JP2008113071A - Automatic tracking device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an automatic tracking device whose tracking precision can be improved. <P>SOLUTION: The automatic tracking device includes an image storage means 3 stored with a group of images picked up by an imaging means 1, a moving contour image generating means 4 of generating a moving contour image by extracting a contour of a moving body from the image group based upon change with time, a template storage means 5 stored with a template for the moving body to be tracked generated based upon the moving contour image, a template matching means 6 of performing template matching using the template to detect the moving body to be tracked from the moving contour image, and a template storage means 8 of updating the template by replacing the template stored in the template storage means 5 with a template T generated by putting the current moving contour image P5 of the moving body to be tracked which is detected by the template matching means 6 together with past moving contour images P1 to P4 of the moving body to be tracked. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an automatic tracking device that automatically tracks (tracks) a specific object in a captured image.

  2. Description of the Related Art In recent years, automatic tracking devices that capture an image of a tracking target and its surrounding area by an image capturing unit such as a camera device and use the captured image are used in many fields because of its convenience.

  As a conventional automatic tracking device, a difference area is obtained from a difference between captured frames (images), the center of the difference area is set as the center of the moving body, and the center of the difference area is zoomed to display an image. For example, and for tracking the moving object by mechanically controlling the direction of the camera device (pan, tilt, etc.) so that the center of the difference area is located at the center of the image.

  However, the size of the difference area extracted by the difference between the frames changes depending on the moving speed of the tracking target moving body (if the moving speed is fast, the size of the difference area becomes large, and if the moving speed is slow, The size of the difference area is reduced). Therefore, in the case where the center of the difference area is zoomed and displayed as in the former case, there are many cases where the tracking target moving body cannot be displayed with high accuracy when the moving speed of the moving body is high.

  In particular, in the latter case in which the camera device orientation (pan, tilt, etc.) is mechanically controlled, the background image changes as the camera device orientation changes. When obtained, a difference area is extracted over the entire captured image, which causes a decrease in tracking accuracy.

  Therefore, an automatic tracking device that performs tracking using a motion vector (optical flow) instead of a difference between frames is provided (for example, Non-Patent Document 1).

In Non-Patent Document 1, when a moving object is detected based on a background difference, a detection frame surrounding the moving object is set. Thereafter, a background motion vector outside the detection frame and a tracking target moving object in the detection frame are set. Respectively, to distinguish the moving object from the background, thereby detecting the moving object. Such a motion vector is obtained by performing block matching using a block composed of a plurality of pixels (for example, a size of 8 × 8). In the case of Non-Patent Document 1, each of the motion vectors is predetermined inside and outside the detection frame. By setting the number of blocks and obtaining the motion vector of each block, the moving object and the background are distinguished.
Tetsuji Hashita, two others, “Tracking people with active cameras using time-averaged silhouettes”, IEICE Transactions D, IEICE, February 1, 2005, Vol.J88-D2, No.2, p.291-301

  However, in the case of detecting a moving object using only a motion vector as in Non-Patent Document 1, if there is another moving object having a similar motion vector, the tracking target moving object and another moving object are detected. It was difficult to distinguish between the body and the tracking result with high accuracy could not be obtained.

  Therefore, in order to improve tracking accuracy, it is conceivable to perform template matching using an image of a moving object to be tracked. However, in the method of extracting a moving object using a motion vector, the moving object has the same motion. Since it is expressed as an aggregate of blocks having vectors, the shape of the moving object is not accurately reflected, and an image of the moving object having a rough shape accuracy is obtained. Even if template matching is performed using an image of such a moving body having a rough shape accuracy, a good result cannot be obtained.

  As described above, the conventional automatic tracking device cannot obtain good tracking accuracy.

  The present invention has been made in view of the above points, and an object thereof is to provide an automatic tracking device capable of improving tracking accuracy.

  In order to solve the above-described problems, in the invention of claim 1, an image storage unit that stores an image captured by the imaging unit, and a moving object based on a temporal change from an image group stored in the image storage unit. A moving contour image generating means for extracting a contour and generating a moving contour image; a template storage means for storing a template of a moving object to be tracked generated based on the moving contour image obtained by the moving contour image generating means; Template matching means for performing template matching using a template stored in the template storage means and detecting a moving object to be tracked from the moving contour image, and detecting a template stored in the template storage means by the template matching means. The moving contour image of the moving object to be tracked this time and the moving contour of the moving object to be tracked in the past Characterized in that it comprises a template updating means for updating the template by replacing a new template created by synthesizing the image.

  According to the first aspect of the present invention, a template is created by synthesizing a moving contour image of a moving object to be tracked in the past and a moving contour image of a moving object to be tracked detected by the current template matching means. Therefore, even if there is a missing part in any moving contour image, the missing part is complemented by another moving contour image, and a template that accurately captures the shape of the moving object to be tracked is obtained. This has the effect of improving the tracking accuracy, and has a stable shape in the tracking target mobile body (part where the movement of the mobile body moves slowly, such as the head and shoulders for humans) Since the contrast is high and the part whose shape is unstable (the part where the movement accompanying the movement of the moving body is intense, such as a hand or a foot in the case of a person) has a low contrast, the shape of the moving body to be tracked And it is a change to perform template matching (not susceptible to influence) robust, also exhibits the effect of improving the tracking accuracy thereby.

  According to a second aspect of the present invention, in the first aspect of the present invention, the moving contour image generating means is configured to generate a moving contour image composed of a binary image.

  According to the second aspect of the present invention, since the moving contour image is a binary image, the capacity required for storing the moving contour image can be reduced, and the memory can be saved.

  According to a third aspect of the present invention, in the first aspect of the present invention, the moving contour image generating means is configured to generate a moving contour image composed of a grayscale image.

  According to the invention of claim 3, since the moving contour image is a grayscale image, a more detailed and accurate template can be obtained, and the tracking accuracy can be improved.

  In the invention of claim 4, in the invention of claim 1, the moving contour image generation means determines whether or not the contrast of the template is low, and if the contrast is not low, generates a moving contour image consisting of a binary image, If the contrast is low, a moving contour image composed of a gray image is generated.

  According to the fourth aspect of the present invention, when the template has a high contrast and sufficient template matching can be performed even with a binary image, the moving contour image is a binary image, so the capacity required for storing the moving contour image is reduced. If the template contrast is low and sufficient template matching cannot be performed with a binary image, the moving contour image is a gray image, so that a more detailed and accurate template can be obtained. The tracking accuracy can be improved.

  In the invention of claim 5, in the invention of claim 4, the moving contour image generation means divides the template into a plurality of regions, calculates an average value of pixel values other than 0 for each of the plurality of regions, and It is configured to determine that the contrast of the template is low when there is a region whose average value is less than a predetermined threshold or a region lower than a predetermined value compared to other regions. .

  According to the fifth aspect of the present invention, when the template has a high contrast and sufficient template matching can be performed even with a binary image, the moving contour image is a binary image, so the capacity required for storing the moving contour image is reduced. If the template contrast is low and sufficient template matching cannot be performed with a binary image, the moving contour image is a gray image, so that a more detailed and accurate template can be obtained. The tracking accuracy can be improved.

  The invention of claim 6 is characterized in that, in the invention of claim 2, 4 or 5, the moving contour image generating means is configured to change the threshold for binarization according to the pixel value of the template. And

  According to the invention of claim 6, since the binarization threshold value is changed according to the pixel value (contrast) of the template, the contour of the moving object in the moving contour image can be obtained even when the pixel value of the template is low. As a result, the tracking accuracy can be improved as compared with the case where the binarization threshold is always constant.

  In the invention of claim 7, in the invention of claim 6, the moving contour image generation means calculates an average value of the pixel values of the template, and when the average value of the pixel values is equal to or less than a predetermined threshold, 2 The present invention is characterized in that the threshold value for valuation is lowered.

  According to the seventh aspect of the present invention, when the contrast of the template is low, the threshold value for binarization is lowered to thicken the contour (edge) of the moving object in the moving contour image. Template matching is facilitated even when the value is low, and as a result, the tracking accuracy can be improved.

  In the invention of claim 8, in the invention of any one of claims 1 to 7, the template updating means includes a moving contour image of a predetermined number of past tracking target moving bodies arranged in time series and the current template matching. A template is created by combining the moving contour image of the tracking target moving body detected by the means.

  According to the invention of claim 8, it is possible to combine the moving contour image of the tracking target moving body in the past with the moving contour image of the tracking target moving body detected by the current template matching means. Therefore, there is an effect that it is possible to gradually follow a change in the shape of the tracking target moving body.

  In the invention of claim 9, in the invention of any one of claims 1 to 7, the template updating means selects and synthesizes a predetermined number of moving contour images of the tracking target moving bodies arranged in chronological order from the newest one. A template is created, and each time a predetermined number of moving contour images are accumulated thereafter, a template is created by synthesizing the newly accumulated predetermined number of moving contour images. It is characterized by.

  According to the ninth aspect of the invention, since the synthesized image created by synthesizing a predetermined number of moving outline images of the tracking target moving bodies arranged in time series is used, the shape of the tracking target moving body is instantaneously Even if it changes greatly, there is an effect that the influence of such a moving contour image can be reduced and the tracking accuracy can be improved.

  In the invention of claim 10, in the invention of claim 8 or 9, the template updating means is configured to thin out unnecessary images from the moving contour image of the tracking target moving body arranged in time series. Features.

  According to the tenth aspect of the present invention, since unnecessary images that adversely affect the template are thinned out from the moving contour images of the tracking target moving bodies arranged in time series, the tracking accuracy can be improved. There is an effect that can be.

  In the invention of claim 11, in the invention of claim 10, the template updating means counts the number of pixels whose pixel value is not 0 from the moving contour image, and the number of the pixels is moved before the moving contour image. The moving contour image of the tracking target moving body obtained from the moving contour image is regarded as the unnecessary image as long as it is increased or decreased by a predetermined value or more compared to the contour image. .

  According to the eleventh aspect of the present invention, unnecessary images that adversely affect the template are thinned out from the moving contour images of the tracking target moving bodies arranged in time series order, so that the tracking accuracy can be improved. There is an effect that can be.

  In the invention of claim 12, in the invention of claim 10, the template updating means calculates a standard deviation of the pixel value of the moving contour image or the image of the imaging means that is the basis of the moving contour image, and the pixel value Is obtained from the moving contour image if the standard deviation of the moving contour image has increased or decreased by a predetermined value or more compared to the moving contour image before the moving contour image or the image of the imaging means that is the source of the previous moving contour image. The moving contour image of the tracking target moving body is regarded as the unnecessary image.

  According to the twelfth aspect of the present invention, unnecessary images that adversely affect the template are thinned out from the moving contour images of the tracking target moving bodies arranged in time series order, so that the tracking accuracy can be improved. There is an effect that can be.

  According to a thirteenth aspect of the present invention, in the tenth aspect of the present invention, the template updating means is configured such that the number of pixels of the contour portion of the moving body in the moving contour image is greater than or equal to a predetermined value compared to the moving contour image before the moving contour image. If it is increased or decreased, the moving contour image of the tracking target moving body obtained from the moving contour image is regarded as the unnecessary image.

  According to the thirteenth aspect of the present invention, unnecessary images that adversely affect the template are thinned out from the moving contour images of the tracking target moving bodies arranged in time series order, thereby improving the tracking accuracy. There is an effect that can be.

  In the present invention, a template is created by synthesizing a moving contour image of a moving object to be tracked in the past and a moving contour image of a moving object to be tracked detected by the template matching unit this time. Even if there is a missing part in the moving contour image, the missing part is complemented by other moving contour images, and a template that accurately captures the shape of the moving object to be tracked can be obtained. There is an effect that the accuracy is improved, and the portion of the tracking target moving body whose shape is stable (the portion where the movement accompanying the movement of the moving body is slow, such as the head or shoulder in the case of a person) has a high contrast, Robustness to changes in the shape of the moving body to be tracked, since the contrast is low for parts with unstable shapes (parts where the movement of the moving body is intense, such as hands and feet if humans) Since it is possible to perform template matching (influence not susceptible to), an effect that can be further improved tracking accuracy.

(Embodiment 1)
As shown in FIG. 1A, the automatic tracking device of the present embodiment includes an imaging unit 1, an A / D conversion unit 2 that performs A / D conversion on an image (input image) captured by the imaging unit 1, and an imaging unit. An image storage unit 3 in which the image group captured by the unit 1 is stored, and a movement that extracts a contour of the moving body from the image group stored in the image storage unit 3 based on temporal changes and generates a moving contour image The contour image generating means 4, the template storage means 5 for storing the template of the tracking target moving body generated based on the moving contour image obtained by the moving contour image generating means 4, and the template storage means 5. Template matching unit 6 that performs template matching (pattern matching) using a template and detects a moving object to be tracked from a moving contour image, and results of template matching unit 6 Based on the image of the imaging unit 1 that is the basis of the moving contour image, the cutting unit 7 that extracts (extracts) an image of the tracking target moving body and the template stored in the template storage unit 5 are updated. A template updating means 8, a control means 9 for controlling these, and an image display means 10 for displaying an image cut out by the cutting means 7 are provided. The template updating means 8 is shown in FIG. The moving outline images P1 to P4 of the moving object to be tracked in the past and the moving outline image P5 of the moving object to be tracked detected by the template matching means 6 this time are combined to create a new template T. The template is updated by replacing the template stored in the storage means 5 with a new template T.

  The imaging means 1 is, for example, a camera device that includes an imaging element (not shown) that continuously images a predetermined area in time series and outputs an image (image data) as an analog signal. As the image pickup means 1, a device that outputs an image as a digital signal, such as a general CMOS camera, may be used. In this case, the A / D conversion means 2 may not be provided. The A / D conversion means 2 is for converting an analog signal output from the imaging means 1 into a digital signal, and the digital signal converted by the A / D conversion means 2 is output to the image storage means 3. .

  The image storage unit 3 is a storage device such as a RAM, and is used for storing an image captured by the imaging unit 1. The image captured by the imaging unit 1 is stored in the image storage unit 3 as an image having a predetermined resolution. Here, as the image information to be stored in the image storage unit 3, a monochrome grayscale image is sufficient considering only the tracking of the moving object, but the image display unit 10 displays an image. Color information is also stored. The image storage unit 3 also stores the moving contour image generated by the moving contour image generation unit 4.

  The moving contour image generation unit 4 is configured to generate a moving contour image composed of binary images based on the image group (image group arranged in time series) stored in the image storage unit 3. For example, it has a binarizing means, a contour extracting means, a moving contour extracting means, and a moving area detecting means.

  The binarization unit is configured to generate a binary image from the grayscale image stored in the image storage unit 3. Specifically, the pixel value (luminance value) for each pixel of the grayscale image is compared with a predetermined binarization threshold value, and the pixel value is determined depending on whether the pixel value is equal to or greater than the binarization threshold value. Are divided into white and black, thereby generating a binary image. The contour extraction unit extracts a contour (edge) by performing a differentiation process using a generally known SOBEL filter (SOBEL operator) based on the binary image obtained by the binarization unit. Thus, a contour image (edge image) is created. The moving contour extracting unit is configured to perform logical synthesis using the contour image created by the contour extracting unit and create a moving contour image in which only the contour of the moving object is extracted. The moving area detecting means is configured to label a moving outline image created by the moving outline extracting means and detect an area corresponding to a moving body (hereinafter referred to as “moving area”). Here, the moving area is represented by a rectangular area circumscribing the outline of the moving body. By doing so, the amount of data is reduced and the calculation is facilitated.

  Therefore, according to the moving contour image generating means 4 described above, as shown in FIG. 2A, the moving contour formed of a binary image in which the pixel value of the pixel indicating the contour portion E of the moving body is 1. An image D is obtained, and the moving contour image D thus obtained is stored in the image storage means 3.

  The template storage unit 5 is a storage device such as a RAM, and is used for storing a template T (see FIG. 1B) used in the template matching unit 6.

  The template matching unit 6 performs template matching (pattern matching) using the template stored in the template storage unit 5, and detects a tracking target moving body from the moving contour image stored in the image storage unit 3 (movement). The image corresponding to the tracking target moving body, ie, the moving contour image of the tracking target moving body is extracted from the contour image), and the template matching result (correlation value information) is output to the control means 9. It is configured as follows.

  The cutout unit 7 cuts out (extracts) an image stored in the image storage unit 3 (an input image obtained from the image pickup unit 1) or a part of the moving contour image generated by the moving contour image generation unit 4. In some cases, the image is enlarged and output to the image display means 10 or the template update means 8.

  The template update unit 8 includes a moving contour image of the moving object to be tracked this time detected by the template matching unit 6 (a moving contour image of the moving object to be tracked latest) and a moving contour image of the moving object to be tracked in the past. Are combined to create a new template, and the template is updated by replacing the template stored in the template storage means 5 with the new template. When there is no moving contour image of the tracking target moving body in the past, the moving contour image of the tracking target moving body cut out by the cutting means 7 is stored in the template storage means 5 as a template as it is. ing.

  Hereinafter, a method for creating a new template in the template updating unit 8 will be described in detail with reference to FIGS. 1B, 2B, and 2C. In the following description, a moving contour image of a tracking target moving body is abbreviated as a tracking moving contour image.

  In the template update unit 8 of the present embodiment, a predetermined number (four in the present embodiment) of the past tracking movement contour images P1 to P4 from the newest of the past tracking movement contour images arranged in chronological order, and the latest tracking. The moving contour image P5 is combined to create a new template T, and the template T combines the past tracking moving contour images P1 to P4 as shown in FIG. After the composite image Pt is created, it is created by synthesizing the tracking movement contour image P5 as shown in FIG. 2C with the composite image Pt. The tracking movement contour image P5 is shown in a simplified manner in FIG.

  The past tracking outline images P1 to P4 are synthesized by aligning the tracking movement outline images P1 to P4, and then integrating the pixel values of the corresponding pixels. As a result, FIG. A composite image Pt as shown in FIG. Although the past tracking movement contour images P1 to P4 are binary images as described above, the synthesized image Pt has a pixel value obtained by integrating the pixel values of the past tracking movement contour images P1 to P4. As shown in b), the image is not a binary image but a grayscale image.

  Here, the tracking movement contour images P1 to P4 are aligned by, for example, matching the position where the correlation value is the highest by pattern matching or by aligning the centers (centers of gravity) of the tracking movement contour images P1 to P4. Do. This also applies to the alignment between the composite image Pt and the latest tracking movement contour image P5. When the number of past tracking movement contour images is less than the predetermined number, all past tracking movement contour images are used (that is, if there are three past tracking movement contour images). Use all three).

  After the composite image Pt is created, the template T is created by aligning the composite image Pt and the latest tracking movement outline image P5 and integrating the pixel values of the corresponding pixels. Here, when the synthesized image Pt and the latest tracking movement contour image are synthesized, the weighting function K is used. For example, the pixel value of the synthesized image Pt is Vp (x, y), and the latest tracking movement contour image. When the pixel value Vn (x, y) of P5 is set, the pixel value Vt (x, y) of the template T is expressed by K · Vn (x, y) + (1−K) · Vp (x, y). To give.

  In this way, it is possible to select which of the latest tracking movement contour image P5 and the synthesized image Pt composed of the past tracking movement contour images P1 to P4 is to be emphasized according to the value of K. For example, if the value of K is increased, the latest tracking moving contour image P5, which is a binary image as shown in FIG. 2C, is strongly reflected in the template T, and if K is decreased, FIG. The influence of the composite image Pt, which is a grayscale image as shown in FIG. Instead of using the weight function, the pixel value of the latest tracking movement contour image P5 may be simply added to the pixel value of the composite image Pt. In this case, the template T is created by each tracking movement contour image. After aligning P1 to P5, the pixel values of the corresponding pixels may be integrated.

  As described above, the template update unit 8 creates a new template T by accumulating the tracking movement outline images. As described above, the tracking movement contour image is a binary image, but the pixel value of each pixel of the template T is given by the integration of the pixel values of the pixels of the tracking movement contour images P1 to P5. As shown in FIG. 1B, a gray image is obtained.

  The template T obtained as described above has the following advantages over the template obtained by cutting out the tracking movement outline image as in the prior art. That is, since the conventional template is an image in which the moving contour image displays only the moved contour portion of the moving body, the non-moving contour portion of the moving body is not reflected in the image, and the shape of the moving body is changed. Although it cannot be accurately captured, in the above-described template T, even if there is a portion lacking in the contour portion of any of the tracking movement contour images P1 to P5, the contour portion is missing due to another tracking movement contour image. Since the existing part is complemented, a template that accurately captures the shape of the tracking target moving body can be obtained.

  In addition, in the template T, as shown in FIG. 1B, a site whose shape is stable in the tracking target mobile body (a site where the movement accompanying the movement of the mobile body is slow, a human head or shoulder, etc.) The contrast becomes high (pixel value becomes large), and the part where the shape is unstable (the part where the movement accompanying the movement of the moving body is intense, such as hands and feet in the case of humans) has low contrast (the pixel value becomes small). Therefore, it is possible to perform template matching that is robust (unaffected) by changes in the shape of the tracking target moving body.

  As described above, the template updating unit 8 creates a new template T, and replaces the template stored in the template storage unit 5 (the template used to obtain the tracking movement outline image P5) with the new template T. . The new template T is used for template matching with the next moving contour image by the template matching means 6.

  Then, as a result of template matching using the new template T in the template matching means 6, a tracking movement contour image P6 is obtained, and when the template updating means 8 acquires a new tracking movement contour image P6, In the template updating means 8, a predetermined number (four in this embodiment) of the past tracking movement contour images P2 to P5 and the latest tracking movement contour image P6 from the newest of the past tracking movement contour images arranged in time series. Are combined to create a template. Then, a template (a template formed by combining the tracking movement contour images P1 to P5) T stored in the template storage unit 5 is a newly generated template (a template formed by combining the tracking movement contour images P2 to P6). ) And the replaced template is used for the next template matching.

  The control means 9 is for controlling the operations of the template matching means 6, the cutting means 7, and the template updating means 8, and is composed of, for example, a microcomputer having a CPU or the like, and includes a tracking target designating means, Tracking target display means and position prediction means are provided. Further, the control means 9 is provided with a console or the like used in the tracking target specifying means.

  The tracking target specifying means is for selecting a tracking target moving body from the image of the imaging means 1 displayed on the image display means 10 in the initial state before specifying the tracking target moving body. It is configured to control the means 7 to output the image stored in the image storage means 3 to the image display means 10 without performing a cutting process or the like. The tracking target designating unit is configured to be able to designate a tracking target moving body from the image of the imaging unit 1 by an input from a user interface such as a console. Note that a configuration for designating a predetermined area of an image by a console or the like can be a well-known configuration and will not be described.

  Further, the tracking target designating unit controls the clipping unit 7 when the tracking target moving body is designated from the image of the imaging unit 1, and the tracking target designation unit calculates the tracking target image from the moving contour image obtained from the image of the imaging unit 1. A moving contour image of a moving body corresponding to the tracking target moving body is acquired, and this image is stored in the template storage means 5 by the template update means 8 as a template.

  By the way, the control means 9 automatically sets the moving contour image of the moving object in the template when detecting the moving object, and does not follow the tracking target designating means. You may comprise. Since such a configuration is well known in the art, a detailed description thereof will be omitted.

  The tracking target display means adopts the position having the highest correlation value in the moving contour image as the position of the tracking target moving body based on the result of template matching received from the template matching means 6, and uses this position as the center (or center of gravity). ) And a moving contour image of a predetermined range area (for example, a moving area) is cut out by the cutting unit 7 as a moving contour image of the tracking target moving body (in some cases, cut out and enlarged), and the template is extracted from the cutting unit 7. The updating means 8 is configured to output.

  Further, the tracking target display means is an image of an area (for example, a moving area) in a predetermined range centered on (or centroid of) the position of the tracking target moving body from the image (input image) that is the basis of the moving contour image. After being cut out by the cutout means 7 and enlarged, it is output to the image display means 10 and displayed with color information on the image display means 10. In addition, when displaying the tracking target moving body on the image display means 10, it may be possible to select which part of the moving body is enlarged and displayed. For example, when the moving body is a person, the face of the moving body may be enlarged and displayed.

  The position prediction means predicts the position of the tracking target moving body based on the position of the tracking target moving body in the past (the previous tracking target moving body position and the previous tracking target moving body position). Thus, the processing time (mainly the processing time in the template matching means 6) is shortened, and the position prediction process and the search range setting process are executed.

  The position prediction process calculates the movement vector of the tracking target moving object based on the position of the tracking target moving object in the previous image and the position of the tracking target moving object in the previous image. This is a process for obtaining the position of the tracking target moving body in the current image (hereinafter referred to as “predicted position”) based on the position of the tracking target moving body.

  For example, in the position prediction process of the present embodiment, it is assumed that the movement vector of the tracking target moving body hardly changes in a short time such as the time between images in time-series order captured by the imaging unit 1. The position moved by the movement vector from the position P2 of the previous tracking target moving body is obtained as the predicted position. The calculation method of the predicted position is not limited to the above method, and may be calculated in consideration of various conditions. In addition, when calculating the predicted position, the prediction may be performed by referring to the history of the past position in addition to the previous position and the previous position, or the prediction may be performed using a neural network or the like. May be.

  The search range setting process is a minimum range that encloses all the movement areas that overlap with the prediction movement range including the prediction position obtained by the position prediction process among the movement areas detected from the movement outline image (in other words, movement related to the prediction movement range). The range obtained by integrating the regions) is set as the search range of the tracking target moving body in the moving contour image. Here, the predicted movement range is a rectangular area centered on the predicted position, and the size thereof is set in consideration of the moving speed of the moving body. The search range information thus obtained is sent to the template matching means 6, and the template matching means 6 that has received the search range information performs template matching within the search range.

  Below, operation | movement of the automatic tracking apparatus of this embodiment is demonstrated. First, in an initial state before the tracking target is set, an image obtained by the imaging unit 1 is stored in the image storage unit 3 and displayed on the image display unit 10 by the tracking countermeasure designating unit of the control unit 9. That is, in the initial state, the image obtained by the imaging unit 1 is displayed on the image display unit 10 as it is.

  In this initial state, when the tracking target is designated from the image displayed on the image display means 10 using the console, the clipping means 7 is controlled by the tracking target designation means of the control means 9, and the moving contour image generating means is controlled. 4, the moving contour image of the tracking target moving body is cut out from the moving contour image generated based on the image of the imaging means 1, and the moving contour image of the tracking target moving body cut out in this way is used as the template updating means. 8 is stored in the template storage means 5 as a template. At this time, since there is no past moving contour image, the template updating unit 8 stores the tracking moving contour image cut out by the cutting unit 7 in the template storage unit 5 as a template as it is.

  Next, the operation of the automatic tracking device after the template is stored in the template storage means 5 will be described with reference to the flowchart shown in FIG.

  First, when an image is input from the imaging unit 1 (step S1), a moving contour image is generated by the moving contour image generation unit 4 (step S2). Then, the predicted position of the tracking target moving body in the moving contour image is obtained by the position predicting means of the control means 9 (step S3), and a search range for performing template matching is set based on the predicted position (step S4). ). However, immediately after the tracking target is designated, the position data of the past tracking target moving body is insufficient, so steps S3 and S4 are omitted. In step S5 described later, template matching is performed on the entire moving contour image. .

  Thereafter, the template matching unit 6 performs template matching on the search range of the moving contour image set in step S 4 using the template stored in the template storage unit 5, and outputs the result to the control unit 9. (Step S5).

  In the tracking target display means of the control means 9, based on the result obtained from the template matching means 6, the position having the highest correlation value in the image (moving contour image subjected to template matching) is set as the position of the tracking target moving body. The image in a predetermined range centered on this position (or the center of gravity) is enlarged and cut out by the cutout means 7, and the cut out image is sent to the template update means 8. Then, the template update means 8 creates a new template T as described above, and the template stored in the template storage means 5 is updated to the template T (step S6).

  Further, the tracking target display means of the control means 9 is a region within a predetermined range (for example, moving) from the image that is the origin of the moving contour image of the control means 9 with the position of the tracking target moving body as the center (or center of gravity). The image of the area) is cut out and enlarged by the cutout means 7, output to the image display means 10, and displayed with color information on the image display means 10 (step S 7).

  Thereafter, the tracking of the tracking target moving body is continued by repeating steps S1 to S7.

  According to the automatic tracking device of the present embodiment described above, a new template T is created by combining the past moving contour images P1 to P4 and the latest moving contour image P5. Even if there is a missing part in the moving contour images P1 to P5, the missing part is complemented by other moving contour images, and a template T that accurately captures the shape of the tracking target moving body can be obtained. As a result, the tracking accuracy is improved.

  In addition, in the tracking target moving body, the part having a stable shape (the part where the movement accompanying the movement of the moving body is slow, such as the head and shoulders in the case of a person) has a high contrast, and the part whose movement is unstable (the movement Since parts of the body that move with the movement of the body, such as human hands and feet, have a low contrast, template matching that is robust (insensitive to changes) in the shape of the tracking target moving body can be performed. This also has the effect of improving the tracking accuracy.

  In addition, since a binary image obtained by binarizing the grayscale image is used as the moving contour image, the capacity necessary for storing the moving contour image can be reduced, so that the memory can be saved. .

  Further, the template updating unit 8 synthesizes a predetermined number of moving outline images of the tracking target moving body arranged in time series and the moving outline image of the tracking target moving object detected by the current template matching unit. Thus, a new template is created, so that the moving contour image of the moving object to be tracked in the past and the moving contour image of the moving object to be tracked detected by the current template matching means are weighted. It is possible to add and combine them, and this has the effect of gradually following the change in the shape of the tracking target moving body.

  By the way, the moving contour image generating means 4 may be configured to change the binarization threshold according to the pixel value of the template. In this way, the threshold for binarization is changed according to the pixel value (contrast) of the template, so that it is possible to obtain the contour of the moving object in the moving contour image even when the pixel value of the template is low. Therefore, the tracking accuracy can be improved as compared with the case where the binarization threshold is always constant.

  As a specific example, the moving contour extracted image generating means 4 calculates the average value of the pixel values of the template, and when the average value of the pixel values is equal to or less than a predetermined threshold value, the threshold value for binarization is lowered. An example of such a configuration is given. In this case, when the contrast of the template is low, the threshold value for binarization is lowered to thicken the contour (edge) of the moving object in the moving contour image. As a result, it is easy to perform template matching, and as a result, the tracking accuracy can be improved.

  On the other hand, the moving contour image generation means 4 generates a moving contour image consisting of a binary image by binarizing the grayscale image, thereby obtaining a merit that memory can be saved. Yes. However, when the binarization process is performed, even if the pixel displaying the outline portion of the moving body is cut off if the contrast is low, there is a tendency to be somewhat accurate. Therefore, the grayscale image may be used as it is without being binarized, and a moving contour image composed of the grayscale image may be generated. In this way, a more detailed and accurate template can be obtained, so that the tracking accuracy can be improved.

  Further, whether the moving contour image is a binary image or a gray image may be selected depending on the situation. For example, the moving contour image generating means 4 determines whether or not the contrast of the template is low. If the contrast is not low, a moving contour image consisting of a binary image is generated. If the contrast is low, a moving contour consisting of a gray image is generated. You may comprise so that an image may be produced | generated.

  Here, as a method for determining the contrast of the template in the moving contour image generation means 4, for example, the template is divided into a plurality of regions, an average value of pixel values other than 0 is calculated for each of the plurality of regions, There is a method of determining that the template has a low contrast when there is a region where the average pixel value is less than a predetermined threshold or when there is a region lower than a predetermined value by a predetermined value or more. The threshold value may be set to an arbitrary value by the user or adjusted with reference to the contrast of the image captured by the imaging unit 1.

  In this way, when the template contrast is high and sufficient template matching can be performed even with a binary image, the capacity necessary for storing the moving contour image can be reduced by making the moving contour image a binary image. Can save memory. On the other hand, when the template contrast is low and it is difficult to perform sufficient template matching with a binary image, the moving contour image is made to be a grayscale image, so that the tracking accuracy can be improved.

  By the way, the template updating means 8 described above, the newest tracking movement outline images P1 to P4 (four in this embodiment) from the newest of the past tracking movement outline images arranged in time series, and the latest tracking movement outline. When a new template T is created by combining the image P5 and a new tracking movement contour image P6 is obtained thereafter, the four tracking tracks from the newest of the past tracking movement contour images arranged in chronological order are used. The movement outline images P2 to P5 and the latest tracking movement outline image P6 are synthesized to create a new template T, that is, the tracking movement outline image that forms the template T every time the latest tracking movement outline image is obtained. The template T is updated by removing the oldest tracking movement contour image from the template, but the template update means 8 may be as follows.

  The template update means 8 creates a template by selecting and synthesizing a predetermined number of tracking movement contour images arranged in chronological order, and each time the tracking movement contour image is accumulated by the predetermined number, A template may be created by synthesizing a predetermined number of tracking movement outline images newly accumulated. As a template synthesis method, the pixel values of the corresponding pixels are integrated after alignment as described above.

  For example, when there are 10 tracking movement contour images, the template update unit 8 creates a template by selecting and synthesizing a predetermined number (for example, five) of the tracking movement contour images from the newest one, and stores the template as a template. The data is stored in the means 5. Thereafter, when template matching is performed by the template matching means 6 and a new tracking movement contour image is obtained, a new template is created every time a new tracking movement contour image is obtained in the above example. In this example, the template is not immediately created, and when a predetermined number (for example, 5) of tracking movement contour images is obtained, these are combined to create a template.

  As described above, in the template update means of this example, since a synthesized image created by synthesizing a predetermined number of tracking movement contour images arranged in time series is used, new tracking movements are successively performed as in the above example. Unlike the case where the template is updated by incorporating the contour image, the same template is used until the tracking movement contour image is accumulated to some extent, so that the shape of the tracking target moving body changes greatly. Even if it exists, there exists an effect that the influence of such a movement outline image can be made small and the improvement of a tracking precision can be aimed at.

  In addition, the template update means 8 creates a new template T by synthesizing the past tracking movement contour images P1 to P4 arranged in chronological order and the latest tracking movement contour image P5, and thereafter a new tracking T is performed. When the moving contour image P6 is obtained, the template T is updated by synthesizing the template T and the tracking moving contour image P6. That is, every time the latest tracking moving contour image is obtained, the template T used this time and the latest The template T to be used next time is created by synthesizing with the tracking movement outline image (that is, the template T is created by accumulating the tracking movement outline images one after another). Also good. Note that the weight function described above may be used when the latest tracking movement contour image and the template T used to obtain the latest tracking movement contour image are combined.

  In the above example, an example in which a moving object is detected using an edge image has been described. However, a moving object may be detected by a temporal change in a grayscale image, and various other known methods may be used. You may make it use.

(Embodiment 2)
The automatic tracking device of the present embodiment has the configuration shown in FIG. 1A as in the first embodiment, but the configuration of the template update unit 8 is different. Since other configurations are the same as those of the first embodiment, description thereof is omitted.

  That is, the template update unit 8 counts the number of pixels whose pixel values are not 0 from the moving contour image generated by the moving contour image generation unit 4, and the number of the pixels is earlier (time) than the moving contour image. If the sequence has increased or decreased by a predetermined value or more compared to the previous moving contour image, it is determined that the moving contour image contains a lot of noise and the tracking target movement obtained from the moving contour image is determined. The moving contour image of the body is configured to be thinned out from the moving contour image of the tracking target moving body arranged in time series. The predetermined value may be set to an arbitrary value.

  In such a template updating unit 8, when the number of pixels has increased or decreased by a predetermined value or more compared to the moving contour image before the moving contour image, the moving contour image is rapidly changed due to noise or the like. Tracking objects that are obtained from a moving contour image that has undergone such a sudden change are regarded as unnecessary images and are tracked in chronological order. The operation of thinning out the moving contour image of the moving body is performed.

  Therefore, according to the automatic tracking device of the present embodiment, the template updating unit 8 is configured to thin out unnecessary images that adversely affect the template from the moving contour image of the tracking target moving body arranged in time series. As a result, the tracking accuracy can be improved. Here, it may be configured such that the number of unnecessary images to be thinned out can be set to an arbitrary number from the moving contour images of the tracking target moving objects arranged in time series order, thereby reducing the number of unnecessary images to be thinned out. Various settings such as limiting and preventing excessive thinning are possible.

  By the way, as another example of the template update means 8 of this embodiment, the following can be cited. For example, the template update unit 8 calculates the standard deviation of the pixel value of the moving contour image or the image of the imaging unit that is the basis of the moving contour image, and the standard deviation of the pixel value is the movement before the moving contour image. If the contour image or the image of the imaging means that is the source of the previous moving contour image is increased or decreased by a predetermined value or more, the moving contour image of the tracking target moving body obtained from the moving contour image is not necessary. It may be configured to be regarded as a correct image.

  Further, the template updating means 8 has a pixel number of the contour portion of the moving body in the moving contour image (or the number of regions which is the number of regions composed of a plurality of pixels) compared to the moving contour image before the moving contour image. The moving contour image of the tracking target moving body obtained from the moving contour image may be regarded as the unnecessary image as long as it is increased or decreased by a predetermined value or more.

  That is, the standard deviation of the pixel value of the moving contour image or the image of the imaging means that is the basis of the moving contour image is calculated, and the standard deviation of the pixel value is the moving contour image before the moving contour image or the previous one When the number of pixels (or the number of areas) of the contour portion of the moving body in the moving contour image is increased or decreased by a predetermined value or more compared to the image of the imaging means that is the source of the moving contour image, the moving contour image When the moving contour image has increased or decreased by a predetermined value or more compared to the previous moving contour image, it is considered that a large amount of noise has occurred in the image of the imaging means, and the movement of the tracking target obtained from the moving contour image The moving contour image of the body is handled as the unnecessary image.

  Even with the template updating means as described above, unnecessary images that adversely affect the template can be thinned out from the moving contour image of the tracking target moving body arranged in chronological order, and the tracking accuracy can be similarly improved. There is an effect that it can be achieved.

(A) is explanatory drawing which shows the block structure of the automatic tracking apparatus of Embodiment 1, (b) is explanatory drawing of the update method of a template. (A) is a schematic explanatory diagram of a moving contour image, (b) is a schematic explanatory diagram of a composite image, and (c) is a schematic explanatory diagram of a moving contour image of a moving object to be tracked. It is a flowchart of operation | movement of an automatic tracking apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image pickup means 3 Image storage means 4 Moving outline image generation means 5 Template storage means 6 Template matching means 8 Template update means P1-P5 Moving outline image of moving object to be tracked T template

Claims (13)

  Image storage means for storing an image picked up by the image pickup means, and moving contour image generation means for generating a moving contour image by extracting a contour of a moving body based on temporal changes from a group of images stored in the image storage means A template storage unit that stores a template of a moving object to be tracked that is generated based on a moving contour image obtained by the moving contour image generation unit, and a template that is stored in the template storage unit. A template matching unit that detects a tracking target moving body from the moving contour image, a template stored in the template storage unit, and a moving contour image of the tracking target moving body detected by the template matching unit and a past Place it on a new template created by combining the moving contour image of the moving object to be tracked Automatic tracking apparatus characterized by comprising a template updating means for updating the template by obtaining.   2. The automatic tracking device according to claim 1, wherein the moving contour image generating means is configured to generate a moving contour image composed of a binary image.   2. The automatic tracking device according to claim 1, wherein the moving contour image generating means is configured to generate a moving contour image composed of grayscale images.   The moving contour image generation means determines whether or not the template has low contrast. If the contrast is not low, the moving contour image generation unit generates a moving contour image composed of a binary image, and if the contrast is low, generates a moving contour image composed of a gray image. The automatic tracking device according to claim 1, wherein the automatic tracking device is configured to do so.   The moving contour image generation means divides the template into a plurality of regions, calculates an average value of pixel values other than 0 for each of the plurality of regions, and the region where the average value of the pixel values is less than a predetermined threshold or other 5. The automatic tracking device according to claim 4, wherein the template is determined to have a low contrast when there is a region lower than a predetermined value by a predetermined value or more.   6. The automatic tracking device according to claim 2, wherein the moving contour image generating means is configured to change a threshold value for binarization according to a pixel value of a template.   The moving contour image generating means is configured to calculate an average value of the pixel values of the template, and to lower the threshold value for binarization when the average value of the pixel values is equal to or less than a predetermined threshold value. The automatic tracking device according to claim 6.   The template updating means synthesizes a moving contour image of a predetermined number of past tracking target moving bodies arranged in time series and a moving contour image of the tracking target moving body detected by the current template matching means to create a template. The automatic tracking device according to claim 1, wherein the automatic tracking device is configured to perform.   The template update means creates a template by selecting and synthesizing a predetermined number of moving contour images of tracking target moving bodies arranged in chronological order, and thereafter, the moving contour image is accumulated by the predetermined number of images. The automatic tracking device according to any one of claims 1 to 7, wherein a template is created by synthesizing a predetermined number of newly accumulated moving contour images every time.   10. The automatic tracking device according to claim 8, wherein the template updating means is configured to thin out unnecessary images from the moving contour images of tracking target moving bodies arranged in time series.   The template updating means counts the number of pixels whose pixel value is not 0 from the moving contour image, and if the number of pixels is increased or decreased by a predetermined value or more compared to the moving contour image before the moving contour image, The automatic tracking device according to claim 10, wherein the moving contour image of the tracking target moving body obtained from the moving contour image is regarded as the unnecessary image.   The template update means calculates a standard deviation of the pixel value of the moving contour image or the image of the imaging means that is the basis of the moving contour image, and the standard deviation of the pixel value is a moving contour before the moving contour image. If the image or the image of the imaging means that is the basis of the previous moving contour image is increased or decreased by a predetermined value or more, the moving contour image of the tracking target moving body obtained from the moving contour image is not necessary. The automatic tracking device according to claim 10, wherein the automatic tracking device is configured to be regarded as a correct image.   The template updating means is obtained from the moving contour image if the number of pixels of the contour portion of the moving body in the moving contour image is increased or decreased by a predetermined value or more compared to the moving contour image before the moving contour image. The automatic tracking device according to claim 10, wherein the moving contour image of a tracking target moving body is regarded as the unnecessary image.