JP2009038777A - Automatic tracking apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To more accurately track and image a tracking object. <P>SOLUTION: An automatic tracking apparatus includes a camera 1 capable of controlling panning, tilting and zooming; and a processing section 2. The processing section 2 includes a tracking processing means which performs tracking processing to track a tracking object on the basis of an image imaged by the camera 1, and a control means for controlling panning, tilting and zooming of the camera 1 so that the camera 1 tracks the tracking object in accordance with the result of the tracking processing by the tracking processing means. The tracking processing means includes a preliminary processing means for selecting, in the beginning of tracking start of the tracking object, either a single-part mode during which tracking processing is performed using a featured value obtained from an entire tracking object area that is an area corresponding to the tracking object, or a multi-part mode during which tracking processing is performed using featured values obtained from respective divided areas of the tracking object area; and a main processing means for performing tracking processing in the mode selected by the preliminary processing means. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an automatic tracking device that automatically tracks and images a tracking target using a camera capable of controlling pan, tilt, and zoom.

Conventionally, as such an automatic tracking device, for example, an automatic tracking device disclosed in Patent Document 1 below has been proposed. The automatic tracking device includes an image input / output unit that inputs a video signal from a television camera and outputs the video signal to a display unit, an image processing unit that performs preprocessing of the video signal from the image input / output unit, and an image A correlation calculation unit that obtains a correlation value between the reference image data and the search image data input via the processing unit and detects a position having the highest correlation on the search image, and the television camera based on a signal from the correlation calculation unit And a central processing unit for controlling each of the above-mentioned parts as well as driving and controlling the swivel device and the zoom lens.
JP-A-11-187387

  However, in the conventional automatic tracking device, since the position of the tracking target is obtained using the correlation calculation, it is weak against changes in the size and shape of the tracking target, the influence of occlusion (concealment) and the influence of image fluctuation due to camera control. Therefore, the tracking target cannot be tracked with high accuracy.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide an automatic tracking device that can track and image a tracking target with higher accuracy.

  In order to solve the above-described problem, an automatic tracking device according to a first aspect of the present invention includes a camera capable of controlling pan, tilt, and zoom, and an image captured by the camera. And processing means for controlling the camera so as to automatically track the image and capture the image. The processing means performs tracking processing for tracking a tracking target based on an image captured by the camera, and the camera tracks the tracking target according to a result of the tracking processing by the tracking processing means. Control means for controlling pan, tilt and zoom of the camera. The tracking processing means is a single feature amount obtained from an entire area of the tracking target area, which is an area corresponding to the tracking target, at the beginning of tracking of the tracking target based on an image captured by the camera. A single part mode for performing tracking processing using part feature amounts, or a multi-part mode for performing tracking processing using multi-part feature amounts, which are feature amounts respectively obtained from the divided regions of the tracking target region, and Pre-processing means for selecting whether to perform, and main processing means for performing tracking processing in the mode selected by the pre-processing means.

  According to the first aspect, the preliminary processing means for selecting a single part mode and a multipart mode based on an image captured by the camera is employed, and the tracking process is performed in the mode selected by the preliminary processing means. Since tracking processing is performed based on more information than in the case of always performing the single part mode, for example, even for tracking targets where it is difficult to distinguish between persons or between a person and a background. The tracking process can be performed with higher accuracy, and as a result, the tracking target can be tracked with higher accuracy and imaged.

  An automatic tracking device according to a second aspect of the present invention is the automatic tracking device according to the first aspect, wherein the feature amount is a histogram. According to the second aspect, since the histogram is used, it is resistant to changes in the size and shape of the tracking target, and is less susceptible to image fluctuations due to camera control. However, the feature amount is not limited to this example in the present invention.

In the automatic tracking device according to a third aspect of the present invention, in the first aspect, the feature amount is a histogram based on a histogram in a predetermined color space. In the third aspect, the example of the feature amount is given, but the feature amount is not limited to this example in the present invention. For example, as the predetermined color space, in addition to the CIE 1976 L ^* u ^* v ^* color space mentioned in the fourth embodiment below, a uniform color space (L ^* , a ^* , b ^* space), RGB color space, CMYK color Various color spaces such as a space, a YUV space, and an XYZ space may be used.

The automatic tracking device according to a fourth aspect of the present invention is the automatic tracking device according to the third aspect, wherein the feature amount is obtained by normalizing a lightness index histogram and a perceptual chromaticity index histogram in a CIE1976L ^* u ^* v ^* color space. It is a histogram obtained by combining and making it one-dimensional. The fourth aspect is an example of the feature amount, but the feature amount is not limited to this example in the present invention.

  The automatic tracking device according to a fifth aspect of the present invention is the automatic tracking device according to any one of the first to fourth aspects, wherein the preliminary processing means is based on an image captured by the camera at the beginning of tracking of the tracking target. When the similarity between the feature quantities of the regions is higher than a predetermined level according to an index indicating the similarity between the feature quantities of the divided regions of the tracking target region, the single part mode is selected. The multipart mode is selected when the similarity between the feature quantities in the respective regions is low below a predetermined level. The fifth aspect is a specific example of selection between the single-part mode and the multi-part mode, but the present invention is not necessarily limited to this example.

  The automatic tracking device according to a sixth aspect of the present invention is the automatic tracking apparatus according to the fifth aspect, wherein the index is a Bhattacharyya distance between the feature quantities of the divided regions. In the sixth aspect, a specific example of an index indicating the degree of similarity is given. However, the fifth aspect is not limited to this example.

  The automatic tracking device according to a seventh aspect of the present invention is the automatic tracking apparatus according to any one of the first to sixth aspects, wherein the processing means is configured to determine the position and size of the tracking target area based on an image captured by the camera. The position and size of the tracking target region are estimated as a tracking result by a particle filter using a plurality of particles with the thickness as a parameter. Then, for each of the particles, the particle filter applies a reference feature amount based on a feature amount of a past tracking target region (single part feature amount in single-part mode, multi-part feature amount in multi-part mode). The likelihood calculated by the degree of difference indicating the degree of difference in the feature amount (single part feature amount in the single part mode, multi part feature amount in the multi part mode) is used.

  According to the seventh aspect, since the position and size of the tracking target region are estimated as the tracking result by the particle filter, tracking failure has occurred because there are a plurality of solution candidates (a plurality of particles) unlike the correlation calculation. Therefore, the tracking process can be performed with higher accuracy and more accurately with respect to occlusion and complicated backgrounds. As a result, the tracking target can be tracked with higher accuracy and imaged. A deterministic method (for example, template matching) cannot recover from a tracking failure because the solution is uniquely determined. In addition, according to the seventh aspect, since particles using the position and size of the tracking target region as parameters are used, the size of the tracking target can be determined at the same time, and panning, tilting, and zooming can be performed. Even a controlled image can be stably tracked.

  The automatic tracking device according to an eighth aspect of the present invention is the automatic tracking device according to the seventh aspect, wherein the degree of difference of each particle is between the reference feature amount and the single part feature amount of the particle in the single part mode. In addition to the Bhattacharyya distance, it is an average value of the Bhattacharyya distance between each of the reference feature quantity and the multipart feature quantity of the particle in the multipart mode. The eighth aspect is a specific example of the degree of difference, but the seventh aspect is not limited to this example.

  The automatic tracking device according to a ninth aspect of the present invention is the automatic tracking device according to any one of the first to sixth aspects, wherein the processing means sets the position of the tracking target area as a parameter based on an image captured by the camera. The position of the tracking target area is estimated as a part of the tracking result by the first particle filter using the plurality of first particles, and the size of the tracking target area is used as a parameter and is estimated by the first particle filter. The size of the tracking target area is estimated as another part of the tracking result by the second particle filter using the plurality of second particles having the tracking target position. Then, the first particle filter relates to the first particles with reference feature amounts based on past feature amounts of the tracking target region (single-part feature amounts in the single-part mode and multi-part feature amounts in the multi-part mode). The likelihood calculated based on the first degree of difference indicating the degree of difference in the feature quantity of the first particle (single part feature quantity in the single part mode and multipart feature quantity in the multipart mode) is used. is there. The second particle filter performs a reference feature amount based on a feature amount of a past tracking target region (single part feature amount in single part mode, multipart feature amount in multipart mode) with respect to each second particle. Thus, the likelihood calculated by the second degree of difference indicating the degree of difference in the feature amount of the second particle (single part feature amount in the single part mode and multipart feature amount in the multipart mode) is used.

  In the seventh aspect, the position and size of the tracking target area are estimated as tracking results by the particle filter, whereas in the ninth aspect, the position of the tracking target is estimated by the first particle filter. The size of the tracking target is estimated by the second particle filter on the assumption of the estimated position. Therefore, according to the ninth aspect, the same advantages as those of the seventh aspect can be basically obtained, and the processing time can be shortened by reducing the amount of calculation compared to the seventh aspect. . However, in the ninth aspect, there is a possibility that the accuracy of the tracking process of the tracking target may be lower than that in the seventh aspect. Therefore, the ninth aspect is particularly effective when the movement of the tracking target is not so fast and the movement amount of the tracking target on the screen is assumed to be small.

  The automatic tracking device according to a tenth aspect of the present invention is the automatic tracking device according to the ninth aspect, wherein the first difference between the first particles is the reference feature amount and the first particle in the single part mode. The Bhattacharyya distance between the single part feature and the average value of the Bhattacharyya distance between each of the reference feature and the multipart feature of the first particle in the multipart mode. The second degree of difference of the second particle is a Bhattacharyya distance between the reference feature quantity and the single part feature quantity of the second particle in the single-part mode, and the reference feature in the multi-part mode. Is the average value of the Bhattacharyya distance between each of the quantity and the multipart feature of the second particle The The tenth aspect is a specific example of each degree of difference, but the ninth aspect is not limited to this example.

  The automatic tracking device according to an eleventh aspect of the present invention is the automatic tracking device according to any one of the first to tenth aspects, wherein the control unit is configured to allow a predetermined time from the current time based on a result of the tracking processing by the tracking processing unit. Predicting means for predicting the position and size of the tracking target area, and the control means corrects the current pan, tilt and zoom control states for the camera according to the prediction result by the predicting means, and It controls the pan, tilt and zoom of the camera.

  According to the eleventh aspect, since predictive control is introduced, for example, the operation time until the camera responds to the control command and enters the command state is longer than the image processing time. However, it is possible to cope with a sudden change in the tracking target, and the tracking target can be tracked with higher accuracy and imaged. Note that if the camera pan, tilt, and zoom control speeds are too fast, the camera pan, tilt, and zoom will change too rapidly when the observer follows the tracking target. Although it may be uncomfortable and unsuitable for monitoring, a camera with a relatively slow control speed can be used, so the camera's pan, tilt and zoom changes are smoothed for tracking that is more suitable for monitoring. Can be realized.

  The automatic tracking device according to a twelfth aspect of the present invention is the automatic tracking apparatus according to the eleventh aspect, wherein the predicting means predicts the position and size of the tracking target area after a predetermined time has elapsed from the present time by a Kalman filter. In the twelfth aspect, since the Kalman filter is used, the position and size of the tracking target region can be predicted with high accuracy, and as a result, the tracking target can be tracked with higher accuracy and imaged. However, in the eleventh aspect, the prediction means is not limited to that using a Kalman filter.

  ADVANTAGE OF THE INVENTION According to this invention, the automatic tracking apparatus which can track and image a tracking object more accurately can be provided.

  Hereinafter, an automatic tracking device according to the present invention will be described with reference to the drawings.

  [First Embodiment]

  FIG. 1 is a block diagram schematically showing an automatic tracking device according to a first embodiment of the present invention. As shown in FIG. 1, the automatic tracking device according to the present embodiment includes a camera 1 that can control pan, tilt, and zoom, a processing unit 2, a distributor 3, a display / recording control unit 4, and a liquid crystal display. A display unit 5 such as a panel and a recording unit 6 are provided.

  The camera 1 includes a camera body 1 a, a zoom lens 1 b that is mounted on the camera body 1 a and sets a magnification according to a control signal that controls zoom from the processing unit 2, and a pan from the processing unit 2. And a turntable 1c for setting pan and tilt of the camera body 1a according to a control signal for controlling the tilt.

  The distributor 3 distributes and supplies the image signal from the camera 1 to the processing unit 2 and the display / recording control unit 4. The image processing unit 2 processes an image captured by the camera 1 based on the image signal from the camera supplied via the distributor 3, and the camera 1 tracks the tracking target 10 (such as an intruder or an intruding object). The pan, tilt, and zoom of the camera 1 are controlled so as to automatically track and capture an image (see FIG. 2 described later). The display / recording control unit 4 displays the image indicated by the image signal from the camera supplied via the distributor 3 on the display unit 5 or records the image on the recording unit 6. The monitor can monitor the image displayed on the display unit 5. Note that when the monitor does not monitor the image, the image signal from the camera 1 may be directly input to the processing unit 2 without providing the distributor 3.

  FIG. 2 is a diagram schematically illustrating an example of the tracking state of the tracking target 10 by the camera 1. FIG. 2 shows a state in which the visual field of the camera 1 is changed by tracking the tracking target 10 such as an intruder. Actually, as the tracking target 10 moves, the pan and tilt of the turntable 1b change to change the direction of the field of view of the camera 1, and the zoom lens 1b operates to enlarge the field of view of the camera 1. Although reduced, in FIG. 2, illustration of each part of the camera 1 is omitted, and only the field of view of the camera 1 is schematically shown.

  Next, an example of the operation of the processing unit 2 of the automatic tracking device according to the present embodiment will be described with reference to FIGS. FIG. 3 is a schematic flowchart illustrating an example of the operation of the processing unit 2. FIG. 4 is a flowchart showing in detail the tracking target detection process (step S2) in FIG. FIG. 5 is a flowchart showing in detail the mode selection process (step S4) in FIG. 6 and 7 are flowcharts showing in detail the main process (step S5) of the tracking process in FIG. 8 to 11 are flowcharts showing in detail the camera control process (step 7) in FIG.

  As shown in FIG. 3, when the processing unit 2 starts operation, first, the processing unit 2 sets the camera 1 in a preset state (step S1). That is, the processing unit 2 changes the pan, tilt, and zoom of the camera 1 to predetermined pan, tilt, and zoom.

  Next, the processing unit 2 performs a tracking target detection process for detecting a tracking target (moving object detection) in a preset state, that is, in a state where the pan, tilt, and zoom of the camera 1 are fixed (step S2). In addition to applying a commonly used method (S101 to S112 in FIG. 4), this detection may be performed by another sensing device such as a laser radar, or a person displayed on the screen may be detected. The monitoring may be performed by specifying with a pointing device such as a mouse as a specifying means.

  Here, an example of the tracking target detection process (step S2) will be described with reference to FIG. Note that the tracking target detection process (step S2) in FIG. 3 is not limited to the example shown in FIG.

  When the tracking target detection process (step S2) is started, as illustrated in FIG. 4, the processing unit 2 first samples two consecutive images captured by the camera 1 (steps S101 and S102), and these images. The difference image (inter-frame difference image) is generated (step S103).

  Next, the processing unit 2 binarizes the difference image generated in step S103 (step S104). The threshold value used for the binarization may be a fixed threshold value or a variable threshold value represented by discriminant analysis method.

  Subsequently, the processing unit 2 labels the image binarized in step S104 (step S105). Then, the processing unit 2 determines whether there is a labeled item (step S106). If there is no labeled item, the processing unit 2 proceeds to step S112. If there is a labeled item, the processing unit 2 proceeds to step S107. To do.

  In step S107, the processing unit 2 acquires feature amounts for all the labeled items (steps S107 and S108). The feature amount referred to here is necessary for accurately detecting the tracking target 10 such as an area or a circularity.

  Thereafter, the processing unit 2 determines whether there is a candidate for the tracking target 10 from the feature values of all the labels acquired in step S107 (step S109). If it does not exist, the process proceeds to step S112. If it exists, the process proceeds to step S110.

  In step S <b> 110, the processing unit 2 determines the tracking target 10 from the tracking target 10 candidates. At this time, if there is one candidate for the tracking target 10, it is determined as the tracking target 10, and if there are a plurality of candidates for the tracking target 10, it is narrowed down to one according to a predetermined judgment criterion, and is selected as the tracking target 10. Determine as.

  After step S111, the processing unit 2 sets a tracking target detection flag indicating whether the tracking target 10 has been detected to 1 (1 indicates that the tracking target 10 has been detected) (step S111). The tracking target detection process (step S2) is terminated, and the process proceeds to step S3 in FIG.

  In step S112, the processing unit 2 sets the tracking target detection flag to 0 (0 indicates that the tracking target 10 has not been detected). Thereafter, the tracking target detection process (step S2) is terminated, and the process proceeds to step S3 in FIG.

  Referring to FIG. 3 again, in step S3, the processing unit 2 determines whether or not the tracking target 10 is detected in step S2. This determination is made depending on whether the tracking target detection flag is 1 or 0. When the tracking target 10 is detected (when the tracking target detection flag is 1), the process proceeds to step S4. On the other hand, when the tracking target 10 is not detected (when the tracking target detection flag is 0), step S2 is performed. Returning to Fig. 4, the tracking target detection process (step S2) is repeated.

  In step S4, the processing unit 2 performs a mode selection process as a preliminary process of the tracking process for tracking the tracking target based on the image captured by the camera 1. In this mode selection process (step S4), the processing unit 2 performs tracking in the tracking target detection process (step S2) in the beginning of tracking of the tracking target based on the image captured by the camera 1 (in this embodiment, tracking target detection processing (step S2)). Immediately after the target 10 is detected, the feature amount (in this embodiment, obtained from the entire area of the tracking target area that is the area corresponding to the tracking target 10 (the circumscribed rectangular area in this embodiment)) A single-part mode that performs tracking processing using a single-part feature quantity (in this embodiment, a single-part histogram) that is a histogram, or each of the areas to be tracked (in this embodiment, Multipart feature quantities (in this embodiment, multi-part feature quantities) in this embodiment (histograms in this embodiment) respectively obtained from two regions divided into upper and lower parts) Selecting either a multi-part mode for performing tracking processing using part histogram).

Here, an example of the mode selection process (step S4) will be described with reference to FIG. Note that the mode selection process (step S4) in FIG. 3 is not limited to the example shown in FIG. In the present embodiment, the histogram handled as the feature value is a histogram in the CIE1976L ^* u ^* v ^* color space. However, in the present invention, the feature amount may be a histogram of other various color spaces, and is not necessarily limited to the histogram.

  When the mode selection process (step S4) is started, as shown in FIG. 5, the processing unit 2 is first detected by the tracking target detection process (step S2) in FIG. 3 (particularly, step S110 in FIG. 4). The tracking target area (in this embodiment, a circumscribed rectangular area) that is an area corresponding to the tracking target 10 is divided into two vertically (step S201). In the present invention, the division pattern is not necessarily limited to this, and any division method such as right and left division may be employed. Further, the number of divisions is not limited to two. This point will be described later.

  Next, the processing unit 2 sets a count value i indicating a divided region number to 1 in order to assign a number to each divided region divided in step S201 (step S202).

Next, the processing unit 2 generates a brightness index histogram (one-dimensional histogram) of the class number m _L in the CIE 1976 L ^* u ^* v ^* color space for the i-th divided region in the image sampled in step S102 in FIG. (Step S203).

  Subsequently, the processing unit 2 normalizes the brightness index histogram generated in step S203 so that the sum of the frequencies of all classes is 0.5 (step S204). That is, the frequency of each class of the brightness index histogram generated in step S203 is divided by the sum of the frequencies of all classes, and further reduced to ½.

Thereafter, the processing unit 2 perceives the chromaticity index histogram (2) of the class number m _u × m _{v in} the CIE1976L ^* u ^* v ^* color space for the i-th divided region in the image sampled in step S102 in FIG. A dimension histogram is generated (step S205).

  Next, the processing unit 2 normalizes the perceptual chromaticity index histogram generated in step S205 so that the sum of the frequencies of all classes becomes 0.5 (step S206). That is, the frequency of each class in the perceptual chromaticity index histogram generated in step S205 is divided by the sum of the frequencies of all classes, and is further halved.

Next, the processing unit 2 generates a one-dimensional histogram p _i by combining the brightness index histogram normalized in step S204 and the perceptual chromaticity index histogram normalized in step S206 according to the following formula 1. (Step S207).

  Thereafter, the processing unit 2 determines whether or not the current divided region number i is 2, so that the generation of the histogram pi is completed for all the divided regions (two divided regions in the present embodiment). It is determined whether or not (step S208). If not completed, the processing unit 2 sets i to 2 (step S209), and returns to step S203. On the other hand, if completed, the process proceeds to step S210.

In step S210, the processing unit 2 calculates the Bhattacharyya distance d between the histograms p ₁ and p ₂ generated for the two divided regions in step S207 according to the following formula 2.

The Bhattacharyya distance d is an index indicating the similarity between the histograms p ₁ and p _{2. The} higher the similarity is, the smaller the value of the distance d is, while the lower the similarity is, the larger the value of the distance d is. In the present invention, the index indicating the similarity between the histograms p ₁ and p ₂ is not necessarily limited to the Bhattacharyya distance d.

Next, the processing unit 2 determines whether the similarity between the histograms p ₁ and p ₂ is lower than a predetermined value by determining whether or not the Bhattacharyya distance d calculated in step S210 is greater than a preset threshold value T. It is determined whether it is higher than a predetermined value (step S211). If the processing unit 2 is larger than the threshold T (if the similarity between the histograms p ₁ and p ₂ is lower than a predetermined value), the processing unit 2 sets the mode flag to 0 (0 indicates that the multipart mode is selected). (Step S212) On the other hand, if it is smaller than the threshold value T (if the similarity between the histograms p ₁ and p ₂ is higher than a predetermined value), the mode flag is set to 1 (1 indicates that the single part mode is selected). (Step S213). After step S212 and after step S213, the mode selection process (step S4) is terminated, and the process proceeds to step S5 in FIG.

Referring to FIG. 3 again, in step S5, the processing unit 2 performs the main process of the tracking process that tracks the tracking target based on the image captured by the camera 1 in the mode selected in the mode selection process (step S4). I do. In the present embodiment, in the main process (step S5) of the tracking process, the processing unit 2 uses a plurality of particles based on the image captured by the camera 1 as parameters for the position and size of the tracking target area. The position and size of the tracking target area are estimated as a tracking result by the filter. The particle filter refers to each particle based on a past feature amount of the tracking target region (single part histogram as a single part feature amount in the single part mode, multi part histogram as a multipart feature amount in the multipart mode). The feature quantity of the particle (single part histogram as a single part feature quantity in single part mode, multi part histogram as a multi part feature quantity in multipart mode) differs from the feature quantity (reference histogram in this embodiment) The likelihood π _t ⁽ⁿ⁾ calculated based on the difference d ⁽ⁿ⁾ indicating the degree to be used is used.

  Here, an example of the main process (step S5) of the tracking process will be described with reference to FIGS. 3 is not limited to the examples shown in FIGS. 6 and 7.

  When the main process (step S5) of the tracking process is started, as shown in FIG. 6, the processing unit 2 first samples one image (step S300).

  Next, as illustrated in FIG. 12, the processing unit 2 sets a tracking target region in the region R as a search range of the tracking target 10 using a region R having a predetermined width and a predetermined height in the image. (In this embodiment, N samples (particles) PS1 to PS5,..., PSn,..., PSN with the position (for example, the center of gravity) and the size (for example, the vertical or horizontal dimension) of the parameters as parameters. Disperse (step S301). The position of each sample is arbitrary as long as it is within the region R, and is initially randomly distributed. The aspect ratio of each sample is the original tracking target area (the tracking target area (the circumscribed rectangular area in the present embodiment) corresponding to the tracking target 10 detected in step S110 in FIG. 4) and the aspect ratio. Are the same. The size of each sample is arbitrary and is initially set at random. In the process of step S301 in the main process (step S5) performed first after the tracking target detection process (step S2) in FIG. 3, N samples are dispersed, but the second and subsequent main processes (step S5). In step S301 in step S301, an insufficient amount obtained by subtracting the number of samples remaining in the previous main process (step S5) from N pieces is added.

  FIG. 12 is a diagram schematically illustrating an example of the distribution state of the sample and the region R that forms the search range of the tracking target 10. In FIG. 12, a hatched rectangular area ES indicates an example of the tracking target area estimated in step S316 described later.

  Next, the processing unit 2 sets a count value n indicating a sample number to 1 in order to assign a number to N samples (step S302).

  Next, the processing unit 2 determines whether or not the mode flag set to the latest in the mode selection process (step S4) in FIG. 3 is 0, so that the currently selected mode is the multipart mode. It is determined whether there is a single part mode or not (step S303). If the multi-part mode is selected (if the mode flag is 0), the processing unit 2 divides the n-th sample area in the same manner as in step S201 in FIG. 2) (step S304), the process proceeds to step S305. On the other hand, if the single part mode is selected (if the mode flag is 1), the process proceeds to step S305 without passing through step S304.

  In step S305, the processing unit 2 sets a count value i indicating an area number to 1. In the multi-part mode, the sample area is divided into two, so i becomes 1, 2, but in the single-part mode, the sample area is not divided, so i is only 1.

Next, the processing unit 2 performs the same processing as in steps S203 to S207 on the i-th region of the n-th sample in the image sampled most recently in step S300, so that the CIE 1976 L ^* u ^* v ^* color space. Histogram q _i obtained by combining the normalized lightness index histogram and perceptual chromaticity index histogram in FIG. 1 to obtain a one-dimensional histogram (i-th region histogram of the sample corresponding to the histogram p _i shown in Equation 1) Generate.

Then, the processing unit 2 in accordance with the number 3, below, between the histogram _{q i} of the histogram _{p i} and the sample of the reference region area, calculates the Bhattacharyya distance _d ^{i (n) (step} S307). Here, the histogram p _i of the reference region may be referred to as reference histogram p _i or reference data, and is a step in the main process (step S5) performed first after the tracking target detection process (step S2) in FIG. in the process of S307 is a histogram _{p i} generated in S207 in FIG. 5, in the processing in step S307 in the second or subsequent the process (step S5), at step S317 in the previous present process (step S5) updated histogram p _{t (provided} that relates the i-th region) is.

  Subsequently, similarly to step S303, the processing unit 2 determines whether the currently selected mode is the multipart mode or the single part mode (step S308). If the multi-part mode is selected (if the mode flag is 0), the process proceeds to step S309. If the single-part mode is selected (if the mode flag is 1), the process proceeds to step S312.

In step S309, the processing unit 2 determines whether or not the current area number i is 2, so that the Bhattacharyya distance d _i ⁽ⁿ ) for all areas (two divided areas in the present embodiment). calculation ^of) determines whether or not it is completed. If not completed, the processing unit 2 sets i to 2 (step S310) and returns to step S306. On the other hand, if completed, the process proceeds to step S311.

In step S311, the processing unit 2 calculates the difference d ⁽ⁿ⁾ of the n-th sample as an average value of the Bhattacharyya distances d _i ⁽ⁿ⁾ of the respective divided regions sequentially calculated in step S307 according to the following equation 4. Calculate and move to step S313.

On the other hand, in the single part mode, in step S312, the processing unit 2 sets the Bhattacharyya distance d _i ⁽ⁿ⁾ obtained in step S307 as the difference d ⁽ⁿ⁾ of the ^nth sample as it is, and proceeds to step S313.

In step S313, the processing unit 2 calculates the likelihood π _t ⁽ⁿ⁾ from the difference d ⁽ⁿ⁾ obtained in step S311 or S312 according to the following formula 5. Here, the likelihood is assumed to follow a Gaussian distribution.

In Equation 5, N is the number of particles (number of samples). In Equation 5, σ ² is normally a variance of Gaussian distribution, but in this calculation, it is a value set in advance.

Thereafter, the processing unit 2, by current sample number n is determined whether the N, for all samples, it is determined whether the calculation of the likelihood [pi _{t ⁽ⁿ⁾} has been completed (step S314). If not completed, the processing unit 2 increments the sample number n by 1 (step S315) and returns to step S303. On the other hand, if completed, the process proceeds to step S316.

In step S316, the processing unit 2, the calculated likelihood [pi _{t ⁽ⁿ⁾} and from each sample position (gravity center position) and size at step S313, in accordance with the number 6 below, tracking target area is a result of estimation of the state A weighted average value E [S _t ] is obtained as an estimation result (that is, a tracking result) of the position and size. The position and size of the tracking target area represented by the weighted average value E [S _t ] is the position and size of the tracking target area that is the tracking result.

Thereafter, the processing unit 2 updates the reference data according to the following equation 7 based on the estimation result obtained in step S316 (step S317). In Equation 7, a histogram _pt-1 indicates a histogram related to the tracking target region estimated at time t-1, and is obtained as an estimation result in Step S315 (Equation 6) in the image sampled most recently in Step S300. The tracking target area is processed in the same manner as steps S203 to S207 to combine the normalized brightness index histogram and perceptual chromaticity index histogram in the CIE1976L ^* u ^* v ^* color space to make one dimension. This is a histogram obtained. In Expression 7, q _t−1 represents a histogram related to the tracking target region before estimation at time t−1.

Next, the processing unit 2 sorts the likelihoods π _t ⁽ⁿ⁾ of all the samples calculated in step S313 in descending order of the values (step S318).

  Next, the processing unit 2 determines whether or not the maximum likelihood value obtained as a result of sorting the likelihood values in step S318 is greater than a preset threshold value V (step S319). If it is larger than the threshold value V, it is determined that the tracking is successful, the tracking result flag is set to 1 (S320), and the process proceeds to step S322. On the other hand, if it is smaller than the threshold value V, it is determined that the tracking has failed, the tracking result flag is set to 0 (S321), and the process proceeds to step S322.

The processing unit 2 repeats the processes of steps S322 to S324 for each sample, and when the processes of steps S322 to S324 are completed for all samples (YES in step S325), the process proceeds to step S326. In step S322, the processing unit 2 determines whether the likelihood π _t ^{(n) of the} sample is larger than a preset threshold value Π. If it is larger than the threshold value Π, the process proceeds to S323, and if it is smaller, the process proceeds to S324. In step S323, the processing unit 2 divides the sample to form a sample for the next time (that is, in the next frame) in accordance with a known method of a particle filter. In step S324, the processing unit 2 causes the sample to disappear so as not to constitute the next sample (that is, in the next frame).

In step S326, the processing unit 2 determines whether or not the number of samples remaining as a result of the division in step S323 or the disappearance of step S324 is N or less for each sample. If the number is N or less, the tracking process is terminated (step S5), and the process proceeds to step S6 in FIG. On the other hand, when there are more than N, the processing unit 2 eliminates the ones that are split in order from the one with the smallest likelihood π _t ⁽ⁿ⁾ (step S327). As a result, the number of remaining samples is N or N-1. After step S327, the tracking process is terminated (step S5), and the process proceeds to step S6 in FIG.

  Referring to FIG. 3 again, in step S6, the processing unit 2 determines whether the current tracking result flag in step S5 is successful by determining whether the current tracking result flag is 1 or not. . If the tracking is successful (if the tracking result flag is 1), the process proceeds to step S7. On the other hand, if tracking fails (if the tracking result flag is 0), the process proceeds to step S8. In step S8, the processing unit 2 determines whether or not the follow-up failure state continues for a certain period of time. If it does not continue for a certain time, the process returns to the main process (step S5) of the tracking process, and the process is repeated. If it continues for a certain period of time, it is considered that the tracking process is unlikely to succeed, and the process returns to step S1 (preset state).

If it is determined in step S6 that the tracking is successful (the tracking result flag is 1), the processing unit 2 determines the tracking result obtained in the main processing (step S5) of the tracking processing, that is, the step in FIG. Camera control processing for controlling pan, tilt, and zoom of the camera 1 is performed so that the camera 1 tracks the tracking target according to the position and size of the tracking target region estimated in S316. In this camera control process, the processing unit 2 predicts the position and size of the tracking target area after the elapse of a predetermined time (after n _f frames) based on the tracking result, and in accordance with the prediction result, the camera 2 The current pan / tilt / zoom control state for 1 is corrected to control the pan / tilt / zoom of the camera 1. Here, in the present embodiment, the position and size of the tracking target region are predicted by a Kalman filter.

  Here, an example of the camera control process (step S7) will be described with reference to FIGS. Note that the camera control process (step S7) in FIG. 3 is not limited to the examples shown in FIGS.

When the camera control process (step S7) is started, as shown in FIG. 8, the processing unit 2 first obtains the result of the tracking process (step S316 in FIG. 6) as information used in the camera control process. The obtained weighted average value E [S _t ], that is, the position and size of the tracking target area estimated in step S316 is acquired (step S501). Since the tracking result is originally possessed by the processing unit 2, the acquisition operation is originally unnecessary, but step S 501 is inserted here for easy understanding.

  Next, the processing unit 2 acquires a pan control flag, a tilt control flag, and a zoom control flag as information indicating the respective pan, tilt, and zoom control states of the camera 1 (step S502). In the present embodiment, the camera 1 performs control operations after receiving control commands from the processing unit 2 for each of pan, tilt, and zoom, and returns a control completion signal to the processing unit 2 when the control operations are completed. It has become. The processing unit 2 sets a pan control flag to 1 when a pan control command is given to the camera 1, and resets the pan control flag to 0 by an interrupt process when a pan control completion signal is received from the camera 1. Further, the processing unit 2 sets the tilt control flag to 1 when giving a tilt control command to the camera 1, and resets the tilt control flag to 0 by interruption processing when receiving a tilt control completion signal from the camera 1. Further, the processing unit 2 sets the zoom control flag to 1 when giving a zoom control command to the camera 1, and resets the zoom control flag to 0 by interrupt processing when receiving a zoom control completion signal from the camera 1. As described above, when each control flag of pan, tilt, and zoom is 1, it indicates that the corresponding operation is being controlled, and when it is 0, it indicates that the corresponding operation is being stopped. As can be understood from the above description, since the processing unit 2 originally has the pan control flag, the tilt control flag, and the zoom control flag, the acquisition operation is originally unnecessary, but here, for easy understanding. This step S502 is inserted.

  Next, the processing unit 2 determines whether all control flags (pan control flag, tilt control flag, zoom control flag) are 0 (step S503). The process proceeds to S519, and if any one or more control flags is 1, the process proceeds to Step S504.

  In step S504, the processing unit 2 compares the tracking result (particularly the position of the tracking target area) acquired in step S501 with the tracking result (particularly the position of the tracking target area) acquired in the previous frame. Determine if you are away from the center. If it is away, the process proceeds to step S506, and if it is approaching, the process proceeds to step S505.

  If it is away from the center of the image, it is determined that the control from the previous frame is not suitable, and the processing unit 2 resets the pan, tilt, and zoom control flags to 0 in step S506, and further performs camera control. Is stopped (step S507).

  In step S505, based on the tracking result acquired in step S501, the processing unit 2 changes the direction of travel and the size of the tracking target up to that point (whether it gradually increases or decreases) Determine if has changed. If there is no change in both the traveling direction and the size, the process proceeds to step S508, and if either has changed, the process proceeds to step S506.

  In step S508, the processing unit 2 determines whether or not the zoom control flag is 1, thereby determining whether or not zoom control is currently in progress. If zoom control is being performed, the process proceeds to step S509. If zoom control is not being performed, the process proceeds to step S511.

  In step S509, the processing unit 2 determines whether the tracking result acquired in step S501 has reached a target size range that has already been set in advance. If it has reached, the process proceeds to S510, and if not, the process proceeds to Step S511.

  In step S510, the processing unit 2 sets the zoom control flag to 0. This is because the zoom control is preferably stopped at that time because the target size range set in advance has been reached.

  In step S511, the processing unit 2 determines whether or not the pan control flag is 1, thereby determining whether or not the pan control is currently in progress. If pan control is being performed, the process proceeds to step S512, and if pan control is not being performed, the process proceeds to step S514.

  In step S512, the processing unit 2 determines whether or not the tracking result acquired in step S501 has reached the target horizontal position range that has been set in advance. If it has reached, the process proceeds to step S513, and if not, the process proceeds to S514.

  In step S513, the processing unit 2 sets the pan control flag to 0. This is because it is preferable to stop the pan control at that time point because the target horizontal position range set in advance has been reached.

  In step S514, it is determined whether or not the tilt control flag is 1, thereby determining whether or not the tilt control is currently in progress. If the tilt control is being performed, the process proceeds to step S515, and if the tilt control is not being performed, the process proceeds to S517.

  In step S515, the processing unit 2 determines whether or not the tracking result acquired in S501 has reached the target vertical position range that has been set in advance. If it has reached, the process proceeds to S516, and if not, the process proceeds to S517.

  In step S516, the processing unit 2 sets the tilt control flag to 0. This is because the tilt control is preferably stopped at that time because the target vertical position range has been set in advance.

  In step S517, the processing unit 2 determines whether any of the pan control flag, the tilt control flag, and the zoom control flag has been changed. If any one of steps S510, S513, and S516 is performed, the process proceeds to S518. If none is performed, the process proceeds to S519 in order to continue the control.

  In step S518, the processing unit 2 changes the control. This is because reaching step S518 indicates that there is a difference between the control of the pan, tilt and zoom of the camera 1 based on the previous prediction and the actual movement of the tracking target.

  In step S <b> 519, the processing unit 2 acquires the current posture of the camera 1 (pan, tilt, zoom position) from the camera 1.

Then, the processing unit 2, a tracking result obtained in step S501, from the results of the previous tracking process to predict the position and size of the tracking target area after n _f frame (step S520). For example, in the case of an NTSC signal, “after n _f frames” corresponds to (n _f / 30) seconds later.

Here, a Kalman filter is used to predict the position and size of the tracking target region after n _f frames.

  Here, the Kalman filter is configured on the assumption that the change in the position and size of the tracking target is constant and the change is smooth. In the Kalman filter, even if the change in the state of the tracking target is not strictly applied to the set model, there are many cases where it can be applied approximately because there is an error term.

The state variable vector x _k at time k is defined as in the following formula 8.

Here, y _k is the vertical coordinate and velocity, s _k and top to dot x _k marked with dots on the x _k is denoted by the dot target rectangle center horizontal coordinates and velocity of the upper and y _k in the image Sk with a _represents a size (product of a horizontal width and a vertical width of a rectangle) and its change.

  A system equation in consideration of the control of the state vector, the camera 1, and the error is defined by the following equation (9).

  In Equation 9, A is a constant matrix shown in Equation 10 below.

The horizontal width of the input image is W _src , the vertical width is H _src , the horizontal angle of view at time k is θ _k , the vertical angle of view is φ _k , the pan angle speed of the camera is P _k , and the tilt angular speed is up When T _k marked with dots, horizontal and vertical variation of the pixel by the pan-tilt control at each time it is represented by the respective by the number 11 and number 12 below.

  If the change in the angle of view by the zoom operation is λ, λ can be expressed by the following equation (13).

From the above elements, the control vector u _k is given by the number 14 below. u _k is a control vector which represents the variation of the image by the camera control.

w _k is a system error, and follows a normal white process with a covariance matrix Q _k and an average of 0, as shown in Equation 15 below.

  Here, the position and size of the rectangle surrounding the tracking target obtained from the tracking process described above are taken as observation values, and the observation vector at time k is defined as in the following equation (16).

  The observation equation is expressed by the following Expression 17.

Here, H is a constant matrix shown in Equation 18 below. Further, as shown in Equation 19, the observation error v _k follows a normal white process having a covariance matrix R _k and an average of 0.

  In the Kalman filter, the estimated amount of the current time is estimated using the observed amount of the current time and the state amount of the previous period. The state of the current time k of the system is represented by two variables represented by the following equation (20). In addition, in this specification, the code | symbol ^ attached | subjected on the symbol means the estimated value.

  The Kalman filter performs two procedures, prediction and update, to advance one time step. In the prediction procedure, the estimated state of the current time is calculated from the estimated state of the previous time. In the update, a more accurate state is estimated by using the observation at the current time and correcting the estimated value.

  Regarding the prediction, the estimated value of the current time is expressed by the following formula 21, and the covariance matrix of the error of the current time is expressed by the following formula 22.

  Regarding the update, the Kalman filter calculates a Kalman gain that minimizes the estimated value of the error after the update using the following equations 23 to 27, and updates the state.

  With the above calculation, it is possible to estimate the state quantity at the current time in consideration of the error.

Here, setting of initial conditions of the Kalman filter will be described. Assuming that the center coordinates of the tracking target rectangle at the start of tracking are (x ₀ , y ₀ ) and the size is s ₀ , the initial value of the state is as shown in Equation 28 below. Here, the speed is zero.

  If there is an error in the initial conditions, an error covariance matrix is given as in Equation 29 below.

  The characteristic of the filter is the variance ratio between the system error and the observation error. The larger the variance ratio, the more accurate the estimated value after filtering is, but the more sensitive the error is. Follow-up performance against is reduced. In the present embodiment, it is possible to quickly cope with a change in the moving direction of the target person, a stoppage, and the like, but a dispersion ratio that is not easily affected by an error included in the tracking result is empirically used.

  The Kalman filter used in the process of step S520 in FIG. 9 has been described above.

After step S520, the processing unit 2 calculates the horizontal angle of view after n _f frames from the size of the tracking target after n _f frames obtained by the prediction at step S520 and the size of the tracking target to be targeted s _i. A vertical angle of view is calculated (step S521). However, the following Equation 30 is used to calculate the horizontal angle of view.

  Next, the processing unit 2 determines whether or not the horizontal angle of view calculated in step S521 has reached the zoom limit (step S522). If the zoom limit has been reached, the process proceeds to step S549. However, when the limit is reached in zooming out, the process proceeds to step S523. If neither zoom-in nor zoom-out has reached the limit, the process proceeds to step S523.

  In step S523, the processing unit 2 compares the current horizontal angle of view from the result obtained in S521, and determines whether or not the zoom control amount is smaller than a predetermined value. If it is smaller, the process proceeds to S524, and if it is larger, the process proceeds to S526.

  In step S524, the processing unit 2 sets the zoom control speed to zero. Subsequently, the processing unit 2 sets the zoom control flag to 0 (step S525), and proceeds to step S530. Thus, zoom control is not performed. Thus, when the control amount of zoom is small, zoom control is not performed. This is because when the zoom control amount is small, if the zoom control is performed, a fine movement is caused, which may cause discomfort to the monitor who monitors the display unit 5.

In step S526, the zoom control speed Z _speed is calculated by the following _equation (31). Here, f _r indicates a frame rate (in the case of NTSC, f _r = 30). Z _pt is the zoom position when the horizontal angle of view is time t.

Thereafter, the processing unit 2 determines whether or not the difference between the zoom control speed Z _speed calculated in step S526 and the _speed currently being controlled is greater than a predetermined value (step S527). If the difference is large, the process proceeds to step S528, and if the difference is small, the process proceeds to step S529. Again, if the difference is not greater than a certain value, the zoom control speed is not changed because the frequent change of the zoom control speed may cause discomfort for the observer who visually monitors the display unit 5. This is to avoid this. In addition, the tracking target only needs to be near the center of the image, and the purpose is not to maintain a state in which there is no deviation from the center of the image.

  In step S528, the processing unit 2 changes the zoom control speed value calculated in step S526. Thereafter, the process proceeds to step S529.

  In step S529, the processing unit 2 sets the zoom control flag to 1. Thereafter, the process proceeds to step S530.

  In step S530, the processing unit 2 determines whether or not the pan control amount is smaller than a predetermined value from the result obtained by the prediction in step S520. If so, the process proceeds to step S531, and if greater, the process proceeds to step S533.

  In step S531, the processing unit 2 sets the pan control speed to zero. Subsequently, the processing unit 2 sets the pan control flag to 0 (step S532), and proceeds to step S538. Thus, pan control is not performed. Thus, when the pan control amount is small, pan control is not performed. This is because if the pan control amount is small, if the pan control is performed, a fine movement occurs, which may cause discomfort to the monitor who monitors the display unit 5.

In step S <b> 533, the processing unit 2 calculates the pan control speed P _speed by the following equation 32. Here, W _src is the horizontal width of the input image.

Thereafter, the processing unit 2 assumes that the pan is controlled at the pan control speed P _speed calculated in step S533, and when the time corresponding to n _f frames has elapsed (after n _f / f _r seconds), The position is calculated, and it is determined whether or not the value reaches the panning limit or exceeds the panning limit (step S534). If the value reaches the panning limit or exceeds the panning limit, the process proceeds to step S549, and if not, the process proceeds to step S535.

In step S535, the processing unit 2 determines whether or not the difference between the pan control speed P _speed calculated in step S533 and the _speed currently being controlled is greater than a predetermined value. If the difference is large, the process proceeds to step S536, and if the difference is small, the process proceeds to step S537.

  In step S536, the processing unit 2 changes the pan control speed value calculated in step S536. Thereafter, the process proceeds to step S537.

  In step S537, the processing unit 2 sets the pan control flag to 1. Thereafter, the process proceeds to step S538.

  In step S538, the processing unit 2 determines whether the tilt control amount is smaller than a predetermined value from the result obtained in the prediction in step S520. If it is smaller, the process proceeds to step S539, and if it is larger, the process proceeds to step S541.

  In step S539, the processing unit 2 sets the tilt control speed to zero. Subsequently, the processing unit 2 sets the tilt control flag to 0 (step S540), and proceeds to step S546. Thus, tilt control is not performed. Thus, when the amount of tilt control is small, tilt control is not performed. This is because when the tilt control amount is small, if the tilt control is performed, a fine movement is caused, which may cause discomfort to the monitor who monitors the display unit 5.

In step S <b> 541, the processing unit 2 calculates the tilt control speed T _speed by the following equation 33. Here, H _src is the height (length in the vertical direction) of the input image.

Thereafter, the processing unit 2 assumes that the tilt is controlled at the tilt control speed T _speed calculated in S541, and when the time corresponding to n _f frames has elapsed (after n _f / f _r seconds), The position is calculated, and it is determined whether or not the value reaches the tilt limit or exceeds the tilt limit (step S542). If the value reaches the tilt limit or exceeds the tilt limit, the process proceeds to step S549, and if not, the process proceeds to step S543.

In step S543, the processing unit 2, the difference between the control speed T _speed of the calculated tilt as the speed of the current control in Step S541 is equal to or greater than a predetermined value. If the difference is large, the process proceeds to S544, and if the difference is small, the process proceeds to S545.

  In step S544, the processing unit 2 changes the value of the tilt control speed calculated in step S541. Thereafter, the process proceeds to step S545.

  In step S545, the processing unit 2 sets the tilt control flag to 1. Thereafter, the process proceeds to step S546.

  In step S546, the processing unit 2 determines whether or not all control flags (pan control flag, tilt control flag, zoom control flag) are zero. If all the control flags are 0, control is not performed, and the process proceeds to step S548 without passing through step S547. If any one or more control flags are 1, it will transfer to step S547.

In step S547, the processing unit 2 gives a control command to the camera 1 so as to perform control according to the control flag that is 1 and the control speed corresponding thereto. Its control is performed n _{f /} f _r seconds depending on the speed, may no wait for the control to complete the process proceeds to step S548 exits this loop. In the present embodiment, a control command is not given to the camera 1 by the control flag itself, but a control command according to the control flag and the control speed is given to the camera 1 by the operation as in step S547. ing.

  In step S548, the processing unit 2 sets the tracking limit flag to 0 (0 indicates that none of pan, tilt, and zoom can reach the limit). This is because step S548 is reached when none of pan, tilt, and zoom can reach the limit. After step S548, the camera control process (step S7) is terminated, and the process proceeds to step S9 in FIG.

  In step S549, the processing unit 2 sets the tracking limit flag to 1 (1 indicates that any of pan, tilt, and zoom may reach the limit). This is because step S549 is reached when there is a possibility that at least one of pan, tilt, and zoom may reach the limit. After step S549, in step S550, the processing unit 2 stops the control if the posture control of the camera 1 is being performed. After step S550, the camera control process (step S7) is terminated, and the process proceeds to step S9 in FIG.

  Referring to FIG. 3 again, in step S9, the processing unit 2 determines whether or not the tracking limit flag is 1. If the tracking limit flag is 1, it is determined that it is difficult to continue tracking of the tracking target, and the process returns to step S1 (preset state). If the tracking limit flag is 0, it is possible to continue tracking the tracking target, so the process returns to step S5 to continue tracking the tracking target.

  According to the present embodiment, either the single-part mode or the multi-part mode is selected based on the image captured by the camera 1, and the tracking process is performed in the selected mode. Therefore, the single-part mode is always performed. Compared to the case, since tracking processing is performed based on more information, for example, it is possible to perform tracking processing with higher accuracy even for tracking targets where it is difficult to distinguish between people or between a person and the background, As a result, the tracking target can be tracked with higher accuracy and imaged.

  The inventor made a prototype of the automatic tracking device according to the present embodiment. In addition, as an automatic tracking device according to the first comparative example, an automatic tracking device in which the automatic tracking device according to the present embodiment is modified so as to always perform only the single part mode was prototyped. Furthermore, as an automatic tracking device according to the second comparative example, an automatic tracking device in which the automatic tracking device according to the present embodiment is modified so as to always perform only the multipart histogram is experimentally manufactured. Then, the number of frames in which tracking of the tracking target was successfully performed for the same video with these automatic tracking devices was obtained. And the kind of the image was changed variously. As a result, in the first comparative example and the second comparative example, depending on the type of video, the first comparative example may have more successfully tracked tracking targets than the second comparative example. The number of frames in which the tracking target was successfully tracked in the second comparative example was larger than that in the first comparative example. On the other hand, the number of frames successfully tracked by the prototype device corresponding to the automatic tracking device according to this embodiment is tracked successfully in the first comparative example and the second comparative example for any type of video. The number of frames that were successfully tracked in the comparative example with the larger number of frames was almost the same. Therefore, it was confirmed that the prototype device corresponding to the automatic tracking device according to the present embodiment can track the tracking target with higher accuracy than any of the first and second comparative examples.

  Further, in the present embodiment, since a histogram is adopted as the feature amount, the histogram is resistant to changes in the size and shape of the tracking target, and is less susceptible to image fluctuations caused by control of the camera 1.

  Furthermore, according to the present embodiment, since the position and size of the tracking target region are estimated as a tracking result by the particle filter, there is a possibility of recovery from the tracking failure because there are a plurality of solution candidates (a plurality of particles). It becomes high and is strong against occlusion and complicated background, and can perform tracking processing with higher accuracy. As a result, the tracking target can be tracked with higher accuracy and imaged. Furthermore, according to the present embodiment, since particles (samples) using the position and size of the tracking target region as parameters are used, the size of the tracking target can be determined at the same time, and pan, tilt Thus, stable tracking is possible even for an image whose zoom is controlled.

  Further, according to the present embodiment, since predictive control is introduced in the camera control process, for example, the operation time until the camera 1 responds to the control command and enters the command state is the image processing time. Even if it is longer than this, it is possible to cope with a sudden change in the tracking target, and it is possible to track and image the tracking target with higher accuracy. Note that if the camera pan, tilt, and zoom control speeds are too fast, the camera pan, tilt, and zoom will change too rapidly when the observer follows the tracking target. Although it is uncomfortable and not suitable for monitoring, a camera with a relatively slow control speed can be used as the camera 1, so that the panning, tilting and zooming changes of the camera 1 are smoothed and more suitable for monitoring. Tracking can be realized.

  Furthermore, according to the present embodiment, since the Kalman filter is used for such prediction, the position and size of the tracking target region can be predicted with high accuracy, and thus the tracking target can be tracked with higher accuracy. Can be taken.

In the present embodiment, a histogram obtained by combining the normalized values of the lightness index histogram and the perceptual chromaticity index histogram in the CIE1976L ^* u ^* v ^* color space and making them one-dimensional is always used as the feature amount. It has been. However, in the present invention, for example, as a feature quantity, only the brightness index histogram (in this case, the brightness index histogram corresponding to the decrease in the number of classes of the perceptual chromaticity index histogram, depending on the environment, time zone (daytime, nighttime, etc.), etc. The combination of the brightness index histogram (one-dimensional) and the perceptual chromaticity index histogram (two-dimensional) may be automatically and selectively used.

By the way, in the present embodiment, the tracking target area is vertically divided into two, but as described above, the number of divisions and the like is not limited to two. For example, it may be further divided into two as required. For example, a division method as shown in FIG. 13 is given in advance. First, divide the area _{p 0} in the region _{p 11, p 12,} and calculates the Bhattacharyya distance for these areas _{p 11, p 12.} If this division is determined to be _{appropriate, p 11} to _{p 21} and _{p 22, p 12} is divided into _{p 31} and _{p 32,} likewise calculates the Bhattacharyya distance. If p ₂₁ and p ₂₂ are determined in advance not to be divided any more, they can be stopped here. Even if p ₃₂ can be further divided into p ₄₁ and p ₄₂ , if it is determined that p ₃₁ cannot be further divided from the value of the Bhattacharyya distance, only p ₃₂ may be divided. In this way, the division method may be changed each time using a predetermined division method. FIG. 13 is a diagram schematically illustrating an example of the division pattern of the region.

  In the present embodiment, as described above, the single part mode and the multipart histogram with the number of divisions of 2 are selected and used. For example, in the case of FIG. The part (particularly, part of FIG. 6) is as shown in FIG. 14, and the difference calculation is as follows.

The region constituting the reference region p ₀ is composed of five regions p ₂₁ , p ₂₂ , p ₃₁ , p ₄₁ , and p ₄₂ from FIG. If the Bhattacharyya distance of each region is d ₁ ⁽ⁿ⁾ , d ₂ ⁽ⁿ⁾ ,..., D ₅ ⁽ⁿ⁾ , the following equation 34 is obtained.

The degree of difference d ⁽ⁿ⁾ is given by the following formula 35 from the five Bhattacharyya distances shown in formula 34.

  [Second Embodiment]

  FIGS. 15 to 17 are flowcharts showing in detail the main process (step S5) of the tracking process in FIG. 3 performed by the processing unit 2 of the automatic tracking device according to the second embodiment of the present invention.

  This embodiment differs from the first embodiment in that the processing unit 2 performs the processing shown in FIGS. 6 and 7 as the main processing (step S5) of the tracking processing in FIG. Only the processing shown in FIGS. 15 to 17 is performed.

  In the processing shown in FIGS. 6 and 7, the aspect ratio of the region is constant, but the particles (samples) are dispersed at any position and size. In this case, if the position is k pattern and the size is j pattern for each position, the number of particles is k × j. If the flowcharts shown in FIGS. 6 and 7 are not always appropriate due to the processing capacity of the computer, etc., the position is estimated with k pattern particles as shown in the flowcharts of FIGS. Even in the process of estimating with particles having a size j pattern, the main process (step S5) of the tracking process in FIG. 3 can be realized. At this time, the total number of particles is k + j, and the amount of calculation can be reduced to shorten the processing time.

  Although the method of estimating the position and size of the tracking target area in two stages as in the present embodiment has the advantage of shortening the processing time, the tracking process is more efficient than in the first embodiment. Accuracy may be reduced. Therefore, this embodiment is particularly effective when the movement of the tracking target is not so fast and the movement amount of the tracking target on the screen is assumed to be small. If the amount of movement is large, there is a possibility that the change in size may become large, and therefore, the processing divided into two stages as shown in FIGS. 15 to 17 is not suitable.

  The processing shown in FIGS. 15 to 17 will be described below.

  When the tracking process is started (step S5), as shown in FIG. 15, the processing unit 2 first samples one image (step S400).

Next, the processing unit 2 uses a region R having a predetermined height and a predetermined height in the image as a search range of the tracking target 10, and includes the tracking target region (in this embodiment, in the present embodiment). position of the rectangle) (e.g., the _{N p} samples that the center of gravity) as parameters (particle) rose seeded (step S401). The position of each sample is arbitrary as long as it is within the region R, and is initially randomly distributed. The size of each sample is the same as the reference area. Here, the reference region is the tracking target detected in step S110 in FIG. 4 in the process of step S307 in the main process (step S5) performed first after the tracking target detection process (step S2) in FIG. 10 is a tracking target area (in the present embodiment, a circumscribed rectangular area). In the process of step S401 in the second and subsequent main processes (step S5), the reference area is the tracking target area estimated in step S442 in the previous main process (step S5). Incidentally, a tracking target detecting process in FIG. 3 in the process of step S401 in (step S2) initially present process performed after (step S5), and sample N _p number rose sow but second and subsequent of the present process (step in the process of step S401 is in S5), dusted rose by adding the shortage minus the number of remaining samples in the previous present process from N _p pieces (step S5).

Next, processing unit 2, to assign a number to N _p samples, sets the count value k, which means sample number to 1 (step S402).

  Next, the processing unit 2 determines whether or not the mode flag set to the latest in the mode selection process (step S4) in FIG. 3 is 0, so that the currently selected mode is the multipart mode. It is determined whether there is a single part mode or not (step S403). If the multi-part mode is selected (if the mode flag is 0), the processing unit 2 divides the k-th sample area in the same manner as in step S201 in FIG. 2) (step S404), the process proceeds to step S405. On the other hand, if the single part mode is selected (if the mode flag is 1), the process proceeds to step S405 without passing through step S404.

  In step S405, the processing unit 2 sets a count value i indicating an area number to 1. In the multi-part mode, the sample area is divided into two, so i becomes 1, 2, but in the single-part mode, the sample area is not divided, so i is only 1.

Then, the processing unit 2, for the i-th region of the k-th sample in the sampled image to date in step S400, by performing the same processes as in steps S203~S207, CIE1976L ^* u ^* v ^* color space the _(histogram of the i-th area of the sample corresponding to the histogram p _i indicated by the number 1) histogram q _pi that lightness index histogram and sensory chromaticity index histograms respectively obtained by one-dimensional by combining those normalized in Generate (step S406).

Next, the processing unit 2 calculates a Bhattacharyya distance d _pi ^(k) between the histogram p _{i of the} reference region and the histogram q _pi of the sample region according to the following Expression 36 (step S407). Here, the histogram p _i of the reference region may be referred to as reference histogram p _i or reference data, and is a step in the main process (step S5) performed first after the tracking target detection process (step S2) in FIG. in the process of S407 is a histogram _{p i} generated in S207 in FIG. 5, in the processing in step S407 in the second or subsequent the process (step S5), at step S443 in the previous present process (step S5) The updated histogram p _t (however, relating to the i-th region).

  Subsequently, as in step S403, the processing unit 2 determines whether the currently selected mode is the multipart mode or the single part mode (step S408). If the multipart mode is selected (if the mode flag is 0), the process proceeds to step S409. If the single part mode is selected (if the mode flag is 1), the process proceeds to step S412.

In step S409, the processing unit 2 determines whether or not the current area number i is 2, so that the Bhattacharyya distance d _pi ^(k ) for all areas (in this embodiment, two divided areas). ⁾ Is determined. If not completed, the processing unit 2 sets i to 2 (step S410), and returns to step S406. On the other hand, if completed, the process proceeds to step S411.

In step S411, the processing unit 2 calculates the dissimilarity d _p ^(k) of the k-th sample according to the following formula 37, and the average value of the Bhattacharyya distance d _pi ^(k) of each divided region sequentially calculated in step S407. And the process proceeds to step S413.

On the other hand, in the single part mode, in step S412, the processing unit 2 sets the Bhattacharyya distance d _pi ^(k) obtained in step S407 as the difference d _p ^(k) of the ^kth sample as it is, and proceeds to step S413. .

In step S413, the processing unit 2 calculates the likelihood π _pt ^(k) from the difference d _p ^(k) obtained in step S411 or S412 according to the following equation (38 ⁾ . Here, the likelihood is assumed to follow a Gaussian distribution.

In Equation 38, the _{N p} is the number of particles (number of samples). Further, in the equation 38, the sigma ² is usually composed of a dispersion of the Gaussian distribution, a value preset in the calculations here.

Thereafter, the processing unit 2 determines whether or not the calculation of the likelihood π _pt ^(k) has been completed for all the samples by determining whether or not the current sample number k is N _p ( Step S414). If not completed, the processing unit 2 increments the sample number k by 1 (step S415) and returns to step S403. On the other hand, if completed, the process proceeds to step S416.

In step S416, the processing unit 2 uses the likelihood π _pt ^(k) calculated in step S413 and the position (center of gravity position) of each sample according to the following equation 39 to determine the position of the tracking target region as a state estimation result. As an estimation result (that is, a part of the tracking result), a weighted average value E _p [S _t ] is obtained. The position of the tracking target area represented by the weighted average value E _p [S _t ] is the position of the tracking target area that is a part of the tracking result.

Next, the processing unit 2 sorts the likelihoods π _pt ^(k) of all the samples calculated in step S413 in descending order of the values (step S417).

Then, the processing unit 2, the maximum value of the likelihood obtained as a result of sorting the values of the likelihood at step S417 determines whether greater than a threshold value V _p set in advance (step S418). Greater than the threshold value V _p, it is determined that the tracking of the far succeeded, to set the tracking result flag to 1 (step S419), the process proceeds to step S421. On the other hand, is smaller than the threshold value V _p, it is determined that the tracking fails, set the tracking result flag to 0 (step S420), the process proceeds to step S421.

The processing unit 2 repeats the processes of steps S421 to S423 for each sample, and when the processes of steps S421 to S423 are completed for all the samples (YES in step S424), the process proceeds to step S425. In step S421, the processing unit 2 determines whether or not the likelihood π _pt ^{(k) of the} sample is larger than a preset threshold value _{ｐ p} . It is greater than the threshold value Π _p proceeds to S422, the process proceeds to S423 is smaller. In step S422, the processing unit 2 divides the sample to form a sample for the next time (that is, in the next frame) according to a known method of a particle filter. In step S423, the processing unit 2 causes the sample to disappear so as not to constitute the next sample (that is, in the next frame).

In step S425, the processing unit 2, extinguished the result of the division or step S423 in step S422 for each sample, the remaining sample is equal to or N _p or less. If N _p or less, the process proceeds to step S427. On the other hand, when there are more than N _p , the processing unit 2 extinguishes including those that are split in order from the one with the smallest likelihood π _pt ^(k) (step S426). As a result, the remaining sample becomes -1 _{N p} pieces or _N p. After step S426, the process proceeds to step S427.

In step S427, the processing unit 2 uses a region R having a predetermined height and a predetermined height in the image as a search range of the tracking target 10, and includes the tracking target region (in this embodiment, in the present embodiment). , a new rectangle) size parameter and the N _s samples (particles) rose sow. The position of each sample is the position estimated in step S416. The size of each sample is arbitrary and is initially randomly distributed. The aspect ratio of each sample is the original tracking target area (the tracking target area (the circumscribed rectangular area in the present embodiment) corresponding to the tracking target 10 detected in step S110 in FIG. 4) and the aspect ratio. Are the same. . Incidentally, a tracking target detecting process in FIG. 3 in the process of step S427 in (step S2) initially present process performed after (step S5), and sample N _s number rose sow but second and subsequent of the present process (step in the process of step S427 is in S5), dusted rose by adding the shortage minus the number of remaining samples in the previous present process from N _s number (step S5).

Next, the processing unit 2 sets a count value j indicating a sample number to 1 in order to assign a number to N _s samples (step S428).

  Next, the processing unit 2 determines whether or not the mode flag set to the latest in the mode selection process (step S4) in FIG. 3 is 0, so that the currently selected mode is the multipart mode. It is determined whether there is a single part mode or not (step S429). If the multi-part mode is selected (if the mode flag is 0), the processing unit 2 divides the j-th sample area in the same manner as in step S201 in FIG. 2) (step S430), the process proceeds to step S431. On the other hand, if the single part mode is selected (if the mode flag is 1), the process proceeds to step S431 without passing through step S430.

  In step S431, the processing unit 2 sets a count value i indicating an area number to 1. In the multi-part mode, the sample area is divided into two, so i becomes 1, 2, but in the single-part mode, the sample area is not divided, so i is only 1.

Next, the processing unit 2 performs the same processing as in steps S203 to S207 on the i-th region of the j-th sample in the image sampled most recently in step S400, so that the CIE 1976 L ^* u ^* v ^* color space. the _(histogram of the i-th area of the sample corresponding to the histogram p _i indicated by the number 1) histogram q _si that lightness index histogram and sensory chromaticity index histograms respectively obtained by one-dimensional by combining those normalized in Generate (step S432).

Next, the processing unit 2 calculates a Bhattacharyya distance d _si ^(j) between the histogram p _{i of the} reference region and the histogram q _si of the sample region according to the following formula 40 (step S433). Here, the histogram p _i of the reference region may be referred to as reference histogram p _i or reference data, and is a step in the main process (step S5) performed first after the tracking target detection process (step S2) in FIG. in the process of S433 is a histogram _{p i} generated in S207 in FIG. 5, in the processing in step S433 in the second or subsequent the process (step S5), at step S443 in the previous present process (step S5) The updated histogram p _t (however, relating to the i-th region).

  Subsequently, similarly to step S429, the processing unit 2 determines whether the currently selected mode is the multi-part mode or the single-part mode (step S434). If the multi-part mode is selected (if the mode flag is 0), the process proceeds to step S435. If the single-part mode is selected (if the mode flag is 1), the process proceeds to step S438.

In step S435, the processing unit 2 determines whether the current region number i is 2 or not, so that the Bhattacharyya distance d _si ^(j ) for all regions (in this embodiment, two divided regions). ⁾ Is determined. If not completed, the processing unit 2 sets i to 2 (step S436), and returns to step S432. On the other hand, if completed, the process proceeds to step S437.

In step S437, the processing unit 2 calculates the difference d _s ^(j) of the j-th sample according to the following formula 41, and the average value of the Bhattacharyya distances d _si ^(j) of the respective divided regions sequentially calculated in step S433. And the process proceeds to step S439.

On the other hand, in the single part mode, in step S438, the processing unit 2 sets the Bhattacharyya distance d _si ^(j) obtained in step S433 as the j-th sample dissimilarity d _s ^(j) as it is, and proceeds to step S439. .

In step S439, the processing unit 2, according to the following numbers 42, using as observed values and the resulting dissimilarity _d ^{s (j)} in step S437 or S438, and calculates the likelihood π _st ^(j). Here, the likelihood is assumed to follow a Gaussian distribution.

In Equation 42, the _{N s} is the number of particles (number of samples). In Equation 41, σ ² is usually a Gaussian distribution, but is assumed to be a value set in advance in this calculation.

Thereafter, the processing unit 2, by current sample number j to determine whether the N _s, for all samples, it is determined whether the calculation of the likelihood [pi _{st ^(j)} is completed ( Step S440). If not completed, the processing unit 2 increments the sample number j by 1 (step S441), and returns to step S429. On the other hand, if completed, the process proceeds to step S442.

In step S442, the processing unit 2 estimates the size of the tracking target region, which is the state estimation result, according to the following equation 43 from the likelihood π _st ^(j) calculated in step S439 and the size of each sample. The weighted average value E _s [S _t ] is obtained as (that is, another part of the tracking result). The size of the tracking target area represented by the weighted average value E _s [S _t ] is the size of the tracking target area that is another part of the tracking result.

Thereafter, the processing unit 2 updates the reference data according to the following equation 44 based on the estimation result obtained in step S442 (step S443). In Equation 44, the histogram _pt-1 indicates a histogram related to the tracking target region estimated at time t-1, and the tracking target region obtained as the estimation result in step S442 in the image sampled latest in step S400. Histogram obtained by combining the normalized values of the lightness index histogram and the perceptual chromaticity index histogram in the CIE 1976L ^* u ^* v ^* color space by performing the same processing as in steps S203 to S207 ^. It is. In Expression 44, q _t−1 represents a histogram related to the tracking target region before estimation at time t−1.

Next, the processing unit 2 sorts the likelihoods π _st ^(j) of all the samples calculated in step S439 in descending order of the values (step S444).

Then, the processing unit 2, the maximum value of the likelihood obtained as a result of sorting the values of the likelihood at step S444 determines whether greater than a threshold value V _s which is set in advance (step S445). Is greater than the threshold value _{V p,} the process proceeds to step S447. On the other hand, is smaller than the threshold value V _s, it is determined that the tracking fails, set the tracking result flag to 0 (step S446), the process proceeds to step S447.

The processing unit 2 repeats the processes of steps S447 to S449 for each sample, and when the processes of steps S427 to S429 are completed for all the samples (YES in step S450), the process proceeds to step S451. In step S447, the processing unit 2 determines whether the sample of the likelihood [pi _{st ^(j)} is greater than the threshold value [pi _s that was set in advance. Is greater than the threshold value Π _s proceeds to S448, the process proceeds to S449 is smaller. In step S <b> 448, the processing unit 2 performs division to form the next sample (that is, in the next frame) according to a known method of the particle filter. In step S449, the processing unit 2 causes the sample to disappear so as not to constitute the next sample (that is, in the next frame).

In step S451, the processing unit 2, extinguished the result of the division or step S449 in step S448 for each sample, the remaining sample is equal to or N _s or less. If N _s or less, the tracking process is terminated (step S5), and the process proceeds to step S6 in FIG. On the other hand, when there are more than N _s , the processing unit 2 extinguishes including those split in order from the smallest likelihood π _st ^(j) (step S452). As a result, the number of remaining samples is N _s or N _s −1. After step S452, the tracking process is terminated (step S5), and the process proceeds to step S6 in FIG.

  The present embodiment can provide the same advantages as the first embodiment. However, as described above, according to the present embodiment, the tracking processing accuracy is higher than that of the first embodiment. Although reduced, there is an advantage that the amount of calculation can be reduced and the processing time can be shortened as compared with the first embodiment.

  As mentioned above, although each embodiment of this invention and its modification were demonstrated, this invention is not limited to these.

1 is a block diagram schematically showing an automatic tracking device according to a first embodiment of the present invention. It is a figure which shows typically the example of the mode of the tracking of the tracking target by a camera. It is a schematic flowchart which shows an example of operation | movement of the process part in FIG. It is a flowchart which shows the tracking object detection process (step S2) in FIG. 3 in detail. It is a flowchart which shows the mode selection process (step S4) in FIG. 3 in detail. It is a flowchart which shows this process (step S5) of the tracking process in FIG. 3 in detail. It is a flowchart following FIG. It is a flowchart which shows the camera control process (step 7) in FIG. 3 in detail. It is a flowchart following FIG. 10 is a flowchart subsequent to FIG. 9. It is a flowchart following FIG. It is a figure which shows typically the example of the area | region which comprises the search range of a tracking object, and the distribution state of a sample. It is a figure which shows the example of the division pattern of an area | region typically. It is a flowchart which shows the example which deform | transformed a part in FIG. It is a flowchart which shows in detail this process of the tracking process which the process part of the automatic tracking apparatus by the 2nd Embodiment of this invention performs. It is a flowchart following FIG. It is a flowchart following FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Camera 1a Camera body 1b Zoom lens 1c Turntable 2 Control part

Claims (12)

A camera capable of controlling pan, tilt and zoom;
Processing means for processing the image captured by the camera and controlling the camera so that the camera automatically tracks and captures a tracking target;
With
The processing means performs tracking processing for tracking a tracking target based on an image captured by the camera, and the camera tracks the tracking target according to a result of the tracking processing by the tracking processing means. Control means for controlling pan, tilt and zoom of the camera, and
The tracking processing means is a single feature amount obtained from an entire area of the tracking target area, which is an area corresponding to the tracking target, at the beginning of tracking of the tracking target based on an image captured by the camera. A single part mode for performing tracking processing using part feature amounts, or a multi-part mode for performing tracking processing using multi-part feature amounts, which are feature amounts respectively obtained from the divided regions of the tracking target region, and Preprocessing means for selecting whether to perform, and main processing means for performing tracking processing in the mode selected by the preprocessing means,
An automatic tracking device characterized by that.   The automatic tracking device according to claim 1, wherein the feature amount is a histogram.   The automatic tracking device according to claim 1, wherein the feature amount is a histogram based on a histogram in a predetermined color space. 4. The feature quantity is a histogram obtained by combining one-dimensional combination of normalized brightness index histograms and perceptual chromaticity index histograms in a CIE 1976 L ^* u ^* v ^* color space. The automatic tracking device described.   The preliminary processing means is based on an image indicating the similarity between the feature quantities of the divided areas of the tracking target area at the beginning of tracking of the tracking target, based on an image captured by the camera. The single-part mode is selected when the similarity between the feature values in each region is higher than a predetermined value, and the multi-part mode is selected when the similarity between the feature values in each region is lower than a predetermined value. The automatic tracking device according to claim 1, wherein the automatic tracking device is selected.   6. The automatic tracking device according to claim 5, wherein the index is a Bhattacharyya distance between the feature quantities of the divided areas. The processing means estimates the position and size of the tracking target region as a tracking result by a particle filter using a plurality of particles using the position and size of the tracking target region as parameters based on an image captured by the camera. And
For each of the particles, the particle filter has a feature amount of the particle with respect to a reference feature amount based on a feature amount of a past tracking target region (single part feature amount in single-part mode, multi-part feature amount in multi-part mode). The likelihood calculated by the degree of difference indicating the degree of difference (single part feature amount in single part mode, multi part feature amount in multi part mode) is used.
The automatic tracking device according to any one of claims 1 to 6, wherein   The degree of difference of each particle is a Bhattacharyya distance between the reference feature quantity and the single part feature quantity of the particle in the single-part mode, and the reference feature quantity and the multi-value of the particle in the multi-part mode. 8. The automatic tracking device according to claim 7, wherein the automatic tracking device is an average value of Bhattacharyya distances between each of the part feature amounts. The processing means uses a first particle filter based on a plurality of first particles using the position of the tracking target region as a parameter based on an image captured by the camera, and sets the tracking target region as a part of the tracking result. A second particle filter based on a plurality of second particles having a tracking target position estimated by the first particle filter using the size of the tracking target region as a parameter, Estimate the size of the tracking target area as a part,
The first particle filter, with respect to each first particle, with respect to a reference feature amount based on a past feature amount of a tracking target region (single part feature amount in single part mode, multi part feature amount in multi part mode). Using the likelihood calculated by the first degree of difference indicating the degree of difference in the feature amount of the first particle (single part feature amount in single part mode, multipart feature amount in multipart mode),
The second particle filter performs a reference feature amount based on a feature amount of a past tracking target region (single part feature amount in single part mode, multipart feature amount in multipart mode) with respect to each second particle. The likelihood calculated by the second degree of difference indicating the degree of difference between the feature quantities of the second particles (single part feature quantity in the single part mode and multipart feature quantity in the multipart mode) is used.
The automatic tracking device according to any one of claims 1 to 6, wherein The first difference degree of each first particle is a Bhattacharyya distance between the reference feature amount and the single part feature amount of the first particle in the single part mode, and in the multipart mode, An average Bhattacharyya distance between each of the reference feature and the multipart feature of the first particle;
The second dissimilarity of each second particle is a Bhattacharyya distance between the reference feature quantity and the single part feature quantity of the second particle in the single-part mode, and in the multi-part mode, An average value of the Bhattacharyya distance between each of the reference feature quantity and the multipart feature quantity of the second particle;
The automatic tracking device according to claim 9. The control means includes prediction means for predicting the position and size of the tracking target area after a predetermined time has elapsed from the present based on the result of the tracking process by the tracking processing means,
The control means controls the pan, tilt and zoom of the camera by correcting a current pan, tilt and zoom control state for the camera according to a prediction result by the prediction means.
The automatic tracking device according to any one of claims 1 to 10, wherein   12. The automatic tracking device according to claim 11, wherein the predicting unit predicts the position and size of the tracking target region after a predetermined time has elapsed from the present time by a Kalman filter.

Images