JP2007148663A - Object-tracking device, object-tracking method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an object-tracking device which very precisely tracks an object at high speed, and also provide an object-tracking method and a program. <P>SOLUTION: A plurality of positions of a hand area of a current-time image are forecasted by a hand-area position forecasting part 43 based on the position and moving speed of a hand area of previous-time image, and a plurality of object positions forecasted by a forecasted-area evaluation part 44 are evaluated based on the current-time image and a preliminarily registered hand area initial model. Then, an object is tracked by a hand-area tracking result estimating part 45 based on each of the evaluated object positions. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an interface between a person and a device, and detects an object such as a human hand region from an input image and tracks the object, and an object tracking method and object tracking process It is related with the program which performs.

  In order to realize a natural interface between humans and devices, information such as human gestures and gestures, including various actions and voices of the operator, is analyzed in an integrated manner, and the presence and intention of humans are determined. It is essential to detect and recognize. Therefore, various techniques related to human behavior analysis have been studied. These can be broadly classified as those that focus on human faces (for example, face orientation, facial expression, line of sight, lip movement, etc.) and human limb movements (hand gestures, pointing, gestures, motion events, walking patterns, etc.) ). Among them, detection and recognition of the hand region in a general environment is one of the most important technologies, and many studies have been made. For example, a hand shaking detection method focusing on a reciprocating motion that moves a human hand to the left and right (for example, see Non-Patent Document 1), a method for attaching a special object such as a data glove (for example, see Non-Patent Document 2), a person Have been proposed (for example, see Non-Patent Document 3 and Non-Patent Document 4).

Irie, Umeda: Detection of hand gestures from images using time-series changes in gray values, P2-8, Symposium on Image Recognition and Understanding (MIRU'02) (Nagoya Institute of Technology (Nagoya City), July 30-August 1st, 2002) Hoshino: Design of robot hand capable of non-verbal communication, Ministry of Education, Culture, Sports, Science and Technology Grant-in-Aid for Scientific Research “Informatics research that pioneers the foundation of IT deepening” Research item A03 “Research on human information processing and its application” No. 4 Kaishira Kenkyukai (Shibaran Kaikan (Kyoto City), July 2, 2003) Ito, Ozeki, Nakamura, Ota: Robust recognition and tracking of hands and grasped objects for video indexing, Symposium on Image Sensing, pp.143-162, (2003) Irie, Umeda: Construction of intelligent room based on gesture recognition, 4th rotating image processing practical use workshop, March 6-7, 1995

  By the way, a person's action such as a gesture cannot be grasped unless the movement of an object such as a person's limb is followed in time. That is, the hand region detection described in Patent Documents 1 to 4 must be performed continuously in time.

  However, when the hand area detection described in Patent Documents 1 to 4 is performed for each input image, the processing burden on the apparatus increases, and it becomes difficult to track the hand area.

  The present invention has been made to solve the above problems, and an object of the present invention is to provide an object tracking device, an object tracking method, and a program capable of tracking an object at high speed and with high accuracy.

  In order to solve the above-described problem, an object tracking device according to the present invention is an object tracking device that tracks an object to be tracked from an image, and a previous time image for an estimated object for estimating the object position of the object. Position prediction means for calculating a prediction range in which the guess object moves from information on the guess object and predicting the guess object position of the current time image based on the forecast range, the guess object position of the guess object of the previous time image, and the motion speed A position evaluation unit that evaluates a plurality of estimated object positions predicted by the position prediction unit based on object similarity between a current time image of the estimated object and a previously registered object image; and the position evaluation unit At each guessed object location evaluated in By providing an estimation unit for estimating the object position at the present time image Zui, to solve the problems described above.

  In addition, the object tracking method according to the present invention is an object tracking method for tracking an object to be tracked from an image, inferring from the estimated object information of the previous time image with respect to the estimated object for estimating the object position of the object. A position prediction step of calculating a prediction range in which the object moves, and predicting the estimated object position of the current time image based on the prediction range, the estimated object position of the estimated object of the previous time image, and the motion speed; A position evaluation step for evaluating the predicted plurality of estimated object positions based on object similarity between the current time image of the estimated object and a pre-registered object image, and each estimated object evaluated in the position evaluation step Obfuscation of the current time image based on position By having the estimating step of estimating the-objects position, to solve the problems described above.

  Further, the program according to the present invention is a program for executing processing for tracking an object to be tracked from an image, and for estimating an object position for estimating the object position of the object from an estimated object information of a previous time image. A position prediction step of calculating a prediction range in which the object moves, and predicting the estimated object position of the current time image based on the prediction range, the estimated object position of the estimated object of the previous time image, and the motion speed; A position evaluation step for evaluating the predicted plurality of estimated object positions based on object similarity between the current time image of the estimated object and a pre-registered object image, and each estimated object evaluated in the position evaluation step Object of current time image based on position By having the estimating step of estimating the bets position, to solve the problems described above.

  According to the present invention, for a guess object for estimating the object position of an object to be tracked, a prediction range in which the guess object moves is calculated from information on the guess object in the previous time image, and the prediction range, the previous time image The estimated object position of the current time image is predicted based on the estimated object position and the motion speed of the estimated object, and a plurality of estimated estimated object positions are determined between the current time image of the estimated object and the object image registered in advance. By evaluating based on the object similarity and estimating the object position of the current time image based on each estimated estimated object position, the object can be tracked at high speed and with high accuracy.

  Hereinafter, specific embodiments of the present invention will be described in detail with reference to the drawings. An object tracking device shown as a specific example of the present invention detects a hand region as an object in an image, calculates a prediction range in which the hand region moves from information on the previous time of the hand region, Predict the estimated hand region position of the current time image based on the position and movement speed of the hand region of the time image, and evaluate the hand region predicted position based on the distribution characteristics of the current time image and the hand region image registered in advance. By doing so, the hand area at the current time is determined.

  FIG. 1 is a diagram illustrating an overall process of a hand region tracking device that tracks a hand region. The current time image captured by the imaging unit 101 is captured by the image capturing unit 102. The current time image captured by the image capturing unit 102 is used for hand region tracking processing and hand region detection processing, as will be described later. The mode determination unit 103 determines whether or not the tracking mode is set.

  In the tracking mode, the tracking result reading means 104 reads the hand region tracking result at the previous time (t−1) and also reads the hand region model 105 registered in advance. Then, tracking of the hand region is performed by the tracking processing unit 106, and the tracking result at the current time (t) is stored by the tracking result storing unit 107. After saving the tracking result at the current time (t), the mode determination unit 103 determines the next mode.

  As will be described later, the tracking processing unit 106 calculates a prediction range in which the hand region moves from the hand region information of the previous time image, and based on the calculated prediction range, the position of the hand region of the previous time image, and the motion speed. Based on the similarity between the current time image and the hand region model image registered in advance, the position predicting unit that predicts a plurality of estimated hand region positions of the current time image, and the plurality of estimated hand region positions predicted by the position predicting unit. Position evaluation means for evaluating and tracking means for tracking the hand area based on each estimated hand area position evaluated by the evaluation means.

  Here, it is preferable that the position predicting unit calculates a prediction range in which the hand region moves based on the size and the movement speed of the hand region of the previous time image. Moreover, it is preferable that a position prediction means acquires the hand region position of a previous time image based on the probability by evaluation of the similarity of a previous time image.

  Further, it is preferable that the position evaluation means evaluate each estimated hand region position based on the similarity between the distribution characteristics of the current time image and the distribution characteristics of the hand region image registered in advance. Further, the position evaluation means determines the similarity between the distribution characteristics of the current time image and the distribution characteristics of the hand area model image registered in advance based on the distance from the center of the estimated hand area position predicted by the position prediction means. It is preferable to evaluate. Further, the position evaluation means preferably evaluates the similarity between the distribution characteristics of the current time image and the distribution characteristics of the hand region model image registered in advance based on a predetermined mask region.

  The tracking means calculates the probability that the estimated hand area position evaluated by the position evaluation means is a hand area based on the evaluation result of the estimated hand area position, and calculates the probability of each estimated hand area position. It is preferable to track the hand region based on it.

  Further, the tracking processing means 106 is a distance information acquisition means for acquiring distance information obtained by estimating the position of each pixel in the three-dimensional space based on images from a plurality of cameras having a predetermined positional relationship, and the distance information acquisition described above. It is preferable to have background removal means for removing the background of the object from the image based on the distance information acquired by the means.

  On the other hand, when the mode determining unit 103 determines that the mode is not the tracking mode, the hand region detecting unit 108 performs processing for detecting the hand region from the image at the current time (t). The hand region detection determination unit 109 determines whether or not the hand region detection unit 108 has detected a hand region. When the hand region is detected, the hand region model 105 is registered in the hand region registration unit 110 and switched to the tracking mode by the mode switching unit 111. When the hand region is not detected, the mode determination unit 103 determines the next mode.

  Next, a case where the mode determining unit 103 determines that the mode is not the tracking mode will be described.

  FIG. 2 is a block diagram showing a configuration of a hand region detection device corresponding to the hand region detection means 108 for detecting the hand region. The hand region detection device 10 includes an imaging unit 11, an image capturing unit 12 that captures an image captured by the imaging unit 11, and a skin color candidate region that selects a skin color candidate region from the image captured by the image capturing unit 12. The selection unit 13, the hand candidate region extraction unit 14 that extracts a hand candidate region from the skin color candidate region selected by the skin color candidate selection unit 13, and the complexity of the shape of the hand candidate region extracted by the hand candidate region extraction unit 14 A shape complexity calculating unit 15 for calculating the degree and a hand region detecting unit 16 for detecting a hand region based on the complexity calculated by the shape complexity calculating unit 15 are configured.

  The imaging unit 11 includes a camera having an imaging element such as a CCD (Charge Coupled Device).

  The image capturing unit 12 captures an image in a format such as JPEG (Joint Photographic Experts Group) from the imaging unit 11.

  The skin color candidate region selection unit 13 selects a skin color candidate region by determining whether each pixel in the image is the color of the skin color model using a skin color model based on human statistical skin color characteristics. This skin color model is obtained by statistical processing using hue, saturation, RGB information, and the like.

  The hand candidate area extraction unit 14 calculates, for example, image distribution characteristics such as hue and saturation distribution for each area selected by the skin color candidate area selection unit 13, and includes, for example, hue and saturation in those areas. A region having the largest pixel having similar color characteristics such as degree is extracted as a hand candidate region.

  For each hand candidate area extracted by the hand candidate area extraction unit 14, the shape complexity calculation unit 15 is based on, for example, the ratio of the area (number of pixels) to the peripheral length or the distance from the center of the hand candidate area. To calculate the shape complexity.

  The hand region detection unit 16 detects the hand region by comparing and evaluating the complexity of each hand candidate region calculated by the shape complexity calculation unit 15 with, for example, a preset threshold value.

  In the configuration of the hand region detection device 10 described above, an image is acquired from the imaging unit 11, but an image may be acquired from various media such as a DVD (Digital Versatile Disc) and the Internet.

  Next, processing for detecting a hand region from an input image will be described with reference to the drawings. 3 to 6 are diagrams illustrating image examples when selecting skin color candidates from an input image using RGB information.

  For example, the skin color candidate region selection unit 13 extracts the document (MJ Jones and JM Rehg: Statistical color models with application to skin detection, Proc. IEEE Conf.) From the input image shown in FIG. Using the RGB information of each pixel described in Computer Vision and Pattern Recognition, vol. 2, pp. 274-280, 1999.), the skin color model color is detected as shown in FIG. Then, by determining whether each pixel is a skin color model color, skin color candidate areas A to D are selected as shown in FIG. Thereby, skin color candidate areas A to D as shown in FIG. 6 can be determined for the input image shown in FIG. In addition to the RGB information, for example, by using statistical information (a priori knowledge) of skin color obtained by learning, it is determined whether each pixel is skin color, and region integration using image filtering processing A skin color candidate region in the image may be selected by the processing.

  The hand candidate area extraction unit 14 calculates a hue image Hue (i, j) and a saturation image Sta (i, j) from the input image as shown in FIG. 3, and converts the input image into a hue image and a saturation image. To do. The hue represents the position of the color on the spectrum, and is represented by an angle of 0 degrees to 360 degrees. The saturation represents the vividness of the color. The value decreases as the color approaches gray, and the value increases as the color approaches the primary color.

The hand candidate area extraction unit 14 also outputs a hue distribution (histogram) h _m (k) and a saturation distribution (histogram) s _m (for each skin color candidate area A to D selected by the skin color candidate area selection unit 13. k) (k = 0, 1, 2,..., K) are calculated by the equations (1) and (2), respectively.

Here, R _m represents each skin color complementary region, and m (m = 1, 2,..., M) is the number of skin color candidate regions. Also, g (Hue (i, j)) and g (Sta (i, j)) are respectively in the pixel (i, j) of the hue image Hue (i, j) and the saturation image Sta (i, j). Indicates the luminance value. Further, δ [x] is 1 when x is 0, and 0 when x is not 0.

For example, when the hue distribution h _m (k) of the skin color candidate region B of the hue image shown in FIG. 7 is calculated using the above equation (1), a histogram as shown in FIG. 8 can be obtained. Moreover, for example, when the saturation distribution s _m (k) of the skin color candidate region B of the saturation image shown in FIG. 9 is calculated using the above equation (2), a histogram as shown in FIG. 10 can be obtained. .

  FIGS. 11 and 12 are examples of flesh color hue areas and saturation areas detected from the above-described histogram. FIGS. 13 and 14 are a hue image and a saturation image in which hand candidate areas are detected based on the hue area and the saturation area of the skin color, respectively.

From the hue distribution h _m (k) and the saturation distribution s _m (k) of each skin color candidate area A to D, the hand candidate area extraction unit 14 has a hue value with the largest number of pixels as shown in FIGS. The h_Level and the saturation value s_Level and their variances h_var and s_var are detected.

  Here, since the flesh color has a low hue value, the hue value in the hue image Hue (i, j) of each flesh color candidate area A to D is within h_Level ± h_var, and the value is the flesh initial hue value H_Initial. The smaller pixel area is defined as a flesh-colored hue area (image) Hm (i, j). As a result, the skin tone area Hm (i, j) of each skin color candidate area A to D as shown in FIG. 13 is obtained. In the skin color candidate regions A to D shown in FIG. 13, the skin color hue region Hm (i, j) is shown in black.

  Further, since the skin color has a high saturation value, the saturation value in the saturation image Sta (i, j) of each skin color candidate region A to D is within s_Level ± s_var, and the value is the saturation of the skin color. A pixel region larger than the initial value S_Initial is defined as a skin color saturation region (image) Sm (i, j). Thus, the skin color saturation region Sm (i, j) of each skin color candidate region A to D as shown in FIG. 14 is obtained. In the skin color candidate areas A to D shown in FIG. 14, the skin color saturation area Sm (i, j) is shown in white.

  A pixel region where the skin tone region Hm (i, j) and the saturation region Sm (i, j) overlap is defined as a hand candidate region Rm. That is, like the hand candidate area shown in FIG. 15, in each skin color candidate area A to D, the black area shown in FIG. 13 and the white area shown in FIG. 14 overlap.

  In this way, by extracting the region where the pixels with the same hue and saturation are the largest, that is, the region indicating the dominant color, as the candidate region of the hand, it is possible to cope with individual differences in human skin color and lighting changes in the real environment. Robustness can be improved.

  The shape complexity calculation unit 15 calculates the complexity of the hand candidate region based on, for example, the area (number of pixels) and the peripheral length of the hand candidate region Rm extracted by the hand candidate region extraction unit 14. .

  Here, L is the perimeter of the hand candidate region, S is the area of the hand candidate region, and δ is the complexity of the shape.

  The hand region detection unit 16 compares and evaluates the complexity of the shape of each hand candidate region calculated by the shape complexity calculation unit 15 with the complexity corresponding to a predetermined hand shape. FIG. 16 is an example of the calculation (learning) result of the complexity according to the hand shape. For example, the shape complexity in the state where five fingers are open, that is, the state where the hand is open (K1) is significantly different from the shape complexity in the state where the hand is not open (K6). Considering this, a threshold value for detecting the hand region is determined empirically. For example, the shape complexity in a state where four or more fingers are open (δ> 0.9) is set as the threshold value of the hand region so that the head region is not detected as the hand region. A case where the complexity of the hand candidate area is larger than a predetermined threshold is detected as a hand area.

  FIG. 17 shows an image in which five fingers are not open, that is, a hand is not open, and FIG. 18 shows the shape complexity of the hand candidate region at that time. When the image shown in FIG. 17 is input, the hand region detection unit 16 compares the complexity of the hand candidate regions b and c with a threshold value. For example, when the complexity threshold is set as δ = 0.9, the complexity of both the hand candidate area b and the hand candidate area c is lower than the threshold, so the hand area detection unit 16 detects these as hand areas. do not do.

  FIG. 19 shows a state in which the two fingers, the index finger and the middle finger, are open, and FIG. 20 shows the shape complexity of the hand candidate region at that time. Even when an image with two fingers open as shown in FIG. 19 is input and the complexity threshold is set to δ = 0.9, the complexity of both the hand candidate area b and the hand candidate area c is a threshold. Therefore, the hand region detection unit 16 does not detect these as hand regions.

  FIG. 21 shows a state where all five fingers are open, that is, a state where the hand is open, and FIG. 22 shows the shape complexity of the hand candidate region at that time. As shown in FIG. 21, when an image with the hand open is input and the threshold value of the complexity of the shape of the hand candidate area is set to δ = 0.9, the complexity of the hand candidate area c is greater than the threshold value. Since the complexity of the hand candidate area b is smaller than the threshold value, the hand area detection unit 16 detects the hand candidate area b as a hand area.

  Note that, as described above, not only the complexity is calculated based on the area and the peripheral length of the hand candidate area, but the center of the hand candidate area is detected based on the distance from the center as shown in FIG. You may calculate. In this case, the distance function d (θ) may be defined as in equation (4).

  FIG. 24 is a graph of the distance function according to equation (4). Here, when the maximum value of the distance function d (θ) is equal to or greater than the threshold (Thd) indicating the shape of the finger, the number of maximum values can be estimated as the number of fingers.

  As shown in FIG. 25, the hand region detection unit 16 detects a hand region according to the number of local maximum values equal to or greater than a threshold value (Thd). For example, when the threshold of the number of maximum values equal to or greater than the threshold (Thd) is 4, that is, when four or more fingers are detected, the hand candidate area is detected as a hand area.

  Also, as shown in FIG. 24, h (k) (k = 1, 2,..., K) may be defined, and if h (k) is equal to or greater than a certain threshold value, it may be estimated as a finger. In this case, the hand region detection unit 16 detects the hand region according to h (k) equal to or greater than the threshold value. For example, when the threshold value is 5, when five fingers are detected, the hand candidate area is detected as a hand area. Note that the threshold for detecting these fingers and the threshold for detecting the hand region may be determined by learning.

  In this way, selecting candidate skin color regions based on human statistical skin color characteristics, extracting hand candidate regions based on image distribution characteristics within those regions, and evaluating the shape complexity of these hand candidate regions Thus, robust hand region detection can be performed even in a real environment without depending on individual differences in human skin color or lighting changes.

  In addition, since the region feature when the hand is open can be distinguished from other region features by evaluating the shape complexity, robust hand region detection can be performed.

  The hand region detection apparatus 10 selects skin color candidate regions based on statistical skin color features and extracts hand candidate regions based on image distribution characteristics in these regions, but is not limited thereto. Instead, the selection of skin color candidate regions and the extraction of hand candidate regions may be integrated, and the hand candidate regions may be narrowed down by performing RGB processing, image distribution processing, and the like in an arbitrary order.

  FIG. 25 is a block diagram showing another configuration (part 1) of the hand region detection apparatus described above. In order to further improve the robustness of the real environment, the hand region detection device 20 uses, for example, difference information between frames, and moves the detected hand region further (region) to the correct hand. Determine as an area. In addition, the same code | symbol is attached | subjected to the structure same as the hand region detection apparatus 10 mentioned above, and description is abbreviate | omitted.

  The hand region detection device 20 includes an imaging unit 11, an image capturing unit 12 that captures an image captured by the imaging unit 11, and a skin color candidate region that selects a skin color candidate region from the image captured by the image capturing unit 12. The selection unit 13, the hand candidate region extraction unit 14 that extracts a hand candidate region from the skin color candidate region selected by the skin color candidate selection unit 13, and the complexity of the shape of the hand candidate region extracted by the hand candidate region extraction unit 14 A shape complexity calculating unit 15 for calculating the degree, a hand region detecting unit 16 for detecting a hand region based on the complexity calculated by the shape complexity calculating unit 15, and a hand region detected by the hand region detecting unit 16 A delay circuit unit 21 that delays an image including a predetermined frame, a motion region detection unit 22 that compares the image delayed by a predetermined frame by the delay circuit unit 21 with an image of the current frame, and detects a motion region; Motion region detector 2 In constructed and a hand region determining unit 23 for determining the hand region based on the detected motion region.

  The delay circuit unit 21 has a buffer that temporarily holds an input image for a predetermined frame. For example, the delay circuit unit 21 receives the input image and an image one frame before that in the motion region detection unit 22. Output.

  For example, the motion region detection unit 22 compares two input images that differ by one frame and detects a motion region. Here, it is preferable that the motion region detection unit 22 detects the motion in the vicinity of the hand region detected by the hand region detection unit 16.

  The hand region determination unit 23 determines whether or not the motion region detected by the motion region detection unit 22 is a hand region detected by the hand region detection unit 16, and only when the detected motion region is a hand region. The area is determined as a hand area. It is determined whether or not the hand region detected by the hand region detection unit 16 is a motion region detected by the motion region detection unit 22, and if the detected hand region is a motion region, it is determined as a hand region. May be.

  Next, the operation of the hand region detection device 20 will be described.

  Since the hand region detection device 20 detects the hand region by the hand region detection unit 16 in the same manner as the hand region detection device 10 described above, detailed description thereof will be omitted. As described above, the hand region detection device 20 selects a skin color candidate region based on a person's statistical skin color feature from the input image, extracts a hand candidate region based on image distribution characteristics in these regions, and extracts the hand region. The hand region is detected based on the shape complexity of the candidate region. The image in which the hand area is detected is input to the delay circuit unit 21.

  FIG. 26 and FIG. 27 show examples of images with different imaging times output from the delay circuit unit 21, respectively. FIG. 28 is an example of an image in which a motion region is detected based on the images of FIGS. 26 and 27 output from the delay circuit 21.

  The delay circuit unit 21 holds the input image in a buffer, and outputs, for example, an image Im (t) as shown in FIG. 26 and an image Im (t + 1) as shown in FIG. 27 to the motion region detection unit 22. .

  The motion region detection unit 22 detects motion information from the input image Im (t) and the image Im (t + 1), for example, by inter-frame difference processing. Thereby, as shown in FIG. 28, a hand part and a face part can be detected.

  The hand region determination unit 23 determines whether or not the detected motion region is a hand region detected by the hand region detection unit 16. For example, as shown in FIG. 28, the hand part that is the hand area detected by the hand area detection unit 16 is determined as the final hand area among the hand part and the face part in which the motion area is detected (see FIG. 29). ).

  FIG. 30 shows an example of selection of a skin color candidate area, and FIG. 31 shows an example of detection of a hand area. By using the motion information described above, the skin color candidate regions G and H depicted on the poster from the hand regions shown in the skin color candidate regions E, F, G, and H selected by the skin color candidate selection unit 13 as shown in FIG. Thus, the final hand region can be reliably detected as shown in FIG.

  On the other hand, when the motion information is not used, as shown in FIGS. 32 and 33, the hand regions in the skin color candidate regions E and G are finally detected from the skin color candidate regions E, F, G, and H. . That is, the hand depicted on the poster is also detected as a hand region.

  The above-described hand region detection device 20 detects the hand region based on the complexity, then detects the motion region, and determines the final hand region from the hand region and the motion region. A hand region may be determined by detecting a motion region from an input image from the capturing unit 12.

  FIG. 34 is a block diagram showing another configuration (part 2) of the hand region detection device described above. In order to further improve the robustness of the real environment, the hand region detection device 30 estimates distance information using, for example, a stereo camera, and removes skin color candidate regions that are at a predetermined distance or more as a background. In addition, the same code | symbol is attached | subjected to the structure same as the hand region detection apparatus 10 mentioned above, and description is abbreviate | omitted.

  The hand region detection device 30 includes an imaging unit 11, an image capturing unit 12 that captures an image captured by the imaging unit 11, and a skin color candidate region that selects a skin color candidate region from the image captured by the image capturing unit 12. The selection unit 13, the hand candidate region extraction unit 14 that extracts a hand candidate region from the skin color candidate region selected by the skin color candidate selection unit 13, and the complexity of the shape of the hand candidate region extracted by the hand candidate region extraction unit 14 A shape complexity calculating unit 15 for calculating the degree, a hand region detecting unit 16 for detecting a hand region based on the complexity calculated by the shape complexity calculating unit 15, a reference imaging unit 31, an imaging unit 11, and a reference Based on the distance information acquisition unit 32 that acquires the distance information of the subject in the image based on the imaging unit 32, the hand region detected by the hand region detection unit 16, and the distance information acquired by the distance information acquisition unit 32. Hand region determination to determine hand region Configured with a 33.

  The reference imaging unit 31 is configured by a camera having an imaging element such as a CCD, and is configured as a stereo camera using the imaging unit 11. In addition, it is preferable to further configure a stereo camera with a plurality of cameras.

  The distance information acquisition unit 32 uses the imaging unit 11 and the reference imaging unit 31 to measure the distance to the object based on the “stereo method”, for example. The stereo method referred to here is a three-dimensional view of each point in a scene, that is, a captured image using images captured from the viewpoints (projection centers) of a plurality of cameras having a predetermined positional relationship based on the principle of so-called “triangulation”. Estimate the position in space and measure the distance to the projection center.

  The hand region determination unit 33 determines the final hand region based on the distance information acquired by the distance information acquisition unit 32 for the hand region detected by the hand region detection unit 16. For example, the final hand region is determined according to whether the hand region detected by the hand region detection unit 16 is equal to or smaller than a predetermined distance, equal to or greater than a predetermined distance, or within a predetermined distance range.

  Next, the operation of the hand region detection device 30 will be described.

  Since the hand region detection device 30 detects the hand region by the hand region detection unit 16 in the same manner as the hand region detection device 10 described above, description thereof is omitted. In other words, skin color candidate regions based on human statistical skin color features are selected from the input image, hand candidate regions based on image distribution characteristics in those regions are extracted, and based on the shape complexity of the hand candidate regions To detect the hand area.

  FIGS. 35 to 39 are diagrams illustrating image examples when skin color candidates are selected from the input image using the distance information. As shown in FIG. 35, when an image in which two persons are spreading their hands is input to the image capturing unit 12, the skin color candidate regions I, J, K, and L are selected by the skin color candidate region selection unit 13. (See FIG. 36). Then, the hand region detection unit 16 detects the hand regions in the skin color candidate regions I and K based on the shape complexity as shown in FIG.

  The distance information acquisition unit 32 acquires distance information from the imaging unit 11 to the subject, for example, based on the images captured by the imaging units 11 and 31, and outputs the distance information to the hand region determination unit 33. The hand region determination unit 33 determines the hand region from the hand candidate regions i, j, k, and l extracted by the hand candidate region extraction unit 14 based on the color shading according to the distance information as shown in FIG. . In FIG. 39, a person's hand area at a position closer to the imaging unit 11 is detected, and a person's hand area at a position farther than that is removed.

  Thus, by using the distance information, it is possible to detect only the hand of a person behind or at a certain distance.

  Note that the hand region detection device 30 described above determines the final hand region based on the distance information after detecting the hand region based on the complexity, but the image capturing unit is based on the distance information. A range for detecting the hand region may be determined from the input image from 12, and the hand region may be determined within the range.

  Next, a hand region tracking device that tracks the hand regions detected by the hand region detection devices 10, 20, and 30 described above will be described.

  FIG. 40 is a block diagram illustrating a configuration of the hand region tracking apparatus. The hand region tracking device 40 includes an imaging unit 41, an image capturing unit 42 that captures an image captured by the imaging unit 41, and a hand region that predicts the estimated hand region position at the current time from the previous hand region tracking result. Based on the position prediction unit 43, the prediction region evaluation unit 44 that evaluates the estimated hand region position predicted by the hand region prediction unit 43 based on the distribution characteristics of the image at the current time, and the evaluation result in the prediction region evaluation unit 44 A hand region tracking result estimation unit 45 that estimates the hand region position at the current time, a tracking result output unit 46 that outputs the hand region position at the current time estimated by the hand region tracking result estimation unit 45, and a tracking result output unit 46 And a tracking result storage unit 47 for storing the hand area of the current time output from the. Here, the imaging unit 41, the image capturing unit 42, and the tracking result storage unit 47 correspond to the image capturing unit 102, the image capturing unit 102, and the tracking result storage unit 107 shown in FIG. Further, the hand region prediction unit 43 and the prediction region evaluation unit 44 include the tracking result reading unit 104 shown in FIG. 1, and the hand region prediction unit 43, the prediction region evaluation unit 44, the hand region tracking result estimation unit 45, and the tracking result output. The unit 46 includes the tracking processing means 106 shown in FIG.

  The imaging unit 41 is configured by a camera having an imaging element such as a CCD (Charge Coupled Device).

  The image capturing unit 42 captures an image in a format such as JPEG (Joint Photographic Experts Group) from the imaging unit 41.

The hand region position predicting unit 43 reads the tracking result of the previous time (t−1) from the tracking result storage unit 47 and the hand region position prediction unit 43 based on the hand motion prediction model 432 prepared in advance. A predicted position calculation unit 433 that calculates a predicted position s _k (t) that can exist at the current time (t).

The tracking result reading unit 431 reads the hand region position s _k (t−1) at the previous time based on the hand region probability π _k (t−1). That is, at the previous time (t−1), a candidate position with a high hand region probability is read, and the tracking position s _k (t) at the current time t is predicted.

For example, the probability π _k (t−1) is normalized to [0, 1]. That is, a hand region position at the previous time having a probability π _k (t−1) larger than the threshold value T is used. In this case, the sum P _K of probabilities of K hand region positions larger than the threshold T is calculated by the equation (5).

Further, the contribution ratio λ _k (t−1) at each hand region position is calculated by the equation (6).

In addition, the total number of predicted positions of the tracking target at the previous time is N (N> K), and the number of times n _k (t−1) at which each hand region position is selected at the previous time is calculated by Expression (7).

That is, a hand region position whose probability π _k (t−1) is larger than a certain threshold T is selected as a candidate n _k (t−1) times, and a predicted position at the current time is calculated as described below. The

The hand region predicted position calculation unit 433 uses the predicted position s _k (t) _where the tracking target at the current time (t) can exist as the tracking target position s _k (t− 1), the movement speed v (t−1) of the tracking target at the previous time, and the range Δd (t) of the predicted position of the tracking target at the current time.

  In this equation (8), α (t−1) and β (t) are weights (coefficients) for v (t−1) and Δd (t), respectively.

FIG. 41 is a diagram schematically illustrating a configuration of a computing unit that calculates the predicted position s _k (t). The predicted position s _k (t) at the current time is determined by the adders 4331 and 4332, the position s _k (t−1) of the tracking target at the previous time, the motion speed v (t−1) of the tracking target at the previous time, and the prediction range. Δd (t) is added.

  The prediction range Δd (t) is randomly given by a normal random number generator 4333 having an average value am = 0 and a variance σ (t) as shown in the equation (9).

  Further, in order to dynamically determine the prediction range of the tracking target according to the amount of motion of the target, the variance σ (t) was calculated as shown in the equation (10). This variance σ (t) depends on the absolute value | v (t−1) | of the magnitude of the speed (motion amount) of the tracking target at the previous time calculated by the absolute value circuit 4334 and the size of the tracking target area. Based on the parameter r2 (t), it is calculated by the calculator 4335.

  In this equation (10), | v (t−1) | indicates the absolute value of the speed (motion amount) of the tracking target at the previous time, and k1 is a constant indicating the contribution of the motion amount. is there.

  Also, r2 (t) is a parameter depending on the size of the tracking target area, and the variance value σ (t) is determined according to the size of the tracking target as shown in equation (11). In other words, the prediction range was decided dynamically.

  In this equation (11), ObjSize (t−1) indicates the size of the tracking target at the previous time. When this ObjSize (t−1) is greater than or equal to a certain threshold value T0, r2 (t) is calculated in proportion to the magnitude. On the other hand, if ObjSize (t−1) is smaller than a certain threshold value TO, r2 (t) is determined by a certain constant Const. This is a constant Const that increases the prediction range because there is a high possibility that the previous time tracking result is incorrect. That is, ObjSize (t−1) can be the area of the skin color area calculated by the binarization process of the hue image Hue and the saturation image Sm at the tracking position at the previous time. For example, as shown in FIG. 42, when the hand region a detected at the previous time is a hand, the skin color region b as shown in FIG. On the other hand, when the hand region c detected at the previous time is not a hand as shown in FIG. 44, the skin color region d as shown in FIG. 45 is calculated to be smaller than a certain threshold T0.

As described above, the hand region position prediction unit 43 reads the hand tracking result at the previous time (t−1) from the tracking result storage unit 47, and based on the hand motion prediction model based on the hand size and the hand movement speed. Thus, by calculating the predicted position s _k (t) of the hand region where the current time (t) can exist, the hand region can be predicted with high speed and high accuracy.

The hand region evaluation unit 44 calculates the HSV distribution of the predicted region image of the hand region predicted position s _k (t) predicted by the hand region position prediction unit 43 and the HSV of the current time image input from the image capturing unit 42. An image distribution calculation unit 441 that calculates the distribution, and an evaluation value calculation unit that evaluates the prediction region based on the HSV distribution image calculated by the image distribution calculation unit 441 and the hand region model 442 stored in the tracking result storage unit 47 443. The hand region model 442 includes, for example, information such as a histogram representing the skin color feature of the hand region image, and is compared with the histogram of the predicted region image.

The hand region evaluation unit 44 converts the predicted region image (24 Bit, RGB) at the input predicted position s _k (t) into an HSV image (8 Bit). In this HSV image, colors are represented by H (Hue / hue, 3 bits), S (Saturation / saturation, 3 bits), and V (value / lightness, 2 bits). For example, when an RGB image as shown in FIG. 46 is converted into an HSV image, an HSV image as shown in FIG. 47 is obtained. FIG. 48 shows a histogram of the hand region a of the HSV image shown in FIG. This histogram shows a characteristic distribution with respect to human skin color.

  Therefore, the distribution characteristic of the HSV image in the prediction region is obtained by the equation (12).

In this equation (12), HSV (i, j) is a luminance value of the HSV image, and δ (x) is a delta function that gives a value of 1 when x = 0, and 0 otherwise. That is, when k takes a value of 0 to 255 and is the same as the brightness value of HSV (i, j), the distribution characteristic function histogram (k) increases by the value of p _{(i, j)} (r).

p _{(i, j)} (r) is inversely proportional to the distance r between the coordinate position (i, j) of the image at the current time and the center position (xc, yc) of the prediction region, as shown in equation (13). The distribution characteristic function histogram (k) is weighted. That is, the pixel near the center has a higher contribution, and the contribution becomes lower as the pixel becomes farther from the center. Thereby, the influence of a background can be reduced.

  In addition, α (i, j) depends on, for example, a preset mask area. For example, when the face area detected by the face detector is used as a mask area and the prediction area includes a part of the face area, the function is adjusted to reduce the probability that the pixel (i, j) is a hand area. . This α (i, j) is determined by whether or not the pixel (i, j) is included in a mask area such as a face area, as shown in equation (15).

  In this way, by adjusting the probability based not only on the histogram of the HSV image but also on the mask area, even if the similarity between the HSV value (characteristic) of the hand area and the HSV value of the face area increases, a hand area tracking error occurs. Can be prevented and robust hand region tracking can be realized.

  Further, the hand region evaluation unit 44 calculates a matching cost between the histogram calculated by the equation (12) and the hand region model registered in the tracking result storage unit 47 as in the equation (16), and calculates the prediction region. evaluate.

  Thus, by evaluating the similarity between the hand region model registered in the tracking result storage unit 47 and the prediction region by matching, robust hand region tracking can be realized.

  The hand region tracking result estimation unit 45 calculates a probability based on the matching cost calculated by the prediction region evaluation unit 44, and a tracking result estimation unit 452 that estimates the tracking result based on the probabilities of all prediction regions. And.

The probability calculation unit 451 uses the matching cost calculated by the prediction region evaluation unit 44 to calculate a probability π _k (t) that is a hand region using Equation (17). The calculated probability π _k (t) is stored in the tracking result storage unit 47 together with the predicted position s _k (t).

The tracking result estimation unit 452 estimates the tracking result (position) according to the equation (18) based on the predicted position s _k (t) of each tracking target at the current time and the probability π _k (t) based on the similarity evaluation.

  Here, S (t) indicates the position of the tracking target (for example, hand or face) at the current time. K indicates the predicted number of positions where the tracking target may exist at the current time.

As described above, the hand region tracking result estimation unit 45 estimates the tracking result (position) by the predicted position s _k (t) of each tracking target at the current time and the probability π _k (t) based on the similarity evaluation, The hand region can be tracked at high speed and with high accuracy.

  The tracking result output unit 46 outputs the position S (t), size, probability, and the like of the hand region estimated by the hand region tracking result estimation unit 45 to the tracking result storage unit 47.

  The tracking result storage unit 47 stores the tracking result output from the tracking result output unit 46. Further, the tracking result of the previous time is read out to the hand region position prediction unit 43. Further, for example, the hand region model at a position with a high estimation probability is read out to the prediction region evaluation unit 44, and the hand region initial model is updated.

  Next, the overall operation of the hand region detection devices 10, 20, 30 and the hand region tracking device 40 described above will be described with reference to the flowchart shown in FIG.

  First, the object tracking device including the hand region detection devices 10, 20, 30 and the hand region tracking device 40 determines whether or not the tracking mode is for tracking the hand region (step S1). That is, it is determined whether or not the hand region to be tracked is detected and registered.

  If it is determined in step S1 that the mode is not the hand region tracking mode, that is, if it is necessary to register the hand region, the process proceeds to step S2 to detect the hand region. This hand region detection can be performed using the hand region detection devices 10, 20, and 30 described above. That is, a hand candidate area is detected from an input image based on a person's statistical skin color feature, a shape complexity of the hand candidate area is calculated, and a hand area is calculated from the hand candidate area based on the shape complexity. To detect.

  In step S3, the object tracking device determines whether or not the hand region has been detected. If the hand area can be detected in step S3, the process proceeds to step S4, and the hand area is registered in the tracking result storage unit 47 as a hand area model. On the other hand, if the hand region cannot be detected, the process returns to step S1 to determine the mode.

  If it is determined in step S1 that the mode is the tracking mode, the hand region position prediction unit 43 reads the tracking result at the previous time (t-1) from the tracking result storage unit 47 (step S5). The hand region evaluation unit 44 reads the hand region model from the tracking result storage unit 47 (step S6).

  In step S7, the hand region tracking device 40 performs a hand region tracking process as shown in FIGS. Here, FIGS. 50 to 53 respectively show an input image example at the previous time (t−1), a hand region predicted position example at the current time (t), an image example at the current time (t), and the current time (t). An example of the hand region estimated position is shown.

First, the hand region position prediction unit 43 calculates the hand region predicted position at the current time (t) based on the hand motion prediction model from the tracking result at the previous time (t-1) read in step S5. . For example, when a tracking result s _k (t−1) as shown in FIG. 50 is inputted, hand region predicted positions s _k1 (t), s _k2 ( t), s _k3 (t), s _k4 (t), and s _k5 (t) are calculated. The prediction region evaluation unit 44 evaluates the similarity of each estimated hand region position predicted by the hand region position prediction unit 43 based on an image at the current time as shown in FIG.

The hand region tracking estimation unit 45 calculates a probability π _k (t) based on similarity evaluation with the hand region model for each predicted position evaluated by the prediction region evaluation unit 44. For example, in the case of the image shown in FIG. 52, the probability calculation result of each predicted position is s _k5 (t), s _k3 (t), s _k2 (t), s _k4 (t), s _k1 ( t). Then, as shown in FIG. 53, the tracking position S (t) is estimated from the predicted position S _k (t) of each tracking target at the current time and the probability π _k (t) based on the similarity evaluation.

  In step S8, the hand region tracking device 40 stores the tracking result such as the tracking position S (t), the size, and the probability estimated in the tracking process in step S7 in the tracking result storage unit 47, and the current time (t). The tracking process of the hand region is terminated. Then, the time is incremented, and the next time tracking process is performed.

  By evaluating the similarity between the image features of the hand region predicted in this way and the image features of the hand region model registered by hand region detection (initialization), an evaluation value (possible that these prediction regions are hand regions) ) And calculating the probability that each prediction area is a hand area based on the evaluation value, and finally determining the tracking result of the hand area from the position and probability of these prediction areas. The hand area can be accurately tracked.

  In the hand region tracking device 40 described above, distance information may be used to reduce the influence of the background, for example, when a background region similar to the HSV distribution characteristic of the hand region exists. FIG. 54 shows a configuration of the hand region tracking apparatus 50 that estimates distance information using a stereo camera, for example, and removes an area that is a predetermined distance or more as a background. In addition, the same code | symbol is attached | subjected to the structure same as the hand region tracking apparatus 40 mentioned above, and description is abbreviate | omitted.

  The hand region tracking device 50 is estimated by the imaging unit 51, the distance information estimation unit 52 that estimates the distance information of the object in the image based on the image captured by the imaging unit 51, and the distance information estimation unit 52. A distance image calculation unit 53 that calculates an image region of an object at a predetermined distance based on the distance information, and a background image removal unit 54 that removes a background image at a distance farther than the distance image calculated by the distance image calculation unit 53 A hand region position prediction unit 43 that predicts the estimated hand region position at the current time from the hand region tracking result at the previous time, and the estimated hand region position predicted by the hand region prediction unit 43 as the distribution characteristics of the image at the current time. A prediction region evaluation unit 44 that evaluates based on the result, a hand region tracking result estimation unit 45 that estimates a hand region position at the current time based on the evaluation result in the prediction region evaluation unit 44, and a hand region tracking result estimation unit 45 A tracking result output unit 46 for outputting a hand region position of the constant has been present time, and a and a tracking result storage unit 47 for storing the hand area of the current time outputted from the tracking result output section 46.

  The imaging unit 51 includes two or more cameras and is installed with a predetermined positional relationship.

  The distance information estimation unit 52 uses the images captured from a plurality of viewpoints (projection centers) having a predetermined positional relationship, and calculates the distance between each point in the captured image and the projection center based on the principle of so-called “triangulation”. Estimate distance information.

Distance image calculation unit 53, by the threshold D ₀ of the distance (Depth) from preset camera, for example, calculates a distance image consisting of pixels at distance less than the threshold value D _0.

  The background image removal unit 54 masks an area other than the distance image calculated by the distance image calculation unit 53 as a background image.

FIGS. 55 to 58 show examples of image processing in the hand region tracking device 50 in which an area that is more than a predetermined distance is removed as a background. For example, when an image as shown in FIG. 55 is picked up by the image pickup unit 51, the hand region tracking device 50 estimates the distance information of each pixel by the distance image estimation unit 52, and calculates the distance image as shown in FIG. cutting out the person area of the distance smaller than the threshold value D ₀ in the part 53 as the foreground. The background image removal unit 54 generates a mask image that masks areas other than the person region cut out by the distance image calculation unit 53. In this mask image, for example, as shown in FIG. Then, the background image removal unit 54 removes the background from the image captured by the imaging unit 51 using the mask image. Therefore, the image at the current time input to the prediction region evaluation unit 44 is an HSV image with the background removed by the mask image as shown in FIG. The subsequent processing is the same as that of the hand region tracking device 40 described above.

  By removing skin color candidate areas that are more than a predetermined distance as a background in this manner, for example, even if there is another person's hand in the background, the hand area within a predetermined distance is tracked, so the robustness of the real environment is further improved. Can be increased.

  It should be noted that the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention. For example, in the present embodiment, the human hand region is tracked as an object in the image, but the face region may be tracked, for example. In this case, as in the hand region tracking, first, a face region is detected and registered by the face detector. Then, the face area can be tracked by the tracking process as described above.

  Further, for example, in the above-described embodiment, the hardware configuration has been described. However, the present invention is not limited to this, and arbitrary processing is realized by causing a CPU (Central Processing Unit) to execute a computer program. It is also possible. In this case, the computer program can be provided by being recorded on a recording medium, or can be provided by being transmitted via the Internet or another transmission medium.

It is a figure for demonstrating the whole process of the hand region tracking apparatus of one Embodiment which concerns on this invention. It is a block diagram which shows the structure of the hand region detection apparatus of one Embodiment which concerns on this invention. It is a figure which shows the example of an input image. It is a figure which shows the example of an image process by RGB information. It is a figure which shows the example of selection of a skin color candidate area | region. It is a figure which shows the example of an image from which the skin color candidate area | region was selected. It is a figure which shows the image processing example by a hue. It is a figure which shows the hue distribution example of a skin color candidate area | region. It is a figure which shows the image processing example by saturation. It is a figure which shows the saturation distribution example of a skin color candidate area | region. It is a figure for demonstrating the detection of the hue area | region of a skin color. It is a figure for demonstrating the detection of the saturation area | region of a skin color. It is a figure which shows the example of a detection of the hue area | region in a hue image. It is a figure which shows the example of a detection of the hue area | region in a saturation image. It is a figure which shows the example of detection of a hand candidate area | region. It is a figure for demonstrating the complexity according to the shape of a hand. It is a figure which shows the example of an input image at the time of closing a hand. It is a figure which shows the calculation result of the shape complexity at the time of closing a hand. It is a figure which shows the example of an input image at the time of opening two fingers. It is a figure which shows the calculation result of the shape complexity at the time of opening two fingers. It is a figure which shows the example of an input image at the time of opening a hand. It is a figure which shows the calculation result of the shape complexity at the time of opening a hand. It is a figure for demonstrating the shape complexity based on the distance from the center of a hand candidate area | region. It is a figure which shows the calculation result of the distance from a center. It is a block diagram which shows the other structure (the 2) in the hand area | region detection apparatus of one Embodiment which concerns on this invention. It is a figure which shows the example of an image in the time t. It is a figure which shows the example of an image in the time t + 1. It is a figure which shows the example of a detection of a motion area | region. It is a figure which shows the example which detected the hand area | region from the movement area | region. It is a figure which shows the example of detection of a skin color candidate area | region. It is a figure which shows the example of determination of a hand area | region. It is a figure which shows the example of detection of the skin color candidate area | region at the time of not detecting a motion area | region. It is a figure which shows the example of a hand area | region detection result at the time of not detecting a movement area | region. It is a block diagram which shows the other structure (the 3) in the hand area | region detection apparatus of one Embodiment which concerns on this invention. It is a figure which shows the example of an input image. It is a figure which shows the example of detection of a skin color candidate area | region. It is a figure which shows the example of a detection result of a hand area | region. It is a figure which shows the example which estimated the distance of the hand candidate area | region from distance information. It is a figure which shows the detection result of the hand area | region based on distance information. It is a block diagram which shows the structure of the hand region tracking apparatus of one Embodiment which concerns on this invention. It is a figure for demonstrating the process in a hand area position estimation part. It is a figure which shows the example of an image when a hand area | region is detected in the previous time. It is a figure which shows the example of an image at the time of measuring the area of a hand area | region by the binarization process. It is a figure which shows the example of an image when a hand area | region is not detected in the previous time. It is a figure which shows the example of an image at the time of measuring the area of a hand area | region by the binarization process. It is a figure which shows the imaged RGB image example. It is a figure which shows the example which converted the RGB image into the HSV image. It is a figure which shows the histogram of the hand area | region in an HSV image. It is a flowchart of the tracking operation | movement in one Embodiment of this invention. It is a figure which shows the example of an image of the hand region position in the previous time. It is a figure which shows the hand region estimated position example in the present time. It is a figure which shows the example of an image at the time of evaluating a hand region estimated position. It is a figure which shows the example of an image of the tracking position of a hand area | region. It is a block diagram which shows the other structure (the 2) in the hand region tracking apparatus of one Embodiment which concerns on this invention. It is a figure which shows the example of an image in the present time. It is a figure which shows the example of a distance image based on distance information. It is a figure which shows the example of a mask image. It is a figure which shows the example of an image which performed the background removal by the mask image.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 hand area detection apparatus, 11 imaging part, 12 image capture part, 13 skin color candidate area selection part, 14 hand candidate area extraction part, 15 shape complexity calculation part, 16 hand area detection part, 20 hand area detection apparatus, 21 Delay circuit unit, 22 motion region detection unit, 23 hand region determination unit, 30 hand region detection device, 31 reference imaging unit, 32 distance information acquisition unit, 33 hand region determination unit 40 hand region tracking device, 41 imaging unit, 42 image Capture unit, 43 hand region position prediction unit, 44 prediction region evaluation unit, 45 hand region tracking result estimation unit, 46 tracking result output unit, 47 tracking result storage unit, 50 hand region tracking device, 51 imaging unit, 52 distance image Estimator, 53 Distance image calculator, 54 Background image remover

Claims (13)

In an object tracking device that tracks an object to be tracked from an image,
With respect to the estimation object for estimating the object position of the object, a prediction range in which the estimation object moves from information of the estimation object of the previous time image is calculated, the prediction range, the estimation object position of the estimation object of the previous time image, and Position prediction means for predicting the estimated object position of the current time image based on the movement speed;
Position evaluation means for evaluating a plurality of estimated object positions predicted by the position prediction means based on object similarity between a current time image of the estimated object and a pre-registered object image;
An object tracking device comprising: estimation means for estimating an object position of a current time image based on each estimated object position evaluated by the position evaluation means.   2. The object tracking device according to claim 1, wherein the position predicting unit calculates the prediction range based on one of the size, color, and shape of the object of the previous time image and the movement speed.   2. The object tracking device according to claim 1, wherein the position predicting unit predicts the estimated object position of the current time image independently from each of the plurality of estimated objects of the previous time image.   The position evaluation means evaluates each estimated object position predicted by the position prediction means based on similarity between the distribution characteristics of the current time image of the estimated object and the distribution characteristics of the object image registered in advance. The object tracking device according to claim 1, wherein:   The position evaluation means sets the similarity between the distribution characteristic of the current time image of the estimated object and the distribution characteristic of the object image registered in advance to the distance from the center of the estimated object position predicted by the position prediction means. 5. The object tracking device according to claim 4, wherein the evaluation is performed based on the evaluation.   The position evaluation means evaluates the similarity between the distribution characteristics of the current time image of the guess object and the distribution characteristics of the object image registered in advance based on other objects tracked at the same time. The object tracking device according to claim 4.   The position evaluation means may change the evaluation of the estimated object position based on the evaluation of the other object when the estimated object position predicted by the position prediction means is included in the object area of the other object. 7. The object tracking device according to claim 6, wherein   7. The object tracking device according to claim 6, wherein the other object is a human face area.   The tracking means calculates a probability that the estimated object position evaluated by the position evaluation means is an object region based on the evaluation result of the object position, and based on the probability of each estimated object position, the object of the current time image The object tracking device according to claim 1, wherein the position is estimated. Distance information acquisition means for acquiring distance information obtained by estimating the position of each pixel in the three-dimensional space based on images from a plurality of cameras having a predetermined positional relationship;
The object tracking device according to claim 1, further comprising: a background removing unit that removes the background of the object from the image based on the distance information acquired by the distance information acquiring unit.   The object tracking apparatus according to claim 1, wherein the object is a human hand area. In an object tracking method for tracking an object to be tracked from an image,
With respect to the estimation object for estimating the object position of the object, a prediction range in which the estimation object moves from information of the estimation object of the previous time image is calculated, the prediction range, the estimation object position of the estimation object of the previous time image, and A position prediction step for predicting the estimated object position of the current time image based on the movement speed;
A position evaluation step for evaluating a plurality of estimated object positions predicted in the position prediction step based on object similarity between a current time image of the estimated object and a pre-registered object image;
An object tracking method comprising: an estimation step of estimating an object position of a current time image based on each estimated object position evaluated in the position evaluation step. In a program that executes a process of tracking an object to be tracked from an image,
With respect to the estimation object for estimating the object position of the object, a prediction range in which the estimation object moves from information of the estimation object of the previous time image is calculated, the prediction range, the estimation object position of the estimation object of the previous time image, and A position prediction step for predicting the estimated object position of the current time image based on the movement speed;
A position evaluation step for evaluating a plurality of estimated object positions predicted in the position prediction step based on object similarity between a current time image of the estimated object and a pre-registered object image;
An estimation step of estimating an object position of a current time image based on each estimated object position evaluated in the position evaluation step.

Images