JP5539102B2 - Object detection apparatus and object detection method - Google Patents

Description

  The present invention relates to an apparatus and method for detecting an object in an image.

  Conventionally, a technique for detecting a human body as a specific object in an image is known. For example, Non-Patent Document 1 discloses a technique for detecting a person in an image by using a brightness gradient histogram in a predetermined range of the image as a feature amount and performing identification by Support Vector Machine. However, although the technique disclosed in Non-Patent Document 1 is effective for detecting a human body in a specific posture, there is a problem that it is difficult to detect when the posture changes.

  Non-Patent Document 2 proposes a human body detection method that solves such problems and allows deformation of the human body. According to Non-Patent Document 2, a technique is proposed in which detection scores of detectors corresponding to human body parts are added in consideration of a relative positional relationship with the center of gravity. That is, first, a detector corresponding to each part of the human body is obtained by learning. Next, detection is performed with respect to the sample image of the human body with those detectors, and the probability distribution of the relative position of the detection point with respect to the center of gravity of the human body is learned for each detector. By using the detector thus learned and the probability distribution of the relative position with respect to the center of gravity of the human body, the human body is detected from the image. In the detection, first, the detection score of each part of the human body is calculated from the target image. The score of the detector of each part of the human body is voted for a bin corresponding to the relative position of the center of gravity of the human body viewed from the detector. The votes from all detectors are summed up to obtain the final human detection result.

  Similarly, the technique disclosed in Patent Document 1 is a human body detection method that allows deformation of the human body. In the technique disclosed in Patent Document 1, candidates that are considered to be human body parts are extracted from an image, the probability of the part likelihood of each candidate, the positional relationship between the part candidates, and the part candidate with a high probability as a detection target. A combination is selected and the humanity is calculated stochastically.

  As described above, in the methods described in Non-Patent Document 2 and Patent Document 1, learning of the arrangement relationship of human body parts is performed. On the other hand, in the method described in Non-Patent Document 3, a model regardless of the geometrical arrangement relationship of each partial feature, that is, a so-called BoK (Bag of Keypoints) model is used in object recognition.

JP 2005-165923 A

Navneet Dalal and Bill Triggs, “Histograms of Oriented Gradients for Human Detection,” IEEE Computer Vision and Pattern Recognition, pp. 886-893, 2005 Lubomir Bourdev and Jitendra Malik, “Poselets: Body Part Detectors Trained Using 3D Human Pose Annotations,” IEEE International Conference on Computer Vision, 2009 Gabriella Csurka, Christopher R. Dance, Lixin Fan, Jutta Willamowski, Cedric Bray, “Visual Categorization with Bags of Keypoints,” In Workshop on Statistical Learning in Computer Vision, European Conference on Computer Vision 2004 Krystian Mikolajczyk and Cordelia Schmidt, “Scale & Affine Invariant Interest Point Detectors,” International Journal of Computer Vision, 60 (1), pp. 63-86, 2004. ) D. Comaniciu, M. Peter, “MeanShift: A Robust Approach Toward Feature Space Analysis,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24 (5), pp. 603-619. (Referred to in the detailed description)

  However, in the object detection method that divides the human body into parts and uses the relative positional relationship between the parts as in the techniques of Patent Document 1 and Non-Patent Document 2 described above, the relative positions of the parts are described as follows. There was a problem that the learning of relationships was complicated.

  As described above, in the technique disclosed in Non-Patent Document 2, the position relationship of each part of the human body is described by describing the positional relationship of each part of the human body as an existence probability distribution in a relative coordinate system based on the center of gravity of the human body image. Has a degree of freedom. However, for example, regarding the upper body part of the human body, there is a difference in the range of motion due to joint structure restrictions between the existence probability distribution when the human body is facing the front and the existence probability distribution when the human body is facing sideways. Therefore, the distribution is different. Therefore, in Non-Patent Document 2, it is necessary to learn the existence probability distribution of each part of the human body at least for each direction of the human body.

  Further, in the technique disclosed in Patent Document 1, candidates that are considered to be human body parts are extracted from an image, the probability of the part likelihood of each candidate, the positional relationship of the part candidates, and the part with a high probability as a detection target A combination of candidates is selected, and the likelihood of human body is calculated stochastically. Therefore, similarly to the technique disclosed in Non-Patent Document 2 described above, the technique disclosed in Patent Document 1 needs to learn the arrangement relationship of human body part candidates at least for each direction of the human body.

  On the other hand, in the object recognition using the model regardless of the geometric arrangement relationship of each partial feature as in Non-Patent Document 3, it is not necessary to learn the existence probability distribution of each part of the human body due to the difference in the orientation of the human body. . However, when each part of the human body has poor features and the background is complicated, the BoK model has a problem that it is difficult to detect.

  The present invention has been made in view of the above-described problems, and detects an object with high detection rate from an image without considering the relative position of each part constituting the object with respect to the object with posture variation. The purpose is to make it possible.

In order to achieve the above object, an object detection apparatus according to an aspect of the present invention has the following arrangement. That is,
An object detection device for detecting a specific object from an image,
A storage unit that stores a pose parameter related to a specific object, a partial feature of the image, and a probability that the partial feature exists in the image in the pose parameter in association with each other;
An acquisition means for acquiring a partial feature by detecting a feature point from an image to be detected of the specific object, extracting a local image including the detected feature point, and calculating a feature amount;
Input means for inputting a plurality of posture parameters irrespective of the image to be detected;
Selection means for selecting a predetermined number of partial features in descending order of probability stored in the storage means for each of the plurality of posture parameters;
Collating means for calculating a matching degree indicating a degree of correlation between the partial feature acquired by the acquiring means and the partial feature selected by the selecting means;
Determination means for determining whether or not the specific object is present in the detection target image based on the matching degree calculated by the matching means with respect to the plurality of posture parameters.

  ADVANTAGE OF THE INVENTION According to this invention, the target object can be detected from an image with a high detection rate, without considering the relative position of each part which comprises the target object with respect to the target object with attitude | position fluctuation | variation.

The figure which shows an example of the image which the object detection apparatus which concerns on embodiment makes object. The figure which shows an example of the image processing process by the object detection apparatus which concerns on embodiment. The block diagram which shows the function structural example of the object detection apparatus which concerns on embodiment. The figure which shows an example of the parameter of the object made into the object detection apparatus which concerns on embodiment. (A) is a figure which shows the data structural example of a correspondence relationship memory | storage part, (b) is a figure which shows the data structural example of a partial feature memory | storage part. The flowchart which shows the process of the object detection apparatus which concerns on embodiment. The flowchart which shows the process of the object detection apparatus which concerns on embodiment.

  Hereinafter, the present invention will be described in detail according to one of its preferred embodiments with reference to the accompanying drawings.

(Overview)
The object detection apparatus according to the present embodiment detects a person as a specific object in an image. Hereinafter, an example is shown using figures. In the present embodiment, the person in the image is the detection target object, but the present invention is not limited to this. For example, an object that can specify its posture and shape with one or more parameters, such as a rigid body whose posture can be defined by camera angle, such as an automobile, or a flexible object that can describe deformation with parameters, such as a spring This can be set as a detection target object.

  FIG. 1 shows an image 100 to be detected in the present embodiment. In the image 100, an object 110 indicates a person to be detected. The detection window area 120 indicates an area that is a target for detecting a specific object (person). The detection window area 130 is a detection target area different from the detection window area 120. An object 140 indicates a background object. As will be described later, in the object detection apparatus, partial features of a person image are learned in advance in association with posture parameters.

  FIG. 2 shows an example of partial features learned in advance in the object detection apparatus according to the present embodiment. The partial feature group 210 is a schematic diagram of partial features of each part of a person. The partial feature 221, the partial feature 222, the partial feature 223, the partial feature 224, and the partial feature 225 are schematic views of the partial feature, respectively.

  The object detection device selects a plurality of partial features, for example, the partial feature 221, the partial feature 222, and the partial feature 223 from the partial feature group 210 based on the posture parameter input from the outside. Such partial feature selection processing is performed based on, for example, the probability that each partial feature exists in the input posture parameter (preliminarily obtained by learning) (details will be described later). The selected partial feature is collated with the partial feature extracted in the detection window area 120. If the sum of the collation degrees of each partial feature is equal to or greater than a threshold value, the corresponding voting parameter space bin is voted. The posture parameter space is a space in which the number of parameters of the posture parameter is the number of dimensions, and the plurality of bins correspond to a plurality of partial spaces obtained by dividing the posture parameter space into a plurality of spaces.

  A new posture parameter is input again from the outside, and the above processing is repeated based on the new posture parameter. That is, based on the new posture parameter, a plurality of partial features such as the partial feature 221, the partial feature 224, and the partial feature 225 are selected from the partial feature group 210, and collation and voting are performed in the same manner. In this manner, the posture parameter input, the partial feature selection, collation, and voting are repeated a predetermined number of times. If the extreme value of the distribution voted in the posture parameter space exceeds a certain threshold after the specified number of repetitions, it is determined that a person exists in the detection window area 120. Further, it is determined that the posture of the person is a posture expressed by a posture parameter corresponding to the extreme value. Similar processing is performed on other regions in the image, such as the detection window region 130, and finally all persons are detected from the image. The above processing will be described in more detail below.

(Constitution)
FIG. 3 is a diagram showing an outline of the object detection apparatus 300 according to this embodiment. In addition, the function of each part shown below is implement | achieved when the CPU (not shown) which the object detection apparatus 300 has runs the computer program stored in the computer-readable memory (not shown). Therefore, the object detection apparatus 300 can be realized using an information processing apparatus such as a personal computer. In the object detection device 300, the image input unit 301 may be, for example, a device that can shoot in real time (for example, a camera), a device that optically reads an image (for example, a scanner), or an image that has been captured or read in advance. May be stored. The image input from the image input unit 301 is output to the feature amount calculation unit 302.

The feature amount calculation unit 302 calculates the feature amount (partial feature) of the part of the image input from the image input unit 301. That is, a feature point is detected from an image to be detected, a partial feature including the detected feature point (a local image near the feature point) is extracted, and a feature amount is calculated to obtain a partial feature. In this embodiment, as this partial feature,
A local image is obtained by normalizing the shape of the AffineInvariant Region near the feature point detected by a corner detector such as Harris Corner Detector as described in Non-Patent Document 4,
It is assumed that Histograms of Oriented Gradients described in Non-Patent Document 1 are calculated for the obtained local image. However, the partial features that can be used in the present embodiment are not limited to this, and those obtained by using other known image feature amount calculation methods can be used.

  The posture parameter input unit 303 receives a posture parameter representing the posture of the target object, and outputs the posture parameter to the partial feature storage unit 304, the partial feature selection unit 305, and the correspondence relationship storage unit 307. As the posture parameter to be input, for example, posture annotation data attached to an image input from the image input unit 301 is used when learning the target object. That is, the posture parameter input unit 303 extracts posture parameters from the posture annotation data added to the image, and outputs the posture parameters to the partial feature storage unit 304, the partial feature selection unit 305, and the correspondence relationship storage unit 307. At the time of detecting the target object, the posture parameter input unit 303 uses data generated by an external program, for example, a random number generated according to a certain probability density, as the posture parameter. That is, the posture parameter input unit 303 inputs a plurality of posture parameters regardless of the detection target image.

In the present embodiment, the detection target is a human body, and the posture parameters include data on camera angles and joint angles. FIG. 4 shows an example of posture parameters when the target is a human body. In FIG. 4, the angle φ _k represents the kth parameter that defines the camera angle in the camera 410, and in FIG. 4, is the pitch with respect to the horizontal direction. The angle θ _j represents the jth parameter that defines the posture of the human body, and in FIG. 4 is the angle formed by the upper arm and the forearm. When the camera angle is fixed, the camera parameter (angle φ _k ) is not required among the attitude parameters. Further, when the specific object to be detected among the posture parameters is not deformed and only the camera parameter is changed, the parameter (angle θ _j ) related to the deformation is unnecessary.

The partial feature storage unit 304 stores the partial feature output from the feature amount calculation unit 302 and the posture parameter output from the posture parameter input unit 303 in association with each other. Cluster all of the partial features obtained by the feature quantity calculation unit 302 for a plurality of sample images, and let the i-th cluster obtained be Π _i and the partial feature at the center of the cluster Π _{i be} π _i . . That is, the partial feature π _i is a partial feature representing the cluster Π _i . Note that the method for determining the partial feature representing the cluster is not limited to this, and for example, an average of all the partial features existing in the cluster may be used. Further, a set of a partial feature π and a posture parameter θ = (φ ₁ , φ ₂ , φ ₃ , θ ₁ , θ ₂ ,..., Θ _n ) assigned thereto is assumed to be x = (π, θ). Under the posture parameter θ, the probability P (Π _i | θ) that a partial feature belonging to the cluster Π _i appears is calculated according to the following equations (1) and (2).

Here, n (X) is the number of elements in set X, theta _k respectively represent the k-th subspace posture parameter space. The implementation of the program, for example, theta _k corresponds to the k-th bin at the time of separated whole posture parameter space into a plurality of bins. That is, the set X ^k _i in the expression (1) represents a set in which the partial feature π of the element x = (π, θ) is included in the cluster Π _i and the posture parameter θ is included in Θ _k. . Partial feature storage unit 304, and the i-th cluster [pi _i partial feature in the center of [pi _i, P calculated by the equation (2) | in association with ([pi _i theta), for example, in FIG. 5 (b ). In FIG. 5B, for example, for the first cluster (center partial feature π ₁ ), the probability p (Π ₁ | θ) existing under each posture parameter θ is recorded in columns 501, 502, etc. The As described above, the partial feature storage unit 304 stores the posture parameter (θ) related to a specific object, the partial feature (π _i ) of the image, and the probability P (Π _i | θ) is stored in association with each other.

The partial feature selection unit 305 receives the posture parameter θ from the posture parameter input unit 303, and based on P (Π _i | θ) stored in the partial feature storage unit 304, a predetermined number (hereinafter, M pieces) of partial features. Select.

The object detection hypothesis generation unit 306 collates the calculated partial feature with the partial feature selected by the partial feature selection unit 305, extracted from the detection window area 120 of the image shown in FIG. Then, a hypothesis of the presence / absence of the detection target object is generated. One of the partial features extracted and calculated by the feature amount calculation unit 302 is π, and a set of all the calculated partial features is Q. When one of the partial features selected by the partial feature selection unit 305 is π ′, the matching degree is given by the following equation (3), for example. According to Expression (3), the matching degree is given as a correlation value when the correlation with π ′ is the maximum for all the partial features belonging to the set Q. That is, the collation degree indicates the degree of correlation between the partial feature acquired by the feature amount calculation unit 302 and the partial feature selected by the partial feature selection unit 305.

  Note that the matching degree is not limited to the method of calculating by Equation (3), and for example, the maximum value of all partial features belonging to the set Q is calculated with the likelihood of the discriminator that has previously learned about π ′. May be. The object detection hypothesis generation unit 306 calculates a matching degree for all M partial features selected by the partial feature selection unit 305, and outputs the sum as an object detection hypothesis.

The correspondence storage unit 307 stores the object detection hypothesis generated by the object detection hypothesis generation unit 306 and the posture parameter input from the posture parameter input unit 303 in association with each other. FIG. 5A shows a data configuration example in the correspondence relationship storage unit 307. FIG. 5A shows a storage area corresponding to the posture parameter. In the example shown in FIG. 5A, the posture parameter is given as a two-dimensional vector composed of θ ₁ and θ _2. However, in an actual device, the number of dimensions necessary for expressing the posture of the target object is set. A vector is used. Each element of the posture parameter is divided into bins having a width Δ. For example, in the cell in the first row and the second column in the table of FIG. 5A, the posture parameters input from the posture parameter input unit 303 are in the range of 0 ≦ θ ₁ <Δ and Δ ≦ θ ₂ <2Δ. When the value is a value, a value obtained by adding all the values output from the object detection hypothesis generation unit 306 is included.

  The detection confirmation unit 308 obtains the extreme values of the data stored in the correspondence relationship storage unit 307, and determines that the target object has been detected when the threshold value exceeds a predetermined threshold value among the extreme values and becomes the maximum value. And the corresponding posture parameter is determined as the posture of the target object. As a method of obtaining the extreme value of the data stored in the correspondence relationship storage unit 307, for example, there is a mean shift method described in Non-Patent Document 5. The above is the configuration part related to the object detection apparatus according to the present embodiment.

(processing)
Next, processing performed by the object detection device 300 of the present embodiment will be described using the flowcharts shown in FIGS. 6A and 6B. Note that the program code according to the flowchart is stored in a memory such as a RAM or a ROM (not shown) in the apparatus of the present embodiment, and is read and executed by a CPU (not shown). First, learning of posture parameters and partial features using the correct sample image is performed in steps S601 to S606 in FIG. 6A, and then object detection in the target image is performed in steps S607 to S616 in FIG. 6B. Note that, for example, the computer for learning processing (information processing apparatus) that executes steps S601 to S606 and the computer for detection processing (information processing apparatus) that executes steps S607 to S616 are not necessarily the same.

First, in step S601, the image input unit 301 reads a positive sample image _Ip1 . Subsequently, in step S602, the posture parameter input unit 303 reads the posture parameter θ _p1 corresponding to the positive sample image I _p1 . Since this process is a learning process, as described above, the attitude parameter input unit 303 reads the attitude parameter θ _p1 from the attitude annotation data given to the positive sample image I _p1 .

Subsequently, in step S603, the feature amount calculation unit 302 extracts partial features of the positive sample image _Ip1 . Subsequently, in step S604, it is confirmed whether or not the processing in steps S601 to S603 has been completed for all positive samples. If there is a positive sample that has not been processed yet, the processing of steps S601 to S603 is performed on that sample. On the other hand, if the above process has been completed for all positive samples, the process proceeds to step S605.

In step S605, the feature amount calculation unit 302 clusters all partial features obtained by the processing in steps S601 to S604 based on the feature similarity. Then, the i-th cluster is set as Πi, and the partial feature representing the cluster is stored in the partial feature storage unit 304 as the partial feature π _{i at} the center of the cluster. Subsequently, in step S606, the feature amount calculation unit 302 calculates, for each partial feature, a probability P (Π _i | θ) that the partial feature belonging to the cluster Π _i obtained in step S605 appears under the posture parameter θ. Store in the feature storage unit 304.

  Through the above steps S601 to S606, learning of the correspondence between the partial features and the posture parameters is completed from the positive sample to be detected, and information as shown in FIG. 5B is stored in the partial feature storage unit 304. The Subsequently, in S607 to S616, a specific object (human body in this example) is detected from the target image.

  First, in step S607, the image input unit 301 inputs an image to be detected. Subsequently, in step S608, the feature amount calculation unit 302 sets a detection target window at an arbitrary position of the image input in step S607. Subsequently, in step S609, the feature amount calculation unit 302 calculates partial features within the detection window set in step S608.

Subsequently, in step S610, the posture parameter input unit 303 inputs the posture parameter θ. Since the target object is being detected, the posture parameter input unit 303 uses data generated by an external program, for example, a random number generated according to a certain probability density such as the likelihood of posture, as the posture parameter. Subsequently, in step S611, the partial feature selection unit 305 selects M partial features corresponding to the posture parameter θ input in step S610 based on P (Π _i | θ) stored in the partial feature storage unit 304. To do. For example, M items are selected in descending order of probability. Subsequently, in step S612, the object detection hypothesis generation unit 306 calculates, for each of the M partial features selected in step S610, a matching degree (detection score) with the partial feature calculated in step S609 (expression ( 3)). Subsequently, in step S613, the object detection hypothesis generation unit 306 stores the sum of the M matching degrees calculated in step S612 in the storage area of the correspondence storage unit 307 corresponding to the posture parameter θ input in step S610. Add to the stored value.

  Subsequently, in step S614, it is determined whether the processes in steps S610 to S613 have been repeated a specified number of times. If not completed, the process returns to step S610. In this way, the processes in steps S610 to S613 are repeated a specified number of times. On the other hand, if the specified number of processes has been completed in step S614, the process proceeds to step S615. In step S615, the detection determination unit 308 determines the presence of the object from the data stored in the correspondence relationship storage unit 307. That is, the detection confirmation unit 308 determines the presence of an object based on whether or not there is an extreme value exceeding the threshold from the result of the addition. If detection is to be performed in an area different from the detection window set in step S608, the process returns to step S608 again, sets the detection window to a new location, and performs the processing of steps S609 to S615 (YES in S616). If the detection is not performed for a different area, the process ends (NO in S616).

  With the above processing, the target object detection processing is completed, and the detection target object and its posture can be detected from the image of FIG. As described above, in this embodiment, a set of human body part features corresponding to posture parameters is selected, and a hypothesis of the presence or absence of a human body is generated regardless of the geometrical arrangement relationship of the selected human body part features. . By generating such hypotheses for a plurality of posture parameters by iterative processing, a plurality of hypotheses are generated, and a human body in an image is detected by integrating the generated hypotheses. That is, according to the present embodiment, it is possible to detect a human body with a varying posture without learning the relative positional relationship between human body parts.

  In the above embodiment, a person is a detection target, but it is obvious that any object in the image can be a detection target.

  Although the embodiment has been described in detail above, the present invention can take an embodiment as a system, apparatus, method, program, storage medium, or the like. Specifically, the present invention may be applied to a system composed of a plurality of devices, or may be applied to an apparatus composed of a single device.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (9)

A storage unit that stores a pose parameter related to a specific object, a partial feature of the image, and a probability that the partial feature exists in the image in the pose parameter in association with each other;
An acquisition means for acquiring a partial feature by detecting a feature point from an image to be detected of the specific object, extracting a local image including the detected feature point, and calculating a feature amount;
Input means for inputting a plurality of posture parameters irrespective of the image to be detected;
Selection means for selecting a predetermined number of partial features in descending order of probability stored in the storage means for each of the plurality of posture parameters;
Collating means for calculating a matching degree indicating a degree of correlation between the partial feature acquired by the acquiring means and the partial feature selected by the selecting means;
Object detection comprising: determination means for determining whether or not the specific object is present in the detection target image based on a matching degree calculated by the matching means with respect to the plurality of posture parameters. apparatus. Learning means for generating information to be stored in the storage means;
The learning means includes
Obtaining posture parameters of the specific object from annotation data of a plurality of sample images;
In each of the plurality of sample images, a feature point is detected, and a partial feature is obtained by extracting a local image including the detected feature point and calculating a feature amount,
Cluster the acquired partial features based on similarity, calculate the probability that the partial features belonging to each cluster exist in each posture parameter,
The object detection apparatus according to claim 1, wherein a posture parameter, a probability of existence of a cluster, and a partial feature representing the cluster are associated with each other and stored in the storage unit.   The determination unit divides the posture parameter space with the number of parameters of the posture parameter as the number of dimensions into a plurality of partial spaces, and determines a plurality of matching degrees calculated by the matching unit with respect to the plurality of posture parameters. Adding to each subspace to which each of the parameters belongs, and determining that the specific object is present in the detection target image when there is an extreme value exceeding a predetermined threshold as a result of the addition. Item 3. The object detection apparatus according to Item 1 or 2.   The said determination means determines that the attitude | position parameter corresponding to the partial space in which the said extreme value exists is an attitude | position of the said specific object which exists in the said image of a detection target. Object detection device.   The object detection apparatus according to claim 1, wherein the posture parameter includes a parameter indicating a camera angle of a camera that has captured an image.   The object detection apparatus according to claim 1, wherein the specific object has a joint, and the posture parameter includes a parameter indicating a joint angle of each joint. An information processing apparatus including a storage unit that stores a posture parameter related to a specific object, a partial feature of the image, and a probability that a local image of the partial feature exists in the posture parameter in the image is identified from the image. An object detection method for detecting an object of
An obtaining step for obtaining a partial feature by extracting a local image from an image to be detected of the specific object;
An input step in which the input means inputs a plurality of posture parameters regardless of the detection target image;
A selection step of selecting a predetermined number of partial features in descending order of probability stored in the storage unit for each of the plurality of posture parameters;
A matching step in which a matching unit calculates a matching degree indicating a degree of correlation between the partial feature acquired in the acquiring step and the partial feature selected in the selection step;
And a determination step of determining whether or not the specific object is present in the detection target image based on the matching degree calculated in the matching step with respect to the plurality of posture parameters. An object detection method. The computer program for functioning a computer as each means of the object detection apparatus of any one of Claims 1 thru | or 6. A computer-readable storage medium storing the computer program according to claim 8.

Images