JP2017033175A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce false recognition of an object which is similar to an object to be tracked, while reducing processing time.SOLUTION: An image processing apparatus includes: an acquisition unit for acquiring a frame from a video; an extraction unit which extracts a plurality of partial areas; a determination unit which determines whether each of the first partial areas is a target area including a target object by means of a classifier; an estimation unit which determines an estimated position of the target object, from the target areas; a positive example addition unit which adds a second partial area located at a distance of a first threshold from the estimated position, as a positive example, to training data when the second partial area is determined to be a positive example by the classifier; a negative example addition unit which adds a third partial area located at a distance of a second threshold from a position of the second partial area added as the positive example, as a negative example, to the training data when the third partial area is determined to be a positive example by the classifier; and an update unit which updates the classifier by use of the updated training data.SELECTED DRAWING: Figure 4

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program.

  A technique for tracking an arbitrary object in a moving image has been conventionally known. In recent years, Tracking by Detection, which incorporates object detection into object tracking, has been successful. In Tracking by Detection, a two-class classifier is learned by using a tracking target object as a positive example and a background area as a negative example, and the position of the tracking target object is detected based on the classifier.

  Non-Patent Document 1 discloses a typical example of a method for tracking an arbitrary object in a moving image. Patent Document 1, Non-Patent Document 3, and Non-Patent Document 4 disclose a method of learning a tracking target object by boosting and detecting the tracking target object. Non-Patent Document 2 discloses a method of learning a tracking target object using a support vector machine (SVM) and detecting the tracking target object.

  In these methods, the background region is learned together with the tracking target object, so that confusion with the background can be reduced and robust tracking can be realized. In addition, since learning is sequentially performed during object tracking, even when the tracking target object changes with time, the change with time can be reflected in the classifier for tracking. Further, Patent Document 2 discloses a method of switching a recognition model (classifier) so as not to be influenced by a learned background when a tracking target object is lost once and retracked.

  As described above, in the conventional object tracking method based on Tracking by Detection, it is possible to follow a change with time of the tracking target object by performing relearning during the object tracking. The method of acquiring the training data of positive examples and negative examples at the time of re-learning includes (1) a method in which a sample of a region to be tracked is a positive example, and other cases as negative examples, and (2) a positive example with a classifier A method of newly adding to the training data a sample determined as a positive example and a sample determined as a negative example as a negative example is known.

  However, in the above methods (1) and (2), the training data increases by the sampled amount and the processing time increases. Specifically, in the method (1), if the position estimation accuracy of the tracking target object is low, the accuracy of the positive example and the negative example is mistaken, leading to a decrease in accuracy. In the method (2), when an object similar to the tracking target object that cannot be identified by the classifier up to that time appears, it may be mistakenly added to the training data as a positive example.

  The present invention has been made in view of the above, and provides an image processing apparatus, an image processing method, and a program capable of reducing processing time and reducing erroneous recognition of an object similar to a tracking target object. With the goal.

  In order to solve the above-described problems and achieve the object, the present invention includes an acquisition unit that acquires a frame from a moving image, and an extraction unit that extracts a plurality of first partial regions indicating a partial region of the frame. Whether each of the first partial areas is a tracking target area including a tracking target object is learned from training data using the tracking target area as a positive example and a non-tracking target area as a negative example. A determination unit that is determined by the classified classifier, an estimation unit that estimates an estimated position of the tracking target object from a plurality of the tracking target regions, and a second part that is within a first threshold from the estimated position When the region is determined to be a positive example by the classifier, a positive example adding unit that adds the second partial region to the training data as a positive example, and the second part added as a positive example From the position of the region to the second threshold value or more When the third partial region at a distance of is determined to be a positive example by the classifier, a negative example addition unit that adds the third partial region to the training data as a negative example; And an updating unit that updates the classifier using the training data updated by the example adding unit and the negative example adding unit.

  According to the present invention, it is possible to reduce the processing time and reduce erroneous recognition of an object similar to the tracking target object.

FIG. 1 is a diagram illustrating an example of a functional configuration of the learning device according to the first embodiment. FIG. 2 is a diagram illustrating an example of a positive example area according to the first embodiment. FIG. 3 is a diagram illustrating an example of a negative example area according to the first embodiment. FIG. 4 is a diagram illustrating an example of a functional configuration of the image processing apparatus according to the first embodiment. FIG. 5 is a flowchart illustrating an operation example of the learning apparatus according to the first embodiment. FIG. 6 is a flowchart illustrating an operation example of the image processing apparatus according to the first embodiment. FIG. 7 is a diagram illustrating an example of a hardware configuration of the learning device and the image processing device according to the first and second embodiments.

  Hereinafter, an image processing apparatus, an image processing method, and a program according to a first embodiment will be described in detail with reference to the accompanying drawings.

(First embodiment)
First, an example of a functional configuration of the learning device according to the first embodiment will be described. The learning device according to the first embodiment learns a classifier used in the image processing device according to the first embodiment.

  FIG. 1 is a diagram illustrating an example of a functional configuration of the learning device 100 according to the first embodiment. The learning device 100 according to the first embodiment includes a storage unit 10, a learning control unit 20, an input unit 30, and a display unit 40. The learning control unit 20 includes an acquisition unit 21, a specification unit 22, a positive example acquisition unit 23, a negative example acquisition unit 24, and a learning unit 25.

  The storage unit 10 stores a moving image to be processed. The acquisition unit 21 acquires one frame included in the moving image to be processed from the storage unit 10. The frame acquired by the acquisition unit 21 may be arbitrary. The frames acquired by the acquisition unit 21 are, for example, the first frame of the moving image to be processed, the frame specified by the user, and the like. The acquisition unit 21 inputs the frame to the designation unit 22.

  When the specification unit 22 receives a frame from the acquisition unit 21, the specification unit 22 displays the frame on the display unit 40. The input unit 30 receives an input for designating a tracking target region to be tracked in the frame displayed on the display unit 40. The method for specifying the tracking target area may be arbitrary. The method for specifying the tracking target area may be, for example, a method of specifying a rectangle including the tracking target object as the tracking target area. When receiving the tracking target region information indicating the tracking target region from the input unit 30, the specifying unit 22 inputs the tracking target region information to the positive example acquisition unit 23 and the negative example acquisition unit 24.

  When receiving the tracking target region information from the specifying unit 22, the positive example acquisition unit 23 acquires a positive example region from a region around the tracking target region indicated by the tracking target region information. A positive example area | region is an area | region which shows the positive example of training data.

  FIG. 2 is a diagram illustrating an example of positive example areas 210a to 210c according to the first embodiment. The example of FIG. 2 illustrates a case where the positive example acquisition unit 23 acquires, as positive example areas 210a to 210c, rectangles obtained by changing the tracking target area 200 in the horizontal direction and the vertical direction. Hereinafter, when the positive example areas 210a to 210c are not distinguished, they are simply referred to as positive example areas 210. The amount of variation when acquiring the positive example area 210 may be arbitrary. The amount of fluctuation in the horizontal direction is, for example, 10% of the horizontal length of the tracking target area 200. Further, the amount of fluctuation in the vertical direction is, for example, 10% of the vertical length of the tracking target area 200.

  The method for acquiring the positive example area 210 is not limited to the method shown in FIG. The positive example acquisition unit 23 may acquire a plurality of divided regions by finely dividing the tracking target region 200, for example, and may acquire the divided regions as the positive example region 210. The shape of the positive example area 210 may not be the same as the shape of the tracking target area 200.

  Returning to FIG. 1, the positive example acquisition unit 23 calculates first feature amount information indicating the characteristics of the positive example region 210. The first feature amount information may be arbitrary. The first feature amount information may be, for example, a feature vector indicating the pixel value of the positive example area. The dimension of this feature vector is 256 dimensions when the size of the positive example area is 16 × 16, for example. The first feature amount information may be a color histogram, a histogram of edge strength, a SIFT feature amount, an HOG feature amount, an output value of a neural network, or the like. Further, the first feature amount information may be a combination of these. The positive example acquisition unit 23 may normalize the size of the positive example region to a specific size and calculate the first feature amount information of the normalized positive example region. The positive example acquisition unit 23 inputs the first feature amount information to the learning unit 25.

  On the other hand, when receiving the tracking target region information from the specifying unit 22, the negative example acquisition unit 24 acquires a negative example region from a region outside the tracking target region indicated by the tracking target region information. A negative example area | region is an area | region which shows the negative example of training data.

  FIG. 3 is a diagram illustrating an example of the negative example regions 220a to 220n of the first embodiment. The example of FIG. 2 illustrates a case where the negative example acquisition unit 24 acquires the negative example regions 220a to 220n by dividing the surrounding region outside the tracking target region 200 into a grid-like region. Hereinafter, when the negative example regions 220a to 220n are not distinguished, they are simply referred to as negative example regions 220.

  The method for obtaining the negative example region 220 is not limited to the method shown in FIG. For example, the negative example acquisition unit 24 divides a surrounding region outside the tracking target region 200 into a lattice region, and then changes the position of the divided lattice region randomly or by a predetermined ratio. The example areas 220a to 220n may be acquired.

  Returning to FIG. 1, the negative example acquisition unit 24 calculates second feature amount information indicating the characteristics of the negative example region 220. The description of the second feature amount information is the same as that in the case of the positive example area 210, and will be omitted. The negative example acquisition unit 24 inputs the second feature amount information of the negative example region 220 to the learning unit 25.

  When the learning unit 25 receives the first feature amount information from the positive example acquisition unit 23 and receives the second feature amount information from the negative example acquisition unit 24, the learning unit 25 performs classification based on the first and second feature amount information. Learn the vessel. In the description of the first embodiment, a case where the learning unit 25 learns a classifier by linear discriminant analysis will be described. A case where the first and second feature quantity information is the above-described feature vector will be described.

Let N _p be the number of feature vectors in the positive example area 210. _Let N _n be the number of feature vectors in the negative example region 220. Learning section 25 first mean vector _{m p} and fluctuation matrix _{S p} of the feature vector of the positive examples region 210, and calculates the average vector _{m n} and fluctuation matrix _{S n} of the feature vector of the negative examples region 220. Then, the learning unit 25 calculates the discrimination coefficient vector a by the following equation (1).

Learning unit 25 _{stores (N p, m p, S} p, N n, m n, S n, a) in the storage unit 10 as the learning result information. Then, the learning unit 25 configures an identification function (discriminant function) that identifies whether or not the region characterized by the feature vector x is a tracking target region by the following equation (2).

  That is, the discriminant function h (x) in the equation (2) is a classifier learned by the learning unit 25. When h (x) is a positive value, it indicates that the region characterized by the feature vector x is a tracking target region. When h (x) is a negative value, it indicates that the region characterized by the feature vector x is a background region.

  Next, an example of a functional configuration of the image processing apparatus according to the first embodiment will be described.

  FIG. 4 is a diagram illustrating an example of a functional configuration of the image processing apparatus 300 according to the first embodiment. The image processing apparatus 300 according to the first embodiment includes a storage unit 310 and an image processing unit 320. The image processing unit 320 includes an acquisition unit 321, an extraction unit 322, a determination unit 323, an estimation unit 324, a positive example addition unit 325, a negative example addition unit 326, and an update unit 327.

The storage unit 310 stores a moving image to be processed. The storage unit 310 stores learning result information and a classifier. Learning result information is described above _{(N p, m p, S} p, N n, m n, S n, a). The classifier is the discriminant function h (x) indicated by the above equation (2).

  The acquisition unit 321 acquires one frame included in the processing target moving image from the storage unit 310 as a current frame. In the first iteration, the acquisition unit 321 acquires one frame for starting tracking of the tracking target object as a current frame. In the second and subsequent iterations, the acquisition unit 321 acquires one new frame (typically the next frame) in chronological order as compared to the previously selected frame as the current frame. The acquisition unit 321 inputs the current frame to the extraction unit 322.

  When receiving the current frame from the acquisition unit 321, the extraction unit 322 extracts a plurality of first partial regions from the current frame. The method for extracting the first partial region may be arbitrary. The extraction unit 322, for example, with reference to the estimated position of the tracking target object estimated in the previous frame, a plurality of rectangular areas extracted by giving a plurality of variations in the horizontal direction and the vertical direction as shown in FIG. Extracted as a plurality of first partial regions.

  At this time, the magnitude of the fluctuation when the extraction unit 322 extracts the first partial region is larger than the magnitude of the fluctuation when the learning device 100 learns the classifier. Specifically, the extraction unit 322 sets the magnitude of fluctuation when extracting the first partial region according to the assumed speed of the tracking target object.

  Note that in the first iteration, there is no information on the previous frame, so the extraction unit 322 may extract a plurality of first partial areas by dividing the entire current frame into a plurality of first partial areas.

  Hereinafter, information indicating the first partial area is referred to as first partial area information. The extraction unit 322 inputs a plurality of pieces of first partial region information to the determination unit 323.

  When the determination unit 323 receives a plurality of pieces of first partial region information from the extraction unit 322, whether or not the first partial region indicated by each first partial region information is a tracking target region including a tracking target object. Determine. Specifically, the determination unit 323 first calculates feature amount information of the first partial area indicated by each first partial area information. Next, the determination unit 323 determines whether the first partial region characterized by the feature amount information is a tracking target region using the above-described classifier (see Expression (2)) learned by the learning device 100. Determine whether or not. Next, when it is determined that the first partial region is a tracking target region, the determination unit 323 inputs tracking target region information indicating the tracking target region to the estimation unit 324.

  The estimation unit 324 receives tracking target area information from the determination unit 323. When receiving a plurality of pieces of tracking target area information from the determination unit 323, the estimation unit 324 estimates the position of the tracking target object based on the tracking target area indicated by each tracking target area information.

  The estimation unit 324 estimates the position of the tracking target object based on, for example, the average of the positions of the tracking target areas indicated by the tracking target area information. Note that a specific method by which the estimation unit 324 estimates the position of the tracking target object may be arbitrary.

The estimation unit 324 may estimate the position of the tracking target object, for example, by taking a weighted average based on the value of the discrimination function h (x) in the above equation (2). The weighted average weight w _i based on the value of the discriminant function h (x) is calculated by the following equation (3), for example.

Here, α (> 0) is a parameter. At this time, the estimation unit 324 calculates the estimated position of the tracking target object by the following expression (4), with the center position of the i-th first partial region as (x _i , y _i ).

As another simpler method, the estimation unit 324 uses the center position (x _i , y _i ) of the i-th first partial region with the largest discrimination function value h _{i as it} is as the estimated position of the tracking target object. It is good.

  The estimation unit 324 determines whether or not the estimation area including the estimated position of the tracking target object is the tracking target area, using the above-described classifier (see Expression (2)) learned by the learning device 100. . The feature quantity x used for determination by the classifier is, for example, the above-described feature vector. The shape of the estimation area may be arbitrary. For example, the estimation unit 324 may use a rectangle centered on the estimated position of the tracking target object as the estimation region.

  The estimation unit 324 may acquire a plurality of estimation regions by changing the position of a rectangle centered on the estimated position of the tracking target object. The estimation unit 324 may acquire, for example, a predetermined number of second partial regions that are within a first threshold from the estimated position as estimated regions. The distance between the estimated position and the second partial area is, for example, the coordinates indicating the estimated position and the coordinates of the center of the second partial area obtained by changing the position of the rectangle centered on the estimated position. Distance.

  When the estimation region is the tracking target region, the estimation unit 324 inputs estimation region information indicating the estimation region to the positive example addition unit 325 and the negative example addition unit 326.

  In order to reduce the processing time, the estimation unit 324 may use the processing result of the determination unit 323. Specifically, the estimation unit 324 indicates a tracking target region whose distance from the estimated position estimated by the estimation unit 324 is equal to or less than a first threshold among the tracking target regions obtained by the determination unit 323 described above. The tracking target area information may be input to the positive example adding unit 325 and the negative example adding unit 326. Note that the distance between the estimated position and the tracking target area is, for example, the distance between the coordinates indicating the estimated position and the coordinates of the center in the tracking target area.

  When the positive example addition unit 325 receives the estimation region information from the estimation unit 324, the positive example addition unit 325 adds the first feature amount information of the estimation region (second partial region) indicated by the estimation region information to the training data as a positive example. . The first feature amount information is, for example, the above-described feature vector that characterizes the estimation region.

  When the negative example addition unit 326 receives the estimation region information from the estimation unit 324, the negative example addition unit 326 determines a negative example region indicating a negative example based on the estimation region indicated by the estimation region information. The negative example adding unit 326 acquires a predetermined number of third partial regions that are at a distance equal to or larger than the second threshold from the estimated region indicated by the estimated region information received from the estimating unit 324. Then, the negative example addition unit 326 converts the second feature amount information of the third partial region determined as the positive example region by the classifier from the plurality of third partial regions into the training data as a negative example. to add.

  The reason why the partial area determined to be the positive example area is added to the training data as the negative example area is that “the tracking target object does not exist in the area at a distance equal to or larger than the second threshold from the estimated position of the tracking target object. "It is based on the precondition. That is, if a tracking target object is detected at a location sufficiently away from the estimated position of the tracking target object, it is considered that the similar object cannot be identified due to insufficient learning of the classifier. Therefore, the negative example adding unit 326 adds a sample erroneously recognized as a tracking target object to the training data as a negative example. Specifically, the negative example addition unit 326 uses the classifier to generate a positive example of the third partial region that is at a distance greater than or equal to the second threshold from the position of the second partial region added as a positive example to the training data. When it determines with it being an area | region, the said 3rd partial area | region is added to training data as a negative example area | region. Thereby, the update part 327 mentioned later can raise the identification accuracy of a tracking target object by updating (relearning) a classifier using the said training data.

  In order to reduce the processing time, the negative example adding unit 326 may use the processing result of the determining unit 323. Specifically, the negative example adding unit 326 has a tracking target region whose distance from the estimated position estimated by the estimating unit 324 is equal to or larger than a second threshold among the tracking target regions obtained by the determination unit 323 described above. The feature amount information may be added to the training data as a negative example.

The updating unit 327 updates the classifier using the training data updated by the positive example adding unit 325 and the negative example adding unit 326. The update method of the classifier by the update unit 327 will be described by taking as an example the case of using a linear discriminant analysis method (binary classification based on linear discriminant analysis) for the classifier. Specifically, the update unit 327, using positive cases and negative examples of the updated training data, the number of samples N _p of the feature vectors of the positive _sample, the mean vectors m _p, and fluctuation matrix S _p, wherein the negative sample sample number _{n n} of the vector, the mean vector _{m n} and fluctuation matrix _{S n,} and updates the determination coefficient vector a. The updating unit 327 updates the discriminant function (classifier) of the above equation (2) from (m _p , m _n, a).

Note that the update unit 327 stores only (N _p , m _p , S _p , N _n , m _n , S _n ) in the storage unit 310, even if the past training data is discarded, only new training data is stored. from can be obtained _{(n p, m p, S} p, n n, m n, S n) discrimination coefficient vector a to update the value of. The updating unit 327, in order to reduce the amount of data in the storage unit 310, the corresponding eigenvalues and specific eigenvectors from the direction eigenvalues of S _p and S _n is large, instead of storing the S _p and S _n, a storage unit 310 may be stored.

  Next, an image processing method according to the first embodiment will be described.

  FIG. 5 is a flowchart illustrating an operation example of the learning apparatus 100 according to the first embodiment. First, the acquisition unit 21 acquires one frame included in the moving image to be processed from the storage unit 10 (step S1). Next, the designation unit 22 receives tracking target area information indicating a tracking target area to be tracked in the frame (step S2). Next, the positive example acquisition unit 23 acquires the positive example area 210 from the area around the tracking target area indicated by the tracking target area information (see FIG. 2) (step S3). Next, the positive example acquisition unit 23 calculates first feature amount information indicating the characteristics of the positive example region 210 (step S4). Next, the negative example acquisition part 24 acquires the negative example area | region 220 from the area | region (refer FIG. 3) outside the tracking object area | region which tracking object area | region information shows (step S5). Next, the negative example acquisition unit 24 calculates second feature amount information indicating the characteristics of the negative example region 220 (step S6). Next, the learning unit 25 learns the classifier (see Expression (2)) based on the first and second feature amount information (step S7).

  FIG. 6 is a flowchart illustrating an operation example of the image processing apparatus 300 according to the first embodiment. First, the acquiring unit 321 acquires one frame included in the processing target moving image from the storage unit 310 as a current frame (step S21). Next, the extraction unit 322 extracts a plurality of first partial areas from the current frame (step S22). Next, the determination unit 323 detects the first partial region determined as the tracking target region by the classifier (see Expression (2)) as the tracking target region including the tracking target object (step S23). Next, when there are a plurality of tracking target areas, the estimation unit 324 estimates the position of the tracking target object based on each tracking target area (step S24).

  Next, when the positive example adding unit 325 determines that the second partial region within the first threshold from the estimated position of the tracking target object is a positive example by the classifier, the second part A region is added to the training data as a positive example (step S25). Next, the negative example addition unit 326 determines that the third partial region located at a distance greater than or equal to the second threshold from the position of the second partial region added as a positive example is a positive example. If so, the third partial region is added to the training data as a negative example (step S26).

  Next, the updating unit 327 updates the classifier using the training data updated by the positive example adding unit 325 in step S25 and the training data updated by the negative example adding unit 326 in step S26 (step S27). ).

  Next, the update unit 327 determines whether the current frame is the final frame of the moving image. If it is not the final frame (step S28, No), the process returns to step S1. If it is the last frame (step S28, Yes), the process ends.

  As described above, according to the image processing apparatus 300 of the first embodiment, by using the classifier that learns the tracking target object and the background, confusion between the tracking target and the background is reduced, and robust object tracking is performed. Can be realized. In particular, when updating the classifier in each frame, only samples that are erroneously determined as tracking target objects are added to the training data as negative examples, so that redundant samples are not added to the training data and the tracking target is not increased. False recognition with an object similar to the object can be reduced.

(Modification 1 of the first embodiment)
Next, Modification 1 of the first embodiment will be described. In the first modification, a case where a two-class classification support vector machine (SVM) is used as a classifier will be described. In the description of the first modification, the description similar to that of the first embodiment will be omitted, and different parts from the first embodiment will be described.

In learning by SVM, M D-dimensional support vectors s ⁽ⁱ⁾ (i = 1, 2,..., M) and corresponding M real weights ω _i (i = 1, 2,...). , M) and an offset c (c is a real number), and the discriminant function h (x) is obtained by the following equation (5).

If h (x) is positive with respect to the D-dimensional feature vector x indicating the feature of the region, it is predicted that the tracking target object is included in the region characterized by the D-dimensional feature vector x, and h (x) Is negative, it is predicted that the region characterized by the D-dimensional feature vector x is the background (not the tracking target object). Here, K is a kernel function R ^D × R ^D → R. ^RD represents a set of D-dimensional real vector spaces. R represents a set of one-dimensional real vector spaces.

  The kernel function is, for example, an RBF kernel represented by the following formula (6). Here, γ is a parameter. Refer to Non-Patent Document 2 for details of the SVM learning method.

  The initial learning of the classifier of the above formula (5) is performed in the same order as the learning device 100 of the first embodiment, but the specific learning process of the classifier is replaced with SVM.

  The operation of the image processing apparatus 300 according to the first modification is the same as that of the image processing apparatus 300 according to the first embodiment. Replaces the vessel.

  However, in Modification 1 using SVM, the processing time required for learning increases as the training data increases. Therefore, the positive example adding unit 325 according to the first modification deletes samples added in the past from the training data by the number of samples newly added as positive examples.

  A specific method of reduction may be arbitrary. For example, the positive example adding unit 325 may sequentially reduce the oldest samples among the samples added to the training data in the past. Further, for example, the positive example adding unit 325 may delete samples other than the support vector from the training data. Further, for example, the positive example adding unit 325 may reduce samples similar to the newly added sample from the training data.

  Similarly, the negative example addition part 326 of the modification 1 deletes the sample added in the past from training data by the number of the samples newly added as a negative example.

  The update unit 327 of Modification 1 updates (relearns) the classifier (SVM) based on the training data updated by the positive example addition unit 325 and the negative example addition unit 326, and stores the updated SVM. Store in 310.

  According to the first modification, by using SVM as a classifier, even when the tracking target object and the background cannot be linearly separated in the feature space, the tracking target object is separated from the background by the non-linear separation capability of the SVM. Tracking can be performed.

(Modification 2 of the first embodiment)
Next, a second modification of the first embodiment will be described. In the second modification, a case where AdaBoost is used as a classifier will be described. In the description of the modified example 2, the description similar to that of the first embodiment will be omitted, and portions different from the first embodiment will be described.

In AdaBoost, the strong classifier h (x) is configured by the following equation (7) by the linear sum of the weights ω _i of the M weak classifiers h _i (x) (i = 1, 2,..., M). .

  For learning of the weak classifier, the same method as in Non-Patent Document 3 can be used.

  The initial learning of the classifier of Equation (7) is performed in the same order as the learning device 100 of the first embodiment, but the specific learning process of the classifier is replaced with AdaBoost.

  The operation of the image processing apparatus 300 according to the second modification is the same as that of the image processing apparatus 300 according to the first embodiment. However, the classifier according to the expression (2) according to the first embodiment performs the classification according to the expression (7) described above. Replaces the vessel.

The update unit 327 of the second modification updates (re-learns) the classifier based on the training data updated by the positive example addition unit 325 and the negative example addition unit 326, and updates the weak classifier h _i ( x) and the updated weight ω _i are stored in the storage unit 310.

  Since AdaBoost can perform online learning as in Non-Patent Document 3, old training data can be discarded. When online learning is not performed, the storage unit 310 needs to store training data. However, the positive example adding unit 325 and the negative example adding unit 326 may reduce the training data in the storage unit 310 in order from an old sample or a similar sample, as in the first modification.

  According to the second modification, by using AdaBoost as the classifier, even if the tracking target object and the background cannot be linearly separated in the feature space, the tracking target object is separated from the background by the non-linear separation capability of AdaBoost. Tracking can be performed.

(Second Embodiment)
Next, a second embodiment will be described. In the description of the second embodiment, a case will be described in which a particle filter is used for motion prediction of a tracking target object. In the description of the second embodiment, the description similar to that of the first embodiment is omitted, and only points different from the first embodiment will be described.

In the particle filter, the state is expressed by a set of N particles having parameters of the tracking symmetry object. The image processing apparatus 300 according to the second embodiment predicts the operation of the tracking target object by updating the particle, and estimates the position of the tracking target object. Hereinafter, a case where the position and speed of a particle are used as parameters will be described. The position of the nth particle is assumed to be (x ⁽ⁿ⁾ , y ⁽ⁿ⁾ ). The speed of the nth particle is (/ x ⁽ⁿ⁾ , / y ⁽ⁿ⁾ ). However, “/” indicates a differential symbol • above the variables x and y. Further, the position and velocity of the nth particle are collectively ^expressed as z ⁽ⁿ⁾ = (x ⁽ⁿ⁾ , y ⁽ⁿ⁾ , / x ⁽ⁿ⁾ , / y ⁽ⁿ⁾ ). The weight of the nth particle is denoted as w ⁽ⁿ⁾ .

  The initial learning of the classifier used in the image processing apparatus 300 of the second embodiment is performed in the same order as the learning apparatus 100 of the first embodiment. Note that the classifiers used in the second embodiment are a linear discriminant (see the first embodiment), SVM (see the first modification of the first embodiment), and AdaBoost (see the second modification of the first embodiment). ). In addition, other algorithms that can be classified into two classes can also be used.

  The operation of the image processing apparatus 300 of the second embodiment is the same as that of the image processing apparatus 300 of the first embodiment, but the operations of the extraction unit 322 and the estimation unit 324 are the operations of the image processing apparatus 300 of the first embodiment. And different.

  The storage unit 310 stores the initial position and initial velocity of particles. The initial positions of the particles are user-specified tracking positions specified by the learning apparatus 100. The initial velocity of the particles is zero.

When receiving the current frame from the acquisition unit 321, the extraction unit 322 extracts a plurality of first partial regions from the current frame based on the particle filter. The extraction unit 322 obtains {z ⁽ⁿ⁾ _tmp } ^N _n−1 by re-sampling N particles, for example, by allowing N particles to overlap from ^N particles {z ⁽ⁿ⁾ } ^N _n−1. . Resampling is performed according to the ratio of the weight w ⁽ⁿ⁾ of each particle. That is, the extraction unit 322 selects the n-th particle with the probability P _n of the following equation (8).

  Then, the extraction unit 322 updates N particles (n = 1, 2,..., N) according to the following equations (9) to (12).

Here, Δt is a parameter, and ε ₁ ,..., Ε ₄ are random numbers. The extraction unit 322 extracts a first partial region including the coordinates indicating the position of one particle as the center.

The estimation unit 324 calculates the weight w ⁽ⁱ⁾ based on the identification function value h _i of the first partial region corresponding to each particle. Calculates the weight w ⁽ⁱ⁾ from the identification function value h _i equation can be used a function w ⁽ⁱ⁾ is increased monotonically with respect to the classification function value h _i. The following equation (13) is an example of an equation for calculating the weight w ⁽ⁱ⁾ from the discriminant function value h _i .

Here, α (> 0) is a parameter. Then, the estimation unit 324 calculates the estimated position of the tracking target object using the following expression (14), where the center position of the i-th first partial region is (x _i , y _i ).

  As described above, according to the image processing apparatus 300 of the second embodiment, it is possible to track the tracking target object by motion prediction based on the particle filter. In the description of the second embodiment, the position and the velocity are used as the particle parameters. However, the acceleration and jerk, the size and shape of the first partial region, and the like may be added as parameters.

  According to the image processing apparatus 300 of the second embodiment, by using a particle filter, it is possible to robustly track a tracking target object that performs a complicated operation with low processing cost.

  Finally, examples of hardware configurations of the learning device 100 and the image processing device 300 according to the first and second embodiments will be described.

  FIG. 7 is a diagram illustrating an example of a hardware configuration of the learning device 100 and the image processing device 300 according to the first and second embodiments. The learning device 100 and the image processing device 300 according to the first and second embodiments include a control device 401, a main storage device 402, an auxiliary storage device 403, a display device 404, an input device 405, and a communication device 406. The control device 401, main storage device 402, auxiliary storage device 403, display device 404, input device 405, and communication device 406 are connected via a bus 410.

  The control device 401 executes the program read from the auxiliary storage device 403 to the main storage device 402. The main storage device 402 is a memory such as a ROM and a RAM. The auxiliary storage device 403 is a memory card, an SSD (Solid State Drive), or the like.

  The display device 404 displays information. The display device 404 is, for example, a liquid crystal display. The input device 405 receives information input. The input device 405 is, for example, a keyboard and a mouse. The communication device 406 communicates with other devices.

  The programs executed by the learning apparatus 100 and the image processing apparatus 300 according to the first and second embodiments are files in an installable format or an executable format, and are CD-ROM, memory card, CD-R, and DVD (Digital Versatile). The program is stored in a computer-readable storage medium such as Disk) and provided as a computer program product.

  The program executed by the learning apparatus 100 and the image processing apparatus 300 according to the first and second embodiments is stored on a computer connected to a network such as the Internet, and is provided by being downloaded via the network. May be. The programs executed by the learning device 100 and the image processing device 300 according to the first and second embodiments may be provided via a network such as the Internet without being downloaded.

  The program executed by the learning apparatus 100 and the image processing apparatus 300 according to the first and second embodiments may be provided by being incorporated in advance in a ROM or the like.

  A program executed by the learning device 100 and the image processing device 300 according to the first and second embodiments is realized by a program among the functional configurations of the learning device 100 and the image processing device 300 according to the first and second embodiments described above. It has a module configuration that includes possible functions.

  Functions realized by the program are loaded into the main storage device 402 when the control device 401 reads the program from a storage medium such as the auxiliary storage device 403 and executes the program. That is, the function realized by the program is generated on the main storage device 402.

  Note that some or all of the functions of the learning device 100 and the image processing device 300 according to the first and second embodiments may be realized by hardware such as an IC (Integrated Circuit).

DESCRIPTION OF SYMBOLS 10 Memory | storage part 20 Learning control part 21 Acquisition part 22 Specification part 23 Positive example acquisition part 24 Negative example acquisition part 25 Learning part 30 Input part 40 Display part 100 Learning apparatus 300 Image processing apparatus 310 Storage part 320 Image processing part 321 Acquisition part 322 Extraction unit 323 determination unit 324 estimation unit 325 positive example addition unit 326 negative example addition unit 327 update unit

Japanese Patent No. 5052670 Japanese Patent No. 5719230

Y. Wu, J .; Lim, M.M. H. Yang, “Object Tracking Benchmark,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Pre-Prints, 2015. S. Avidan, “Support Vector Tracking,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 8, pp. 1064-1072, 2004. H. Grabner, et al. “On-line Boosting and Vision,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2006. B. Babenko, et al. “Visual Tracking with Online Multiple Instance Learning,” Proceedings of the IEEE Conferencing on Computer Vision and Pattern Recognition (CVPR), 2009.

Claims (10)

An acquisition unit for acquiring frames from a video;
An extraction unit for extracting a plurality of first partial areas indicating partial areas of the frame;
Whether each of the first partial areas is a tracking target area including a tracking target object is learned from training data using the tracking target area as a positive example and a non-tracking target area as a negative example. A determination unit for determining by the classifier,
An estimation unit that estimates an estimated position of the tracking target object from a plurality of tracking target regions;
When the second partial area located within the first threshold from the estimated position is determined to be a positive example by the classifier, the second partial area is added to the training data as a positive example. Positive example addition part,
When the third partial area located at a distance equal to or larger than the second threshold from the position of the second partial area added as a positive example is determined as a positive example by the classifier, the third part A negative example addition unit for adding a region as a negative example to the training data;
An updating unit for updating the classifier using the training data updated by the positive example adding unit and the negative example adding unit;
An image processing apparatus comprising: The classifier is a discriminant function that performs binary classification based on linear discriminant analysis.
The image processing apparatus according to claim 1. The classifier is an identification function that performs binary classification based on a support vector machine.
The image processing apparatus according to claim 1. The classifier is an identification function that performs binary classification based on a combination of weak classifiers.
The image processing apparatus according to claim 1. The estimation unit estimates the tracking target region having the maximum value of the identification function as an estimated position of the tracking target object;
The image processing apparatus according to claim 2. The estimation unit estimates an estimated position of the tracking target object based on an average position of the tracking target area;
The image processing apparatus according to claim 2. The estimation unit estimates an estimated position of the tracking target object by a weighted average based on an identification function value of the tracking target region;
The image processing apparatus according to claim 2. The extraction unit extracts the first partial region based on a particle filter;
The image processing apparatus according to claim 1. An image processing device acquiring a frame from a moving image;
An image processing apparatus extracting a plurality of first partial areas indicating a partial area of the frame;
The image processing apparatus determines whether each of the first partial areas is a tracking target area including a tracking target object, using the tracking target area as a positive example and a non-tracking target area as a negative example. Determining by a classifier learned from training data;
An image processing device estimating an estimated position of the tracking target object from a plurality of tracking target regions;
When the image processing apparatus determines that the second partial region at a distance within the first threshold from the estimated position is a positive example by the classifier, the second partial region is used as a positive example. Adding to the training data;
When it is determined by the classifier that the third partial region at a distance equal to or greater than a second threshold from the position of the second partial region added as a positive example is a positive example, Adding the third partial region to the training data as a negative example;
The image processing apparatus adds the second partial region to the training data as a positive example, and adds the third partial region to the training data as a negative example. Updating the classifier with data;
An image processing method including: Computer
An acquisition unit for acquiring frames from a video;
An extraction unit for extracting a plurality of first partial areas indicating partial areas of the frame;
Whether each of the first partial areas is a tracking target area including a tracking target object is learned from training data using the tracking target area as a positive example and a non-tracking target area as a negative example. A determination unit for determining by the classifier,
An estimation unit that estimates an estimated position of the tracking target object from a plurality of tracking target regions;
When the second partial area located within the first threshold from the estimated position is determined to be a positive example by the classifier, the second partial area is added to the training data as a positive example. Positive example addition part,
When the third partial area located at a distance equal to or larger than the second threshold from the position of the second partial area added as a positive example is determined as a positive example by the classifier, the third part A negative example addition unit for adding a region as a negative example to the training data;
An updating unit for updating the classifier using the training data updated by the positive example adding unit and the negative example adding unit;
Program to function as.

Images