JP2015230559A - Object identification device, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the identification performance by considering a deformation portion as one of features of DPM, in an approach to identification performance degradation of an object identification model base on the DPM due to the difference between an original dataset and a target dataset.SOLUTION: An appearance feature quantity is extracted using an intermediate dictionary that becomes gradually close from an original dictionary to a target dictionary. The appearance feature quantity is changed to a quantity appropriate for a target dataset, and a coefficient for a vector obtained by interconnecting the appearance feature quantity and the deformation feature quantity is learned. Therefore, an identifier for putting the deformation portion into a state appropriate for the target dataset can be produced.

Description

  The present invention relates to an object identification device, method, and program, and more particularly, to a technique for improving object identification accuracy.

  A Deformable Part Model (hereinafter referred to as DPM) is known as a de facto standard for an object identification model (see, for example, FIG. 11).

  The appearance feature quantity used in DPM: Histogram of Oriented Gradient (hereinafter referred to as HOG) is now the de facto standard for appearance feature quantities. However, in recent years, techniques for extracting appearance feature quantities based on dictionaries (= bases) obtained by dictionary learning represented by Histogram of Sparse Code (hereinafter referred to as HSC) have emerged (for example, (See FIG. 12).

  Further, after obtaining a dictionary of learning sample data sets (hereinafter referred to as original data sets) and a dictionary of evaluation sample data sets (hereinafter referred to as target data sets) (hereinafter referred to as target dictionaries), Image characteristics of the original data set and the target data set are extracted by extracting the intermediate representation of the dictionary and the target dictionary (hereinafter referred to as the intermediate dictionary) and learning the feature values expressed using all of the original dictionary, the target dictionary, and the intermediate dictionary. There is already known a technique for suppressing deterioration of the identification performance due to the difference (object orientation, shooting viewpoint, resolution, etc.).

  FIG. 13 shows an example of the original dictionary, the target dictionary, and the intermediate dictionary. If the left end in the figure is the original dictionary and the right end is the target dictionary, the images arranged between them are the intermediate dictionary. In FIG. 13, “person” is an identification target, and “object orientation” is a difference in image characteristics between the original dictionary and the target dictionary.

  However, the object identification model based on DPM so far is based on the characteristics of DPM in the approach for the degradation of the identification performance due to the difference in image characteristics (object orientation, shooting viewpoint, resolution, etc.) between the original data set and the target data set. The Deformable Part (hereinafter referred to as a deformed part), which is one of the above, is not taken into consideration, and there is a problem that the identification performance improvement rate is limited.

  As an example, consider the identification target as a person, the difference in image characteristics between the original data set and the target data set as the person's orientation, and the deformed part as the left arm (see FIG. 14). In this example, it can be seen that the position of the left arm from the center of the image has changed between the sample of the original data set and the sample of the target data set. When DPM is optimized with the left arm position of the original data set sample as "probable as the left arm position", the likelihood of the left arm position is low in the target data set sample, and as a result it may not be identified as a person is there.

  The DPM receives a sample of the target data set as an input, and outputs a sample identification target likelihood (personality in this example) as a score. For example, the output score value increases for a sample identified as human, and the output score value decreases for a sample identified as not human. The final identification result is determined as “detected” if the output score of the DPM is equal to or greater than an arbitrary threshold, and “undetected” if it is less than the threshold. Therefore, it is required to output a high score when a human image sample is input. If a high output score is not output when a human image sample is input, the result of “detection” can be obtained by lowering the above-mentioned threshold, but an image sample other than a human image is input. In particular, since the probability of obtaining a result of “detection” (that is, a misidentification result) increases, it is required to output a high score only when a sample of a human image is input in order to improve the identification performance.

  The DPM is an object identification model that improves the identification performance by allowing the above-described position change (= deformation) of the part with a penalty. The penalty means that the output score value is deducted, that is, the output score value becomes smaller. However, if the DPM is optimized based on the position of the original data set without considering the position of the target data set, the penalty will be large if the position of the position is extremely different between the original data set and the target data set. Therefore, there is a case where the requirement for improving the identification performance of “outputting a high score when a human image sample is input” may not be satisfied.

  Patent Document 1 discloses an object identification model that evaluates image features for each part for the purpose of improving object identification accuracy. It is similar to the present invention in that the image feature is certainly evaluated for each part. However, the problem that the deformation | transformation site | part mentioned above cannot be considered has not been solved.

  Non-Patent Document 2 extracts an intermediate dictionary between an original dictionary and a target dictionary after obtaining an original dictionary and a target dictionary for the purpose of suppressing degradation of identification performance due to a difference in characteristics between the original data set and the target data set. A technique is disclosed. The present invention is certainly similar in that it generates an original dictionary, a target dictionary, and an intermediate dictionary. However, when an object identification model based on DPM is considered, deformation sites cannot be considered.

  Therefore, in the approach to the degradation of the identification performance due to the difference in the characteristics of the original data set and the target data set in the object identification model based on DPM, the deformation performance which is one of the features of DPM is considered and the identification performance is improved. This is the issue.

  That is, an object of the present invention is to improve identification performance so that an object can be identified even when there is a deformed part in the identification target.

  The present invention that achieves the above-described object provides, as a first aspect, a computer, an original dictionary generation unit that generates an original dictionary based on a first image in which a certain identification object is captured, and the identification object is captured, A target dictionary generating unit that generates a target dictionary based on a second image that has an image characteristic different from that of the first image, and an intermediate that generates a plurality of intermediate dictionaries based on the original dictionary and the target dictionary As a classifier learning unit for inputting a feature quantity of the first image and a label added to the first image to a classifier and causing the classifier to learn using the original dictionary The classifier learning unit inputs an image group in which the first image and the second image are mixed, and a label attached to each of the images to the classifier, and sets each of the intermediate dictionaries. Using the discriminator to function to learn, The image identification and the second image include a part of the identification target and a deformed part that is different in appearance in the first image and the second image. Provide a program.

  According to the present invention, it is possible to improve the identification performance so that an object can be identified even when there is a deformed part in the identification target.

It is a flowchart figure which shows the learning flow of embodiment. It is a conceptual diagram of the process of L1 of FIG. It is a conceptual diagram of the process of L2 of FIG. It is a conceptual diagram of the process of L3 of FIG. FIG. 3 is a conceptual diagram (part 1) of the process of L4 in FIG. 1; FIG. 3 is a conceptual diagram (part 2) of the process of L4 in FIG. It is a conceptual diagram of the learning process of embodiment. It is a flowchart figure which shows the identification flow of embodiment. It is a figure which shows the hardware structural example of an Example. It is a block diagram which shows the function structure of an Example. It is a figure used when demonstrating a deformation | transformation site | part model (DPM). It is a figure used when explaining an appearance feature-value (HSC). It is a figure for demonstrating the concept of an original dictionary, a target dictionary, and an intermediate dictionary. It is a figure for demonstrating that object identification may fail because there exists a deformation | transformation site | part.

  The invention disclosed below considers a deformed part as an approach for suppressing degradation of identification performance due to a difference in characteristics between an original data set and a target data set in an object identification model based on DPM. It is possible to suppress degradation of the identification performance due to the difference in the characteristics of the data set.

  In the following, an algorithm of an object identification program that specifically realizes the object identification model as described above will be described (FIGS. 1 to 8). Next, the object identification program is executed by a general-purpose computer, and the computer is An embodiment in the case of functioning as an identification device will be described (FIGS. 9 and 10).

  The object identification model of the present embodiment learns an object that is known in advance and then identifies an object that is unknown. The learning process will be described with reference to FIGS. 1 to 7, and the identification process will be described with reference to FIG.

<Learning flow>
Considering the case where all of the original data set and a part of the target data set are labeled (person = 1, otherwise = 0), the L1 to L4 processes shown in FIG. 1 and FIGS. This will be described below.

・ L1: Generation of original dictionary and target dictionary (Fig. 2)
In this step, the original dictionary is calculated from the original data set, and the target dictionary is calculated from the target data set. The original data set, the original dictionary, the target data set, and the target dictionary are expressed as follows (where n ^{s is the} number of samples in the original data set, n ^{t is the} number of samples in the target data set, m is the number of dimensions of each sample) ).

  There are various methods for the calculation algorithm of the original dictionary and the target dictionary. For example, the method described in Non-Patent Document 1 can be cited. Based on this method, the original dictionary / target dictionary can be calculated as in the following equation. Note that Γ in the following formula is a sparse code obtained in each dictionary.

The dimension number m of each sample is determined by the image size. When learning the original dictionary and the target dictionary, it is necessary to unify the number of dimensions for all samples, but the image size of each sample may be different. In this case, for example, the resample process is performed on all samples by the following method to determine the number of dimensions.
The sample is scaled to a predetermined fixed size.
A part of the sample is cut out with a predetermined fixed-size frame (see Non-Patent Document 4 for determining the position of the frame).

L2: Generation of intermediate dictionary (Fig. 3)
In this step, an intermediate dictionary: D (k) (where k = 1 to K−1) is generated. The algorithm for calculating the displacement: ΔD (k) between the intermediate dictionaries is performed by the method described in Non-Patent Document 2.

L3: Learning the initial model using the original dictionary (Fig. 4)
In this step, the appearance feature quantity and label for each sample of the original data set are input using the original dictionary, the following objective function is learned for the coefficient: β (0), and the coefficient: β (0) is obtained as a learning result. . The appearance feature value and the label can be expressed as follows.

  There are various methods for calculating the appearance feature value using the original dictionary. For example, the method described in Non-Patent Document 3 can be cited. The learning method of the coefficient: β (0) is also described in Non-Patent Document 4.

L4: Sequential learning of models using intermediate dictionary and target dictionary (FIGS. 5 and 6)
First, an appearance feature value and a label are calculated using D (k) for each sample obtained by mixing the original data set and the target data set. (Fig. 5)

  The calculated appearance feature value and label are input, the following objective function is learned with respect to the coefficient: β (k), and the coefficient: β (k) is obtained as a learning result. Let β (k−1) be the initial value of β (k). This process is repeated from k = 1 to K. (Fig. 6)

In the process of L4, the appearance feature value is extracted using an intermediate dictionary that gradually approaches the target dictionary from the original dictionary, thereby changing the appearance feature value to be suitable for the target data set, and the appearance feature value. By learning the coefficient for the vector in which the deformed feature quantity is linked, it becomes possible to generate a discriminator that makes the deformed part suitable for the target data set.
FIG. 7 shows an image of the learning process of L1 to L4.

  In the above description, the same dictionary is used when calculating the feature values of the overall image (Root) and the deformable part (Deformable Part). However, each dictionary is prepared at the time point L1 and each time when the feature value is calculated. It is possible to use a dictionary of For example, when calculating the deformed part dictionary, the original data set, the target data set, and the intermediate dictionary are calculated after the samples of the original data set and the target data set are resampled at a certain image size. In addition, when calculating the dictionary for the whole image, the original data set and the target data set sample are resampled at a resolution half the sampling image size of the deformed part, and then the original dictionary, target dictionary, intermediate Calculate the dictionary.

  In the above description, it is assumed that the image sizes of all samples used for learning of the original dictionary and the target dictionary are the same. For example, the original data set is converted into a plurality of sample groups (hereinafter referred to as the aspect ratio of the image). It is also possible to divide the target data set into a plurality of sample groups (hereinafter referred to as target sub data set) according to the same criteria and calculate a dictionary for each sub data set (aspect ratio). (See Non-Patent Document 4 for the method of dividing the sample group in accordance with the above).

  For example, when the original data set / target data set is divided into two original sub data sets and two target sub data sets according to the aspect ratio, in the processing of L1, the dictionary of the first original sub data set ( Thereafter, the sub-source dictionary 1), the second original sub-data set dictionary (hereinafter referred to as the sub-original dictionary 2), the first target sub-data set dictionary (hereinafter referred to as the sub-target dictionary 1), the second A dictionary of the target sub-data set (hereinafter referred to as sub-target dictionary 2) is calculated. In the process of L2, an intermediate dictionary between the sub-source dictionary 1 and the sub-target dictionary 1 (intermediate dictionary 1) and an intermediate dictionary between the sub-source dictionary 2 and the sub-target dictionary 2 (intermediate dictionary 2) are calculated. Model learning after L3 is multi-component model learning (see Non-Patent Document 4). Components using sub-source dictionary 1, intermediate dictionary 1, sub-target dictionary 1, sub-source dictionary 2, intermediate dictionary 2, A component using the sub-target dictionary 2 is learned.

<Identification flow>
A process for identifying an object using the classifier generated by the learning flow described above will be described with reference to FIG.
D1: Feature value extraction using target dictionary Appearance feature values using the target dictionary generated in L1 are extracted for the evaluation target image. The appearance feature amount extraction method is the same as the L4 processing.
D2: Input Feature Value to Discriminator Based on the learned coefficient β (K), the discriminator input with the feature amount calculates an identification score. See Non-Patent Document 4 for the method of calculating the identification score.
D3: Threshold determination of identification score The calculated identification score is subjected to threshold determination and identified as “detected” if it is equal to or greater than the threshold, and “undetected” if it is less than the threshold.

  According to the present embodiment described above, the appearance feature value is changed to a value suitable for the target data set, and the coefficient for the vector in which the appearance feature value and the deformed feature value are connected is learned. A discriminator is generated that puts the region in a state suitable for the target data set. Since an object is identified using such a classifier, the identification accuracy does not deteriorate even when a deformed part exists.

<Example>
FIG. 9 shows a hardware configuration example of the embodiment. The object identification device 1 according to the embodiment has the same configuration as a general information processing terminal. That is, in the object identification device 1 according to the embodiment, a CPU (Central Processing Unit) 10, a RAM (Random Access Memory) 11, a ROM (Read Only Memory) 12, an HDD (Hard Disk Drive) 13 and an I / F 14 are buses. Connected through. Further, an LCD (Liquid Crystal Display) 15 and an operation unit 16 are connected to the I / F 14.

  The CPU 10 is a calculation means and controls the operation of the entire apparatus. The RAM 11 is a volatile storage medium capable of reading and writing information at high speed, and is used as a work area when the CPU 10 processes information. The ROM 12 is a read-only nonvolatile storage medium, and stores programs such as firmware. The HDD 13 is a non-volatile storage medium that can read and write information, and stores an OS (Operating System), various control programs, application programs, and the like. The I / F 14 connects and controls a bus and various hardware and networks. The LCD 15 is a visual user interface for the user to check the state of the apparatus. The operation unit 16 is a user interface such as a keyboard and a mouse for a user to input information to the apparatus.

  By the information processing by the software program using the hardware resources as described above, the functional configuration described below is realized in the control unit formed by the CPU 10 or the like. Hereinafter, functional blocks will be described.

  FIG. 10 shows a functional configuration of the object identification device 1. As illustrated, the object identification device 1 includes various storage units in addition to the classifier learning function unit 110 and the identification function unit 120.

The discriminator learning function unit 110 is a functional block that bears the discriminator learning function shown in FIG.
The classifier learning function unit 110 includes a dictionary learning image input unit 111, an original dictionary generation unit 112, a target dictionary generation unit 113, an intermediate dictionary generation unit 114, a learning image / label input unit 115, a feature vector generation unit 116, and classifier learning. A generation unit 117 is provided.

The dictionary learning image input unit 111 is an image input block necessary for the processing of L1 and L2.
The original dictionary generation unit 112 is an original dictionary generation block of L1.
The target dictionary generation unit 113 is an L1 target dictionary generation block.
The intermediate dictionary generation unit 114 is an L2 intermediate dictionary generation block.
The learning image / label input unit 115 is a data set sample input unit for each of L3 and L4.
The feature vector generation unit 116 is a feature amount extraction block using the L3 and L4 dictionaries.
The discriminator learning / generation unit 117 is a coefficient learning block of L3 and L4.
The dictionary storage unit 118 is a dictionary storage unit generated by L1 and L2.
The classifier storage unit 119 is a classifier storage unit generated in L3 and L4.

The identification function unit 120 is a functional block responsible for the identification processing function shown in FIG.
The identification function unit 120 includes an evaluation image input unit 121, a feature vector generation unit 123, an identification score calculation unit 124, and a threshold determination unit 127.

The evaluation image input unit 121 is an identification target image input block.
The feature vector generation unit 123 is a D1 processing block.
The identification score calculation unit 124 is a D2 processing block.
The threshold value determination unit 127 is a D3 processing block.
The identification result storage unit 130 is a block that stores the identification result obtained by the D3 process.

  With the apparatus having the above-described configuration, the processing of the above embodiment can be performed, and an object can be identified even when there is a deformed part in the identification target.

DESCRIPTION OF SYMBOLS 1 Object identification device 110 Classifier learning function part 111 Dictionary learning image input part 112 Original dictionary generation part 113 Target dictionary generation part 114 Intermediate dictionary generation part 115 Learning image and label input part 116 Feature vector generation part 117 Classifier learning and generation part 118 Dictionary Storage Unit 119 Classifier Storage Unit 120 Identification Function Unit 121 Evaluation Image Input Unit 123 Feature Vector Generation Unit 124 Discrimination Score Calculation Unit 127 Threshold Determination Unit 130 Identification Result Storage Unit

JP 2005-004612 A

M. Aharon, M. Elad, and A. Bruckstein.K-SVD: An algorithm for designing of overcomplete dictionaries for sparse representation.IEEE Transactions on Signal Processing, 54 (11): 4311-4322, 2006. 2, 3, 5 , 6 Ni, Jie, Qiang Qiu, and Rama Chellappa. "Subspace Interpolation via Dictionary Learning for Unsupervised Domain Adaptation." 2013 IEEE Conference on Computer Vision and Pattern Recognition Ren, Xiaofeng, and Deva Ramanan. "Histograms of Sparse Codes for Object Detection.", 2013 IEEE Conference on Computer Vision and Pattern Recognition Felzenszwalb, Pedro F., et al. "Object detection with discriminatively trained part-based models." Pattern Analysis and Machine Intelligence, IEEE Transactions on 32.9 (2010): 1627-1645.

Claims (5)

Computer
An original dictionary generation unit that generates an original dictionary based on a first image in which a certain identification object is captured;
A target dictionary generating unit that generates a target dictionary based on a second image that has an image characteristic different from that of the first image in which the identification target is captured;
An intermediate dictionary generating unit that generates a plurality of intermediate dictionaries based on the original dictionary and the target dictionary;
The feature amount of the first image and the label added to the first image are input to a discriminator, and function as a discriminator learning unit that causes the discriminator to learn using the original dictionary,
The classifier learning unit inputs an image group in which the first image and the second image are mixed, and a label attached to each group to the classifier, and uses each of the intermediate dictionaries to Let the classifier function to learn,
The first image and the second image include a deformed portion that is a part of the identification target and has a different appearance between the first image and the second image. , Object identification program. An original dictionary generation unit that generates an original dictionary based on a first image in which a certain identification object is captured;
A target dictionary generating unit that generates a target dictionary based on a second image that has an image characteristic different from that of the first image in which the identification target is captured;
An intermediate dictionary generating unit that generates a plurality of intermediate dictionaries based on the original dictionary and the target dictionary;
A discriminator learning unit that inputs the feature amount of the first image and a label added to the first image to a discriminator, and causes the discriminator to learn using the original dictionary;
The classifier learning unit inputs an image group in which the first image and the second image are mixed, and a label attached to each group to the classifier, and uses each of the intermediate dictionaries to Let the classifier learn,
The first image and the second image include a deformed portion that is a part of the identification target and has a different appearance between the first image and the second image. , Object identification device. The original dictionary generation unit generates an original dictionary for a deformed part based on a third image showing the deformed part,
The target dictionary generation unit generates a target dictionary for a deformed part based on a fourth image showing the deformed part,
The object identification device according to claim 2, wherein the intermediate dictionary generation unit generates a plurality of intermediate dictionaries for deformed parts based on the original dictionary and the target dictionary for the deformed parts.   4. The object identification device according to claim 2, wherein the dictionary group is generated for each aspect ratio of the image. An original dictionary generation step of generating an original dictionary based on a first image in which a certain identification object is shown;
A target dictionary generating step for generating a target dictionary based on a second image that is an image having a different image characteristic from the first image in which the identification target is captured;
An intermediate dictionary generating step for generating a plurality of intermediate dictionaries based on the original dictionary and the target dictionary;
A first discriminator learning step of inputting a feature amount of the first image and a label added to the first image to a discriminator, and causing the discriminator to learn using the original dictionary;
An image group in which the first image and the second image are mixed and a label attached to each image are input to the discriminator, and the discriminator learns using each of the intermediate dictionaries. Two discriminator learning steps;
The first image and the second image include a deformed portion that is a part of the identification target and has a different appearance between the first image and the second image. , Object identification method.

Images