JPWO2015008567A1 - Face impression degree estimation method, apparatus, and program - Google Patents

Abstract

In order to accurately estimate the impression level of both ends of a person, the impression level estimation method for estimating the impression level of a person captured in image data includes an extraction step of extracting a face image from the image data, and a facial feature from the face image The calculation process for calculating the vector and the magnitude relationship between the impressions between the two samples were modeled by a class of least-squares stochastic discriminators, and the test data and the learning data were already obtained from the facial image vector as an explanatory variable. An estimation step for estimating the impression level of a person by scoring based on the certainty of each class obtained by sequentially comparing a plurality of comparison base data with a plurality of classes of ranking least squares stochastic discriminators; ,including.

Description

  The present invention relates to a method, an apparatus, and a program for estimating an attribute of a person, and more particularly to a method, an apparatus, and a program for estimating an impression level of a person from a face image of the person.

Background information

In recent years, research for estimating the attributes (gender, age, facial expression, etc.) of a person from the face image of the person has been remarkably developed. In particular, gender and age estimation is applied to marketing strategies, security, amusement, and other commercial products.
For example, Patent Document 1 discloses an “age estimation apparatus, method, and program” that can obtain a recognition result close to the result perceived by humans. In the age estimation device disclosed in Patent Document 1, when an age estimation model is created by regression analysis, the younger generation's estimation accuracy is improved by strengthening the younger group's learning weight. Specifically, in Patent Document 1, a kernel regularized weighted least squares (KRWLS) is used in a supervised regression problem that predicts the true age of test data from which feature vectors are extracted. The age estimation function is modeled by a linear combination of positive definite kernels.
A classifier having a high learning efficiency called a least square probabilistic classifier (LSPC) is also known (see, for example, Non-Patent Document 1 and Non-Patent Document 2). LSPC is an identification method for learning a class posterior probability model under a square loss, and its greatest feature is that a solution can be calculated analytically. In addition, since LSPC directly estimates the posterior probability for each class in the form of a density ratio, it also has a feature that it is resistant to an imbalance in the number of learning data of each class. In LSPC, the posterior probability is learned using the square loss. As a result, the LSPC can shorten the learning time several hundred times while maintaining pattern recognition accuracy comparable to that of the conventional method. In addition, LSPC is not easily affected by a deviation in the number of data of a specific class.
In addition, ranking learning is also known. Here, “ranking learning” is a technique of optimization based on supervised learning so that a high score can be given to data according to the degree of relevance and permutation. For example, as a representative example of ranking learning based on the pair-wise approach, ranking SVM (ranking Support Vector Machine) is known (see, for example, Non-Patent Document 3). In ranking SVM, the loss to the pair of learning data is considered, thereby reducing the two-class identification problem by SVM and optimizing the score function.

Japanese Patent No. 4742192

Sugiyama, M, “Superfast-trainable multi-class probable classifier by last-square posteriorfitting,” IEICE Transactions Information Institute. E93-D, no. 10, pp. 2690-2701, 2010 Yasuyuki Ihara, Masaru Sugiyama, Kazuya Ueki, Mitsuhiro Fujita, “Multi-Racial Age Identification by Weighted Integration of Multiple Classifiers”, Dynamic Image Processing Realization Workshop 2011 (DIA2011) Proceedings, pp. 317-322, Tokushima, 2011.3.3-4 R. Herbrich, T .; Graepel and K.M. Obermayer: “Large margin rank boundaries for ordinal regression”, Advances in Large Margin Classifiers, Cambridge, MA, MIT Press, pp. 115-132 (2000)

Recently, efforts are being made on new attribute areas such as “cute” and “bright and refreshing” to estimate the “appearance of a person's appearance” that is deeply related to the taste of makeup. The impression of a person includes, for example, five types of impressions of “bright and refreshing”, “adorable”, “business”, “friendly”, and “healthy”. As the evaluation (impression degree), each impression is evaluated (estimated) in a plurality of stages (for example, five stages).
However, in the “regression analysis” used in Patent Document 1 or the like, it is difficult to accurately estimate the impression degree of a face image corresponding to the accuracy of both ends (very cute, not pretty at all). This problem is a universal problem peculiar to regression analysis in which it is difficult to ensure the estimation accuracy of the values of the dependent variable (objective variable) for data near both ends, and facial images corresponding to impressions at both ends are sufficient. This is thought to be caused by data collection problems such as difficulty in collecting data.
Impressions at both ends (very cute, not pretty at all) are more memorable to human memory than average impressions (average cuteness). What can be estimated is important.
Even in the LSPC disclosed in Non-Patent Document 1 and Non-Patent Document 2, there is a problem that the estimation accuracy of impression degree at both ends of a person is poor.
In addition, the conventional optimization method using ranking learning has a problem that it is difficult to reflect the degree of difference between two dependent variables (object variables) in the optimization problem.
As described above, with the conventional method, it is difficult to accurately estimate the “degree of impression at both ends” of a person.
[Object of invention]
Accordingly, an object of the present invention is to provide a method, an apparatus, and a program for accurately estimating the degree of impression of both ends of a person.

  One aspect of the present invention is an impression level estimation method for estimating an impression level of a person captured in image data using an impression level estimation device, an extraction step of extracting a face image from image data, and a face image The calculation process of calculating the facial feature vector from the model, and the magnitude relationship between the impressions between the two samples are modeled by a class of least-squares stochastic discriminators, and test data and learning data with the facial image vector as an explanatory variable are already used Estimating the impression of a person by scoring based on the certainty of each class obtained by sequentially comparing the obtained plurality of comparison base data with a plurality of ranking least square probabilistic classifiers. An estimation step.

  The effect of the present invention is that the degree of impression of both ends of a person can be accurately estimated.

FIG. 1 is an image diagram of scoring used in the impression degree estimation method according to the embodiment of the present invention.
FIG. 2 is a block diagram showing a configuration of an impression degree estimation apparatus according to the first embodiment of the present invention.
FIG. 3 is a block diagram showing a configuration of a neural network (continuous amount estimation means) in regression analysis mode used in the impression degree estimation apparatus of FIG.
FIG. 4 is a block diagram showing a configuration of a ranking least square probabilistic discriminator (order estimation means) used in the impression degree estimation apparatus of FIG.
FIG. 5 is a block diagram showing the configuration of an impression degree estimation apparatus according to the second embodiment of the present invention.
FIG. 6 is a block diagram showing a configuration of a ranking least square probabilistic discriminator (order estimation means) used in the impression degree estimation apparatus of FIG.
FIG. 7 is a block diagram showing a configuration of an impression degree estimation apparatus according to the third embodiment of the present invention.
FIG. 8 is a table showing the experimental results of the recognition rate of each class in each method when “pretty impression” is solved as an identification problem.
FIG. 9 is a table showing the experimental results of the absolute error average (MAE) in each method when “pretty impression” is solved as a regression problem.

ただし、ｐ（ｘ）はサンプルの確率密度、ｐ（ｘ，ｙ）は同時確率密度である。最終的には、事後確率が最大になるクラスを、次の式（２）で表される推定クラスとする。

ＬＳＰＣでは、事後確率ｐ（ｙ｜ｘ）を二乗損失を用いて学習を行う。これにより、ＬＳＰＣでは、回帰分析の従来手法（カーネル正則化最小二乗法：ＫＲＬＳ）と同程度のパターン認識精度を維持しながら、学習時間を数百倍短縮することが可能となる。
また、ＬＳＰＣでは、事後確率を上記数１で表される密度比の形で推定するため、訓練データにおける、特定のクラスのデータ数の偏りの影響を受けにくい。
まず、ＬＳＰＣでは、クラスｙの事後確率ｐ（ｙ｜ｘ）を、次の式（３）で表される線形モデルでモデル化する。

る。
ＬＳＰＣでは、パラメータα＝（α_１，…，α_ｂ）^Ｔを、下記の式（４）で表される二乗誤差Ｊ_０が最小となるように学習する。

なお、Ｔは転置行列である。
二乗誤差Ｊ_０の期待値の近似値を標本平均で考えることにより、上記式（４）の解は、解析的に次の式（５）で与えられる。

ここで、

である。
ＬＳＰＣでは、最後に、全クラスの事後確率の総和が１になるように正規化補正することにより、下記の式（６）で示されるように、事後確率の解を得る。

しかしながら、上述したように、ＬＳＰＣでは、人物の両端の印象度の推定精度が悪いという問題がある。
［関連技術２］
次に、第２の関連技術として、ランキング学習の定式化について説明する。
上述したように、ランキング学習とは、関連度の高さや順列に応じて、データに高いスコアを付与出来る様、教師付き学習の枠組みで最適化する手法である。近年、ランキング学習は、クエリと関連性の高いデータを優先的に抽出する情報検索や、ユーザに関心の高い商品情報を優先的に提示する推薦システムなどの領域で、盛んに応用されている。
データ空間をχとするとき、ｎ個の教師付き学習データ

ここで、ベクトルｘ_ｉは説明変数（顔画像の特徴量）であり、スカラｙ_ｉは目的変数（印象度の大小をあらわしたもの）である。例えば、‘かわいらしい’の印象度を表す場合、ｙ_ｉの値域を０．０〜４．０（全くかわいくない：０．０、とてもかわいい：４．０）で定め、‘かわいらしい’の度合いが大きいほど、数値が大きくなる様に取る。
従来のランキング学習では、印象度が高い順に出力値がソートされ、つまり、下記の式（７）

となる様な、スコア関数ｆ：χ→Ｒを最適化する。
この場合、Ｎｏｒｍａｌｉｚｅｄ Ｄｉｓｃｏｕｎｔｅｄ Ｃｕｍｕｌａｔｉｖｅ Ｇａｉｎ（ＮＤＣＧ）や、Ｍｅａｎ Ｒｅｃｉｐｒｏｃａｌ Ｒａｎｋ（ＭＲＲ）などが、モデルの評価基準として用いられるが、最適化問題を容易にするため、便宜的に学習データのペアやリストに対する損失を考えることが多い。
次に、第２の関連技術として、上記非特許文献３に記載されている、ペアワイズ・アプローチに基づくランキング学習の代表的な例として、ｒａｎｋｉｎｇ ＳＶＭについて述べる。
ｒａｎｋｉｎｇ ＳＶＭでは、学習データのペアに対する損失を考えることで、ＳＶＭ（Ｓｕｐｐｏｒｔ Ｖｅｃｔｏｒ Ｍａｃｈｉｎｅ）による２クラス識別問題に帰着させ、スコア関数ｆの最適化を行う。
ｒａｎｋｉｎｇ ＳＶＭでは、スコア関数ｆ：χ→Ｒを、下記の式（８）で表されるように、正定値カーネルｋ（ｘ’，ｘ）の線形結合でモデル化する。

集合を用いても良い。そして、下記パラメータα＝（α_１，…，α_ｎ）^Ｔを、次の関係が成り立つ様に最適化する。

ここで、Φ（ｘ）＝（ｋ（ｘ_１，ｘ），…ｋ（ｘ_ｎ，ｘ））^Ｔである。このとき、ｒａｎｋｉｎｇ ＳＶＭの主問題は次の式（１０）の形で定式化される。

は分離可能で無い場合を考慮して導入したスラック変数である。
ランキング学習の従来の定式化では、上述したように、下記の式（１１）

となる様な、スコア関数ｆ：χ→Ｒを最適化することを考えた。しかしながら、この最適化手法には、２つの従属変数（目的変数）ｙ_ｉ，ｙ_ｊの差の大小の度合いを最適化問題に反映させることが難しいという問題点がある。
［実施の形態］
次に、本発明の一実施形態に係る印象度推定方法について説明する。
本実施形態に係る印象度推定方法は、「二標本間の印象度の大小関係」を多クラスのＬＳＰＣによりモデル化（ｒａｎｋｉｎｇ ＬＳＰＣの導入）し、ＬＳＰＣの出力するクラスの確信度を元に、出力のスコアリングを行うことによって、人物の両端の印象度を精度良く推定する。換言すれば、本実施の形態に係る印象度推定方法におけるランキング学習（ペアワイズ・アプローチ）では、識別問題に帰着させて解く際に、多クラス確率的識別器であるＬＳＰＣを使用する。
すなわち、本実施形態に係る印象度推定方法は、比較的正解の得られやすい‘二標本間の印象度の大小関係’に着目して、二標本間の印象度の大小関係をランキング学習によりモデル化し、このモデルを用いた推定結果（ベアワイズで比較した時の印象度の大小）の集計により、両端の印象度に相当する顔画像に対する推定精度を改善する手法である。
上記第２の関連技術でおいて述べた従来のランキング学習手法と、本実施の形態に係るランキング学習手法との間には、次に述べるような相違点がある。
まず、従来のランキング学習手法では、ペアワイズで解くアプローチ（例えば、２データ間の比較で考えるアプローチ）を考える際、２クラスの識別問題に帰着させて解いている。したがって、従来のランキング学習手法は、どちらが「上又は下」の識別のみである。また、従来のランキング学習手法ではＳＶＭを使用して解くことが多い。
これに対して、本実施の形態に係るランキング学習手法では、ペアワイズで解くアプローチ（例えば、２データ間の比較で考えるアプローチ）を考える際、多クラスの識別問題に帰着させて解く。したがって、本実施の形態に係るランキング学習手法は、どちらが「上又は下」の判断だけでなく、差の大小も加味する。そして、本実施の形態に係るランキング学習手法ではＬＳＰＣを採用する。
本発明の実施形態では、次の形でランキング学習を再定式化する。
まず、サンプルのペア（ｘ_ｉ，ｙ_ｉ）、（ｘ_ｊ、ｙ_ｊ）が与えられた時に、従属変数（目的変数）の差（ｙ_ｉ−ｙ_ｊ）を、大小の度合いに応じて、Ｃクラスにクラス分けする。具体的には、２クラス（Ｃ＝２）の場合には、下記の式（１２）

といったクラス分けを考え、４クラス（Ｃ＝４）の場合には、下記の式（１３）

といったクラス分けを考える。
Ｌ（ｙ_ｉ，ｙ_ｊ）は差（ｙ_ｉ−ｙ_ｊ）の属するクラスラベルを表す。この時、クラス毎に、スコア関数ｆｋ：χ×χ→Ｒ（ｋ＝１，…，Ｃ）を定義し、それぞれ、下記の式（１４）となる様にそれぞれ最適化する。

このスコア関数は、上記第２の関連技術における従来のスコア関数の特別な場合（線形性を持つ場合）の一般化と考えてよい。例えば、上記第２の関連技術において述べたｒａｎｋｉｎｇ ＳＶＭにおいて、スコア関数ｆの正定値カーネルの線形カーネルを用い、従属変数（目的変数）の差を上記式（１２）のように２クラスに分類する場合を考える。この時、２組のスコア関数をそれぞれ、下記の式（１５）

と定義すれば、ランキング学習は、上記式（１４）で表される最適化問題に帰着することに注意されたい。
スコア関数の最適化問題を考える際は、最適化問題を容易にするため、学習データのペアに対する損失を考えるペアワイズ・アプローチを便宜的に採用する。具体的には、本実施形態では、以下に説明するｒａｎｋｉｎｇ ＬＳＰＣを考えることで、多クラス識別問題に帰着させて、スコア関数の最適化を行う。
次に、ｒａｎｋｉｎｇ ＬＳＰＣについて説明する。ｒａｎｋｉｎｇ ＬＳＰＣは、上記第２の関連技術において述べたｒａｎｋｉｎｇ ＳＶＭと同様に、ペアワイズ・アプローチに基づくものであり、学習データのペアに対する損失を考えることで、上記第１の関連技術において述べたＬＳＰＣによる多クラス識別問題に帰着させ、下記の式（１６）で表されるスコア関数ｆ_ｋの最適化を行う。

このように、本実施の形態において、識別問題に関して、従来手法でよく使われるＳＶＭでなく、ＬＳＰＣを用いた理由は、次の４点である。
（１）ＬＳＰＣは、二乗損失のもとでクラスの事後確率モデルを学習する識別手法であり、従来手法（第２の関連技術）と同程度のパターン認識精度を維持しながら、学習時間を数百倍短縮することが出来る。一方、ペアワイズ・アプローチに基づくランキング学習は、学習データ数が飛躍的に増大する。元お学習データがｎ個の場合、ランキング学習で使用する学習データはｎ^２倍となる。ＬＳＰＣのこの学習効率の良さは、ランキング学習における大規模サンプル数の問題を緩和することが出来る。
（２）ＬＳＰＣでは、クラス毎に事後確率をモデル化するため、各クラスの学習データのアンバランスに強いという特徴も併せ持つ。従属変数（目的変数）の差分の大小によりクラス分類する問題を考える場合、各クラスの学習データ数にアンバランスが生じやすい。しかしながら、ＬＳＰＣのこの特徴は、データ数のアンバランスの認識精度に与える悪影響を緩和するのに好都合である。
（３）ＬＳＰＣは、多クラスの非線形識別問題を解くことが出来、クラス毎に事後確率をモデル化するため、上述したように、クラス毎のスコア関数のモデル化が可能である。特に、ＬＳＰＣでは、後述する多クラスのランキング学習を考えた場合、クラスによっては‘正解率の高い出力’が得られやすい。これはランキング学習の出力を集計して、回帰問題を解く際に有益である。
（４）ランキング学習の出力は、ペアワイズ的なアプローチの場合、通常、クラス識別結果の出力（予測クラス）のみであり、出力の確信度の情報は含まれない。これに対して、ＬＳＰＣの場合、クラス識別結果を各クラスの確信度で出力する為、後述するように、ランキング学習の出力を集計して、回帰問題を解く際に好都合である。
次に、ｒａｎｋｉｎｇ ＬＳＰＣの定式化について説明する。換言すれば、ｒａｎｋｉｎｇ ＬＳＰＣの「学習フェーズ」について説明する。

（目的変数））、従属変数（目的変数）の差（ｙ_ｉ−ｙ_ｊ）を、大小の度合いに応じて、Ｃクラスにクラス分けする。ここで、前述したように、ベクトルｘ_ｉは顔画像の特徴量を表わす説明変数であり、ｙ_ｉは印象度の大小を数値化したものを表す目的変数（従属変数）である。ここで、ｙ_ｉの値は、主観的に（例えば多数決などにより）印象度として既に決められた値（既知の値）である。
具体的には、２クラスであれば上記式（１２）、４クラスであれば上記式（１３）の形を考える。
この時、ｒａｎｋｉｎｇ ＬＳＰＣでは、各クラスのスコア関数

正定値カーネルφ_ｉ，ｊの線形結合で、下記の式（１７）で表されるように、それぞれモデル化する。

る最小二乗誤差Ｊ_０が最小になる様に学習する。

ＰＣと同様である。
以上の説明が、ｒａｎｋｉｎｇ ＬＳＰＣの「学習フェーズ」の説明である。
次に、本発明の実施の形態に係る印象度推定方法の核心部分である、ｒａｎｋｉｎｇ ＬＳＰＣの推定結果を元に、回帰分析の問題を解く手法について説明する。換言すれば、ｒａｎｋｉｎｇ ＬＳＰＣを用いた「認識（推定）フェーズ」について説明する。
まず、テストデータ（ｘ^ｔｅ，ｙ^ｔｅ）との従属変数（目的変数）の差を比較するため

する。ここで、テストデータの特徴変数（ベクトル）ｘ^ｔｅは、顔画像から特徴ベクトル算出部（後述する）によって算出された顔特徴ベクトルである。一方、テストデータの目的変数ｙ^ｔｅは、未知の変数である。比較用基底データ数ｂの大きさを適宜変えることで、モデル学習時の計算負荷を調整出来る。
また、印象度ｙは、下限値０．０と上限値４．０をそれぞれ持つ、すなわち、０．０≦ｙ≦４．０であるとする。
簡単のため、まず２クラスのｒａｎｋｉｎｇ ＬＳＰＣの場合について説明する。従属変数（目的変数）の差のクラス区分は、上記数２２に従うものとする。
テストデータ（ｘ^ｔｅ，ｙ^ｔｅ）と比較用基底データ（ｂ_１、ｙ_１）のペアを入力した際のｒａｎｋｉｎｇ ＬＳＰＣの出力が、‘クラス１，クラス２の確信度はそれぞれ０．７，０．３’であったとする（ｙ^ｔｅ＜ｙ_１である確率が０．７）。
この時、下記の式（１９）で表される、ベクトル形式の印象度スコア

を印象度ｙの値に応じて次の式（２０）のように割り当てる（ｙの値の取り方は任意で決めて良いが、ｙの全値域に対して密になる様取得するのが望ましい）。

図１は、このスコアリングのイメージ図である。図１において、横軸は印象度スコアを示し、縦軸は確信度を示す。印象度スコアは、０．０〜４．０の間の範

で表されるベクトルｇ_１に正規化処理しておく。

次に、他の比較用基底データ（ｂ_２、ｙ_２）、（ｂ_３、ｙ_３）に対しても、同様にス

_３に正規化しておく。
この時、次の式（２２）

の最大成分に該当するｙ^＊、つまり、下記の式（２３）

となる様なｙ^＊を、テストデータ（ｘ^ｔｅ，ｙ^ｔｅ）を入力した場合の印象度の推定結果とする。
以上の説明が、２クラスのｒａｎｋｉｎｇ ＬＳＰＣを用いた「認識（推定）フェーズ」の説明である。
次に、４クラスのｒａｎｋｉｎｇ ＬＳＰＣの場合について説明する。従属変数（目的変数）の差のクラス区分は、上記式（１３）に従うものとする。
テストデータ（ｘ^ｔｅ，ｙ^ｔｅ）と比較用基底データ（ｂ_１、ｙ_１）のペアを入力した際のｒａｎｋｉｎｇ ＬＳＰＣの出力が、‘クラス１〜クラス４の確信度がそれぞれ０．６５，０．２，０．１，０．０５’であったとする（ｙ^ｔｅ＜ｙ_１−０．５である確率が最も高い（０．６５））。
この時、下記の式（２４）で表されるベクトル形式の印象度スコア

を、印象度ｙの値に応じて次の式（２５）のように割り当てる。

以後は、上述した２クラスのｒａｎｋｉｎｇ ＬＳＰＣの場合と同様に、比較用基底データ毎にスコア化を行い、テストデータ（ｘ^ｔｅ，ｙ^ｔｅ）に対する印象度の推定値を求める。
差分の大きなクラス（クラス１，４）に属するデータは認識率が高く、逆に、差分の小さなクラス（クラス２，３）に属するデータは認識率が低いことに注意されたい。そこで、ε−Ｓｕｐｐｏｒｔ Ｖｅｃｔｏｒ Ｒｅｇｒｅｓｓｉｏｎと同様、許容誤差を設定し（ここでは０．５）、これをクラス２，３の値域設定に用いる。
以上の説明が、２クラスのｒａｎｋｉｎｇ ＬＳＰＣを用いた「認識（推定）フェーズ」の説明である。In order to facilitate understanding of the present invention, related techniques will be described.
[Related technology 1]
First, LSPC described in Non-Patent Documents 1 and 2 will be described as a first related technique.
LSPC is an identification method for learning a class posterior probability model under a square loss, and its greatest feature is that a solution can be calculated analytically.
In addition, since LSPC directly estimates the posterior probability for each class in the form of a density ratio, it also has a feature of being strong against imbalance in the number of learning data of each class. This feature of LSPC is advantageous when solving ranking LSPC according to embodiments of the invention described below.
In LSPC, the posterior probability distribution p (y | x) of class y in the input vector (face feature amount) x is estimated in the form of the density ratio of the following equation (1).

However, p (x) is the probability density of the sample, and p (x, y) is the joint probability density. Finally, the class with the maximum posterior probability is set as the estimated class represented by the following equation (2).

In LSPC, the posterior probability p (y | x) is learned using the square loss. Thereby, in LSPC, it is possible to shorten the learning time by several hundred times while maintaining pattern recognition accuracy comparable to that of the conventional method of regression analysis (kernel regularized least square method: KRLS).
In LSPC, since the posterior probability is estimated in the form of the density ratio represented by the above equation 1, it is difficult to be affected by the deviation in the number of data of a specific class in the training data.
First, in LSPC, the posterior probability p (y | x) of class y is modeled by a linear model expressed by the following equation (3).

The
In LSPC, the parameter α = (α ₁ ,..., Α _b ) ^T is learned so that the square error J ₀ expressed by the following equation (4) is minimized.

T is a transposed matrix.
By considering an approximation of the expected value of the square error J ₀ at sample mean, the solution of the equation (4) is given by analytically following equation (5).

here,

It is.
In LSPC, finally, a normalization correction is performed so that the sum of the posterior probabilities of all classes becomes 1, thereby obtaining a solution of the posterior probabilities as shown in the following equation (6).

However, as described above, the LSPC has a problem that the estimation accuracy of the impression degree at both ends of a person is poor.
[Related Technology 2]
Next, formulating ranking learning will be described as a second related technique.
As described above, ranking learning is a technique of optimization in a supervised learning framework so that a high score can be given to data according to the degree of relevance and permutation. In recent years, ranking learning has been actively applied in areas such as information retrieval that preferentially extracts data highly relevant to a query and a recommendation system that preferentially presents product information that is highly interested in users.
When the data space is χ, n supervised learning data

Here, the vector x _i is an explanatory variable (feature amount of the face image), and the scalar y _i is an objective variable (representing the magnitude of the impression degree). For example, when expressing the degree of impression of “pretty”, the range of y _i is set to 0.0 to 4.0 (not pretty: 0.0, very cute: 4.0), and the degree of “pretty” is large As the number increases, take it.
In conventional ranking learning, output values are sorted in descending order of impression, that is, the following equation (7)

Optimize the score function f: χ → R such that
In this case, Normalized Discounted Cumulative Gain (NDCG), Mean Reproductive Rank (MRR), and the like are used as model evaluation criteria. However, in order to facilitate optimization problems, for the sake of convenience, loss of learning data pairs and lists is lost. I often think of.
Next, as a second related technique, ranking SVM will be described as a representative example of ranking learning based on the pair-wise approach described in Non-Patent Document 3 above.
In the ranking SVM, by considering the loss of the pair of learning data, the score function f is optimized by reducing to a two-class identification problem by SVM (Support Vector Machine).
In ranking SVM, the score function f: χ → R is modeled by a linear combination of positive definite kernels k (x ′, x) as represented by the following equation (8).

A set may be used. Then, the following parameter α = (α ₁ ,..., Α _n ) ^T is optimized so that the following relationship is established.

Here, Φ (x) = (k (x ₁ , x),... K (x _n , x)) ^T. At this time, the main problem of ranking SVM is formulated in the form of the following equation (10).

Is a slack variable introduced in consideration of the case where separation is not possible.
In the conventional formulation of ranking learning, as described above, the following equation (11)

It was considered to optimize the score function f: χ → R such that However, this optimization method has a problem that it is difficult to reflect the degree of difference between two dependent variables (object variables) y _i and y _{j in} the optimization problem.
[Embodiment]
Next, an impression level estimation method according to an embodiment of the present invention will be described.
The impression degree estimation method according to the present embodiment models the “size relationship between two specimens” by multi-class LSPC (introduction of ranking LSPC), and based on the certainty of the class output by LSPC, By performing output scoring, the degree of impression at both ends of the person is accurately estimated. In other words, in ranking learning (pairwise approach) in the impression degree estimation method according to the present embodiment, LSPC, which is a multi-class probabilistic classifier, is used when solving the classification problem.
That is, the impression degree estimation method according to the present embodiment is based on the model of the magnitude relation between the two samples by focusing on “the magnitude relation between the two specimens”, which is relatively easy to obtain a correct answer. This is a technique for improving the estimation accuracy for the face image corresponding to the impression degree at both ends by summing up the estimation results using this model (the magnitude of the impression degree when compared with bearwise).
There is the following difference between the conventional ranking learning method described in the second related technology and the ranking learning method according to the present embodiment.
First, in the conventional ranking learning method, when considering a pair-wise approach (for example, an approach considered by comparison between two data), the problem is solved by reducing to a two-class identification problem. Therefore, the conventional ranking learning method only identifies which is “upper or lower”. Also, conventional ranking learning methods often use SVM to solve.
On the other hand, in the ranking learning method according to the present embodiment, when considering a pair-wise approach (for example, an approach considered by comparison between two data), it is solved by reducing to a multi-class identification problem. Therefore, the ranking learning method according to the present embodiment takes into consideration not only the determination of which is “up or down” but also the magnitude of the difference. The ranking learning method according to the present embodiment employs LSPC.
In the embodiment of the present invention, the ranking learning is reformulated in the following form.
First, when a pair of samples (x _i , y _i ) and (x _j , y _j ) is given, the difference (y _i −y _j ) of the dependent variable (object variable) is set according to the degree of magnitude, Classify into C classes. Specifically, in the case of 2 classes (C = 2), the following formula (12)

In the case of 4 classes (C = 4), the following formula (13)

Consider the classification.
L (y _i , y _j ) represents the class label to which the difference (y _i −y _j ) belongs. At this time, a score function fk: χ × χ → R (k = 1,..., C) is defined for each class, and optimized so as to satisfy the following equation (14).

This score function may be considered as a generalization of the special case (with linearity) of the conventional score function in the second related technique. For example, in the ranking SVM described in the second related technology, the linear kernel of the positive definite kernel of the score function f is used, and the difference of the dependent variable (target variable) is classified into two classes as in the above equation (12). Think about the case. At this time, each of the two sets of score functions is expressed by the following equation (15).

Note that ranking learning results in an optimization problem expressed by the above equation (14).
When considering the optimization problem of the score function, in order to facilitate the optimization problem, a pair-wise approach that considers the loss to the pair of learning data is conveniently adopted. Specifically, in the present embodiment, a ranking LSPC described below is considered, and the score function is optimized by reducing to a multi-class identification problem.
Next, ranking LSPC will be described. The ranking LSPC is based on the pair-wise approach, similar to the ranking SVM described in the second related technology, and is based on the LSPC described in the first related technology by considering the loss to the pair of learning data. The score function f _k expressed by the following equation (16) is optimized by reducing to the multi-class identification problem.

Thus, in this embodiment, regarding the identification problem, the reason for using LSPC instead of SVM often used in the conventional method is the following four points.
(1) LSPC is an identification method for learning a posterior probability model of a class under a square loss, and the number of learning times is reduced while maintaining the same pattern recognition accuracy as the conventional method (second related technology). Can be shortened by a factor of 100. On the other hand, in the ranking learning based on the pair-wise approach, the number of learning data increases dramatically. When the original learning data is n, the learning data used in the ranking learning is n ² times. This good learning efficiency of LSPC can alleviate the problem of large-scale sample number in ranking learning.
(2) In LSPC, since the posterior probability is modeled for each class, the LSPC also has a feature of being strong against unbalance of learning data of each class. When considering the problem of classifying according to the difference of the dependent variable (objective variable), imbalance tends to occur in the number of learning data of each class. However, this feature of LSPC is advantageous to mitigate the negative impact on the accuracy of recognition of the number of data imbalances.
(3) Since the LSPC can solve a multi-class nonlinear identification problem and models the posterior probability for each class, the score function for each class can be modeled as described above. In particular, in the LSPC, when considering multi-class ranking learning, which will be described later, an “output with a high accuracy rate” is easily obtained depending on the class. This is useful when calculating the output of ranking learning and solving regression problems.
(4) In the case of the pair-wise approach, the output of ranking learning is normally only the output of the class identification result (predicted class), and does not include information on the reliability of the output. On the other hand, in the case of LSPC, since the class identification result is output with the certainty factor of each class, as will be described later, it is convenient when calculating the ranking learning outputs and solving the regression problem.
Next, the formulation of ranking LSPC will be described. In other words, the “learning phase” of ranking LSPC will be described.

(Object variable)) and the difference (y _i -y _j ) of the dependent variable (object variable) are classified into C classes according to the degree of magnitude. Here, as described above, the vector x _i is an explanatory variable representing the feature quantity of the face image, and y _i is an objective variable (dependent variable) representing the numerical value of the degree of impression. Here, the value of y _i is a value (known value) that has already been determined as an impression level subjectively (for example, by majority vote).
Specifically, consider the form of the above formula (12) for 2 classes and the above formula (13) for 4 classes.
At this time, in ranking LSPC, the score function of each class

Each is modeled by a linear combination of positive definite kernels φ _{i, j} as represented by the following equation (17).

Minimum square error J ₀ to learn so as to become a minimum that.

It is the same as PC.
The above description is the description of the “learning phase” of ranking LSPC.
Next, a technique for solving the regression analysis problem based on the estimation result of ranking LSPC, which is the core part of the impression degree estimation method according to the embodiment of the present invention, will be described. In other words, the “recognition (estimation) phase” using ranking LSPC will be described.
First, to compare the difference of the dependent variable (objective variable) with the test data (x ^te , y ^te )

To do. Here, the feature variable (vector) ^xte of the test data is a face feature vector calculated by a feature vector calculation unit (described later) from the face image. On the other hand, the objective variable y ^te of the test data is an unknown variable. The calculation load at the time of model learning can be adjusted by appropriately changing the size of the comparison basis data number b.
The impression level y has a lower limit value 0.0 and an upper limit value 4.0, that is, 0.0 ≦ y ≦ 4.0.
For the sake of simplicity, the case of a 2-class ranking LSPC will be described first. The class classification of the difference of the dependent variable (objective variable) shall be according to the above formula 22.
The output of ranking LSPC when a pair of test data (x ^te , y ^te ) and comparison base data (b ₁ , y ₁ ) is input, and the certainty of class 1 and class 2 is 0.7 and 0, respectively. 3 ′ (the probability that y ^te <y ₁ is 0.7).
At this time, the impression score of the vector format represented by the following formula (19)

Is assigned as shown in the following equation (20) according to the value of the impression degree y (how to obtain the value of y may be arbitrarily determined, but it is desirable to obtain the value so as to be dense with respect to the entire range of y) ).

FIG. 1 is an image diagram of this scoring. In FIG. 1, the horizontal axis represents the impression score, and the vertical axis represents the certainty factor. Impression score is a range between 0.0 and 4.0.

Normalization processing is performed on the vector g ₁ represented by

Next, for the other comparison basis data (b ₂ , y ₂ ), (b ₃ , y ₃ )

Normalize to ₃ .
At this time, the following equation (22)

Y ^* corresponding to the maximum component of, that is, the following formula (23)

Y ^* such that becomes the impression result when the test data (x ^te , y ^te ) is input.
The above description is the description of the “recognition (estimation) phase” using two classes of ranking LSPC.
Next, the case of 4-class ranking LSPC will be described. The class classification of the difference of the dependent variable (object variable) shall be according to the above equation (13).
The output of ranking LSPC when a pair of test data (x ^te , y ^te ) and comparison base data (b ₁ , y ₁ ) is input, the certainty of class 1 to class 4 is 0.65, 0, respectively. , 2, 0.1, 0.05 ′ (the probability that y ^te <y ₁ −0.5 is highest (0.65)).
At this time, the impression score of the vector format represented by the following formula (24)

Is assigned according to the value of impression degree y as in the following equation (25).

Thereafter, as in the case of the above-described two classes of ranking LSPC, scoring is performed for each comparison base data, and an estimated value of the impression degree for the test data (x ^te , y ^te ) is obtained.
It should be noted that data belonging to a class with a large difference (classes 1 and 4) has a high recognition rate, and conversely, data belonging to a class with a small difference (classes 2 and 3) has a low recognition rate. Therefore, as in the case of ε-Support Vector Regression, an allowable error is set (here, 0.5), and this is used for class 2 and 3 range setting.
The above description is the description of the “recognition (estimation) phase” using two classes of ranking LSPC.

Next, the impression level estimation apparatus 10 according to the first embodiment of the present invention will be described with reference to FIG.
The impression level estimation device 10 shown in the figure is a device that evaluates (estimates) the following five types of impressions of a person in five levels (5 classes) of “0, 1, 2, 3, 4”.
1. 1. Bright and refreshing 2. Cute Business 4. 4. Easy. Healthy And, for example, when expressing the impression level of “cute”, the range of the objective variable (dependent variable) y described later is 0.0 to 4.0 (“not cute at all” is set to 0.0, “very cute” ”Is defined in 4.0), and the numerical value increases as the degree of“ pretty ”increases.
In the first embodiment, the above five types are given as the impression of the person. However, the present invention is not limited to these, and at least one impression may be selected from these five types of impressions. Of course, other impressions may be used.
The impression level estimation device 10 classifies the impression level (objective variable) y into the following five classes (evaluated in five stages) according to the magnitude.
(Class 1) 0.0 ≦ y <0.5
(Class 2) 0.5 ≦ y <1.5
(Class 3) 1.5 ≦ y <2.5
(Class 4) 2.5 ≦ y <3.5
(Class 5) 3.5 ≦ y ≦ 4.0
In the first embodiment, the impression level is classified into five classes (evaluated in five stages), but the present invention is not limited to this, and is classified into two or more classes (evaluated in two or more stages). Of course, this may be done.
The impression level estimation device 10 includes a head detection unit 12, a face detection unit 14, a neural network 16 in a regression analysis mode, and a ranking least square probabilistic classifier (ranking LSPC) 18.
The head detection unit 12 estimates the positions of eyes, nose, and mouth from a color still image file obtained by photographing a person using a head detection program. The face detection unit 14 extracts a face image normalized to 64 × 64 pick cells based on position information of face parts such as eyes. Therefore, the combination of the head detection unit 12 and the face detection unit 14 serves as a data acquisition unit (face image extraction unit) 32 that extracts a face image from a person image.
As is well known in the art, the neural network 16 comprises an input layer 162, an intermediate layer 164, and an output layer 166, as shown in FIG.
The data (intermediate layer data) of the intermediate layer 164 of the neural network 16 is supplied to a ranking least square probabilistic classifier (ranking LSPC) 18 as a face feature vector. Therefore, the combination of the input layer 162 and the intermediate layer 164 of the neural network 16 serves as a feature vector calculation unit 34 that calculates a face feature vector from the extracted face image.
In the first embodiment, the feature vector calculation unit 34 in the neural network 16 is used as means for calculating the face feature vector. However, the present invention is not limited to this, and other well-known various types. Of course, the face feature vector calculating means may be used.
The ranking least square stochastic discriminator 18 shown comprises two classes of ranking least square stochastic discriminators. Therefore, the ranking least-squares stochastic discriminator 18 outputs y ^* expressed by the above equation (23) as an impression degree estimation result when test data (x ^te , y ^te ) is input.
FIG. 4 is a block diagram showing the configuration of the two-class ranking least squares probabilistic classifier 18. The two-class ranking least-squares probabilistic classifier 18 includes a magnitude comparison / determination unit 182 using a two-class ranking LSPC, a scoring processing unit 184, and a score sum processing unit 186.
One test data (x ^te , y ^te ) and a plurality of comparison base data (x ₁ , y ₁ ) are supplied to the magnitude comparison determination unit 182 using two classes of ranking LSPC.
The test data (x ^te , y ^te ) is intermediate layer data (face feature vector) when the test data in the face image format is input to the neural network 16 in the regression analysis mode. Hereinafter, this is used as an explanatory variable for test of test data and expressed as ^xte . Further, at this stage, an unknown test objective variable (for example, a correct value of the impression degree related to “adorable”) is represented by y ^te .
The plurality of comparison basis data (x ₁ , y ₁ ) is an intermediate layer when the comparison basis data in the face image format (a plurality obtained from the learning data) is input to the neural network 16 in the regression analysis mode. Data (face feature vector). These are already obtained at the model learning stage of the two classes of ranking LSPC, and are not newly calculated at the stage of estimation processing for test data. Hereinafter, these are used as comparison explanatory variables of the comparison base data, and are referred to as x ₁ , x ₂ ,..., X _k ,. In addition, comparison objective variables corresponding to x ₁ , x ₂ ,..., X _k ,... (For example, correct values of impression degree regarding “pretty”) are respectively represented by y ₁ , y ₂ ,. .., y _k ,...
Magnitude comparison determination unit 182 using the ranking LSPC of 2 classes, using a ranking LSPC two classes, the comparative base data _{(x k} ^{(a x te} with the test explanatory variable) test data and the comparative explanatory variables It is determined whether the impression level is “have” (the same processing is repeated for all comparison base data x _k ).
In this example, since the size determination by rank LSPC in two classes is performed, the size comparison determination unit 182 using the rank LSPC of two classes has a difference (x ^te −x) between the test data and the comparison base data. _k )), the size of the objective variable between the two data is estimated (y ^te <y _k or y ^te ≧ y _k is estimated), and is classified into two as shown in the above equation (12).
Then, the magnitude comparison / determination unit 182 using two classes of ranking LSPC outputs the magnitude determination result in the form of confidence of each class. For ^example, in the case of comparison between ^{x te} and _{x k,} magnitude comparison determination unit 182 using the ranking LSPC of 2 classes, for example, Class 1, confidence of class 2 are respectively 0.7,0.3 It outputs as there is (refer FIG. 1).
The scoring processing unit 184 performs scoring processing according to the above equations (19) to (21) using the magnitude determination comparison results output for each comparison base data.
The score total processing unit 186 adds all the scores calculated for each comparison base data, and outputs the portion where the calculated score is the maximum as the estimation result of the impression degree as shown in the above formula 44. The impression degree estimation result is “estimation result of objective variable (for example, impression degree estimation result regarding“ adorable ”) by two classes of ranking LSPC when test data x ^te is input”.
When the impression degree estimation apparatus 10 according to the first embodiment is configured by an electronic device, the impression degree estimation apparatus 10 can be realized by a computer that operates by program control. Although not shown, this type of computer, as is well known, includes an input device for inputting data, a data processing device, an output device for outputting processing results in the data processing device, and an auxiliary memory serving as various databases. Device. Then, the data processing device stores the program in a read-only memory (ROM), a random access memory (RAM) used as a work memory for temporarily storing data, and a program stored in the ROM. It consists of a central processing unit (CPU) that processes stored data.
In this case, the input device operates as a device (not shown) for inputting a person image (image data). The data processing apparatus operates as a data acquisition unit 32, a neural network (continuous quantity estimation unit) 16 in regression analysis mode, and a ranking least square probabilistic classifier (order estimation unit) 18. The auxiliary storage device functions as storage means (not shown) for storing a plurality of comparison base data.
In other words, each unit of the impression level estimation device 10 according to the first embodiment may be realized using a combination of hardware and software. In a form in which hardware and software are combined, each unit is realized as various means by operating hardware such as a control unit (CPU) based on an impression degree estimation program stored in the ROM. The impression degree estimation program may be recorded on a recording medium and distributed. The impression degree estimation program recorded on the recording medium is read into the memory via the wired, wireless, or recording medium itself, and operates the control unit and the like. Examples of the recording medium include an optical disk, a magnetic disk, a semiconductor memory device, and a hard disk.
The impression level estimation device 10 having such a configuration can accurately estimate the impression levels of both ends of a person.

Next, with reference to FIG. 5, an impression level estimation apparatus 10A according to a second embodiment of the present invention will be described.
The illustrated impression level estimation apparatus 10A has the same configuration as the impression level estimation apparatus 10 shown in FIG. 2 and operates except that the configuration of the ranking least squares stochastic discriminator is different. Therefore, the reference sign of 18A is attached to the ranking least squares stochastic discriminator. Constituent elements similar to those shown in FIG. 2 are denoted by the same reference numerals, and description thereof is omitted for the sake of simplicity.
The ranking least square stochastic discriminator 18A shown in the figure is composed of four classes of ranking least square stochastic discriminators.
FIG. 6 is a block diagram showing a configuration of a 4-class ranking least squares probabilistic classifier 18A. The 4-class ranking least-squares probabilistic classifier 18A includes a magnitude comparison / determination unit 182A using a 4-class ranking LSPC, a scoring processing unit 184A, and a score sum processing unit 186.
That is, the 4-class ranking least-squares stochastic discriminator 18A is different from the 2-class ranking least-squares stochastic discriminator 18 in FIG. 4 between the magnitude comparison determination unit and the scoring processing unit using ranking LSPC. Configuration and operation are different. Hereinafter, only the differences will be described for the sake of simplicity.
4 magnitude comparison determination unit 182A using the ranking LSPC classes 4 using class ranking LSPC, the comparative base data _{(x k} ^{(a x te} with the test explanatory variable) test data and the comparative explanatory variables It is determined whether the impression level is “have” (the same processing is repeated for all comparison base data x _k ).
In this example, since the size determination is performed by ranking LSPC in 4 classes, the size comparison determination unit 182A using the ranking LSPC of 4 classes determines the difference (x ^te −x) between the test data and the comparison base data. _k ), the size of the objective variable between the two data is estimated (y ^te <y _k or y ^te ≧ y _k is estimated), and is classified into four as shown in the above equation (13).
Then, the magnitude comparison / determination unit 182A using the four classes of ranking LSPC outputs the magnitude determination result in the form of the certainty factor of each class. For ^example, in the comparative case of ^{x te} and _{x k,} magnitude comparison determination unit 182A using the ranking LSPC of 4 classes, for example, confidence of class 1 to class 4, respectively 0.65,0.2,0 ., Output as 0.05.
The scoring processing unit 184A performs scoring processing according to the above formulas (24), (25), and (21) using the magnitude determination comparison results output for each comparison base data.
Then, the score total processing unit 186 adds all the scores calculated for each comparison base data, and, as shown in the above equation (23), the location where the calculated score is the maximum is obtained as the impression degree estimation result. Output. The impression degree estimation result is “an estimation result of an objective variable by ranking LSPC when test data x ^te is input (for example, impression degree estimation result regarding“ adorable ”)”.
When the impression degree estimation apparatus 10A according to the second embodiment is configured by an electronic device, the impression degree estimation apparatus 10A can be realized by a computer that operates under program control. Although not shown, this type of computer, as is well known, includes an input device for inputting data, a data processing device, an output device for outputting processing results in the data processing device, and an auxiliary memory serving as various databases. Device. Then, the data processing device stores the program in a read-only memory (ROM), a random access memory (RAM) used as a work memory for temporarily storing data, and a program stored in the ROM. It consists of a central processing unit (CPU) that processes stored data.
In this case, the input device operates as a device (not shown) for inputting a person image (image data). The data processing apparatus operates as a data acquisition unit 32, a neural network (continuous amount estimation unit) 16 in regression analysis mode, and a ranking least squares probabilistic discriminator (order estimation unit) 18A. The auxiliary storage device functions as storage means (not shown) for storing a plurality of comparison base data.
In other words, each unit of the impression level estimation device 10A according to the second embodiment may be realized using a combination of hardware and software. In a form in which hardware and software are combined, each unit is realized as various means by operating hardware such as a control unit (CPU) based on an impression degree estimation program stored in the ROM. The impression degree estimation program may be recorded on a recording medium and distributed. The impression degree estimation program recorded on the recording medium is read into the memory via the wired, wireless, or recording medium itself, and operates the control unit and the like. Examples of the recording medium include an optical disk, a magnetic disk, a semiconductor memory device, and a hard disk.
The impression level estimation device 10A having such a configuration can accurately estimate the impression levels of both ends of a person.
In the second embodiment as well, the feature vector calculation unit 34 in the neural network 16 is used as means for calculating the face feature vector. However, the present invention is not limited to this, and various other known face feature vectors. Calculation means may be used.

する。ここでは、推定誤差の標準偏差を０．５とした。各印象度ｚ（０．０≦ｚ≦４．０）（０．２５間隔）に対し、一度、下記の式（２６）

によりスコアを割り当てたのち、スコアを下記の式（２７）で正規化する。

そして、ベクトルｆ_１＝｛ｆ_１（ｚ）｝_{０．０≦ｚ≦４．０}をニューラルネットワーク（回帰モード）１６の出力スコア（第１の印象度スコア）とする。
上述したように、回帰分析モードのニューラルネットワーク１６は、人物の印象度を連続量として推定する回帰問題として解くので、連続量推定手段とも呼ばれる。
次に、最小二乗確率的識別器（ＬＳＰＣ）２０について説明する。最小二乗確率的識別器（ＬＳＰＣ）２０は、ニューラルネットワーク１６の中間層１６４の出力を顔特徴ベクトルとして、印象度推定問題を識別問題として解く。この場合、人物の印象度の値域を、例えば（０．０〜／０．５〜／１．５〜／２．５〜／３．５〜）の様に、数クラスに区分し、これらのクラスを識別する様にする。印象度の大小に従って区分を行っているため、これは、ポイントワイズ・アプローチに基づくランキング学習の１種と解釈することも出来る。
引き続いて、最小二乗確率的識別器（ＬＳＰＣ）２０の出力のスコア化について説明する。
最小二乗確率的識別器（ＬＳＰＣ）２０は、各クラスの確信度を確率形式で出力する。ここでは、前述したニューラルネットワーク（回帰モード）１６の出力に対する第１の印象度スコア（ベクトル）ｆ_１の形式に合わせる様に、スコア化を行う。
例えば、印象度を０．０〜／０．５〜／１．５〜／２．５〜／３．５〜４．０の５クラスで識別を行う時、最小二乗確率的識別器（ＬＳＰＣ）２０の出力した確信度が順にｐ_１，ｐ_２，・・・，ｐ_５であったする。この時、各印象度ｚに対し、一旦、次の（式２８）

（ｉは、印象度ｚの属するクラスに対応）によりスコアを割り当てたのち、スコアを下記の式（２９）で正規化する。

そして、ベクトルｆ_２＝｛ｆ_２（ｚ）｝_{０．０≦ｚ≦４．０}を最小二乗確率的識別器（ＬＳＰＣ）２０の出力スコア（第２の印象度スコア）とする。
最小二乗確率的識別器（ＬＳＰＣ）２０では、人物の印象度を離散量として推定するので、離散量推定手段とも呼ばれる。
４クラスのランキング最小二乗確率的識別器１８Ａでは、上記数４３で表されるベクトルｆ_３をその出力スコア（第３の印象度スコア）とする。このスコアの形式は、上記ベクトルｆ_１，ｆ_２に合わせるものとする。
ランキング最小二乗確率的識別器１８Ａは、人物の印象度を、上述したように、テストデータと複数の比較用基底データとを順序的に比較することにより推定するので、順序推定手段とも呼ばれる。
次に、識別器出力の重み付き統合部２２について説明する。
識別器出力の重み付き統合部２２は、重みｗ_１、ｗ_２、ｗ_３を用いて、各印象度ｙ（０．０≦ｙ≦４．０，０．２５間隔）における各識別器１６，２０，１８Ａの出力スコア（第１乃至第３の印象度スコア）を、下記の式（３０）で表せるように、重み付きで加算する。

そして、識別器出力の重み付き統合部２２は、下記の式（３１）で表されるｙ^＊

となる様なｙ^＊を、印象度の推定結果として出力する。
識別器出力の重み付き統合部２２は、統合手段と呼ばれる。
また、回帰分析モードのニューラルネットワーク１６、最小二乗確率的識別器（ＬＳＰＣ）２０、ランキング最小二乗確率的識別器１８Ａ、および識別器出力の重み付き統合部２２の組み合わせは、統合識別器（１６，２０，１８Ａ，２２）と呼ばれる。
次に、重みの探索方法について説明する。
検証データ（モデル学習に用いていないデータ）を用いて、最適な重みｗ_１、ｗ_２、ｗ_３を虱潰しに探索する。具体的には、数値幅を、０≦ｗ_ｉ≦１、探索間隔を０．０５に設定し、検証用データで、統合識別器（１６，２０，１８Ａ，２２）の評価を実施する。検証用データで評価した時のスコア（各カテゴリの認識率の平均値）が最も高くなる様な重み（ｗ_１、ｗ_２、ｗ_３）を採択する。
電子機器で第３の実施例に係る印象度推定装置１０Ｂを構成する場合、印象度推定装置１０Ｂは、プログラム制御により動作するコンピュータで実現可能である。図示はしないが、この種のコンピュータは、周知のように、データを入力する入力装置と、データ処理装置と、データ処理装置での処理結果を出力する出力装置と、種々のデータベースとして働く補助記憶装置とを備えている。そして、データ処理装置は、プログラムを記憶するリードオンリメモリ（ＲＯＭ）と、データを一時的に記憶するワークメモリとして使用されるランダムアクセスメモリ（ＲＡＭ）と、ＲＯＭに記憶されたプログラムに従って、ＲＡＭに記憶されているデータを処理する中央処理装置（ＣＰＵ）とから構成される。
この場合、入力装置は、人物画像（画像データ）を入力するための装置（図示せず）として動作する。データ処理装置は、データ取得手段３２、回帰分析モードのニューラルネットワーク（連続量推定手段）１６、最小二乗確率的識別器（連続量推定手段）２０、ランキング最小二乗確率的識別器（順序推定手段）１８Ａ、および識別器出力の重み付き統合部（統合手段）２２として動作する。そして、補助記憶装置が、複数個の比較用基底データを蓄積する蓄積手段（図示せず）として働く。
換言すれば、第３の実施例に係る印象度推定装置１０Ｂの各部は、ハードウェアとソフトウェアとの組み合わせを用いて実現すればよい。ハードウェアとソフトウェアとを組み合わせた形態では、ＲＯＭに記憶された印象度推定プログラムに基づいて制御部（ＣＰＵ）等のハードウェアを動作させることによって、各部を各種手段として実現する。また、該印象度推定プログラムは、記録媒体に記録されて頒布されても良い。当該記録媒体に記録された印象度推定プログラムは、有線、無線、又は記録媒体そのものを介して、メモリに読込まれ、制御部等を動作させる。尚、記録媒体を例示すれば、オプティカルディスクや磁気ディスク、半導体メモリ装置、ハードディスクなどが挙げられる。
次に、人の顔画像を用いて、‘かわいらしい’印象度を推定した場合の実験結果（すなわち、第３の実施例による印象度推定装置１０Ｂの効果）について説明する。
以下では、回帰分析モードのニューラルネットワーク（連続量推定手段）１６を使用した手法を「手法１」と呼び、最小二乗確率的識別器（ＬＳＰＣ）（離散量推定手段）２０を使用した手法を「手法２」と呼び、２クラスのランキング最小二乗確率的識別器（順序推定手段）１８を使用した手法を「手法３’」と呼び、４クラスのランキング最小二乗確率的識別器（順序推定手段）１８Ａを使用した手法を「手法３」と呼ぶことにする。
また、本例では、統合識別器（１６，２０，１８Ａ，２２）における、（手法１：手法２：手法３）の統合比を、０．３５：０．１５：０．５０とした。
印象度区分が（０．０〜／０．５〜／１．５〜／２．５〜／３．５〜４．０）の５クラス識別問題として解いたとする。図８はその識別問題として解いた場合の各クラスの認識率を示し、図９は回帰問題として解いた場合のＭＡＥ（絶対誤差平均）を示す。認識率は、１００％であれば誤りなく認識されたことを示す。また、ＭＡＥ（絶対誤差平均）は、その小さいほど誤差が小さいことを示す。
図８および図９から、手法３において、人物の両端の印象度の推定精度が従来手法（手法１、手法２）より改善していることが分かる。また、（手法３）の４クラスのｒａｎｋｉｎｇ ＬＳＰＣの方が、（手法３’）の２クラスのｒａｎｋｉｎｇ ＬＳＰＣより、やや良いことが見て取れる。
このように、本第３の実施例に係る印象度推定装置１０Ｂでは、複数の識別器１６、２０、１８Ａの出力を重み付きで統合しているので、各識別器の弱点を相補することが出来、単体の識別器では成し得なかった安定した識別精度を実現することが出来る。
尚、本第３の実施例では、順序推定手段として、４クラスのランキング最小二乗確率的識別器１８Ａを使用しているが、その代わりに２クラスのランキング最小二乗確率的識別器１８を使用してもよい。
以上、実施の形態を参照して本願発明を説明したが、本願発明は上記実施の形態に限定されるものではない。本願発明の構成や詳細には、本願発明のスコープ内で当業者が理解し得る様々な変更をすることができる。
例えば、上記第３の実施例では、３つの識別器（連続量推定手段；離散量推定手段；順序推定手段）を統合した場合の例を示しているが、本発明は２つの識別器を統合するようにしても良い。具体的には、本発明の印象度推定装置は、連続量推定手段１６と順序推定手段１８Ａ（又は１８）とを統合しても良いし、離散量推定手段２０と順序推定手段１８Ａ（又は１８）とを統合しても良い。また、上記実施例では、ランキング最小二乗確率的識別器として、２クラスのランキング最小二乗確率的識別器１８と２クラスのランキング最小二乗確率的識別器１８Ａとの例のみを挙げて説明しているが、一般的に、複数クラス（２クラス以上）のランキング最小二乗確率的識別器を使用してよい。Next, an impression level estimation device 10B according to a third example of the present invention will be described with reference to FIG.
The impression level estimation device 10B shown in the figure is the impression level estimation device 10A shown in FIG. 5 except that a least square probabilistic classifier (LSPC) 20 and a weighted integration unit 22 for classifier output are further added. It operates in the same way as the above. Therefore, the same components as those shown in FIG. 5 are denoted by the same reference numerals, and the description thereof is omitted for the sake of simplicity.
As described above, by introducing a multi-class ranking LSPC, the accuracy of the impression level at both ends of the person is improved, but the accuracy of the impression level near the middle of the person is not good.
Therefore, in the impression level estimation apparatus 10B according to the third embodiment, in addition to the classifier by multi-class ranking LSPC, a classifier for regression analysis and class identification, which is a conventional technique, and Is used to stabilize the impression level estimation result.
First, the neural network 16 in the regression analysis mode will be described. The neural network 16 in the regression analysis mode is a neural network in the regression mode that is solved as a regression problem that estimates the impression level of a person as a continuous quantity. When dealing with multiple types of impression estimation problems, there are cases where multitask learning is performed.
Subsequently, scoring of the output of the neural network 16 will be described.

To do. Here, the standard deviation of the estimation error is 0.5. For each impression degree z (0.0 ≦ z ≦ 4.0) (0.25 interval), the following equation (26)

After assigning the score, the score is normalized by the following equation (27).

The vector f ₁ = {f ₁ (z)} _{0.0 ≦ z ≦ 4.0} is set as an output score (first impression score) of the neural network (regression mode) 16.
As described above, the neural network 16 in the regression analysis mode solves as a regression problem that estimates the impression level of a person as a continuous quantity, and is also called continuous quantity estimation means.
Next, the least square probabilistic classifier (LSPC) 20 will be described. The least square probabilistic classifier (LSPC) 20 solves the impression estimation problem as a discrimination problem using the output of the intermediate layer 164 of the neural network 16 as a face feature vector. In this case, the range of the impression level of a person is divided into several classes, for example (0.0 to /0.5 to /1.5 to /2.5 to /3.5 to), Try to identify the class. Since classification is performed according to the degree of impression, this can be interpreted as a kind of ranking learning based on a point-wise approach.
Subsequently, scoring of the output of the least square probabilistic classifier (LSPC) 20 will be described.
The least square probabilistic classifier (LSPC) 20 outputs the confidence of each class in a probability format. Here, scoring is performed so as to match the format of the first impression degree score (vector) f ₁ with respect to the output of the neural network (regression mode) 16 described above.
For example, when classifying impressions with five classes of 0.0 to /0.5 to /1.5 to /2.5 to /3.5 to 4.0, the least square probabilistic classifier (LSPC) Assume that the certainty levels output by 20 are p ₁ , p ₂ ,..., P ₅ in order. At this time, for each impression degree z, the following (Formula 28)

After assigning a score according to (i corresponds to the class to which the impression degree z belongs), the score is normalized by the following equation (29).

The vector f ₂ = {f ₂ (z)} _{0.0 ≦ z ≦ 4.0} is set as the output score (second impression score) of the least square probabilistic classifier (LSPC) 20.
Since the least square probabilistic classifier (LSPC) 20 estimates the impression level of a person as a discrete quantity, it is also called a discrete quantity estimation means.
4 Class The Index least square stochastic classifier 18A of the vector f ₃ represented by the number 43 and its output score (third impression score). The score format is adjusted to the vectors f ₁ and f ₂ .
The ranking least square probabilistic classifier 18A estimates the person's impression level by comparing the test data with a plurality of comparison base data in order as described above, and is also called order estimation means.
Next, the weighted integration unit 22 for discriminator output will be described.
The weighted integration unit 22 of the discriminator output uses the weights w ₁ , w ₂ , and w _3, and the discriminators 16, The output scores (first to third impression scores) of 20 and 18A are added with weights so that they can be expressed by the following equation (30).

Then, the weighted integration unit 22 of the discriminator output is y ^* represented by the following equation (31) ^.

The such y ^* a, to output as the estimated result of the impression.
The weighted integration unit 22 for discriminator output is called integration means.
The combination of the neural network 16 in the regression analysis mode, the least square probabilistic classifier (LSPC) 20, the ranking least squares probabilistic classifier 18A, and the weighted integration unit 22 of the classifier output is an integrated classifier (16, 20, 18A, 22).
Next, a weight search method will be described.
Using the verification data (data not used for model learning), the optimal weights w ₁ , w ₂ , and w ₃ are searched in a collapsed manner. Specifically, the numerical value width is set to 0 ≦ w _i ≦ 1 and the search interval is set to 0.05, and the integrated discriminator (16, 20, 18A, 22) is evaluated with the verification data. The weights (w ₁ , w ₂ , w ₃ ) are selected so that the score (average value of the recognition rate of each category) when evaluated with the verification data is the highest.
When the impression degree estimation apparatus 10B according to the third embodiment is configured by an electronic device, the impression degree estimation apparatus 10B can be realized by a computer that operates by program control. Although not shown, this type of computer, as is well known, includes an input device for inputting data, a data processing device, an output device for outputting processing results in the data processing device, and an auxiliary memory serving as various databases. Device. Then, the data processing device stores the program in a read-only memory (ROM), a random access memory (RAM) used as a work memory for temporarily storing data, and a program stored in the ROM. It consists of a central processing unit (CPU) that processes stored data.
In this case, the input device operates as a device (not shown) for inputting a person image (image data). The data processing apparatus includes a data acquisition means 32, a regression analysis mode neural network (continuous quantity estimation means) 16, a least squares stochastic discriminator (continuous quantity estimation means) 20, a ranking least squares stochastic discriminator (order estimation means). 18A and a weighted integration unit (integration means) 22 of the discriminator output. The auxiliary storage device functions as storage means (not shown) for storing a plurality of comparison base data.
In other words, each unit of the impression level estimation device 10B according to the third embodiment may be realized using a combination of hardware and software. In a form in which hardware and software are combined, each unit is realized as various means by operating hardware such as a control unit (CPU) based on an impression degree estimation program stored in the ROM. The impression degree estimation program may be recorded on a recording medium and distributed. The impression degree estimation program recorded on the recording medium is read into the memory via the wired, wireless, or recording medium itself, and operates the control unit and the like. Examples of the recording medium include an optical disk, a magnetic disk, a semiconductor memory device, and a hard disk.
Next, an experimental result (that is, the effect of the impression level estimation device 10B according to the third example) when a “pretty” impression level is estimated using a human face image will be described.
Hereinafter, a technique using the neural network (continuous quantity estimating means) 16 in the regression analysis mode is referred to as “method 1”, and a technique using the least square probabilistic classifier (LSPC) (discrete quantity estimating means) 20 is referred to as “method 1”. Called “Method 2”, a method using a two-class ranking least-squares stochastic discriminator (order estimation means) 18 is called “Method 3 ′”, and a four-class ranking least-squares probability classifier (order estimation means). The method using 18A will be referred to as “method 3”.
In this example, the integration ratio of (Method 1: Method 2: Method 3) in the integrated classifier (16, 20, 18A, 22) is set to 0.35: 0.15: 0.50.
It is assumed that the impression class is solved as a 5-class identification problem with (0.0 to /0.5 to /1.5 to /2.5 to /3.5 to 4.0). FIG. 8 shows the recognition rate of each class when solved as the identification problem, and FIG. 9 shows MAE (absolute error average) when solved as a regression problem. If the recognition rate is 100%, it indicates that the recognition has been made without error. Further, MAE (absolute error average) indicates that the smaller the error, the smaller the error.
From FIG. 8 and FIG. 9, it can be seen that, in Method 3, the accuracy of estimation of the impression degree at both ends of the person is improved from the conventional methods (Method 1 and Method 2). Also, it can be seen that the 4-class ranking LSPC of (Method 3) is slightly better than the 2-class ranking LSPC of (Method 3 ′).
Thus, in the impression level estimation device 10B according to the third embodiment, the outputs of the plurality of discriminators 16, 20, and 18A are integrated with weights, so that the weak points of each discriminator can be complemented. It is possible to achieve stable discrimination accuracy that cannot be achieved by a single discriminator.
In the third embodiment, the four-class ranking least-squares probabilistic classifier 18A is used as the order estimation means. Instead, a two-class ranking least-squares probability classifier 18 is used. May be.
Although the present invention has been described with reference to the embodiments, the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.
For example, in the third embodiment, an example in which three discriminators (continuous quantity estimation means; discrete quantity estimation means; order estimation means) are integrated is shown, but the present invention integrates two discriminators. You may make it do. Specifically, the impression degree estimation apparatus of the present invention may integrate the continuous quantity estimation means 16 and the order estimation means 18A (or 18), or the discrete quantity estimation means 20 and the order estimation means 18A (or 18). ) May be integrated. Further, in the above-described embodiment, only examples of the two-class ranking least-squares probability classifier 18 and the two-class ranking least-squares probability classifier 18A are described as ranking least-squares probability classifiers. However, in general, multiple classes (two or more classes) of least-squares stochastic discriminators may be used.

  The present invention can be used for sales support of cosmetics, glasses, etc. that change the impression of the face. For example, in the case of web sales of cosmetics, the consumer can check the impression change at the time of makeup at home with a smart device such as a smartphone / tablet. At this time, it is possible to encourage purchase by recommending makeup items used in the simulation. In addition to use at home, it can also be used at drugstore stores and cafes where you can make up. Since the first impression of a person is often interested in social exchanges and activities, the present invention is expected to expand in various usage scenes in the future.

DESCRIPTION OF SYMBOLS 10, 10A, 10B Impression degree estimation apparatus 12 Head detection part 14 Face detection part 16 Neural network of regression analysis mode (continuous quantity estimation means)
162 Input layer 164 Intermediate layer 166 Output layer 18, 18A Ranking least square probabilistic classifier (order estimation means)
182, 182A Size comparison determination unit 184, 184A Scoring processing unit 186 Score total processing unit 20 Least squares probabilistic discriminator (discrete quantity estimation means)
22 Weighted integration unit (integration means) for discriminator output
34 Feature Vector Calculation Unit This application claims priority based on Japanese Patent Application No. 2013-149123 filed on July 18, 2013, the entire disclosure of which is incorporated herein.

Claims (24)

An impression level estimation method for estimating an impression level of a person photographed in image data using an impression level estimation device,
An extraction step of extracting a face image from the image data;
A calculation step of calculating a face feature vector from the face image;
A plurality of comparative base data already obtained from test data and learning data using the face image vector as an explanatory variable, modeled by a class least-squares stochastic discriminator of the degree of impression between two samples. And estimating the impression level of the person by scoring based on the certainty of each class obtained by sequentially comparing the plurality of classes by the least squares stochastic discriminator,
Impression degree estimation method including Each of the plurality of comparison base data includes a learning explanatory variable that is a feature amount of a face image and a known comparative variable that is a numerical value of the degree of impression corresponding to the comparison explanatory variable. Data,
The test data consists of test explanatory variables for the face feature vector and unknown test objective variables corresponding to the test explanatory variables,
The estimation step includes
The test data and the plurality of comparison base data are sequentially compared using the plurality of classes of least-squares stochastic discriminators, and the test explanatory variable of the test data is compared with each comparison base data. Classifying the difference between the test objective variable of the test data and the comparison objective variable of each comparison base data into a plurality of classes according to the degree of magnitude. Obtaining a certainty of
Scoring the certainty of each class for each comparison base data to obtain a score;
Adding all the scores calculated for each comparison base data, and estimating the position where the calculated total score is the maximum as the person's impression degree;
The impression degree estimation method according to claim 1, wherein:   The impression degree estimation method according to claim 2, wherein the plurality of comparison base data are already obtained in a model learning stage of the plurality of classes of least squares stochastic discriminators.   The impression degree estimation method according to any one of claims 1 to 3, wherein the plurality of classes of least-squares stochastic classifiers comprises two classes of least-squares stochastic classifiers.   The impression degree estimation method according to any one of claims 1 to 3, wherein the plurality of classes of least-squares stochastic discriminators comprises three or more classes of least-square stochastic discriminators.   The impression level of the person represents at least one impression level selected from impressions including “bright and refreshing”, “adorable”, “business”, “friendly”, and “healthy”. 6. The impression degree estimation method according to any one of 5 above.   The impression level estimation method according to claim 6, wherein the estimation step evaluates the at least one impression in a plurality of stages as the impression level of the person. An impression level estimation device that estimates the impression level of a person captured in image data,
Data acquisition means for extracting a face image from the image data;
Feature vector calculation means for calculating a face feature vector from the face image;
A plurality of comparative base data already obtained from test data and learning data using the face image vector as an explanatory variable, modeled by a class least-squares stochastic discriminator of the degree of impression between two samples. And order estimation means for estimating the impression degree of the person by scoring based on the certainty of each class obtained by sequentially comparing the plurality of classes by the least-square ranking probabilistic classifier,
Impression degree estimation device. Each of the plurality of comparison base data includes a learning explanatory variable that is a feature amount of a face image and a known comparative variable that is a numerical value of the degree of impression corresponding to the comparison explanatory variable. Data,
The test data consists of test explanatory variables for the face feature vector and unknown test objective variables corresponding to the test explanatory variables,
The order estimation means includes
The test data and the plurality of comparison base data are sequentially compared using the plurality of classes of least-squares stochastic discriminators, and the test explanatory variable of the test data is compared with each comparison base data. Classifying the difference between the test objective variable of the test data and the comparison objective variable of each comparison base data into a plurality of classes according to the degree of magnitude. A size comparison and determination unit that obtains certainty of
A scoring processing unit that obtains a score by scoring the certainty of each class for each comparison base data;
A score total processing unit that adds all the scores calculated for each comparison base data and estimates the calculated total score as the maximum impression level of the person,
The impression degree estimation device according to claim 8, comprising:   The impression degree estimation device according to claim 9, wherein the plurality of comparison base data are already obtained in a model learning stage of the plurality of classes of least squares stochastic discriminators.   The impression level estimation device according to any one of claims 8 to 10, wherein the plurality of classes of least squares stochastic discriminators comprises two classes of least squares stochastic discriminators.   The impression level estimation device according to any one of claims 8 to 10, wherein the plurality of classes of least-squares stochastic discriminators comprises three or more ranking least-squares stochastic discriminators.   9. The impression level of the person represents at least one impression level selected from impressions including “bright and refreshing”, “adorable”, “business”, “friendly”, and “healthy”. 13. Impression degree estimation apparatus of any one of 12.   The impression degree estimation device according to claim 13, wherein the order estimation unit evaluates the at least one impression in a plurality of stages as the impression degree of the person. An impression degree estimation program for causing a computer to estimate an impression degree of a person captured in image data, the computer comprising:
An extraction procedure for extracting a face image from the image data;
A calculation procedure for calculating a face feature vector from the face image;
A plurality of comparative base data already obtained from test data and learning data using the face image vector as an explanatory variable, modeled by a class least-squares stochastic discriminator of the degree of impression between two samples. And an estimation procedure for estimating the impression degree of the person by scoring based on the certainty of each class obtained by sequentially comparing the plurality of classes by the least-square ranking probabilistic classifier,
Impression degree estimation program to execute. Each of the plurality of comparison base data includes a learning explanatory variable that is a feature amount of a face image and a known comparative variable that is a numerical value of the degree of impression corresponding to the comparison explanatory variable. Data,
The test data consists of test explanatory variables for the face feature vector and unknown test objective variables corresponding to the test explanatory variables,
The estimation procedure is performed on the computer,
The test data and the plurality of comparison base data are sequentially compared using the plurality of classes of least-squares stochastic discriminators, and the test explanatory variable of the test data is compared with each comparison base data. Classifying the difference between the test objective variable of the test data and the comparison objective variable of each comparison base data into a plurality of classes according to the degree of magnitude. The steps to get confidence
A procedure for scoring the certainty of each class for each comparison base data to obtain a score;
A procedure for adding all the calculated scores for each comparison base data and estimating the impression level of the person, where the calculated total score is maximum,
The impression degree estimation program according to claim 15, wherein:   The impression degree estimation program according to claim 16, wherein the plurality of comparison base data are already obtained in a model learning stage of the plurality of classes of least squares stochastic discriminators.   The impression degree estimation program according to any one of claims 15 to 17, wherein the plurality of classes of least-squares stochastic discriminators comprises two classes of least-squares stochastic discriminators.   The impression degree estimation program according to any one of claims 15 to 17, wherein the plurality of classes of least-squares stochastic discriminators comprises three or more ranking least-squares stochastic discriminators.   16. The impression level of the person represents at least one impression level selected from impressions including “bright and refreshing”, “adorable”, “business”, “friendly” and “healthy”. The impression degree estimation program according to any one of 19 above.   21. The impression degree estimation program according to claim 20, wherein the estimation procedure causes the computer to evaluate the at least one impression in a plurality of stages as an impression degree of the person. An impression level estimation method for estimating an impression level of a person photographed in image data using an impression level estimation device,
An extraction step of extracting a face image from the image data;
A first impression degree score is estimated by scoring an estimated impression degree, which is a continuous quantity obtained by inputting the face image to a neural network, and the face image is input to the neural network. A first estimation step of outputting the intermediate layer data obtained from the intermediate layer as a face feature vector;
A second estimation step of estimating a second impression score by scoring the certainty of each class which is a discrete quantity obtained by inputting the face feature vector into a least squares probabilistic classifier;
A plurality of comparative base data already obtained from test data and learning data using the face image vector as an explanatory variable, modeled by a class least-squares stochastic discriminator of the degree of impression between two samples. And a third estimation step of estimating a third impression score by scoring based on the certainty of each class obtained by sequentially comparing the plurality of classes with a ranking least squares probabilistic classifier of the plurality of classes,
An integration step of integrating the first to third impression degree scores with weights to estimate the impression degree of the person;
Impression degree estimation method including An impression level estimation device that estimates the impression level of a person captured in image data,
Data acquisition means for extracting a face image from the image data;
A first impression degree score is estimated by scoring an estimated impression degree, which is a continuous quantity obtained by inputting the face image to a neural network, and the face image is input to the neural network. Continuous quantity estimating means for outputting the intermediate layer data obtained from the intermediate layer as a face feature vector;
Discrete quantity estimation means for estimating a second impression score by scoring the certainty of each class, which is a discrete quantity obtained by inputting the face feature vector into a least squares probabilistic classifier;
A plurality of comparative base data already obtained from test data and learning data using the face image vector as an explanatory variable, modeled by a class least-squares stochastic discriminator of the degree of impression between two samples. Order estimation means for estimating the third impression score by scoring based on the certainty of each class obtained by sequentially comparing the plurality of classes by the least-square ranking least square probabilistic classifier,
Integrating means for integrating the first to third impression degree scores with weights to estimate the impression degree of the person;
Impression degree estimation device. An impression degree estimation program for causing a computer to estimate an impression degree of a person captured in image data, the computer comprising:
An extraction procedure for extracting a face image from the image data;
A first impression degree score is estimated by scoring an estimated impression degree, which is a continuous quantity obtained by inputting the face image to a neural network, and the face image is input to the neural network. A first estimation procedure for outputting intermediate layer data obtained from the intermediate layer as a face feature vector;
A second estimation procedure for estimating a second impression score by scoring the certainty of each class which is a discrete quantity obtained by inputting the face feature vector into a least squares probabilistic classifier;
A plurality of comparative base data already obtained from test data and learning data using the face image vector as an explanatory variable, modeled by a class least-squares stochastic discriminator of the degree of impression between two samples. And a third estimation procedure for estimating a third impression score by scoring on the basis of the certainty of each class obtained by sequentially comparing the plurality of classes by the least-square ranking least square probabilistic classifier,
An integration procedure for estimating the impression level of the person by integrating the first to third impression level scores with weights;
Impression degree estimation program to execute.

Images