JP2004145625A - Device for preparing portrait - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device for preparing portrait suitable for transmitting feelings by using a portrait through a portable telephone. <P>SOLUTION: This device for preparing portrait from the featured points of the respective face parts of eyebrows, eyes, nose, mouth, ears and face outline and the image of a hair style is provided with a wrinkle describing means for describing wrinkles corresponding to specific feelings at the time of plotting expressions showing specific feelings and a winkle plotting means for plotting the described wrinkles in a portrait to be prepared. Also, this preparing device may be provided with a wrinkle describing means for describing wrinkles corresponding to the specific feelings based on an expressionless portrait at the time of plotting expressions showing the specific feelings and a wrinkle plotting means for plotting the described wrinkles in the expressionless portrait. <P>COPYRIGHT: (C)2004,JPO

Description

【００４８】
記述が終了すると、表情認識結果を参照し、記述された曲線に従って、表１の配置位置に皺を描画する（ステップＳ２１０）。皺の線の太さは任意であるが、一般的には顔部品を描画する線の太さと同等かそれ以下とする。但し、特に皺を強調したい場合には、皺の線を顔部品を描画する線の太さよりも太く設定することにより、より誇張された表情が得られる。また、ここでは、線の濃さすなわち濃度は一定としているが、特定の位置の皺を描画する線分全体、または、皺を描画する個々の線分の一部分の濃さを変える、すなわち、皺を描画する線分に濃淡をつけることによって、よりリアルな表情を得ることができる。上記一部分の濃度を変える場合は、例えば上から下に緩やかな濃度勾配を設けるなど、連続的な濃度変化をもたせることにより、より自然な表現が可能である。
【００４９】
皺の配置は表１の配置位置をもとに顔の左右の対称性にしたがい、顔の中心対称にならって配置するが、表情によって左右の対象性が崩れている場合や、顔が傾いている場合には、形を含め非対称に配置すれば、よりリアルな似顔絵を作成することができる。
【００５０】
表１に記載した皺の記述例１を用いて作成した各表情に対する似顔絵の作成例を図５に示す。図５（ａ）は「怒り」に対応した皺を描画したものである。「怒り」に対応した似顔絵を作成する場合、瞳の記述で得られた瞳径を故意に小さくし、瞳中心を、下側のまぶたに瞳が接するように移動して、上まぶたの曲線と瞳の間に空白を生じさせ、又は瞳径はそのままで、瞳の上端に接する上まぶた端点を上方に移動させることにより空白を生じさせ、眉の外側を持ち上げるなどの加工を行うことにより図５（ｂ）に示すように、より誇張された表情を得ることも可能である。
【００５１】
「喜び」に対応する似顔絵の作成例を図５（ｃ）に、「驚き」に対応する似顔絵の作成例を図５（ｄ）に示す。図５（ｄ）では、図５（ｂ）と同様、瞳と上まぶたの間に隙間を作り、誇張を行った。また、唇の端点で、上唇と下唇を切り離し、間に隙間を設けることも効果的である。
【００５２】
図５（ｅ）は、「悲しみ」に対する似顔絵の作成例である。悲しみの場合は、表１の「怒り」と同じ皺の記述を用いても、同様の効果が得られる。
【００５３】
また、図５（ｆ）では、目が閉じている場合の似顔絵の作成例である。目が閉じている場合、図５（ｆ）では閉じられたまぶたの上側に、４個の節点からなる波形の皺を付加している。図中の目の上の左側の皺には、節点がわかるように記載したが、実際には描画しない。また、眉間の皺は図５（ｅ）と同様である。
【００５４】
これにより、まぶたが閉じた状態でも、悲しい顔が表現できる。また、皺の付加位置、およびその大きさについては、表示後にユ−ザ−のキ−操作などにより変更可能とすることもできる。この場合は、個人の好みにより皺や顔部品を表す線の太さや濃さも含め、その程度を変更することにより、よりリアルな似顔絵とすることができる。
【００５５】
以上で説明した似顔絵の作成方法を用いてカメラで撮影した顔画像から似顔絵を作成し、表情識別の効果をみる実験を行った。実験は、「喜び」、「怒り」、「驚き」、「悲しみ」及び「無表情」の５種類の表情について、男女計５人の顔画像と、顔画像から生成された似顔絵について評価した。すなわち、顔画像と似顔絵について、「喜び」、「怒り」、「驚き」、「悲しみ」及び「無表情」から表情を選んでもらうアンケ−ト方式の評価を５０人に対して行った。
【００５６】
【表２】

【００５７】
表２の（ａ）は元となるカメラで撮影した顔画像を用いた評価結果である。表は、左端に表示されている無表情を含む５種類の画像を提示したときに、それぞれの表情が、評価者によって、どの表情に認識されたかを、横方向に示している。例えば、「怒り」を表している顔画像を提示したとき、８８％の人が「怒り」と認識し、１２％の人は「無表情」と認識したことを示している。この実験の結果、似顔絵のもととなる顔画像の提示では全ての表情に対して、７割以上の高い正答率が得られている。
【００５８】
表２の（ｂ）は皺の無い似顔絵を用いた場合の評価結果であり、特に「怒り」や「悲しみ」、「驚き」などで正答率が落ちている。
【００５９】
表２の（ｃ）は、皺を付加した場合の結果である。表２の（ｂ）と比較すると、いずれの表情も認識率、つまり正答率が向上しており、「怒り」や「悲しみ」で顕著な効果が得られた。
【００６０】
なお、本例で、皺の記述として、３つの節点からなる記述を用いたが、端点のみの２点を用いた直線を使用しても最低限の効果が得られる。また、本例で「怒り」「喜び」「驚き」「悲しみ」の４表情を認識するものとしたが、図２の表情認手段にこれ以外の表情たとえば、上記のうちの２者の中間的な表情を認識させることも学習などにより可能である。
【００６１】
実施の形態２．
実施の形態１では、カメラ等により入力された顔画像の表情を認識して、似顔絵を作成する方法について説明したが、実施の形態２では、予め得られている、無表情の顔画像に対して、感情表現を有する似顔絵を作成する手法について説明する。
【００６２】
無表情の顔画像は、図１のＲＡＭ１１またはＲＯＭ１２に格納されている。また、この顔画像に対し、似顔絵を作成したことがある場合は、その特徴点や顔部品、皺等の曲線に関する記述情報をＲＡＭ１１やＲＯＭ１２等に記録しておき、その情報を次回の似顔絵の作成に再利用することも可能である。
【００６３】
また、こうした画像や特徴点、曲線の記述情報はメモリチップやサ−バなど、外部デ−タとして保存しておくことも可能である。
【００６４】
図６は実施の形態２における似顔絵の作成処理のフロ−チャ−トを示す。実施の形態１では、カメラ等から入力された顔画像から表情の認識を行ったが、本例では、入力される顔画像ａは無表情のものとし（ステップＳ１００）、図３の表情認識処理（ステップＳ１３０）に代えて、表情入力処理（ステップＳ２２０）及び特徴点移動処理（ステップＳ２３０）を行う。
【００６５】
表情入力処理（ステップＳ２２０）ではユ−ザ−が、図１の入力装置１４を用いて「怒り」「喜び」「驚き」「悲しみ」等の表情を入力する。特徴点移動処理（ステップＳ２３０）では、この入力結果に基づき、既存の手法たとえば、楊大昭ら：「ヒューマンインタフェースにおける顔表情の認識・合成手法に関する基礎研究」ヒューマンインターフェースシンポジウム’９９論文集，ｐ．１４１（１９９９）などに記載された手法によって、表情付けのための特徴点の移動を行う。
【００６６】
このあと、顔部品の記述処理と描画を行い（ステップＳ１４０〜Ｓ１６０）、髪の描画（ステップＳ１７０）、皺の記述処理と描画処理（ステップＳ１８０〜Ｓ２１０）を実施の形態１と同様の方法で実行し、入力された表情に対応する似顔絵を作成することができる。
【００６７】
本例における機能ブロック図を図７に示す。本例が実施の形態１の機能ブロック図と異なる点は表情認識手段１５０に代え、特徴点移動手段１８０が設けられている点である。この特徴点移動手段１８０は表情入力があると、特徴点抽出手段１００によって抽出された特徴点を移動し、顔部品記述手段１１０及び皺描画手段１７０に与え、「無表情」の顔画像と移動した特徴点に基づいて顔部品の記述及び皺の描画がなされるようになっている。
【００６８】
本例では、個々の顔画像に対して特徴点描出を行った後で特徴点の移動を行ったが、予め多数の顔画像からの似顔絵作成例に基づいて顔部品または皺の記述データに関して曲線の移動量を表情ごとにルール化しておき、無表情の顔画像から作成された顔部品の記述手段の結果や、皺の記述手段の結果をこのルールに基づいて移動させることにより、表情を有する似顔絵を作成することもできる。
【００６９】
この場合の機能ブロック図を図８に示す。本例が図７の機能ブロック図と異なる点は、顔部品記述手段１１０と顔部品描画手段１２０の間に特徴点移動手段１９０が、皺記述手段１６０と皺描画手段１７０との間に節点移動手段２００が設けられている点である。
【００７０】
顔部品または皺の記述データに関して曲線の移動量を表情ごとにルール化する場合、そのルールは予め作成してＲＯＭ等に格納しておく。予め作成したルールを用いる場合の処理フローは図７、図８から明らかなように、顔部品または皺記述手段の前に特徴点や接点の移動を行うか、同記述手段の後に特徴点や節点の移動を行うかの違いのみである。
【００７１】
その相違は容易に理解できるので、ここでは詳細な記述を省略するが、図７では、入力装置１４からの表情入力は皺記述手段１６０と特徴点移動手段１８０の双方に伝達され、皺描画のための位置情報は特徴点移動手段１８０の結果が入力される。
【００７２】
これに対して、図８では、表情入力は、皺記述手段１６０に入力され、必要な曲線記述が作成されるとともに、節点移動手段２００と特徴点移動手段１９０は、表情入力に応じて、移動ルールを参照して移動処理を実行する。その後、皺描画手段１７０は、移動後の特徴点情報を参照して、皺の描画を実行する。
【００７３】
実施の形態３．
以上の例では、表１に示すおおまかな皺の配置位置を設定していたが、個人の顔部品の配置や表情の程度などによっては、大まかな設定では不自然な印象を受ける似顔絵になることがある。こうした場合、顔部品位置をもとに皺の位置や大きさを決めることで、自然な似顔絵を作成することができる。
【００７４】
以下では、「怒り」「喜び」「驚き」を例にとり、その設定方法の一例を説明する。
【００７５】
図９の（ａ）は「怒り」の表情における皺の配置例を説明するための図である。ここでは表１の記述例２の皺を用いた。図において、眉の最も高い点と低い点の間の高さをＨ１とし、左右の眉で高さが異なる場合にはこれを平均するが、場合によっては、小さい方または大きい方の値を採用してもよい。両眉の間の距離をＬ１とする。
【００７６】
左側の皺はａ１、ａ２、ａ３の節点、右側の皺はｂ１、ｂ２、ｂ３の各節点から記述される。皺の最下端は、両眉の最下端に揃える。このとき、左右個別に眉と皺の最下点をあわせてもよいし、左右眉の下端の平均、つまり中点にａ１，ｂ１の高さをそろえてもよい。皺の高さはＨ１の０．８倍とした。左右の位置は、ａ１、ａ３が左眉の右端を基準に、右へ（０．３ｘＬ１）の位置、ａ２が同じ位置を基準に（０．３５ｘＬ１）の位置とする。右の皺についても同様にｂ１、ｂ３を右眉の左端を基準に左へ（０．３ｘＬ１）の位置、ｂ２を同位置を基準に（０．３５ｘＬ１）左の位置とする。
【００７７】
このように配置することにより、眉の位置との整合性のとれた皺の配置が可能となり、自然な似顔絵が作成できる。
【００７８】
図９の（ｂ）は、「喜び」の表情における皺の配置例を説明するための図である。図において、向かって左目の内側の目じりと唇の左端、向かって右目の内側の目じりと唇の右端を各々結んだ直線の高さをＨ２とする。左右の直線の高さが違う場合は高さの平均とした。また、鼻下の点から唇左端までの水平距離をＬ２ａ、鼻下の点から唇右端までの水平距離をＬ２ｂとする。
【００７９】
頬のしわは、内側から左側がａ４、ａ５、ａ６の各節点で、右側がｂ４、ｂ５、ｂ６の各節点で記述される。内側の節点の高さは、ここでは鼻下の高さとした。例えば唇の端から０．３ｘＨ２の高さとしてもよい。節点ａ４の左右位置は、向かって左目の内側の目じりと唇の左端を結ぶ直線との交点、節点ｂ４も右側で同様に右目の内側の目じりと唇の右端を結ぶ直線との交点とする。節点ａ６の左右位置は、鼻下の点を基準にＬ２ａの１．６倍の長さ左へ進んだ位置、同様に節点ｂ６の位置は水平距離Ｌ２ｂの１．６倍の長さを右へ進んだ位置とし、節点ａ５、ｂ５の左右位置は、それぞれ、節点ａ４，ａ６及びｂ４，ｂ６の中点とした。節点ａ６，ｂ６の高さ位置は、唇の端点の高さを基準に上に０．２５ｘＨ２の位置、節点ａ５，ｂ５の位置は同様に唇短点の高さから上に０．２３ｘＨ２の位置とした。
【００８０】
図９の（ｃ）は、「喜び」の表情における皺の配置例を説明するための図である。図において、両目の目頭の高さ方向位置から眉の内側の端点までの高さをＨ３とする。左右で目頭から眉の端点までの高さが異なるときは、その平均をとる。高さが大きく異なる場合は左右別々に設定してもよい。目頭間の距離をＬ３とする。
【００８１】
左側の皺は下から上にａ７、ａ８、ａ９の各節点で、右側の皺は下から上へｂ７、ｂ８、ｂ９の各節点で記述される。皺の最下端ａ７，ｂ７の高さ位置は、目頭の高さ位置とする。節点ａ７からａ９までの高さ、節点ｂ７からｂ９までの高さは、０．７５ｘＨ３とし、左右別々に高さを設定する場合は、目頭から眉までの高さの０．７５倍とする。節点ａ８、ｂ８の高さ位置は、それぞれ節点ａ７、ａ９の中間及び節点ｂ７、ｂ９の中間である。皺の左右位置は、まず節点ａ７、ａ８の横位置を、左の目頭を基準として右へ０．２２ｘＬ３の位置、節点ａ９の横位置を同じく左目頭を基準として、右へ０．０７ｘＬ３の位置とする。節点ｂ７、ｂ８についても、同様に右目頭から左へ０．２２ｘＬ３、ｂ９の位置を右目頭から左へ０．０７ｘＬ３の位置とする。
【００８２】
「悲しみ」の場合は、「怒り」と同様の手法を用いることにより、皺位置を決定することができる。
【００８３】
このように構成することにより、自動で、違和感の無い皺位置の設定が可能となる。
【００８４】
なお、本例における節点位置は、ユーザーの好みに応じて微調整できるように、ユーザーのキー入力等により、高さＨや距離Ｌの係数を調節できるように構成されている。
【００８５】
実施の形態４．
これまで、表情の認識または、ユーザーによる表情種別の入力結果によって皺の有無と描画位置を定める方法について説明してきたが、特徴点の位置情報から、皺の有無とその位置を決めることも可能である。以下、「怒り」の場合を例にとり、その方法について説明する。
【００８６】
図１０は実施の形態４を説明するための図である。眉の内側の端点はａ１０とｂ１０であり、予め、皺が発生する位置を定めておく。本例では、左のしわの節点ａ１１、ａ１２の水平位置をａ１０から右に０．３５ｘＬ１、ａ１１の高さ位置を節点ａ１０と同じ高さとした。右のしわの節点ｂ１１、ｂ１２についても、節点ａ１１、ａ１２と節点ｂ１１、ｂ１２が顔の中心線に対し対称になるように、同じ手法で皺が発生する位置を定めた。
【００８７】
ここで、入力された顔画像に対し、特徴点の動きを調べたところ、ある「怒り」の表情では、「無表情」に比べて節点ａ１０、ｂ１０間の距離が１５％程度減少した。個人差が大きいが、この例では上記距離の減少率が５％程度から眉間のしわの発生がみられており、閾値を１０％に設定することで、皺の発生の有無を決定することが出来た。また、節点ａ１０、ｂ１０距離が１０％のときに皺の高さ、すなわち節点ａ１１とａ１２及び節点ｂ１１とｂ１２間の距離Ｈ１１を０とし、距離１５％の時にＨ１１が０．８ｘＨ１となるように、連続的に変化させることにより、特徴点の動きに応じて皺の大きさを変化させることができた。
【００８８】
本例では、上記のように皺の発生位置を定めたが、発生位置は、実際に顔画像上でしわの出る位置を多人数の顔画像で求めておき、設定してもよい。また、ここでは皺を直線としたが、これに限定されるものではなく、例えば図９の（ｂ）に示した３個の節点ａ１、ａ２、ａ３やｂ１、ｂ２、ｂ３の場合であれば、節点ａ２、ｂ２を皺の発生場所として、節点ａ２とａ１、節点ａ２とａ３及び、節点ｂ２とｂ１、節点ｂ２とｂ３の距離を特徴点の動きに合わせて変化させるようにしてもよい。
【００８９】
また、時間軸上で、例えば赤松茂：「表情コミュニケーションにおけるダイナミクスを探る」画像ラボ，ｖｏｌ２，ｐ．１４（２０００．１２）．中に記載されているような、動的特性を参照し、予め、表情に応じた特徴点移動を定義しておき、それに応じた移動を行うことにより、連続した皺の描画が可能になり、動画においても、より自然な似顔絵の作成が可能になる。
【００９０】
また、上記の例では、高さＨ１１を距離Ｌ１に応じて変化させたが、距離Ｌ１の変化を皺の有無の判定にのみ用いて、皺の大きさすなわち高さＨ１１を、ユーザーの入力する表情のつよさによって変化させてもよい。たとえば、「すこし怒っている」に対応する高さが０．３ｘＨ１、「すごく怒っている」のときに０．８ｘＨ１などである。
【００９１】
さらに、上記の例では、特徴点間の距離の変化が１０％以下の場合は「皺なし」としたが、年齢に応じて、最初の状態すなわち無表情の状態でも皺を小さく描画しておき、表情が現れるにしたがい、大きくしていくことも可能である。
【００９２】
たとえば図９の（ｂ）に示されるような皺であれば、ユーザーに年齢を入力させ、４０歳以下ならば、初期のしわの大きさを０、４０歳以上であれば、初期の皺の大きさを皺の大きさの最大値の例えば２０％と設定し、これ以降は上記の例と同様に例えば高さＨ２に応じて皺の大きさを変えるようにしてもよい。
【００９３】
また、上記の例では、直線的に特徴点間の距離と皺の大きさを対応させたが、変化が大きくなるにつれて、皺の大きさの変化率を増やすようにしてもよい。
【００９４】
またさらに、皺の大きさを上記の例では長さで表現したが、長さの代わりに太さや濃さで表現することも可能である。
【００９５】
【発明の効果】
以上のように、この発明によれば、特定の感情に対応した皺を記述し、作成する似顔絵に皺を描画するようにしたので、似顔絵に特定の感情を示す表情が現れ、似顔絵を見たときの感情の識別が容易になる。
【図面の簡単な説明】
【図１】本発明に係る似顔絵の作成装置の好ましい実施の形態１を示す図である。
【図２】上記実施の形態１の機能ブロックを示す図である。
【図３】上記実施の形態１における似顔絵の作成処理のフローチャートを示す図である。
【図４】上記実施の形態１における髪形の画像（ａ）、各顔部品の記述（ｂ）、無表情における似顔絵作成画像（ｃ）を示す図である。
【図５】上記実施の形態１における「怒り」に対応した似顔絵の第一の作成例（ａ）、「怒り」に対応した似顔絵の第二の作成例（ｂ）、「喜び」に対応した似顔絵作成画像（ｃ）、「驚き」に対応した似顔絵作成画像（ｄ）、「悲しみ」に対応した似顔絵作成画像（ｅ）、「悲しみ」に対応した第二の似顔絵作成画像（ｆ）を示す図である。
【図６】実施の形態２における似顔絵の作成処理のフローチャートを示す図である。
【図７】上記実施の形態２の機能ブロックを示す図である。
【図８】上記実施の形態２で予め作成されたルールに基づき顔部品や皺の記述手段結果に対し処理を行う場合の機能ブロックを示す図である。
【図９】実施の形態３の「怒り」の表情における皺の配置（ａ），「怒り」の表情における皺の配置（ｂ），「驚き」の表情における皺の配置（ｃ）を説明するための図である。
【図１０】実施の形態４における皺の記述を説明するための図である。
【符号の説明】
１０　ＣＰＵ（皺記述手段、皺描画手段）、　１１　ＲＡＭ、　１２　ＲＯＭ、　１３　ディスプレイ、　１４　入力装置、　１５　通信装置、　１００　特徴点抽出手段、　１１０　顔部品記述手段、　１２０　顔部品描画手段、　１３０　髪型抽出手段、　１４０　髪型描画手段、　１５０　特徴点抽出手段、　１６０　皺記述手段、　１７０　皺描画手段、　１８０　特徴点移動手段、　１９０　特徴点移動手段　２００　節点移動手段。[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an apparatus for creating a portrait, for example, an apparatus suitable for transmitting emotions using a portrait on a mobile phone or the like.
[0002]
[Prior art]
A method of creating a portrait by combining a previously registered pattern of face parts, that is, an image image, when creating a portrait is disclosed (see Patent Document 1). However, in such a method, an enormous number of parts need to be stored in a memory in order to generally describe the characteristics of an individual's face, for example.
[0003]
On the other hand, the eyebrow and eyes are extracted as 4 to 10 feature points, and the nose and mouth are extracted as 2 to 3 feature points, and the face model is adjusted to the basic model of the face by adjusting the area and position of the face parts After transforming a face part, a face part is drawn by using a drawing rule using a feature point, and a portrait is generated with the outline of the hair and face as it is (Patent Document 2). ,reference).
[0004]
[Patent Document 1] JP-A-07-239945
[Patent Document 2] JP-A-2000-67218
[0005]
[Problems to be solved by the invention]
However, in the method described in Patent Literature 2, since the movement of the facial expression and the movement of the facial part do not correspond one-to-one, it is difficult to synthesize the facial expression based on the facial image captured by the camera.
[0006]
In addition, when creating a portrait from a facial image having a facial expression or performing facial expression by moving a feature point of a facial part to a facial expression without expression, only a facial part based on information of the characteristic point is used. Since the facial expression is displayed, there is a problem that the difference from the non-expression is difficult to understand.
[0007]
Furthermore, even when it is recognized that a facial expression is appearing, it is difficult to identify the facial expression, and a facial expression different from the intended facial expression may be recognized. In particular, in the facial expression of a portrait synthesized based on a face image of an unknown person, there is a problem that the facial expression that is different from the original intention of the person is likely to be mistaken.
[0008]
On the other hand, in the method described in Patent Document 1, wrinkle parts are also prepared, but as a built-in process, wrinkles are added according to gender and age. Since it is left to the choice of the user, the user has to generate a portrait while checking the effect of wrinkles by trial and error.
[0009]
SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problems, and in a portrait generating method using a small number of feature points, a simple operation allows a richer and easier-to-understand expression expression. It is an issue.
[0010]
[Means for Solving the Problems]
Therefore, a portrait creating apparatus according to the present invention is a portrait creating apparatus that creates a portrait from feature points of each face part such as eyebrows, eyes, nose, mouth, ears, and face contours and a hairstyle image. Wrinkle description means for describing wrinkles corresponding to the specific emotion when drawing an expression representing the emotion of the above, and wrinkle drawing means for drawing the wrinkles described above in a portrait to be created. And
[0011]
One of the features of the present invention is that when a portrait to be created presents an expression representing a specific emotion, wrinkle information corresponding to the emotion is added. This makes it easy to identify the emotion when looking at the portrait.
[0012]
In the above, when the captured face image has a facial expression that expresses a specific emotion, wrinkles are drawn from that expression, but wrinkles are drawn based on non-expression, and a caricature that expresses emotion is created Even if it does, emotion identification becomes easy similarly to the above.
[0013]
That is, the portrait creating apparatus according to the present invention is a portrait creating apparatus that creates a portrait from feature points of each face part such as eyebrows, eyes, nose, mouth, ears, and facial contours and a hairstyle image. Wrinkle description means for describing wrinkles corresponding to the above specific emotions in a caricature created based on an expressionless face image when drawing an expression representing the emotion of the face, and creating the image based on the expressionless face image Wrinkle drawing means for drawing the wrinkles described above on the portrait.
[0014]
One of the features of the present invention is that the facial expression is represented by wrinkles corresponding to specific emotions, so that drawing by eye parts of eyebrows, eyes, nose, mouth, ears, and face contours is a feature. It is only necessary to be able to create a portrait from points and hairstyle images, and there is no particular limitation, but it is possible to create a portrait using a small number of feature points, and to make a one-to-one correspondence between the movement of facial expressions and the movement of facial parts. It is preferable to adopt the following configuration.
[0015]
That is, a caricature creation apparatus for creating a caricature from feature points of face parts such as eyebrows, eyes, nose, mouth, ears, and facial contours and a hairstyle image. The eyes are described by complementing at least 4 to 6 feature points containing at least the following points with a function curve, and the midpoints of both edges of the lips, the midpoints of the upper and lower edges of the lips, and the midpoints of the upper and lower edges of the upper lip A face part description which describes a mouth by complementing a function curve with at least 8 to 10 feature points including at least one point on one side set on the line connecting the middle point and the lip end point of the upper ridge of the upper lip Means and the upper eyelid are drawn in accordance with the above-described curve, and only a part of the curve connecting both eyes and the lower end of the pupil that is in contact with the lower end of the pupil is drawn, while the mouth is drawn in accordance with the above-described curve. Face parts drawing means to draw and hairstyle images And hairstyles drawing means demarcating, may be provided with. This makes it possible to draw a realistic expression.
[0016]
Further, it is preferable that the position where the wrinkle is provided is set based on the position of the face part. This prevents wrinkles from coming out of place and allows a portrait with a natural expression.
[0017]
In addition, instead of the position of the face part, whether to draw wrinkles and the size of wrinkles can be determined based on movement information of feature points at the time of expression generation. This enables more natural expression and more realistic expression not only in still images but also in moving images. As described above, according to the present invention, since means for automatically drawing wrinkles in association with facial expressions is provided, the user can easily create a portrait with expressive power.
[0018]
By installing the portrait creation program in a data processing device such as a mobile phone, the portrait creation device according to the present invention can be easily constructed. The portrait creation program may be recorded on a recording medium such as a memory chip and installed in a data processing device, or may be stored in a server and stored in the server. You may make it download to a processing apparatus.
[0019]
The caricature creation program according to the present invention is a program for causing a computer to create a caricature from feature points of face parts of eyebrows, eyes, nose, mouth, ears, and facial contours and a hairstyle image. The computer functions as wrinkle description means for writing wrinkles corresponding to the specific emotion when drawing an expression representing a specific emotion, and wrinkle drawing means for drawing the wrinkles described above in a portrait to be created It is characterized in that it is made to be.
[0020]
Further, the portrait creation program according to the present invention is a program for causing a computer to create a portrait from feature points of each face part such as eyebrows, eyes, nose, mouth, ears, and face contours and a hairstyle image. Wrinkle description means for writing wrinkles corresponding to the specific emotion in a portrait created based on a non-expression facial image when the computer draws an expression representing a specific emotion, and a facial expressionless face A wrinkle drawing means for drawing the wrinkles described above in a portrait created based on the image is pointed out as a brain.
[0021]
The portrait drawing program may further include a program for controlling the computer so that the wrinkle drawing unit sets the drawing position of the wrinkle based on the position of the feature point of the face part.
[0022]
In addition, the computer causes the computer to function as a feature point moving unit that moves a feature point of the face part at the time of generating an expression, and determines whether or not the wrinkle drawing unit draws wrinkles based on the movement information of the face part. Preferably, the apparatus further comprises a program for controlling the computer to determine the size.
[0023]
In addition, the portrait creating method according to the present invention includes a facial expression representing a specific emotion when creating a portrait from feature points of each face part such as eyebrows, eyes, nose, mouth, ears, and facial contours and a hairstyle image. When drawing, wrinkles corresponding to the specific emotion are described, and the wrinkles described above are drawn in a portrait to be created.
[0024]
Further, the portrait creation method according to the present invention includes a facial expression representing a specific emotion when creating a portrait from the feature points of the face parts such as eyebrows, eyes, nose, mouth, ears, and facial contours and the image of the hairstyle. When drawing, a wrinkle corresponding to the specific emotion is described in a portrait created based on a faceless expression image, and the wrinkle described above is drawn in a portrait created based on the expressionless face image. It is characterized by doing so.
[0025]
Also in these portrait creation methods, it is preferable to set the drawing position of the wrinkle based on the position of the feature point of the face part. Further, it is preferable to determine whether or not to draw wrinkles and determine the size of the wrinkles based on the movement information of the face parts while moving the feature points of the face parts when generating the facial expression.
[0026]
BEST MODE FOR CARRYING OUT THE INVENTION
Embodiment 1 FIG.
First, a method for creating a portrait with a small number of feature points used in the present invention will be described. 1 to 3 show a preferred embodiment of a portrait creating apparatus according to the present invention. This embodiment is an example applied to a mobile phone. FIG. 1 shows an example of an apparatus configuration of a portrait drawing apparatus of the present embodiment. In the drawing, the portrait creating apparatus basically comprises a CPU 10, a RAM 11, a ROM 12, a display 13, an input device 14 such as a numeric keypad, a jog dial, a touch panel, a microphone, and a communication device 15 such as a modem.
[0027]
The ROM 12 stores a portrait creation program.
[0028]
Upon receiving a user's instruction, the CPU 10 captures the face image, extracts the hairstyle image and the feature points of the face parts, and complements the feature points of the face parts with a function curve on the display 13 to describe the face parts. Then, a face part is drawn according to the described curve, and a hairstyle image is drawn to create a portrait.
[0029]
FIG. 2 is a functional block diagram of the caricature creation device of the present embodiment. The portrait creating apparatus is functionally characterized by feature point extraction means 100, face part description means 110, face part drawing means 120, hairstyle extraction means 130, hairstyle drawing means 140, facial expression recognition means 150, wrinkle description means 160, and wrinkles. It comprises drawing means 170.
[0030]
The feature point extracting means 100 specifies the face parts, specifically the eyebrows, eyes, nose, mouth, ears, and the shape of the face outline by a known extraction method, and extracts the feature points.
[0031]
The face part description means 110 describes the face part by complementing the extracted feature points with a function curve. Although any function curve can be used, a B-spline curve is used in this example. The description based on the feature points of each face part uses the method described above.
[0032]
The face part drawing means 120 draws the face part on the display 13 according to the described curve.
[0033]
Hairstyle extracting means 130 extracts a hairstyle image from a face image by a known method. There are various extraction methods, but in this example, the color image is converted using the RGB color system, the R image is binarized, then labeling is performed, and a portion having the widest area is extracted to extract the hairstyle. Image.
[0034]
The hairstyle drawing means 140 draws the image of the extracted hairstyle on the display 13.
[0035]
The facial expression recognition means 150 recognizes a facial expression from the input face image. Facial expression recognition is described in a known method, for example, in Shigeru Akamatsu: "Face Recognition by Computer-Survey", IEICE Transactions, vol., J80-ANo, 8, 1215 ('97). Can be used. In this case, it is possible to generate a portrait with a small number of feature points in order to generate a portrait. However, if more feature point information is required to improve facial expression recognition accuracy, this processing part Only more feature points may be used. In this example, four types of facial expressions of “anger”, “joy”, “surprise”, and “sadness” are recognized.
[0036]
The wrinkle description means 160 describes wrinkles according to the recognized expression. Wrinkle description is described using two or three points per wrinkle. In the case of two pieces, a straight line is used, and in the case of three pieces, it is complemented and described using a function curve, similarly to the face part. In this example, the description is made using a B-spline curve. If the length or undulation is large, such as a wrinkle, the score may be increased as necessary.
[0037]
The wrinkle drawing means 170 draws wrinkles according to the described curve.
[0038]
When the image of the prepared face part, hairstyle, and wrinkles is displayed on the display 13, a portrait is created.
[0039]
The portrait creation program is stored in the ROM or the like in advance, and when an execution instruction is given, the portrait creation program is expanded on the RAM and the processing described in the program is executed. Instead of the ROM, the data may be compressed and stored, for example, in a portable recording medium such as a memory chip, and inserted into a slot or the like of a mobile phone to give an instruction to expand the data in the RAM and execute the processing. . In the case of a personal computer (PC), the program can be stored on a hard disk.
[0040]
Further, a portrait creation program may be stored in a server, and a portable telephone may be connected to the server to download the program, developed on a RAM, and executed.
[0041]
Next, the operation will be described with reference to FIG. FIG. 3 shows a flowchart of the portrait creation process in this example. When creation of a portrait is selected on the mobile phone and an instruction is given with a numeric keypad or the like, first, a face image a is captured from a camera or the like (step S100), and hairstyle extraction processing is performed on this image (step S110). The obtained hairstyle image data is temporarily stored in a RAM or the like. FIG. 4A shows an example of the obtained hairstyle.
[0042]
When the hairstyle is extracted, the feature points of the facial parts are extracted from the face image (step S120), and the information of the obtained feature points is temporarily stored in a RAM or the like. Existing methods such as edge extraction, pattern matching, and active contour extraction can be used for feature point extraction.
[0043]
Next, facial expression recognition processing is performed using an existing method such as a neural network (step S130), and the result of classification into "anger", "joy", "surprise", "sadness", and "expressionless" is stored in a RAM or the like. To be stored temporarily.
[0044]
A predetermined number of feature points are used for each of the face parts thus obtained, and the feature points are complemented by a function curve to describe the shape of each face part (step S140). FIG. 4B shows an example of a face part in the described “expressionless”. The description of the face parts is made for each part of the outline, eyebrows, ears, nose, eyes including pupils, and mouth. When the description is completed, each face part is displayed on the display 13 and drawn according to the described curve (steps S160, S170).
[0045]
Referring to the previous expression recognition result, if the recognition result is “no expression”, the image of the hairstyle is drawn and the processing ends (step S180). FIG. 4C shows an example of the created portrait in this case.
[0046]
If the result is other than “no expression”, a description of the wrinkle according to the expression is created (step S190). Table 1 shows a wrinkle addition position for each expression used in this example and a description example thereof. Wrinkles are described using two or three points. This point is called a node to distinguish it from a feature point of the face. The nodes do not necessarily have to match the feature points of the face. Nodes at both ends are end points. Similarly to the description using the feature points, the space between the nodes is complemented by a function curve. In this example, a B-spline curve was used as a function.
[0047]
[Table 1]

[0048]
When the description is completed, wrinkles are drawn at the arrangement positions in Table 1 according to the described curve with reference to the expression recognition result (step S210). The thickness of the wrinkle line is arbitrary, but is generally equal to or less than the thickness of the line for drawing the face part. However, when wrinkles are particularly emphasized, a more exaggerated expression can be obtained by setting the wrinkle line thicker than the thickness of the line for drawing the face part. Also, here, the density of the line, that is, the density is constant, but the density of the entire line segment for drawing a wrinkle at a specific position or the density of a part of an individual line segment for drawing a wrinkle is changed. A more realistic expression can be obtained by adding light and shade to the line segment for drawing. When changing the density of the above part, a more natural expression is possible by giving a continuous density change, for example, by providing a gentle density gradient from top to bottom.
[0049]
The arrangement of the wrinkles is arranged according to the center symmetry of the face according to the left-right symmetry of the face based on the arrangement position in Table 1, but when the right-left symmetry is lost due to the expression, or the face is inclined. In this case, a more realistic portrait can be created by asymmetrically including the shape.
[0050]
FIG. 5 shows an example of creating a portrait for each facial expression created using the description example 1 of wrinkles described in Table 1. FIG. 5A illustrates a wrinkle corresponding to “anger”. When creating a caricature corresponding to "anger", the pupil diameter obtained in the description of the pupil is intentionally reduced, the center of the pupil is moved so that the pupil touches the lower eyelid, and the curve of the upper eyelid A blank is generated between the pupils, or a blank is generated by moving the upper eyelid end point in contact with the upper end of the pupil upward while keeping the pupil diameter as it is. As shown in (b), a more exaggerated expression can be obtained.
[0051]
An example of creating a portrait corresponding to "joy" is shown in FIG. 5C, and an example of creating a portrait corresponding to "surprise" is shown in FIG. 5D. In FIG. 5D, as in FIG. 5B, a gap is formed between the pupil and the upper eyelid, and exaggeration is performed. It is also effective to separate the upper lip and the lower lip at the end points of the lips and provide a gap between them.
[0052]
FIG. 5E is an example of creating a portrait for “sadness”. In the case of sadness, the same effect can be obtained by using the same wrinkle description as “anger” in Table 1.
[0053]
FIG. 5F shows an example of creating a portrait when the eyes are closed. When the eyes are closed, in FIG. 5 (f), a wrinkle of a waveform composed of four nodes is added to the upper side of the closed eyelid. In the left wrinkle above the eyes in the figure, the nodes are described so as to be recognized, but are not actually drawn. The wrinkles between the eyebrows are the same as in FIG.
[0054]
Thus, a sad face can be expressed even when the eyelids are closed. Also, the wrinkle addition position and its size can be changed by a user's key operation or the like after the display. In this case, a more realistic portrait can be obtained by changing the degree including the thickness and density of the lines representing wrinkles and facial parts according to personal preference.
[0055]
Using the portrait creation method described above, a portrait was created from a face image taken by a camera, and an experiment was performed to see the effect of facial expression identification. In the experiment, five types of facial expressions, "joy", "anger", "surprise", "sadness", and "expressionless" were evaluated for face images of a total of five men and women and caricatures generated from the face images. That is, for the face image and the portrait, the questionnaire system in which the facial expressions were selected from "joy", "anger", "surprise", "sadness" and "expressionless" was evaluated for 50 persons.
[0056]
[Table 2]

[0057]
(A) of Table 2 shows the evaluation result using the face image captured by the original camera. The table shows, in the horizontal direction, which facial expression was recognized by the evaluator when each of the five types of images including the facial expression displayed at the left end was presented. For example, when a face image representing "anger" is presented, it indicates that 88% of the persons recognized as "anger" and 12% of the persons recognized as "expressionless". As a result of this experiment, a high correct answer rate of 70% or more was obtained for all facial expressions in the presentation of a face image as a basis for a portrait.
[0058]
(B) of Table 2 shows the evaluation results when a portrait without wrinkles is used. In particular, the correct answer rate is low for "anger", "sadness", "surprise", and the like.
[0059]
(C) in Table 2 shows the result when wrinkles are added. As compared with (b) in Table 2, the recognition rate, that is, the correct answer rate was improved for each expression, and a remarkable effect was obtained in "anger" and "sadness".
[0060]
In this example, the description of three nodes is used as the description of the wrinkle. However, the minimum effect can be obtained even if a straight line using only two end points is used. In this example, the four expressions of "anger", "joy", "surprise", and "sadness" are recognized. However, the facial expression recognition means shown in FIG. It is also possible by learning etc. to recognize a different facial expression.
[0061]
Embodiment 2 FIG.
In the first embodiment, a method of recognizing the facial expression of a face image input by a camera or the like and creating a portrait is described. In the second embodiment, however, a previously obtained expressionless face image is used. Next, a method of creating a portrait having an emotional expression will be described.
[0062]
The expressionless face image is stored in the RAM 11 or the ROM 12 in FIG. If a portrait has been created for this face image, descriptive information on the characteristic points, facial parts, and curves such as wrinkles is recorded in the RAM 11 or ROM 12 and the information is stored in the next portrait. It can be reused for creation.
[0063]
Further, the description information of such images, feature points, and curves can be stored as external data such as a memory chip or a server.
[0064]
FIG. 6 is a flowchart of a portrait creation process according to the second embodiment. In the first embodiment, the facial expression is recognized from the facial image input from the camera or the like. However, in this example, the input facial image a is assumed to have no facial expression (step S100), and the facial expression recognition processing in FIG. Instead of (Step S130), an expression input process (Step S220) and a feature point moving process (Step S230) are performed.
[0065]
In the facial expression input process (step S220), the user inputs facial expressions such as "anger", "joy", "surprise", and "sadness" using the input device 14 of FIG. In the feature point moving process (step S230), based on this input result, an existing method such as, for example, D. Yang, et al .: "Basic Research on Recognition / Synthesis Method of Facial Expression in Human Interface", Human Interface Symposium '99 Transactions, p. 141 (1999) or the like, feature points for expression are moved.
[0066]
Thereafter, the face part description processing and drawing are performed (steps S140 to S160), the hair drawing (step S170), the wrinkle description processing and the drawing processing (steps S180 to S210) are performed in the same manner as in the first embodiment. It is possible to execute and create a portrait corresponding to the input facial expression.
[0067]
FIG. 7 shows a functional block diagram in this example. This example is different from the functional block diagram of the first embodiment in that a feature point moving unit 180 is provided instead of the facial expression recognition unit 150. When a facial expression is input, the feature point moving unit 180 moves the feature point extracted by the feature point extracting unit 100 and gives the feature point to the face part description unit 110 and the wrinkle drawing unit 170. The description of the face parts and the drawing of wrinkles are performed based on the feature points thus set.
[0068]
In this example, the feature points were moved after the feature points were drawn for each of the face images.However, curves of the description data of the face parts or wrinkles were previously obtained based on an example of creating a portrait from many face images. The amount of movement of each facial expression is made into a rule, and the result of the description means of the face parts and the result of the description means of the wrinkles created from the expressionless face image are moved based on this rule, thereby having the expression. You can also create portraits.
[0069]
FIG. 8 shows a functional block diagram in this case. This example is different from the functional block diagram of FIG. 7 in that the feature point moving means 190 moves between the face part description means 110 and the face part drawing means 120, and the node moves between the wrinkle description means 160 and the wrinkle drawing means 170. The point is that the means 200 is provided.
[0070]
When the movement amount of the curve is described for each facial expression with respect to the description data of the face part or the wrinkle, the rule is created in advance and stored in a ROM or the like. As is clear from FIGS. 7 and 8, the processing flow when using a rule created in advance is to move a feature point or a contact point before a face part or wrinkle description means, or to move a feature point or a node after the description means. The only difference is whether to move.
[0071]
Since the difference can be easily understood, a detailed description is omitted here. However, in FIG. 7, the facial expression input from the input device 14 is transmitted to both the wrinkle description means 160 and the feature point moving means 180, and the The result of the feature point moving means 180 is input as the position information for this.
[0072]
On the other hand, in FIG. 8, the facial expression input is input to the wrinkle description unit 160 to create a necessary curve description, and the node moving unit 200 and the feature point moving unit 190 move in accordance with the facial expression input. Execute the move process by referring to the rules. After that, the wrinkle drawing means 170 executes wrinkle drawing with reference to the feature point information after the movement.
[0073]
Embodiment 3 FIG.
In the above example, the rough wrinkle arrangement positions shown in Table 1 were set. However, depending on the arrangement of individual facial parts and the degree of facial expression, rough settings may result in a caricature that gives an unnatural impression. There is. In such a case, a natural portrait can be created by determining the position and size of the wrinkle based on the face part position.
[0074]
Hereinafter, an example of the setting method will be described with reference to “anger”, “joy”, and “surprise”.
[0075]
FIG. 9A is a diagram for explaining an example of arrangement of wrinkles in the expression of “anger”. Here, the wrinkles of description example 2 in Table 1 were used. In the figure, the height between the highest point and the low point of the eyebrows is defined as H1, and when the height differs between the left and right eyebrows, this is averaged, but depending on the case, the smaller or larger value is adopted. May be. The distance between both eyebrows is L1.
[0076]
The left wrinkle is described from nodes a1, a2, and a3, and the right wrinkle is described from nodes b1, b2, and b3. The lowermost ends of the wrinkles are aligned with the lowermost ends of both eyebrows. At this time, the lowermost points of the eyebrows and wrinkles may be individually set to the left and right, or the heights of a1 and b1 may be aligned with the average of the lower ends of the left and right eyebrows, that is, the middle point. The height of the wrinkles was set to 0.8 times H1. As for the left and right positions, a1 and a3 are positions to the right (0.3 × L1) based on the right end of the left eyebrow, and a2 is a position (0.35 × L1) based on the same position. Similarly, for the right wrinkle, b1 and b3 are set to the left (0.3 × L1) position with reference to the left end of the right eyebrow, and b2 is set to the left position (0.35 × L1) with reference to the same position.
[0077]
This arrangement makes it possible to arrange wrinkles with consistency with the position of the eyebrows, and to create a natural portrait.
[0078]
FIG. 9B is a diagram for explaining an example of the arrangement of wrinkles in the expression of “joy”. In the drawing, the height of a straight line connecting the inner edge of the left eye and the left end of the lip and the inner edge of the right eye and the right end of the lip are H2. When the height of the right and left straight lines was different, the average of the heights was used. The horizontal distance from the point under the nose to the left end of the lip is L2a, and the horizontal distance from the point under the nose to the right end of the lip is L2b.
[0079]
The wrinkles of the cheek are described by nodes a4, a5, and a6 on the left side from the inside and by nodes b4, b5, and b6 on the right side. Here, the height of the inner node was set to the height below the nose. For example, the height may be 0.3 × H2 from the end of the lips. The left and right positions of the node a4 are defined as the intersection between the inner edge of the left eye and the straight line connecting the left end of the lip, and the node b4 is also the right side of the intersection between the inner edge of the right eye and the straight line connecting the right end of the lip. The left and right position of the node a6 is a position that is 1.6 times the length of L2a to the left with respect to the point under the nose, and the position of the node b6 is 1.6 times the horizontal distance L2b to the right. The advanced position was set, and the left and right positions of the nodes a5 and b5 were set to the middle points of the nodes a4 and a6 and b4 and b6, respectively. The height positions of the nodes a6 and b6 are 0.25 × H2 above the lip end point, and the positions of the nodes a5 and b5 are also 0.23 × H2 above the height of the lip short point. And
[0080]
FIG. 9C is a diagram for explaining an example of arrangement of wrinkles in the expression of “joy”. In the drawing, the height from the height direction position of the inner corners of the eyes to the end point inside the eyebrows is H3. If the height from the inner corner of the eye to the end point of the eyebrow differs between the left and right, take the average. If the heights are significantly different, they may be set separately for the left and right. The distance between the inner corners is L3.
[0081]
The left wrinkle is described by nodes a7, a8, and a9 from bottom to top, and the right wrinkle is described by nodes b7, b8, and b9 from bottom to top. The height position of the lowermost ends a7 and b7 of the wrinkles is the height position of the inner corner of the eye. The height from the nodes a7 to a9 and the height from the nodes b7 to b9 are set to 0.75 × H3, and when the heights are set separately for the left and right sides, the height is set to 0.75 times the height from the inner corner of the eye to the eyebrows. The height positions of the nodes a8 and b8 are respectively located between the nodes a7 and a9 and between the nodes b7 and b9. The left and right positions of the wrinkles are, first, the horizontal positions of the nodes a7 and a8 to the right with respect to the left inner corner at a position of 0.22 × L3, and the horizontal positions of the nodes a9 to the right with the left inner corner as a reference at the position of 0.07 × L3. And Similarly, with respect to the nodes b7 and b8, the positions of 0.22 × L3 and b9 from the right eye to the left are 0.07 × L3 from the right eye to the left.
[0082]
In the case of "sadness", the wrinkle position can be determined by using the same method as in "anger".
[0083]
With this configuration, it is possible to automatically set a wrinkle position without a sense of discomfort.
[0084]
The node position in this example is configured so that the coefficient of the height H and the distance L can be adjusted by a user's key input or the like so that the node position can be finely adjusted according to the user's preference.
[0085]
Embodiment 4 FIG.
Until now, the method of determining the presence or absence of wrinkles and the drawing position based on the recognition of facial expressions or the input result of the facial expression type by the user has been described. However, it is also possible to determine the presence or absence of wrinkles and its position from the position information of the feature points. is there. Hereinafter, the method will be described taking the case of “anger” as an example.
[0086]
FIG. 10 is a diagram for explaining the fourth embodiment. The inner end points of the eyebrows are a10 and b10, and positions where wrinkles occur are determined in advance. In this example, the horizontal positions of the left wrinkle nodes a11 and a12 are 0.35 × L1 to the right from a10, and the height position of a11 is the same height as the node a10. With respect to the right wrinkle nodes b11 and b12, positions where wrinkles are generated are determined by the same method so that the nodes a11 and a12 and the nodes b11 and b12 are symmetric with respect to the center line of the face.
[0087]
Here, when the movement of the feature points was examined with respect to the input face image, the distance between the nodes a10 and b10 was reduced by about 15% in the expression of a certain "anger" compared to the "no expression". Although there is a large individual difference, in this example, wrinkles between the eyebrows are seen from the rate of decrease in the distance of about 5%. By setting the threshold value to 10%, it is possible to determine the presence or absence of wrinkles. done. Also, when the distance between the nodes a10 and b10 is 10%, the height of the wrinkles, that is, the distance H11 between the nodes a11 and a12 and the nodes b11 and b12 is 0, and when the distance is 15%, H11 is 0.8xH1. , The size of the wrinkles could be changed in accordance with the movement of the feature point.
[0088]
In this example, the wrinkle generation position is determined as described above, but the wrinkle generation position may be set by previously obtaining the wrinkle position on the face image using the face image of a large number of people. Although the wrinkles are straight lines in this case, the present invention is not limited to this. For example, in the case of three nodes a1, a2, a3 and b1, b2, b3 shown in FIG. The distances between the nodes a2 and a1, the nodes a2 and a3, the nodes b2 and b1, and the nodes b2 and b3 may be changed in accordance with the movement of the feature points, with the nodes a2 and b2 as wrinkle occurrence locations.
[0089]
On the time axis, for example, Shigeru Akamatsu: “Exploring Dynamics in Facial Expression Communication” Image Lab, vol2, p. 14 (2000.12). As described in the above, referring to the dynamic characteristics, in advance, defining the feature point movement according to the expression, by performing the movement according to it, it becomes possible to draw continuous wrinkles, Even in the case of moving images, a more natural portrait can be created.
[0090]
In the above example, the height H11 is changed in accordance with the distance L1, but the change in the distance L1 is used only for determining the presence or absence of wrinkles, and the size of the wrinkles, that is, the height H11 is input by the user. It may be changed depending on the expression. For example, the height corresponding to "slightly angry" is 0.3xH1, and the height corresponding to "very angry" is 0.8xH1.
[0091]
Furthermore, in the above example, when the change in the distance between the feature points is 10% or less, “no wrinkle” is set. However, depending on the age, wrinkles are drawn small even in the initial state, ie, the state of no expression. However, it is possible to increase the size as the expression appears.
[0092]
For example, in the case of wrinkles as shown in FIG. 9B, the user inputs the age. If the age is 40 or younger, the size of the initial wrinkles is 0; The size may be set to, for example, 20% of the maximum value of the wrinkle size, and thereafter, the size of the wrinkle may be changed according to, for example, the height H2 as in the above-described example.
[0093]
In the above example, the distance between the feature points and the size of the wrinkle are linearly associated with each other. However, as the change increases, the rate of change in the size of the wrinkle may be increased.
[0094]
Further, the size of the wrinkle is represented by the length in the above example, but may be represented by a thickness or a density instead of the length.
[0095]
【The invention's effect】
As described above, according to the present invention, wrinkles corresponding to specific emotions are described, and wrinkles are drawn on a portrait to be created. Therefore, a facial expression indicating a specific emotion appears in the portrait, and the portrait is viewed. It is easy to identify the emotion at the time.
[Brief description of the drawings]
FIG. 1 is a diagram showing a preferred embodiment 1 of a portrait creating apparatus according to the present invention.
FIG. 2 is a diagram showing functional blocks of the first embodiment.
FIG. 3 is a diagram showing a flowchart of a portrait creation process in the first embodiment.
FIG. 4 is a diagram showing a hairstyle image (a), a description of each face part (b), and a facial expression creation image with no expression in the first embodiment.
FIG. 5 corresponds to a first example of creation of a portrait corresponding to “anger” in the first embodiment (a), a second example of creation of a portrait corresponding to “anger” (b), and corresponds to “joy”. A portrait image (c), a portrait image (d) corresponding to "surprise", a portrait image (e) corresponding to "sadness", and a second portrait image (f) corresponding to "sadness" are shown. FIG.
FIG. 6 is a diagram showing a flowchart of a portrait creation process according to the second embodiment.
FIG. 7 is a diagram showing functional blocks of the second embodiment.
FIG. 8 is a diagram showing functional blocks in a case where processing is performed on a result of a description unit of a face part or a wrinkle based on a rule created in advance in the second embodiment.
FIG. 9 illustrates the arrangement of wrinkles in the expression of “anger” (a), the arrangement of wrinkles in the expression of “anger” (b), and the arrangement of wrinkles in the expression of “surprise” (c) in the third embodiment. FIG.
FIG. 10 is a diagram for describing the description of wrinkles in the fourth embodiment.
[Explanation of symbols]
10 CPU (wrinkle description means, wrinkle drawing means), 11 RAM, 12 ROM, 13 display, 14 input device, 15 communication device, 100 feature point extraction means, 110 face part description means, 120 face part drawing means, 130 hairstyle extraction Means, 140 hairstyle drawing means, 150 feature point extracting means, 160 wrinkle description means, 170 wrinkle drawing means, 180 feature point moving means, 190 feature point moving means 200 node moving means.

Claims (12)

A caricature creation device that creates a caricature from feature points of each face part of eyebrows, eyes, nose, mouth, ears, and facial contours and a hairstyle image,
Wrinkle description means for describing wrinkles corresponding to the specific emotion when drawing an expression representing a specific emotion;
Wrinkle drawing means for drawing the wrinkles described above on the portrait to be created,
An apparatus for creating a caricature, comprising: A caricature creation device that creates a caricature from feature points of each face part of eyebrows, eyes, nose, mouth, ears, and facial contours and a hairstyle image,
Wrinkle description means for describing wrinkles corresponding to the specific emotion in a portrait created based on a facial expression without expression when drawing an expression representing a specific emotion,
Wrinkle drawing means for drawing the wrinkles described above in a portrait created based on the expressionless face image,
An apparatus for creating a caricature, comprising: 3. The caricature creation device according to claim 1, wherein the wrinkle drawing means sets a wrinkle drawing position based on a position of a feature point of the face part. The image processing apparatus further includes a feature point moving unit that moves a feature point of the face part at the time of generating an expression, and the wrinkle drawing unit determines whether to draw a wrinkle based on the movement information of the face part and determines a size of the wrinkle. 4. The apparatus for creating a portrait according to claim 1, wherein: A program for causing a computer to create a portrait from feature points of each face part of eyebrows, eyes, nose, mouth, ears, and facial contours and a hairstyle image,
The above computer,
Wrinkle description means for describing wrinkles corresponding to the specific emotion when drawing an expression representing a specific emotion; and
Wrinkle drawing means for drawing the wrinkles described above on the portrait to be created,
A caricature creation program characterized by being made to function as. A program for causing a computer to create a portrait from feature points of each face part of eyebrows, eyes, nose, mouth, ears, and facial contours and a hairstyle image,
The above computer,
Wrinkle description means for describing wrinkles corresponding to the specific emotion in a caricature created based on an expressionless face image when drawing an expression representing a specific emotion, and
Wrinkle drawing means for drawing the wrinkles described above in a portrait created based on the expressionless face image,
A portrait creation program characterized by functioning as a portrait. 7. The caricature creation program according to claim 5, further comprising a program for controlling the computer so that the wrinkle drawing means sets a wrinkle drawing position based on the position of a feature point of the face part. The computer causes the computer to function as a feature point moving unit that moves a feature point of the face part when generating an expression, and determines whether or not the wrinkle drawing unit draws wrinkles based on the movement information of the face part. 8. The caricature creation program according to claim 5, further comprising a program for controlling the computer to determine a size. In creating a caricature from the feature points of each face part such as eyebrows, eyes, nose, mouth, ears and face contours and hairstyle images,
A facial portrait creation method, characterized in that wrinkles corresponding to the specific emotions are described when a facial expression representing a specific emotion is drawn, and the wrinkles described above are drawn in a portrait to be created. In creating a caricature from the feature points of each face part such as eyebrows, eyes, nose, mouth, ears and face contours and hairstyle images,
When drawing an expression that expresses a specific emotion, describe the wrinkle corresponding to the specific emotion in a portrait created based on the expressionless face image, and add the above A method for creating a portrait, wherein the described wrinkles are drawn. The method for creating a portrait according to claim 9 or 10, wherein the drawing position of the wrinkle is set based on the position of the feature point of the face part. 12. The method according to claim 9, wherein a feature point of the face part is moved when the facial expression is generated, and whether or not to draw a wrinkle and the size of the wrinkle are determined based on the movement information of the face part. How to create a caricature described in.

Images