JP4114966B2 - Outline extracting apparatus and outline extracting method - Google Patents

Description

で表されるように構成する。ここでＨは、グラデイエントベクトルＧ（ｑ）の方向より＋π／２ずれた方向を示す単位ベクトルであり、acos（（ｑ−ｐ）／｜ｑ−ｐ｜・Ｈ）は、輪郭の進む方向と、ｐからｑへ進む局所経路の方向のなす角度を表す。
【００５０】
ここで上述のようにグラデイエントベクトルＧ（ｑ）の方向は連続なので、輪郭に沿つて一方向に進む経路ほどコストが低く、逆に輪郭に沿つていても途中で逆方向に方向が変化する経路はコストが悪くなる。
従つて経路探査部２６は、局所関数ｇを用いることにより、常に輪郭に沿つて一方向に進むような経路探査を行うことができる。
【００５１】
（Ｂ）エツジピーク通過の評価関数
経路が輪郭の中央をなるべく通過するようにする評価関数は本発明に有効である。同様の機能をもつ評価関数が上述の「Intelligent Scissors for Image Composition」においては画像のラプラシアンを用いて実現されている。ラプラシアンはエツジの中心を検出する能力は高いが、ノイズに敏感であるので、平滑化フイルタと組み合わせて使用される。しかしながらどんなサイズの平滑化フイルタを用いるかは、得たい輪郭に合わせてその都度決定しなければならない。
【００５２】
この実施例における中継点連結部２３は、検出したい輪郭の大まかな位置と方向、すなわちエツジの位置とエツジの方向とを用いて、エツジ強度がピークとなるエツジ点を検出するエツジ点検出処理を行つており、これによりエツジ強度を用いたエツジ点検出でもエツジ中心の検出能力を向上させることができるので、評価関数の信頼性を向上させることができる。またラプラシアンのように平滑化フイルタを用いる必要がないので、その分中継点連結部２３の構成を簡略化することができる。
【００５３】
このような局所評価関数は、経路探査中の８近傍において、隣接する中継点間に存在する隣接する２画素の位置座標ｐ、ｑを入力とし、エツジ点２値画像ＥＰにおける座標ｐ、ｑでの画素値Ｅ（ｐ）、Ｅ（ｑ）と、座標ｐ、ｑとの関数ｅを、次式（２）
【数２】

で表されるように構成する。ここでＥ＝０又は１である。すなわちこの局所評価関数ｅは、エツジ強度がピークとなる位置を通過する画素を評価する関数である。従つて経路探査部２３は、局所関数ｅを用いることにより、エツジ強度がピークとなる位置を評価する経路探査を行うことができる。
【００５４】
（Ｃ）評価関数の組み合わせと８近傍距離の補正
この実施例では上述の評価関数ｇ及び評価関数ｅ（すなわちグラデイエントベクトルＧＳ及びエツジ２値画像ＥＰ）を以下のように組み合わせて用いる。
評価関数ｄは、評価関数ｇと評価関数ｅとを評価項とし、その係数付き線形和に、隣接する中継点間に存在する隣接する２画素の座標ｐ、ｑ間の距離（｜ｑ−ｐ｜）を乗ずることによつて、各評価項がそれぞれ８近傍距離による評価補正を受ける次式（３）
【数３】

となるｄで表されるように構成する。ここで係数ｃ１とｃ２は予め適当に定められた係数である。
【００５５】
またこの局所関数ｄを一般的に表したものを次式（４）
【数４】

で表す。この場合、ｆ_i はｉ番目の評価項を表し、ｃ_i は番目の係数を表す。
従つて経路探査部２３は、複数の評価項がそれぞれ８近傍距離による評価補正を受けることができる。
【００５６】
かくして輪郭座標リスト生成部２０は、連続値をもつ画像を２値化して対象物の輪郭を中継点でなる輪郭座標リストＰとして生成することにより輪郭の形状の次数を調べやすいようにし、各中継点を順次連結して輪郭形状を一本の経路として抽出することにより他の輪郭などの不要な情報を削除し得るようになされている。
【００５７】
（２−１−２）曲線生成部の構成
曲線生成部２１の構成を図１４に示す。曲線生成部２１は、曲線生成手段として、輪郭座標リスト生成部２０から供給される輪郭座標リストＰを、それぞれ入力座標分割回路２１Ａ、連結条件算出回路２１Ｂ及び曲線近似回路２１Ｃで受ける。入力座標分割回路２１Ａは、輪郭座標リストＰによつて表される形状を分割する分割点の座標のリスト（以下、これを分割点座標リストと呼ぶ）Ｆを生成し、これを連結条件算出回路２１Ｂ及び曲線近似回路２１Ｃに送出する。
【００５８】
すなわち入力座標分割回路２１Ａは、輪郭座標リストＰの離散的な並びから、求める形状の次数を局所的に推定する。この場合、輪郭座標リストＰは分割された各セグメントにおける形状の次数が大きくならないように分割する。例えば各分割点を曲率の大きさに基づいて決定することより、座標データ数に依存せずに各セグメントにおける次数を決定することができので、各セグメントにおける次数が大きくなることを防止することができる。
【００５９】
連結条件算出回路２１Ｂは、輪郭座標リストＰ及び分割点座標リストＦに基づいて、各分割点における連結条件、すなわちＧ１連続性を満たす連結条件を算出し、連結条件リストＲＶとして曲線近似回路２１Ｃに送出する。すなわち連結条件算出回路２１Ｂは、各分割点の連結位置を一致させるための連結位置の座標ｒ_i と、連結位置ｒ_i における速度方向を一致させるための速度ベクトルｖ_i との対（すなわち接線）を連結条件リストＲＶとして算出する。この場合、連結位置の座標と分割点の座標とは必ずしも一致しない。
【００６０】
曲線近似回路２１Ｃは、隣接する分割点で区切られる各セグメントを、両端点における連結条件を満たすように最小２乗法を用いて曲線近似することにより、隣接する分割点間における近似曲線ｃ_i を算出し、近似曲線ｃ_i でなる近似曲線群Ｃを輪郭曲線情報Ｃとしてキー算出部３に送出する。
【００６１】
曲線生成部２１における曲線生成処理について図１５に示すフローチヤートを用いて説明する。
曲線生成部２１は、ステツプＳＰ１より曲線生成処理を開始し、ステツプＳＰ２において、輪郭座標リスト生成部２０から輪郭座標リストＰを受け、ステツプＳＰ３において、輪郭座標リストＰを基に分割点座標リストＦを作成する。
【００６２】
続いて曲線生成部２１は、ステツプＳＰ４において、各分割点近傍における形状に近似した近似曲線を算出し、当該近似曲線における分割点近傍での接線ｒ_i 、ｖ_i を各セグメントを曲線近似するための連結条件ＲＶとして算出する。次いで曲線生成部２１は、ステツプＳＰ５において、隣接する分割点で区切られる各セグメントを、両端点における連結条件を満たすように最小２乗法を用いて曲線近似することにより近似曲線ｃ_i を算出し、ステツプＳＰ６において、近似曲線ｃ_i でなる近似曲線群を輪郭曲線情報Ｃとして出力し、ステツプＳＰ７において曲線生成処理を終了する。
【００６３】
（２−２）実施例の動作及び効果
以上の構成において、輪郭曲線生成部７は、図１６に示すステツプＳＰ１より輪郭曲線生成処理を開始し、ステツプＳＰ２において、輪郭候補領域決定部６から輪郭候補領域情報Ｉ_i を受けた後、ステツプＳＰ３において、輪郭候補領域情報Ｉ_i からエツジ強度が最大となるエツジ点を中継点として算出し、当該各中継点を順次連結することにより、対象物の輪郭として８近傍線図形を表す輪郭座標リストＰを生成する。
【００６４】
次いで輪郭曲線生成部７は、ステツプＳＰ４において、輪郭座標リストＰで表される形状を複数のセグメントに分割した後、各セグメントを連結する連結条件を算出して当該連結条件を満たすように各セグメントを曲線近似することにより近似曲線を生成し、ステツプＳＰ５において、当該近似曲線群を輪郭曲線情報Ｃとしてキー算出部３に送出し、ステツプＳＰ６において輪郭曲線生成処理を終了する。
【００６５】
従つてこの輪郭曲線生成部７は、連続値でなる画像から対象物の輪郭を、複数の離散的な座標データでなる輪郭座標リストＰとして生成しているので、対象物の輪郭の形状の次数を容易に調べることができる。
またこの輪郭曲線生成部７は、対象物の輪郭を一本の経路として抽出しているので、抽出したい輪郭以外の他の輪郭などの不要な情報を削除することができ、これにより対象物の輪郭を精度良く抽出することができる。
【００６６】
またこの輪郭曲線生成部７は、画像から対象物の輪郭を曲線として抽出する問題を、曲線近似で解きやすい問題にすることができるので、対象物の輪郭を曲線として容易に抽出することができる。
【００６７】
以上の構成によれば、輪郭候補領域情報Ｉ_i からエツジ強度が最大となるエツジ点を中継点として算出し、当該各中継点を順次連結することにより、対象物の輪郭として８近傍線図形を表す輪郭座標リストＰを生成した後、輪郭座標リストＰで表される形状を複数のセグメントに分割すると共に、各セグメントを連結する連結条件を算出し、当該連結条件を満たすように各セグメントを曲線近似して近似曲線を生成したことにより、対象物の輪郭の形状の次数を容易に調べることができると共に、抽出したい輪郭以外の他の輪郭などの不要な情報を削除することができる。
かくして画像中から対象物の輪郭の形状の次数に一致した輪郭曲線を精度良く抽出することのできる輪郭抽出部７及び輪郭抽出方法を実現することができる。
【００６８】
また上述の構成によれば、対象物の輪郭の特徴点を基準に輪郭座標リストＰによつて表される形状を分割して各セグメントの次数を決定すると共に、各近似曲線における分割点近傍での接線を連結条件として算出し、当該連結条件を維持するように各近似曲線を曲線近似したので、輪郭座標リストＰによつて表される形状の次数と一致し、かつ滑らかに連続する輪郭曲線を生成することができる。
【００６９】
さらに上述の構成によれば、輪郭上から複数の中継点ｑ_i を抽出し、各中継点ｑ_i を経路探査処理によつて連結することにより８近傍線図形を生成したので、曲線生成部２１における曲線近似処理の近似精度を向上させることができる。また輪郭上から複数の中継点ｑ_i を抽出したので、高速な経路探査処理を行うことができる。
【００７０】
さらに上述の構成によれば、隣接する中継点間の経路を、輪郭の各小部分における最短経路探査問題に帰着させることができるので、輪郭８近傍線図形２画像を生成することができ、これにより輪郭を一本の経路として抽出することができる。従つて対象物の輪郭を精度良く抽出することができる。
【００７１】
さらに上述の構成によれば、画像のグラデイエントと、エツジ強度の極値位置情報とを用いた局所評価関数を用いて経路探査処理を行うようにしたので、ノイズの影響を回避することができる。また隣接する２画素間の局所評価値を算出するための局所評価関数を用いたので、高速な最短経路探査アルゴリズムを適用することができる。
【００７２】
さらに上述の構成によれば、グラデイエントベクトルＧＳ及びエツジ点２値画像ＥＰに基づいて、エツジ点のうちエツジ強度が最大となるエツジ点を中継点ｑ_i として抽出したので、輪郭の探査処理を行う際にオペレータが探査に必要な点を与える必要がなく、その分輪郭抽出作業を簡略化することができ、また輪郭上の点を確実に抽出することができる。
【００７３】
さらに上述の構成によれば、局所評価関数ｄを用いているので、エツジ強度の勾配や、他の物体の輪郭に起因するエツジ強度の勾配の影響を回避することができるので、画像中から対象物の輪郭を輪郭曲線として精度良く抽出することができる。
【００７４】
さらに上述の構成によれば、色差の大小に依存しない規格化されたグラデイエントベクトルＧＳを用いてエツジ点を検出したので、輪郭全周に亘つて均等に中継点を検出することができ、これにより対象物の輪郭を精度良く抽出することができる。
【００７５】
（３）他の実施例
なお上述の実施例においては、中継点抽出手段として中継点抽出部２２を用いた場合について述べたが、本発明はこれに限らず、図５及び図８との対応部分に同一符号を付した図１７に示すような中継点抽出部３０を中継点抽出手段として用いてもよい。
【００７６】
中継点抽出部３０における中継点抽出処理について図１８に示すフローチヤートを用いて説明する。
中継点抽出部３０は、ステツプＳＰ１より中継点抽出処理を開始し、ステツプＳＰ２において、対応する線分ｌ_j 及びマスク画像Ｉ_mjの各対でなる輪郭候補領域情報Ｉ_i をグラデイエントベクトル算出部２４及びエツジ検出部２２Ｂで受ける。
【００７７】
次いでステツプＳＰ３において、中継点抽出部３０は、グラデイエントベクトル算出部２４において、規格化されたグラデイエントベクトルＧＳを算出した後、ステツプＳＰ４において、エツジ点２値画像Ｅをエツジ検出部２２Ｂにおいて生成し、グラデイエントベクトルＧＳ及びエツジ点２値画像Ｅを強エツジ点抽出部２２Ｃに送出する。この場合、グラデイエントベクトル算出部２４は、算出したグラデイエントベクトルＧを規格化せずに、当該グラデイエントベクトルＧを強エツジ点抽出部２２Ｃに送出するようにしてもよい。
【００７８】
続いてステツプＳＰ５において、中継点抽出部３０は、強エツジ点抽出部２２Ｃにおいて、グラデイエントベクトルＧＳ及びエツジ点２値画像Ｅに基づいて、中継点座標リストＱを生成した後、ステツプＳＰ６において、当該中継点座標リストＱを中継点連結部２３に送出し、ステツプＳＰ７において中継点抽出処理を終了する。ここで強エツジ点抽出部２２Ｃは、グラデイエントベクトルＧＳ及びエツジ点２値画像Ｅに基づいて、エツジ点のうちエツジ強度が最大となるエツジ点を中継点として算出し、当該中継点でなる中継点座標リストＱを生成するようにしてもよい。
【００７９】
この場合、入力画像における色差の大小に依存せずにエツジ強度を均一に検出することができるので、中継点を対象物の輪郭全周に亘つてほぼ均等に抽出することができ、これにより最終的に得られる輪郭曲線の精度を一段と向上させることができる。またこの場合、入力手段４から対象物の輪郭の大まかな形状を指定する曲線情報を入力したので、輪郭上の中継点の信頼性を向上させることができる。
【００８０】
また上述の実施例においては、中継点連結手段として中継点連結部２３を用いた場合について述べたが、本発明はこれに限らず、図５及び図８との対応部分に同一符号を付して示す図１９に示すような中継点連結部４０を用いてもよい。
この中継点連結部４０は、折れ線又は曲線でなる輪郭候補領域情報Ｉ_i をグラデイエントベクトル算出部４１及びエツジ検出部２２Ｂで受ける。
グラデイエントベクトル算出部４１は、一般に知られているグラデイエントベクトル算出方法を用いて、輪郭候補領域情報Ｉ_i に応じた輪郭候補領域におけるグラデイエントベクトルＧを算出し、当該グラデイエントベクトルＧを経路探査部２６に送出する。
【００８１】
さらに上述の実施例においては、中継点連結手段として中継点連結部２３を用いた場合について述べたが、本発明はこれに限らず、経路探査処理を行わずに上述の「画像合成のための対象物抽出方法」及び「領域抽出方法」に記載されている細線化処理を用いて中継点を連結するようにしてもよい。
【００８２】
さらに上述の実施例においては、経路評価方法として局所評価関数ｇ及び局所評価関数ｄを組み合わせた局所評価関数ｄを用いた場合について述べたが、本発明はこれに限らず、要は一意に決まる通過コストの大きさを表す評価関数であれば、経路評価方法としてこの他種々の評価関数を用いてもよい。この場合、この評価関数が最小となるように、隣接する中間点間の経路を決定する。また経路評価方法として画像のラプラシアンを用いてもよく、探査方法として「Intelligent Scissors for Image Composition」や「離散最適化法とアルゴリズム」を用いてよい。
【００８３】
さらに上述の実施例においては、局所評価関数ｇとして（１）式で構成される局所評価関数ｇを用いた場合について述べたが、本発明はこれに限らず、次式（５）
【数５】

に示すように、要は経路探査中の８近傍において、隣接する中継点間に存在する隣接する２画素の位置座標におけるグラデイエントベクトルＧ（ｐ）、Ｇ（ｑ）と、２画素の位置座標ｐ、ｑとの関数で表される評価項をもつ局所評価関数ｇを経路探査部２６にもたせるようにしてもよい。これによりエツジの進む方向を考慮した経路探査を行うことができる。
【００８４】
さらに上述の実施例においては、局所評価関数ｅとして（２）式で構成される局所評価関数ｅを用いた場合について述べたが、本発明はこれに限らず、次式（６）
【数６】

に示すように、要は、経路探査中の８近傍において、隣接する中継点間に存在する隣接する２画素の位置座標における画素値（エツジ点２値画像ＥＰにおける画素値）Ｅ（ｐ）、Ｅ（ｑ）と、２画素の位置座標ｐ、ｑとの関数で表される評価項をもつ局所評価関数ｅを経路探査部２６にもたせるようにしてもよい。これによりエツジのピーク位置を考慮した経路探査を実現することができる。
【００８５】
さらに上述の実施例においては、局所評価関数ｇ及び局所評価関数ｄを組み合わせた局所評価関数ｄを用いた場合について述べたが、本発明はこれに限らず、局所評価関数ｇ又は局所評価関数ｅを用いるようにしてもよく、要は状況に応じて局所評価関数を用いればよい。
【００８６】
さらに上述の実施例においては、オペレータが入力手段４を用いて線分、折れ線又は曲線を入力した場合について述べたが、本発明はこれに限らず、上述したように、現画像ｉと、既に輪郭曲線情報Ｃとして輪郭曲線が得られている１フレーム前の画像ｉ−１と、当該画像ｉ−１の輪郭曲線情報Ｃとに基づいて現画像ｉの推定輪郭情報Ｓ２を得るようにしてもよい（図１）。これにより、作業効率を格段的に向上させることができる。
【００８７】
さらに上述の実施例においては、推定輪郭情報Ｓ２に応じた曲線を各特徴点を頂点とする折れ線に近似し、折れ線の各線分ｌ_i をエツジ強度規格回路２４Ａに送出したた場合について述べたが、本発明はこれに限らず、推定輪郭情報Ｓ２に応じた曲線を各特徴点で分割し、各特徴点で分割した曲線でなるセグメントをエツジ強度規格化回路２４Ａに送出するようにしてもよい。
【００８８】
【発明の効果】
上述のように本発明によれば、対象物の輪郭が含まれる輪郭候補領域情報Ｉｉからエツジ強度を基に中継点ｑ _ｉ を抽出し、各中継点ｑ _ｉ を結ぶ経路を探査し順次連結することにより複数の離散的な座標データでなる線図形として生成し、線図形によつて表される形状を近似する曲線を生成することにより、輪郭の形状の次数を容易に調べることができると共に、輪郭形状を最終的に曲線で得ることができる。
かくして対象物の輪郭の形状に一致した輪郭曲線を得ることのできる輪郭抽出装置及び輪郭抽出方法を実現することができる。
【図面の簡単な説明】
【図１】本発明を適用したキー信号生成装置の全体構成を示すブロツク図である。
【図２】輪郭曲線生成部の構成を示すブロツク図である。
【図３】輪郭座標リスト生成部の構成を示すブロツク図である。
【図４】輪郭座標リスト生成処理の処理手順の説明に供するフローチヤートである。
【図５】中継点抽出部の構成を示すブロツク図である。
【図６】中継点抽出処理の処理手順の説明に供するフローチヤートである。
【図７】中継点の順序付け処理の説明に供する略線図である。
【図８】中継点連結部の構成を示すブロツク図である。
【図９】グラデイエントベクトル算出部及び輪郭候補領域決定部の構成を示すブロツク図である。
【図１０】エツジ検出部の構成を示すブロツク図である。
【図１１】経路探査処理の処理手順の説明に供するフローチヤートである。
【図１２】最短経路探査アルゴリズムを示す図表である。
【図１３】最短経路探査アルゴリズムを示す図表である。
【図１４】曲線生成部の構成を示すブロツク図である。
【図１５】曲線生成処理の処理手順の説明に供するフローチヤートである。
【図１６】輪郭曲線生成処理の処理手順の説明に供するフローチヤートである。
【図１７】他の実施例による中継点抽出部の構成を示すブロツク図である。
【図１８】他の実施例による中継点抽出部における中継点抽出処理の処理手順の説明に供するフローチヤートである。
【図１９】他の実施例による中継点連結部の構成を示すブロツク図である。
【符号の説明】
１……キー信号生成装置、２……輪郭抽出部、３……キー算出部、４……入力手段、５……推定輪郭算出部、６……輪郭候補領域決定部、６Ａ……特徴点抽出回路、６Ｂ……曲線分割回路、６Ｃ……領域分割回路、７……輪郭曲線生成部、８……動きベクトル推定部、９……遅延回路、１０……両端曲線算出部、１１……ソフトキー生成部、２０……輪郭座標リスト生成部、２１……曲線生成部、２１Ａ……入力座標分割回路、２１Ｂ……連結条件算出回路、２１Ｃ……曲線近似回路、２２、３０……中継点抽出部、２２Ａ……エツジ強度算出部、２２Ｂ、２５……エツジ検出部、２２Ｃ……強エツジ点抽出部、２３、４０……中継点連結部、２４、４１……グラデイエントベクトル算出部、２４Ａ……エツジ強度規格化回路、２４Ｂ……画像合成部、２５Ａ……しきい値算出回路、２５Ｂ……エツジ点検出回路。２６……経路探査部。[0001]
【table of contents】
The present invention will be described in the following order.
TECHNICAL FIELD OF THE INVENTION
Conventional technology
Problems to be solved by the invention
Means for solving the problem
BEST MODE FOR CARRYING OUT THE INVENTION
(1) Overall configuration (Fig. 1)
(2) Example (FIGS. 2 to 18)
(2-1) Basic configuration of contour curve generator (FIG. 2)
(2-1-1) Configuration of Outline Coordinate List Generation Unit (FIGS. 3 and 4)
(2-1-1-1) Configuration of relay point extraction unit (FIGS. 5 to 7)
(2-1-1-2) Configuration of relay point connecting portion (FIGS. 8 to 13)
(2-1-2) Configuration of Curve Generation Unit (FIGS. 14 and 15)
(2-2) Operation and effect of the embodiment (FIG. 16)
(3) Other embodiments (FIGS. 17 and 18)
The invention's effect
[0002]
BACKGROUND OF THE INVENTION
The present invention relates to a contour extraction apparatus and a contour extraction method, and is particularly suitable for application to a contour extraction apparatus and a contour extraction method for extracting a contour of an object from an image in special effect processing in video production such as television and movies. Is.
[0003]
[Prior art]
Conventionally, in this type of special effect processing, accuracy in extracting an arbitrary partial image from an image is important. In this extraction work, processing such as edge detection, area detection, and association with a known object is necessary. Especially when the background and the object in the image are complicated, the edges constituting the contour of the object are detected. Must be detected and tracked with high accuracy.
[0004]
Here, as a first contour extraction method for extracting the contour of an object from an image, the document “Object Extraction Method for Image Synthesis” (Seiki Inoue, IEICE Transactions, vol.J74-D-II). No. 10, pp 1411-1418, 1991) and “region extraction method” (Japanese Patent Laid-Open No. 3-17680). In this first contour extraction method, an approximate contour of an object is given as information by tracing the contour of the object with a thick line, and the edge intensity (degree of change in pixel value) of this thick line region is indicated. An outline figure binary image in which large pixels are left is obtained.
[0005]
As a second contour extraction method for extracting the contour of an object from an image, the document “Intelligent Scissors for Image Composition” (Eric N. Mortensen and William A. Barrett, Computer Graphics Proceedings, Annual Conference Series, 1995, ACM SIGGRAPH Pp.191-198).
[0006]
In this second contour extraction method, when two arbitrary points of the contour are designated interactively, a contour line figure binary image between the two points is calculated. By repeating this operation, the object is extracted. Get the contour. The contour between two given points is calculated by solving the shortest path search problem by setting a route evaluation function using the strength of the edge and the direction of the edge. According to the document "Discrete Optimization Method and Algorithm" (Toshihide Ibaraki, Iwanami Shoten, 1993), an algorithm that solves the shortest path exploration problem at high speed has been proposed, which is also used in this contour extraction method. .
[0007]
Furthermore, as a third contour extraction method for extracting the contour of an object from an image, “SNAKES: Active Contour models” (Kass M., Witikin A., Terzopoulos D., Proc. 1st ICCV, pp259-268, 1987) There is. This active contour model called “snakes” is a chain of control points that move so as to converge to a contour in an image, depending on constraint conditions. Examples of applications of this active contour model include "Automatic extraction of complex objects based on region segmentation" (Eizo Satoshi, Shirai Yoshiaki, NICOGRAPH '92 papers, pp. 8-17, 1992), "Region extraction device" ( Japanese Patent Laid-Open No. 5-61977), “Contour Tracking Method Based on Elastic Contour Model and Energy Minimization Principle” (Nobue Ueda, Kenji Mase, Yasuhito Suenaga, Journal of Science, vol.J-75-D-II, No. 1, pp 111-120, 1992), and “contour tracking method of moving object” (Japanese Patent Laid-Open No. 5-12443).
[0008]
The contour extraction method using an active contour model called “snakes” is as follows. First, the active contour model is obtained by connecting a plurality of control points using a spline curve or the like, and is not always a line figure binary image. The feature is that a smooth contour can be obtained. Second, the convergence of the dynamic contour model is performed by a method called local exploration in which the update of the evaluation function of the model is iteratively processed while moving the control points little by little. It has the feature of finding a solution.
[0009]
[Problems to be solved by the invention]
By the way, in the method of extracting the contour of the object in the image, when the extracted contour is finally obtained as a curve, various feature amounts in the contour can be easily calculated, and the shape has subpixel accuracy. As a result, anti-aliasing or the like becomes possible, which is advantageous for image composition processing or the like, and subsequent image processing becomes easy. Therefore, it is desirable that the contour shape of the object is obtained as a curve.
However, in the first and second contour extraction methods described above, the contour of the object is calculated as a line figure binary image, and the contour shape is not finally obtained with a curve. Therefore, the extracted contour is finally obtained with a curve. There is a problem that the above-mentioned advantages obtained in the case of the above cannot be obtained.
[0010]
In addition, the second contour extraction method can suppress an increase in processing time such as sort processing for solving the problem as compared with the first contour extraction method, but the operator gives various points on the contour. As a result, there is a problem that the contour extraction work becomes complicated. In the second contour extraction method, the second derivative of the image is used for the path evaluation function used to solve the shortest path search problem, and the term of the second derivative of the image is the most important term in the evaluation function. However, the second derivative of the image is vulnerable to noise, and the contour curve cannot be accurately extracted.
[0011]
On the other hand, the third and fourth contour extraction methods using the dynamic contour model can obtain the contour as a spline curve. However, the number of control points is determined in advance based on the principle of the active contour model, and the degree of freedom of the shape of the spline curve is limited by the number of control points. The number of control points must be set so that the order of the curve does not decrease. However, it is difficult to directly examine the order of the contour shape from the image without performing contour extraction, and a contour curve that matches the order of the contour shape cannot be obtained.
[0012]
In the case of the third and fourth contour extraction methods, the dynamic contour model is driven by an evaluation function using the edge strength gradient as an index. Therefore, noise of the gradient of the edge strength and other objects that are difficult to distinguish from the desired contour are obtained. It is easy to be influenced by the gradient of the edge intensity caused by the contour, and the contour curve cannot be extracted with high accuracy.
Further, since the third and fourth contour extraction methods are algorithms based on local search, there is a possibility that an optimal solution cannot be obtained depending on the initial position of the active contour model.
[0013]
The present invention has been made in consideration of the above points, and intends to propose a contour extraction apparatus and a contour extraction method capable of obtaining a contour curve that matches the order of the contour shape of an object from an image. is there.
[0014]
[Means for Solving the Problems]
  In order to solve such a problem, in the present invention, among a plurality of edge points detected in a contour candidate region where the contour of an object is considered to exist, an edge point whose edge intensity is greater than a predetermined threshold is used as a relay point. Extract andThe relay points are ordered along the contours, and the shortest path between relay points in order is determined so that the evaluation function representing the sum of the passing costs set for each local path is minimized. do itA route connecting each relay point is searched, a line figure representing an outline is generated by sequentially connecting the routes, and a curve approximating the shape represented by the line figure is generated.
  By using route search, each route connecting relay points can be searched with high accuracy, and the line figure connecting the searched routes in order is approximated by a curve, so that the contour shape can finally be obtained as a curve. .
[0015]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.
[0016]
(1) Overall configuration
In FIG. 1, reference numeral 1 denotes a key signal generating apparatus to which the present invention is applied as a whole, and generates soft keys for objects existing in a plurality of consecutive images.
The key signal generation device 1 includes a contour extraction unit 2 and a key calculation unit 3, and based on the contour curve of the object calculated by the contour extraction unit 2 and the gradient vector of the contour of the object, The calculation unit 3 generates a soft key for the object.
[0017]
The key signal generating device 1 is a mask that is input by an operator using, for example, the mouse 4 as input means, and shows a rough shape of a contour, line information, curve information, or a mask drawn with a thick brush. The image information S1 is received by the estimated contour calculation unit 5 of the contour extraction unit 2. The estimated contour calculation unit 5 obtains estimated contour information S2 corresponding to the estimated shape of the contour of the object (hereinafter referred to as the estimated contour) based on the line segment information, the broken line information, the curve information, or the mask image information S1. The estimated contour information S2 is sent to the contour candidate region determination unit 6.
[0018]
The contour candidate region determination unit 6 calculates a region where a contour is considered to exist (hereinafter referred to as a contour candidate region) based on the estimated contour information S2, and contour candidate region information I corresponding to the contour candidate region._iIs sent to the contour curve generator 7.
The contour curve generator 7 generates contour candidate area information I_iAfter calculating the gradient vector in the contour candidate region based on the above and normalizing the gradient vector, and generating a list P of pixels passing through the center of the contour (hereinafter referred to as a contour coordinate list) P, A curve that approximates the shape represented by the contour coordinate list P is generated, and the contour curve information C and the standardized gradient vector GS corresponding to the curve are sent to the key calculation unit 3.
[0019]
Here, in this key signal generation device 1, the contour curve information S4 of the image i-1 which is the previous frame is obtained separately from the method of obtaining the estimated contour information S2 from the polygonal line information, the curve information or the mask image information S1. The estimated contour information S2 in the current image i can be obtained by estimating where the contour of the image i-1 has moved in the current image i by using the motion vector estimation process.
[0020]
That is, the motion vector estimation unit 8 responds to the contour curve information C using the contour curve information C obtained via the contour candidate region determination unit 6 and the contour curve generation unit 7 for the image i-1 one frame before. It is estimated where the contour curve has moved in the current image i, and the result is sent to the contour candidate region determination unit 6 as estimated contour information S2. In this case, the processing in the contour candidate region determination unit 6 and the contour curve generation unit 7 is performed by the same method as described above.
Therefore, when the motion vector estimation unit 8 is selected as a method for obtaining the estimated contour information S2, the contour curve generation unit 7 feeds back the contour curve information C to the motion vector estimation unit 8 via the delay circuit 9. ing.
[0021]
The key calculation unit 3 receives the gradient vector GS and the contour curve information C at the both-end curve calculation unit 10. Based on the gradient vector GS and the contour curve information C, the both-end curve calculation unit 10 calculates the width between two curves representing both ends of the contour (hereinafter referred to as “end-end curves”), Both-end curve information S3 corresponding to the width of the key is sent to the soft key generation unit 11. The soft key generation unit 11 generates a curved surface in which the height smoothly changes from “0” to “1” between the two end curves based on the both end curve information S3. A process for writing to the image as the value of is performed and a soft key is calculated and output.
[0022]
Thus, the key signal generating apparatus 1 can generate soft keys for objects existing in the image.
[0023]
(2) Examples
(2-1) Basic configuration of contour curve generator
The configuration of the contour curve generator 7 according to the embodiment of the present invention is shown in FIG. The contour curve generation unit 7 includes contour candidate region information I supplied from the contour candidate region determination unit 6._iIs received by the contour coordinate list generation unit 20.
[0024]
The contour coordinate list generation unit 20 serves as contour coordinate information generation means, as candidate contour region information I._iThe contour of the object is generated as a contour coordinate list P composed of a plurality of discrete coordinate data, and the contour coordinate list P is sent to the curve generation unit 21. The curve generation unit 21 generates a curve C that approximates the shape represented by the contour coordinate list P, and sends the curve C to the key calculation unit 3 as the contour curve information C.
Hereinafter, specific configurations of the contour coordinate list generation unit 20 and the curve generation unit 21 in this embodiment will be described in order.
[0025]
(2-1-1) Configuration of outline coordinate list generation unit
A specific configuration of the contour coordinate list generation unit 20 is shown in FIG. 3, and the contour coordinate list generation processing in the contour coordinate list generation unit 20 will be described using the flowchart shown in FIG.
The contour coordinate list generation unit 20 starts the contour coordinate list generation processing from step SP1, and in step SP2, the contour candidate area information I_i(X, y) is received by the relay point extracting unit 22 and the relay point connecting unit 23.
[0026]
Next, in step SP3, the contour coordinate list generation unit 20 uses the contour candidate area information I in the relay point extraction unit 22 as relay point extraction means._iAfter detecting the contour of the object from inside, a plurality of points are relayed from the detected contour q_iAre extracted along the contour as_iThe relay point coordinate list Q consisting of is sent to the relay point connecting unit 23. In the case of this embodiment, the relay point extraction unit 22 determines a point where the edge strength is clearly larger than the vicinity as will be described later._iExtract as
[0027]
Subsequently, in step SP4, the contour coordinate list generation unit 20 uses the contour candidate area information I in the relay point connection unit 23 as a relay point connection unit._iAnd each relay point q based on the relay point coordinate list Q_iAre sequentially connected to create an 8-neighbor line figure, and at step SP5, the contour coordinate list P representing the 8-neighbor line figure is sent to the curve generation unit 21, and at step SP6, the outline coordinate list generation process ends.
[0028]
(2-1-1-1) Configuration of relay point extraction unit
A specific configuration of the relay point extraction unit 22 is shown in FIG. 5, and relay point extraction processing in the relay point extraction unit 22 will be described using the flowchart shown in FIG.
The relay point extraction unit 22 starts the relay point extraction process from step SP1, and in step SP2, the contour candidate area information I_iIs received by the edge strength calculation unit 22A and the edge detection unit 22B.
[0029]
Next, at step SP3, the relay point extracting unit 22 uses the edge candidate area information I in the edge intensity calculating unit 22A as the edge intensity calculating means._iThe edge strength image G (x, y) corresponding to the edge strength is generated based on the edge strength region, and the edge strength image G is sent to the strong edge point extraction unit 22C. Here, the edge strength calculation unit 22A calculates the edge strength by a method described in, for example, “Image Analysis Handbook” (Mikio Takagi, Yoshihisa Shimoda, University of Tokyo Press, 1991).
[0030]
Subsequently, at step SP4, the relay point extraction unit 22 uses the edge candidate region information I at the edge detection unit 22B as the edge detection means._iThen, an edge point is detected to generate an edge point binary image E (x, y) for designating the position of the edge point, and the edge point binary image E is sent to the strong edge point extraction unit 22C. Here, the edge point binary image is an image in which the value of the pixel corresponding to the edge position is represented by “1” and the values of the other pixels are represented by “0”. The edge detection unit 22B detects edge points by the method described in, for example, the “image analysis handbook” described above.
[0031]
Next, at step SP5, the relay point extracting unit 22 determines that the edge strength of the edge points is based on the edge intensity image G and the edge point binary image E in the strong edge point extracting unit 22C as the strong edge point extracting means. A pixel at a position corresponding to an edge point greater than a predetermined threshold value T is represented as a relay point q._iAnd the extracted relay point q_iThe relay point coordinate list Q is generated by ordering along the contours. The relay point extracting unit 22 sends the relay point coordinate list Q to the relay point connecting unit 23 at step SP6, and ends the relay point extracting process at step SP7.
[0032]
Here, the relay point q in the strong edge point extraction unit 22C._iWill be described with reference to FIG.
Relay point q_iIs obtained at intervals not so large along the contour,_iWhile avoiding the path (polygonal line) that follows the loop (the object contour does not become a loop), a nearby relay point q_iCan be ordered along the contours. That is, the relay point q_iIf a route that makes one round at the shortest is selected from among the routes that follow, a loop does not occur.
[0033]
Here, as shown in FIG. 7A, in a route with a loop, an intersection always occurs somewhere in the route, and such a route is not the shortest route, as shown in FIG. 7B. It can be proved that the route obtained by correcting is clearly shorter than the route shown in FIG. That is, the line segment [q in FIG._i, Q_{i + 1}] And line [q_j, Q_{j + 1}] And the line segment [q in FIG._i, Q_{i + 1}] And line [q_j, Q_{j + 1}], The relationship of the sum of one side of the triangle and the other two sides is established, and it can be seen that there is always a shorter route in the route where there is an intersection. There is no waste.
[0034]
Therefore, the relay point q in the strong edge point extraction unit 22C._iThe ordering process of all relay points q_iCan be replaced with the problem of finding the shortest path to follow. According to “Discrete Optimization Method and Algorithm” (Toshihide Ibaraki, Iwanami Shoten, 1993), this problem is a kind of traveling salesman problem. By solving this problem, all relay points q_iThe shortest route to follow can be obtained.
[0035]
(2-1-1-2) Configuration of junction point connecting part
  A specific configuration of the relay point connecting portion 23 is shown in FIG. The relay point connecting unit 23 functions as a relay point connecting unit, which is the contour candidate area information I supplied from the contour candidate area determining unit 6._iIs received by the gradient vector calculation unit 24. The gradient vector calculation unit 24, as will be described later, contour candidate region information I_iAfter the gradient vector G is generated from the input image, the gradient vector G is normalized to generate a normalized gradient vector GS that does not depend on the magnitude of the color difference in the input image, and the gradient vector GS is Detection unit 25, route search unit26And to the key calculation unit 3.
[0036]
FIG. 9 shows the configuration of the contour candidate region determination unit 6 and the gradient vector calculation unit 24 in this embodiment. In the case of this embodiment, it is assumed that continuous curve information S1 along the contour is input from the input means 4 as information specifying the approximate position and direction of the contour to be detected.
[0037]
The contour candidate region determination unit 6 receives the estimated contour information S2 including the curve supplied from the estimated contour calculation unit 5 in the feature point extraction circuit 6A and the curve division circuit 6B. The feature point extraction circuit 6A extracts feature points from the curve corresponding to the estimated contour information S2, and sends the feature point information CR corresponding to the feature points to the curve dividing circuit 6B. Here, the feature point is a point indicating a unique point on the curve, for example, a bending point where the shape of the curve is bent or a color change point where the color of the image changes on the locus of the curve.
[0038]
The curve dividing circuit 6B approximates a curve according to the estimated contour information S2 to a continuous broken line L along an edge having each feature point as a vertex, and sends the bent line L to the area dividing circuit 6C. The area dividing circuit 6C is configured so that each line segment l of the broken line L is based on the broken line L supplied from the curved line dividing circuit 6B._jMask image I that specifies neighboring pixels of_mjFor each line segment l_jGenerated for each line segment corresponding to each_jAnd mask image I_mjAre sent to the gradient vector calculation unit 24 as information on the direction and position of the contour.
[0039]
The gradient vector calculation unit 24 serves as the gradient vector calculation means as contour candidate region information I_iLine segment supplied as_jAnd mask image I_mjAre received by the edge intensity normalization circuit 24A provided by the number of the pairs, and each mask image I_mjIs received by the image composition unit 24B. Each edge strength standard circuit 24A has a line segment l_jAnd mask image I_mjBased on the gradient vector G_jIs generated, and the gradient vector G_j, Line segment l_jAnd mask image I_mjBased on the above, the normalized gradient vector G is obtained by performing a normalization process for setting the magnitude of the color difference to “1”._sjAnd each normalized gradient vector G_sjIs sent to the image composition unit 24B.
The image composition unit 24B uses the corresponding gradient vector G_sjAnd mask image I_mjIs used to generate a standardized gradient vector GS and send the gradient vector GS to the edge detection unit 25, the route search unit 26, and the key calculation unit 3.
[0040]
Here, the specific contents of the normalization processing in the edge intensity image normalization circuit 24A will be described.
The edge strength normalization circuit 24A calculates a transverse line that vertically crosses the edge based on a mask image and a line segment pair given as information on the edge position and edge direction, and the edge strength on the transverse line is calculated. After calculating the edge strength maximum point at which the maximum is the edge strength, the edge strength is smaller than the edge strength at the edge strength maximum point on the half line on one side starting from the edge strength maximum point on the transverse line and The point P0 that is less than or equal to the threshold value is detected, and the edge intensity is smaller than the edge intensity at the edge intensity maximum point on the half line opposite to the above half line starting from the edge intensity maximum point on the transverse line. A point P1 that is equal to or less than a predetermined threshold is detected, and a straight line connecting the points P0 and P1 is defined as an integration path PI.
[0041]
Subsequently, the edge intensity normalization circuit 24A obtains edge intensity values at a plurality of arbitrary points from the end point P0 to the end point P1 on the integration path PI, and calculates the sum of the edge intensity values at these points as a normalization coefficient S. After that, the gradient vector G with the normalization factor S_jBy dividing the normalized gradient vector GS_jIs generated.
[0042]
Next, the configuration of the edge detection unit 25 in this embodiment is shown in FIG. The edge detection unit 25 receives, as an edge detection means, the normalized coefficient S and the integration path PI from the gradient vector calculation unit 24 by the threshold value calculation circuit 25A, and receives the normalized gradient vector GS from the edge point detection circuit. Receive at 25B.
The threshold value calculation circuit 25A sets the value obtained by dividing the normalization coefficient S by the length of the integration path PI (that is, the distance between the points P0 and P1) as a threshold value H to the edge point detection circuit 25B. Send it out.
[0043]
The edge point detection circuit 25B performs threshold value processing on the standardized gradient vector GS using the threshold value H, thereby detecting an edge point at which the edge intensity reaches a peak, and The edge point binary image EP representing the peak is sent to the route search unit 26.
Thus, the standardized gradient vector GS and edge point binary image EP are input to the route search unit 26.
[0044]
The route searching process in the route searching unit 26 will be described using the flowchart shown in FIG.
The route search unit 26 starts route search processing from step SP1 as route search means. In step SP2, step SP3, and step SP4, the relay point coordinate list Q, the gradient vector GS, and the edge point binary image EP are respectively obtained. receive. Next, in step SP5, the route search unit 26 initializes the contour coordinate list P to be output, and sets q to the first coordinate.₀Enter.
[0045]
Subsequently, in step SP6, the route searching unit 26 uses the local evaluation function d using the gradient vector GS and the edge point binary image EP, as will be described later, and the relay point q._iAnd relay point q_{i + 1}After calculating the shortest path Pt between the shortest path Pt and the end point q_{i + 1}Is input to the contour coordinate list P. When the contour coordinate list P is completed in this way, the contour coordinate list P is sent to the curve generation unit 21 at step SP7, and the route search process is terminated at step SP8. The shortest path Pt is a coordinate list representing eight neighboring line figures.
[0046]
Here, the process in step SP6 of the route search unit 26 will be described. Although the shortest path search can be obtained by various methods, the path cost function of every local path is uniquely determined, and the total cost of the entire path is the sum of these local path costs, i.e., unique. By setting a path evaluation function that represents the magnitude of the passage cost determined by the above, a high-speed algorithm such as “discrete optimization method and algorithm” or “Intelligent Scissors for Image Composition” described above can be used. The shortest path search algorithm in this “Intelligent Scissors for Image Composition” is shown in FIGS.
[0047]
However, the local evaluation function that determines the local passage cost needs to be set according to the specific problem to be applied. Here, a local evaluation function suitable for the purpose of the present invention will be described.
[0048]
(A) Evaluation function for the direction of travel
Since the gradient is directed in the direction in which the color gradient increases, when the color difference of the contour changes depending on the location, the gradient direction changes discontinuously on the contour. However, in this embodiment, the relay point connecting unit 23 uses the rough position and direction of the contour to be detected, that is, the edge position and the edge direction, so that the direction of the gradient vector is always smooth on the contour to be detected. The gradient is calculated to change. This makes it possible to create a local evaluation function that evaluates the cost of a route that always travels clockwise or counterclockwise along a contour lower than a route that goes back in the middle of the contour.
[0049]
Such a local evaluation function receives, as an input, the position coordinates p and q of two adjacent pixels existing between adjacent relay points in the vicinity of 8 during the route search, and a vector G indicating the direction of the gradient at the coordinates p and q. A function g between (p) and G (q) and coordinates p and q is expressed by the following equation (1).
[Expression 1]

It is configured to be represented by Here, H is a unit vector indicating a direction deviated by + π / 2 from the direction of the gradient vector G (q), and acos ((qp) / | q−p | · H) is a direction in which the contour advances. And the angle formed by the direction of the local route going from p to q.
[0050]
Here, since the direction of the gradient vector G (q) is continuous as described above, the cost of a route traveling in one direction along the contour is lower, and conversely the direction changes in the opposite direction even along the contour. The cost of the route is worse.
Therefore, the route search unit 26 can perform route search that always proceeds in one direction along the contour by using the local function g.
[0051]
(B) Edge peak passing evaluation function
An evaluation function that makes the path pass through the center of the contour as much as possible is effective for the present invention. An evaluation function having the same function is realized by using the Laplacian of the image in the “Intelligent Scissors for Image Composition” described above. Laplacian has a high ability to detect the center of an edge, but is sensitive to noise and is used in combination with a smoothing filter. However, the size of the smoothing filter to be used must be determined each time according to the contour to be obtained.
[0052]
In this embodiment, the relay point connecting unit 23 performs edge point detection processing for detecting an edge point at which the edge intensity reaches a peak using the rough position and direction of the contour to be detected, that is, the edge position and the edge direction. As a result, the edge center detection capability using the edge intensity can be improved, so that the reliability of the evaluation function can be improved. Further, since it is not necessary to use a smoothing filter as in Laplacian, the configuration of the relay point connecting portion 23 can be simplified accordingly.
[0053]
Such a local evaluation function receives, as inputs, the position coordinates p and q of two adjacent pixels existing between adjacent relay points in the vicinity of 8 during the route search, and is based on the coordinates p and q in the edge point binary image EP. A function e of the pixel values E (p) and E (q) and the coordinates p and q is expressed by the following equation (2).
[Expression 2]

It is configured to be represented by Here, E = 0 or 1. That is, the local evaluation function e is a function for evaluating a pixel passing through a position where the edge intensity reaches a peak. Therefore, the route search unit 23 can perform route search for evaluating the position where the edge intensity reaches a peak by using the local function e.
[0054]
(C) Evaluation function combination and 8-neighbor distance correction
In this embodiment, the above-described evaluation function g and evaluation function e (that is, the gradient vector GS and the edge binary image EP) are used in combination as follows.
The evaluation function d uses the evaluation function g and the evaluation function e as evaluation terms, and the distance between the coordinates p and q of two adjacent pixels existing between adjacent relay points (| q−p) By multiplying |), each evaluation term is subjected to an evaluation correction by 8 neighborhood distances (3)
[Equation 3]

To be represented by d. Here, the coefficients c1 and c2 are coefficients appropriately determined in advance.
[0055]
A general representation of this local function d is given by the following equation (4)
[Expression 4]

Represented by In this case, f_iRepresents the i-th evaluation term and c_iRepresents the th coefficient.
Accordingly, the route searching unit 23 can receive an evaluation correction for each of the plurality of evaluation terms based on the 8-neighbor distance.
[0056]
Thus, the contour coordinate list generation unit 20 binarizes an image having continuous values and generates the contour of the object as a contour coordinate list P composed of relay points, thereby making it easy to check the order of the shape of the contour. By connecting the points sequentially and extracting the contour shape as one path, unnecessary information such as other contours can be deleted.
[0057]
(2-1-2) Configuration of curve generation unit
The configuration of the curve generation unit 21 is shown in FIG. The curve generation unit 21 receives the contour coordinate list P supplied from the contour coordinate list generation unit 20 as a curve generation unit by the input coordinate division circuit 21A, the connection condition calculation circuit 21B, and the curve approximation circuit 21C, respectively. The input coordinate dividing circuit 21A generates a list of coordinates of division points (hereinafter referred to as a division point coordinate list) F that divides the shape represented by the outline coordinate list P, and uses this as a connection condition calculation circuit. 21B and the curve approximation circuit 21C.
[0058]
That is, the input coordinate dividing circuit 21A locally estimates the order of the shape to be obtained from the discrete arrangement of the contour coordinate list P. In this case, the outline coordinate list P is divided so that the order of the shape in each divided segment does not increase. For example, by determining each division point based on the magnitude of the curvature, it is possible to determine the order in each segment without depending on the number of coordinate data, thereby preventing the order in each segment from increasing. it can.
[0059]
Based on the contour coordinate list P and the division point coordinate list F, the connection condition calculation circuit 21B calculates a connection condition at each division point, that is, a connection condition that satisfies G1 continuity, and supplies the connection condition list RV to the curve approximation circuit 21C. Send it out. That is, the connection condition calculation circuit 21B has a connection position coordinate r for matching the connection positions of the respective dividing points._iAnd connecting position r_iVelocity vector v for matching the velocity direction at_iAnd a pair (that is, a tangent line) is calculated as a connection condition list RV. In this case, the coordinates of the connection position and the coordinates of the dividing points do not necessarily match.
[0060]
The curve approximating circuit 21C approximates each segment delimited by adjacent dividing points using the least square method so as to satisfy the connection condition at both end points, thereby approximating an approximate curve c between adjacent dividing points._iAnd approximate curve c_iIs sent to the key calculation unit 3 as contour curve information C.
[0061]
The curve generation processing in the curve generation unit 21 will be described using the flowchart shown in FIG.
The curve generation unit 21 starts the curve generation process from step SP1, receives the contour coordinate list P from the contour coordinate list generation unit 20 in step SP2, and in step SP3, based on the contour coordinate list P, the division point coordinate list F Create
[0062]
Subsequently, in step SP4, the curve generation unit 21 calculates an approximate curve that approximates the shape in the vicinity of each division point, and the tangent r in the vicinity of the division point in the approximate curve._i, V_iIs calculated as a connection condition RV for curve approximation of each segment. Next, in step SP5, the curve generation unit 21 approximates each segment delimited by the adjacent division points using the least square method so as to satisfy the connection condition at both end points, thereby approximating the approximate curve c._iAnd the approximate curve c is calculated at step SP6._iIs output as contour curve information C, and the curve generation process is terminated at step SP7.
[0063]
(2-2) Operation and effect of the embodiment
In the above configuration, the contour curve generation unit 7 starts the contour curve generation processing from step SP1 shown in FIG. 16, and the contour candidate region information I is received from the contour candidate region determination unit 6 in step SP2._iIn step SP3, the contour candidate area information I is received._iThen, the edge point having the maximum edge strength is calculated as a relay point, and the relay points are sequentially connected to generate a contour coordinate list P representing eight neighboring line figures as the contour of the object.
[0064]
Next, in step SP4, the contour curve generation unit 7 divides the shape represented by the contour coordinate list P into a plurality of segments, calculates a connection condition for connecting the segments, and satisfies each connection condition. Are approximated to generate an approximate curve, and in step SP5, the approximate curve group is sent to the key calculation unit 3 as contour curve information C, and the contour curve generation process is terminated in step SP6.
[0065]
Accordingly, the contour curve generation unit 7 generates the contour of the object as an outline coordinate list P composed of a plurality of discrete coordinate data from an image composed of continuous values, so the order of the shape of the contour of the object Can be easily examined.
Further, since the contour curve generation unit 7 extracts the contour of the target object as a single route, unnecessary information such as other contours other than the contour to be extracted can be deleted. The contour can be extracted with high accuracy.
[0066]
Further, the contour curve generation unit 7 can make the problem of extracting the contour of the object from the image as a curve easy to solve by curve approximation, so that the contour of the object can be easily extracted as a curve. .
[0067]
According to the above configuration, the contour candidate area information I_iThen, an edge point having the maximum edge strength is calculated as a relay point, and each of the relay points is sequentially connected to generate a contour coordinate list P representing eight neighboring line figures as the contour of the object. The object represented by dividing the shape represented by P into a plurality of segments, calculating a connection condition for connecting the segments, and approximating each segment so as to satisfy the connection condition and generating an approximate curve. It is possible to easily check the order of the shape of the contour, and it is possible to delete unnecessary information such as a contour other than the contour to be extracted.
Thus, it is possible to realize the contour extracting unit 7 and the contour extracting method that can accurately extract the contour curve that matches the order of the shape of the contour of the object from the image.
[0068]
In addition, according to the above-described configuration, the shape represented by the contour coordinate list P is determined based on the feature points of the contour of the object, and the order of each segment is determined. Is calculated as a connection condition, and each approximate curve is approximated so as to maintain the connection condition. Therefore, a contour curve that matches the order of the shape represented by the contour coordinate list P and that is smoothly continuous Can be generated.
[0069]
Furthermore, according to the above-described configuration, a plurality of relay points q from the contour._i, And each relay point q_iSince the eight neighboring line figures are generated by connecting the two by the route search process, the approximation accuracy of the curve approximation process in the curve generation unit 21 can be improved. A plurality of relay points q from the contour_iTherefore, high-speed route search processing can be performed.
[0070]
Furthermore, according to the above-described configuration, the route between adjacent relay points can be reduced to the shortest route search problem in each small portion of the contour, so that the contour 8 neighboring line figure 2 image can be generated. Thus, the contour can be extracted as a single route. Therefore, the contour of the object can be extracted with high accuracy.
[0071]
Furthermore, according to the above configuration, the route search process is performed using the local evaluation function using the gradient of the image and the extreme value position information of the edge intensity, so that the influence of noise can be avoided. In addition, since a local evaluation function for calculating a local evaluation value between two adjacent pixels is used, a fast shortest path search algorithm can be applied.
[0072]
Further, according to the above-described configuration, the edge point having the maximum edge strength among the edge points is determined as the relay point q based on the gradient vector GS and the edge point binary image EP._iAs a result, it is not necessary for the operator to give points necessary for exploration when performing contour exploration processing, and the contour extraction work can be simplified accordingly, and points on the contour can be extracted reliably. Can do.
[0073]
Further, according to the above configuration, since the local evaluation function d is used, it is possible to avoid the influence of the gradient of the edge strength and the gradient of the edge strength caused by the contour of another object. The contour of an object can be accurately extracted as a contour curve.
[0074]
  Furthermore, according to the above-described configuration, since the edge point is detected using the standardized gradient vector GS that does not depend on the magnitude of the color difference, it is evenly distributed over the entire contour.Relay pointThus, the contour of the object can be extracted with high accuracy.
[0075]
(3) Other embodiments
In the above-described embodiment, the case where the relay point extracting unit 22 is used as the relay point extracting unit has been described. However, the present invention is not limited to this, and the same reference numerals are given to the corresponding parts in FIGS. You may use the relay point extraction part 30 as shown in FIG. 17 as a relay point extraction means.
[0076]
The relay point extraction processing in the relay point extraction unit 30 will be described with reference to the flowchart shown in FIG.
The relay point extraction unit 30 starts the relay point extraction process from step SP1, and in step SP2, the corresponding line segment l_jAnd mask image I_mjContour candidate area information I consisting of each pair of_iIs received by the gradient vector calculation unit 24 and the edge detection unit 22B.
[0077]
Next, at step SP3, the relay point extraction unit 30 calculates a normalized gradient vector GS at the gradient vector calculation unit 24, and then generates an edge point binary image E at the edge detection unit 22B at step SP4. Then, the gradient vector GS and the edge point binary image E are sent to the strong edge point extraction unit 22C. In this case, the gradient vector calculation unit 24 may send the gradient vector G to the strong edge point extraction unit 22C without normalizing the calculated gradient vector G.
[0078]
Subsequently, in step SP5, the relay point extraction unit 30 generates a relay point coordinate list Q based on the gradient vector GS and the edge point binary image E in the strong edge point extraction unit 22C, and then in step SP6. The relay point coordinate list Q is sent to the relay point coupling unit 23, and the relay point extraction process is terminated at step SP7. Here, based on the gradient vector GS and the edge point binary image E, the strong edge point extraction unit 22C calculates, as a relay point, an edge point having the maximum edge intensity among the edge points, and relays at the relay point. The point coordinate list Q may be generated.
[0079]
In this case, since the edge intensity can be detected uniformly without depending on the magnitude of the color difference in the input image, the relay points can be extracted almost evenly over the entire contour of the object. Therefore, the accuracy of the contour curve obtained can be further improved. In this case, since the curve information for specifying the rough shape of the contour of the object is input from the input unit 4, the reliability of the relay point on the contour can be improved.
[0080]
In the above-described embodiment, the case where the relay point connecting unit 23 is used as the relay point connecting means has been described. However, the present invention is not limited to this, and the same reference numerals are given to the corresponding parts in FIGS. A relay point connecting part 40 as shown in FIG. 19 may be used.
This relay point connecting part 40 is contour candidate area information I consisting of a polygonal line or a curve._iIs received by the gradient vector calculation unit 41 and the edge detection unit 22B.
The gradient vector calculation unit 41 uses the generally known gradient vector calculation method to use the contour candidate area information I._iThe gradient vector G in the contour candidate region corresponding to the above is calculated, and the gradient vector G is sent to the route search unit 26.
[0081]
Further, in the above-described embodiment, the case where the relay point connecting unit 23 is used as the relay point connecting means has been described. However, the present invention is not limited to this, and the above-described “for image composition” is performed without performing the route search processing. You may make it connect a relay point using the thinning process described in the "object extraction method" and the "area | region extraction method."
[0082]
Furthermore, in the above-described embodiment, the case where the local evaluation function d combining the local evaluation function g and the local evaluation function d is used as the route evaluation method has been described. However, the present invention is not limited to this, and the point is uniquely determined. Any other evaluation function may be used as the route evaluation method as long as the evaluation function represents the magnitude of the passing cost. In this case, a path between adjacent intermediate points is determined so that this evaluation function is minimized. Further, an image Laplacian may be used as a route evaluation method, and “Intelligent Scissors for Image Composition” or “Discrete Optimization Method and Algorithm” may be used as a search method.
[0083]
Further, in the above-described embodiment, the case where the local evaluation function g configured by the expression (1) is used as the local evaluation function g has been described, but the present invention is not limited to this, and the following expression (5)
[Equation 5]

As shown, the gradient vectors G (p) and G (q) in the position coordinates of two adjacent pixels existing between adjacent relay points and the position coordinates of the two pixels in the vicinity of 8 during the route search You may make it give the route search part 26 the local evaluation function g which has the evaluation term represented by the function with p and q. As a result, a route search can be performed in consideration of the direction in which the edge travels.
[0084]
Furthermore, in the above-described embodiment, the case where the local evaluation function e configured by the expression (2) is used as the local evaluation function e has been described, but the present invention is not limited to this, and the following expression (6)
[Formula 6]

As shown in the figure, the point is that in the vicinity of 8 during the route search, the pixel value (pixel value in the edge point binary image EP) E (p) in the position coordinates of two adjacent pixels existing between adjacent relay points, The route search unit 26 may be provided with a local evaluation function e having an evaluation term represented by a function of E (q) and the position coordinates p and q of two pixels. This makes it possible to realize a route search in consideration of the peak position of the edge.
[0085]
Furthermore, in the above-described embodiment, the case where the local evaluation function d obtained by combining the local evaluation function g and the local evaluation function d is described. However, the present invention is not limited to this, and the local evaluation function g or the local evaluation function e is used. In short, a local evaluation function may be used depending on the situation.
[0086]
Furthermore, in the above-described embodiment, the case where the operator inputs a line segment, a broken line, or a curve using the input unit 4 has been described. However, the present invention is not limited to this, and as described above, the current image i is already The estimated contour information S2 of the current image i may be obtained on the basis of the image i-1 one frame before the contour curve obtained as the contour curve information C and the contour curve information C of the image i-1. Good (Figure 1). Thereby, work efficiency can be improved markedly.
[0087]
Further, in the above-described embodiment, the curve corresponding to the estimated contour information S2 is approximated to a polygonal line having each feature point as a vertex, and each line segment l of the polygonal line is obtained._iIs sent to the edge strength standard circuit 24A, but the present invention is not limited to this, and a segment is formed by dividing a curve corresponding to the estimated contour information S2 at each feature point and dividing the curve at each feature point. May be sent to the edge strength normalization circuit 24A.
[0088]
【The invention's effect】
  As described above, according to the present invention, the contour of an objectRelay point q based on edge strength from contour candidate area information Ii containing _i , And each relay point q _i By exploring and connecting the routes connectingBy generating a line figure consisting of multiple discrete coordinate data and generating a curve that approximates the shape represented by the line figure, the contour shape order can be easily examined and the contour shape Can finally be obtained as a curve.
  Thus, it is possible to realize a contour extraction apparatus and a contour extraction method that can obtain a contour curve that matches the contour shape of the object.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an overall configuration of a key signal generation apparatus to which the present invention is applied.
FIG. 2 is a block diagram showing a configuration of a contour curve generation unit.
FIG. 3 is a block diagram showing a configuration of a contour coordinate list generation unit.
FIG. 4 is a flowchart for explaining a processing procedure of contour coordinate list generation processing;
FIG. 5 is a block diagram showing a configuration of a relay point extraction unit.
FIG. 6 is a flowchart for explaining a processing procedure of a relay point extraction process.
FIG. 7 is a schematic diagram for explaining relay point ordering processing;
FIG. 8 is a block diagram showing a configuration of a relay point connecting portion.
FIG. 9 is a block diagram showing a configuration of a gradient vector calculation unit and a contour candidate region determination unit.
FIG. 10 is a block diagram showing a configuration of an edge detection unit.
FIG. 11 is a flowchart for explaining a processing procedure of route search processing;
FIG. 12 is a chart showing a shortest path search algorithm.
FIG. 13 is a chart showing a shortest path search algorithm.
FIG. 14 is a block diagram showing a configuration of a curve generation unit.
FIG. 15 is a flowchart for explaining a processing procedure of curve generation processing;
FIG. 16 is a flowchart for explaining the processing procedure of the contour curve generation processing;
FIG. 17 is a block diagram showing a configuration of a relay point extraction unit according to another embodiment.
FIG. 18 is a flowchart for explaining a processing procedure of a relay point extraction process in a relay point extraction unit according to another embodiment.
FIG. 19 is a block diagram illustrating a configuration of a relay point coupling unit according to another embodiment.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Key signal generation apparatus, 2 ... Contour extraction part, 3 ... Key calculation part, 4 ... Input means, 5 ... Estimated outline calculation part, 6 ... Contour candidate area | region determination part, 6A ... Feature point Extraction circuit, 6B... Curve division circuit, 6C... Area division circuit, 7... Contour curve generation unit, 8... Motion vector estimation unit, 9. Soft key generation unit, 20 ... contour coordinate list generation unit, 21 ... curve generation unit, 21A ... input coordinate division circuit, 21B ... connection condition calculation circuit, 21C ... curve approximation circuit, 22, 30 ... relay Point extraction unit, 22A... Edge strength calculation unit, 22B, 25... Edge detection unit, 22C. , 24A ... Edge strength normalization circuit, 24B ... Image Combining unit, 25A ...... threshold calculation circuit, 25B ...... edge point detection circuit. 26 …… Route search section.

Claims (9)

In a contour extraction device that extracts the contour of an object from an image using route search,
A relay point extracting means for extracting a plurality of points along the contour in the contour candidate region where the contour of the object is considered to exist as a relay point, and ordering the relay points along the contour ;
The route connecting the relay points is searched by determining the shortest route between the relay points in the above sequence so that the evaluation function representing the sum of the passing costs set for each local route is minimized. , a relay point connecting means for generating a line figure more representative of the contour to sequentially connecting the route,
Curve generating means for generating a curve approximating the shape represented by the line figure,
The relay point extracting means includes strong edge point extracting means for extracting, as the relay point, the edge point having an edge strength greater than a predetermined threshold value among a plurality of edge points detected in the contour candidate region. A contour extraction apparatus characterized by the above. The relay point extracting means is
Edge strength calculating means for calculating and outputting the edge strength from the image,
Edge detection means for detecting an edge point from the image and generating and outputting an edge point binary image for designating the position of the edge point;
The strong edge point extracting means extracts, as the relay point, the edge point having the edge intensity greater than a predetermined threshold among the edge points based on the outputs of the edge intensity calculating means and the edge detecting means. The contour extracting apparatus according to claim 1, wherein: The relay point extracting means is
Gradient vector calculation means for calculating and outputting a gradient vector from the image using contour candidate area information consisting of a continuous polygonal line or curve along an estimated contour curve roughly showing the contour of the object;
Edge detection means for detecting an edge point from the image in the contour candidate region, and generating and outputting an edge point binary image designating the position of the edge point;
With
The strong edge point extracting means is based on the output of the gradient vector calculating means and the edge detecting means, each line segment of the broken line, each line segment that approximates each segment obtained by dividing the curve, or the curve. 2. The contour extracting apparatus according to claim 1, wherein the edge point having the maximum edge strength among the edge points is extracted as the relay point for each neighborhood of each segment obtained by dividing the segment. The route exploration means is
A gradient vector calculating means for calculating a gradient vector from the image;
Based on the gradient vector, the gradient vector at the position coordinates of the two pixels adjacent to each other existing between the relay points in the above sequence in the vicinity of 8 in the route search is calculated. 2. The route between relay points in which the order is continuous is determined using a local evaluation function having an evaluation term expressed by a function of the gradient vector and the position coordinates of the two pixels. The contour extracting device according to claim 1. The route exploration means is
Between the relay points having a local evaluation function for calculating a local evaluation value between two adjacent pixels that exist between the relay points in which the order is continuous, and in which the order is continuous using the local evaluation function The contour extracting apparatus according to claim 1 , wherein the path is determined. The curve generation means includes
A dividing point calculating means for calculating and outputting a dividing point for dividing the shape represented by the line figure into a plurality of segments;
Based on the line figure and each division point, a connection condition calculation means for calculating and outputting a connection condition for connecting the segments for each division point;
A curve approximating means for approximating each segment in a curve so as to satisfy the connection condition at each division point based on the line figure, each division point, and each connection condition. The contour extracting apparatus according to 1. Contour estimation means for estimating the contour of the object as the estimated contour based on the rough shape of the contour specified by the operator, and the relay point extracting means is based on the estimated contour estimated by the contour estimation means. The contour extracting apparatus according to claim 1, wherein the contour candidate region is calculated and the relay point is extracted. Contour estimation means for estimating the contour of the object in the current image as an estimated contour based on the output of the curve generation means, and the relay point extraction means is based on the estimated contour estimated by the contour estimation means. Calculating the contour candidate area in the current image and extracting the relay point;
The said relay point connection means produces | generates the outline of the said target object in the said current image as the said line figure by exploring the path | route which connects the said relay points, and connecting sequentially. Contour extractor. In a contour extraction method for extracting a contour of an object from an image using route search,
A relay point extraction step for extracting a plurality of points along the contour in the contour candidate area where the contour of the object is considered to exist as relay points and ordering the relay points along the contour ;
A relay point connection step for generating a line figure representing the outline by exploring and sequentially connecting routes connecting the relay points;
A curve generation step for generating a curve approximating the shape represented by the line figure,
The relay point extraction step includes a strong edge point extraction step for extracting, as the relay point, the edge point having an edge strength greater than a predetermined threshold among a plurality of edge points detected in the contour candidate region,
The relay point connection step has an evaluation function that represents a sum of passage costs set for each local route, and the shortest path between the relay points in the order is continuous so that the evaluation function is minimized. A contour extraction method comprising a route search step for determining a path .

Images