JP3541774B2 - Vehicle type identification system - Google Patents

Description

【００５７】
このようにしてＹｏをＺｏ，ＹｅをＺｅに変換し、速度Ｖを式２で求める。
【００５８】
【式２】

【００５９】
ここで、車両の追跡処理は、特開平10−307988号「交通流監視装置」に記載のような手法について説明したが、車両のパターンを抽出し、その末尾座標の位置変動から速度を求める方式など色々と考えられるが、検知領域を通過した車両の速度を計測できる手法であれば、その方式は限定しない。また、画素と実距離の変換方法は他にも色々な方法があり、上記の変換式に限定するものではない。
【００６０】
次に車種判別手段８について説明する。車種判別手段８は図１２のように動作する。まず車長を求め、車長がＬＢより長い時は大型となり、大型と判別した場合は、画像濃度変化，エッジ変化検出・記憶手段４，５に記憶されているデータを用いて形状判別（ここではバス，貨物判別）を行い、車長がＬＢより短い時は小型となり、同様にして形状判別（貨物，乗用車の区別）を行う。ここで、上記例は小型，大型を分類するものであるが、車長によっては小型，中型，大型の３車種に分類し、これらについてそれぞれ貨物、乗用などと分類することも可能である。この車長の計測は車両の通過速度Ｖと車両存在確認手段３で求まった存在時間TimeSを用いて車長＝速度Ｖ＊存在時間TimeSにより算出することができる。
【００６１】
入力画像全体から車長を求めることは、車両が重なった場合の車両の分離が困難なため、正確には行えなかったが、このように、検知領域を設定し、その検知領域における車両の通過速度と存在時間を求めると車長を正確に求めることができる。
【００６２】
なお、図３のような視野に設定しているので、画面の下部では車両が重なることがなく（ほぼ真下を撮影）、検知領域の情報は車両１台分の情報であり、画像から車長を正確に求めることが可能となっている。
【００６３】
ところで、カメラの向きから検知領域を通過している車両の時間と車長の関係は図１３のようになるため、補正することが精度の向上につながる。すなわち、（車長＋検知幅）＝速度Ｖ＊存在時間TimeSであるため、よって、車長＝速度Ｖ＊存在時間TimeS−検知幅で算出する。なお、検知幅とは車線方向の検知領域の長さである。
【００６４】
また、画像濃度変化，エッジ濃度変化による形状チェックは、図１４（ａ）のように車両存在回数分記憶されているデータを、例えば１０個に正規化し図１４（ｂ）のように変換する。この処理は、車長及び速度によってデータの個数が大きく変化するため判別処理が複雑になるのを回避するために正規化するものである。正規化法は単純にｎ個（車長，通過時間によって変動）のデータを１０個に間引きする処理でも良いし、更に正確なものとして補間付きの圧縮処理でも良い。
【００６５】
正規化した画像濃度データをGray［０］〜Gray［９］とするとバスの場合は、図１５のように後部に窓が有るので濃度値がGray［７］〜Gray［９］当たりで下がる。また、天井の濃度はほぼ一様なのでGray［０］からGray［６］当たりの濃度は均一となる。このため、画像濃度の時系列データの前半が均一か否かの判別を行う。
【００６６】
車長で大型と判別した場合は、以下の判別条件でバスと判別する。
（１）Gray［７］からGray［９］が最小濃度である（図１５ Ｓ８６−１）。
（２）Gray［０］からGray［６］の平均濃度Ａｖｅとの差の絶対値の累積が所定値以下である（Ｓ８６−２）。
【００６７】
更に、エッジ変化検出・記憶手段５によって求まる情報も同様にして１０個にデータを正規化しEdge［０］からEdge［９］に記憶する。バスの場合は、車両の先頭により最初に大きなエッジ濃度が発生し、最後の位置に窓による大きなエッジ（濃度）が発生する。これに対し、トラックなどでは、途中に運転席と荷台の間に大きなエッジ濃度があるためこの情報が発生する。
【００６８】
したがって、バスの場合は以下の条件で判別する。
（１）Edge［０］からEdge［４］の平均よりEdge［５］からEdge［９］の平均濃度が高いか、中央のエッジ濃度は一様（Ｓ８６−３）。
【００６９】
同様に、車長がＬＢより短い場合は小型車とするが、濃度変化，エッジ濃度変化チェック処理により、貨物か乗用かを判断する。セダンなどでは、大きなエッジ濃度が窓の部分で発生するが、小型貨物ではエッジ濃度が途中小さい。この違いから小型乗用車，小型貨物の判別が可能である。
【００７０】
このように、車両検知領域を通過する車両の車長，画像濃度の時系列データ，エッジ濃度の時系列データを用いることで、さまざまな形状の分類を正確に行うことが可能となる。２次元の画像パターンを記憶して分類する方式は、記憶容量の問題のほかに、車両の形状を正確に切り出すことが非常に困難であるが、本発明では、検知領域を通過する車両の画像濃度，エッジ濃度を時系列に求めるだけで形状判別が可能である。
【００７１】
車種判別手段は以上のように種々のしきい値を設定し判別することが可能であるが、より安定的に判断する処理としてニューラルネットワークの利用がある。ニューラルネットワークを利用する場合のネットワークの構造を図１６に示す。ネットワークは３層構造のニューラルネットワークであり、ネットワークの入力層に、画像濃度を１０個（例えば）に正規化したGray［０］からGray［９］を入力する。さらにエッジ濃度を１０個に正規化したEdge［０］からEdge［９］を入力する。さらに、車長を入力する。出力層は、形状毎の出力を用意する。ここで、ニューラルネットワークの扱うデータ範囲は０から１の実数であるため、Gray［ｉ］は画像濃度の最大値で除して、Edge［ｉ］はエッジ濃度の最大値で除して０から１の実数に変換することが必要である。また、車長については例えば１５ｍを最大として車長を１５ｍで除した数値（１以上は１に設定）する。
【００７２】
このような入力データを入力し、学習時は教師データとして車種毎の値を与えることで学習する。例えば、車種を５車種（小型乗用，小型貨物，大型貨物，バス，その他）に判別する場合、小型乗用のデータを入力した場合は、出力の教師データとして小型乗用を１その他を０に設定し与える。
【００７３】
認識時は車種毎の出力を求め、最大出力の車種を認識結果として用いる。また、ニューラルネットワークを小型，大型の２種類用意する方法もある。図１７は小型の例であるが、入力は図１６の車長を除いた濃度，エッジ情報を入力し、出力は小型乗用，小型貨物を用意する。このように大型，小型のニューラルネットワーク分けることで、小型車両の車長であるにもかかわらず少なくとも大型に判断する誤りは防ぐことが可能となる。また、図１６，図１７ではエッジ濃度は垂直エッジ強調の微分画像の平均濃度を時系列に求めた情報を入力しているが、水平エッジ濃度を加えた情報を入力することが必要な場合も同様な処理で対応可能である（Edge［０］からEdge［９］が垂直エッジ強調情報、Edge［１０］から
Edge［１９］が水平エッジ強調情報とすると入力層は濃度情報と合わせ３０個設ければ良い）。
【００７４】
このように、ニューラルネットワークを利用すると、画像濃度，エッジ濃度の形状判別にしきい値などを設定する必要が無く、きわめて簡単に車種の判別が可能となる。また、エッジ濃度を水平，垂直両方用いる場合、データ数が増えプログラム的に車種判別の処理を作り込むのは容易でないが、ニューラルネットワークを用いることで、判別に用いる時系列パターンの数が増えても車種判別が高精度にできるメリットがある。
【００７５】
また、あらかじめ形状毎の画像濃度の時系列データをパターン，エッジ濃度の時系列データを形状数（例えば、バス，大型貨物）記憶しておき、画像から得られる変化情報との類似度を画像濃度，エッジ濃度毎に求め、これらの類似度の総和が最大のパターンを認識結果とすることもできる。記憶容量は時系列データを分類形状数だけ用意するだけであり、ごく少ないデータで良い。
【００７６】
いずれにしても、車両が検知領域を通過している間の画像濃度の時系列データ，エッジ濃度の時系列データ、及び車長を求めることで、これまでの大型，小型の判別だけでなく、車両の形状判別も可能となる。
【００７７】
以上の処理の流れを詳細に説明する。図１８に全体フローを示す。まず、画像処理装置の初期化や各種テーブルの初期化を実行するＳ０。検知領域設定手段で車両検知領域を設定するＳ１。車両の存在を求めるベースになる背景画像を作成するＳ２。以下が処理のループになる。
【００７８】
画像を入力Ｓ３し、車両存在確認Ｓ４で検知領域上に車両が存在しているかを確認する。また、検知領域内の濃度の変化、エッジの変化情報を求める。車両が検知領域を抜けた場合はＳ５で車両の末尾領域をテンプレート画像に登録する
Ｓ５。登録したテンプレート画像と似ている場所を入力画像から探し追跡処理するＳ６。追跡が終了したら、車両の速度を算出するＳ７。全ての情報が求まった場合は、車種の判別Ｓ８を実行する。以下、各処理の詳細フローを示す。
【００７９】
図１９に車両存在確認処理Ｓ４のフローを示す。まず、背景画像と入力画像の差分処理Ｓ４−１を実行し、この画像を２値化Ｓ４−２し、２値の面積を求めるＳ４−３。この面積が所定値以上の場合は、車両が存在していると判断しCountを増加するＳ４−５。また、検知領域内の入力画像の平均濃度および微分画像の平均濃度を求め、テーブルに格納する（Ｓ４−６からＳ４−９）。微分処理で、水平，垂直の両方の情報を用いる場合は配列を２倍用意する。
【００８０】
面積が小さい場合は車両が存在しないと判断するが、Count(存在回数)が所定値以下ならノイズで存在，非存在が変動したと判断しCountをクリアする。Countが所定値以上の場合は車両が通過したと判断し、車両末尾座標の検出Ｓ４−１１を実行し（図８，図９の処理）車両末尾が求まった場合はｘｃ(Ｘ方向中心座標)、ｙｃ（Ｙ方向中心座標）にその座標を格納する。また、Ｓ４−７，９で格納した濃度，エッジ濃度の時系列データを退避するＳ４−１２，１３。これは、次の車両が進入した時にデータが消去されないようにするためである。車種判別のためには図３のような視野に設定した場合、車両が画面に２台（１車線）までしか入らないので、このような処理で良い。また、通過時間を格納Ｓ４−１４する。なお、車両が通過したと判断した場合の回数を求めることで、通過台数を算出することができる。
【００８１】
図２０はテンプレート登録のフローである。車両が通過した時に車両の末尾座標を求めるが、この座標ｘｃ，ｙｃが求まっている場合に、テンプレート画像の登録処理を実行する。テンプレートは複数個登録できるようにしているため（多車線対応）、図２０のように、未使用の番号を探し（Track［ｉ］…hold が０の場合未使用）、未使用の場合はそのテーブルに所定情報を格納する。格納データは、追跡座標のＸ中心，Ｙ中心（ｘｃ，ｙｃ）、その初期値（ｏｘｃ，ｏｙｃ），追跡回数(count），テーブルの状態(hold＝２）、時刻(starttime，endtime）である。さらに、ｘｃ，ｙｃを中心として入力画像から所定のＸ幅，Ｙ幅の大きさの画像をテンプレートに登録する。
【００８２】
ここで、登録データは図２１のような管理テーブルである。
【００８３】
次に、図２２に追跡処理フローを示す。テーブル番号を０からスタートし、管理テーブル（Track［ｉ］）のhold が２の場合は、現時刻でテンプレートが登録されたばかりなので、holdを１にして次の入力画像まで待つＳ６−３。holdが１の場合は登録してから新しい画像が入力されているのでこのデータについて追跡処理する。まず、Track［ｉ］．ｘｃ，ｙｃ の値を基準に探索領域（サーチエリア）を設定するＳ６−５。これは図１０の１００のように設定する。サーチエリアが決まれば、その領域内で最も登録しているテンプレート画像に類似している位置を探すＳ６−６。位置が見つかった場合は、管理テーブルのｘｃ，ｙｃ（ｏｘｃ，ｏｙｃは更新しない）、時刻（endtime）を更新しＳ６−９、登録画像のテンプレート画像をｘｃ，ｙｃの位置の近傍画像に更新するＳ６−１０。一方、登録パターンが見つからない場合は、holdを３に設定し追跡が終了したことにするＳ６−８。このような処理をテーブル数ループする。通常、２車線の場合、テーブル数は４個程度有れば十分である。
【００８４】
なお、テンプレート画像の記憶番号は管理テーブルの番号ｉと同じである。
【００８５】
次に図２３に速度算出フローを示す。前述したように車両追跡終了時はholdを３に設定しているので、holdが３になったテーブルに記憶されている、座標，時刻から速度を算出するＳ７−３。速度を計算したら管理テーブルのデータを初期化するＳ７−４。
【００８６】
車種判別Ｓ８のフローを図２４に示す。車両追跡処理により速度が求まった場合に、速度，存在時間TimeS から車長を求めＳ８−２、これを基に車種判別するＳ８−３。速度が求めっていない場合は、車両が通過していないので何も実行しない。なお、車種判別の具体例は、図１２，図１５で説明した通りである。検知領域を車両が通過してから車両の速度が求まるまでには、速度が速い場合は、数回の追跡処理、例えば３３ミリ秒×１０回＝３３０ミリ秒後に速度が求まるので、検知領域通過後瞬時に車種判別が可能であるが、遅い車の場合は画面を通過するまで速度計算しないため、数十秒後に速度が求まる場合などがある。このため、車両追跡回数が所定値以上（例えば、２０回）になったら速度を求め、車種判別するようにしても良い。この場合は、図２２の管理テーブルの更新Ｓ６−９の後に、追跡回数Track［ｉ］．countが所定値以上ならholdを３に設定し、速度を求めるようにすればよい。
【００８７】
以上の説明は車線数が１本の場合について述べたが、車線数が２本，３本の場合は、各処理を車線数分ループ処理し、各テーブルを車線数分の配列で記憶すれば、容易に多車線対応が可能である。
【００８８】
ところで、上記車種判別手法は渋滞した場合に問題が発生する。すなわち、渋滞すると、検知領域に車両が停止し、車両存在時間が大きく延びる。この車両は最終的には画面上から消えるため、ある速度で移動するが、車長を計算する処理は、存在時間と速度から求めているため、車長が非常に大きくなってしまう。
【００８９】
そこで、存在時間の計測処理を図２５のように改善する。車両が移動している場合だけ、存在時間としてカウントするようにするため（存在確認処理としては存在と判断するが記憶カウンタとしてはカウントしない）、検知領域について現在時刻（ｔ）の入力画像ｆ（ｔ）と所定時刻前（ｔ−ｄｔ）の入力画像ｆ（ｔ−ｄｔ）の画像間の差分処理を実行しＳ４−１６、差分画像を所定のしきい値で２値化した時の面積を算出する。この面積が所定値以下なら移動していないと判断しＳ４−１７、存在カウンタを更新しない。
【００９０】
この処理により、渋滞していても存在時間がほぼ正しく求まるため車長の計測が可能となる。なお、移動速度Ｖが加減速する場合は車長に誤差が発生するが、速度算出の範囲を検知領域の近くに設定することで誤差を低減可能である。
【００９１】
また、記憶している背景画像は更新が必要であるが、画像処理で一般的な移動物体が無い場合の画像の平均化（新しい背景画像＝古い背景画像と入力画像の平均）などで対応可能である（検知領域だけの狭い範囲なので処理は安定する）。
【００９２】
なお、速度算出のための車両追跡処理にパターンマッチング処理を用いた例を示しているが、そのほかの速度算出の方式も流用可能である。また、存在確認処理は背景画像と入力画像の類似度で判別する方式が安定するが、背景差分の平均濃度を監視したりする方式などが考えられる。
【００９３】
また、夜間においてはテールランプだけしか見えない状態では、本発明の効果は発生しないため（車長が求まらない）、夜間時においては、ＬＥＤ照明などによる照明を少なくとも検知領域内に照射することが必要である。それ以外の場所での処理はテールランプを検出し追跡処理し速度を求めることが出来るので問題ない。将来的にはカメラの感度が向上し、夜間でも照明が不用になる可能性があるが、現在では照明が必要である。
【００９４】
以上の例は車両が同一方向に移動する場合、特に車両を後方から撮影し、台数，速度，車種を判別するものである。
【００９５】
しかし、道路には片側１車線の道路が対面通行している場所も多い。この場合に検知領域を図２６のように左車線は画面の下側Ｄ１，右車線は画面の上側Ｄ２に設置し、車両をそれぞれ追跡する（１０１，１０２は追跡のパターンを示す）。ここで、左車線の車両検知，通過，追跡などの時間経過は図２７に示すように、検知領域を通過した時点から車両の追跡処理が始まり、追跡終了後、速度算出，車種判別が行われる。これに対し、右車線の場合は、図２８のように車両が検知領域に進入した瞬間から車両追跡が始まり、追跡終了後、速度算出，車種判別が行われる。これは、検知領域通過直後に車両のパターンを追跡すると、車両の屋根などを追跡することになり、この車高が影響し速度計測精度が低下するので、できるだけ路面との高さが少ない場所を追跡する必要がるためである。このため、検知領域に進入した時に車両前面のパターンを抽出し追跡する。このように、速度追跡処理方法を変更することで対面通行に対応可能である。
【００９６】
ところで、カメラの視野を図２９のように真下を撮影し、検知領域は画面の中央に設定するＤ１，Ｄ２。このようにすれば、上り，下り車線とも同じ処理で台数，速度，車種の判別が可能である。すなわち、検知領域を通過してから車両の速度を上り下りとも同じように求めることが可能である。また、一般的なテレビカメラは水平，垂直の比率が４：３であるので、カメラを横向きに設置して撮影すれば、図３０のように車線方向に長い撮影視野をとれるので、より好ましい。
【００９７】
上り，下り対面通行の場合には、カメラを真下に向けることが有効である。これは、設置上非常に簡単であり効果と、認識処理を同じに出来るという効果がある。
【００９８】
また、カメラなどで道路を撮影し、その映像に車両検知領域を設け、この領域を通過する車両の存在，非存在情報，濃度の時系列情報，エッジ濃度の時系列データ、および通過速度を求め、これら情報で車長を基準に大型，小型を判別し、さらに濃度，エッジ濃度の時系列データから貨物，バスなどの形状判別を行うことが可能である。
【００９９】
また、１枚の画像から車両を切り出し車種を判別することは非常に困難であるが、検知領域を通過する車両の時間的変化を用いるので、車両の特徴を正確に抽出できるため容易に車種判別が可能である。
【０１００】
また、画像処理であるため、認識結果の検証も容易であり、また、車長をベースに判別するため正確な車種判別が可能である。なお、車長の設定値によっては大型，中型，小型の分類も可能である。
【０１０１】
【発明の効果】
本発明によれば、車種判別装置における車種判別の精度を向上させることができる。
【図面の簡単な説明】
【図１】本発明の処理手段の構成を示す図である。
【図２】本発明に使用する画像処理ハードウエアの一例を示す図である。
【図３】画像感知器の撮影例を示す図である。
【図４】車両存在有無を判別する処理の一例を示す図である。
【図５】車両存在有無を判別する処理で微分処理を用いた例を示す図である。
【図６】車両存在有無の状態を記憶した場合を説明する図である。
【図７】車両存在時に求める入力濃度の時系列データを説明する図である。
【図８】車両が通過した時に車両の末尾座標を求める処理の一例を示す図である。
【図９】車両末尾座標の近傍をテンプレート画像に登録する処理を示した図である。
【図１０】車両を追跡する処理を説明する図である。
【図１１】カメラの空間位置を説明する図である。
【図１２】車種判別のフローを示す図である。
【図１３】カメラアングルと車両存在時間の関係を説明する図である。
【図１４】濃度変化情報個数の正規化を説明する図である。
【図１５】車種判別に用いる濃度変化，エッジ変化の判別処理フローを示す図である。
【図１６】濃度変化，エッジ変化をニューラルネットワークで処理し車種を判別する場合の構成を示す図である。
【図１７】小型車専用のニューラルネットワークによる車種判別の構成を示す図である。
【図１８】車種判別処理の全体処理を示す図である。
【図１９】車両存在確認処理の処理フローを示す図である。
【図２０】車両追跡時のテンプレート登録処理フローを示す図である。
【図２１】車両追跡時の管理テーブルの内容を示す図である。
【図２２】車両追跡処理フローを示す図である。
【図２３】通過速度算出フローを示す図である。
【図２４】車種判別処理フローを示す図である。
【図２５】渋滞時に対応した車両存在確認処理のフローを示す図である。
【図２６】対面通行の場合に、本発明を適用した図である。
【図２７】左車線の車両検知，通過，追跡などの時間経過を示す図である。
【図２８】右車線の車両検知，通過，追跡などの時間経過を示す図である。
【図２９】カメラの視野を真下に設定した態様をあらわした図である。
【図３０】カメラを横向きに設定した場合のカメラの視野を示す図である。
【符号の説明】
３…車両存在確認手段、４…濃度変化検出・記憶手段、５…エッジ変化検出・記憶手段、６…車両追跡手段、７…速度算出手段、８…車種判別手段、２０…画像処理装置。[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a vehicle type discrimination system that processes an image of a television camera and determines a vehicle type of a vehicle passing on a road.
[0002]
[Prior art]
As a vehicle type determination system for determining the type of a vehicle on a road, there is a system using an optical sensor.
[0003]
As a vehicle type discrimination system using the optical sensor, Japanese Patent Laid-Open No. 11-296784 discloses that a road is irradiated with light, and the height of a passing vehicle is obtained from a phase difference between the irradiated light and reflected light. It describes that vehicle types such as small, large, freight, and bus are distinguished from patterns of height change.
[0004]
Further, as an apparatus for determining the type of a vehicle on a road, there is an apparatus using an image sensor. Japanese Patent Application Laid-Open No. 11-353581 discloses a method of removing the density of a road surface from an input image, extracting a pattern of a vehicle, detecting an edge of the extracted vehicle pattern to determine a leading position and a width of the vehicle. It describes that a vehicle silhouette is extracted from a vehicle, and a large-sized passenger, a large-sized cargo, a small-sized passenger, and a small-sized cargo are discriminated based on the leading width of the vehicle and the vehicle silhouette.
[0005]
Japanese Patent Application Laid-Open No. H11-232588 describes a method in which a front luminance pattern for each vehicle type is stored in advance, and the vehicle type is determined by comparing with a luminance pattern of an input image.
[0006]
[Problems to be solved by the invention]
In a conventional vehicle type discriminating system using an optical sensor, other checking means such as a video camera are required to check the discrimination result.
[0007]
Also, when a video camera is installed, there are problems that it is necessary to match the detection area of the optical sensor and that the time of the sensor and the time of the video need to be accurately corrected.
[0008]
On the other hand, when the image sensor is used, it can be used for vehicle type determination and confirmation of the determination result, so that such a problem can be solved. In other words, verification is facilitated by processing the captured image to determine the vehicle type, superimposing the determination result on the camera video, and recording this video.
[0009]
However, Japanese Patent Application Laid-Open No. H11-353581, which is an apparatus using the image sensor, extracts the luminance pattern of the vehicle by removing the road surface density of the entire area of the input screen. If the brightness of the vehicle pattern to be extracted or the brightness of the road surface has large unevenness, it is difficult to accurately extract the vehicle pattern, and the accuracy of the vehicle type determination is reduced. Further, the difference in length of the vehicle width depending on the vehicle type is small, and it is difficult to determine the vehicle type based on the vehicle width. These problems become a serious problem when discriminating the type of vehicle according to the size such as large, normal, small, or the like, or when discriminating the type of vehicle whether it is cargo or passenger.
[0010]
Similarly, in the method described in Japanese Patent Application Laid-Open No. H11-232588, "speed violation control device", since the front luminance pattern of the vehicle is extracted from the entire image, the accuracy of vehicle type determination is reduced. This problem becomes a serious problem when discriminating the type of vehicle according to the size such as large, ordinary, small, or the like, or when discriminating the type of vehicle whether it is cargo or passenger.
[0011]
As described above, an object of the present invention is to improve the accuracy of vehicle type determination in an image monitoring device.
[0012]
[Means for Solving the Problems]
As described above, in order to determine the vehicle type of the vehicle with the image sensor, in the vehicle type determination system of the present invention, an area is set on the image, and the image of the area while the vehicle passes through the area is set. Time-series data of density (density of image brightness) and time-series data of edge density (density of edge brightness obtained by differentiating image brightness) of the area are stored, and the traveling speed of the vehicle and The method of determining the vehicle type from the time series data is adopted.
[0013]
That is, the length of the vehicle is measured from the passing time of the vehicle passing through the detection area (the time when the detection area is hidden by the vehicle) and the passing speed of the vehicle, and the type of the size of the vehicle is obtained. The vehicle type is determined by specifying the type of cargo, passenger, or the like from the time series data (pattern) and the edge density time series data (pattern). Normally, buses have the same color and uniform ceiling, so the first half of the image density time-series data is flat, and the edge density time-series data has strong edges in the second half. It becomes possible. The “concentration” in this specification is
It means “luminance density”.
[0014]
The following means are also provided as aspects of the present invention.
[0015]
(Means 1)
In a vehicle type discrimination system that processes a video of a television camera for photographing a road to determine a vehicle type, an image input unit that captures the video of the television camera into an image memory, and a detection area for detecting a vehicle on the input image is a vehicle in each lane. Detection area information management means provided on the approach side for storing vehicle presence information, input image density change pattern information, and edge density change pattern information of an image obtained by differentiating the input image, and timing when a vehicle passes through the detection area And the speed detection means for obtaining the passing speed by tracking the position of the vehicle and determining the vehicle type of the passing vehicle based on the information of the detection area information managing means and the passing speed information obtained by the speed detecting means. A vehicle type discriminating system comprising:
[0016]
(Means 2)
In the vehicle type discrimination system of the means 1, the vehicle presence / absence information of the detection area information management means determines the presence or absence of a vehicle in the detection area, and differentiates the input image average density and the input image in the detection area when the vehicle is present. A vehicle type discriminating system characterized by storing an average density of an image in a time series for the number of times of vehicle existence.
[0017]
(Means 3)
In the vehicle type discrimination system of the means 2, the vehicle presence / absence information of the detection area information management means determines the presence or absence of a vehicle in the detection area, and detects a fluctuation area by time difference processing of the detection area. A vehicle type discriminating system characterized by storing an average density of an input image in a detection area and an average density of a video obtained by differentiating the input image in a time series for the number of times of vehicle presence only when the input image fluctuates with time.
[0018]
(Means 4)
In the vehicle type discriminating system of the means 2, in order to determine the presence or absence of a vehicle in the detection area, a difference process between a background image stored in advance and the input image is executed in the detection area, and the difference image is determined by a predetermined method. A vehicle type discriminating system wherein a vehicle is determined to be present when an area when binarized by a threshold value is equal to or greater than a predetermined value, and is determined to be absent when the area is not greater than a predetermined value.
[0019]
(Means 5)
In the vehicle type discriminating system of the means 2, in order to determine the presence or absence of a vehicle in the detection area, a difference process between a background image stored in advance and the input image is executed in the detection area, and the average of the difference images is calculated. A vehicle type discriminating system for discriminating that a vehicle is present when the density is equal to or higher than a predetermined value and that the vehicle is not present when the density is equal to or lower than the predetermined value.
[0020]
(Means 6)
In the vehicle type discriminating system of the means 2, in order to discriminate the presence or absence of the vehicle in the detection area, a similarity is calculated in the detection area from a previously stored background image and the input image, and the similarity is determined. A vehicle type discriminating system for discriminating that the vehicle is present when the value is equal to or less than a predetermined value, and that the vehicle is not present when the value is equal to or more than the predetermined value.
[0021]
(Means 7)
In the vehicle type discrimination system of the means 2, the input image is differentiated in order to determine the presence or absence of the vehicle in the detection area, and the vehicle is present when the average density of the differential image is equal to or more than a predetermined value, A vehicle type discrimination system characterized by discriminating non-existence.
[0022]
(Means 8)
In the vehicle type discrimination system of the means 1, the vehicle type discrimination means calculates the length of the passing vehicle from the time passing through the detection area obtained from the vehicle presence information of the detection area information management means and the speed obtained by the speed detection means. And a shape discrimination means for discriminating the shape of the vehicle from the input image density change pattern and the edge density change pattern of the detection area information management means.
[0023]
(Means 9)
In the vehicle type discriminating system of the means 8, the shape discriminating means of the vehicle type discriminating means inputs the input image density change pattern and the edge density change pattern of the detection area information management means to the input layer of the neural network, and outputs cargo, buses, etc. A shape discrimination system using a three-layer neural network having an output corresponding to the shape of a vehicle.
[0024]
(Means 10)
In the vehicle type discriminating system of the means 8, the shape discriminating means of the vehicle type discriminating means stores several types of density change patterns and edge density change patterns for each vehicle shape in advance, and the input image density change pattern and the edge density change pattern of the detection area information management means. A vehicle type discrimination system, wherein a shape similarity is determined by calculating a degree of similarity with a pattern, and obtaining a pattern having the highest sum.
[0025]
(Means 11)
In a vehicle type discrimination system that processes a video of a TV camera that photographs a road and determines a vehicle type, an image input unit that captures a video of the TV camera into an image memory, a unit that sets a detection area for detecting a vehicle in the input image, Means for obtaining, from the image of the detection area, a vehicle length of a vehicle passing through the detection area, time-series data of image density of the detection area, and time-series data of edge density in the image of the detection area; A vehicle type discriminating system comprising: means for discriminating a vehicle type from series data, time series data of edge density, and a vehicle length.
[0026]
(Means 12)
In the vehicle type discrimination system of the means 11, the vehicle length of the vehicle passing through the detection area, the time-series data of the image density of the detection area, and the time-series data of the edge density in the image of the detection area are obtained from the image of the detection area. The means to ask is
Means for obtaining, from the image of the detection area, the speed of the vehicle passing through the detection area, time-series data of the image density of the detection area, and time-series data of the edge density in the image of the detection area; Means for determining the time at which the vehicle passes through the detection area from the data, means for determining the speed of the vehicle passing through the detection area from the time series data of the edge density, and the vehicle passing through the detection area from the obtained time and speed. Means for determining the vehicle length of the vehicle.
[0027]
(Means 13)
In the vehicle type discrimination system of the means 12, the means for determining the speed of the vehicle passing through the detection area from the time series data of the edge density includes means for specifying the end of the vehicle from the edge density, a change in the position of the end, A vehicle type discrimination system comprising: means for calculating a time required for a change in the position; and means for calculating a speed of the vehicle from an amount of change in the position and a time required for a change in the position.
[0028]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0029]
FIG. 1 is a diagram for explaining processing means of the vehicle type identification system of the present invention.
[0030]
Image input means 1 for inputting an image picked up by a video image pick-up means such as a camera, detection area setting means 2 for measuring vehicle information on the image, and measuring whether or not a vehicle exists in the detection area The vehicle presence confirmation means 3 and the density change detection / storage means 4 for obtaining the density information of the input image when the vehicle is present in the detection area. An edge change detection / storage means 5 for obtaining edge density information of an image inputted by the user, a vehicle tracking means 6 for tracking a vehicle passing through the detection area, a speed calculation means 7 for obtaining a passing speed of the vehicle from the tracking result, It comprises a vehicle type discriminating means 8 for discriminating a vehicle type by using data stored in the density change detecting / storing means 4 and the edge change detecting / storing means and the speed information obtained by the vehicle speed calculating means 7.
[0031]
FIG. 2 is a diagram illustrating a hardware configuration. An image processing device 20 used in the present invention is an A / D converter for converting an image of a television camera 21 into digital data.
22 and an image memory 23 for storing the converted image data and a D / A converter 25 for converting digital data for displaying the data in the image memory 23 as video into analog data for display on a monitor television 26. Configuration. The processing operation is controlled by a computer 29 (such as a personal computer) including a CPU 27 and a RAM 28.
[0032]
A plurality of image memories 23 are provided and used as a memory for storing background images and input images, a work memory for storing various image processing results, and the like.
[0033]
Note that the image processor 24 has a spatial product-sum operation for performing a differentiation process and the like, an inter-image operation for performing a difference process between two images, a histogram processing function for obtaining a frequency distribution of image densities, and the like. Functions such as normalized correlation processing for calculating similarity are provided.
[0034]
The operation of each unit in FIG. 1 will be described below.
[0035]
FIG. 3 shows an example of the input image. The detection area setting means 2 sets detection areas D1 and D2 in an image 10 of a two-lane road taken from a television camera and obtains data of these areas. Hereinafter, a case where only one left lane of FIG. 3 is processed will be described for simplicity.
[0036]
First, the detection area setting means 2 sets the detection area D1 on the side where the vehicle enters, as shown in FIG. 3, in the input image obtained from the image input means 1. The size of the detection area is set so that the length in the lane direction is approximately 1/10 (for example, approximately 50 cm) of the vehicle length of the ordinary vehicle, and the direction perpendicular to the lane is set to approximately the width of the lane. If the length in the lane direction is excessively widened, a case where a plurality of vehicles are simultaneously present in the detection area may occur, and the vehicle must be separated.
[0037]
Further, if the width is too narrow, the variation in image density and edge density increases, so that about 50 cm is set empirically to fit in the detection area.
[0038]
Further, if the camera is set in a field of view as shown in FIG. 3, the vehicles do not overlap at the lower part of the screen (approximately right below), so that the information of the detection area is information of one vehicle. .
[0039]
In this setting method, coordinates on the screen are set off-line and the coordinates are stored.
[0040]
In the present embodiment, the detection area is set directly. However, if the center white line is recognized by image processing and set to the left of the white line, automatic setting is also possible.
[0041]
Note that the distance between the detection areas D1 and D2 of each lane in FIG. 3 is adjusted each time according to the angle of view to be photographed, the road width, and the passing characteristics of the vehicle (they vary depending on the location such as frequently passing the left side of the lane). .
[0042]
The vehicle presence confirming means 3 measures whether or not a vehicle is present in the area D1 set by the detection area setting means 2 (the processing cycle is about 33 milliseconds).
[0043]
As a processing method, as shown in FIG. 4, an image of only the road is stored in advance 41, a difference process between the image 41 and the input image 42 is executed in the detection area, and the obtained difference image is determined by a predetermined threshold. The area at the time of binarization 43 with the value is obtained. If the area is smaller than a predetermined value, it is determined that the vehicle does not exist, and if the area is larger than the predetermined value, it is determined that the vehicle exists (background difference processing method).
[0044]
Alternatively, as shown in FIG. 5, there is a method of differentiating the detection area of the input image and determining that the vehicle does not exist if the average density of the differential image 51 is equal to or less than a predetermined value, and that the vehicle exists if the average density is equal to or more than the predetermined value.
[0045]
Further, as described in Japanese Patent Application Laid-Open No. H10-307988 “Traffic flow monitoring device”, a similarity (regularity) between a previously stored road background image pattern and an input image is used so as to cope with a change in the density of the input image. If the similarity is equal to or less than a predetermined value, it is possible to determine that a vehicle is present.
[0046]
In the normalized correlation processing, the similarity is obtained by matching the densities of the registered pattern and the search image pattern. Therefore, even if the brightness of the input image fluctuates, the similarity is not significantly affected. Therefore, in the case of outdoors, this processing is stable. In any case, the effect can be obtained by any method as long as the information on the presence or absence of the vehicle can be stored.
[0047]
Although it is difficult to accurately determine the area of the vehicle from the image of the entire screen, by limiting the detection area to a predetermined size, the presence / absence information of the vehicle can be stably determined.
[0048]
Such presence / absence information is stored in chronological order as shown in FIG. 6 (existence is set to 1 and non-existence is set to 0). At this time, since the total time of the existence is required, the existence time TimeS is obtained using the existence start time and the existence end time.
[0049]
The density change detection / storage means 4 obtains the average density in the detection area of the input image when the vehicle is present, using the result of the determination of the presence or absence of the vehicle, as shown in FIG. They are stored in chronological order. The number of stored data is the same as the number of data with and without the vehicle.
[0050]
Similarly, the edge change detection / storage means 5 calculates the edge strength of the input image when the vehicle is present, using the result of the determination of the presence or absence of the vehicle. Is differentiated, and the average density of the differential image is stored in a time series. The number to be stored is the same as the data of the vehicle presence / absence. The differentiation process includes a process of enhancing the edge of a horizontal stripe (the edge of a car has many horizontal lines) and a process of enhancing a vertical line. These time-series data are used for shape discrimination when a later-described passing speed is obtained.
[0051]
As described above, by setting the detection area to a predetermined size, obtaining the image density and the average density of the edge of the detection area, and using the average density of the edges for the process of determining the type of vehicle, the accuracy can be improved more than before.
[0052]
Next, the vehicle tracking means 6 will be described. As shown in FIG. 8, when the vehicle is determined to have passed through the detection area D1 (the moment when the vehicle is changed from being present to being absent by the vehicle presence confirmation means 3; this is referred to as time t), the vehicle tracking processing is performed in the detection area D1. A processing area 82 is provided in the upper part, and the end position of the vehicle is detected in this area. When the speed is obtained by image processing, the vehicle is in contact with the road surface in order to accurately measure the speed. For example, when a vehicle roof is tracked, the apparent speed increases, so here, the moment the vehicle passes through the detection area in order to detect the end coordinates of the vehicle. Position coordinates I want to ask. In the detection processing of the end coordinates, a differential process 82 is performed in the detection area, the differential image 83 is projected in the horizontal direction 84, a position Yo of the maximum projection distribution is obtained, and an image near the position Yo is shown in FIG. To the template image Tmp. Here, 80 in the figure indicates an input image, and 81 indicates a passing vehicle. As shown in FIG. 8, a place where the projection distribution is equal to or more than the predetermined threshold may be detected from the lower side of the detection area 82 instead of the maximum point.
[0053]
Next, in the tracking processing, the matching processing is executed using the input image at the time (t1) and the registered template image. As shown in FIG. 10, a search area 100 is set in order to search for a place that is most similar to the template image, and a template matching process (normalized correlation process) is executed within the search area 100, and the similarity is high. If the position can be detected, the image near the coordinates yp (t) is registered again in the template image, and this process is sequentially repeated (in FIG. 10, the coordinates of Yo move to yp (t1), yp (t2)). are doing). When the timing at which the vehicle leaves the screen or when yp (t) is higher than a predetermined position, the tracking is terminated and the coordinates are set to Ye (the process is also terminated when a position with a high similarity cannot be detected). The initial template registration coordinates Yo, end coordinates Ye, tracking start time time0 and tracking end time time1 obtained by such vehicle tracking processing are stored.
[0054]
The speed calculating means 7 for calculating the passing speed of the vehicle calculates the speed from the registered coordinates Yo and the tracking coordinates Ye in the vehicle tracking processing and the times time0 and time1. Here, the coordinates of the image are on a pixel-by-pixel basis, but it is necessary to convert between the pixel and the actual distance in order to determine the speed. Here, the conversion is performed by Expression 1.
[0055]
The camera orientation θ, the focal length F of the camera lens, and the camera height H are obtained in advance, and the coordinates (x, y) on the image can be converted to the real coordinates (X, Z). This calculation formula can be converted by Expression 1 by using the method described in “Discrimination of Vehicle Type of Passing Vehicle Based on Moving Object Detection”, IEICE Road Traffic Study Group Material (RTA-95-7, 95.3). Here, the relationship between the camera and the road is shown in FIG.
[0056]
(Equation 1)

[0057]
In this way, Yo is converted to Zo, and Ye is converted to Ze, and the speed V is obtained by Expression 2.
[0058]
[Equation 2]

[0059]
Here, the vehicle tracking process has been described with reference to a method described in Japanese Patent Application Laid-Open No. H10-307988 “Traffic flow monitoring device”. However, a method of extracting a vehicle pattern and obtaining a speed from a position change of the end coordinates thereof is used. Although various methods can be considered, the method is not limited as long as the method can measure the speed of the vehicle passing through the detection area. There are various other methods for converting the actual distance from the pixel, and the method is not limited to the above conversion formula.
[0060]
Next, the vehicle type determining means 8 will be described. The vehicle type determining means 8 operates as shown in FIG. First, the vehicle length is obtained. If the vehicle length is longer than LB, the vehicle becomes large. If it is determined that the vehicle is large, the shape is determined using the data stored in the image density change / edge change detection / storage means 4 and 5 (here. Then, when the vehicle length is shorter than LB, the size is reduced, and the shape is determined in the same manner (the distinction between the cargo and the passenger car). Here, the above example classifies small and large vehicles, but it is also possible to classify them into three types of small, medium and large vehicles depending on the vehicle length, and to classify these as cargo, passenger, etc., respectively. The measurement of the vehicle length can be calculated by vehicle length = speed V * existence time TimeS using the passing speed V of the vehicle and the existence time TimeS obtained by the vehicle existence confirmation means 3.
[0061]
Determining the vehicle length from the entire input image could not be performed accurately because it is difficult to separate the vehicles when they overlap, but in this way, the detection area is set and the vehicle passes through the detection area. If the speed and the existence time are obtained, the vehicle length can be accurately obtained.
[0062]
Since the field of view is set as shown in FIG. 3, the vehicle does not overlap at the lower part of the screen (immediately below the image), and the information of the detection area is information of one vehicle. Can be obtained accurately.
[0063]
By the way, since the relationship between the time and the vehicle length of the vehicle passing through the detection area from the direction of the camera is as shown in FIG. 13, the correction leads to an improvement in accuracy. That is, since (vehicle length + detection width) = speed V * existing time TimeS, the vehicle length = speed V * existing time TimeS−detection width is calculated. Note that the detection width is the length of the detection area in the lane direction.
[0064]
In the shape check based on the change in image density and the change in edge density, data stored as many times as the number of times of vehicle existence as shown in FIG. 14A is normalized to, for example, 10 data and converted as shown in FIG. 14B. This processing is performed to normalize in order to avoid complicating the discrimination processing because the number of data greatly changes depending on the vehicle length and the speed. The normalization method may be a process of simply thinning out n (variable depending on vehicle length and transit time) data to ten, or a more accurate compression process with interpolation.
[0065]
Assuming that the normalized image density data is Gray [0] to Gray [9], in the case of a bus, since there is a window at the rear as shown in FIG. 15, the density value decreases per Gray [7] to Gray [9]. Further, since the density of the ceiling is almost uniform, the density per Gray [0] to Gray [6] becomes uniform. For this reason, it is determined whether or not the first half of the time series data of the image density is uniform.
[0066]
When the vehicle length is determined to be large, the vehicle is determined to be a bus under the following determination conditions.
(1) Gray [7] to Gray [9] have the minimum density (S86-1 in FIG. 15).
(2) The accumulation of the absolute value of the difference from the average density Ave of Gray [0] to Gray [6] is equal to or less than a predetermined value (S86-2).
[0067]
Further, the information obtained by the edge change detection / storage means 5 is similarly normalized to ten pieces of data and stored in Edge [0] to Edge [9]. In the case of a bus, a large edge density occurs first at the head of the vehicle, and a large edge (density) due to a window occurs at the last position. On the other hand, in a truck or the like, this information is generated because there is a large edge density between the driver's seat and the carrier on the way.
[0068]
Therefore, in the case of a bus, it is determined under the following conditions.
(1) The average density of Edge [5] to Edge [9] is higher than the average of Edge [0] to Edge [4], or the central edge density is uniform (S86-3).
[0069]
Similarly, if the vehicle length is shorter than LB, the vehicle is determined to be a small vehicle, but it is determined whether the vehicle is a cargo or a passenger by checking density change and edge density change. In sedan etc., large edge density occurs in the window area, but in small cargo, the edge density is small in the middle. From this difference, small passenger cars and small cargo can be distinguished.
[0070]
As described above, by using the vehicle length, the image density time series data, and the edge density time series data of the vehicle passing through the vehicle detection area, it is possible to accurately classify various shapes. In the method of storing and classifying two-dimensional image patterns, it is very difficult to accurately extract the shape of the vehicle in addition to the problem of storage capacity. Shape determination is possible only by obtaining the density and edge density in a time series.
[0071]
The vehicle type determination means can set and determine various threshold values as described above, but there is use of a neural network as a more stable determination process. FIG. 16 shows the structure of a network when a neural network is used. The network is a neural network having a three-layer structure. Gray [0] to Gray [9] whose image densities are normalized to 10 (for example) are input to the input layer of the network. Further, Edge [9] to Edge [9] whose edge densities are normalized to 10 are input. Further, a vehicle length is input. The output layer prepares an output for each shape. Here, since the data range handled by the neural network is a real number from 0 to 1, Gray [i] is divided by the maximum value of the image density, and Edge [i] is divided by 0 from the maximum value of the edge density. It is necessary to convert to a real number of 1. For the vehicle length, for example, a value obtained by dividing the vehicle length by 15 m with 15 m as the maximum (1 or more is set to 1).
[0072]
Such input data is input, and at the time of learning, learning is performed by giving a value for each vehicle type as teacher data. For example, when discriminating a vehicle type into five types (small passenger, small cargo, large cargo, bus, etc.), when data of small passenger is input, small passenger is set to 1 and others to 0 as output teacher data. give.
[0073]
At the time of recognition, the output for each vehicle type is obtained, and the vehicle type having the maximum output is used as the recognition result. There is also a method of preparing two types of small and large neural networks. FIG. 17 shows an example of a small-sized vehicle. The input is density and edge information excluding the vehicle length shown in FIG. 16, and the output is small-sized passenger and small-sized cargo. By dividing the large and small neural networks in this way, it is possible to prevent at least an error in determining a large vehicle despite being the length of a small vehicle. In FIGS. 16 and 17, the edge density is information obtained by chronologically calculating the average density of the differential image in which vertical edge enhancement is performed. However, it is sometimes necessary to input information obtained by adding the horizontal edge density. The same processing can be used (Edge [0] to Edge [9] are vertical edge enhancement information, and Edge [10] is
If Edge [19] is horizontal edge enhancement information, 30 input layers may be provided together with the density information).
[0074]
As described above, when the neural network is used, it is not necessary to set a threshold value or the like for determining the image density and the edge density, and the vehicle type can be determined very easily. Further, when both the horizontal and vertical edge densities are used, the number of data increases and it is not easy to create a vehicle type discriminating process programmatically, but by using a neural network, the number of time series patterns used for discrimination increases. This also has the advantage that the vehicle type can be determined with high accuracy.
[0075]
Further, the time series data of the image density for each shape is stored in advance as a pattern, and the time series data of the edge density is stored in the number of shapes (for example, a bus or a large cargo), and the degree of similarity with the change information obtained from the image is determined. , For each edge density, and a pattern having the maximum sum of these similarities can be used as the recognition result. As for the storage capacity, only time-series data is prepared for the number of classification shapes, and very small data may be used.
[0076]
In any case, by obtaining the time series data of the image density, the time series data of the edge density, and the vehicle length while the vehicle is passing through the detection area, not only the conventional large and small size discrimination, but also It is also possible to determine the shape of the vehicle.
[0077]
The flow of the above processing will be described in detail. FIG. 18 shows the overall flow. First, initialization of the image processing apparatus and initialization of various tables are executed S0. S1 in which a detection area setting means sets a vehicle detection area. S2 for creating a background image serving as a base for determining the presence of the vehicle. The following is the processing loop.
[0078]
An image is input S3, and a vehicle presence confirmation S4 confirms whether a vehicle exists in the detection area. Further, information on a change in density and a change in edge in the detection area is obtained. If the vehicle has left the detection area, the end area of the vehicle is registered in the template image in S5.
S5. A place similar to the registered template image is searched from the input image and tracking processing is performed S6. When the tracking is completed, the speed of the vehicle is calculated S7. If all the information has been obtained, the vehicle type determination S8 is executed. Hereinafter, a detailed flow of each process will be described.
[0079]
FIG. 19 shows the flow of the vehicle presence confirmation process S4. First, a difference process S4-1 between the background image and the input image is executed, the image is binarized S4-2, and a binary area is obtained S4-3. If the area is equal to or larger than the predetermined value, it is determined that a vehicle is present, and the count is increased (S4-5). Further, the average density of the input image and the average density of the differential image in the detection area are obtained and stored in a table (S4-6 to S4-9). When both horizontal and vertical information are used in the differentiation process, the array is prepared twice.
[0080]
If the area is small, it is determined that the vehicle does not exist. However, if the Count (the number of times) is equal to or less than a predetermined value, it is determined that the presence or absence of the vehicle has changed due to noise, and the Count is cleared. If the Count is equal to or more than the predetermined value, it is determined that the vehicle has passed, and the detection of the vehicle end coordinates S4-11 is executed (the processing in FIGS. 8 and 9). , Yc (Y direction center coordinates). Also, S4-12 and S13 save the time series data of the density and the edge density stored in S4-7 and S4-9. This is to prevent data from being erased when the next vehicle enters. If the field of view is set as shown in FIG. 3 for vehicle type discrimination, only two vehicles (one lane) can enter the screen, so this processing is sufficient. The transit time is stored in S4-14. The number of passing vehicles can be calculated by obtaining the number of times when it is determined that the vehicle has passed.
[0081]
FIG. 20 shows a flow of template registration. When the vehicle passes, the end coordinates of the vehicle are obtained. When the coordinates xc and yc have been obtained, the registration processing of the template image is executed. Since a plurality of templates can be registered (multi-lane correspondence), as shown in FIG. 20, an unused number is searched (unused when Track [i]... Hold is 0). The predetermined information is stored in the table. The stored data is the X center and the Y center (xc, yc) of the tracking coordinates, their initial values (oxc, oyc), the number of times of tracking (count), the state of the table (hold = 2), and the time (starttime, endtime). . Further, an image having a predetermined X width and Y width from the input image centering on xc and yc is registered in the template.
[0082]
Here, the registration data is a management table as shown in FIG.
[0083]
Next, FIG. 22 shows a tracking processing flow. If the table number starts from 0 and the hold of the management table (Track [i]) is 2, since the template has just been registered at the current time, the hold is set to 1 and the process waits until the next input image S6-3. If the hold is 1, since a new image has been input after registration, this data is tracked. First, Track [i]. S6-5 for setting a search area (search area) based on the values of xc and yc. This is set as 100 in FIG. When the search area is determined, a position similar to the most registered template image in the area is searched (S6-6). If the position is found, xc, yc (oxc, oyc are not updated) and time (endtime) of the management table are updated, and S6-9, and the template image of the registered image is updated to an image near the position of xc, yc. S6-10. On the other hand, if the registered pattern is not found, the hold is set to 3 and the tracking ends (S6-8). This processing is looped for the number of tables. Usually, in the case of two lanes, it is sufficient if there are about four tables.
[0084]
The storage number of the template image is the same as the management table number i.
[0085]
Next, FIG. 23 shows a speed calculation flow. Since the hold is set to 3 at the end of the vehicle tracking as described above, the speed is calculated from the coordinates and the time stored in the table in which the hold is 3 (S7-3). After calculating the speed, the data in the management table is initialized (S7-4).
[0086]
FIG. 24 shows the flow of the vehicle type determination S8. When the speed is obtained by the vehicle tracking process, a vehicle length is obtained from the speed and the existence time TimeS, and the vehicle type is determined based on the obtained vehicle length in S8-3. If the speed is not determined, nothing is executed because the vehicle has not passed. A specific example of the vehicle type determination is as described with reference to FIGS. If the speed is high after the vehicle passes through the detection area and the speed of the vehicle is obtained, the speed is obtained after several tracking processes, for example, 33 milliseconds × 10 times = 330 milliseconds. Although it is possible to determine the vehicle type instantaneously later, in the case of a slow car, the speed is not calculated until the vehicle passes through the screen, so that the speed may be obtained several tens of seconds later. For this reason, when the number of times of vehicle tracking becomes equal to or more than a predetermined value (for example, 20 times), the speed may be obtained and the vehicle type may be determined. In this case, after the update S6-9 of the management table in FIG. 22, the number of tracking times Track [i]. If the count is equal to or more than a predetermined value, the hold may be set to 3 and the speed may be obtained.
[0087]
In the above description, the case where the number of lanes is one has been described. However, when the number of lanes is two or three, if each process is looped for the number of lanes and each table is stored in an array for the number of lanes. Multi-lane support is easily possible.
[0088]
By the way, the above-mentioned vehicle type discrimination method has a problem when traffic jams. That is, when traffic is congested, the vehicle stops in the detection area, and the vehicle existence time is greatly extended. Since this vehicle eventually disappears from the screen, it moves at a certain speed. However, since the process of calculating the vehicle length is obtained from the existence time and the speed, the vehicle length becomes very large.
[0089]
Therefore, the processing for measuring the existence time is improved as shown in FIG. Only when the vehicle is moving, it is counted as the existence time (determined as existence as existence confirmation processing, but not counted as a storage counter), so that the input image f ( t) and the difference processing between the images of the input image f (t-dt) before the predetermined time (t-dt) is executed, S4-16, and the area when the difference image is binarized with the predetermined threshold value is calculated. calculate. If the area is equal to or smaller than the predetermined value, it is determined that the area has not moved, and the existence counter is not updated at S4-17.
[0090]
With this processing, the vehicle length can be measured because the existence time is almost correctly obtained even in a traffic jam. When the moving speed V accelerates or decelerates, an error occurs in the vehicle length. However, the error can be reduced by setting the speed calculation range near the detection area.
[0091]
Also, the stored background image needs to be updated, but it can be handled by averaging the image when there is no general moving object in image processing (new background image = average of old background image and input image) (The processing is stable because it is a narrow range of only the detection area).
[0092]
Although an example in which pattern matching processing is used in the vehicle tracking processing for speed calculation is shown, other speed calculation methods can be used. In addition, in the presence check processing, a method of determining based on the similarity between the background image and the input image is stable, but a method of monitoring the average density of the background difference may be considered.
[0093]
In addition, in the state where only the tail lamp is visible at night, the effect of the present invention does not occur (the vehicle length is not determined). is necessary. Processing in other places is not a problem because the tail lamp can be detected and tracked to determine the speed. In the future, the sensitivity of the camera will improve, and it may be unnecessary to use the lighting even at night, but it is necessary now.
[0094]
In the above example, when the vehicle moves in the same direction, particularly, the vehicle is photographed from behind, and the number, speed, and vehicle type are determined.
[0095]
However, there are many places where one-lane roads on one side are face-to-face. In this case, as shown in FIG. 26, the detection area is set on the lower side D1 of the screen in the left lane and on the upper side D2 of the screen as shown in FIG. 26, and the vehicle is tracked (101 and 102 indicate tracking patterns). Here, as shown in FIG. 27, the lapse of time such as vehicle detection, passage, and tracking in the left lane starts the vehicle tracking processing from the time when the vehicle passes through the detection area, and after the tracking is completed, speed calculation and vehicle type determination are performed. . On the other hand, in the case of the right lane, vehicle tracking starts at the moment when the vehicle enters the detection area as shown in FIG. 28, and after the tracking is completed, speed calculation and vehicle type determination are performed. This means that if the vehicle pattern is tracked immediately after passing the detection area, it will track the roof of the vehicle, etc., which will affect the vehicle height and reduce the speed measurement accuracy. This is because they need to be tracked. Therefore, when the vehicle enters the detection area, the pattern on the front of the vehicle is extracted and tracked. In this way, by changing the speed tracking processing method, it is possible to cope with face-to-face traffic.
[0096]
By the way, as shown in FIG. 29, the field of view of the camera is photographed directly below and the detection area is set at the center of the screen at D1 and D2. This makes it possible to determine the number, speed, and vehicle type of the up and down lanes by the same processing. That is, it is possible to obtain the speed of the vehicle as well as going up and down after passing through the detection area. Further, since a general television camera has a horizontal / vertical ratio of 4: 3, it is more preferable to install the camera horizontally and take a picture since a long field of view can be taken in the lane direction as shown in FIG.
[0097]
In the case of ascending or descending two-way traffic, it is effective to point the camera directly below. This is very simple in installation, and has an effect that the recognition processing can be made the same.
[0098]
In addition, the road is photographed with a camera or the like, and a vehicle detection area is provided in the image, and the presence / absence information of vehicles passing through this area, time series information of density, time series data of edge density, and passing speed are obtained. Based on these information, it is possible to determine whether the vehicle is large or small based on the vehicle length, and to determine the shape of a cargo or a bus from the time series data of density and edge density.
[0099]
Further, it is very difficult to cut out a vehicle from one image and determine the type of the vehicle. However, since the temporal change of the vehicle passing through the detection area is used, the characteristics of the vehicle can be accurately extracted, so that the type of the vehicle can be easily determined. Is possible.
[0100]
In addition, since the image processing is used, verification of the recognition result is easy, and accurate vehicle type determination is possible because the determination is performed based on the vehicle length. Depending on the set value of the vehicle length, classification into large, medium, and small is also possible.
[0101]
【The invention's effect】
ADVANTAGE OF THE INVENTION According to this invention, the accuracy of vehicle type discrimination in a vehicle type discrimination apparatus can be improved.
[Brief description of the drawings]
FIG. 1 is a diagram showing a configuration of a processing unit of the present invention.
FIG. 2 is a diagram illustrating an example of image processing hardware used in the present invention.
FIG. 3 is a diagram illustrating a photographing example of an image sensor.
FIG. 4 is a diagram illustrating an example of a process of determining the presence or absence of a vehicle;
FIG. 5 is a diagram showing an example in which a differentiation process is used in the process of determining the presence or absence of a vehicle.
FIG. 6 is a diagram illustrating a case where a state of presence / absence of a vehicle is stored.
FIG. 7 is a diagram illustrating time-series data of an input density obtained when a vehicle is present.
FIG. 8 is a diagram illustrating an example of a process of obtaining the end coordinates of the vehicle when the vehicle has passed.
FIG. 9 is a diagram illustrating a process of registering the vicinity of the vehicle end coordinates in a template image.
FIG. 10 is a diagram illustrating a process of tracking a vehicle.
FIG. 11 is a diagram illustrating a spatial position of a camera.
FIG. 12 is a diagram illustrating a flow of vehicle type determination.
FIG. 13 is a diagram illustrating a relationship between a camera angle and a vehicle existence time.
FIG. 14 is a diagram illustrating normalization of the number of density change information.
FIG. 15 is a diagram illustrating a flow of a process of determining a density change and an edge change used for vehicle type determination.
FIG. 16 is a diagram showing a configuration in a case where a density change and an edge change are processed by a neural network to determine a vehicle type.
FIG. 17 is a diagram showing a configuration of vehicle type discrimination by a neural network dedicated to a small vehicle.
FIG. 18 is a diagram illustrating an overall process of a vehicle type determination process.
FIG. 19 is a diagram showing a processing flow of a vehicle presence confirmation processing.
FIG. 20 is a diagram showing a template registration processing flow during vehicle tracking.
FIG. 21 is a diagram showing the contents of a management table during vehicle tracking.
FIG. 22 is a diagram showing a vehicle tracking processing flow.
FIG. 23 is a diagram illustrating a flow of calculating a passing speed.
FIG. 24 is a diagram showing a vehicle type determination processing flow.
FIG. 25 is a diagram showing a flow of a vehicle presence confirmation process corresponding to a traffic jam.
FIG. 26 is a diagram to which the present invention is applied in the case of two-way traffic.
FIG. 27 is a diagram showing a lapse of time such as vehicle detection, passage, and tracking in the left lane.
FIG. 28 is a diagram showing a lapse of time such as vehicle detection, passage, and tracking in the right lane.
FIG. 29 is a diagram showing an aspect in which the field of view of the camera is set right below.
FIG. 30 is a diagram illustrating a field of view of the camera when the camera is set to be in a horizontal position.
[Explanation of symbols]
Reference numeral 3 denotes a vehicle presence confirmation unit, 4 denotes a density change detection / storage unit, 5 denotes an edge change detection / storage unit, 6 denotes a vehicle tracking unit, 7 denotes a speed calculation unit, 8 denotes a vehicle type determination unit, and 20 denotes an image processing device.

Claims (3)

In a vehicle type discrimination system that processes images of a television camera that photographs a road and determines a vehicle type,
Image input means for capturing the image of the television camera into the image memory,
Means for setting a detection area for detecting a vehicle in the input image;
Detecting means for obtaining, from the image of the detection area, time-series data of the image density of the detection area and time-series data of the edge density in the image of the detection area;
Vehicle speed measurement means for determining the speed of a vehicle passing through the detection area;
Vehicle type identification system comprising: the time-series data of the image density, the time-series data of the edge density, and vehicle type discriminating means for discriminating the vehicle type from said vehicle speed. In claim 1,
The detection unit includes, from the image of the detection region, a unit that obtains time-series data of density of an image in which a vehicle is present in the detection region, and a unit that obtains time-series data of edge density in the image.
The vehicle type determination means obtains a time when the vehicle passes through the detection area from the time-series data of the image density in the detection area,
A vehicle type discrimination system wherein a length of a vehicle passing through a detection area is obtained from the passing time and the speed obtained by the vehicle speed measuring means . In claim 2,
The vehicle speed measurement means,
Means for identifying the end of the vehicle from the edge density,
Means for calculating the passing speed of the vehicle from the amount of change in the position of the tail determined by tracking the detected tail .

Images