JP2004173108A - Database generation method, check support system, and program for check support of sensor information - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To record sensor information such as sound automatically arranging. <P>SOLUTION: A check support system 2 comprises a means 9 which detects a terminal for working 2 which acquires sensor information that is information representing a target status including at least video image information, and variation in the video image information, divides the sensor information other than the video image information based on the variation, and records them in the data base 3. <P>COPYRIGHT: (C)2004,JPO

Description

【００５２】
ここで、ｃ（ｋ，ｘ，ｙ）はｋ番目のフレーム画像における位置ｘ，ｙの画素の色値を表す。ｄｉｓｔ（ ）は色値のユークリッド空間での距離である。色空間として、例えば本実施形態では、最も標準的なＲＧＢ空間（０≦Ｒ，Ｇ，Ｂ≦２５５）を採用する。但し、色空間は上記ＲＧＢ空間に限定されるものではなく、例えば撮影環境や検出対象物が予め想定できる場合は、色空間を適宜工夫することで感度を上げることも可能である。
【００５３】
次に、過去ｎフレームの変化量ｄ_ｋの平均μと標準偏差σを計算する。そして、以下の条件式に基づいてｋ番目のフレームの映像状態が「活性」「沈静」「定常」のいずれにあるかを判定する。
【００５４】
【数２】
もし ｄ_ｋ＞μ＋ασ ならば 「活性」状態と判定する。
【数３】
もし ｄ_ｋ＜μ−βσ ならば 「沈静」状態と判定する。
【数４】
「活性」および「沈静」の何れの状態にも該当しなければ「定常」状態と判定する。
【００５５】
αおよびβは「定常」状態の範囲を指定するためのパラメータである（α＞０，β＞０）。そして、現在の映像状態とは異なる状態のフレームが過去ｎフレームの内にｍ（ｍ≦ｎ）回出現した場合に、映像の状態が変わったと判定する。これらパラメータ（ｎ，ｍ）は状態変化の恒常性に影響する。
【００５６】
ここで、巡視作業は、設備２１，２１間の移動と、移動せずに各設備２１を点検する作業に分けられる。「沈静」状態が検出された時点（「沈静」以外の状態から「沈静」に遷移したと判定された時点）とは、移動から点検に移った時点と推定される。
【００５７】
そこで、例えば本実施形態では、映像情報と音情報とを連続的に記録するようにし、且つ映像情報において「沈静」以外の状態から「沈静」に遷移したと判定された時点で、音情報を区切るようにしている。これにより、センサ情報を点検対象毎に自動で整理することが可能になる。従って、収集したセンサ情報を後で活用できるように、録音テープなどの記録媒体の録音箇所を示す情報（カウンタなど）を作業者が一々記録しておくといった手間のかかる作業は不要となる。
【００５８】
ここで、本実施形態では、区切られたセンサ情報を「センサ情報ブロック」と呼び、各「センサ情報ブロック」に一意に識別できる識別番号ｉｄを付与するようにしている。そして、識別番号ｉｄと、センサ情報ブロックの先頭箇所を表す情報（例えば区切時点ｔ）との対応表を作成するようにしている。従って、上記対応表により、識別番号ｉｄから対応するセンサ情報ブロックを呼出すことができる。さらに、区切時点ｔにおける映像を静止画像として記録し、その静止画像の識別番号ＩＤと、識別番号ｉｄまたは区切時点ｔとの対応表を作成するようにしている。従って、上記対応表により、センサ情報ブロックの先頭箇所に対応する静止画像も呼出すことができる。尚、補足的な機能として、映像そのもの（動画像）をデータベース３に記録するようにしても良い。この場合は、例えば記録映像における区切時点ｔのフレームｆｔと、識別番号ｉｄまたは区切時点ｔとの対応表を作成するようにする。
【００５９】
記録映像は全撮影時間にわたって同じファイルに記録しても良く、各区切時点ｔで区切って別々のファイルに記録しても良い。また、音情報も全録音時間にわたって同じファイルに記録し当該ファイルに区切り情報（タグ）を挿入するようにしても良く、各区切時点ｔで区切って別々のファイルに記録しても良い。センサ情報のファイル構成を如何にするかは、計算機の性能および利用環境等によって適宜選択して良い。尚、映像情報において「沈静」以外の状態から「沈静」状態が判定された時点で、作業用端末２において、録音を開始し、映像情報において「沈静」状態から「沈静」以外の状態が判定された時点で、録音を停止するようにしても良い。
【００６０】
さらに、例えば本実施形態では、点検対象としての設備２１を識別する情報（例えば点検対象を一意に識別するための番号である。以下、本実施形態では「対象識別情報」と呼ぶ。）を、区切られたセンサ情報であるセンサ情報ブロックのそれぞれに関連付けてデータベース３に記録するようにしている。このために、本実施形態の点検支援システム１では、設備２１を識別する手段と、対象識別情報をセンサ情報ブロックのそれぞれに関連付けて記録する対象識別情報付与手段１０を有している。点検支援用プログラムは、対象識別情報付与手段１０としてコンピュータを機能させる。尚、本実施形態における当該コンピュータは例えばデータベース３である。したがって、センサ情報ブロックがどの設備２１に属するかを対象識別情報により特定することができ、設備２１毎にセンサ情報を整理することができる。
【００６１】
設備２１を識別する手段としては、必ずしも限定されないが、例えば以下の手段が挙げられる。第１の手段としては、点検作業員が作業用端末２の入力装置８を用いて、現在取得されているセンサ情報がどの設備２１に関するものであるか示す対象識別情報を入力する。第２の手段としては、ＲＦＩＤ（Ｒａｄｉｏ Ｆｒｅｑｕｅｎｃｙ ＩＤ，非接触式ＩＣチップとも呼ばれる）などの設備識別用ＩＣチップを各設備２１に装着し、当該ＩＣチップとの通信用アンテナを作業用端末２に装備し、作業用端末２により当該ＩＣチップに記録された対象識別情報を自動で読み取る。第３の手段としては、バーコードや２次元バーコード或いは情報識別用マーカ（例えば特開２００２−５６３７１号公報参考）などを各設備２１に添付しておき、それを映像情報から読み取って設備２１を自動認識し、対応する対象識別情報を特定する。第４の手段としては、現在撮影されている画像情報と、過去に登録された既知の設備２１の画像情報とを照合し、画像検索により設備２１を自動認識し（例えば特開２００１−１０９８８４号公報参考）、対応する対象識別情報を特定する。設備２１の見た目が異なる場合に特に有効である。第５の手段としては、ＧＰＳ（衛星航法システム）などの位置情報システムを利用して作業用端末２の位置を特定し、特定された位置に基づいて現在点検対象となっている設備２１を自動認識し、対応する対象識別情報を特定する。第６の手段としては、音声認識手段を作業用端末２に装備し、点検作業員が設備名を発声することで、当該設備名から対応する対象識別情報を特定する。
【００６２】
さらに、例えば本実施形態では、対象毎に整理された各対象の状態を表すセンサ情報の集合に対し、個別のセンサ情報からそれぞれ周波数分布を求めて、当該求めた各周波数分布について、周波数分布方向に連続するｎ個の周波数成分を組み合わせて構成される周波数ｎグラムをすべて抽出し、さらに同一の対象に属するセンサ情報の集合について、共通する周波数ｎグラムの頻度を求めて、周波数ｎグラムの種別とその頻度との組の集合である周波数ｎグラムヒストグラムを得て、周波数ｎグラムヒストグラムを索引とし（図２のステップ３）、異なる対象間で共通する周波数ｎグラムの頻度に基づいて、ある対象と類似する対象を検索できるようにしている（図２のステップ４）。
【００６３】
このために、本実施形態の点検支援システム１では、対象毎に整理された各対象の状態を表すセンサ情報の集合に対し、個別のセンサ情報からそれぞれ周波数分布を求めて、当該求めた各周波数分布について、周波数分布方向に連続するｎ個の周波数成分を組み合わせて構成される周波数ｎグラムをすべて抽出し、さらに同一の対象に属するセンサ情報の集合について、共通する周波数ｎグラムの頻度を求めて、周波数ｎグラムの種別とその頻度との組の集合である周波数ｎグラムヒストグラムを得て、上記周波数ｎグラムヒストグラムを対応する対象の索引としてデータベース３に記録する対象毎索引作成手段１１と、異なる対象間で共通する周波数ｎグラムの頻度に基づいて、ある対象と類似する対象を検索する類似対象検索手段１２とを有するようにしている。点検支援用プログラムは、対象毎索引作成手段１１と、類似対象検索手段１２としてコンピュータを機能させる。尚、本実施形態における当該コンピュータは例えばデータベース３である。
【００６４】
ある設備２１に関して収録された特定の種別の個別センサ情報ｓのすべての集まりをセンサ情報集合Ｓとする。例えばある変圧器の１年分の音情報などがセンサ情報集合Ｓに該当する。個別センサ情報ｓは、例えば上述した処理により得られるセンサ情報ブロックである。
【００６５】
センサ情報集合Ｓに属するすべての個別センサ情報ｓに対して、例えばＦＦＴ（高速フーリエ変換）を適用し、周波数分布情報ｈに変換する。センサ情報集合Ｓに対応する、周波数分布情報ｈの集まりを周波数分布情報集合Ｈとする。尚、ＦＦＴのパラメータである信号処理時区間、周波数分解能、各周波数帯域の信号分解能は、目的に応じて設定するものとする。これらのパラメータは、同じ周波数分布情報集合Ｈに属するすべての周波数分布情報ｈで、同一とすることが好ましい。
【００６６】
ここで、一つの個別センサ情報ｓから一つの周波数分布情報ｈを算出しても良いが、例えばフーリエ変換の対象とする時区間をずらすことで、一つの個別センサ情報ｓから一定時間間隔毎に複数の周波数分布情報ｈ’を算出することもできる。そこで、一つの個別センサ情報ｓから算出された複数の周波数分布情報ｈ’について、統計量（例えば、同じ周波数帯域あたりでの平均を求めたもの）を算出し、当該統計量を周波数分布情報ｈとして用いても良い。
【００６７】
周波数分布情報ｈの周波数帯域数をｍとし、低周波帯域からｋ番目の帯域のパワーをｐ_ｋ（１≦ｋ≦ｍ）とする。（ｋ，ｐ_ｋ）というベクトルを単特徴ｓｋとする。そして、１＜ｋ≦ｍなるｋに対して定義される（ｓ_ｋ−１，ｓ_ｋ）の組を周波数バイグラムとする。同様に、１＜ｋ＜ｍなるｋに対して定義される（ｓ_ｋ−１，ｓ_ｋ，ｓ_ｋ＋１）の組を周波数トリグラムとする。同様に、連続するｎ個の単特徴の組を周波数ｎグラムとする。ここでｎを、周波数ｎグラムの次元数と呼ぶ。類似性の高いセンサ情報を精度良く検索するためには、次元数の大きな周波数ｎグラムを用いることが好ましい。ただし、周波数ｎグラムの次元数をあまり大きくすると、信号の細かな特徴が現れ過ぎるため、検索に求められる曖昧さが達成できない場合がある。その場合は、より低い次元数の周波数ｎグラムを用いる必要がある。本願発明者等が種々実験検討した結果、一般に回転機設備音の場合は周波数トリグラムを用いることで精度良い検索が実現できることが知見された。そこで、本実施形態では、次元数３の周波数トリグラムを用いた例について特に説明する。尚、他の次元数の場合も同様の処理となる。
【００６８】
単特徴ｓ_ｋ＝（ｋ，ｐ_ｋ）のｐ_ｋは、検索精度に応じた量子化がなされているものとする。量子化数が多いほど細かなパワー特徴が反映され、量子化数が少ないほど暖昧さを加味した特徴となる。
【００６９】
ある周波数分布情報集合Ｈに属する周波数分布情報ｈから計算されたすべての周波数トリグラムの集まりを周波数トリグラム集合Ｔとする。そして、各周波数トリグラムに一意の識別番号ｔｉｄを付与する。ただし、共通する周波数トリグラムの識別番号ｔｉｄは等しいものとする。ここで、周波数トリグラムが共通するとは、例えば本実施形態では、構成要素である三つの単特徴の位置と値とがすべて等しい場合をいうものとする。ただし、周波数トリグラムの共通性を緩やかにするために、例えば三つの単特徴のうちの二つについて位置と値が等しい場合に、「周波数トリグラムが共通する」と定義しても良い。
【００７０】
そして、周波数トリグラム集合Ｔ内における共通する周波数トリグラムの数をｆＴとする。周波数トリグラム集合Ｔに属する異なるすべての周波数トリグラムに対して、（ｔｉｄ，ｆＴ）の組をすべて計算し、それらの組の集合を周波数トリグラムヒストグラムＴｆとする。
【００７１】
計算された周波数トリグラムヒストグラムＴｆは、類似設備検索用の索引として、各設備２１毎に作成され、データベース３に記録される。索引の構成方法は、特に限定はされないが、Ｒ−Ｔｒｅｅなどの一般的な多次元索引を利用するのが適切である。周波数帯域数とパワーの量子化数とを一定に押さえることができるので、予めテーブルの規模を定めた三段階のツリー構造を用いるのが効果的である。例えば、周波数トリグラムの場合では、三つの単特徴のうちの一番目の単特徴を用いて第一の索引を構成し、当該第一の索引の参照する領域に、二番目の単特徴を用いて第二の索引を構成し、当該第二の索引の参照する領域に、三番目の単特徴を用いて第三の索引を構成し、単特徴の個数分のツリーを構成する。最後のリーフには、その周波数トリグラムの頻度ｆｔと、上述した対象識別情報を記録する。このように索引を構成することで、一つの周波数トリグラムの検索を、せいぜい次元数回の探索で行うことができる。また、周波数トリグラムの分布には大きな偏りが存在し得るため、当該偏りに応じてデータベース３の外部記憶装置２０内部の索引配置を考慮することも有効である。また、単特徴に関する索引をデータベース３の主記憶装置上に保有することも近年の計算機性能を考慮すれば可能であり、高速な探索を助ける。さらに、検索では頻出する周波数トリグラムが偏る傾向も在り得るため、キャッシュ機構を取り入れることで、さらに効率的な索引を構成することができる。
【００７２】
ある点検対象に関して作成された周波数トリグラムヒストグラムＴｆ１と、別の点検対象に関して作成された周波数トリグラムヒストグラムＴｆ２との類似性は、例えば以下のようにして計算できる。即ち、Ｔｆ１，Ｔｆ２の双方に属する周波数トリグラムの識別番号ｔｉｄについて、Ｔｆ１，Ｔｆ２におけるその周波数トリグラムの頻度をｆＴ１，ｆＴ２とする。ｆＴ１，ｆＴ２の二つの値のうちの最小値をｍｉｎｆＴとする。そして、Ｔｆ１，Ｔｆ２の双方に属するすべての周波数トリグラムの識別番号ｔｉｄについて、最小値ｍｉｎｆＴの総和を求め、当該総和値をＴｆ１とＴｆ２の類似度とする。
【００７３】
以上の処理により、例えば作業用端末２で対象識別情報を入力すれば、データベース３にて当該対象識別情報に対応する点検対象の周波数トリグラムヒストグラムを検索キーとして類似度が最も大きな周波数トリグラムヒストグラムを持つ点検対象を検索し、検索結果を作業用端末２に出力することができる。また、類似度の大きい方から順に整列した類似設備のリストを出力することもできる。さらに、すべての点検対象間の類似度を計算し、予め定めた閾値以上の類似度を有する点検対象群を抽出し、類似したセンサ情報を発している点検対象群として整理することもできる。
【００７４】
ここで、例えばフーリエ変換の対象とする時区間をずらすことで、一つの個別センサ情報ｓから一定時間間隔毎に複数の周波数分布情報を算出することも可能である。例えば４８ＫＨｚでサンプリングされたデジタル音信号は、一秒間あたり４８０００の情報から構成されている。一般にＦＦＴでは１０２４のように２の累乗個の情報を１度に扱うので、一秒間の音情報を連続する例えば１０２４毎の時区間に区切る。区切られた各時区間での周波数分布情報を計算することで、一つの個別センサ情報ｓから複数の周波数分布情報が得られる。このような処理を行った場合、連続する時区間から計算された周波数分布情報には音情報の時間方向の変化が反映されている。当該時間方向の変化は、瞬発音や音声などはもとより、設備２１のうなりなどを反映できるため、点検目的によっては重要な情報となる。
【００７５】
そこで、一つの個別センサ情報ｓから得られる複数の周波数分布情報をも扱えるように、例えば以下のようにして、上述の周波数ｎグラムヒストグラムの構成方法を拡張しても良い。即ち、信号情報ｓの連続する時区間から計算された周波数分布をｈ_１，ｈ_２，ｈ_３，…，ｈ_ｍａｘｉとする。ｋ番目の周波数分布ｈ_ｋの、ｊ番目の周波数区間のパワーをｐ_ｋ，ｊとし、拡張単特徴をｓ_ｋｊ＝（ｋ，ｊ，ｐ_ｋ，ｊ）とする。ここで、ｊを周波数方向引数、ｋを時間方向引数とする。拡張周波数ｎグラムとは、時間方向と周波数分布方向の一方または双方に連続するｎ個の拡張単特徴の組であり、任意の組み合わせが可能である。例えば、時間方向および周波数分布方向の双方にまたがる（ｓ_ｋ，ｊ，ｓ_{ｋ−１，ｊ}，ｓ_{ｋ，ｊ−１}）なるＬ字型の拡張周波数トリグラムとしても良い。つまり上述した周波数ｎグラムでは、拡張周波数ｎグラムにおいて時間方向引数が同一であり、かつ周波数方向引数が連続した場合の拡張単特徴の組となる。尚、拡張周波数ｎグラムについての拡張周波数ｎグラムヒストグラムの作成方法、拡張周波数ｎグラムヒストグラムを索引として記録する方法、拡張周波数ｎグラムヒストグラムを索引として類似する点検対象を検索する方法は、既に説明した周波数ｎグラムの場合と同様である。
【００７６】
さらに、例えば本実施形態では、同一の対象について纏められた当該対象の状態を表すセンサ情報の集合に対して、当該集合に属する個別のセンサ情報からそれぞれの周波数分布を求めて、求められた周波数分布の集合に対して、同じ周波数帯域または予め設定した周波数区間の一方または双方についての統計量を算出するようにしている（図２のステップ５）。この統計量とは、例えば、同じ周波数帯域または予め設定した周波数区間の一方または双方についての平均値や分散または標準偏差などが該当し、点検対象固有の特徴を表すものである。
【００７７】
さらに、例えば本実施形態では、現時点で取得されたセンサ情報である現センサ情報から周波数分布を求めて、現センサ情報と同一の対象について算出された上記統計量を用いて閾値を設定し、当該閾値を超える上記現センサ情報の周波数成分を抽出するようにしている（図２のステップ６）。従って、点検対象固有の特徴と、当該点検対象について現時点で取得された現センサ情報とを比較して、現センサ情報についての過去とは異なる特徴の有無を、容易に確認することができる。
【００７８】
このために、本実施形態の点検支援システム１では、同一の対象について纏められた当該対象の状態を表すセンサ情報の集合に対して、当該集合に属する個別のセンサ情報からそれぞれの周波数分布を求めて、求められた周波数分布の集合に対して、同じ周波数帯域または予め設定した周波数区間の一方または双方についての統計量（以下、本実施形態では「対象固有特徴」と呼ぶ。）を算出してデータベース３に記録する対象固有特徴抽出手段１３と、現時点で取得されたセンサ情報である現センサ情報から周波数分布を求めて、上記現センサ情報と同一の対象について算出された上記統計量を用いて閾値を設定し、当該閾値を超える上記現センサ情報の周波数成分（以下、本実施形態では「個別特徴」と呼ぶ。）を抽出してデータベース３に記録する個別特徴抽出手段１４と、上記データベース３に記録された対象固有特徴および個別特徴を表示する特徴表示手段１５を有するようにしている。点検支援用プログラムは、対象固有特徴抽出手段１３と個別特徴抽出手段１４と特徴表示手段１５としてコンピュータを機能させる。尚、本実施形態における当該コンピュータは例えばデータベース３である。
【００７９】
例えば本実施形態では、対象固有特徴として、以下に説明する情報を算出するようにしている。先ず、同一の設備２１について纏められた周波数分布情報集合Ｈに属するすべての周波数分布情報ｈについて、同じ周波数帯域あたりの平均をとったベクトルを平均周波数特徴ｈｖとして求める。また、同周波数分布情報集合Ｈに属するすべての周波数分布情報ｈについて、同じ周波数帯域あたりの標準偏差を計算したベクトルを標準偏差周波数特徴ｈｓとして求める。
【００８０】
次に、平均周波数特徴ｈｖを用いて、一定の周波数区間ｆでの平均として、区間平均特徴ｈｖ２を計算する。ここで、周波数分布情報ｈの周波数帯域数をｍａｘｉとする。また、平均周波数特徴ｈｖの周波数分布方向のｉ番目の要素をｈｖ_ｉとする。また、区間平均特徴ｈｖ２の周波数分布方向のｉ番目の要素をｈｖ２_ｉとする。周波数区間ｆは、例えば周波数分布情報ｈの全体周波数帯域Ｆよりも小さく設定する。ここで、周波数分解能をｄｆとし、ａ＝ｆ／（２ｄｆ）とすると、区間平均特徴ｈｖ２のｉ番目の要素ｈｖ２_ｉは、例えば以下の数式５および数式６に基づいて計算される。
【００８１】
【数５】
ａ≦ｉ≦ｍａｘｉ−ａの場合、

【数６】
ｉ＜ａ，ｉ＞ｍａｘｉ−ａの場合、
ｈｖ２_ｉ＝０
【００８２】
以下に、ｈｖ２_ｉを算出する処理を例を挙げて説明する。周波数分布情報ｈの全体周波数帯域Ｆが、１０Ｈｚ〜１０１０Ｈｚまでの１０００Ｈｚの帯域であるとする。周波数分解能ｄｆを１０Ｈｚとする。この場合、添え字ｉは１〜１００の範囲をとる。周波数区間ｆを１００Ｈｚの帯域として設定したとする。この場合、ａ＝ｆ／（２ｄｆ）＝５となる。例えばｈｖ２の５０番目の要素ｈｖ２_５０は、ｈｖ_４６からｈｖ_５５までの平均値となる。
【００８３】
次に、平均周波数特徴ｈｖおよび区間平均特徴ｈｖ２を用いて、周波数区間ｆでの標準偏差を求めて区間標準偏差特徴ｈｓ２とする。区間標準偏差特徴ｈｓ２の周波数分布方向のｉ番目の要素をｈｓ２_ｉとすると、ｈｓ２_ｉは、例えば以下の数式７および数式８に基づいて計算される。
【００８４】
【数７】
２ａ≦ｉ≦ｍａｘｉ−２ａの場合、

【数８】
ｉ＜２ａ，ｉ＞ｍａｘｉ−２ａの場合、
ｈｓ２_ｉ＝０
【００８５】
次に、平均周波数特徴ｈｖと区間平均特徴ｈｖ２と区間標準偏差特徴ｈｓ２とを用いて、区間平均から逸脱したピーク値であるピーク特徴ｈｐを計算する。ピーク特徴ｈｐの周波数分布方向のｉ番目の要素をｈｐ_ｉとすると、ｈｐ_ｉは、例えば以下の条件式に基づいて計算される。
【００８６】
【数９】
もし ｉ＜ａ または ｉ＞ｍａｘｉ−ａ ならば
ｈｐ_ｉ＝０
【数１０】
さもなければ、
もし ａ≦ｉ＜２ａ かつ ｈｖ_ｉ＞ｈｖ２_ｉ＋ｂ・ｈｓ２_２ａ＋ｃ ならば
ｈｐ_ｉ＝ｈｖ_ｉ−ｈｖ２_ｉ
【数１１】
さもなければ、
もし ｍａｘｉ−２ａ＜ｉ≦ｍａｘｉ−ａ
かつ ｈｖ_ｉ＞ｈｖ２_ｉ＋ｂ・ｈｓ２_{ｍａｘｉ −２ａ}＋ｃ ならば
ｈｐ_ｉ＝ｈｖ_ｉ−ｈｖ２_ｉ
【数１２】
さもなければ、
もし ｈｖ_ｉ＞ｈｖ２_ｉ＋ｂ・ｈｓ２_ｉ＋ｃ ならば
ｈｐ_ｉ＝ｈｖ_ｉ−ｈｖ２_ｉ
【数１３】
さもなければ、
ｈｐ_ｉ＝０
【００８７】
ここで、ｂ，ｃは閾値設定のパラメータである。可聴音を対象とした場合の目安としては、ｂ＝１，ｃ＝１０ｄＢ程度である。また、本願発明者等が種々実験検討した結果、ｂを２以上に設定すると、音波に限らず、殆んどの特徴信号が消えてしまうことが知見されたため、特殊な場合を除き、ｂは２以下に設定することが望ましい。
【００８８】
以上の処理により、対象固有特徴として、平均周波数特徴ｈｖ、標準偏差周波数特徴ｈｓ、区間平均特徴ｈｖ２、区間標準偏差特徴ｈｓ２、ピーク特徴ｈｐが算出される。尚、これらの対象固有特徴の算出方法は必ずしも上記の例に限定されない。例えば、上記処理では、要素ｉを中心として周波数区間ｆを設定したが、場合によっては、要素ｉを先頭または終端として周波数区間ｆを設定するようにしても良い。
【００８９】
また本実施形態では、現センサ情報ｓの個別特徴として、当該現センサ情報ｓと、その現センサ情報ｓに対応する設備２１に関して計算された、区間平均特徴ｈｖ２と区間標準偏差特徴ｈｓ２とピーク特徴ｈｐとを用いて、以下に説明する一時信号特徴ｈｔを計算するようにしている。現センサ情報ｓから計算された周波数分布情報ｈの周波数分布方向のｉ番目の要素をｈ_ｉとし、一時信号特徴ｈｔの周波数分布方向のｉ番目の要素をｈｔ_ｉとすると、ｈｔ_ｉは、例えば以下の条件式に基づいて計算される。
【００９０】
【数１４】
もし ｉ＜ａ または ｉ＞ｍａｘｉ−ａならば
ｈｔ_ｉ＝０
【数１５】
さもなければ、
もし ａ≦ｉ＜２ａ かつ ｈ_ｉ＞ｈｖ２_ｉ＋ｂ’・ｈｓ２_２ａ＋ｃ’
かつ ｈ_ｉ−ｈｖ２_ｉ＞ｈｐ_ｉ ならば
ｈｔ_ｉ＝ｈ_ｉ−ｈｖ２_ｉ
【数１６】
さもなければ、
もし ｍａｘｉ−２ａ＜ｉ≦ｍａｘｉ−ａ
かつ ｈ_ｉ＞ｈｖ２_ｉ＋ｂ’・ｈｓ２_{ｍａｘｉ −２ａ}＋ｃ’
かつ ｈ_ｉ−ｈｖ２_ｉ＞ｈｐ_ｉ ならば
ｈｔ_ｉ＝ｈ_ｉ−ｈｖ２_ｉ
【数１７】
さもなければ、
もし ｈ_ｉ＞ｈｖ２_ｉ＋ｂ’・ｈｓ２_ｉ＋ｃ’
かつ ｈ_ｉ−ｈｖ２_ｉ＞ｈｐ_ｉ ならば
ｈｔ_ｉ＝ｈ_ｉ−ｈｖ２_ｉ
【数１８】
さもなければ、
ｈｔ_ｉ＝０
【００９１】
ここで、ｂ’，ｃ’は閾値設定のパラメータである。可聴音を対象とした場合の目安としては、ｂ’＝１，ｃ’＝１０ｄＢ程度である。また、本願発明者等が種々実験検討した結果、ｂ’を２以上に設定すると、音波に限らず、殆んどの特徴信号が消えてしまうことが知見されたため、特殊な場合を除き、ｂ’は２以下に設定することが望ましい。
【００９２】
以上の処理により、現センサ情報ｓの個別特徴として、一時信号特徴ｈｔが算出される。尚、個別特徴の算出方法は必ずしも上記の例に限定されない。例えば、現センサ情報ｓから計算された周波数分布情報ｈと、現センサ情報ｓに対応する設備２１に関して計算されたピーク特徴ｈｐとの差を、現センサ情報ｓの個別特徴として求めるようにしても良い。
【００９３】
さらに、例えば本実施形態では、個別の信号である各センサ情報についても、上述した周波数ｎグラムヒストグラムを利用して、索引を作成し（図２のステップ７）、類似のセンサ情報を検索できるようにしている（図２のステップ８）。個別センサ情報に係る周波数ｎグラムの構成としては、例えば以下に説明する複数の形態が挙げられる。これらの形態はそれぞれに特徴があり、目的に応じて選択して良い。尚、周波数ｎグラムヒストグラムの作成方法、周波数ｎグラムヒストグラムを索引として記録する方法、周波数ｎグラムヒストグラムを索引として類似する情報を検索する方法は、既に説明した周波数ｎグラムの場合と同様である。
【００９４】
先ず、第１の形態について説明する。この場合、対象の状態を表すセンサ情報を時分割して且つそれぞれ周波数分布を求めて、当該求めた各周波数分布について、周波数分布方向に連続するｎ個の周波数成分を組み合わせて構成される周波数ｎグラムをすべて抽出し、さらに同一のセンサ情報に属する上記周波数ｎグラムの集合について、共通する周波数ｎグラムの頻度を求めて、周波数ｎグラムの種別とその頻度との組の集合である周波数ｎグラムヒストグラムを得て、当該周波数ｎグラムヒストグラムを索引とし、異なるセンサ情報間で共通する周波数ｎグラムの頻度に基づいて、あるセンサ情報と類似する他のセンサ情報を検索できるようにする。
【００９５】
このために、点検支援システム１では、対象の状態を表すセンサ情報を時分割して且つそれぞれ周波数分布を求めて、当該求めた各周波数分布について、周波数分布方向に連続するｎ個の周波数成分を組み合わせて構成される周波数ｎグラムをすべて抽出し、さらに同一のセンサ情報に属する上記周波数ｎグラムの集合について、共通する周波数ｎグラムの頻度を求めて、周波数ｎグラムの種別とその頻度との組の集合である周波数ｎグラムヒストグラムを得て、上記周波数ｎグラムヒストグラムを対応するセンサ情報の索引としてデータベース３に記録するセンサ情報毎索引作成手段１６と、異なるセンサ情報間で共通する周波数ｎグラムの頻度に基づいて、あるセンサ情報と類似する他のセンサ情報を検索する類似センサ情報検索手段１７とを有するようにしている。点検支援用プログラムは、センサ情報毎索引作成手段１６と類似センサ情報検索手段１７としてコンピュータを機能させる。尚、本実施形態における当該コンピュータは例えばデータベース３である。
【００９６】
上記第１の形態では、個別のセンサ情報ｓから計算できる複数の周波数分布情報ｈを用いて、周波数ｎグラムが直接計算される。多量の周波数ｎグラムが生成され得るため、索引作成および検索に要する計算処理が多くなる場合がある。その一方で、細かな特徴が反映された周波数ｎグラムヒストグラムが得られるため、細かな周波数特徴まで考慮した検索が可能となる。
【００９７】
次に、第２の形態について説明する。この場合、センサ情報毎索引作成手段１６により、以下の処理を実行するようにする。即ち、対象の状態を表すセンサ情報を時分割して且つそれぞれ周波数分布を求めて、上記周波数分布の時系列を複数の群に分け、上記群のそれぞれについて、時間方向と周波数分布方向の一方または双方に連続するｎ個の周波数成分を組み合わせて構成される拡張周波数ｎグラムをすべて抽出し、さらに同一のセンサ情報に属する上記拡張周波数ｎグラムの集合について、共通する拡張周波数ｎグラムの頻度を求めて、拡張周波数ｎグラムの種別とその頻度との組の集合である拡張周波数ｎグラムヒストグラムを得て、当該拡張周波数ｎグラムヒストグラムを対応するセンサ情報の索引としてデータベース３に記録するようにする。この第２の形態によれば、センサ情報の時間方向の変化を考慮した類似検索を行なえる。当該時間方向の変化は、瞬発音や音声などはもとより、設備２１のうなりなどを反映できる。
【００９８】
次に、第３の形態について説明する。この場合、センサ情報毎索引作成手段１６により、以下の処理を実行するようにする。即ち、現センサ情報の特徴的な周波数成分について、周波数分布方向に連続するｎ個の周波数成分を組み合わせて構成される周波数ｎグラムをすべて抽出し、さらに同一のセンサ情報に属する上記周波数ｎグラムの集合について、共通する周波数ｎグラムの頻度を求めて、周波数ｎグラムの種別とその頻度との組の集合である周波数ｎグラムヒストグラムを得て、当該周波数ｎグラムヒストグラムを対応するセンサ情報の索引としてデータベース３に記録するようにする。
【００９９】
周波数ｎグラムの抽出に用いる現センサ情報の特徴的な周波数成分として、例えば、現センサ情報ｓの周波数分布情報ｈと、当該現センサ情報ｓと対応する設備２１の平均周波数特徴ｈｖとの差分に対し、閾値処理を施した情報を用いても良い。現センサ情報ｓの周波数分布情報ｈと、当該現センサ情報ｓと対応する設備２１の平均周波数特徴ｈｖとの差分を差分周波数分布情報ｈｄとし、ｈｄの周波数分布方向のｉ番目の要素をｈｄ_ｉとし、設定した閾値（例えば１０ｄＢを初期値として変化させ、目的に対してノイズとなる信号が消去され、目的の信号が消去されない値を設定する）以下となるｈｄ_ｉを０とする。上記閾値処理後の差分周波数分布情報をｈｄ’とし、このｈｄ’から周波数ｎグラムを計算する。この場合、平均から逸脱したピーク値のみに基づいた検索が可能となり、計算処理量を削減できる。尚、パワーがゼロの単特徴のみを含む周波数ｎグラムが多数生成され得るため、索引作成および検索では、そのような周波数ｎグラムを除外して計算することが好ましい。
【０１００】
また、周波数ｎグラムの抽出に用いる現センサ情報の特徴的な周波数成分として、上述した一時信号特徴ｈｔを用いても良い。この場合、さらに少数の特徴情報のみに基づいた検索が可能となり、計算処理量を一層削減できる。ただし細かな周波数特徴は省略されるため、細かな周波数特徴を考慮した検索には不向きといえる。尚、この場合も、パワーがゼロの単特徴のみを含む周波数ｎグラムが多数生成され得るため、索引作成および検索では、そのような周波数ｎグラムを除外して計算することが好ましい。
【０１０１】
次に、第４の形態について説明する。この場合、センサ情報毎索引作成手段１６により、以下の処理を実行するようにする。即ち、現センサ情報の特徴的な周波数成分（例えば上述した閾値処理を施した差分周波数分布情報ｈｄ’や一時音特徴ｈｔなど）の時系列を複数の群に分け、上記群のそれぞれについて、時間方向と周波数分布方向の一方または双方に連続するｎ個の周波数成分を組み合わせて構成される拡張周波数ｎグラムをすべて抽出し、さらに同一のセンサ情報に属する上記拡張周波数ｎグラムの集合について、共通する拡張周波数ｎグラムの頻度を求めて、拡張周波数ｎグラムの種別とその頻度との組の集合である拡張周波数ｎグラムヒストグラムを得て、当該拡張周波数ｎグラムヒストグラムを対応するセンサ情報の索引としてデータベース３に記録するようにする。上記第４の形態では、センサ情報の時間方向の変化を考慮した類似検索を行なえるという第２の形態の特徴と、少数の特徴情報のみに基づいた検索により計算処理量を削減できるという第３の形態の特徴との双方の特徴を有する。
【０１０２】
次に、以上のように構成される点検支援システム１により、点検作業員等の利用者に対して情報提示を行なう処理（図２のステップ９）の一例について説明する。図４に、作業用端末２の表示装置６に表示される画面の一例を示す。
【０１０３】
上記画面の左側部分６ａには、映像情報において「沈静」状態が検出された時点の静止画像がアルバム状に表示される。当該静止画像は、例えば画面上方から下方に向かって古い順に整列されている。このような表示を行うことで、点検作業がどのような順序で進行したかを簡単に確認することができる。また、上記画面の中央上部分６ｂには現在ビデオカメラ４からキャプチャされつつある映像（動画像）が表示される。一方、画面の中央下部分６ｃには現在キャプチャされつつある音情報を処理した結果（例えば周波数分布情報ｈや一時音特徴ｈｔなど）が表示される。ここで、周波数分布情報ｈや一時音特徴ｈｔなどの履歴を、濃淡情報として画面に表示するようにしても良い。この場合、例えば新しい情報ほど濃く、古い情報ほど淡く表示するようにする。これにより、点検作業員はセンサ情報の変化を聴覚だけでなく視覚によっても確認することができる。
【０１０４】
さらに、点検作業員が画面の左側部分６ａの静止画像を選択することによって、対応する収録音が作業用端末２の音出力装置７より再生されるようにする。さらに、収録音と対応する収録信号（ｈ，ｈｖ，ｈｖ２，ｈｓ，ｈｓ２，ｈｐ，ｈｔなど）が作業用端末２の表示装置６に表示されるようにする。これにより、点検作業員は、過去のセンサ情報を簡易に確認することができる。
【０１０５】
画面の右側部分６ｄには、巡視点検対象の設備２１が配置されている建物の平面図が表示されている。当該平面図中のラジオボタンは現在どの設備２１を点検中であるのかを示す。図４の例では、点検作業員がマウスをクリック等することで、ラジオボタンを指定して点検中の設備２１をデータベース３に知らせるようにしている。尚、画面右端上部にある複数のチェックボックスは、表示機能や記録機能の状態を表し、点検作業員により変更可能であるようにしている。
【０１０６】
また、点検対象となっている設備２１が識別されると、当該設備２１に対応する過去のセンサ情報群から計算された対象固有特徴（平均周波数特徴ｈｖ，標準偏差周波数特徴ｈｓ、区間平均特徴ｈｖ２、区間標準偏差特徴ｈｓ２、ピーク特徴ｈｐ）がグラフ化されて表示される。これにより、点検対象設備２１に特有の特徴を視覚的に確かめることができる。また、これらの特徴に基づいて計算された一時信号特徴ｈｔもグラフ化されて表示される。一時信号特徴ｈｔおよびその履歴を表示することで、現在収録しつつある設備２１のセンサ情報が、過去の履歴に対してどういった違いを持つのかを、点検作業員は視覚的に確認できる。現在収録しつつある設備２１のセンサ情報が、過去に記録された同設備２１のセンサ情報とは異なる特徴を持つ場合には、その変化が明確に表示されるので、点検作業員による異常発見を補助することができる。
【０１０７】
さらに、その現場で、現在のセンサ情報と類似した過去のセンサ情報を検索して、点検作業員に提示する（例えば作業用端末２の音出力装置７から収録音を再生する、若しくは収録音に対応する信号を表示装置６にグラフとして表示する）こともできる。これは、現時点で検知した異常信号に類似した過去の事例を参照するのに役立つ。さらに、点検対象間の類似検索により、同種の症状を呈し得る設備グループを特定し、当該設備グループの範囲で、過去のセンサ情報を検索し、現在のセンサ情報と比較することもできる。さらに、同じ設備や類似する設備についての過去のセンサ情報を取得日時順に整列して、作業用端末２の表示装置６にリスト表示することもできる。この場合、前回や前々回の点検で取得されたセンサ情報と現在のセンサ情報に係る収録音を、作業用端末２の音出力装置７から再生して聞き比べたり、当該収録音に対応する収録信号（ｈ，ｈｖ，ｈｖ２，ｈｓ，ｈｓ２，ｈｐ，ｈｔなど）を表示装置６にグラフとして表示して見比べたりすることができる。
【０１０８】
尚、各設備２１に添付されたマーカ等に基づいて作業用端末２から入力される映像情報から前回点検におけるセンサ情報の収録位置を特定して、次回点検時に上記収録位置に点検作業員を誘導するための音声情報や映像情報を作業用端末２に出力するようにしても良い。この場合、各収録回の収録状況（音源までの距離など）を揃えることができる。
【０１０９】
本発明のセンサ情報のデータベース化方法および点検支援システム１および方法並びに点検支援用プログラムによれば、点検業務時の音などのセンサ情報が、映像情報とともに点検対象ごとに整理された形で、自動的にデータベース３に蓄えられる。また、各点検対象ごとに纏められたセンサ情報群に、周波数ｎグラムヒストグラムを索引として付与しているので、ある点検対象に類似する点検対象を迅速に検索できる。また、各センサ情報にも、周波数ｎグラムヒストグラムを索引として付与しているので、あるセンサ情報に類似するセンサ情報を迅速に検索できる。さらに、これらの検索結果を実際の設備２１を前にした現場で即座に点検作業員に提示することができる。また、各設備２１に固有の特徴と、現時点で得られたセンサ情報に固有の特徴とを、それぞれ可視化して表示することで、異常の有無を視覚によっても確認することができる。これらの機能により、従来、作業者の五感に頼っていた音や臭いによる点検作業を支援できる。これは、異常音検出の精度向上、作業の効率化、熟練者でなくとも作業を精度良く行なえるといったメリットに結びつく。
【０１１０】
なお、上述の実施形態は本発明の好適な実施の一例ではあるがこれに限定されるものではなく、本発明の要旨を逸脱しない範囲において種々変形実施可能である。例えば、センサ情報は音情報に限らず、振動情報、温度情報、臭い情報、超音波等であっても良い。これらの情報も波形信号として取得できるため、上述の実施形態と同様の取扱が可能である。
【０１１１】
また、上述の実施形態では、映像情報の変化に基づいて、映像情報以外のセンサ情報を区切ってデータベース３に記録するようにしたが、例えば次に説明する処理によってセンサ情報を区切って整理しデータベース３に記録するようにしても良い。即ち、音などのセンサ情報ｓに関する信号差分ｄｓ（単位時間あたりの平均信号と、直前の単位時間で計算した値との差分）もしくはセンサ情報ｓそのものを入力ｄ_ｋとし、入力ｄ_ｋの平均μおよび標準偏差σを用いて、上述した数式２から数式４によって、センサ情報の変化時点を検出し、当該変化に基づいて、センサ情報を区切ってデータベース３に記録するようにする。この場合、定点観測やビデオカメラ４を固定して得られる監視映像情報など、映像情報に変化が殆んど無い場合でも、センサ情報を区切って整理しデータベース３に記録することができる。また、監視映像情報などから異音発生時点を検出するなどの機能を実現できる。
【０１１２】
尚、信号差分ｄｓは、例えば以下のように求められる。入力信号としてのセンサ情報のサンプリングレートをｒとする。つまり一秒間に入力される信号の個数がｒとなる。計測開始から、ある計測時刻までに入力された信号の総数をｋとする。この場合、経過した時間は、ｋ・（１／ｒ）秒となる。ｋ番目の信号の値をｓ_ｋとする。ｋ＋１番目の信号の値はｓ_ｋ＋１となる。ここで単位時間をｔｂとする。単位時間ｔｂは、１／ｒの整数倍である。単位時間あたりの信号数ｋｂは、ｋｂ＝ｔｂ／（１／ｒ）となる。単位時間の設定は、時間方向の分解能をいくつとしたいかによって異なり、数ｍｓから数十ｍｓの間で設定するのが一般的である。ある計測時刻ｔ１での単位時間あたりの平均信号とは、その計測時刻までに入力された信号総数をｋ１とした場合に、次式で表される。
【０１１３】
【数１９】

【０１１４】
直前の単位時間あたりの平均信号はｓｖ（ｋ１−ｋｂ）となる。したがって、計測時刻ｔ１での信号差分はｄｓ＝ｓｖ（ｋ１）−ｓｖ（ｋ１−ｋｂ）となる。
【０１１５】
【発明の効果】
以上の説明から明らかなように、請求項１記載のセンサ情報のデータベース化方法および請求項３記載の点検支援システム並びに請求項５記載の点検支援用プログラムによれば、音等のセンサ情報を自動で区切って整理しながら記録することができる。また、整理蓄積されたセンサ情報群の中から特定のセンサ情報を取り出すことも容易にできる。録音テープなどにおける録音箇所を示すカウンタなどを一々利用者が記録しておくといった手間のかかる作業は不要となる。
【０１１６】
さらに、請求項２記載のセンサ情報のデータベース化方法および請求項４記載の点検支援システム並びに請求項６記載の点検支援用プログラムによれば、音等のセンサ情報を点検対象ごとに自動で整理することが可能になる。
【０１１７】
さらに、請求項７記載のセンサ情報のデータベース化方法および請求項１０記載の点検支援システム並びに請求項１３記載の点検支援用プログラムによれば、点検対象ごとに整理され蓄積されたセンサ情報に基づいて作成された索引を利用して、ある点検対象と類似したセンサ情報を発している他の点検対象を高速に検索することが可能となる。したがって、当該検索結果を、実際の点検対象を前にした現場で即座に点検作業員に提示することもできる。
【０１１８】
また、請求項８記載のセンサ情報のデータベース化方法および請求項１１記載の点検支援システム並びに請求項１４記載の点検支援用プログラムによれば、点検対象ごとの統計的特徴に基づいて作成された索引を利用して、ある点検対象と類似したセンサ情報を発している他の点検対象を高速に検索することが可能となる。したがって、当該検索結果を、実際の点検対象を前にした現場で即座に点検作業員に提示することもできる。
【０１１９】
また、請求項９記載のセンサ情報のデータベース化方法および請求項１２記載の点検支援システム並びに請求項１５記載の点検支援用プログラムによれば、類似点検対象の検索用の索引に、センサ情報の時間方向の変化（例えば瞬発音や音声、うなりなど）を反映することができる。したがって、センサ情報の時間方向の変化を考慮した類似点検対象の高速検索が可能となる。
【０１２０】
さらに、請求項１６記載のセンサ情報のデータベース化方法および請求項１７記載の点検支援システム並びに請求項１８記載の点検支援用プログラムによれば、点検対象固有の特徴である統計量と、当該点検対象について現時点で取得された現センサ情報とを可視化して視覚的に比較することができる。これにより、点検作業員による異常発見を補助することができ、従来、作業者の五感に頼っていた音や臭いによる点検作業を支援できる。これは、異常音検出の精度向上、作業の効率化、熟練者でなくとも作業を精度良く行なえるといったメリットに結びつく。
【０１２１】
さらに、請求項１９記載のセンサ情報のデータベース化方法および請求項２３記載の点検支援システム並びに請求項２７記載の点検支援用プログラムによれば、センサ情報に基づいて作成されセンサ情報の特徴が反映された索引を利用して、現時点で取得されたセンサ情報と類似する過去に取得されたセンサ情報を高速に検索することが可能となる。したがって、当該検索結果を、実際の点検対象を前にした現場で即座に点検作業員に提示することもできる。
【０１２２】
また、請求項２０記載のセンサ情報のデータベース化方法および請求項２４記載の点検支援システム並びに請求項２８記載の点検支援用プログラムによれば、類似センサ情報の検索用の索引に、センサ情報の時間方向の変化（例えば瞬発音や音声、うなりなど）を反映することができる。したがって、センサ情報の時間方向の変化を考慮した類似センサ情報の高速検索が可能となる。また、高速検索が可能であるから、検索結果を実際の点検対象を前にした現場で即座に点検作業員に提示することもできる。
【０１２３】
また、請求項２１記載のセンサ情報のデータベース化方法および請求項２５記載の点検支援システム並びに請求項２９記載の点検支援用プログラムによれば、センサ情報の抽出された特徴に特化した類似センサ情報の検索が可能となり、検索に要する計算処理量を削減できる。また、高速検索が可能であるから、検索結果を実際の点検対象を前にした現場で即座に点検作業員に提示することもできる。
【０１２４】
また、請求項２２記載のセンサ情報のデータベース化方法および請求項２６記載の点検支援システム並びに請求項３０記載の点検支援用プログラムによれば、センサ情報の抽出された特徴に特化した類似センサ情報の検索が可能となり、検索に要する計算処理量を削減できる。また、類似センサ情報の検索用の索引に、センサ情報の時間方向の変化（例えば瞬発音や音声、うなりなど）を反映できるので、当該センサ情報の時間方向の変化を考慮した類似センサ情報の高速検索が可能となる。また、高速検索が可能であるから、検索結果を実際の点検対象を前にした現場で即座に点検作業員に提示することもできる。
【図面の簡単な説明】
【図１】本発明の点検支援システムの実施の一形態を示す概略機能ブロック図である。
【図２】本発明のデータベース化方法の処理の一例を示す概略フローチャートである。
【図３】本発明の点検支援システムの実施のイメージを示す概念図である。
【図４】作業用端末の表示装置に表示される画面の一例を示す図である。
【符号の説明】
１ 点検支援システム
２ 作業用端末（センサ情報を取得する手段）
３ データベース[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a method for converting information such as voice into a database, an inspection support system using the database, and an inspection support program.
[0002]
[Prior art]
Regular inspections are indispensable for the long-term sound operation and use of power equipment such as generators, transformers and circuit breakers. For this reason, the power industry and other equipment industries invest a great deal of labor and capital in inspection work for huge amounts of equipment. Therefore, it is an important issue how to reduce these costs without lowering the inspection accuracy. Inspection work includes disassembly and inspection, in which the equipment is stopped and then disassembled and inspected in detail, and patrol inspection in which the appearance and the like are inspected while the equipment is operating. Of these, the disassembly and inspection must stop the equipment, so it is desirable to reduce the frequency as much as possible to reduce maintenance costs. For this reason, attempts have been made to shift to a situation-dependent inspection in which signs of abnormality are detected early during operation and patrol inspection, and disassembly inspection is performed accordingly.
[0003]
As a conventional patrol inspection support technology, there is a technology of collecting various data of a patrol object using a portable terminal device (handy terminal) and registering the data in a database (see Patent Document 1). Further, an information storage and retrieval method and an information storage and retrieval device for collecting, accumulating, and retrieving a large amount of various data related to control, operation, maintenance, inspection, and the like of facilities such as plants have been proposed (patented patents). Reference 2). There is also a technique for accumulating video and audio data in order to facilitate creation of know-how data in inspection and maintenance work (see Patent Document 3).
[0004]
[Patent Document 1]
JP-A-6-4539
[Patent Document 2]
JP 2000-90106 A
[Patent Document 3]
JP-A-2000-222026
[0005]
[Problems to be solved by the invention]
The reality is that many patrols rely on the patrol members' five senses. For example, hearing, that is, inspection using sound, is performed in many equipment patrols, and is one of important inspection items. Therefore, it is desired that the five senses information received by the patrol member during the patrol be organized and recorded so that it can be used during the subsequent patrol. However, the prior art requires a great deal of effort to organize and record information such as sound so that it can be easily used later.
[0006]
Therefore, an object of the present invention is to provide a method of creating a database of sensor information that can easily organize and record sensor information corresponding to five senses and sensor information (for example, ultrasonic waves) exceeding the five senses. . Still another object of the present invention is to provide a system for supporting inspection work and a program for supporting inspection using a database in which information such as sounds is arranged and recorded.
[0007]
[Means for Solving the Problems]
In order to achieve the above object, a method for creating a database of sensor information according to claim 1 obtains sensor information that is information indicating a target state and includes at least video information, and detects a change in video information. Then, based on the change, sensor information other than video information is recorded separately. According to a third aspect of the present invention, there is provided an inspection support system configured to acquire sensor information including information representing a state of a target and including at least video information, and to detect a change in the video information, and to perform a detection based on the change. Means for separately recording sensor information other than video information. According to a fifth aspect of the present invention, there is provided an inspection support program for detecting a change in video information with respect to sensor information that is information indicating a target state and includes at least video information, and based on the change, detects video information. The computer is made to function as means for recording sensor information other than information in a divided manner. Therefore, sensor information such as sound can be recorded while being automatically divided and arranged.
[0008]
According to a second aspect of the invention, in the sensor information database method according to the first aspect, information for identifying an object is recorded in association with each of the divided sensor information. According to a fourth aspect of the present invention, in the inspection support system of the third aspect, there is further provided a means for identifying an object, and a means for recording information for identifying the object in association with each of the divided sensor information. Like that. According to a sixth aspect of the present invention, in the inspection support program according to the fifth aspect, the computer is caused to function as means for recording information for identifying an object in association with each of the divided sensor information. . Therefore, sensor information such as sound can be automatically arranged for each inspection target.
[0009]
In addition, according to the method for creating a database of sensor information according to claim 7, a frequency distribution is obtained from individual sensor information for a set of sensor information representing the state of each object arranged for each object, and the obtained frequency distribution is obtained. With respect to the frequency distribution, all frequency n-grams formed by combining n frequency components continuous in the frequency distribution direction are extracted, and the frequency of a common frequency n-gram is determined for a set of sensor information belonging to the same target. Then, a frequency n-gram histogram, which is a set of a type of frequency n-gram and its frequency, is obtained, and an object similar to a certain object is searched based on the frequency of the frequency n-gram common between different objects. I have to.
[0010]
In addition, the inspection support system according to claim 10 obtains a frequency distribution from individual sensor information for a set of sensor information indicating the state of each object arranged for each object, and calculates a frequency distribution for each of the obtained frequency distributions. , Extracting all frequency n-grams formed by combining n frequency components continuous in the frequency distribution direction, and further calculating the frequency of a common frequency n-gram for a set of sensor information belonging to the same object. means for obtaining a frequency n-gram histogram, which is a set of sets of n-gram types and their frequencies, and recording the frequency n-gram histogram as an index of a corresponding object; frequency n-gram frequency common to different objects And means for searching for an object similar to a certain object on the basis of.
[0011]
In the inspection support program according to the thirteenth aspect, a frequency distribution is obtained from individual sensor information for a set of sensor information representing the state of each object arranged for each object, and the obtained frequency distribution is obtained. For all of the frequency n-grams that are formed by combining n frequency components that are continuous in the frequency distribution direction, and further, for a set of sensor information belonging to the same target, the frequency of a common frequency n-gram is obtained, Means for obtaining a frequency n-gram histogram, which is a set of sets of frequency n-gram types and their frequencies, and recording the frequency n-gram histogram as an index of a corresponding object; The computer is caused to function as means for searching for a target similar to a target based on the frequency.
[0012]
Therefore, it is possible to use an index created based on accumulated sensor information for each inspection target to quickly search for another inspection target that emits sensor information similar to a certain inspection target. Become.
[0013]
In addition, according to the sensor information database method of the present invention, for a set of sensor information indicating the state of each object arranged for each object, individual sensor information is time-divided and the frequency distribution is obtained. Further, a statistic is calculated for a set of frequency distributions belonging to the same sensor information, and for the statistic, all frequency n-grams formed by combining n frequency components continuous in the frequency distribution direction are extracted, Further, for the set of the frequency n-grams belonging to the same object, the frequency of the common frequency n-gram is obtained, and the frequency n-gram histogram which is a set of the type of the frequency n-gram and the frequency is obtained. An object similar to a certain object is searched for based on the frequency of the common frequency n-gram.
[0014]
In the inspection support system according to the eleventh aspect, with respect to a set of sensor information indicating the state of each object arranged for each object, individual sensor information is time-divided and the frequency distribution is obtained, and the same is obtained. Calculate statistics about a set of frequency distributions belonging to the sensor information, extract all frequency n-grams composed of a combination of n frequency components continuous in the frequency distribution direction, and further extract the same statistic. For the set of the frequency n-grams belonging to the object, the frequency of the common frequency n-gram is determined, and the frequency n-gram histogram which is a set of the type of the frequency n-gram and the frequency is obtained. Means for recording an index as a corresponding object, and similar to an object based on the frequency of a frequency n-gram common to different objects. Have a means to have at least to find that object.
[0015]
In addition, the inspection support program according to claim 14, for a set of sensor information representing the state of each object arranged for each object, time-division of individual sensor information and obtain a frequency distribution respectively, Calculate statistics for a set of frequency distributions belonging to the same sensor information, extract all frequency n-grams composed of a combination of n frequency components continuous in the frequency distribution direction for the statistics, and With respect to the set of the frequency n-grams belonging to the above-mentioned object, the frequency of the common frequency n-gram is obtained, and the frequency n-gram histogram which is a set of the type of the frequency n-gram and the frequency is obtained. A means for recording the histogram as an index of a corresponding object; and an object based on a frequency of a common frequency n-gram between different objects. So that causes a computer to function as means for searching an object to be similar.
[0016]
Therefore, using an index created based on the statistical characteristics of each inspection target, it is possible to quickly search for another inspection target that emits sensor information similar to a certain inspection target.
[0017]
According to a ninth aspect of the present invention, there is provided a database for sensor information, wherein, for a set of sensor information indicating the state of each object arranged for each object, individual sensor information is time-divided and a frequency distribution is obtained. Further, with respect to the time series of the frequency distribution belonging to the same sensor information, all extended frequency n-grams formed by combining n frequency components continuous in one or both of the time direction and the frequency distribution direction are extracted, and further the same For a set of sensor information belonging to the target, the frequency of a common extended frequency n-gram is obtained, and an extended frequency n-gram histogram that is a set of the type of the extended frequency n-gram and its frequency is obtained. The search for an object similar to a certain object is performed based on the frequency of the common extended frequency n-gram.
[0018]
In the inspection support system according to the twelfth aspect, with respect to a set of sensor information indicating the state of each object arranged for each object, individual sensor information is time-divided and the frequency distribution is obtained, and the same is obtained. For the time series of the frequency distribution belonging to the sensor information, extract all the extended frequency n-grams that are formed by combining n frequency components continuous in one or both of the time direction and the frequency distribution direction, and further extract the same target. For the set of sensor information to which it belongs, the frequency of the common extended frequency n-gram is obtained, and the extended frequency n-gram histogram, which is a set of the type of the extended frequency n-gram and the frequency, is obtained. Based on the frequency of an extended frequency n-gram common to different objects. It is to have at least a means for retrieving an object similar to.
[0019]
In addition, the inspection support program according to claim 15, for a set of sensor information representing the state of each object arranged for each object, time-divides individual sensor information and obtains a frequency distribution, respectively. With respect to the time series of the frequency distribution belonging to the same sensor information, all extended frequency n-grams composed of a combination of n frequency components continuous in one or both of the time direction and the frequency distribution direction are extracted, and further the same object The frequency of the common extended frequency n-gram is obtained for the set of sensor information belonging to the extended frequency n-gram, and the extended frequency n-gram histogram which is a set of the type of the extended frequency n-gram and the frequency is obtained. Based on the means for recording the histogram as an index of the corresponding object and the frequency of the extended frequency n-gram common to different objects, So that causes a computer to function as means for searching an object similar to the subject.
[0020]
In this case, the change in the time direction of the sensor information (for example, instantaneous sound, voice, beat, etc.) is reflected in the index for searching for the similar check target. Therefore, a high-speed search for similar inspection targets in consideration of the change in the sensor information in the time direction becomes possible.
[0021]
Further, in the sensor information database creation method according to claim 16, for a set of sensor information indicating the state of the target collected for the same target, each frequency distribution is obtained from individual sensor information belonging to the set. For the obtained set of frequency distributions, a statistic for one or both of the same frequency band and a preset frequency section is calculated, and the frequency is calculated from the current sensor information that is the sensor information acquired at the present time. A distribution is obtained, a threshold is set using the statistics calculated for the same target as the current sensor information, and a frequency component of the current sensor information exceeding the threshold is extracted.
[0022]
Further, in the inspection support system according to claim 17, for a set of sensor information indicating the state of the target combined for the same target, each frequency distribution is obtained from individual sensor information belonging to the set. For a set of obtained frequency distributions, means for calculating a statistic for one or both of the same frequency band or a preset frequency section, and a frequency distribution from current sensor information which is sensor information acquired at the present time. Calculating, setting a threshold using the statistic calculated for the same target as the current sensor information, extracting a frequency component of the current sensor information exceeding the threshold, the calculated statistic and Means for displaying the extracted frequency components.
[0023]
In the inspection support program according to the eighteenth aspect, for a set of sensor information that represents the state of the target grouped for the same target, each frequency distribution is obtained from individual sensor information belonging to the set. Means for calculating a statistic for one or both of the same frequency band or a preset frequency section for the determined set of frequency distributions, and a frequency distribution based on current sensor information which is sensor information acquired at the present time. Means for setting a threshold using the statistic calculated for the same target as the current sensor information, extracting frequency components of the current sensor information exceeding the threshold, and calculating the statistic The computer is caused to function as means for displaying the extracted frequency components.
[0024]
Therefore, it is possible to visualize the statistic, which is a characteristic unique to the inspection target, and the current sensor information obtained at the present time for the inspection target, and visually compare them.
[0025]
Further, in the method for creating a database of sensor information according to claim 19, the sensor information representing the target state is time-divided and frequency distributions are obtained, and for each of the obtained frequency distributions, n continuous in the frequency distribution direction. All frequency n-grams formed by combining the frequency components are extracted, and the frequency n-gram frequency common to the set of the frequency n-grams belonging to the same sensor information is obtained. A frequency n-gram histogram, which is a set of combinations of the frequency and the frequency, is obtained, and another sensor information similar to a certain sensor information is searched based on the frequency of the frequency n-gram common to different sensor information. .
[0026]
In the inspection support system according to claim 23, the sensor information representing the state of the target is time-divided and frequency distributions are obtained, and for each of the obtained frequency distributions, n consecutive frequencies in the frequency distribution direction are obtained. All the frequency n-grams composed by combining the components are extracted, and the frequency n-gram frequency common to the set of the frequency n-grams belonging to the same sensor information is obtained. Means for obtaining a frequency n-gram histogram, which is a set of sets, and recording the frequency n-gram histogram as an index of corresponding sensor information, and a sensor based on a frequency of a frequency n-gram common to different sensor information. Means for searching for other sensor information similar to the information.
[0027]
In addition, the inspection support program according to claim 27 divides the sensor information representing the state of the object by time division and obtains the respective frequency distributions. All the frequency n-grams composed by combining the frequency components are extracted, and the frequency n-gram frequency common to the set of the frequency n-grams belonging to the same sensor information is obtained. A means for obtaining a frequency n-gram histogram which is a set of the above-mentioned sets and recording the frequency n-gram histogram as an index of the corresponding sensor information, and a frequency n-gram frequency common to different sensor information. The computer is caused to function as means for searching for other sensor information similar to the sensor information.
[0028]
Therefore, by using the index created based on the sensor information and reflecting the characteristics of the sensor information, it is possible to quickly search for previously acquired sensor information similar to the sensor information acquired at the present time. .
[0029]
Further, in the sensor information database creation method according to claim 20, the sensor information representing the target state is time-divided and the frequency distribution is obtained, and the time series of the frequency distribution is divided into a plurality of groups. , Extract all extended frequency n-grams formed by combining n frequency components continuous in one or both of the time direction and the frequency distribution direction, and further extract the extended frequency n-grams belonging to the same sensor information. For the set, the frequency of the common extended frequency n-gram is obtained, and the extended frequency n-gram histogram which is a set of the type of the extended frequency n-gram and the frequency is obtained, and the extended frequency n common to the different sensor information is obtained. Based on the frequency of the gram, other sensor information similar to certain sensor information is searched.
[0030]
In the inspection support system according to claim 24, the sensor information representing the state of the object is time-divided and the frequency distribution is obtained, and the time series of the frequency distribution is divided into a plurality of groups. , Extracting all extended frequency n-grams composed of a combination of n frequency components continuous in one or both of the time direction and the frequency distribution direction, and further regarding the set of extended frequency n-grams belonging to the same sensor information, The frequency of the common extended frequency n-gram is obtained, the extended frequency n-gram histogram which is a set of the type of the extended frequency n-gram and the frequency is obtained, and the extended frequency n-gram histogram is indexed to the sensor information corresponding thereto. Based on the frequency of the extended frequency n-gram common to different sensor information, Have a means to have at least to find other sensor information.
[0031]
In addition, the inspection support program according to claim 28, time-divides the sensor information representing the state of the object and obtains each frequency distribution, divides the time series of the frequency distribution into a plurality of groups, , Extract all extended frequency n-grams composed of a combination of n frequency components continuous in one or both of the time direction and the frequency distribution direction, and further extract a set of the extended frequency n-grams belonging to the same sensor information. , The frequency of the common extended frequency n-gram is obtained, and an extended frequency n-gram histogram which is a set of the type of the extended frequency n-gram and its frequency is obtained. Means for recording as an index, and certain sensor information based on the frequency of the extended frequency n-gram common to different sensor information. So that causes a computer to function as means for retrieving other sensor information similar.
[0032]
In this case, a change in the time direction of the sensor information (for example, instantaneous sound, voice, beat, etc.) is reflected in the search index for the similar sensor information. Therefore, high-speed search for similar sensor information in consideration of a change in the sensor information in the time direction becomes possible.
[0033]
According to a twenty-first aspect of the present invention, in the sensor information database creation method according to the sixteenth aspect, the frequency components of the extracted current sensor information are combined with n consecutive frequency components in the frequency distribution direction. All frequency n-grams are extracted, and a frequency n-gram frequency common to the set of frequency n-grams belonging to the same sensor information is obtained. A frequency n-gram histogram is obtained, and other sensor information similar to certain sensor information is searched based on the frequency of the frequency n-gram common to different sensor information.
[0034]
According to a twenty-fifth aspect of the present invention, in the inspection support system according to the seventeenth aspect, a frequency n configured by combining n frequency components continuous in the frequency distribution direction with respect to the frequency components of the extracted current sensor information. The frequency n-gram, which is a set of the type of the frequency n-gram and the frequency, is obtained by extracting all of the frequency n-grams and obtaining the frequency of the common frequency n-gram for the set of the frequency n-grams belonging to the same sensor information. Means for obtaining a histogram and recording the frequency n-gram histogram as an index of the corresponding sensor information; and, based on the frequency of the frequency n-gram common to different sensor information, other sensor information similar to certain sensor information. And means for searching.
[0035]
According to a twenty-ninth aspect of the present invention, in the inspection support program according to the eighteenth aspect, a frequency configured by combining n frequency components continuous in the frequency distribution direction with respect to the frequency components of the extracted current sensor information. All n-grams are extracted, and the frequency n-grams belonging to the same sensor information are extracted to determine the frequency of the common frequency n-grams. A means for obtaining a gram histogram and recording the frequency n-gram histogram as an index of corresponding sensor information; and other sensor information similar to certain sensor information based on a frequency of a frequency n-gram common to different sensor information. The computer is made to function as means for searching for.
[0036]
In this case, a search for similar sensor information specialized for the extracted feature of the sensor information becomes possible, and the amount of calculation processing required for the search can be reduced.
[0037]
According to a twenty-second aspect of the present invention, in the sensor information database creation method according to the sixteenth aspect, the time series of the frequency component of the extracted current sensor information is divided into a plurality of groups, and each of the groups is subjected to a time direction. , And extract all extended frequency n-grams formed by combining n frequency components continuous in one or both of the frequency distribution directions, and further, for a set of extended frequency n-grams belonging to the same sensor information, The frequency of the frequency n-gram is obtained to obtain an extended frequency n-gram histogram which is a set of a set of the type and the frequency of the extended frequency n-gram, and based on the frequency of the extended frequency n-gram common to different sensor information. In addition, another sensor information similar to a certain sensor information is searched.
[0038]
According to a twenty-sixth aspect of the present invention, in the inspection support system according to the seventeenth aspect, the time series of the frequency components of the extracted current sensor information is divided into a plurality of groups, and the time direction and the frequency distribution are determined for each of the groups. Extracts all extended frequency n-grams formed by combining n frequency components continuous in one or both directions, and further sets a common extended frequency n-gram for the set of extended frequency n-grams belonging to the same sensor information. Means for obtaining an extended frequency n-gram histogram which is a set of a set of the type of the extended frequency n-gram and its frequency, and recording the extended frequency n-gram histogram as an index of the corresponding sensor information; Based on the frequency of the extended frequency n-gram common to different sensor information, other sensor information similar to certain sensor information And search means are designed to have at least.
[0039]
According to a thirtieth aspect of the present invention, in the inspection support program according to the eighteenth aspect, the time series of the frequency component of the extracted current sensor information is divided into a plurality of groups, and the time direction and the frequency All extended frequency n-grams composed of a combination of n frequency components continuous to one or both of the distribution directions are extracted, and a set of the extended frequency n-grams belonging to the same sensor information is subjected to a common extended frequency n. Means for obtaining the frequency of the gram, obtaining an extended frequency n-gram histogram which is a set of a set of the type and the frequency of the extended frequency n-gram, and recording the extended frequency n-gram histogram as an index of the corresponding sensor information. Based on the frequency of the extended frequency n-gram common to different sensor information, and other sensors similar to certain sensor information So that causes a computer to function as means for retrieving broadcast.
[0040]
In this case, a search for similar sensor information specialized for the extracted feature of the sensor information becomes possible, and the amount of calculation processing required for the search can be reduced. In addition, since the change in the time direction of the sensor information (for example, instantaneous sound, voice, beat, etc.) is reflected in the index for searching the similar sensor information, the similar sensor information of the sensor information in consideration of the change in the time direction of the sensor information is considered. High-speed search becomes possible.
[0041]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, the configuration of the present invention will be described in detail based on an embodiment shown in the drawings.
[0042]
FIG. 1 to FIG. 4 show an embodiment of a sensor information database creation method, an inspection support system, and an inspection support program according to the present invention. For example, in the present embodiment, an example in which the present invention is applied to a patrol inspection of the power equipment 21 will be described.
[0043]
For example, the inspection support system 1 according to the present embodiment includes a work terminal 2 carried by an inspection worker, and a computer 3 functioning as a database in which sensor information collected by the work terminal 2 and information obtained by processing the sensor information are recorded. And is configured.
[0044]
The “sensor information” in this specification includes all kinds of information indicating the state of the “target”. As the “target”, for example, the individual equipment 21 to be checked corresponds. The sensor information includes, for example, information corresponding to the human five senses (visual, auditory, olfactory, skin sensation such as temperature and vibration), and information beyond the five senses (eg, ultrasonic waves). The work terminal 2 is provided with a unit for acquiring sensor information. For example, the working terminal 2 of the present embodiment includes a video camera 4 for acquiring video information and a microphone 5 for acquiring sound information. Accordingly, in the present embodiment, the work terminal 2 acquires the video information and the sound information as the sensor information. However, the sensor information is not limited to the example of the present embodiment. For example, a vibration sensor, a temperature sensor, an odor sensor, an ultrasonic sensor, and the like are provided in the work terminal 2 so that the work terminal 2 acquires vibration information, temperature information, odor information, ultrasonic waves, and the like as sensor information. Is also good.
[0045]
The computer 3 functioning as a database (hereinafter simply referred to as a database 3) includes, for example, a central processing unit (CPU), a main storage device, an external storage device 20 (for example, a sensor information and an information obtained by processing the sensor information). A computer that has hardware resources such as a communication interface with the work terminal 2, an operating system that controls these hardware resources, and software resources such as an inspection support program. It is. The database 3 also has a function of receiving a request from a user, interactively performing a process, and outputting information required by the user.
[0046]
The work terminal 2 may include an output device such as a display device 6 (for example, a liquid crystal display or a head mounted display) or a sound output device 7 (for example, a speaker, an earphone, a headphone, or the like). In this case, the sensor information stored in the database 3 and information obtained by processing the sensor information can be output to the work terminal 2. Further, the work terminal 2 may include an input device 8 such as a mouse or a keyboard. In this case, the inspection worker can interactively perform processing with the database 3, and the inspection worker can appropriately record necessary information in the database 3 together with the sensor information. Further, the work terminal 2 is provided with hardware resources such as a central processing unit (CPU), a main storage device, and an external storage device, and an operating system for controlling these hardware resources. Some or all of the processing involved in processing the sensor information, operating the database 3, etc. may be borne. Of course, all of these processes may be performed in the database 3.
[0047]
Although the form of the work terminal 2 is not particularly limited, for example, (1) a video camera 4 and a microphone 5 are mounted on an inspection work helmet, and (2) a video is displayed on a head-mounted display device 6 (head-mounted display). (3) Use a commercially available handy video camera 4; (4) Hold the video camera 4 and the microphone 5 separately; (5) Put the video camera 4 on the clothes of the inspection worker. There is a mode in which the microphone 5 is provided, and (6) a portable computer (for example, a notebook personal computer or a wearable computer) including the video camera 4 and the microphone 5 is used. These modes and the input / output devices 6, 7, and 8 included in the work terminal 2 may be appropriately selected according to the usage environment.
[0048]
The work terminal 2 and the database 3 may be always connected wirelessly or by wire to perform real-time communication, or the work terminal 2 may be operated offline without being connected to the database 3. The sensor information is temporarily recorded on a recording medium (for example, a magnetic tape or a hard disk) provided in the work terminal 2, and the work terminal 2 and the database 3 are connected later by wireless or wire to perform communication. May be.
[0049]
In the present embodiment, the sensor information is arranged and recorded in the database 3 as follows, for example (steps 1 and 2 in FIG. 2). That is, among the acquired sensor information, a change in the video information is detected, and based on the change, the sensor information other than the video information (sound information in the present embodiment) is divided and recorded in the database 3. The inspection support system 1 detects a change in video information with respect to the work terminal 2 as a means for acquiring sensor information and the sensor information acquired by the work terminal 2, and based on the change, detects other than video information. And at least sensor information dividing means 9 for recording the sensor information in the database 3 in a divided manner. The inspection support program causes the computer to function as the sensor information dividing means 9. The computer in the present embodiment is, for example, the database 3.
[0050]
For example, in the present embodiment, a change in video information is detected by the processing described below. First, with respect to input video information, a change amount d between successive frame images_kIs calculated based on the following equation.
[0051]
(Equation 1)

[0052]
Here, c (k, x, y) represents the color value of the pixel at position x, y in the k-th frame image. dist () is the distance of the color value in the Euclidean space. As the color space, for example, in this embodiment, the most standard RGB space (0 ≦ R, G, B ≦ 255) is adopted. However, the color space is not limited to the RGB space. For example, when a shooting environment or a detection target can be assumed in advance, the sensitivity can be increased by appropriately devising the color space.
[0053]
Next, the change amount d of the past n frames_kIs calculated and the standard deviation σ is calculated. Then, it is determined whether the video state of the k-th frame is “active”, “quiet”, or “steady” based on the following conditional expression.
[0054]
(Equation 2)
If d_kIf> μ + ασ, it is determined to be in the “active” state.
(Equation 3)
If d_kIf <μ−βσ, it is determined that the state is “calm”.
(Equation 4)
If it does not correspond to any of the "active" and "quiet" states, it is determined to be the "steady" state.
[0055]
α and β are parameters for specifying the range of the “steady state” (α> 0, β> 0). Then, when a frame in a state different from the current video state appears m (m ≦ n) times in the past n frames, it is determined that the video state has changed. These parameters (n, m) affect the state change homeostasis.
[0056]
Here, the patrol work is divided into a movement between the facilities 21 and 21 and a work for inspecting each facility 21 without moving. The point in time when the “calm down” state is detected (the point in time when it is determined that the state has transitioned from a state other than “calm down” to “calm down”) is estimated to be the point in time when the movement is shifted to the inspection.
[0057]
Therefore, for example, in the present embodiment, the video information and the sound information are continuously recorded, and when it is determined that the video information has transitioned from a state other than “calm” to “calm”, the sound information is They are separated. This makes it possible to automatically arrange the sensor information for each inspection target. This eliminates the need for the operator to record information (such as a counter) indicating a recording location on a recording medium such as a recording tape one by one so that the collected sensor information can be used later.
[0058]
Here, in the present embodiment, the separated sensor information is called a “sensor information block”, and an identification number id that can be uniquely identified is assigned to each “sensor information block”. Then, a correspondence table between the identification number id and information indicating the head of the sensor information block (for example, the separation time t) is created. Accordingly, the corresponding sensor information block can be called from the identification number id according to the correspondence table. Further, the video at the separation time t is recorded as a still image, and a correspondence table between the identification number ID of the still image and the identification number id or the separation time t is created. Therefore, the still image corresponding to the head position of the sensor information block can be called from the correspondence table. Note that the video itself (moving image) may be recorded in the database 3 as a supplementary function. In this case, for example, a correspondence table between the frame ft at the separation time t in the recorded video and the identification number id or the separation time t is created.
[0059]
The recorded images may be recorded in the same file over the entire shooting time, or may be recorded in separate files at each division time t. Also, sound information may be recorded in the same file over the entire recording time, and delimiter information (tag) may be inserted into the file. Alternatively, sound information may be recorded in separate files at each division time t. The file configuration of the sensor information may be appropriately selected depending on the performance of the computer, the usage environment, and the like. When the video information is determined to be in a state other than “calm” from a state other than “calm”, the work terminal 2 starts recording, and in the video information, a state other than “calm” to “calm” is determined. At this point, the recording may be stopped.
[0060]
Further, for example, in the present embodiment, information for identifying the equipment 21 as the inspection target (for example, a number for uniquely identifying the inspection target. Hereinafter, in this embodiment, referred to as “target identification information”). The data is recorded in the database 3 in association with each of the separated sensor information blocks as the sensor information. For this purpose, the inspection support system 1 of the present embodiment includes a unit for identifying the equipment 21 and a target identification information providing unit 10 for recording the target identification information in association with each of the sensor information blocks. The inspection support program causes a computer to function as the target identification information providing unit 10. The computer in the present embodiment is, for example, the database 3. Therefore, it is possible to specify to which equipment 21 the sensor information block belongs by the target identification information, and it is possible to arrange the sensor information for each equipment 21.
[0061]
The means for identifying the equipment 21 is not necessarily limited, but includes, for example, the following means. As a first means, the inspection worker uses the input device 8 of the work terminal 2 to input target identification information indicating which equipment 21 the currently acquired sensor information relates to. As a second means, a facility identification IC chip such as an RFID (Radio Frequency ID, also referred to as a non-contact type IC chip) is attached to each facility 21, and a communication antenna with the IC chip is attached to the work terminal 2. The work terminal 2 automatically reads the target identification information recorded on the IC chip. As a third means, a barcode, a two-dimensional barcode, an information identification marker (for example, refer to JP-A-2002-56371) is attached to each facility 21, and it is read from the video information to read the facility 21. Is automatically recognized, and the corresponding target identification information is specified. As a fourth means, the image information currently captured is compared with the image information of the known equipment 21 registered in the past, and the equipment 21 is automatically recognized by an image search (for example, Japanese Patent Application Laid-Open No. 2001-109884). Refer to the gazette) and specify the corresponding target identification information. This is particularly effective when the appearance of the equipment 21 is different. As a fifth means, the position of the work terminal 2 is specified by using a position information system such as a GPS (satellite navigation system), and the equipment 21 currently under inspection is automatically determined based on the specified position. Recognize and specify the corresponding target identification information. As a sixth means, the work terminal 2 is provided with a voice recognition means, and the inspection worker utters the name of the equipment, thereby identifying the corresponding target identification information from the equipment name.
[0062]
Further, for example, in the present embodiment, for a set of sensor information representing the state of each object arranged for each object, a frequency distribution is obtained from individual sensor information, and the frequency distribution direction is obtained for each obtained frequency distribution. To extract all the frequency n-grams composed of a combination of n frequency components that are continuous with each other, and further, for a set of sensor information belonging to the same target, determine the frequency of the common frequency n-gram, and determine the type of the frequency n-gram. A frequency n-gram histogram is obtained as a set of a set of the frequency n-gram histogram, and the frequency n-gram histogram is used as an index (step 3 in FIG. 2). It is possible to search for an object similar to (step 4 in FIG. 2).
[0063]
For this purpose, in the inspection support system 1 of the present embodiment, for a set of sensor information indicating the state of each object arranged for each object, a frequency distribution is obtained from individual sensor information, and the obtained frequency distribution is obtained. With respect to the distribution, all frequency n-grams formed by combining n frequency components continuous in the frequency distribution direction are extracted, and further, for a set of sensor information belonging to the same target, the frequency of a common frequency n-gram is determined. And a frequency index n-gram histogram, which is a set of pairs of frequency n-gram types and their frequencies, and records the frequency n-gram histogram in the database 3 as an index of a corresponding target. Similar object search means 12 for searching for an object similar to a certain object based on the frequency of the frequency n-gram common to the objects; It is as to have. The inspection support program causes the computer to function as the index creating means 11 for each object and the similar object searching means 12. The computer in the present embodiment is, for example, the database 3.
[0064]
A set of all individual sensor information s of a specific type recorded for a certain facility 21 is referred to as a sensor information set S. For example, one year of sound information of a certain transformer corresponds to the sensor information set S. The individual sensor information s is, for example, a sensor information block obtained by the above-described processing.
[0065]
For example, FFT (Fast Fourier Transform) is applied to all the individual sensor information s belonging to the sensor information set S to convert the individual sensor information s into frequency distribution information h. A set of frequency distribution information h corresponding to the sensor information set S is referred to as a frequency distribution information set H. Note that the signal processing section, frequency resolution, and signal resolution of each frequency band, which are the parameters of the FFT, are set according to the purpose. These parameters are preferably the same for all the frequency distribution information h belonging to the same frequency distribution information set H.
[0066]
Here, one piece of frequency distribution information h may be calculated from one piece of individual sensor information s. For example, by shifting the time interval to be subjected to Fourier transform, one piece of individual sensor information s can be calculated at regular time intervals. A plurality of pieces of frequency distribution information h ′ can be calculated. Therefore, for a plurality of pieces of frequency distribution information h ′ calculated from one piece of individual sensor information s, a statistic (for example, an average obtained over the same frequency band) is calculated, and the statistic is used as the frequency distribution information h. You may use as.
[0067]
Let m be the number of frequency bands of the frequency distribution information h and p be the power of the k-th band from the low frequency band._k(1 ≦ k ≦ m). (K, p_k) Is defined as a single feature sk. Then, k is defined for 1 <k ≦ m (s_k-1, S_k) Is a frequency bigram. Similarly, it is defined for k such that 1 <k <m (s_k-1, S_k, S_{k + 1}) Is a frequency trigram. Similarly, a set of n consecutive single features is defined as a frequency n gram. Here, n is called the number of dimensions of the frequency n-gram. In order to accurately retrieve sensor information having high similarity, it is preferable to use a frequency n-gram having a large number of dimensions. However, if the number of dimensions of the frequency n-gram is too large, fine features of the signal may appear too much, and the ambiguity required for the search may not be achieved. In that case, it is necessary to use a frequency n-gram of a lower dimension. As a result of various experiments and studies conducted by the inventors of the present application, it has been found that generally accurate search can be realized by using a frequency trigram in the case of rotating machine equipment sound. Therefore, in the present embodiment, an example using a frequency trigram having three dimensions will be particularly described. Note that the same processing is performed for other dimensions.
[0068]
Single feature s_k= (K, p_k) P_kIs quantized according to the search accuracy. As the number of quantizations increases, the finer power characteristics are reflected, and as the number of quantizations decreases, the characteristics take into account the warmth.
[0069]
A set of all frequency trigrams calculated from the frequency distribution information h belonging to a certain frequency distribution information set H is referred to as a frequency trigram set T. Then, a unique identification number tid is assigned to each frequency trigram. However, it is assumed that the identification numbers tid of the common frequency trigrams are equal. Here, the fact that the frequency trigram is common means, for example, in the present embodiment, the case where the positions and values of the three single features that are the constituent elements are all equal. However, in order to loosen the commonality of the frequency trigrams, for example, when the position and the value are the same for two of the three simple features, it may be defined as “the frequency trigrams are common”.
[0070]
Then, the number of common frequency trigrams in the frequency trigram set T is defined as fT. For all different frequency trigrams belonging to the frequency trigram set T, all the sets of (tid, fT) are calculated, and the set of the sets is set as a frequency trigram histogram Tf.
[0071]
The calculated frequency trigram histogram Tf is created for each facility 21 as an index for searching for similar facilities, and recorded in the database 3. The method of constructing the index is not particularly limited, but it is appropriate to use a general multidimensional index such as R-Tree. Since the number of frequency bands and the number of power quantizations can be kept constant, it is effective to use a three-stage tree structure in which the size of the table is determined in advance. For example, in the case of a frequency trigram, a first index is formed using the first single feature of the three single features, and a second single feature is used in a region referred to by the first index. A second index is formed, and a third index is formed by using the third single feature in a region referred to by the second index, thereby forming trees for the number of single features. In the last leaf, the frequency ft of the frequency trigram and the above-described target identification information are recorded. By configuring the index in this way, a search for one frequency trigram can be performed by searching at most several times. In addition, since there may be a large deviation in the distribution of the frequency trigram, it is also effective to consider the index arrangement in the external storage device 20 of the database 3 according to the deviation. It is also possible to keep an index relating to a single feature in the main storage device of the database 3 in consideration of recent computer performance, which helps a high-speed search. Further, since a frequency trigram that frequently appears in a search may be biased, a more efficient index can be configured by incorporating a cache mechanism.
[0072]
The similarity between the frequency trigram histogram Tf1 created for a certain inspection object and the frequency trigram histogram Tf2 created for another inspection object can be calculated, for example, as follows. That is, for the identification number tid of the frequency trigram belonging to both Tf1 and Tf2, the frequency of the frequency trigram at Tf1 and Tf2 is fT1 and fT2. The minimum value of the two values fT1 and fT2 is defined as minfT. Then, for the identification numbers tid of all the frequency trigrams belonging to both Tf1 and Tf2, the sum of the minimum values minfT is obtained, and the sum is defined as the similarity between Tf1 and Tf2.
[0073]
According to the above processing, for example, when the target identification information is input at the work terminal 2, the frequency trigram histogram having the highest similarity is used as the search key in the database 3 using the frequency trigram histogram of the inspection target corresponding to the target identification information. Can be searched, and the search result can be output to the work terminal 2. In addition, a list of similar facilities arranged in descending order of similarity can be output. Furthermore, it is also possible to calculate the similarity between all the inspection targets, extract an inspection target group having a similarity greater than or equal to a predetermined threshold value, and arrange the inspection target groups that emit similar sensor information.
[0074]
Here, it is also possible to calculate a plurality of pieces of frequency distribution information at regular time intervals from one piece of individual sensor information s, for example, by shifting the time interval to be subjected to the Fourier transform. For example, a digital sound signal sampled at 48 KHz is composed of 48,000 information per second. Generally, in FFT, information of a power of 2 is treated at once like 1024, so that sound information for one second is divided into continuous time intervals of, for example, 1024. By calculating the frequency distribution information in each divided time section, a plurality of frequency distribution information can be obtained from one individual sensor information s. When such processing is performed, the change in the time direction of the sound information is reflected in the frequency distribution information calculated from the continuous time sections. The change in the time direction can reflect the beating of the equipment 21 as well as the instantaneous sound and voice, and is therefore important information depending on the inspection purpose.
[0075]
In order to handle a plurality of pieces of frequency distribution information obtained from one piece of individual sensor information s, for example, the above-described method of constructing the frequency n-gram histogram may be extended as follows. That is, the frequency distribution calculated from the continuous time interval of the signal information s is represented by h₁, H₂, H₃, ..., h_maxiAnd k-th frequency distribution h_k, The power of the j-th frequency section is p_{k, j}And the extended single feature is s_kj= (K, j, p_{k, j}). Here, j is a frequency direction argument and k is a time direction argument. The extended frequency n-gram is a set of n extended single features continuous in one or both of the time direction and the frequency distribution direction, and any combination is possible. For example, both the time direction and the frequency distribution direction (s_{k, j}, S_{k-1, j}, S_{k, j-1}) May be an L-shaped extended frequency trigram. That is, the above-mentioned frequency n-gram is a set of extended single features when the time direction argument is the same and the frequency direction argument is continuous in the extended frequency n-gram. The method of creating the extended frequency n-gram histogram for the extended frequency n-gram, the method of recording the extended frequency n-gram histogram as an index, and the method of searching for a similar inspection target using the extended frequency n-gram histogram as the index have already been described. This is similar to the case of the frequency n-gram.
[0076]
Further, for example, in the present embodiment, for a set of sensor information that represents the state of the target collected for the same target, the respective frequency distributions are obtained from individual sensor information belonging to the set, and the obtained frequency is obtained. For a set of distributions, statistics are calculated for one or both of the same frequency band and a preset frequency section (Step 5 in FIG. 2). The statistic corresponds to, for example, an average value, a variance, or a standard deviation of one or both of the same frequency band and a preset frequency section, and represents a characteristic unique to the inspection target.
[0077]
Further, for example, in the present embodiment, a frequency distribution is obtained from the current sensor information that is the sensor information acquired at the present time, and a threshold is set using the statistic calculated for the same target as the current sensor information. A frequency component of the current sensor information exceeding the threshold is extracted (step 6 in FIG. 2). Therefore, by comparing the characteristic unique to the inspection target with the current sensor information acquired at the present time for the inspection target, it is possible to easily confirm the presence or absence of a characteristic of the current sensor information different from the past.
[0078]
For this reason, in the inspection support system 1 of the present embodiment, for a set of sensor information that represents the state of the target collected for the same target, each frequency distribution is obtained from individual sensor information belonging to the set. Then, with respect to the obtained set of frequency distributions, a statistic (hereinafter, referred to as “target unique feature” in the present embodiment) for one or both of the same frequency band and a preset frequency section is calculated. A frequency distribution is obtained from the target unique feature extraction means 13 recorded in the database 3 and the current sensor information which is the sensor information acquired at the present time, and the above-described statistics calculated for the same target as the current sensor information are used. A threshold value is set, and a frequency component of the current sensor information exceeding the threshold value (hereinafter, referred to as “individual feature” in the present embodiment) is extracted and stored in a database. To the individual feature extraction means 14 for recording, and to have a characteristic display means 15 for displaying the target-specific characteristics and individual features are recorded in the database 3. The inspection support program causes a computer to function as the target unique feature extraction unit 13, the individual feature extraction unit 14, and the feature display unit 15. The computer in the present embodiment is, for example, the database 3.
[0079]
For example, in the present embodiment, information described below is calculated as the target unique feature. First, for all the frequency distribution information h belonging to the frequency distribution information set H collected for the same equipment 21, a vector obtained by averaging the same frequency band is obtained as an average frequency characteristic hv. Further, for all the frequency distribution information h belonging to the same frequency distribution information set H, a vector obtained by calculating the standard deviation per the same frequency band is obtained as a standard deviation frequency feature hs.
[0080]
Next, using the average frequency feature hv, a section average feature hv2 is calculated as an average in a certain frequency section f. Here, the number of frequency bands of the frequency distribution information h is maxi. The i-th element in the frequency distribution direction of the average frequency feature hv is hv_iAnd Also, the i-th element in the frequency distribution direction of the section average feature hv2 is hv2_iAnd The frequency section f is set smaller than, for example, the entire frequency band F of the frequency distribution information h. Here, assuming that the frequency resolution is df and a = f / (2df), the i-th element hv2 of the section average feature hv2_iIs calculated based on, for example, Equations 5 and 6 below.
[0081]
(Equation 5)
When a ≦ i ≦ maxi-a,

(Equation 6)
In the case of i <a, i> maxi-a,
hv2_i= 0
[0082]
Below, hv2_iThe process of calculating is described using an example. It is assumed that the entire frequency band F of the frequency distribution information h is a band of 1000 Hz from 10 Hz to 1010 Hz. The frequency resolution df is set to 10 Hz. In this case, the subscript i ranges from 1 to 100. It is assumed that the frequency section f is set as a band of 100 Hz. In this case, a = f / (2df) = 5. For example, the 50th element hv2 of hv2₅₀Is hv₄₆From hv₅₅It becomes the average value up to.
[0083]
Next, using the average frequency feature hv and the section average feature hv2, the standard deviation in the frequency section f is obtained and set as the section standard deviation feature hs2. The i-th element in the frequency distribution direction of the section standard deviation feature hs2 is hs2_iThen, hs2_iIs calculated based on, for example, the following Expressions 7 and 8.
[0084]
(Equation 7)
When 2a ≦ i ≦ maxi-2a,

(Equation 8)
In the case of i <2a, i> maxi-2a,
hs2_i= 0
[0085]
Next, a peak feature hp that is a peak value deviating from the section average is calculated using the average frequency feature hv, the section average feature hv2, and the section standard deviation feature hs2. The ith element in the frequency distribution direction of the peak feature hp is hp_iThen, hp_iIs calculated based on, for example, the following conditional expression.
[0086]
(Equation 9)
If i <a or i> maxi-a
hp_i= 0
(Equation 10)
Otherwise,
If a ≦ i <2a and hv_i> Hv2_i+ B · hs2_2a+ C
hp_i= Hv_i-Hv2_i
(Equation 11)
Otherwise,
If maxi-2a <i ≦ maxi-a
And hv_i> Hv2_i+ B · hs2_{maxi -2a}+ C
hp_i= Hv_i-Hv2_i
(Equation 12)
Otherwise,
If hv_i> Hv2_i+ B · hs2_i+ C
hp_i= Hv_i-Hv2_i
(Equation 13)
Otherwise,
hp_i= 0
[0087]
Here, b and c are threshold setting parameters. As a guideline for an audible sound, b = 1 and c = about 10 dB. In addition, as a result of various experiments and studies conducted by the inventors of the present application, it has been found that when b is set to 2 or more, not only sound waves but also most of the characteristic signals disappear, so that b is 2 except in special cases. It is desirable to set the following.
[0088]
Through the above processing, the average frequency feature hv, the standard deviation frequency feature hs, the section average feature hv2, the section standard deviation feature hs2, and the peak feature hp are calculated as the target unique features. Note that the method of calculating these target unique features is not necessarily limited to the above example. For example, in the above processing, the frequency section f is set with the element i as the center, but in some cases, the frequency section f may be set with the element i at the beginning or end.
[0089]
In the present embodiment, as individual features of the current sensor information s, the section average feature hv2, the section standard deviation feature hs2, and the peak feature calculated with respect to the current sensor information s and the equipment 21 corresponding to the current sensor information s. hp is used to calculate a temporary signal feature ht described below. The i-th element in the frequency distribution direction of the frequency distribution information h calculated from the current sensor information s is h_iAnd the ith element in the frequency distribution direction of the temporary signal feature ht is ht_iThen ht_iIs calculated based on, for example, the following conditional expression.
[0090]
[Equation 14]
If i <a or i> maxi-a
ht_i= 0
[Equation 15]
Otherwise,
If a ≦ i <2a and h_i> Hv2_i+ B '· hs2_2a+ C '
And h_i-Hv2_i> Hp_i  If
ht_i= H_i-Hv2_i
(Equation 16)
Otherwise,
If maxi-2a <i ≦ maxi-a
And h_i> Hv2_i+ B '· hs2_{maxi -2a}+ C '
And h_i-Hv2_i> Hp_i  If
ht_i= H_i-Hv2_i
[Equation 17]
Otherwise,
If h_i> Hv2_i+ B '· hs2_i+ C '
And h_i-Hv2_i> Hp_i  If
ht_i= H_i-Hv2_i
(Equation 18)
Otherwise,
ht_i= 0
[0091]
Here, b 'and c' are threshold setting parameters. As a guideline for an audible sound, b '= 1 and c' = about 10 dB. Also, as a result of various experiments and studies conducted by the inventors of the present application, it has been found that when b ′ is set to 2 or more, not only sound waves but also most of the characteristic signals disappear. Is desirably set to 2 or less.
[0092]
Through the above processing, the temporary signal feature ht is calculated as the individual feature of the current sensor information s. Note that the method of calculating the individual features is not necessarily limited to the above example. For example, a difference between the frequency distribution information h calculated from the current sensor information s and the peak characteristic hp calculated for the equipment 21 corresponding to the current sensor information s may be obtained as an individual characteristic of the current sensor information s. good.
[0093]
Further, for example, in the present embodiment, an index is created for each sensor information, which is an individual signal, using the above-described frequency n-gram histogram (step 7 in FIG. 2) so that similar sensor information can be searched. (Step 8 in FIG. 2). The configuration of the frequency n-gram related to the individual sensor information includes, for example, a plurality of forms described below. Each of these forms has its own characteristics and may be selected according to the purpose. The method for creating the frequency n-gram histogram, the method for recording the frequency n-gram histogram as an index, and the method for searching for similar information using the frequency n-gram histogram as the index are the same as those for the frequency n-gram described above.
[0094]
First, the first embodiment will be described. In this case, the sensor information indicating the target state is time-divided and frequency distributions are obtained, and for each of the obtained frequency distributions, a frequency n configured by combining n frequency components continuous in the frequency distribution direction. The frequency n-gram, which is a set of the type of the frequency n-gram and the frequency, is obtained by extracting all of the frequency n-grams and obtaining the frequency of the common frequency n-gram for the set of the frequency n-grams belonging to the same sensor information. By obtaining a histogram and using the frequency n-gram histogram as an index, it is possible to search for other sensor information similar to certain sensor information based on the frequency of the frequency n-gram common to different sensor information.
[0095]
For this purpose, the inspection support system 1 time-divides sensor information representing the state of the object and obtains the respective frequency distributions. For each of the obtained frequency distributions, n frequency components continuous in the frequency distribution direction are obtained. All the frequency n-grams that are combined are extracted, and the frequency n-gram frequency common to the set of the frequency n-grams belonging to the same sensor information is obtained. And an index generating unit 16 for each sensor information which records the frequency n-gram histogram in the database 3 as an index of the corresponding sensor information, and a frequency n-gram histogram common to different sensor information. Similar sensor information searching means 1 for searching for other sensor information similar to certain sensor information based on frequency It is to have a door. The inspection support program causes the computer to function as the sensor information index creating unit 16 and the similar sensor information searching unit 17. The computer in the present embodiment is, for example, the database 3.
[0096]
In the first embodiment, the frequency n-gram is directly calculated using a plurality of pieces of frequency distribution information h that can be calculated from the individual sensor information s. Since a large number of frequency n-grams can be generated, the calculation processing required for index creation and search may increase. On the other hand, since a frequency n-gram histogram in which the fine features are reflected is obtained, it is possible to perform a search in consideration of even the fine frequency features.
[0097]
Next, a second embodiment will be described. In this case, the following processing is executed by the index creating unit 16 for each sensor information. That is, the sensor information representing the target state is time-divided and the frequency distribution is obtained, and the time series of the frequency distribution is divided into a plurality of groups. For each of the groups, one of the time direction and the frequency distribution direction or All extended frequency n-grams composed of a combination of n consecutive frequency components are extracted, and the frequency of a common extended frequency n-gram is determined for the set of extended frequency n-grams belonging to the same sensor information. Then, an extended frequency n-gram histogram, which is a set of sets of the type of the extended frequency n-gram and its frequency, is obtained, and the extended frequency n-gram histogram is recorded in the database 3 as an index of the corresponding sensor information. According to the second embodiment, a similarity search can be performed in consideration of a change in the sensor information in the time direction. The change in the time direction can reflect not only the instantaneous sound and voice, but also the beat of the equipment 21 and the like.
[0098]
Next, a third embodiment will be described. In this case, the following processing is executed by the index creating unit 16 for each sensor information. That is, for the characteristic frequency components of the current sensor information, all the frequency n-grams formed by combining n frequency components continuous in the frequency distribution direction are extracted, and further, the frequency n-grams belonging to the same sensor information are extracted. For the set, the frequency of the common frequency n-gram is obtained, and the frequency n-gram histogram which is a set of the type of the frequency n-gram and the frequency is obtained, and the frequency n-gram histogram is used as an index of the corresponding sensor information. It is recorded in the database 3.
[0099]
As a characteristic frequency component of the current sensor information used for extracting the frequency n-gram, for example, a difference between the frequency distribution information h of the current sensor information s and the average frequency characteristic hv of the equipment 21 corresponding to the current sensor information s is given. On the other hand, information that has been subjected to threshold processing may be used. The difference between the frequency distribution information h of the current sensor information s and the average frequency characteristic hv of the equipment 21 corresponding to the current sensor information s is defined as difference frequency distribution information hd, and the i-th element in the frequency distribution direction of hd is hd._iHd that is equal to or less than a set threshold value (for example, by changing 10 dB as an initial value, a signal that becomes noise for a purpose is eliminated, and a target signal is not erased)_iIs set to 0. The difference frequency distribution information after the threshold processing is defined as hd ', and a frequency n-gram is calculated from the hd'. In this case, a search based on only the peak value deviating from the average becomes possible, and the amount of calculation processing can be reduced. In addition, since many frequency n-grams including only a single feature with zero power can be generated, it is preferable to exclude the frequency n-grams in the index creation and search.
[0100]
Further, the above-mentioned temporary signal feature ht may be used as a characteristic frequency component of the current sensor information used for extracting the frequency n-gram. In this case, a search based on only a small number of pieces of feature information becomes possible, and the amount of calculation processing can be further reduced. However, since the detailed frequency characteristics are omitted, it can be said that the search is not suitable for the search in consideration of the fine frequency characteristics. In this case, too, a large number of frequency n-grams including only a single feature with zero power can be generated. Therefore, it is preferable to exclude the frequency n-gram in the index creation and search.
[0101]
Next, a fourth embodiment will be described. In this case, the following processing is executed by the index creating unit 16 for each sensor information. That is, the time series of characteristic frequency components of the current sensor information (for example, the difference frequency distribution information hd ′ subjected to the above-described threshold processing and the temporary sound feature ht) are divided into a plurality of groups, and the time All extended frequency n-grams composed of a combination of n frequency components continuous in one or both of the direction and the frequency distribution direction are extracted, and the set of extended frequency n-grams belonging to the same sensor information is common. The frequency of the extended frequency n-gram is obtained, the extended frequency n-gram histogram which is a set of the type of the extended frequency n-gram and the frequency is obtained, and the extended frequency n-gram histogram is used as an index of the corresponding sensor information in the database. Record it in 3. In the fourth embodiment, the feature of the second embodiment that the similarity search can be performed in consideration of the change in the sensor information in the time direction, and the third feature that the calculation processing amount can be reduced by the search based on only a small number of feature information. It has both features of the feature of the form.
[0102]
Next, an example of a process (step 9 in FIG. 2) of presenting information to a user such as an inspection worker by the inspection support system 1 configured as described above will be described. FIG. 4 shows an example of a screen displayed on the display device 6 of the work terminal 2.
[0103]
On the left part 6a of the screen, a still image at the time when the "quiet" state is detected in the video information is displayed in the form of an album. The still images are arranged in chronological order from the top to the bottom of the screen, for example. By performing such a display, it is possible to easily confirm in what order the inspection work has progressed. The video (moving image) currently being captured from the video camera 4 is displayed in the upper center portion 6b of the screen. On the other hand, the result of processing the currently captured sound information (for example, frequency distribution information h, temporary sound feature ht, etc.) is displayed in the lower center portion 6c of the screen. Here, a history such as the frequency distribution information h and the temporary sound feature ht may be displayed on the screen as shading information. In this case, for example, newer information is displayed darker, and older information is displayed lighter. Thereby, the inspection worker can confirm the change of the sensor information not only by hearing but also visually.
[0104]
Further, when the inspection worker selects a still image in the left portion 6a of the screen, the corresponding recorded sound is reproduced from the sound output device 7 of the work terminal 2. Further, a recording signal (h, hv, hv2, hs, hs2, hp, ht, etc.) corresponding to the recording sound is displayed on the display device 6 of the work terminal 2. Thereby, the inspection worker can easily check the past sensor information.
[0105]
In the right part 6d of the screen, a plan view of the building in which the equipment 21 to be inspected is arranged is displayed. Radio buttons in the plan view indicate which equipment 21 is currently under inspection. In the example of FIG. 4, the inspection worker clicks a mouse or the like to specify a radio button to notify the database 3 of the equipment 21 under inspection. Note that a plurality of check boxes at the upper right end of the screen indicate states of the display function and the recording function, and can be changed by an inspection worker.
[0106]
When the equipment 21 to be inspected is identified, the target unique features (average frequency feature hv, standard deviation frequency feature hs, section average feature hv2) calculated from the past sensor information group corresponding to the facility 21 , The section standard deviation feature hs2 and the peak feature hp) are graphed and displayed. As a result, it is possible to visually confirm the characteristics unique to the inspection target equipment 21. Further, the temporary signal features ht calculated based on these features are also displayed in a graph. By displaying the temporary signal feature ht and its history, the inspection worker can visually confirm what difference the sensor information of the equipment 21 currently recording has with respect to the past history. If the sensor information of the facility 21 currently being recorded has different characteristics from the sensor information of the same facility 21 recorded in the past, the change is clearly displayed, so that the inspection worker can find the abnormality. Can assist.
[0107]
Further, at the site, the past sensor information similar to the current sensor information is searched and presented to the inspection worker (for example, the recorded sound is reproduced from the sound output device 7 of the work terminal 2 or the recorded sound is reproduced). The corresponding signal can be displayed as a graph on the display device 6). This is useful for referring to past cases similar to the abnormal signal detected at the present time. Furthermore, it is also possible to specify a facility group that can exhibit the same kind of symptoms by performing a similarity search between inspection targets, search past sensor information within the range of the facility group, and compare it with current sensor information. Further, past sensor information on the same facility or similar facility can be arranged in order of acquisition date and time, and displayed in a list on the display device 6 of the work terminal 2. In this case, the sensor information acquired in the previous or second-last inspection and the recording sound related to the current sensor information are reproduced from the sound output device 7 of the work terminal 2 and compared, or the recording signal corresponding to the recording sound is compared. (H, hv, hv2, hs, hs2, hp, ht, etc.) can be displayed as a graph on the display device 6 and compared.
[0108]
The recording position of the sensor information in the previous inspection is specified from the video information input from the work terminal 2 based on the marker attached to each facility 21 and the inspection worker is guided to the recording position in the next inspection. For example, audio information or video information for performing the operation may be output to the work terminal 2. In this case, the recording status (distance to the sound source and the like) of each recording can be made uniform.
[0109]
According to the sensor information database creation method, the inspection support system 1 and the inspection support program, and the inspection support program of the present invention, sensor information such as sound at the time of inspection work is automatically arranged together with video information for each inspection object. Is temporarily stored in the database 3. In addition, since the frequency n-gram histogram is assigned as an index to the sensor information group compiled for each inspection target, an inspection target similar to a certain inspection target can be quickly searched. Also, since the frequency n-gram histogram is assigned to each sensor information as an index, sensor information similar to certain sensor information can be quickly searched. Further, these search results can be immediately presented to the inspector at the site in front of the actual equipment 21. In addition, by visualizing and displaying the characteristic unique to each facility 21 and the characteristic unique to the sensor information obtained at the present time, the presence or absence of an abnormality can be visually confirmed. With these functions, it is possible to support the inspection work based on the sound and smell, which has conventionally relied on the five senses of the worker. This leads to the advantages of improving the accuracy of abnormal sound detection, increasing the efficiency of the work, and enabling the work to be performed accurately even by non-experts.
[0110]
The above embodiment is an example of a preferred embodiment of the present invention, but the present invention is not limited to this, and various modifications can be made without departing from the gist of the present invention. For example, the sensor information is not limited to sound information, but may be vibration information, temperature information, odor information, ultrasonic waves, or the like. Since such information can also be acquired as a waveform signal, the same handling as in the above-described embodiment is possible.
[0111]
In the above-described embodiment, the sensor information other than the video information is divided and recorded in the database 3 based on the change of the video information. However, for example, the sensor information is divided and arranged by the processing described below. 3 may be recorded. That is, the signal difference ds (the difference between the average signal per unit time and the value calculated in the immediately preceding unit time) relating to the sensor information s such as sound or the sensor information s itself is input d._kAnd input d_kUsing the average μ and the standard deviation σ, the change time of the sensor information is detected by the above-described Expressions 2 to 4, and the sensor information is divided and recorded in the database 3 based on the change. In this case, even when there is almost no change in video information, such as fixed-point observation and monitoring video information obtained by fixing the video camera 4, the sensor information can be divided and arranged and recorded in the database 3. Further, it is possible to realize a function of detecting a point of occurrence of abnormal noise from monitoring video information or the like.
[0112]
The signal difference ds is obtained, for example, as follows. Let r be the sampling rate of sensor information as an input signal. That is, the number of signals input in one second is r. Let k be the total number of signals input from the start of measurement to a certain measurement time. In this case, the elapsed time is k · (1 / r) seconds. Let the value of the kth signal be s_kAnd The value of the (k + 1) th signal is s_{k + 1}Becomes Here, the unit time is set to tb. The unit time tb is an integral multiple of 1 / r. The number of signals kb per unit time is kb = tb / (1 / r). The setting of the unit time differs depending on the desired resolution in the time direction, and is generally set between several ms and several tens of ms. The average signal per unit time at a certain measurement time t1 is represented by the following equation, where k1 is the total number of signals input up to the measurement time.
[0113]
[Equation 19]

[0114]
The average signal per unit time immediately before is sv (k1-kb). Therefore, the signal difference at the measurement time t1 is ds = sv (k1) -sv (k1-kb).
[0115]
【The invention's effect】
As is clear from the above description, according to the sensor information database creation method of claim 1, the inspection support system of claim 3, and the inspection support program of claim 5, sensor information such as sound is automatically output. You can record while organizing by separating with. Also, it is possible to easily extract specific sensor information from the sensor information group that has been arranged and stored. It is not necessary for the user to record a counter or the like indicating a recording location on a recording tape or the like one by one.
[0116]
Furthermore, according to the sensor information database creation method of claim 2, the inspection support system of claim 4, and the inspection support program of claim 6, sensor information such as sound is automatically arranged for each inspection object. It becomes possible.
[0117]
Furthermore, according to the sensor information database creation method of claim 7, the inspection support system of claim 10, and the inspection support program of claim 13, based on the sensor information arranged and stored for each inspection object. Using the created index, it is possible to quickly search for another inspection target that emits sensor information similar to a certain inspection target. Therefore, the search result can be immediately presented to the inspection worker at the site where the actual inspection target is located.
[0118]
According to the sensor information database creation method of the eighth aspect, the inspection support system of the eleventh aspect, and the inspection support program of the fourteenth aspect, the index created based on the statistical characteristics of each inspection target. , It is possible to quickly search for other inspection objects that emit sensor information similar to a certain inspection object. Therefore, the search result can be immediately presented to the inspection worker at the site where the actual inspection target is located.
[0119]
According to the sensor information database creation method of the ninth aspect, the inspection support system of the twelfth aspect, and the inspection assistance program of the fifteenth aspect, the time of sensor information is stored in the index for searching for similar inspection targets. A change in direction (for example, instantaneous sound, voice, beat, etc.) can be reflected. Therefore, a high-speed search for similar inspection targets in consideration of the change in the sensor information in the time direction becomes possible.
[0120]
Further, according to the method for creating a database of sensor information according to claim 16, the inspection support system according to claim 17, and the inspection support program according to claim 18, a statistic that is a characteristic unique to the inspection target and the inspection target Can be visualized with the current sensor information acquired at the present time and visually compared. As a result, it is possible to assist the inspection worker in finding an abnormality, and it is possible to support the inspection operation based on the sound and smell that has conventionally relied on the five senses of the operator. This leads to the advantages of improving the accuracy of abnormal sound detection, increasing the efficiency of the work, and enabling the work to be performed accurately even by non-experts.
[0121]
Furthermore, according to the sensor information database creation method described in claim 19, the inspection support system described in claim 23, and the inspection support program described in claim 27, the characteristics of the sensor information created based on the sensor information are reflected. By using the index, it is possible to quickly search for sensor information acquired in the past that is similar to sensor information acquired at the present time. Therefore, the search result can be immediately presented to the inspection worker at the site where the actual inspection target is located.
[0122]
According to the sensor information database creation method described in claim 20, the inspection support system described in claim 24, and the inspection support program described in claim 28, the index of the similar sensor information is stored in the index for searching for similar sensor information. A change in direction (for example, instantaneous sound, voice, beat, etc.) can be reflected. Therefore, high-speed search for similar sensor information in consideration of a change in the sensor information in the time direction becomes possible. In addition, since a high-speed search is possible, the search result can be immediately presented to the inspection worker at the site where the actual inspection target is located.
[0123]
According to the sensor information database method, the inspection support system according to the twenty-fifth aspect, and the inspection support program according to the twenty-seventh aspect, the similar sensor information specialized in the extracted feature of the sensor information is provided. Can be searched, and the amount of calculation processing required for the search can be reduced. In addition, since a high-speed search is possible, the search result can be immediately presented to the inspection worker at the site where the actual inspection target is located.
[0124]
According to the sensor information database creation method, the inspection support system according to the twenty-sixth aspect, and the inspection assistance program according to the thirty-second aspect, the similar sensor information specialized in the extracted feature of the sensor information. Can be searched, and the amount of calculation processing required for the search can be reduced. Further, since the change in the time direction of the sensor information (for example, instantaneous sound, voice, beat, etc.) can be reflected in the index for searching for the similar sensor information, the high-speed processing of the similar sensor information in consideration of the change in the time direction of the sensor information. Search becomes possible. In addition, since a high-speed search is possible, the search result can be immediately presented to the inspection worker at the site before the actual inspection target.
[Brief description of the drawings]
FIG. 1 is a schematic functional block diagram showing an embodiment of an inspection support system according to the present invention.
FIG. 2 is a schematic flowchart showing an example of processing of a database creation method of the present invention.
FIG. 3 is a conceptual diagram showing an image of an embodiment of an inspection support system according to the present invention.
FIG. 4 is a diagram illustrating an example of a screen displayed on a display device of the work terminal.
[Explanation of symbols]
1 Inspection support system
2 Work terminal (means for acquiring sensor information)
3 database

Claims (30)

Acquiring sensor information that is information representing the state of the object and at least including video information, detecting a change in the video information, and recording sensor information other than the video information separately based on the change. A method for creating a database of sensor information, characterized in that: 2. The method according to claim 1, wherein information for identifying the target is recorded in association with each of the divided sensor information. Means for acquiring sensor information that is information representing the state of the object and that includes at least video information, detecting a change in the video information, and recording the sensor information other than the video information separately based on the change An inspection support system comprising: The inspection support system according to claim 3, further comprising: means for identifying the object; and means for recording information for identifying the object in association with each of the separated sensor information. Means for detecting a change in the video information with respect to sensor information that is information representing the state of the object and that includes at least the video information, and records the sensor information other than the video information separately based on the change. An inspection support program that causes a computer to function as a computer. The computer-readable storage medium according to claim 5, wherein the program causes a computer to function as means for recording the information for identifying the object in association with each of the separated sensor information. For each set of sensor information representing the state of each object arranged for each object, a frequency distribution is obtained from individual sensor information, and for each obtained frequency distribution, n frequency components continuous in the frequency distribution direction are obtained. Are extracted, and the frequency of the common frequency n-gram is obtained for a set of sensor information belonging to the same target, and the set of the type of the frequency n-gram and the frequency is obtained. A method for creating a database of sensor information, comprising: obtaining a frequency n-gram histogram, and searching for an object similar to a certain object based on a frequency of a frequency n-gram common to different objects. For a set of sensor information indicating the state of each target arranged for each target, individual sensor information is time-divided and frequency distributions are obtained, and statistics on a set of frequency distributions belonging to the same sensor information are further obtained. Amount, and for the statistic, extract all frequency n-grams formed by combining n frequency components that are continuous in the frequency distribution direction. Further, for the set of the frequency n-grams belonging to the same object, The frequency of the frequency n-gram to be obtained is obtained, the frequency n-gram histogram which is a set of the type of the frequency n-gram and the frequency is obtained, and based on the frequency of the frequency n-gram common to different objects, A method of creating a database of sensor information, characterized by searching for a target similar to. For a set of sensor information representing the state of each object arranged for each object, individual sensor information is time-divided and frequency distributions are obtained respectively, and further, for a time series of frequency distribution belonging to the same sensor information, Extract all extended frequency n-grams composed of a combination of n frequency components that are continuous in one or both of the time direction and the frequency distribution direction. Further, for a set of sensor information belonging to the same target, a common extended frequency n The frequency of the gram is obtained, an extended frequency n-gram histogram which is a set of the type of the extended frequency n-gram and the frequency is obtained, and based on the frequency of the extended frequency n-gram common to different objects, a certain object is obtained. A method of creating a database of sensor information, characterized by searching for a target similar to. For each set of sensor information representing the state of each object arranged for each object, a frequency distribution is obtained from individual sensor information, and for each obtained frequency distribution, n frequency components continuous in the frequency distribution direction are obtained. Are extracted, and the frequency of the common frequency n-gram is obtained for a set of sensor information belonging to the same target, and the set of the type of the frequency n-gram and the frequency is obtained. Means for obtaining a frequency n-gram histogram, and recording the frequency n-gram histogram as an index of a corresponding object; and searching for an object similar to a certain object based on the frequency of a frequency n-gram common to different objects. An inspection support system comprising: For a set of sensor information indicating the state of each target arranged for each target, individual sensor information is time-divided and frequency distributions are obtained, and statistics on a set of frequency distributions belonging to the same sensor information are further obtained. Amount, and for the statistic, extract all frequency n-grams formed by combining n frequency components that are continuous in the frequency distribution direction. Further, for the set of the frequency n-grams belonging to the same object, Means for obtaining the frequency of the frequency n-gram to be obtained, obtaining a frequency n-gram histogram which is a set of a set of the type of the frequency n-gram and the frequency, and recording the frequency n-gram histogram as an index of a corresponding object; Means for searching for an object similar to a certain object based on the frequency of a common frequency n-gram between different objects Inspection support system which is characterized the door. For a set of sensor information representing the state of each object arranged for each object, individual sensor information is time-divided and frequency distributions are obtained respectively, and further, for a time series of frequency distribution belonging to the same sensor information, Extract all extended frequency n-grams composed of a combination of n frequency components that are continuous in one or both of the time direction and the frequency distribution direction. Further, for a set of sensor information belonging to the same target, a common extended frequency n Means for obtaining the frequency of the gram, obtaining an extended frequency n-gram histogram which is a set of a set of the type of the extended frequency n-gram and the frequency, and recording the extended frequency n-gram histogram as an index of a corresponding object; Means for searching for an object similar to a certain object based on the frequency of the extended frequency n-gram common to different objects. Inspection support system characterized in that it has. For a set of sensor information indicating the state of each object arranged for each object, a frequency distribution is obtained from individual sensor information, and for each obtained frequency distribution, n frequency components continuous in the frequency distribution direction are obtained. Is extracted, and the frequency of the common frequency n-gram is obtained for a set of sensor information belonging to the same target, and the set of the type of the frequency n-gram and the frequency is obtained. Means for obtaining a frequency n-gram histogram, and recording the frequency n-gram histogram as an index of a corresponding object; and searching for an object similar to a certain object based on a frequency of a frequency n-gram common to different objects. An inspection support program characterized by causing a computer to function as means for performing inspection. For a set of sensor information indicating the state of each target arranged for each target, individual sensor information is time-divided and frequency distributions are obtained, and statistics on a set of frequency distributions belonging to the same sensor information are further obtained. Amount, and for the statistic, extract all frequency n-grams formed by combining n frequency components that are continuous in the frequency distribution direction. Further, for the set of the frequency n-grams belonging to the same object, Means for obtaining the frequency of the frequency n-gram to be obtained, obtaining a frequency n-gram histogram which is a set of a set of the type of the frequency n-gram and the frequency, and recording the frequency n-gram histogram as an index of a corresponding object; A computer is used as a means for searching for an object similar to a certain object based on the frequency of a common frequency n-gram between different objects. Inspection support for the program, characterized in that to function. For a set of sensor information representing the state of each object arranged for each object, individual sensor information is time-divided and frequency distributions are obtained respectively, and further, for a time series of frequency distribution belonging to the same sensor information, Extract all extended frequency n-grams composed of a combination of n frequency components that are continuous in one or both of the time direction and the frequency distribution direction. Further, for a set of sensor information belonging to the same target, a common extended frequency n Means for obtaining the frequency of the gram, obtaining an extended frequency n-gram histogram which is a set of a set of the type of the extended frequency n-gram and the frequency, and recording the extended frequency n-gram histogram as an index of a corresponding object; As a means for searching for an object similar to a certain object based on the frequency of the extended frequency n-gram common to different objects, Inspection support for the program, characterized in that the functioning of the over data. For a set of sensor information that represents the state of the target combined for the same target, each frequency distribution is obtained from individual sensor information belonging to the set, and the same is set for the set of obtained frequency distributions. Calculate statistics for one or both of the frequency band and the preset frequency section, and obtain the frequency distribution from the current sensor information that is the sensor information acquired at the present time, and calculate the same target as the current sensor information. A threshold value is set using the obtained statistic, and a frequency component of the current sensor information exceeding the threshold value is extracted, and a database method of sensor information is provided. For a set of sensor information that represents the state of the target combined for the same target, each frequency distribution is obtained from individual sensor information belonging to the set, and the same is set for the set of obtained frequency distributions. Means for calculating a statistic for one or both of a frequency band and a preset frequency section, and obtaining a frequency distribution from current sensor information which is sensor information acquired at the present time, for the same target as the current sensor information A means for setting a threshold using the calculated statistic and extracting a frequency component of the current sensor information exceeding the threshold, and a means for displaying the calculated statistic and the extracted frequency component. Inspection support system characterized by having at least. For a set of sensor information that represents the state of the target combined for the same target, each frequency distribution is obtained from individual sensor information belonging to the set, and the same is set for the set of obtained frequency distributions. Means for calculating a statistic for one or both of a frequency band and a preset frequency section, and obtaining a frequency distribution from current sensor information which is sensor information acquired at the present time, for the same target as the current sensor information A computer is used as a means for setting a threshold using the calculated statistic and extracting a frequency component of the current sensor information exceeding the threshold, and a means for displaying the calculated statistic and the extracted frequency component. Inspection support program characterized by functioning. The frequency distribution is obtained by time-dividing the sensor information representing the state of the object and obtaining the respective frequency distributions, and for each of the obtained frequency distributions, all the frequency n-grams formed by combining n frequency components continuous in the frequency distribution direction. With respect to the set of the frequency n-grams extracted and further belonging to the same sensor information, the frequency of the common frequency n-gram is obtained, and the frequency n-gram histogram which is a set of the type of the frequency n-gram and the frequency is obtained. And searching for other sensor information similar to certain sensor information based on the frequency of a common frequency n-gram between different sensor information. The sensor information indicating the state of the object is time-divided and the frequency distribution is obtained respectively, and the time series of the frequency distribution is divided into a plurality of groups, and for each of the groups, one or both of the time direction and the frequency distribution direction Extracting all extended frequency n-grams composed of a combination of n consecutive frequency components, and further calculating the common extended frequency n-gram frequency for the set of extended frequency n-grams belonging to the same sensor information, An extended frequency n-gram histogram, which is a set of a set of the type of the extended frequency n-gram and its frequency, is obtained, and based on the frequency of the extended frequency n-gram common to different sensor information, another similarity to certain sensor information is obtained. A method for creating a database of sensor information, characterized by searching for sensor information. With respect to the frequency components of the extracted current sensor information, all frequency n-grams formed by combining n frequency components continuous in the frequency distribution direction are extracted, and a set of the frequency n-grams belonging to the same sensor information is further extracted. , The frequency of the common frequency n-gram is obtained, the frequency n-gram histogram which is a set of the type of the frequency n-gram and the frequency is obtained, and based on the frequency of the common frequency n-gram between different sensor information. 17. The method according to claim 16, wherein another sensor information similar to a certain sensor information is searched. The time series of the frequency components of the extracted current sensor information is divided into a plurality of groups, and each of the groups is configured by combining n frequency components continuous in one or both of the time direction and the frequency distribution direction. All of the extended frequency n-grams are extracted, and for the set of the extended frequency n-grams belonging to the same sensor information, the frequency of the common extended frequency n-gram is obtained. 17. An extended frequency n-gram histogram as a set is obtained, and another sensor information similar to a certain sensor information is searched based on a frequency of an extended frequency n-gram common to different sensor information. Database method of the described sensor information. The frequency distribution is obtained by time-dividing the sensor information representing the state of the object and obtaining the respective frequency distributions, and for each of the obtained frequency distributions, all the frequency n-grams formed by combining n frequency components continuous in the frequency distribution direction. With respect to the set of the frequency n-grams extracted and further belonging to the same sensor information, the frequency of the common frequency n-gram is obtained, and the frequency n-gram histogram which is a set of the type of the frequency n-gram and the frequency is obtained. Means for recording the frequency n-gram histogram as an index of the corresponding sensor information, and means for searching for other sensor information similar to certain sensor information based on the frequency of the frequency n-gram common to different sensor information. An inspection support system comprising at least: The sensor information representing the state of the object is time-divided and the frequency distribution is obtained respectively, and the time series of the frequency distribution is divided into a plurality of groups, and each of the groups is divided into one or both of the time direction and the frequency distribution direction. Extracting all extended frequency n-grams composed of a combination of n consecutive frequency components, and further calculating the common extended frequency n-gram frequency for the set of extended frequency n-grams belonging to the same sensor information, Means for obtaining an extended frequency n-gram histogram, which is a set of pairs of types and frequencies of extended frequency n-grams, and recording the extended frequency n-gram histogram as an index of corresponding sensor information; Means for searching for other sensor information similar to certain sensor information based on the frequency of the Inspection support system characterized in that it comprises also. With respect to the frequency components of the extracted current sensor information, all frequency n-grams formed by combining n frequency components continuous in the frequency distribution direction are extracted, and a set of the frequency n-grams belonging to the same sensor information is further extracted. , The frequency of the common frequency n-gram is obtained, the frequency n-gram histogram which is a set of the type of the frequency n-gram and the frequency is obtained, and the frequency n-gram histogram is recorded as an index of the corresponding sensor information. 18. The inspection support according to claim 17, further comprising means for performing a search and another sensor information similar to a certain sensor information based on a frequency of a frequency n-gram common to different sensor information. system. The time series of the frequency components of the extracted current sensor information is divided into a plurality of groups, and each of the groups is configured by combining n frequency components continuous in one or both of the time direction and the frequency distribution direction. All of the extended frequency n-grams are extracted, and for the set of the extended frequency n-grams belonging to the same sensor information, the frequency of the common extended frequency n-gram is obtained. A means for obtaining an extended frequency n-gram histogram as a set and recording the extended frequency n-gram histogram as an index of corresponding sensor information; and a sensor based on a frequency of an extended frequency n-gram common to different sensor information. 18. A point according to claim 17, further comprising means for searching for other sensor information similar to the information. Support system. The frequency distribution is obtained by time-dividing the sensor information representing the state of the object and obtaining the respective frequency distributions, and for each of the obtained frequency distributions, all the frequency n-grams formed by combining n frequency components continuous in the frequency distribution direction. With respect to the set of the frequency n-grams extracted and further belonging to the same sensor information, the frequency of the common frequency n-gram is obtained, and the frequency n-gram histogram which is a set of the type of the frequency n-gram and the frequency is obtained. Means for recording the frequency n-gram histogram as an index of the corresponding sensor information, and means for searching for other sensor information similar to certain sensor information based on the frequency of the frequency n-gram common to different sensor information. An inspection support program that causes a computer to function as a computer. The sensor information indicating the state of the object is time-divided and the frequency distribution is obtained respectively, and the time series of the frequency distribution is divided into a plurality of groups, and for each of the groups, one or both of the time direction and the frequency distribution direction Extracting all extended frequency n-grams composed of a combination of n consecutive frequency components, and further calculating the common extended frequency n-gram frequency for the set of extended frequency n-grams belonging to the same sensor information, Means for obtaining an extended frequency n-gram histogram, which is a set of pairs of types and frequencies of extended frequency n-grams, and recording the extended frequency n-gram histogram as an index of corresponding sensor information; As a means for searching for sensor information similar to certain sensor information based on the frequency of the Inspection support for the program, characterized in that the functioning of the computer. With respect to the frequency components of the extracted current sensor information, all frequency n-grams formed by combining n frequency components continuous in the frequency distribution direction are extracted, and a set of the frequency n-grams belonging to the same sensor information is further extracted. , The frequency of the common frequency n-gram is obtained, the frequency n-gram histogram which is a set of the type of the frequency n-gram and the frequency is obtained, and the frequency n-gram histogram is recorded as an index of the corresponding sensor information. 19. The inspection according to claim 18, wherein the computer is caused to function as means for searching and for searching for other sensor information similar to certain sensor information based on the frequency of a frequency n-gram common to different sensor information. Support program. The time series of the frequency components of the extracted current sensor information is divided into a plurality of groups, and each of the groups is configured by combining n frequency components continuous in one or both of the time direction and the frequency distribution direction. All of the extended frequency n-grams are extracted, and for the set of the extended frequency n-grams belonging to the same sensor information, the frequency of the common extended frequency n-gram is obtained. A means for obtaining an extended frequency n-gram histogram as a set and recording the extended frequency n-gram histogram as an index of corresponding sensor information; and a sensor based on a frequency of an extended frequency n-gram common to different sensor information. 2. A computer functioning as means for searching for other sensor information similar to the information. Inspection support for the program described.

Images