JP4136044B2 - Image processing apparatus and image processing method therefor - Google Patents

Description

ここで、ａｉは画素値、Ａはテンプレート候補内の画素値平均であり、テンプレート候補内全体に対して和を取る。また、具体的な式は挙げないが、２次の画素値分散や、エッジ検出結果も特徴量として好ましい値である。このとき、前記値Ｔが次式、
【００３１】
【数２】

のように、ある規定値Ｔthより小さい場合は、信頼性が低いとして採用しない。例えば、図３（ｂ）における空の部分のように、画素値の変化が小さい部分では一般に後述するマッチングの精度が落ちるが、本発明の方法では（２）式のような値Ｔをもつ特徴点候補位置には特徴点が設定されず、位置関係パラメータの精度低下を防ぐことができる。
【００３２】
図４は、対応点探索部３５の構成を説明する図である。
サーチエリア設定部５１では、特徴点設定部３３から読み出した特徴点座標を基準に、対応点を探索する領域の位置と大きさを設定する。相関演算部５２では、サーチエリア設定部５１で設定されたサーチエリア内の情報に基づいて、特徴点設定部３３から入力されたテンプレートと同じ大きさを持つように対応点候補位置周囲の画像データをフレームメモリ３２ｂ中の画像データから切り出し、特徴点設定部３３から入力されたテンプレートデータとの相関値を計算する。
【００３３】
相関値判定部５３では、テンプレート抽出部４６で抽出されたテンプレートと最も相関が高い位置を判定し、結果を対応点位置として対応点位置メモリ３６に出力する。
【００３４】
このとき、一般に対応点候補は、１画素毎に移動して相関値を求めるが、画素以下の精度で対応点位置を検出するために、周囲の相関値を参照して補間により対応点位置を決定しても良い。その相関値としては、次の（３）式に示す差分の絶対値和、（４）式に示す相互相関、（５）式に示す正規化相互相関等を使うことができる。
【００３５】
【数３】

ここで、ｂｉ，Ｂは、画像Ｂのサーチエリア内で移動する小領域内の画素値とその平均であり、σ_A ，σ_B はテンプレートデータの標準偏差であり、σ_A ² ，σ_B ² がその分散値となる。
【００３６】
前記（２）〜（４）式を計算する場合、ある代表色のみについて計算すれば、高速に処理することができる。
また、Ｒ，Ｇ，Ｂ各色の相関値を加算して対応点を探索すればより高精度に対応点検索を行うことが可能になる。
【００３７】
そして補正パラメータ算出部３７では、特徴点座標データ及び、対応点座標データから、画像Ａ，Ｂ間の位置関係を決定する。求めるパラメータは（６）式の cosθ， sinθ，ｓｘ，ｓｙ，Ｍである。
【００３８】
【数４】

ここで、（ｘ，ｙ）は特徴点位置、（Ｘ，Ｙ）は対応点位置を示し、Ｍは画像間の相対的な倍率に相当する。
前記特徴点位置メモリ３４、対応点位置メモリ３６には、複数のデータが格納されているので、最小二乗的にパラメータを求めると、（７）式のようになる。
【００３９】
【数５】

ここで、Ｎは特徴点数。（８）式より、Ｍ， cosθ， sinθは（９）式の様に決定される。
【００４０】
【数６】

【００４１】
また、本来 cosθ， sinθは、 cos² θ＋ sin² θ＝１の関係があるため、対象画像に倍率変化が無いと仮定できる場合、（９）式の結果で、Ｍ＝１．０とし、（１０）式のような関係になった場合、（１１）式のように cosθ， sinθを補正したほうがよい。ｓｘ，ｓｙは補正した cosθ， sinθより計算する。
【００４２】
【数７】

【００４３】
前述した（６）〜（９）式は、任意の原点を画像の回転中心としたが、設定・探索された特徴点・対応点位置それぞれの重心は一致すると考えても構わない。特徴点の重心位置は（ /ｘ， /ｙ）（ここで、/ は、平均を意味する記号として用いている。尚、各式のイメージ情報にあるように、各式中では、この平均はオーバーバーとして表している）、対応点の重心位置は（ /Ｘ， /Ｙ）、で与えられ、 cosθ， sinθ，ｓｘ，ｓｙはそれぞれ（１２）、（１３）式で与えられる。
【００４４】
【数８】

【００４５】
以上説明したように、本実施形態は、画像中から特徴点を複数抽出し、自動的に位置あわせを行うため、業務用の高価な器材を用いることなく、一般ユーザーでもダイナミックレンジ拡大処理、被写界深度拡大処理の効果を最大限利用することができる。但し、処理の対象画像に歪曲収差が生じている場合には、任意の手法であらかじめ補正しておくことが必要となる。
【００４６】
次に、本発明による第２の実施形態について説明する。
本実施形態は、前述した第１の実施形態に比較して、特徴点設定部３３の内部の構成が異なるものである。
【００４７】
図５（ａ）は、特徴点設定部３３の概念的な構成を示す図である。この構成は、図１に示した構成に比べて領域指定部８１が加わっている点が異なる。
図５（ｂ）は、領域指定部８１の構成例を示す図である。この領域指定部８１では、入力された画像データに対し、領域指示部８３において、注目する領域を複数指示する。特徴点は、ここで指定された領域の内部に前述の方法で設定される。
【００４８】
この表示制御部８４では、指定された領域（後述する領域位置メモリ８５からのデータ）を、例えば、図６（ａ），（ｂ）のように入力画像データ上にオーバーレイし、表示部８２に、そのデータを表示する。この表示部８２は、ＰＣに付属のディスプレイでも構わないし、別に専用のものを設けても構わない。また領域指示部８３は、ＰＣに付属のマウスで指示しても構わないし、別に設けても構わない。前記表示制御部８４は、領域指定部８１の構成の一部として説明したが、パーソナルコンピュータ等のオペレーティングシステム（ＯＳ）の機能を利用するような構成でも構わない。
【００４９】
さらに、領域指示部８３では、画像中の座標で注目領域を指定しても構わない。 そして領域位置メモリ部８５は、領域指示部８３で指示された領域の位置・サイズを記憶し、領域分割部４１に出力する。領域分割部４１では、領域位置メモリ部８５からの領域位置情報をもとに、指示された領域内部に前記した方法で特徴点を設定する。
【００５０】
本実施形態の構成により、ユーザーが注目したい部分の位置合わせ精度を上げことができると共に、あまり必要でない部分では特徴点探索を必要としないので、計算時間を短縮することができる。
【００５１】
また、人間や乗り物など、動く被写体が撮影画像に含まれた場合、動いた部分の画像を計算に用いると画像全体の位置合わせに誤差が生じるが、本実施例形態の構成では、そのような部分のデータを利用しないようにできる。
【００５２】
さらに、領域指示部８３で指示した複数領域のうち特定の領域を指示し、その内部だけ他の領域より多く特徴点を設定することで、その特定領域の位置合わせ精度を更に向上することが可能である。
【００５３】
また、ダイナミックレンジ拡大技術の場合は露出値の順に、被写界深度拡大技術の場合には焦点位置の順に画像を整列し、隣り合う画像間で上記の操作を実行することで、ランダムに画像を選択して実行する場合に比べて精度を向上することができる。
【００５４】
図７は、本発明の第３の実施形態を示す図である。
図７に示した画像補正部３０の構成は、図１に示した画像補正部３０の構成に加えて、表示部８２、表示制御部８４、特徴点指示部９０が含まれる。複数の特徴点位置は前述の方法で設定され、特徴点メモリ３４に記憶される。
【００５５】
この特徴点メモリ３４に記憶された特徴点座標は特徴点指示部９０に送られ、表示制御部８４を通して表示部８２に対象画像上にその位置がオーバーレイされる（例えば、図３（ｂ））
このとき、特徴点が人間・乗り物等動く被写体（移動物体）上に設定されていると、（６）〜（１０）式で計算する補正係数の誤差の原因となる。そこで、特徴点指示部９０では、移動物体が含まれている場合は、その物体を検出し、移動物体上に設定された特徴点は計算に使うデータから除外し、特徴点メモリ３４からデータを削除する。
【００５６】
この操作は、表示されたデータをユーザーが判断し、特徴点指示部９０から除外するデータを示するようにしてもよい。また、対応点探索後、特徴点メモリ３４と対応点メモリ３６のデータを同時に表示部８２に表示し、対応が取れていないとユーザーが判断した場合は、特徴点指示部９０により、そのデータを除外することを指示できる用にしても構わない。
【００５７】
次に図８を参照して、本発明による第４の実施形態について説明する。この第４の実施形態は、図５の領域指定部８１を図８に示すような構成に変更したものである。
【００５８】
図８に示される領域指定部８１は、領域指示部８３、領域位置メモリ８５、状態判定部１０１から構成される。
本実施形態では、画像の統計的・物理的な性質に基づいて特徴点設定領域に指定する範囲を決定する。
【００５９】
まず、画像全体を図９（ａ）のように予め決められた大きさに分割する。このとき分割するサイズは、特徴点設定部で切り出されるテンプレートより大きい範囲とする。
【００６０】
分割された各部分は状態判定部１０１に送られ、特徴点設定領域に指定するかどうかを判断する。状態判定部１０１では入力された画像データを走査し、飽和した画素が一定の割合を超えている場合は特徴点設定領域として適さないと判断する。
【００６１】
また、状態判定部１０１では、入力データの平均値がある閾値を超えている場合は、特徴点設定領域として適さないと判断しても構わない。
領域指示部８３は、図９（ｂ）のように、分割した領域の画像データを順番に状態判定部１０１に出力し、特徴点設定領域として適していると判断された場合は、その範囲を含むように領域位置メモリ８５に記録する。
【００６２】
前述した説明では、各分割領域ごとに判断したが、状態判定部に出力する画像データとして、領域位置メモリ８５に記録されている範囲全体にしても構わない。また、図９（ｃ）に示したように、領域指示部８３で、ユーザーがある分割領域を指定し、その周辺の領域から順番に特徴点指定領域に含んでいくようにしても良い。
【００６３】
さらに、図１０に示すように、領域指示部８３で、ユーザーがある１点を指定し、その周りに拡張した矩形を仮定し、その画像データを状態判定部１０１で判断するようにしてもよい。
【００６４】
また、飽和した画素値を判断基準に用いたが、状態判定部１０１では、入力した画像の離散フーリエ変換、離散コサイン変換といった直交変換を画像に施した時の高周波成分のパワーや、ウェーブレット変換の成分によって判断しても構わない。
【００６５】
さらに、状態判定部１０１では、入力されたデータのコントラストや画素値最大と最小の差を判断に用いても構わない。
次に図１１を参照して、本発明による第５の実施形態について説明する。
【００６６】
本実施形態は、３枚以上の画像が処理対象になっている例である。ここで、図示するうち、前述した図１と同等の構成部位には、同じ参照符号を付してその説明を省略する。
【００６７】
本実施形態において、画像データ再生部３１で再生された画像データは、画像選択指示部１１４の指示により切換動作する画像切替部１１１によっていずれかのフレームメモリ３２ａ，３２ｂに入力される。この画像選択指示部１１４は、入力された同一被写体が撮影された複数の画像を順次処理できるように装置の処理時間等を見計らって、２つのフレームメモリに対して交互に入力されるように画像切換部１１１を制御するものである。
【００６８】
前記フレームメモリ３２ａ，３２ｂに格納された画像から前述した方法で特徴点・対応点を検出し、補正パラメータ算出部３７で画像間の位置関係（平行移動量・回転角度・相対倍率）を算出する。補正パラメータ算出部３７で算出されたパラメーターは、画像ＩＤ、参照画像ＩＤと共に補正パラメータ記憶部１１３に記憶される。
【００６９】
前記補正パラメータ変換部１１２は、補正パラメータ記憶部１１３に記憶されたデータを読み出し、それぞれのデータを予め指定された補正の基準となる画像からみた平行移動量・回転角度に変換する。続いて、補間演算部３８では、補正パラメータ変換部１１２で補正基準画像からみたパラメータに従って画像を補正する。
【００７０】
ここで補正パラメータ変換部１１２でのデータ変換及び補間演算部３８での補正は、画像切替部１１１の操作毎に行っても構わないし、対象画像全てについて相対的な位置関係を算出した後、改めてフレームメモリ３２に画像を読み込んで行っても構わない。
【００７１】
また、本実施形態ではフレームメモリ３２を２つとし、信号切替部１１１により入力画像を切り替えるように説明したが、対象画像全てを格納できる数のフレームメモリを用意しても構わない。
【００７２】
以上の実施形態について説明したが、本明細書には以下のような発明も含まれている。
（１） 撮影条件の異なる複数の画像において、
画像を複数の分割領域に分割し、前記分割領域中におのおのの分割領域と同じまたは小さい特徴点探索領域を設け、それぞれの特徴点探索領域から画像の統計量に基づいて被写体の特徴的な位置情報を抽出し、前記複数の画像のうち、被写体の特徴的な位置情報を抽出した画像とは異なる画像中で対応する点を探索し、前記特徴的な位置情報及び対応する点の位置情報より補正パラメータ（画像間の回転・移動・倍率変化等）を検出し、前記補正パラメータとしての画像間の回転・移動・倍率変化に基づいて画像を補正する画像処理装置。
【００７３】
本発明は、第１の実施形態に対応する。
これにより、複数の画像を重ねる際に、より正確な画像の重ね合わせ行うことができるようになる。
（２） 前記（１）項に記載の画像処理装置において、
前記回転・移動・倍率変化に基づいて画像を補正する際に特定の基準画像から見た回転・移動・倍率変化を表わすようにパラメータを変換してから補正することを特徴とする。
【００７４】
本発明は第５の実施形態に対応する。
これにより、複数の画像を重ねる際に、それぞれの画像に対して適当な任意の補正パラメータを設定でき、撮影者の意図に応じた、より正確な画像の重ね合わせを行うことができるようになる。
（３） 前記複数の分割領域について、
分割する領域を画像の一部に制限することを特徴とする１に記載の画像処理装置。
【００７５】
本発明は、第２の実施形態に対応する。
これにより、複数の画像を重ねる際に、画像処理の速度が向上すると共に、それぞれの画像に対して適当な領域を設定でき、撮影者の意図に応じた、より正確な画像の重ね合わせを行うことができるようになる。
（４） 前記（１）項または（３）項に記載の画像処理装置において、
前記画像の統計量に基づく被写体の特徴的な位置情報のうち、複数の画像間で位置が移動している被写体上に設定された位置情報は画像間の回転・移動・倍率変化の検出に用いないことを特徴とする。
【００７６】
本発明は、第３の実施形態に対応する。
これにより、複数の画像を重ねる際に、移動物体が存在する領域を排除しているので、より正確な画像の重ね合わせを行うことができるようになる。
（５） 前記（３）項に記載の画像処理装置において、
分割する領域を画像の一部に制限する制限領域は、直交変換したときの高周波成分の割合、飽和した画素の割合、画素値の平均値、最大及び最小の画素値の差、コントラストのいずれかの値を基準に制限されることを特徴とする。
【００７７】
本発明は、第４の実施形態に対応する。
これにより、複数の画像を重ねる際に、画像処理の速度が向上すると共に、それぞれの画像に対して適当な領域を自動的に設定することができ、撮影者の意図に応じた、より正確な画像の重ね合わせを行うことができるようになる。
（６） 前記（５）項に記載の画像処理装置において、
特定点を中心に制限領域を広げていくことを特徴とする。
【００７８】
本発明は第４の実施形態に対応する。
これにより、複数の画像を重ねる際に、特定点を中心に、それぞれの画像に対して適当な領域を設定するので、より正確な画像の重ね合わせを行うことができるようになる。
【００７９】
【発明の効果】
以上詳述したように本発明によれば、ほぼ同一の構図で露出や焦点位置など撮影条件が異なる複数の画像間の位置関係を検出し、検出した位置関係を示すパラメータの値によって、正確に画像が重なるように補正する画像処理装置及びその画像処理方法を提供することができる。
【図面の簡単な説明】
【図１】第１の実施形態としての画像処理装置の構成例を示す図である。
【図２】図１に示した特徴点設定部の構成例を示す図である。
【図３】図３（ａ）は、画像を複数の分割した分割領域と、各分割領域に設定された特徴点探索領域を示す図、図３（ｂ）は、特徴点について説明するための図である。
【図４】図１に示した対応点探索部の構成例を示す図である。
【図５】図５（ａ）は、第２の実施形態としての画像処理装置の構成例を示す図、図５（ｂ）は、領域指定部の構成例を示す図である。
【図６】図６（ａ）は、画像上に表示された指定領域の一例を示す図、図６（ｂ）は、画像上に表示された注目位置を示す図である。
【図７】第３の実施形態としての画像処理装置の構成例を示す図である。
【図８】第４実施形態としての画像処理装置に用いられる領域指定部の構成例を示す図である。
【図９】図９（ａ）は、複数の分割領域に分割された入力画像を示す図、図９（ｂ）は、走査による特徴点設定領域の指定について説明するための図、図９（ｃ）は、拡張による特徴点設定領域の指定について説明するための図である。
【図１０】指定点から拡張した矩形を特徴点設定領域として指定することについて説明するための図である。
【図１１】第５実施形態としての画像処理装置の構成例を示す図である。
【図１２】従来の広ダイナミックレンジ画像を作成するための構成例を示す図である。
【図１３】従来の深い被写界深度を有する画像を作成するための構成例を示す図である。
【符号の説明】
３０…画像補正部
３１…画像データ再生部
３２…フレームメモリ
３３…特徴点設定部
３４…特徴点位置メモリ
３５…対応点探索部
３６…対応点位置メモリ
３７…補正パラメータ算出部
３８…補間演算部
３９…画像合成部[0001]
BACKGROUND OF THE INVENTION
The present invention relates to composition processing of images that exceed the dynamic range and depth of field of an input device from a plurality of images obtained by shooting the same subject under different shooting conditions, and in particular, the subject matches when the images are superimposed. It relates to an image processing apparatus and an image processing how to align corrected to.
[0002]
[Prior art]
In general, an image processing technique for creating an image having a dynamic range that is greater than or equal to a dynamic range unique to an imaging element such as a CCD (Charge Coupled Device) by combining a plurality of images obtained by photographing the same subject with different exposures (For example, Japanese Patent Application Laid-Open No. 63-232591) or a plurality of images obtained by photographing the same subject at different focal positions, and an image having a depth of field deeper than the depth of field of the input optical system. There is an image processing technique to be created (Japanese Patent Laid-Open No. 1-309478).
[0003]
This technique for creating a wide dynamic range image is realized by the configuration shown in FIG.
Images A and B shown in the figure are images taken with different exposures, and the relationship between the amount of incident light incident on the image sensor and the signal value of the image obtained by the image input device is shown in FIG. “[A] Long time exposure”, “[B] Short time exposure”.
[0004]
In the adder 11, the pixel values of the images A and B are added, and a value indicated as “[C] addition signal” in FIG. 12B is stored in the frame memory 12.
The shape of this “[C] addition signal” is determined by the exposure ratio of the two images. The linear conversion unit 13 estimates the incident light amount I from the added signal value S, and the matrix circuit 15 converts the signal values of R, B, and G colors into luminance signal values Y.
[0005]
Then, the luminance compression unit 16 compresses the luminance value so that the estimated incident light quantity falls within the predetermined number of gradations, and outputs the compressed luminance signal value Y ′. In the divider 17, Y ′ / Y is calculated, multiplied by the signals of R, G, and B, which are the outputs of the linear converter 16, by the multiplier 18, and the result is stored in the frame memory 19.
[0006]
Since this image processing technique uses a plurality of image data, an image with good color reproduction and less noise can be obtained as compared with a case where a short-time exposure image is simply subjected to tone curve processing.
[0007]
Further, the image processing technique for creating an image having a depth of field deeper than the depth of field of the input optical system described above is realized, for example, with the configuration shown in FIG.
First, images taken with different focal positions are weighted and added by the adder 21 and stored in the frame memory 22. It is known that blurring of the entire image is made uniform by weighted addition of an image with the focal position changed according to the focal position.
[0008]
Therefore, the recovery filter setting circuit 25 determines a blur recovery filter in the image after the addition, and outputs it to the matrix operation circuit 23. Here, the method for creating the blur recovery filter is not a part related to the essence of the present invention, and therefore will be omitted. The matrix operation circuit 23 applies a recovery filter to the added image and stores the result in the frame memory 24.
As a result, the image stored in the frame memory 24 is an image having a deeper depth of field than the imaging optical system.
[0009]
[Problems to be solved by the invention]
In the image processing technique described above, a captured image is temporarily added or weighted for each pixel, and then a series of processing such as dynamic range compression processing and recovery processing is performed. For this reason, if the images to be added are misaligned with each other, they are shifted when added, and an accurate result cannot be obtained.
[0010]
For this reason, in the case of dynamic range expansion processing, nondestructive readout is possible, so in principle, a device using a CMD (Charge Modurated Device) element that does not cause a positional shift while taking an image with multiple exposures. In the process of using and increasing the depth of field, an expensive apparatus for business use is required such as a solid stage is required. Further, when shooting with the focal position changed, a subtle change in magnification for each focal position also affects.
[0011]
On the other hand, digital cameras, scanners, tripods, etc. used by general users are mostly simple ones that cannot be expected to be accurate compared to commercial equipment. Therefore, the above-described advantages of the image processing technique cannot be fully utilized.
[0012]
For example, when shooting with a digital camera on a tripod, the camera moves when you press the shutter, or when you take an image taken with a silver salt camera with a film scanner, they will deviate from each other depending on how you insert the film Etc.
[0013]
Therefore, the present invention detects a positional relationship between a plurality of images having substantially the same composition and different shooting conditions such as exposure and focal position, and corrects the images so that they are accurately overlapped by a parameter value indicating the detected positional relationship. An object is to provide an image processing apparatus and an image processing method thereof.
[0014]
[Means for Solving the Problems]
In order to achieve the above object, the present invention is an image processing apparatus for performing alignment processing between a plurality of images obtained by photographing substantially the same composition under different photographing conditions. A search area setting unit that divides the image into divided areas and provides a search area for feature points that represent the features of the image in each of the divided areas; and based on the features of the image from the search areas of the feature points, Position information that extracts characteristic position information of the subject in the composition excluding position information related to the coordinates of the divided area where the object moving is detected in relation to the coordinates for each divided area of the divided image An image processing apparatus comprising: an extracting unit .
Further, an image processing method for performing alignment processing between a plurality of images obtained by capturing substantially the same composition under different shooting conditions, wherein at least one of the input images is divided into a plurality of divided regions, A search area setting step for providing a search area for a feature point representing an image feature in the area, and a division in which an object moving in the image is detected from the search area of each feature point based on the feature amount of the image A position information extracting step of extracting characteristic position information of the subject in the composition excluding position information related to the coordinates of the area in association with coordinates for each divided area of the divided image. I will provide a.
[0015]
The image processing apparatus configured as described above selects one of a plurality of input images, divides the image into a plurality of regions, sets feature points from the feature amounts of the divided regions, The coordinate position of the feature point is stored. Based on the coordinates of the feature point, for each input image that was not selected, search for the corresponding point, extract the coordinate position of the corresponding point, After calculating the positional relationship (parallel movement amount, rotation angle) and correcting one or both of the images so that the subject matches, dynamic range compression processing, depth of field expansion processing, etc. Apply.
[0016]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 shows a configuration example of an image processing apparatus according to the first embodiment of the present invention.
[0017]
In the present embodiment, an image taken with a digital camera (not shown) or an image taken with a silver halide camera and taken with a scanner or the like is recorded in a storage medium such as a hard disk, a portable floppy disk, or a memory card.
[0018]
The present embodiment can be broadly divided into an image data reproduction unit 31 that reads an image recorded on the hard disk, floppy disk, memory card, etc., and performs processing such as decompression, and an input image from the image data reproduction unit 31. An image correction unit 30 that corrects the positional relationship of the subject of the image by performing an interpolation operation by a method that will be described later, and the above-described dynamic range expansion and depth of field for the image whose positional relationship is corrected by the image correction unit 30 The image composition unit 39 executes enlargement.
[0019]
The image correction unit 30 also includes a frame memory 32 that temporarily stores the image data that has been decompressed by the image data reproduction unit 31, and a plurality of feature points (coordinates) that will be described later for one selected image. ), A feature point position memory 34 for storing the feature point coordinates, and a corresponding point in another image from the feature point coordinates and image data around the feature point cut out as a template. Corresponding point search unit 35 for searching, corresponding point position memory 36 for storing the obtained corresponding point coordinate position, and positional relationship (parallel movement amount, rotation angle) between two images from the feature point coordinates and the corresponding point coordinates A correction parameter calculation unit 37 for calculating the image and an interpolation calculation unit 38 that corrects one or both images from the input positional relationship and outputs the corrected image to the image composition unit 39.
[0020]
In the image processing method of the present embodiment configured as described above, an image read from the hard disk, memory card, or the like is subjected to processing such as decompression by the image data reproduction unit 31, and the frame of the image correction unit 30 Stored in the memory 32. Here, the data output from the image data reproducing unit 31 includes not only the pixel value of the image but also additional data such as the image size, the number of colors, and a color map.
[0021]
Next, one of the input images is selected (here, image A) and input to the feature point setting unit 33.
The feature point setting unit 33 sets a plurality of feature points (coordinates) by a method described later, and the feature point coordinates are input to the feature point position memory 34. The set feature point coordinates are also input to the corresponding point search unit 35, and simultaneously, image data around the feature point is cut out as a template and input to the corresponding point search unit 35.
[0022]
The corresponding point search unit 35 searches for a point corresponding to the input template in an image different from the image input to the feature point setting unit 33 (here, the image B). A method for searching for corresponding points will be described later.
[0023]
As a result, the corresponding point coordinate position is output to the corresponding point position memory 36. The correction parameter calculation unit 37 reads the feature point coordinates in the feature point position memory 34 and the corresponding point coordinates in the corresponding point position memory 36, and calculates the positional relationship (parallel movement amount, rotation angle) between the two images. The interpolation calculation unit 38 corrects one or both images from the input positional relationship and outputs the corrected image to the image composition unit 39.
[0024]
The image composition unit 39 performs the above-described dynamic range expansion processing and depth-of-field expansion processing. However, since the target image is an image that has been aligned by the previous image correction unit 30, it is naturally accurate. An image superimposed on is obtained as a result image.
[0025]
FIG. 2 shows and describes the configuration of the feature point setting unit 33.
The feature point setting unit 33 determines a plurality of feature point positions. However, when many feature points are concentrated on a part of the image, an alignment error may increase in other parts. In view of this, the region dividing unit 41 divides the image into a plurality of regions and sets a feature point search region therein so that the feature points are appropriately scattered in the image. Here, FIG. 3A shows an example in which the whole is divided equally.
[0026]
A small region that is a template candidate is moved in this region, and the position where the feature amount is maximized is output as one of the feature points. However, since it takes a very long processing time to search the whole, it is desirable to set a feature point search area smaller than the divided area as shown in FIG. The size of the feature point search area is determined in advance based on the image size, for example, 5% of the image size. Alternatively, the size of the divided small area may be used as a reference.
[0027]
Next, the template position determination unit 42 determines a template candidate position in the feature point search region, and the template extraction unit 43 cuts out image data in the template candidate based on the position. At this time, the template position determination unit 42 sequentially moves the position of the template candidate within the feature search area.
[0028]
Since the change in the feature amount of the template candidate that is close in position may not be so large, in such a case, the position of the template candidate can be shifted every several pixels instead of every pixel. The amount of calculation can be reduced.
[0029]
The feature amount calculation unit 44 calculates the feature amount of the template candidate, and the feature amount determination unit 45 determines the template position where the feature amount is maximized. The template extraction unit 46 extracts data around the position where the feature amount is maximized as a template. The template extracting unit 46 may serve as the template extracting unit 43.
As this feature quantity, for example, pixel value dispersion as shown in equation (1) can be used.
[0030]
[Expression 1]

Here, ai is the pixel value, A is the average pixel value in the template candidates, and is summed over the entire template candidates. Although no specific expression is given, secondary pixel value dispersion and edge detection results are also preferable values as feature amounts. At this time, the value T is expressed by the following equation:
[0031]
[Expression 2]

As described above, when the value is smaller than a predetermined value Tth, it is not adopted because the reliability is low. For example, the accuracy of matching, which will be described later, generally decreases at a portion where the change in the pixel value is small, such as an empty portion in FIG. 3B, but the method of the present invention has a feature having a value T as shown in equation (2). No feature points are set at the point candidate positions, and it is possible to prevent a decrease in accuracy of the positional relationship parameters.
[0032]
FIG. 4 is a diagram illustrating the configuration of the corresponding point search unit 35.
The search area setting unit 51 sets the position and size of the area for searching for the corresponding points based on the feature point coordinates read from the feature point setting unit 33. In the correlation calculation unit 52, based on the information in the search area set by the search area setting unit 51, image data around the corresponding point candidate position so as to have the same size as the template input from the feature point setting unit 33. Is extracted from the image data in the frame memory 32b, and a correlation value with the template data input from the feature point setting unit 33 is calculated.
[0033]
The correlation value determination unit 53 determines a position having the highest correlation with the template extracted by the template extraction unit 46 and outputs the result to the corresponding point position memory 36 as a corresponding point position.
[0034]
At this time, generally, the corresponding point candidate moves for each pixel to obtain the correlation value. In order to detect the corresponding point position with an accuracy of less than the pixel, the corresponding point position is determined by interpolation with reference to the surrounding correlation value. You may decide. As the correlation value, the sum of absolute values of differences shown in the following equation (3), cross-correlation shown in equation (4), normalized cross-correlation shown in equation (5), and the like can be used.
[0035]
[Equation 3]

Here, bi and B are the pixel values in the small area moving within the search area of the image B and the average thereof, σ _A and σ _B are the standard deviations of the template data, and σ _A ² and σ _B ² Is the variance value.
[0036]
When calculating the equations (2) to (4), if only a representative color is calculated, the processing can be performed at high speed.
In addition, if the corresponding points are searched by adding the correlation values of the R, G, and B colors, the corresponding points can be searched with higher accuracy.
[0037]
Then, the correction parameter calculation unit 37 determines the positional relationship between the images A and B from the feature point coordinate data and the corresponding point coordinate data. The parameters to be obtained are cos θ, sin θ, sx, sy, and M in equation (6).
[0038]
[Expression 4]

Here, (x, y) represents a feature point position, (X, Y) represents a corresponding point position, and M corresponds to a relative magnification between images.
Since a plurality of data is stored in the feature point position memory 34 and the corresponding point position memory 36, when the parameter is obtained in the least squares, the equation (7) is obtained.
[0039]
[Equation 5]

Here, N is the number of feature points. From equation (8), M, cos θ, and sin θ are determined as in equation (9).
[0040]
[Formula 6]

[0041]
In addition, since cosθ and sinθ originally have a relationship of cos ² θ + sin ² θ = 1, when it can be assumed that there is no magnification change in the target image, M = 1.0 is obtained as a result of equation (9), When the relationship is as shown in equation (10), it is better to correct cos θ and sin θ as in equation (11). sx and sy are calculated from corrected cos θ and sin θ.
[0042]
[Expression 7]

[0043]
In the above-described equations (6) to (9), an arbitrary origin is set as the rotation center of the image. However, it may be considered that the centroids of the set / searched feature points and corresponding point positions coincide with each other. The centroid position of the feature point is (/ x, / y) (where / is used as a symbol meaning the average. In addition, as shown in the image information of each equation, this average is The barycentric position of the corresponding point is given by (/ X, / Y), and cos θ, sin θ, sx, sy are given by equations (12) and (13), respectively.
[0044]
[Equation 8]

[0045]
As described above, the present embodiment extracts a plurality of feature points from an image and automatically aligns them. Therefore, even a general user can perform dynamic range expansion processing and processing without using expensive business equipment. The effect of the depth of field expansion process can be utilized to the maximum extent. However, when distortion is generated in the processing target image, it is necessary to correct it in advance by an arbitrary method.
[0046]
Next, a second embodiment according to the present invention will be described.
This embodiment differs from the first embodiment described above in the internal configuration of the feature point setting unit 33.
[0047]
FIG. 5A is a diagram illustrating a conceptual configuration of the feature point setting unit 33. This configuration is different from the configuration shown in FIG. 1 in that an area specifying unit 81 is added.
FIG. 5B is a diagram illustrating a configuration example of the area specifying unit 81. In the area designating unit 81, a plurality of target areas are instructed in the area instruction unit 83 for the input image data. The feature points are set in the area specified here by the method described above.
[0048]
The display control unit 84 overlays the designated area (data from the area position memory 85 described later) on the input image data as shown in FIGS. , Display the data. The display unit 82 may be a display attached to the PC, or a dedicated display may be provided. The area instruction unit 83 may be instructed by a mouse attached to the PC or may be provided separately. The display control unit 84 has been described as a part of the configuration of the area specifying unit 81, but may be configured to use an operating system (OS) function of a personal computer or the like.
[0049]
Further, the area instruction unit 83 may designate a region of interest with coordinates in the image. Then, the area position memory unit 85 stores the position / size of the area designated by the area instruction unit 83 and outputs it to the area dividing unit 41. Based on the region position information from the region position memory unit 85, the region dividing unit 41 sets feature points in the designated region by the method described above.
[0050]
With the configuration of the present embodiment, it is possible to increase the alignment accuracy of the portion that the user wants to pay attention to, and it is possible to reduce the calculation time because the feature point search is not required in the portion that is not so necessary.
[0051]
In addition, when a moving subject such as a person or a vehicle is included in the photographed image, an error occurs in the alignment of the entire image when the image of the moved part is used for the calculation. You can avoid using partial data.
[0052]
Furthermore, it is possible to further improve the alignment accuracy of the specific area by instructing a specific area among a plurality of areas instructed by the area instructing unit 83 and setting more feature points only in the inside than other areas. It is.
[0053]
In the case of the dynamic range expansion technique, images are arranged in the order of exposure values, and in the case of the depth of field expansion technique, the images are arranged in the order of the focal positions, and the above operation is performed between adjacent images, so that the images are randomly selected. The accuracy can be improved as compared with the case of selecting and executing.
[0054]
FIG. 7 is a diagram showing a third embodiment of the present invention.
The configuration of the image correction unit 30 illustrated in FIG. 7 includes a display unit 82, a display control unit 84, and a feature point instruction unit 90 in addition to the configuration of the image correction unit 30 illustrated in FIG. A plurality of feature point positions are set by the method described above and stored in the feature point memory 34.
[0055]
The feature point coordinates stored in the feature point memory 34 are sent to the feature point instruction unit 90, and the position is overlaid on the target image on the display unit 82 through the display control unit 84 (for example, FIG. 3B).
At this time, if the feature point is set on a moving subject (moving object) such as a person or a vehicle, it causes an error of the correction coefficient calculated by the equations (6) to (10). Therefore, the feature point instruction unit 90 detects a moving object if it is included, excludes the feature point set on the moving object from the data used for calculation, and stores the data from the feature point memory 34. delete.
[0056]
In this operation, the user may determine the displayed data and indicate data to be excluded from the feature point instruction unit 90. Further, after the corresponding point search, the data of the feature point memory 34 and the corresponding point memory 36 are simultaneously displayed on the display unit 82, and when the user determines that the correspondence is not achieved, the feature point indicating unit 90 stores the data. It may be used for instructing exclusion.
[0057]
Next, a fourth embodiment according to the present invention will be described with reference to FIG. In the fourth embodiment, the area designating unit 81 shown in FIG. 5 is changed to the configuration shown in FIG.
[0058]
The area designation unit 81 shown in FIG. 8 includes an area instruction unit 83, an area position memory 85, and a state determination unit 101.
In the present embodiment, the range to be designated as the feature point setting area is determined based on the statistical and physical properties of the image.
[0059]
First, the entire image is divided into a predetermined size as shown in FIG. The size to be divided at this time is larger than the template cut out by the feature point setting unit.
[0060]
Each of the divided parts is sent to the state determination unit 101, and it is determined whether or not to specify the feature point setting area. The state determination unit 101 scans the input image data, and determines that it is not suitable as a feature point setting region if the saturated pixels exceed a certain ratio.
[0061]
Further, the state determination unit 101 may determine that it is not suitable as a feature point setting area when the average value of input data exceeds a certain threshold.
As shown in FIG. 9B, the region instruction unit 83 sequentially outputs the image data of the divided regions to the state determination unit 101, and when it is determined that the region is appropriate as the feature point setting region, the range is displayed. It records in the area position memory 85 so as to include it.
[0062]
In the above description, the determination is made for each divided region, but the entire range recorded in the region position memory 85 may be used as the image data output to the state determination unit. Further, as shown in FIG. 9C, the user may designate a certain divided area by the area instructing unit 83 and include it in the feature point designation area in order from the surrounding area.
[0063]
Further, as shown in FIG. 10, the user may designate a certain point in the area instruction unit 83 and assume a rectangle expanded around the point, and the state determination unit 101 may determine the image data. .
[0064]
In addition, although the saturated pixel value is used as a determination criterion, the state determination unit 101 uses the power of a high-frequency component when the image is subjected to orthogonal transform such as discrete Fourier transform or discrete cosine transform of the input image, or wavelet transform. You may judge by a component.
[0065]
Further, the state determination unit 101 may use the contrast of input data or the difference between the maximum and minimum pixel values for determination.
Next, a fifth embodiment of the present invention will be described with reference to FIG.
[0066]
This embodiment is an example in which three or more images are processed. Here, in the figure, the same components as those in FIG. 1 described above are denoted by the same reference numerals, and the description thereof is omitted.
[0067]
In the present embodiment, the image data reproduced by the image data reproducing unit 31 is input to one of the frame memories 32 a and 32 b by the image switching unit 111 that performs a switching operation according to an instruction from the image selection instruction unit 114. The image selection instruction unit 114 estimates the processing time of the apparatus so that a plurality of images taken of the same subject that have been input can be sequentially processed, and the images are input alternately to the two frame memories. The switching unit 111 is controlled.
[0068]
The feature points / corresponding points are detected from the images stored in the frame memories 32a and 32b by the above-described method, and the positional relationship (parallel movement amount / rotation angle / relative magnification) between the images is calculated by the correction parameter calculation unit 37. . The parameters calculated by the correction parameter calculation unit 37 are stored in the correction parameter storage unit 113 together with the image ID and the reference image ID.
[0069]
The correction parameter conversion unit 112 reads data stored in the correction parameter storage unit 113, and converts each data into a parallel movement amount / rotation angle viewed from an image serving as a reference for correction specified in advance. Subsequently, in the interpolation calculation unit 38, the correction parameter conversion unit 112 corrects the image according to the parameters viewed from the correction reference image.
[0070]
Here, the data conversion in the correction parameter conversion unit 112 and the correction in the interpolation calculation unit 38 may be performed for each operation of the image switching unit 111. After calculating the relative positional relationship for all the target images, the data conversion is performed again. The image may be read into the frame memory 32.
[0071]
In the present embodiment, two frame memories 32 are used and the input image is switched by the signal switching unit 111. However, a number of frame memories that can store all target images may be prepared.
[0072]
Although the above embodiments have been described, the present invention includes the following inventions.
(1) In multiple images with different shooting conditions,
An image is divided into a plurality of divided areas, and feature point search areas that are the same as or smaller than each of the divided areas are provided in the divided areas, and the characteristic position of the subject is determined from each feature point search area based on image statistics. Information is extracted, and a corresponding point in the image different from the image from which the characteristic position information of the subject is extracted is searched from among the plurality of images, and the characteristic position information and the position information of the corresponding point are searched. An image processing apparatus that detects a correction parameter (rotation / movement / magnification change between images) and corrects an image based on rotation / movement / magnification change between images as the correction parameter.
[0073]
The present invention corresponds to the first embodiment.
As a result, more accurate images can be superimposed when a plurality of images are superimposed.
(2) In the image processing apparatus according to (1),
When the image is corrected based on the rotation / movement / magnification change, the correction is performed after converting the parameters so as to represent the rotation / movement / magnification change seen from a specific reference image.
[0074]
The present invention corresponds to the fifth embodiment.
As a result, when a plurality of images are overlaid, any appropriate correction parameter can be set for each image, and more accurate image superposition according to the photographer's intention can be performed. .
(3) For the plurality of divided regions,
2. The image processing apparatus according to 1, wherein a region to be divided is limited to a part of an image.
[0075]
The present invention corresponds to the second embodiment.
As a result, the speed of image processing is improved when a plurality of images are superimposed, and an appropriate area can be set for each image, and more accurate images can be superimposed according to the intention of the photographer. Will be able to.
(4) In the image processing apparatus according to (1) or (3),
Of the characteristic position information of the subject based on the statistics of the image, the position information set on the subject whose position is moving between a plurality of images is used to detect rotation, movement, and magnification change between images. It is characterized by not.
[0076]
The present invention corresponds to the third embodiment.
Thereby, when overlapping a plurality of images, the area where the moving object exists is excluded, so that more accurate image overlapping can be performed.
(5) In the image processing apparatus according to (3),
The restricted area that limits the area to be divided to a part of the image is one of the ratio of high frequency components, the ratio of saturated pixels, the average value of pixel values, the difference between maximum and minimum pixel values, and contrast It is limited on the basis of the value of.
[0077]
The present invention corresponds to the fourth embodiment.
As a result, when a plurality of images are overlaid, the speed of image processing is improved, and an appropriate area can be automatically set for each image, which is more accurate according to the photographer's intention. Images can be superimposed.
(6) In the image processing device according to (5),
It is characterized by expanding the restricted area around a specific point.
[0078]
The present invention corresponds to the fourth embodiment.
As a result, when a plurality of images are overlapped, an appropriate region is set for each image with a specific point as the center, so that more accurate image overlap can be performed.
[0079]
【The invention's effect】
As described above in detail, according to the present invention, the positional relationship between a plurality of images having different photographing conditions such as exposure and focal position with the substantially same composition is detected, and the value of the parameter indicating the detected positional relationship is accurately determined. the image processing apparatus and an image processing how to correct such images overlap can be provided.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a configuration example of an image processing apparatus according to a first embodiment.
FIG. 2 is a diagram illustrating a configuration example of a feature point setting unit illustrated in FIG. 1;
FIG. 3A is a diagram showing a plurality of divided regions obtained by dividing an image and feature point search regions set in each divided region, and FIG. 3B is a diagram for explaining feature points. FIG.
4 is a diagram illustrating a configuration example of a corresponding point search unit illustrated in FIG. 1;
FIG. 5A is a diagram illustrating a configuration example of an image processing apparatus according to a second embodiment, and FIG. 5B is a diagram illustrating a configuration example of an area designating unit.
FIG. 6A is a diagram illustrating an example of a designated area displayed on an image, and FIG. 6B is a diagram illustrating a target position displayed on the image.
FIG. 7 is a diagram illustrating a configuration example of an image processing apparatus as a third embodiment.
FIG. 8 is a diagram illustrating a configuration example of an area designating unit used in an image processing apparatus as a fourth embodiment.
9A is a diagram showing an input image divided into a plurality of divided regions, FIG. 9B is a diagram for explaining designation of a feature point setting region by scanning, and FIG. c) is a diagram for explaining the specification of a feature point setting region by extension.
FIG. 10 is a diagram for describing designation of a rectangle expanded from a designated point as a feature point setting area;
FIG. 11 is a diagram illustrating a configuration example of an image processing apparatus as a fifth embodiment.
FIG. 12 is a diagram illustrating a configuration example for creating a conventional wide dynamic range image.
FIG. 13 is a diagram illustrating a configuration example for creating a conventional image having a deep depth of field.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 30 ... Image correction part 31 ... Image data reproduction part 32 ... Frame memory 33 ... Feature point setting part 34 ... Feature point position memory 35 ... Corresponding point search part 36 ... Corresponding point position memory 37 ... Correction parameter calculation part 38 ... Interpolation calculation part 39. Image composition unit

Claims (13)

An image processing apparatus that performs an alignment process between a plurality of images obtained by shooting substantially the same composition under different shooting conditions,
Search area setting means for dividing at least one of the input images into a plurality of divided areas and providing a search area for feature points representing the features of the image in each of the divided areas;
The characteristic position of the subject in the composition excluding position information related to the coordinates of the divided area where the object moving in the image is detected based on the feature amount of the image from the search area of each feature point Position information extracting means for extracting information in association with coordinates for each divided region of the divided image;
An image processing apparatus comprising: The position information extracting means further searches for a point corresponding to the characteristic position information for each of the input images other than the divided images, and a corresponding point obtained by the search is searched for the image. The image processing apparatus according to claim 1, wherein the image processing apparatus is extracted as position information associated with coordinates. The image processing apparatus further includes a correction parameter detection unit that detects a correction parameter based on a deviation between the coordinates of the characteristic position information extracted by the position information extraction unit and the position information of the corresponding point.
Image correcting means for correcting the image so that the subject positions of the images match based on the detected correction parameters;
The image processing apparatus according to claim 2, further comprising: The position information extraction means, for each divided area in the image, the ratio of high-frequency components when orthogonally transformed, the ratio of saturated pixels, the average value of pixel values, the difference between the maximum and minimum pixel values, and The image processing apparatus according to claim 3, wherein coordinate-related position information for a desired divided region is extracted from the input image on the basis of any value of contrast. The search area setting means, for each divided area in the image, the ratio of high-frequency components when orthogonally transformed, the ratio of saturated pixels, the average value of pixel values, the difference between the maximum and minimum pixel values, and 5. A state determination unit that determines whether or not to search for a feature point based on any one of the contrast values, according to any one of claims 1 to 4. The image processing apparatus described. An image processing apparatus that corrects a plurality of images with different shooting conditions,
Area dividing means for dividing an image into a plurality of divided areas ;
A feature point search area setting means for providing a feature point search area in the divided area that is the same as or smaller than each of the divided areas;
Feature position information extracting means for extracting characteristic position information of the subject from the respective feature point search areas based on image statistics;
Corresponding point search means for searching for a corresponding point in an image different from the image from which the characteristic position information of the subject is extracted among the plurality of images ;
Among the characteristic position information of the subject based on the statistics of the image, the position information set on the subject that does not move between the plurality of images is used for detection of rotation / movement / magnification change between images. Correction parameter detecting means for detecting correction parameters including rotation, movement, and magnification change between images from the characteristic position information and position information of corresponding points;
Image correction means for correcting an image based on rotation, movement, and magnification change between images as the correction parameter;
An image processing apparatus comprising: An image processing apparatus that performs an alignment process between a plurality of images obtained by shooting substantially the same composition under different shooting conditions,
Area dividing means for dividing at least one of the input images into a plurality of divided areas;
Search area setting means for providing a search area for feature points representing image features in each of the divided areas;
Position information extraction means for extracting characteristic position information of a subject in the composition in association with coordinates for each divided area of the divided image based on the feature amount of the image from the search area of each feature point;
Comprising
  The area dividing means, when dividing into a plurality of divided areas, a division possible range set on the image centered on a specific point based on a user designation or centered on a divided area based on a user designation. An image processing apparatus characterized by expanding a rectangular frame to be shown. An image processing method for performing alignment processing between a plurality of images obtained by photographing substantially the same composition under different shooting conditions,
A search area setting step of dividing at least one of the input images into a plurality of divided areas, and providing a search area for feature points representing the features of the image in each of the divided areas;
The characteristic position of the subject in the composition excluding position information related to the coordinates of the divided area where the object moving in the image is detected based on the feature amount of the image from the search area of each feature point A position information extraction step of extracting information in association with coordinates for each divided region of the divided image;
An image processing method comprising: An image processing method for correcting a plurality of images with different shooting conditions,
A region dividing step of dividing the image into a plurality of divided regions;
A feature point search region setting step for providing a feature point search region that is the same as or smaller than each of the divided regions in the divided region;
A feature position information extraction step for extracting characteristic position information of the subject based on image statistics from each of the feature point search areas ;
A corresponding point search step of searching for a corresponding point in an image different from the image from which the characteristic position information of the subject is extracted among the plurality of images;
Among the characteristic position information of the subject based on the statistics of the image, the position information set on the subject that does not move between the plurality of images is used for detection of rotation / movement / magnification change between images. A correction parameter detecting step of detecting a correction parameter including rotation, movement, and magnification change between images from the characteristic position information and the position information of the corresponding point;
An image correction step for correcting an image based on rotation, movement, and magnification change between images as the correction parameter;
An image processing method comprising: The image correction step converts the correction parameter so as to represent a rotation / movement / magnification change viewed from a specific reference image when correcting the image based on the rotation / movement / magnification change of the correction parameter. The image processing method according to claim 9, wherein the correction is performed afterwards. The image processing method according to claim 9, wherein the region dividing step limits a region to be divided to a part of the image . The region dividing step is based on any one of a ratio of a high-frequency component when orthogonally transformed, a ratio of saturated pixels, an average value of pixel values, a difference between maximum and minimum pixel values, and a contrast. 12. The image processing method according to claim 11, wherein a region to be divided is limited. An image processing method for performing alignment processing between a plurality of images obtained by photographing substantially the same composition under different shooting conditions,
A region dividing step of dividing at least one of the input images into a plurality of divided regions;
A search area setting step for providing a search area for feature points representing the characteristics of the image in each of the divided areas;
A position information extracting step of extracting characteristic position information of the subject in the composition in association with coordinates for each divided area of the divided image based on the feature amount of the image from the search area of each feature point;
Comprising
In the region dividing step, when dividing into a plurality of divided regions, a range that can be divided set on the image is centered on a specific point based on a user designation or centered on a divided region based on a user designation. An image processing method characterized by expanding a rectangular frame to be shown.

Images