JP2005011269A - Image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor of a small circuit scale and a low power consumption capable of correcting distortion and chromatic aberration. <P>SOLUTION: This image processor for processing an electronic image data constituted of a plurality of components image-picked up by an optical system is provided with a distortion correcting processing part having a single distortion correction coefficient calculating part 22a for calculating a distortion correction coefficient for correcting a distortion aberration caused by the optical system the component by the component, based on a distance from the distortion center position, distortion correction circuits 7A1, 7B1, 7C1 for distortion-correcting the image data of the every component using the distortion correction coefficient calculated by the distortion correction coefficient calculating part 22a, and a grant synchronization circuit 27 for conducting control to output a permission signal e_grant to the distortion correction coefficient calculating part 22a after distortion correcting processing start request signals trok from all the distortion correction circuits 7A1, 7B1, 7C1 are all present, so as to start distortion correcting processing in all the distortion correction circuits 7A1, 7B1, 7C1. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

ここに、ｋは着目点をＸ方向にΔＸだけ移動させる際にインクリメントされる変数、ｌは着目点をＹ方向にΔＹだけ移動させる際にインクリメントされる変数である。
【００５６】
なお、上記補間開始位置（ＸＳＴ，ＹＳＴ）は、画像内の任意の位置に設定することが可能である。また、上記補間ステップ（ΔＸ，ΔＹ）は、上記ＣＰＵ３が適宜に設定することにより、画像の拡大や縮小を行うことが可能である。
【００５７】
上記歪補正座標変換部２２は、基本的に、上記補間位置生成部２１から出力される歪補正処理後の着目点の座標（Ｘ，Ｙ）から、歪補正処理前の着目点の座標（Ｘ’，Ｙ’）を次のように算出するものである。
【００５８】
すなわち、まず、処理対象の画像における歪中心位置の座標（Ｘｄ ，Ｙｄ ）と、歪補正処理後に該歪補正に起因して被写体の位置が撮影時とずれてしまうのを補正するための中心ずれ補正量（Ｘｏｆｆ ，Ｙｏｆｆ ）と、光学系の光学的な性質を示すパラメータを用いて歪補正処理を行った場合に補正後の画像が画像データとして必要な範囲からはみ出したり不足したりするのを補正するための範囲補正倍率Ｍと、を用いて、次の数式２に示すような中間的な算出値（Ｘ（ドット），Ｙ（ドット））（ここに、文字の上に付す点を文章中では（ドット）などとして表している。）を求める。
【数２】

【００５９】
なお、上記歪中心位置の座標（Ｘｄ ，Ｙｄ ）は、上記ＣＣＤ１に被写体光像を結像する光学系の光軸が交わる画像上の位置に相当する座標である。
【００６０】
また、上記歪中心位置の座標（Ｘｄ ，Ｙｄ ）、中心ずれ補正量（Ｘｏｆｆ ，Ｙｏｆｆ ）、範囲補正倍率Ｍは、上記ＣＰＵ３により、上記制御レジスタ７ａに設定されるようになっている。
【００６１】
次に、求めた算出値（Ｘ（ドット），Ｙ（ドット））と、上記歪中心位置の座標（Ｘｄ ，Ｙｄ ）と、画像データを間引いて取り込んだ場合などに画像の縦方向と横方向とで空間的なサンプリングが異なるのを補正するための係数（ＳＸ ，ＳＹ ）と、を用いて、歪中心からの距離を示すＺ（より正確にはＺの２乗）を次の数式３に示すように計算する。
【数３】

【００６２】
こうして算出されたＺと、上記ＣＰＵ３から上記制御レジスタ７ａに設定される光学系の歪曲収差に関する光学的性質を示すパラメータである歪補正係数Ａ，Ｂ，Ｃと、上記算出値（Ｘ（ドット），Ｙ（ドット））と、上記歪中心位置の座標（Ｘｄ ，Ｙｄ ）と、を用いて、歪補正処理後の着目点の座標（Ｘ，Ｙ）に対応する歪補正処理前の着目点の座標（Ｘ’，Ｙ’）を、次の数式４に示すように算出する。
【数４】

【００６３】
ここで図４は、格子状の被写体を光学系を介して撮像したときの歪曲収差の例を示す図である。まず、図４（Ａ）は、格子状の被写体の例を示している。従来は、Ｚの２次項までしか考慮していないために、図４（Ａ）に示すような被写体を光学系を介して撮像したときに生じ得る、図４（Ｂ）に示すような樽型の歪曲収差や、図４（Ｃ）に示すような糸巻き型の歪曲収差をある程度補正することは可能であったが、図４（Ｄ）に示すような陣笠型の歪曲収差を補正することはできなかった。しかし、本実施形態では、この数式４に示すように、Ｚの２次項を越える次数まで、つまり例えば４次項、または６次項まで考慮するようにしているために、このようなより高次の収差も高精度に補正することが可能となっている。なお、ここでは６次項まで考慮する例を示したが、８次項、１０次項等の、より一層高次の収差も補正するようにしても構わない。
【００６４】
上述したような補間位置生成部２１により算出された座標（Ｘ，Ｙ）、または歪補正座標変換部２２により算出された座標（Ｘ’，Ｙ’）は、セレクタ２３に入力されて、歪補正を行うか否かに応じて、必要なものが選択される。
【００６５】
次に、回路規模を縮小することができるようにした実用的な数式について説明する。
【００６６】
まず、上記数式３に数式２および数式１を代入して、次の数式５に示すように変形する。
【数５】

【００６７】
そして、数式６に示すような新たな定数式を導入して、上記数式５を数式７に示すように変形する。
【数６】

【数７】

【００６８】
ここに、数式７におけるＺ（２ドット）は、次の数式８に示すように定義され、Ｘ２，Ｙ２（補間座標データ）は次の数式９に示すように定義される。
【数８】

【数９】

【００６９】
この数式８および数式７に示すように定義されたＺ（２ドット）を用いると、上記数式４は次の数式１０に示すように変形される。
【数１０】

【００７０】
ここに、この数式１０においては、次の数式１１から数式１４に示すように定義された定数式または変数式を用いている。
【数１１】

【数１２】

【数１３】

【数１４】

【００７１】
上記数式１〜数式４に示したような基本的な数式に沿って演算を行う場合には、係数が１４個で乗算回数が１３回となる。すなわち、係数は、歪中心位置（Ｘｄ ，Ｙｄ ）、中心ずれ補正量（Ｘｏｆｆ ，Ｙｏｆｆ ）、範囲補正倍率Ｍ、係数（Ｓｘ，Ｓｙ）、歪補正係数Ａ，Ｂ，Ｃ、補間開始位置（ＸＳＴ，ＹＳＴ）、補間ステップ（ΔＸ，ΔＹ）の合計１４個である。また、乗算回数は、数式２におけるＭとの乗算が２回、数式３におけるＳｘ，Ｓｙとの各乗算と２つの２乗式とで乗算が４回、数式４における歪補正係数Ａ，Ｂ，Ｃとの乗算が３回、Ｚの４乗および６乗の算出に乗算が２回、大括弧と小括弧との乗算が２回、で合計１３回となる。
【００７２】
これに対して、上記数式５〜数式１４に示したような実用的な数式に沿って演算を行う場合には、係数が１３個で乗算回数が９回となる。すなわち、係数は、Ｘ（２ドット）ＳＴ，Ｙ（２ドット）ＳＴ，ΔＸ（２ドット），ΔＹ（２ドット），Ｘ（３ドット），Ｙ（３ドット），ΔＸ（３ドット），ΔＹ（３ドット），Ａ（ドット），Ｂ（ドット），Ｃ（ドット），Ｘｄ ，Ｙｄ の合計１３個である。また、乗算回数は、数式７の中括弧内における２つの２乗式で２回、数式１４におけるＡ（ドット），Ｂ（ドット），Ｃ（ドット）との乗算が３回、該数式１４におけるＺ（ドット）の４乗および６乗の算出に乗算が２回、数式１０におけるＦとの乗算が２回、で合計９回となる。
【００７３】
このような演算で扱う数は、ダイナミックレンジが大きく、計算回数が少し増えるだけでも回路規模が大きくなってしまうために、上記に示したような実用的な数式に沿って演算を行うことにより、乗算器の個数を減少させるとともに、係数を設定するためのレジスタを削減することができ、回路規模を効果的に縮小することが可能となる。
【００７４】
このように変形された実用的な数式に基づいて、上記補間位置生成部２１に対応する補間位置算出回路２１ａ（後述する図１４参照）は、上記ＣＰＵ３から上記制御レジスタ７ａに設定される補間開始位置（Ｘ（３ドット），Ｙ（３ドット））および補間ステップ（ΔＸ（３ドット），ΔＹ（３ドット））を用いて、上記数式１３に示すように、補間を行う着目点の座標（補間位置）（Ｘ１，Ｙ１）を算出するようになっている。
【００７５】
また、上記歪補正座標変換部２２の歪補正係数算出手段たる歪補正係数算出回路２２ａ（後述する図１１参照）は、上記ＣＰＵ３から上記制御レジスタ７ａに設定されるＡ（ドット），Ｂ（ドット），Ｃ（ドット）を用いて、上記数式１４に示すように歪補正係数Ｆを算出するようになっている。なお、後述するように、該図１１に示す構成に代えて、図１３に示すような構成により、歪補正係数Ｆを算出することも可能となっている。
【００７６】
さらに、上記歪補正座標変換部２２の補間位置補正回路２２ｂ（後述する図１４参照）は、算出された歪補正係数Ｆと、上記ＣＰＵ３から上記制御レジスタ７ａに設定される歪中心位置の座標（Ｘｄ ，Ｙｄ ）と、を用いて上記着目点の座標（補間位置）（Ｘ１，Ｙ１）から上記数式１０により歪補正処理前の着目点の座標（補間座標データ）（Ｘ’，Ｙ’）を算出するようになっている。
【００７７】
次に、上記補間演算部２６は、上記セレクタ２３から出力される座標に基づき、該座標の近傍の画素の画像データＤ０ 〜Ｄ１５を上記内部メモリ部２５から読み出して、次のような数式１５を用いることにより、該着目点に係る歪補正処理後の画像データＤｏｕｔ を算出して、後段の処理ブロックへ出力する。
【数１５】

ここに、ｋｘ０〜ｋｘ３，ｋｙ０〜ｋｙ３は、例えばＣｕｂｉｃ補間を行う際に定められた所定の補間係数である。
【００７８】
図７は、光学系により画像を撮影したときに発生する色収差の一例を示す図である。
【００７９】
この図７に示す例においては、糸巻き型の歪曲収差が発生するとともに、赤Ｒが歪中心から見て緑Ｇの内側に、青Ｂが歪中心から見て緑Ｇの外側に、それぞれずれるような色収差が発生している。そして、この色収差によるずれは、歪中心からの距離が遠いほど大きくなっている。
【００８０】
このように、色収差は各色毎に異なるために、補間処理も上記図２に示したように、各色毎に行っている。図１０は、上記歪補正処理部７のより詳細な構成を示すブロック図である。
【００８１】
上記歪補正処理部７ＡはＣｈ．０用歪補正回路７Ａ１を、上記歪補正処理部７ＢはＣｈ．１用歪補正回路７Ｂ１を、上記歪補正処理部７ＣはＣｈ．２用歪補正回路７Ｃ１を、それぞれ歪補正演算手段として有して構成されている。
【００８２】
また、歪補正係数Ｆを算出するための歪補正係数算出回路２２ａは、上記歪補正座標変換部２２の一部を構成するものであり、歪補正処理部７Ａ，７Ｂ，７Ｃにおいて共通化することができるために、３つのチャンネルに共通な単一の回路として設けられていて、Ｃｈ．０用の歪補正係数Ｆ０、Ｃｈ．１用の歪補正係数Ｆ１、Ｃｈ．２用の歪補正係数Ｆ２を算出して、上記Ｃｈ．０用歪補正回路７Ａ１、Ｃｈ．１用歪補正回路７Ｂ１、Ｃｈ．２用歪補正回路７Ｃ１へそれぞれ出力するようになっている。
【００８３】
また、歪補正処理や拡大縮小処理に係る補間処理を行った後に、後段の処理ブロックに対して画像データを出力する際には、該後段の処理ブロックの処理内容によっては、３つのチャンネルの画像データが同時に必要となる場合がある。ところが、上記図７に示したように、色収差がある場合には注目画素の位置が色によって異なるために、３つのデータが同時に揃って出力されるとは限らない。
【００８４】
これを図８および図９を参照して説明する。図８はＢに係る補間データを算出可能となった時点での内部メモリ部に蓄積された画像データの様子を示す図、図９はＲ，Ｇ，Ｂのそれぞれに係る補間データを算出可能となった時点での内部メモリ部に蓄積された画像データの様子を示す図である。
【００８５】
３つのチャンネルで補間データの算出処理を同時に開始した場合には、前段の処理ブロックからの画像データが、各チャンネルの内部メモリ部２５（後述する図６に示す２ポートＳＲＡＭ２５ａ）に順次記憶され始める。このときに、既にデータとして記憶されている部分が、斜線で示している部分である。図８に示す時点では、青Ｂに関する補間データの算出が可能なところまで画像データの蓄積が行われているが、緑Ｇおよび赤Ｒに関する補間データの算出を行うことができる量にはまだ至っていない。
【００８６】
また、図９に示す時点では、赤Ｒ、緑Ｇ、青Ｂに関する補間データの算出を行うことが可能なところまで画像データの蓄積が行われている。
【００８７】
このように、３つのチャンネルの補間データを算出可能になった時点で、後段の処理ブロックへの画像データの転送を許可するように同期させるのが、グラント同期手段たるグラント同期回路２７である。
【００８８】
Ｃｈ．０がＲ、Ｃｈ．１がＧ、Ｃｈ．２がＢである場合を例にとると、上記図８に示すような状態になったところで上記Ｃｈ．０用歪補正回路７Ａ１が歪補正処理の開始を要求する要求信号ｔｒｏｋ＿０を出力し、図示はしないがＣｈ．１用歪補正回路７Ｂ１が歪補正処理可能となった時点で歪補正処理の開始を要求する要求信号ｔｒｏｋ＿１を出力し、上記図９に示すような状態になったところでＣｈ．２用歪補正回路７Ｃ１が歪補正処理の開始を要求する要求信号ｔｒｏｋ＿２を出力するようになっている。
【００８９】
グラント同期回路２７は、これらの要求信号ｔｒｏｋ＿０，ｔｒｏｋ＿１，ｔｒｏｋ＿２が全て揃った後に、すなわち、３つのチャンネルの全てで補間データを送信することが可能になったところで、後段の処理ブロックの、Ｃｈ．０に係る処理を行う部分に対して許可信号ＧＲＡＮＴ＿Ｎ０を、Ｃｈ．１に係る処理を行う部分に対して許可信号ＧＲＡＮＴ＿Ｎ１を、Ｃｈ．２に係る処理を行う部分に対して許可信号ＧＲＡＮＴ＿Ｎ２を、それぞれ出力するとともに、上記Ｃｈ．０用歪補正回路７Ａ１、Ｃｈ．１用歪補正回路７Ｂ１、Ｃｈ．２用歪補正回路７Ｃ１に対して後述するように歪補正処理を同時に開始させるよう制御するようになっている。
【００９０】
なお、該グラント同期回路２７は、後述するように、内部にスイッチを有しており、３つのチャンネルの出力を同期させるか否かを切り換えることができるようになっている。
【００９１】
図１１は、上記歪補正係数算出回路２２ａの構成を示すブロック図である。
【００９２】
この歪補正係数算出回路２２ａは、上記数式１４によって、歪補正係数Ｆを算出するためのものであり、該数式１４における光学系の光学的性質を反映するＡ（ドット），Ｂ（ドット），Ｃ（ドット）は、各チャンネル毎に、つまり例えばＲＧＢの各色成分毎に、上記ＣＰＵ３から制御レジスタ７ａに設定されるようになっている。
【００９３】
なお、この図１１および後述する図１２，図１３において、そして以下に示す図１１から図１３までの説明においては、Ｚは数式８の左辺に示すＺ（２ドット）のことを指し、係数Ａ，Ｂ，Ｃは数式１１の左辺に示すＡ（ドット），Ｂ（ドット），Ｃ（ドット）のことを指している。
【００９４】
歪補正係数算出回路２２ａは、上記グラント同期回路２７から許可信号ｅ＿ｇｒａｎｔが入力されたところで上記数式９に基づき補間座標Ｘ２，Ｙ２を算出する歪補正用座標算出回路３１と、この歪補正用座標算出回路３１により算出された補間座標Ｘ２を浮動小数に変換する浮動小数化回路３２ａと、上記歪補正用座標算出回路３１により算出された補間座標Ｙ２を浮動小数に変換する浮動小数化回路３２ｂと、上記浮動小数化回路３２ａにより浮動小数化された補間座標Ｘ２を２乗する２乗計算器３３ａと、上記浮動小数化回路３２ｂにより浮動小数化された補間座標Ｙ２を２乗する２乗計算器３３ｂと、上記２乗計算器３３ａにより算出された補間座標Ｘ２の２乗と上記２乗計算器３３ｂにより算出された補間座標Ｙ２の２乗とを加算することによりＺ（より正確には、上述したようにＺ（２ドット）。以下同様）の２乗を算出する加算器３４と、この加算器３４からの出力を２乗することによりＺの４乗を算出する２乗計算器３３ｃと、上記加算器３４からの出力を遅延させる遅延回路３５ｂと、上記２乗計算器３３ｃにより算出されたＺの４乗とこの遅延回路３５ｂにより遅延されタイミングを合わせて出力されるＺの２乗とを乗算することによりＺの６乗を算出する乗算器３６と、上記加算器３４からの出力を遅延させるとともに上記２乗計算器３３ｃからの出力を遅延させる遅延回路３５ａと、この遅延回路３５ａからタイミングを合わせて出力されるＺの２乗およびＺの４乗と上記乗算器３６から出力されるＺの６乗とを用いてＣｈ．０用，Ｃｈ．１用，Ｃｈ．２用の歪補正係数Ｆ０，Ｆ１，Ｆ２をそれぞれ算出する距離依存係数算出回路３７ａ，３７ｂ，３７ｃと、を有して構成されている。
【００９５】
図１２は上記距離依存係数算出回路３７ａ，３７ｂ，３７ｃの任意の１つである距離依存係数算出回路３７の構成を示すブロック図である。
【００９６】
この距離依存係数算出回路３７は、上記遅延回路３５ａから遅延されタイミングを合わせて出力されるＺの２乗に上記ＣＰＵ３から制御レジスタ７ａに設定される係数Ａ（より正確には、上述したようにＡ（ドット）。以下同様）を乗算する乗算器４１ａと、上記遅延回路３５ａから遅延されタイミングを合わせて出力されるＺの４乗に上記ＣＰＵ３から制御レジスタ７ａに設定される係数Ｂ（より正確には、上述したようにＢ（ドット）。以下同様）を乗算する乗算器４１ｂと、上記乗算器３６から出力されるＺの６乗に上記ＣＰＵ３から制御レジスタ７ａに設定される係数Ｃ（より正確には、上述したようにＣ（ドット）。以下同様）を乗算する乗算器４１ｃと、上記乗算器４１ａからの出力に上記ＣＰＵ３から制御レジスタ７ａに設定される上記係数Ａの符号ｓｉｇｎＡを与えて固定小数に変換する固定小数化回路４２ａと、上記乗算器４１ｂからの出力に上記ＣＰＵ３から制御レジスタ７ａに設定される上記係数Ｂの符号ｓｉｇｎＢを与えて固定小数に変換する固定小数化回路４２ｂと、上記乗算器４１ｃからの出力に上記ＣＰＵ３から制御レジスタ７ａに設定される上記係数Ｃの符号ｓｉｇｎＣを与えて固定小数に変換する固定小数化回路４２ｃと、上記固定小数化回路４２ａからの出力と上記固定小数化回路４２ｂからの出力と上記固定小数化回路４２ｃからの出力と上記ＣＰＵ３から制御レジスタ７ａに設定される定数１．０とを加算することにより上記数式１４に基づく歪補正係数Ｆ（すなわち、歪補正係数Ｆ０，Ｆ１，Ｆ２の何れか）を算出する加算器４３と、を有して構成されている。
【００９７】
なお、図１１および図１２において、２重線で示している部分が、浮動小数による演算処理を行っている部分であり、ダイナミックレンジの広いＸ２，Ｙ２やＺなどを浮動小数で扱うことにより、精度を保ちながら回路規模を縮小するようになっている。
【００９８】
このとき、上記図１１に示したように、歪補正係数算出回路２２ａの一部、すなわち、歪補正用座標算出回路３１、浮動小数化回路３２ａ，３２ｂ、２乗計算器３３ａ，３３ｂ，３３ｃ、加算器３４、遅延回路３５ａ，３５ｂ、乗算器３６は、チャンネルによらない共通した部分として構成されている。これにより、さらに回路規模の縮小を図って、消費電力を低減することができるようになっている。
【００９９】
また、図１３は、上記歪補正係数算出回路２２ａの構成の他の例を示すブロック図である。
【０１００】
この図１３に示す歪補正係数算出回路２２ａは、上記歪補正用座標算出回路３１と、上記浮動小数化回路３２ａと、上記浮動小数化回路３２ｂと、上記２乗計算器３３ａと、上記２乗計算器３３ｂと、上記加算器３４と、この加算器３４から出力される浮動小数化されたＺ（より正確には、上述したようにＺ（２ドット）。以下同様）の２乗を用いて参照を行うことにより固定小数化された歪補正係数Ｆ０，Ｆ１，Ｆ２をそれぞれ出力するＬＵＴ（ルックアップテーブル）３９ａ，３９ｂ，３９ｃと、を有して構成されている。
【０１０１】
この図１３においても、２重線で示している部分が、浮動小数による演算処理を行っている部分である。
【０１０２】
このように、この図１３に示す構成例は、ルックアップテーブルを用いることにより、歪補正係数Ｆ０，Ｆ１，Ｆ２を算出するための処理時間を短縮し回路の消費電力を低減するものとなっている。
【０１０３】
なお、この構成例を適用する場合には、チャンネル毎の上記係数Ａ，Ｂ，ＣはＣＰＵ３から設定されるものではなく、固定値となっている。もちろん、ルックアップテーブルの規模が大きくなっても構わない場合には、上記係数Ａ，Ｂ，Ｃの複数の組み合わせに対応するルックアップテーブルをチャンネル毎に用意しておいて、適切なものを用いるようにしても構わない。
【０１０４】
次に、図１４は、上記Ｃｈ．０用歪補正回路７Ａ１の構成を示すブロック図である。Ｃｈ．１用歪補正回路７Ｂ１、Ｃｈ．２用歪補正回路７Ｃ１の構成も、このＣｈ．０用歪補正回路７Ａ１の構成と同様であるために、ここではＣｈ．０用歪補正回路７Ａ１についてのみ説明する。
【０１０５】
補間位置算出回路２１ａは、上記着目点の座標（Ｘ１，Ｙ１）を上述したように算出するためのものであり、上記補間位置生成部２１に対応する回路部分である。
【０１０６】
補間位置補正回路２２ｂは、上記歪補正係数算出回路２２ａにより算出された歪補正係数Ｆ０を用いて上記数式１０に示したように歪補正処理前の着目点の座標（Ｘ’，Ｙ’）を算出するためのものであり、上記歪補正座標変換部２２の一部となっている。
【０１０７】
上記セレクタ２３は、歪補正処理を行わない場合には（拡大縮小処理を行うか否かに関わらず）上記補間位置算出回路２１ａからの座標（Ｘ１，Ｙ１）を選択し、歪補正処理を行う場合には（拡大縮小処理を伴うか否かに関わらず）上記補間位置補正回路２２ｂからの座標（Ｘ’，Ｙ’）を選択するものである。
【０１０８】
このセレクタ２３により選択された座標（Ｘ１，Ｙ１）または座標（Ｘ’，Ｙ’）には、制御レジスタ７ａを介して上記ＣＰＵ３により設定される歪中心位置の座標（Ｘｄ ，Ｙｄ ）が、加算器２７ａ，２７ｂによりそれぞれ加算されるようになっている。
【０１０９】
これら加算器２７ａ，２７ｂの出力は、読出しアドレス生成回路２４ａとバッファ開放量算出回路２４ｃとへ出力されるようになっている。
【０１１０】
この読出しアドレス生成回路２４ａには、さらに、上記補間位置算出回路２１ａにより算出された座標（Ｘ１，Ｙ１）も上記セレクタ２３を介する経路とは別途に入力されるようになっている。そして、該読出しアドレス生成回路２４ａは、この座標（Ｘ１，Ｙ１）に関連する画素データ（１６点補間を行う場合には、該座標（Ｘ１，Ｙ１）を中心とする１６個の画素データ）が記憶されているアドレスＡＤＲを２ポートＳＲＡＭ２５ａに出力するとともに、補間係数（例えば、上記数式１５に示したような補間係数ｋｘ０〜ｋｘ３，ｋｙ０〜ｋｙ３）と出力する画素データがＤ０ 〜Ｄ１５のどの位置の画素データであるかを示すデータ列制御信号とを補間回路２６ａに出力するようになっている。さらに、該読出しアドレス生成回路２４ａは、書き込み許可信号ＷＥ＿Ｎ０を後段のＣｈ．０に対応する処理ブロックへ出力する。
【０１１１】
上記２ポートＳＲＡＭ２５ａは、上記図３に示した内部メモリ部２５に対応する回路部分である。
【０１１２】
補間回路２６ａは、上記図３に示した補間演算部２６に対応するものであり、上記２ポートＳＲＡＭ２５ａから読み出された画像データを用いて上記数式１５に示したような補間処理を行い、処理後の画像データを後段の処理ブロックに出力するものである。
【０１１３】
上記バッファ開放量算出回路２４ｃは、上記補間位置算出回路２１ａからの出力と、上記加算器２７ａ，２７ｂを介した上記セレクタ２３からの出力と、に基づいて、上記２ポートＳＲＡＭ２５ａにおいて開放可能な記憶容量（バッファ開放量）を算出するものである。
【０１１４】
バッファ空き容量監視回路２４ｄは、このバッファ開放量算出回路２４ｃの出力を参照して、上記２ポートＳＲＡＭ２５ａの空き容量の状況を把握するものである。
【０１１５】
書込みアドレス生成回路２４ｂは、前段の処理ブロックからのＣｈ．０に係る画像データを受けて、上記２ポートＳＲＡＭ２５ａに記録するものである。
【０１１６】
データ送信可否判定回路２４ｅは、Ｃｈ．０に係る後段の処理ブロックからの画像データを要求する要求信号ＲＥＱ＿Ｎ０を受けて、上記書込みアドレス生成回路２４ｂとバッファ開放量算出回路２４ｃとの出力に基づき画像データの送信が可能であるか否かを判断し、可能である場合に要求信号ｔｒｏｋ＿０を上記グラント同期回路２７へ出力するものである。
【０１１７】
次に、この図１４に示したような歪補正処理部７の動作について説明する。
【０１１８】
バッファ空き容量監視回路２４ｄは、バッファ開放量算出回路２４ｃを介して２ポートＳＲＡＭ２５ａの空き容量を監視し、所定の空き容量があると、Ｃｈ．０に係る前段の処理ブロックに対して所定のブロック単位の画像データ（以下、適宜、ユニットライン（ＵＬ）データという。）を送信するように要求信号ＲＥＱ＿０を出す（ステップＳ１）。
【０１１９】
Ｃｈ．０に係る前段の処理ブロックは、この要求信号ＲＥＱ＿０を受けて、画像データが送信可能になったところで許可信号ＧＲＡＮＴ＿０を送信し、上記バッファ空き容量監視回路２４ｄは、この許可信号ＧＲＡＮＴ＿０を受信する（ステップＳ２）。
【０１２０】
該バッファ空き容量監視回路２４ｄは、２ポートＳＲＡＭ２５ａの空き容量を、内部に保持するカウンタで把握するようになっており、許可信号ＧＲＡＮＴを受信すると同時に、この内部カウンタの値を１つ減らす。該バッファ空き容量監視回路２４ｄは、この内部カウンタが０になったら、上記要求信号ＲＥＱ＿０を取り下げるように動作する（ステップＳ３）。
【０１２１】
そして、Ｃｈ．０に係る前段の処理ブロックから、書き込み許可信号ＷＥ＿０が書込みアドレス生成回路２４ｂに入力され、続いて画像データが入力される。これに応じて、書込みアドレス生成回路２４ｂは、制御信号を２ポートＳＲＡＭ２５ａへ出力して、画像データＤＡＴＡを該２ポートＳＲＡＭ２５ａの、アドレスＡＤＤＲＥＳＳで指定される領域に書き込む。また、書込みアドレス生成回路２４ｂは、１ＵＬデータが入力される毎に、ＢＬＣカウンタ（内部バッファである２ポートＳＲＡＭ２５ａにどれだけのデータが溜められたかを示すカウンタ）をアップして、データ送信可否判定回路２４ｅへ出力する（ステップＳ４）。
【０１２２】
データ送信可否判定回路２４ｅは、Ｃｈ．０に係る後段の処理ブロックから要求信号ＲＥＱ＿Ｎ０を受信すると、次に送信することができるＵＬデータが２ポートＳＲＡＭ２５ａ内にあるか否かを判定し、あると判定した場合には、要求信号ｔｒｏｋ＿０を上記グラント同期回路２７へ送信する（ステップＳ５）。
【０１２３】
グラント同期回路２７は、後述するように、画像データの送信条件が整ったところで各チャンネルに係る後段の処理ブロックおよび各チャンネルに係る補間位置算出回路２１ａに対して許可信号ＧＲＡＮＴ＿Ｎ０，ＧＲＡＮＴ＿Ｎ１，ＧＲＡＮＴ＿Ｎ２を出力する。
【０１２４】
Ｃｈ．０に係る補間位置算出回路２１ａは、この許可信号ＧＲＡＮＴ＿Ｎ０を受けて動作を開始し、補間位置である座標（Ｘ１，Ｙ１）の算出動作を１ＵＬ分行ったところで、次のＵＬ先頭座標を算出して終了する（ステップＳ６）。
【０１２５】
上記グラント同期回路２７は、補間位置算出回路２１ａが動作を開始して座標（Ｘ１，Ｙ１）を出力するのと同期して歪補正係数算出回路２２ａが歪補正係数Ｆ０を出力することができるようなタイミングで、許可信号ｅ＿ｇｒａｎｔを該歪補正係数算出回路２２ａへ出力する（ステップＳ７）。
【０１２６】
歪補正係数算出回路２２ａは、許可信号ｅ＿ｇｒａｎｔを受けると、各チャンネルに係る歪補正係数Ｆ０，Ｆ１，Ｆ２を上記数式１４に基づき算出し、各チャンネルの補間位置補正回路２２ｂへ出力する。この歪補正係数算出回路２２ａも、上記補間位置算出回路２１ａと同様に、１ＵＬ分の動作を行ったところで、次のＵＬ先頭座標を算出して終了する（ステップＳ８）。
【０１２７】
Ｃｈ．０に係る補間位置補正回路２２ｂは、この歪補正係数算出回路２２ａから受け取った歪補正係数Ｆ０と、上記補間位置算出回路２１ａから受け取った座標（Ｘ１，Ｙ１）と、を用いて、上記数式１０に基づき座標（Ｘ’，Ｙ’）を算出する。この補間位置補正回路２２ｂも、上記歪補正係数算出回路２２ａに従って、１ＵＬ分の動作を行ったところで、次のＵＬ先頭座標を算出して終了する（ステップＳ９）。
【０１２８】
セレクタ２３は、上記制御レジスタ７ａを介して上記ＣＰＵ３により設定された動作モードに従って、歪補正処理を行う場合には上記補間位置補正回路２２ｂからの座標（Ｘ’，Ｙ’）を選択し、歪補正処理を行わない場合には上記補間位置算出回路２１ａからの座標（Ｘ１，Ｙ１）を選択する（ステップＳ１０）。
【０１２９】
加算器２７ａ，２７ｂは、このセレクタ２３により選択された座標（Ｘ１，Ｙ１）または座標（Ｘ’，Ｙ’）に、歪中心位置の座標（Ｘｄ ，Ｙｄ ）をそれぞれ加算する（ステップＳ１１）。
【０１３０】
読み出しアドレス生成回路２４ａは、加算器２７ａ，２７ｂから受け取った座標に基づいて、補間に用いるために２ポートＳＲＡＭ２５ａから読み出す画素データのアドレスＡＤＲを該２ポートＳＲＡＭ２５ａに出力するとともに、補間係数とデータ列制御信号とを補間回路２６ａに出力する（ステップＳ１２）。
【０１３１】
補間回路２６ａは、読み出しアドレス生成回路２４ａから受け取った補間係数およびデータ列制御信号と、２ポートＳＲＡＭ２５ａから受け取った画素データと、を用いて、上記数式１５に示すように、補間された画素データを算出し、Ｃｈ．０に係る後段の処理ブロックへ出力する（ステップＳ１３）。
【０１３２】
バッファ開放量算出回路２４ｃは、上記補間位置算出回路２１ａおよび加算器２７ａ，２７ｂの出力に基づいて、ＵＬデータを最後まで出力したことが確認されたら、今処理を終えたＵＬ先頭座標と、次のＵＬ先頭座標との差を計算して、必要なくなったデータが蓄積されているバッファ（２ポートＳＲＡＭ２５ａ内の領域）を開放するために、バッファ開放量をバッファ空き容量監視回路２４ｄへ出力するとともに、次のＵＬ処理を行うために後どれだけのデータをＣｈ．０に係る前段の処理ブロックから受けることが必要であるかを、上記データ送信可否判定回路２４ｅへ送信する（ステップＳ１４）。
【０１３３】
バッファ空き容量監視回路２４ｄは、上記ステップＳ１４において内部バッファである２ポートＳＲＡＭ２５ａに記憶領域の空きができたことが確認されたところで、上記ステップＳ１へ戻って、上述したような処理を繰り返して行う（ステップＳ１５）。
【０１３４】
データ送信可否判定回路２４ｅは、上記書込みアドレス生成回路２４ｂからのＢＬＣカウンタの値と、バッファ開放量算出回路２４ｃからの出力と、に基づいて、次のＵＬデータを後段の処理ブロックに送信することができるか否かを判定し、送信することができると判定した場合には、上記ステップＳ５の処理を行う（ステップＳ１６）。
【０１３５】
図１５はグラント同期回路２７の構成を示すブロック図、図１６はグラント同期回路２７の動作を説明するためのタイミングチャートである。
【０１３６】
まず、上記制御レジスタ７ａを参照することにより得られるＤＴ＿ＯＮは、歪補正を行うか否かを示すブール値を保持するものであり、０（偽）の場合にはローレベル、１（真）の場合にはハイレベルの信号として取得されるように構成されている。
【０１３７】
このグラント同期回路２７は、Ｃｈ．０用歪補正回路７Ａ１からの要求信号ｔｒｏｋ＿０とＣｈ．１用歪補正回路７Ｂ１からの要求信号ｔｒｏｋ＿１とＣｈ．２用歪補正回路７Ｃ１からの要求信号ｔｒｏｋ＿２との論理積をとるためのＡＮＤ回路５１と、上記要求信号ｔｒｏｋ＿０，ｔｒｏｋ＿１，ｔｒｏｋ＿２とＡＮＤ回路５１の出力との立ち上がりを検出するための微分回路５２と、ＤＴ＿ＯＮがローレベルである場合には要求信号ｔｒｏｋ＿０に係る微分回路５２の出力側に接続され該ＤＴ＿ＯＮがハイレベルである場合には上記ＡＮＤ回路５１に係る微分回路５２の出力側に接続されるように切り換えられるスイッチ５３ａと、ＤＴ＿ＯＮがローレベルである場合には要求信号ｔｒｏｋ＿１に係る微分回路５２の出力側に接続され該ＤＴ＿ＯＮがハイレベルである場合には上記ＡＮＤ回路５１に係る微分回路５２の出力側に接続されるように切り換えられるスイッチ５３ｂと、ＤＴ＿ＯＮがローレベルである場合には要求信号ｔｒｏｋ＿２に係る微分回路５２の出力側に接続され該ＤＴ＿ＯＮがハイレベルである場合には上記ＡＮＤ回路５１に係る微分回路５２の出力側に接続されるように切り換えられるスイッチ５３ｃと、上記ＡＮＤ回路５１に係る微分回路５２の出力とＤＴ＿ＯＮとの論理積をとって上記歪補正係数算出回路２２ａの歪補正用座標算出回路３１へ許可信号ｅ＿ｇｒａｎｔとして出力するＡＮＤ回路５４と、を有して構成されている。
【０１３８】
このようなグラント同期回路２７の動作は、次のようになっている。
【０１３９】
まず、ＤＴ＿ＯＮがローレベルであるとき、つまり歪補正を行わない場合には、上記スイッチ５３ａ，５３ｂ，５３ｃは、要求信号ｔｒｏｋ＿０，ｔｒｏｋ＿１，ｔｒｏｋ＿２に係る検出する微分回路５２の出力側にそれぞれ切り換えられている。
【０１４０】
このときに、図１６の左側に示すように、要求信号ｔｒｏｋ＿０にハイレベルの信号が入力されると、微分回路５２でその立ち上がりが検出されて、ＧＲＡＮＴ＿Ｎ０として出力される。次に、要求信号ｔｒｏｋ＿１にハイレベルの信号が入力されると、微分回路５２でその立ち上がりが検出されて、ＧＲＡＮＴ＿Ｎ１として出力され、さらにその後に、要求信号ｔｒｏｋ＿２にハイレベルの信号が入力されると、微分回路５２でその立ち上がりが検出されて、ＧＲＡＮＴ＿Ｎ２として出力される。このように、ＤＴ＿ＯＮがローレベルであるときには、３つのチャンネルからの画像データの出力は、各チャンネル毎にそれぞれのタイミングで行われる。
【０１４１】
また、ＤＴ＿ＯＮがローレベルとなっているときには、ＡＮＤ回路５１の出力如何に関わらず、ＡＮＤ回路５４の出力はローレベルのままとなり、許可信号ｅ＿ｇｒａｎｔは出力されない（あるいは不許可の出力となる）ために、上記歪補正係数算出回路２２ａは歪補正係数Ｆ０，Ｆ１，Ｆ２の算出を行わない。従って、各チャンネルに係る補間位置補正回路２２ｂも動作せず、結局、歪補間処理は行われないことになる。
【０１４２】
次に、ＤＴ＿ＯＮがハイレベルであるとき、つまり歪補正を行う場合には、上記スイッチ５３ａ，５３ｂ，５３ｃは、ＡＮＤ回路５１に係る微分回路５２の出力側に切り換えられている。
【０１４３】
このときに、図１６の右側に示すように、要求信号ｔｒｏｋ＿０にハイレベルの信号が入力されても、その時点では、要求信号ｔｒｏｋ＿１，ｔｒｏｋ＿２はローレベルのままであるために、該ＡＮＤ回路５１の出力はローレベルのまま維持される。
【０１４４】
その後、要求信号ｔｒｏｋ＿０がハイレベルであって、かつ要求信号ｔｒｏｋ＿１がハイレベルとなった時点でも、要求信号ｔｒｏｋ＿２がローレベルである間は、ＡＮＤ回路５１の出力はローレベルのままである。
【０１４５】
そして、要求信号ｔｒｏｋ＿０，ｔｒｏｋ＿１，ｔｒｏｋ＿２の全てがハイレベルとなった時点で、ＡＮＤ回路５１の出力がハイレベルとなり、微分回路５２でその立ち上がりが検出される。
【０１４６】
この微分回路５２の出力は、許可信号ＧＲＡＮＴ＿Ｎ０，ＧＲＡＮＴ＿Ｎ１，ＧＲＡＮＴ＿Ｎ２として同時に出力されるとともに、ＡＮＤ回路５４にも出力されて、ハイレベルとなっているＤＴ＿ＯＮと論理積がとられ、許可信号ｅ＿ｇｒａｎｔとして上記歪補正係数算出回路２２ａへ出力される。
【０１４７】
このようにして、ＤＴ＿ＯＮがハイレベルであるときには、３つのチャンネルからの画像データの出力は、同期したタイミングで行われる。
【０１４８】
このような実施形態によれば、歪補正と色収差補正とを行い得る回路規模が小さく低消費電力な画像処理装置となる。
【０１４９】
特に、歪補正係数算出回路の少なくとも一部を複数のチャンネルに対して共通化することにより、回路規模を効果的に小さくし、低消費電力化を図ることができる。このとき、ダイナミックレンジが広い数を浮動小数として扱って演算を行っているために、精度を保ちながら回路規模を縮小することが可能となる。そして、歪補正係数をルックアップテーブルを参照して求める場合には、処理時間を短縮しながら回路規模を縮小することが可能となる。
【０１５０】
また、グラント同期回路を設けて、各チャンネルからの画像データ出力を同時に行うことができるようにしたために、後段の処理ブロックにおいて複数チャンネルの画像データを同時に必要とする場合に良好に対応することができる。このとき、グラント同期回路は、スイッチングにより複数チャンネルの画像データがそれぞれのタイミングで出力されるように処理することも可能であるために、必要に応じた選択を行うことが可能となる。
【０１５１】
加えて、基本的な数式に基づく場合よりも乗算の回数が減るように変形した実用的な数式に基づいて、補間座標データを生成するようにしたために、補間位置生成部や歪補正座標変換部の内部に設ける乗算器の個数を削減することができ、回路規模の縮小を図ることができる。
【０１５２】
さらに、拡大縮小処理も、歪補正処理と同一の回路で同時に行うことが可能となる利点がある。
【０１５３】
なお、本発明は上述した実施形態に限定されるものではなく、発明の主旨を逸脱しない範囲内において種々の変形や応用が可能であることは勿論である。
【０１５４】
【発明の効果】
以上説明したように本発明によれば、歪補正と色収差補正とを行い得る回路規模が小さく低消費電力な画像処理装置となる。
【図面の簡単な説明】
【図１】本発明の一実施形態における画像処理装置の構成を示すブロック図。
【図２】上記実施形態における歪補正処理部の構成の概要を示すブロック図。
【図３】上記実施形態において、１つのチャンネルに係る歪補正処理部の構成を示すブロック図。
【図４】上記実施形態において、格子状の被写体を光学系を介して撮像したときの歪曲収差の例を示す図。
【図５】上記実施形態において、歪補正を含む補間処理の概要を説明するための図。
【図６】上記実施形態において、１６点補間による処理を説明するための図。
【図７】上記実施形態において、光学系により画像を撮影したときに発生する色収差の一例を示す図。
【図８】上記実施形態において、Ｂに係る補間データを算出可能となった時点での内部メモリ部に蓄積された画像データの様子を示す図。
【図９】上記実施形態において、Ｒ，Ｇ，Ｂのそれぞれに係る補間データを算出可能となった時点での内部メモリ部に蓄積された画像データの様子を示す図。
【図１０】上記実施形態における歪補正処理部のより詳細な構成を示すブロック図。
【図１１】上記実施形態における歪補正係数算出回路の構成を示すブロック図。
【図１２】上記実施形態における距離依存係数算出回路の構成を示すブロック図。
【図１３】上記実施形態における歪補正係数算出回路の構成の他の例を示すブロック図。
【図１４】上記実施形態におけるＣｈ．０用歪補正回路の構成を示すブロック図。
【図１５】上記実施形態におけるグラント同期回路の構成を示すブロック図。
【図１６】上記実施形態におけるグラント同期回路の動作を説明するためのタイミングチャート。
【図１７】上記実施形態の画像処理装置における一般的な画像処理の手順を示す図。
【図１８】上記図１７に示したような一般的な画像処理を行うための従来の画像処理装置の構成を示すブロック図。
【図１９】従来において歪補正処理を行うために必要なメモリ量を説明するための図。
【符号の説明】
１…ＣＣＤ
２…プリプロセス部
３…ＣＰＵ
４…フレームメモリ
６…イメージプロセス部（画像処理部）
７，７Ａ，７Ｂ，７Ｃ…歪補正処理部（歪補正処理手段、画像処理部）
７ａ…制御レジスタ
７Ａ１…Ｃｈ．０用歪補正回路（歪補正演算手段、画像処理部）
７Ｂ１…Ｃｈ．１用歪補正回路（歪補正演算手段、画像処理部）
７Ｃ１…Ｃｈ．２用歪補正回路（歪補正演算手段、画像処理部）
９…ＪＰＥＧ処理部（画像処理部）
１０…メモリカード等
１１…バス
２０…補間座標生成部
２１…補間位置生成部（補間座標生成部の一部）
２１ａ…補間位置算出回路
２２…歪補正座標変換部（補間座標生成部の一部）
２２ａ…歪補正係数算出回路（歪補正係数算出手段）
２２ｂ…補間位置補正回路
２３…セレクタ（補間座標生成部の一部）
２４…メモリ制御部
２４ａ…読出しアドレス生成回路
２４ｂ…書込みアドレス生成回路
２４ｃ…バッファ開放量算出回路
２４ｄ…バッファ空き容量監視回路
２４ｅ…データ送信可否判定回路
２５…内部メモリ部
２５ａ…２ポートＳＲＡＭ（内部バッファ）
２６…補間演算部
２６ａ…補間回路
２７…グラント同期回路（グラント同期手段）
３１…歪補正用座標算出回路
３２ａ，３２ｂ…浮動小数化回路
３７，３７ａ，３７ｂ，３７ｃ…距離依存係数算出回路
４２ａ，４２ｂ，４２ｃ…固定小数化回路
３９ａ，３９ｂ，３９ｃ…ＬＵＴ（ルックアップテーブル）[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus, and more particularly to an image processing apparatus that processes electronic image data captured through an optical system.
[0002]
[Prior art]
In an electronic image pickup apparatus such as a digital camera, a subject image formed by an optical system is photoelectrically converted by an image pickup device such as a CCD to obtain image pickup data, and various image processing is performed on the image pickup data. In general, the image data is compressed by a compression method such as JPEG and recorded on a recording medium such as a memory card, and the electronic imaging device such as the digital camera also serves as an image processing device.
[0003]
FIG. 17 is a diagram showing a general image processing procedure in the image processing apparatus.
[0004]
An imaging element such as a CCD photoelectrically converts an optical subject image formed by an optical system to generate an electrical imaging signal. This imaging signal is stored in the frame memory after preprocessing such as pixel defect correction and A / D conversion.
[0005]
Next, the image data stored in the frame memory is read out and converted from a single plate signal to a three plate signal by the first image process, the second image process,..., The Nth image process, etc. Various image processing such as processing, low-pass filter processing, edge enhancement processing, and enlargement / reduction processing are performed.
[0006]
The image signal after the image processing is further compressed by a compression method such as JPEG, and is recorded on the memory card as an image file.
[0007]
FIG. 18 is a block diagram showing a configuration of a conventional image processing apparatus for performing general image processing as shown in FIG.
[0008]
This image processing apparatus includes a CCD 91, a preprocessing unit 92, a frame memory 94, a first image processing unit 95a, a second image processing unit 95b,..., An Nth image processing unit 95n, a JPEG processing unit 96, A bus 98 to which a memory card or the like 97, each circuit excluding the CCD 91 and a CPU 93 to be described later are connected, and a CPU 93 that controls the image processing apparatus including the above-described circuits in an integrated manner. It is configured.
[0009]
When the processing shown in FIG. 17 is performed by the image processing apparatus configured as shown in FIG. 18, the procedure is as follows.
[0010]
First, the image data from the preprocessing unit 92 is temporarily stored in the frame memory 94 via the bus 98.
[0011]
Next, the image data is read from the frame memory 94 and input to the first image processing unit 95a via the bus 98, the first image processing is performed, and the processed image data is written on the frame memory 94. .
[0012]
Similarly, the image data after the first image processing is read out from the frame memory 94 and is input to the second image processing unit 95b via the bus 98, and the second image processing is performed. A process of writing data on the frame memory 94 is performed, and the same process is repeated for each image process unit.
[0013]
As described above, when image processing is performed, the image data flows through the bus 98 many times. However, since the image data generally has a large data size, the bus 98 is heavily loaded. Become. Such a large load on the bus 98 is more conspicuous when the continuous shooting function is used.
[0014]
From this point of view, a technique for reducing the bus load by connecting a plurality of image processing units so as to enable pipeline processing and pipeline processing of images from the frame memory is disclosed in, for example, Japanese Patent Laid-Open No. 2000-2000. Japanese Patent No. 312441 discloses that image processing including enlargement / reduction processing can be performed in real time without increasing the memory capacity while reducing the bus load.
[0015]
Furthermore, Japanese Patent Laid-Open No. 2000-31327 also describes a technique for reducing the buffer amount when performing pipeline processing by reading an image stored in the frame memory in a predetermined direction (column direction) in units of blocks. Therefore, an image processing apparatus with low power consumption and memory saving can be configured.
[0016]
By the way, in the optical system of a camera including a digital camera or a silver salt camera, distortion is generally generated regardless of the size. This distortion aberration is observed as a barrel shape, a pincushion shape, etc., for example, when a lattice-like subject is photographed (FIGS. 4A, 4B, and 4C according to the embodiment of the present invention). reference). In addition, many cameras that are currently on sale can perform optical zoom, but such zoomable optical systems have a state of distortion when the focal length is changed within the zoom range from the wide end to the tele end. Often changes.
[0017]
For such a phenomenon, a technique for correcting distortion as part of image processing has been developed, and an example thereof is described in Japanese Patent Laid-Open No. 6-181530. In normal image processing as described in the publication, data is read from the frame memory, for example, in units of lines.
[0018]
As another technique for correcting distortion as part of image processing, for example, Japanese Patent Application Laid-Open No. 10-224695 describes a technique in which each image processing unit randomly accesses a frame memory. According to this technique, there is no need to provide a buffer in the image process unit, so there is an advantage that the circuit scale of the image process unit can be reduced.
[0019]
Furthermore, it is known that chromatic aberration occurs in the optical system of the camera. This chromatic aberration is caused by the difference in refractive index depending on the wavelength of light when light is incident on the optical system. When an optical image is formed by the optical system, it is imaged for each wavelength. It appears as a phenomenon in which the optical image is slightly shifted. The optical system is designed so that the chromatic aberration is as small as possible. However, it is difficult to eliminate all the chromatic aberration from the viewpoint of arrangement space, weight, cost, and the like.
[0020]
[Patent Document 1]
JP 2000-311241 A
[0021]
[Patent Document 2]
JP 2000-31327 A
[0022]
[Patent Document 3]
JP-A-6-181530
[0023]
[Patent Document 4]
JP-A-10-224695
[0024]
[Problems to be solved by the invention]
In the one described in Japanese Patent Laid-Open No. 6-181530, in order to perform distortion correction as image processing for one line of an image after correction, image data before correction as shown in FIG. Therefore, it is necessary to read out the entire image in the horizontal direction by a plurality of lines necessary for distortion correction. FIG. 19 is a diagram for explaining a memory amount necessary for performing distortion correction processing in the related art. Since the image data for a plurality of lines is temporarily stored in a buffer provided in the image processing unit and then processed, a relatively large capacity is required as a buffer to obtain a corrected image for one line. As a result, the circuit scale increases and the manufacturing cost increases and the power consumption increases. Furthermore, the processable image size is limited by the buffer memory capacity in the image process unit.
[0025]
In addition, in the one described in the above-mentioned Japanese Patent Application Laid-Open No. 10-224695, if random access is made to a frame memory composed of SDRAM or the like, burst transfer cannot be performed, so address transfer is required each time. As a result, the bus load eventually increases and the power consumption also increases. Furthermore, since random access is performed, the data transfer time becomes a factor of increasing the overall processing time as compared with burst transfer that can be read from the SDRAM at high speed.
[0026]
Further, it is desirable that the above-described chromatic aberration can be corrected satisfactorily without significantly increasing the cost.
[0027]
The present invention has been made in view of the above circumstances, and an object thereof is to provide an image processing apparatus having a small circuit scale and low power consumption capable of performing distortion correction and chromatic aberration correction.
[0028]
[Means for Solving the Problems]
In order to achieve the above object, an image processing apparatus according to a first invention processes electronic image data obtained by taking an image through an optical system and is composed of a plurality of components. A distortion correction coefficient calculating unit that calculates a distortion correction coefficient for each component based on a distance from a strain center position, and a distortion correction coefficient that is used to correct distortion due to the optical system. Distortion correction processing means configured to include distortion correction calculation means for correcting distortion of the image data for each component using the distortion correction coefficient for each component calculated by the coefficient calculation means.
[0029]
The image processing apparatus according to the second invention is the image processing apparatus according to the first invention, wherein a plurality of the distortion correction calculation means are provided corresponding to all the components constituting the image data on a one-to-one basis. Is.
[0030]
The image processing apparatus according to a third aspect of the present invention is the image processing apparatus according to the second aspect of the present invention, wherein the distortion correction calculating means includes an internal buffer for storing image data, and distortion correction processing is performed. Is output when the image data necessary for performing the correction is accumulated in the internal buffer, and after the distortion correction processing start requests from all of the plurality of distortion correction calculation means are prepared, Grant synchronization means for controlling all the distortion correction calculation means to start the distortion correction processing is further provided.
[0031]
An image processing apparatus according to a fourth invention is the image processing apparatus according to the third invention, wherein the grant synchronization means outputs a permission signal for permitting a distortion correction coefficient calculation process to the distortion correction coefficient calculation means. Thus, by starting the calculation of the distortion correction coefficient relating to all the components by the distortion correction coefficient calculating means, control is performed so that all the distortion correction calculating means start the distortion correction processing.
[0032]
An image processing apparatus according to a fifth invention is the image processing apparatus according to any one of the first to fourth inventions, wherein at least a part of the distortion correction coefficient calculation means calculates the distortion correction coefficient for each component. It is configured as a part used in common for all components.
[0033]
An image processing apparatus according to a sixth invention is the image processing apparatus according to the fifth invention, wherein the distortion correction coefficient calculation means calculates a square value of a distance from the distortion center position and based on the square value. The distortion correction coefficient is obtained for each component, and the portion used in common for all the components is a portion for calculating the square value of the distance from the strain center position.
[0034]
An image processing apparatus according to a seventh aspect is the image processing apparatus according to the sixth aspect, wherein the distortion correction coefficient calculating means shows a correspondence relationship between a square value of a distance from the distortion center position and the distortion correction coefficient. A lookup table to be stored is provided for each component, and the distortion correction coefficient is obtained for each component by referring to the lookup table for each component based on the square value of the calculated distance. It is.
[0035]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings.
1 to 16 show an embodiment of the present invention, and FIG. 1 is a block diagram showing a configuration of an image processing apparatus.
[0036]
This image processing apparatus includes a CCD 1 as an image pickup device that photoelectrically converts an optical subject image formed by an optical system to generate an electric image pickup signal, and correction of pixel defects in the image pickup signal output from the CCD 1. A preprocessing unit 2 that performs preprocessing such as A / D conversion, a frame memory 4 that stores a frame image that has been processed by the preprocessing unit 2, and image data stored in the frame memory 4 In order to perform distortion correction processing and enlargement / reduction processing on the image processing unit 6 that is an image processing unit that reads out predetermined blocks through a bus 11 to be described later and performs predetermined image processing, and image data processed by the image processing unit 6 A distortion correction processing unit 7 as a distortion correction processing unit, and image data output from the distortion correction processing unit 7 is compressed by JPEG or the like The JPEG processing unit 9 which is an image processing unit to be further compressed, and the image data compressed by the JPEG processing unit 9 are once written in the frame memory 4 via the bus 11 described later, and the written image data is transmitted via the bus 11. Including a memory card 10 which is a non-volatile storage means for reading out, inputting and storing as an image file, a bus 11 to which each circuit except the CCD 1 and a CPU 3 described later are connected, and each circuit described above And a CPU 3 which is a control means for comprehensively controlling the image processing apparatus.
[0037]
Here, the image processing unit 6, the distortion correction processing unit 7, and the JPEG processing unit 9 are connected so as to be able to perform pipeline processing through an information transmission path different from the bus 11 without going through the bus 11. Image data is transferred and processed in predetermined block units in a two-dimensional pixel array. As a result, image data with a large amount of data is not transferred over and over the bus 11 every process, so that the load on the bus 11 can be greatly reduced and the processing is performed in units of blocks. This makes it possible to reduce the capacity of an internal buffer (an internal memory unit 25 shown in FIG. 3 described later or a 2-port SRAM 25a shown in FIG. 14) of the image processing unit.
[0038]
In the example shown in FIG. 1, only one image processing unit 6 for performing image processing is provided, but a plurality of image processing units corresponding to a plurality of image processings are arranged on the pipeline processing path. Of course, it does not matter even if it makes it. Examples of image processing include conversion processing from a single plate signal to a three plate signal, low pass filter processing, edge enhancement processing, enlargement / reduction processing, and the like, as described above. The arrangement at this time may be on the upstream side of the distortion correction processing unit 7 or on the downstream side.
[0039]
FIG. 2 is a block diagram showing an outline of the configuration of the distortion correction processing unit 7.
[0040]
Color image data is usually decomposed into three (or more) independent signal components such as RGB and YCbCr, and processing is performed for each component. Here, a path (channel) in which each of three independent signal components is processed is defined as Ch. 0, Ch. 1, Ch. 2 will be described specifically, for example, Ch. 0, the signal to be processed is R, Ch. 1 is G, Ch. The signal processed in 2 is B or the like.
[0041]
In this distortion correction processing unit 7, the distortion correction processing unit for each channel transmits a request to the processing block of the corresponding channel in the previous stage, and the image data transmitted from the previous processing block in response to the request is sent to the predetermined block. After being received in units and subjected to distortion correction, it is output to the processing block of the corresponding channel in the subsequent stage. Specifically, Ch. The distortion correction processing unit for processing 0 data is 7A, Ch. The distortion correction processing unit for processing one data is 7B, Ch. The distortion correction processing unit for processing two data is 7C, and these are included in the distortion correction processing unit 7. When the distortion correction processing units 7A, 7B, and 7C receive a request for transmitting image data from the subsequent block, the image data is transmitted in units of predetermined blocks when the transmission is enabled. The data is transmitted to the subsequent block.
[0042]
In the configuration example shown in FIG. 1, the preceding processing block corresponds to the image processing unit 6 and the subsequent processing block corresponds to the JPEG processing unit 9, and these preceding and following processings are performed. The block is also configured to perform processing for each channel.
[0043]
The distortion correction processing unit 7 is provided with a control register 7a, and setting values and various data for the distortion correction processing unit 7 from the CPU 3 are set in the distortion correction processing units 7A, 7B, and 7C. At the same time, the status of the processing result can be read from the CPU 3.
[0044]
The outline of the process of the distortion correction processing unit 7 relating to one channel is roughly as follows. FIG. 5 is a diagram for explaining an outline of interpolation processing including distortion correction, and FIG. 6 is a diagram for explaining processing by 16-point interpolation.
[0045]
First, an image coordinate system (X, Y) after distortion correction processing as shown in FIG. 5B is prepared in advance. As a matter of course, nothing is required for the image data in the coordinate system (X, Y) before the distortion correction processing is started.
[0046]
A point of interest (pixel of interest) in the coordinate system (X, Y) (this corresponds to the coordinates of each pixel in the image after distortion correction processing and is similarly represented as (X, Y)). Then, the coordinates (interpolated coordinate data) (X ′, Y ′) of the image data corresponding to the point of interest (X, Y) are obtained by coordinate conversion (see FIG. 5A). The correspondence between (X, Y) and (X ′, Y ′) is determined by the optical properties of the optical system for forming the subject image on the CCD 1, and the parameters for defining the correspondence, etc. Is obtained in advance from the design value of the optical system or from the inspection of the optical system after manufacture, and is stored in a non-illustrated nonvolatile memory or the like. The CPU 3 reads out parameters from the nonvolatile memory or the like and sets them in the control register 7a.
[0047]
When the coordinates (X ′, Y ′) are obtained in this way, the coordinates of the peripheral image data necessary for obtaining the image data at the coordinates (X ′, Y ′) are determined. For example, when performing cubic interpolation processing, as shown in FIG. 6, the coordinates of 16 points around the coordinates (X ′, Y ′) (the point where the image data in FIG. 6 is Dout) are shown. Is determined.
[0048]
Therefore, by obtaining the image data Dout at the coordinates (X ′, Y ′) indicated by the white circles from the image data D0 to D15 at the coordinates of these 16 points using a predetermined interpolation formula, It becomes image data at the point of interest (X, Y) of the image after the correction processing.
[0049]
Image data after distortion correction is generated by calculating all the image data in a necessary range while moving the point of interest (X, Y).
[0050]
FIG. 3 is a block diagram illustrating a configuration of a distortion correction processing unit related to one channel.
[0051]
As shown in FIG. 3, the distortion correction processing unit 7 for one channel for performing such processing includes an interpolation position generation unit 21 for generating the coordinates (X, Y) of the point of interest, and this interpolation. A distortion correction coordinate conversion unit 22 for calculating the coordinates (X ′, Y ′) of the image data before the distortion correction processing from the coordinates (X, Y) of the target point generated by the position generation unit 21; If not, the coordinates (X, Y) output from the interpolation position generation unit 21 are selected and the distortion correction processing is performed. When the distortion correction processing is performed, the coordinates (X ′, Y output from the distortion correction coordinate conversion unit 22). The selector 23 for selecting ') and the reading of the image data from the previous processing block are controlled, and the image data of the peripheral pixels necessary for performing the interpolation processing corresponding to the coordinates output from the selector 23 are transmitted. As will be described later, the internal memory unit 2 5 and the image data from the previous processing block are accumulated, and the image data of peripheral pixels necessary for interpolation is output to the interpolation calculation unit 26 described later under the control of the memory control unit 24. The image data at the point of interest is, for example, Cubic as described above based on the internal memory unit 25 that performs the processing, the image data in the vicinity of the point of interest output from the internal memory unit 25, and the coordinates of the point of interest output from the selector 23. And an interpolation calculation unit 26 that obtains the result by interpolation and outputs it to a subsequent processing block.
[0052]
Among these constituent blocks, the interpolation position generation unit 21, the distortion correction coordinate conversion unit 22, and the selector 23 are elements constituting the interpolation coordinate generation unit 20.
[0053]
Here, in the interpolation position generation unit 21 and the distortion correction coordinate conversion unit 22, basic mathematical formulas for calculating coordinates, and practical use in which the circuit scale can be reduced by modifying the mathematical formulas. Mathematical formulas will be described.
[0054]
First, basic mathematical formulas will be described.
[0055]
The interpolation position generation unit 21 basically uses the interpolation start position (XST, YST) and the interpolation step (ΔX, ΔY) set by the CPU 3 in the control register 7a as shown in the following formula 1. In addition, the coordinates (X, Y) of the point of interest to be interpolated are calculated.
[Expression 1]

Here, k is a variable incremented when the point of interest is moved by ΔX in the X direction, and l is a variable incremented when the point of interest is moved by ΔY in the Y direction.
[0056]
The interpolation start position (XST, YST) can be set at an arbitrary position in the image. The interpolation step (ΔX, ΔY) can be enlarged or reduced by appropriately setting the CPU 3.
[0057]
The distortion correction coordinate conversion unit 22 basically uses the coordinates (X, Y) of the target point after the distortion correction processing output from the interpolation position generation unit 21 to the coordinates (X, Y) of the target point before the distortion correction processing. ', Y') is calculated as follows.
[0058]
That is, first, the coordinates (Xd, Yd) of the distortion center position in the image to be processed and the center deviation for correcting the deviation of the subject position from the time of shooting due to the distortion correction after the distortion correction processing. When distortion correction processing is performed using the correction amount (Xoff, Yoff) and a parameter indicating the optical properties of the optical system, the corrected image may protrude from the necessary range as image data or may be insufficient. Using the range correction magnification M for correction, intermediate calculated values (X (dots), Y (dots)) as shown in the following formula 2 (where the points added on the characters are written (Represented as (dots) etc.).
[Expression 2]

[0059]
Note that the coordinates (Xd, Yd) of the distortion center position are coordinates corresponding to the position on the image where the optical axes of the optical system that forms the subject optical image on the CCD 1 intersect.
[0060]
Further, the coordinates (Xd, Yd) of the distortion center position, the center deviation correction amounts (Xoff, Yoff), and the range correction magnification M are set in the control register 7a by the CPU 3.
[0061]
Next, when the calculated values (X (dots), Y (dots)), the coordinates (Xd, Yd) of the distortion center position, and image data are thinned out and taken in, the vertical and horizontal directions of the image And Z (more precisely, the square of Z) indicating the distance from the distortion center using the coefficients (SX, SY) for correcting the difference in spatial sampling between and Calculate as shown.
[Equation 3]

[0062]
Z calculated in this way, distortion correction coefficients A, B, and C, which are parameters indicating optical properties relating to distortion of the optical system set by the CPU 3 in the control register 7a, and the calculated value (X (dot)) , Y (dot)) and the coordinates (Xd, Yd) of the distortion center position, the coordinates of the point of interest before distortion correction processing corresponding to the coordinates (X, Y) of the point of interest after distortion correction processing are used. The coordinates (X ′, Y ′) are calculated as shown in Equation 4 below.
[Expression 4]

[0063]
Here, FIG. 4 is a diagram illustrating an example of distortion when a lattice-shaped subject is imaged through an optical system. First, FIG. 4A shows an example of a grid-like subject. Conventionally, since only the second order term of Z is taken into consideration, a barrel shape as shown in FIG. 4 (B) that can be generated when a subject as shown in FIG. 4 (A) is imaged through the optical system. 4 and the pincushion distortion as shown in FIG. 4C can be corrected to some extent, but it is possible to correct the Jinkasa distortion as shown in FIG. could not. However, in this embodiment, as shown in Equation 4, since the order exceeding the second-order term of Z, that is, for example, the fourth-order term or the sixth-order term is considered, such higher order aberrations are considered. Can be corrected with high accuracy. Although an example in which up to the 6th order term is considered is shown here, higher order aberrations such as an 8th order term and a 10th order term may be corrected.
[0064]
The coordinates (X, Y) calculated by the interpolation position generation unit 21 as described above or the coordinates (X ′, Y ′) calculated by the distortion correction coordinate conversion unit 22 are input to the selector 23 for distortion correction. Depending on whether or not to perform, the necessary one is selected.
[0065]
Next, a practical mathematical formula capable of reducing the circuit scale will be described.
[0066]
First, Formula 2 and Formula 1 are substituted into Formula 3 above, and a transformation is performed as shown in Formula 5 below.
[Equation 5]

[0067]
Then, a new constant formula as shown in Formula 6 is introduced, and Formula 5 is modified as shown in Formula 7.
[Formula 6]

[Expression 7]

[0068]
Here, Z (2 dots) in Equation 7 is defined as shown in Equation 8 below, and X2 and Y2 (interpolated coordinate data) are defined as shown in Equation 9 below.
[Equation 8]

[Equation 9]

[0069]
When Z (2 dots) defined as shown in Equation 8 and Equation 7 is used, Equation 4 above is transformed as shown in Equation 10 below.
[Expression 10]

[0070]
Here, in the formula 10, a constant formula or a variable formula defined as shown in the following formula 11 to formula 14 is used.
[Expression 11]

[Expression 12]

[Formula 13]

[Expression 14]

[0071]
When the calculation is performed according to the basic formulas as shown in the above formulas 1 to 4, the number of multiplications is 13 with 14 coefficients. That is, the coefficients are the distortion center position (Xd, Yd), center deviation correction amount (Xoff, Yoff), range correction magnification M, coefficient (Sx, Sy), distortion correction coefficients A, B, C, interpolation start position (XST). , YST) and 14 interpolation steps (ΔX, ΔY) in total. The number of multiplications is 2 for M in Formula 2, 2 for Sx and Sy in Formula 3, and 4 for 2 squares, and distortion correction coefficients A, B, in Formula 4. The multiplication with C is 3 times, the multiplication of Z to the 4th and 6th powers is 2 times, and the multiplication with square brackets is 2 times, for a total of 13 times.
[0072]
On the other hand, when the calculation is performed according to a practical mathematical formula as shown in the mathematical formulas 5 to 14, the number of multiplications is nine with 13 coefficients. That is, the coefficients are X (2 dots) ST, Y (2 dots) ST, ΔX (2 dots), ΔY (2 dots), X (3 dots), Y (3 dots), ΔX (3 dots), ΔY A total of 13 dots (3 dots), A (dots), B (dots), C (dots), Xd and Yd. The number of multiplications is two for the two squares in the curly brackets of Equation 7, twice for A (dot), B (dot), and C (dot) in Equation 14, and in Equation 14 The multiplication of Z (dot) to the fourth and sixth powers is two times, and the multiplication with F in Formula 10 is two times, for a total of nine times.
[0073]
The number handled in such an operation has a large dynamic range, and even if the number of calculations is slightly increased, the circuit scale becomes large, so by performing the calculation according to a practical formula as shown above, In addition to reducing the number of multipliers, it is possible to reduce the registers for setting the coefficients, thereby effectively reducing the circuit scale.
[0074]
Based on the practical expression modified in this way, the interpolation position calculation circuit 21a (see FIG. 14 described later) corresponding to the interpolation position generation unit 21 starts interpolation set in the control register 7a from the CPU 3. Using the position (X (3 dots), Y (3 dots)) and the interpolation step (ΔX (3 dots), ΔY (3 dots)), as shown in the above equation 13, the coordinates of the point of interest for interpolation ( Interpolation position) (X1, Y1) is calculated.
[0075]
Further, a distortion correction coefficient calculation circuit 22a (see FIG. 11 described later) serving as a distortion correction coefficient calculation means of the distortion correction coordinate conversion unit 22 is set to A (dots) and B (dots) set in the control register 7a from the CPU 3. ) And C (dots), the distortion correction coefficient F is calculated as shown in Equation 14 above. As will be described later, the distortion correction coefficient F can be calculated by a configuration as shown in FIG. 13 instead of the configuration shown in FIG.
[0076]
Further, the interpolation position correction circuit 22b (see FIG. 14 described later) of the distortion correction coordinate conversion unit 22 calculates the calculated distortion correction coefficient F and the coordinates of the distortion center position set in the control register 7a from the CPU 3 (see FIG. Xd, Yd) and the coordinates (interpolated coordinate data) (X ′, Y ′) of the point of interest before distortion correction processing from the coordinates (interpolated position) (X1, Y1) of the point of interest using Equation 10 above. It comes to calculate.
[0077]
Next, the interpolation calculation unit 26 reads out image data D0 to D15 of pixels in the vicinity of the coordinates from the internal memory unit 25 based on the coordinates output from the selector 23, and the following formula 15 is obtained. By using it, the image data Dout after the distortion correction processing related to the point of interest is calculated and output to the subsequent processing block.
[Expression 15]

Here, kx0 to kx3 and ky0 to ky3 are predetermined interpolation coefficients determined when performing cubic interpolation, for example.
[0078]
FIG. 7 is a diagram illustrating an example of chromatic aberration that occurs when an image is captured by the optical system.
[0079]
In the example shown in FIG. 7, a pincushion type distortion aberration is generated, and red R is shifted to the inside of the green G when viewed from the strain center, and blue B is shifted to the outside of the green G when viewed from the strain center. Chromatic aberration occurs. The deviation due to this chromatic aberration increases as the distance from the strain center increases.
[0080]
As described above, since the chromatic aberration is different for each color, the interpolation processing is also performed for each color as shown in FIG. FIG. 10 is a block diagram showing a more detailed configuration of the distortion correction processing unit 7.
[0081]
The distortion correction processing unit 7A is a Ch. 0 distortion correction circuit 7A1, the distortion correction processing unit 7B is a Ch. 1 distortion correction circuit 7B1, the distortion correction processing unit 7C is a Ch. The second distortion correction circuit 7C1 is provided as distortion correction calculation means.
[0082]
The distortion correction coefficient calculation circuit 22a for calculating the distortion correction coefficient F constitutes a part of the distortion correction coordinate conversion unit 22, and is shared by the distortion correction processing units 7A, 7B, and 7C. Is provided as a single circuit common to the three channels. 0 distortion correction coefficient F0, Ch. 1 distortion correction coefficient F1, Ch. 2 distortion correction coefficient F2 is calculated, and Ch. 0 distortion correction circuit 7A1, Ch. 1 distortion correction circuit 7B1, Ch. 2 is output to the distortion correction circuit 7C1 for 2 respectively.
[0083]
In addition, when image data is output to a subsequent processing block after performing interpolation processing related to distortion correction processing or enlargement / reduction processing, depending on the processing content of the subsequent processing block, images of three channels Data may be needed at the same time. However, as shown in FIG. 7, when there is chromatic aberration, the position of the pixel of interest differs depending on the color, and therefore, the three data are not always output at the same time.
[0084]
This will be described with reference to FIGS. FIG. 8 is a diagram showing the state of the image data stored in the internal memory unit at the time when the interpolation data related to B can be calculated, and FIG. 9 shows that the interpolation data related to each of R, G, and B can be calculated. It is a figure which shows the mode of the image data accumulate | stored in the internal memory part at the time of becoming.
[0085]
When interpolation data calculation processing is started simultaneously for three channels, image data from the previous processing block starts to be sequentially stored in the internal memory unit 25 of each channel (a 2-port SRAM 25a shown in FIG. 6 described later). . At this time, the portion already stored as data is a portion indicated by hatching. At the time shown in FIG. 8, the image data has been accumulated to the extent that the interpolation data for blue B can be calculated. However, the amount of interpolation data for green G and red R can still be calculated. Not in.
[0086]
Further, at the time shown in FIG. 9, the image data has been stored up to the point where the interpolation data regarding red R, green G, and blue B can be calculated.
[0087]
As described above, when the interpolation data of the three channels can be calculated, the grant synchronization circuit 27 as the grant synchronization means synchronizes so as to permit the transfer of the image data to the subsequent processing block.
[0088]
Ch. 0 is R, Ch. 1 is G, Ch. Taking the case where 2 is B as an example, when the state shown in FIG. 0 distortion correction circuit 7A1 outputs a request signal trok_0 requesting the start of the distortion correction processing. When the distortion correction circuit for B1 7B1 becomes capable of the distortion correction process, the request signal trok_1 requesting the start of the distortion correction process is output, and when the state becomes as shown in FIG. The second distortion correction circuit 7C1 outputs a request signal trok_2 requesting the start of the distortion correction process.
[0089]
The grant synchronization circuit 27, after all of these request signals trok_0, trok_1, and trok_2 are ready, that is, when it becomes possible to transmit the interpolation data on all three channels, the Ch. 0 to the part that performs the processing related to 0, the grant signal GRANT_N0 is sent to Ch. 1 for the part that performs the processing related to No. 1 and Ch. 2 is output to each of the parts performing the processing according to 2 and the Ch. 0 distortion correction circuit 7A1, Ch. 1 distortion correction circuit 7B1, Ch. As described later, the second distortion correction circuit 7C1 is controlled to start distortion correction processing simultaneously.
[0090]
As will be described later, the grant synchronization circuit 27 has a switch inside, and can switch whether or not the outputs of the three channels are synchronized.
[0091]
FIG. 11 is a block diagram showing a configuration of the distortion correction coefficient calculation circuit 22a.
[0092]
This distortion correction coefficient calculation circuit 22a is for calculating the distortion correction coefficient F by the above-described equation 14, and A (dot), B (dot), which reflects the optical properties of the optical system in the equation 14. C (dot) is set in the control register 7a from the CPU 3 for each channel, that is, for each color component of RGB, for example.
[0093]
In FIG. 11, FIG. 12 and FIG. 13, which will be described later, and in the following description from FIG. 11 to FIG. 13, Z indicates Z (2 dots) shown on the left side of Equation 8, and coefficient A , B, and C indicate A (dot), B (dot), and C (dot) shown on the left side of Expression 11.
[0094]
The distortion correction coefficient calculation circuit 22a receives the permission signal e_grant from the grant synchronization circuit 27, and calculates a distortion correction coordinate calculation circuit 31 that calculates the interpolation coordinates X2 and Y2 based on the above formula 9, and this distortion correction coordinate calculation. A floating number reduction circuit 32a for converting the interpolation coordinates X2 calculated by the circuit 31 into a floating point number; a floating number reduction circuit 32b for converting the interpolation coordinates Y2 calculated by the distortion correction coordinate calculation circuit 31 into a floating point number; A square calculator 33a that squares the interpolation coordinate X2 that has been floated by the floating-point circuit 32a, and a square calculator 33b that squares the interpolation coordinate Y2 that has been floated by the floating-point circuit 32b. And the square of the interpolation coordinate X2 calculated by the square calculator 33a and the square of the interpolation coordinate Y2 calculated by the square calculator 33b Thus, an adder 34 for calculating the square of Z (more precisely, Z (2 dots) as described above, and so on), and the output from the adder 34 is squared to obtain 4 of Z The square calculator 33c for calculating the power, the delay circuit 35b for delaying the output from the adder 34, the fourth power of Z calculated by the square calculator 33c, and the delay circuit 35b delays the timing. The multiplier 36 for calculating the sixth power of Z by multiplying the Z square output together with the output and the output from the adder 34 are delayed and the output from the square calculator 33c is delayed. Using the delay circuit 35a, the Z square and the fourth power of Z output from the delay circuit 35a in accordance with the timing, and the sixth power of Z output from the multiplier 36. For 0, Ch. 1, Ch. 2 and distance-dependent coefficient calculation circuits 37a, 37b, and 37c for calculating distortion correction coefficients F0, F1, and F2 for 2, respectively.
[0095]
FIG. 12 is a block diagram showing a configuration of a distance dependency coefficient calculation circuit 37 which is an arbitrary one of the distance dependency coefficient calculation circuits 37a, 37b and 37c.
[0096]
The distance-dependent coefficient calculation circuit 37 is a coefficient A (more precisely, as described above) set in the control register 7a from the CPU 3 to the square of Z that is delayed from the delay circuit 35a and output in synchronization. A multiplier 41a for multiplying A (dot), and so on) and a coefficient B (more accurate) set in the control register 7a from the CPU 3 to the fourth power of Z output from the delay circuit 35a with the timing being delayed. , The multiplier 41b that multiplies B (dot) as described above, and the coefficient C (from the CPU 3 set in the control register 7a to the sixth power of Z output from the multiplier 36) To be exact, the multiplier 41c for multiplying C (dot) as described above, and the output from the multiplier 41a is set in the control register 7a from the CPU 3. The fixed decimal number reduction circuit 42a for giving the sign signA of the coefficient A to be converted into a fixed decimal number and the output from the multiplier 41b are fixed by giving the sign signB of the coefficient B set in the control register 7a from the CPU 3 A fixed decimal number circuit 42b for converting to a decimal number; a fixed decimal number circuit 42c for converting the output from the multiplier 41c to a fixed decimal number by applying the sign C of the coefficient C set in the control register 7a from the CPU 3 to the output; By adding the output from the fixed decimal circuit 42a, the output from the fixed decimal circuit 42b, the output from the fixed decimal circuit 42c, and the constant 1.0 set by the CPU 3 to the control register 7a. An adder 43 that calculates a distortion correction coefficient F (that is, any one of the distortion correction coefficients F0, F1, and F2) based on the formula 14; It is configured to include.
[0097]
In FIG. 11 and FIG. 12, the portion indicated by a double line is a portion that performs arithmetic processing using a floating point number. By handling X2, Y2, Z, etc., which have a wide dynamic range, using a floating point number, The circuit scale is reduced while maintaining accuracy.
[0098]
At this time, as shown in FIG. 11, a part of the distortion correction coefficient calculation circuit 22a, that is, the distortion correction coordinate calculation circuit 31, the floating-point reduction circuits 32a and 32b, and the square calculators 33a, 33b, and 33c, The adder 34, the delay circuits 35a and 35b, and the multiplier 36 are configured as a common part independent of the channel. As a result, the circuit scale can be further reduced, and the power consumption can be reduced.
[0099]
FIG. 13 is a block diagram showing another example of the configuration of the distortion correction coefficient calculation circuit 22a.
[0100]
The distortion correction coefficient calculation circuit 22a shown in FIG. 13 includes the distortion correction coordinate calculation circuit 31, the floating-point reduction circuit 32a, the floating-point reduction circuit 32b, the square calculator 33a, and the square. Using the calculator 33b, the adder 34, and the square of the floating-point Z output from the adder 34 (more precisely, Z (2 dots) as described above, and so on). LUTs (look-up tables) 39a, 39b, and 39c that output distortion correction coefficients F0, F1, and F2 that have been reduced to a fixed decimal number by reference, respectively.
[0101]
Also in FIG. 13, the portion indicated by a double line is a portion where arithmetic processing using a floating point number is performed.
[0102]
As described above, the configuration example shown in FIG. 13 uses the lookup table to shorten the processing time for calculating the distortion correction coefficients F0, F1, and F2 and reduce the power consumption of the circuit. Yes.
[0103]
When applying this configuration example, the coefficients A, B, and C for each channel are not set by the CPU 3 but are fixed values. Of course, when the size of the lookup table may be increased, a lookup table corresponding to a plurality of combinations of the coefficients A, B, and C is prepared for each channel, and an appropriate one is used. It doesn't matter if you do.
[0104]
Next, FIG. FIG. 6 is a block diagram showing a configuration of a zero distortion correction circuit 7A1. Ch. 1 distortion correction circuit 7B1, Ch. 2 distortion correction circuit 7C1 is also configured in this Ch. Since the configuration is the same as that of the distortion correction circuit 7A1 for 0, the Ch. Only the zero distortion correction circuit 7A1 will be described.
[0105]
The interpolation position calculation circuit 21a is for calculating the coordinates (X1, Y1) of the point of interest as described above, and is a circuit portion corresponding to the interpolation position generation unit 21.
[0106]
The interpolation position correction circuit 22b uses the distortion correction coefficient F0 calculated by the distortion correction coefficient calculation circuit 22a to obtain the coordinates (X ′, Y ′) of the point of interest before the distortion correction processing as shown in the above equation 10. This is for calculation, and is a part of the distortion correction coordinate conversion unit 22.
[0107]
When the distortion correction process is not performed, the selector 23 selects the coordinates (X1, Y1) from the interpolation position calculation circuit 21a (regardless of whether the enlargement / reduction process is performed) and performs the distortion correction process. In this case, the coordinates (X ′, Y ′) from the interpolation position correction circuit 22b are selected (regardless of whether the enlargement / reduction process is involved).
[0108]
The coordinates (Xd, Yd) of the strain center position set by the CPU 3 via the control register 7a are added to the coordinates (X1, Y1) or coordinates (X ′, Y ′) selected by the selector 23. The units 27a and 27b are added.
[0109]
The outputs of the adders 27a and 27b are output to the read address generation circuit 24a and the buffer release amount calculation circuit 24c.
[0110]
Further, the coordinates (X1, Y1) calculated by the interpolation position calculation circuit 21a are also input to the read address generation circuit 24a separately from the path through the selector 23. Then, the read address generation circuit 24a obtains pixel data related to the coordinates (X1, Y1) (when performing 16-point interpolation, 16 pixel data centered on the coordinates (X1, Y1)). The stored address ADR is output to the 2-port SRAM 25a, and the interpolation coefficients (for example, interpolation coefficients kx0 to kx3 and ky0 to ky3 as shown in the above equation 15) and the pixel data to be output are at any position of D0 to D15. A data string control signal indicating whether or not the pixel data is present is output to the interpolation circuit 26a. Further, the read address generation circuit 24a sends a write permission signal WE_N0 to the Ch. Output to the processing block corresponding to 0.
[0111]
The 2-port SRAM 25a is a circuit portion corresponding to the internal memory unit 25 shown in FIG.
[0112]
The interpolation circuit 26a corresponds to the interpolation calculation unit 26 shown in FIG. 3, and performs an interpolation process as shown in the equation 15 using the image data read from the 2-port SRAM 25a, The subsequent image data is output to a subsequent processing block.
[0113]
Based on the output from the interpolation position calculation circuit 21a and the output from the selector 23 via the adders 27a and 27b, the buffer opening amount calculation circuit 24c is a memory that can be opened in the 2-port SRAM 25a. The capacity (buffer opening amount) is calculated.
[0114]
The buffer free capacity monitoring circuit 24d refers to the output of the buffer open amount calculation circuit 24c to grasp the status of the free capacity of the 2-port SRAM 25a.
[0115]
The write address generation circuit 24b receives the Ch. The image data related to 0 is received and recorded in the 2-port SRAM 25a.
[0116]
The data transmission permission / inhibition determination circuit 24e is connected to the Ch. In response to a request signal REQ_N0 for requesting image data from a subsequent processing block related to 0, whether or not image data can be transmitted based on the outputs of the write address generation circuit 24b and the buffer release amount calculation circuit 24c. The request signal trok_0 is output to the grant synchronization circuit 27 when it is possible.
[0117]
Next, the operation of the distortion correction processing unit 7 as shown in FIG. 14 will be described.
[0118]
The buffer free capacity monitoring circuit 24d monitors the free capacity of the 2-port SRAM 25a via the buffer opening amount calculation circuit 24c, and if there is a predetermined free capacity, the Ch. A request signal REQ_0 is issued so as to transmit image data in a predetermined block unit (hereinafter, referred to as unit line (UL) data as appropriate) to the preceding processing block related to 0 (step S1).
[0119]
Ch. The preceding processing block related to 0 receives this request signal REQ_0 and transmits a permission signal GRANT_0 when image data can be transmitted, and the buffer free space monitoring circuit 24d receives the permission signal GRANT_0 ( Step S2).
[0120]
The buffer free capacity monitoring circuit 24d grasps the free capacity of the 2-port SRAM 25a by a counter stored therein, and simultaneously decreases the value of the internal counter by 1 upon receipt of the permission signal GRANT. When the internal counter reaches 0, the buffer free capacity monitoring circuit 24d operates to withdraw the request signal REQ_0 (step S3).
[0121]
And Ch. The write enable signal WE_0 is input to the write address generation circuit 24b from the previous processing block related to 0, and then the image data is input. In response to this, the write address generation circuit 24b outputs a control signal to the 2-port SRAM 25a and writes the image data DATA in the area specified by the address ADDRESS of the 2-port SRAM 25a. The write address generation circuit 24b increases a BLC counter (a counter indicating how much data is stored in the 2-port SRAM 25a, which is an internal buffer) every time 1UL data is input, and determines whether data can be transmitted. Output to the circuit 24e (step S4).
[0122]
The data transmission permission / inhibition determination circuit 24e is connected to the Ch. When the request signal REQ_N0 is received from the subsequent processing block related to 0, it is determined whether or not UL data that can be transmitted next is in the 2-port SRAM 25a. If it is determined that there is, the request signal trok_0 is set. The data is transmitted to the grant synchronization circuit 27 (step S5).
[0123]
As will be described later, the grant synchronization circuit 27 outputs permission signals GRANT_N0, GRANT_N1, and GRANT_N2 to the subsequent processing block related to each channel and the interpolation position calculation circuit 21a related to each channel when the transmission conditions of the image data are satisfied. To do.
[0124]
Ch. The interpolation position calculation circuit 21a related to 0 starts the operation upon receiving this permission signal GRANT_N0, and calculates the next UL head coordinate when the calculation operation of the coordinates (X1, Y1) as the interpolation position is performed for 1 UL. (Step S6).
[0125]
The grant synchronization circuit 27 is configured so that the distortion correction coefficient calculation circuit 22a can output the distortion correction coefficient F0 in synchronization with the operation of the interpolation position calculation circuit 21a to output the coordinates (X1, Y1). At a proper timing, the permission signal e_grant is output to the distortion correction coefficient calculation circuit 22a (step S7).
[0126]
When the distortion correction coefficient calculation circuit 22a receives the permission signal e_grant, the distortion correction coefficient calculation circuit 22a calculates distortion correction coefficients F0, F1, and F2 related to each channel based on the above-described equation 14 and outputs the distortion correction coefficients F0, F1, and F2 to the interpolation position correction circuit 22b of each channel. Similarly to the interpolation position calculation circuit 21a, the distortion correction coefficient calculation circuit 22a calculates the next UL head coordinate when the operation for 1 UL is performed, and ends (step S8).
[0127]
Ch. The interpolation position correction circuit 22b according to 0 uses the distortion correction coefficient F0 received from the distortion correction coefficient calculation circuit 22a and the coordinates (X1, Y1) received from the interpolation position calculation circuit 21a, and the above equation 10 The coordinates (X ′, Y ′) are calculated based on The interpolating position correction circuit 22b also calculates the next UL head coordinate when the operation for 1 UL is performed according to the distortion correction coefficient calculation circuit 22a, and ends (step S9).
[0128]
The selector 23 selects the coordinates (X ′, Y ′) from the interpolation position correction circuit 22b when performing distortion correction processing according to the operation mode set by the CPU 3 via the control register 7a, and performs distortion. When correction processing is not performed, the coordinates (X1, Y1) from the interpolation position calculation circuit 21a are selected (step S10).
[0129]
The adders 27a and 27b add the coordinates (Xd, Yd) of the strain center position to the coordinates (X1, Y1) or coordinates (X ′, Y ′) selected by the selector 23 (step S11).
[0130]
Based on the coordinates received from the adders 27a and 27b, the read address generation circuit 24a outputs an address ADR of pixel data read from the 2-port SRAM 25a to be used for interpolation to the 2-port SRAM 25a, and also includes an interpolation coefficient and a data string. The control signal is output to the interpolation circuit 26a (step S12).
[0131]
The interpolation circuit 26a uses the interpolation coefficient and data string control signal received from the read address generation circuit 24a and the pixel data received from the 2-port SRAM 25a, as shown in Equation 15 above, to convert the interpolated pixel data. And Ch. The data is output to the subsequent processing block related to 0 (step S13).
[0132]
When it is confirmed that the buffer opening amount calculation circuit 24c has output the UL data to the end based on the outputs of the interpolation position calculation circuit 21a and the adders 27a and 27b, the UL head coordinates that have just been processed, the next In order to calculate the difference from the first UL coordinate of the data and release the buffer (area in the 2-port SRAM 25a) in which unnecessary data is stored, the buffer release amount is output to the buffer free space monitoring circuit 24d. After that, how much data is stored in Ch. Whether or not it is necessary to receive from the previous processing block related to 0 is transmitted to the data transmission availability determination circuit 24e (step S14).
[0133]
When it is confirmed in step S14 that the 2-port SRAM 25a, which is an internal buffer, has a free storage area, the buffer free capacity monitoring circuit 24d returns to step S1 and repeats the processing described above. (Step S15).
[0134]
The data transmission availability determination circuit 24e transmits the next UL data to the subsequent processing block based on the value of the BLC counter from the write address generation circuit 24b and the output from the buffer release amount calculation circuit 24c. If it is determined whether or not transmission is possible, the process of step S5 is performed (step S16).
[0135]
FIG. 15 is a block diagram showing the configuration of the grant synchronization circuit 27, and FIG. 16 is a timing chart for explaining the operation of the grant synchronization circuit 27.
[0136]
First, DT_ON obtained by referring to the control register 7a holds a Boolean value indicating whether or not to perform distortion correction. When 0 (false), low level, 1 (true) In some cases, the signal is acquired as a high level signal.
[0137]
This grant synchronization circuit 27 is connected to Ch. Request signal trok — 0 from Ch. 0 distortion correction circuit 7A1 and Ch. Request signal trok_1 from the distortion correction circuit 7B1 and Ch. An AND circuit 51 for taking a logical product with the request signal trok_2 from the second distortion correction circuit 7C1, and a differentiating circuit 52 for detecting the rise of the request signals trok_0, trok_1, trok_2 and the output of the AND circuit 51; When DT_ON is low level, it is connected to the output side of the differentiation circuit 52 related to the request signal trok_0, and when DT_ON is high level, it is connected to the output side of the differentiation circuit 52 related to the AND circuit 51. The switch 53a to be switched as described above is connected to the output side of the differentiation circuit 52 related to the request signal trok_1 when DT_ON is at a low level, and the differentiation circuit 52 related to the AND circuit 51 when the DT_ON is at a high level. The switch 53b that is switched so as to be connected to the output side of the DT_ON When it is -level, it is connected to the output side of the differentiation circuit 52 related to the request signal trok_2, and when DT_ON is high level, it is switched to be connected to the output side of the differentiation circuit 52 related to the AND circuit 51. An AND circuit 54 that outputs a logical product of the output of the differentiation circuit 52 and the DT_ON of the AND circuit 51 and the DT_ON to the distortion correction coordinate calculation circuit 31 of the distortion correction coefficient calculation circuit 22a as the permission signal e_grant. , And is configured.
[0138]
The operation of the grant synchronization circuit 27 is as follows.
[0139]
First, when DT_ON is at a low level, that is, when distortion correction is not performed, the switches 53a, 53b, and 53c are respectively switched to the output side of the differentiating circuit 52 that detects the request signals trok_0, trok_1, and trok_2. ing.
[0140]
At this time, as shown on the left side of FIG. 16, when a high level signal is input to the request signal trok_0, its rise is detected by the differentiating circuit 52 and output as GRANT_N0. Next, when a high level signal is input to the request signal trok_1, its rise is detected by the differentiating circuit 52 and output as GRANT_N1, and then a high level signal is input to the request signal trok_2. The rising edge is detected by the differentiating circuit 52 and output as GRANT_N2. As described above, when DT_ON is at the low level, the output of the image data from the three channels is performed at each timing for each channel.
[0141]
Further, when DT_ON is at a low level, the output of the AND circuit 54 remains at a low level regardless of the output of the AND circuit 51, and the permission signal e_grant is not output (or is an unpermitted output). In addition, the distortion correction coefficient calculation circuit 22a does not calculate the distortion correction coefficients F0, F1, and F2. Accordingly, the interpolation position correction circuit 22b associated with each channel does not operate, and consequently, the distortion interpolation process is not performed.
[0142]
Next, when DT_ON is at a high level, that is, when distortion correction is performed, the switches 53 a, 53 b, and 53 c are switched to the output side of the differentiating circuit 52 related to the AND circuit 51.
[0143]
At this time, as shown on the right side of FIG. 16, even if a high level signal is input to the request signal trok_0, the request signals trok_1 and trok_2 remain at the low level at that time. Is maintained at a low level.
[0144]
Thereafter, even when the request signal trok_0 is at a high level and the request signal trok_1 is at a high level, the output of the AND circuit 51 remains at a low level while the request signal trok_2 is at a low level.
[0145]
When all of the request signals trok_0, trok_1, and trok_2 become high level, the output of the AND circuit 51 becomes high level, and the differentiation circuit 52 detects the rising edge.
[0146]
The output of the differentiating circuit 52 is simultaneously output as the permission signals GRANT_N0, GRANT_N1, and GRANT_N2, and is also output to the AND circuit 54, and is logically ANDed with DT_ON which is at the high level, and the above-mentioned as the permission signal e_grant This is output to the distortion correction coefficient calculation circuit 22a.
[0147]
In this way, when DT_ON is at a high level, output of image data from the three channels is performed at a synchronized timing.
[0148]
According to such an embodiment, an image processing apparatus with a small circuit scale and low power consumption that can perform distortion correction and chromatic aberration correction.
[0149]
In particular, by sharing at least a part of the distortion correction coefficient calculation circuit for a plurality of channels, the circuit scale can be effectively reduced and the power consumption can be reduced. At this time, since the calculation is performed by treating a number having a wide dynamic range as a floating point number, the circuit scale can be reduced while maintaining accuracy. When the distortion correction coefficient is obtained with reference to a lookup table, the circuit scale can be reduced while reducing the processing time.
[0150]
In addition, since the grant synchronization circuit is provided so that the image data output from each channel can be performed simultaneously, it is possible to cope well with the case where the image data of a plurality of channels is required simultaneously in the subsequent processing block. it can. At this time, the grant synchronization circuit can perform processing so that image data of a plurality of channels is output at respective timings by switching, and therefore, selection can be made as necessary.
[0151]
In addition, since the interpolation coordinate data is generated based on a practical mathematical formula that is modified so that the number of multiplications is reduced as compared with the case based on the basic mathematical formula, the interpolation position generation unit and the distortion correction coordinate conversion unit The number of multipliers provided inside the circuit can be reduced, and the circuit scale can be reduced.
[0152]
Further, the enlargement / reduction process can be performed simultaneously with the same circuit as the distortion correction process.
[0153]
It should be noted that the present invention is not limited to the above-described embodiments, and various modifications and applications can be made without departing from the spirit of the invention.
[0154]
【The invention's effect】
As described above, according to the present invention, an image processing apparatus having a small circuit scale and low power consumption that can perform distortion correction and chromatic aberration correction.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of an image processing apparatus according to an embodiment of the present invention.
FIG. 2 is a block diagram showing an outline of a configuration of a distortion correction processing unit in the embodiment.
FIG. 3 is a block diagram showing a configuration of a distortion correction processing unit related to one channel in the embodiment.
FIG. 4 is a diagram showing an example of distortion when a lattice-like subject is imaged through an optical system in the embodiment.
FIG. 5 is a diagram for explaining an outline of interpolation processing including distortion correction in the embodiment.
FIG. 6 is a diagram for explaining processing by 16-point interpolation in the embodiment.
FIG. 7 is a diagram showing an example of chromatic aberration that occurs when an image is taken by an optical system in the embodiment.
FIG. 8 is a diagram showing a state of image data stored in an internal memory unit at the time when interpolation data related to B can be calculated in the embodiment.
FIG. 9 is a diagram showing a state of image data stored in an internal memory unit at the time when interpolation data related to each of R, G, and B can be calculated in the embodiment.
FIG. 10 is a block diagram showing a more detailed configuration of a distortion correction processing unit in the embodiment.
FIG. 11 is a block diagram showing a configuration of a distortion correction coefficient calculation circuit in the embodiment.
FIG. 12 is a block diagram showing a configuration of a distance dependent coefficient calculation circuit in the embodiment.
FIG. 13 is a block diagram showing another example of the configuration of the distortion correction coefficient calculation circuit in the embodiment.
FIG. 14 shows Ch. The block diagram which shows the structure of the distortion correction circuit for 0. FIG.
FIG. 15 is a block diagram showing a configuration of a grant synchronization circuit in the embodiment.
FIG. 16 is a timing chart for explaining the operation of the grant synchronization circuit in the embodiment.
FIG. 17 is a diagram showing a general image processing procedure in the image processing apparatus according to the embodiment;
18 is a block diagram showing a configuration of a conventional image processing apparatus for performing general image processing as shown in FIG.
FIG. 19 is a diagram for explaining a memory amount necessary for performing distortion correction processing in the related art.
[Explanation of symbols]
1 ... CCD
2 ... Pre-process part
3 ... CPU
4 ... Frame memory
6 ... Image process section (image processing section)
7, 7A, 7B, 7C ... distortion correction processing section (distortion correction processing means, image processing section)
7a: Control register
7A1 ... Ch. 0 distortion correction circuit (distortion correction calculation means, image processing unit)
7B1 ... Ch. 1 distortion correction circuit (distortion correction calculation means, image processing unit)
7C1 ... Ch. 2 distortion correction circuit (distortion correction calculation means, image processing unit)
9 ... JPEG processing unit (image processing unit)
10 ... Memory card, etc.
11 ... Bus
20 ... Interpolated coordinate generator
21 ... Interpolation position generation unit (part of the interpolation coordinate generation unit)
21a: Interpolation position calculation circuit
22: Distortion correction coordinate conversion unit (part of interpolation coordinate generation unit)
22a ... distortion correction coefficient calculation circuit (distortion correction coefficient calculation means)
22b: Interpolation position correction circuit
23 ... Selector (part of the interpolation coordinate generator)
24. Memory control unit
24a ... Read address generation circuit
24b ... Write address generation circuit
24c: Buffer opening amount calculation circuit
24d: Buffer free space monitoring circuit
24e: Data transmission availability determination circuit
25. Internal memory section
25a 2 port SRAM (internal buffer)
26. Interpolation calculation unit
26a ... interpolation circuit
27 ... Grant synchronization circuit (grant synchronization means)
31 ... Coordinate calculation circuit for distortion correction
32a, 32b... Floating number reduction circuit
37, 37a, 37b, 37c ... distance dependent coefficient calculation circuit
42a, 42b, 42c ... fixed decimal circuit
39a, 39b, 39c ... LUT (look-up table)

Claims (7)

In an image processing apparatus that processes electronic image data obtained by taking an image through an optical system, which is composed of a plurality of components,
A distortion correction coefficient calculating means for calculating a distortion correction coefficient for use in correcting the distortion caused by the optical system for each of the components based on a distance from a strain center position;
Distortion correction calculation means for correcting distortion of the image data for each component using the distortion correction coefficient for each component calculated by the distortion correction coefficient calculation means;
An image processing apparatus comprising a distortion correction processing unit configured to include: 2. The image processing apparatus according to claim 1, wherein a plurality of the distortion correction calculation means are provided corresponding to all the components constituting the image data on a one-to-one basis. The distortion correction calculation means is configured to have an internal buffer for storing image data, and the distortion correction processing start request is made when the image data necessary for performing the distortion correction processing is accumulated in the internal buffer. Is output,
After the distortion correction processing start requests from all of the plurality of distortion correction calculation means are prepared, the apparatus further includes grant synchronization means for controlling all of the distortion correction calculation means to start the distortion correction processing. The image processing apparatus according to claim 2. The grant synchronization means outputs a permission signal for permitting a distortion correction coefficient calculation process to the distortion correction coefficient calculation means, and starts calculating distortion correction coefficients for all components by the distortion correction coefficient calculation means. The image processing apparatus according to claim 3, wherein all the distortion correction calculation means are controlled to start distortion correction processing. The distortion correction coefficient calculating means is configured such that at least a part thereof is configured as a part that is commonly used for all components when calculating the distortion correction coefficient for each component. The image processing apparatus according to any one of claims 1 to 4. The distortion correction coefficient calculating means calculates a square value of a distance from the distortion center position, and obtains the distortion correction coefficient for each component based on the square value.
The image processing apparatus according to claim 5, wherein the portion used in common for all the components is a portion for calculating a square value of a distance from the distortion center position. The distortion correction coefficient calculating means is configured to have a lookup table for storing the correspondence relationship between the square value of the distance from the distortion center position and the distortion correction coefficient for each component.
The image processing apparatus according to claim 6, wherein the distortion correction coefficient is obtained for each component by referring to a lookup table for each component based on the square value of the calculated distance.

Images