JP4465844B2 - Image processing apparatus and method, and recording medium - Google Patents

Description

なお、エフェクトパラメータａ₁₁乃至ａ₃₃は、次式に示すように逆行列Ａ^-1の要素である。
【数２】

また、ライティング変調軸Ｔの回転係数ｐ，ｑは、ｐ＝cosθ、ｑ＝sinθである。
【００８８】
このように、リードアドレス(Xm,Ym,T)は、スクリーンアドレス(H,V)をパラメータとする関数値X(H,V)，Y(H,V)，T(H,V)，Z(H,V)を用いて演算される。
【００８９】
ところで、リードアドレスは、順次走査するスクリーンアドレスの画素毎（クロック毎）に算出されることになるが、スクリーンアドレスの全ての画素について関数値X(H,V)，Y(H,V)，T(H,V)，Z(H,V)を演算し、リードアドレスを算出していたのでは、その演算量は膨大なものとなり、演算専用の回路が必要となる。
【００９０】
そこで、図３０に示すように、スクリーンアドレスの４端点、すなわち左上点(0,0)、左下点(0,539)、右上点(1919,0)、および右下点(1919,539)、について、予め関数値X(0,0)，Y(0,0)，T(0,0)，Z(0,0)，X(0,539)，Y(0,539)，T(0,539)，Z(0,539)，X(1919,0)，Y(1919,0)，T(1919,0)，Z(1919,0)，X(1919,539)，Y(1919,539)，T(1919,539)，Z(1919,539)（以下、関数値X(0,0)乃至Z(1919,539)と記述する）を演算するようにし、スクリーンアドレスの他の画素についての関数値X(H,V)，Y(H,V)，T(H,V)，Z(H,V)は、当該４端点について演算した関数値X(0,0)乃至Z(1919,539)を用いて補間し、対応するリードアドレスを算出するようにする。
【００９１】
このようにスクリーンアドレスの４端点に対応する関数値X(0,0)乃至Z(1919,539)を用いて、他の画素についての関数値X(H,V)，Y(H,V)，T(H,V)，Z(H,V)を補間する処理を、以下、スーパインタポレーション(Super Interpolation)と称し、特に、左上点(0,0)と左下点(0,539)の垂直方向の補間処理、または右上点(1919,0)と右下点(1919,539)の垂直方向の補間処理をスーパインタポレーション（Ｖ）と称し、スーパインタポレーション（Ｖ）の結果などの水平走査線上の左右両端点の関数値を用いた水平方向の補間処理をスーパインタポレーション（Ｈ）と称する。
【００９２】
次に、スーパインタポレーションの処理タイミングについて、図３１を参照して説明する。あるフィールド画像についてスーパインタポレーションを施す場合、スクリーンアドレスの４端点のそれぞれについて、当該フィールド画像の１フィールド前までに関数値X(0,0)乃至Z(1919,539)を予め演算して所定のレジスタ（後述）に保持する。そして、タイミング信号VMIXのEnableに同期して水平帰線期間(BLANK(H))の初期においてスーパインタポレーション（Ｖ）を実行し、タイミング信号HMIXのEnableに同期してスクリーンアドレスを水平走査する期間（ACTIVE AREA)において、クロック毎にスーパインタポレーション（Ｈ）を実行する。
【００９３】
このように、スーパインタポレーション（Ｈ）とスーパインタポレーション（Ｖ）は、実行タイミングが異なる。
【００９４】
図３２は、アドレスジェネレータ２１の構成例を示している。レジスタ演算ブロック９１は、スクリーンアドレスの４端点の関数値X(0,0)乃至Z(1919,539)を演算してスーパインタポレーションブロック９３に供給する。ミキサ係数ブロック９２は、内蔵するレジスタに予め保持されているミキサ係数をスーパインタポレーションブロック９３に供給する。
【００９５】
スーパインタポレーションブロック９３は、レジスタ演算ブロック９１から供給されるスクリーンアドレスの４端点の関数値(0,0)乃至Z(1919,539)、および、ミキサ係数ブロック９２から供給されるミキサ係数を用いて、スーパインタポレーション（Ｈ）およびスーパインタポレーション（Ｖ）を実行し、得られるスクリーンアドレス上の４端点以外の画素にそれぞれ対応する関数値X(H,V)，Y(H,V)，T(H,V)，Z(H,V)を補間してリードアドレス演算ブロック９４に出力する。
【００９６】
リードアドレス演算ブロック９４は、スーパインタポレーションブロック９３から入力されるスクリーンアドレスの全ての画素にそれぞれ対応する関数値X(H,V)，Y(H,V)，T(H,V)，Z(H,V)を用い、リードアドレスを生成してバッファ２０に出力する。
【００９７】
図３３は、スーパインタポレーションブロック９３の構成例を示している。スーパインタポレーションブロック９３は、関数値X(H,V)を補間するブロック、関数値Y(H,V)を補間するブロック、関数値T(H,V)を補間するブロック、および関数値Z(H,V)を補間するブロックから構成される。
【００９８】
関数値X(H,V)を補間するブロックのREG_V_START_XLレジスタ１０１−Ｘは、レジスタ演算ブロック９１から供給される左上点(0,0)についての関数値X(0,0)を保持し、セレクタ１０７−ＸのＡ端子に出力する。REG_V_START_XRレジスタ１０２−Ｘは、レジスタ演算ブロック９１から供給される右上点(1919,0)についての関数値X(1919,0)を保持し、セレクタ１０７−ＸのＢ端子に出力する。FF_H_START_Xレジスタ１０３−Ｘは、セレクタ１１２−ＸのＡ端子を介して入力されるミキサ１１１−Ｘの出力を保持し、セレクタ１０８−ＸのＢ端子に出力する。FF_H_END_Xレジスタ１０４−Ｘは、セレクタ１１２−ＸのＢ端子を介して入力されるミキサ１１１−Ｘの出力を保持し、セレクタ１１０−ＸのＢ端子に出力する。REG_V_END_XLレジスタ１０５−Ｘは、レジスタ演算ブロック９１から供給される左下点(0,539)についての関数値X(0,539)を保持し、セレクタ１０９−ＸのＢ端子に出力する。REG_V_END_XRレジスタ１０６−Ｘは、レジスタ演算ブロック９１から供給される右下点(1919,539)についての関数値X(1919,539)を保持し、セレクタ１０９−ＸのＡ端子に出力する。
【００９９】
セレクタ１０７−Ｘ乃至１１０−Ｘは、Ａ端子またはＢ端子への入力を後段に出力する。セレクタ１１２−Ｘは、ミキサ１１１−Ｘの出力をFF_H_START_Xレジスタ１０３−Ｘ、またはFF_H_END_Xレジスタ１０４−Ｘに出力する。ミキサ１１１−Ｘは、Ａ端子に入力されるセレクタ１０８−Ｘから出力をＡとし、Ｂ端子に入力されるセレクタ１１０−Ｘから出力をＢとし、ミキサ係数ブロック９２から供給されるミキサ係数をｋ_nとした場合、クロック毎に次式を用いて補間値Ｃを後段に出力する。
補間値Ｃ＝Ａ・（１．０−ｋ_n）＋Ｂ・ｋ_n
ただし、実際には乗算回数を１回減少させるために次式が用いられる。
補間値Ｃ＝ｋ_n（Ｂ−Ａ）＋Ａ
【０１００】
なお、関数値Y(H,V)，T(H,V)，Z(H,V)のそれぞれを補間する各ブロックの構成は、関数値X(H,V)を演算するブロックの構成と同様であるので、その説明は省略する。ただし、REG_V_START_XLレジスタ１０１−Ｘ乃至REG_V_END_ZRレジスタ１０６−Ｚと、それらに保持させる関数値X(0,0)乃至Z(1919,539)との対応関係は、図３４に示すとおりである。
【０１０１】
図３５は、ミキサ係数ブロック９２に内蔵されているレジスタと、そこに保持されているミキサ係数との対応関係を示している。
【０１０２】
次に、スーパインタポレーションブロック９３の動作について説明する。なお、REG_V_START_XLレジスタ１０１−Ｘ乃至REG_V_END_ZRレジスタ１０６−Ｚには、対応する関数値(0,0)乃至Z(1919,539)がレジスタ演算ブロック９１から供給されているとする。
【０１０３】
始めに、スクリーンアドレスの垂直成分Ｖが初期化されてＶ＝０とされ、タイミング信号VMIXのEnableに同期してスーパインタポレーション（Ｖ）が開始される。まず、スクリーンアドレスの左端点(0,V)のスーパインタポレーション（Ｖ）を実行するために、ミキサ１１１−Ｘ乃至１１１−Ｚへの関数値の入力元および出力先が、図３６に示すとおりとなるように、各ブロックにおいてスイッチング等がなされる。
【０１０４】
具体的には、例えば関数値X(H,V)を補間するブロックでは、図３７に示すように、セレクタ１０７−Ｘ乃至１１０−Ｘ，１１２−Ｘがスイッチングされる。これにより、ミキサ１１１−ＸのＡ端子には、REG_V_START_XLレジスタ１０１−Ｘに保持されている左上点(0,0)についての関数値X(0,0)が入力され、Ｂ端子には、REG_V_END_XLレジスタ１０５−Ｘに保持されている左下点(0,539)についての関数値X(0,539)が入力される。ミキサ１１１−Ｘにはさらに、ミキサ係数供給ブロック９２からミキサ係数が供給される。ミキサ１１１−Ｘは、スクリーンアドレスの左端点(0,V)についての関数値X(0,V)を補間する。補間さえた関数値X(0,V)は、セレクタ１１２−Ｘを介してFF_H_START_Xレジスタ１０３−Ｘにラッチされる。
【０１０５】
なお、他のブロックにおいても同様の処理がなされ、スクリーンアドレスの左端点(0,V)についての関数値Y(0,V)、関数値T(0,V)、関数値Z(0,V)が、それぞれ対応するFF_H_START_Xレジスタ１０３−Ｙ乃至１０３−Ｚにラッチされる。
【０１０６】
次に、スクリーンアドレスの右端点(1919,V)のスーパインタポレーション（Ｖ）を実行するために、ミキサ１１１−Ｘ乃至１１１−Ｚへの関数値の入力元および出力先が、図３８に示すとおりとなるように、各ブロックにおいてスイッチング等がなされる。
【０１０７】
具体的には、例えば関数値X(H,V)を補間するブロックでは、図３９に示すように、セレクタ１０７−Ｘ乃至１１０−Ｘ，１１２−Ｘがスイッチングされる。これにより、ミキサ１１１−ＸのＡ端子には、REG_V_START_XRレジスタ１０２−Ｘに保持されている右上点(1919,539)についての関数値X(1919,539)が入力され、Ｂ端子には、REG_V_END_XRレジスタ１０６−Ｘに保持されている右下点(1919,539)についての関数値X(1919,539)が入力される。ミキサ１１１−Ｘにはさらに、ミキサ係数供給ブロック９２からミキサ係数が供給される。ミキサ１１１−Ｘは、スクリーンアドレスの右端点(1919,V)についての関数値X(1919,V)を補間する。補間された関数値X(1919,V)は、セレクタ１１２−Ｘを介してFF_H_END_X１０４−Ｘにラッチされる。
【０１０８】
なお、他のブロックにおいても同様の処理がなされ、スクリーンアドレスの右端点(1919,V)についての関数値Y(1919,V)、関数値T(1919,V)、関数値Z(1919,V)が、それぞれ対応するFF_H_END_Xレジスタ１０４−Ｙ乃至１０４−Ｚにラッチされる。
【０１０９】
ここまでの処理は、水平帰線期間において実行される。
【０１１０】
その後、タイミング信号HMIXのEnableに同期し、スクリーンアドレスの水平成分Ｈが初期化されてＨ＝０とされ、スーパインタポレーション（Ｈ）が開始される。スーパインタポレーション（Ｈ）を実行するために、ミキサ１１１−Ｘ乃至１１１−Ｚへの関数値の入力元および出力先が、図４０に示すとおりとなるように、各ブロックにおいてスイッチング等がなされる。
【０１１１】
具体的には、例えば関数値X(H,V)を補間するブロックでは、図４１に示すように、セレクタ１０８−Ｘ，１１０−Ｘがスイッチングされる。これにより、ミキサ１１１−ＸのＡ端子には、FF_H_START_Xレジスタ１０３−Ｘに保持されている左端点(0,V)についての関数値X(0,V)が入力され、Ｂ端子には、FF_H_END_Xレジスタ１０４−Ｘに保持されている右端点(1919,V)についての関数値X(1919,V)が入力される。ミキサ１１１−Ｘにはさらに、クロック毎に、ミキサ係数供給ブロック９２からミキサ係数が供給される。ミキサ１１１−Ｘは、クロック毎に順次、左端点(0,V)から右端点(1919,V)についての関数値X(H,V)を補間してリードアドレス演算ブロック９４に供給する。
【０１１２】
なお、他のブロックにおいても同様の処理がなされ、クロック毎に左端点(0,V)から、順次、右端点(1919,V)までについての関数値Y(H,V)、関数値T(H,V)、関数値Z(H,V)が補間されてリードアドレス演算ブロック９４に供給される。
【０１１３】
スクリーンアドレスの水平成分Ｈが初期化された後、ここまでの処理は、水平走査期間に実行される。
【０１１４】
その後、垂直成分Ｖが１だけインクリメントされ、上述したスーパインタポレーション（Ｖ）以降の処理が繰り返される。そして、垂直成分Ｖが５４０に達した場合、処理中のフィールドに対するスーパインタポレーションは終了されて、次のフィールドが処理の対象とされる。
【０１１５】
以上説明のように、水平帰線期間においてスーパインタポレーション（Ｖ）を実行し、水平走査期間においてスーパインタポレーション（Ｈ）を実行するようにしたので、スーパインタポレーション（Ｖ）とスーパインタポレーション（Ｈ）を同一の回路（スーパインタポレーションブロック９３）を共用して実行することが可能となる。
【０１１６】
次に、補間回路２２について図４２を参照して説明する。補間回路２２は、バッファ２０にバッファリングされている映像信号がＨＤフォーマットである場合、４画素の映像信号を用いる４点補間処理を動作周波数７４．２５MHzで実行する。また、バッファ２０に、ＳＤフォーマットの映像信号がフィールド・フレーム変換されてバッファリングされている場合、１６画素の映像信号を用いる１６点補間処理を、動作周波数５４MHz（ＳＤフォーマットの映像信号を処理する際の通常の動作周波数１３．５MHzの４倍速）で実行する。
【０１１７】
図４２は、補間回路２２の構成例を示している。補間回路２２は、バッファ２０のユニットＵ０，Ｌ０から同時に入力される垂直方向に隣接する２画素の映像信号を比例分配して、当該２画素の間の位置に対応する映像信号の補間値ＴＡを演算する垂直方向比例分配回路１２１、バッファ２０のユニットＵ１，Ｌ１から同時に入力される垂直方向に隣接する２画素の映像信号を比例分配して、当該２画素の間の位置に対応する映像信号の補間値ＴＢを演算する垂直方向比例分配回路１２２、および、垂直方向比例分配回路１２１から入力される補間値ＴＡと垂直方向比例分配回路１２２から入力される補間値ＴＢを比例分配する水平方向比例分配回路１２３から構成される。
【０１１８】
図４３は、垂直方向比例分配回路１２１の構成例を示している。垂直方向比例分配回路１２１には、バッファ２０のユニットＵ０，Ｌ０から同時に入力される垂直方向に隣接する２画素の映像信号の他、および、当該２画素の間の補間点の垂直方向の位置を示す４ビットの位置情報ｒ、および、セレクタ１４３，１４４を制御するsel信号が入力される。
【０１１９】
ユニットＵ０からの映像信号は遅延回路（Ｄ）１４１に入力され、ユニットＬ０からの映像信号は遅延回路１４２に入力される。位置情報ｒは、遅延回路１４８に入力される。sel信号は、遅延回路１５２に入力される。
【０１２０】
遅延回路（Ｄ）１４１は、ユニットＵ０からの映像信号を所定のクロック周期だけ遅延して、セレクタ１４３のａ端子およびセレクタ１４４のｂ端子に出力する。遅延回路１４２は、ユニットＬ０からの映像信号を所定のクロック周期だけ遅延して、セレクタ１４３のｂ端子およびセレクタ１４４のａ端子に出力する。
【０１２１】
セレクタ１４３は、遅延回路１５２から入力されるsel信号に基づき、ａ端子に入力されるユニットＵ０からの映像信号、またはｂ端子に入力されるユニットＬ０からの映像信号を乗算器１４５に出力する。セレクタ１４４は、NOT回路１５３から入力される反転されたsel信号に基づき、ａ端子に入力されるユニットＬ０からの映像信号、またはｂ端子に入力されるユニットＵ０からの映像信号を乗算器１４６に出力する。したがって、乗算器１４５，１４６の一方にはユニットＵ０からの映像信号が入力され、他方にはユニットＬ０からの映像信号が入力される。ここで、乗算器１４５に入力される映像信号の値をＡとし、乗算器１４６に入力される映像信号の値をＢとする。
【０１２２】
乗算器１４５は、遅延回路１５１から入力される値（１６−ｒ）に、セレクタ１４３から入力される映像信号の値Ａを乗算して演算器１４７に出力する。乗算器１４６は、遅延回路１４９から入力される位置情報の値ｒに、セレクタ１４４から入力される映像信号の値Ｂを乗算して演算器１４７に出力する。演算器１４７は、乗算器１４５の出力と乗算器１４６の出力を加算して１６で除算する。
【０１２３】
以上説明したように構成される垂直方向比例分配回路１２１は、次式で示される垂直方向の補間値ＴＡを水平方向比例分配回路１２３に出力する。
補間値ＴＡ＝（Ａ＊（１６−ｒ）＋Ｂ＊ｒ）／１６
【０１２４】
なお、垂直方向比例分配回路１２２の構成は、垂直方向比例分配回路１２１と同様であるので、その説明は省略する。
【０１２５】
図４４は、水平方向比例分配回路１２３の構成例を示している。水平方向比例分配回路１２３には、垂直方向比例分配回路１２１からの垂直方向の補間値ＴＡ、垂直方向比例分配回路１２２からの垂直方向の補間値ＴＢの他、補間点の水平方向の位置を示す４ビットの位置情報ｒ’が補間係数供給回路１７１，１７２に入力される。
【０１２６】
乗算器１６１は、垂直方向比例分配回路１２１からの垂直方向の補間値ＴＡと、補間係数供給回路１７１から入力される補間係数Ｃｉを乗算してレジスタ（Ｒ０）１６３に出力する。乗算器１６２は、垂直方向比例分配回路１２２からの垂直方向の補間値ＴＢと、補間係数供給回路１７２から入力される補間係数Ｃｉを乗算してレジスタ（Ｒ１）１６４に出力する。
【０１２７】
加算器１６５は、レジスタ（Ｒ０）１６３の出力とレジスタ（Ｒ１）１６４の出力を加算してレジスタ（Ｒ２）１６６に出力する。加算器１６７は、レジスタ（Ｒ２）１６６の出力と、自身の１クロック周期前の出力を保持しているレジスタ（Ｒ３）１６８の出力を加算して、レジスタ（Ｒ３）１６８および除算器１６９に出力する。
【０１２８】
除算器１６９は、加算器１６７の出力（所定に期間における加算器１６５の主力の累算値）を補間係数の総和ΣＣｉで除算してレジスタ（Ｒ４）１７０に出力する。
【０１２９】
レジスタ（Ｒ０）１６３，レジスタ（Ｒ１）１６４，レジスタ（Ｒ２）１６６、およびレジスタ（Ｒ３）１６８は、上段からの入力を所定のクロック周期だけ遅延して出力する。レジスタ（Ｒ３）１６８は、RSR_R信号に対応してリセットする。保持している値を初期化する。レジスタ（Ｒ４）１７０は、EN信号に対応して保持している値を出力する。
【０１３０】
補間係数供給回路１７１，１７２は、補間点の水平方向の位置を示す４ビットの位置情報ｒ’に対応する補間係数Ｃｉを、それぞれ乗算器１６１，１６２に供給する。
【０１３１】
以上説明したように構成される水平方向比例分配回路１２３は、次式で示される水平方向の補間値Ｘを出力する。
補間値Ｘ＝Σ（Ｃｉ＊Ｔｉ）／ΣＣｉ
ここでｉは、４点補間処理の場合、ｉ＝０，１であり、１６点補間処理の場合、ｉ＝０，１，２，・・・，７である。
【０１３２】
図４５は、フィールド・フレーム変換されたＳＤフォーマットの映像信号を１６点補間処理するときの補間係数Ｃｉの値を示す。
【０１３３】
次に、補間回路２２の動作について説明する。始めに、ＨＤフォーマットの映像信号が、バッファ２０にバッファリングされている場合における４点補間処理について説明する。
【０１３４】
この場合、図２１に示したように、ＨＤフォーマットの映像信号は、フィールド画像単位で、かつ、上下左右に隣接する４画素がそれぞれ分かれてバッファ２０のユニットＵ０，Ｕ１，Ｌ０，Ｌ１に格納されているので、例えば、図４６（Ａ）に示すEVENフィールドの「×」印で示す補間点に対応する映像信号を補間するとき、補間点の上下左右の４画素の映像信号を１クロック周期で同時に読み出すことができる。
【０１３５】
ユニットＵ０，Ｕ１，Ｌ０，Ｌ１から１クロック周期で同時に読み出された映像信号（以下、それぞれを信号Ｕ０，Ｕ１，Ｌ０，Ｌ１と記述する）のうち、信号Ｕ０，Ｌ０は垂直方向比例分配回路１２１に供給され、信号Ｕ１，Ｌ１は垂直方向比例分配回路１２２に供給される。
【０１３６】
垂直方向比例分配回路１２１は、補間点の垂直方向の位置情報ｒに応じて信号Ｕ０，Ｌ０を比例分配し、得られた垂直方向の補間値ＴＡを水平方向比例分配回路１２３に出力する。垂直方向比例分配回路１２２は、補間点の垂直方向の位置情報ｒに応じて信号Ｕ１，Ｌ１を比例分配し、得られた垂直方向の補間値ＴＢを水平方向比例分配回路１２３に出力する。
【０１３７】
水平方向比例分配回路１２３は、補間点の水平方向の位置情報ｒ’に応じて、垂直方向の補間値ＴＡ，ＴＢを比例分配し、「×」印で示す補間点に対応する補間値を得る。
【０１３８】
なお、図４６（Ｂ）に示すODDフィールドの映像信号に対する動作も同様であるので、その説明は省略する。
【０１３９】
次に、フィールド・フレーム変換されたＳＤフォーマットの映像信号が、バッファ２０に格納されている場合における１６点補間処理について説明するが、その前に、格納されているＳＤフォーマットの映像信号について、４８０ｉ×７２０のＳＤフォーマットの映像信号を一例に図４７を参照して説明する。
【０１４０】
ＳＤフォーマットの映像信号は、バッファ２０に入力される前、スキャンコンバータ１５のコンバータ６７によって、EVENフィールドの画像（図４７（Ａ）に○で示される画素から成る）と、ODDフィールドの画像（図４７（Ａ）に□で示される画素から成る）が合成されて、図４７（Ａ）に示すような４８０×７２０のフレーム画像に変換されており、さらに、上下に隣接する○で示される画素と□で示される画素から、その２画素の間に位置する画素（図４７（Ｂ）に△で示される画素）が補間されて９６０×７２０のフレーム画像に変換されている。
【０１４１】
９６０×７２０のフレーム画像に変換されたＳＤフォーマットの映像信号は、図４８に示すように、すなわち、ＨＤフォーマットのフィールド画像がバッファ２０に格納される状態（図２１）と同様に、上下左右に隣接する４画素のそれぞれがバッファ２０のユニットＵ０，Ｕ１，Ｌ０，Ｌ１に分かれて格納される。
【０１４２】
補間回路２２の１６点補間処理の動作の概念について説明する。例えば、図４９の「×」印で示す補間点に対応する映像信号を補間する場合、補間点「×」の上段の隣接した８画素と、上段の隣接した８画素から成る１６画素（同図に示した水平方向に長い長方形で囲まれた１６画素）の映像信号が読み出されて比例分配が行われる。
【０１４３】
具体的には、図５０に示すように、上段の８画素の映像信号と、それらにそれぞれに対応する下段の８画素の映像信号が垂直方向比例分配回路１２１，１２２によってそれぞれ比例分配され、垂直方向の補間値Ｔ０乃至Ｔ７が算出される。垂直方向の補間値Ｔ０乃至Ｔ７は、水平方向比例分配回路１２３によって、図５１に示すように、補間値Ｔ０乃至Ｔ７に補間係数Ｃ０乃至Ｃ７がそれぞれ乗算され、その総和Σ（Ｔｉ＊Ｃｉ）が補間係数Ｃｉの総和ΣＣｉで除算されて、補間点「×」の補間値が算出される。ただし、いまの場合、ｉ＝０，１，２，・・・，７である。
【０１４４】
次に、動作周波数５４MHzで実行される補間回路２２の１６点補間処理の動作タイミングについて、図５２乃至図５４を参照して説明する。上述したように、バッファ２０のユニットＵ０，Ｕ１，Ｌ０，Ｌ１は同時読み出しが可能であるので、１６点補間処理においては、１クロック周期毎に４画素ずつ順次読み出される。
【０１４５】
すなわち、図５２（Ａ）に示すように、バッファ２０のユニットＵ０，Ｕ１，Ｌ０，Ｌ１に分かれて格納されている補間に用いる１６画素の映像信号をａ０乃至ａ１５と記述することにすれば、第０番目のタイミング(cycle0)において、同図（Ｂ）に示す映像信号ａ０，ａ１，ａ８，ａ９が読み出され、第２番目のタイミング(cycle1)において、同図（Ｃ）に示す映像信号ａ２，ａ３，ａ１０，ａ１１が読み出され、第３番目のタイミング(cycle2)において、同図（Ｄ）に示す映像信号ａ４，ａ５，ａ１２，ａ１３が読み出され、第４番目のタイミング(cycle3)において、同図（Ｅ）に示す映像信号ａ６，ａ７，ａ１４，ａ１５が読み出される。
【０１４６】
第０番目のタイミング(cycle0)において読み出された映像信号ａ０，ａ８は、垂直方向比例分配回路１２１に入力され、映像信号ａ１，ａ９は、垂直方向比例分配回路１２２に入力される。第１番目のタイミング(cycle1)において読み出された映像信号ａ２，ａ１０は、垂直方向比例分配回路１２１に入力され、映像信号ａ３，ａ１１は、垂直方向比例分配回路１２２に入力される。第２番目のタイミング(cycle2)において読み出された映像信号ａ４，ａ１２は、垂直方向比例分配回路１２１に入力され、映像信号ａ５，ａ１３は、垂直方向比例分配回路１２２に入力される。さらに、第３番目のタイミング(cycle3)において読み出された映像信号ａ６，ａ１４は、垂直方向比例分配回路１２１に入力され、映像信号ａ７，ａ１５は、垂直方向比例分配回路１２２に入力される。
【０１４７】
図５３（Ａ），（Ｂ）は、それぞれ垂直方向比例分配回路１２１，１２２の動作タイミングを示している。垂直方向比例分配回路１２１は、バッファ２０のユニットＵ０，Ｌ０からの入力タイミングに４クロック周期だけ遅延したタイミングで、順次、垂直方向の補間値ＴＡを水平方向比例分配回路１２３に出力する。
【０１４８】
具体的には、第５番目のタイミング(cycle5)において、映像信号ａ０，ａ８を比例分配した補間値Ｔａ０を出力し、第６番目のタイミング(cycle6)において、映像信号ａ２，ａ１０を比例分配した補間値Ｔａ２を出力し、第７番目のタイミング(cycle7)において、映像信号ａ４，ａ１２を比例分配した補間値Ｔａ４を出力し、さらに、第８番目のタイミング(cycle8)において、映像信号ａ６，ａ１４を比例分配した補間値Ｔａ６を出力する。
【０１４９】
同様に、垂直方向比例分配回路１２２は、バッファ２０のユニットＵ１，Ｌ１からの入力タイミングに４クロック周期だけ遅延したタイミングで、順次、垂直方向の補間値ＴＢを水平方向比例分配回路１２３に出力する。
【０１５０】
具体的には、第５番目のタイミング(cycle5)において、映像信号ａ１，ａ９を比例分配した補間値Ｔａ１を出力し、第６番目のタイミング(cycle6)において、映像信号ａ３，ａ１１を比例分配した補間値Ｔａ３を出力し、第７番目のタイミング(cycle7)において、映像信号ａ５，ａ１３を比例分配した補間値Ｔａ５を出力し、さらに、第８番目のタイミング(cycle8)において、映像信号ａ７，ａ１５を比例分配した補間値Ｔａ７を出力する。
【０１５１】
図５４は、水平方向比例分配回路１２３の動作タイミングを示している。垂直方向比例分配回路１２３は、４クロック周期毎に補間値Ｘを出力する。
【０１５２】
具体的には、乗算器１６１は、第５乃至８番目のタイミングで順次入力された垂直方向の補間値Ｔａ０，Ｔａ２，Ｔａ４，Ｔａ６に、それぞれ補間係数Ｃ０，Ｃ２，Ｃ４，Ｃ６を乗算してレジスタ（Ｒ０）１６３に出力する。レジスタ（Ｒ０）１６３は、入力タイミングから３クロック周期だけ遅延した第８乃至１１番目のタイミングで、乗算値Ｔａ０＊Ｃ０，Ｔａ２＊Ｃ２，Ｔａ４＊Ｃ４，Ｔａ６＊Ｃ６を加算器１６５に順次出力する。
【０１５３】
同様に、乗算器１６２は、第５乃至８番目のタイミングで順次入力された垂直方向の補間値Ｔａ１，Ｔａ３，Ｔａ５，Ｔａ７に、それぞれ補間係数Ｃ１，Ｃ３，Ｃ５，Ｃ７を乗算してレジスタ（Ｒ１）１６４に出力する。レジスタ（Ｒ１）１６４は、入力タイミングから３クロック周期だけ遅延した第８乃至１１番目のタイミングで、乗算値Ｔａ１＊Ｃ１，Ｔａ３＊Ｃ３，Ｔａ５＊Ｃ５，Ｔａ７＊Ｃ７を加算器１６５に順次出力する。
【０１５４】
加算器１６５は、第８乃至１１番目のタイミングで乗算器１６１，１６２のそれぞれから順次入力された乗算値Ｔａ０＊Ｃ０と乗算値Ｔａ１＊Ｃ１、乗算値Ｔａ２＊Ｃ２と乗算値Ｔａ３＊Ｃ３、乗算値Ｔａ４＊Ｃ４と乗算値Ｔａ５＊Ｃ５、乗算値Ｔａ６＊Ｃ６と乗算値Ｔａ７＊Ｃ７を、それぞれ加算して、レジスタ（Ｒ２）１６６に出力する。レジスタ（Ｒ２）１６６は、入力タイミングから１クロック周期だけ遅延した第９乃至１２番目のタイミングで、加算値Ｔａ０＊Ｃ０＋Ｔａ１＊Ｃ１，Ｔａ２＊Ｃ２＋Ｔａ３＊Ｃ３，Ｔａ４＊Ｃ４＋Ｔａ５＊Ｃ５，Ｔａ６＊Ｃ６＋Ｔａ７＊Ｃ７を、加算器１６７に順次出力する。
【０１５５】
加算器１６７は、第９乃至１２番目のタイミングで加算器１６５から順次入力された加算値と、レジスタ（Ｒ３）１６８から入力される１クロック周期前の加算器１６７の出力を加算してレジスタ（Ｒ３）１６８および除算器１６９に出力する。なお、レジスタ（Ｒ３）１６８は、４クロック周期毎に入力されるRST_R信号に同期して初期化される。よって、第１３番目のタイミングにおいて、加算器１６７は、第９乃至１２番目のタイミングで加算器１６５から順次入力された加算値の累計値を出力することになる。
【０１５６】
除算器１６９は、加算器１６７からの累算値を補間係数の総和ΣＣｉで除算してレジスタ（Ｒ４）１７０に出力する。レジスタ（Ｒ４）１７０は、４クロック周期毎（いまの場合、第１３番目のタイミング）に入力されるＥＮ信号に同期して、乗算器１６９からの除算値、すなわち補間点「×」の補間値Ｘを出力する。
【０１５７】
以上説明したように、本実施の形態においては、ＨＤフォーマットの映像信号とＳＤフォーマットの映像信号を同一の回路、すなわち、補間回路２２によって補間することができる。なお、ＨＤフォーマットの映像信号に対しては４点補間処理を実行し、ＳＤフォーマットの映像信号に対しては、４点補間処理ではなく、１６点補間処理を実行するようにしたので、従来型ＳＤ専用機器と同等品質の補間値を得ることができる。
【０１５８】
なお、本発明は、映像信号を処理するあらゆる機器に適用することが可能である。
【０１５９】
ところで、上述した一連の処理は、ハードウェアにより実行させることもできるが、ソフトウェアにより実行させることもできる。一連の処理をソフトウェアにより実行させる場合には、そのソフトウェアを構成するプログラムが、専用のハードウェアに組み込まれているコンピュータ、または、各種のプログラムをインストールすることで、各種の機能を実行することが可能な、例えば汎用のパーソナルコンピュータなどに、記録媒体からインストールされる。
【０１６０】
この記録媒体は、図１に示すように、コンピュータとは別に、ユーザにプログラムを提供するために配布される、プログラムが記録されている磁気ディスク６（フロッピディスクを含む）、光ディスク７（CD-ROM(Compact Disc-Read Only Memory)、DVD(Digital Versatile Disc)を含む）、光磁気ディスク８（ＭＤ(Mini Disc)を含む）、もしくは半導体メモリ９などよりなるパッケージメディアにより構成されるだけでなく、コンピュータに予め組み込まれた状態でユーザに提供される、プログラムが記録されているROMやハードディスクなどで構成される。
【０１６１】
なお、本明細書において、記録媒体に記録されるプログラムを記述するステップは、記載された順序に従って時系列的に行われる処理はもちろん、必ずしも時系列的に処理されなくとも、並列的あるいは個別に実行される処理をも含むものである。
【０１６２】
また、本明細書において、システムとは、複数の装置により構成される装置全体を表すものである。
【０１６３】
【発明の効果】
以上のように、本発明の画像処理装置および方法、並びに記録媒体のプログラムによれば、入力された画像に対応する映像信号をメモリに記憶させ、変形後の画像が重畳されるフレームの４端点のそれぞれに対応するパラメータを算出し、算出した４端点のそれぞれに対応するパラメータのうち、垂直方向に対向する２端点のそれぞれに対応するパラメータを用いて、２組の２端点のそれぞれの間の水平方向に対向する２点のそれぞれに対応するパラメータを補間し、補間した水平方向に対向する２点のそれぞれに対応するパラメータを用いて、変形後の画像上の点に対応するパラメータを補間し、補間した変形後の画像上の点に対応するパラメータに基づいて、メモリの読み出しアドレスを生成するが、垂直方向の補間処理と水平方向の補間処理を、共通の回路により実行するようにしたので、省スペース化および低コスト化を実現することが可能となる。
【図面の簡単な説明】
【図１】本発明を適用した画像合成装置の構成例を示すブロック図である。
【図２】 DME３の構成例を示すブロック図である。
【図３】Ｈフィルタ１４における色差信号Ｕ，Ｖを補間する処理の概念を説明するための図である。
【図４】Ｈフィルタ１４の色差信号Ｕ，Ｖを補間する処理に関わる部分の構成例を示すブロック図である。
【図５】図４の４点補間回路４０の構成例を示すブロック図である。
【図６】スキャンコンバータ１５の構成例を示すブロック図である。
【図７】スキャンコンバータ１５のフィールド単位で走査方向を垂直に変換する処理の大まかな時間推移を示す図である。
【図８】フィールド単位で記録した映像信号を垂直走査の順序でバースト(burst)転送するSDRAM６４−１，６４−２と、バースト転送された映像信号をキャッシュするSRAM６５との関係を模式的に示す図である。
【図９】 SDRAM６４の２種類のバンクに対する交互バーストでの連続アクセスのタイミングの一例を示す図である。
【図１０】 SDRAM６４への連続アクセス（書き込み）を説明するための図である。
【図１１】 SDRAM６４への連続アクセス（読み出し）を説明するための図である。
【図１２】 SDRAM６４に対する、ＨＤフォーマット（１０８０ｉ×１９２０）の映像信号の２バンク４ワードバーストにおけるアドレスの２次元割り当ての一例を示す図である。
【図１３】 SDRAM６４に対する書き込みアドレスを生成するカウンタ機構を説明するための図である。
【図１４】 SDRAM６４からの連続読み出しの順序を示す図である。
【図１５】 SDRAM６４に対する読み出しアドレスを生成するカウンタ機構を説明するための図である。
【図１６】 SRAM６５を構成するメモリをリングとして用いる概念を示する図である。
【図１７】 SRAM６５を構成する４個のメモリを４重のリングとして用いる概念を示する図である。
【図１８】コンバータ６７が色差信号Ｕ，Ｖの値を８ビットに削減して映像信号（Ｙ／Ｕ／Ｖ／Ｋ）を３６ビット幅に変換する処理を説明するための図である。
【図１９】バッファ２０の構成例を示すブロック図である。
【図２０】バッファ２０のユニットＵ０の構成例を示すブロック図である。
【図２１】スキャンコンバータ１５から入力されるＨＤフォーマットの映像信号のユニットＵ０乃至Ｌ１に対する割り付けを示す図である。
【図２２】バッファ２０に設定されるリードアドレスの座標系を示す図である。
【図２３】バッファ２０のデータ領域にEVENフィールドの映像信号が書き込まれている状態を示す図である。
【図２４】４点補間処理に用いる４個の画素の位置を示す図である。
【図２５】４点補間処理に用いる４個の画素が存在しない例を示す図である。
【図２６】バッファ２０の有効アクセス領域に設ける領域外データバンドを示す図である。
【図２７】バッファ２０のデータ領域にEVENフィールドの映像信号が書き込まれ、その周囲に領域外データバンドが設定されている状態を示す図である。
【図２８】バッファ２０に領域外データバンドが設定されたことによって４点補間処理が可能となることを説明するための図である。
【図２９】スクリーンアドレスとリードアドレスとの関係を説明するための図である。
【図３０】アドレスジェネレータ２１によるスーパインタポレーションを説明するための図である。
【図３１】スーパインタポレーションの処理タイミングを説明するための図である。
【図３２】アドレスジェネレータ２１の構成例を示すブロック図である。
【図３３】スーパインタポレーションブロック９３の構成例を示すブロック図である。
【図３４】 REG_V_START_XLレジスタ１０１−Ｘ乃至REG_V_END_ZRレジスタ１０６−Ｚに保持させる関数値X(0,0)乃至Z(1919,539)を示す図である。
【図３５】ミキサ係数ブロック９２に内蔵されているレジスタと、そこに保持されているミキサ係数との対応関係を示す図である。
【図３６】ミキサ１１１−Ｘ乃至１１１−Ｚへの関数値の入力元および出力先を示す図である。
【図３７】図３６に対応する関数値X(H,V)を補間するブロックの状態を示す図である。
【図３８】ミキサ１１１−Ｘ乃至１１１−Ｚへの関数値の入力元および出力先を示す図である。
【図３９】図３８に対応する関数値X(H,V)を補間するブロックの状態を示す図である。
【図４０】ミキサ１１１−Ｘ乃至１１１−Ｚへの関数値の入力元および出力先を示す図である。
【図４１】図４０に対応する関数値X(H,V)を補間するブロックの状態を示す図である。
【図４２】補間回路４２の構成例を示すブロック図である。
【図４３】垂直方向比例分配回路１２１の構成例を示すブロック図である。
【図４４】水平方向比例分配回路１２３の構成例を示すブロック図である。
【図４５】１６点補間処理に用いる補間係数Ｃ０乃至Ｃ７の値を示す図である。
【図４６】ＨＤフォーマットの映像信号に対する４点補間処理を説明するための図である。
【図４７】スキャンコンバータ１５のコンバータ６７による、ＳＤフォーマットの映像信号に対するフィールド・フレーム変換を説明するための図である。
【図４８】スキャンコンバータ１５から入力されるフィールド・フレーム変換されたＳＤフォーマットの映像信号のユニットＵ０乃至Ｌ１に対する割り付けを示す図である。
【図４９】ＳＤフォーマットの映像信号に対する１６点補間処理を説明するための図である。
【図５０】１６点補間処理における垂直方向比例分配回路１２１，１２２の動作を説明するための図である。
【図５１】１６点補間処理における水平方向比例分配回路１２３の動作を説明するための図である。
【図５２】１６点補間処理における映像信号の読み出しタイミングを説明するための図である。
【図５３】１６点補間処理における垂直方向比例分配回路１２１，１２２の動作タイミングを説明するための図である。
【図５４】１６点補間処理における水平方向比例分配回路１２３の動作タイミングを説明するための図である。
【符号の説明】
１ レバーアーム， ２ 制御回路， ３ DME， ４ 合成回路， ５ ドライブ， ６ 磁気ディスク， ７ 光ディスク， ８ 光磁気ディスク， ９半導体メモリ， １１ HDFF， １２乃至１４ Ｈフィルタ， １５ スキャンコンバータ， １６ VDFF ， １７乃至１９ Ｖフィルタ， ２０ バッファ， ２１ アドレスジェネレータ， ２２ 補間回路， ６４ SDRAM， ６５ SRAM， ６７ コンバータ， ７３ SRAM， ９３ スーパインタポレーションブロック， １２１，１２２ 垂直方向比例分配回路， １２３ 水平方向比例分配回路[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus and method, and a recording medium. For example, the present invention relates to an image processing apparatus and method, and a recording medium that are suitable for use in arbitrarily changing the shape of an image for display.
[0002]
[Prior art]
The advent of digital storage has greatly contributed to the technical progress of television program production techniques. DRAM (Dinamic Random Access Memory) in digital storage has a recording capacity of one scanning line, one field image, one frame image, and a series of multiple images. It has been gradually increased. Further, even in consideration of the manufacturing cost, circuit scale, power consumption, etc., it is economically practical.
[0003]
As an application example of digital storage such as DRAM, there is so-called DME (Digital Multi Effects) which is used when an image is deformed or moved to an arbitrary shape when a television program is produced.
[0004]
In a DME or the like, a video signal is temporarily stored in an image memory such as a DRAM, and a deformed image is output by reading out the stored video signal in accordance with a read address corresponding to the shape of the deformed image. Can do.
[0005]
Note that the read address needs to be generated in units of pixels corresponding to the shape of the image after deformation, but usually corresponds to the four end points (4 pixels) of the image after deformation in order to reduce the amount of calculation. After the calculation for generating the read address is executed, the calculation result for generating the read address corresponding to the position other than the four end points is interpolated using the calculation result.
[0006]
[Problems to be solved by the invention]
By the way, this interpolation needs to be executed independently in the vertical and horizontal directions. However, both a vertical direction interpolation circuit and a horizontal direction interpolation circuit are provided, or a dedicated CPU or the like is provided for executing vertical direction interpolation processing and horizontal direction interpolation processing by software. This is not efficient, and there is a problem that the scale of the entire apparatus becomes large.
[0007]
The present invention has been made in view of such a situation. By performing a vertical interpolation process and a horizontal interpolation process when generating a read address with a common circuit, space saving and low cost are achieved. The purpose is to enable realization.
[0008]
[Means for Solving the Problems]
The image processing apparatus according to the present invention includes a storage unit that stores a video signal corresponding to an input image in a memory, a designation unit that designates a deformed shape of the image, and four frames on which the deformed image is superimposed. The calculation means for calculating the parameter corresponding to each of the end points, and among the parameters corresponding to each of the four end points calculated by the calculation means, two sets of parameters are used, corresponding to each of the two end points opposed in the vertical direction. The vertical direction interpolation means for interpolating the parameters corresponding to each of the two points facing each other in the horizontal direction between the two end points, and the parameters corresponding to each of the two points facing each other in the horizontal direction interpolated by the vertical direction interpolation means. The horizontal direction interpolation means for interpolating the parameters corresponding to the points on the transformed image, and the points on the transformed image interpolated by the horizontal direction interpolation means A generation unit that generates a read address of the memory based on the parameters, and a read unit that reads out a video signal from the memory according to the read address generated by the generation unit, and the vertical direction interpolation unit and the horizontal direction interpolation unit have a common circuit Consists of
[0009]
The vertical direction interpolation means can be operated during a horizontal blanking period, and the horizontal direction interpolation means can be operated during a horizontal scanning period.
[0010]
The image processing method of the present invention includes a storage step for storing a video signal corresponding to an input image in a memory, a designation step for designating a deformed shape of the image, and four frames on which the deformed image is superimposed. A calculation step for calculating a parameter corresponding to each of the end points, and a parameter corresponding to each of the two end points opposed in the vertical direction among the parameters corresponding to each of the four end points calculated in the processing of the calculation step, A vertical interpolation step for interpolating parameters corresponding to two horizontally opposed points between each of two sets of two end points, and two horizontally opposed pixels interpolated in the vertical interpolation step processing A horizontal interpolation step for interpolating the parameters corresponding to the points on the transformed image using the corresponding parameters, and the horizontal interpolation Based on the parameters corresponding to the points on the transformed image interpolated in the step processing, a generation step for generating a read address of the memory, and a video signal is read from the memory according to the read address generated in the processing of the generation step A vertical interpolation step processing and a horizontal interpolation step processing are executed by a common circuit.
[0011]
The recording medium program of the present invention includes a storage step for storing a video signal corresponding to an input image in a memory, a designation step for designating a deformed shape of the image, and a frame on which the deformed image is superimposed. A calculation step for calculating a parameter corresponding to each of the four end points, and a parameter corresponding to each of the two end points opposed in the vertical direction among the parameters corresponding to each of the four end points calculated in the processing of the calculation step. A vertical interpolation step for interpolating parameters corresponding to two horizontally opposed points between each of two sets of two end points, and two points opposed to the horizontal direction interpolated in the vertical interpolation step processing A horizontal interpolation step for interpolating parameters corresponding to points on the transformed image using parameters corresponding to A generation step for generating a read address of the memory based on parameters corresponding to points on the transformed image interpolated by the direction interpolation step processing, and a video signal from the memory according to the read address generated by the generation step processing Read out, and the vertical interpolation step processing and the horizontal interpolation step processing are executed by a common circuit.
[0012]
In the image processing apparatus and method and the recording medium program of the present invention, the video signal corresponding to the input image is stored in the memory, and the deformed shape of the image is designated. Further, parameters corresponding to each of the four end points of the frame on which the deformed image is superimposed are calculated, and among the calculated parameters corresponding to each of the four end points, corresponding to each of the two end points facing in the vertical direction. Parameters are used to interpolate the parameters corresponding to each of the two horizontally opposed points between each of the two sets of two end points, and the parameters corresponding to each of the two horizontally opposed points interpolated are Used to interpolate parameters corresponding to the points on the transformed image, and to generate a memory read address based on the interpolated parameters on the transformed image point, and to generate the read address Accordingly, the video signal is read from the memory. The vertical interpolation process and the horizontal interpolation process are executed by a common circuit.
[0013]
DETAILED DESCRIPTION OF THE INVENTION
An image composition apparatus to which the present invention is applied will be described with reference to FIG. FIG. 1 is a block diagram illustrating a configuration example of an image composition apparatus. This image composition device is used, for example, in the production of a television broadcast program, and synthesizes a video input A image and a video input B image that have undergone digital image processing such as deformation and movement. Output.
[0014]
The image synthesizing apparatus detects a user's operation and outputs a corresponding operation signal to the control circuit 2 and controls the drive 5 to control a magnetic disk 6 (including a floppy disk), an optical disk 7 (CD-ROM ( The control program stored in the compact disc-read only memory (DVD) (including digital versatile disc), magneto-optical disk 8 (including MD (mini disc)), or semiconductor memory 9 is read and read. A control circuit 2 for controlling the entire image synthesizing apparatus based on an output control program, an operation signal from the lever arm 1, etc., a DME 3 for performing digital image processing on the video input A and outputting it to the synthesizing circuit 4; The video input A video is superimposed on the video input A video subjected to digital image processing and output to the subsequent stage.
[0015]
Next, the operation of the image composition device will be described. The video of the video input A is subjected to digital image processing corresponding to the user's operation on the lever arm 1 by the DME 3 and is superimposed on the video of the video input B by the synthesis circuit 4 and output.
[0016]
FIG. 2 shows a configuration example of the DME 3. The video input A input to the DME 3 is a 4: 2: 2: 4 (Y / U / V / K) HD format (eg, 1080i × 1920) video signal (30 bits wide), that is, 10 bits wide Image signal composed of a luminance signal Y, color difference signals U and V each having a width of 5 bits, and a key signal K having a width of 10 bits. As the video input A, an HD (High Definition) format video signal, an SD (Standered Definition) format video signal (for example, 480i × 720), and other format video signals can be input. It is.
[0017]
In FIG. 2, the luminance signal Y is displayed as a signal Y, the 5-bit wide color difference signals U and V are integrated and displayed as a 10-bit wide C signal, and the key signal K is displayed as a K signal. ing.
[0018]
A horizontal defocus filter (hereinafter referred to as HDFF (Horizontal Defocus Filter)) 11 of the DME 3 is an H filter 12 that performs one-dimensional low-pass filter processing on the luminance signal Y input in the horizontal scanning order, and in the horizontal scanning order. After interpolating the H filter 13 that performs one-dimensional low-pass filter processing on the input key signal K and the color difference signals U and V input in the order of horizontal scanning (see FIGS. 3 to 6 for details) It is configured by an H filter 14 that performs one-dimensional low-pass filter processing (described later). Therefore, the 4: 4: 4: 4 video signal (40-bit width) is supplied to the scan converter 15 subsequent to the HDFF 11.
[0019]
The scan converter 15 holds video signals input from the HDFF 11 in the order of horizontal scanning, scans the held video signals in the vertical direction, that is, converts the scanning direction from the horizontal direction to the vertical direction, and The image is output to a focus filter (hereinafter referred to as VDFF (Vertical Defocus Filter)) 16.
[0020]
Further, the scan converter 15 reduces the bit width of the color difference signals U and V in the video signal (40-bit width) returned from the VDFF 16 in the order of the vertical scanning and outputs the reduced signal to the buffer 20.
[0021]
Further, when the video signal returned from the VDFF 16 is in the SD format, the scan converter 15 converts the field image into a frame image, interpolates, and outputs the frame image to the buffer 20. When the video signal input from the VDFF 16 is in the HD format, it is output to the buffer 20 in the state of a field image.
[0022]
The VDFF 16 is a V filter 17 that performs one-dimensional low-pass filter processing on the luminance signal Y input in the vertical scanning order, a V filter 18 that performs one-dimensional low-pass filtering processing on the key signal K input in the vertical scanning order, and The color difference signals U and V inputted in the order of vertical scanning are constituted by a V filter 19 which applies a one-dimensional low-pass filter process. The VDFF 16 returns the 4: 4: 4: 4 video signal (40-bit width) subjected to the one-dimensional low-pass filter processing in the vertical direction to the scan converter 15.
[0023]
The buffer 20 is composed of ZBT SRAM (Zero Bus Turnaround Static Random Access Memory), writes the video signal supplied from the scan converter 15 in accordance with the write address supplied from the address generator 21, and the address generator 21. Are read out in units of four pixels in accordance with the read address (read adrs) supplied from, and output to the interpolation circuit 22.
[0024]
The interpolation circuit 22 uses the video signal input from the buffer 20 in units of 4 pixels, interpolates the video signal corresponding to a predetermined position inside the 4 pixels, and outputs it to the synthesis circuit 4.
[0025]
The address generator 15 and the interpolation circuit 22 may be configured by an FPGA (Field Programmable Gate Array).
[0026]
Next, the color difference signals U and V of the 4: 2: 2: 4 video signal input to the HDFF 11 are interpolated by the H filter 14 and converted to a 4: 4: 4: 4 video signal. This will be described with reference to FIGS. In the following description, the notation regarding the key signal K is omitted, and it is also described as a 4: 2: 2 (Y / U / V) video signal or a 4: 4: 4 video signal.
[0027]
FIG. 3 shows a concept of processing for interpolating the color difference signals U and V.
[0028]
FIG. 3A shows a video signal input to the HDFF 11 in the order of horizontal scanning. That is, the HDFF 11 has a luminance signal Y corresponding to a certain pixel (the 0th pixel).₀And the color difference signal U corresponding to the 0th pixel₀Are input at the same time. In the next clock, the luminance signal Y corresponding to the 0.5th pixel located to the right of 0th_0.5And the color difference signal V corresponding to the 0th pixel.₀Are input at the same time. In the next clock, the luminance signal Y corresponding to the first pixel located to the right of the 0.5th₁And the color difference signal U corresponding to the first pixel₁Are input at the same time. Similarly, the luminance signal Y corresponding to the Nth pixel_NAnd the color difference signal U corresponding to the Nth pixel_NAt the same time, and in the next clock, the luminance signal Y corresponding to the (N + 0.5) th pixel located on the right of the Nth pixel_{N + 0.5}And the color difference signal V corresponding to the Nth pixel._NAre input at the same time.
[0029]
As shown in FIG. 3A, the luminance signal Y corresponding to the Nth pixel._NAnd color difference signal U_N, V_NAre not simultaneously input, and the color difference signal U corresponding to the (N + 0.5) th pixel_{N + 0.5}, V_{N + 0.5}Does not exist. Therefore, the luminance signal Y_NAnd color difference signal U_N, V_NAnd the color difference signal U corresponding to the (N + 0.5) th pixel in order to unify the spatial frequency of the luminance signal and the color difference signal._{N + 0.5}, V_{N + 0.5}Is interpolated.
[0030]
Color difference signal U corresponding to the (N + 0.5) th pixel_{N + 0.5}3B, as shown in FIG. 3B, the color difference signals of the pixels adjacent to the left and right, that is, the color difference signal U corresponding to the (N-1) th pixel._N-1, The color difference signal U corresponding to the Nth pixel_N, The color difference signal U corresponding to the (N + 1) th pixel_{N + 1}, And the color difference signal U corresponding to the (N + 2) th pixel_{N + 2}Interpolate using.
[0031]
Similarly, the color difference signal V corresponding to the (N + 0.5) th pixel_{N + 0.5}3, as shown in FIG. 3C, the color difference signals of the pixels adjacent to the left and right, that is, the color difference signal V corresponding to the (N−1) th pixel._N-1, The color difference signal V corresponding to the Nth pixel_N, The color difference signal V corresponding to the (N + 1) th pixel_{N + 1}, And the color difference signal V corresponding to the (N + 2) th pixel_{N + 2}Interpolate using.
[0032]
Next, FIG. 4 shows a configuration example of a portion related to the process of interpolating the color difference signals U and V of the H filter 14.
[0033]
The selector 31 outputs the color difference signals U and V sequentially input from all stages to the delay circuit (D) 32 and the four-point interpolation circuit 40 in synchronization with the clock. The delay circuits 32 to 37 and 42 delay the color difference signal input from the previous stage by one clock cycle and output it. The delay circuits 33, 35, and 37 also output the color difference signal delayed by one clock cycle to the four-point interpolation circuit 40.
[0034]
When the control signal S from the D flip-flop 41 is 0, the selector 39 outputs the output of the delay circuit 38 input to its own DA terminal from the QA terminal to the delay circuit 42 and also to its own DB terminal. The output of the input 4-point interpolation circuit 40 is output from the QB terminal. On the other hand, when the control signal S from the D flip-flop 41 is 1, the output of the delay circuit 38 input to its own DA terminal is output from the QB terminal and is input to its own DB terminal. The output of the interpolation circuit 40 is output from the QA terminal to the delay circuit 42. The selector 39 receives the color difference signal U as shown in FIG._NAnd color difference signal V_N-0.5Are output at the same time.
[0035]
The 4-point interpolation circuit 40 interpolates the color difference signal by pipeline processing that requires 4 clock cycles using the following equation.
Interpolated color difference signal value
= (T₀* C₀+ T₁* C₁+ T₂* C₂+ T_Three* C_Three) / (C₀+ C₁+ C₂+ C_Three)
[0036]
Where t₀Is the color difference signal U corresponding to the (N−1) th pixel inputted from the delay circuit 37._N-1(Or V_N-1) And t₁Is the color difference signal U corresponding to the Nth pixel input from the delay circuit 35._N(Or V_N) And t₂Is the color difference signal U corresponding to the (N + 1) th pixel input from the delay circuit 33._{N + 1}(Or V_{N + 1}) And t_ThreeIs the color difference signal U corresponding to the (N + 2) th pixel input from the selector 31._{N + 2}(Or V_{N + 2}) Value. Here, the interpolation coefficient C₀Thru C_FourAre −163, 1187, 1187, and −163, respectively.
[0037]
The D flip-flop 41 alternately outputs 0 and 1 as the control signal S to the selector 39 every clock.
[0038]
FIG. 5 shows a configuration example of the four-point interpolation circuit 40. The four-point interpolation circuit 40 includes multipliers 51 to 54 and adders 55 to 57.
[0039]
The multiplier 51 receives the color difference signal U corresponding to the (N−1) th pixel input from the selector 31._N-1(Or V_N-1) Value t₀Interpolation coefficient C₀Multiplication value t multiplied by₀* C₀Is output to the adder 55. The multiplier 52 receives the color difference signal U corresponding to the Nth pixel input from the delay circuit 33._N(Or V_N) Value t₁Interpolation coefficient C₁Multiplication value t multiplied by₁* C₁Is output to the adder 55. The multiplier 53 receives the color difference signal U corresponding to the (N + 1) th pixel input from the delay circuit 35._{N + 1}(Or V_{N + 1}) Value t₂Interpolation coefficient C₂Multiplication value t multiplied by₂* C₂Is output to the calculator 56. The multiplier 54 receives the color difference signal U corresponding to the (N + 2) th pixel input from the delay circuit 37._{N + 2}(Or V_{N + 2}) Value t_ThreeInterpolation coefficient C_ThreeMultiplication value t multiplied by_Three* C_ThreeIs output to the calculator 56.
[0040]
The adder 55 is a multiplication value t from the multiplier 51.₀* C₀And the multiplication value t from the multiplier 52₁* C₁And the addition value t₀* C₀+ T₁* C₁Is output to the adder 57. The adder 56 is a multiplication value t from the multiplier 53.₂* C₂And the multiplication value t from the multiplier 54_Three* C_ThreeAnd the addition value t₂* C₂+ T_Three* C_ThreeIs output to the adder 57. The adder 57 adds the added value t from the adder 55.₀* C₀+ T₁* C₁And the addition value t from the adder 56₂* C₂+ T_Three* C_ThreeAnd the interpolation coefficient C₀Thru C_ThreeAnd the color difference signal U corresponding to the (N + 0.5) th pixel._{N + 0.5}(Or V_{N + 0.5}) Value.
[0041]
Next, the operation of the functional block related to the process of interpolating the color difference signals U and V will be described.
[0042]
For example, as shown in FIG. 4, at the clock timing t0, the selector 31, the delay circuit 32 to the delay circuit 38 are respectively connected to the color difference signal V._Four, U_Four, V_Three, U_Three, V₂, U₂, V₁, U₁Is output to the subsequent stage, the color difference signal V is obtained by the four-point interpolation circuit 40 in the subsequent four clock cycles._2.5Is interpolated. Therefore, at the clock timing t4, the color difference signal U is sent from the delay circuit 38 to the DA terminal of the selector 39._ThreeThe color difference signal V interpolated by the 4-point interpolation circuit 40 is input to the DB terminal._2.5Is entered.
[0043]
At this time, the selector 39 corresponds to the control signal S = 0 from the D flip-flop 41, and the color difference signal U input to the DA terminal._ThreeFrom the QA terminal to the delay circuit 42 and the interpolated color difference signal V input to the DB terminal._2.5Is output from the QB terminal. In synchronism with this, from the delay circuit 42, the color difference signal U which has been interpolated and delayed by one clock before._2.5Is output. Therefore, from the H filter 14, as shown in FIG. 3E, the color difference signal U corresponding to the (N + 0.5) th pixel._{N + 0.5}And color difference signal V_{N + 0.5}Will be output at the same time.
[0044]
As described above, the HDFF 11 removes the high-frequency component of the 4: 2: 2 video signal and interpolates the color difference signal to convert it to a 4: 4: 4 video signal. The luminance signal Y and the color difference signals U and V can be handled at the same spatial frequency. Thereby, for example, as a process related to the color operation of the video, it is possible to perform a colored spotlight process, a trail process with a color change, and the like.
[0045]
Next, FIG. 6 shows a configuration example of the scan converter 15 that converts the scanning direction of the 4: 4: 4 video signal input from the HDFF 11 in the order of horizontal scanning from the horizontal direction to the vertical direction. The scan converter 15 includes a scan conversion IC 61 made of an FPGA or the like, and SRAMs (Synchronous Dynamic Random Access Memory) 64-1 and 64-2.
[0046]
The V scan generator 62 of the scan conversion IC 61 generates a signal indicating the corresponding vertical scanning timing based on the horizontal scanning timing indicated by the REF signal supplied from the outside, and outputs the signal to the SDRAM controller 63 and the SRAM controller 66.
[0047]
The SDRAM controller 63 switches the 4: 4: 4: 4 video signal input from the HDFF 11 in units of fields and records them in the SDRAMs 64-1 and 64-2. The SDRAM controller 63 also reads the video signals recorded in the SDRAMs 64-1 and 64-2 in a predetermined order (described later) and outputs them to an SRAM (Static Random Access Memory) 65.
[0048]
The SRAM 65 has a capacity of 2 bits * 2048 per block, and uses four memories of 20 blocks. Based on the control of the SRAM controller 66, the SRAM 65 caches a video signal input from the SDRAM controller 63. Output to VDFF16.
[0049]
The converter 67 reduces the information amount of the color difference signals U and V in the video signal input from the VDFF 16 in the vertical scanning order from 10-bit width to 8-bit width (see FIG. 18 for details). Will be described later). When the video signal input from the VDFF 16 is in the SD format, the converter 67 converts the field image into a frame image (details will be described later with reference to FIG. 47).
[0050]
FIG. 7 shows a rough time transition of the process of converting the scanning direction to vertical in field units. In the following figures, buffer A corresponds to one of SDRAMs 64-1 and 64-2, and buffer B corresponds to the other.
[0051]
FIG. 8 schematically shows the relationship between the SDRAMs 64-1 and 64-2 that burst-transfer video signals recorded in field units in the vertical scanning order, and the SRAM 65 that caches the burst-transferred video signals. ing. That is, the SRAM 65 operates as if moving horizontally from the left to the right of the image like a caterpillar while scanning the video signals recorded in the SDRAMs 64-1 and 64-2 in the vertical direction. Cache the signal. If the transfer amount per unit time input to the SDRAMs 64-1 and 64-2 in the horizontal scanning order is equal to the transfer amount per unit time output to the SRAM 65 in the vertical scanning order, the system is stable. Operation is guaranteed. That is, in the SDRAMs 64-1 and 64-2, a situation where the read address overtakes the write address does not occur.
[0052]
Note that the SDRAMs 64-1 and 64-2 (hereinafter simply referred to as the SDRAM 64 when there is no need to distinguish between the SDRAMs 64-1 and 64-2) have a plurality of (for example, two types) SDRAM characteristics. When burst transfer with an appropriate width (auto pre-charge 4 word burst) is alternately performed on a bank, continuous access (read or write) is guaranteed. FIG. 9 shows an example of the timing of successive access (write burst) in alternating bursts for two types of banks using such characteristics.
[0053]
Specifically, as shown in FIG. 10, the video signal written in the horizontal scanning order to the SDRAM 64 is burst-transferred in units of 8 words, that is, 4 words to each bank. As shown in FIG. 11, the video signal read from the SDRAM 64 in the order of vertical scanning is also burst-transferred in units of 8 words, that is, 4 words from each bank.
[0054]
Here, one word indicates a 40-bit information amount indicating a luminance signal Y (10 bits), color difference signals U and V (10 bits each), and a key signal K (10 bits) corresponding to one pixel. ing.
[0055]
As described above, the video signal cached by burst transfer to the SRAM 65 in units of 8 words (for two rectangular areas in FIG. 11) is read from the SRAM 65 after being delayed by a time corresponding to four vertical scans. For example, the SRAM 65 can have a minimum capacity (capacity corresponding to 0.4% of one field for two rectangular areas in FIG. 11).
[0056]
FIG. 12 shows an example of two-dimensional allocation of addresses in a 2-bank 4-word burst of an HD format (1080i × 1920) video signal to the SDRAM 64.
[0057]
As shown in the figure, at the time of writing, the burst head address is controlled so that writing is performed at an address in accordance with horizontal scanning. In this case, at the time of reading, it is accessed as a band having a burst size width.
[0058]
A write address for the SDRAM 64 is generated by a counter mechanism including an upper counter (ROW) and a lower counter (COLUMN) as shown in FIG.
[0059]
(1) Count-up is performed by a state machine that outputs a 4-word burst once every two times. The next count is performed during the bank access on the late side.
(2) Counter reload and 2K up are performed at 0x3c0. For each lower 0x3c0 count, 1k is added to the reload data.
(3) When the value of the upper counter reaches 540, one field is completed.
[0060]
FIG. 14 shows the order of continuous reading from the SDRAM 64. As shown in the figure, by using 4-word burst 2 bank pingpong access, the writing area to the SRAM 65 is a strip unit (rectangular area (4 words) consisting of 540 rectangles in FIG. 11). (n): n = 1, 2,..., 1df) and must be performed continuously.
[0061]
Generation of a read address from the SDRAM 64 is performed by a counter mechanism including an upper counter (ROW) and a lower counter (COLUMN) as shown in FIG.
[0062]
(1) The lower counter is reloaded every time. The reload value is incremented by 4 words when the value of the upper counter reaches 540.
(2) When the value of the lower counter reaches 0x3c0 and the value of the upper counter reaches 540, the process is terminated.
(3) Reloading of the upper counter and 2K up are performed every 4 word burst ping-pong.
(4) When the upper count reaches 540, the reloading of one strip unit (4 words * 540 lines) is completed.
[0063]
Video signals read from the SDRAM 64 in strip units are accessed after being cached in the SRAM 65. However, since the order of access is constant and complete synchronization, it is not an associative structure like a general-purpose cache, but complete synchronization predictive control.
[0064]
FIG. 16 shows a concept of using four memories (2 bits * 2048 * 20 blocks) constituting the SRAM 65 as a ring having a width of 40 bits (1 word) * 2048. In practice, as shown in FIG. 17A, four rings shown in FIG. 16 are stacked to form a caterpillar (FIG. 8B) having a width of 160 bits (4 words) * 2048.
[0065]
Specifically, the rectangular areas (4 words) read from the SDRAM 64 in the order shown in FIG. 15 are written sequentially into the rings 0 to 3 one by one as shown in FIG. 17B, for only three vertical scans. Delay and read out in the circumferential direction of the ring as shown in FIG.
[0066]
Through the operation as described above, the scanning direction of the video signal can be converted from the horizontal direction to the vertical direction in real time, and can be output to the VDFF 16 at the subsequent stage.
[0067]
As described above, the video signal input to the VDFF 16 is subjected to vertical one-dimensional low-pass filter processing, input to the scan converter 15 again, and supplied to the converter 67.
[0068]
Next, for the purpose of adapting to the buffer 20 in which eight 36-bit ZBT SRAMs in the subsequent stage are used, the 40-bit width of 4: 4: 4: 4 inputted from the VDFF 16 in the order of vertical scanning is used. Processing of the converter 67 that converts the video signal into a 36-bit video signal will be described.
[0069]
As shown in FIG. 18, the converter 67 is a video signal (10-bit luminance signal Y, 10-bit color difference signal U, 10-bit luminance signal Y, 4: 4: 4: 4) that is input from the VDFF 16 in the order of vertical scanning. Among the 10-bit color difference signal V and the 10-bit key signal K), the values of the color difference signals U and V that finally return the spatial frequency characteristics to ½ are obtained by, for example, rounding down or rounding off. The video signal (Y / U / V / K) is converted to a 36-bit width by reducing each to 8 bits and output to the buffer 20 at the subsequent stage.
[0070]
Note that the reduction of the bit widths of the color difference signals U and V is not limited to the 8-bit width as described above. For example, the color difference signal U is reduced to 9 bits and the color difference signal is reduced. The reduction range may be changed as appropriate, for example, V is reduced to 7 bits.
[0071]
As described above, in the converter 67, the video signal (without reducing the bit width) without deteriorating the information amount of the luminance signal Y of the video signal and the key signal K important in the digital video effect. The bit width of (Y / U / V / K) can be adapted to the subsequent buffer 20 (36-bit width ZBT SRAM).
[0072]
Next, FIG. 19 shows a detailed configuration example of the buffer 20. The buffer 20 includes four units U0, U1, L0, and L1 that can be read simultaneously. A configuration example of the unit U0 is shown in FIG. The unit U0 is configured as a double buffer with an A buffer composed of SRAM 73-U0-A and a B buffer composed of SRAM 73-U0-B. Thereby, the unit U0 realizes two-dimensional reading and simultaneous writing. Similarly, each of the units U0 to L1 has a double buffer configuration, and two-dimensional reading and simultaneous writing are realized. Note that SRAM 73-U0-A to 73-L1-B are simply referred to as SRAM 73 when it is not necessary to distinguish them individually.
[0073]
FIG. 21 shows allocation of video signals input from the scan converter 15 to the units U0 to L1. That is, when the video signal of the EVEN field output from the scan converter 15 is written into the buffer 20, as shown in FIG. 4A, the mth (m = 0, 2, 4,...) Horizontal Four pixels composed of two adjacent pixels on the scanning line and two adjacent pixels on the (m + 2) th horizontal scanning line immediately below are written in the A buffers of different units U0 to L1, respectively. When the video signal of the ODD field output from the scan converter 15 is written to the buffer 20, as shown in FIG. 5B, the (m + 1) th (m = 0, 2, 4,...) Horizontal Four pixels composed of two adjacent pixels on the scanning line and two adjacent pixels on the m + 3th horizontal scanning line immediately below are written in the B buffers of different units U0 to L1.
[0074]
In this way, by writing the four pixels adjacent vertically and horizontally to different units U0 to L1, they can be read out simultaneously, so that the pixel located at the center of the four pixels is interpolated using the four-pixel video signal. The processing to be performed can be executed efficiently.
[0075]
Next, in the effective access area (access area) of the buffer 20, the out-of-area data band (black area) set around the data area (real image area) in which the video signal is written will be described with reference to FIGS. I will explain.
[0076]
FIG. 22 shows a coordinate system of read addresses (also described as linear addresses) set in the buffer 20, and FIG. 23 shows the video signal of the EVEN field written in the data area (real image area) of FIG. It shows the state.
[0077]
In general, when a video signal written in the buffer 20 is read, based on an address on the display (hereinafter referred to as a screen address) when the video signal to which the digital effect is applied in the DME 3 is displayed on the display, The read address of the buffer 20 is determined. However, details of the relationship between the screen address and the read address will be described later with reference to FIG.
[0078]
When the read address [X, Y] is determined at the position indicated by “x” in FIG. 24, the video signals of four pixels on the top, bottom, left, and right of the position of the read address [X, Y] are read out and the interpolation circuit 22, the video signal of the pixel corresponding to the read address [X, Y] is interpolated (however, the interpolation process using four pixels is for the HD format video signal, and the SD format video signal Is applied with an interpolation process using 16 pixels).
[0079]
By the way, when the position indicated by the mark “x” in FIG. 25 is the read address [X, Y], there are no four pixels above, below, left, and right. Requires different processing. Therefore, when a read address [X, Y] is given, it is necessary to determine whether the read address [X, Y] can be applied with normal interpolation processing. Necessary. Therefore, an out-of-area data band is set in the image buffer 20 for the purpose of omitting such a determination circuit.
[0080]
Specifically, as shown in FIGS. 26 and 27, in the effective access area (access area) of the buffer 20, dummy for two pixels is provided on each of the upper, lower, left and right sides of the data area (real image area) into which the video signal is written. The out-of-area data band (black area) is set by writing the video signal. FIG. 27 shows a state in which the video signal of the EVEN field is written in the data area (real image area) of FIG. 26 and an out-of-area data band is set around the video signal.
[0081]
Here, it is shown that it is possible to set a data area and an out-of-area data band in which video signals are written in the buffer 20 in terms of storage capacity.
[0082]
The buffer 20 is provided with eight SRAMs 73-U0-A to 73-L1-B as shown in FIG. 19, and four of them store the video signal of the field image. However, one effective access area of the SRAM 73 is 256k words = 256 * 1024 words = 262144 words. Since the data area and the out-of-area data band to be written therein are ¼ of the field image video signal (540 × 1920) and the dummy video signal for two pixels in the upper, lower, left and right directions, the necessary capacity is 544 *. 1924/4 = 261664 words, which are completely stored in one effective access area of the SRAM 73. Therefore, an out-of-area data band can be set in the buffer 20 in terms of storage capacity.
[0083]
In this manner, by setting the data area and the out-of-area data band in the effective access area of the buffer 20, for example, when the position indicated by “x” in FIG. 28 is the read address [X, Y]. However, since there are four pixels on the top, bottom, left, and right, it is possible to apply a normal interpolation process using four pixels. Therefore, when the read address [X, Y] is given, it is not necessary to determine whether or not normal interpolation processing can be applied to the read address [X, Y]. It is possible to omit a dedicated circuit.
[0084]
The read address [X, Y] that can be generated at this time is
-960.5 <X <960.5
-540.5 <Y <540.5
It is.
[0085]
Next, the address generator 21 that supplies the read address to the buffer 20 will be described. Before that, the details of the relationship between the screen address and the read address will be described with reference to FIG. FIG. 29A shows a coordinate system of a read address (Xm, Ym, T) (equivalent to the above-described read address [X, Y]) set in the buffer 20. In the coordinate system of the read address, the origin is provided at the center of the image. T denotes a lighting modulation axis (T axis) designated when lighting is added to the video. FIG. 29B shows the coordinate system of the screen address (H, V). In the screen address coordinate system, the origin is provided at the upper left of the image. The points a to d in the coordinate system of the read address correspond to the points a 'to d' in the coordinate system of the screen address, respectively.
[0086]
Since the screen address (H, V) is obtained by converting the read address (Xm, Ym, T) using the 3 × 3 conversion matrix A, the screen address (H, V) is sequentially scanned. Inverse matrix A of transformation matrix A^-1Can be read to calculate the read address (Xm, Ym, T).
[0087]
Specifically, the read address (Xm, Ym, T) is calculated as shown in the following equation.
[Expression 1]

Effect parameter a₁₁Thru a₃₃Is the inverse matrix A as shown in the following equation:^-1Elements.
[Expression 2]

The rotation coefficients p and q of the lighting modulation axis T are p = cos θ and q = sin θ.
[0088]
Thus, the read address (Xm, Ym, T) is a function value X (H, V), Y (H, V), T (H, V), Z with the screen address (H, V) as a parameter. Calculated using (H, V).
[0089]
By the way, the read address is calculated for each pixel (clock) of the screen address to be sequentially scanned. For all the pixels of the screen address, the function values X (H, V), Y (H, V), If T (H, V) and Z (H, V) are calculated and the read address is calculated, the amount of calculation becomes enormous, and a circuit dedicated to the calculation is required.
[0090]
Therefore, as shown in FIG. 30, about the four end points of the screen address, that is, the upper left point (0,0), the lower left point (0,539), the upper right point (1919,0), and the lower right point (1919,539), Function values X (0,0), Y (0,0), T (0,0), Z (0,0), X (0,539), Y (0,539), T (0,539), Z (0,539) , X (1919,0), Y (1919,0), T (1919,0), Z (1919,0), X (1919,539), Y (1919,539), T (1919,539), Z (1919,539) (hereinafter referred to as function values X (0,0) to Z (1919,539)) is calculated, and function values X (H, V) for other pixels of the screen address , Y (H, V), T (H, V), Z (H, V) are interpolated using the function values X (0,0) to Z (1919,539) calculated for the four end points, The corresponding read address is calculated.
[0091]
Thus, using function values X (0,0) to Z (1919,539) corresponding to the four end points of the screen address, function values X (H, V) and Y (H, V) for other pixels are used. , T (H, V), Z (H, V) is referred to as Super Interpolation, and the upper left point (0,0) and lower left point (0,539) Interpolation processing of the direction, or vertical interpolation processing of the upper right point (1919,0) and lower right point (1919,539) is called super interpolation (V), and the result of super interpolation (V), etc. The horizontal interpolation process using the function values of the left and right end points on the horizontal scanning line is referred to as super interpolation (H).
[0092]
Next, processing timing of super interpolation will be described with reference to FIG. When super-interpolation is performed for a field image, function values X (0,0) to Z (1919,539) are calculated in advance for each of the four end points of the screen address one field before the field image. It is held in a predetermined register (described later). Then, super-interpolation (V) is executed at the initial stage of the horizontal blanking period (BLANK (H)) in synchronization with the enable of the timing signal VMIX, and the screen address is horizontally scanned in synchronization with the enable of the timing signal HMIX. In the period (ACTIVE AREA), super-interpolation (H) is executed for each clock.
[0093]
As described above, the super interpolation (H) and the super interpolation (V) have different execution timings.
[0094]
FIG. 32 shows a configuration example of the address generator 21. The register calculation block 91 calculates the function values X (0,0) to Z (1919,539) at the four end points of the screen address and supplies them to the super interpolation block 93. The mixer coefficient block 92 supplies the mixer coefficient previously stored in the built-in register to the super interpolation block 93.
[0095]
The super interpolation block 93 receives the function values (0, 0) to Z (1919, 539) of the four end points of the screen address supplied from the register calculation block 91 and the mixer coefficient supplied from the mixer coefficient block 92. Are used to execute super interpolation (H) and super interpolation (V), and function values X (H, V) and Y (H, respectively) corresponding to pixels other than the four end points on the obtained screen address. V), T (H, V), and Z (H, V) are interpolated and output to the read address calculation block 94.
[0096]
The read address calculation block 94 has function values X (H, V), Y (H, V), T (H, V), T (H, V), respectively corresponding to all pixels of the screen address input from the super interpolation block 93. Using Z (H, V), a read address is generated and output to the buffer 20.
[0097]
FIG. 33 shows a configuration example of the super interpolation block 93. The super interpolation block 93 is a block that interpolates a function value X (H, V), a block that interpolates a function value Y (H, V), a block that interpolates a function value T (H, V), and a function value Consists of blocks that interpolate Z (H, V).
[0098]
The REG_V_START_XL register 101-X of the block that interpolates the function value X (H, V) holds the function value X (0,0) for the upper left point (0,0) supplied from the register operation block 91, and the selector Output to the A terminal of 107-X. The REG_V_START_XR register 102 -X holds the function value X (1919,0) for the upper right point (1919,0) supplied from the register operation block 91 and outputs it to the B terminal of the selector 107 -X. The FF_H_START_X register 103-X holds the output of the mixer 111-X input via the A terminal of the selector 112-X and outputs it to the B terminal of the selector 108-X. The FF_H_END_X register 104-X holds the output of the mixer 111-X input via the B terminal of the selector 112-X and outputs it to the B terminal of the selector 110-X. The REG_V_END_XL register 105-X holds the function value X (0,539) for the lower left point (0,539) supplied from the register operation block 91 and outputs it to the B terminal of the selector 109-X. The REG_V_END_XR register 106-X holds the function value X (1919,539) for the lower right point (1919,539) supplied from the register operation block 91 and outputs it to the A terminal of the selector 109-X.
[0099]
The selectors 107-X to 110-X output the input to the A terminal or the B terminal to the subsequent stage. The selector 112-X outputs the output of the mixer 111-X to the FF_H_START_X register 103-X or the FF_H_END_X register 104-X. The mixer 111 -X sets the output from the selector 108 -X input to the A terminal to A, sets the output from the selector 110 -X input to the B terminal to B, and sets the mixer coefficient supplied from the mixer coefficient block 92 to k._nIn this case, the interpolation value C is output to the subsequent stage using the following equation for each clock.
Interpolated value C = A · (1.0−k_n) + B ・ k_n
However, the following equation is actually used to decrease the number of multiplications by one.
Interpolated value C = k_n(B-A) + A
[0100]
The configuration of each block that interpolates each of the function values Y (H, V), T (H, V), and Z (H, V) is the same as the configuration of the block that calculates the function value X (H, V). Since it is the same, the description is abbreviate | omitted. However, the correspondence between the REG_V_START_XL registers 101-X to REG_V_END_ZR registers 106-Z and the function values X (0,0) to Z (1919,539) held by them is as shown in FIG.
[0101]
FIG. 35 shows the correspondence between the registers built in the mixer coefficient block 92 and the mixer coefficients held therein.
[0102]
Next, the operation of the super interpolation block 93 will be described. It is assumed that corresponding function values (0, 0) to Z (1919, 539) are supplied from the register operation block 91 to the REG_V_START_XL registers 101-X to REG_V_END_ZR registers 106-Z.
[0103]
First, the vertical component V of the screen address is initialized to V = 0, and super-interpolation (V) is started in synchronization with Enable of the timing signal VMIX. First, in order to execute super interpolation (V) at the left end point (0, V) of the screen address, input sources and output destinations of function values to the mixers 111-X to 111-Z are shown in FIG. Switching is performed in each block so as to be as described above.
[0104]
Specifically, for example, in a block for interpolating the function value X (H, V), the selectors 107-X to 110-X, 112-X are switched as shown in FIG. As a result, the function value X (0,0) for the upper left point (0,0) held in the REG_V_START_XL register 101-X is input to the A terminal of the mixer 111-X, and the REG_V_END_XL is input to the B terminal. The function value X (0,539) for the lower left point (0,539) held in the register 105-X is input. The mixer coefficient is further supplied from the mixer coefficient supply block 92 to the mixer 111 -X. The mixer 111-X interpolates the function value X (0, V) for the left end point (0, V) of the screen address. The interpolated function value X (0, V) is latched in the FF_H_START_X register 103-X via the selector 112-X.
[0105]
The same processing is performed in the other blocks, and the function value Y (0, V), function value T (0, V), function value Z (0, V) for the left end point (0, V) of the screen address. ) Are latched in the corresponding FF_H_START_X registers 103-Y to 103-Z, respectively.
[0106]
Next, in order to execute the super interpolation (V) of the right end point (1919, V) of the screen address, the input source and output destination of the function values to the mixers 111-X to 111-Z are shown in FIG. As shown, switching and the like are performed in each block.
[0107]
Specifically, for example, in the block for interpolating the function value X (H, V), the selectors 107-X to 110-X, 112-X are switched as shown in FIG. Thus, the function value X (1919,539) for the upper right point (1919,539) held in the REG_V_START_XR register 102-X is input to the A terminal of the mixer 111-X, and the REG_V_END_XR is input to the B terminal. The function value X (1919,539) for the lower right point (1919,539) held in the register 106-X is input. The mixer coefficient is further supplied from the mixer coefficient supply block 92 to the mixer 111 -X. The mixer 111-X interpolates the function value X (1919, V) for the right end point (1919, V) of the screen address. The interpolated function value X (1919, V) is latched into FF_H_END_X 104-X via the selector 112-X.
[0108]
The same processing is performed in the other blocks, and the function value Y (1919, V), function value T (1919, V), function value Z (1919, V) for the right end point (1919, V) of the screen address is performed. ) Are latched in the corresponding FF_H_END_X registers 104-Y to 104-Z, respectively.
[0109]
The processing so far is executed in the horizontal blanking period.
[0110]
Thereafter, in synchronization with the Enable of the timing signal HMIX, the horizontal component H of the screen address is initialized to H = 0, and superinterpolation (H) is started. In order to execute super interpolation (H), switching and the like are performed in each block so that the input source and output destination of the function values to the mixers 111-X to 111-Z are as shown in FIG. The
[0111]
Specifically, for example, in the block that interpolates the function value X (H, V), the selectors 108-X and 110-X are switched as shown in FIG. As a result, the function value X (0, V) for the left end point (0, V) held in the FF_H_START_X register 103-X is input to the A terminal of the mixer 111-X, and FF_H_END_X is input to the B terminal. The function value X (1919, V) for the right end point (1919, V) held in the register 104-X is input. The mixer coefficient is further supplied to the mixer 111-X from the mixer coefficient supply block 92 for each clock. The mixer 111 -X interpolates the function value X (H, V) for the right end point (1919, V) from the left end point (0, V) sequentially for each clock and supplies the result to the read address calculation block 94.
[0112]
The same processing is performed in the other blocks, and the function value Y (H, V) and the function value T (from the left end point (0, V) to the right end point (1919, V) in order for each clock. H, V) and function value Z (H, V) are interpolated and supplied to the read address calculation block 94.
[0113]
After the horizontal component H of the screen address is initialized, the processing so far is executed in the horizontal scanning period.
[0114]
Thereafter, the vertical component V is incremented by 1, and the processing after the super interpolation (V) described above is repeated. When the vertical component V reaches 540, super-interpolation for the field being processed is terminated, and the next field is the target of processing.
[0115]
As described above, since super interpolation (V) is executed in the horizontal blanking period and super interpolation (H) is executed in the horizontal scanning period, super interpolation (V) and super interpolation are executed. It becomes possible to execute the interpolation (H) by sharing the same circuit (super interpolation block 93).
[0116]
Next, the interpolation circuit 22 will be described with reference to FIG. When the video signal buffered in the buffer 20 is in the HD format, the interpolation circuit 22 executes a 4-point interpolation process using a 4-pixel video signal at an operating frequency of 74.25 MHz. Further, when an SD format video signal is field-frame converted and buffered in the buffer 20, a 16-point interpolation process using a 16-pixel video signal is performed at an operating frequency of 54 MHz (an SD format video signal is processed). At a normal operating frequency of 13.5 MHz).
[0117]
FIG. 42 shows a configuration example of the interpolation circuit 22. The interpolation circuit 22 proportionally distributes the video signals of two pixels adjacent in the vertical direction that are simultaneously input from the units U0 and L0 of the buffer 20, and obtains the interpolation value TA of the video signal corresponding to the position between the two pixels. The vertical proportional distribution circuit 121 to be calculated and the video signals of two pixels adjacent in the vertical direction simultaneously input from the units U1 and L1 of the buffer 20 are proportionally distributed, and the video signal corresponding to the position between the two pixels is distributed. A vertical proportional distribution circuit 122 that calculates the interpolation value TB, and a horizontal proportional distribution that proportionally distributes the interpolation value TA input from the vertical proportional distribution circuit 121 and the interpolation value TB input from the vertical proportional distribution circuit 122. The circuit 123 is configured.
[0118]
FIG. 43 shows a configuration example of the vertical proportional distribution circuit 121. In the vertical proportional distribution circuit 121, in addition to the video signals of two pixels adjacent in the vertical direction that are simultaneously input from the units U0 and L0 of the buffer 20, the vertical position of the interpolation point between the two pixels is also displayed. The 4-bit position information r shown and the sel signal for controlling the selectors 143 and 144 are input.
[0119]
The video signal from the unit U0 is input to the delay circuit (D) 141, and the video signal from the unit L0 is input to the delay circuit 142. The position information r is input to the delay circuit 148. The sel signal is input to the delay circuit 152.
[0120]
The delay circuit (D) 141 delays the video signal from the unit U0 by a predetermined clock cycle, and outputs it to the a terminal of the selector 143 and the b terminal of the selector 144. The delay circuit 142 delays the video signal from the unit L0 by a predetermined clock cycle, and outputs it to the b terminal of the selector 143 and the a terminal of the selector 144.
[0121]
Based on the sel signal input from the delay circuit 152, the selector 143 outputs to the multiplier 145 the video signal from the unit U0 input to the a terminal or the video signal from the unit L0 input to the b terminal. Based on the inverted sel signal input from the NOT circuit 153, the selector 144 inputs to the multiplier 146 the video signal from the unit L0 input to the a terminal or the video signal from the unit U0 input to the b terminal. Output. Accordingly, the video signal from the unit U0 is input to one of the multipliers 145 and 146, and the video signal from the unit L0 is input to the other. Here, the value of the video signal input to the multiplier 145 is A, and the value of the video signal input to the multiplier 146 is B.
[0122]
The multiplier 145 multiplies the value (16-r) input from the delay circuit 151 by the value A of the video signal input from the selector 143, and outputs the result to the calculator 147. The multiplier 146 multiplies the position information value r input from the delay circuit 149 by the video signal value B input from the selector 144 and outputs the result to the calculator 147. The arithmetic unit 147 adds the output of the multiplier 145 and the output of the multiplier 146 and divides the result by 16.
[0123]
The vertical proportional distribution circuit 121 configured as described above outputs the vertical interpolation value TA represented by the following equation to the horizontal proportional distribution circuit 123.
Interpolated value TA = (A * (16−r) + B * r) / 16
[0124]
Note that the configuration of the vertical proportional distribution circuit 122 is the same as that of the vertical proportional distribution circuit 121, and a description thereof will be omitted.
[0125]
FIG. 44 shows a configuration example of the horizontal proportional distribution circuit 123. The horizontal proportional distribution circuit 123 indicates the horizontal position of the interpolation point in addition to the vertical interpolation value TA from the vertical proportional distribution circuit 121 and the vertical interpolation value TB from the vertical proportional distribution circuit 122. 4-bit position information r ′ is input to the interpolation coefficient supply circuits 171 and 172.
[0126]
The multiplier 161 multiplies the vertical direction interpolation value TA from the vertical direction proportional distribution circuit 121 by the interpolation coefficient Ci input from the interpolation coefficient supply circuit 171 and outputs the result to the register (R0) 163. The multiplier 162 multiplies the vertical interpolation value TB from the vertical proportional distribution circuit 122 by the interpolation coefficient Ci input from the interpolation coefficient supply circuit 172, and outputs the result to the register (R1) 164.
[0127]
The adder 165 adds the output of the register (R0) 163 and the output of the register (R1) 164 and outputs the result to the register (R2) 166. The adder 167 adds the output of the register (R2) 166 and the output of the register (R3) 168 holding the output of one clock cycle before, and outputs the result to the register (R3) 168 and the divider 169. To do.
[0128]
The divider 169 divides the output of the adder 167 (accumulated value of the main power of the adder 165 in a predetermined period) by the total interpolation coefficient ΣCi and outputs the result to the register (R4) 170.
[0129]
The register (R0) 163, the register (R1) 164, the register (R2) 166, and the register (R3) 168 delay the input from the upper stage by a predetermined clock period and output it. The register (R3) 168 is reset in response to the RSR_R signal. Initialize the stored value. The register (R4) 170 outputs the value held corresponding to the EN signal.
[0130]
The interpolation coefficient supply circuits 171 and 172 supply the interpolation coefficient Ci corresponding to the 4-bit position information r ′ indicating the horizontal position of the interpolation point to the multipliers 161 and 162, respectively.
[0131]
The horizontal proportional distribution circuit 123 configured as described above outputs the horizontal interpolation value X expressed by the following equation.
Interpolated value X = Σ (Ci * Ti) / ΣCi
Here, i = 0, 1 in the case of 4-point interpolation processing, and i = 0, 1, 2,..., 7 in the case of 16-point interpolation processing.
[0132]
FIG. 45 shows the value of the interpolation coefficient Ci when 16-point interpolation processing is performed on a field-frame converted SD format video signal.
[0133]
Next, the operation of the interpolation circuit 22 will be described. First, a four-point interpolation process when an HD format video signal is buffered in the buffer 20 will be described.
[0134]
In this case, as shown in FIG. 21, the HD format video signal is stored in the units U0, U1, L0, and L1 of the buffer 20 in the field image unit, and the four pixels adjacent vertically and horizontally are separated. Therefore, for example, when interpolating the video signal corresponding to the interpolation point indicated by the “x” mark in the EVEN field shown in FIG. 46A, the video signals of the four pixels above, below, left, and right of the interpolation point are generated in one clock cycle. They can be read simultaneously.
[0135]
Of the video signals (hereinafter referred to as signals U0, U1, L0, and L1) read out simultaneously from the units U0, U1, L0, and L1 in one clock cycle, the signals U0 and L0 are vertical proportional distribution circuits. The signals U 1 and L 1 are supplied to the vertical proportional distribution circuit 122.
[0136]
The vertical proportional distribution circuit 121 proportionally distributes the signals U0 and L0 in accordance with the vertical position information r of the interpolation point, and outputs the obtained vertical interpolation value TA to the horizontal proportional distribution circuit 123. The vertical proportional distribution circuit 122 proportionally distributes the signals U1 and L1 according to the vertical position information r of the interpolation point, and outputs the obtained vertical interpolation value TB to the horizontal proportional distribution circuit 123.
[0137]
The horizontal proportional distribution circuit 123 proportionally distributes the interpolation values TA and TB in the vertical direction in accordance with the horizontal position information r ′ of the interpolation point, and obtains an interpolation value corresponding to the interpolation point indicated by “x”. .
[0138]
Since the operation for the video signal in the ODD field shown in FIG. 46B is the same, the description thereof is omitted.
[0139]
Next, the 16-point interpolation process when the field-frame converted SD format video signal is stored in the buffer 20 will be described. Before that, the stored SD format video signal is 480i. An example of a x720 SD format video signal will be described with reference to FIG.
[0140]
Before the SD format video signal is input to the buffer 20, the converter 67 of the scan converter 15 converts the EVEN field image (consisting of pixels indicated by a circle in FIG. 47A) and the ODD field image (see FIG. 47). 47 (A) composed of pixels indicated by □) is synthesized and converted into a frame image of 480 × 720 as shown in FIG. 47A, and pixels indicated by ○ adjacent to the top and bottom From the pixels indicated by □ and □, the pixel located between the two pixels (the pixel indicated by Δ in FIG. 47B) is interpolated and converted into a 960 × 720 frame image.
[0141]
The SD format video signal converted into a 960 × 720 frame image is vertically and horizontally as shown in FIG. 48, that is, in the same manner as when the HD format field image is stored in the buffer 20 (FIG. 21). Each of the four adjacent pixels is stored separately in units U0, U1, L0, and L1 of the buffer 20.
[0142]
The concept of the operation of the 16-point interpolation process of the interpolation circuit 22 will be described. For example, in the case of interpolating a video signal corresponding to the interpolation point indicated by “×” in FIG. 49, 16 pixels (the same figure) including 8 pixels adjacent to the upper stage of the interpolation point “×” and 8 pixels adjacent to the upper stage. (16 pixels surrounded by a long rectangle in the horizontal direction) is read out and proportionally distributed.
[0143]
Specifically, as shown in FIG. 50, the upper 8-pixel video signal and the lower 8-pixel video signal corresponding to them are proportionally distributed by the vertical proportional distribution circuits 121 and 122, respectively, so that the vertical Directional interpolation values T0 to T7 are calculated. As shown in FIG. 51, the interpolation values T0 to T7 in the vertical direction are respectively multiplied by the interpolation coefficients C0 to C7 by the horizontal proportional distribution circuit 123, and the total sum Σ (Ti * Ci) is obtained. The interpolation value of the interpolation point “×” is calculated by dividing by the sum ΣCi of the interpolation coefficients Ci. However, in this case, i = 0, 1, 2,.
[0144]
Next, the operation timing of the 16-point interpolation process of the interpolation circuit 22 executed at the operation frequency of 54 MHz will be described with reference to FIGS. As described above, the units U0, U1, L0, and L1 of the buffer 20 can be read simultaneously, so that in the 16-point interpolation process, 4 pixels are sequentially read every clock cycle.
[0145]
That is, as shown in FIG. 52A, if the 16-pixel video signals used for interpolation stored separately in the units U0, U1, L0, and L1 of the buffer 20 are described as a0 to a15, At the 0th timing (cycle0), the video signals a0, a1, a8, and a9 shown in FIG. 5B are read, and at the second timing (cycle1), the video signals shown in FIG. a2, a3, a10, a11 are read out, and at the third timing (cycle 2), the video signals a4, a5, a12, a13 shown in FIG. ), The video signals a6, a7, a14, and a15 shown in FIG.
[0146]
The video signals a0 and a8 read at the 0th timing (cycle0) are input to the vertical proportional distribution circuit 121, and the video signals a1 and a9 are input to the vertical proportional distribution circuit 122. The video signals a2 and a10 read at the first timing (cycle1) are input to the vertical proportional distribution circuit 121, and the video signals a3 and a11 are input to the vertical proportional distribution circuit 122. The video signals a4 and a12 read at the second timing (cycle2) are input to the vertical proportional distribution circuit 121, and the video signals a5 and a13 are input to the vertical proportional distribution circuit 122. Further, the video signals a6 and a14 read out at the third timing (cycle 3) are input to the vertical proportional distribution circuit 121, and the video signals a7 and a15 are input to the vertical proportional distribution circuit 122.
[0147]
53A and 53B show the operation timings of the vertical proportional distribution circuits 121 and 122, respectively. The vertical proportional distribution circuit 121 sequentially outputs the interpolation value TA in the vertical direction to the horizontal proportional distribution circuit 123 at a timing delayed by 4 clock cycles from the input timing from the units U 0 and L 0 of the buffer 20.
[0148]
Specifically, at the fifth timing (cycle 5), the interpolation value Ta0 obtained by proportionally distributing the video signals a0 and a8 is output, and at the sixth timing (cycle 6), the video signals a2 and a10 are proportionally distributed. The interpolation value Ta2 is output, and at the seventh timing (cycle 7), the interpolation value Ta4 obtained by proportionally distributing the video signals a4 and a12 is output. At the eighth timing (cycle 8), the video signals a6 and a14 are output. Is output in proportion to the interpolated value Ta6.
[0149]
Similarly, the vertical proportional distribution circuit 122 sequentially outputs the interpolation value TB in the vertical direction to the horizontal proportional distribution circuit 123 at a timing delayed by 4 clock cycles from the input timing from the units U 1 and L 1 of the buffer 20. .
[0150]
Specifically, the interpolation value Ta1 obtained by proportionally distributing the video signals a1 and a9 is output at the fifth timing (cycle5), and the video signals a3 and a11 are proportionally distributed at the sixth timing (cycle6). The interpolation value Ta3 is output, and at the seventh timing (cycle 7), the interpolation value Ta5 obtained by proportionally distributing the video signals a5 and a13 is output. At the eighth timing (cycle 8), the video signals a7 and a15 are output. Is output in proportion to the interpolation value Ta7.
[0151]
FIG. 54 shows the operation timing of the horizontal proportional distribution circuit 123. The vertical proportional distribution circuit 123 outputs an interpolation value X every 4 clock cycles.
[0152]
Specifically, the multiplier 161 multiplies the vertical interpolation values Ta0, Ta2, Ta4, and Ta6 sequentially input at the fifth to eighth timings by interpolation coefficients C0, C2, C4, and C6, respectively. The data is output to the register (R0) 163. The register (R0) 163 sequentially outputs the multiplication values Ta0 * C0, Ta2 * C2, Ta4 * C4, Ta6 * C6 to the adder 165 at the eighth to eleventh timings delayed by three clock cycles from the input timing. .
[0153]
Similarly, the multiplier 162 multiplies the interpolation values C1, C3, C5, and C7 in the vertical direction sequentially input at the fifth to eighth timings by interpolation coefficients C1, C3, C5, and C7, respectively. R1) output to 164. The register (R1) 164 sequentially outputs the multiplication values Ta1 * C1, Ta3 * C3, Ta5 * C5, Ta7 * C7 to the adder 165 at the eighth to eleventh timings delayed by three clock cycles from the input timing. .
[0154]
The adder 165 sequentially receives the multiplication value Ta0 * C0 and the multiplication value Ta1 * C1, the multiplication value Ta2 * C2 and the multiplication value Ta3 * C3, which are sequentially input from the multipliers 161 and 162 at the eighth to eleventh timings. The value Ta4 * C4 and the multiplication value Ta5 * C5, the multiplication value Ta6 * C6 and the multiplication value Ta7 * C7 are added together and output to the register (R2) 166. The register (R2) 166 has addition values Ta0 * C0 + Ta1 * C1, Ta2 * C2 + Ta3 * C3, Ta4 * C4 + Ta5 * C5, Ta6 * C6 + Ta7 * C7 at the ninth to twelfth timing delayed by one clock cycle from the input timing. Are sequentially output to the adder 167.
[0155]
The adder 167 adds the addition value sequentially input from the adder 165 at the ninth to twelfth timings and the output of the adder 167 one clock period before input from the register (R3) 168 to add a register ( R3) Output to 168 and divider 169. The register (R3) 168 is initialized in synchronization with the RST_R signal input every four clock cycles. Therefore, at the thirteenth timing, the adder 167 outputs the cumulative value of the addition values sequentially input from the adder 165 at the ninth to twelfth timings.
[0156]
The divider 169 divides the accumulated value from the adder 167 by the total ΣCi of interpolation coefficients and outputs the result to the register (R4) 170. The register (R4) 170 synchronizes with the EN signal input every 4 clock cycles (in this case, the 13th timing), that is, the division value from the multiplier 169, that is, the interpolation value of the interpolation point “×”. X is output.
[0157]
As described above, in the present embodiment, the HD format video signal and the SD format video signal can be interpolated by the same circuit, that is, the interpolation circuit 22. Since the 4-point interpolation processing is executed for the HD format video signal and the 16-point interpolation processing is executed for the SD format video signal instead of the 4-point interpolation processing. Interpolated values with the same quality as the SD dedicated device can be obtained.
[0158]
The present invention can be applied to any device that processes video signals.
[0159]
By the way, the series of processes described above can be executed by hardware, but can also be executed by software. When a series of processing is executed by software, a program constituting the software may execute various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a recording medium in a general-purpose personal computer.
[0160]
As shown in FIG. 1, the recording medium is distributed to provide a program to a user separately from a computer, and includes a magnetic disk 6 (including a floppy disk) on which a program is recorded, an optical disk 7 (CD- It is not only composed of package media consisting of ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disc)), magneto-optical disc 8 (including MD (Mini Disc)), or semiconductor memory 9 etc. It is configured by a ROM, a hard disk or the like on which a program is recorded, which is provided to the user in a state of being pre-installed in the computer.
[0161]
In the present specification, the step of describing the program recorded in the recording medium is not limited to the processing performed in time series according to the described order, but is not necessarily performed in time series, either in parallel or individually. The process to be executed is also included.
[0162]
Further, in this specification, the system represents the entire apparatus constituted by a plurality of apparatuses.
[0163]
【The invention's effect】
As described above, according to the image processing apparatus and method and the recording medium program of the present invention, the video signal corresponding to the input image is stored in the memory, and the four end points of the frame on which the deformed image is superimposed. The parameter corresponding to each of the four endpoints is calculated, and the parameter corresponding to each of the two endpoints facing each other in the vertical direction is used among the parameters corresponding to each of the calculated four endpoints. Interpolate the parameters corresponding to each of the two points facing in the horizontal direction, and use the parameters corresponding to each of the two points facing in the horizontal direction to interpolate the parameters corresponding to the points on the transformed image. The memory read address is generated based on the parameters corresponding to the interpolated and deformed points on the image, but the vertical interpolation processing and the horizontal interpolation are performed. Processing, since to be executed by a common circuit, it is possible to realize a space saving and cost reduction.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration example of an image composition apparatus to which the present invention is applied.
FIG. 2 is a block diagram illustrating a configuration example of a DME 3;
3 is a diagram for explaining the concept of processing for interpolating color difference signals U and V in an H filter 14. FIG.
FIG. 4 is a block diagram illustrating a configuration example of a portion related to a process of interpolating color difference signals U and V of an H filter 14;
5 is a block diagram showing a configuration example of a four-point interpolation circuit 40 in FIG.
6 is a block diagram illustrating a configuration example of a scan converter 15. FIG.
FIG. 7 is a diagram showing a rough time transition of processing for converting the scanning direction to vertical in the field unit of the scan converter 15;
FIG. 8 schematically shows the relationship between SDRAMs 64-1 and 64-2 that burst-transfer video signals recorded in field units in the order of vertical scanning, and SRAM 65 that caches burst-transferred video signals. FIG.
FIG. 9 is a diagram illustrating an example of continuous access timing in alternate bursts with respect to two types of banks of the SDRAM 64;
FIG. 10 is a diagram for explaining continuous access (write) to SDRAM 64;
FIG. 11 is a diagram for explaining continuous access (reading) to SDRAM 64;
FIG. 12 is a diagram illustrating an example of two-dimensional allocation of addresses in a 2-bank 4-word burst of an HD format (1080i × 1920) video signal to SDRAM 64;
FIG. 13 is a diagram for explaining a counter mechanism for generating a write address for SDRAM 64;
14 is a diagram showing the order of continuous reading from SDRAM 64. FIG.
FIG. 15 is a diagram for explaining a counter mechanism that generates a read address for SDRAM 64;
FIG. 16 is a diagram showing a concept of using a memory constituting SRAM 65 as a ring.
FIG. 17 is a diagram illustrating a concept of using four memories constituting the SRAM 65 as a quadruple ring.
FIG. 18 is a diagram for explaining processing in which a converter 67 converts the video signal (Y / U / V / K) into a 36-bit width by reducing the values of the color difference signals U and V to 8 bits.
FIG. 19 is a block diagram illustrating a configuration example of a buffer 20;
20 is a block diagram illustrating a configuration example of a unit U0 of the buffer 20. FIG.
FIG. 21 is a diagram illustrating allocation of HD format video signals input from the scan converter 15 to units U0 to L1.
22 is a diagram showing a coordinate system of a read address set in a buffer 20. FIG.
FIG. 23 is a diagram showing a state where an EVEN field video signal is written in the data area of the buffer 20;
FIG. 24 is a diagram illustrating the positions of four pixels used in a four-point interpolation process.
FIG. 25 is a diagram illustrating an example in which there are not four pixels used for the four-point interpolation process.
FIG. 26 is a diagram illustrating an out-of-area data band provided in an effective access area of the buffer 20;
FIG. 27 is a diagram showing a state where an EVEN field video signal is written in the data area of the buffer 20 and an out-of-area data band is set around the video signal.
FIG. 28 is a diagram for explaining that a four-point interpolation process can be performed when an out-of-area data band is set in the buffer 20;
FIG. 29 is a diagram for explaining a relationship between a screen address and a read address.
FIG. 30 is a diagram for explaining super-interpolation by the address generator 21;
FIG. 31 is a diagram for explaining processing timing of super interpolation;
32 is a block diagram illustrating a configuration example of an address generator 21. FIG.
33 is a block diagram showing a configuration example of a super interpolation block 93. FIG.
FIG. 34 is a diagram illustrating function values X (0,0) to Z (1919,539) held in REG_V_START_XL registers 101-X to REG_V_END_ZR registers 106-Z.
FIG. 35 is a diagram illustrating a correspondence relationship between a register built in the mixer coefficient block 92 and a mixer coefficient held therein.
FIG. 36 is a diagram illustrating an input source and an output destination of function values to the mixers 111-X to 111-Z.
FIG. 37 is a diagram illustrating a state of a block for interpolating a function value X (H, V) corresponding to FIG.
FIG. 38 is a diagram illustrating input sources and output destinations of function values to the mixers 111-X to 111-Z.
FIG. 39 is a diagram illustrating a state of a block for interpolating a function value X (H, V) corresponding to FIG.
FIG. 40 is a diagram illustrating an input source and an output destination of function values to the mixers 111-X to 111-Z.
41 is a diagram illustrating a state of a block that interpolates a function value X (H, V) corresponding to FIG. 40. FIG.
42 is a block diagram illustrating a configuration example of an interpolation circuit 42. FIG.
43 is a block diagram illustrating a configuration example of a vertical proportional distribution circuit 121. FIG.
44 is a block diagram showing a configuration example of a horizontal proportional distribution circuit 123. FIG.
FIG. 45 is a diagram illustrating values of interpolation coefficients C0 to C7 used for 16-point interpolation processing.
FIG. 46 is a diagram for describing a four-point interpolation process for an HD format video signal.
47 is a diagram for explaining field / frame conversion for an SD format video signal by the converter 67 of the scan converter 15. FIG.
FIG. 48 is a diagram showing the allocation of field format frame-converted SD format video signals input from the scan converter 15 to the units U0 to L1.
[Fig. 49] Fig. 49 is a diagram for describing 16-point interpolation processing for an SD format video signal.
FIG. 50 is a diagram for explaining the operation of the vertical proportional distribution circuits 121 and 122 in 16-point interpolation processing.
FIG. 51 is a diagram for explaining the operation of the horizontal proportional distribution circuit 123 in 16-point interpolation processing;
FIG. 52 is a diagram for explaining video signal readout timing in 16-point interpolation processing;
FIG. 53 is a diagram for explaining the operation timing of the vertical proportional distribution circuits 121 and 122 in 16-point interpolation processing.
FIG. 54 is a diagram for describing the operation timing of the horizontal proportional distribution circuit 123 in the 16-point interpolation process.
[Explanation of symbols]
1 lever arm, 2 control circuit, 3 DME, 4 synthesis circuit, 5 drive, 6 magnetic disk, 7 optical disk, 8 magneto-optical disk, 9 semiconductor memory, 11 HDFF, 12-14H filter, 15 scan converter, 16 VDFF, 17 to 19 V filter, 20 buffer, 21 address generator, 22 interpolation circuit, 64 SDRAM, 65 SRAM, 67 converter, 73 SRAM, 93 super interpolation block, 121, 122 vertical proportional distribution circuit, 123 horizontal proportional distribution circuit

Claims (4)

In an image processing apparatus that changes the shape of an image,
Storage means for storing a video signal corresponding to the input image in a memory;
Designating means for designating the deformed shape of the image;
Calculating means for calculating a parameter corresponding to each of the four end points of the frame on which the transformed image is superimposed;
Of the parameters corresponding to each of the four end points calculated by the calculating means, using the parameters corresponding to the two opposite end points in the vertical direction, the horizontal direction between each of the two sets of the two end points. Vertical direction interpolation means for interpolating parameters corresponding to each of two opposing points;
Horizontal interpolation means for interpolating parameters corresponding to points on the image after deformation, using the parameters corresponding to the two points facing the horizontal direction interpolated by the vertical direction interpolation means;
Generating means for generating a read address of the memory based on the parameter corresponding to the point on the image after the transformation interpolated by the horizontal direction interpolation means;
Reading means for reading out the video signal from the memory according to the read address generated by the generating means,
The image processing apparatus according to claim 1, wherein the vertical direction interpolation unit and the horizontal direction interpolation unit are configured by a common circuit. The vertical interpolation means operates in a horizontal blanking period;
The image processing apparatus according to claim 1, wherein the horizontal direction interpolation unit operates during a horizontal scanning period. In the image processing method of the image processing apparatus for deforming the shape of the image,
Storing a video signal corresponding to the input image in a memory;
A designating step of designating a deformed shape of the image;
A calculation step for calculating parameters corresponding to each of the four end points of the frame on which the transformed image is superimposed;
Among the parameters corresponding to each of the four end points calculated in the processing of the calculating step, using the parameters corresponding to each of the two end points opposed in the vertical direction, between the two sets of the two end points. A vertical interpolation step for interpolating parameters corresponding to each of two points facing in the horizontal direction;
Horizontal interpolation step of interpolating parameters corresponding to points on the image after the deformation using the parameters corresponding to the two points facing the horizontal direction interpolated in the processing of the vertical direction interpolation step When,
A generation step of generating a read address of the memory based on the parameter corresponding to the point on the image after the transformation interpolated in the process of the horizontal direction interpolation step;
A step of reading out the video signal from the memory according to the read address generated in the process of the generation step,
The image processing method according to claim 1, wherein the vertical direction interpolation step and the horizontal direction interpolation step are executed by a common circuit. An image processing program for changing the shape of an image,
Storing a video signal corresponding to the input image in a memory;
A designating step of designating a deformed shape of the image;
A calculation step for calculating parameters corresponding to each of the four end points of the frame on which the transformed image is superimposed;
Among the parameters corresponding to each of the four end points calculated in the processing of the calculating step, using the parameters corresponding to each of the two end points opposed in the vertical direction, between the two sets of the two end points. A vertical interpolation step for interpolating parameters corresponding to each of two points facing in the horizontal direction;
Horizontal interpolation step of interpolating parameters corresponding to points on the image after the deformation using the parameters corresponding to the two points facing the horizontal direction interpolated in the processing of the vertical direction interpolation step When,
A generation step of generating a read address of the memory based on the parameter corresponding to the point on the image after the transformation interpolated in the process of the horizontal direction interpolation step;
A step of reading out the video signal from the memory according to the read address generated in the process of the generation step,
A recording medium on which a computer-readable program is recorded, wherein the vertical interpolation step processing and the horizontal interpolation step processing are executed by a common circuit.

Images