JP2004064194A - Device for special effect image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device for special effect images capable of preventing image quality deterioration in an output image subjected to a special effect obtained by deforming an input image. <P>SOLUTION: The device for special effect images comprising: a reduction rate generating unit (11) for generating a reduction rate at each position of the input image attendant on deformation of the input image in a mode of deformation of the input image by using effect information denoting the mode; and a filter processing unit (12) performing the filter processing with a filter characteristic decided on the basis of the reduction rate with respect to each position of the input image, a memory buffer (14) stores image information of the input image subjected to the filter processing by the filter processing unit (12) in a prescribed order, and the image information is read according to the read address generated on the basis of the effect information. <P>COPYRIGHT: (C)2004,JPO

Description

【００２７】
【数２】

【００２８】
フィルタ処理ユニット１２は、後述するような３次元アフィン変換による入力画像の変形により発生することが予想される歪み（例えば、折返し歪み）を抑制するためのフィルタ処理を、変形前の入力画像を表す入力映像信号に対して行う。画像の歪み具合（縮小率）がわかっていれば、一般的にその歪みを抑制するためのフィルタ特性を決めることができる。従って、このフィルタ処理ユニット１２は、入力映像信号の画素に対応した部分毎に、その画素位置に対して上記のように生成される３次元アフィン変換縮小率生成ユニット１１からの水平及び垂直方向の縮小率Ｒｘ、Ｒｙに基づいて定められたフィルタ特性にてフィルタ処理を行う。
【００２９】
上記フィルタ処理ユニット１２は、例えば、連続的または段階的にフィルタ係数の調整が可能なデジタルフィルタとなる水平方向フィルタ及び垂直方向フィルタにて構成することができる。この場合、フィルタ処理の対象となる画素位置（ｘ，ｙ）の上記水平及び垂直方向の縮小率Ｒｘ、Ｒｙに基づいて適正なフィルタ特性となるように水平方向フィルタ及び垂直方向フィルタそれぞれのフィルタ係数が調整される。そして、その調整により得られたフィルタ特性にて上記水平方向フィルタ及び垂直方向フィルタが当該画素位置（ｘ，ｙ）に対応した入力映像信号の部分に対するフィルタ処理を行う。
【００３０】
また、上記フィルタ処理ユニット１２は、特性の異なる複数のデジタルフィルタ回路を備えた水平方向フィルタ及び垂直方向フィルタにて構成することができる。この場合、フィルタ処理の対象となる画素位置（ｘ，ｙ）の上記水平及び垂直方向の縮小率Ｒｘ、Ｒｙに基づいて水平方向フィルタ及び垂直方向フィルタそれぞれにおけるデジタルフィルタ回路の選択切替えがなされる。そして、その選択切替えにより有効となる上記水平方向フィルタ及び垂直方向フィルタそれぞれのデジタルフィルタ回路が当該画素位置（ｘ，ｙ）に対応した入力映像信号の部分に対するフィルタ処理を行う。
【００３１】
上記読出しアドレス生成ユニット１３は、上記シーケンシャルアドレス生成ユニット１０からのアドレスで表される画素位置（ｘ，ｙ）の座標変換を上記効果パラメータ（Ａｘ、Ａｙ、Ｂｘ、Ｂｙ、Ｃｘ、Ｃｙ、Ｄ、Ｅ）を用いて次式に従って行い（３次元アフィン変換）、その変換後の画素位置（Ｘ，Ｙ）に対応した読取りアドレスを出力する。
【００３２】
【数３】

【００３３】
【数４】

【００３４】
上述したような画像特殊効果装置は次のように動作する。
シーケンシャルアドレス生成ユニット１０から画像フレーム上の画素位置（ｘ，ｙ）に対応したアドレスが３次元アフィン変換縮小率生成ユニット１１に供給される毎に、３次元アフィン変換縮小率生成ユニット１１がその画素位置（ｘ，ｙ）での水平及び垂直方向の縮小率Ｒｘ、Ｒｙを演算して（上記式（１）、（２）参照）フィルタ処理ユニット１２に供給する。そして、フィルタ処理ユニット１２が、３次元アフィン変換縮小率生成ユニット１１からその画素位置（ｘ，ｙ）での縮小率Ｒｘ、Ｒｙを入力すると、入力映像信号の当該画素位置（ｘ，ｙ）に対応した部分に対してその縮小率Ｒｘ、Ｒｙに基づいて定められるフィルタ特性にてフィルタ処理を行う。
【００３５】
これにより、シーケンシャルアドレス生成ユニット１０から１フレーム分のアドレスが出力されると、フィルタ処理ユニット１２により１フレーム分の入力映像信号に対してフィルタ処理がなされる。そのフィルタ処理ユニット１２から出力されるフィルタ処理後の１フレーム分の映像信号が、上記３次元アフィン変換縮小率生成ユニット１１及びフィルタ処理ユニット１２における処理時間を遅延として考慮しつつ上記シーケンシャルアドレス生成ユニット１０から供給されるアドレス（書込みアドレス）に従ってフレームメモリバッファ１４に書き込まれる。
【００３６】
その後、読出しアドレス生成ユニット１３が、シーケンシャルスアドレス生成ユニット１０からのアドレス（ｘ，ｙ）が供給される毎に、３次元アフィン変換の座標変換処理（上記式（３）、（４）参照）により得られた読出しアドレス（Ｘ，Ｙ）をフレームメモリバッファ１４に供給する。その結果、フレームメモリバッファ１４からその読出しアドレス（Ｘ，Ｙ）に対応した映像信号値が読み出される。これにより、シーケンシャルアドレス生成ユニット１０から１フレーム分のアドレスが出力されると、フレームメモリバッファ１４から１フレーム分の出力映像信号が出力される。
【００３７】
上記のようにして、フィルタ処理後の入力映像信号をシーケンシャルなアドレスに従ってフレームメモリバッファ１４に書込み、その後、そのシーケンシャルなアドレスを３次元アフィン変換より変換して得られる読出しアドレスに従ってそのフレームメモリバッファ１４から映像信号を読みだすことにより出力映像信号を生成するようにしているので、その出力映像信号にて表される出力画像Ｉｏは、上記３次元アフィン変換により図２に示すように奥行き感を出すような特殊効果がかけられた画像となる。
【００３８】
また、入力画像に対して予めフィルタ処理がなされているので、その入力画像を変形させて得られた出力画像の縮小部分での折返し歪が抑制される。更に、そのフィルタ処理が入力画像全面に一様な特性でなされるものではなく、各位置に対して予想される（効果パラメータに基づいて演算される）縮小率に応じたフィルタ特性にてそのフィルタ処理がなされるので、特殊効果のかけられたより高い画質の出力画像Ｉｏを得ることができるようになる。
【００３９】
更に、また、出力画像と入力画像の比較から各位置での縮小率を得るのではなく、出力画像を得る前に各位置での縮小率を予想し（効果パラメータに基づいて演算し）、その予想された各位置での縮小率に応じたフィルタ特性にてフィルタ処理を行うようにしているので、この画像特殊効果装置は、入力映像信号がフレームごとに連続的に処理される動画像に対してリアルタイム性を損なうことなく特殊効果をかけることができるようになる。
【００４０】
次に、本発明の第二の実施の形態に係る画像特殊効果装置について説明する。
【００４１】
本発明の第二の実施の形態に係る画像特殊効果装置は、図３に示すように構成される。この画像特殊効果装置は、入力画像を変形させて図５に示すようにページをめくるような特殊効果（以下、ページターン効果という）のかけられた出力画像Ｉｏを得るものである。
【００４２】
図３に示す第二の実施の形態に係る画像特殊効果装置は、ＣＰＵ１５がページターン効果が得られるような入力画像の変形態様を表す効果パラメータを出力する点、上記３次元アフィン変換縮小率生成ユニット１１に代えてページターン縮小率生成ユニット２０を設けた点、読出しアドレス生成ユニット１３がそのページターン効果に対応した効果パラメータに基づいて読出しアドレスを生成する点、新たに背景レベル出力ユニット２１を設けた点で、上述した第一の実施の形態に係る画像特殊効果装置（図１参照）と相違する。
【００４３】
ＣＰＵ１５から読出しアドレス生成ユニット１３及びページターン縮小率生成ユニット２０に供給される効果パラメータは、図５に示すようなページめくりによる湾曲（ページターン効果）の中心位置を表すｘｃ、ｙｃ、その湾曲の半径ｒ、及びそのページターン効果の方向ｘ１を表す角度θから構成される。
【００４４】
ページターン縮小率生成ユニット２０は、シーケンシャルアドレス生成ユニット１０からの画素位置（ｘ，ｙ）に対応したアドレスが入力される毎に、ＣＰＵ１５からの上記効果パラメータｘｃ、ｙｃ、ｒ、θを用いて当該画素位置（ｘ，ｙ）での水平方向の縮小率Ｒｘと垂直方向の縮小率Ｒｙを次式に従って演算する。
【００４５】
【数５】

【００４６】
【数６】

【００４７】
【数７】

【００４８】
上記式において、ｘ１＜０の領域は、図５に示すように効果後の画像において平面部分の領域を表し、０≦ｘ１≦ｒの領域は、効果後の画像において曲面状になる部分の領域を表す。即ち、効果後に平面部分となる領域の各位置での縮小率は「１」として得られ（式（６）参照）、効果後に変形のある曲面状となる領域の各位置での縮小率がその曲面の湾曲の程度ｒとその効果の方向θに依存したものとなる（式（７）参照）。
【００４９】
フィルタ処理ユニット１２は、前述した第一の実施の形態と同様に、入力映像信号の各画素位置に対応した部分に対して、その位置に対して上記ページターン縮小生成ユニット２０から供給される縮小率Ｒｘ、Ｒｙに対応したフィルタ特性にて水平方向及び垂直方向のフィルタ処理を行う。このフィルタ処理後の映像信号が、前述した第一の実施の形態と同様に、シーケンシャルアドレス生成ユニット１０からシーケンシャルに出力されるアドレスに従ってフレームメモリバッファ１４に書き込まれる。
【００５０】
上記読出しアドレス生成ユニット１３は、例えば、図４に示すように構成される。図４において、この読出しアドレス生成ユニット１３は、効果方向位置制御ユニット３０、上面変化量生成ユニット３１、下面変化量生成ユニット３２、上面ｘアドレス生成ユニット３３、上面ｙアドレス生成ユニット３４、下面ｘアドレス生成ユニット３５、下面ｙアドレス生成ユニット３６、及び有効領域判定ユニット３７を有している。効果方向位置制御ユニット３０は、シーケンシャルアドレス生成ユニット１０から画素位置（ｘ，ｙ）を表すアドレスが入力される毎に、ＣＰＵ１５から供給される効果パラメータｘｃ、ｙｃ、θを用いた上記式（５）によって、当該入力アドレス（ｘ，ｙ）を効果の中心（ｘｃ，ｙｃ）を基準にした効果の方向θにおける位置ｘ１を表すアドレスに変換する。
【００５１】
下面変化量生成ユニット３２は、効果方向位置制御ユニット３０からの座標変換後のアドレスｘ１とＣＰＵ１５からの効果パラメータｒとを用いて、画像下面Ｂ（図５参照）上の位置ｘ１（アドレス）のｒの湾曲による元の位置からの変化量Ｆａ（ｘ１）を次式に従って演算する。
【００５２】
【数８】

【００５３】
上記変化量Ｆａ（ｘ１）は、画像下面Ｂのｘ１＜０の領域では湾曲がないので、０の変化量（変化しない）を表し、ｘ１≧０の領域ではｒの湾曲があるので、そのｒに基づいた変化量を表す。
【００５４】
上記下面ｘアドレス生成ユニット３５は、上記下面変化量生成ユニット３２から供給される変化量Ｆａ（ｘ１）を用いた次式に従って、画素位置（ｘ，ｙ）を表す座標系におけるｘ方向の位置ｘの変位先の位置ｘ２（ｘアドレス）を演算する。
【００５５】
【数９】

【００５６】
また、上記下面ｙアドレス生成ユニット３６は、上記下面変化量生成ユニット３２から供給される変化量Ｆａ（ｘ１）を用いた次式に従って、画素位置（ｘ，ｙ）を表す座標系におけるｙ方向の位置ｙの変化先の位置ｙ２（ｙアドレス）を演算する。
【００５７】
【数１０】

【００５８】
上記のようにして、読出しアドレス生成ユニット１３は、シーケンシャルアドレス生成ユニット１０からシーケンシャルに供給される画素位置（ｘ，ｙ）に対応したアドレスを入力すると、上記式（５）、（８）〜（１０）で示されるような効果後の画像の下面Ｂ上における位置（ｘ２，ｙ２）に対応した下面読出しアドレスを生成する。
【００５９】
一方、上記上面変化量生成ユニット３１は、効果方向位置制御ユニット３０からの座標変換後のアドレスｘ１とＣＰＵ１５からの効果パラメータｒとを用いて、画像上面Ｔ（図５参照）上の位置ｘ１（アドレス）のｒの湾曲による元の位置からの変化量Ｆｂ（ｘ１）を次式に従って演算する。
【００６０】
【数１１】

【００６１】
上記変化量Ｆｂ（ｘ１）は、画像上面Ｔ上のｘ１＜０の領域では、ｘ１＝πｒ／２を基準とした折返しにより定まる元の位置からの変化量を表し、ｘ１≧０の領域では、湾曲ｒ及び折返しの双方を考慮して定まる元の位置からの変化量を表している。
【００６２】
上記上面ｘアドレス生成ユニット３３は、上記上面変化量生成ユニット３１から供給される変化量Ｆｂ（ｘ１）を用いた次式に従って、画素位置（ｘ，ｙ）を表す座標系におけるｘ方向の位置ｘの変位先の位置ｘ３（ｘアドレス）を演算する。
【００６３】
【数１２】

【００６４】
また、上記上面ｙアドレス生成ユニット３４は、上記上面変化量生成ユニット３１から供給される変化量Ｆｂ（ｘ１）を用いた次式に従って、画素位置（ｘ，ｙ）を表す座標系におけるｙ方向の位置ｙの変位先の位置ｙ３（ｙアドレス）を演算する。
【００６５】
【数１３】

【００６６】
上記のようにして、読出しアドレス生成ユニット１３は、シーケンシャルアドレス生成ユニット１０からシーケンシャルに供給される画素位置（ｘ，ｙ）に対応したアドレスを入力すると、上記式（５）、（１１）〜（１３）で示されるような効果後の画像の上面Ｔ上における位置（ｘ３，ｙ３）に対応した上面読出しアドレスを生成する。
【００６７】
なお、上記効果後の画像下面Ｂに対して適用される変化量Ｆａ（ｘ１）と画像上面Ｔに対して適用される変化量Ｆｂ（ｘ１）の特性は、図６（ａ）、（ｂ）に示すようになる。
【００６８】
上記有効領域判定ユニット３７は、上記上面ｘアドレス生成ユニット３３及び上面ｙアドレス生成ユニット３４にて生成された上面読出しアドレス（ｘ３，ｙ３）が画面内に存在するか否かを判定する。次いで、上記有効領域判定ユニット３７は、上記下面ｘアドレス生成ユニット３５及び下面ｙアドレス生成ユニット３６にて生成された下面読出しアドレス（ｘ２，ｙ２）がその画面内に存在するか否かを判定する。
【００６９】
上面読出しアドレス（ｘ３，ｙ３）が画面内に存在しているとの判定がなされると、上記有効領域判定ユニット３７は、その上面読出しアドレス（ｘ３，ｙ３）を読出しアドレス（Ｘ，Ｙ）として出力すると共に「１」を表すキー（ｋｅｙ）信号を出力する。また、上面読出しアドレス（ｘ３，ｙ３）が画面内に存在せず、下面読出しアドレス（ｘ２，ｙ２）が画面内に存在するとの判定がなされると、上記有効領域判定ユニット３７は、その下面読出しアドレス（ｘ２，ｙ２）を読み出しアドレス（Ｘ，Ｙ）として出力すると共に「１」を表すキー（ｋｅｙ）信号を出力する。
【００７０】
一方、上記上面読出しアドレス（ｘ３，ｙ３）と下面読出しアドレス（ｘ２，ｙ２）の双方が画面内に存在しないとの判定がなされると、上記有効領域判定ユニット３７は、「０」を表すキー（ｋｅｙ）信号を出力する。この「０」を表すキー（ｋｅｙ）信号は、図５に示すような画面内における背景部分Ｇ（上面Ｔ及び下面Ｂのいずれでもない部分）を表す信号となる。
【００７１】
上記のようにして、読出しアドレス生成ユニット１３は、シーケンシャルアドレス生成ユニット１０からシーケンシャルに生成される画素位置（ｘ，ｙ）を表すアドレスを入力する毎に、読出しアドレス（Ｘ，Ｙ）とキー（ｋｅｙ）信号を出力する。
【００７２】
図３に戻って、背景レベル出力ユニット２１は、フレームメモリバッファ１４から読出しアドレス（Ｘ，Ｙ）に従って読み出される映像信号を入力し、上記読出しアドレス生成ユニット１３から出力されるキー（ｋｅｙ）信号が「１」である場合に、その映像信号をそのまま出力する。一方、そのキー（ｋｅｙ）信号が「０」である場合に、その映像信号に代えて背景レベル（例えば、黒）を表す信号を出力する。そして、この背景レベル出力ユニット２１から出力される信号が出力映像信号となる。
【００７３】
上記のような画像特殊効果装置では、ページターン効果による入力画像（１フレーム分の入力映像信号）の各画素位置での縮小率が生成され（式（５）、（６）、（７）参照）、その縮小率に基づいて定まるフィルタ特性にて当該入力映像信号のフィルタ処理がなされる。そして、そのフィルタ処理後の入力映像信号がシーケンシャルアドレス生成ユニット１０からシーケンシャルに出力されるアドレスに従ってフレームメモリバッファ１４に書き込まれる。その後、そのシーケンシャルなアドレスをページターン効果に基づいて変換して得られる読出しアドレス（式（８）〜（１３）参照）に従ってそのフレームメモリバッファ１４から映像信号を読みだすと共に背景画像の信号への置き換えを行っていくことにより出力映像信号が生成される。
【００７４】
従って、その出力映像信号にて表される出力画像Ｉｏは、上記ページターン効果により図５に示すようなページをめくるような特殊効果がかけられた画像となる。また、この第二の実施の形態に係る画像特殊効果装置も、上述した第一の実施の形態と同様に、その入力画像の各画素位置に対して縮小率に基づいて決まるフィルタ特性でのフィルタ処理がなされることにより、より高い画質の出力画像Ｉｏを得ることができると共に、動画像に適したものとなる。
【００７５】
次に、本発明の第三の実施の形態に係る画像特殊効果装置について説明する。
【００７６】
本発明の第三の実施の形態に係る画像特殊効果装置は、図７図に示すように構成される。この画像特殊効果装置は、入力画像を変形させて図８に示すように奥行き感を出すような特殊効果（３次元アフィン変換）とページターン効果の双方の特殊効果がかけられた出力画像Ｉｏを得るものである。
【００７７】
図７に示す第三の実施の形態に係る画像特殊効果装置は、ＣＰＵ１５が上記３次元アフィン変換の効果パラメータ及び上記ページターン効果に係る効果パラメータの双方を出力する点、前述した３次元アフィン変換縮小率生成ユニット１１及びページターン縮小率生成ユニット２０の双方を備えている点、読出しアドレス生成ユニット１３が３次元アフィン変換の効果パラメータとページターン効果に係る効果パラメータの双方に基づいて読出しアドレスを生成する点、及び上記３次元アフィン変換縮小率生成ユニット１１から出力される縮小率とページターン縮小生成ユニット２０から出力される縮小率とを合成して合成縮小率を出力する合成ユニット４０を新たに設けた点で、上述した第一及び第二の実施の形態に係る画像特殊効果装置（図１、図３参照）と相違する。
【００７８】
３次元アフィン変換縮小率生成ユニット１１は、シーケンシャルアドレス生成ユニット１０からシーケンシャルに出力される画素位置（ｘ，ｙ）に対応したアドレスを入力する毎に、ＣＰＵ１５から供給される３次元アフィン変換の効果パラメータ（Ａｘ、Ａｙ、Ｂｘ、Ｂｙ、Ｃｘ、Ｃｙ、Ｄ，Ｅ）を用いて、上記式（１）、（２）に従って各画素位置での水平方向の縮小率ＲｘＡ（式（１）におけるＲｘ）と垂直方向の縮小率ＲｙＡ（式（２）におけるＲｙ）を演算する。また、ページターン縮小率生成ユニット２０は、シーケンシャルアドレスユニット１０からシーケンシャルに出力されるアドレスを入力する毎に、ＣＰＵ１５から供給されるページターン効果に係る効果パラメータ（ｒ、θ、ｘｃ、ｙｃ）を用いて、上記式（５）、（６）、（７）に従って各画素位置での水平方向の縮小率ＲｘＢ（式（７）におけるＲｘ）と垂直方向の縮小率ＲｙＢ（式（７）におけるＲｙ）を演算する。
【００７９】
上記合成ユニット４０は、上記３次元アフィン変換縮小率生成ユニット１１からの水平方向の縮小率ＲｘＡ、垂直方向の縮小率ＲｙＡと、上記ページターン縮小率生成ユニット２０からの水平方向の縮小率ＲｘＢ、垂直方向の縮小率ＲｙＢを用いて、次式に従って水平方向の合成縮小率Ｒｘ及び垂直方向の合成縮小率Ｒｙを算出する。
【００８０】
【数１４】

【００８１】
このようにして、シーケンシャルアドレス生成ユニット１０からシーケンシャルに画素位置（ｘ，ｙ）に対応したアドレスが出力される毎に、合成ユニット４０は、その画素位置に対応した水平方向の合成縮小率Ｒｘ及び垂直方向の合成縮小率Ｒｙを出力する。そして、フィルタ処理ユニット１２が、前述した第一及び第二の実施の形態の場合と同様の手法に従って、上記合成縮小率Ｒｘ、Ｒｙに基づいて定まるフィルタ特性の水平方向フィルタ及び垂直方向フィルタにより入力映像信号に対するフィルタ処理を行う。そして、そのフィルタ処理後の１フレーム分の映像信号は、上記ページターン縮小率生成ユニット２０、合成ユニット４０およびフィルタ処理ユニット１２での処理時間を遅延として考慮して上記シーケンシャルアドレス生成ユニット１０からシーケンシャルに出力される画素位置（ｘ，ｙ）に対応したアドレスに従ってフレームメモリバッファ１４に順次書き込まれる。
【００８２】
読出しアドレス生成ユニット１３は、シーケンシャルアドレス生成ユニット１０からシーケンシャルに出力される画素位置（ｘ，ｙ）に対応するアドレスを入力する毎に、まず、上記式（３）及び（４）に従って３次元アフィン変換されたアドレスＸＡ（式（３）におけるＸ）及びＹＡ（式（４）におけるＹ）を生成する。
【００８３】
そのように生成されたアドレス（ＸＡ、ＸＢ）に対してページターン効果に対応した処理がなされる。
【００８４】
即ち、上記式（５）に対応した下式
【００８５】
【数１５】

【００８６】
に従って、当該アドレス（ＸＡ，ＹＡ）をページターン効果の中心（ｘｃ，ｙｃ）を基準にしたその効果の方向θにおける位置ｘ１を表すアドレスに変換する。そして、上述した式（８）及び下式に従って下面読出しアドレス（ｘ２，ｙ２）が生成される。
【００８７】
【数１６】

【００８８】
【数１７】

【００８９】
また、上述した式（１１）及び下式に従って上面読出しアドレス（ｘ３，ｙ３）が生成される。
【００９０】
【数１８】

【００９１】
【数１９】

【００９２】
そして、上記第二の実施の形態の場合と同様に、読出しアドレス生成ユニット１３の有効領域判定ユニット３７（図４参照）により、上記下面読出しアドレス（ｘ２，ｙ２）及び上面読出しアドレス（ｘ３、ｙ３）のいずれかが最終的な読出しアドレス（Ｘ，Ｙ）として選択されると共にキー（ｋｅｙ）信号が生成される。その読出しアドレス生成ユニット１３から出力される読出しアドレスに従って、上述したようにフレームメモリバッファ１４に格納された映像信号が読み出されると共に、「０」となるキー（ｋｅｙ）信号に基づいて背景画像の信号への置き換えがなされ、最終的な出力映像信号が生成される。
【００９３】
上述したような画像特殊効果装置では、シーケンシャルアドレス生成ユニット１０からシーケンシャルに出力されるアドレス（ｘ，ｙ）から３次元アフィン変換による座標変換により得られたアドレス（ＸＡ，ＹＡ）が、更に、ページターン効果に基づいた変換処理によりアドレス（ｘ２，ｙ２）またはアドレス（ｘ３，ｙ３）の読出しアドレスに変換される。従って、フレームメモリバッファ１４にシーケンシャルに生成されるアドレス（ｘ，ｙ）に従って蓄積された映像信号を上記読出しアドレスに従って読み出して得られる出力映像信号は、図８に示すような奥行き感を出すような特殊効果（３次元アフィン変換）とページターン効果の双方の特殊効果がかけられた出力画像Ｉｏを表すことになる。
【００９４】
また、上記３次元アフィン変換による入力画像の変形に伴う各画素位置での縮小率と、ページターン効果による入力画像の変形に伴う各画素位置での縮小率とが合成され、それにより得られた合成縮小率に基づいて定められるフィルタ特性にて上記フィルタ処理ユニット１２が入力画像の各画素位置に対するフィルタ処理を行う。このため、上記のように奥行き感を出すような特殊効果（３次元アフィン変換）とページターン効果の双方の特殊効果がかけられた出力画像Ｉｏにおける歪み（折返し歪み）が抑制されるようになる。
【００９５】
更に、本発明の第四の実施の形態に係る画像特殊効果装置について説明する。
【００９６】
本発明の第四の実施の形態に係る画像特殊効果装置は、図９に示すように構成される。この画像特殊効果装置は、３次元アフィン変換縮小率生成ユニット１１からの縮小率をフィルタ特性を指定するための情報に変換するデコードユニット５０を新たに設けた点で第一の実施の形態に係る画像特殊効果装置（図１参照）と相違する。
【００９７】
上記デコードユニット５０は、シーケンシャルアドレス生成ユニット１０からシーケンシャルに出力される画素位置（ｘ，ｙ）に対応したアドレス毎に３次元アフィン変換縮小率生成ユニット１１にて生成されるその画素位置（ｘ，ｙ）での水平方向の縮小率Ｒｘと垂直方向の縮小率Ｒｙを次式に従ってフィルタ特性の指定情報Ｄｅｃｘ、Ｄｅｃｙに変換する。
【００９８】
【数２０】

【００９９】
【数２１】

【０１００】
上記各式（２０）、（２１）において、ｉｎｔ（）は、（）内の除算における商を表す。
【０１０１】
フィルタ処理ユニット１２は、複数のフィルタ特性を有する水平方向フィルタと複数のフィルタ特性を有する垂直方向フィルタとを有しており、水平方向フィルタの複数のフィルタ特性と指定情報Ｄｅｃｘの各値との関係及び垂直方向フィルタの複数のフィルタ特性と指定情報Ｄｅｃｙの各値との関係を管理している。
【０１０２】
上記のような画像特殊効果装置では、シーケンシャルアドレス生成ユニット１０から画素位置（ｘ，ｙ）に対応したアドレスが出力される毎に、３次元アフィン変換縮小率生成ユニット１１にて生成される縮小率Ｒｘ、Ｒｙに対応した指定情報Ｄｅｃｘ、Ｄｅｃｙがデコードユニット５０から出力される。その各指定情報Ｄｅｃｘ、Ｄｅｃｙを入力するフィルタ処理ユニット１２では、水平方向フィルタがその指定情報Ｄｅｃｘにて指定されるフィルタ特性にて入力映像信号の当該画素位置（ｘ，ｙ）に対応した部分に対するフィルタ処理を行うと共に、垂直方向フィルタがその指定情報Ｄｅｃｙにて指定されるフィルタ特性にて入力映像信号の当該画素位置（ｘ，ｙ）に対応した部分に対するフィルタ処理を行う。その結果、第一の実施の形態に係る画像特殊効果装置と同様に、３次元アフィン変換により図２に示すように奥行き感を出すような特殊効果がかけられた出力画像Ｉｏを得ることができる。
【０１０３】
この画像特殊効果装置によれば、前述した第一の実施の形態と同様の効果が得られることに加え、フィルタ処理ユニット１２でのフィルタ特性の選択に係る処理を効率的に行うことができるようになる。
【０１０４】
なお、上記のようなデコードユニット５０は、前述した第三の実施の形態に係る画像特殊効果装置にも適用することができる。この場合、デコードユニット５０は、合成ユニット４０から出力される合成縮小率Ｒｘ、Ｒｙを指定情報Ｄｅｃｘ、Ｄｅｃｙに変換し、その各指定情報Ｄｅｃｘ、Ｄｅｃｙをフィルタ処理ユニット１２に供給する。
【０１０５】
前述した各実施の形態に係る画像特殊効果装置では、縮小率が入力画像の画素単位に生成されるものであったが、本発明では、縮小率（変形度合情報）は必ずしも入力画像の画素単位に生成されるものでなくてもよい。効率的な処理を行いうるという観点から、複数の画素により形成される領域毎に縮小率（変形度合情報）を生成することもできる。この場合、その領域に含まれる位置に対してその領域について生成された縮小率（変形度合情報）に基づいて定められるフィルタ特性でのフィルタ処理がなされる。
【０１０６】
また、前述した各実施の形態に係る画像特殊効果装置は、動画像の処理に適用されたものであったが、静止画に対しても当然適用することは可能である。
【０１０７】
更に、入力画像の奥行き感を出すような変形（図２、図８参照）、及び入力画像のページをめくるような変形（図５、図８参照）について説明したが、入力画像を変形させる態様は、これに限られるものではない。その変形の態様を効果情報として表しうるものであれば任意の変形（例えば、画像の折返しの先端を巻き込むような変形（ロールターン）、画像を円筒に巻き込むような変形など）について同様の処理を行うことができる。
【０１０８】
【発明の効果】
以上、説明したように、本発明によれば、入力画像を変形させる態様を表す効果情報に基づいて生成された前記入力画像の各位置での変形度合情報に基づいて定められるフィルタ特性にて前記入力画像に対するフィルタ処理を行い、そのフィルタ処理後の画像情報から前記入力画像を前記態様にて変形させた出力画像を得るようにしているので、入力画像を変形させて得られる特殊効果がかけられた出力画像の画質劣化を抑制することのできる画像特殊効果装置を提供することが可能となる。
【図面の簡単な説明】
【図１】本発明の第一の実施の形態に係る画像特殊効果装置を示すブロック図
【図２】特殊効果のかけられた画像の第一の例を示す図
【図３】本発明の第二の実施の形態に係る画像特殊効果装置を示すブロック図
【図４】図３に示す画像特殊効果装置における読出しアドレス生成ユニットの構成例を示すブロック図
【図５】特殊効果のかけられた画像の第二の例を示す図
【図６】図５に示す画像変形における変化量の特性の一例を示す特性図
【図７】本発明の第三の実施の形態に係る画像特殊効果装置を示すブロック図
【図８】特殊効果がかけられた画像の第三の例を示す図
【図９】本発明の第四の実施の形態に係る画像特殊効果装置を示すブロック図
【符号の説明】
１０　シーケンシャルアドレス生成ユニット
１１　３次元アフィン変換縮小率生成ユニット
１２　フィルタ処理ユニット
１３　読出しアドレス生成ユニット
１４　フレームメモリバッファ
１５　ＣＰＵ
２０　ページターン縮小率生成ユニット
２１　背景レベル出力ユニット
３０　効果方向位置制御ユニット
３１　上面変化量生成ユニット
３２　下面変化量生成ユニット
３３　上面ｘアドレス生成ユニット
３４　上面ｙアドレス生成ユニット
３５　下面ｘアドレス生成ユニット
３６　下面ｙアドレス生成ユニット
３７　有効領域判定ユニット
４０　合成ユニット
５０　デコードユニット[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image special effect device which transforms an input image to obtain an output image to which a special effect has been applied.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, as an image special effect device for obtaining an output image to which a special effect is applied by deforming an input image, for example, there is one disclosed in JP-A-7-298136. This image special effect device is modified from the original image (by applying a special effect) by manipulating the read address when reading the image signal from the frame memory buffer in which the image signal related to the original image is written in a predetermined address order. T) have obtained the output image.
[0003]
[Problems to be solved by the invention]
However, such an image special effect device has a problem in that the image quality of a portion reduced by deformation of an image is deteriorated due to aliasing distortion of an image signal frequency. If a uniform filtering process is performed on the entire image to solve such a problem of image quality deterioration, the image quality of a portion reduced by deformation is improved, but an unreduced portion becomes unclear. Such as a new problem of image quality deterioration.
[0004]
The present invention has been made in order to solve such a problem, and provides an image special effect device capable of suppressing image quality deterioration of an output image to which a special effect obtained by deforming an input image is applied. Things.
[0005]
[Means for Solving the Problems]
An image special effect device according to the present invention reads out image information of an input image stored in a memory buffer in a predetermined address order according to a read address generated based on effect information representing a mode of deforming the input image. An image special effect device configured to obtain an output image obtained by deforming the input image in the mode in the above manner, wherein each position of the input image accompanying the deformation of the input image in the mode using the effect information Deformation degree information generating means for generating deformation degree information representing the degree of deformation in, and filter characteristics determined based on the deformation degree information generated by the deformation degree information generation means for each position of the input image Filter processing means for performing a filtering process on the input image, wherein the image information of the input image filtered by the Having the configuration as store according to the predetermined order.
[0006]
With such a configuration, before obtaining an output image obtained by actually deforming the input image, the deformation degree information at each position of the input image accompanying the deformation of the input image in the mode represented by the effect information is obtained. Each position of the input image generated using the effect information is subjected to a filtering process with a filter characteristic determined based on the generated deformation degree information. Then, image information of the input image that has been subjected to the filtering process for each position is stored in a memory buffer in a predetermined address order, and the memory is stored in accordance with a read address generated based on effect information representing a mode of deforming the input image. By reading the image information from the buffer, an output image obtained by deforming the input image in the above-described manner can be obtained.
[0007]
In this way, the degree of deformation associated with the deformation expected at each position with respect to the input image is previously generated based on the effect information indicating the mode of the deformation, and the generated deformation information is generated for each position of the input information. Since the filter processing is performed with the filter characteristics determined based on the degree information, the distortion at each position of the output image obtained by deforming the input image after the filter processing can be appropriately suppressed.
[0008]
In the present invention, in what unit of the input image the deformation degree information is generated for each position is not particularly limited.
[0009]
The image special effect device according to the present invention can be configured such that the deformation degree information generating means generates the deformation degree information for each position of the input image in pixel units.
[0010]
With such a configuration, the deformation degree information is generated for each pixel of the input image, and a filtering process can be performed for each pixel with a filter characteristic determined based on the deformation degree information. Can be suppressed with higher accuracy.
[0011]
Further, in the present invention, the deformation degree information is not particularly limited as long as it is information that quantitatively represents the degree of deformation. It may represent the amount of deformation.
[0012]
The image special effect device according to the present invention is configured such that the deformation degree information generating means includes a reduction rate calculation means for calculating a reduction rate at each position of the input image as deformation degree information using the effect information. be able to.
[0013]
With such a configuration, it is possible to effectively suppress aliasing distortion caused by image reduction.
[0014]
Also, the image special effect device according to the present invention is configured to generate a read address for the memory buffer based on effect information representing a plurality of deformation modes of the input image, A plurality of deformation degree information generating means for generating deformation degree information at each position of the input image associated with the deformation of the input image in each of the modes using the effect information representing the mode; Synthesizing means for synthesizing deformation degree information for each position of the input image obtained by the information generating means to generate synthetic deformation degree information for each position, wherein the filter processing means The filter processing may be performed with a filter characteristic determined based on the combined deformation degree information generated by the combining unit.
[0015]
With such a configuration, even when the input image is deformed in a plurality of deformation modes to obtain an output image, the deformation degree information at each position of the input image associated with each of the plurality of deformation modes is synthesized. Since filter processing is performed on each position of the input image with filter characteristics determined based on the obtained combined deformation degree information, distortion at each position of the output image can be reliably suppressed. Become.
[0016]
In the present invention, there is no particular limitation on the method of determining the filter characteristics when the filter processing unit performs the filter processing on each position of the input image based on the deformation degree information.
[0017]
In the image special effect device according to the present invention, the filter processing unit has a plurality of filter characteristics determined in accordance with the deformation degree information, and includes a plurality of filter characteristics for each position of the input image. A configuration may be adopted in which a filter characteristic corresponding to the deformation degree information generated by the deformation degree information generating means is selected, and a filtering process is performed using the selected filter characteristic.
[0018]
With such a configuration, the filter processing means can select a filter characteristic corresponding to the deformation degree information generated for each position of the input image from a plurality of filter characteristics determined corresponding to the deformation degree information. Can be determined, so that the filter processing can be performed efficiently.
[0019]
As described above, when configured to be able to select a filter characteristic corresponding to the deformation degree information generated for each position of the input image from a plurality of filter characteristics, the image special effect device according to the present invention, A design information generating means for converting the deformation degree information for each position of the input image generated by the deformation degree information generating means into specification information for specifying a corresponding filter characteristic in the filter processing means; The processing unit may be configured to perform a filter process on each position of the input image with a filter characteristic specified by the specification information generated by the specification information generation unit.
[0020]
With such a configuration, the filter processing unit performs the filter processing on each position of the input image with the filter characteristics specified by the specification information without considering the deformation degree information generated for each position of the input image. Since the filter processing can be performed, the filter processing by the filter processing means can be efficiently performed.
[0021]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0022]
The image special effect device according to the first embodiment of the present invention is configured as shown in FIG. This image special effect device obtains an output image Io to which a special effect such as giving a sense of depth as shown in FIG. 2 is obtained by deforming an input image by three-dimensional affine transformation.
[0023]
In FIG. 1, the image special effect device includes a sequential address generation unit 10, a three-dimensional affine transformation reduction ratio generation unit 11, a filter processing unit 12, a read address generation unit 13, a frame memory buffer 14, and a CPU (Central Processing Unit) 15. It has. In this image special effect device, for example, a video signal from an imaging device such as a television camera is sequentially supplied as an input video signal, and the input video signal is processed in units of frames, so that a special An output video signal in frame units representing the output image Io to which the effect has been applied is output. This output video signal is provided, for example, for broadcasting.
[0024]
The sequential address generation unit 10 sequentially generates addresses corresponding to pixel positions (x, y) on an image frame. The CPU 15 converts the effect parameter of the three-dimensional affine transformation, which is information representing a mode of transforming the input image (represented by one frame of the input video signal) with a sense of depth, into a three-dimensional affine transformation reduction rate generation unit 11 and the read address generation unit 13. The effect parameters of the three-dimensional affine transformation are composed of eight parameters Ax, Ay, Bx, By, Cx, Cy, D and E.
[0025]
The three-dimensional affine transformation reduction rate generation unit 11 uses the address from the sequential address generation unit 10 of the input image accompanying the deformation of the input image in such a manner as to give a sense of depth as described above using the effect parameter. A reduction ratio at each specified pixel position (x, y) is generated. For example, the horizontal reduction ratio Rx and the vertical reduction ratio Ry at each pixel position (x, y) of the input image to be transformed by the three-dimensional affine transformation are calculated according to the following equations.
[0026]
(Equation 1)

[0027]
(Equation 2)

[0028]
The filter processing unit 12 performs a filter process for suppressing distortion (for example, aliasing distortion) expected to occur due to deformation of the input image due to three-dimensional affine transformation, which will be described later, and represents the input image before deformation. Performed on the input video signal. If the degree of image distortion (reduction ratio) is known, it is generally possible to determine a filter characteristic for suppressing the distortion. Therefore, the filter processing unit 12 outputs the horizontal and vertical directions from the three-dimensional affine transformation reduction ratio generation unit 11 generated as described above to the pixel position for each portion corresponding to the pixel of the input video signal. Filter processing is performed with filter characteristics determined based on the reduction ratios Rx and Ry.
[0029]
The filter processing unit 12 can be composed of, for example, a horizontal filter and a vertical filter that are digital filters capable of adjusting a filter coefficient continuously or stepwise. In this case, the filter coefficients of the horizontal filter and the vertical filter are adjusted so that appropriate filter characteristics are obtained based on the horizontal and vertical reduction rates Rx and Ry of the pixel position (x, y) to be filtered. Is adjusted. Then, the horizontal direction filter and the vertical direction filter perform a filtering process on a portion of the input video signal corresponding to the pixel position (x, y) based on the filter characteristics obtained by the adjustment.
[0030]
Further, the filter processing unit 12 can be constituted by a horizontal filter and a vertical filter provided with a plurality of digital filter circuits having different characteristics. In this case, the digital filter circuits in the horizontal filter and the vertical filter are selectively switched based on the horizontal and vertical reduction ratios Rx and Ry of the pixel position (x, y) to be filtered. Then, the digital filter circuits of the horizontal filter and the vertical filter, which are enabled by the selection switching, perform a filtering process on a portion of the input video signal corresponding to the pixel position (x, y).
[0031]
The read address generation unit 13 converts the coordinate conversion of the pixel position (x, y) represented by the address from the sequential address generation unit 10 into the effect parameter (Ax, Ay, Bx, By, Cx, Cy, D, E) is performed according to the following equation (three-dimensional affine transformation), and a read address corresponding to the pixel position (X, Y) after the transformation is output.
[0032]
[Equation 3]

[0033]
(Equation 4)

[0034]
The image special effect device as described above operates as follows.
Every time an address corresponding to a pixel position (x, y) on an image frame is supplied from the sequential address generation unit 10 to the three-dimensional affine transformation reduction ratio generation unit 11, the three-dimensional affine conversion reduction ratio generation unit 11 The reduction ratios Rx and Ry in the horizontal and vertical directions at the position (x, y) are calculated (see the above equations (1) and (2)) and supplied to the filter processing unit 12. When the filter processing unit 12 receives the reduction ratios Rx and Ry at the pixel position (x, y) from the three-dimensional affine transformation reduction ratio generation unit 11, the filter processing unit 12 adds the pixel data to the pixel position (x, y) of the input video signal. Filter processing is performed on the corresponding portion with filter characteristics determined based on the reduction ratios Rx and Ry.
[0035]
As a result, when the address for one frame is output from the sequential address generation unit 10, the filtering processing is performed on the input video signal for one frame by the filter processing unit 12. One frame of the filtered video signal output from the filter processing unit 12 is converted into the sequential address generation unit while considering the processing time in the three-dimensional affine transformation reduction rate generation unit 11 and the filter processing unit 12 as a delay. The data is written to the frame memory buffer 14 in accordance with the address (write address) supplied from 10.
[0036]
Thereafter, every time the read address generation unit 13 is supplied with the address (x, y) from the sequential address generation unit 10, the read address generation unit 13 performs coordinate conversion processing of three-dimensional affine transformation (see the above equations (3) and (4)). Is supplied to the frame memory buffer 14. As a result, a video signal value corresponding to the read address (X, Y) is read from the frame memory buffer 14. Thus, when an address for one frame is output from the sequential address generation unit 10, an output video signal for one frame is output from the frame memory buffer 14.
[0037]
As described above, the input video signal after the filter processing is written into the frame memory buffer 14 in accordance with the sequential address, and then the frame memory buffer 14 is written in accordance with the read address obtained by converting the sequential address by the three-dimensional affine transformation. Since the output video signal is generated by reading the video signal from the output video signal, the output image Io represented by the output video signal gives a sense of depth as shown in FIG. 2 by the three-dimensional affine transformation. The resulting image has such special effects.
[0038]
In addition, since the input image has been subjected to the filter processing in advance, the aliasing in the reduced portion of the output image obtained by deforming the input image is suppressed. Further, the filter processing is not performed with uniform characteristics over the entire input image, but is performed with filter characteristics according to a reduction ratio expected (calculated based on an effect parameter) for each position. Since the processing is performed, it is possible to obtain an output image Io of higher image quality to which a special effect has been applied.
[0039]
Furthermore, instead of obtaining the reduction ratio at each position from the comparison between the output image and the input image, the reduction ratio at each position is predicted (calculated based on the effect parameter) before obtaining the output image, and Since the filter processing is performed with the filter characteristic according to the expected reduction ratio at each position, this image special effect device is applied to a moving image in which an input video signal is continuously processed for each frame. It is possible to apply special effects without impairing the real-time property.
[0040]
Next, an image special effect device according to a second embodiment of the present invention will be described.
[0041]
The image special effect device according to the second embodiment of the present invention is configured as shown in FIG. The image special effect device obtains an output image Io to which a special effect (hereinafter referred to as a page turn effect) such as turning a page as shown in FIG. 5 is applied by deforming an input image.
[0042]
The image special effect device according to the second embodiment shown in FIG. 3 is different from the image special effect device according to the second embodiment in that the CPU 15 outputs an effect parameter representing a deformation mode of an input image such that a page turn effect is obtained. A point that a page turn reduction rate generation unit 20 is provided in place of the unit 11, a point that the read address generation unit 13 generates a read address based on an effect parameter corresponding to the page turn effect, and that a background level output unit 21 is newly added. This is different from the image special effect device according to the first embodiment described above (see FIG. 1).
[0043]
The effect parameters supplied from the CPU 15 to the read address generation unit 13 and the page turn reduction ratio generation unit 20 are xc and yc representing the center position of the page turning effect (page turn effect) as shown in FIG. It comprises a radius r and an angle θ representing the direction x1 of the page turn effect.
[0044]
The page turn reduction rate generation unit 20 uses the effect parameters xc, yc, r, and θ from the CPU 15 every time an address corresponding to the pixel position (x, y) is input from the sequential address generation unit 10. The horizontal reduction ratio Rx and the vertical reduction ratio Ry at the pixel position (x, y) are calculated according to the following equation.
[0045]
(Equation 5)

[0046]
(Equation 6)

[0047]
(Equation 7)

[0048]
In the above formula, the area of x1 <0 represents a plane area in the image after the effect as shown in FIG. 5, and the area of 0 ≦ x1 ≦ r is an area of a curved surface in the image after the effect. Represents That is, the reduction ratio at each position of the region that becomes the flat portion after the effect is obtained as “1” (see Equation (6)), and the reduction ratio at each position of the curved surface region that is deformed after the effect is obtained. It depends on the degree of curvature r of the curved surface and the direction θ of the effect (see equation (7)).
[0049]
As in the first embodiment described above, the filter processing unit 12 performs, for a portion corresponding to each pixel position of the input video signal, a reduction supplied from the page turn reduction generation unit 20 for that position. Filter processing in the horizontal and vertical directions is performed using filter characteristics corresponding to the rates Rx and Ry. The video signal after the filtering process is written to the frame memory buffer 14 in accordance with the addresses sequentially output from the sequential address generation unit 10 as in the first embodiment described above.
[0050]
The read address generation unit 13 is configured, for example, as shown in FIG. 4, the read address generation unit 13 includes an effect direction position control unit 30, an upper surface change amount generation unit 31, a lower surface change amount generation unit 32, an upper surface x address generation unit 33, an upper surface y address generation unit 34, and a lower surface x address. It has a generation unit 35, a lower surface y address generation unit 36, and an effective area determination unit 37. Each time the address indicating the pixel position (x, y) is input from the sequential address generation unit 10, the effect direction position control unit 30 uses the above equation (5) using the effect parameters xc, yc, and θ supplied from the CPU 15. ), The input address (x, y) is converted into an address representing the position x1 in the direction θ of the effect based on the center (xc, yc) of the effect.
[0051]
The lower surface change amount generation unit 32 uses the address x1 after the coordinate conversion from the effect direction position control unit 30 and the effect parameter r from the CPU 15 to calculate the position x1 (address) on the image lower surface B (see FIG. 5). The amount of change Fa (x1) from the original position due to the curvature of r is calculated according to the following equation.
[0052]
(Equation 8)

[0053]
The change amount Fa (x1) has no curvature in the region of x1 <0 on the lower surface B of the image, and thus represents a change amount of 0 (no change). In the region of x1 ≧ 0, there is a curvature of r. Represents the amount of change based on.
[0054]
The lower surface x address generation unit 35 calculates the position x in the x direction in the coordinate system representing the pixel position (x, y) according to the following equation using the change amount Fa (x1) supplied from the lower surface change amount generation unit 32. The position x2 (x address) of the displacement destination is calculated.
[0055]
(Equation 9)

[0056]
In addition, the lower surface y address generation unit 36 calculates the pixel position (x, y) in the y direction in the coordinate system representing the pixel position (x, y) according to the following equation using the change amount Fa (x1) supplied from the lower surface change amount generation unit 32. The position y2 (y address) at which the position y changes is calculated.
[0057]
(Equation 10)

[0058]
As described above, when the read address generation unit 13 receives the address corresponding to the pixel position (x, y) sequentially supplied from the sequential address generation unit 10, the read address generation unit 13 obtains the expressions (5), (8) to (8). A lower surface read address corresponding to the position (x2, y2) on the lower surface B of the image after the effect as shown in 10) is generated.
[0059]
On the other hand, the upper surface change amount generation unit 31 uses the address x1 after coordinate conversion from the effect direction position control unit 30 and the effect parameter r from the CPU 15 to position x1 (see FIG. 5) on the image upper surface T (see FIG. 5). The amount of change Fb (x1) from the original position due to the curvature of r of the address) is calculated according to the following equation.
[0060]
[Equation 11]

[0061]
The change amount Fb (x1) represents a change amount from an original position determined by folding based on x1 = πr / 2 in a region of x1 <0 on the upper surface T of the image, and in a region of x1 ≧ 0, It represents the amount of change from the original position determined in consideration of both the curvature r and the return.
[0062]
The upper surface x address generation unit 33 calculates the position x in the x direction in the coordinate system representing the pixel position (x, y) according to the following equation using the variation Fb (x1) supplied from the upper surface variation generation unit 31. The position x3 (x address) of the displacement destination is calculated.
[0063]
(Equation 12)

[0064]
Also, the upper surface y address generation unit 34 calculates the pixel position (x, y) in the y direction in the coordinate system representing the pixel position (x, y) according to the following equation using the change amount Fb (x1) supplied from the upper surface change amount generation unit 31. The position y3 (y address) at the displacement destination of the position y is calculated.
[0065]
(Equation 13)

[0066]
As described above, when the read address generation unit 13 receives the address corresponding to the pixel position (x, y) sequentially supplied from the sequential address generation unit 10, the read address generation unit 13 obtains the expressions (5), (11) to (11). An upper surface read address corresponding to the position (x3, y3) on the upper surface T of the image after the effect as shown in 13) is generated.
[0067]
The characteristics of the change amount Fa (x1) applied to the lower surface B of the image and the change amount Fb (x1) applied to the upper surface T of the image after the above effects are shown in FIGS. It becomes as shown in.
[0068]
The effective area determination unit 37 determines whether the upper surface read address (x3, y3) generated by the upper surface x address generation unit 33 and the upper surface y address generation unit 34 exists in the screen. Next, the effective area determination unit 37 determines whether or not the lower surface read address (x2, y2) generated by the lower surface x address generation unit 35 and the lower surface y address generation unit 36 exists in the screen. .
[0069]
When it is determined that the upper surface read address (x3, y3) exists in the screen, the effective area determination unit 37 sets the upper surface read address (x3, y3) as the read address (X, Y). And a key signal representing "1". If it is determined that the upper surface read address (x3, y3) does not exist in the screen and the lower surface read address (x2, y2) exists in the screen, the effective area determination unit 37 reads the lower surface read address. The address (x2, y2) is output as the read address (X, Y), and a key signal representing "1" is output.
[0070]
On the other hand, when it is determined that both the upper surface read address (x3, y3) and the lower surface read address (x2, y2) do not exist in the screen, the valid area determination unit 37 sets the key representing “0”. (Key) signal is output. The key signal representing "0" is a signal representing a background portion G (a portion other than the upper surface T or the lower surface B) in the screen as shown in FIG.
[0071]
As described above, every time the read address generating unit 13 inputs an address representing a pixel position (x, y) sequentially generated from the sequential address generating unit 10, the read address (X, Y) and the key ( key) signal.
[0072]
Referring back to FIG. 3, the background level output unit 21 receives a video signal read from the frame memory buffer 14 in accordance with the read address (X, Y), and outputs a key signal output from the read address generation unit 13. If it is "1", the video signal is output as it is. On the other hand, when the key signal is “0”, a signal representing a background level (for example, black) is output instead of the video signal. The signal output from the background level output unit 21 is an output video signal.
[0073]
In the image special effect device as described above, the reduction ratio at each pixel position of the input image (input video signal for one frame) by the page turn effect is generated (see equations (5), (6), and (7)). ), The input video signal is filtered with filter characteristics determined based on the reduction ratio. Then, the input video signal after the filtering is written to the frame memory buffer 14 in accordance with the addresses sequentially output from the sequential address generation unit 10. Thereafter, the video signal is read out from the frame memory buffer 14 in accordance with the read address (see Equations (8) to (13)) obtained by converting the sequential address based on the page turn effect, and the background image signal is converted to the background image signal. An output video signal is generated by performing the replacement.
[0074]
Therefore, the output image Io represented by the output video signal is an image to which a special effect such as turning a page as shown in FIG. 5 has been applied by the page turn effect. Further, the image special effect device according to the second embodiment also has a filter with a filter characteristic determined based on the reduction ratio for each pixel position of the input image, similarly to the first embodiment. By performing the process, an output image Io of higher image quality can be obtained, and the image is suitable for a moving image.
[0075]
Next, an image special effect device according to a third embodiment of the present invention will be described.
[0076]
The image special effect device according to the third embodiment of the present invention is configured as shown in FIG. This image special effect device transforms an input image into an output image Io to which both a special effect (three-dimensional affine transformation) for giving a sense of depth as shown in FIG. 8 and a page turn effect are applied. What you get.
[0077]
The image special effect device according to the third embodiment shown in FIG. 7 is different from the image special effect device according to the third embodiment in that the CPU 15 outputs both the effect parameter of the three-dimensional affine transformation and the effect parameter of the page turn effect. The point that both the reduction rate generation unit 11 and the page turn reduction rate generation unit 20 are provided is that the read address generation unit 13 reads the read address based on both the effect parameter of the three-dimensional affine transformation and the effect parameter relating to the page turn effect. And a combining unit 40 that combines the reduction rate output from the three-dimensional affine transformation reduction rate generation unit 11 and the reduction rate output from the page turn reduction generation unit 20 to output a combined reduction rate. The image special effect device according to the first and second embodiments described above 1, see FIG. 3) and different.
[0078]
The three-dimensional affine transformation reduction ratio generation unit 11 performs the effect of the three-dimensional affine transformation supplied from the CPU 15 every time an address corresponding to the pixel position (x, y) sequentially output from the sequential address generation unit 10 is input. Using the parameters (Ax, Ay, Bx, By, Cx, Cy, D, E), the horizontal reduction ratio RxA at each pixel position (Rx in Expression (1)) at each pixel position according to the above Expressions (1) and (2). ) And the reduction ratio RyA in the vertical direction (Ry in equation (2)). Also, the page turn reduction rate generation unit 20 converts the effect parameters (r, θ, xc, yc) relating to the page turn effect supplied from the CPU 15 every time an address sequentially output from the sequential address unit 10 is input. Using the above equations (5), (6), and (7), the horizontal reduction rate RxB (Rx in equation (7)) and the vertical reduction rate RyB (Ry in equation (7)) at each pixel position are used. ) Is calculated.
[0079]
The synthesizing unit 40 includes a horizontal reduction ratio RxA and a vertical reduction ratio RyA from the three-dimensional affine transformation reduction ratio generation unit 11, a horizontal reduction ratio RxB from the page turn reduction ratio generation unit 20, Using the vertical reduction ratio RyB, the horizontal reduction ratio Rx and the vertical reduction ratio Ry are calculated according to the following equations.
[0080]
[Equation 14]

[0081]
In this manner, every time the sequential address generation unit 10 sequentially outputs an address corresponding to a pixel position (x, y), the synthesizing unit 40 sets the horizontal synthesis reduction ratio Rx and The composite reduction ratio Ry in the vertical direction is output. Then, the filter processing unit 12 uses a horizontal filter and a vertical filter having filter characteristics determined based on the composite reduction ratios Rx and Ry in accordance with the same method as in the first and second embodiments described above. Performs filter processing on the video signal. Then, the video signal for one frame after the filter processing is sequentially sent from the sequential address generation unit 10 in consideration of the processing time in the page turn reduction rate generation unit 20, the synthesis unit 40, and the filter processing unit 12 as a delay. Are sequentially written to the frame memory buffer 14 according to the address corresponding to the pixel position (x, y) output to the frame memory buffer 14.
[0082]
Each time the read address generation unit 13 inputs an address corresponding to a pixel position (x, y) sequentially output from the sequential address generation unit 10, the read address generation unit 13 first performs a three-dimensional affine according to the above equations (3) and (4). The converted addresses XA (X in equation (3)) and YA (Y in equation (4)) are generated.
[0083]
Processing corresponding to the page turn effect is performed on the address (XA, XB) generated in this manner.
[0084]
That is, the following equation corresponding to the above equation (5)
[0085]
[Equation 15]

[0086]
, The address (XA, YA) is converted into an address representing the position x1 in the direction θ of the page turn effect with respect to the center (xc, yc) of the effect. Then, the lower surface read address (x2, y2) is generated according to the above equation (8) and the following equation.
[0087]
(Equation 16)

[0088]
[Equation 17]

[0089]
Further, the upper surface read address (x3, y3) is generated according to the above equation (11) and the following equation.
[0090]
(Equation 18)

[0091]
[Equation 19]

[0092]
Then, as in the case of the second embodiment, the lower surface read address (x2, y2) and the upper surface read address (x3, y3) are obtained by the effective area determination unit 37 (see FIG. 4) of the read address generation unit 13. ) Is selected as the final read address (X, Y), and a key signal is generated. In accordance with the read address output from the read address generating unit 13, the video signal stored in the frame memory buffer 14 is read as described above, and the signal of the background image is generated based on the key signal "0". And a final output video signal is generated.
[0093]
In the image special effect device as described above, an address (XA, YA) obtained by coordinate conversion by three-dimensional affine transformation from an address (x, y) sequentially output from the sequential address generation unit 10 is further converted to a page. The read address of the address (x2, y2) or (x3, y3) is converted by a conversion process based on the turn effect. Accordingly, an output video signal obtained by reading out the video signals stored in the frame memory buffer 14 in accordance with the sequentially generated addresses (x, y) in accordance with the readout address gives a sense of depth as shown in FIG. An output image Io to which both special effects (three-dimensional affine transformation) and page turn effects are applied is shown.
[0094]
Further, the reduction ratio at each pixel position due to the deformation of the input image due to the above three-dimensional affine transformation and the reduction ratio at each pixel position due to the deformation of the input image due to the page turn effect are synthesized and obtained. The filter processing unit 12 performs a filter process on each pixel position of the input image with a filter characteristic determined based on the synthesis reduction ratio. For this reason, distortion (foldback distortion) in the output image Io to which both the special effect (three-dimensional affine transformation) that gives a sense of depth and the page turn effect as described above are applied is suppressed. .
[0095]
Further, an image special effect device according to a fourth embodiment of the present invention will be described.
[0096]
An image special effect device according to the fourth embodiment of the present invention is configured as shown in FIG. This image special effect device according to the first embodiment is different from the first embodiment in that a decoding unit 50 for converting the reduction ratio from the three-dimensional affine transformation reduction ratio generation unit 11 into information for specifying a filter characteristic is newly provided. This is different from the image special effect device (see FIG. 1).
[0097]
The decoding unit 50 outputs the pixel position (x, y) generated by the three-dimensional affine transformation reduction rate generation unit 11 for each address corresponding to the pixel position (x, y) sequentially output from the sequential address generation unit 10. The horizontal reduction ratio Rx and the vertical reduction ratio Ry in y) are converted into filter characteristic designation information Decx and Decy in accordance with the following equation.
[0098]
(Equation 20)

[0099]
(Equation 21)

[0100]
In each of the above equations (20) and (21), int () represents a quotient in the division in ().
[0101]
The filter processing unit 12 has a horizontal filter having a plurality of filter characteristics and a vertical filter having a plurality of filter characteristics, and a relationship between the plurality of filter characteristics of the horizontal filter and each value of the designation information Decx. And a relationship between a plurality of filter characteristics of the vertical filter and each value of the designation information Decy.
[0102]
In the image special effect device as described above, every time an address corresponding to a pixel position (x, y) is output from the sequential address generation unit 10, the reduction ratio generated by the three-dimensional affine transformation reduction ratio generation unit 11 is output. The designation information Decx, Decy corresponding to Rx, Ry is output from the decoding unit 50. In the filter processing unit 12 for inputting the respective specification information Decx and Decy, the horizontal direction filter performs the filtering on the portion corresponding to the pixel position (x, y) of the input video signal by the filter characteristic specified by the specification information Decx. In addition to performing the filter processing, the vertical filter performs the filter processing on the portion corresponding to the pixel position (x, y) of the input video signal with the filter characteristics specified by the specification information Decy. As a result, similarly to the image special effect device according to the first embodiment, it is possible to obtain an output image Io to which a special effect giving a sense of depth is applied by three-dimensional affine transformation as shown in FIG. .
[0103]
According to this image special effect device, in addition to obtaining the same effects as those of the above-described first embodiment, it is possible to efficiently perform the processing related to the selection of the filter characteristics in the filter processing unit 12. become.
[0104]
Note that the decoding unit 50 as described above can be applied to the image special effect device according to the third embodiment described above. In this case, the decoding unit 50 converts the combined reduction ratios Rx and Ry output from the combining unit 40 into designated information Decx and Decy, and supplies the designated information Decx and Decy to the filter processing unit 12.
[0105]
In the image special effect device according to each of the above-described embodiments, the reduction ratio is generated for each pixel of the input image. However, in the present invention, the reduction ratio (deformation degree information) is not necessarily set for each pixel of the input image. May not be generated. From the viewpoint that efficient processing can be performed, a reduction ratio (deformation degree information) can be generated for each region formed by a plurality of pixels. In this case, a filtering process is performed on a position included in the region with a filter characteristic determined based on the reduction ratio (deformation degree information) generated for the region.
[0106]
Further, the image special effect device according to each of the above-described embodiments is applied to processing of a moving image, but can be applied to a still image as a matter of course.
[0107]
Furthermore, a description has been given of the deformation that gives a sense of depth to the input image (see FIGS. 2 and 8) and the deformation that turns the page of the input image (see FIGS. 5 and 8). Is not limited to this. The same processing is performed for arbitrary deformations (for example, a deformation (roll turn) involving the folded end of the image, a deformation involving the image in a cylinder, etc.) as long as the mode of the deformation can be represented as effect information. It can be carried out.
[0108]
【The invention's effect】
As described above, according to the present invention, according to the present invention, the filter characteristics are determined based on the deformation degree information at each position of the input image generated based on the effect information representing the mode of deforming the input image. Since the input image is subjected to the filtering process and the output image obtained by deforming the input image in the above-described manner is obtained from the image information after the filtering process, a special effect obtained by deforming the input image is applied. It is possible to provide an image special effect device capable of suppressing the deterioration of the output image quality.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an image special effect device according to a first embodiment of the present invention.
FIG. 2 is a diagram showing a first example of an image to which a special effect has been applied;
FIG. 3 is a block diagram showing an image special effect device according to a second embodiment of the present invention.
FIG. 4 is a block diagram showing a configuration example of a read address generation unit in the image special effect device shown in FIG. 3;
FIG. 5 is a diagram showing a second example of an image to which a special effect has been applied;
6 is a characteristic diagram illustrating an example of a characteristic of a change amount in the image deformation illustrated in FIG. 5;
FIG. 7 is a block diagram showing an image special effect device according to a third embodiment of the present invention.
FIG. 8 is a diagram showing a third example of an image to which a special effect has been applied;
FIG. 9 is a block diagram showing an image special effect device according to a fourth embodiment of the present invention.
[Explanation of symbols]
10 Sequential address generation unit
11 3D affine transformation reduction ratio generation unit
12 Filter processing unit
13 Read address generation unit
14 Frame memory buffer
15 CPU
20 page turn reduction rate generation unit
21 Background level output unit
30 Effect direction position control unit
31 Top surface change generation unit
32 Lower surface change generation unit
33 Top surface x address generation unit
34 Top y address generation unit
35 Lower surface x address generation unit
36 bottom y address generation unit
37 Effective area judgment unit
40 synthesis unit
50 decoding unit

Claims (6)

The image information of the input image stored in the memory buffer according to the predetermined address order is read in accordance with the read address generated based on the effect information representing the mode of deforming the input image, whereby the input image is read in the mode. An image special effect device configured to obtain a deformed output image,
Deformation degree information generating means for generating deformation degree information indicating the degree of deformation at each position of the input image associated with the deformation of the input image in the aspect using the effect information,
Filter processing means for performing a filter process with a filter characteristic determined based on the deformation degree information generated by the deformation degree information generation means for each position of the input image,
An image special effect device, wherein image information of an input image filtered by the filter processing means is stored in the memory buffer in the predetermined address order. 2. The image special effect device according to claim 1, wherein the deformation degree information generating means generates the deformation degree information for each position of the input image for each pixel. 3. The image specialization apparatus according to claim 1, wherein the deformation degree information generation unit includes a reduction ratio calculation unit that calculates a reduction ratio at each position of the input image as deformation degree information using the effect information. 4. Effect device. To generate a read address for the memory buffer based on the effect information representing a plurality of modified aspects of the input image,
The deformation degree information generating means generates deformation degree information at each position of the input image associated with the deformation of the input image in each mode using the effect information representing the plurality of deformation modes. Information generation means;
Synthesizing means for generating deformation degree information for each position by synthesizing deformation degree information for each position of the input image obtained by the plurality of mode-specific deformation degree information generating means,
2. The filter processing unit according to claim 1, wherein the filter processing unit performs a filter process on each position of the input image with a filter characteristic determined based on the synthetic deformation degree information generated by the synthesis unit. 4. The image special effect device according to any one of claims 1 to 3. The filter processing means has a plurality of filter characteristics determined corresponding to the deformation degree information, and is generated by the deformation degree information generation means among the plurality of filter characteristics for each position of the input image. The image special effect device according to any one of claims 1 to 4, wherein a filter characteristic corresponding to the information on the degree of deformation is selected, and a filter process is performed using the selected filter characteristic. Designation information generating means for converting deformation degree information for each position of the input image generated by the deformation degree information generating means into specification information for specifying a corresponding filter characteristic in the filter processing means,
The filter processing unit selects a filter characteristic specified by the specification information generated by the specification information generation unit, and performs a filter process on each position of the input image with the selected filter characteristic. 6. The image special effect device according to claim 5, wherein:

Images