JP2004240732A - Image compositing method, image compositing device, image compositing program and image recording device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To alleviate the artificiality of a composite image by a method having a small number of steps. <P>SOLUTION: An image compositing means 30 composites image data G1 and G2 of an original image to create image data P of a composite image, and creates composite end part information. A post-composite processing means 40 determines image converting conditions for conversion to the image data P of the composite image using the composite end part information obtained by the image compositing means 30, and converts the image data P of the composite image based on the determined image converting conditions to create image data P' of the composite image after post-processing. For example, the blurring processing by a space filter is conducted in an image position described in the composite end part information. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

【数２】

【数３】

上記式（３）で、ａはウェーブレット関数のスケールを表し、ｂはウェーブレット関数の位置を示す。図１３に例示するように、スケールａの値が大きいほどウェーブレット関数ψ_ａ，ｂ（ｘ）の周波数は小さくなり、また位置ｂの値に従ってウェーブレット関数ψ_ａ，ｂ（ｘ）が振動する位置が移動する。従って上記式（３）は、入力信号ｆ（ｘ）を種々のスケールと位置を持つウェーブレット関数ψ_ａ，ｂ（ｘ）の総和に分解することを意味している。
【０１１３】
ウェーブレット変換を行う際に用いられるウェーブレット関数としては、多くのものが知られているが、画像処理分野では特に計算を高速に行うことができる直交ウェーブレット（ｏｒｔｈｏｇｏｎａｌ ｗａｖｅｌｅｔ）変換、双直交ウェーブレット（ｂｉｏｒｔｈｏｇｏｎａｌ ｗａｖｅｌｅｔ）変換が広く用いられている。
【０１１４】
以下、直交ウェーブレット変換、双直交ウェーブレット変換の概要を説明する。直交ウェーブレット変換及び双直交ウェーブレット変換のウェーブレット関数は下記式（４）のように定義される。
【数４】

ただし、ｉは自然数である。
【０１１５】
前記式（４）と前記式（１）とを比較すると、直交ウェーブレット変換、双直交ウェーブレット変換では、スケールａの値が２のｉ乗で離散的に定義され、また位置ｂの最小移動単位が２^ｉで離散的に定義されていることが判る。このｉの値はレベルと呼ばれる。
【０１１６】
また実用的にはレベルｉを有限な上限Ｎまでに制限して、入力信号を下記式（５）、式（６）、式（７）のように変換することが行われる。
【数５】

【数６】

【数７】

【０１１７】
上記式（５）の第２項は、レベル１のウェーブレット関数ψ_１，ｊ（ｘ）の総和で表せない残差の低周波数帯域成分を、レベル１のスケーリング関数φ_１，ｊ（ｘ）の総和で表したものである。スケーリング関数はウェーブレット関数に対応して適切なものを用いる。上記式（５）に示す１レベルのウェーブレット変換により入力信号ｆ（ｘ）≡Ｓ_０は、レベル１の高周波数帯域成分Ｗ_１と低周波数帯域成分Ｓ_１に信号分解されたことになる。
【０１１８】
なお、画像においては、高周波数帯域成分は、例えば、髪の毛やまつ毛のような微細な構造を表現する成分であり、低周波数帯域成分は、例えば、頬のように信号強度の変化の緩やかな構造を示す成分である。
【０１１９】
ウェーブレット関数ψ_ｉ，ｊ（ｘ）の最小移動単位単位は２^ｉなので、入力信号Ｓ_０の信号量に対して高周波数帯域成分Ｗ_１と低周波数帯域成分Ｓ_１の信号量は各々１／２となり、高周波数帯域成分Ｗ_１と低周波数帯域成分Ｓ_１の信号量の総和は、入力信号Ｓ_０の信号量と等しくなる。レベル１の低周波数帯域成分Ｓ_１は式（６）において、レベル２の高周波数帯域成分Ｗ_２と低周波数帯域成分Ｓ_２に分解され、以下同様にレベルＮ迄の変換を繰り返すことで、入力信号Ｓ_０は、式（７）に示すように各レベル１〜Ｎの高周波数帯域成分の総和とレベルＮの低周波数帯域成分の和に分解される。
【０１２０】
ここで、前記式（６）で示す１レベルのウェーブレット変換は、図１４に示すようなフィルタ処理で計算することができる。図１４において、ＬＰＦはローパスフィルタ、ＨＰＦはハイパスフィルタを示している。ローパスフィルタＬＰＦとハイパスフィルタＨＰＦのフィルタ係数は、ウェーブレット関数に応じて適切に定められる。また、２↓は、信号を１つおきに間引くダウンサンプリング処理を示す。
【０１２１】
画像信号のような２次元信号における１レベルのウェーブレット変換は、図１５に示すようなフィルタ処理で計算される。図１５に示すように、まず低周波数帯域成分Ｓ_ｎ−１をｘ方向のローパスフィルタＬＰＦｘ、ハイパスフィルタＨＰＦｘによりフィルタ処理を行い、ｘ方向にダウンサンプリング処理を行う。これにより入力信号Ｓ_ｎ−１はＳＸ_ｎとＷＸ_ｎとに分解される。このＳＸ_ｎとＷＸ_ｎに対して、それぞれｙ方向のローパスフィルタＬＰＦｙ、ハイパスフィルタＨＰＦｙによるフィルタ処理を行い、ｙ方向にダウンサンプリング処理を行う。
【０１２２】
この１レベルのウェーブレット変換により、低周波数帯域成分Ｓ_ｎ−１は３つの高周波数帯域成分Ｗｖ_ｎ，Ｗｈ_ｎ，Ｗｄ_ｎと１つの低周波数帯域成分Ｓ_ｎに分解される。１回のウェーブレット変換により生成されるＷｖ_ｎ，Ｗｈ_ｎ，Ｗｄ_ｎ，Ｓ_ｎの各々の信号量は、分解前のＳ_ｎ−１に比べて縦横ともに１／２となるので、分解後の４成分の信号量の総和は、分解前のＳ_ｎ−１の信号と等しくなる。
【０１２３】
なお、図１５において、ＬＰＦｘ，ＨＰＦｘ，２↓ｘのように添え字で示したｘはｘ方向の処理を示し、ＬＰＦｙ，ＨＰＦｙ，２↓ｙのように添え字で示したｙはｙ方向の処理を示す。
【０１２４】
図１６に、入力信号Ｓ_０が３レベルのウェーブレット変換で信号分解される過程の模式図を示す。レベル数ｉが増えるに従って、ダウンサンプリング処理により画像信号が間引かれ、分解画像が小さくなっていくことがわかる。
【０１２５】
また、図１７に示すように、分解で生成したＷｖ_ｎ，Ｗｈ_ｎ，Ｗｄ_ｎ，Ｓ_ｎにフィルタ処理で計算されるウェーブレット逆変換をほどこすことにより、分解前の信号Ｓ_ｎ−１を完全再構成できることが知られている。図１７において、ＬＰＦ’は逆変換用のローパスフィルタ、ＨＰＦ’は逆変換用のハイパスフィルタを示している。また、２↑は、信号に１つおきにゼロを挿入するアップサンプリング処理を示す。また、ＬＰＦ’ｘ，ＨＰＦ’ｘ，２↑ｘのように添え字で示したｘはｘ方向の処理を示し、ＬＰＦ’ｙ，ＨＰＦ’ｙ，２↑ｙのように添え字で示したｙはｙ方向の処理を示す。
【０１２６】
図１７に示すように、Ｓ_ｎをｙ方向にアップサンプリング処理及び逆変換用のローパスフィルタＬＰＦ’ｙによるフィルタ処理を施すことにより得られる信号と、Ｗｈ_ｎをｙ方向にアップサンプリング処理及び逆変換用のハイパスフィルタＨＰＦ’ｙによるフィルタ処理を施すことにより得られる信号とを加算してＳＸ_ｎを得る。これと同様にして、Ｗｖ_ｎとＷｄ_ｎからＷＸ_ｎを生成する。
【０１２７】
さらに、ＳＸ_ｎをｘ方向にアップサンプリング処理及び逆変換用のローパスフィルタＬＰＦ’ｘによるフィルタ処理を施すことにより得られる信号と、ＷＸ_ｎをｘ方向にアップサンプリング処理及び逆変換用のハイパスフィルタＨＰＦ’ｘによるフィルタ処理を施すことにより得られる信号とを加算することにより分解前の信号Ｓ_ｎ−１を得ることができる。
【０１２８】
この逆変換の際に用いられるフィルタ係数は、直交ウェーブレットの場合にはウェーブレット変換に用いた係数と同じ係数が使用されるが、双直交ウェーブレットの場合にはウェーブレット変換に用いた係数とは異なる係数が使用される。
【０１２９】
図１２に示すように、歪み量算出手段４２０は、直交ウェーブレット変換手段４２１と、歪みデータ抽出手段４２２と、歪み量ランク分け手段４２３と、から構成される。
直交ウェーブレット変換手段４２１は、合成画像の輝度データＢに２レベルの直交ウェーブレット変換を施す。すなわち、図１５に示すような直交ウェーブレット変換を２回繰り返すことにより、レベル１の高周波数帯域成分Ｗｖ_１，Ｗｈ_１，Ｗｄ_１と、レベル２の高周波数帯域成分Ｗｖ_２，Ｗｈ_２，Ｗｄ_２と低周波数帯域成分Ｓ_２が得られる。
【０１３０】
歪みデータ抽出手段４２２は、輝度データＢから求められた高周波数帯域成分Ｗｖ_１，Ｗｈ_１，Ｗｄ_１，Ｗｖ_２，Ｗｈ_２，Ｗｄ_２から歪みデータを抽出する。歪みデータとは、例えば、合成端部情報に記された合成端部の画素位置における高周波数帯域成分Ｗｖ_１，Ｗｈ_１，Ｗｄ_１，Ｗｖ_２，Ｗｈ_２，Ｗｄ_２の画像データ値である。
【０１３１】
歪み量ランク分け手段４２３は、この高周波数帯域成分Ｗｖ_１，Ｗｈ_１，Ｗｄ_１，Ｗｖ_２，Ｗｈ_２，Ｗｄ_２の画像データ値に応じてランク分けし、歪み量とする。なお、画像データ値をそのままランク値としてもよい。
【０１３２】
画像データ変換手段４３０は、ランク別画像データ変換手段４３１と、直交ウェーブレット逆変換手段４３２と、から構成される。
【０１３３】
ランク別画像データ変換手段４３１は、例えば、歪み情報に記された画素位置において、高周波数帯域成分Ｗｖ_１，Ｗｈ_１，Ｗｄ_１，Ｗｖ_２，Ｗｈ_２，Ｗｄ_２の画像データを減少させる処理を行う。このとき、高周波数帯域成分Ｗｖ_１，Ｗｈ_１，Ｗｄ_１，Ｗｖ_２，Ｗｈ_２，Ｗｄ_２の画像データの減少比は、歪み量のランクに応じて変える。高周波数帯域成分Ｗｖ_１，Ｗｈ_１，Ｗｄ_１，Ｗｖ_２，Ｗｈ_２，Ｗｄ_２は、それぞれ、変換後の高周波数帯域成分Ｗｖ_１’，Ｗｈ_１’，Ｗｄ_１’，Ｗｖ_２’，Ｗｈ_２’，Ｗｄ_２’に変換される。
【０１３４】
次に、直交ウェーブレット逆変換手段４３２は、前記変換後の高周波数帯域成分Ｗｖ_２’，Ｗｈ_２’，Ｗｄ_２’と低周波数帯域成分Ｓ_２に対して、図１７に示すような直交ウェーブレット逆変換を行い、低周波数帯域成分Ｓ_１’を得る。続いて、変換後の高周波数帯域成分Ｗｖ_１’，Ｗｈ_１’，Ｗｄ_１’と低周波数帯域成分Ｓ_１’に対して直交ウェーブレット逆変換を行い、修正後の輝度データＢ’を得ることができる。
【０１３５】
合成後処理手段４０の詳細例４では、直交ウェーブレット変換により得られた解像度が異なる複数の画像データを用いて、特定の位置に特定の周波数帯域への処理を行うことで、より精度よく、歪みを修正することができる。そのため、画像全体の印象を保ったまま効率的に合成端部の不自然さを軽減させるのに効果的であり、好ましい。また、直交ウェーブレット変換を用いているため、処理を高速に行うことができ、好ましい。
また、直交ウェーブレット変換された画像データを、合成画像データの歪み量の算出に用いるとともに、歪み量に基づく画像データ変換に用いることにより、処理を効率化することができる。
【０１３６】
なお、合成後処理手段４０の詳細例４において行われる直交ウェーブレット変換は双直交ウェーブレット変換でもよい。
【０１３７】
（合成後処理手段４０の詳細例５）
図１８に、合成後処理手段４０の詳細例５を示す。合成後処理手段４０は、輝度／色差変換手段４１０と、歪み量算出手段４２０と、画像データ変換手段４３０と、輝度／色差逆変換手段４４０と、を備える。
【０１３８】
合成後処理手段４０の詳細例５は、合成後処理手段４０の詳細例３における輝度データＢの中間データＢｔｍｐ１として、二項ウェーブレット変換を施して得られた高周波数帯域成分Ｗｘ_ｎ，Ｗｙ_ｎを用いた例である。
輝度／色差変換手段４１０、輝度／色差逆変換手段４４０については、合成後処理手段４０の詳細例２と同様であるので、説明を省略する。
【０１３９】
ここで、二項ウェーブレット変換について概要を説明する。二項ウェーブレット（Ｄｙａｄｉｃ Ｗａｖｅｌｅｔ）変換は、直交ウェーブレット変換や双直交ウェーブレットと比較すると、画像を間引くことがないので、より高精細に画像を処理することができる。二項ウェーブレット変換については“Ｓｉｎｇｕｌａｒｉｔｙ ｄｅｔｅｃｔｉｏｎ ａｎｄ ｐｒｏｃｅｓｓｉｎｇ ｗｉｔｈ ｗａｖｅｌｅｔｓ” ｂｙ Ｓ．Ｍａｌｌａｔ ａｎｄ Ｗ．Ｌ．Ｈｗａｎｇ， ＩＥＥＥ Ｔｒａｎｓ． Ｉｎｆｏｒｍ． Ｔｈｅｏｒｙ ３８ ６１７ （１９９２）や“Ｃｈａｒａｃｔｅｒｉｚａｔｉｏｎ ｏｆ ｓｉｇｎａｌｓ ｆｒｏｍ ｍｕｌｔｉｓｃａｌｅ ｅｄｇｅｓ” ｂｙ Ｓ．Ｍａｌｌａｔ ａｎｄ Ｓ．Ｚｈｏｎｇ， ＩＥＥＥ Ｔｒａｎｓ． Ｐａｔｔｅｒｎ Ａｎａｌ． Ｍａｃｈｉｎｅ Ｉｎｔｅｌ． １４７１０ （１９９２） や“Ａ ｗａｖｅｌｅｔ ｔｏｕｒ ｏｆ ｓｉｇｎａｌ ｐｒｏｃｅｓｓｉｎｇ ２ｅｄ” ｂｙ Ｓ．Ｍａｌｌａｔ， Ａｃａｄｅｍｉｃ Ｐｒｅｓｓに詳細な説明がある。
【０１４０】
二項ウェーブレット変換のウェーブレット関数は下記式（８）のように定義される。
【数８】

ただし、ｉは自然数である。
【０１４１】
前述した直交ウェーブレット変換や双直交ウェーブレット変換のウェーブレット関数はレベルｉにおける位置の最小移動単位が２^ｉで離散的に定義されていたのに対し、二項ウェーブレット変換のウェーブレット関数はレベルｉにかかわらず位置の最小移動単位が一定である。この相違により、二項ウェーブレット変換には下記の特徴が生じる。
【０１４２】
第一の特徴として、下記式（９）に示す１レベルの二項ウェーブレット変換で生成する、高周波数帯域成分Ｗ_ｉと低周波数帯域成分Ｓ_ｉの各々の信号量は、変換前の信号Ｓ_ｉ−１と同一である。
【数９】

【０１４３】
第二の特徴として、スケーリング関数φ_ｉ，ｊ（ｘ）とウェーブレット関数ψ_ｉ，ｊ（ｘ）の間に下記式（１０）の関係が成立する。
【数１０】

したがって、二項ウェーブレット変換で生成される高周波数帯域成分Ｗ_ｉは、低周波数帯域成分Ｓ_ｉの一回微分（勾配）を表す。
【０１４４】
第三の特徴として、ウェーブレット変換のレベルｉに応じて定められた係数γ_ｉを高周波数帯域成分に乗じたＷ_ｉ・γ_ｉ（以下、これを補正済高周波数帯域成分という。）について、入力信号の信号変化の特異性（ｓｉｎｇｕｌａｒｉｔｙ）に応じて、該変換後の補正済高周波数帯域成分Ｗ_ｉ・γ_ｉの信号強度のレベル間の関係が一定の法則に従う。すなわち、図１９の“１”や“４”に示すようななだらかな（微分可能な）信号変化に対応する補正済高周波数帯域成分Ｗ_ｉ・γ_ｉはレベル数ｉが増大するほど信号強度が増大するのに対して、図１９の“２”に示すステップ状の信号変化に対応する補正済高周波数帯域成分Ｗ_ｉ・γ_ｉはレベル数ｉに関わらず信号強度が一定となり、図１９の“３”に示すδ関数状の信号変化に対応する補正済高周波数帯域成分Ｗ_ｉ・γ_ｉはレベル数ｉが増大するほど信号強度が減少する。
【０１４５】
第四の特徴として、画像信号のような２次元信号における１レベルの二項ウェーブレット変換の方法は、前述の直交ウェーブレット変換や双直交ウェーブレット変換の方法と異なり、図２０に示すような方法で行われる。図２０に示すように、１レベルのウェーブレット変換において、低周波数帯域成分Ｓ_ｎ−１をｘ方向のローパスフィルタＬＰＦｘ及びｙ方向のローパスフィルタＬＰＦｙで処理することにより、低周波数帯域成分Ｓ_ｎが得られる。また、低周波数帯域成分Ｓ_ｎ−１をｘ方向のハイパスフィルタＨＰＦｘで処理することにより、高周波数帯域成分Ｗｘ_ｎが得られ、ｙ方向のハイパスフィルタＨＰＦｙで処理することにより、高周波数帯域成分Ｗｙ_ｎが得られる。
【０１４６】
このように、１レベルの二項ウェーブレット変換により、低周波数帯域成分Ｓ_ｎ−１は、２つの高周波数帯域成分Ｗｘ_ｎ，Ｗｙ_ｎと１つの低周波数帯域成分Ｓ_ｎに分解される。２つの高周波数帯域成分Ｗｘ_ｎ，Ｗｙ_ｎは低周波数帯域成分Ｓ_ｎの２次元における変化ベクトルＶ_ｎのｘ成分とｙ成分に相当する。変化ベクトルＶ_ｎの大きさＭ_ｎと偏角Ａ_ｎは下記式（１１）、式（１２）で与えられる。
【数１１】

【数１２】

【０１４７】
二項ウェーブレット変換で得られた２つの高周波数帯域成分Ｗｘ_ｎ，Ｗｙ_ｎと１つの低周波数帯域成分Ｓ_ｎに、図２１に示す二項ウェーブレット逆変換を施すことにより、変換前のＳ_ｎ−１を再構成することができる。すなわち、低周波数帯域成分Ｓ_ｎに対してｘ方向におけるローパスフィルタＬＰＦｘ及びｙ方向におけるローパスフィルタＬＰＦｙで処理することにより得られる信号と、高周波数帯域成分Ｗｘ_ｎに対してｘ方向における逆変換用のハイパスフィルタＨＰＦ’ｘ及びｙ方向における逆変換用のローパスフィルタＬＰＦ’ｙで処理することにより得られる信号と、高周波数帯域成分Ｗｙ_ｎに対してｘ方向における逆変換用のローパスフィルタＬＰＦ’ｘ及びｙ方向における逆変換用のハイパスフィルタＨＰＦ’ｙで処理することにより得られる信号と加算することにより分解前の信号Ｓ_ｎ−１を得ることができる。
【０１４８】
図１８に示すように、歪み量算出手段４２０は、二項ウェーブレット変換手段４２４と、歪みデータ抽出手段４２２と、歪み量ランク分け手段４２３と、から構成される。
画像データ変換手段４３０は、ランク別画像データ変換手段４３１と、二項ウェーブレット逆変換手段４３３と、から構成される。
【０１４９】
合成後処理手段４０の詳細例５は、合成後処理手段４０の詳細例４における直交ウェーブレット変換を二項ウェーブレット変換に置き換えたもので、扱うデータが、二項ウェーブレット変換を施して得られた高周波数帯域成分Ｗｘ_ｎ，Ｗｙ_ｎであること以外は、合成後処理手段４０の詳細例４と同様であるので、説明を省略する。
【０１５０】
合成後処理手段４０の詳細例５では、二項ウェーブレット変換により得られた解像度が異なる複数の画像データを用いて、特定の位置に特定の周波数帯域への処理を行うことで、より精度よく、歪みを修正することができる。そのため、より精度よく合成画像の不自然さを軽減させることができ、好ましい。また、部分処理であるから、画像全体の印象を保つことができ、合成画像の微修正に適しているとともに、極めて少ない作業工数で効率的に処理することができる。
【０１５１】
また、合成後処理手段４０の詳細例５で用いた二項ウェーブレット変換は、変換時に画像を間引くことがないので、画像を再構成するのに好ましい。そのため、画像を逆変換して戻した後の画像に、アーティファクトが出現しにくく、好ましい。
【０１５２】
（合成後処理手段４０の詳細例６）
図２２に、合成後処理手段４０の詳細例６を示す。合成後処理手段４０は、歪み量算出手段４２０と、画像データ変換手段４３０と、を備える。
【０１５３】
合成後処理手段４０の詳細例６は、合成画像の画質評価値に基づいて歪み情報を生成する場合である。ここで、画質評価値とは、階調性、鮮鋭性、粒状性等をいう。
【０１５４】
歪み量算出手段４２０として、画質評価値に基づいて、歪み情報を生成する方法を用いる。
歪み量算出手段４２０は、画質評価手段４２５と、歪み情報算出手段４２６と、から構成される。
【０１５５】
画質評価手段４２５は、合成端部情報を用いて、適宜目的に応じた画像領域において、合成画像の画像データＰから画質評価値を求める。画質評価値として、階調性については、例えば、合成画像の画像データＰのヒストグラムからコントラストを求める。また、鮮鋭性については、例えば、合成画像の画像データＰをウェーブレット変換して得られた複数の高周波数帯域成分の画像データの大きさを比較して求める。また、粒状性については、例えば、合成画像の画像データＰをウェーブレット変換して得られた複数の高周波数帯域成分の画像データの大きさを比較して求める。なお、画質評価値として階調性、鮮鋭性、粒状性等を求める際に、他の可能な方法を用いてもよい。
【０１５６】
歪み情報算出手段４２６は、画質評価手段４２５により求められた画質評価値に基づいて、歪み情報を算出する。
【０１５７】
画像データ変換手段４３０は、各画質評価値に基づく歪み情報に基づき変換を行う。例えば、階調性についていえば、階調変換等を行い、鮮鋭性、粒状性についていえば、例えば、スムージング処理や、アンシャープマスキング処理等の変換を行う。
【０１５８】
合成後処理手段４０の詳細例６では、合成画像の階調性、鮮鋭性、粒状性等の画質評価値に基づいて歪み情報を生成し、合成画像の不自然さを軽減することができ、好ましい。
【０１５９】
（合成後処理手段４０の詳細例７）
図２３に、合成後処理手段４０の詳細例７を示す。合成後処理手段４０は、輝度／色差変換手段４１０と、歪み量算出手段４２０と、画像データ変換手段４３０と、輝度／色差逆変換手段４４０と、を備える。
【０１６０】
合成後処理手段４０の詳細例７は、合成後処理手段４０の詳細例５と同様に、二項ウェーブレット変換を用いた例である。また、合成後処理手段４０の詳細例７は、合成後処理手段４０の詳細例６と同様に、画質評価値に基づいて歪み情報を生成する例であり、合成後処理手段４０の詳細例５において、歪みデータ抽出手段４２２に、粒状性評価を用いる場合の例である。
輝度／色差変換手段４１０と、輝度／色差逆変換手段４４０については、合成後処理手段４０の詳細例２と同様であるので、説明を省略する。
【０１６１】
歪み量算出手段４２０は、二項ウェーブレット変換手段４２４と、歪みデータ抽出手段４２２と、歪み量ランク分け手段４２３と、から構成される。合成後処理手段４０の詳細例７では、歪みデータ抽出手段４２２として、粒状性評価手段４２７を用いる。この例では、合成端部情報により合成画像データを複数の領域に分けて、領域毎に別の処理を行う方法を用いる。例えば、図６（ｄ）において背景と人物に領域分けして別々の処理を行う方法が挙げられる。
二項ウェーブレット変換手段４２４については、合成後処理手段４０の詳細例５と同様であるので、説明を省略する。
【０１６２】
粒状性評価手段４２７について説明する。
合成画像の輝度データＢを二項ウェーブレット変換して得られたレベル１の高周波数帯域成分Ｗｘ_１，Ｗｙ_１の画像データと、レベル２の高周波数帯域成分Ｗｘ_２，Ｗｙ_２の画像データの大きさを比較して、合成端部情報に記された領域毎の粒状性の特性値を求める。
【０１６３】
レベル１の高周波数帯域成分Ｗｘ_１，Ｗｙ_１の画像データの大きさをＡＨ１とし、レベル２の高周波数帯域成分Ｗｘ_２，Ｗｙ_２の画像データの大きさをＡＨ２とする。ＡＨ１＞ＡＨ２×α（０＜α＜１）を満たす画素を、粒状評価画素として各領域毎に抽出する。そして、領域毎に粒状評価画素におけるレベル１の高周波数帯域成分Ｗｘ_１，Ｗｙ_１の画像データの大きさＡＨ１の平均値を、粒状性評価値ＡＲｉとして求める（ｉは領域の番号）。
次に、すべての領域ｉにおける粒状性評価値ＡＲｉの平均を取り、粒状性評価平均値ＡＲａｖｇを算出する。粒状性評価値ＡＲｉの粒状性評価平均値ＡＲａｖｇに対する割合を比粒状性評価値ＲＲｉ＝ＡＲｉ／ＡＲａｖｇとして、各領域毎に求める。
なお、粒状性評価手段４２７は、この方法に限定されない。
【０１６４】
歪み量ランク分け手段４２３は、比粒状性評価値ＲＲｉに応じてランク分けして、各領域毎の歪み量とする。そして、歪み情報として、領域ｉ毎の粒状評価画素の位置と領域ｉ毎の歪み量を生成する。
【０１６５】
画像データ変換手段４３０は、ランク別画像データ変換手段４３１と、二項ウェーブレット逆変換手段４３３と、から構成される。
ランク別画像データ変換手段４３１は、歪み量のランクに応じて画像データを変換する。この例においては、領域ｉ毎の歪み量に応じて領域ｉ毎に粒状評価画素の高周波数帯域成分Ｗｘ_１，Ｗｙ_１，Ｗｘ_２，Ｗｙ_２に所定の変換を行い、各領域ｉ毎の比粒状性評価値ＲＲｉが所定の値内に入るようにする。
二項ウェーブレット逆変換手段４３３については、合成後処理手段４０の詳細例５と同様であるので、説明を省略する。
【０１６６】
合成後処理手段４０の詳細例７では、合成端部情報を利用して合成画像のデータを領域に分けて、領域毎に別の処理を行うことができるので、各領域に適した処理を行うことができる。したがって、合成画像をより自然に修正することができ、好ましい。
なお、この例では、粒状性を評価したが、鮮鋭性、階調性等、他の画質評価値に基づいて歪み量を求め、対応する画像データ変換を行ってもよい。
【０１６７】
合成後処理手段４０の詳細例４，５，７では、直交ウェーブレット変換、二項ウェーブレット変換において、レベル２までの高周波数帯域成分を用いた場合を説明したが、レベル１だけでもよいし、より多いレベルまでの高周波数帯域成分を用いて歪み量を算出することとしてもよい。例えば、ｎレベルまでの高周波数帯域成分を用いる場合、１〜ｍ（ｎ＜ｍ）レベルまでは直交エーブレット変換又は双直交ウェーブレット変換を用い、ｍ＋１〜ｎレベルに二項ウェーブレット変換を用いると、精度を大きく落とすことなく、処理速度の低下を抑えることができる。よって、精度と処理速度の両立の点で好ましい。
また、多重解像度変換での例として、ウェーブレット変換を示したが、他の多重解像度変換でもよい。また、複数の多重解像度変換の併用でもよい。
【０１６８】
［実施の形態４］
図２４に、実施の形態４として画像合成処理の例を示す。図２４に示す画像処理部１４３は、図２における画像記録装置１の画像処理部１４の一例である。
図２４に示すように、画像処理部１４３は、入力画像変換手段２０と、合成前処理手段６０と、画像合成手段３０と、合成後処理手段４０と、出力画像変換手段５０と、を備える。
【０１６９】
実施の形態４では、元画像の画像データＧ１，Ｇ２を合成する前に、元画像間の特性の違いを軽減させるための前処理を行う。
入力画像変換手段２０、合成後処理手段４０、出力画像変換手段５０については、実施の形態１と同様であるので、説明を省略する。
【０１７０】
合成前処理手段６０は、元画像の画像データＧ１，Ｇ２に、元画像間の特性の違いを軽減させるための画像処理を行い、前処理後の画像データＧ１’，Ｇ２’を生成する。ここで、特性とは、主に画質に関する特性をいい、色、階調性、鮮鋭性、粒状性等をいう。
【０１７１】
合成前処理としては、種々の処理があるが、例えば、画質評価値の違いを求め、特性を近づける処理を行う。
画質評価値として、階調性については、例えば、元画像の画像データＧ１，Ｇ２のヒストグラムからコントラストを求める。また、鮮鋭性については、例えば、元画像の画像データＧ１，Ｇ２をウェーブレット変換して得られた複数の高周波数帯域成分の画像データの大きさを比較して求める。また、粒状性については、例えば、元画像の画像データＧ１，Ｇ２をウェーブレット変換して得られた複数の高周波数帯域成分の画像データの大きさを比較して求める。なお、画質評価値として階調性、鮮鋭性、粒状性等を求める際に、他の可能な方法を用いてもよい。
【０１７２】
元画像の画像データＧ１，Ｇ２から求めた画質評価値の違いに基づいて、元画像の画像データＧ１，Ｇ２に画像処理を行う。例えば、階調性についていえば、階調変換等を行い、鮮鋭性、粒状性についていえば、例えば、スムージング処理や、アンシャープマスキング処理等の変換を行う。元画像の画像データＧ１，Ｇ２の両方に対して変換を行うこととしてもよいし、元画像の画像データＧ１，Ｇ２のうち、一方を他方に合わせるように変換を行うこととしてもよい。
【０１７３】
画像合成手段３０は、前処理後の画像データＧ１’，Ｇ２’を合成して合成画像の画像データＰを生成するとともに、合成端部情報を生成する。扱うデータが前処理後の画像データＧ１’，Ｇ２’であること以外は、実施の形態１と同様であるので、説明を省略する。
【０１７４】
（合成前処理手段６０の詳細例１）
図２５に、二項ウェーブレット変換を用いた合成前処理手段６０の例として、合成前処理手段６０の詳細例１を示す。合成前処理手段６０は、輝度／色差変換手段６１０と、二項ウェーブレット変換手段６２０と、前処理用画像データ変換手段６３０と、二項ウェーブレット逆変換手段６４０と、輝度／色差逆変換手段６５０と、を備える。
【０１７５】
輝度／色差変換手段６１０は、元画像の画像データＧ１を輝度データＶ１と色差データＵ１に変換し、元画像の画像データＧ２を輝度データＶ２と色差データＵ２に変換する。
【０１７６】
二項ウェーブレット変換手段６２０は、輝度データＶ１を二項ウェーブレット変換して、レベル１の高周波数帯域成分Ｗｘ_１１，Ｗｙ_１１と、レベル２の高周波数帯域成分Ｗｘ_２１，Ｗｙ_２１と、低周波数帯域成分Ｓ_２１を生成する。同様に、輝度データＶ２を二項ウェーブレット変換して、レベル１の高周波数帯域成分Ｗｘ_１２，Ｗｙ_１２と、レベル２の高周波数帯域成分Ｗｘ_２２，Ｗｙ_２２と、低周波数帯域成分Ｓ_２２を生成する。
【０１７７】
前処理用画像データ変換手段６３０は、高周波数帯域成分Ｗｘ_１１，Ｗｙ_１１，Ｗｘ_２１，Ｗｙ_２１と、低周波数帯域成分Ｓ_２１を変換して、高周波数帯域成分Ｗｘ_１１’，Ｗｙ_１１’，Ｗｘ_２１’，Ｗｙ_２１’と、低周波数帯域成分Ｓ_２１’を生成する。同様に、高周波数帯域成分Ｗｘ_１２，Ｗｙ_１２，Ｗｘ_２２，Ｗｙ_２２と、低周波数帯域成分Ｓ_２２を変換して、高周波数帯域成分Ｗｘ_１２’，Ｗｙ_１２’，Ｗｘ_２２’，Ｗｙ_２２’と、低周波数帯域成分Ｓ_２２’を生成する。
【０１７８】
図２６に、前処理用画像データ変換手段６３０として、粒状性に基づいて変換を行う例を示す。前処理用画像データ変換手段６３０は、粒状性評価手段６３１と、画像変換手段６３２と、から構成される。
【０１７９】
粒状性評価手段６３１は、輝度データＶ１を二項ウェーブレット変換して得られたレベル１の高周波数帯域成分Ｗｘ_１１，Ｗｙ_１１の画像データの大きさをＢＨ１とし、レベル２の高周波数帯域成分Ｗｘ_２１，Ｗｙ_２１の画像データの大きさをＢＨ２として、ＢＨ１＞ＢＨ２×β（０＜β＜１）を満たす画素を、粒状評価画素として抽出する。そして、粒状評価画素におけるレベル１の高周波数帯域成分Ｗｘ_１１，Ｗｙ_１１の画像データの大きさＢＨ１の平均値を、粒状性評価値ＢＲ１として求める。
同様に、輝度データＶ２を二項ウェーブレット変換して得られたレベル１の高周波数帯域成分Ｗｘ_１２，Ｗｙ_１２の画像データ、レベル２の高周波数帯域成分Ｗｘ_２２，Ｗｙ_２２の画像データについても処理を行い、粒状性評価値ＢＲ２を求める。
【０１８０】
次に、２つの元画像における粒状性評価値ＢＲ１，ＢＲ２の平均を取り、粒状性評価平均値ＢＲａｖｇを算出する。粒状性評価値ＢＲ１，ＢＲ２の粒状性評価平均値ＢＲａｖｇに対する割合を比粒状性評価値ＢＲＲ１＝ＢＲ１／ＢＲａｖｇ，ＢＲＲ２＝ＢＲ２／ＢＲａｖｇとして求める。
そして、粒状性評価手段６３１は、比粒状性評価値ＢＲＲ１，ＢＲＲ２に応じて、それぞれの元画像に対する画像変換条件を算出し、各画像の画像変換条件を画像変換手段６３２に送る。
なお、粒状性評価手段６３１は、この方法に限定されない。
【０１８１】
画像変換手段６３２は、粒状性評価手段６３１により求められた各画像の画像変換条件に基づいて高周波数帯域成分Ｗｘ_１１，Ｗｙ_１１，Ｗｘ_２１，Ｗｙ_２１と、低周波数帯域成分Ｓ_２１を変換して、高周波数帯域成分Ｗｘ_１１’，Ｗｙ_１１’，Ｗｘ_２１’，Ｗｙ_２１’と、低周波数帯域成分Ｓ_２１’を生成する。同様に、高周波数帯域成分Ｗｘ_１２，Ｗｙ_１２，Ｗｘ_２２，Ｗｙ_２２と、低周波数帯域成分Ｓ_２２を変換して、高周波数帯域成分Ｗｘ_１２’，Ｗｙ_１２’，Ｗｘ_２２’，Ｗｙ_２２’と、低周波数帯域成分Ｓ_２２’を生成する。この変換により、比粒状性評価値ＢＲＲ１，ＢＲＲ２が所定の値内に入るようにする。
【０１８２】
図２５に示すように、二項ウェーブレット逆変換手段６４０は、高周波数帯域成分Ｗｘ_１１’，Ｗｙ_１１’，Ｗｘ_２１’，Ｗｙ_２１’と、低周波数帯域成分Ｓ_２１’を用いて二項ウェーブレット逆変換を行い、前処理後の輝度データＶ１’を生成する。同様に、高周波数帯域成分Ｗｘ_１２’，Ｗｙ_１２’，Ｗｘ_２２’，Ｗｙ_２２’と、低周波数帯域成分Ｓ_２２’を用いて二項ウェーブレット逆変換を行い、前処理後の輝度データＶ２’を生成する。
【０１８３】
輝度／色差逆変換手段６５０は、二項ウェーブレット逆変換手段６４０において生成された前処理後の輝度データＶ１’と色差データＵ１に輝度／色差逆変換を施し、前処理後の画像データＧ１’を生成する。同様に、前処理後の輝度データＶ２’と色差データＵ２から前処理後の画像データＧ２’を生成する。この輝度／色差逆変換は、輝度／色差変換手段６１０における輝度／色差変換の逆変換である。
【０１８４】
実施の形態４では、合成前に画像の基本特性をそろえておき、合成後に合成境界部の不自然さを修正するため、合成前と合成後の両方の処理による複合効果により、合成画像をより自然な印象にすることができる。
【０１８５】
なお、実施の形態１〜４においては、２つの元画像から合成画像を作成する例を示したが、３つ以上の元画像から合成画像を作成することとしてもよい。また、合成完了後の画像に、さらに追加合成してもよい。
また、実施の形態１〜４においては、画像データのみを用いているが、必要に応じて、画像データ以外の画像付属データを用いてもよい。
【０１８６】
また、実施の形態１〜４において、画像データ変換処理は、変換する画像全体への処理でもよいし、部分処理でもよい。部分処理としては、合成端部及びその周辺だけに処理を行う方法や、合成領域に分離してそれぞれの領域に別の処理を行う方法等がある。また、画像データを輝度データと色差データに分け、輝度は部分処理し、色差は全体処理する等、部分処理と全体処理の組み合わせでもよい。また、合成画像の画像データや中間データを任意に組み合わせて用いてもよい。
【０１８７】
また、実施の形態１〜４において、変換するデータとしては、画像データや、画像データから変換された輝度データや、輝度データから変換された中間データ等の例を挙げたが、色差データや、色差データから変換された中間データ等、他のデータを目的に応じて用いればよく、限定されるものではない。
【０１８８】
また、実施の形態１〜４の各ブロックは、画像処理部の機能の理解を助けるために設けたブロックであり、必ずしも物理的に独立している必要はなく、例えば、単一のＣＰＵにおけるソフトウェア処理の種類のブロックとして実現されてもよい。
【０１８９】
【発明の効果】
上述したように、請求項１、７、１３又は１９に記載の発明によれば、合成端部情報を用いることにより、合成画像の各領域毎に適切な変換ができるので、合成画像の不自然さを軽減させることができる。特に、合成画像の境界部やその周辺に対して合成後の画像データを変換することができるので、合成画像の境界部の不自然さを軽減させることができる。したがって、極めて少ない作業工数で合成画像を自然な画像にすることができる。
【０１９０】
上述したように、請求項２、８、１４又は２０に記載の発明によれば、歪み量に基づいて画像データを変換するので、必要な部分に必要な程度の処理を行うことができる。そのため、極めて少ない作業工数で合成画像の不自然さを軽減させることができる。
【０１９１】
上述したように、請求項３、９、１５又は２１に記載の発明によれば、合成端部情報を用いて歪み量を算出するので、効率よく歪み量を算出することができる。したがって、より精度よく合成画像の不自然さを軽減させることができる。
【０１９２】
上述したように、請求項４、１０、１６又は２２に記載の発明によれば、多重解像度変換により、解像度が異なる複数の画像データを用いて、より精度よく歪み量を算出することができる。したがって、より精度よく合成画像の不自然さを軽減させることができる。
【０１９３】
上述したように、請求項５、１１、１７又は２３に記載の発明によれば、多重解像度変換された画像データを、合成画像データの歪み量の算出に用いるとともに、歪み量に基づく画像データ変換に用いることにより、処理を効率化することができる。
【０１９４】
上述したように、請求項６、１２、１８又は２４に記載の発明によれば、合成前に画像の基本特性をそろえておき、合成後に合成境界部の不自然さを修正するので、合成前と合成後の両方の処理による複合効果により、合成画像をより自然な印象にすることができる。
【図面の簡単な説明】
【図１】本実施の形態の画像記録装置の概観構成を示す図である。
【図２】画像記録装置の機能構成を示すブロック図である。
【図３】実施の形態１の画像合成処理を示す図である。
【図４】合成画像の合成端部を説明するための図である。
【図５】背景画像に人物を貼り付ける場合の例を示す図である。
【図６】図４の合成画像における部分処理の処理領域の例を示す図である。
【図７】実施の形態２の画像合成処理を示す図である。
【図８】合成後処理手段４０の詳細例１を示す図である。
【図９】実施の形態３の画像合成処理を示す図である。
【図１０】合成後処理手段４０の詳細例２を示す図である。
【図１１】合成後処理手段４０の詳細例３を示す図である。
【図１２】合成後処理手段４０の詳細例４を示す図である。
【図１３】ウェーブレット変換において用いられるウェーブレット関数を示した図である。
【図１４】直交ウェーブレット変換又は双直交ウェーブレット変換におけるフィルタ処理を示す図である。
【図１５】２次元信号における１レベルの直交ウェーブレット変換又は双直交ウェーブレット変換を示す図である。
【図１６】３レベルの直交ウェーブレット変換又は双直交ウェーブレット変換で信号分解される過程を示す模式図である。
【図１７】直交ウェーブレット変換又は双直交ウェーブレット変換で分解された信号をウェーブレット逆変換により再構成する方法を示す図である。
【図１８】合成後処理手段４０の詳細例５を示す図である。
【図１９】ウェーブレット変換される信号の特性を示す図である。
【図２０】二項ウェーブレット変換を示す図である。
【図２１】二項ウェーブレット逆変換を示す図である。
【図２２】合成後処理手段４０の詳細例６を示す図である。
【図２３】合成後処理手段４０の詳細例７を示す図である。
【図２４】実施の形態４の画像合成処理を示す図である。
【図２５】合成前処理手段６０の詳細例１を示す図である。
【図２６】前処理用画像データ変換手段６３０の例を示す図である。
【符号の説明】
１ 画像記録装置
２ ＣＰＵ
３ ＲＯＭ
４ ＲＡＭ
５ 操作部
６ フィルムスキャナ部
７ 反射原稿入力部
８ 画像読込部
９ ＣＲＴ
１０ プリント作成部
１１ 画像書込部
１２ 記憶部
１３ 通信部
１４ 画像処理部
１５ トレー
１６ マガジン
２０ 入力画像変換手段
３０ 画像合成手段
４０ 合成後処理手段
５０ 出力画像変換手段
６０ 合成前処理手段
４１０ 輝度／色差変換手段
４２０ 歪み量算出手段
４２１ 直交ウェーブレット変換手段
４２２ 歪みデータ抽出手段
４２３ 歪み量ランク分け手段
４２４ 二項ウェーブレット変換手段
４２５ 画質評価手段
４２６ 歪み情報算出手段
４２７ 粒状性評価手段
４３０ 画像データ変換手段
４３１ ランク別画像データ変換手段
４３２ 直交ウェーブレット逆変換手段
４３３ 二項ウェーブレット逆変換手段
４４０ 輝度／色差逆変換手段
６１０ 輝度／色差変換手段
６２０ 二項ウェーブレット変換手段
６３０ 前処理用画像データ変換手段
６３１ 粒状性評価手段
６３２ 画像変換手段
６４０ 二項ウェーブレット逆変換手段
６５０ 輝度／色差逆変換手段[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image synthesizing method, an image synthesizing device, an image synthesizing program, and an image recording device, and relates to a process for creating a synthetic image having a natural impression.
[0002]
[Prior art]
In recent years, with the spread of digital cameras and scanners, opportunities for handling digital image data have been increasing. These image data are easy to process, such as combining all or a part of the plurality of image data into one image data. For example, a specific portion such as a person is extracted from an image and pasted on a background image, a CG (Computer Graphics) image of a car or an airplane is pasted on a background image of a natural image, or a specific portion such as a person is extracted from a natural image. It can be extracted and pasted on a background image of a CG such as an imaginary room or a futuristic city, or the design of a signboard of a store photograph can be changed. Here, the natural image is an image obtained by photographing a real thing. As a method of synthesizing images, there are replacement synthesis that replaces a part of an image, watermark synthesis that overlaps a part of one image on another image, and the like.
[0003]
When synthesizing images, there is a problem how to make the image quality of a natural impression without discomfort. For example, there has been proposed a method of extracting a person from a certain image and pasting it to another background image to adjust the granularity and sharpness characteristics of the original image before combining the images (for example, See Patent Document 1.). This eliminates the apparent discomfort due to the different granularity and sharpness of each region in the composite image.
[0004]
[Patent Document 1]
JP-A-11-275343
[0005]
[Problems to be solved by the invention]
However, in the method described in Patent Literature 1, since the image is not adjusted with respect to the synthesized boundary portion, unnaturalness of the boundary remains. Further, even if pre-processing such as the method described in Patent Document 1 is performed before synthesis, it may be inappropriate or insufficient. In addition, there is a limit in adjustment before synthesis, and unnaturalness may remain in the overall impression after synthesis.
As a post-synthesis process, a method of manually performing image processing such as blurring on a boundary portion of the synthesis is known, but there is a problem that a large number of work steps are required.
[0006]
The present invention has been made in view of the above-described problems in the related art, and has an image combining method, an image combining apparatus, an image combining program, and an image combining method capable of reducing unnaturalness of a combined image with a small number of work steps. It is an object to provide an image recording device.
[0007]
[Means for Solving the Problems]
According to a first aspect of the present invention, there is provided an image combining method for creating a combined image using two or more images, the combined edge information relating to a boundary where the two or more images are in contact with each other. A post-synthesis processing step of determining an image conversion condition for the synthesized image data using the image conversion method, and converting the synthesized image data based on the determined image conversion condition. It is.
[0008]
According to the first aspect of the present invention, by using the composite edge information, appropriate conversion can be performed for each region of the composite image, so that unnaturalness of the composite image can be reduced. In particular, since the synthesized image data can be converted into and around the boundary of the composite image, the unnaturalness of the boundary of the composite image can be reduced. Therefore, the synthesized image can be made a natural image with a very small number of work steps.
[0009]
According to a second aspect of the present invention, in the image combining method of creating a combined image using two or more images, a distortion amount calculating step of calculating a distortion amount from the combined image data, and the calculated distortion amount An image data conversion step of determining an image conversion condition for the image data after synthesis based on the image conversion condition, and converting the image data after synthesis based on the determined image conversion condition. This is an image synthesis method.
[0010]
According to the second aspect of the present invention, since image data is converted based on the amount of distortion, necessary processing can be performed on a required portion. Therefore, the unnaturalness of the composite image can be reduced with a very small number of work steps.
[0011]
According to a third aspect of the present invention, in the image synthesizing method according to the second aspect, in the distortion amount calculating step, the distortion amount is calculated using synthesized edge information regarding a boundary where the two or more images are in contact with each other. An image synthesizing method characterized in that:
[0012]
According to the third aspect of the present invention, since the distortion amount is calculated using the combined edge information, the distortion amount can be calculated efficiently. Therefore, it is possible to more accurately reduce the unnaturalness of the composite image.
[0013]
The invention according to claim 4 is the image composition method according to claim 3, wherein the distortion amount calculating step includes a multi-resolution conversion.
[0014]
According to the fourth aspect of the present invention, it is possible to more accurately calculate the distortion amount by using a plurality of image data having different resolutions by the multi-resolution conversion. Therefore, it is possible to more accurately reduce the unnaturalness of the composite image.
[0015]
According to a fifth aspect of the present invention, in the image synthesizing method according to the fourth aspect, the conversion to the image data after the combination in the image data conversion step is conversion to the image data after the multi-resolution conversion. This is an image synthesizing method characterized by the following.
[0016]
According to the fifth aspect of the present invention, the image data subjected to the multi-resolution conversion is used for calculating the amount of distortion of the combined image data and used for image data conversion based on the amount of distortion, thereby increasing the efficiency of processing. Can be.
[0017]
According to a sixth aspect of the present invention, in the image synthesizing method according to any one of the first to fifth aspects, a pre-synthesis process using multi-resolution conversion is performed on at least one image before the synthesis. An image synthesizing method including a preprocessing step.
[0018]
According to the sixth aspect of the present invention, the basic characteristics of the images are aligned before the synthesis, and the unnaturalness of the synthesis boundary is corrected after the synthesis. Thus, the synthesized image can be made more natural.
[0019]
8. The image synthesizing apparatus according to claim 7, wherein the image combining device creates a combined image by using two or more images. An image synthesizing apparatus comprising: a post-synthesis processing unit that determines an image conversion condition for the image data and converts the synthesized image data based on the determined image conversion condition.
[0020]
According to the seventh aspect of the present invention, since the appropriate conversion can be performed for each area of the composite image by using the composite edge information, the unnaturalness of the composite image can be reduced. In particular, since the synthesized image data can be converted into and around the boundary of the composite image, the unnaturalness of the boundary of the composite image can be reduced. Therefore, the synthesized image can be made a natural image with a very small number of work steps.
[0021]
The invention according to claim 8, wherein in an image synthesizing apparatus that creates a composite image using two or more images, a distortion amount calculation unit that calculates a distortion amount from image data after synthesis, and the calculated distortion amount. An image data conversion unit that determines an image conversion condition to the combined image data based on the image data, and converts the combined image data based on the determined image conversion condition. Image synthesizing device.
[0022]
According to the eighth aspect of the present invention, since image data is converted based on the amount of distortion, necessary processing can be performed on a required portion. Therefore, the unnaturalness of the composite image can be reduced with a very small number of work steps.
[0023]
According to a ninth aspect of the present invention, in the image synthesizing apparatus according to the eighth aspect, the distortion amount calculating means calculates the amount of distortion using synthesized edge information regarding a boundary where the two or more images are in contact with each other. An image synthesizing apparatus characterized in that:
[0024]
According to the ninth aspect of the present invention, since the distortion amount is calculated using the combined edge information, the distortion amount can be efficiently calculated. Therefore, it is possible to more accurately reduce the unnaturalness of the composite image.
[0025]
According to a tenth aspect of the present invention, in the image synthesizing apparatus according to the ninth aspect, the distortion amount calculating means performs multi-resolution conversion.
[0026]
According to the tenth aspect, it is possible to more accurately calculate the amount of distortion by using a plurality of image data having different resolutions by the multi-resolution conversion. Therefore, it is possible to more accurately reduce the unnaturalness of the composite image.
[0027]
According to an eleventh aspect of the present invention, in the image synthesizing apparatus according to the tenth aspect, the conversion to the image data after the synthesis by the image data conversion means is conversion to the image data after the multi-resolution conversion. An image synthesizing apparatus characterized by the following.
[0028]
According to the eleventh aspect of the present invention, the image data subjected to the multi-resolution conversion is used for calculating the amount of distortion of the combined image data and is used for image data conversion based on the amount of distortion, thereby improving the efficiency of the processing. Can be.
[0029]
According to a twelfth aspect of the present invention, in the image synthesizing apparatus according to any one of the seventh to eleventh aspects, a pre-synthesis process using multi-resolution conversion is performed on at least one image before the synthesis. An image synthesizing apparatus comprising a preprocessing means.
[0030]
According to the twelfth aspect of the present invention, the basic characteristics of the images are aligned before the synthesis, and the unnaturalness of the synthesis boundary portion is corrected after the synthesis. Thus, the synthesized image can be made more natural.
[0031]
According to a thirteenth aspect of the present invention, a computer for executing a process of creating a composite image using two or more images is composed using composite edge information relating to a boundary where the two or more images contact each other. An image synthesizing program for realizing a post-synthesis processing function of determining image conversion conditions for post-image data and converting the synthesized image data based on the determined image conversion conditions.
[0032]
According to the thirteenth aspect, by using the composite edge information, an appropriate conversion can be performed for each region of the composite image, so that the unnaturalness of the composite image can be reduced. In particular, since the synthesized image data can be converted into and around the boundary of the composite image, the unnaturalness of the boundary of the composite image can be reduced. Therefore, the synthesized image can be made a natural image with a very small number of work steps.
[0033]
A computer for executing a process of creating a composite image using two or more images includes a distortion amount calculation function of calculating a distortion amount from image data after synthesis, An image data conversion function of determining image conversion conditions for the combined image data based on the determined distortion amount, and converting the combined image data based on the determined image conversion conditions. Image synthesizing program.
[0034]
According to the fourteenth aspect of the present invention, since image data is converted based on the amount of distortion, necessary processing can be performed on a required portion. Therefore, the unnaturalness of the composite image can be reduced with a very small number of work steps.
[0035]
According to a fifteenth aspect of the present invention, in the image synthesizing program according to the fourteenth aspect, the distortion amount calculating function calculates the amount of distortion using synthesized edge information on a boundary where the two or more images are in contact with each other. An image synthesizing program characterized in that:
[0036]
According to the fifteenth aspect, since the distortion amount is calculated using the combined edge information, the distortion amount can be efficiently calculated. Therefore, it is possible to more accurately reduce the unnaturalness of the composite image.
[0037]
The invention according to claim 16 is the image composition program according to claim 15, wherein the distortion amount calculation function includes multi-resolution conversion.
[0038]
According to the sixteenth aspect, it is possible to more accurately calculate the amount of distortion by using a plurality of image data having different resolutions by the multi-resolution conversion. Therefore, it is possible to more accurately reduce the unnaturalness of the composite image.
[0039]
According to a seventeenth aspect of the present invention, in the image synthesizing program according to the sixteenth aspect, the conversion to the image data after the combination in the image data conversion function is conversion to the image data after the multi-resolution conversion. An image synthesizing program characterized by the following.
[0040]
According to the seventeenth aspect, the multi-resolution converted image data is used for calculating the amount of distortion of the combined image data, and is used for image data conversion based on the amount of distortion, thereby improving processing efficiency. Can be.
[0041]
According to an eighteenth aspect of the present invention, in the image composing program according to any one of the thirteenth to seventeenth aspects, the computer is configured to provide the computer with at least one image before synthesis using multi-resolution conversion before synthesis. This is an image synthesis program that realizes a pre-synthesis processing function for performing processing.
[0042]
According to the eighteenth aspect of the present invention, the basic characteristics of the images are aligned before the synthesis, and the unnaturalness of the synthesis boundary portion is corrected after the synthesis. Thus, the synthesized image can be made more natural.
[0043]
20. An image recording apparatus according to claim 19, wherein a composite image is created by using two or more images, and the composite image is recorded on a recording medium. An image characterized by comprising post-synthesis processing means for determining image conversion conditions for image data after synthesis using information and converting the image data after synthesis based on the determined image conversion conditions. It is a recording device.
[0044]
According to the nineteenth aspect, by using the composite edge information, appropriate conversion can be performed for each region of the composite image, so that the unnaturalness of the composite image can be reduced. In particular, since the synthesized image data can be converted into and around the boundary of the composite image, the unnaturalness of the boundary of the composite image can be reduced. Therefore, the synthesized image can be made a natural image with a very small number of work steps.
[0045]
According to the twentieth aspect of the present invention, in an image recording apparatus for creating a composite image using two or more images and recording the composite image on a recording medium, a distortion amount calculation is performed from the image data after the combination. Means for determining image conversion conditions for the combined image data based on the calculated amount of distortion, and converting the combined image data based on the determined image conversion conditions And an image recording apparatus.
[0046]
According to the twentieth aspect, the image data is converted based on the distortion amount, so that a necessary degree of processing can be performed on a necessary portion. Therefore, the unnaturalness of the composite image can be reduced with a very small number of work steps.
[0047]
According to a twenty-first aspect of the present invention, in the image recording apparatus according to the twentieth aspect, the distortion amount calculating means calculates the distortion amount using composite edge information on a boundary where the two or more images are in contact with each other. An image recording apparatus characterized in that:
[0048]
According to the twenty-first aspect, since the distortion amount is calculated using the combined edge information, the distortion amount can be calculated efficiently. Therefore, it is possible to more accurately reduce the unnaturalness of the composite image.
[0049]
The invention according to claim 22 is the image recording apparatus according to claim 21, wherein the distortion amount calculating means performs multi-resolution conversion.
[0050]
According to the invention described in claim 22, the amount of distortion can be calculated with higher accuracy by using a plurality of image data having different resolutions by the multi-resolution conversion. Therefore, it is possible to more accurately reduce the unnaturalness of the composite image.
[0051]
According to a twenty-third aspect of the present invention, in the image recording device according to the twenty-second aspect, the conversion to the image data after the combination by the image data conversion means is conversion to the image data after the multi-resolution conversion. An image recording apparatus characterized in that:
[0052]
According to the twenty-third aspect, the multi-resolution converted image data is used for calculating the amount of distortion of the combined image data and used for image data conversion based on the amount of distortion, thereby increasing the efficiency of processing. Can be.
[0053]
According to a twenty-fourth aspect of the present invention, in the image recording apparatus according to any one of the nineteenth to twenty-third aspects, a synthesis pre-process using multi-resolution conversion is performed on at least one image before the synthesis. An image recording apparatus comprising a preprocessing unit.
[0054]
According to the invention described in claim 24, the basic characteristics of the images are aligned before the synthesis, and the unnaturalness of the synthesis boundary portion is corrected after the synthesis. Thus, the synthesized image can be made more natural.
[0055]
BEST MODE FOR CARRYING OUT THE INVENTION
An embodiment of the present invention will be described below with reference to the drawings.
First, FIG. 1 shows an outline configuration of an image recording apparatus 1 according to the present embodiment.
As shown in FIG. 1, the image recording apparatus 1 includes an operation unit 5, a film scanner unit 6, a reflection original input unit 7, an image reading unit 8, a CRT (Cathode Ray Tube) 9, a print creation unit 10, and an image writing unit. 11, a tray 15, and a magazine 16 are provided.
[0056]
Next, FIG. 2 shows a functional configuration of the image recording apparatus 1.
As shown in FIG. 2, the image recording apparatus 1 includes a CPU (Central Processing Unit) 2, a ROM (Read Only Memory) 3, a RAM (Random Access Memory) 4, an operation unit 5, a film scanner unit 6, and a reflection document input unit. 7, an image reading unit 8, a CRT 9, a print creation unit 10, an image writing unit 11, a storage unit 12, a communication unit 13, and an image processing unit 14.
[0057]
The CPU 2 reads various control programs stored in the ROM 3 or the storage unit 12 in response to signals input from the operation unit 5 and the communication unit 13 and expands the control programs in the RAM 4, and cooperates with the developed control programs. Various processes are executed by the operation, and each unit of the image recording apparatus 1 is made to function.
[0058]
The ROM 3 is a read-only semiconductor memory, and stores programs executed by the CPU 2, data, and the like.
The RAM 4 is a storage medium in which data is temporarily stored, a program area for expanding a program to be executed by the CPU 2, data input from the operation unit 5 and the communication unit 13, various processing results by the CPU 2, and the like. , A data area for storing image data input from the film scanner unit 6, the reflection document input unit 7, the image reading unit 8 or the communication unit 13, and the like. Here, the image data refers to data of each pixel obtained by dividing an image into a number of small regions (pixels) and expressing the brightness and the like of each color component by signal intensity for each pixel.
[0059]
The operation unit 5 includes numeric keys and various function keys, and includes, for example, a touch panel. When any of these keys is pressed, a signal indicating that the key is pressed is output to the CPU 2.
[0060]
The film scanner unit 6 is a device that reads a transparent original. The film scanner unit 6 reads image data from a color negative film, a color reversal film, or the like, which is photographed and developed by an analog camera under the control of the CPU 2. The reflection document input unit 7 is a device that reads an image from a reflection document. The reflection document input unit 7 reads image data from a reflection document such as a color photographic print or a printed matter under the control of the CPU 2.
[0061]
The image reading section 8 includes a PC card adapter 8a, a floppy (registered trademark) disk adapter 8b, and an optical disk adapter 8c, and a PC card, a floppy disk, and an optical disk can be inserted therein. The media is not limited to these, and may be an MO (Magneto Optical disk), Zip (registered trademark), or the like, as long as a corresponding adapter is provided. Various media stores, for example, a plurality of image data, image attached data, order information (for example, DPOF (Digital Print Order Format), etc.) captured by a digital camera, and the like. The image attached data refers to data such as a header and a tag attached to the image data. The image reading unit 8 reads image data, image attached data, and the like recorded on various media under the control of the CPU 2.
[0062]
The CRT 9 displays image data read from the film scanner unit 6, the reflection original input unit 7, the image reading unit 8 and image data received from the communication unit 13 on the screen under the control of the CPU 2.
[0063]
Under the control of the CPU 2, the print creating unit 10 exposes the photographic photosensitive material drawn from the magazine 16 based on the image data, and develops the exposed photographic photosensitive material to create a print. The created print is discharged to the tray 15. The print creation unit 10 may be any device that forms an image from the read image data, and may be, for example, an inkjet, electrophotographic, heat-sensitive, or sublimation print creation device.
[0064]
The image writing unit 11 includes a floppy disk adapter 11a, an MO adapter 11b, and an optical disk adapter 11c, so that a floppy disk, an MO, and an optical disk can be inserted. The media is not limited to these, and may be a PC card, Zip, or the like, as long as a corresponding adapter is provided. The image writing unit 11 writes image data, image attached data, and the like on an image recording medium under the control of the CPU 2. Note that the image reading unit 8 and the image writing unit 11 may be used for both reading and writing with one adapter.
[0065]
The storage unit 12 includes, for example, an HDD (Hard Disk Drive), and records programs, data, and the like in the HDD under the control of the CPU 2. Further, when an instruction to read a program or data is issued from the CPU 2, the instructed information is read from the HDD and output to the CPU 2.
[0066]
The communication unit 13 receives image data, image-attached data, print commands, and the like from a computer in the facility or a distant computer via the Internet or the like. That is, the image recording device 1 can function as a network printer. In addition, the communication unit 13 transmits image data, image attached data, accompanying order information, and the like to another computer in the facility, another network printer, a distant computer via the Internet, or the like.
[0067]
The image processing unit 14 performs appropriate image processing on the image data read from the film scanner unit 6, the reflection document input unit 7 or the image reading unit 8 and the image data received from the communication unit 13 as necessary. For example, calibration processing, gray balance adjustment, contrast adjustment suitable for each input method, and negative / positive reversal processing for a negative original are performed. The image data may be any of RGB color image data, monochrome image data, YMCK 4-color image data, and the like. The number of gradations is also arbitrary. For example, in the case of RGB color image data, arbitrary image data such as 8 bits for each color and 12 bits for each color can be used. The image may be in any format (JPEG, Tiff, bmp, etc.). In the case of a format having image data and image attached data, such as JPEG and Tiff, processing corresponding to each format for separating the image data and image attached data and extracting necessary information from the image attached data is performed.
[0068]
The image processing unit 14 is realized by software processing in cooperation with an image processing program stored in the ROM 3 and the CPU 2. Details of an example of the processing according to the present invention will be described later (FIGS. 3, 7, 9, and 24).
[0069]
Further, the image processing unit 14 performs image processing corresponding to the output destination, and the image data subjected to the image processing is displayed on the CRT 9, printed by the print creation unit 10, or image-recorded by the image writing unit 11. It is recorded on a medium or transmitted from the communication unit 13 to another device or the like. For example, calibration processing, color matching, pixel number change processing, etc., suitable for each output method are performed. The image data may be any of RGB color image data, monochrome image data, YMCK 4-color image data, and the like. The number of gradations is also arbitrary. For example, in the case of RGB color image data, arbitrary image data such as 8 bits for each color and 12 bits for each color can be used. The image may be in any format (JPEG, Tiff, bmp, etc.). In the case of a format having image data and image attached data, such as JPEG and Tiff, processing corresponding to each format is performed using the image data and image attached data in correspondence.
[0070]
The image recording apparatus 1 shown in FIG. 1 has a structure in which an operation unit 5, a CRT 9, a film scanner unit 6, a reflection document input unit 7, an image reading unit 8, and a print creation unit 10 are integrated. However, one or more of them may be provided separately. Further, another film scanner, a reflection original input device, a printer, or the like may be connected. In that case, the image processing unit 14 performs a process unique to each device.
[0071]
Next, image processing performed by the image processing unit 14 of the image recording device 1 will be described.
[0072]
[Embodiment 1]
FIG. 3 shows an example of the image synthesis processing as the first embodiment. The image processing unit 140 shown in FIG. 3 is an example of the image processing unit 14 of the image recording device 1 in FIG.
As shown in FIG. 3, the image processing unit 140 includes an input image conversion unit 20, an image synthesis unit 30, a post-synthesis processing unit 40, and an output image conversion unit 50.
[0073]
The input image converting means 20 converts the original image F1 input from the film scanner section 6, the reflection original input section 7, the image reading section 8 or the communication section 13 by means corresponding to each image, and converts the original image F1. The image data G1 is generated. Similarly, conversion is performed by means corresponding to the original image F2 to generate image data G2 of the original image. If the image data G1 and G2 of the original image are three colors of RGB, the following processing is performed for each of them.
[0074]
The image combining means 30 combines the image data G1 and G2 of the original image to generate image data P of the combined image, and also generates combined edge information.
Here, the composite edge refers to the boundary of the composite image or the boundary and its periphery. For example, as shown in FIG. 4A, when a person is combined with the background image, the boundary between the person and the background shown in FIG. 4B is the combining end.
The composite edge information includes pixel position information (coordinates) on the image data of the composite edge. For example, the image data may be image data in which the detected image data other than the composite end is 0, or may be binary image data in which the image data of the composite end is 1 and the image data other than the composite end is 0.
[0075]
FIG. 5 shows an example in which a person is pasted on a background image. A person is extracted from an original image F1 including a person as shown in FIG. 5A, and a person image is superimposed on a predetermined position of the original image F2 shown in FIG. .
As a method for extracting a person, for example, various methods such as automatic extraction by a method such as matching with a predetermined person pattern, manual extraction by giving an instruction from the operation unit 5 while looking at an image displayed on the CRT 9, and the like are described. There are methods and are not limited. A person is extracted by one of the methods (FIG. 5C), and the shape (relative position) of the composite end is obtained from the information on the contour at the time of extraction (FIG. 5D). The relative position is represented by relative coordinates from a point Q on the contour.
The above-described image synthesizing unit 30 is an example of generating the synthesized edge information, and is not limited to this.
[0076]
There are various methods for determining the position to be pasted, such as automatic determination in which the position to be pasted is determined in advance, and manual position adjustment, such as giving an instruction from the operation unit 5 while looking at the image displayed on the CRT 9. , But not limited to. The attachment position is determined by one of the methods (FIG. 5E), and the combined edge information is obtained from the shape of the combined edge and the attached position (FIG. 5F). The shape of the composite end portion represented by the relative coordinates from the point Q on the contour is converted into coordinates from the origin (0,0).
[0077]
As described above, the image combining means 30 creates a combined image (FIG. 5G).
As the synthesizing method, there are various methods such as replacement synthesis and watermark synthesis, but it is not limited thereto.
After the combination, the finish may be confirmed on the CRT 9, and a redo or a correction of the position or the like may be performed.
[0078]
The post-synthesis processing unit 40 determines an image conversion condition of the synthesized image into the image data P using the synthesized edge information obtained by the image synthesis unit 30, and based on the determined image conversion condition, The data P is converted to generate image data P ′ of the composite image after the post-processing. This conversion includes partial processing on the composite end and its surroundings, and overall processing such as blurring the whole.
[0079]
FIG. 6 shows an example of the processing area of the partial processing in the composite image of FIG. In FIGS. 6A to 6D, it is assumed that a process is executed on a portion indicated by a halftone dot. FIG. 6A shows the inside of a person, FIG. 6B shows the boundary between the person and the background and its peripheral portion, and FIG. 6C shows the background portion. Further, as shown in FIG. 6D, different partial processes may be executed depending on regions, such as executing a process of increasing sharpness inside a person and executing a process of decreasing sharpness of a background portion.
[0080]
As the image data conversion in the post-synthesis processing means 40, a spatial filter can be used. At the pixel position described in the combined edge information and its surroundings, a process of blurring the image data P of the combined image with a spatial filter is performed. As the spatial filter, various filters known in the art, such as a smoothing filter, may be appropriately used according to the purpose.
[0081]
The output image conversion unit 50 performs conversion corresponding to the output destination on the image data P ′ of the post-processed composite image, and converts the post-processed composite image D ′ into the CRT 9, the print creation unit 10, the image writing unit 11 or the communication unit 13.
[0082]
In the first embodiment, by using the composite edge information, appropriate conversion can be performed for each region of the composite image, so that the unnaturalness of the composite image can be reduced. In particular, since the synthesized image data can be converted into and around the boundary of the composite image, the unnaturalness of the boundary of the composite image can be reduced. Therefore, the synthesized image can be made a natural image with a very small number of work steps. In addition, since the partial processing is performed, the impression of the entire image can be maintained, which is suitable for fine correction of the composite image.
[0083]
[Embodiment 2]
FIG. 7 shows an example of an image synthesis process according to the second embodiment. The image processing unit 141 illustrated in FIG. 7 is an example of the image processing unit 14 of the image recording device 1 in FIG.
As shown in FIG. 7, the image processing unit 141 includes an input image conversion unit 20, an image synthesis unit 30, a post-synthesis processing unit 40, and an output image conversion unit 50.
[0084]
In the first embodiment, the image conversion condition of the composite image into the image data P is determined using the composite edge information. In the second embodiment, the image data P of the composite image is determined without using the composite edge information. The image data P of the composite image is converted based on the distortion information obtained from.
Since the input image conversion means 20 and the output image conversion means 50 are the same as those in the first embodiment, the description will be omitted.
[0085]
The image combining means 30 combines the image data G1 and G2 of the original image to generate image data P of the combined image. In the second embodiment, the image synthesizing unit 30 does not generate the synthesizing edge information, but the other parts are the same as those in the first embodiment, and thus the description is omitted.
[0086]
The post-synthesis processing unit 40 calculates the amount of distortion from the image data P of the synthesized image and generates distortion information. Then, using the distortion information, an image conversion condition of the synthesized image into the image data P is determined, and the image data P of the synthesized image is converted based on the determined image conversion condition. Generate image data P '.
[0087]
The distortion amount is related to the unnaturalness of an image, and may be calculated based on the magnitude of a change in image data, a change in gradient of image data, a change in S / N ratio, a change in sharpness, or the like. . The larger the amount of distortion, the more unnatural the image looks. When the amount of distortion is large, it is sometimes called discontinuity.
The distortion information includes pixel position information (coordinates) on the image data and a distortion amount at the pixel position.
[0088]
(Detailed example 1 of post-synthesis processing means 40)
FIG. 8 shows a detailed example 1 of the post-synthesis processing means 40 as an example of the post-synthesis processing means 40. The post-synthesis processing unit 40 includes a distortion amount calculation unit 420 and an image data conversion unit 430.
[0089]
The distortion amount calculation means 420 calculates a distortion amount from the image data P of the composite image and generates distortion information. An example in which a differential filter is used as the distortion amount calculation means 420 will be described. As the differential filter, various filters known in the art, such as a Laplacian filter, may be used as appropriate for the purpose. For example, the gradient is obtained by converting the image data P of the composite image at each pixel position by a differential filter or the like. The obtained gradient amounts are ranked and used as distortion amounts. For example, a rank value 1 is set below the gradient amount k, and a rank value 2 is set above the gradient amount k. The value of k is appropriately set according to the size of the composite image, the purpose, the use, and the like.
Note that the gradient amount may be classified not only into two ranks but also into n ranks.
Further, the gradient amount may be directly used as the rank value.
Further, the distortion amount calculation means 420 is not limited to the processing using the differential filter, and may calculate the distortion amount by another method.
[0090]
The image data conversion unit 430 determines the image conversion condition of the composite image into the image data P using the distortion information obtained by the distortion amount calculation unit 420, and based on the image conversion condition, determines the image data P of the composite image. To generate image data P ′ of the composite image after the post-processing.
Here, the image data P of the composite image is converted based on the distortion amount. The conversion based on the distortion amount refers to a conversion performed under a conversion condition corresponding to the distortion amount, but the distortion amount itself may be the conversion condition. The conversion based on the amount of distortion may be not only a conversion for reducing or removing the amount of distortion, but also a conversion that does not reduce or remove the amount of distortion but looks natural as if the amount of distortion is reduced.
[0091]
An example in which a spatial filter is used as the image data conversion unit 430 will be described. For example, at the pixel position described in the distortion information, a process of blurring the image data P of the composite image with a spatial filter is performed. At this time, the blur amount is determined according to the rank of the distortion amount. For example, when blurring with a 5 × 5 filter, the filter coefficient is changed according to the rank of the distortion amount. As the spatial filter, various filters known in the art such as a smoothing filter may be appropriately used according to the purpose, and the filter coefficient may be changed according to the rank.
Further, the processing using the spatial filter has been described as the image data conversion means 430, but another method may be used.
[0092]
In the second embodiment, the amount of distortion is calculated from the image data P of the combined image, and the image data P of the combined image is converted based on the distortion information including the amount of distortion and the pixel position information. Can be accurately reduced.
Further, since image data is converted based on the amount of distortion, necessary processing can be performed on a required portion. Therefore, the unnaturalness of the composite image can be corrected with a very small number of work steps. In addition, since the partial processing is performed, the impression of the entire image can be maintained, which is suitable for fine correction of the composite image.
[0093]
[Embodiment 3]
FIG. 9 shows an example of an image combining process according to the third embodiment. The image processing unit 142 illustrated in FIG. 9 is an example of the image processing unit 14 of the image recording device 1 in FIG.
As shown in FIG. 9, the image processing unit 142 includes an input image conversion unit 20, an image synthesis unit 30, a post-synthesis processing unit 40, and an output image conversion unit 50.
[0094]
In the second embodiment, the amount of distortion is calculated from the image data P of the composite image without using the composite edge information. In the third embodiment, however, the distortion is calculated from the image data P of the composite image using the composite edge information. Calculate the amount.
The input image converting means 20, the image synthesizing means 30, and the output image converting means 50 are the same as those in the first embodiment, and thus the description is omitted.
[0095]
The post-synthesis processing unit 40 includes a distortion amount calculation unit 420 and an image data conversion unit 430.
The distortion amount calculating unit 420 calculates the amount of distortion from the image data P of the synthesized image using the synthesized edge information obtained by the image synthesizing unit 30, and generates distortion information. For example, image data other than the composite end may be set to 0, and image data of the composite end may be set as image data having a rank value corresponding to the distortion amount.
The image data conversion means 430 determines an image conversion condition of the composite image into the image data P using the distortion information, converts the image data P of the composite image based on the determined image conversion condition, and performs post-processing. Is generated.
[0096]
Hereinafter, detailed examples 2 to 7 will be described as examples of the post-synthesis processing unit 40. It should be noted that the data handled by the distortion amount calculation means 420 and the image data conversion means 430 is not limited to the image data P of the composite image, but includes data obtained by converting the image data P of the composite image.
[0097]
(Detailed example 2 of post-synthesis processing means 40)
FIG. 10 shows a detailed example 2 of the post-synthesis processing means 40. The post-synthesis processing unit 40 includes a luminance / color difference conversion unit 410, a distortion amount calculation unit 420, an image data conversion unit 430, and a luminance / color difference inverse conversion unit 440.
[0098]
In the detailed example 2 of the post-synthesis processing means 40, after converting the image data P of the synthesized image into the luminance data B and the color difference data C, the distortion information is generated using the luminance data B, and the distortion amount is calculated for the luminance data B. Perform a conversion based on
[0099]
First, the luminance / color difference conversion unit 410 will be described. The luminance / color difference conversion unit 410 converts the image data P of the composite image into luminance data B and color difference data C. As the luminance / color difference conversion for converting the image data P of the composite image into the luminance data B and the color difference data C, when the image data is data based on the sRGB standard, CIE (Commission Internationale de l'Eclairage: International Commission on Illumination). There is an operation of converting to an L * a * b * base or an L * u * v * base recommended in 1976. In this case, L * corresponds to luminance. Alternatively, conversion may be performed such that the average value of sRGB is luminance data B and two axes orthogonal to the luminance data B are color difference signals. Moreover, you may convert from sRGB to sYCC. In this case, Y corresponds to luminance.
[0100]
The distortion amount calculation unit 420 calculates the distortion amount using the luminance data B of the composite image and the composite edge information obtained by the image composition unit 30, and generates distortion information. Specifically, the luminance data B of the composite image is converted by a differential filter or the like at the pixel position of the composite edge described in the composite edge information to obtain a gradient. Except for the fact that the data to be handled is the luminance data B and the use of the combined edge information, this is the same as the detailed example 1 of the post-synthesis processing means 40 in the second embodiment, and the description is omitted.
[0101]
In addition, the pixel position information of the distortion information may be the same as the pixel position of the combined edge described in the combined edge information, or may be a portion where the pixel position of the combined edge described in the combined edge information is broken. The shape of the contour, such as connection, may be shaped to change the pixel position.
[0102]
The image data conversion unit 430 converts the luminance data B of the composite image using the distortion information obtained by the distortion amount calculation unit 420, and generates corrected luminance data B '. Except that the data to be converted is the luminance data B, it is the same as the detailed example 1 of the post-synthesis processing means 40 in the second embodiment, and the description is omitted.
[0103]
The luminance / color difference inverse conversion means 440 performs the luminance / color difference inverse conversion on the corrected luminance data B ′ and color difference data C, and generates image data P ′ of the post-processed composite image. This inverse luminance / color difference conversion is an inverse conversion of the luminance / color difference conversion in the luminance / color difference conversion unit 410.
[0104]
In the detailed example 2 of the post-synthesis processing means 40, since the distortion amount is calculated using the synthesized edge information, the distortion amount can be calculated efficiently. In addition, the distortion amount calculation unit 420 and the image data conversion unit 430 handle only the luminance data B, thereby simplifying the process of correcting the composite image. Therefore, processing can be performed at high speed, which is preferable.
[0105]
(Detailed example 3 of post-synthesis processing means 40)
FIG. 11 shows a detailed example 3 of the post-synthesis processing means 40. The post-synthesis processing unit 40 includes a luminance / color difference conversion unit 410, a distortion amount calculation unit 420, an image data conversion unit 430, and a luminance / color difference inverse conversion unit 440.
[0106]
In the detailed example 2 of the post-synthesis processing means 40, after converting the image data P of the synthesized image into luminance data B and color difference data C, distortion information is generated using the luminance data B and the synthesized edge information, and the luminance data B On the other hand, the conversion based on the distortion amount was performed, but in the detailed example 3 of the post-synthesis processing means 40, the distortion information is generated using the luminance data B and the synthesized edge information, and the luminance generated to obtain the distortion information is generated. The conversion based on the distortion amount is performed on the intermediate data Btmp1 of the data B.
The luminance / color difference conversion unit 410, the distortion amount calculation unit 420, and the luminance / color difference inverse conversion unit 440 are the same as those in the detailed example 2 of the post-synthesis processing unit 40, and a description thereof will be omitted.
[0107]
The image data conversion unit 430 converts the intermediate data Btmp1 generated to obtain the distortion information by the distortion amount calculation unit 420 using the distortion information obtained by the distortion amount calculation unit 420, and outputs the corrected luminance data B 'Is generated. Except that the data to be converted is the intermediate data Btmp1, it is the same as the detailed example 2 of the post-synthesis processing means 40, and the description is omitted.
[0108]
In the detailed example 3 of the post-synthesis processing unit 40, the distortion amount calculation unit 420 handles the luminance data B, and the image data conversion unit 430 handles the intermediate data Btmp1 of the luminance data B, thereby using the result of the previous process. The correction processing of the composite image can be simplified. Therefore, processing can be performed quickly and efficiently, which is preferable.
[0109]
(Detailed example 4 of post-synthesis processing means 40)
FIG. 12 shows a detailed example 4 of the post-synthesis processing means 40. The post-synthesis processing unit 40 includes a luminance / color difference conversion unit 410, a distortion amount calculation unit 420, an image data conversion unit 430, and a luminance / color difference inverse conversion unit 440.
[0110]
The detailed example 4 of the post-synthesis processing means 40 is a high-frequency band obtained by performing orthogonal wavelet transform, which is one of multiple resolution conversions, as the intermediate data Btmp1 of the luminance data B in the detailed example 3 of the post-synthesis processing means 40. Component Wv _n , Wh _n , Wd _n This is an example using.
The luminance / color difference conversion unit 410 and the luminance / color difference inverse conversion unit 440 are the same as those in the detailed example 2 of the post-synthesis processing unit 40, and thus description thereof is omitted.
[0111]
First, the multi-resolution conversion will be described.
The multi-resolution conversion is a general term of a method represented by a wavelet transform, a complete reconstruction filter bank, a Laplacian pyramid, etc., and converts an input signal into a low frequency band component signal and a high frequency band component signal by one conversion operation. This is a technique in which the same operation is performed on the separated and obtained low frequency band component signals to obtain a multi-resolution signal including a plurality of signals having different frequency bands. If the obtained multi-resolution signal is subjected to inverse multi-resolution conversion without processing, the original signal is reconstructed (“Wavelet and Filter Banks” by G. Strang & T. Nguyen, Wellesley-Cambridge Press (Japanese translation) Wavelet Analysis and Filter Bank ", G. Strang and T. Nguyen, Baifukan).) Here, an outline of a wavelet transform that is a typical multi-resolution transform will be described.
[0112]
The wavelet transform refers to a wavelet function that oscillates in a finite range as illustrated in FIG. 13 (see the following equation (1)), and a wavelet transform coefficient <f, に 対 す る for the input signal f (x). _{a, b} > By the following equation (2) to decompose the input signal into a sum of wavelet functions represented by the following equation (3).
(Equation 1)

(Equation 2)

[Equation 3]

In the above equation (3), a represents the scale of the wavelet function, and b represents the position of the wavelet function. As illustrated in FIG. 13, as the value of the scale a increases, the wavelet function ψ _{a, b} The frequency of (x) becomes smaller, and the wavelet function ψ _{a, b} The position where (x) vibrates moves. Therefore, the above equation (3) shows that the input signal f (x) has a wavelet function を 持 つ having various scales and positions. _{a, b} (X) is decomposed into the sum.
[0113]
There are many known wavelet functions used for performing the wavelet transform. In the image processing field, in particular, orthogonal wavelet (orthogonal wavelet) transform and biorthogonal wavelet (bioorthogonal wavelet) capable of performing calculation at high speed. ) Transforms are widely used.
[0114]
Hereinafter, the outline of the orthogonal wavelet transform and the biorthogonal wavelet transform will be described. The wavelet functions of the orthogonal wavelet transform and the biorthogonal wavelet transform are defined as in the following equation (4).
(Equation 4)

Here, i is a natural number.
[0115]
Comparing Expression (4) with Expression (1), in the orthogonal wavelet transform and the biorthogonal wavelet transform, the value of the scale a is discretely defined by the power of 2 and the minimum movement unit of the position b is 2 ⁱ It can be seen that is defined discretely by. This value of i is called a level.
[0116]
In practice, the level i is limited to a finite upper limit N, and the input signal is converted as in the following equations (5), (6), and (7).
(Equation 5)

(Equation 6)

(Equation 7)

[0117]
The second term of the above equation (5) is a wavelet function of level 1 １ _{1, j} The low frequency band component of the residual that cannot be expressed by the sum of (x) is converted to a level 1 scaling function φ. _{1, j} (X). An appropriate scaling function is used corresponding to the wavelet function. The input signal f (x) ≡S by the one-level wavelet transform shown in the above equation (5) ₀ Is the level 1 high frequency band component W ₁ And low frequency band component S ₁ That is, the signal is decomposed into
[0118]
In the image, the high-frequency band component is a component that expresses a fine structure such as hair or eyelashes, and the low-frequency band component is a structure with a gradual change in signal intensity such as a cheek, for example. It is a component which shows.
[0119]
Wavelet function ψ _{i, j} The minimum movement unit of (x) is 2 ⁱ Therefore, the input signal S ₀ High frequency band component W ₁ And low frequency band component S ₁ Are の each, and the high frequency band component W ₁ And low frequency band component S ₁ Is the sum of the input signals S ₀ Signal amount. Level 1 low frequency band component S ₁ Is the high frequency band component W of level 2 in equation (6). ₂ And low frequency band component S ₂ , And by repeating the conversion up to the level N in the same manner, the input signal S ₀ Is decomposed into the sum of the high frequency band components of levels 1 to N and the low frequency band component of level N as shown in equation (7).
[0120]
Here, the one-level wavelet transform represented by the equation (6) can be calculated by a filter process as shown in FIG. In FIG. 14, LPF indicates a low-pass filter, and HPF indicates a high-pass filter. The filter coefficients of the low-pass filter LPF and the high-pass filter HPF are appropriately determined according to the wavelet function. 2 ↓ indicates a downsampling process for thinning out every other signal.
[0121]
One-level wavelet transform in a two-dimensional signal such as an image signal is calculated by a filter process as shown in FIG. As shown in FIG. 15, first, the low frequency band component S _n-1 Is filtered by a low-pass filter LPFx and a high-pass filter HPFx in the x direction, and downsampling is performed in the x direction. Thereby, the input signal S _n-1 Is SX _n And WX _n And is decomposed into This SX _n And WX _n , A filter process is performed by a low-pass filter LPFy and a high-pass filter HPFy in the y-direction, and a down-sampling process is performed in the y-direction.
[0122]
By this one-level wavelet transform, the low frequency band component S _n-1 Are the three high frequency band components Wv _n , Wh _n , Wd _n And one low frequency band component S _n Is decomposed into Wv generated by one wavelet transform _n , Wh _n , Wd _n , S _n Are the signal amounts before decomposition. _n-1 , The sum of the signal amounts of the four components after decomposition is S _n-1 Signal.
[0123]
In FIG. 15, x indicated by a suffix such as LPFx, HPFx, 2 ↓ x indicates processing in the x direction, and y indicated by a suffix such as LPFy, HPFy, 2 ↓ y indicates a process in the y direction. Indicates processing.
[0124]
FIG. 16 shows the input signal S ₀ Is a schematic diagram showing a process in which the signal is decomposed by a three-level wavelet transform. It can be seen that as the number of levels i increases, the image signal is thinned out by the downsampling process, and the decomposed image becomes smaller.
[0125]
In addition, as shown in FIG. _n , Wh _n , Wd _n , S _n Is subjected to the inverse wavelet transform calculated by the filter processing to obtain the signal S before decomposition. _n-1 It is known that can be completely reconstructed. In FIG. 17, LPF ′ indicates a low-pass filter for inverse transform, and HPF ′ indicates a high-pass filter for inverse transform. In addition, 2 ア ッ プ indicates an upsampling process of inserting a zero into every other signal. X indicated by a suffix such as LPF′x, HPF′x, 2 ↑ x indicates processing in the x direction, and y indicated by a suffix such as LPF′y, HPF′y, 2 ↑ y Indicates processing in the y direction.
[0126]
As shown in FIG. _n Is obtained by performing up-sampling processing in the y direction and filtering processing by a low-pass filter LPF′y for inverse conversion, and Wh _n And a signal obtained by performing up-sampling processing in the y-direction and filtering processing by a high-pass filter HPF'y for inverse conversion, and SX _n Get. Similarly, Wv _n And Wd _n From WX _n Generate
[0127]
Furthermore, SX _n And a signal obtained by performing up-sampling processing in the x direction and filtering processing by a low-pass filter LPF′x for inverse conversion, and WX _n Is added to a signal obtained by performing an up-sampling process in the x direction and a filter process by a high-pass filter HPF'x for inverse conversion to obtain a signal S before decomposition. _n-1 Can be obtained.
[0128]
The filter coefficients used in the inverse transform are the same as those used in the wavelet transform in the case of the orthogonal wavelet, but are different from the coefficients used in the wavelet transform in the case of the bi-orthogonal wavelet. Is used.
[0129]
As shown in FIG. 12, the distortion amount calculation unit 420 includes an orthogonal wavelet transformation unit 421, a distortion data extraction unit 422, and a distortion amount ranking unit 423.
The orthogonal wavelet transform unit 421 performs two-level orthogonal wavelet transform on the luminance data B of the composite image. That is, by repeating the orthogonal wavelet transform twice as shown in FIG. 15, the level 1 high frequency band component Wv ₁ , Wh ₁ , Wd ₁ And the high frequency band component Wv of level 2 ₂ , Wh ₂ , Wd ₂ And low frequency band component S ₂ Is obtained.
[0130]
The distortion data extracting means 422 outputs the high frequency band component Wv obtained from the luminance data B. ₁ , Wh ₁ , Wd ₁ , Wv ₂ , Wh ₂ , Wd ₂ From the distortion data. The distortion data is, for example, the high-frequency band component Wv at the pixel position of the combining edge described in the combining edge information. ₁ , Wh ₁ , Wd ₁ , Wv ₂ , Wh ₂ , Wd ₂ Is the image data value.
[0131]
The distortion amount ranking means 423 calculates the high frequency band component Wv ₁ , Wh ₁ , Wd ₁ , Wv ₂ , Wh ₂ , Wd ₂ Are classified according to the image data value of the image data, and are set as distortion amounts. Note that the image data value may be directly used as the rank value.
[0132]
The image data conversion means 430 includes rank-specific image data conversion means 431 and orthogonal wavelet inverse transformation means 432.
[0133]
The rank-based image data conversion unit 431 may, for example, use a high-frequency band component Wv at a pixel position described in the distortion information. ₁ , Wh ₁ , Wd ₁ , Wv ₂ , Wh ₂ , Wd ₂ Of the image data is reduced. At this time, the high frequency band component Wv ₁ , Wh ₁ , Wd ₁ , Wv ₂ , Wh ₂ , Wd ₂ Is changed according to the rank of the distortion amount. High frequency band component Wv ₁ , Wh ₁ , Wd ₁ , Wv ₂ , Wh ₂ , Wd ₂ Are the converted high frequency band components Wv, respectively. ₁ ', Wh ₁ ', Wd ₁ ', Wv ₂ ', Wh ₂ ', Wd ₂ Is converted to '.
[0134]
Next, the orthogonal wavelet inverse transform means 432 outputs the converted high frequency band component Wv. ₂ ', Wh ₂ ', Wd ₂ 'And low frequency band component S ₂ Is subjected to an inverse orthogonal wavelet transform as shown in FIG. ₁ Get ' Subsequently, the converted high frequency band component Wv ₁ ', Wh ₁ ', Wd ₁ 'And low frequency band component S ₁ ′ Is subjected to an inverse orthogonal wavelet transform to obtain corrected luminance data B ′.
[0135]
In the detailed example 4 of the post-synthesis processing means 40, the processing to a specific frequency band is performed at a specific position using a plurality of image data having different resolutions obtained by the orthogonal wavelet transform, so that the distortion can be improved with higher accuracy. Can be modified. Therefore, it is effective and preferable to efficiently reduce the unnaturalness of the combined edge while maintaining the impression of the entire image. Further, since the orthogonal wavelet transform is used, the processing can be performed at high speed, which is preferable.
Further, by using the orthogonal wavelet-transformed image data for calculating the distortion amount of the composite image data and using the image data for the image data conversion based on the distortion amount, the processing can be made more efficient.
[0136]
The orthogonal wavelet transform performed in the detailed example 4 of the post-synthesis processing means 40 may be a bi-orthogonal wavelet transform.
[0137]
(Detailed example 5 of post-synthesis processing means 40)
FIG. 18 shows a detailed example 5 of the post-synthesis processing means 40. The post-synthesis processing unit 40 includes a luminance / color difference conversion unit 410, a distortion amount calculation unit 420, an image data conversion unit 430, and a luminance / color difference inverse conversion unit 440.
[0138]
The detailed example 5 of the post-synthesis processing means 40 is a high-frequency band component Wx obtained by performing a binomial wavelet transform as the intermediate data Btmp1 of the luminance data B in the detailed example 3 of the post-synthesis processing means 40. _n , Wy _n This is an example using.
The luminance / color difference conversion unit 410 and the luminance / color difference inverse conversion unit 440 are the same as those in the detailed example 2 of the post-synthesis processing unit 40, and thus description thereof is omitted.
[0139]
Here, the outline of the binomial wavelet transform will be described. Compared with the orthogonal wavelet transform and the bi-orthogonal wavelet, the binomial wavelet (Dyadic Wavelet) transform does not thin out the image, so that the image can be processed with higher definition. For the binomial wavelet transform, see “Singularity detection and processing with wavelengths” by S.H. Mallat and W.M. L. Hwang, IEEE Trans. Inform. Theory 38 617 (1992) and "Characterization of signals from multiscale edges" by S.E. Mallat and S.M. Zhong, IEEE Trans. Pattern Anal. Machine Intel. 14710 (1992) and "A wavelet tour of signal processing 2ed" by S.M. Mallat, Academic Press has a detailed description.
[0140]
The wavelet function of the binomial wavelet transform is defined as the following equation (8).
(Equation 8)

Here, i is a natural number.
[0141]
The wavelet function of the above-described orthogonal wavelet transform and biorthogonal wavelet transform has a minimum movement unit of the position at level i of 2 ⁱ In the wavelet function of the binomial wavelet transform, the minimum movement unit of the position is constant regardless of the level i. This difference results in the following features of the binomial wavelet transform:
[0142]
As a first feature, a high-frequency band component W generated by a one-level binomial wavelet transform represented by the following equation (9) _i And low frequency band component S _i Is the signal S before conversion. _i-1 Is the same as
(Equation 9)

[0143]
As a second feature, the scaling function φ _{i, j} (X) and wavelet function ψ _{i, j} The relationship of the following expression (10) is established between (x).
(Equation 10)

Therefore, the high frequency band component W generated by the binomial wavelet transform _i Is the low frequency band component S _i Represents the first derivative (gradient).
[0144]
The third feature is that the coefficient γ is determined according to the level i of the wavelet transform. _i Multiplied by the high frequency band component _i ・ Γ _i (Hereinafter, this is referred to as a corrected high-frequency band component.) In accordance with the singularity of the signal change of the input signal, the converted corrected high-frequency band component W _i ・ Γ _i The relationship between the signal strength levels follows a certain law. That is, the corrected high frequency band component W corresponding to a gentle (differentiable) signal change as shown by "1" or "4" in FIG. _i ・ Γ _i 19 shows that the signal intensity increases as the number of levels i increases, whereas the corrected high frequency band component W corresponding to the step-like signal change shown in "2" of FIG. _i ・ Γ _i Is the corrected high frequency band component W corresponding to the signal change of the δ function shown in “3” in FIG. 19, regardless of the level number i. _i ・ Γ _i The signal intensity decreases as the number of levels i increases.
[0145]
As a fourth feature, the one-level binomial wavelet transform method for a two-dimensional signal such as an image signal is different from the above-described orthogonal wavelet transform or biorthogonal wavelet transform method in that a method shown in FIG. Is As shown in FIG. 20, in the one-level wavelet transform, the low frequency band component S _n-1 Is processed by the low-pass filter LPFx in the x direction and the low-pass filter LPFy in the y direction to obtain a low-frequency band component S _n Is obtained. Further, the low frequency band component S _n-1 Is processed by a high-pass filter HPFx in the x direction to obtain a high frequency band component Wx _n Is processed by the y-direction high-pass filter HPFy to obtain the high-frequency band component Wy. _n Is obtained.
[0146]
Thus, the low-frequency band component S is obtained by the one-level binomial wavelet transform. _n-1 Are two high frequency band components Wx _n , Wy _n And one low frequency band component S _n Is decomposed into Two high frequency band components Wx _n , Wy _n Is the low frequency band component S _n Change vector V in two dimensions _n X component and y component. Change vector V _n Size M _n And declination A _n Is given by the following equations (11) and (12).
[Equation 11]

(Equation 12)

[0147]
Two high frequency band components Wx obtained by the binomial wavelet transform _n , Wy _n And one low frequency band component S _n Is subjected to the inverse binomial wavelet transform shown in FIG. _n-1 Can be reconstructed. That is, the low frequency band component S _n And a high-frequency band component Wx obtained by processing with a low-pass filter LPFx in the x-direction and a low-pass filter LPFy in the y-direction. _n And a high-frequency band component Wy obtained by processing with a high-pass filter HPF′x for inverse transformation in the x direction and a low-pass filter LPF′y for inverse transformation in the y direction. _n Is added to a signal obtained by processing with a low-pass filter LPF′x for inverse transformation in the x direction and a high-pass filter HPF′y for inverse transformation in the y direction to obtain a signal S before decomposition. _n-1 Can be obtained.
[0148]
As shown in FIG. 18, the distortion amount calculating unit 420 includes a binomial wavelet transform unit 424, a distortion data extracting unit 422, and a distortion amount ranking unit 423.
The image data conversion means 430 includes rank-based image data conversion means 431 and inverse binomial wavelet conversion means 433.
[0149]
The detailed example 5 of the post-synthesis processing means 40 is obtained by replacing the orthogonal wavelet transform in the detailed example 4 of the post-synthesis processing means 40 with the binomial wavelet transform. Frequency band component Wx _n , Wy _n Other than that, since it is the same as the detailed example 4 of the post-synthesis processing means 40, the description is omitted.
[0150]
In the detailed example 5 of the post-synthesis processing means 40, by using a plurality of image data having different resolutions obtained by the binomial wavelet transform and performing processing to a specific frequency band at a specific position, more accurate The distortion can be corrected. Therefore, the unnaturalness of the composite image can be more accurately reduced, which is preferable. In addition, since the partial processing is performed, the impression of the entire image can be maintained, which is suitable for fine correction of the synthesized image, and the processing can be efficiently performed with extremely few work steps.
[0151]
The binomial wavelet transform used in the detailed example 5 of the post-synthesis processing means 40 is preferable for reconstructing an image because the image is not thinned out at the time of conversion. Therefore, artifacts are less likely to appear in the image after the image is converted back and returned, which is preferable.
[0152]
(Detailed example 6 of post-synthesis processing means 40)
FIG. 22 shows a detailed example 6 of the post-synthesis processing means 40. The post-synthesis processing unit 40 includes a distortion amount calculation unit 420 and an image data conversion unit 430.
[0153]
The detailed example 6 of the post-synthesis processing means 40 is a case where distortion information is generated based on the image quality evaluation value of the synthesized image. Here, the image quality evaluation value refers to gradation, sharpness, granularity, and the like.
[0154]
As the distortion amount calculating means 420, a method of generating distortion information based on the image quality evaluation value is used.
The distortion amount calculation unit 420 includes an image quality evaluation unit 425 and a distortion information calculation unit 426.
[0155]
The image quality evaluation unit 425 obtains an image quality evaluation value from the image data P of the composite image in an image region appropriately corresponding to the purpose using the composite edge information. As the image quality evaluation value, for the gradation, for example, the contrast is obtained from the histogram of the image data P of the composite image. Further, the sharpness is determined, for example, by comparing the sizes of image data of a plurality of high frequency band components obtained by performing a wavelet transform on the image data P of the composite image. The granularity is determined, for example, by comparing the sizes of image data of a plurality of high frequency band components obtained by performing a wavelet transform on the image data P of the composite image. It should be noted that other possible methods may be used when obtaining gradation, sharpness, granularity, and the like as image quality evaluation values.
[0156]
The distortion information calculation unit 426 calculates distortion information based on the image quality evaluation value obtained by the image quality evaluation unit 425.
[0157]
The image data conversion means 430 performs conversion based on distortion information based on each image quality evaluation value. For example, in terms of gradation, gradation conversion is performed, and in terms of sharpness and granularity, for example, conversion such as smoothing processing and unsharp masking processing is performed.
[0158]
In the detailed example 6 of the post-synthesis processing unit 40, distortion information is generated based on image quality evaluation values such as gradation, sharpness, and graininess of a synthesized image, and unnaturalness of the synthesized image can be reduced. preferable.
[0159]
(Detailed example 7 of post-synthesis processing means 40)
FIG. 23 shows a detailed example 7 of the post-synthesis processing means 40. The post-synthesis processing unit 40 includes a luminance / color difference conversion unit 410, a distortion amount calculation unit 420, an image data conversion unit 430, and a luminance / color difference inverse conversion unit 440.
[0160]
The detailed example 7 of the post-synthesis processing means 40 is an example using the binomial wavelet transform, similarly to the detailed example 5 of the post-synthesis processing means 40. Further, the detailed example 7 of the post-synthesis processing unit 40 is an example of generating distortion information based on the image quality evaluation value, similarly to the detailed example 6 of the post-synthesis processing unit 40. Is an example in which graininess evaluation is used for the distortion data extraction means 422.
The luminance / color difference conversion unit 410 and the luminance / color difference inverse conversion unit 440 are the same as those in the detailed example 2 of the post-synthesis processing unit 40, and a description thereof will be omitted.
[0161]
The distortion amount calculation unit 420 includes a binomial wavelet transformation unit 424, a distortion data extraction unit 422, and a distortion amount ranking unit 423. In the detailed example 7 of the post-synthesis processing unit 40, a graininess evaluation unit 427 is used as the distortion data extraction unit 422. In this example, a method is used in which the combined image data is divided into a plurality of regions based on the combined edge information, and another process is performed for each region. For example, in FIG. 6D, there is a method in which a region is divided into a background and a person and different processes are performed.
Since the binomial wavelet transform unit 424 is the same as the detailed example 5 of the post-synthesis processing unit 40, the description is omitted.
[0162]
The graininess evaluation means 427 will be described.
Level 1 high frequency band component Wx obtained by performing binomial wavelet transform on luminance data B of the composite image ₁ , Wy ₁ Image data and the level 2 high frequency band component Wx ₂ , Wy ₂ By comparing the size of the image data, the characteristic value of the granularity for each area described in the composite edge information is obtained.
[0163]
High frequency band component Wx of level 1 ₁ , Wy ₁ Is the size of the image data of AH1, and the level 2 high frequency band component Wx ₂ , Wy ₂ Is AH2. Pixels satisfying AH1> AH2 × α (0 <α <1) are extracted for each area as granular evaluation pixels. Then, the high-frequency band component Wx of level 1 in the granularity evaluation pixel for each region ₁ , Wy ₁ The average value of the size AH1 of the image data is obtained as the graininess evaluation value ARi (i is the number of the area).
Next, an average of the graininess evaluation values ARi in all the regions i is calculated, and a graininess evaluation average value ARavg is calculated. The ratio of the granularity evaluation value ARi to the average granularity evaluation value ARavg is determined for each region as a specific granularity evaluation value RRi = ARi / ARavg.
The granularity evaluation means 427 is not limited to this method.
[0164]
The distortion amount ranking means 423 performs a ranking according to the specific granularity evaluation value RRi to obtain a distortion amount for each region. Then, as the distortion information, the position of the granular evaluation pixel for each region i and the distortion amount for each region i are generated.
[0165]
The image data conversion means 430 includes rank-specific image data conversion means 431 and inverse binomial wavelet conversion means 433.
The rank-specific image data conversion means 431 converts the image data according to the rank of the distortion amount. In this example, the high frequency band component Wx of the granular evaluation pixel is set for each region i according to the distortion amount for each region i. ₁ , Wy ₁ , Wx ₂ , Wy ₂ Is performed so that the specific graininess evaluation value RRi for each region i falls within the predetermined value.
The binomial wavelet inverse transform unit 433 is the same as the detailed example 5 of the post-synthesis processing unit 40, and thus the description is omitted.
[0166]
In the detailed example 7 of the post-synthesis processing unit 40, the data of the synthesized image can be divided into regions by using the synthesized edge information, and different processing can be performed for each region, so that processing suitable for each region is performed. be able to. Therefore, the synthesized image can be corrected more naturally, which is preferable.
In this example, the graininess is evaluated. However, the amount of distortion may be obtained based on other image quality evaluation values such as sharpness and gradation, and corresponding image data conversion may be performed.
[0167]
In the detailed examples 4, 5, and 7 of the post-synthesis processing unit 40, in the orthogonal wavelet transform and the binomial wavelet transform, the case where the high frequency band components up to the level 2 are used has been described, but only the level 1 may be used. The distortion amount may be calculated using high frequency band components up to a large number of levels. For example, when a high frequency band component up to n levels is used, an orthogonal wavelet transform or a bi-orthogonal wavelet transform is used up to 1 to m (n <m) levels, and a binomial wavelet transform is used at m + 1 to n levels. It is possible to suppress a decrease in processing speed without greatly reducing accuracy. Therefore, it is preferable in terms of both accuracy and processing speed.
Although the wavelet transform has been described as an example of the multi-resolution conversion, another multi-resolution conversion may be used. Further, a plurality of multi-resolution conversions may be used in combination.
[0168]
[Embodiment 4]
FIG. 24 shows an example of the image synthesis processing as the fourth embodiment. The image processing unit 143 illustrated in FIG. 24 is an example of the image processing unit 14 of the image recording device 1 in FIG.
As shown in FIG. 24, the image processing unit 143 includes an input image conversion unit 20, a pre-synthesis processing unit 60, an image synthesis unit 30, a post-synthesis processing unit 40, and an output image conversion unit 50.
[0169]
In the fourth embodiment, before combining the image data G1 and G2 of the original image, preprocessing for reducing the difference in characteristics between the original images is performed.
The input image conversion means 20, the post-synthesis processing means 40, and the output image conversion means 50 are the same as those in the first embodiment, and a description thereof will be omitted.
[0170]
The pre-synthesis processing unit 60 performs image processing on the image data G1 and G2 of the original image to reduce the difference in characteristics between the original images, and generates pre-processed image data G1 ′ and G2 ′. Here, the characteristics mainly refer to characteristics relating to image quality, such as color, gradation, sharpness, and granularity.
[0171]
As the pre-synthesis process, there are various processes. For example, a process of obtaining a difference between image quality evaluation values and bringing characteristics close to each other is performed.
As the image quality evaluation value, for the gradation, for example, the contrast is obtained from the histogram of the image data G1 and G2 of the original image. The sharpness is obtained by comparing the sizes of image data of a plurality of high frequency band components obtained by performing wavelet transform on the image data G1 and G2 of the original image, for example. The graininess is obtained by comparing the sizes of image data of a plurality of high frequency band components obtained by performing wavelet transform on the image data G1 and G2 of the original image, for example. It should be noted that other possible methods may be used when obtaining gradation, sharpness, granularity, and the like as image quality evaluation values.
[0172]
Image processing is performed on the image data G1 and G2 of the original image based on the difference between the image quality evaluation values obtained from the image data G1 and G2 of the original image. For example, in terms of gradation, gradation conversion is performed, and in terms of sharpness and granularity, for example, conversion such as smoothing processing and unsharp masking processing is performed. The conversion may be performed on both the image data G1 and G2 of the original image, or the conversion may be performed such that one of the image data G1 and G2 of the original image is adjusted to the other.
[0173]
The image synthesizing means 30 synthesizes the preprocessed image data G1 'and G2' to generate image data P of a synthesized image and also generates synthesized edge information. Except that the data to be handled is the preprocessed image data G1 'and G2', the description is omitted because it is the same as that of the first embodiment.
[0174]
(Detailed example 1 of synthesis pre-processing means 60)
FIG. 25 shows a detailed example 1 of the pre-synthesis processing means 60 as an example of the pre-synthesis processing means 60 using the binomial wavelet transform. The pre-synthesis unit 60 includes a luminance / color difference conversion unit 610, a binomial wavelet conversion unit 620, a preprocessing image data conversion unit 630, a binomial wavelet inverse conversion unit 640, and a luminance / color difference inverse conversion unit 650. , Is provided.
[0175]
The luminance / color difference conversion unit 610 converts the image data G1 of the original image into luminance data V1 and color difference data U1, and converts the image data G2 of the original image into luminance data V2 and color difference data U2.
[0176]
The binomial wavelet transform unit 620 performs a binomial wavelet transform of the luminance data V1 to obtain a level 1 high frequency band component Wx ₁₁ , Wy ₁₁ And the high frequency band component Wx of level 2 ₂₁ , Wy ₂₁ And the low frequency band component S ₂₁ Generate Similarly, the luminance data V2 is subjected to a binomial wavelet transform to obtain a level 1 high frequency band component Wx ₁₂ , Wy ₁₂ And the high frequency band component Wx of level 2 ₂₂ , Wy ₂₂ And the low frequency band component S ₂₂ Generate
[0177]
The pre-processing image data conversion means 630 outputs the high-frequency band component Wx ₁₁ , Wy ₁₁ , Wx ₂₁ , Wy ₂₁ And the low frequency band component S ₂₁ Is converted to a high frequency band component Wx ₁₁ ', Wy ₁₁ ', Wx ₂₁ ', Wy ₂₁ 'And the low frequency band component S ₂₁ 'Is generated. Similarly, the high frequency band component Wx ₁₂ , Wy ₁₂ , Wx ₂₂ , Wy ₂₂ And the low frequency band component S ₂₂ Is converted to a high frequency band component Wx ₁₂ ', Wy ₁₂ ', Wx ₂₂ ', Wy ₂₂ 'And the low frequency band component S ₂₂ 'Is generated.
[0178]
FIG. 26 shows an example in which the pre-processing image data conversion unit 630 performs conversion based on granularity. The preprocessing image data conversion unit 630 includes a graininess evaluation unit 631 and an image conversion unit 632.
[0179]
The granularity evaluation means 631 calculates the level 1 high frequency band component Wx obtained by performing binomial wavelet transform on the luminance data V1. ₁₁ , Wy ₁₁ Is the size of the image data of BH1, and the level 2 high frequency band component Wx ₂₁ , Wy ₂₁ Assuming that the size of the image data of BH2 is BH2, pixels satisfying BH1> BH2 × β (0 <β <1) are extracted as granular evaluation pixels. Then, the level 1 high frequency band component Wx in the granularity evaluation pixel ₁₁ , Wy ₁₁ The average value of the image data size BH1 is determined as the graininess evaluation value BR1.
Similarly, a high frequency band component Wx of level 1 obtained by performing a binomial wavelet transform on the luminance data V2 ₁₂ , Wy ₁₂ Image data, high frequency band component Wx of level 2 ₂₂ , Wy ₂₂ The processing is also performed on the image data of No. to obtain the graininess evaluation value BR2.
[0180]
Next, an average of the graininess evaluation values BR1 and BR2 in the two original images is calculated, and a graininess evaluation average value BRavg is calculated. The ratio of the granularity evaluation values BR1 and BR2 to the average granularity evaluation value BRavg is determined as a specific granularity evaluation value BRR1 = BR1 / BRavg and BRR2 = BR2 / BRavg.
Then, the granularity evaluation unit 631 calculates image conversion conditions for each original image according to the specific granularity evaluation values BRR1 and BRR2, and sends the image conversion conditions of each image to the image conversion unit 632.
Note that the graininess evaluation means 631 is not limited to this method.
[0181]
The image conversion unit 632 performs the high frequency band component Wx based on the image conversion condition of each image obtained by the granularity evaluation unit 631. ₁₁ , Wy ₁₁ , Wx ₂₁ , Wy ₂₁ And the low frequency band component S ₂₁ Is converted to a high frequency band component Wx ₁₁ ', Wy ₁₁ ', Wx ₂₁ ', Wy ₂₁ 'And the low frequency band component S ₂₁ 'Is generated. Similarly, the high frequency band component Wx ₁₂ , Wy ₁₂ , Wx ₂₂ , Wy ₂₂ And the low frequency band component S ₂₂ Is converted to a high frequency band component Wx ₁₂ ', Wy ₁₂ ', Wx ₂₂ ', Wy ₂₂ 'And the low frequency band component S ₂₂ 'Is generated. By this conversion, the specific graininess evaluation values BRR1 and BRR2 are set to fall within predetermined values.
[0182]
As shown in FIG. 25, the inverse binomial wavelet transform unit 640 outputs the high frequency band component Wx ₁₁ ', Wy ₁₁ ', Wx ₂₁ ', Wy ₂₁ 'And the low frequency band component S ₂₁ Is used to perform inverse binomial wavelet transform to generate pre-processed luminance data V1 '. Similarly, the high frequency band component Wx ₁₂ ', Wy ₁₂ ', Wx ₂₂ ', Wy ₂₂ 'And the low frequency band component S ₂₂ ′ Is used to perform an inverse binomial wavelet transform to generate pre-processed luminance data V2 ′.
[0183]
The luminance / chrominance inverse transform unit 650 performs luminance / chrominance inverse transform on the pre-processed luminance data V1 ′ and chrominance data U1 generated by the binomial wavelet inverse transform unit 640, and converts the pre-processed image data G1 ′. Generate. Similarly, pre-processed image data G2 'is generated from pre-processed luminance data V2' and color difference data U2. This inverse luminance / color difference conversion is an inverse conversion of the luminance / color difference conversion in the luminance / color difference conversion unit 610.
[0184]
In the fourth embodiment, in order to adjust the basic characteristics of the image before the synthesis and to correct the unnaturalness of the synthesis boundary portion after the synthesis, the synthesized image is improved by the combined effect of both the processing before and after the synthesis. It can give a natural impression.
[0185]
In the first to fourth embodiments, an example has been described in which a composite image is created from two original images. However, a composite image may be created from three or more original images. Further, additional synthesis may be performed on the image after the synthesis is completed.
Further, in the first to fourth embodiments, only the image data is used. However, if necessary, image attached data other than the image data may be used.
[0186]
In the first to fourth embodiments, the image data conversion processing may be processing for the entire image to be converted or partial processing. As the partial processing, there are a method in which processing is performed only on the synthesis end portion and the periphery thereof, a method in which the processing is separated into synthesis regions, and another processing is performed on each region. Further, the image data may be divided into luminance data and color difference data, the luminance may be partially processed, and the color difference may be entirely processed. Further, the image data and the intermediate data of the composite image may be arbitrarily combined and used.
[0187]
In the first to fourth embodiments, examples of data to be converted include image data, luminance data converted from image data, intermediate data converted from luminance data, and the like. Other data such as intermediate data converted from the color difference data may be used according to the purpose, and is not limited.
[0188]
Further, each block of the first to fourth embodiments is a block provided to help understanding of the function of the image processing unit, and does not necessarily have to be physically independent. It may be implemented as a processing type block.
[0189]
【The invention's effect】
As described above, according to the first, seventh, thirteenth, or nineteenth aspects of the present invention, the appropriate conversion can be performed for each area of the composite image by using the composite edge information, so that the unnatural Can be reduced. In particular, since the synthesized image data can be converted into and around the boundary of the composite image, the unnaturalness of the boundary of the composite image can be reduced. Therefore, the synthesized image can be made a natural image with a very small number of work steps.
[0190]
As described above, according to the second, eighth, fourteenth, or twenty-second aspect, image data is converted based on the amount of distortion, so that necessary processing can be performed on required portions. Therefore, the unnaturalness of the composite image can be reduced with a very small number of work steps.
[0191]
As described above, according to the third, ninth, fifteenth, or twenty-first aspect, the amount of distortion is calculated using the combined edge information, so that the amount of distortion can be calculated efficiently. Therefore, it is possible to more accurately reduce the unnaturalness of the composite image.
[0192]
As described above, according to the invention as set forth in claim 4, 10, 16 or 22, it is possible to more accurately calculate the amount of distortion by using a plurality of image data having different resolutions by the multi-resolution conversion. Therefore, it is possible to more accurately reduce the unnaturalness of the composite image.
[0193]
As described above, according to the fifth, eleventh, seventeenth, or twenty-third aspect of the present invention, the multi-resolution converted image data is used for calculating the distortion amount of the composite image data, and the image data conversion based on the distortion amount , The processing can be made more efficient.
[0194]
As described above, according to the invention as set forth in claim 6, 12, 18 or 24, the basic characteristics of the images are aligned before the synthesis, and the unnaturalness of the synthesis boundary portion is corrected after the synthesis. The combined effect of both the post-combination and post-combination processes can make the combined image look more natural.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a general configuration of an image recording apparatus according to an embodiment.
FIG. 2 is a block diagram illustrating a functional configuration of the image recording apparatus.
FIG. 3 is a diagram illustrating an image synthesis process according to the first embodiment;
FIG. 4 is a diagram for explaining a composite end of a composite image.
FIG. 5 is a diagram illustrating an example in which a person is pasted on a background image.
FIG. 6 is a diagram illustrating an example of a processing area of a partial process in the composite image of FIG. 4;
FIG. 7 is a diagram illustrating an image combining process according to the second embodiment.
FIG. 8 is a diagram showing a detailed example 1 of the post-synthesis processing means 40;
FIG. 9 is a diagram illustrating an image combining process according to the third embodiment;
10 is a diagram showing a detailed example 2 of the post-synthesis processing means 40. FIG.
11 is a diagram illustrating a third detailed example of the post-synthesis processing unit 40. FIG.
12 is a diagram showing a detailed example 4 of the post-synthesis processing means 40. FIG.
FIG. 13 is a diagram showing a wavelet function used in the wavelet transform.
FIG. 14 is a diagram illustrating filter processing in orthogonal wavelet transform or bi-orthogonal wavelet transform.
FIG. 15 is a diagram illustrating one-level orthogonal wavelet transform or bi-orthogonal wavelet transform in a two-dimensional signal.
FIG. 16 is a schematic diagram showing a process of signal decomposition by three-level orthogonal wavelet transform or bi-orthogonal wavelet transform.
FIG. 17 is a diagram illustrating a method of reconstructing a signal decomposed by orthogonal wavelet transform or biorthogonal wavelet transform by inverse wavelet transform.
FIG. 18 is a diagram illustrating a detailed example 5 of the post-synthesis processing means 40.
FIG. 19 is a diagram illustrating characteristics of a signal subjected to wavelet transform.
FIG. 20 is a diagram showing a binomial wavelet transform.
FIG. 21 is a diagram showing an inverse binomial wavelet transform.
22 is a diagram illustrating a detailed example 6 of the post-synthesis processing unit 40. FIG.
FIG. 23 is a diagram showing a detailed example 7 of the post-synthesis processing means 40.
FIG. 24 is a diagram illustrating an image combining process according to the fourth embodiment.
FIG. 25 is a diagram showing a detailed example 1 of the pre-synthesis processing means 60.
FIG. 26 is a diagram illustrating an example of a preprocessing image data conversion unit 630.
[Explanation of symbols]
1 Image recording device
2 CPU
3 ROM
4 RAM
5 Operation section
6 Film scanner
7 Reflection document input section
8 Image reading unit
9 CRT
10 Print making section
11 Image writing unit
12 Storage unit
13 Communication unit
14 Image processing unit
15 trays
16 Magazine
20 Input image conversion means
30 Image synthesis means
40 Post-synthesis processing means
50 Output image conversion means
60 synthesis preprocessing means
410 brightness / color difference conversion means
420 distortion amount calculation means
421 orthogonal wavelet transform means
422 distortion data extraction means
423 Distortion amount ranking means
424 Binomial wavelet transform means
425 Image quality evaluation means
426 Distortion information calculation means
427 Graininess evaluation means
430 Image data conversion means
431 Image data conversion means by rank
432 Orthogonal wavelet inverse transform means
433 Binomial wavelet inverse transform means
440 luminance / color difference inverse conversion means
610 luminance / color difference conversion means
620 Binomial wavelet transform means
630 Preprocessing image data conversion means
631 Graininess evaluation means
632 Image conversion means
640 Binary wavelet inverse transform means
650 luminance / color difference inverse conversion means

Claims (24)

In an image composition method for creating a composite image using two or more images,
Determining an image conversion condition for the synthesized image data using the synthesized edge information regarding a boundary where the two or more images are in contact with each other; and converting the synthesized image data based on the determined image conversion condition. An image synthesizing method, comprising a post-synthesis processing step. In an image composition method for creating a composite image using two or more images,
A distortion amount calculating step of calculating a distortion amount from the image data after synthesis,
An image data conversion step of determining an image conversion condition to the synthesized image data based on the calculated distortion amount, and converting the synthesized image data based on the determined image conversion condition,
An image synthesizing method comprising: The image synthesizing method according to claim 2,
An image combining method, wherein in the distortion amount calculation step, an amount of distortion is calculated using composite edge information on a boundary where the two or more images are in contact with each other. The image synthesizing method according to claim 3,
The image synthesizing method, wherein the distortion amount calculating step includes a multi-resolution conversion. The image synthesizing method according to claim 4,
An image synthesizing method, wherein the conversion into image data after synthesis in the image data conversion step is conversion into image data after multi-resolution conversion. The image synthesizing method according to any one of claims 1 to 5,
An image synthesizing method, comprising: a pre-synthesis processing step of performing pre-synthesis processing using multi-resolution conversion on at least one image before synthesis. In an image synthesizing apparatus that creates a synthetic image using two or more images,
Determining an image conversion condition for the synthesized image data using the synthesized edge information regarding a boundary where the two or more images are in contact with each other; and converting the synthesized image data based on the determined image conversion condition. An image synthesizing apparatus, comprising: a post-synthesis processing unit that performs a synthesizing process. In an image synthesizing apparatus that creates a synthetic image using two or more images,
Distortion amount calculating means for calculating the distortion amount from the image data after the combination,
Image data conversion means for determining an image conversion condition to the combined image data based on the calculated distortion amount, and converting the combined image data based on the determined image conversion condition,
An image synthesizing apparatus comprising: The image synthesizing device according to claim 8,
The image synthesizing apparatus, wherein the distortion amount calculating means calculates an amount of distortion by using synthesized edge information regarding a boundary where the two or more images are in contact with each other. The image synthesizing device according to claim 9,
The image synthesizing device, wherein the distortion amount calculating means performs multi-resolution conversion. The image synthesizing device according to claim 10,
An image synthesizing apparatus, wherein the conversion into the image data after the synthesis by the image data conversion means is the conversion into the image data after the multi-resolution conversion. The image synthesizing device according to any one of claims 7 to 11,
An image synthesizing apparatus, comprising: a pre-synthesis processing unit that performs pre-synthesis processing using multi-resolution conversion on at least one image before synthesis. A computer for executing a process of creating a composite image using two or more images,
The image conversion condition to the synthesized image data is determined using the synthesized edge information regarding the boundary where the two or more images are in contact with each other, and the synthesized image data is converted based on the determined image conversion condition. An image composition program for realizing the post-composition processing function. A computer for executing a process of creating a composite image using two or more images,
A distortion amount calculation function for calculating a distortion amount from the image data after synthesis;
An image data conversion function of determining an image conversion condition to the combined image data based on the calculated distortion amount, and converting the combined image data based on the determined image conversion condition,
An image synthesis program for realizing. The image composition program according to claim 14,
An image synthesis program, wherein the distortion amount calculation function calculates an amount of distortion using synthesized edge information on a boundary where the two or more images are in contact with each other. The image composing program according to claim 15, wherein
The image synthesis program, wherein the distortion amount calculation function includes a multi-resolution conversion. The image synthesizing program according to claim 16,
An image synthesizing program, wherein the conversion into image data after synthesis in the image data conversion function is conversion into image data after multi-resolution conversion. The image synthesizing program according to any one of claims 13 to 17,
To the computer,
An image synthesizing program for realizing a pre-synthesis processing function of performing pre-synthesis processing using multi-resolution conversion on at least one image before synthesis. In an image recording apparatus that creates a composite image using two or more images and records the composite image on a recording medium,
Determining an image conversion condition for the synthesized image data using the synthesized edge information regarding a boundary where the two or more images are in contact with each other; and converting the synthesized image data based on the determined image conversion condition. An image recording apparatus, comprising: a post-synthesis processing unit that performs the processing. In an image recording apparatus that creates a composite image using two or more images and records the composite image on a recording medium,
Distortion amount calculating means for calculating the distortion amount from the image data after the combination,
Image data conversion means for determining an image conversion condition to the combined image data based on the calculated distortion amount, and converting the combined image data based on the determined image conversion condition,
An image recording apparatus comprising: The image recording apparatus according to claim 20,
The image recording apparatus according to claim 1, wherein the distortion amount calculating unit calculates the amount of distortion using combined edge information regarding a boundary where the two or more images are in contact with each other. The image recording apparatus according to claim 21,
The image recording apparatus according to claim 1, wherein the distortion amount calculation unit performs multi-resolution conversion. The image recording apparatus according to claim 22,
The image recording apparatus according to claim 1, wherein the conversion into the image data after the synthesis by the image data conversion means is a conversion into the image data after the multi-resolution conversion. The image recording apparatus according to any one of claims 19 to 23,
An image recording apparatus comprising: a pre-synthesis processing unit that performs pre-synthesis processing using multi-resolution conversion on at least one image before synthesis.

Images