JP4129097B2 - Image processing apparatus and image processing method - Google Patents

Description

この場合、補間画像は平均化されたものとなりやすくエッジ等が不鮮明になりやすいので、元の原画像より抽出されたエッジ情報を使った補間画素位置や補間データの調整処理が行われることがある。図１５においては、エッジ抽出手段１６００でエッジ情報を抽出された後、エッジ補間手段１６０１で所望の拡大率に合わせて拡大エッジを求める。そして、この拡大エッジ情報と画素間補間手段１１００による原画像の補間拡大画像の結果を掛け合わせることで、所望の拡大画像を生成し推定拡大画像出力手段１４０２で出力するようになっている。
【００１２】
【発明が解決しようとする課題】
しかし、従来の電子スチルカメラ等の画像処理装置では、１度圧縮され記憶された画像データをもう１度伸張してやる必要があり、しかも圧縮方法として量子化処理を伴う非可逆圧縮方法を用いるため、一般に元の原画像には戻ることがなく、ノイズや色ずれを多少生じることがあった。
【００１３】
また、レビュー用に表示される縮小画像（以下、サムネイル画像と呼ぶ）生成に必要な解像度変換時における間引き処理により色ずれやモアレを生じてしまうという欠点があり、そのためにメジアンフィルタや平均値フィルタによる平滑化処理が不可欠であった。そのため、サムネイル画像の鮮明さが失われたり、これらの平滑化処理に非常に時間がかかってしまうというような問題点が指摘されていた。
【００１４】
さらにスチルカメラのCCD の低解像度をプリントしたりディスプレイに表示したりする際に必要とされる従来の画像拡大においては、次のような問題点があった。
【００１５】
まず、（１）の周波数変換した空間での補間による方法の場合、一般に画像のサイズは任意であり、そしてＤＣＴにかかるサイズが大きくなればなるほど処理時間が長くなるという問題があった。このような処理速度等の問題を解決するために、一般には画像サイズ全体を１度に直交変換を掛けることはせず、４画素から１６画素程度のブロックサイズ単位で行う方法が取られたが、このような方法によって拡大処理をすると、出来上がった拡大画像のブロック間の不連続性（ブロック歪み）が境界部分に生じることがあった。
【００１６】
また、（２）の単純に画素間を補間する場合には、このような低解像度のデバイスで自然画像を取った場合、その被写体の持つカラーバランスによっては、得られた原画像からの図１５におけるエッジ抽出が精度良く行われないことが多く、その結果、画素補間された位置やその補間データの調整がうまくできないという問題があった。これを避けるために、できた拡大画像にエッジ強調フィルタによる処理を複数繰り返すことが行われたが、従来のエッジ強調フィルタ処理の場合、適正なフィルタを選ばないと、その繰り返し回数は非常に大きくなったり、全くエッジ強調がされない等の欠点があった。
【００１７】
【課題を解決するための手段】
上記課題を解決するために、本発明は、以下の手段を採用している。すなわち、本発明はディジタルカメラ等より得られる画像データを加工する画像処理装置において、まず図１に示すように、画像入力手段１０より得られる画像データに対して、それを原画像直交変換手段１１にて周波数空間のデータとする直交変換を施して原画像の周波数成分である原画像周波数成分を生成し、そして低成分抽出手段１００で、該原画像周波数成分からその低周波成分を抽出する。次いで、高成分符号化手段１３で、前記低成分抽出手段１００にて抽出された残りである前記原画像周波数成分の高周波成分を前記低周波成分と関連づけて、その情報である関連情報を符号化するとともに、符号合成手段１４で前記低周波成分を表したデータと前記関連情報を表したデータとを合成して、前記原画像の画像データよりデータサイズの小さい画像データである簡易画像データを生成するようになっている。
【００１８】
上記のように生成された簡易画像データは、そのデータから、低成分復号手段１６によって低周波成分を表したデータが抽出されるとともに、高成分復号手段１７によって、前記関連情報を表したデータが取り出されて、その関連情報を表したデータと前記低周波成分を表したデータとに基づいて該高周波成分を表したデータが復号され、そして、原画像出力手段１８によって、前記低周波成分を表したデータと高周波成分を表したデータとが結合されたものが周波数成分とされて、その周波数成分に逆直交変換が施されて、原画像が復元されるものである。
【００１９】
これによって、原画像の画像データよりデータサイズの小さい簡易画像データを扱うことができるとともに、原画像に近い画質の復元画像を得ることができることになる。
【００２０】
前記簡易画像データは直接、パソコン等の画像を復元できる手段に入力して処理をすることも可能であるが、一旦記憶媒体１５に記憶させることも可能である。また、前記簡易画像データにおいて、前記低周波成分のデータ量を図３に示すように低成分圧縮手段３００で更に圧縮することも可能であるが、この場合可逆性を持った圧縮方法を用いるのが好ましい。
【００２１】
前記低周波成分として抽出する基準の周波数には、縮小画像生成手段１０１で規定される、画像サイズが原画像より縮小されたサイズである縮小サイズ（例えばプレビューのサイズ）に対応した周波数を用いてもよく、該低周波成分を表したデータに逆直交変換を施すことで該縮小サイズの縮小画像を生成することができる。
【００２２】
また、本発明は、原画像を所望の画像サイズに拡大する際に、図５に示すように不足成分推定手段５００が、原画像の周波数成分に補うべき高周波成分を前記原画像の周波数成分、及び該原画像の周波数成分と補うべき該高周波成分との関連を示した情報である関連情報を基に推定し、拡大画像出力手段５０１によって、前記原画像の周波数成分と、前記不足成分推定手段にて推定された前記高周波成分とを結合したものを拡大した画像の周波数成分として、その周波数成分に逆直交変換を施すことで所望の画像サイズに拡大した画像を出力するように構成してもよい。
【００２３】
更に、本発明は図７に示すように、画像入力手段１０から入力された画像データを加工する画像処理装置において、原画像直交変換手段１１で、原画像として入力された画像データに直交変換を施して原画像周波数成分を生成するとともに、該原画像周波数成分を、拡大周波数推定手段８００により、該原画像を所望の拡大率で拡大した拡大画像の周波数成分である拡大周波数成分における低周波成分とみなして、その拡大周波数成分を、前記原画像周波数成分及び該拡大周波数成分における低周波成分と残りの高周波成分との関連を示した情報である関連情報を基に推定する。上述のようにして得られた原画像周波数成分と複数の拡大率についての拡大周波数成分とから、基本成分抽出手段８０７が予め画像サイズが指定された基本画像を生成するのに必要となる周波数成分を基本成分として抽出し、多重画像符号化手段８０２で、推定された前記拡大周波数成分を前記基本成分と関連づけて、その情報である関連情報を符号化する。そして、このようにして得られた基本成分を表したデータと前記関連情報を表したデータとを多重符号合成手段８０３で合成して、前記原画像の画像データよりデータサイズの小さい多重簡易画像データを生成するように構成してもよい。ここで、基本画像とは、一の原画像から拡大率の異なるすなわち画像サイズの異なる複数の画像に拡大するためにその拡大する元として共通した画像サイズの画像のことである。
【００２４】
これによって、画像データを符号化する側で複数のサイズに対応する多重簡易画像データを生成することが可能となる。
【００２５】
このようにして生成された多重簡易画像データからは、前記基本成分を表したデータと前記関連情報を表したデータとが抽出されて、この両者に基づいて拡大画像を生成できることになる。
【００２６】
前記多重簡易画像データにおいても、前記基本成分のデータ量を圧縮することによって、更に、データサイズを小さくすることができることになる。
【００２７】
また、本発明は、図１０に示すように、画像入力手段１０から入力された画像データを画素間補間手段１１００が所望の拡大率に応じて原画像の画素間を新たな画素で補間するものとしてもよい。これにより得られた補間拡大画像はエッジ部分が不鮮明になるが、該補間拡大画像に対して畳み込み手段１１０１がエッジ部分を強調する畳み込み計算を行うことで、処理時間のかかる周波数変換を経ずに鮮明なエッジを持つ拡大画像を生成する。
【００２８】
【発明の実施の形態】
以下、本発明の実施の形態について図面を参照しながら説明する。なお、これ以降で、座標値の単位には全て画素単位が用いられることとする。
【００２９】
（第１の実施の形態）
まず、本発明の第１の実施の形態である画像処理装置について説明する。
【００３０】
図１において、CCD 撮像素子などの画像入力手段１０で得られたN 画素×N 画素のサイズを持つ原画像は、原画像直交変換手段１１で周波数空間における成分データ（周波数成分データ）に変換される。この際、直交変換が使用され、その変換方式としては、アダマール変換や高速フーリエ変換(FFT; Fast Fourier Transform) 、離散コサイン変換(DCT; Discrete Cosine Transform)、スラント変換、ハール変換等が挙げられるが、ここでは、ＤＣＴを使用することとする。しかし、本発明はこの変換方式に限ったものではなく、他の直交変換を使用した場合にも成立するものである。
【００３１】
ＤＣＴの内、特に画像を扱うため２次元ＤＣＴになるが、その変換式は（数２）のようになる。
【００３２】
【数２】

ここで、F[u,v]が成分位置(u,v) におけるＤＣＴ成分であり、D[x,y]は画素位置(x,y) における画像データを表す。また、はK ＿x はx 方向の画素数を、K ＿y はy 方向の画素数を表すが、ここではN ×N の正方形画像を対象としているので、K ＿x = K ＿y = N となる。
【００３３】
低成分抽出手段１００では、図２（ｂ）が示すように原画像の周波数成分データの低周波成分を抽出する。
【００３４】
高成分符号化手段１３では、図２（ｂ）の原画像の周波数成分から低周波成分（ｃ）の分を除いたときに残る、高周波成分領域（ｅ）の符号化を行う。ここでは、図２（ｅ）に示すように例えば、前記高周波成分の領域をH1からH5の小ブロックに分割する。同様に図２（ｆ）に示すように、低周波成分（ｃ）の領域もL1からL4のブロックに分割する。そして、前記高周波成分（ｅ）のブロックH1からH5の各ブロック内の周波数成分と低周波成分（ｆ）の各ブロックL1からL4とを相互に関連づけるようにする。例えば、（数３）のようにある定数係数を乗算した１次式の形でも良いし、各ブロックL1からL4の周波数成分を変数に持つある多次元関数Ψ(L1,L2,L3,L4)で各ブロックH1からH5内の周波数成分を近似しても良い。
【００３５】
【数３】

また、各ブロック内においても一律の係数を持たせる必要はなく、各ブロックH1からH5に対し係数マトリックスM ＿C1からM ＿C5を用意し、それとL1からL4のブロック内の周波数データより構成され低周波成分マトリックスM ＿L1からM ＿L4を使ってH1からH5のブロック内の周波数データより構成され高周波成分マトリックスM ＿H1からM ＿H5を表現することも考えられる。いずれの手段でも、図２（ｃ）の低周波成分データを使って、画像の鮮明さやエッジ情報を表す残りの図２（ｅ）の高周波成分を精度良くかつルール化することで、高周波成分のデータ容量を削減できる。
【００３６】
符号化記憶手段１４は、高成分符号化手段１３で得られたルール記述、もしくは該高成分符号化手段１３で用いた近似式の係数等のデータより得られる、前記低周波成分と高周波成分の関連情報の符号列と、元の原画像の低周波成分データとを合成し、簡易画像データとして記憶媒体１５に記憶する処理を行う。なお、次に読み出して復元する処理のために、記憶媒体内のヘッダー情報に、記憶した画像データ数、各画像データのサイズ、記憶された各データにおける低周波成分データの占める割合や高周波成分を表す符号の占める割合等の情報を記述することで、各データの抽出が効率的になる。
【００３７】
（第２の実施の形態）
次に、本発明の第２の実施の形態である画像処理装置について説明する。
【００３８】
図１において、画像入力手段１０及び原画像直交変換手段１１の行う処理は第１の実施の形態の画像処理装置と同様なので説明を省略する。低成分抽出手段１００は、サムネイル画像を表示する縮小画像表示手段１０２の画素数に応じた低周波成分を抽出し、縮小画像生成手段１０１で該低周波成分に逆直交変換を施し、サムネイル画像を前記縮小画像表示手段に表示する。
【００３９】
（第３の実施の形態）
次に、本発明の第３の実施の形態である画像処理装置について説明する。
【００４０】
図１において、ユーザは本発明の第１の実施の形態である画像処理装置により記憶媒体１５に記憶された簡易画像データの中から、高解像度のレーザプリンタやインクジェットプリンタへ出力したり、CRT上で編集を行う所望の画像を取り出すために、指示信号を与える。この指示信号をもとに、まず低成分復号手段１６が記憶媒体１５より、中心となる低周波成分を取り出す。そして、高成分復号手段１７が該低成分復号手段１６で得られた低周波成分と、記憶媒体１５内の該低周波成分と高周波成分との関連情報とより高周波成分を復号する。そして、原画像出力手段１８が、この低周波成分データと高周波成分データを結合し、対応する画像サイズの逆直交変換を行うことで、CRTへの表示やプリンタへの出力等、他の画像処理装置で扱われるデータとして出力する。
【００４１】
記憶媒体１５に符号化され、記憶されている画像データは、量子化処理を受けていないために、この方法により従来より鮮明な原画像を得ることができる。
【００４２】
（第４の実施の形態）
次に、本発明の第４の実施の形態である画像処理装置について説明する。
【００４３】
図３において、原画像直交変換１１、低成分抽出手段１００、高成分符号化手段１３、符号合成手段１４及び記憶媒体１５の行う処理は第１の実施の形態の画像処理装置と同様なので説明を省略する。低成分抽出手段１００で得られた原画像の低周波数成分データの容量がそれ自身大きなサイズを占めることや、記憶媒体の容量サイズによる制限等を考慮して該低周波成分データを低成分圧縮手段３００により圧縮する。
【００４４】
図４に示す低成分圧縮手段３００の処理方式は、ＪＰＥＧにおける空間的予測(Spatial)方式と呼ばれ、通常のＤＣＴと量子化、エントロピー符号化とを用いた圧縮処理（ベースライン方式とも呼ぶ）に代わって、差分符号化（DifferentialPCM；DPCM)とエントロピー符号化とを使用したものであり、圧縮、伸張によってひずみが生じない可逆方式である。
【００４５】
この方式では、隣接する画素値を用いて予測器によって予測値を計算し、符号化する画素値からその予測値を減ずるものであり、この予測器には（数４）に示すように、例えば７種類の関係式が用意されている。
【００４６】
【数４】

図４（ａ）に示すように、符号化すべき画素値をDxとし、それに隣接する３つの画素値をD1, D2, D3とする。このとき、これら３つの画素値より計算される予測値dDx は、（数４）で定義され、いずれの式を使ったかはヘッダー情報に記載される。符号化処理では図４（ｃ）に示すように、１つの予測式が選択され、Exが計算され、このExをエントロピー符号化する。このような予測誤差の符号化により、前記低周波成分を可逆的に圧縮することが可能となる。
【００４７】
低成分圧縮手段３００により圧縮された前記低周波成分データを基に、高成分符号化手段１３は残りの高周波成分を第一の実施の形態と同様に（数３）に示すような方法などでルール化する。
【００４８】
符号合成手段１４は圧縮された前記低周波成分データと、高成分符号化手段１３で得られたルール記述、もしくは該高成分符号化手段１３で用いた近似式の係数等のデータから得られる、前記低周波成分と高周波成分との関連情報の符号列とを合成し、簡易画像データとして記憶媒体１５に記憶する処理を行う。
【００４９】
このような本実施例の形態にすることで、第１の装置のデータ容量で最も大きな部分を占めると思われる低周波成分を１／２から１／４のデータ量に圧縮することが可能となる。また、この圧縮に可逆な方法を用いることで原画像の持つ周波数成分を損なうことなく記憶することが可能となる。
【００５０】
なお、ここでは前記低周波成分データは画像の細部の鮮明さを表す高周波成分を復元するための基準画像であることから、できるだけ鮮明でエッジの明瞭な画像出力を実現するために原画像の低周波数成分に可逆な圧縮方法を適用したが、ここに通常のＪＰＥＧのベースライン方式を適用することも可能である。この場合は非可逆圧縮であるので画像の鮮明さは失われるが、データ量はおよそ１／１０から１／２０に圧縮されるので、記憶メディアにおいて保持される読み取り画像容量をできるだけ小さくして多くの画像データを記憶したりするような場合に取り得る方法である。
【００５１】
（第５の実施の形態）
次に本発明の第５の実施の形態である画像処理装置について説明する。
【００５２】
図５において、記憶媒体１５、低成分復号手段１６及び高成分復号手段１７までの処理は、本発明の第３の実施形態と同様なので説明を省略する。
【００５３】
不足成分推定手段５００は、読み取られた原画像を高解像度のCRT等に表示する場合において、所望の画素サイズ（画像サイズ）に拡大した際に不足する高周波成分データを推定する手段であり、拡大画像出力手段５０１は、不足成分推定手段５００で推定された前記不足高周波成分と、低成分復号手段１６及び高成分復号手段１７で記憶媒体１５より復号された原画像の周波数成分とを結合し、拡大サイズに対応する逆直交変換手段を行うことで、読み取られた原画像の拡大画像を生成する手段である。
【００５４】
不足成分推定手段５００及び拡大画像出力手段５０１では、図６に示すように処理が行われる。まず図６（ａ）に示すN画素×N画素のサイズを持つ原画像を直交変換して得られる図６（ｂ）に示すような周波数成分データを、所望の拡大サイズ(sN画素×sN画素）に対応する係数サイズを持つ拡大画像の周波数成分の低域に埋め込む（図６（ｃ））。そして、この際に生じる不足成分FH1からFH3を図６（ｂ）に示す原画像の周波数成分より推定するのである。その推定方法には、例えば、
（１）特願平１１−６８１５１のようにラジアル基底関数ネットワーク（RadialBaseFunctionNetwork;RBFN）の非線形近似能力を適用する方法
（２）同じく特願平１１−６８１５１のように原画像のエッジ画像の周波数成分より、その拡大エッジ画像の持つ周波数成分を線形近似もしくはラジアル基底関数ネットワークを使って精度良く推定することで、原画像で削除されていた拡大画像の高周波成分を推定する方法
（３）特開平８−２９４００１のように、原画像の直交変換成分を低周波領域に埋め込み、高周波領域は予め準備された予測ルールに基づいて得られる周波数情報を埋める方法
（４）特開平６−５４１７２のように画像信号を直交変換を用いて正変換とその逆変換を繰り返す過程（Gerchberg-Papoulis反復による方法）で、高周波成分の復元を行う方法
等が挙げられる。
【００５５】
これ以外にも多くの周波数領域を使った画像拡大手法があるが、この不足成分推定手段５００では、拡大時に細部が鮮明でエッジが明瞭な画像の生成に必要な高周波成分を精度良く推定する手法が求められるのであり、その要望を満たす手法であれば同様に成立するものである。
【００５６】
そして、拡大画像出力手段５０１では図６（ｄ）に示すようにsN×sNの係数サイズに対応する逆直交変換手段を行うことで、不足成分推定手段５００で推定された拡大画像の周波数成分を実空間データに戻し、CRT 等に表示したり、プリンタ等の出力装置に渡したり、他の画像処理装置で扱われるデータとして出力する。
【００５７】
以上のような構成により、原画像の周波数成分を使って拡大画像出力時に必要とされる高周波成分データを推定し、補うことで、従来の画素補間による拡大時に発生するエッジのボケや細部の不鮮明さを解消することができる。
【００５８】
（第６の実施の形態）
次に本発明の第６の実施の形態である画像処理装置について、図７及び図８を参照しながら説明する。
【００５９】
図７において、画像入力手段９及び原画像直交変換手段１１の行う処理は、本発明の第１の実施形態と同様なので説明を省略する。まず原画像直交変換手段１１で得られた原画像の周波数成分を使い、拡大周波数推定手段８００では複数の画像サイズに拡大した際の周波数成分データが推定される。この推定方式は、本発明の第５の実施の形態である画像処理装置の構成要素の１つである図５の不足成分推定手段５００と同様の手法を取ることができる。
【００６０】
次に基本成分の抽出処理が基本成分抽出手段８０７で行われる。この処理を模式的に表したものが図８である。図８（ａ）は原画像の周波数成分、図８（ｂ）は原画像サイズを縦に２倍、横に２倍に拡大した画像の周波数成分、図８（ｃ）は原画像サイズを縦に３倍、横に３倍に拡大した画像の周波数成分を示している。なお、さらに多くの画像サイズを用意しても良いし、その際に用意される画像サイズも整数倍である必要はない。図８（ａ），（ｂ），（ｃ）において、低域の周波数成分は画像の全体的特徴を表すものであり、各倍率に共通な定性的な傾向にある。そこでこの似た傾向のある低域の周波数成分L00を図８（ｄ）に示すように基本成分と考える。このように、基本成分とは、周波数成分であって、一の原画像から拡大などされた画像サイズの異なる複数の画像について、それらの周波数成分における似た（定性的な共通性を有する）傾向にある低周波領域のことであり、言い換えれば一の原画像から画像サイズの異なる複数の画像に拡大などするためにその拡大する元として共通した画像サイズの画像（基本画像）の周波数成分のことである。なお、処理の簡単化のために図８（ａ）に示す原画像の周波数データを基本成分と見なしても構わない。
【００６１】
図７における多重符号化手段８０２は、本発明の第１及び第４の実施の形態の画像処理装置における高成分符号化手段１３と同様に、該基本成分と図８（ｂ）に示す各ブロックH11からH13とを関連付け、ルール化する。図８（ｃ）に示す各ブロックH21からH25についても、同様に該基本成分と直接関連付けることも可能であるが、ここでは図８（ｅ）に示すように、図８（ｂ）の各ブロックH11からH13と図８（ｃ）の各ブロックH21からH25を関連付けることで、該基本成分と図８（ｃ）の各ブロックH21からH25を関連付け、ルール化する。これは周波数領域の近いブロック同士の方が定性的な相違が少ないため、関連付けの精度やその手間を減少させることができるからである。
【００６２】
多重符号合成手段８０３は本発明の第１及び第４の実施の形態における符号合成手段１４と同様に、抽出された前記基本成分と、多重符号化手段８０２で得られた該基本成分と各拡大周波数成分との関連情報を合成し、多重簡易画像データとして記憶媒体１５への記憶処理を行う。この際、多重に用意した拡大画像の個数や各サイズのデータの始まりのフラグ信号等を明確にヘッダー等で記述する必要がある。
【００６３】
以上のように、予め出力装置の解像度に合わせて複数の画像サイズに応じた拡大周波数成分を用意しておくことで、記憶媒体から読み出されたデータを基に所望の拡大サイズの画像を出力する際に不足する、高周波数成分の推定にかかる時間を短縮することができ、ユーザインターフェースの改善になる。
【００６４】
（第７の実施の形態）
次に本発明の第７の実施の形態である画像処理装置について説明する。
【００６５】
図７において、画像入力手段１０、原画像直交変換手段１１、拡大周波数推定手段８００及び基本成分抽出手段８０７の行う処理は、本発明の第６の実施の形態の画像処理装置と同様なので説明を省略する。
【００６６】
基本画像生成手段８０８は、基本成分抽出手段８０７にて抽出された基本成分に逆直交変換を施し、サムネイル画像を基本画像表示手段８０９に表示するが、ここで、基本成分抽出手段８０７にて抽出された周波数の係数サイズがレビュー用のディスプレイの画像サイズより大きい場合には、その基本成分の低域からその解像度に合う周波数成分をさらに抽出させてその周波数成分で表示用の基本画像を生成する。逆に基本成分抽出手段で抽出された周波数の係数サイズがレビュー用のディスプレイより画像サイズより小さい場合には、基本成分の係数で不足する部分に図１６に示すものと同様な”０”成分埋め込みを行ってサムネイル画像を生成することとする。しかし、このような場合には、拡大周波数推定手段８００のようにその画像サイズに応じて、不足分を推定してレビュー用画像サイズに拡大する処理を、該基本画像生成手段８０８に加えることも可能である。
【００６７】
（第８の実施の形態）
次に本発明の第８の実施の形態である画像処理装置について説明する。
【００６８】
図７において、ユーザは本発明の第６の実施の形態の画像処理装置により記憶媒体１５に記憶された画像データの中から、出力・編集対象の画像を選択するとともに、対象とする画像サイズを指定することができる。この際、記憶媒体１５に記憶された画像サイズに応じた選択情報が提示されるか、もしくはユーザにより、原画像サイズを基準とした拡大率が入力されるか、また出力される機器の持つ解像度に合わせて自動的に選択する等の手法が考えられる。
【００６９】
先ず、記憶媒体１５に記憶された画像サイズに応じた選択情報が提示される場合の処理を説明する。ユーザは記憶媒体１５から、所望の画像と所望の画像サイズを選択指示し、基本成分復号手段８０４はこの指示に対応した多重周波数データから該画像の基本成分を取り出す。次に対象周波数復号手段８０５が選択された該画像サイズに相当する高周波成分データの表現符号列を取り出し、該符号列と該基本成分から、対象とする拡大画像サイズの該基本成分以外の高周波成分を復号する。対象画像出力手段８０６では該高周波成分と該基本成分を結合し、逆直交変換を施して、所望の拡大画像を出力する。
【００７０】
次に、原画像サイズを基準とした拡大率が入力されるか、また出力される機器の持つ解像度に合わせて自動的に選択する場合の処理を説明する。記憶メディア１５に多重的に保持されていない画像サイズの場合には最も近いサイズの画像データが選択され、基本成分復号手段８０４、対象周波数復号手段８０５及び対象画像出力手段８０６への処理へ移る。この際、対象画像出力手段８０６における逆直交変換時には、前記の最も近い画像サイズに対応した逆直交変換処理が取られるものとする。
【００７１】
しかし、この段階で、選択される画像サイズを所望の画像サイズより小さめに選び、不足した分を第５の実施の形態における不足成分推定手段５００を加えることも可能である。
【００７２】
以上のように、複数の拡大サイズの周波数成分の共通と見なせる低周波成分を基本成分として抽出し、この基本成分を中心として各拡大サイズの残りの高周波成分を符号化することで、複数の拡大サイズに関する画像の周波数データを用意することができる。そのため、ユーザの指示により目的とした画像サイズの拡大画像の再生時には、指示のたびに所望の拡大サイズに不足する高周波成分を推定することなく高速に拡大画像を再現することができる。
【００７３】
（第９の実施の形態）
次に本発明の第９の実施の形態である画像処理装置について説明する。
【００７４】
図９において、画像入力手段１０、原画像直交変換手段１１、拡大周波数推定手段８００及び基本成分抽出手段８０７までの処理は、本発明の第６の実施の形態である画像処理装置のものと同様であるので、説明を省略する。
【００７５】
基本成分圧縮手段１０００では、基本成分抽出手段８０７により抽出された基本成分を圧縮処理するが、該圧縮処理の工程は本発明の第４の実施の形態の画像処理装置における低成分圧縮手段３００と同様であるので、説明を省略する。
【００７６】
また、基本成分圧縮手段１０００に続く多重符号化手段８０２、多重符号合成手段８０３及び記憶媒体１５における処理は、本発明の第６の実施の形態である画像処理装置のものと同様であるので、説明を省略する。
【００７７】
以上のような構成により、複数の画像サイズを保持する際の問題点である画像データ容量の低減化を図ることができる。
【００７８】
（第１０の実施の形態）
最後に本発明の第１０の実施の形態である画像処理装置について説明する。図１０はその構成を表すものである。
【００７９】
まず画像入力手段１０で読み取られた原画像に図１６に示すような補間方式で画素間を補間することで、画素間補間手段１１００が拡大画像を生成する。次に該拡大画像に対して畳み込み手段１１０１が畳み込み処理を繰り返すことにより画素強調を行う。
【００８０】
図１１は前記畳み込み処理の工程を表す。図１１（ｂ）に示すようにエッジが不明瞭な画像の場合、エッジについての画像データが抽出できず、図１５に示すような処理工程で画像の拡大処理を行うことは困難である。しかし、本発明のようにすることで平均化された画素値を簡単に強調することができる。なお、畳み込み手段１１０１の処理は、拡大画像にエッジ強調フィルタによる処理を複数回繰り返すことと同じような効果を持つものである。従来のエッジ強調フィルタ処理の場合、適正なフィルタを選ばないと、その繰り返し回数は非常に大きくなったり、全くエッジ強調がされない等の問題があるが、畳み込み手段１１０１では画素値自身による畳み込みを行うため、その心配もない。図１１（ｃ）のＰ点におけるK回目の畳み込み処理をする際の画素値をDp[K]とすると、そのK+1回目の畳み込み処理によるＰ点の画素値Dp[K+1]は（数５）のように定義する。これは、この画素近傍における畳み込み値の平均値を求めるものである。ここで、Ωは畳み込み対象画素の範囲を、U＿GasoはそのΩ内の総画素数を、ｑはΩ内の任意画素を表す。
【００８１】
【数５】

この処理を拡大補間画像に含まれる画素の集合をΛとした場合、このΛ内の画素全部に処理が行われる。
【００８２】
次に収束判定手段１１０２では、この畳み込みによる処理の収束判定が行われる。それは（数６）のように(K-1) 回目で得られた画素値Dp[K-1] とK 回目の画素値Dp[K] の２乗誤差のΛ内の平均値をもとにそれが、予め用意された収束判定値Thred より小さい場合に、収束したとして畳み込み処理を終え、推定拡大画像出力手段１３１２で推定拡大画像として出力されるのである。
【００８３】
【数６】

（数４）でT ＿GasoはΛ内の総画素数を表す。
【００８４】
以上のように本発明の第１０における画像処理装置は、補間画像データの畳み込み演算処理により拡大画像の強調をするものであり、処理時間のかかる周波数変換を行う必要がなく、手軽に拡大画像を実現できる。また、補間画像のボケ解消に元の画像のエッジ情報が使用する際に問題となるエッジが不明瞭な原画像に対しても、簡単にエッジ情報を持つ拡大画像を得ることができる。
【００８５】
なお最後に、以上において、原画像がスキャナーや電子スチルカメラ等により取り込まれた画像である場合について説明したが、必ずしもその必要はなく、磁気ディスク等に保持されている特定の画像等であってもよい。
【００８６】
また本発明の第１及び第４の実施の形態の画像処理装置における高成分符号化手段１３により得られる低周波成分と高周波成分の関連情報、並びに、第６及び第９の実施の形態である画像処理装置における多重符号化手段８０２により得られる基本成分と各拡大画像の高周波成分の関連情報についても、Spatial 方式等の可逆の圧縮方式で圧縮することが可能であることを付け加えておく。
【００８７】
【発明の効果】
以上のように、本発明における第１から第５までの画像処理装置によれば、原画像の周波数成分から低周波成分を抽出、圧縮して、残りの高周波成分は該低周波成分との関連情報に基づいて表現するため、記憶媒体に保持される画像データ容量が低減され、且つ、前記圧縮方法が可逆的であるために画像復元時に鮮明さが失われない。また抽出された前記低周波成分によりサムネイル画像を生成するため、従来のような画素間引きによるノイズを低減することが可能となる。更に記憶媒体に保持された画像データを拡大して出力する際にも、従来のような画素補間によるエッジのボケや細部の不鮮明さを解消することができる。
【００８８】
次に、本発明における第６から第９までの画像処理装置によれば、原画像に対して予め複数の拡大率の画像を用意し、これらの中から共通と見なせる低周波成分を基本成分として抽出、圧縮した後、残りの各拡大画像の高周波成分は該基本成分との関連情報に基づいて表現するため、記憶媒体に保持される画像データ容量が低減され、且つ、前記圧縮方法が可逆的であるために画像復元時に鮮明さが失われない。また所望のサイズの拡大画像を出力する際には、予め用意されている複数の拡大率の画像の中から、同一または最も近いサイズの画像を選択するために、画像復元の高速化が図られる。
【００８９】
最後に、本発明の第１０における画像処理装置によれば、入力された画像に画素間補間を行い、得られた補間画像データを複数回畳み込み演算する処理を適用したものであり、処理時間のかかる周波数変換を行う必要がなく、手軽に拡大画像を実現できる。また、補間画像のボケ解消として元の画像のエッジ情報等が使用されるが、エッジ情報の抽出が困難な原画像に対しても、鮮明なエッジを持つ拡大画像を得ることができる。
【図面の簡単な説明】
【図１】本発明の第１、第２及び第３の実施の形態の構成を示すブロック図。
【図２】本発明の第１の実施の形態の処理工程を表す模式図。
【図３】本発明の第４の実施の形態である画像処理装置を含む概略構成図。
【図４】本発明の第４の実施の形態に使用する空間予測圧縮の手順を示す模式図。
【図５】本発明の第５の実施の形態である画像処理装置を含む概略構成図。
【図６】本発明の第５の実施の形態に使用する不足成分推定手段の手順の模式図
【図７】本発明の第６、第７及び第８の実施の形態の構成を表すブロック図。
【図８】本発明の第６の実施の形態に使用する基本成分抽出手段と多重符号化手段の処理工程を表す模式図。
【図９】本発明の第９の実施の形態である画像処理装置を含む概略構成図。
【図１０】本発明の第１０の実施の形態の構成を表すブロック図。
【図１１】本発明の第１０の実施の形態である画像処理装置の処理工程を表す模式図。
【図１２】従来の電子スチルカメラの構成を表すブロック図。
【図１３】従来の周波数領域に変換して拡大する画像拡大装置の構成を表すブロック図。
【図１４】従来の周波数領域に変換して拡大する例を示す説明図。
【図１５】従来の画素補間により拡大する画像拡大装置の構成を表すブロック図。
【図１６】従来の画素間補間を表す模式図。
【符号の説明】
１０ 画像入力手段
１１ 原画像直交変換手段
１３ 高成分符号化手段
１４ 符号合成手段
１５ 記憶媒体
１６ 低成分復号手段
１７ 高成分復号手段
１８ 原画像出力手段
１００ 低成分抽出手段
１０１ 縮小画像生成手段
１０２ 縮小画像表示手段
３００ 低成分圧縮手段
５００ 不足成分推定手段
５０１ 拡大画像出力手段
８００ 拡大周波数推定手段
８０２ 多重符号化手段
８０３ 多重符号合成手段
８０４ 基本成分復号手段
８０５ 対象周波数復号手段
８０６ 対象画像出力手段
８０７ 基本成分抽出手段
８０８ 基本画像生成手段
８０９ 基本画像表示手段
１０００ 基本成分圧縮手段
１１００ 画素間補間手段
１１０１ 畳み込み手段
１１０２ 収束判定手段
１３００ 圧縮手段
１３０１ 量子化テーブル
１３０２ エントロピー符号化テーブル
１３０３ 伸張手段
１３０４ 画素間引き手段
１３０５ 縮小画像表示手段
１３０６ ＤＣＴ変換手段
１３０７ 量子化手段
１３０８ エントロピー符号化手段
１３０９ ＩＤＣＴ変換手段
１３１０ 逆量子化手段
１３１１ エントロピー復号化手段
１３１２ 画像出力手段
１４００ ”０”成分埋め込み手段
１４０１ 逆直交変換手段
１４０２ 推定拡大画像出力手段[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus for reducing the storage size of image data input by a scanner, an electronic still camera, or the like and outputting the data clearly.
[0002]
[Prior art]
In recent years, with technological innovation, electronic imaging devices that store images in a storage medium such as a magnetic medium or a memory instead of a silver salt film have come into practical use. In particular, electronic still cameras that use CCD (Charge Coupled Devices) image sensors to read light shading and colors and store them in semiconductor memory, etc., are becoming popular in ordinary households due to their ease and low price. The resolution of CCD elements is also increasing from 1 million pixels to 2 million pixels. In this case, although it has a resolution capability of about 1280 pixels × 980 pixels depending on the effective number of pixels in the CCD, the capacity of the image data read by the CCD image sensor becomes very large. As this compression processing, a color still image coding standard method (JPEG; Joint Photographic Coding Experts Group) is usually adopted. A conventional image processing apparatus that performs compression by JPEG can be represented by a block diagram as shown in FIG.
[0003]
The original image is read by the image input means 10 composed of a CCD image sensor or the like. The compression unit 1300 executes lossy compression using JPEG on the original image. That is, first, DCT (Discrete Cosine Transform) transforming means 1306 performs discrete cosine transform on the image to transform it into a frequency domain signal, and the obtained transform coefficient is quantized by quantization means 1307 using quantization table 1301. . The quantized result is converted into a symbol string based on the entropy encoding table 1302 in the entropy encoding means 1308 and stored in the storage medium 15. This process is performed until all the original images are compressed. The JPEG-compressed image is an irreversible compression that does not accurately return to the original original image even when decompressed due to the influence of quantization processing and the effect of rounding error due to DCT. Regarding this irreversibility, the influence of quantization processing is particularly great among the above two factors.
[0004]
By the way, an image processing apparatus such as an electronic still camera is generally provided with a display device such as a liquid crystal display in order to review captured images on the spot or to perform data editing. At this time, as described above, the resolution of the CCD is higher than the resolution of the review display device. When the compressed image data stored in the storage medium 15 is displayed on these display devices, the compressed image data is displayed. After decompressing, a pixel thinning process is performed for resolution conversion.
[0005]
First, the decompression means 1303 in FIG. 12 performs decompression work on the compressed image data stored in the storage medium 15. This process returns to the transform coefficient quantized by the entropy decoding means 1311 and returns to the DCT coefficient by the inverse quantization means 1310.
Then, an inverse discrete cosine transform (IDCT) conversion unit 1309 executes an inverse discrete cosine transform on the DCT coefficient and expands it to a value in the original image data space. The expanded image data is output from the original image output means 1312 to a printer, CRT, or the like in accordance with an instruction signal from the user.
[0006]
On the other hand, the pixel thinning means 1304 thins out pixels from all the image data decompressed by the decompressing means 1303 in accordance with the resolution of the reduction display means for review display, and this thinned image is represented by the reduced image display means 1305. The review is displayed.
[0007]
By the way, if you consider reading an A4 document using an electronic still camera with a 2 million-pixel CCD element, it cannot be said that it depends on its focal length, but it can be estimated only from the number of pixels of the CCD image sensor. It can be seen that the electronic still camera has only a resolution capability of about 100 (dot / inch). This resolution is much coarser than an image obtained from a silver salt camera, and is still insufficient for describing edge information and detail sharpness in a natural image. Therefore, when outputting an image input with an electronic still camera to a high-resolution printer such as a laser printer or an ink-jet printer, or displaying it on a high-resolution display connected to a computer for data editing, the low resolution It is necessary to convert the image data into high resolution data. Since high-resolution conversion and image enlargement are equivalent, a conventional method for performing accurate image enlargement will be described below.
[0008]
As a conventional image enlargement method,
(1) As described in JP-A-2-76472 or JP-A-5-167920, an image signal in a real space is converted into an image signal in a frequency space using orthogonal transform such as FFT (Fast Fourier Transform) or DCT. How to enlarge after conversion
(2) Simple interpolation method between pixels
Can be mentioned.
[0009]
FIG. 13 is a block diagram showing the configuration of the conventional example of the method (1), and FIG. 14 is a diagram schematically showing the processing steps. First, an original image in real space (n × n pixels) as shown in FIG. 14A is converted into an image (n × n) in a frequency space as shown in FIG. n pixels). At this time, the image signal in this frequency space is represented by an n × n matrix, and the matrix after the frequency conversion becomes a low frequency component as it goes to the upper left in the drawing, and the right direction and the downward direction along the arrow As it goes to, it becomes a high frequency component. Next, in the “0” component embedding means 1400, an area (sn × sn area shown in FIG. 14C) obtained by multiplying the area converted into the frequency space image by s is prepared. The n × n frequency region shown in FIG. 14B obtained by the orthogonal transformation is copied to the low frequency component region portion of the region, and “0” is interpolated to the remaining high frequency component region portion. Is done. Finally, an inverse orthogonal transform is performed on the frequency domain of sn × sn in the inverse orthogonal transform unit 1401 to obtain an image signal in real space multiplied by s as shown in FIG. The enlarged image estimated by the image output means 1402 is output.
[0010]
On the other hand, a block diagram as shown in FIG. 15 can be given as a conventional example of the method of interpolating between pixels of (2). In the original image obtained by the image input means 10, interpolated pixels are filled between the pixels according to the following (Equation 1). In (Expression 1), Da represents pixel data at point A in FIG. 16, Db represents pixel data at point B, Dc represents pixel data at point C, and Dd represents pixel data at point D. E is the interpolation point desired.
[0011]
[Expression 1]

In this case, the interpolated image tends to be averaged, and the edges and the like tend to become unclear, so that the interpolated pixel position and interpolated data may be adjusted using the edge information extracted from the original original image. . In FIG. 15, after edge information is extracted by the edge extraction unit 1600, the edge interpolation unit 1601 obtains an enlarged edge in accordance with a desired enlargement ratio. Then, by multiplying the enlarged edge information by the interpolated enlarged image result of the original image by the inter-pixel interpolation unit 1100, a desired enlarged image is generated and output by the estimated enlarged image output unit 1402.
[0012]
[Problems to be solved by the invention]
However, in a conventional image processing apparatus such as an electronic still camera, it is necessary to decompress the image data that has been compressed once and stored, and since an irreversible compression method involving quantization processing is used as the compression method, In general, the original image does not return to the original image, and some noise and color shift may occur.
[0013]
In addition, there is a disadvantage that color shift and moire are caused by thinning processing at the time of resolution conversion necessary for generating a reduced image (hereinafter referred to as a thumbnail image) displayed for review. For this reason, a median filter or an average value filter is used. Smoothing treatment by was indispensable. For this reason, problems have been pointed out that the sharpness of the thumbnail image is lost or that the smoothing process takes a very long time.
[0014]
Furthermore, the conventional image enlargement required for printing or displaying the low resolution of a still camera CCD has the following problems.
[0015]
  First, in the case of the method of interpolation in the frequency-converted space of (1), the image size is generally arbitrary, and the size required for DCTButThere is a problem that the processing time becomes longer as the size increases. In order to solve such problems as processing speed, generally, the entire image size is not subjected to orthogonal transformation at a time, but a method of performing block size units of about 4 to 16 pixels has been adopted. When enlargement processing is performed by such a method, discontinuity (block distortion) between blocks of the completed enlarged image may occur at the boundary portion.
[0016]
Further, in the case of simply interpolating between pixels in (2), when a natural image is taken with such a low-resolution device, depending on the color balance of the subject, FIG. In many cases, the edge extraction is not performed with high accuracy, and as a result, there is a problem in that the pixel interpolated position and the interpolation data cannot be adjusted well. In order to avoid this, it was performed to repeat the process using the edge enhancement filter on the enlarged image, but in the case of the conventional edge enhancement filter process, the number of repetitions is very large unless an appropriate filter is selected. There are disadvantages such as no edge enhancement.
[0017]
[Means for Solving the Problems]
the aboveTaskTheResolutionTherefore, the present invention employs the following means. That is, according to the present invention, in an image processing apparatus for processing image data obtained from a digital camera or the like, first, as shown in FIG.for,ItOriginal image orthogonal transform means 11Use frequency space data atWith orthogonal transformationThe frequency component of the original imageGenerate the original image frequency component,AndIn the low component extraction means 100, the original image frequency componentFrom thatExtract low frequency components. Next, the high component encoding means 13The remainder extracted by the low component extraction means 100High frequency component of the original image frequency componentThe low frequency componentWhenAssociateofInformationThe related information is encoded and the low frequency component is encoded by the code synthesizing unit 14.Data representingAnd related informationData representingAndThe image data is smaller in data size than the image data of the original imageSimple image data-Data is generated.
[0018]
  The simple image data generated as described above isFrom that data,Low frequency component by the low component decoding means 16The data representingExtractionIsAnd the high component decoding means 17,Related informationThe data representingTake outBeen,Data representing that related information andLow frequency componentAnd data representingOn the basis of theTheHigh frequency componentThe data representingDecryptionIs,AndBy the original image output means 18, the low frequency componentData representingAnd high frequency componentsThe data representingJoinIs the frequency component.The, Its frequency componentInverse orthogonal transformButOutIsThe,Original imageButRestoreIsRuStuffThe
[0019]
  By this, from the image data of the original imagedataSimple image data with a small size can be handled, and a restored image with an image quality close to that of the original image can be obtained.
[0020]
  The simple image data can be directly input to a means such as a personal computer for restoring the image and processed, but can also be temporarily stored in the storage medium 15. Further, in the simplified image data, the low-frequency component-It is possible to further compress the data amount by the low component compression means 300 as shown in FIG. 3, but in this case, it is preferable to use a compression method having reversibility.
[0021]
  As the low frequency componentThe reference frequency to be extractedStipulated by the reduced image generation means 101, The image size is smaller than the original imageReduced size (eg preview size)versusMeetShiUsing different frequenciesMayThe low frequency componentRepresentedInverse orthogonal transform to dataApplyByOf the reduced sizeA reduced image can be generated.
[0022]
  The present invention also provides:originalDesired imageimageWhen expanding to the size, as shown in FIG.Should be compensated for the frequency components of the original imageHigh frequency componentoriginalImage frequency components, And related information that is information indicating the relationship between the frequency component of the original image and the high frequency component to be supplementedThe enlarged image output means 501 determines the frequency component of the original image and the insufficient component estimation means.EstimatedThe high frequency componentWhenCombineAs a frequency component of an enlarged image,In the frequency componentBy applying inverse orthogonal transformation, the desiredimageOutput an image enlarged to sizeConfigureTheGood.
[0023]
  Furthermore, as shown in FIG.FromEnteredTheIn the image processing apparatus for processing the image data, the original image orthogonal transformation means 11Input as original imageThe image data is subjected to orthogonal transformation to generate an original image frequency component, and the original image frequency componentTheBy means of the expanded frequency estimation means 800,TheOriginal image with desired magnificationsoEnlargedEnlarged imageFrequency componentsThe expanded frequency component is regarded as a low frequency component in the expanded frequency componentThe, Based on related information which is information indicating the relationship between the low frequency component and the remaining high frequency component in the original image frequency component and the enlarged frequency componentpresume. UpPredicateThe original image frequency component obtained asAbout multiple magnificationsExpanded frequency componentAnd fromBasic component extraction means 807In advanceimagesizeButThe specified basic imageGenerationThe frequency component necessary for the extraction is extracted as a basic component and is estimated by the multiple image encoding means 802.SaidExpanded frequency componentThe basic ingredientsWhenAssociateofInformationEncode related information.AndBasic components obtained in this wayData representingAnd related informationData representingAnd the multiple code combining means 803The,The data size is smaller than the image data of the original imageMultiple simple image data-To generateMay be configured.Here, the basic image is an image having a common image size as an enlargement source in order to enlarge from one original image to a plurality of images having different enlargement rates, that is, different image sizes.
[0024]
  As a result, multiple simplified image data corresponding to a plurality of sizes are encoded on the image data encoding side.-Data can be generated.
[0025]
  The multiplexed simple image data generated in this way-TFrom,AboveBasic ingredientsData representingWhenAboveRelated informationThe data representingExtractionBeenBased on bothExpansionthe imageGenerationIt will be possible.
[0026]
  The multiple simplified image data-In the case of the-By compressing the data amount, the data size can be further reduced.
[0027]
  The present invention also provides,As shown in FIG. 10, the image input means 10FromEnteredTheInterpolation unit 1100 interpolates image data between pixels of the original image according to a desired enlargement ratio.With a pixelInterpolateMay be good. As a result, the edge portion of the interpolated magnified image is unclear, but the convolution means 1101 performs a convolution calculation for emphasizing the edge portion of the interpolated magnified image without performing frequency conversion that requires processing time. Generate an enlarged image with sharp edges.
[0028]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following, pixel units are all used as coordinate value units.
[0029]
(First embodiment)
First, the image processing apparatus according to the first embodiment of the present invention will be described.
[0030]
In FIG. 1, an original image having a size of N pixels × N pixels obtained by an image input means 10 such as a CCD image sensor is converted into component data (frequency component data) in a frequency space by an original image orthogonal transform means 11. The In this case, orthogonal transform is used, and examples of the transform method include Hadamard transform, Fast Fourier Transform (FFT), Discrete Cosine Transform (DCT), Slant transform, Haar transform, etc. Here, DCT is used. However, the present invention is not limited to this conversion method, and can be realized when other orthogonal transforms are used.
[0031]
Among the DCTs, a two-dimensional DCT is used to handle an image in particular.
[0032]
[Expression 2]

Here, F [u, v] is the DCT component at the component position (u, v), and D [x, y] represents the image data at the pixel position (x, y). Furthermore, K_x represents the number of pixels in the x direction, and K_y represents the number of pixels in the y direction. Here, since N × N square images are targeted, K_x = K_y = N.
[0033]
The low component extraction unit 100 extracts the low frequency component of the frequency component data of the original image as shown in FIG.
[0034]
  The high component encoding means 13 encodes the high frequency component region (e) that remains when the low frequency component (c) is removed from the frequency component of the original image in FIG. Here, as shown in FIG.HighDivide the frequency component area into small blocks from H1 to H5. Similarly, as shown in FIG. 2F, the low frequency component (c) region is also divided into blocks L1 to L4. Then, the frequency components in the blocks H1 to H5 of the high frequency component (e) and the blocks L1 to L4 of the low frequency component (f) are associated with each other. For example, it may be in the form of a linear expression such as (Equation 3) multiplied by a certain constant coefficient, or a multidimensional function Ψ (L1, L2, L3, L4) having the frequency components of each block L1 to L4 as variables. Thus, the frequency components in each of the blocks H1 to H5 may be approximated.
[0035]
[Equation 3]

Also, it is not necessary to have a uniform coefficient in each block, and a coefficient matrix M_C1 to M_C5 is prepared for each block H1 to H5, and this is composed of frequency data in the blocks L1 to L4. It is also conceivable to express the high frequency component matrices M_H1 to M_H5 that are composed of the frequency data in the blocks H1 to H5 using the component matrices M_L1 to M_L4. In any means, by using the low-frequency component data in FIG. 2C, the remaining high-frequency components in FIG. Data capacity can be reduced.
[0036]
  The encoding storage means 14 stores the low-frequency component and the high-frequency component obtained from the rule description obtained by the high-component encoding means 13 or data such as the coefficient of the approximate expression used by the high-component encoding means 13. Code string of related information and low frequency component data of original original imageWhenA simple image-The data is stored in the storage medium 15 as a data. For the processing to be read and restored next, the number of stored image data, the size of each image data, the proportion of low frequency component data in each stored data and the high frequency component are included in the header information in the storage medium. By describing information such as the proportion of the code to be represented, each data can be extracted efficiently.
[0037]
(Second Embodiment)
Next, an image processing apparatus according to the second embodiment of the present invention will be described.
[0038]
In FIG. 1, the processing performed by the image input unit 10 and the original image orthogonal transform unit 11 is the same as that of the image processing apparatus of the first embodiment, and a description thereof will be omitted. The low component extraction unit 100 extracts a low frequency component corresponding to the number of pixels of the reduced image display unit 102 that displays the thumbnail image, and the reduced image generation unit 101 performs inverse orthogonal transformation on the low frequency component to convert the thumbnail image into a thumbnail image. Displayed on the reduced image display means.
[0039]
(Third embodiment)
Next, an image processing apparatus according to the third embodiment of the present invention will be described.
[0040]
  In FIG.-The simple image data stored in the storage medium 15 by the image processing apparatus according to the first embodiment of the present invention.-High resolution recording-An instruction signal is given to output a desired image to be output to the printer or inkjet printer or to be edited on the CRT. Based on this instruction signal, the low component decoding means 16 first extracts the central low frequency component from the storage medium 15. Then, the high component decoding means 17 relates to the low frequency component obtained by the low component decoding means 16 and the related information between the low frequency component and the high frequency component in the storage medium 15.WhenDecode higher frequency components. Then, the original image output means 18 combines the low-frequency component data and the high-frequency component data, and performs inverse orthogonal transform of the corresponding image size, thereby performing other image processing such as display on a CRT or output to a printer. Output as data handled by the device.
[0041]
  Image data encoded and stored in the storage medium 15-Since the data is not subjected to quantization processing, a clearer original image can be obtained by this method.
[0042]
(Fourth embodiment)
Next, an image processing apparatus according to a fourth embodiment of the present invention will be described.
[0043]
In FIG. 3, the processing performed by the original image orthogonal transform 11, the low component extraction unit 100, the high component encoding unit 13, the code synthesis unit 14 and the storage medium 15 is the same as that of the image processing apparatus according to the first embodiment. Omitted. In consideration of the fact that the volume of the low frequency component data of the original image obtained by the low component extraction means 100 occupies a large size, the limitation due to the capacity size of the storage medium, etc. Compress by 300.
[0044]
  The processing method of the low-component compression unit 300 shown in FIG. 4 is called a spatial prediction (Spatial) method in JPEG, and is a normal DCT, quantization, and entropy coding.WhenDifferential coding (DifferentialPCM; DPCM) and entropy coding instead of compression processing (also called baseline method)WhenThis is a reversible method in which no distortion occurs due to compression or expansion.
[0045]
In this method, a prediction value is calculated by a predictor using adjacent pixel values, and the prediction value is subtracted from the pixel value to be encoded. As shown in (Expression 4), for example, Seven types of relational expressions are prepared.
[0046]
[Expression 4]

As shown in FIG. 4A, a pixel value to be encoded is Dx, and three adjacent pixel values are D1, D2, and D3. At this time, the predicted value dDx calculated from these three pixel values is defined by (Equation 4), and which formula is used is described in the header information. In the encoding process, as shown in FIG. 4C, one prediction formula is selected, Ex is calculated, and this Ex is entropy encoded. By encoding such a prediction error, the low frequency component can be reversibly compressed.
[0047]
  Based on the low-frequency component data compressed by the low-component compression unit 300, the high-component encoding unit 13 uses the method shown in (Equation 3) for the remaining high-frequency components as in the first embodiment. Le-Turn into
[0048]
  The code synthesizing unit 14 compresses the compressed low frequency component data.-And data such as the rule description obtained by the high component encoding means 13 or the coefficients of the approximate expression used by the high component encoding means 13FromObtained low frequency component and high frequency componentWhenAre combined with a code string of related information of-The data is stored in the storage medium 15 as a data.
[0049]
  By adopting such a form of the present embodiment, the low frequency component which is supposed to occupy the largest part of the data capacity of the first device is reduced from 1/2 to 1/4.-It is possible to compress the amount of data. Further, by using a reversible method for the compression, it becomes possible to store the frequency component of the original image without damaging it.
[0050]
  Here, since the low frequency component data is a reference image for restoring a high frequency component representing the sharpness of the details of the image, the low-frequency component data is low in order to realize an image output that is as clear and clear as possible. Although a reversible compression method is applied to the frequency component, a normal JPEG baseline method can also be applied thereto. In this case, the loss of image clarity is lost because of lossy compression.-Since the amount of data is compressed from about 1/10 to 1/20, this is a method that can be used when a large amount of image data is stored with the read image capacity held in the storage medium as small as possible.
[0051]
(Fifth embodiment)
Next, an image processing apparatus according to a fifth embodiment of the present invention will be described.
[0052]
In FIG. 5, the processing up to the storage medium 15, the low component decoding unit 16 and the high component decoding unit 17 is the same as that of the third embodiment of the present invention, and the description thereof will be omitted.
[0053]
  The deficient component estimation unit 500 displays a desired pixel size when displaying the read original image on a high-resolution CRT or the like.(Image size)Is a means for estimating high-frequency component data that is insufficient when the image is enlarged, and the enlarged image output means 501,The insufficient high frequency component estimated by the insufficient component estimation unit 500 and the frequency component of the original image decoded from the storage medium 15 by the low component decoding unit 16 and the high component decoding unit 17.WhenAnd performing inverse orthogonal transform means corresponding to the enlarged size,This is means for generating an enlarged image of the read original image.
[0054]
  The deficient component estimation unit 500 and the enlarged image output unit 501 perform processing as shown in FIG. First, as shown in FIG. 6B obtained by orthogonal transformation of an original image having a size of N pixels × N pixels shown in FIG.NaThe wave number component data is embedded in a low frequency region of an enlarged image having a coefficient size corresponding to a desired enlargement size (sN pixel × sN pixel) (FIG. 6C). Then, the insufficient components FH1 to FH3 generated at this time are estimated from the frequency components of the original image shown in FIG. 6B. The estimation method includes, for example,
(1) Method of applying the nonlinear approximation capability of a radial base function network (RBFN) as in Japanese Patent Application No. 11-68151
(2) Similarly, as in Japanese Patent Application No. 11-68151, the frequency component of the enlarged edge image is accurately estimated from the frequency component of the edge image of the original image using a linear approximation or a radial basis function network. Method for estimating high-frequency components of enlarged images that have been deleted from images
(3) A method of embedding an orthogonal transform component of an original image in a low-frequency region and embedding frequency information obtained based on a prediction rule prepared in advance in a high-frequency region as disclosed in JP-A-8-294001
(4) A method of restoring high-frequency components in a process of repeating normal transformation and inverse transformation using orthogonal transformation of an image signal as in Japanese Patent Laid-Open No. 6-54172 (method by iteration of Gerchberg-Papoulis)
Etc.
[0055]
In addition to this, there are image enlargement methods using many frequency regions, but this insufficient component estimation unit 500 accurately estimates high-frequency components necessary for generating an image with clear details and clear edges when enlarged. If it is a method that satisfies the request, it is similarly established.
[0056]
Then, the enlarged image output means 501 performs the inverse orthogonal transform means corresponding to the coefficient size of sN × sN as shown in FIG. 6D, so that the frequency component of the enlarged image estimated by the insufficient component estimation means 500 is obtained. Return to real space data, display on CRT, pass to output device such as printer, or output as data handled by other image processing devices.
[0057]
With the above configuration, high-frequency component data required for enlarged image output is estimated and supplemented using the frequency components of the original image, and blurring of edges and details that are blurred during conventional pixel interpolation are blurred. Can be eliminated.
[0058]
(Sixth embodiment)
Next, an image processing apparatus according to a sixth embodiment of the present invention will be described with reference to FIGS.
[0059]
In FIG. 7, the processing performed by the image input unit 9 and the original image orthogonal transform unit 11 is the same as that of the first embodiment of the present invention, and a description thereof will be omitted. First, using the frequency component of the original image obtained by the original image orthogonal transform means 11, the enlarged frequency estimation means 800 estimates frequency component data when the image is enlarged to a plurality of image sizes. This estimation method can take the same method as the insufficient component estimation unit 500 of FIG. 5 which is one of the components of the image processing apparatus according to the fifth embodiment of the present invention.
[0060]
  Next, basic component extraction processing is performed by the basic component extraction means 807. FIG. 8 schematically shows this processing. FIG. 8A shows the frequency component of the original image, FIG. 8B shows the frequency component of the image obtained by enlarging the original image size twice vertically and twice horizontally, and FIG. 8C shows the original image size vertically. The frequency components of the image enlarged three times and three times horizontally are shown. It should be noted that a larger number of image sizes may be prepared, and the image size prepared at that time need not be an integer multiple. In FIGS. 8A, 8B, and 8C, the low-frequency component represents the overall characteristics of the image, and has a qualitative tendency common to each magnification. Therefore, the low frequency component L00 having a similar tendency is considered as a basic component as shown in FIG.As described above, the basic component is a frequency component, and a plurality of images having different image sizes enlarged from one original image have similar (qualitative common) tendencies in the frequency components. In other words, the frequency component of an image (basic image) with a common image size as the source of enlargement in order to enlarge from one original image to a plurality of images having different image sizes. It is.For simplification of processing, the frequency data of the original image shown in FIG. 8A may be regarded as a basic component.
[0061]
  In the same way as the high-component encoding unit 13 in the image processing apparatuses according to the first and fourth embodiments of the present invention, the multiplex encoding unit 802 in FIG. 7 includes the basic component and each block shown in FIG. Associate H11 with H13-Turn into Similarly, each of the blocks H21 to H25 shown in FIG. 8 (c) can be directly associated with the basic component, but here, as shown in FIG. 8 (e), each block of FIG. By associating H11 to H13 with each block H21 to H25 in FIG. 8C, the basic component is associated with each block H21 to H25 in FIG.-Turn into This is because the qualitative differences between the blocks in the frequency domain are less qualitative, so that the accuracy and time of association can be reduced.
[0062]
  The multiplex code synthesis means 803 is the same as the code synthesis means 14 in the first and fourth embodiments of the present invention, the extracted basic component, the basic component obtained by the multiplex coding means 802, and each expansion. By combining related information with frequency components, multiple simple image data-The storage processing to the storage medium 15 is performed as a data. At this time, it is necessary to clearly describe the number of enlarged images prepared in a multiplexed manner, a flag signal at the start of data of each size, and the like in a header.
[0063]
As described above, an image with a desired enlargement size is output based on the data read from the storage medium by preparing in advance enlargement frequency components corresponding to multiple image sizes according to the resolution of the output device. It is possible to reduce the time required to estimate the high frequency component, which is insufficient when doing so, and to improve the user interface.
[0064]
(Seventh embodiment)
Next, an image processing apparatus according to a seventh embodiment of the present invention will be described.
[0065]
In FIG. 7, the processing performed by the image input unit 10, the original image orthogonal transform unit 11, the enlarged frequency estimation unit 800, and the basic component extraction unit 807 is the same as that of the image processing apparatus according to the sixth embodiment of the present invention. Omitted.
[0066]
  Basic image generation means 808,Basic component extraction means 807TheThe extracted basic component is subjected to inverse orthogonal transformation, and a thumbnail image is displayed on the basic image display unit 809. Here, the basic component extraction unit 807 is displayed.AtIf the extracted frequency coefficient size is larger than the image size of the review displayIn,ThatBasic component low rangeFromMatch the resolutionUFrequency componentFurtherExtractedLet the frequency componentBasic image for displayTheGenerationYouThe On the other hand, if the frequency coefficient size extracted by the basic component extraction means is smaller than the image size than the review display, the “0” component similar to that shown in FIG. To generate a thumbnail image. However, in such a case, processing for estimating the shortage and enlarging the image size for review according to the image size, such as the enlarged frequency estimating means 800, may be added to the basic image generating means 808. Is possible.
[0067]
(Eighth embodiment)
Next, an image processing apparatus according to an eighth embodiment of the present invention will be described.
[0068]
  In FIG.-The image data stored in the storage medium 15 by the image processing apparatus according to the sixth embodiment of the present invention.-The image to be output / edited can be selected from the data and the target image size can be designated. At this time, selection information corresponding to the image size stored in the storage medium 15 is presented, or the user inputs an enlargement ratio based on the original image size, and the resolution of the output device A method such as automatically selecting according to the above is conceivable.
[0069]
First, processing when selection information corresponding to the image size stored in the storage medium 15 is presented will be described. The user instructs to select a desired image and a desired image size from the storage medium 15, and the basic component decoding unit 804 extracts the basic component of the image from the multi-frequency data corresponding to this instruction. Next, the target frequency decoding means 805 takes out the expression code string of the high-frequency component data corresponding to the selected image size, and from the code string and the basic component, high-frequency components other than the basic component of the target enlarged image size Is decrypted. The target image output means 806 combines the high frequency component and the basic component, performs inverse orthogonal transformation, and outputs a desired enlarged image.
[0070]
  Next, a description will be given of processing when an enlargement ratio based on the original image size is input or automatically selected according to the resolution of the output device. In the case of an image size that is not stored in a multiplexed manner on the storage medium 15, the image data of the closest size is stored.-And the process proceeds to processing to the basic component decoding unit 804, the target frequency decoding unit 805, and the target image output unit 806. At this time, at the time of inverse orthogonal transformation in the target image output means 806, the inverse orthogonal transformation processing corresponding to the closest image size is taken.
[0071]
However, at this stage, it is also possible to select the image size to be selected to be smaller than the desired image size and add the shortage component estimation means 500 in the fifth embodiment to the shortage.
[0072]
As described above, a low-frequency component that can be regarded as common among frequency components of a plurality of enlargement sizes is extracted as a basic component, and the remaining high-frequency components of each enlargement size are encoded around this basic component. Image frequency data relating to size can be prepared. Therefore, at the time of reproducing an enlarged image having a target image size according to a user instruction, the enlarged image can be reproduced at high speed without estimating a high-frequency component that is insufficient for a desired enlargement size each time an instruction is issued.
[0073]
(Ninth embodiment)
Next, an image processing apparatus according to a ninth embodiment of the present invention will be described.
[0074]
In FIG. 9, the processing up to the image input means 10, the original image orthogonal transform means 11, the enlarged frequency estimation means 800 and the basic component extraction means 807 is the same as that of the image processing apparatus according to the sixth embodiment of the present invention. Therefore, explanation is omitted.
[0075]
The basic component compression unit 1000 compresses the basic component extracted by the basic component extraction unit 807, and the compression processing step is the same as the low component compression unit 300 in the image processing apparatus according to the fourth embodiment of the present invention. Since it is the same, description is abbreviate | omitted.
[0076]
Further, the processing in the multiplex encoding unit 802, the multiplex code combining unit 803 and the storage medium 15 following the basic component compression unit 1000 is the same as that of the image processing apparatus according to the sixth embodiment of the present invention. Description is omitted.
[0077]
With the configuration as described above, it is possible to reduce the image data capacity, which is a problem when holding a plurality of image sizes.
[0078]
(Tenth embodiment)
Finally, an image processing apparatus according to the tenth embodiment of the present invention will be described. FIG. 10 shows the configuration.
[0079]
First, the inter-pixel interpolation unit 1100 generates an enlarged image by interpolating pixels between the original image read by the image input unit 10 using an interpolation method as shown in FIG. Next, the convolution unit 1101 repeats the convolution process on the enlarged image to perform pixel enhancement.
[0080]
  FIG. 11 shows the steps of the convolution process. In the case of an image with unclear edges as shown in FIG.-The image cannot be extracted, and it is difficult to perform the image enlargement process in the processing steps as shown in FIG. However, the averaged pixel value can be easily emphasized by using the present invention. Note that the process of the convolution means 1101 has the same effect as repeating the process using the edge enhancement filter a plurality of times on the enlarged image. In the case of the conventional edge enhancement filter processing, there is a problem that if the appropriate filter is not selected, the number of repetitions becomes very large or edge enhancement is not performed at all. However, the convolution means 1101 performs convolution with the pixel value itself. Therefore, there is no worry about that. If the pixel value at the time of the Kth convolution process at the P point in FIG. 11C is Dp [K], the pixel value Dp [K + 1] at the P point by the K + 1th convolution process is ( It is defined as in Equation 5). This is to obtain an average value of convolution values in the vicinity of this pixel. Here, Ω represents the range of pixels to be convolved, U_Gaso represents the total number of pixels in Ω, and q represents an arbitrary pixel in Ω.
[0081]
[Equation 5]

When this processing is set to Λ for a set of pixels included in the enlarged interpolation image, processing is performed on all the pixels in this Λ.
[0082]
Next, the convergence determination means 1102 determines the convergence of the process by this convolution. It is based on the average value within Λ of the square error of the pixel value Dp [K-1] obtained at the (K-1) th time and the pixel value Dp [K] at the Kth time as shown in (Equation 6). When it is smaller than the convergence determination value Thred prepared in advance, the convolution processing is finished as it has converged, and the estimated enlarged image output unit 1312 outputs the estimated enlarged image.
[0083]
[Formula 6]

In (Equation 4), T_Gaso represents the total number of pixels in Λ.
[0084]
As described above, the image processing apparatus according to the tenth aspect of the present invention emphasizes an enlarged image by convolution calculation processing of interpolated image data, and does not need to perform frequency conversion that requires processing time, and can easily enlarge an enlarged image. realizable. Further, an enlarged image having edge information can be easily obtained even with respect to an original image in which the edge which is a problem when the edge information of the original image is used to eliminate the blur of the interpolated image.
[0085]
  Finally, in the above, the original image is-However, this is not always necessary, and a specific image or the like held on a magnetic disk or the like may be used.
[0086]
Further, it is the sixth and ninth embodiments as well as the related information of the low frequency component and the high frequency component obtained by the high component encoding means 13 in the image processing apparatus of the first and fourth embodiments of the present invention. It is added that the basic component obtained by the multiplex encoding means 802 in the image processing apparatus and the related information of the high frequency component of each enlarged image can be compressed by a reversible compression method such as the Spatial method.
[0087]
【The invention's effect】
  As described above, according to the first to fifth image processing apparatuses of the present invention, the low frequency component is extracted from the frequency component of the original image and compressed, and the remaining high frequency component is related to the low frequency component. Since the data is expressed based on the information, the capacity of the image data held in the storage medium is reduced, and the compression method is reversible, so that the sharpness is not lost at the time of image restoration. Further, since the thumbnail image is generated by the extracted low frequency component, it is possible to reduce noise caused by pixel thinning as in the conventional case. In addition, the image data stored in the storage medium-Even when the data is enlarged and output, it is possible to eliminate blurring of edges and blurring of details due to pixel interpolation as in the prior art.
[0088]
  Next, according to the sixth to ninth image processing apparatuses of the present invention, an image having a plurality of magnifications is prepared in advance for the original image, and a low frequency component that can be regarded as common among these is used as a basic component After extraction and compression, the high-frequency component of each remaining enlarged image is expressed based on the information related to the basic component, so that the amount of image data held in the storage medium is reduced, and the compression method is reversible. Therefore, the sharpness is not lost at the time of image restoration. When outputting an enlarged image of a desired size, a plurality of prepared imagesMagnificationIn order to select an image of the same size or the closest size from among the images, the image restoration can be speeded up.
[0089]
Finally, according to the image processing apparatus of the tenth aspect of the present invention, the input image is subjected to inter-pixel interpolation, and the obtained interpolated image data is subjected to a convolution operation a plurality of times. There is no need to perform such frequency conversion, and an enlarged image can be easily realized. Further, the edge information of the original image is used to eliminate the blur of the interpolated image, but an enlarged image having a clear edge can be obtained even for an original image in which extraction of edge information is difficult.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of first, second, and third embodiments of the present invention.
FIG. 2 is a schematic diagram illustrating a processing step according to the first embodiment of the present invention.
FIG. 3 is a schematic configuration diagram including an image processing apparatus according to a fourth embodiment of the present invention.
FIG. 4 is a schematic diagram showing a procedure of spatial prediction compression used in the fourth embodiment of the present invention.
FIG. 5 is a schematic configuration diagram including an image processing apparatus according to a fifth embodiment of the present invention.
FIG. 6 is a schematic diagram of a procedure of an insufficient component estimation unit used in the fifth embodiment of the present invention.
FIG. 7 is a block diagram showing a configuration of sixth, seventh, and eighth embodiments of the present invention.
FIG. 8 is a schematic diagram showing processing steps of a basic component extraction unit and a multiplex encoding unit used in the sixth embodiment of the present invention.
FIG. 9 is a schematic configuration diagram including an image processing apparatus according to a ninth embodiment of the present invention.
FIG. 10 is a block diagram showing a configuration of a tenth embodiment of the present invention.
FIG. 11 is a schematic diagram showing processing steps of the image processing apparatus according to the tenth embodiment of the present invention.
FIG. 12 is a block diagram illustrating a configuration of a conventional electronic still camera.
FIG. 13 is a block diagram showing the configuration of a conventional image enlarging apparatus that converts and enlarges the frequency domain.
FIG. 14 is an explanatory diagram showing an example of expansion by converting to a conventional frequency domain.
FIG. 15 is a block diagram illustrating a configuration of an image enlargement apparatus that performs enlargement by conventional pixel interpolation.
FIG. 16 is a schematic diagram illustrating conventional inter-pixel interpolation.
[Explanation of symbols]
10 Image input means
11 Original image orthogonal transform means
13 High component encoding means
14 Code synthesis means
15 storage media
16 Low component decoding means
17 High component decoding means
18 Original image output means
100 Low component extraction means
101 Reduced image generation means
102 Reduced image display means
300 Low component compression means
500 Insufficient component estimation means
501 Enlarged image output means
800 Expanded frequency estimation means
802 Multiple encoding means
803 Multiple code synthesis means
804 Basic component decoding means
805 Target frequency decoding means
806 Target image output means
807 Basic component extraction means
808 Basic image generation means
809 Basic image display means
1000 Basic component compression means
1100 Interpolating means between pixels
1101 Convolution means
1102 Convergence determining means
1300 Compression means
1301 Quantization table
1302 Entropy coding table
1303 Stretching means
1304 Pixel thinning means
1305 Reduced image display means
1306 DCT conversion means
1307 Quantization means
1308 Entropy encoding means
1309 IDCT conversion means
1310 Inverse quantization means
1311 Entropy decoding means
1312 Image output means
1400 "0" component embedding means
1401 Inverse orthogonal transform means
1402 Estimated enlarged image output means

Claims (21)

In an image processing apparatus that inputs and processes image data,
Original image orthogonal transform means for performing orthogonal transform on the image data input as the original image to generate the original image frequency component which is the frequency component of the original image by performing orthogonal transform using the data as frequency space data;
Low component extraction means for extracting the low frequency component from the generated original image frequency component;
High component encoding means for associating a high frequency component of the original image frequency component that is the remainder extracted by the low component extraction means with the low frequency component and encoding related information that is information thereof;
Code synthesizing means for synthesizing the data representing the low frequency component and the data representing the related information to generate simple image data which is image data having a data size smaller than the image data of the original image An image processing apparatus. Further comprising a low component compression means for compressing the amount of data representing the low frequency component;
The code synthesizing unit generates the simplified image data by synthesizing data representing a low frequency component whose data amount is compressed by the low component compression unit and data representing the related information. The image processing apparatus according to claim 1. The image processing apparatus has a storage medium, the simplified image data generated by the code synthesizing means is the image processing apparatus Ru is stored in the storage medium, the image processing apparatus according to claim 1 . In an image processing apparatus that inputs image data and displays a reduced image obtained by reducing the image size to a reduced size that is a size reduced from the original image,
Reduced image generating means for generating the reduced image by performing inverse orthogonal transform on the data representing the low frequency component;
Reduced image display means for displaying the reduced image;
Further comprising
The image processing apparatus according to claim 1, wherein the low component extraction unit extracts a low frequency component corresponding to the reduced size. In an image processing apparatus that restores an original image from simple image data that is image data having a data size smaller than the image data of the original image,
The simplified image data is obtained by subjecting the image data of the original image to orthogonal transformation using the frequency space data, that is, the low frequency component of the original image frequency component which is the frequency component of the original image. Low component decoding means for extracting data representing the low frequency component from the simplified image data,
The simplified image data further includes data representing related information, which is information relating the high-frequency component, which is the remainder obtained by removing the low-frequency component from the original image frequency component, to the low-frequency component. High-component decoding that extracts data representing the relevant information from the simplified image data and decodes the data representing the high-frequency component based on the data representing the relevant information and the data representing the low-frequency component Means,
An original image output means for restoring the original image by performing inverse orthogonal transform on the frequency component obtained by combining the low frequency component and the high frequency component, and outputting the original image; An image processing apparatus. The image processing apparatus has a storage medium, image processing according to the simplified image data in claim 5 in which the image processing apparatus reads from the storage medium to the low component decoding means or the high component decoding means apparatus. In an image processing apparatus that enlarges an original image to an image of a desired image size and outputs the image,
When the image data of the original image is enlarged, the high frequency component to be supplemented to the frequency component of the original image, which is obtained by performing orthogonal transformation on the image data of the original image as frequency space data, A component, and insufficient component estimation means for estimating based on related information, which is information indicating a relationship between the frequency component of the original image and the high frequency component to be supplemented;
A desired image size can be obtained by performing inverse orthogonal transformation on the frequency component as a frequency component of an enlarged image obtained by combining the frequency component of the original image and the high frequency component estimated by the insufficient component estimation unit. And an enlarged image output means for outputting an enlarged image. The image processing apparatus has a storage medium, the frequency components and the related information of the original image, the image processing apparatus can decode the, respectively then data stored in the storage medium, wherein Item 8. The image processing device according to Item 7. In an image processing apparatus that inputs and processes image data,
Original image orthogonal transform means for performing orthogonal transform on the image data input as the original image to generate the original image frequency component which is the frequency component of the original image by performing orthogonal transform using the data as frequency space data;
The generated original image frequency component is regarded as a low frequency component in an enlarged frequency component that is a frequency component of an enlarged image obtained by enlarging the original image at a desired enlargement ratio, and the enlarged frequency component is regarded as the original image frequency component. And an expanded frequency estimation means for estimating based on related information, which is information indicating the relationship between the low frequency component and the remaining high frequency component in the expanded frequency component,
The original image frequency component and the plurality of enlargement frequency components respectively estimated for a plurality of enlargement factors are enlarged to enlarge from one original image to a plurality of images having different enlargement factors, that is, different image sizes. Basic component extraction means for extracting, as a basic component, a frequency component necessary for generating a basic image that is an image having a common image size as a source;
Multiple image encoding means for associating the expanded frequency component estimated by the expanded frequency estimation means with the basic component and encoding related information that is information thereof;
Multiplex code synthesizing means for synthesizing the data representing the basic component and the data representing the related information to generate multiple simple image data having a data size smaller than the image data of the original image. An image processing apparatus. Further comprising basic component compression means for compressing the amount of data representing the basic component;
The multiplex code synthesizing unit generates the multiplex simple image data by synthesizing data representing the basic component whose data amount is compressed by the basic component compressing unit and data representing the related information. The image processing apparatus according to claim 9, wherein the image processing apparatus is characterized. The image processing apparatus has a storage medium, the multiple simplified image data generated by the multiplexing code combining means the image processing apparatus Ru is stored in the storage medium, image according to claim 9 Processing equipment. Basic image generation means for generating the basic image by performing inverse orthogonal transform on the data representing the basic component;
The image processing apparatus according to claim 9, further comprising basic image display means for displaying the basic image. In an image processing apparatus that generates and outputs an enlarged image obtained by enlarging an original image to a desired size from multiple simple image data that is image data having a data size smaller than the image data of the original image.
The multiple simplified image data is obtained by subjecting image data to orthogonal transformation using the frequency space data, that is, a plurality of images that are frequency components of an image and have different image sizes from one original image. Therefore, a basic component that is a frequency component necessary for generating a basic image that is an image having a common image size is included as a source of enlargement, and the basic component is represented from the multiplexed simple image data. Basic component decoding means for retrieving the processed data;
The multiple simplified image data further includes data representing related information, which is information relating the high-frequency component, which is the remaining frequency component of the magnified image with the basic component removed, to the basic component. And extracting from the multiplexed simple image data the data representing the related information about the enlarged image of a desired image size, and the high-frequency component based on the data representing the related information and the data representing the basic component A target frequency decoding means for decoding the data representing
A target image output means for generating a magnified image of a desired size by performing inverse orthogonal transform on the frequency component obtained by combining the basic component and the high-frequency component, and generating a magnified image of a desired size; An image processing apparatus comprising the image processing apparatus. The image processing apparatus has a storage medium, the multiple simplified image data is the image processing apparatus is characterized in that read from the storage medium to the basic component decoding means or the target frequency decoding means, wherein Item 14. The image processing apparatus according to Item 13. Claim 6,8 the image processing apparatus, the processing of reading the frequency components and the related information of the simplified image data or the original image from the storage medium, characterized that you started by an instruction from the user, Or the image processing apparatus as described in any one of 14. In an image processing apparatus for inputting an image data and obtaining an enlarged image obtained by enlarging the image,
Inter-pixel interpolation means for interpolating between pixels of the original image with new pixels according to a desired enlargement ratio for the input original image;
A convolution means for performing a convolution calculation for performing pixel enhancement on the interpolated enlarged image that is an image interpolated by the inter-pixel interpolation means;
The value based on the difference between the pixel value in the enlarged image obtained by the Kth convolution process and the pixel value in the enlarged image obtained by the K-1th convolution process is a pre-set convergence condition. Convergence determining means for determining whether or not satisfied,
14. An enlarged image output means for outputting an image obtained by the convolution process when the convergence determination means determines that the image has converged, further comprising: an enlarged image output means. The image processing apparatus according to any one of the above.   Extracting the low frequency component from the frequency component of the input original image, associating the extracted high frequency component with the low frequency component to obtain related information as the information, and the low frequency component An image processing method for generating simple image data that is image data having a data size smaller than the image data of the original image by synthesizing data representing the related information and data representing the related information.   The data representing the related information, which is information relating the high-frequency component, which is the remainder of the low-frequency component extracted from the frequency component of the image, to the low-frequency component, and the data representing the low-frequency component are combined. The data representing the low frequency component and the data representing the related information are extracted from the simple image data which is image data having a data size smaller than the image data of the original image, and the related information is represented. Based on the data and the data representing the low frequency component, the data representing the high frequency component is decoded and combined with the data representing the low frequency component, and the combined frequency component is used as the frequency component. An image processing method for outputting the original image by performing inverse orthogonal transform on components.   When enlarging an original image to an image having a desired image size, a high frequency component to be supplemented with a frequency component of the original image is obtained by combining the frequency component of the original image and the frequency component of the original image with the high frequency component to be supplemented. Estimated on the basis of related information, which is information indicating the relationship, and a frequency component of the enlarged image obtained by combining the frequency component of the original image and the estimated high frequency component is inversely orthogonal to the frequency component. An image processing method for outputting an image obtained by enlarging the original image to a desired image size by performing conversion.   A frequency component of an original image having a predetermined image size and a frequency component of a plurality of enlarged images obtained by enlarging the original image of the image size for a plurality of image sizes are regarded as common to the frequency components of the images of the plurality of image sizes. Extracting the low-frequency component as a basic component, associating the high-frequency component that has been extracted for each frequency component with the basic component, obtaining related information that is data and information representing the information, and An image processing method for generating multiplexed simple image data having a data size smaller than the image data of the original image by combining the basic component and data representing the related information.   A frequency component of an original image having a predetermined image size and a frequency component of a plurality of enlarged images obtained by enlarging the original image of the image size with respect to a plurality of image sizes. The data representing the related information, which is information relating the low-frequency component that is the remaining low-frequency component extracted as the basic component to the low-frequency component, and the data representing the basic component Data representing the basic component and data representing the related information are extracted from the multiplexed simple image data, which is image data that has been combined and is smaller than the image data of the original image, and the related information Based on the data representing the low frequency component and the data representing the low frequency component, the data representing the high frequency component is decoded to represent the low frequency component Combined with chromatography data, and image processing method thereof as a frequency component which bound to the original image by performing inverse orthogonal transformation to the frequency components to expand to the desired image size output.

Images