JP3006872B2 - High-resolution image synthesis processing method - Google Patents
High-resolution image synthesis processing methodInfo
- Publication number
- JP3006872B2 JP3006872B2 JP2300466A JP30046690A JP3006872B2 JP 3006872 B2 JP3006872 B2 JP 3006872B2 JP 2300466 A JP2300466 A JP 2300466A JP 30046690 A JP30046690 A JP 30046690A JP 3006872 B2 JP3006872 B2 JP 3006872B2
- Authority
- JP
- Japan
- Prior art keywords
- image
- camera
- images
- resolution
- horizontal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Lifetime
Links
- 230000015572 biosynthetic process Effects 0.000 title description 3
- 238000003786 synthesis reaction Methods 0.000 title description 3
- 238000003672 processing method Methods 0.000 title description 2
- 238000000034 method Methods 0.000 claims description 7
- 230000002194 synthesizing effect Effects 0.000 claims description 3
- 238000003384 imaging method Methods 0.000 description 10
- 238000000605 extraction Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 230000002596 correlated effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
- Color Television Image Signal Generators (AREA)
Description
【発明の詳細な説明】 〔産業上の利用分野〕 本発明は，従来のTVカメラで撮像し，該TVカメラの解
像度の水平，垂直方向各々２倍の解像度の画像を得るこ
とができる，高解像度画像合成処理方式に関するもので
ある。DETAILED DESCRIPTION OF THE INVENTION [Industrial Application Field] The present invention relates to a high-performance television camera capable of capturing an image with a conventional TV camera and obtaining an image having a resolution twice as high in the horizontal and vertical directions as the resolution of the TV camera. It relates to a resolution image synthesis processing method.
従来のこの種の撮像方式としては,CCDなどの固体撮像
素子を水平方向に物理的に振動させ，該固体撮像素子の
水平解像度を実効的に増大させる方法が報告されている
（テレビジョン学会技術報告TEBS94−3 ED772）。当該
方法は,CCD固体撮像素子を水平方向に1/2画素ピッチの
振幅でフィールドごとに物理的に振動させ，従来のほぼ
２倍の水平限界解像度を得る手法である。As a conventional imaging method of this type, a method has been reported in which a solid-state imaging device such as a CCD is physically vibrated in the horizontal direction to effectively increase the horizontal resolution of the solid-state imaging device (Technology of the Institute of Television Engineers of Japan). Report TEBS94-3 ED772). In this method, the CCD solid-state imaging device is physically vibrated in the horizontal direction at an amplitude of 1/2 pixel pitch for each field to obtain a horizontal limit resolution that is almost twice that of the conventional one.
当該従来の方法ではCCD固体撮像素子を物理的に振動
させるために,TVカメラ自身に特殊な振動機能を付加さ
せなければならない。また垂直方向に高解像度化されて
いない，という欠点がある。In the conventional method, a special vibration function must be added to the TV camera itself to physically vibrate the CCD solid-state imaging device. Another drawback is that the resolution is not increased in the vertical direction.
本発明の目的は，従来技術の欠点である特殊な固体撮
像素子振動機能を付加したTVカメラを使用しなければな
らないことや，垂直方向には高解像度化されていないこ
となどの欠点を解決するため，既存のTVカメラを使用
し，該TVカメラの外部から擾乱による振動を利用して水
平，垂直方向それぞれ高解像度をもつ画像を得ようとす
るものである。An object of the present invention is to solve the disadvantages of the prior art, such as the necessity of using a TV camera with a special solid-state imaging device vibration function and the fact that the resolution is not increased in the vertical direction. Therefore, an existing TV camera is used, and an image having a high resolution in each of a horizontal direction and a vertical direction is obtained by utilizing vibration caused by disturbance from outside the TV camera.
本発明では，既存のTVカメラを使用して静止画を撮像
する際，該TVカメラの設置状態における外部からの擾乱
により該TVカメラが振動することを利用して，基準とし
た画像から水平，垂直，斜め方向に半ドットずれた画像
を検索し，基準画像と併せて高解像度の画像を得る。According to the present invention, when a still image is captured using an existing TV camera, the TV camera vibrates due to external disturbance in the installation state of the TV camera, and the horizontal and horizontal positions are taken from the reference image. An image shifted by half a dot in the vertical and oblique directions is searched, and a high-resolution image is obtained together with the reference image.
既存のTVカメラでｎ枚の画像を撮像し，撮像された画
像の中から１枚を基準とし，基準とした画像から水平，
垂直，斜め方向に半ドットずれた３枚の画像を生成し，
撮像されたｎ枚の画像から該３枚の半ドットずれた画像
と相関の高い画像を３枚検索し，前記基準画像と検索さ
れた３枚の画像を組み合わせて高解像度の画像を得るよ
うにする。従来の技術では，固体撮像素子自身を振動さ
せて高解像度な撮像を行っているが，本発明では既存の
TVカメラを使用し，外部からの擾乱による該TVカメラ自
身の振動を利用して高解像度な撮像を行う点で異なる。An existing TV camera captures n images and one of the captured images is used as a reference.
Generate three images that are shifted half a dot vertically and diagonally,
In order to obtain a high-resolution image by retrieving three images having high correlation with the three half-dot-shifted images from the n captured images, and combining the reference image and the retrieved three images. I do. In the conventional technology, high-resolution imaging is performed by vibrating the solid-state imaging device itself.
The difference is that high-resolution imaging is performed using a TV camera and utilizing the vibration of the TV camera itself due to external disturbance.
第１図は本発明の実施例を示す。図中,1はTVカメラ,2
はA/D変換部,3はフレームメモリ部,4は基準画像抽出，
参照画像生成部,5は高相関画像検索部,6は基準画像，高
相関画像合成部,7はD/A変換部である。FIG. 1 shows an embodiment of the present invention. In the figure, 1 is a TV camera, 2
Is the A / D converter, 3 is the frame memory, 4 is the reference image extraction,
A reference image generation unit, 5 is a high correlation image search unit, 6 is a reference image, a high correlation image synthesis unit, and 7 is a D / A conversion unit.
TVカメラ１により撮像された画像はｎ枚だけA/D変換
部２でディジタル化され，フレームメモリ部３に蓄積さ
れる。フレームメモリ部３にｎ枚蓄積されている画像
は,TVカメラ１に加わった外部からの擾乱によりわずか
に変動しているものとする。フレームメモリ部３に蓄積
されているｎ枚の画像は，基準画像抽出，参照画像生成
部４にてｎ−ｋフレーム画像を基準画像として抽出され
る。同時に基準画像抽出，参照画像生成部４では該抽出
された基準画像から水平方向，垂直方向，斜め方向に半
ドットずれた３枚の参照画像を内挿処理により生成す
る。Only n images captured by the TV camera 1 are digitized by the A / D converter 2 and stored in the frame memory 3. It is assumed that the n images stored in the frame memory unit 3 slightly fluctuate due to external disturbance applied to the TV camera 1. The n images stored in the frame memory unit 3 are extracted by the reference image extraction and reference image generation unit 4 using the nk frame images as reference images. At the same time, the reference image extraction and reference image generation unit 4 generates three reference images shifted by half a dot in the horizontal, vertical, and oblique directions from the extracted reference image by interpolation.
第２図は内挿処理について説明する図であり,PS（i,
j）は，基準画像における水平方向ｉ番目，垂直方向ｊ
番目の画素の輝度値,ISは画素Ｐの集合で作られる１フ
レームの基準画像である。ここで，画素数はたとえば水
平方向ｌ個，垂直方向ｍ個とする。また,P′Ｈ（i,j）,
P′Ｖ（i,j）,P′Ｄ（i,j）は，水平方向，垂直方向，
斜め方向に半ドットずれた画素の輝度値,I′H,I′V,I′
Ｄはそれぞれ画素Ｐ′Ｈ（i,j）,P′Ｖ（i,j）,P′
Ｄ（i,j）の集合で作られる１フレームの参照画像であ
る。Ｐ′Ｈ（i,j）,P′Ｖ（i,j）,P′Ｄ（i,j）はそれ
ぞれ以下の式で表現される。FIG. 2 is a diagram for explaining the interpolation processing, and P S (i,
j) is the ith horizontal direction and the vertical direction j in the reference image.
Luminance value of th pixel, I S is the reference image of one frame made with the set of pixels P. Here, the number of pixels is, for example, 1 in the horizontal direction and m in the vertical direction. Also, P ′ H (i, j),
P ′ V (i, j) and P ′ D (i, j) are horizontal, vertical,
Luminance values of the pixels shifted by a half dot in the oblique direction, I 'H, I' V , I '
D is a pixel P ′ H (i, j), P ′ V (i, j), P ′
D (i, j) is a reference image of one frame made of a set. P ′ H (i, j), P ′ V (i, j) and P ′ D (i, j) are expressed by the following equations, respectively.
Ｐ′Ｈ（i,j）＝（PS（i,j）＋PS（ｉ＋1,j））/2 ……（１） Ｐ′Ｖ（i,j）＝（PS（i,j）＋PS（i,j＋１））/2 ……（２） Ｐ′Ｄ（i,j）＝（PS（i,j）＋PS（ｉ＋1,j）＋PS（i,j
＋１） PS（i,j＋１）＋PS（ｉ＋1,j＋１））/4 ……（３） 式（１），（２），（３）を該基準画像のすべての画
素に関して計算し,3枚の参照画像Ｉ′H,I′V,I′Ｄを生
成する。 P 'H (i, j) = (P S (i, j) + P S (i + 1, j)) / 2 ...... (1) P' V (i, j) = (P S (i, j) + P S (i, j + 1) ) / 2 ...... (2) P 'D (i, j) = (P S (i, j) + P S (i + 1, j) + P S (i, j
+1) P S (i, j + 1) + P S (i + 1, j + 1)) / 4 (3) Formulas (1), (2), and (3) are calculated for all pixels of the reference image, and three images are calculated. , The reference images I ′ H , I ′ V , and I ′ D are generated.
上述した方法により生成された参照画像Ｉ′H,I′V,
I′Ｄは，高相関画像検索部５に転送されて，フレーム
メモリ部３に蓄積されている基準画像ISを除くｎ−１枚
の画像から１番相関の高い画像IH,IV,IDを検索する。相
関の高い画像の検索方法はいくつかあるが，例えば参照
画像Ｉ′Ｈと一番相関が高い画像を検索する場合，検索
対象となる画像の輝度値をＰ（i,j）とすると，距離Ｄ
は以下の式で表現される。The reference images I ′ H , I ′ V ,
I ′ D is transferred to the high-correlation image search unit 5 and the highest correlation image I H , I V , out of n−1 images excluding the reference image I S stored in the frame memory unit 3 Search for ID . While searching method highly correlated images are several, for example the reference image I 'if the most correlated with H searches for high image, the brightness value P (i, j) of an image to be searched when the distance D
Is represented by the following equation.
Ｄが最小となる画像をｎ−ｋフレームの中から検索す
る方法がある。このようにして参照画像Ｉ′H,I′V,I′
Ｄと一番相関の高い画像IH,IV,IDを検索することができ
る。 There is a method of searching for an image in which D becomes the minimum from nk frames. In this way, the reference images I ′ H , I ′ V , I ′
D and highest correlation image I H, it is possible to find the I V, I D.
次に，上記方法などで検索された一番相関の高い画像
IH,IV,IDは，基準画像，高相関画像合成部６に転送さ
れ，基準画像ISの水平，垂直，斜め方向の画素間に挿入
されて2l×2mの画素数の画像が合成される。Next, the image with the highest correlation searched by the above method
I H , I V , and I D are transferred to the reference image / high correlation image synthesizing unit 6 and inserted between the horizontal, vertical, and oblique pixels of the reference image I S to form an image having a 2l × 2m pixel number. Synthesized.
第３図は，基準画像ISの水平，垂直，斜め方向の画素
間に，一番相関の高い画像IH,IV,IDがそれぞれ挿入され
て合成された画像を示す図である。検索された一番相関
の高い画像IH,IV,IDの画素をそれぞれPH（i,j）,PV（i,
j）,PD（i,j）とすると，まず，水平方向PS（i,j）,PS
（ｉ＋1,j）の間にPH（i,j）を，垂直方向PS（i,j）,PS
（i,j＋１）の間にPV（i,j）を挿入する。次に斜め方向
として,PS（i,j）,PS（ｉ＋1,j）,PS（i,j＋１）,P
S（ｉ＋1,j＋１）で囲まれる四角の中心にPD（i,j）を
挿入して所望の高解像度画像を合成する。該合成された
画像は,D/A変換部７に転送され高解像度画像として出力
される。Figure 3 is a horizontal reference image I S, vertical, between the diagonal direction of the pixel, the most high correlation image I H, is a diagram showing an image I V, I D is synthesized are inserted respectively. The pixels of the searched images I H , I V , and I D having the highest correlation are denoted by P H (i, j), P V (i,
j), P D (i, j), first, the horizontal direction P S (i, j), P S
P H (i, j) between (i + 1, j) and P S (i, j), P S
P V (i, j) is inserted between (i, j + 1). Next, as an oblique direction, P S (i, j), P S (i + 1, j), P S (i, j + 1), P
A desired high-resolution image is synthesized by inserting P D (i, j) at the center of the square surrounded by S (i + 1, j + 1). The synthesized image is transferred to the D / A converter 7 and output as a high-resolution image.
第４図は,TVカメラ１が書画像用として設置されてい
る構成を示す図であり,8はTVカメラスタンドである。該
TVカメラスタンド８を使用して机上にある文書などを撮
像する場合，机の振動などはTVカメラスタンド８を伝わ
ってTVカメラ１に伝達される。該振動は，通常の書画像
用TVカメラとしての使用法では書画像に悪影響を及ぼす
が，本発明では該振動を逆に利用して，高解像度の画像
を得ることができる。また,TVカメラ１は振動しながら
ｎ枚の画像を撮像するため，高速のシャッターが付加さ
れているTVカメラを使用すると，より高解像度の撮像が
可能となる。FIG. 4 is a diagram showing a configuration in which the TV camera 1 is installed for a book image, and 8 is a TV camera stand. The
When a document or the like on a desk is imaged using the TV camera stand 8, vibration of the desk or the like is transmitted to the TV camera 1 via the TV camera stand 8. The vibration adversely affects a book image when used as a normal book image TV camera, but in the present invention, a high-resolution image can be obtained by using the vibration in reverse. Further, since the TV camera 1 captures n images while vibrating, if a TV camera provided with a high-speed shutter is used, higher-resolution imaging can be performed.
また，以上では静止画をTVカメラスタンド８に取り付
けたTVカメラで撮像し高解像度の画像を作り出す場合を
例にとって説明したが，本発明は三脚に取り付けた望遠
レンズ付きTVカメラで入力した遠景の画像を鮮鋭化する
場合など，いろいろな場合に適用されることは言うまで
もない。In the above description, a case has been described in which a still image is captured by a TV camera attached to the TV camera stand 8 to produce a high-resolution image. However, the present invention is directed to a distant view input by a TV camera with a telephoto lens attached to a tripod. It goes without saying that the present invention is applied to various cases such as when sharpening an image.
以上説明したように，本発明によれば，既存のTVカメ
ラを使用して，該TVカメラに加わる外部からの擾乱によ
る当該TVカメラ自身の振動を利用して高解像度の画像を
得る方式であるから，既存のTVカメラを使用できる。水
平，垂直方向ともに高解像度の画像を得ることができ
る，という利点がある。本方式はテレビ会議，テレビ電
話などの書画TVカメラとして，実施例で述べたようなTV
カメラスタンドを使用すると高解像度の書画像を提供す
ることができる。As described above, according to the present invention, a high-resolution image is obtained using an existing TV camera and utilizing the vibration of the TV camera itself due to external disturbance applied to the TV camera. From, existing TV cameras can be used. There is an advantage that high-resolution images can be obtained in both the horizontal and vertical directions. This system is used as a document TV camera for videoconferencing, videophone, etc.
The use of the camera stand can provide a high-resolution document image.
第１図は本発明の実施例，第２図は内挿処理について説
明する図，第３図は基準画像ISの水平，垂直，斜め方向
の画素間に，一番相関の高い画像IH,IV,IDがそれぞれ挿
入されて合成された画像を示す図，第４図はTVカメラ１
が設置されている構成を示す図である。 １……TVカメラ,2……A/D変換部,3……フレームメモリ
部,4……基準画像抽出，参照画像生成部,5……高相関画
像検索部,6……基準画像，高相関画像合成部,7……D/A
変換部,8……TVカメラスタンド。Example of Figure 1 according to the present invention, FIG. FIG. 2 for explaining the interpolation process, a horizontal Figure 3 is a reference image I S, vertical, between the diagonal direction of the pixel, the highest correlation image I H , I V , and I D are respectively inserted into the TV camera 1 and FIG.
It is a figure showing composition in which is installed. 1 ... TV camera, 2 ... A / D converter, 3 ... Frame memory unit, 4 ... Reference image extraction and reference image generation unit, 5 ... High correlation image search unit, 6 ... Reference image, height Correlation image synthesizer, 7 ... D / A
Conversion unit, 8 …… TV camera stand.
Claims (1)
してディジタル化してメモリに蓄積する処理手段と， ｎ−ｋ番目（ｋ＝0,1,2,…,n−１）フレームの画像を基
準にして，水平方向，垂直方向，斜め方向に半ドットず
れた３枚の参照画像を内挿処理により生成する処理手段
と， 該生成された３枚の参照画像と一番相関の高い画像をｎ
−ｋ番目のフレームを除く1,2,…,nフレームの画像から
３枚検索する処理手段と， 該検索された３枚の高相関画像を基準画像の水平，垂
直，斜め方向の画素間に挿入し,1枚の画像に合成する処
理手段とを有する ことを特徴とする高解像度画像合成処理方式。1. A processing means for inputting an image for n (n ≧ 4) frame periods, digitizing the image, and storing it in a memory, and an (nk) th (k = 0, 1, 2,..., N−1) Processing means for generating, by interpolation, three reference images shifted by half a dot in the horizontal, vertical, and oblique directions with respect to the image of the frame, and a correlation with the three generated reference images Image with high
Processing means for retrieving three images from 1, 2,..., N-frame images excluding the k-th frame, and retrieving the three retrieved high-correlation images between pixels in the horizontal, vertical, and oblique directions of the reference image. A high-resolution image synthesizing method, comprising: processing means for inserting and synthesizing into one image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2300466A JP3006872B2 (en) | 1990-11-06 | 1990-11-06 | High-resolution image synthesis processing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2300466A JP3006872B2 (en) | 1990-11-06 | 1990-11-06 | High-resolution image synthesis processing method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| JPH04172778A JPH04172778A (en) | 1992-06-19 |
| JP3006872B2 true JP3006872B2 (en) | 2000-02-07 |
Family
ID=17885136
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| JP2300466A Expired - Lifetime JP3006872B2 (en) | 1990-11-06 | 1990-11-06 | High-resolution image synthesis processing method |
Country Status (1)
| Country | Link |
|---|---|
| JP (1) | JP3006872B2 (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP3190220B2 (en) * | 1994-12-20 | 2001-07-23 | シャープ株式会社 | Imaging device |
| JP3907729B2 (en) * | 1996-02-05 | 2007-04-18 | オリンパス株式会社 | Still image pickup device |
| TW583603B (en) | 2003-02-21 | 2004-04-11 | Inventec Appliances Corp | Method for producing enhanced-resolution image by use of a plurality of low-resolution images |
| JP4184319B2 (en) | 2004-06-10 | 2008-11-19 | オリンパス株式会社 | Imaging device |
-
1990
- 1990-11-06 JP JP2300466A patent/JP3006872B2/en not_active Expired - Lifetime
Also Published As
| Publication number | Publication date |
|---|---|
| JPH04172778A (en) | 1992-06-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7876980B2 (en) | Imaging apparatus and imaging method for outputting a specified number of pixels in a specified area | |
| US6678000B1 (en) | High resolution still-image capture apparatus that shifts pixels by plus or minus two-thirds pixel pitch | |
| JPH05260264A (en) | Image processor | |
| KR101251220B1 (en) | Image pickup apparatus, solid-state imaging device, and image generating method | |
| US7812875B2 (en) | Solid-state imaging device | |
| JPH11112956A (en) | Image composite and communication equipment | |
| US6876386B1 (en) | Digital camera with downsampling and zoom processing and method of controlling operation of same | |
| JP2006166400A (en) | Imaging apparatus and imaging method | |
| EP1460837A2 (en) | Digital still camera | |
| JP2018074361A (en) | Imaging apparatus, imaging method, and program | |
| JP3006872B2 (en) | High-resolution image synthesis processing method | |
| US6876387B1 (en) | Digital zoom-out processing apparatus | |
| JPS61198879A (en) | Television signal processor | |
| JP4001233B2 (en) | Panorama shooting method and image processing apparatus and image pickup apparatus capable of using the method | |
| JP3346481B2 (en) | Video input device | |
| JPH06292052A (en) | Still picture image pickup device | |
| JP3395195B2 (en) | Image distortion correction method | |
| US6452630B1 (en) | Method for arranging image numbers | |
| JPH09224180A (en) | Image pickup device | |
| JPH05276452A (en) | Image pickup device | |
| JPH1198395A (en) | Picture processor | |
| JPH06105232A (en) | Picture synthesis device | |
| JPH0918763A (en) | Image pickup device and video signal processing unit | |
| JP2703687B2 (en) | Electronic zoom method | |
| JP2003018470A (en) | Imaging apparatus, imaging method, program, and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20071126 Year of fee payment: 8 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20081126 Year of fee payment: 9 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20091126 Year of fee payment: 10 |
|
| FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20101126 Year of fee payment: 11 |
|
| EXPY | Cancellation because of completion of term |