JPH11136693A - Electron endoscope - Google Patents

Electron endoscope

Info

Publication number
JPH11136693A
JPH11136693A JP9301019A JP30101997A JPH11136693A JP H11136693 A JPH11136693 A JP H11136693A JP 9301019 A JP9301019 A JP 9301019A JP 30101997 A JP30101997 A JP 30101997A JP H11136693 A JPH11136693 A JP H11136693A
Authority
JP
Japan
Prior art keywords
signal
luminance
color
band
correction
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
JP9301019A
Other languages
Japanese (ja)
Other versions
JP3917733B2 (en
Inventor
Shoichi Amano
正一 天野
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Olympus Corp
Original Assignee
Olympus Optical Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Olympus Optical Co Ltd filed Critical Olympus Optical Co Ltd
Priority to JP30101997A priority Critical patent/JP3917733B2/en
Publication of JPH11136693A publication Critical patent/JPH11136693A/en
Application granted granted Critical
Publication of JP3917733B2 publication Critical patent/JP3917733B2/en
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Landscapes

  • Processing Of Color Television Signals (AREA)
  • Instruments For Viewing The Inside Of Hollow Bodies (AREA)
  • Endoscopes (AREA)
  • Color Television Image Signal Generators (AREA)

Abstract

PROBLEM TO BE SOLVED: To prevent deterioration of an S/N with a simple structure and to provide a luminance reproducibility by generating a high band luminance signal from an image pickup signal outputted from a solid-state image pickup element, generating plural low band color signals, detecting a hue by using at least more than one low band color signal out of the low band color signals and correcting the high band luminance signal with a correction amount output corresponding to the hue. SOLUTION: A luminance generation part 20 generates a high band luminance signal YH from an image pickup signal by an LPF 21, and applies an outline compensation to the generated high band luminance signal YH by an enhancement circuit 22, whereas a γ correction circuit 23 executes a γ correction of a luminance element. A band pass filter 31 extracts a color carrier element from the image pickup signal, and generates a line sequential color difference signals CR/CB, while a 1H delay circuit 32 performs one horizontal period delay the line sequential color difference signals CR/CB, and a selector 33 switches for every horizontal period the delayed line sequential color difference signals CR/CB and the non-delayed line sequential color difference signals CR/CB thus a matrix circuit 34 converts them into primary colors RGB.

Description

【発明の詳細な説明】DETAILED DESCRIPTION OF THE INVENTION

【0001】[0001]

【発明の属する技術分野】本発明は電子内視鏡装置、更
に詳しくは補色フィルタを有する固体撮像素子からの撮
像信号による輝度信号の生成部分に特徴のある電子内視
鏡装置に関する。
BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an electronic endoscope apparatus, and more particularly to an electronic endoscope apparatus characterized by a portion for generating a luminance signal based on an imaging signal from a solid-state imaging device having a complementary color filter.

【0002】[0002]

【従来の技術】従来より電子内視鏡装置の映像信号処理
回路では、挿入部先端に設けられた固体撮像素子(以
下、CCDと略す)により被写体を撮像し、映像信号処
理部において映像信号を生成して観察画像をモニタに表
示している。
2. Description of the Related Art Conventionally, in a video signal processing circuit of an electronic endoscope apparatus, a subject is imaged by a solid-state image pickup device (hereinafter abbreviated as CCD) provided at the tip of an insertion section, and a video signal is processed in a video signal processing section. The generated and observed image is displayed on the monitor.

【0003】CCDの前面に補色フィルタを有する単板
形式の電子内視鏡からの撮像信号を信号処理する映像信
号処理回路においては、図9に示すように、CCD10
1から読み出された撮像信号はローパスフィルタ(以
下、LPFと略記する)102によって色キャリア成分
が除去され、高帯域輝度信号YHが得られる。また、L
PF103は前記LPF102よりも低いカットオフ周
波数のものが用いられ、これにより低帯域輝度信号YL
が得られる。バンドパスフィルタ(BPF)104で
は、前記撮像信号から色キャリア成分を抽出が用いら
れ、これにより線順次色差信号CR/CBが得られる。
In a video signal processing circuit for processing an image signal from a single-plate type electronic endoscope having a complementary color filter in front of the CCD, as shown in FIG.
The low-pass filter (hereinafter abbreviated as LPF) 102 removes the color carrier component from the imaging signal read from 1 to obtain a high-band luminance signal YH. Also, L
The PF 103 has a cut-off frequency lower than that of the LPF 102, whereby the low-band luminance signal YL is used.
Is obtained. In the bandpass filter (BPF) 104, extraction of a color carrier component from the image signal is used, whereby a line-sequential color difference signal CR / CB is obtained.

【0004】前記線順次色差信号CR/CBは1H遅延線
105とセレクタ106によって同時化され、色差信号
CR(=2R−G)、CB(=2B−G)が得られる。前
記YL及びCR、CB信号は、第1のマトリクス回路10
7に入力され、以下の行列式によって原色RGB信号に
変換される。
The line-sequential color difference signals CR / CB are synchronized by the 1H delay line 105 and the selector 106 to obtain color difference signals CR (= 2R-G) and CB (= 2B-G). The YL, CR, and CB signals are supplied to the first matrix circuit 10.
7 and converted into primary color RGB signals by the following determinant.

【0005】[0005]

【数1】 上式において、K11〜K33は、接続されるCCD101
の分光感度特性を考慮し、例えば赤を撮像した場合には
R信号が大きくなるように設定する。
(Equation 1) In the above equation, K 11 to K 33 are the connected CCD 101
In consideration of the spectral sensitivity characteristics described above, for example, when red is captured, the R signal is set to be large.

【0006】前記原色RGB信号は、ホワイトバランス
補正回路108において、光源の色温度変化に対する補
色フィルタの分光感度特定のばらつきを補正している。
The white balance correction circuit 108 corrects a variation in the spectral sensitivity of the complementary color filter with respect to a change in the color temperature of the light source from the RGB signals of the primary colors.

【0007】ホワイトバランス補正回路108でのホワ
イトバランス後のRGB信号は、γ補正回路109にお
いてR、G、B独立にγ補正が施され、第2のマトリク
ス回路110によって、以下に示すようなNTSC方式
の規格にあった変換によって再度輝度信号YL’、色差
信号R−Y、B−Y信号に変換される。
The RGB signals after the white balance in the white balance correction circuit 108 are subjected to γ correction independently for R, G, and B in the γ correction circuit 109, and the second matrix circuit 110 performs the following NTSC The luminance signal YL ', the chrominance signal RY, and the BY signal are converted again by the conversion conforming to the standard of the system.

【0008】[0008]

【数2】 上記第2のマトリクス回路110によって得られた輝度
信号YL’は次のような特徴を持っている。
(Equation 2) The luminance signal YL 'obtained by the second matrix circuit 110 has the following characteristics.

【0009】(1)ホワイトバランス補正後のRGB信
号より生成されるため、輝度再現性が良い。
(1) Since luminance is generated from RGB signals after white balance correction, luminance reproducibility is good.

【0010】(2)色差信号との演算により混色が生じ
ており、S/Nが悪い。
(2) Color mixing occurs due to the calculation with the color difference signal, and the S / N is poor.

【0011】一方、前記高帯域輝度信号YHは、エンハ
ンス回路111により輪郭補償された後、γ補正回路1
12において輝度成分についてのγ補正が施される。
On the other hand, after the high-band luminance signal YH is subjected to contour compensation by the enhancement circuit 111, the γ correction circuit 1
At 12, the luminance component is subjected to γ correction.

【0012】ここで、前記高帯域輝度信号YH信号はC
CD101の撮像信号から色キャリア成分を除去して得
られるものであるため、画素の混色はなく、S/Nの劣
化は生じていない。
Here, the high-band luminance signal YH is C
Since it is obtained by removing the color carrier component from the image pickup signal of the CD 101, there is no color mixing of pixels and no deterioration of S / N occurs.

【0013】しかし、その分光感度特性は、補色フィル
タの分光感度に依存し、実際の目の視感度とは異なるた
め輝度再現性の悪いものである。
However, the spectral sensitivity characteristic depends on the spectral sensitivity of the complementary color filter and differs from the actual visual sensitivity of the eye, so that the luminance reproducibility is poor.

【0014】そのため、従来は、S/Nは悪いが輝度再
現性の良い前記YL’信号を、加算器113によって、
輝度再現性は悪いがS/Nの良い前記YH信号に加算す
ることでこの問題を改善し、エンコーダ114によって
前記R−Y、B−Y信号と、補正後の高帯域輝度信号Y
H’を用いてビデオ信号に変換していた。
Therefore, conventionally, the YL ′ signal, which has a poor S / N but good luminance reproducibility, is
This problem is remedied by adding to the YH signal, which has poor luminance reproducibility but good S / N, and the RY and BY signals by the encoder 114 and the corrected high-band luminance signal Y
It was converted to a video signal using H '.

【0015】ところが、前述したように、前記YL’信
号はS/Nが悪く、補正によって輝度再現性とS/Nが
トレードオフになるという問題があった。
However, as described above, the S / N of the YL 'signal is poor, and there is a problem that the luminance reproducibility and the S / N are traded off by the correction.

【0016】そこで、例えば特開平2−288574号
公報においては、色の評価手段を設け、S/Nは良いが
輝度再現性の悪いYH信号と、S/Nは悪いが輝度再現
性の良いYL’信号の混合比を色相により制御して補正
信号を生成し、S/Nの劣化を防ぐ提案がなされてい
る。
For example, in Japanese Patent Application Laid-Open No. 2-288574, a color evaluation means is provided, and a YH signal having a good S / N but poor luminance reproducibility and a YL signal having a poor S / N but good luminance reproducibility are provided. 'A proposal has been made to generate a correction signal by controlling the mixing ratio of the signals based on the hue, thereby preventing the S / N from deteriorating.

【0017】すなわち、図9に示した構成と同様の構成
には同一の符号を付し詳細な説明は省略するが、特開平
2−288574号公報では、図10に示すように、第
2のマトリクス回路110によって得られた色差信号R
−Y、B−Yは、評価回路121に入力されて色の飽和
度、色温度等の評価を行い、その評価結果を加算器12
2に出力する。加算器122では、高帯域輝度信号YH
と、前記YL’を、前記評価結果に応じて異なる混合比
にて加算する。これにより、被写体の色情報に応じてS
/Nの良いYH信号と輝度再現性の高いYL’信号の利点
を生かしながら、S/N劣化が少なく輝度再現性の高い
輝度信号を得ることを可能としている。
That is, the same components as those shown in FIG. 9 are denoted by the same reference numerals, and detailed description thereof is omitted. However, in Japanese Patent Laid-Open No. 2-288574, as shown in FIG. The color difference signal R obtained by the matrix circuit 110
-Y and BY are input to an evaluation circuit 121 to evaluate the color saturation, color temperature, and the like, and the evaluation results are added to an adder 12.
Output to 2. In the adder 122, the high-band luminance signal YH
And YL ′ are added at different mixing ratios according to the evaluation result. As a result, according to the color information of the subject, S
While taking advantage of the YH signal with good / N and the YL 'signal with high luminance reproducibility, it is possible to obtain a luminance signal with little S / N deterioration and high luminance reproducibility.

【0018】[0018]

【発明が解決しようとする課題】上記特開平2−288
574号公報においては、原色RGB信号によってホワ
イトバランス補正を行った後に、再度低帯域輝度信号Y
L’を生成し、これを低帯域輝度信号YLとある比率で混
合し補正量を生成することで、S/Nの劣化を防止する
試みがなされている。
SUMMARY OF THE INVENTION The above-mentioned Japanese Patent Application Laid-Open No. 2-288 is disclosed.
In Japanese Patent Application Laid-Open No. 574, 574, after performing white balance correction using the primary color RGB signals, the low band luminance signal Y
Attempts have been made to prevent S / N degradation by generating L 'and mixing it with the low-band luminance signal YL at a certain ratio to generate a correction amount.

【0019】しかしながら、評価回路121や混合比の
選択を行うための加算器122等の手段が、非常に複雑
になるという問題がある。
However, there is a problem that means such as the evaluation circuit 121 and the adder 122 for selecting the mixture ratio become very complicated.

【0020】また、補正の自由度も低いため、特定の色
相について着目した場合、十分な補正が得られないとい
った問題もある。特に、医用電子内視鏡装置において
は、被写体は赤系の色相がほとんどであり、輝度再現性
は、赤系の色相で非常に重要となる。また、出血部位な
ど彩度の高い赤色を近接から撮像する場合が多く、この
ような場合には、色再現性を重視するために輝度再現性
が特に損なわれがちとなっていた。このような場合に輝
度再現性が損なわれると、観察部位の起伏が観察しづら
くなるため、検査に支障をきたす虞がある。
Further, since the degree of freedom of correction is low, there is also a problem that a sufficient correction cannot be obtained when paying attention to a specific hue. In particular, in a medical electronic endoscope apparatus, most subjects have a reddish hue, and luminance reproducibility is very important for a reddish hue. In many cases, a highly saturated red color, such as a bleeding part, is imaged from close-up. In such a case, the luminance reproducibility tends to be particularly impaired because the color reproducibility is emphasized. In such a case, if the luminance reproducibility is impaired, it is difficult to observe the undulation of the observation site, which may hinder the inspection.

【0021】本発明は、上記事情に鑑みてなされたもの
であり、所望の色相についての輝度再現性に自由度を与
え、色相に応じて最適な補正を施し、簡単な構成にてS
/Nの劣化を防止すると共に高い輝度再現性を得ること
ができ、内視鏡検査における診断性と安全性を向上させ
ることのできる電子内視鏡装置を提供することを目的と
している。
The present invention has been made in view of the above circumstances, and provides a degree of freedom in luminance reproducibility for a desired hue, performs optimal correction in accordance with the hue, and implements a simple configuration.
It is an object of the present invention to provide an electronic endoscope apparatus that can prevent deterioration of / N and obtain high luminance reproducibility, and can improve diagnostic performance and safety in endoscopy.

【0022】[0022]

【課題を解決するための手段】本発明の電子内視鏡装置
は、挿入部の先端部に補色フィルタを有する固体撮像素
子を有した電子内視鏡と、前記固体撮像素子から出力さ
れる撮像信号から映像信号を生成する映像信号処理部と
を備えた電子内視鏡装置において、前記固体撮像素子か
ら出力される撮像信号から高帯域輝度信号を生成する輝
度信号生成手段と、前記固体撮像手段から出力される撮
像信号から複数の低帯域色信号を生成する色信号生成手
段と、前記低帯域色信号のうち少なくとも1つ以上の低
帯域色信号を用いて色相を検出する色相検出手段と、前
記低帯域色信号の少なくとも1つ以上の低帯域色信号を
用いて色相に応じた補正量を生成する補正量生成手段
と、前記色相検出手段及び前記補正量生成手段の出力を
受けて前記高帯域輝度信号を補正する輝度補正手段とを
備えて構成される。
According to an electronic endoscope apparatus of the present invention, an electronic endoscope having a solid-state image sensor having a complementary color filter at a distal end of an insertion portion, and an image output from the solid-state image sensor are provided. An electronic endoscope apparatus comprising: a video signal processing unit configured to generate a video signal from a signal; a luminance signal generating unit configured to generate a high-band luminance signal from an imaging signal output from the solid-state imaging device; and the solid-state imaging unit. Color signal generation means for generating a plurality of low-band color signals from the imaging signal output from the, hue detection means for detecting hue using at least one or more low-band color signals of the low-band color signals, A correction amount generating unit that generates a correction amount corresponding to a hue using at least one or more low band color signals of the low band color signal; and receiving the output of the hue detecting unit and the correction amount generating unit, Band brightness Constructed and a luminance correcting means for correcting the signal.

【0023】本発明の電子内視鏡装置では、前記色相検
出手段が前記低帯域色信号のうち少なくとも1つ以上の
低帯域色信号を用いて色相を検出し、前記補正量生成手
段が前記低帯域色信号の少なくとも1つ以上の低帯域色
信号を用いて色相に応じた補正量を生成し、前記輝度補
正手段が前記色相検出手段及び前記補正量生成手段の出
力を受けて前記高帯域輝度信号を補正することで、所望
の色相についての輝度再現性に自由度を与え、色相に応
じて最適な補正を施し、簡単な構成にてS/Nの劣化を
防止すると共に高い輝度再現性を得ることができ、内視
鏡検査における診断性と安全性を向上させることを可能
とする。
In the electronic endoscope apparatus according to the present invention, the hue detecting means detects a hue using at least one of the low-band color signals among the low-band color signals, and the correction amount generating means detects the hue. A correction amount corresponding to the hue is generated using at least one or more low-band color signals of the band color signal, and the luminance correction unit receives the outputs of the hue detection unit and the correction amount generation unit, and receives the high-band luminance. By correcting the signal, the degree of freedom in luminance reproducibility for a desired hue is given, optimal correction is performed according to the hue, and the S / N is prevented from deteriorating with a simple configuration, and high luminance reproducibility is achieved. This makes it possible to improve diagnostic performance and safety in endoscopy.

【0024】[0024]

【発明の実施の形態】以下、図面を参照しながら本発明
の実施の形態について述べる。
Embodiments of the present invention will be described below with reference to the drawings.

【0025】図1ないし図8は本発明の一実施の形態に
係わり、図1は電子内視鏡装置の構成を示す構成図、図
2は図1の補色フィルタの構成を示す構成図、図3は図
1の映像信号処理装置の構成を示す構成図、図4は図3
の色相検出回路の構成を示す構成図、図5は図3の補正
量生成回路の構成を示す構成図、図6は図4の色相検出
回路の変形例の構成を示す構成図、図7は図5の補正量
生成回路の第1の変形例の構成を示す構成図、図8は図
5の補正量生成回路の第2の変形例の構成を示す構成図
である。
FIGS. 1 to 8 relate to an embodiment of the present invention. FIG. 1 is a block diagram showing a configuration of an electronic endoscope apparatus, and FIG. 2 is a block diagram showing a configuration of a complementary color filter of FIG. 3 is a block diagram showing the configuration of the video signal processing device of FIG. 1, and FIG.
FIG. 5 is a configuration diagram showing a configuration of the correction amount generation circuit of FIG. 3, FIG. 6 is a configuration diagram showing a configuration of a modification of the hue detection circuit of FIG. 4, and FIG. FIG. 8 is a configuration diagram showing a configuration of a first modification of the correction amount generation circuit of FIG. 5, and FIG. 8 is a configuration diagram showing a configuration of a second modification of the correction amount generation circuit of FIG.

【0026】図1に示すように、本実施の形態の電子内
視鏡装置1は、体腔内に挿入する可撓性を有する挿入部
2及び挿入部2の基端側に設けられた操作部3よりなり
体腔内の観察部位を撮像する電子内視鏡4と、この電子
内視鏡4に照明光を供給する光源装置5と、電子内視鏡
4からの撮像信号を信号処理しモニタ6に前記観察部位
の画像を表示させる映像信号処理装置7とを備えて構成
される。
As shown in FIG. 1, an electronic endoscope apparatus 1 according to the present embodiment has a flexible insertion section 2 to be inserted into a body cavity and an operation section provided on the base end side of the insertion section 2. 3, an electronic endoscope 4 for imaging an observation site in a body cavity, a light source device 5 for supplying illumination light to the electronic endoscope 4, and a monitor 6 for processing image signals from the electronic endoscope 4 for signal processing. And a video signal processing device 7 for displaying an image of the observation site.

【0027】電子内視鏡4は、操作部3から延出するユ
ニバーサルケーブル8により映像信号処理装置7に接続
され、さらにライトガイドケーブル9を介して光源装置
5に接続されている。
The electronic endoscope 4 is connected to a video signal processing device 7 by a universal cable 8 extending from the operation section 3, and further connected to a light source device 5 via a light guide cable 9.

【0028】光源装置5から供給される照明光は、前記
のライトガイドケーブル9、ユニバーサルケーブル8及
び電子内視鏡4の挿入部2内を挿通するライトガイド1
0により挿入部2の先端に伝送され、ライトガイド10
の出射端面より図示しない観察部位に照射されるように
なっている。
The illumination light supplied from the light source device 5 is transmitted through the light guide cable 9, the universal cable 8 and the insertion section 2 of the electronic endoscope 4.
0 is transmitted to the distal end of the insertion portion 2 and the light guide 10
Are irradiated to an observation site (not shown) from the emission end surface of the light emitting device.

【0029】観察部位の光学像は、電子内視鏡4の挿入
部2の先端面に設けられた、図2に示すマゼンダ(M
g)、グリーン(G)、シアン(Cy)、イエロー(Y
e)の各色フィルタを配列した色コーディング用の補色
フィルタ11により光学的に色分離された後、結像レン
ズ12により固体撮像素子、例えばCCD13の受光面
に結像するようになっている。
The optical image of the observation site is provided on the distal end surface of the insertion section 2 of the electronic endoscope 4 by a magenta (M) shown in FIG.
g), green (G), cyan (Cy), yellow (Y
The optical color separation is performed by a complementary color filter 11 for color coding in which the respective color filters are arranged, and then an image is formed on a light receiving surface of a solid-state imaging device, for example, a CCD 13 by an imaging lens 12.

【0030】そして、CCD13により光電変換され得
られた撮像信号は、挿入部2及びユニバーサルケーブル
8内を挿通する図示しない信号線により前記映像信号処
理装置7に伝送されるようになっている。
The imaging signal obtained by the photoelectric conversion by the CCD 13 is transmitted to the video signal processing device 7 through a signal line (not shown) passing through the insertion section 2 and the universal cable 8.

【0031】図3に示すように、映像信号処理装置7で
は、CCD13から出力された撮像信号は、輝度生成部
20及び色生成部30に入力される。
As shown in FIG. 3, in the video signal processing device 7, the image pickup signal output from the CCD 13 is input to the luminance generation unit 20 and the color generation unit 30.

【0032】輝度生成部20では、LPF21によって
撮像信号から高帯域輝度信号YHを生成し、LPF21
によって生成された高帯域輝度信号YHに対してエンハ
ンス回路22により輪郭補償が施され、さらにγ補正回
路23によって輝度成分についてのγ補正が施される。
In the luminance generating section 20, the LPF 21 generates a high-band luminance signal YH from the image pickup signal,
The high-band luminance signal YH generated as described above is subjected to contour compensation by the enhancement circuit 22, and further to γ correction of the luminance component by the γ correction circuit 23.

【0033】色生成部30では、バンドパスフィルタ
(以下、BPFと略記する)31によって撮像信号か
ら、カラーキャリア成分を抽出し、線順次色差信号CR
/CBを生成する。
The color generator 30 extracts a color carrier component from the image pickup signal by a band-pass filter (hereinafter, abbreviated as BPF) 31 and outputs a line-sequential color difference signal CR.
/ CB is generated.

【0034】さらに色生成部30では、1H遅延回路3
2によって線順次色差信号CR/CBに対して1Hの遅延
を行い、セレクタ33によって遅延された線順次色差信
号CR/CBと遅延されない線順次色差信号CR/CBを1
H毎に切り換える。これにより、前記線順次色差信号C
R/CBは同時化され出力される(CR、CB)。
Further, the color generator 30 includes a 1H delay circuit 3
2, the line sequential color difference signal CR / CB is delayed by 1H, and the line sequential color difference signal CR / CB delayed by the selector 33 and the line sequential color difference signal CR / CB that is not delayed by 1
Switch every H. Thereby, the line-sequential color difference signal C
R / CB is synchronized and output (CR, CB).

【0035】そして、前記CR、CB信号及び輝度生成部
20のLPF21によって色キャリア成分が除去された
高帯域輝度信号YHは、第1のマトリクス回路34に入
力され、以下の行列式によって原色RGB信号に変換さ
れる。
The CR and CB signals and the high-band luminance signal YH from which the color carrier component has been removed by the LPF 21 of the luminance generation section 20 are input to a first matrix circuit 34, and the primary color RGB signals are calculated by the following determinant. Is converted to

【0036】[0036]

【数3】 このとき、k11〜k33は、接続されるCCD13の分光
感度特性を考慮し、例えば赤を撮像したときはR信号が
大きくなるように設定する。
(Equation 3) At this time, k 11 to k 33 are set in consideration of the spectral sensitivity characteristics of the connected CCD 13 so that, for example, when an image of red is taken, the R signal becomes large.

【0037】以上のようにして得られたRGB信号は、
LPF35a、35b、35cによって低帯域に制限さ
れる。
The RGB signals obtained as described above are
The band is limited to a low band by the LPFs 35a, 35b, and 35c.

【0038】LPF35a、35b、35cを介したR
GB信号は、ホワイトバランス回路37に入力され、ホ
ワイトバランス回路37において、白を撮像したときに
R:G:B=1:1:1となるように、R,B信号に対
してゲイン調整を行い、光源の色温度変化に対して白の
バランスがとられる。
R through LPFs 35a, 35b and 35c
The GB signal is input to the white balance circuit 37, and the white balance circuit 37 adjusts the gain of the R and B signals so that R: G: B = 1: 1: 1 when capturing white. Thus, white balance is obtained with respect to a change in the color temperature of the light source.

【0039】ホワイトバランス回路37によりホワイト
バランス補償がなされたRGB信号は、次段のγ補正回
路38、色相検出回路39及び補正量生成回路40に出
力される。
The RGB signals subjected to white balance compensation by the white balance circuit 37 are output to the γ correction circuit 38, hue detection circuit 39, and correction amount generation circuit 40 at the next stage.

【0040】γ補正回路38は色成分のγ補正を行い第
2のマトリクス回路41に出力し、第2のマトリクス回
路41によって再度色差信号R−Y、B−Yに変換され
る。
The gamma correction circuit 38 performs gamma correction on the color components and outputs the result to the second matrix circuit 41, which converts the color components into color difference signals RY and BY again.

【0041】色相検出回路39は、図4に示すように、
ホワイトバランス回路37より入力されたG信号を係数
器39aによりk倍しG’信号を得、またB信号を係数
器39bによりm倍しB’信号を得る。このG’及び
B’信号は加算器39cによって加算され、判別回路3
9dに入力される。判別回路39dは、加算器39cに
よって加算されたG’+B’とR信号の大きさについて
比較を行い、次に示す判別式によって、色相の判別を行
う。
The hue detection circuit 39, as shown in FIG.
The G signal input from the white balance circuit 37 is multiplied by k by a coefficient unit 39a to obtain a G 'signal, and the B signal is multiplied by m by a coefficient unit 39b to obtain a B' signal. The G ′ and B ′ signals are added by the adder 39c,
9d. The discrimination circuit 39d compares the G '+ B' added by the adder 39c with the magnitude of the R signal, and discriminates the hue by the following discriminant.

【0042】 判別式:R≧k×G+m×Bのとき、補正指示信号発生 R<k×G+m×Bのとき、補正指示信号なし 補正量生成回路40は、図5に示すように、ホワイトバ
ランス回路37より入力されたG信号を係数器40aに
よりk倍しG’信号を得、またB信号を係数器40bに
よってm倍しB’信号を得る。係数倍されたG’及び
B’信号は加算器40cによって加算され、加算器40
dに入力される。加算器40dは、加算器40cの出力
とR信号の演算を行い、以下の信号Cを発生する。
Discriminant: When R ≧ k × G + m × B, a correction instruction signal is generated. When R <k × G + m × B, no correction instruction signal is provided. As shown in FIG. The G signal input from the circuit 37 is multiplied by k by the coefficient unit 40a to obtain a G 'signal, and the B signal is multiplied by m by the coefficient unit 40b to obtain a B' signal. The G ′ and B ′ signals multiplied by the coefficients are added by an adder 40c,
is input to d. The adder 40d calculates the output of the adder 40c and the R signal, and generates the following signal C.

【0043】C=R−(k×G+m×B) ここで、k=m=1の場合を考える。被写体色が赤原色
の場合、G=B=0であり、C=Rとなる。次に被写体
色が黄色の場合はR=G、B=0となり、C=R−G=
0となる。同様に、被写体がマゼンダの場合もC=0と
なる。
C = R− (k × G + m × B) Here, the case where k = m = 1 is considered. When the subject color is the primary red color, G = B = 0 and C = R. Next, when the subject color is yellow, R = G and B = 0, and C = RG−
It becomes 0. Similarly, when the subject is magenta, C = 0.

【0044】つまり、被写体色が赤の場合補正信号Cは
最大値Rとなり、黄色、マゼンダに近付くにつれて小さ
くなる。さらに、G、Cy、Bの方面の色となると、R
<G+BとなるためにC=0となる。つまり、被写体の
色相がRに近いほど補正信号Cの値が大きくなる。この
補正信号Cは、係数器40eによってn倍されて最適化
が行われた後にC’として出力される。
That is, when the subject color is red, the correction signal C has the maximum value R, and becomes smaller as approaching yellow or magenta. Further, when the color in the direction of G, Cy, B is obtained, R
Since G <B, C = 0. That is, the value of the correction signal C increases as the hue of the subject approaches R. The correction signal C is output as C 'after being n-times optimized by the coefficient unit 40e and optimized.

【0045】なお、色相検出回路39及び補正量生成回
路40の上記構成例においては、係数器39aと係数器
40a、係数器39bと係数器40b及び加算器39c
と加算器40cは、動作が同様であり、共通とすること
ができるため、回路の簡易化が可能である。
In the above-described configuration examples of the hue detection circuit 39 and the correction amount generation circuit 40, the coefficient units 39a and 40a, the coefficient units 39b and 40b, and the adder 39c
The operation of the adder 40c is the same as that of the adder 40c, so that the circuit can be simplified.

【0046】図3に戻り、前記色相検出回路39によっ
て得られた補正指示信号及び前記補正信号作成回路40
より出力された補正信号C’は、セレクタ42に入力さ
れる。セレクタ42は、色相検出回路39からの補正指
示信号に基づき所望の色相が検出された場合についての
み前記補正信号C’を輝度補正部43に出力する。
Returning to FIG. 3, the correction instruction signal obtained by the hue detection circuit 39 and the correction signal generation circuit 40
The output correction signal C ′ is input to the selector 42. The selector 42 outputs the correction signal C ′ to the luminance correction unit 43 only when a desired hue is detected based on the correction instruction signal from the hue detection circuit 39.

【0047】輝度補正部43は、例えばYH’=YH+
C’のようにして高帯域輝度信号YHと補正量C’を加
算し、これにより輝度レベルの補正を行う。なお、輝度
補正部43を乗算器等より構成し、YH’=YH×C’と
することでレベル補正してもよい。
The luminance correction unit 43 calculates, for example, YH '= YH +
As in C ', the high-band luminance signal YH and the correction amount C' are added, and thereby the luminance level is corrected. Note that the luminance correction section 43 may be configured by a multiplier or the like, and the level may be corrected by setting YH ′ = YH × C ′.

【0048】そして、第2のマトリクス回路41によっ
て得られた再度色差信号R−Y、B−Yと共に、補正後
の高帯域輝度信号YH’が同時にエンコーダ44に入力
され、エンコーダ44により標準テレビジョン信号へ変
換されてモニタ6に出力される。
Then, the corrected high-band luminance signal YH 'together with the chrominance signals RY and BY again obtained by the second matrix circuit 41 are simultaneously input to the encoder 44, and the standard television The signal is converted to a signal and output to the monitor 6.

【0049】このように本実施の形態では、色相検出回
路39が検出した所望の色相が検出された場合について
のみに、補正信号作成回路40が作成した補正信号C’
を輝度補正部43に出力することで、輝度補正部43で
高帯域輝度信号YHと補正量C’を加算し輝度レベルの
補正を行うので、所望の色相についての輝度再現性に自
由度を与え、色相に応じて最適な補正を施し、簡単な構
成にてS/Nの劣化を防止すると共に高い輝度再現性を
得ることができ、内視鏡検査における診断性と安全性を
向上させることができる。
As described above, in the present embodiment, the correction signal C ′ generated by the correction signal generation circuit 40 only when the desired hue detected by the hue detection circuit 39 is detected.
Is output to the luminance correction unit 43, and the luminance correction unit 43 adds the high-band luminance signal YH and the correction amount C ′ to correct the luminance level, so that the degree of freedom in luminance reproducibility for a desired hue is given. Optimum correction can be performed according to the hue, S / N can be prevented from deteriorating with a simple structure, and high luminance reproducibility can be obtained, and diagnostic performance and safety in endoscopy can be improved. it can.

【0050】なお、本実施の形態では、ホワイトバラン
ス補正後のRGB信号により色相の検出及び補正信号の
作成を行っているが、ホワイトバランス補正前のRGB
信号により行ってもよいし、第2のマトリクス回路41
より得られた色差信号R−Y、B−Yを用いて行っても
よい。いずれの場合においても、所望の色相を検出し
て、その色相によって最適な補正量を生成することで、
同様の効果を得ることができる。
In this embodiment, the hue is detected and the correction signal is generated based on the RGB signals after the white balance correction. However, the RGB signals before the white balance correction are generated.
It may be performed by a signal, or the second matrix circuit 41
This may be performed using the obtained color difference signals RY and BY. In any case, by detecting a desired hue and generating an optimal correction amount based on the hue,
Similar effects can be obtained.

【0051】また、図6に示すように、外部の図示しな
い指示手段からの制御信号a、bにより乗算係数u、v
を可変できる係数器51、52を係数器39a、39b
の代わりに構成した色相検出回路39でもよく、この場
合、乗算係数u、vを可変することで、補正する色相を
マゼンダ方向、Ye方向に変化させることが可能とな
る。
As shown in FIG. 6, the multiplication coefficients u and v are controlled by control signals a and b from an external instruction means (not shown).
Coefficient units 51a and 52b can be changed to coefficient units 39a and 39b.
In this case, the hue to be corrected can be changed in the magenta direction and the Ye direction by changing the multiplication coefficients u and v.

【0052】さらに、図7に示すように、外部の図示し
ない指示手段からの制御信号Lによりゲインwを可変で
きる係数器53を係数器40eの代わりに構成した補正
信号作成回路40でもよく、この場合、係数器53のゲ
インを外部からの制御信号Lにより変化させることで、
被写体の輝度成分の補正量を観察者の好みに応じて選択
することが可能となる。
Further, as shown in FIG. 7, a correction signal generating circuit 40 in which a coefficient unit 53 capable of varying the gain w by a control signal L from an external not shown indicating means may be used instead of the coefficient unit 40e may be used. In this case, by changing the gain of the coefficient unit 53 by the external control signal L,
The correction amount of the luminance component of the subject can be selected according to the viewer's preference.

【0053】また、図8に示すように、加算器40dと
係数器40eとの間に、LPFやコアリングにより構成
されるノイズ低減回路54を設けて補正信号作成回路4
0を構成しても良く、これにより補正信号Cについて、
さらに低ノイズ化を図ることが可能となる。
As shown in FIG. 8, a noise reduction circuit 54 composed of an LPF or coring is provided between the adder 40d and the coefficient unit 40e to provide a correction signal generation circuit 4.
0 may be configured so that the correction signal C
Further, noise can be reduced.

【0054】[付記] (付記項1) 挿入部の先端部に補色フィルタを有する
固体撮像素子を有した電子内視鏡と、前記固体撮像素子
から出力される撮像信号から映像信号を生成する映像信
号処理部とを備えた電子内視鏡装置において、前記固体
撮像素子から出力される撮像信号から高帯域輝度信号を
生成する輝度信号生成手段と、前記固体撮像手段から出
力される撮像信号から複数の低帯域色信号を生成する色
信号生成手段と、前記低帯域色信号のうち少なくとも1
つ以上の低帯域色信号を用いて、色相を検出する色相検
出手段と、前記低帯域色信号の少なくとも1つ以上の低
帯域色信号を用いて色相に応じた補正量を生成する補正
量生成手段と、前記色相検出手段及び前記補正量生成手
段の出力を受けて、前記高帯域輝度信号を補正する輝度
補正手段とを備えたことを特徴とする電子内視鏡装置。
[Supplementary note] (Supplementary note 1) An electronic endoscope having a solid-state imaging device having a complementary color filter at the tip of the insertion portion, and an image for generating a video signal from an imaging signal output from the solid-state imaging device In an electronic endoscope apparatus including a signal processing unit, a luminance signal generation unit configured to generate a high-band luminance signal from an imaging signal output from the solid-state imaging device, and a plurality of imaging signals output from the solid-state imaging unit. A color signal generating means for generating a low-band color signal; and at least one of the low-band color signals.
Hue detection means for detecting a hue by using one or more low-band color signals, and correction amount generation for generating a correction amount corresponding to the hue using at least one low-band color signal of the low-band color signals An electronic endoscope apparatus comprising: a luminance correction unit configured to receive the output of the hue detection unit and the correction amount generation unit and correct the high-band luminance signal.

【0055】(付記項2) 前記色信号生成手段は、ホ
ワイトバランス補正手段を有し、ホワイトバランス補正
前またはホワイトバランス補正後の前記低帯域色信号
を、原色RGB信号もしくは色差信号R−Y、B−Y信
号として出力可能であることを特徴とする付記項1に記
載の電子内視鏡装置。
(Additional Item 2) The color signal generation means has a white balance correction means, and converts the low band color signal before or after white balance correction into a primary color RGB signal or a color difference signal RY, 2. The electronic endoscope apparatus according to claim 1, wherein the electronic endoscope apparatus can be output as a BY signal.

【0056】[0056]

【発明の効果】以上説明したように本発明の電子内視鏡
装置によれば、色相検出手段が低帯域色信号のうち少な
くとも1つ以上の低帯域色信号を用いて色相を検出し、
補正量生成手段が低帯域色信号の少なくとも1つ以上の
低帯域色信号を用いて色相に応じた補正量を生成し、輝
度補正手段が色相検出手段及び補正量生成手段の出力を
受けて高帯域輝度信号を補正するので、所望の色相につ
いての輝度再現性に自由度を与え、色相に応じて最適な
補正を施し、簡単な構成にてS/Nの劣化を防止すると
共に高い輝度再現性を得ることができ、内視鏡検査にお
ける診断性と安全性を向上させることができるという効
果がある。
As described above, according to the electronic endoscope apparatus of the present invention, the hue detecting means detects the hue using at least one low-band color signal among the low-band color signals,
The correction amount generating means generates a correction amount corresponding to the hue using at least one or more low-band color signals of the low-band color signals, and the luminance correcting means receives the output of the hue detecting means and the correction amount generating means, and Since the band luminance signal is corrected, the degree of freedom in luminance reproducibility for a desired hue is given, optimal correction is performed according to the hue, and S / N is prevented from deteriorating with a simple configuration, and high luminance reproducibility is achieved. And the diagnostic performance and safety in endoscopy can be improved.

【図面の簡単な説明】[Brief description of the drawings]

【図1】本発明の一実施の形態に係る電子内視鏡装置の
構成を示す構成図
FIG. 1 is a configuration diagram showing a configuration of an electronic endoscope apparatus according to an embodiment of the present invention.

【図2】図1の補色フィルタの構成を示す構成図FIG. 2 is a configuration diagram showing a configuration of a complementary color filter of FIG. 1;

【図3】図1の映像信号処理装置の構成を示す構成図FIG. 3 is a configuration diagram showing a configuration of the video signal processing device of FIG. 1;

【図4】図3の色相検出回路の構成を示す構成図FIG. 4 is a configuration diagram showing a configuration of a hue detection circuit of FIG. 3;

【図5】図3の補正量生成回路の構成を示す構成図FIG. 5 is a configuration diagram showing a configuration of a correction amount generation circuit of FIG. 3;

【図6】図4の色相検出回路の変形例の構成を示す構成
FIG. 6 is a configuration diagram showing a configuration of a modification of the hue detection circuit of FIG. 4;

【図7】図5の補正量生成回路の第1の変形例の構成を
示す構成図
7 is a configuration diagram showing a configuration of a first modification of the correction amount generation circuit of FIG. 5;

【図8】図5の補正量生成回路の第2の変形例の構成を
示す構成図
8 is a configuration diagram showing a configuration of a second modification of the correction amount generation circuit of FIG. 5;

【図9】第1の従来例の映像信号処理装置の構成を示す
構成図
FIG. 9 is a configuration diagram showing a configuration of a video signal processing device of a first conventional example.

【図10】第2の従来例の映像信号処理装置の構成を示
す構成図
FIG. 10 is a configuration diagram showing a configuration of a video signal processing device of a second conventional example.

【符号の説明】[Explanation of symbols]

1…電子内視鏡装置 4…電子内視鏡 5…光源装置 6…モニタ 7…映像信号処理装置 11…補色フィルタ 13…CCD 20…輝度生成部 21、35a、35b、35c…LPF 22…エンハンス回路 23、38…γ補正回路 30…色生成部 31…BPF 32…1H遅延回路 33、42…セレクタ 34…第1のマトリクス回路 37…ホワイトバランス回路 39…色相検出回路 39a、39b、40a、40b、40e…係数器 39c、40c、40d…加算器 39d…判別回路 40…補正量生成回路 41…第2のマトリクス回路 43…輝度補正部 44…エンコーダ DESCRIPTION OF SYMBOLS 1 ... Electronic endoscope apparatus 4 ... Electronic endoscope 5 ... Light source apparatus 6 ... Monitor 7 ... Video signal processing apparatus 11 ... Complementary color filter 13 ... CCD 20 ... Luminance generation parts 21, 35a, 35b, 35c ... LPF 22 ... Enhance Circuits 23, 38 γ correction circuit 30 Color generator 31 BPF 32 1H delay circuit 33, 42 Selector 34 First matrix circuit 37 White balance circuit 39 Hue detection circuits 39a, 39b, 40a, 40b , 40e... Coefficient units 39c, 40c, 40d... Adders 39d... Discrimination circuits 40... Correction amount generation circuits 41.

Claims (1)

【特許請求の範囲】[Claims] 【請求項1】 挿入部の先端部に補色フィルタを有する
固体撮像素子を有した電子内視鏡と、前記固体撮像素子
から出力される撮像信号から映像信号を生成する映像信
号処理部とを備えた電子内視鏡装置において、 前記固体撮像素子から出力される撮像信号から高帯域輝
度信号を生成する輝度信号生成手段と、 前記固体撮像手段から出力される撮像信号から複数の低
帯域色信号を生成する色信号生成手段と、 前記低帯域色信号のうち少なくとも1つ以上の低帯域色
信号を用いて、色相を検出する色相検出手段と、 前記低帯域色信号の少なくとも1つ以上の低帯域色信号
を用いて色相に応じた補正量を生成する補正量生成手段
と、 前記色相検出手段及び前記補正量生成手段の出力を受け
て、前記高帯域輝度信号を補正する輝度補正手段とを備
えたことを特徴とする電子内視鏡装置。
1. An electronic endoscope having a solid-state imaging device having a complementary color filter at a distal end of an insertion portion, and a video signal processing unit for generating a video signal from an imaging signal output from the solid-state imaging device. In the electronic endoscope device, a luminance signal generating unit that generates a high-band luminance signal from an imaging signal output from the solid-state imaging device; and a plurality of low-band color signals from the imaging signal output from the solid-state imaging unit. A color signal generating means for generating; a hue detecting means for detecting a hue by using at least one low-band color signal among the low-band color signals; and at least one low band of the low-band color signal A correction amount generation unit configured to generate a correction amount corresponding to a hue using a color signal; and a luminance correction unit configured to receive the output of the hue detection unit and the correction amount generation unit and correct the high-band luminance signal. Electronic endoscope apparatus characterized by a.
JP30101997A 1997-10-31 1997-10-31 Electronic endoscope device Expired - Fee Related JP3917733B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP30101997A JP3917733B2 (en) 1997-10-31 1997-10-31 Electronic endoscope device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP30101997A JP3917733B2 (en) 1997-10-31 1997-10-31 Electronic endoscope device

Publications (2)

Publication Number Publication Date
JPH11136693A true JPH11136693A (en) 1999-05-21
JP3917733B2 JP3917733B2 (en) 2007-05-23

Family

ID=17891872

Family Applications (1)

Application Number Title Priority Date Filing Date
JP30101997A Expired - Fee Related JP3917733B2 (en) 1997-10-31 1997-10-31 Electronic endoscope device

Country Status (1)

Country Link
JP (1) JP3917733B2 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006135425A (en) * 2004-11-02 2006-05-25 Sony Corp Signal processor and processing method of solid state image sensor and imaging apparatus
US7215366B2 (en) 2000-09-18 2007-05-08 Sanyo Electric Co., Ltd. Tone correcting circuit and hue correcting circuit
JP2008178481A (en) * 2007-01-23 2008-08-07 Hoya Corp Image processor
WO2010016166A1 (en) * 2008-08-04 2010-02-11 パナソニック株式会社 Imaging device, image processing method, image processing program and semiconductor integrated circuit
US7880937B2 (en) 2004-10-20 2011-02-01 Fujinon Corporation Electronic endoscope apparatus
JP2011194111A (en) * 2010-03-23 2011-10-06 Olympus Corp Image processing device, image processing method, and program

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7215366B2 (en) 2000-09-18 2007-05-08 Sanyo Electric Co., Ltd. Tone correcting circuit and hue correcting circuit
US7880937B2 (en) 2004-10-20 2011-02-01 Fujinon Corporation Electronic endoscope apparatus
JP2006135425A (en) * 2004-11-02 2006-05-25 Sony Corp Signal processor and processing method of solid state image sensor and imaging apparatus
JP4661168B2 (en) * 2004-11-02 2011-03-30 ソニー株式会社 Signal processing apparatus and method for solid-state imaging device, and imaging apparatus
JP2008178481A (en) * 2007-01-23 2008-08-07 Hoya Corp Image processor
US8233038B2 (en) 2007-01-23 2012-07-31 Hoya Corporation Image-signal processing unit
WO2010016166A1 (en) * 2008-08-04 2010-02-11 パナソニック株式会社 Imaging device, image processing method, image processing program and semiconductor integrated circuit
US8223219B2 (en) 2008-08-04 2012-07-17 Panasonic Corporation Imaging device, image processing method, image processing program and semiconductor integrated circuit
JP5280448B2 (en) * 2008-08-04 2013-09-04 パナソニック株式会社 Image processing apparatus, image processing method, image processing program, and semiconductor integrated circuit
JP2011194111A (en) * 2010-03-23 2011-10-06 Olympus Corp Image processing device, image processing method, and program
US9095269B2 (en) 2010-03-23 2015-08-04 Olympus Corporation Image processing device, image processing method, and program
US9629525B2 (en) 2010-03-23 2017-04-25 Olympus Corporation Image processing device, image processing method, and program

Also Published As

Publication number Publication date
JP3917733B2 (en) 2007-05-23

Similar Documents

Publication Publication Date Title
JP4009626B2 (en) Endoscope video signal processor
US7221388B2 (en) Endoscope image sensing method and apparatus
US8979741B2 (en) Endoscopic apparatus
US8773522B2 (en) Endoscope apparatus
US7248296B2 (en) Automatic gain control device for electronic endoscope
JP4502577B2 (en) Electronic endoscope device
US8854445B2 (en) Endoscope apparatus
KR960016851B1 (en) Luminance signal forming circuit
EP2548498B1 (en) Endoscopic device
JP3565723B2 (en) Color image processing equipment
JP3917733B2 (en) Electronic endoscope device
JP4606838B2 (en) Electronic endoscope device
JP2000209605A (en) Video signal processing unit
JP2682626B2 (en) Transendoscopic spectroscopic diagnostic equipment
US5087967A (en) Color image pickup device having a level correcting circuit for correcting level variations in color image signals
JP2006061621A (en) Light control signal generator for endoscope
JP4086572B2 (en) Video signal processing device
JP2837913B2 (en) Color shift reduction device
JP2000350231A (en) Video signal processor
JP2003070009A (en) Imaging apparatus
JP3193416B2 (en) Color shift detection device
JP2846317B2 (en) Color imaging device
JP2010279457A (en) Electronic endoscope, electronic endoscopic system, and color adjusting method
JP3655679B2 (en) Endoscope device
KR101008488B1 (en) Image Processing Apparatus and Control Method Thereof

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20040906

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20061106

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20061114

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20070112

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20070206

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20070209

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20110216

Year of fee payment: 4

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20110216

Year of fee payment: 4

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20120216

Year of fee payment: 5

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20120216

Year of fee payment: 5

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20130216

Year of fee payment: 6

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20140216

Year of fee payment: 7

LAPS Cancellation because of no payment of annual fees