JPH10274564A - Non-cooled infrared two-dimensional sensor camera with shading correcting function - Google Patents

Non-cooled infrared two-dimensional sensor camera with shading correcting function

Info

Publication number
JPH10274564A
JPH10274564A JP9506097A JP9506097A JPH10274564A JP H10274564 A JPH10274564 A JP H10274564A JP 9506097 A JP9506097 A JP 9506097A JP 9506097 A JP9506097 A JP 9506097A JP H10274564 A JPH10274564 A JP H10274564A
Authority
JP
Japan
Prior art keywords
shutter
shading
infrared
signal
dimensional sensor
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
JP9506097A
Other languages
Japanese (ja)
Other versions
JP3373387B2 (en
Inventor
Akira Sema
章 瀬間
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nippon Avionics Co Ltd
Original Assignee
Nippon Avionics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nippon Avionics Co Ltd filed Critical Nippon Avionics Co Ltd
Priority to JP09506097A priority Critical patent/JP3373387B2/en
Priority to US08/949,719 priority patent/US5994701A/en
Priority to EP97117829A priority patent/EP0837600A3/en
Publication of JPH10274564A publication Critical patent/JPH10274564A/en
Application granted granted Critical
Publication of JP3373387B2 publication Critical patent/JP3373387B2/en
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Landscapes

  • Radiation Pyrometers (AREA)

Abstract

PROBLEM TO BE SOLVED: To automatically make a shading correction including the influence of a lens in a short time without an operator needing to set a uniform plane heat source manually in an infrared camera. SOLUTION: A shutter 2 easily opened/closed in an instant by a shutter controller 3 is provided in the optical pupil position of a combination lens 1. Thermal image data at the time of closing the shutter 2 is story in a shading memory 8, and a subtracter 7 subtracts stored shading memory data from the thermal image data at the time of photographing the infrared image of the external work in the opened state of the shutter 2 every corresponding picture element, and outputs the result to a frame memory 9.

Description

【発明の詳細な説明】DETAILED DESCRIPTION OF THE INVENTION

【0001】[0001]

【発明の属する技術分野】本発明は、赤外線2次元セン
サを用いた赤外線カメラにおけるシェーディングを補正
する技術の分野に関する。
The present invention relates to the field of techniques for correcting shading in an infrared camera using an infrared two-dimensional sensor.

【0002】[0002]

【従来の技術】赤外線カメラにおいては測定対象物とは
無関係にカメラ筐体内における迷光に起因して、2次元
センサ上に入射される赤外線エネルギー量が、環境温度
が高い場合には中心部から周辺にいくにつれてエネルギ
ー量が多くなり、逆に環境温度が低い場合には中心部か
ら周辺にいくにつれてエネルギー量が少なくなるという
いわゆるシェーディングと呼ばれる現象(温度勾配)を
生ずる。これは特に非冷却の赤外線カメラにおいて顕著
に現れる。本来温度勾配があってはならないところにこ
のような温度勾配があっては赤外線像から正確な温度測
定ができなくなるので、このシェーディングによる影響
を補正するための操作が行われていた。
2. Description of the Related Art In an infrared camera, the amount of infrared energy incident on a two-dimensional sensor due to stray light in a camera housing irrespective of an object to be measured increases from a central portion to a peripheral portion when an environmental temperature is high. In contrast, when the environmental temperature is low, a phenomenon (temperature gradient) called so-called shading occurs in which the amount of energy decreases from the center to the periphery when the environmental temperature is low. This is particularly noticeable in uncooled infrared cameras. If there is such a temperature gradient in a place where there should not be a temperature gradient, accurate temperature measurement cannot be performed from an infrared image. Therefore, an operation for correcting the influence of the shading has been performed.

【0003】その第1の方法は、手作業でレンズの前に
温度の均一な面熱源を置いたり或いはレンズにキャップ
を被せたうえで赤外線2次元センサの各画素の出力が均
一になるように補正していた。第2の方法は、レンズと
赤外線2次元センサとの間に、メカ駆動により均一な面
熱源であるプレートを挿入して補正する方法である。
The first method is to manually place a surface heat source having a uniform temperature in front of the lens or cover the lens so that the output of each pixel of the infrared two-dimensional sensor becomes uniform. Had been corrected. The second method is a method in which a plate, which is a uniform surface heat source, is inserted between the lens and the infrared two-dimensional sensor by mechanical driving to perform correction.

【0004】[0004]

【発明が解決しようとする課題】しかしながら、第1の
方法では、手作業で面熱源をセットするため補正に要す
る時間が長くかかるという問題があるし、第2の方法で
は、レンズの存在により生ずるシェーディングの影響を
補正することができないという問題がある。
However, in the first method, there is a problem that it takes a long time for correction because the surface heat source is set manually, and in the second method, it is caused by the presence of a lens. There is a problem that the influence of shading cannot be corrected.

【0005】本発明の目的は、上記従来技術の問題点に
鑑みて、カメラの操作者がわざわざ手作業で均一平面熱
源をセットする必要がなく、短時間で、レンズによるシ
ェーディングの影響をも補正し得るシェーディング補正
機能付非冷却赤外線2次元センサカメラを提供すること
にある。
SUMMARY OF THE INVENTION In view of the above-mentioned problems of the prior art, an object of the present invention is to eliminate the need for a camera operator to manually set a uniform planar heat source, and to correct the effects of shading by a lens in a short time. An object of the present invention is to provide an uncooled infrared two-dimensional sensor camera with a shading correction function that can be performed.

【0006】[0006]

【課題を解決するための手段】上記の目的を達成するた
めに本発明のシェーディング補正機能付非冷却赤外線2
次元センサカメラは次の各構成を具備することを特徴と
する。 (イ)光学瞳位置にシャッタを内蔵する赤外線像形成用
の組合わせレンズ (ロ)前記シャッタの開閉を制御するシャッタコントロ
ーラ (ハ)前記組合わせレンズによる赤外線像が結像される
2次元センサ (ニ)前記2次元センサに対する画素走査により出力さ
れるアナログ電気信号を画素毎にディジタル信号に変換
するA/D変換器 (ホ)前記A/D変換器の出力信号を一方入力端の入力
信号とし、他方入力端へ入力された信号との間で減算を
行う減算器 (ヘ)前記シャッタを閉じたときのA/D変換器の出力
を1フレーム分格納し、その後シャッタが開となった後
は、その格納された信号が、前記減算器の一方入力端に
入力されている信号の画素位置と一致させて読み出さ
れ、前記減算器の他方入力端へ入力されるシェーディン
グメモリ (ト)前記減算器の出力を1画面単位で格納するフレー
ムメモリ
In order to achieve the above object, an uncooled infrared ray 2 with a shading correction function according to the present invention is provided.
The dimensional sensor camera is characterized by having the following components. (A) Combination lens for forming an infrared image having a built-in shutter at the optical pupil position. (B) Shutter controller for controlling opening and closing of the shutter. (C) Two-dimensional sensor on which an infrared image is formed by the combination lens. D) an A / D converter that converts an analog electric signal output by pixel scanning to the two-dimensional sensor into a digital signal for each pixel; (e) an output signal of the A / D converter is used as an input signal at one input terminal A subtractor for performing subtraction with a signal input to the other input terminal. (F) After storing the output of the A / D converter when the shutter is closed for one frame, and then after opening the shutter, Are read out in such a manner that the stored signal is made coincident with the pixel position of the signal input to one input terminal of the subtractor, and the shading memo is input to the other input terminal of the subtractor. (G) a frame memory for storing an output of said subtractor with each screen

【0007】[0007]

【発明の実施の形態】本発明の実施の形態は、光学瞳位
置に外部から開閉制御が容易なシャッタを内蔵した組合
わせレンズを用い、シャッタを閉にしレンズを通しての
赤外線を遮断した状態で、迷光によるシェーディングの
影響を受けた状態の赤外線2次元センサの画素走査を行
って、得られた信号を画素毎にシェーディングメモリに
記憶させ、次いでシャッタを開にして外界からの赤外線
による画像を赤外線2次元センサ上に結像させ、これに
画素走査を行って画素毎の信号をとり出し、この信号か
ら画素毎に、シェーディングメモリに記憶させてあった
対応する画素の信号を読み出して、減算器にて減算する
ことによりシェーディングの影響を除去するというもの
である。
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention uses a combination lens having a built-in shutter which can be easily opened and closed from the outside at an optical pupil position, and closes the shutter to block infrared rays through the lens. Pixel scanning of the infrared two-dimensional sensor under the influence of shading due to stray light is performed, and the obtained signal is stored in a shading memory for each pixel. An image is formed on a dimensional sensor, pixel scanning is performed on the image, a signal for each pixel is taken out, and a signal of a corresponding pixel stored in a shading memory is read from this signal for each pixel, and the signal is sent to a subtractor. In this case, the effect of shading is removed by subtraction.

【0008】このように、シャッタを閉にしてシェーデ
ィングメモリに記憶させるようにすること即ちシェーデ
ィングメモリの更新は、予め定めた一定の周期毎に自動
的に行わせてもよいし、時間の経過に対する温度変化が
予め定めた値を越えたときに行わせるようにしてもよ
い。これはまた、操作者の観測によって行わせてもよい
し、自動的に行わせてもよい。
As described above, the shutter is closed and stored in the shading memory, that is, the updating of the shading memory may be automatically performed at predetermined intervals, or may be performed in response to the passage of time. It may be performed when the temperature change exceeds a predetermined value. This may be performed by the operator's observation or automatically.

【0009】[0009]

【実施例】以下、本発明の赤外線カメラの実施例を図面
を参照して説明する。図1は、本発明の赤外線カメラの
実施例の構成を示すブロック図である。外界からの赤外
線を集光する組合わせレンズ1の光学瞳の位置にシャッ
タ2が設けられており、その開閉はシャッタコントロー
ラ3によって行われるようになっている。シャッタ2が
開の場合、集光された赤外線は赤外線2次元センサ4の
面上に結像する。この赤外線2次元センサからは一定の
走査(例えばラスタスキャン)によって画素毎に赤外線
強度に応じた電気信号が取り出される。取り出された赤
外線画像信号は増幅器5で後の処理に必要なレベルまで
増幅される。増幅された信号はA/D変換器6で、要求
される階調度を満足するビット数のディジタル信号に変
換される。得られたディジタル信号は、これからシェー
ディングによる影響分を減ずるための減算器7の一方入
力端へ入力され、シェーディングによる影響が除去され
た形でフレームメモリ9へ出力される。
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the infrared camera according to the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the configuration of an embodiment of the infrared camera of the present invention. A shutter 2 is provided at a position of an optical pupil of a combination lens 1 for condensing infrared rays from the outside. The shutter 2 is opened and closed by a shutter controller 3. When the shutter 2 is open, the collected infrared light forms an image on the surface of the infrared two-dimensional sensor 4. From the infrared two-dimensional sensor, an electric signal corresponding to the infrared intensity is extracted for each pixel by a fixed scan (for example, raster scan). The extracted infrared image signal is amplified by the amplifier 5 to a level necessary for the subsequent processing. The amplified signal is converted by the A / D converter 6 into a digital signal having a bit number that satisfies the required gradation. The obtained digital signal is input to one input terminal of a subtractor 7 for reducing the influence of shading, and is output to the frame memory 9 in a form in which the influence of shading has been removed.

【0010】一方シェーディングメモリ8が、減算器7
の出力を格納可能なように接続されており、また、その
格納された信号を読み出して減算器7の他方入力端へ入
力できるように接続されている。その作用は、シャッタ
2を閉にしたときのA/D変換器6の出力が1画面分減
算器7を経由してシェーディングメモリ8に格納され
る。このときシェーディングメモリ8からの読み出しは
行われず減算器7の他方入力端へは信号は入力されない
から、減算器7では減算が行われずA/D変換器6から
の入力がそのまま出力されてシェーディングメモリ8に
画素毎に格納される。こうして、格納されたデータはシ
ャッタ2以降のレンズおよび筐体における迷光に起因す
るシェーディング熱画像のデータということになる。
On the other hand, the shading memory 8 includes a subtractor 7
Are connected so that the output of the subtractor 7 can be read out and input to the other input terminal of the subtractor 7. The effect is that the output of the A / D converter 6 when the shutter 2 is closed is stored in the shading memory 8 via the subtracter 7 for one screen. At this time, since the reading from the shading memory 8 is not performed and no signal is input to the other input terminal of the subtractor 7, the subtractor 7 does not perform the subtraction and the input from the A / D converter 6 is output as it is and the shading memory is output. 8 is stored for each pixel. Thus, the stored data is shading thermal image data caused by stray light in the lens and the housing after the shutter 2.

【0011】一方、シャッタ2が開となって外界からの
赤外線を集光したとき赤外線2次元センサ4上に形成さ
れる熱画像は、外界からの赤外線によるものとシェーデ
ィングによるものとが加算された状態のものである。従
って、外界からの赤外線のみによる熱画像を得るには、
赤外線2次元センサ上に形成された熱画像からシェーデ
ィング熱画像分を減ずればよいことになる。
On the other hand, the thermal image formed on the infrared two-dimensional sensor 4 when the shutter 2 is opened and the infrared rays from the outside are condensed is obtained by adding the infrared rays from the outside and the shading. It is in the state. Therefore, to obtain a thermal image using only infrared rays from the outside world,
The shading thermal image should be subtracted from the thermal image formed on the infrared two-dimensional sensor.

【0012】そこで、シャッタ2を閉にしたときに格納
してあったシェーディングメモリ8のデータを読み出し
て減算器7の他方入力端へ入力し、A/D変換器6から
の入力信号から対応する画素毎に減ずることにより、シ
ェーディングの影響を除去した外界からの赤外線のみに
よる熱画像データが得られることになる。
Therefore, the data stored in the shading memory 8 stored when the shutter 2 is closed is read out and input to the other input terminal of the subtracter 7, and the corresponding data is input from the A / D converter 6. By reducing for each pixel, it is possible to obtain thermal image data using only infrared rays from the outside, from which the influence of shading has been removed.

【0013】こうして得られた熱画像データをフレーム
メモリ9へ格納し、後の処理段階へ備えることになる。
なお、シャッタコントローラ3の動作とシェーディング
メモリ8への格納および読出し動作の連動はCPUを利
用して行っている。
The thermal image data obtained in this way is stored in the frame memory 9 to prepare for a later processing stage.
Note that the operation of the shutter controller 3 and the operation of storing and reading the data in and from the shading memory 8 are linked using a CPU.

【0014】[0014]

【発明の効果】以上説明したように、本発明のシェーデ
ィング補正機能付赤外線2次元センサカメラは、赤外線
集光用の組合わせレンズの光学瞳の位置に、シャッタコ
ントローラによる外部からのコントロールによって開閉
が瞬時に容易に行えるシャッタを設け、シャッタを閉に
したときの熱画像データをシェーディングメモリに格納
しておき、シャッタを開にして外界の赤外線像を撮影し
たときに形成された熱画像データから格納しておいたシ
ェーディングデータを画素毎に減ずるようにしたので、
カメラの操作者が手作業で均一平面熱源をセットしたり
或いはレンズにキャップを被せたりする必要がなく、短
時間で、シャッタの後方にあるレンズによるシェーディ
ングの影響も含めてシェーディングの影響を自動的に補
正することができるという利点がある。
As described above, the infrared two-dimensional sensor camera with the shading correction function of the present invention can be opened and closed at the position of the optical pupil of the combination lens for condensing infrared rays by external control by the shutter controller. A shutter that can be instantly and easily provided is provided, thermal image data when the shutter is closed is stored in a shading memory, and thermal image data formed when an infrared image of the outside world is taken with the shutter opened is stored. Since the shading data that was set is reduced for each pixel,
There is no need for the camera operator to manually set a uniform flat heat source or put a cap on the lens, and the effects of shading, including the effects of the lens behind the shutter, are automatically reduced in a short time. There is an advantage that correction can be made to

【図面の簡単な説明】[Brief description of the drawings]

【図1】本発明のシェーディング補正機能付非冷却赤外
線2次元センサカメラの実施例の構成を示すブロック図
である。
FIG. 1 is a block diagram illustrating a configuration of an embodiment of an uncooled infrared two-dimensional sensor camera with a shading correction function according to the present invention.

【符号の説明】[Explanation of symbols]

1 組合わせレンズ 2 シャッタ 3 シャッタコントローラ 4 赤外線2次元センサ 5 増幅器 6 A/D変換器 7 減算器 8 シェーディングメモリ 9 フレームメモリ 10 CPUバス DESCRIPTION OF SYMBOLS 1 Combination lens 2 Shutter 3 Shutter controller 4 Infrared two-dimensional sensor 5 Amplifier 6 A / D converter 7 Subtractor 8 Shading memory 9 Frame memory 10 CPU bus

Claims (1)

【特許請求の範囲】[Claims] 【請求項1】 次の各構成を具備することを特徴とする
シェーディング補正機能付非冷却赤外線2次元センサカ
メラ。 (イ)光学瞳位置にシャッタを内蔵する赤外線像形成用
の組合わせレンズ (ロ)前記シャッタの開閉を制御するシャッタコントロ
ーラ (ハ)前記組合わせレンズによる赤外線像が結像される
2次元センサ (ニ)前記2次元センサに対する画素走査により出力さ
れるアナログ電気信号を画素毎にディジタル信号に変換
するA/D変換器 (ホ)前記A/D変換器の出力信号を一方入力端の入力
信号とし、他方入力端へ入力された信号との間で減算を
行う減算器 (ヘ)前記シャッタを閉じたときのA/D変換器の出力
を1フレーム分格納し、その後シャッタが開となった後
は、その格納された信号が、前記減算器の一方入力端に
入力されている信号の画素位置と一致させて読み出さ
れ、前記減算器の他方入力端へ入力されるシェーディン
グメモリ (ト)前記減算器の出力を1画面単位で格納するフレー
ムメモリ
1. An uncooled infrared two-dimensional sensor camera having a shading correction function, comprising: (A) Combination lens for forming an infrared image having a built-in shutter at the optical pupil position. (B) Shutter controller for controlling opening and closing of the shutter. (C) Two-dimensional sensor on which an infrared image is formed by the combination lens. D) an A / D converter that converts an analog electric signal output by pixel scanning with respect to the two-dimensional sensor into a digital signal for each pixel; (e) an output signal of the A / D converter is used as an input signal at one input terminal A subtractor for performing subtraction with a signal input to the other input terminal. (F) After storing the output of the A / D converter when the shutter is closed for one frame, and then after opening the shutter, Is read out in such a manner that the stored signal matches the pixel position of the signal input to one input terminal of the subtractor, and the shading memo is input to the other input terminal of the subtractor. (G) a frame memory for storing an output of said subtractor with each screen
JP09506097A 1996-10-15 1997-03-28 Uncooled infrared 2D sensor camera with shading correction function Expired - Fee Related JP3373387B2 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP09506097A JP3373387B2 (en) 1997-03-28 1997-03-28 Uncooled infrared 2D sensor camera with shading correction function
US08/949,719 US5994701A (en) 1996-10-15 1997-10-14 Infrared sensor device with temperature correction function
EP97117829A EP0837600A3 (en) 1996-10-15 1997-10-15 Infrared sensor device with temperature correction function

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP09506097A JP3373387B2 (en) 1997-03-28 1997-03-28 Uncooled infrared 2D sensor camera with shading correction function

Publications (2)

Publication Number Publication Date
JPH10274564A true JPH10274564A (en) 1998-10-13
JP3373387B2 JP3373387B2 (en) 2003-02-04

Family

ID=14127496

Family Applications (1)

Application Number Title Priority Date Filing Date
JP09506097A Expired - Fee Related JP3373387B2 (en) 1996-10-15 1997-03-28 Uncooled infrared 2D sensor camera with shading correction function

Country Status (1)

Country Link
JP (1) JP3373387B2 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100345441B1 (en) * 2000-01-14 2002-07-26 국방과학연구소 Nonuniformity correction method and apparatus for thermal imaging system
US6700124B1 (en) 1999-01-14 2004-03-02 Matsushita Electric Industrial Co., Ltd. Infrared imaging device, vehicle having the same installed therein, and infrared image adjustment device
US8442347B2 (en) 2009-12-14 2013-05-14 Sony Corporation Information processing apparatus, information processing method, program, and imaging apparatus including optical microscope

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6700124B1 (en) 1999-01-14 2004-03-02 Matsushita Electric Industrial Co., Ltd. Infrared imaging device, vehicle having the same installed therein, and infrared image adjustment device
US6855934B2 (en) 1999-01-14 2005-02-15 Matsushita Electric Industrial Co., Ltd. Infrared imaging device, vehicle having the same installed therein, and infrared image adjustment device
US7030377B2 (en) 1999-01-14 2006-04-18 Matsushita Electric Industrial Co., Ltd. Infrared imaging device, vehicle having the same installed therein, and infrared image adjustment device
KR100345441B1 (en) * 2000-01-14 2002-07-26 국방과학연구소 Nonuniformity correction method and apparatus for thermal imaging system
US8442347B2 (en) 2009-12-14 2013-05-14 Sony Corporation Information processing apparatus, information processing method, program, and imaging apparatus including optical microscope
US9071761B2 (en) 2009-12-14 2015-06-30 Sony Corporation Information processing apparatus, information processing method, program, and imaging apparatus including optical microscope

Also Published As

Publication number Publication date
JP3373387B2 (en) 2003-02-04

Similar Documents

Publication Publication Date Title
US6940550B2 (en) Digital camera and a method for correcting dust images in the camera
TW574827B (en) Image capturing device and method for removing noise
JPH0851571A (en) Electronic image pickup device
US20090033762A1 (en) Color photographing apparatus
JP2009044367A (en) Imaging method and imaging apparatus
CN100377574C (en) Image processing device and electronic camera
CN103004179A (en) Tracking device, and tracking method
JPH07284011A (en) Digital electronic still camera
JP3373387B2 (en) Uncooled infrared 2D sensor camera with shading correction function
CN101521826B (en) Electronic camera
KR101429512B1 (en) Apparatus and method for reducing banding noise
JP4046244B2 (en) Electronic still camera
JP2001292377A (en) Imaging apparatus
JP4892802B2 (en) Electronic camera
JP5885023B2 (en) Imaging apparatus, white balance control method, and white balance control program
US20040151488A1 (en) Image pickup apparatus, control method therefor, and computer-readable program for implementing the control method
JP3495571B2 (en) Uncooled infrared 2D sensor camera with shading correction function
JPH0951459A (en) Video camera
JP5263764B2 (en) Electronic camera
JP2006140767A (en) Imaging apparatus and method
JPH0440763A (en) Image pickup device
JP2002257627A (en) Method and apparatus for discriminating between different kinds of light sources
US20070070222A1 (en) Method of eliminating horizontal pattern noise
JP2005260573A (en) Electronic camera, camera system and black point correcting method of electronic camera
JP2007174199A (en) Image pickup device, image processing method, program and recording medium

Legal Events

Date Code Title Description
S531 Written request for registration of change of domicile

Free format text: JAPANESE INTERMEDIATE CODE: R313531

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20071122

Year of fee payment: 5

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20081122

Year of fee payment: 6

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20091122

Year of fee payment: 7

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20101122

Year of fee payment: 8

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20101122

Year of fee payment: 8

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20111122

Year of fee payment: 9

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20111122

Year of fee payment: 9

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20121122

Year of fee payment: 10

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20121122

Year of fee payment: 10

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20131122

Year of fee payment: 11

LAPS Cancellation because of no payment of annual fees