JP3262835B2 - Image discriminating apparatus and method - Google Patents

Image discriminating apparatus and method

Info

Publication number
JP3262835B2
JP3262835B2 JP09933592A JP9933592A JP3262835B2 JP 3262835 B2 JP3262835 B2 JP 3262835B2 JP 09933592 A JP09933592 A JP 09933592A JP 9933592 A JP9933592 A JP 9933592A JP 3262835 B2 JP3262835 B2 JP 3262835B2
Authority
JP
Japan
Prior art keywords
image
value
signal
difference signal
primary color
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
JP09933592A
Other languages
Japanese (ja)
Other versions
JPH05300390A (en
Inventor
彰浩 宇佐美
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Canon Inc
Original Assignee
Canon Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Canon Inc filed Critical Canon Inc
Priority to JP09933592A priority Critical patent/JP3262835B2/en
Publication of JPH05300390A publication Critical patent/JPH05300390A/en
Application granted granted Critical
Publication of JP3262835B2 publication Critical patent/JP3262835B2/en
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Landscapes

  • Image Analysis (AREA)
  • Facsimile Image Signal Circuits (AREA)
  • Color Image Communication Systems (AREA)

Description

【発明の詳細な説明】DETAILED DESCRIPTION OF THE INVENTION

【0001】[0001]

【産業上の利用分野】本発明は画像判別装置およびその
方法に関し、例えば、カラー画像読取装置から出力され
る画像信号によって、原稿画像が有彩色か無彩色かを判
定する画像処理に関する。
BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image discriminating apparatus and a method thereof, for example, to an image processing for judging whether a document image is chromatic or achromatic based on an image signal output from a color image reading apparatus.

【0002】[0002]

【従来の技術】従来の画像判別装置においては、原稿が
有彩色か無彩色かの判別は、カラー画像読取装置から出
力されたRGB輝度信号の差信号、あるいは、L**
*などの均等色空間に変換した信号を用いていた。
2. Description of the Related Art In a conventional image discriminating apparatus, whether a document is chromatic or achromatic is determined by a difference signal between RGB luminance signals output from a color image reading apparatus or L * a * b.
A signal converted to a uniform color space such as * was used.

【0003】[0003]

【発明が解決しようとする課題】しかし、上記従来例に
おいては、次のような問題点があつた。すなわち、従来
の画像判別装置においては、RGB輝度信号を使つてい
るので、人間の視覚特性に合わなくなり、暗い色におい
ては、判別ができないという欠点があつた。
However, the above-described conventional example has the following problems. That is, in the conventional image discriminating apparatus, since the RGB luminance signal is used, it does not conform to human visual characteristics, and there is a disadvantage that dark colors cannot be discriminated.

【0004】また、従来の画像判別装置において、均等
色空間に変換するためには、RGB信号を一旦XYZ信
号に変換した後、L***信号に変換するために回路
が複雑化し、装置コストが上昇する欠点もあつた。
Further, in the conventional image discriminating apparatus, in order to convert to a uniform color space, a circuit is complicated to convert an RGB signal to an XYZ signal and then to an L * a * b * signal. There was also a drawback that the equipment cost increased.

【0005】本発明は、上述の問題を解決するためのも
のであり、画像の有彩色/無彩色の判定を低コスト、高
精度、かつ人間の視覚特性に応じて行うことを目的とす
る。
An object of the present invention is to solve the above-mentioned problem, and it is an object of the present invention to determine a chromatic / achromatic color of an image at low cost, with high accuracy, and in accordance with human visual characteristics.

【0006】[0006]

【課題を解決するための手段】本発明は、前記の目的を
達成する一手段として、以下の構成を備える。本発明に
かかる画像判別装置は、画像信号の各原色色成分を視覚
特性に応じた値へ変換するために、画像暗部の原色色成
分のレベル差を拡大する特性を有する関数に従い、前記
画像信号を非線型変換する変換手段と、前記各原色色成
分の変換結果の最大値および最小値を検出して、検出さ
れる最大値と最小値との差信号を出力する差信号出力手
段と、前記差信号の値と、予め設定された閾値との関係
が、前記閾値よりも前記差信号の値が大きい場合は前記
画像信号が表す画像が有彩色であると判定し、前記差信
号の値が前記閾値以下の場合は前記画像信号が表す画像
が無彩色であると判定する判定手段とを有することを特
徴とする。本発明にかかる画像判別方法は、画像信号の
各原色色成分を視覚特性に応じた値へ変換するために、
画像暗部の原色色成分のレベル差を拡大する特性を有す
る関数に従い、前記画像信号を非線型変換し、前記各原
色色成分の変換結果の最大値および最小値を検出し、検
出される最大値と最小値との差信号を出力し、前記差信
号の値と、予め設定された閾値との関係が、前記閾値よ
りも前記差信号の値が大きい場合は前記画像信号が表す
画像が有彩色であると判定し、前記差信号の値が前記閾
値以下の場合は前記画像信号が表す画像が無彩色である
と判定することを特徴とする。
The present invention has the following arrangement as one means for achieving the above object. The image discriminating apparatus according to the present invention is configured to convert each primary color component of an image signal into a value corresponding to a visual characteristic, in accordance with a function having a characteristic of enlarging a level difference between primary color components of an image dark portion, the image signal Conversion means for non-linearly converting, a difference signal output means for detecting a maximum value and a minimum value of the conversion result of each primary color component, and outputting a difference signal between the detected maximum value and the minimum value, The relationship between the value of the difference signal and a preset threshold value, when the value of the difference signal is greater than the threshold value, determines that the image represented by the image signal is a chromatic color, and the value of the difference signal is A determination unit that determines that the image represented by the image signal is achromatic when the value is equal to or less than the threshold value. The image determination method according to the present invention is to convert each primary color component of the image signal into a value according to the visual characteristic,
According to a function having a characteristic of enlarging a level difference between primary color components of an image dark portion, the image signal is nonlinearly converted, and a maximum value and a minimum value of a conversion result of each primary color component are detected, and a detected maximum value And a difference signal between the difference signal and the minimum value. If the value of the difference signal is greater than the threshold value, the image represented by the image signal is chromatic. Is determined, and when the value of the difference signal is equal to or smaller than the threshold, it is determined that the image represented by the image signal is achromatic.

【0007】[0007]

【実施例】以下、図面を参照して本発明に係る一実施例
を詳細に説明する。
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment according to the present invention will be described below in detail with reference to the drawings.

【0008】[0008]

【第1実施例】図1は本発明に係る第1実施例の画像判
別装置の構成例を示すブロツク図である。図1におい
て、1〜3はそれぞれルツクアツプテーブルLUTで、
入力されたRGB輝度信号に、それぞれ非線型な処理を
施す。なお、LUT1〜3は、ROMなどで構成されて
いて、例えば8ビツトのR信号を(1)式で処理した
R’信号を出力する。G信号,B信号も略同様に処理さ
れてG’信号,B’信号となる。
FIG. 1 is a block diagram showing a configuration example of an image discriminating apparatus according to a first embodiment of the present invention. In FIG. 1, reference numerals 1 to 3 denote lookup table LUTs, respectively.
Non-linear processing is performed on the input RGB luminance signals. Each of the LUTs 1 to 3 is composed of a ROM or the like, and outputs an R ′ signal obtained by processing, for example, an 8-bit R signal according to equation (1). The G signal and the B signal are processed in substantially the same manner to become the G ′ signal and the B ′ signal.

【0009】 R’=255×(R/255)r ・・・(1) ただし、r=0.1〜0.9の定数 4は差信号生成回路で、LUT1〜3から入力された
R’G’B’信号の最大値MAXと最小値MINを検出
して、その差信号S=MAX−MINを出力する。
R ′ = 255 × (R / 255) r (1) where r = 0.1 to 0.9, a constant 4 is a difference signal generation circuit, and R ′ input from LUTs 1 to 3 A maximum value MAX and a minimum value MIN of the G'B 'signal are detected, and a difference signal S = MAX-MIN is output.

【0010】5は判定回路で、予め決められた閾値S0
と、差信号生成回路4から入力された差信号Sとを比較
して、有彩色/無彩色の判定結果を出力する。すなわ
ち、判定回路5は、S>S0の場合は有彩色を表す
‘1’を、S≦S0の場合は無彩色を表す‘0’を出力
する。なお、本実施例のLUT1〜3の変換テーブル
は、(1)式に限定されるものではなく、例えば、
(2)式に示すような対数変換テーブルを用いることも
できる。
Reference numeral 5 denotes a judgment circuit, which is a predetermined threshold value S0
And a difference signal S input from the difference signal generation circuit 4 to output a chromatic / achromatic determination result. That is, the determination circuit 5 outputs “1” representing a chromatic color when S> S0, and outputs “0” representing an achromatic color when S ≦ S0. Note that the conversion tables of the LUTs 1 to 3 of the present embodiment are not limited to the expression (1).
A logarithmic conversion table as shown in equation (2) can be used.

【0011】 R’=(255/Δ)log(R/255) ・・・(2) ただし、Δ=1〜5の定数 また、本実施例の差信号生成回路4の出力は、差信号S
=MAX−MINに限定されるものではなく、例えば、
LUT1〜3から入力されたR’G’B’信号から、色
差信号R’−G’とG’−B’を生成した後、(3)式
または(4)式に従つて信号Sを出力してもよい。
R ′ = (255 / Δ) log (R / 255) (2) where Δ = 1 to 5 is a constant. The output of the difference signal generation circuit 4 of the present embodiment is the difference signal S
= MAX-MIN is not limited to, for example,
After generating the color difference signals R'-G 'and G'-B' from the R'G'B 'signals input from the LUTs 1 to 3, the signal S is output according to the equation (3) or (4). May be.

【0012】 S=|R’−G’|+|G’−B’| ・・・(3) S=√{(R’−G’)2+(G’−B’)2} ・・・(4) 以上説明したように、本実施例によれば、RGB信号を
各々非線型処理した後に差信号などとして、差信号など
の大きさによつて有彩色/無彩色の判別をするので、低
コストで判別精度が高い有彩色/無彩色の判別ができ、
とくに、濃い青色の判別に顕著な効果がある。
S = | R′−G ′ | + | G′−B ′ | (3) S = {{(R′−G ′) 2 + (G′−B ′) 2 }. (4) As described above, according to the present embodiment, the chromatic / achromatic color is determined according to the magnitude of the difference signal or the like as a difference signal after each of the RGB signals is nonlinearly processed. , High-precision chromatic / achromatic color discrimination at low cost
In particular, there is a remarkable effect on the determination of dark blue.

【0013】さらに、本実施例によれば、原稿に黒文字
などのエッジ部があると、R,G,Bの読取部の画素ずれな
どの影響で色のにじみが出る場合などに、非線型処理が
効果を示す。例えば、図2(a)に示すような画像信号は、
図2(b)に示すような画像信号に変換され、色のにじみを
抑えることができ、かつ色のにじみによる誤判別を防ぐ
働きもする。
Further, according to the present embodiment, if the original has an edge portion such as a black character, the color processing becomes blurred due to the pixel shift of the R, G, B reading section. Indicates an effect. For example, an image signal as shown in FIG.
The image signal is converted into an image signal as shown in FIG. 2 (b), which can suppress color bleeding and also functions to prevent erroneous determination due to color bleeding.

【0014】[0014]

【第2実施例】図3は本発明に係る第2実施例の色判別
装置の構成例を示すブロツク図である。図3において、
第1実施例と略同様の構成については、同一符号を付し
て詳細説明を省略する。
Second Embodiment FIG. 3 is a block diagram showing a configuration example of a color discriminating apparatus according to a second embodiment of the present invention. In FIG.
About the structure substantially the same as that of 1st Example, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

【0015】図3において、6〜8はそれぞれルツクア
ツプテーブルLUTで、入力されたRGB輝度信号に、
それぞれ非線型な処理を施す。なお、LUT6〜8は、
RAMなどで構成されていて、例えば8ビツトのR信号
を(1)式または(2)式で処理したR’信号を出力す
る。G信号,B信号も略同様に処理されてG’信号,
B’信号となる。
In FIG. 3, reference numerals 6 to 8 denote lookup tables LUT, respectively, for input RGB luminance signals.
Each is subjected to non-linear processing. The LUTs 6 to 8 are
It is composed of a RAM or the like, and outputs an R 'signal obtained by processing, for example, an 8-bit R signal by the equation (1) or (2). The G signal and the B signal are processed in substantially the same manner, and the G ′ signal,
B ′ signal.

【0016】9はCPU、10はROMで、CPU9はROM10に記憶さ
れた変換テーブルを、LUT6〜8へ記憶させるなどの処理
を実行する。また、CPU9は、ROM10に記憶されたデータ
によって、LUT6〜8の変換テーブルを計算し、計算結果
をLUT6〜8へ記憶させることもできる。以上説明したよ
うに、本実施例によれば、第1実施例と略同様の効果が
あるほか、LUT6〜8の変換テーブルを任意に設定できる
ので、判別しようとする画像の特性に合わせた非線型処
理として、より判別精度が高い有彩色/無彩色の判別が
できる。
Reference numeral 9 denotes a CPU, and reference numeral 10 denotes a ROM. The CPU 9 executes processing such as storing the conversion table stored in the ROM 10 in the LUTs 6 to 8. Further, the CPU 9 can calculate the conversion tables of the LUTs 6 to 8 based on the data stored in the ROM 10 and store the calculation results in the LUTs 6 to 8. As described above, according to the present embodiment, in addition to substantially the same effects as those of the first embodiment, since the conversion tables of the LUTs 6 to 8 can be set arbitrarily, a non-optimizing method suitable for the characteristics of the image to be determined is used. As the linear processing, a chromatic color / achromatic color can be distinguished with higher distinction accuracy.

【0017】[0017]

【第3実施例】図4は本発明に係る第3実施例の色判別
装置の構成例を示すブロツク図である。図4において、
第1実施例および第2実施例と略同様の構成について
は、同一符号を付して詳細説明を省略する。
Third Embodiment FIG. 4 is a block diagram showing a configuration example of a color discriminating apparatus according to a third embodiment of the present invention. In FIG.
About the structure substantially the same as 1st Example and 2nd Example, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

【0018】図4において、11はn入力1出力のセレクタ
で、LUT2が出力したG'信号に応じて、入力された閾値S0
〜Snの1つを選択して出力する。セレクタ11は、例え
ば、G'i≦G'<G'i+1の場合、閾値Siを選択して出力す
る。セレクタ11から出力された閾値Siは、判定回路5へ
入力される。すなわち、判定回路5は、S>Siの場合は有
彩色を表す‘1’を、S≦Siの場合は無彩色を表す‘0’
を出力する。
In FIG. 4, reference numeral 11 denotes an n-input / 1-output selector, which receives an input threshold value S0 according to the G ′ signal output from the LUT2.
Select and output one of ~ Sn. For example, when G′i ≦ G ′ <G′i + 1, the selector 11 selects and outputs the threshold value Si. The threshold value Si output from the selector 11 is input to the determination circuit 5. That is, the determination circuit 5 outputs “1” representing a chromatic color when S> Si, and “0” representing an achromatic color when S ≦ Si.
Is output.

【0019】なお、前記の説明および図4においては、
選択信号としてG’信号を用いる例を説明したが、本実
施例はこれに限定されるものではなく、例えば、R’信
号やB’信号を選択信号とすることもできるし、(R’
+G’+B’)/3を選択信号とすることもできる。以
上説明したように、本実施例によれば、第1実施例と略
同様の効果があるほか、判別しようとする画素の明度に
合わせた閾値とすることができるので、より判別精度が
高い有彩色/無彩色の判別ができる。
In the above description and FIG.
Although an example in which the G 'signal is used as the selection signal has been described, the present embodiment is not limited to this. For example, an R' signal or a B 'signal can be used as the selection signal, or (R'
+ G '+ B') / 3 can be used as the selection signal. As described above, according to the present embodiment, in addition to the effects substantially similar to those of the first embodiment, since the threshold value can be set in accordance with the brightness of the pixel to be determined, the determination accuracy is higher. Colored / achromatic colors can be distinguished.

【0020】また、図3に示した構成例にセレクタ11
を追加しても、略同様の効果が期待できる。なお、本発
明は、複数の機器から構成されるシステムに適用して
も、1つの機器からなる装置に適用してもよい。また、
本発明は、システムあるいは装置にプログラムを供給す
ることによつて達成される場合にも適用できることはい
うまでもない。
The selector 11 shown in FIG.
A similar effect can be expected by adding. The present invention may be applied to a system including a plurality of devices or to an apparatus including a single device. Also,
It is needless to say that the present invention can be applied to a case where the present invention is achieved by supplying a program to a system or an apparatus.

【0021】[0021]

【発明の効果】以上説明したように、本発明によれば、
画像の有彩色/無彩色の判定を低コスト、高精度、かつ
人間の視覚特性に応じて行うことができる。
As described above, according to the present invention,
The chromatic / achromatic color of an image can be determined at low cost, with high accuracy, and in accordance with human visual characteristics.

【図面の簡単な説明】[Brief description of the drawings]

【図1】本発明に係る一実施例の画像判別装置の構成例
を示すブロツク図である。
FIG. 1 is a block diagram showing a configuration example of an image discriminating apparatus according to one embodiment of the present invention.

【図2】本実施例による効果の一例を説明する図であ
る。
FIG. 2 is a diagram illustrating an example of an effect according to the embodiment.

【図3】本発明に係る第2実施例の色判別装置の構成例
を示すブロツク図である。
FIG. 3 is a block diagram illustrating a configuration example of a color discriminating apparatus according to a second embodiment of the present invention.

【図4】本発明に係る第3実施例の色判別装置の構成例
を示すブロツク図である。
FIG. 4 is a block diagram showing a configuration example of a color discriminating apparatus according to a third embodiment of the present invention.

【符号の説明】[Explanation of symbols]

1〜3 ルツクアツプテーブルLUT 4 差信号生成回路 5 判定回路 6〜8 ルツクアツプテーブルLUT 9 CPU 10 ROM 11 セレクタ 1-3 Lookup table LUT 4 Difference signal generation circuit 5 Judgment circuit 6-8 Lookup table LUT 9 CPU 10 ROM 11 Selector

Claims (2)

(57)【特許請求の範囲】(57) [Claims] 【請求項1】 画像信号の各原色色成分を視覚特性に応
じた値へ変換するために、画像暗部の原色色成分のレベ
ル差を拡大する特性を有する関数に従い、前記画像信号
を非線型変換する変換手段と、 前記各原色色成分の変換結果の最大値および最小値を検
出して、検出される最大値と最小値との差信号を出力す
る差信号出力手段と、 前記差信号の値と、予め設定された閾値との関係が、前
記閾値よりも前記差信号の値が大きい場合は前記画像信
号が表す画像が有彩色であると判定し、前記差信号の値
前記閾値以下の場合は前記画像信号が表す画像が無彩
色であると判定する判定手段とを有することを特徴とす
る画像判別装置。
1. A method for converting each primary color component of an image signal into a value corresponding to a visual characteristic, according to a function having a characteristic of enlarging a level difference between primary color components of an image dark portion. Converting means for detecting the maximum value and the minimum value of the conversion result of each primary color component, and outputting a difference signal between the detected maximum value and the minimum value; and a value of the difference signal And, the relationship between the preset threshold, if the value of the difference signal is greater than the threshold, it is determined that the image represented by the image signal is a chromatic color, the value of the difference signal is equal to or less than the threshold A determination unit that determines that the image represented by the image signal is an achromatic color.
【請求項2】 画像信号の各原色色成分を視覚特性に応
じた値へ変換するために、画像暗部の原色色成分のレベ
ル差を拡大する特性を有する関数に従い、前記画像信号
を非線型変換し、 前記各原色色成分の変換結果の最大値および最小値を検
出し、 検出される最大値と最小値との差信号を出力し、 前記差信号の値と、予め設定された閾値との関係が、前
記閾値よりも前記差信号の値が大きい場合は前記画像信
号が表す画像が有彩色であると判定し、前記差信号の値
前記閾値以下の場合は前記画像信号が表す画像が無彩
色であると判定することを特徴とする画像判別方法。
2. A method for converting each primary color component of an image signal into a value corresponding to a visual characteristic according to a function having a characteristic of enlarging a level difference between primary color components of an image dark portion. Detecting a maximum value and a minimum value of the conversion result of each primary color component, outputting a difference signal between the detected maximum value and the minimum value, and comparing the difference signal value with a preset threshold value. When the value of the difference signal is larger than the threshold, the image represented by the image signal is determined to be chromatic, and when the value of the difference signal is equal to or less than the threshold, the image represented by the image signal is An image discriminating method characterized by judging that an image is achromatic.
JP09933592A 1992-04-20 1992-04-20 Image discriminating apparatus and method Expired - Fee Related JP3262835B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP09933592A JP3262835B2 (en) 1992-04-20 1992-04-20 Image discriminating apparatus and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP09933592A JP3262835B2 (en) 1992-04-20 1992-04-20 Image discriminating apparatus and method

Publications (2)

Publication Number Publication Date
JPH05300390A JPH05300390A (en) 1993-11-12
JP3262835B2 true JP3262835B2 (en) 2002-03-04

Family

ID=14244763

Family Applications (1)

Application Number Title Priority Date Filing Date
JP09933592A Expired - Fee Related JP3262835B2 (en) 1992-04-20 1992-04-20 Image discriminating apparatus and method

Country Status (1)

Country Link
JP (1) JP3262835B2 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7995242B2 (en) 2007-05-31 2011-08-09 Sharp Laboratories Of America, Inc. Systems and methods for attenuation of near-neutral image colors
KR101783259B1 (en) * 2010-12-31 2017-10-10 삼성디스플레이 주식회사 Method for compensating data, compensating apparatus for performing the method and display device having the compensating apparatus

Also Published As

Publication number Publication date
JPH05300390A (en) 1993-11-12

Similar Documents

Publication Publication Date Title
JP3625074B2 (en) Color conversion system and color conversion method
JP3384580B2 (en) Background removal method
JP2756280B2 (en) Color image processing equipment
US20080253650A1 (en) Image Processing Device, Image Processing Method, Image Processing Program Product, and Image-Capturing Device
KR100911815B1 (en) Picture signal contrast control
US4975861A (en) Color conversion image processing system with modified intensity information calculation
JPH07170420A (en) Color image processing system
AU624760B2 (en) Gradation correcting apparatus
JPH0832827A (en) Gradation correction divice for digital image
JP3255258B2 (en) Image display device
JP2003230051A (en) Electronic camera
JP3262835B2 (en) Image discriminating apparatus and method
US5267031A (en) Color image processing apparatus
JPH07152904A (en) Image processor
JP2848558B2 (en) Color image processing equipment
JP3080019B2 (en) Video signal processing device
JP3482083B2 (en) Color image brightness converter
JP3105689B2 (en) Image processing apparatus and method
JPH06292006A (en) Picture processing unit
JP2954230B2 (en) Image processing device
JP2001251526A (en) Picture processor
JP2001352456A (en) Image-processing method and image-processing unit, and image-forming device provided with them
JPS62165687A (en) Cursor display system
JP2863947B2 (en) Color image processing method
JP3732315B2 (en) Image processing device

Legal Events

Date Code Title Description
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20011126

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20081221

Year of fee payment: 7

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20081221

Year of fee payment: 7

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20091221

Year of fee payment: 8

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20091221

Year of fee payment: 8

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20101221

Year of fee payment: 9

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20111221

Year of fee payment: 10

LAPS Cancellation because of no payment of annual fees