JP3791592B2 - Image processing apparatus, image processing method, program, and recording medium - Google Patents

Image processing apparatus, image processing method, program, and recording medium Download PDF

Info

Publication number
JP3791592B2
JP3791592B2 JP2001134450A JP2001134450A JP3791592B2 JP 3791592 B2 JP3791592 B2 JP 3791592B2 JP 2001134450 A JP2001134450 A JP 2001134450A JP 2001134450 A JP2001134450 A JP 2001134450A JP 3791592 B2 JP3791592 B2 JP 3791592B2
Authority
JP
Japan
Prior art keywords
hue
conversion
image processing
rgb
processing apparatus
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
JP2001134450A
Other languages
Japanese (ja)
Other versions
JP2002330298A (en
Inventor
政史 金井
Original Assignee
セイコーエプソン株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by セイコーエプソン株式会社 filed Critical セイコーエプソン株式会社
Priority to JP2001134450A priority Critical patent/JP3791592B2/en
Publication of JP2002330298A publication Critical patent/JP2002330298A/en
Application granted granted Critical
Publication of JP3791592B2 publication Critical patent/JP3791592B2/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to image processing for printers, projectors, and the like, in particular, color conversion processing of image data expressed in three colors of R (red), G (green), and B (blue) according to the use device, user preference, and the like The present invention relates to an image processing apparatus, an image processing method, a program, and a recording medium.
[0002]
[Prior art]
Color conversion in the printing process is an essential technique for correcting image quality deterioration and maintaining good color reproducibility. Also, in a display device such as a monitor / projector, color conversion processing is performed in order to perform desired color reproduction according to usage conditions.
[0003]
[Problems to be solved by the invention]
However, in the case of matrix conversion color conversion, suitable color conversion cannot be realized due to the non-linearity of the print. In addition, since the hue conversion amount between the six hues (reference hues) of red, blue, green, yellow (Y), cyan (C), and magenta (M) cannot be independently corrected, good color conversion is possible in the entire color space. Cannot be realized.
[0004]
The present invention has been made to solve the above-described problems. An image processing apparatus, an image processing method, a program, and a recording medium capable of independently correcting a hue conversion amount in an area between reference hues for each reference hue. The issue is to provide.
[0005]
[Means for Solving the Problems]
In view of the above problems, the invention according to claim 1 is an image processing apparatus that performs desired image processing on RGB image input data in an RGB color space, and the pre-conversion color point indicated by the RGB image input data A hue determination unit that determines a hue of the color conversion unit, and a hue conversion unit that performs a hue conversion of the color point before conversion to a color point on a straight line parallel to a side of the RGB color space according to a determination result of the hue determination unit. And the hue conversion unit is configured to perform hue conversion of the RGB image input data.
[0006]
According to the image processing apparatus configured to perform the desired image processing on the RGB image input data in the RGB color space configured as described above, the hue determination unit determines the color point before conversion indicated by the RGB image input data. The hue is determined, and the hue conversion unit converts the hue point before conversion into a color point on a straight line parallel to the side of the RGB color space according to the determination result of the hue determination unit. Then, hue conversion of the RGB image input data is performed by the hue conversion means.
[0007]
The invention according to claim 2 is the image processing apparatus according to claim 1, wherein the hue determination unit includes a plurality of pre-conversion color points indicated by the RGB image input data in the RGB color space. It is configured to determine which of the divided hue regions belongs, and the hue conversion unit performs the hue conversion according to the determination result.
[0008]
Furthermore, the invention described in claim 3 is the image processing apparatus according to claim 2, wherein the hue determination unit is configured such that the pre-conversion color point indicated by the RGB image input data is an RY hue region, Y -G hue region, GC hue region, CB hue region, BM hue region, or MR hue region is configured to be determined.
[0009]
According to a fourth aspect of the present invention, there is provided the image processing apparatus according to any one of the first to third aspects, wherein the hue conversion unit inputs a hue conversion parameter that defines a hue conversion amount. It is comprised so that an input means may be provided.
[0010]
Further, the invention described in claim 5 is the image processing apparatus according to claim 4, wherein the hue conversion parameter can be set independently for each hue of RGBCMY.
[0011]
The invention according to claim 6 is the image processing apparatus according to claim 4 or 5, wherein the RGB image input data is based on a determination result by the hue determination unit and an input hue conversion parameter. A hue conversion amount calculating means for calculating the hue conversion amount of
[0012]
Furthermore, the invention according to claim 7 is the image processing apparatus according to any one of claims 1 to 6, wherein a set of color points having the same hue is converted into a set of color points having the same hue even after hue conversion. As described above, a desired weight is applied to the hue conversion amount by the hue conversion means, and the hue conversion of the RGB image input data is performed based on the weighted hue conversion amount.
[0013]
The invention according to claim 8 is the image processing apparatus according to claim 7, wherein the weighting is decreased as the pre-conversion color point is closer to the gray axis.
[0014]
Furthermore, the invention according to claim 9 is the image processing apparatus according to any one of claims 1 to 8, wherein the color point is converted into an adjacent hue region as a result of the hue conversion by the hue conversion means. In such a case, the direction of hue conversion is changed at the adjacent boundary.
[0015]
According to a tenth aspect of the present invention, there is provided an image processing method for performing desired image processing on RGB image input data in an RGB color space, wherein the hue of a pre-conversion color point indicated by the RGB image input data is determined. A hue determination step for determining, and a hue conversion step for hue-converting the pre-conversion color point to a color point on a straight line parallel to a side of the RGB color space according to the determination result of the hue determination step, It is configured to perform hue conversion of the RGB image input data by a hue conversion process.
[0016]
The invention according to claim 11 is a program for causing a computer to execute desired image processing performed on RGB image input data in the RGB color space, the conversion indicated by the RGB image input data. Hue determination processing for determining the hue of the previous color point, and hue conversion processing for hue-converting the pre-conversion color point to a color point on a straight line parallel to the side of the RGB color space according to the determination result of the hue determination processing And is configured to cause the computer to execute image processing for performing hue conversion of the RGB image input data by the hue conversion processing.
[0017]
A twelfth aspect of the present invention is a computer-readable recording medium on which the program according to the eleventh aspect is recorded.
[0018]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. In this embodiment, image processing in the projector will be described.
[0019]
FIG. 1 is a functional block diagram of an image processing unit 100 in a projector according to an embodiment of the present invention.
[0020]
An image processing unit 100 in a projector according to an embodiment of the present invention includes an A / D conversion unit 110 that converts an analog image input signal into a digital signal, and a hue determination unit that determines a hue based on RGB image input data. 115, a hue conversion unit 120 that performs a desired hue conversion on an RGB image input signal, a D / A conversion unit 130 for converting a digital signal into an analog signal, and a liquid crystal light valve to drive an image projection And an L / V (light valve) driving unit 140 for performing display.
[0021]
Further, the hue conversion unit 120 is configured to input the hue conversion parameter for each hue, the hue image determined by the hue determination unit 115 and the input hue conversion parameter based on the RGB image. And a conversion amount calculation unit 120b for calculating the hue conversion amount of the input data.
[0022]
FIG. 2 shows six hue regions for explaining the processing by the hue conversion unit 120. All conversions by the hue conversion unit 120 are performed in the RGB space.
[0023]
The hue is divided into six areas as shown in FIG. 2, and hue conversion is performed for each area. That is, the hue determination unit 115 determines the hue based on the RGB image input data, and performs hue conversion for each region based on the determination result.
[0024]
During this hue conversion, the parameter input unit 120a inputs a hue conversion parameter that defines the hue conversion amount for each of the hues R, G, B, C, M, and Y. The range of the input hue conversion parameter is from −128 to 127. In each area of FIG. 2, the hue conversion parameters for two colors of RGBCMY are related to the calculation of the hue conversion amount. For example, in region 1 in FIG. 2, the hue conversion parameters of R and Y are related to the calculation of the hue conversion amount. In other words, each RGBCMY hue conversion amount affects two adjacent hue regions. For example, if R, region 1 and region 6 are affected.
[0025]
As shown in FIG. 3, for example, in the case of R, weighting is performed so that each RGBCMY conversion amount becomes maximum for each primary color (the vertex of the RGB space) and decreases as it approaches the adjacent hue or gray axis. By performing such weighting, the continuity of the RGB space is maintained even after the hue conversion. The weighting process will be described in detail below.
[0026]
Operations of Hue Determination Unit 115 and Hue Conversion Unit 120 Next, operations of the hue determination unit 115 and the hue conversion unit 120 in the image processing unit 100 of the projector according to an embodiment of the present invention will be described with reference to FIG. To do.
[0027]
The hue conversion processing by the hue determination unit 115 and the hue conversion unit 120 described below is performed by executing an image processing program recorded in a program storage unit (not shown) of the projector. The program storage unit constitutes a medium on which an image processing program is recorded. Further, the image processing program itself is also included within the scope of the present invention.
[0028]
First, when the use of the projector is started, the hue determination unit 115 obtains a magnitude relationship between the R, G, and B values of the RGB image input data (S10), and the RGB image input data is displayed in the hue regions 1 to 1 shown in FIG. 6 or a gray region is determined. That is, the hue determination unit 115
(1) When R ≧ G> B, it is determined that the region is the RY region (hue region 1 in FIG. 2), and hue conversion processing is performed in the RY region (S12).
(2) When G> R ≧ B, it is determined that the region is the YG region (hue region 2 in FIG. 2), and a hue conversion process is performed in the YG region (S14).
(3) When G ≧ B> R, it is determined that the region is the GC region (hue region 3 in FIG. 2), and hue conversion processing is performed in the GC region (S16).
(4) When B> G ≧ R, it is determined that the region is the CB region (hue region 4 in FIG. 2), and a hue conversion process is performed in the CB region (S18).
(5) When B ≧ R> G, it is determined that the region is the BM region (hue region 5 in FIG. 2), and a hue conversion process is performed in the BM region (S20).
(6) When R> B ≧ G, it is determined that the region is the MR region (hue region 6 in FIG. 2), and hue conversion processing is performed in the MR region (S22).
(7) When R = G = B, it is determined that the region is a gray region (non-conversion region), and hue conversion processing is not performed (S24).
[0029]
Next, processing performed by the hue conversion unit 120 in each hue region will be described with reference to FIG. In the present embodiment, a hue conversion process in the case of R ≧ G> B, that is, the RY area (hue area 1 in FIG. 2) will be described. The hue conversion process is the same as the hue area 1 for the other hue areas 2 to 6 shown in FIG.
[0030]
As shown in FIG. 5, first, a parameter that defines a hue conversion amount (referred to as “hue conversion parameter” in the specification) is input from the parameter input unit 120a to the conversion amount calculation unit 120b (S30). . Hue conversion parameters are
Rh: R hue conversion parameter (−128 ≦ Rh ≦ 127)
Gh: G hue conversion parameter (−128 ≦ Gh ≦ 127)
Bh: B hue conversion parameter (−128 ≦ Bh ≦ 127)
Ch: C hue conversion parameter (−128 ≦ Ch ≦ 127)
Mh: M hue conversion parameter (−128 ≦ Mh ≦ 127)
Yh: Y hue conversion parameter (−128 ≦ Yh ≦ 127)
Is defined.
[0031]
When the hue conversion parameter is positive, the hue is rotated in the direction of R → Y → G → C → B → M → R.
[0032]
Next, a hue conversion amount calculation process is performed by the conversion amount calculation unit 120b (S32).
[0033]
The color of each vertex of RGBCMY moves along the side of R-Y-G-C-B-M-R by the hue conversion amount. For the color points other than the vertices, the hue conversion amount is weighted. That is, the hue point of the pre-conversion color point is converted to a color point on a straight line parallel to the side of the RGB color space. As an example of the weighting process, Rh is weighted (R−G) / 255 in the RY region and (R−B) / 255 in the MR region. When such weighting is performed, the hue conversion amount becomes smaller as the color point before saturation conversion is closer to the gray axis. Therefore, as shown in FIG. 6, a set of color points having the same hue before conversion is converted. After that, it becomes a set of color points of the same hue.
[0034]
When moving in parallel with the side of R-Y-G-C-B-M-R, any one component of RGB may be increased or decreased. In the case shown in FIG. 6, it is only necessary to increase the value of G while keeping the values of R and B as they are. That is, when the hue conversion amount is (ΔR, ΔG, ΔB),
ΔR = 0
ΔG = {(RG) / 255} × Rh
ΔB = 0
It becomes.
[0035]
The calculation of the hue conversion amount is affected by the hue parameters of the two hue areas in each area. For example, the hue conversion amount in the RY area is obtained by superimposing the hue conversion amounts by Rh and Yh.
(1) As a result of hue conversion, when the color point does not move from the RY area to the adjacent area (YG area or MR area) (in the case of FIG. 7A: BG <H < RG)
If the hue conversion amount in the RY region is expressed by an equation,
ΔR = 0
ΔG = {(RG) / 255} × Rh + {(GB) / 255} × Yh
ΔB = 0
It becomes.
[0036]
In the following description:
ΔG = {(RG) / 255} × Rh + {(GB) / 255} × Yh = H
And
[0037]
However, as shown in FIG. 7B or 7C, when the color point moves from the RY region to the adjacent region (YG region or MR region) as a result of the hue conversion, conversion is performed. The direction is changed at the boundary with the adjacent region.
(2) When color points are converted from the RY area to the adjacent YG area (in the case of FIG. 7B: RG <H)
When the hue conversion amount is expressed by an equation, ΔR = − {H− (R−G)}
ΔG = R−G
ΔB = 0
It becomes.
(3) When a color point is converted from an RY area to an adjacent MR area (in the case of (C) in FIG. 7: H <BG <0)
If the hue conversion amount is expressed by an equation, ΔR = 0
ΔG =-(GB)
ΔB = (− H) − (GB)
It becomes.
[0038]
The above cases (1) to (3) can be summarized as one expression. Hue conversion from the point P (R, G, B) to the point P ′ (R ′, G ′, B ′) in the RY region is performed. When the hue conversion amount is ΔR, ΔG, ΔB,
R ′ = R + ΔR
G ′ = G + ΔG
B ′ = B + ΔB
And
ΔR = min [R−G−H, 0]
ΔG = mid [BR, H, RG]
ΔB = max [B−G−H, 0]
H = {(R−G) / 255} × Rh + {(GB) / 255} × Yh
It becomes. Here, min [R−G−H, 0] means that R−G−H or the minimum value of 0 is taken, and mid [B−R, H, R−G] is B− This means taking the median value among R, H, and R-G, and max [B-G-H, 0] means taking the maximum value of B-G-H or 0.
[0039]
Similar hue conversion processing is performed in other hue regions.
[0040]
As described above, according to the image processing apparatus according to the present embodiment, the hue conversion amount in the region between the reference hues can be set independently for each reference hue and so as not to lose continuity with the adjacent region. Therefore, good color conversion can be realized in the entire color space.
[Brief description of the drawings]
FIG. 1 is a functional block diagram of an image processing unit 100 in a projector according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating six hue regions for explaining processing by the hue conversion unit 120;
FIG. 3 is a diagram for explaining weighting processing by a conversion amount calculation unit 120b.
FIG. 4 is a flowchart for explaining operations of a hue determination unit 115 and a hue conversion unit 120 according to an embodiment of the present invention.
FIG. 5 is a flowchart for explaining the operation of the hue conversion unit 120 in each hue region;
FIG. 6 is a diagram (1) illustrating a hue conversion amount calculation process performed by a conversion amount calculation unit 120b.
FIG. 7 is a diagram (2) illustrating a hue conversion amount calculation process performed by the conversion amount calculation unit 120b.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 100 Image processing part 110 A / D conversion part 115 Hue determination part 120 Hue conversion part 120a Parameter input part 120b Conversion amount calculation part 130 D / A conversion part 140 L / V drive part

Claims (12)

  1. An image processing apparatus that performs desired image processing on RGB image input data in an RGB color space,
    Hue determination means for determining the hue of the pre-conversion color point indicated by the RGB image input data;
    Hue conversion means for hue-converting the pre-conversion color point by increasing or decreasing any one component of RGB according to the determination result of the hue determination means;
    An image processing apparatus that performs hue conversion of the RGB image input data by the hue conversion means.
  2. The image processing apparatus according to claim 1,
    The hue determination means;
    Determining whether the pre-conversion color point indicated by the RGB image input data belongs to a plurality of hue regions divided in the RGB color space;
    An image processing apparatus in which the hue conversion means performs hue conversion according to the determination result.
  3. The image processing apparatus according to claim 2,
    The hue determination means;
    The pre-conversion color point indicated by the RGB image input data is any one of an RY hue area, a YG hue area, a GC hue area, a CB hue area, a BM hue area, and an MR hue area. Image processing apparatus for determining whether it belongs to
  4. An image processing apparatus according to any one of claims 1 to 3, wherein
    An image processing apparatus comprising: a parameter input unit for the hue conversion unit to input a hue conversion parameter that defines a hue conversion amount.
  5. The image processing apparatus according to claim 4,
    The image processing apparatus, wherein the hue conversion parameter can be set independently for each hue of RGBCMY.
  6. The image processing apparatus according to claim 4 or 5, wherein
    An image processing apparatus comprising: a hue conversion amount calculation unit that calculates a hue conversion amount of the RGB image input data based on a determination result by the hue determination unit and an input hue conversion parameter.
  7. The image processing apparatus according to any one of claims 1 to 6,
    Applying a desired weight to the hue conversion amount by the hue conversion means so that a set of color points of the same hue becomes a set of color points of the same hue even after the hue conversion,
    An image processing apparatus that performs hue conversion of RGB image input data based on the weighted hue conversion amount.
  8. The image processing apparatus according to claim 7,
    An image processing apparatus that reduces the weighting as the pre-conversion color point is closer to the gray axis.
  9. An image processing apparatus that performs desired image processing on RGB image input data in an RGB color space,
    The hues of pre-conversion color points indicated by the RGB image input data are RY hue area, YG hue area, GC hue area, CB hue area, BM hue area, MR hue area. Hue determination means for determining which one of
    Hue conversion means for performing hue conversion on the pre-conversion color point;
    With
    The hue conversion means until the desired hue conversion amount H is achieved,
    Based on which hue region the hue of the pre-conversion color point belongs to, the assigned hue region to which the hue of the pre-conversion color point belongs, and the affiliation by increasing / decreasing any one component of RGB Assuming that the hue conversion is performed until the boundary between the hue area and the adjacent hue area is reached, any one component of RGB at the post-conversion color point (however, the component is included in the adjacent hue area). Is a hue conversion equivalent to increasing / decreasing).
    Image processing device.
  10. An image processing method for performing desired image processing on RGB image input data in an RGB color space,
    A hue determination step of determining a hue of a color point before conversion indicated by the RGB image input data;
    A hue conversion step for hue-converting the pre-conversion color point by increasing or decreasing any one component of RGB according to the determination result of the hue determination step;
    An image processing method for performing hue conversion of the RGB image input data by the hue conversion step.
  11. A program for causing a computer to execute desired image processing performed on RGB image input data in an RGB color space,
    A hue determination process for determining a hue of a color point before conversion indicated by the RGB image input data;
    A hue conversion process for performing a hue conversion on the pre-conversion color point by increasing or decreasing any one component of RGB according to the determination result of the hue determination process;
    A program for causing a computer to perform image processing for performing hue conversion of the RGB image input data by the hue conversion processing.
  12. A computer-readable recording medium on which the program according to claim 11 is recorded.
JP2001134450A 2001-05-01 2001-05-01 Image processing apparatus, image processing method, program, and recording medium Expired - Fee Related JP3791592B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2001134450A JP3791592B2 (en) 2001-05-01 2001-05-01 Image processing apparatus, image processing method, program, and recording medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2001134450A JP3791592B2 (en) 2001-05-01 2001-05-01 Image processing apparatus, image processing method, program, and recording medium

Publications (2)

Publication Number Publication Date
JP2002330298A JP2002330298A (en) 2002-11-15
JP3791592B2 true JP3791592B2 (en) 2006-06-28

Family

ID=18982131

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2001134450A Expired - Fee Related JP3791592B2 (en) 2001-05-01 2001-05-01 Image processing apparatus, image processing method, program, and recording medium

Country Status (1)

Country Link
JP (1) JP3791592B2 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPWO2006095496A1 (en) * 2005-03-10 2008-08-14 シャープ株式会社 Color conversion device, program, image display device, and portable terminal device
JP4609674B2 (en) 2007-08-09 2011-01-12 セイコーエプソン株式会社 Image processing system, display device, program, and information storage medium

Also Published As

Publication number Publication date
JP2002330298A (en) 2002-11-15

Similar Documents

Publication Publication Date Title
US8773340B2 (en) Color signal converter, display unit, color signal conversion program, computer-readable storage medium storing color signal conversion program, and color signal conversion method
US8310723B2 (en) Image processing apparatus, printing apparatus, and image processing method
DE69727414T2 (en) Image processing device and method
US7054035B2 (en) Method and system for management of color through conversion between color spaces
US6876764B2 (en) Method and apparatus for generating white component and controlling the brightness in display devices
KR100600228B1 (en) Image processing system, projector, information storage medium, and image processing method
US7505052B2 (en) Method and apparatus for displaying image and computer-readable recording medium for storing computer program
JP4126564B2 (en) Image processing system, projector, program, information storage medium, and image processing method
JP4046647B2 (en) Color adjustment apparatus and method
US7599096B2 (en) Tint adjustment for monochrome image printing
US6421142B1 (en) Out-of-gamut color mapping strategy
EP1047263B1 (en) Color image reproduction with accurate inside-gamut colors and enhanced outside-gamut colors
JP4200365B2 (en) Correspondence definition data creation grid point determination method, correspondence definition data creation grid point determination apparatus, correspondence definition data creation grid point determination program, print control apparatus, print control method, print control program, and image data processing apparatus
JP4705917B2 (en) Method and apparatus for converting from a source color space to a target color space
EP1301025B1 (en) Color correction table forming method and apparatus, control program and storage medium
JP5179829B2 (en) Gray data correction apparatus and method
JP4990507B2 (en) Image main color component adjustment apparatus, image main color component adjustment method, and image main color component adjustment program
JP4594185B2 (en) Color processing method and apparatus
US7944594B2 (en) Image processing device and an image processing method for curbing the amount of color material consumed and suppressing a deterioration in image quality
US7542167B2 (en) Smoothing lattice positions
US8335015B2 (en) Systems and methods for building a color lookup table for a printer
KR100736939B1 (en) Intelligence type color gamut management method
US7015929B2 (en) Color conversion method and image processor
EP1463338A2 (en) System, method and software for processing a projected image
KR100565810B1 (en) Color signal processing apparatus and method of using the same

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20040412

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20051121

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20051125

A521 Written amendment

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20060123

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20060315

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20060328

R150 Certificate of patent or registration of utility model

Free format text: JAPANESE INTERMEDIATE CODE: R150

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20090414

Year of fee payment: 3

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20100414

Year of fee payment: 4

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20110414

Year of fee payment: 5

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20110414

Year of fee payment: 5

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20120414

Year of fee payment: 6

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20130414

Year of fee payment: 7

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20130414

Year of fee payment: 7

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20140414

Year of fee payment: 8

S531 Written request for registration of change of domicile

Free format text: JAPANESE INTERMEDIATE CODE: R313531

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350

LAPS Cancellation because of no payment of annual fees