JP3630015B2 - Projection display apparatus and information storage medium - Google Patents

Projection display apparatus and information storage medium Download PDF

Info

Publication number
JP3630015B2
JP3630015B2 JP11363099A JP11363099A JP3630015B2 JP 3630015 B2 JP3630015 B2 JP 3630015B2 JP 11363099 A JP11363099 A JP 11363099A JP 11363099 A JP11363099 A JP 11363099A JP 3630015 B2 JP3630015 B2 JP 3630015B2
Authority
JP
Japan
Prior art keywords
person
area
information
display area
display
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
JP11363099A
Other languages
Japanese (ja)
Other versions
JP2000305481A (en
Inventor
和憲 平松
Original Assignee
セイコーエプソン株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by セイコーエプソン株式会社 filed Critical セイコーエプソン株式会社
Priority to JP11363099A priority Critical patent/JP3630015B2/en
Publication of JP2000305481A publication Critical patent/JP2000305481A/en
Application granted granted Critical
Publication of JP3630015B2 publication Critical patent/JP3630015B2/en
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Images

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a projection display device such as a liquid crystal projector and an information storage medium.
[0002]
[Background Art and Problems to be Solved by the Invention]
In recent years, with the widespread use of projection display devices such as personal computers and liquid crystal projectors, presentations in the form of enlarging and projecting electronic video information have been widely performed.
[0003]
In the presentation, the presenter can directly point to a part of the display area, so that the main point of the display content can be indicated, and a more effective and efficient presentation is realized. Therefore, sometimes the presenter is located in the display area and the presentation is performed while watching the audience. At this time, when the projection display device is used, the projection display device and the audience are in the same direction when viewed from the presenter's position. Therefore, the image light output from the projection display device is not visible to the presenter's eyes. Direct incidence.
[0004]
In order to obtain sufficient brightness of the projected image even in a bright environment, the projection light quantity of the projection display device is improving year by year. Therefore, when the image light of the projection display device directly enters the eyes of the presenter, there is a problem that the presenter feels dazzling and uncomfortable.
[0005]
At this time, the face of the presenter may change due to the presenter squinting due to glare, which may adversely affect the audience's impression of the presenter.
[0006]
The present invention has been made in view of such problems, and an object of the present invention is to provide a presenter anti-glare measure for controlling the brightness of image light in an area where a person is located in a display area of a projection display device. It is an object of the present invention to provide a projection display device having a function and an information storage medium of a system having the projection display device.
[0007]
[Means for Solving the Problems]
In order to achieve the above object, the first invention provides:
Image light projection means for projecting image light onto a display area, imaging means for imaging the display area, and person position detection for detecting position information of a person located in the display area from imaging information obtained by the imaging means A brightness adjustment area determining means for determining a brightness adjustment area of the video light from the position information of the person, and a video light brightness adjusting means for adjusting the brightness of the video light in the brightness adjustment area. This is a characteristic projection display device.
[0008]
For this reason, anti-glare measures for a person in the display area, that is, a presenter, can be performed by controlling the image light from the position information of the person in the display area.
[0009]
According to a second invention, in the first invention, the brightness adjustment region determination means sets a region overlapping the person position detected by the person position detection means as the brightness adjustment range of the video light. Type display device.
[0010]
For this reason, the luminance control of the image light at the position of the eyes of the person can be reliably performed by performing the luminance control of the image light in the region overlapping with the person position. Compared with the case where only the position of the person's eyes is detected, the position of the person itself has a larger area and can be easily and reliably detected, and an effective anti-glare measure can be taken. The image light projected on the person position is an area where the image viewer cannot see the image due to the presence of the person, and controlling the brightness of the image light at the entire person position has a significant effect on the display function. Does not give.
[0011]
A third invention is the projection display device according to the first invention, wherein the imaging means uses the display area and its peripheral area as an imaging range.
[0012]
Therefore, it is possible to detect a person position even in the vicinity of the display area. In order to detect a person in the display area, it is necessary to separate the display image and the person image. However, the periphery of the display area is not necessary, and the person image can be detected easily.
[0013]
In a fourth aspect based on the third aspect, the person position detecting means detects a horizontal range of the person position in the display area based on imaging information in the lower part of the display area imaged by the imaging means,
The brightness adjustment range determining means is a projection display device characterized in that the display area in the horizontal range of the person position is set as a brightness adjustment range.
[0014]
For this reason, it is possible to easily detect a human image from imaging information of a limited area below the display area. The reason why the position can be detected in the lower part of the display area is that a person who is a presenter usually has an upright posture. Therefore, when eyes or faces are present in the display area, a torso or legs are present in the lower part of the display area. This is because the horizontal position of the person's display area can be detected by detecting this. By controlling the brightness of the image light with respect to the entire horizontal position detected in the display area, the brightness of the image light at the eye position can also be controlled when the presenter is upright.
[0015]
A fifth invention is the projection display device according to the first invention, wherein the image light luminance adjusting means sets the luminance of the image light in the luminance adjustment region to a black level.
[0016]
For this reason, the image light directly incident on the presenter's eyes can be greatly reduced. Note that the image light projected on the person position is an area where the image viewer cannot see the image due to the presence of the person, and even if the image light of the entire person position is black, the display function is greatly affected. There is nothing.
[0017]
According to a sixth aspect of the present invention, based on the imaging information of the display area of the projection display device, the position detection control information for detecting the position information of the person located in the display area and the projection type from the position information of the person. The brightness adjustment area determination control information for determining the brightness adjustment area of the image light projected by the display device and the image light brightness adjustment control information for adjusting the brightness of the image light in the brightness adjustment area are stored. A computer-readable information storage medium.
[0018]
In a seventh aspect based on the sixth aspect, the brightness adjustment area determination control information includes control information in which an area overlapping with a position of a person located in the display area is set as the brightness control range of the video light. A characteristic information storage medium.
[0019]
In an eighth aspect based on the sixth aspect, the person position detection control information detects a horizontal range of the person position in the display area based on the image information in the lower part of the display area of the projection display device.
The luminance adjustment range determination control information is an information storage medium including control information in which the display area in the horizontal range of the person position is a luminance adjustment range.
[0020]
A ninth invention is the information storage medium according to the sixth invention, wherein the video light luminance adjustment control information includes control information for setting the luminance of the video light in the luminance adjustment region to a black level. .
[0021]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0022]
(1) Overall Description FIG. 1 is a schematic explanatory diagram of a usage pattern of a projection display device to which the present invention is applied.
[0023]
On the screen 4, image light is projected onto the display area 3 from the projector 1, which is a projection display device provided in the front. In FIG. 1, a person 5 who is a presenter is present in the display area 3 to point to a predetermined position, and a presentation is made while watching the audience 7. The projector 1 is provided with a CCD camera 2 that is an imaging means, and detects the position of a person on the display area. Then, the image light in the image light luminance adjustment area 6 is set to the black level according to the detection result of the person position. Therefore, the incident amount of the image light projected from the projector 1 with respect to the eyes of the person 5 is reduced, and the person 5 does not feel dazzling due to the image light.
[0024]
Here, the display information that should be originally displayed is not displayed in the image light brightness adjustment area 6, but the person 5 prevents the image light from being reflected on the screen 4, so the brightness of the image light is adjusted. Even if there is not, a normal image cannot be obtained at the position where the person 5 is located. Therefore, when the image light brightness adjustment area 6 substantially coincides with the position of the person 5, the display information is reduced by the image light brightness adjustment 6. There are few.
[0025]
FIG. 2 is a functional block diagram of the projection display apparatus system of the present embodiment.
[0026]
The projection display device 11 includes an imaging unit 14 and a control unit 12 in addition to the image light projection unit 13.
[0027]
The imaging means 14 is means for converting light into an electrical signal, and uses a CCD camera. The imaging area is an area including the display area of the image light projection means 13.
[0028]
The control unit 12 is a part that performs luminance control of the image light, and includes a person position detection unit 15, a luminance adjustment region determination unit 16, and an image light luminance adjustment unit 17 for each function. The person position detection unit 15 performs a process of detecting the position of the person on the display area from the imaging information obtained by the imaging unit 14. The brightness adjustment area determination unit 16 performs processing for determining which area of the display area is to be adjusted to be low based on the position information of the person obtained by the person position detection unit 15. The video light luminance adjustment unit 17 performs a process of reducing the luminance of the video input 18 indicating the original display image by setting the luminance of the luminance adjustment region obtained by the luminance adjustment region determination unit 16 to the black level or the like. .
[0029]
Note that the imaging unit 14 and the control unit 12 are not necessarily integrated with the image light projection unit 13 in the projection display device 11. For example, only the control unit 12 may exist outside. Further, some functions of the control unit 12, for example, only the image light luminance adjustment unit 17 may exist in the projection display device 11, and the other functions may exist outside. Moreover, the control part 12 can implement | achieve a function also in exclusive hardware or a computer.
[0030]
(2) Description of Anti-Glare Process Details An example of the anti-glare process, which is a series of processes for reducing the image light luminance at the person position on the display area, will be described. This anti-glare process is performed by the control unit 12.
[0031]
FIG. 3 is a flowchart showing the entire anti-glare process of this embodiment, and FIG. 4 is a timing chart showing the operation of the anti-glare process of this embodiment. This will be described with reference to FIGS.
[0032]
First, in step S01, the imaging information of the display area peripheral lower area 20 is extracted from the imaging information obtained by the imaging means. That is, only the information on the area where the person position is detected is extracted from the imaging information of the entire imaging area 22. The purpose of detecting the horizontal position of the person position from the imaging information of the display area peripheral lower area 20 is to separate the display image and the person image when performing person detection in the display area. This is because it is not necessary in the vicinity of, and a human image can be easily detected. Further, the reason why it is possible to detect the horizontal position of the person position from the imaging information of the display area peripheral lower area 20 is that the person who is the presenter stands upright, and thus there are eyes or faces in the display area. In this case, there is a trunk or legs in the lower part around the display area. Therefore, by detecting it, the horizontal position of the person's display area can be detected. Even when only the horizontal position of a person is detected, if the presenter is standing upright by controlling the brightness of the image light over the entire horizontal position detected in the display area, the image of the eye position is displayed. The brightness of the light can also be controlled, and the purpose of preventing glare of the presenter can be achieved.
[0033]
The imaging signal 31 indicates one horizontal line of imaging information of the display area peripheral lower area 20 of FIG. 4 obtained by the process of step S01. The luminance level of the image pickup signal 31 in the portion where the person 5 exists is lower than the portion where the person 5 does not exist. This is because when the display peripheral area is a screen, the person 5 is darker because it reflects less light than the screen.
[0034]
Next, at step S02, the imaging information obtained at step S01 is binarized to generate person position information. That is, the imaging signal 31 is binarized by the binarization threshold 32 to obtain a person position signal 33 indicating the horizontal position of the person.
[0035]
Note that the processing in steps S01 and S02 corresponds to the processing performed by the person position detection unit 15 in FIG.
[0036]
Next, in step S03, luminance adjustment area information of the image light is generated from the person position detection information. Specifically, the brightness adjustment region signal 34 is generated by expanding the pulse width, that is, the position information, with respect to the person position signal 33 indicating the position information of the person in the horizontal direction obtained in step S02. The reason why the brightness adjustment area is made wider than the original person position area is to provide a margin so that the image light control of the face area of the person can be performed even when the person is not completely upright.
[0037]
Note that if the brightness adjustment area is larger than necessary, the normal display area becomes unnecessarily narrow. Therefore, it is desirable that the brightness adjustment area be adjustable so that the brightness adjustment area is wider than the person position area.
[0038]
The process in step S03 corresponds to the process performed by the brightness adjustment area determination unit in FIG.
[0039]
Finally, in step S04, the video signal in the luminance adjustment area is converted to a black level to generate projection video information. Specifically, the video input signal 35, which is the video information originally projected by the video light projection means, is clamped to the black level at the timing of the luminance adjustment area signal 34 obtained in step S03, and the projected video signal Get 36.
[0040]
Here, FIG. 4 shows a process for one horizontal line, but the luminance adjustment area signal 34 is held for one frame, and the luminance adjustment area signal is obtained for each horizontal line of the video input signal 35. Black level clamp processing is performed at the timing of 34 to obtain a projection video signal 36 for one frame. Since the black level region 23 is generated in the display at that time, the direct incident amount of the image light to the eyes of the person 5 is reduced, and an antiglare effect is obtained.
[0041]
Although one embodiment of the anti-glare processing has been described above, the anti-glare processing method of the present embodiment detects the horizontal position of the person position around the display area, and the image light in the horizontal range with the display area is black level. Compared with the case where the position of a person in the display area is strictly detected, there is a feature that an anti-glare effect can be surely obtained with a simple process.
[0042]
The present invention is not limited to the above-described embodiment, and various modifications can be made within the scope of the gist of the present invention.
[0043]
For example, only the lower area 20 around the display area of the imaging means may be used as the imaging area.
[0044]
Further, the imaging unit and the control unit may be provided separately from the projection display device, and the control unit may be realized by a personal computer.
[Brief description of the drawings]
FIG. 1 is a schematic explanatory diagram of a usage form of a projection display device to which the present invention is applied.
FIG. 2 is a functional block diagram of the projection display apparatus system according to the present embodiment.
FIG. 3 is a flowchart showing the entire antiglare process of the present embodiment.
FIG. 4 is a timing chart illustrating an operation of an antiglare process according to the present exemplary embodiment.
[Explanation of symbols]
1. Projector 2. CCD camera 3. 3. Display area Screen 5. Person 6. 6. Image light brightness adjustment area Audience 11. Projection display device 12. Control unit 13. Image light projection means 14. Imaging means 15. Human position detector 16. Brightness adjustment region determination unit 17. Video light brightness adjustment unit 18. Video input20. Display area peripheral lower area 22. Total imaging area 23. Black level region 31. Imaging signal 32.2 threshold value 33. Person position signal 34. Brightness adjustment area signal 35. Video input signal 36. Projected image signal

Claims (9)

  1. Image light projection means for projecting image light onto a display area, imaging means for imaging the display area, and person position detection for detecting position information of a person located in the display area from imaging information obtained by the imaging means A brightness adjustment area determining means for determining a brightness adjustment area of the video light from the position information of the person, and a video light brightness adjusting means for adjusting the brightness of the video light in the brightness adjustment area. A projection display device characterized.
  2. The projection display device according to claim 1, wherein the brightness adjustment region determination unit sets a region overlapping the person position detected by the person position detection unit as a brightness adjustment range of the video light.
  3. The projection display device according to claim 1, wherein the imaging unit uses the display area and its peripheral area as an imaging range.
  4. In Claim 3, the said person position detection means detects the horizontal direction range of the person position in a display area based on the imaging information of the display area lower part which the said imaging means imaged,
    The projection display apparatus, wherein the luminance adjustment range determination unit sets the display area in a horizontal range of the person position as a luminance adjustment range.
  5. 2. The projection display device according to claim 1, wherein the image light luminance adjusting means sets the luminance of the image light in the luminance adjustment region to a black level.
  6. Based on imaging information of the display area of the projection display device, person position detection control information for detecting position information of a person located in the display area, and an image projected by the projection display apparatus from the position information of the person Computer-readable information storage storing brightness adjustment area determination control information for determining a light brightness adjustment area and video light brightness adjustment control information for adjusting the brightness of the image light in the brightness adjustment area Medium.
  7. 7. The information storage medium according to claim 6, wherein the brightness adjustment area determination control information includes control information in which an area overlapping with a position of a person located in the display area is set as a brightness control range of the video light.
  8. In Claim 6, the person position detection control information detects a horizontal range of the person position in the display area based on the image information in the lower part of the display area of the projection display device,
    The brightness adjustment range determination control information includes control information in which the display area in the horizontal range of the person position is a brightness adjustment range.
  9. 7. The information storage medium according to claim 6, wherein the video light brightness adjustment control information includes control information for setting the brightness of the video light in the brightness adjustment area to a black level.
JP11363099A 1999-04-21 1999-04-21 Projection display apparatus and information storage medium Expired - Fee Related JP3630015B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP11363099A JP3630015B2 (en) 1999-04-21 1999-04-21 Projection display apparatus and information storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP11363099A JP3630015B2 (en) 1999-04-21 1999-04-21 Projection display apparatus and information storage medium

Publications (2)

Publication Number Publication Date
JP2000305481A JP2000305481A (en) 2000-11-02
JP3630015B2 true JP3630015B2 (en) 2005-03-16

Family

ID=14617111

Family Applications (1)

Application Number Title Priority Date Filing Date
JP11363099A Expired - Fee Related JP3630015B2 (en) 1999-04-21 1999-04-21 Projection display apparatus and information storage medium

Country Status (1)

Country Link
JP (1) JP3630015B2 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9172931B2 (en) 2011-07-06 2015-10-27 Ricoh Company, Limited Projection display device, information processing device, projection display system, and program

Families Citing this family (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA2438739C (en) * 2001-02-16 2009-12-29 Imatte, Inc. Interactive teleconferencing display system
US6361173B1 (en) * 2001-02-16 2002-03-26 Imatte, Inc. Method and apparatus for inhibiting projection of selected areas of a projected image
AUPR807601A0 (en) * 2001-10-04 2001-10-25 Monash University Flare elimination in video conferencing
JP4591720B2 (en) * 2002-05-20 2010-12-01 セイコーエプソン株式会社 Projection-type image display system, projector, program, information storage medium, and image projection method
JP4009850B2 (en) 2002-05-20 2007-11-21 セイコーエプソン株式会社 Projection-type image display system, projector, program, information storage medium, and image projection method
JP4591719B2 (en) * 2002-05-20 2010-12-01 セイコーエプソン株式会社 Projection-type image display system, projector, program, information storage medium, and image projection method
US6789903B2 (en) * 2003-02-18 2004-09-14 Imatte, Inc. Generating an inhibit signal by pattern displacement
JP3849654B2 (en) 2003-02-21 2006-11-22 株式会社日立製作所 Projection display
JP3735624B2 (en) 2003-09-26 2006-01-18 Necビューテクノロジー株式会社 Projection display device
US6964481B2 (en) * 2003-11-24 2005-11-15 The Boeing Company Apparatus and method for projecting identifying information onto seats
US6984039B2 (en) 2003-12-01 2006-01-10 Eastman Kodak Company Laser projector having silhouette blanking for objects in the output light path
JP4747524B2 (en) * 2004-07-13 2011-08-17 セイコーエプソン株式会社 projector
US7325933B2 (en) * 2004-08-09 2008-02-05 Sanyo Electric Co., Ltd Projection type video display apparatus
JP2006091121A (en) * 2004-09-21 2006-04-06 Sanyo Electric Co Ltd Projection video display
JP5298425B2 (en) * 2006-12-12 2013-09-25 株式会社ニコン Projector and camera
GB2454685A (en) * 2007-11-14 2009-05-20 Everest Display Inc Projector having silhouette or black mask that tracks the user to prevent eye damage.
US20090147272A1 (en) * 2007-12-05 2009-06-11 Microvision, Inc. Proximity detection for control of an imaging device
JP5495389B2 (en) * 2008-03-24 2014-05-21 カシオ計算機株式会社 Projection apparatus, projection method, and program
JP5147016B2 (en) * 2008-03-24 2013-02-20 カシオ計算機株式会社 Projection apparatus, projection method, and program
DE102008060110A1 (en) * 2008-12-03 2010-06-10 Sennheiser Electronic Gmbh & Co. Kg Projection apparatus for projecting video signal to projection surface, has projection unit for generating broadcasting light signal based on video input signal
JP2011043689A (en) * 2009-08-21 2011-03-03 Ntt Docomo Inc Projector device and projector control method
CN102237013A (en) * 2010-04-29 2011-11-09 鸿富锦精密工业(深圳)有限公司 Display screen adjustment system and method, and advertisement board with adjustment system
JP4819960B1 (en) * 2010-08-24 2011-11-24 暁 吉井 Full-field projection device and full-field image system
US9641826B1 (en) 2011-10-06 2017-05-02 Evans & Sutherland Computer Corporation System and method for displaying distant 3-D stereo on a dome surface
JP2013156324A (en) 2012-01-27 2013-08-15 Seiko Epson Corp Projector and image projection method by the same
JPWO2013186994A1 (en) * 2012-06-15 2016-02-04 日本電気株式会社 Projection type projection device, light anti-glare method, and light anti-glare program
WO2013186993A1 (en) * 2012-06-15 2013-12-19 日本電気株式会社 Projection-type projector, anti-glare method, and program for anti-glare
JP6083338B2 (en) * 2013-07-08 2017-02-22 株式会社デンソー Projection display
JPWO2017149943A1 (en) * 2016-03-02 2018-12-27 ソニー株式会社 Image display control device, image display control method, and program
DE102016215746A1 (en) 2016-08-23 2018-03-01 Robert Bosch Gmbh Projector with non-contact control
KR101831712B1 (en) 2017-06-22 2018-02-23 영남대학교 산학협력단 Module for video processing, aparratus and method for video output
JP2017191340A (en) * 2017-07-04 2017-10-19 カシオ計算機株式会社 Irradiation control device, irradiation control method, and program

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9172931B2 (en) 2011-07-06 2015-10-27 Ricoh Company, Limited Projection display device, information processing device, projection display system, and program

Also Published As

Publication number Publication date
JP2000305481A (en) 2000-11-02

Similar Documents

Publication Publication Date Title
JP2018023158A (en) Preview image display method and device, and terminal
CN105554413B (en) A kind of screen light filling photographic method, device and mobile terminal
US9508005B2 (en) Method for warning a user about a distance between user' s eyes and a screen
RU2616567C2 (en) Device and method for displayed image rotation control
TWI545947B (en) Display device with image capture and analysis module
US7430369B2 (en) Image capture apparatus and control method therefor
US8891867B2 (en) Image processing method
US6075557A (en) Image tracking system and method and observer tracking autostereoscopic display
KR100948572B1 (en) Method and apparatus for inhibiting projection of selected areas of a projected image
TWI380232B (en)
US7248289B2 (en) Image-pickup signal processor and method of detecting flicker
US9143695B2 (en) Systems and methods for imaging objects
US7349020B2 (en) System and method for displaying an image composition template
US7284864B2 (en) Projector type display apparatus
US7791652B2 (en) Image processing apparatus, image capture apparatus, image output apparatus, and method and program for these apparatus
CN101499253B (en) Output picture regulation method and apparatus
DE10315462B4 (en) Image correction system and method
EP1704711B1 (en) Method and apparatus for optimizing capture device settings through depth information
CN201993777U (en) Digital camera
JP4232042B2 (en) Projection control system, projector, program, information storage medium, and projection control method
US7137707B2 (en) Projector-camera system with laser pointers
US8538147B2 (en) Methods and appartuses for restoring color and enhancing electronic images
US6278491B1 (en) Apparatus and a method for automatically detecting and reducing red-eye in a digital image
TWI362214B (en) Camera apparatus, recording medium in which camera apparatus control program is recorded, and method for controlling camera apparatus
TWI336420B (en) Method and apparatus for generating binary matte signal

Legal Events

Date Code Title Description
TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20041124

R150 Certificate of patent or registration of utility model

Free format text: JAPANESE INTERMEDIATE CODE: R150

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20041207

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20081224

Year of fee payment: 4

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20091224

Year of fee payment: 5

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20101224

Year of fee payment: 6

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20101224

Year of fee payment: 6

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20111224

Year of fee payment: 7

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20111224

Year of fee payment: 7

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20121224

Year of fee payment: 8

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20121224

Year of fee payment: 8

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20131224

Year of fee payment: 9

S531 Written request for registration of change of domicile

Free format text: JAPANESE INTERMEDIATE CODE: R313531

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350

LAPS Cancellation because of no payment of annual fees