JPH0530503A - Image display/pickup device - Google Patents

Image display/pickup device

Info

Publication number
JPH0530503A
JPH0530503A JP3204530A JP20453091A JPH0530503A JP H0530503 A JPH0530503 A JP H0530503A JP 3204530 A JP3204530 A JP 3204530A JP 20453091 A JP20453091 A JP 20453091A JP H0530503 A JPH0530503 A JP H0530503A
Authority
JP
Japan
Prior art keywords
semi
image
display
transparent mirror
pickup device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
JP3204530A
Other languages
Japanese (ja)
Other versions
JP3087915B2 (en
Inventor
Masato Kuriki
真人 久力
Hitoshi Arai
均 新井
Shiro Suyama
史朗 陶山
Shigenobu Sakai
重信 酒井
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nippon Telegraph and Telephone Corp
Original Assignee
Nippon Telegraph and Telephone Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nippon Telegraph and Telephone Corp filed Critical Nippon Telegraph and Telephone Corp
Priority to JP03204530A priority Critical patent/JP3087915B2/en
Priority to US07/840,559 priority patent/US5317405A/en
Priority to DE69211385T priority patent/DE69211385T2/en
Priority to EP92103573A priority patent/EP0503432B1/en
Publication of JPH0530503A publication Critical patent/JPH0530503A/en
Application granted granted Critical
Publication of JP3087915B2 publication Critical patent/JP3087915B2/en
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Abstract

PURPOSE:To prevent images other than a reflected image to be a picked-up image from being entered into an image display/pickup device by attaining the coincidence of visual fields by means of a semitransparent mirror. CONSTITUTION:In the image display/pickup device constituted of arranging the semi-transparent mirror 4 in the front of a display device 1 and picking up the image of an object M by an image pickup device 2 as a reflected image from the mirror 4, the mirror 4 is constituted of a visual field selecting transparent base so that an object A other than the object M is not entered.

Description

【発明の詳細な説明】Detailed Description of the Invention

【0001】[0001]

【産業上の利用分野】本発明は、TV電話やTV会議シ
ステム等に用いられる表示装置と撮像装置が組み合わさ
った表示・撮像装置に関するものである。
BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a display / image pickup device in which a display device and an image pickup device used in a TV telephone or a TV conference system are combined.

【0002】[0002]

【従来の技術】近年、画像符号化技術等の通信技術の発
展により、遠隔地間を映像・音声通信網で結んだTV電
話やTV会議システム等の双方向映像通信が急速に普及
してきている。今後は、ISDN網の充実と伝送網の高
帯域化等により、より高臨場感に溢れた映像がサービス
できることが期待されている。
2. Description of the Related Art In recent years, two-way video communication such as a TV telephone and a TV conference system in which remote locations are connected by a video / audio communication network has rapidly spread due to the development of communication technology such as image coding technology. . In the future, it is expected that a more realistic video service can be provided by enhancing the ISDN network and increasing the bandwidth of the transmission network.

【0003】図8に現在双方向映像通信に用いられる表
示・撮像装置を示す。撮像装置2がCRTディスプレイ
等の表示装置1の上面または側面にあり、使用者の撮像
を行う。しかし、このような装置を利用する者は、通常
の表示装置1を見ているため、撮像装置2で撮像を行っ
ても、使用者は撮像装置2の方に視線を向けることはな
い。そのため、せっかくの高臨場感溢れる映像が送信可
能となっても、対話者同士の視線が一致しない不自然な
会話になるという欠点があった。
FIG. 8 shows a display / imaging device currently used for two-way video communication. The image pickup device 2 is on the upper surface or the side surface of the display device 1 such as a CRT display, and picks up an image of the user. However, since a person who uses such a device looks at the normal display device 1, the user does not turn his or her gaze toward the imaging device 2 even when the imaging device 2 captures an image. Therefore, there is a drawback in that even if it is possible to transmit a highly realistic image, it becomes an unnatural conversation in which the line of sight of the interlocutors do not match.

【0004】対話者同士の視線を一致させる方法に、半
透鏡(ハーフミラー)を用いる方式がある。図9に半透
鏡を用いた表示・撮像装置の構成図を示す。表示装置1
の前方に半透鏡3をある角度θだけ傾けて奥行きをdと
して設置してある。被写体Mからの像は、半透鏡3で反
射され、半透鏡3の上方または下方に設置された撮像装
置2により撮像されることになる。その結果、使用者の
視線は表示装置1に向けられているため、通話者と表示
画面に映し出された通話相手との視線が一致することに
なる。
As a method of matching the line of sight of the interlocutors, there is a method using a semi-transparent mirror (half mirror). FIG. 9 shows a block diagram of a display / imaging device using a semi-transparent mirror. Display device 1
The semi-transparent mirror 3 is tilted by a certain angle θ in front of, and the depth is set to d. The image from the subject M is reflected by the semi-transparent mirror 3 and is imaged by the imaging device 2 installed above or below the semi-transparent mirror 3. As a result, since the line of sight of the user is directed to the display device 1, the line of sight of the caller and the line of sight of the other party displayed on the display screen match.

【0005】[0005]

【発明が解決しようとする課題】しかしながら、半透鏡
3を用いる場合、反射像の他に半透鏡3を通した像も撮
像されてしまう欠点がある。例えば、図9の半透鏡3の
下方にある物体Aが被写体Mと一緒に撮像されるため、
図10に示すように撮像画面に物体Aがゴーストとして
表示され、撮像画像の画質劣化となる。実際のTV電話
装置としては、図11に示すように、1つの箱8内に配
置されるため、外部の物体像が映り込むことはないが、
半透鏡3により反射された表示画面が箱8の底面に反射
して撮像装置2に入ることがある。そのため、撮像画像
の画質を向上させるためには、このような反射像以外に
像の映り込みを防止することが急務であった。
However, when the semi-transparent mirror 3 is used, there is a drawback that an image through the semi-transparent mirror 3 is also captured in addition to the reflected image. For example, since the object A below the semi-transparent mirror 3 in FIG. 9 is imaged together with the subject M,
As shown in FIG. 10, the object A is displayed as a ghost on the image pickup screen, and the image quality of the picked-up image deteriorates. As shown in FIG. 11, the actual TV telephone device is arranged in one box 8 so that an external object image is not reflected, but
The display screen reflected by the semi-transparent mirror 3 may be reflected on the bottom surface of the box 8 and enter the imaging device 2. Therefore, in order to improve the quality of the captured image, it is an urgent task to prevent the reflection of an image other than such a reflected image.

【0006】本発明は、TV電話やTV会議システム等
に用いられる、半透鏡を用いて視線一致を実現した表示
・撮像装置において、撮像画像である反射像以外の像の
映り込みを防止し、映像画像の画質向上を図ることを目
的とする。
The present invention prevents reflection of an image other than a reflected image, which is a captured image, in a display / imaging device used in a TV telephone, a TV conference system, etc., which realizes line-of-sight matching using a semi-transparent mirror, The purpose is to improve the image quality of video images.

【0007】[0007]

【課題を解決するための手段】本発明に係る表示・撮像
装置は、表示装置の前方に半透鏡を設置し、被写体をこ
の半透鏡の反射像として撮像装置で撮像するものにおい
て、半透鏡を視野選択性透明基板を用いて構成した、ま
たは撮像装置前面ないし表示装置と撮像装置の双方の前
面に偏光子を配置して構成したものである。
In the display / imaging device according to the present invention, a semi-transparent mirror is installed in front of the display device, and a semi-transparent mirror is used for imaging a subject as a reflected image of the semi-transparent mirror. It is configured by using a field-selective transparent substrate, or by arranging a polarizer on the front surface of the image pickup device or both of the display device and the image pickup device.

【0008】[0008]

【作用】本発明は、半透鏡に視野選択性を有する材料を
用いた、または偏光子を通して表示・撮像を行うことに
より、反射像以外の像の映り込みを防止し、撮像画像の
画質を簡便に向上させる。
The present invention prevents the reflection of images other than the reflection image by using a semi-transparent mirror made of a material having a field-of-view selectivity or through a polarizer to display and pick up the image, and thus the image quality of the picked-up image is simplified. Improve to.

【0009】[0009]

【実施例】図1に本発明による表示・撮像装置の一実施
例を示す構成図を示す。図5に示した従来の表示・撮像
装置と構成はほぼ同じであるが、材料に視野選択性を有
する透明基板(例えば、日本板硝子(株)製 商品名ア
ングル21)を用いて半透鏡4を構成していることが大
きく異なる。このような視野選択性機能を有する透明基
板を用いた半透鏡4では、図2(a)に示したように、
正面の視点に対しては、ある視野方向に対してのみ不透
明に見え、図2(b)に示すように視点が上方あるいは
下方にある場合には、透明基板全体が不透明に見える。
そのため、図1において撮像装置2を不透明に見える方
向、すなわち図2(b)と同様に半透鏡4の中心からφ
方向に撮像装置2を設置すれば、撮像装置2は反射像の
み撮像することが可能となり、撮像画像の画質の向上が
測れる。
1 is a block diagram showing an embodiment of a display / imaging device according to the present invention. Although the configuration is almost the same as that of the conventional display / imaging device shown in FIG. 5, the semi-transparent mirror 4 is formed by using a transparent substrate (for example, an angle name 21 manufactured by Nippon Sheet Glass Co., Ltd.) having a field-selective material. The composition is very different. In the semi-transparent mirror 4 using the transparent substrate having such a visual field selection function, as shown in FIG.
From the front viewpoint, it looks opaque only in a certain viewing direction, and when the viewpoint is above or below as shown in FIG. 2B, the entire transparent substrate looks opaque.
Therefore, in the direction in which the imaging device 2 appears opaque in FIG. 1, that is, from the center of the semi-transparent mirror 4 as in FIG.
If the image pickup device 2 is installed in the direction, the image pickup device 2 can pick up only the reflected image, and the quality of the picked-up image can be improved.

【0010】図3は本発明による表示・撮像装置の他の
実施例を示すもので、複数の微小半透鏡を組み合わせて
構成した半透鏡4の断面図を示す。図3の半透鏡4の構
成としては、視野選択性透明基板(日本板硝子(株)製
商品名アングル21)4Aの表面に従来の複数の微小
半透鏡5が作製してある。微小半透鏡5を組み合わせて
構成する場合、表示装置1と撮像装置2が同一直線上に
配置されるため、表示画面が撮像画面に映り込む欠点が
あった。しかし、撮像装置2を不透明に見える方向に設
置し、微小半透鏡5の配列をその方向に被写体Mの反射
像を反射するように構成することにより、撮像時の表示
画面の映り込みを防止することができる。
FIG. 3 shows another embodiment of the display / imaging device according to the present invention, and is a sectional view of a semi-transparent mirror 4 constructed by combining a plurality of minute semi-transparent mirrors. As the structure of the semi-transparent mirror 4 in FIG. 3, a plurality of conventional micro-semi-transparent mirrors 5 are formed on the surface of a field-selective transparent substrate (product name: Angle 21 manufactured by Nippon Sheet Glass Co., Ltd.) 4A. When the micro-semitransparent mirror 5 is combined and configured, the display device 1 and the image pickup device 2 are arranged on the same straight line, so that there is a drawback that the display screen is reflected on the image pickup screen. However, by disposing the image pickup device 2 in a direction in which it looks opaque and by arranging the array of the minute semi-transparent mirrors 5 so as to reflect the reflected image of the subject M in that direction, reflection of the display screen at the time of image pickup is prevented. be able to.

【0011】図4に、図3に示した半透鏡4を用いた場
合の表示・撮像装置の構成図を示す。半透鏡4が不透明
に見える視野方向に撮像装置2を設置してある。具体的
には、視野選択性透明基板4Aに前記のアングル21を
用いた場合、不透明に見える視野方向は、半透鏡の法線
に対し30〜60°方向であるため、半透鏡4の微小半
透鏡5の傾きθを20〜25°の範囲に設定することに
より、上記の視野方向に撮像装置2が設置できるように
してある。
FIG. 4 shows a block diagram of a display / imaging device when the semi-transparent mirror 4 shown in FIG. 3 is used. The image pickup device 2 is installed in the visual field direction in which the semi-transparent mirror 4 looks opaque. Specifically, when the angle 21 is used for the view-selective transparent substrate 4A, the view direction that appears to be opaque is 30 to 60 ° with respect to the normal to the semi-transparent mirror, and thus the semi-transparent mirror 4 has a small semi-transparent direction. By setting the inclination θ of the transparent mirror 5 in the range of 20 to 25 °, the image pickup device 2 can be installed in the above-mentioned visual field direction.

【0012】図5に偏光子を用いた本発明の一実施例を
示す。6,7は偏光子であり、その作用を図6に示す。
偏光子6を通った光は特定の方向のみに電場が振動する
ため、その方向から90度ずれた偏光方向の偏光子7を
通過することができない。この効果を用いて、図5のよ
うに、表示装置1の全面と撮像装置2の全面に互いに図
6のような関係で偏光子6,7をおけば、図5中の実線
で示した表示画像の映り込みは生じない。但し、被写体
Mから見た表示が面の明るさおよび撮像装置2への入射
光量は減少することになる。
FIG. 5 shows an embodiment of the present invention using a polarizer. Reference numerals 6 and 7 are polarizers, and their operation is shown in FIG.
The light passing through the polarizer 6 vibrates in the electric field only in a specific direction, and therefore cannot pass through the polarizer 7 having a polarization direction deviated by 90 degrees from that direction. By using this effect, as shown in FIG. 5, if the polarizers 6 and 7 are provided on the entire surface of the display device 1 and the entire surface of the image pickup device 2 in the relationship as shown in FIG. 6, the display shown by the solid line in FIG. No image reflection occurs. However, the brightness of the surface viewed from the subject M and the amount of light incident on the imaging device 2 are reduced.

【0013】また、図7のように、表示装置として液晶
ディスプレイ11を用いる場合には、本来液晶ディスプ
レイ内部に偏光子があるので、その偏光方向に対してず
れた偏光方向の偏光子6を撮像装置2の前面に置くだけ
で同様の効果が得られれる。この場合には、表示画面の
明るさには変化がないことはいうまでもない。
Further, when the liquid crystal display 11 is used as the display device as shown in FIG. 7, since the polarizer is originally inside the liquid crystal display, the polarizer 6 having a polarization direction deviated from the polarization direction is imaged. The same effect can be obtained simply by placing it on the front surface of the device 2. In this case, it goes without saying that the brightness of the display screen does not change.

【0014】以上の効果は、45度に配置した半透鏡3
の場合で説明したが、視野選択性透明基板4Aを用いた
半透鏡4の場合と同様、複数の微小半透鏡5を組み合わ
せて構成した半透鏡4の場合にも有効であることはいう
までもない。
The above effect is obtained by the semi-transparent mirror 3 arranged at 45 degrees.
However, as with the case of the semi-transparent mirror 4 using the field-selective transparent substrate 4A, it is needless to say that it is also effective in the case of the semi-transparent mirror 4 configured by combining a plurality of minute semi-transparent mirrors 5. Absent.

【0015】[0015]

【発明の効果】本発明は以上説明したように、半透鏡を
視野選択性透明基板で構成し、または撮像装置の全面な
いしは撮像装置と表示装置の双方の前面に偏光子を配置
して構成したことにより、撮像画像への表示画面の映り
込みを防止することができる。
As described above, according to the present invention, the semi-transparent mirror is composed of the field-selective transparent substrate, or the polarizer is arranged on the entire surface of the image pickup device or on the front surface of both the image pickup device and the display device. As a result, the reflection of the display screen on the captured image can be prevented.

【図面の簡単な説明】[Brief description of drawings]

【図1】本発明の表示・撮像装置の一実施例を示す構成
図である。
FIG. 1 is a configuration diagram showing an embodiment of a display / imaging device of the present invention.

【図2】本発明で用いた視野選択性の機能を有する透明
基板を説明するための図である。
FIG. 2 is a diagram illustrating a transparent substrate having a function of visual field selectivity used in the present invention.

【図3】本発明で用いる半透鏡の他の実施例を示す断面
図である。
FIG. 3 is a sectional view showing another embodiment of the semi-transparent mirror used in the present invention.

【図4】図3に示した半透鏡を用いた本発明の表示・撮
像装置の構成図である。
4 is a configuration diagram of a display / imaging device of the present invention using the semi-transparent mirror shown in FIG.

【図5】本発明の他の実施例を示す構成図である。FIG. 5 is a configuration diagram showing another embodiment of the present invention.

【図6】図5の実施例における偏光子の作用を説明する
図である。
FIG. 6 is a diagram for explaining the action of the polarizer in the embodiment of FIG.

【図7】本発明のさらに他の実施例を示す構成図であ
る。
FIG. 7 is a configuration diagram showing still another embodiment of the present invention.

【図8】従来の表示・撮像装置の構成図である。FIG. 8 is a configuration diagram of a conventional display / imaging device.

【図9】半透鏡を用いた従来の表示・撮像装置の断面図
である。
FIG. 9 is a sectional view of a conventional display / imaging device using a semi-transparent mirror.

【図10】図9の従来例の撮像画像の画質劣化を説明す
るための図である。
FIG. 10 is a diagram for explaining image quality deterioration of the captured image of the conventional example of FIG.

【図11】従来の表示・撮像装置の問題点を説明するた
めの図である。
FIG. 11 is a diagram for explaining a problem of a conventional display / imaging device.

【符号の説明】[Explanation of symbols]

1 表示装置 2 撮像装置 3 半透鏡 4 視野選択性透明基板を用いた半透鏡 4A 視野選択性透明基板 5 微小半透鏡 6 偏光子 7 偏光子 8 箱 11 液晶ディスプレイ 1 Display device 2 Imaging device 3 semi-transparent mirror 4 Semi-transparent mirror using field-selective transparent substrate 4A Field-selective transparent substrate 5 Micro semi-transparent mirror 6 Polarizer 7 Polarizer 8 boxes 11 LCD display

フロントページの続き (72)発明者 酒井 重信 東京都千代田区内幸町一丁目1番6号 日 本電信電話株式会社内Continued front page    (72) Inventor Shigenobu Sakai             1-6 Uchisaiwaicho, Chiyoda-ku, Tokyo             Inside Telegraph and Telephone Corporation

Claims (2)

【特許請求の範囲】[Claims] 【請求項1】表示装置と撮像装置および半透鏡から構成
され、前記表示装置の前面に前記半透鏡を設置し、前記
表示装置前方の被写体を前記半透鏡からの反射像として
前記表示装置の周囲に設置された前記撮像装置により撮
像を行う表示・撮像装置において、前記半透鏡を視野選
択性透明基板を用いて構成したことを特徴とする表示・
撮像装置。
1. A display device, an imaging device, and a semi-transparent mirror, wherein the semi-transparent mirror is installed in front of the display device, and a subject in front of the display device is surrounded by the semi-transparent mirror as a reflection image from the semi-transparent mirror. In the display / image pickup device for picking up an image by the image pickup device installed in, a display characterized by comprising the semi-transparent mirror using a field-selective transparent substrate.
Imaging device.
【請求項2】表示装置と撮像装置および半透鏡から構成
され、前記表示装置の全面に前記半透鏡を設置し、前記
表示装置前方の被写体を前記半透鏡からの反射像として
前記表示装置の周囲に設置された前記撮像装置により撮
像を行う表示・撮像装置において、前記撮像装置の前面
または前記表示装置と前記撮像装置双方の前面に偏光子
を配置したことを特徴とする表示・撮像装置。
2. A display device, an image pickup device, and a semi-transparent mirror, wherein the semi-transparent mirror is installed on the entire surface of the display device, and a subject in front of the display device is surrounded by the semi-transparent mirror as a reflection image from the semi-transparent mirror. In the display / imaging device for performing imaging with the imaging device installed in, a display / imaging device in which a polarizer is disposed on the front surface of the imaging device or both the display device and the front surface of the imaging device.
JP03204530A 1991-03-08 1991-07-22 Display / imaging device Expired - Lifetime JP3087915B2 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
JP03204530A JP3087915B2 (en) 1991-07-22 1991-07-22 Display / imaging device
US07/840,559 US5317405A (en) 1991-03-08 1992-02-25 Display and image capture apparatus which enables eye contact
DE69211385T DE69211385T2 (en) 1991-03-08 1992-03-02 Image capture and playback device that allows eye contact
EP92103573A EP0503432B1 (en) 1991-03-08 1992-03-02 Display and image capture apparatus which enables eye contact

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP03204530A JP3087915B2 (en) 1991-07-22 1991-07-22 Display / imaging device

Publications (2)

Publication Number Publication Date
JPH0530503A true JPH0530503A (en) 1993-02-05
JP3087915B2 JP3087915B2 (en) 2000-09-18

Family

ID=16492067

Family Applications (1)

Application Number Title Priority Date Filing Date
JP03204530A Expired - Lifetime JP3087915B2 (en) 1991-03-08 1991-07-22 Display / imaging device

Country Status (1)

Country Link
JP (1) JP3087915B2 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5560204A (en) * 1992-08-04 1996-10-01 Kabushiki Kaisha Komatsu Seisakusho Compensation system for hydraulic circuit of hydraulically driven vehicle for straight traveling
US5651390A (en) * 1992-10-23 1997-07-29 Kabushiki Kaisha Komatsu Seisakusho Pressurized fluid supply system

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5560204A (en) * 1992-08-04 1996-10-01 Kabushiki Kaisha Komatsu Seisakusho Compensation system for hydraulic circuit of hydraulically driven vehicle for straight traveling
US5651390A (en) * 1992-10-23 1997-07-29 Kabushiki Kaisha Komatsu Seisakusho Pressurized fluid supply system
US5784885A (en) * 1992-10-23 1998-07-28 Kabushiki Kaisha Komatsu Seisakusho Pressurized fluid supply system
US5845678A (en) * 1992-10-23 1998-12-08 Kabushiki Kaisha Komatsu Seisakusho Pressurized fluid supply system

Also Published As

Publication number Publication date
JP3087915B2 (en) 2000-09-18

Similar Documents

Publication Publication Date Title
JP3027065B2 (en) Display / imaging device
US7821561B2 (en) Display apparatus with camera and communication apparatus
US5317405A (en) Display and image capture apparatus which enables eye contact
TW200307460A (en) Data processing device, data processing system and method for displaying conversation parties
JPH11504780A (en) Eye contact device using directional transmission layer for video conferencing
US5953053A (en) Teleconference system with visual display terminal
JP3122531B2 (en) Display / imaging device
GB2273411A (en) Video conferencing terminal
JPH0530503A (en) Image display/pickup device
JP2859414B2 (en) Display / imaging device
JPH04213287A (en) Display and image pickup device
JPH05199518A (en) Video telephone system
JP3058369B2 (en) Display / imaging device
JP2859417B2 (en) Display / imaging device
JP3209790B2 (en) Display / imaging device
JPH06233290A (en) Image pickup and display device
JP3122543B2 (en) Display / imaging device
JP3122529B2 (en) Display / imaging device
JP2859420B2 (en) Display / Imaging device
JPH0413391A (en) Picture display device with image pickup device
JP3001251B2 (en) Display / imaging device
JPH0564183A (en) Video telephone set
JPH05145912A (en) Display/image pickup device
JPH0556423A (en) Display/image pickup device
JPS58107786A (en) Picture display image pickup device

Legal Events

Date Code Title Description
FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20070714

Year of fee payment: 7

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20080714

Year of fee payment: 8

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20080714

Year of fee payment: 8

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20090714

Year of fee payment: 9

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20090714

Year of fee payment: 9

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20100714

Year of fee payment: 10

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20100714

Year of fee payment: 10

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20110714

Year of fee payment: 11

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20120714

Year of fee payment: 12

EXPY Cancellation because of completion of term
FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20120714

Year of fee payment: 12