JP3508271B2 - Imaging device pointing device - Google Patents

Imaging device pointing device

Info

Publication number
JP3508271B2
JP3508271B2 JP04276395A JP4276395A JP3508271B2 JP 3508271 B2 JP3508271 B2 JP 3508271B2 JP 04276395 A JP04276395 A JP 04276395A JP 4276395 A JP4276395 A JP 4276395A JP 3508271 B2 JP3508271 B2 JP 3508271B2
Authority
JP
Japan
Prior art keywords
visual axis
target
image pickup
size
drive
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
JP04276395A
Other languages
Japanese (ja)
Other versions
JPH08242402A (en
Inventor
博之 渋谷
久男 難波
秀和 西内
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujitsu Ltd
Original Assignee
Fujitsu Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujitsu Ltd filed Critical Fujitsu Ltd
Priority to JP04276395A priority Critical patent/JP3508271B2/en
Publication of JPH08242402A publication Critical patent/JPH08242402A/en
Application granted granted Critical
Publication of JP3508271B2 publication Critical patent/JP3508271B2/en
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Landscapes

  • Image Analysis (AREA)
  • Image Processing (AREA)

Description

【発明の詳細な説明】Detailed Description of the Invention

【0001】[0001]

【産業上の利用分野】本発明は、TVカメラ等の撮像機
により得た一定視野を有する画面内の目標物体の位置
を、其の視野の中心となる様に,撮像機の視軸中心を基
にした位置情報をジンバル等の視軸変更装置にフィード
バックして, 撮像機の方向を制御する撮像機指向装置に
関する。
BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention sets the position of a target object in a screen having a constant visual field obtained by an image pickup device such as a TV camera so that the center of the visual axis of the image pickup device becomes the center of the visual field. The present invention relates to an image pickup device directing device that controls the direction of an image pickup device by feeding back position information based on the information to a visual axis changing device such as a gimbal.

【0002】[0002]

【従来の技術】従来の撮像機指向装置は、図6の構成図
に示す如く、ジンバル等の視軸変更装置に取付けられた
撮像機の撮像画面の中心である視軸中心を基準として得
た目標物体の位置情報から,視軸変更装置であるジンバ
ルへの駆動信号を生成し,其の駆動信号を視軸変更装置
へ, 目標物体の位置が視軸の中心に来るようにフィード
バックし制御し駆動していた。
2. Description of the Related Art As shown in the configuration diagram of FIG. 6, a conventional image pickup apparatus directing device is obtained with reference to the visual axis center which is the center of the image pickup screen of the image pickup apparatus attached to the visual axis changing device such as a gimbal. Based on the position information of the target object, a drive signal to the gimbal, which is the visual axis changing device, is generated, and the drive signal is fed back to the visual axis changing device so that the position of the target object is in the center of the visual axis and controlled. I was driving.

【0003】[0003]

【発明が解決しようとする課題】従来技術では、上記の
如く、撮像機の視軸中心を基準として得た目標物体の位
置の誤差を、単に視軸変更装置にフィードバックし制御
して零となる様に駆動していた為に、視軸変更装置であ
るジンバルの応答特性によっては、図7の様に,目標位
置が視軸中心に収斂する迄の追尾期間に振動現象を発生
し、視軸中心と目標位置とのズレが大きくなり,図7で
は点線で示す外部から与えられた追尾ゲート幅内に目標
物体が位置しない状況となって、所謂自動追尾が中断し
て継続しない事があった。本発明の目的は、視軸変更装
置であるジンバルの応答特性が如何なるものであって
も、目標位置が一定画面の中心の視軸に一致する迄の自
動追尾の期間に、振動現象を発生せず視軸と目標位置と
のズレが余り大きくならず,自動追尾が中断することの
無いような撮像機指向装置を実現することにある。
In the prior art, as described above, the position error of the target object obtained with the center of the visual axis of the image pickup device as a reference is simply fed back to the visual axis changing device and controlled to be zero. As shown in FIG. 7, a vibration phenomenon occurs during the tracking period until the target position converges on the center of the visual axis, depending on the response characteristics of the gimbal that is the visual axis changing device. There is a large deviation between the center and the target position, and in FIG. 7, the target object is not located within the tracking gate width given from the outside indicated by the dotted line, and so-called automatic tracking may be interrupted and may not continue. . An object of the present invention is to generate a vibration phenomenon during automatic tracking until the target position coincides with the visual axis at the center of a fixed screen, regardless of the response characteristics of the gimbal that is the visual axis changing device. The aim is to realize an imager-oriented device in which the deviation between the visual axis and the target position does not become too large and automatic tracking is not interrupted.

【0004】[0004]

【課題を解決するための手段】この目的達成のための本
発明の撮像機指向装置の基本構成は、図1の原理的な構
成図に示す如く、TVカメラ等の撮像手段から入力する
画像のデータを記憶する画像メモリ(1) と、其の出力の
画像データを処理する一定領域を設定する処理領域設定
手段(2) と、其の一定処理領域内の目標物体の位置を算
出する目標位置算出手段(3) と前記の算出された目標位
置を基にして撮像機の視軸方向を変える視軸変向装置へ
の駆動信号を生成する視軸変向駆動指令生成手段(5)
と、前記の算出された目標位置と外部から与えられた追
尾可能なゲート幅とを基に,視軸変向装置への駆動信号
の大きさを制限する視軸変向駆動指令制限手段(6)と、
其の大きさを制限された駆動信号により駆動される視軸
変向手段(7) とを具える様に構成する(請求項1)。そ
して前記視軸変向装置(7)への駆動信号の大きさを制限
する視軸変向駆動指令制限手段(6) には、前記駆動信号
の大きさを前記目標位置算出手段(3) の出力の目標位置
を使用して,外部から与えられた追尾可能なゲート幅以
下に制限するもの(請求項2)と、目標位置算出手段
(3) の他に更に目標物体のサイズを算出する目標サイズ
算出手段(4) を設け、前記駆動信号の大きさを,前記の
与えられた追尾可能なゲート幅から前記の算出された目
標サイズを差引いた差分以下に制限するもの(請求項
3)とがある。
To achieve this object, the basic structure of an image pickup apparatus directing device of the present invention is as shown in FIG. An image memory (1) for storing data, processing area setting means (2) for setting a certain area for processing the output image data, and a target position for calculating the position of a target object within the certain processing area Calculating means (3) and a visual axis changing drive command generating means (5) for generating a drive signal to a visual axis changing device that changes the visual axis direction of the image pickup device based on the calculated target position.
And a visual axis deflection drive command limiting means (6) for limiting the magnitude of the drive signal to the visual axis deflection device based on the calculated target position and the trackable gate width given from the outside. )When,
And a visual axis changing means (7) driven by a drive signal of which size is limited (claim 1). Then, the visual axis deflection drive command limiting means (6) for limiting the magnitude of the drive signal to the visual axis deflection device (7) is controlled by the target position calculation means (3). An output target position is used to limit the gate width that can be tracked externally and below (claim 2), and target position calculation means.
In addition to (3), a target size calculating means (4) for calculating the size of the target object is further provided, and the magnitude of the drive signal is calculated from the given trackable gate width to the calculated target size. There is one that is limited to the difference obtained by subtracting (Claim 3).

【0005】[0005]

【作用】本発明の請求項1の構成では、TVカメラ等の
撮像手段から入力された画像のデータを記憶する画像メ
モリ(1) の中に, 画面として処理する一定領域を設定す
る処理領域設定手段(2) を用いて一定処理領域を設定
し、其の一定領域内の目標物体の位置を目標位置算出手
段(3) が算出する、又は其の目標物体のサイズを目標サ
イズ算出手段(4) が算出する。そして目標位置算出手段
(3) で算出された目標位置から視軸変向駆動指令生成手
段(5) が視軸変向装置へ送る駆動信号を生成する。同時
に、目標サイズ算出手段(4) で算出された目標サイズか
ら決定され外部から与えられた追尾可能なゲート幅を,
設定領域内に設定し、視軸変向駆動指令制限手段(6)に
て,視軸変向装置(7)への駆動信号の大きさを制限す
る。斯うして制限された結果として適切な大きさの駆動
信号が例えばジンバル等の視軸変向手段(7) を駆動し、
撮像機の向きを変えて,目標位置が撮像機の画面の中央
の視軸中心位置に一致する迄,フィードバック制御す
る。そして視軸変向駆動制限手段(6) としては、目標
位置算出手段(3) で算出した目標物体の中心位置を用い
る方法(図2の請求項2)と、目標サイズ算出手段
(4) で算出した目標物体のサイズを用いる方法(図3の
請求項3)とがあるが、の目標位置を用いる方法で
は、視軸変向装置への駆動信号の大きさを,外部から与
えられた追尾可能なゲート幅以下に制限し、の目標サ
イズを用いる方法では、駆動信号の大きさを,追尾可能
なゲート幅から目標サイズを差引いた値となる。図中、
dxは目標位置を左右方向に移動させる駆動信号の大きさ
を表し、dyは目標位置を上下方向に移動させる駆動信号
の大きさを表す。図4に本発明の撮像機指向装置の動作
を説明する図を示す。上述の視軸変向駆動制限手段(6)
を用いると、与えられた追尾可能なゲート幅内に入った
目標物体の目標位置と目標サイズとが算出される限り、
与えられた追尾可能なゲート幅の内から外へ出てしまう
ことが無いので、目標位置が視軸中心に一致する迄の自
動追尾が中断することは無く、視軸変向装置の応答特性
の如何に拘わらず,図4の説明図に示す如く、画面上の
目標物体の位置を其の画面の中心に真直ぐに移動させる
ことが出来る。
According to the first aspect of the present invention, the processing area setting for setting a certain area to be processed as a screen in the image memory (1) for storing the data of the image input from the image pickup means such as a TV camera. A fixed processing area is set using the means (2) and the position of the target object within the fixed area is calculated by the target position calculation means (3), or the size of the target object is calculated by the target size calculation means (4 ) Is calculated. And target position calculation means
From the target position calculated in (3), the visual axis changing drive command generating means (5) generates a drive signal to be sent to the visual axis changing device. At the same time, the trackable gate width determined from the target size calculated by the target size calculation means (4) and given from the outside is
It is set in the setting area, and the magnitude of the drive signal to the visual axis changing device (7) is limited by the visual axis changing drive command limiting means (6). Thus, as a result of the limitation, a drive signal of an appropriate magnitude drives the visual axis changing means (7) such as a gimbal,
The orientation of the image pickup device is changed, and feedback control is performed until the target position coincides with the center position of the visual axis at the center of the image pickup device screen. As the visual axis deflection drive limiting means (6), a method of using the center position of the target object calculated by the target position calculating means (3) (claim 2 in FIG. 2) and target size calculating means
There is a method of using the size of the target object calculated in (4) (Claim 3 of FIG. 3). However, in the method of using the target position of, the magnitude of the drive signal to the visual axis deflecting device is changed from the outside. In the method in which the target size is limited to a given trackable gate width or less and the target size of is used, the magnitude of the drive signal is a value obtained by subtracting the target size from the trackable gate width. In the figure,
dx represents the magnitude of the drive signal that moves the target position in the left-right direction, and dy represents the magnitude of the drive signal that moves the target position in the up-down direction. FIG. 4 shows a diagram for explaining the operation of the image pickup device directing apparatus of the present invention. The above-mentioned visual axis deflection drive limiting means (6)
Using, the target position and target size of the target object that has entered within the given trackable gate width,
Since it does not go out from within the given gate width that can be tracked, automatic tracking will not be interrupted until the target position coincides with the visual axis center, and the response characteristics of the visual axis deflecting device Regardless of the method, as shown in the explanatory view of FIG. 4, the position of the target object on the screen can be moved straight to the center of the screen.

【0006】[0006]

【実施例】図5は本発明の実施例の撮像機指向装置の構
成図である。撮像手段であるTVカメラ(1) から入力さ
れたアナログの画像信号を、A/D変換器(2) でディジ
タルのデータに変換し其のディジタルの画像データを,
例えば 256 x 256画素の画像メモリ(3) に格納する。外
部入力のジョイスティック(9) を操作し、処理領域設定
回路(10)にて目標物体の位置決定の為の一定大きさの画
面の処理領域を設定する為に, 其の出力で該画像メモリ
(3) 内の単位画面に相当する, 例えば 32 x32画素の処
理画像メモリ(4) を設定する。そして、其の処理画像メ
モリ(4) の出力から, 視軸中心を基準とした目標物体の
面積重心の位置( 図2の dx, dy)を,面積重心算出回路
(5) にて算出し、其の重心位置の算出値を基にして, 目
標物体の位置を目標位置算出回路(7) にて算出する。そ
れと同時に、処理画像メモリ(4) の出力から, 視軸中心
を基準とした目標物体の周辺までの距離(図3の dx,d
y)を算出する事により, 目標物体のサイズを, 目標サイ
ズ算出回路(6) にて算出する。そして、目標位置算出回
路(7) にて算出された目標位置から, 最終的にはTVカ
メラ(1) の視野の中心方向を変えるジンバル(8) を駆動
する駆動信号の大きさ(駆動量,駆動レート信号と呼
ぶ)を, 駆動レート算出回路(13)にて算出し、其の算出
された駆動レート信号と, 先に目標サイズ算出回路(6)
にて算出された目標サイズと, 処理領域設定回路(10)に
て設定された追尾可能なゲート幅の3つの信号により、
駆動レート制限回路(12)にて大きさが制限された駆動レ
ート信号を生成し、駆動レート算出回路(13)からの大き
さが制限されない駆動レート信号と共に駆動レート信号
切換回路(11)へ出力する。駆動レート信号切換回路(11)
は、モード切換スイッチ(14)で操作されたモード切換回
路(15)からのモード切換信号により、前記の大きさが制
限されない駆動レート信号と, 大きさが制限された駆動
レート信号とを切り換えて、其の出力の駆動レート信号
をジンバル(8) へ送り、該ジンバル(8) を駆動して、目
標物体の位置を,TVカメラ(1) の視野の中心方向に移
動させ、目標物体の像を撮像画面の中心で捕らえること
が出来るようにする。
FIG. 5 is a block diagram of an image pickup device directing apparatus according to an embodiment of the present invention. The analog image signal input from the TV camera (1) which is an image pickup means is converted into digital data by the A / D converter (2), and the digital image data is converted into digital data.
For example, it is stored in the image memory (3) of 256 x 256 pixels. Operate the joystick (9) of external input and set the processing area of the screen of a certain size to determine the position of the target object in the processing area setting circuit (10).
Set the processing image memory (4) of 32 x 32 pixels, which corresponds to the unit screen in (3). Then, from the output of the processed image memory (4), the position of the area centroid of the target object (dx, dy in Fig. 2) with respect to the center of the visual axis is calculated as the area centroid calculation circuit.
The position of the target object is calculated by the target position calculation circuit (7) based on the calculated value of the position of the center of gravity. At the same time, the distance from the output of the processed image memory (4) to the periphery of the target object based on the center of the visual axis (dx, d in Fig. 3).
By calculating y), the size of the target object is calculated by the target size calculation circuit (6). Then, based on the target position calculated by the target position calculation circuit (7), the magnitude of the drive signal (drive amount, drive amount, which finally drives the gimbal (8) that changes the direction of the center of the visual field of the TV camera (1), Drive rate signal) is calculated by the drive rate calculation circuit (13), and the calculated drive rate signal and the target size calculation circuit (6)
Based on the target size calculated in, and the three signals of the trackable gate width set in the processing area setting circuit (10),
The drive rate limiting circuit (12) generates a drive rate signal whose size is limited, and outputs it to the drive rate signal switching circuit (11) together with the drive rate signal from the drive rate calculating circuit (13) whose size is not limited. To do. Drive rate signal switching circuit (11)
Is switched between the drive rate signal whose size is not limited and the drive rate signal whose size is limited by the mode switching signal from the mode switching circuit (15) operated by the mode switch (14). , The output drive rate signal is sent to the gimbal (8), the gimbal (8) is driven to move the position of the target object toward the center of the visual field of the TV camera (1), and the image of the target object is displayed. To be able to be captured at the center of the imaging screen.

【0007】[0007]

【発明の効果】以上説明した如く、本発明によれば、撮
像機が撮像した画面上の目標物体の位置を画面の中心と
なる様に制御する自動追尾に於けるジンバル等の視軸変
向装置の応答特性に対する要求条件を緩和することが出
来るとともに、目標物体の追尾を可能とする機会を従来
よりも増大することが出来る効果が得られる。
As described above, according to the present invention, the visual axis deflection of the gimbal or the like in the automatic tracking for controlling the position of the target object on the screen imaged by the imaging device to be the center of the screen. The effect that the requirement for the response characteristic of the device can be relaxed and the chances of enabling the tracking of the target object can be increased as compared with the conventional case can be obtained.

【図面の簡単な説明】[Brief description of drawings]

【図1】 本発明の撮像機指向装置の基本構成を示す原
理図
FIG. 1 is a principle diagram showing a basic configuration of an image pickup device directing apparatus of the present invention.

【図2】 本発明の請求項2に対応する実施例の視軸変
向装置への駆動量の制限手段の動作を説明する為の説明
FIG. 2 is an explanatory view for explaining the operation of the drive amount limiting means for the visual axis deflecting device of the embodiment corresponding to claim 2 of the present invention.

【図3】 本発明の請求項3に対応する実施例の視軸変
向装置への駆動量の制限手段の動作を説明する為の説明
FIG. 3 is an explanatory diagram for explaining the operation of the drive amount limiting means for the visual axis deflecting device of the embodiment corresponding to claim 3 of the present invention.

【図4】 本発明の実施例の撮像機指向装置の指向動作
を説明する図
FIG. 4 is a diagram for explaining pointing operation of the image pickup apparatus pointing device according to the embodiment of the present invention.

【図5】 本発明の実施例の撮像機指向装置の構成図FIG. 5 is a configuration diagram of an image pickup device directing apparatus according to an embodiment of the present invention.

【図6】 従来の撮像機指向装置の構成例の図FIG. 6 is a diagram of a configuration example of a conventional image pickup device directing device.

【図7】 従来の撮像機指向装置の動作の問題点を説明
する為の説明図
FIG. 7 is an explanatory diagram for explaining problems in the operation of the conventional imaging device-oriented device.

【符号の説明】[Explanation of symbols]

図1にて、(1) は画像メモリ、(2) は処理領域設定手
段、(3) は目標位置算出手段、(4) は目標サイズ算出手
段、(5) は視軸変向駆動指令生成手段、(6) は視軸変向
駆動指令制限手段、(7) は撮像機の視軸変向手段で,例
えばジンバルである。
In FIG. 1, (1) is an image memory, (2) is a processing area setting means, (3) is a target position calculation means, (4) is a target size calculation means, and (5) is a visual axis deflection drive command generation. Means, (6) is a visual axis changing drive command limiting means, and (7) is a visual axis changing means of the image pickup device, for example, a gimbal.

───────────────────────────────────────────────────── フロントページの続き (56)参考文献 特開 昭57−132489(JP,A) 特開 昭59−208983(JP,A) (58)調査した分野(Int.Cl.7,DB名) H04N 5/232 ─────────────────────────────────────────────────── ─── Continuation of front page (56) References JP-A-57-132489 (JP, A) JP-A-59-208983 (JP, A) (58) Fields investigated (Int.Cl. 7 , DB name) H04N 5/232

Claims (3)

(57)【特許請求の範囲】(57) [Claims] 【請求項1】 撮像機から入力する画像のデータを記憶
する画像メモリ(1)と、其の出力の画像データを処理す
る一定領域を設定する処理領域設定手段(2)と、其の設
定した一定領域内の目標物体の位置を算出する目標位置
算出手段(3)と、前記の算出された目標位置を基に撮像
機の視軸方向を変える視軸変向装置への駆動信号を生成
する視軸変向駆動指令生成手段(5) と、前記算出された
目標位置と,外部から与えられた追尾可能なゲート幅と
を基に,視軸変向装置への駆動信号の大きさを制限する
視軸変向駆動制限手段(6)と、其の大きさを制限された
駆動信号により駆動される視軸変向手段(7) とを具えた
ことを特徴とする撮像機指向装置。
1. An image memory (1) for storing image data input from an image pickup device, a processing area setting means (2) for setting a certain area for processing image data of the output, and the setting Target position calculation means (3) for calculating the position of the target object within a certain area, and a drive signal to the visual axis changing device for changing the visual axis direction of the image pickup device based on the calculated target position. The magnitude of the drive signal to the visual axis direction changing device is limited based on the visual axis direction changing drive command generating means (5), the calculated target position, and the trackable gate width given from the outside. An apparatus for directing an image pickup device, comprising: a visual axis changing drive limiting means (6) and a visual axis changing means (7) driven by a drive signal whose size is limited.
【請求項2】 前記の視軸変向駆動指令制限手段(6)
が、目標位置算出手段(3) で算出した目標物体の中心位
置の情報を用いて前記駆動信号の大きさを前記追尾可能
なゲート幅以下に制限するものであることを特徴とした
前記請求項1記載の撮像機指向装置。
2. The visual axis deflection drive command limiting means (6)
Wherein the size of the drive signal is limited to be equal to or smaller than the trackable gate width by using the information of the center position of the target object calculated by the target position calculation means (3). 1. The image pickup device directing device according to 1.
【請求項3】 前記の目標位置算出手段(3) に更に目標
物体のサイズを検出する目標サイズ検出手段(4) を設
け、視軸変向駆動制限手段(6) が前記駆動信号の大きさ
を前記追尾可能なゲート幅から前記目標サイズを差引い
た差分以下に制限することを特徴とした前記請求項1記
載の撮像機指向装置。
3. The target position calculating means (3) is further provided with a target size detecting means (4) for detecting the size of the target object, and the visual axis deflection drive limiting means (6) controls the magnitude of the drive signal. 2. The image pickup apparatus directing apparatus according to claim 1, wherein the target width is limited to a difference equal to or smaller than the trackable gate width minus the target size.
JP04276395A 1995-03-02 1995-03-02 Imaging device pointing device Expired - Fee Related JP3508271B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP04276395A JP3508271B2 (en) 1995-03-02 1995-03-02 Imaging device pointing device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP04276395A JP3508271B2 (en) 1995-03-02 1995-03-02 Imaging device pointing device

Publications (2)

Publication Number Publication Date
JPH08242402A JPH08242402A (en) 1996-09-17
JP3508271B2 true JP3508271B2 (en) 2004-03-22

Family

ID=12645026

Family Applications (1)

Application Number Title Priority Date Filing Date
JP04276395A Expired - Fee Related JP3508271B2 (en) 1995-03-02 1995-03-02 Imaging device pointing device

Country Status (1)

Country Link
JP (1) JP3508271B2 (en)

Also Published As

Publication number Publication date
JPH08242402A (en) 1996-09-17

Similar Documents

Publication Publication Date Title
US6489993B1 (en) Image pickup apparatus with optical and electronic zoom
US4566036A (en) Remote control apparatus
US20020080257A1 (en) Focus control system and process
US6078751A (en) Image-shake correcting device
KR960008931B1 (en) Image stabilizing apparatus for a portable video camera
US6100927A (en) Photographing apparatus having a blurring correcting apparatus
EP0425243B1 (en) Lens control apparatus
JP3508271B2 (en) Imaging device pointing device
JPH042281A (en) Automatic focusing device
JPH10294890A (en) Automatic/manual photographic camera system
JP2003219414A (en) Video camera system
JPS63284990A (en) Tracing device for video camera
JP3357628B2 (en) Viewfinder control device and television camera
JPH05227463A (en) Picture processor
JP2988676B2 (en) Imaging device
US6268967B1 (en) Zoom lens apparatus
JPH11275431A (en) Image pickup device
JPH11142719A (en) Camera device
JPH01158882A (en) Method and device for adjusting focus of television camera
JP3437263B2 (en) Remote control head
JPH1034571A (en) Robot dislocation correcting device
JP2549080B2 (en) Zoom lens drive for TV camera
JPH05130481A (en) Image pickup device
JP4040121B2 (en) Pan / tilt apparatus and pan / tilt method
JP4019196B2 (en) Camera zoom position display device

Legal Events

Date Code Title Description
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20031202

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20031215

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20080109

Year of fee payment: 4

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20090109

Year of fee payment: 5

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20100109

Year of fee payment: 6

LAPS Cancellation because of no payment of annual fees