JP3672065B2 - Video collection and distribution device - Google Patents

Video collection and distribution device Download PDF

Info

Publication number
JP3672065B2
JP3672065B2 JP22079397A JP22079397A JP3672065B2 JP 3672065 B2 JP3672065 B2 JP 3672065B2 JP 22079397 A JP22079397 A JP 22079397A JP 22079397 A JP22079397 A JP 22079397A JP 3672065 B2 JP3672065 B2 JP 3672065B2
Authority
JP
Japan
Prior art keywords
image
video
video transmission
user station
transmission device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
JP22079397A
Other languages
Japanese (ja)
Other versions
JPH1153682A (en
Inventor
博幸 山川
幸夫 山本
俊宏 森
昭二 横山
裕記 石川
誠一 鈴木
敏雄 高市
Original Assignee
アイシン・エィ・ダブリュ株式会社
株式会社エクォス・リサーチ
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by アイシン・エィ・ダブリュ株式会社, 株式会社エクォス・リサーチ filed Critical アイシン・エィ・ダブリュ株式会社
Priority to JP22079397A priority Critical patent/JP3672065B2/en
Publication of JPH1153682A publication Critical patent/JPH1153682A/en
Application granted granted Critical
Publication of JP3672065B2 publication Critical patent/JP3672065B2/en
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Images

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a video collection / delivery device that transmits video .
[0002]
[Prior art]
In a monitoring system for monitoring a situation in a remote place, video captured via a video camera or the like is transmitted via a network.
[0003]
[Problems to be solved by the invention]
The applicant further advances the monitoring system, collects the status of a plurality of remote locations imaged with a video camera in the base station, and designates it via a public line in response to a request from a user station (moving vehicle) Devised a system to distribute the video of the remote location.
[0004]
Here, in such a system, even if an image is displayed on the monitor of the user station (moving vehicle) based on the transmitted data, it is difficult to immediately determine information required by the driver. For example, even if a driver who wants to know the traffic situation at a specific intersection is shown an image of the intersection, it is difficult to instantly grasp the traffic situation from the image. Further, even if an operator who wants to know the weather of a certain saddle is shown the image of the saddle, it is difficult to determine the weather from the image. In particular, it has been difficult for a driver to show an image in a moving vehicle and to understand various situations such as weather included in the image.
[0005]
SUMMARY OF THE INVENTION An object of the present invention is to provide a video collection / delivery device that transmits so that the content of information contained in a video can be immediately discriminated.
[0006]
[Means for Solving the Problems]
Claim 1 is a video collection and distribution device that collects video from a plurality of video transmission devices and distributes the collected video to a user station in order to achieve the above object ,
Image request receiving means for receiving an image request signal for requesting transmission of an image at a predetermined location from a user station;
Image requesting means for requesting a video transmission apparatus that captures the requested predetermined location to transmit an image;
Image data receiving means for receiving an image captured by the video transmission device and an output of a sensor included in the video transmission device from the video transmission device;
The graphic and the image transmitted from the video transmission device so that the graphic indicating the weather determined by the output of the sensor and the image transmitted from the video transmission device are displayed on the same screen at the user station. Transmitting means for transmitting to the user station;
And technical features in that it comprises.
[0007]
The invention of claim 2 is a video collection and distribution device that collects video from a plurality of video transmission devices and distributes the collected video to a user station,
Image request receiving means for receiving an image request signal for requesting transmission of an image at a predetermined location from a user station;
Image requesting means for requesting a video transmission apparatus that captures the requested predetermined location to transmit an image;
Image data receiving means for receiving an image captured by the video transmission device from the video transmission device;
Determining means for analyzing the received image and determining whether or not there is a traffic jam;
A transmission means for transmitting to the user station a code for allowing the user station to display a graphic corresponding to the determined result and an image sent from the video transmission device;
And technical features in that it comprises.
[0008]
According to a third aspect of the present invention, in the second aspect of the present invention, the judging means calculates the traveling speed of the vehicle based on a plurality of images sent from the video transmission device, or the number of vehicles per predetermined area in the image. It is a technical feature to determine whether or not the traffic is congested by the method of counting.
[0010]
In the invention of claim 1, together with the image of the captured requested place, by transmitting a figure indicating the weather to the user station, the receiving side, by the figure (e.g. icons) can be easily verified weather Can be displayed.
[0011]
In the invention of claim 2, a code for displaying a graphic corresponding to the result of determining whether or not there is a traffic jam is sent together with an image obtained by imaging the requested predetermined location. By displaying a corresponding figure ( icon ) or the like in the image, it can be displayed so that it can be easily determined whether or not there is a traffic jam .
[0012]
In the invention of claim 3, since the method of calculating the traveling speed of the vehicle based on the image or the method of counting the number of vehicles per predetermined area in the image is used, it is possible to determine whether or not the vehicle is congested. .
[0014]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, the video collection and delivery method according to the first embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a configuration diagram of a video collection / distribution method, and FIG. 2 is an explanatory diagram showing an overview of the video collection / distribution method of the first embodiment.
[0015]
2A shows the video transmission apparatus 10, FIG. 2B shows the base station 20 that collects video from the plurality of video transmission apparatuses 10, and FIG. The user station 30 requesting a specific video is shown in FIG. For example, the video transmission device 10 is arranged at each junction of an expressway, and the camera 12 captures a congestion state at the junction and transmits the video to the base station 20 side via the communication device 14. The communication device 14 is connected to a general public line (for example, ISDN or analog line), and is a communication device for transmitting and receiving data through this line. It is a terminal adapter. The same applies to communication devices 24A, 24B, and 36 described later. The video transmission device 10 is provided with sensors such as an illuminometer, a thermometer, and a hygrometer (not shown). On the base station 20 side, the video collection and distribution device 22 temporarily holds the video received via the communication device 24A. The user station (vehicle) 30 is provided with a communication device 36, a request issuing device 32 that requests video transfer to the base station 20, and a video display 34 that displays the transferred video. Here, the request issuing device 32 and the video display 34 can be configured using a vehicle-mounted navigation device.
[0016]
The description of the video collection and distribution method will be further continued with reference to FIG. A video transmission apparatus 10 shown in FIG. 2A is disposed at a location N1 where highway junctions and traffic jams are likely to occur, and is connected to the base station 20 via a general public line. The base station 20 is provided with a number N2 of communication devices 24A smaller than N1, and is arranged so as to receive image data from the video transmission apparatus 10.
[0017]
The video collection / distribution device 22 of the base station 20 includes a computer, a video collection unit 22a that collects image data from the video transmission device 10, a video storage unit 22b that stores the collected data, and a video storage unit 22b. A determination unit 22c that determines whether each image data stored in the image data is new, and a data request frequency of each video transmission device 10, and a distribution unit 22d that distributes the image data requested by the user station. I have. The video collection / distribution device 22 includes N3 communication devices 24B for distributing image data to a user station.
[0018]
On the other hand, as described above with reference to FIG. 2C, the user station transmits the communication device 36 such as a mobile phone, a request issuing device that requests video transfer to the base station 20, and the transferred video. A video display 34 for display is provided. There are N4 user stations. Here, the number N4 (tens of thousands) of user stations is the largest, followed by the number N3 of communication devices 24B for communication with the user stations on the base station 20 side. Next to this, the number N1 of video transmission devices 10 is large. The number N2 of communication devices 24A for communication with the video transmission device 10 of the base station 20 is the smallest.
[0019]
An outline of the video collection and distribution method will be described with reference to FIG.
FIG. 3A and FIG. 3B show display screens by the video display 34 formed of a touch panel. FIG. 3A shows a video request selection screen. In the selection screen, an outline of the city expressway and first to fifth junctions provided on the expressway are shown. Here, for example, if the driver confirms the traffic situation and weather of the first junction, is not too crowded, and if it is not raining, if the driver wants to go up to the city high speed from the first junction, Touch the first junction. As a result, an image captured by the video transmission device 10 arranged at the first junction is sent from the base station 20, and a video is displayed together with a weather icon as shown in FIG. Based on this video, the driver determines the traffic situation and weather of the first junction.
[0020]
FIG. 4 is a flowchart illustrating processing in the user station 30, the base station 20, and the video transmission device 10 in the video collection and distribution method according to the first embodiment. First, as described above with reference to FIG. 3 (A), when the driver touches the first junction in the display screen of the video display 34 composed of a touch panel and requests transfer of the image of the first junction, The user station determines whether the image transmission request is in step 12, and the user station calls the base station using the communication device 36 such as a mobile phone and sends an image request to transmit the image of the first junction. (S14).
[0021]
Here, when the base station receives an image transmission request from the user station, the determination in step 22 is Yes. First, the latest image data (for example, 5 minutes) obtained by imaging the first junction on the base station side. It is determined whether (image data captured before) is held (S24). Here, as long as the image data is not held (S24 is No), the video transmission device 10 side that images the first junction is called by the communication device 24A, and an image request is transmitted so as to transmit the image. (S26).
[0022]
When receiving an image request from the base station (Yes in S30), the video transmission apparatus 10 first obtains the output of a sensor such as the illuminometer, thermometer, hygrometer described above (S32). Next, an image obtained by capturing the first junction is captured and compressed by the camera 12 (S34). Based on the information (illuminance, humidity, temperature) acquired by the above-described sensor, first, the weather (clear, cloudy, rain, or snow) is determined, and the determined information (for example, a code indicating clear) and detection of the sensor The value (temperature, humidity) is added to the image data as a header (S36). FIG. 5A shows image data to which this information (image additional information) is added. In addition to the image additional information, a file management unit for identifying the image data is added to the image data. Finally, the video transmission apparatus 10 sends the image additional information shown in FIG. 5A and the image data to which the file management unit is added to the base station 20 side (S38).
[0023]
When the base station 20 receives the image data from the video transmission device 10 side (Yes in S40), the base station 20 transfers the received image to the user station 30 side (S42). In the determination of step 24 described above, when the latest image data obtained by imaging the first junction is held on the base station side, it is held without acquiring new image data from the video transmission device. The image additional information and the image data added by the file management unit are transferred to the user station (S42).
[0024]
When the user station receives the image additional information and the image data to which the file management unit is added from the base station 20 side (S50: Yes), the received image is displayed on the video display 34 as shown in FIG. (S52). Here, as shown in the figure corresponding to the image additional information (code indicating sunny) shown in FIG. 5A, an icon (sun figure) indicating sunny is used in the image and the superimpose function is used. To display. Here, when the first junction is cloudy, a code indicating cloudiness is transmitted, so that the cloud figure shown in the figure corresponding to this code is the umbrella figure corresponding to the code indicating rain when it is raining. In the case of snow, a snow dharma figure corresponding to a code indicating snow is displayed. For this reason, the driver can immediately determine the weather that is difficult to determine from the image alone from the graphic (icon). The weather may be notified by voice based on the image additional information.
[0025]
In the first embodiment, on the basis of image additional information (code indicating weather) added on the video transmission apparatus 10 side, a weather display icon is used in the image on the user station side using a superimpose function. However, instead of this, an image in which the weather display icons are superimposed on the base station side can be created and transmitted to the user station side.
[0026]
In the first embodiment described above, the weather or the like is determined from the detection value of the sensor on the video transmission device side. However, it is also possible to determine the weather or the like by processing the captured image, and the video transmission. It is also possible to transmit the sensor value detected on the device side to the base station side, determine the weather at the base station, and add a code for weather display at the user station. Furthermore, it is also possible to transmit the image captured on the video transmission device side to the base station side, the operator can judge the weather at the base station, and the base station side can add a code for weather display at the user station It is.
[0027]
Subsequently, a video collection and delivery method according to the second embodiment of the present invention will be described. In the first embodiment described above, the information included in the image is added on the video transmission apparatus 10 side. However, in the second embodiment, the information included in the image is determined and added on the base station side.
[0028]
FIG. 6 is a flowchart showing processing in the user station 30, the base station 20, and the video transmission device 10 in the video collection and distribution method of the second embodiment. First, when the driver requests the transfer of the image of the first junction, the determination as to whether or not the image transmission is requested in step 112 of the user station is Yes, and the user station bases the image request to transmit the image of the first junction. The data is sent to the station (S114).
[0029]
Here, when the base station receives an image transmission request from the user station, the determination in step 122 is Yes. First, is the latest image data obtained by imaging the first junction stored in the base station? Is determined (S124). Here, unless the image data is held (No in S124), an image request is transmitted so as to transmit an image to the video transmission apparatus 10 side that images the first junction (S126).
[0030]
When receiving an image request from the base station (Yes in S130), the video transmission apparatus 10 first captures and compresses an image obtained by capturing the first junction by the camera 12 (S134). Then, the compressed image data is sent to the base station 20 side (S136). As shown in FIG. 5B, a file management unit for identifying the image data is attached to the image data from the video transmission apparatus 10.
[0031]
When the base station 20 receives the image data from the video transmission device 10 side (Yes in S140), the base station 20 analyzes the content of the image to determine whether or not there is a current traffic jam, and determines the traffic jam information (indicating whether or not there is a traffic jam). Code) is added to the image data as a header (S142). Here, the traffic jam is automatically determined by image processing, but it is also possible for the operator to determine whether there is a traffic jam based on the image and add the traffic jam information. The image data to which the traffic jam information (image additional information) is added is shown in FIG. Finally, the image data including the additional information is transferred to the user station 30 side (S144). In the determination of step 124 described above, when the latest image data (for example, image data captured five minutes ago) of the first junction is held on the base station side, a new transmission is performed from the video transmission device. The image data to which the additional information is added without acquiring the image data is transferred to the user station side (S144). Here, it is determined whether or not there is a traffic jam by a method of calculating the traveling speed of the vehicle based on a plurality of images or a method of counting the number of vehicles per predetermined area in the image. Here, the operator of the base station can determine the traffic jam based on the image of each point, and can add a code indicating the presence / absence of the traffic jam.
[0032]
When the user station receives image data including additional information from the base station 20 side (S150 is Yes), the received image is displayed on the video display 34 as shown in FIG. 7A (S152). Here, as shown on the left side of the figure corresponding to the image additional information (code indicating that there is traffic jam) shown in FIG. 5C, an icon indicating that there is traffic jam is displayed in the image using the superimpose function. To do. Here, when the first junction is not congested, a code indicating no traffic jam is sent, so an icon indicating a free road as shown on the left side in FIG. 7B corresponding to the no traffic jam code. Is displayed. For this reason, the driver can immediately discriminate from the figure (icon) whether there is a traffic jam that is difficult to judge from the image alone. Note that the presence or absence of traffic congestion may be notified by voice based on the image additional information.
[0033]
Subsequently, a third embodiment of the present invention will be described. In the first and second embodiments described above, the video transmission method of the present invention is used for transmission of traffic information. In contrast, in the third embodiment, the video transmission system of the present invention is used for sightseeing guidance. That is, as shown in FIG. 8A, the video transmission device 110 is arranged so as to photograph a sightseeing spot (amusement park, temple, etc.), and the computer placed in each home as shown in FIG. 8B. A desired video transmission device 110 from (user station) 130 is called and video is transmitted. In the third embodiment, as shown in FIG. 8, an image captured by the video transmission apparatus 10 is directly sent to the user station (computer) without going through the base station.
[0034]
Processing in the video transmission apparatus 110 and the user station 130 will be described with reference to the flowchart in FIG.
When the operator operates the keyboard to request image transfer to the video transmission device 110 disposed in a desired sightseeing spot, the user station determines whether the image transmission is requested in step 212, and the user station 130 Makes a call by dialing the telephone number of the video transmission device 110 that images the sightseeing spot (here, a temple), and sends an image request to transmit the image (S214).
[0035]
When receiving the image request (Yes in S220), the video transmission apparatus 110 first obtains the output of a sensor such as an illuminometer, a thermometer, a hygrometer (S222). Next, the camera 12 captures and compresses an image of the temple (S224). Then, based on the information acquired by the above-described sensor, first, it is determined whether the weather (sunny, cloudy, rain, snow) and the current vacation are suitable, and the information (code) on the determined weather and travel suitability is used as a header. It is added to the image data (S226). Finally, the video transmission apparatus 110 sends the image data to which the code is added to the base station 20 side (S228). Here, whether or not it is suitable for a vacation is determined to be suitable for a vacation when the illuminance is high (sunny or cloudy) and the humidity is low.
[0036]
When the user station 130 receives the image data to which the code is added from the video transmission apparatus 110 side (S230 is Yes), the received image is displayed on the video display 34 as shown in FIG. 8C (S232). . Here, we use the superimpose function in the image to show the clear icon (sun figure) shown in the figure corresponding to the image additional information (code) and the clover figure that is suitable for sightseeing And display.
[0037]
Here, when suitable for sightseeing, cherry blossoms as shown in the figure can be displayed in the spring, juice can be shown in the summer, and sparrows can be shown in the fall. In the third embodiment, it is possible to immediately determine whether the weather, which is difficult to judge from an image alone, by a graphic (icon) indicating the sun and by a graphic such as a clover, is suitable for a vacation.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a video collection and delivery method according to a first embodiment of the present invention.
FIGS. 2A, 2B, and 2C are explanatory diagrams showing an overview of a video collection and delivery method according to the first embodiment of the present invention.
FIGS. 3A and 3B are explanatory diagrams showing a display screen by a video display device. FIGS.
FIG. 4 is a flowchart showing processing according to a video collection and delivery method according to the first embodiment.
FIG. 5A, FIG. 5B, and FIG. 5C are explanatory diagrams showing the format of image data.
FIG. 6 is a flowchart showing processing in the video collection and delivery method according to the second embodiment.
FIGS. 7A and 7B are explanatory diagrams of images in the video collection and distribution apparatus of the second embodiment.
8A is an explanatory diagram illustrating a configuration of a video transmission apparatus according to the third embodiment, and FIG. 8B is an explanatory diagram illustrating a configuration of a user station, and FIG. ) Is an explanatory diagram of an image.
FIG. 9 is a flowchart showing processing in the video collection and delivery method according to the third embodiment.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10 Video transmission apparatus 12 Camera 14 Communication apparatus 20 Base station 22 Video collection / distribution apparatus 24A Communication apparatus 24B Communication apparatus 30 User station 32 Request issuing apparatus 34 Video display device 36 Communication apparatus (vehicle telephone)

Claims (3)

  1. A video collection and distribution device that collects video from a plurality of video transmission devices and distributes the collected video to a user station,
      Image request receiving means for receiving an image request signal for requesting transmission of an image at a predetermined location from a user station;
      Image requesting means for requesting a video transmission apparatus that captures the requested predetermined location to transmit an image;
      Image data receiving means for receiving an image captured by the video transmission device and an output of a sensor included in the video transmission device from the video transmission device;
      The graphic and the image transmitted from the video transmission device so that the graphic indicating the weather determined by the output of the sensor and the image transmitted from the video transmission device are displayed on the same screen at the user station. Transmitting means for transmitting to the user station;
    A video collection and distribution device comprising:
  2. A video collection and distribution device that collects video from a plurality of video transmission devices and distributes the collected video to a user station,
      Image request receiving means for receiving an image request signal for requesting transmission of an image at a predetermined location from a user station;
      Image requesting means for requesting a video transmission apparatus that captures the requested predetermined location to transmit an image;
      Image data receiving means for receiving an image captured by the video transmission device from the video transmission device;
      Determining means for analyzing the received image and determining whether or not there is a traffic jam;
      A transmission means for transmitting to the user station a code for allowing the user station to display a graphic corresponding to the determined result and an image sent from the video transmission device;
    A video collection and distribution device comprising:
  3. Whether the determination means is congested by a method of calculating the traveling speed of the vehicle based on a plurality of images sent from the video transmission device or a method of counting the number of vehicles per predetermined area in the image. The video collection / distribution apparatus according to claim 2, wherein the determination is made.
JP22079397A 1997-07-31 1997-07-31 Video collection and distribution device Expired - Fee Related JP3672065B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP22079397A JP3672065B2 (en) 1997-07-31 1997-07-31 Video collection and distribution device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP22079397A JP3672065B2 (en) 1997-07-31 1997-07-31 Video collection and distribution device

Publications (2)

Publication Number Publication Date
JPH1153682A JPH1153682A (en) 1999-02-26
JP3672065B2 true JP3672065B2 (en) 2005-07-13

Family

ID=16756663

Family Applications (1)

Application Number Title Priority Date Filing Date
JP22079397A Expired - Fee Related JP3672065B2 (en) 1997-07-31 1997-07-31 Video collection and distribution device

Country Status (1)

Country Link
JP (1) JP3672065B2 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4776329B2 (en) * 2005-10-17 2011-09-21 アルパイン株式会社 Navigation device
JP4945543B2 (en) * 2008-10-31 2012-06-06 株式会社東芝 Road traffic information providing system and method
CN105096599B (en) * 2015-08-05 2017-05-10 南京感动科技有限公司 Multifunctional command and dispatch system

Also Published As

Publication number Publication date
JPH1153682A (en) 1999-02-26

Similar Documents

Publication Publication Date Title
US9918001B2 (en) Crowd sourcing exterior vehicle images of traffic conditions
ES2341629T3 (en) Instant traffic monitoring system.
US7260472B2 (en) GPS-based traffic monitoring system
JP4453796B2 (en) Map display method and system for position information display terminal
DE102012022207B3 (en) A method for providing route information by means of at least one motor vehicle
US20150149070A1 (en) System for providing traffic information
EP1357529B1 (en) A driver assist information transmitter, a driver assist information receiver, and a driver assist information providing system
JP4321128B2 (en) Image server, image collection device, and image display terminal
EP1162435B1 (en) Navigation apparatus for a vehicle
JP3941312B2 (en) Road traffic system and information processing method thereof
US6850269B2 (en) Mobile traffic camera system
DE60032969T2 (en) Traffic information system for vehicles
JP4153246B2 (en) Information provision system using real-time information
US6587777B1 (en) System and method for location based traffic reporting
KR100488746B1 (en) Data output controller and method of providing data output service
JP2005253070A (en) Apparatus and method for providing ambient parameter data and for determining weather information
EP1542031B1 (en) System and method of offering data relating to a specific location of a device to a requesting terminal
US9262916B2 (en) Server for providing traffic image to user device, and the user device
JP4287181B2 (en) Image display method
US8169505B2 (en) Image management apparatus for displaying images based on geographical environment
US8000888B2 (en) System and method for information supplying service
DE112011104739B4 (en) Information management device and information management system for mobile information terminals
US20060148488A1 (en) Method for handling location data
US20100095065A1 (en) Field device communications
US9885782B2 (en) Vehicle operator and/or occupant information apparatus and method

Legal Events

Date Code Title Description
A521 Written amendment

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20040708

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20040708

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20050224

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20050412

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20050413

R150 Certificate of patent or registration of utility model

Free format text: JAPANESE INTERMEDIATE CODE: R150

RD04 Notification of resignation of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7424

Effective date: 20050908

A072 Dismissal of procedure

Free format text: JAPANESE INTERMEDIATE CODE: A072

Effective date: 20051209

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20090428

Year of fee payment: 4

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20090428

Year of fee payment: 4

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20100428

Year of fee payment: 5

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20110428

Year of fee payment: 6

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20120428

Year of fee payment: 7

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20130428

Year of fee payment: 8

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20130428

Year of fee payment: 8

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20140428

Year of fee payment: 9

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

LAPS Cancellation because of no payment of annual fees