KR102266223B1 - 화상 처리 장치, 비행체 및 프로그램 - Google Patents

화상 처리 장치, 비행체 및 프로그램 Download PDF

Info

Publication number
KR102266223B1
KR102266223B1 KR1020207026536A KR20207026536A KR102266223B1 KR 102266223 B1 KR102266223 B1 KR 102266223B1 KR 1020207026536 A KR1020207026536 A KR 1020207026536A KR 20207026536 A KR20207026536 A KR 20207026536A KR 102266223 B1 KR102266223 B1 KR 102266223B1
Authority
KR
South Korea
Prior art keywords
image
camera
altitude
vehicle
flying object
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
KR1020207026536A
Other languages
English (en)
Korean (ko)
Other versions
KR20200115648A (ko
Inventor
아키히코 다지카
Original Assignee
소프트뱅크 가부시키가이샤
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 소프트뱅크 가부시키가이샤 filed Critical 소프트뱅크 가부시키가이샤
Publication of KR20200115648A publication Critical patent/KR20200115648A/ko
Application granted granted Critical
Publication of KR102266223B1 publication Critical patent/KR102266223B1/ko
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04B—TRANSMISSION
    • H04B7/00—Radio transmission systems, i.e. using radiation field
    • H04B7/14—Relay systems
    • H04B7/15—Active relay systems
    • H04B7/185—Space-based or airborne stations; Stations for satellite systems
    • H04B7/18502—Airborne stations
    • H04B7/18506—Communications with or from aircraft, i.e. aeronautical mobile service
    • H04B7/18508—Communications with or from aircraft, i.e. aeronautical mobile service with satellite system used as relay, i.e. aeronautical mobile satellite service
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/174—Segmentation; Edge detection involving the use of two or more images
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64C—AEROPLANES; HELICOPTERS
    • B64C27/00—Rotorcraft; Rotors peculiar thereto
    • B64C27/04—Helicopters
    • B64C27/08—Helicopters with two or more rotors
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64C—AEROPLANES; HELICOPTERS
    • B64C39/00—Aircraft not otherwise provided for
    • B64C39/02—Aircraft not otherwise provided for characterised by special use
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64C—AEROPLANES; HELICOPTERS
    • B64C39/00—Aircraft not otherwise provided for
    • B64C39/02—Aircraft not otherwise provided for characterised by special use
    • B64C39/024—Aircraft not otherwise provided for characterised by special use of the remote controlled vehicle type, i.e. RPV
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64D—EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
    • B64D47/00—Equipment not otherwise provided for
    • B64D47/08—Arrangements of cameras
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U20/00—Constructional aspects of UAVs
    • B64U20/80—Arrangement of on-board electronics, e.g. avionics systems or wiring
    • B64U20/87—Mounting of imaging devices, e.g. mounting of gimbals
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F41—WEAPONS
    • F41H—ARMOUR; ARMOURED TURRETS; ARMOURED OR ARMED VEHICLES; MEANS OF ATTACK OR DEFENCE, e.g. CAMOUFLAGE, IN GENERAL
    • F41H11/00—Defence installations; Defence devices
    • F41H11/02—Anti-aircraft or anti-guided missile or anti-torpedo defence installations or systems
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/0094—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/04—Control of altitude or depth
    • G05D1/042—Control of altitude or depth specially adapted for aircraft
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/77—Retouching; Inpainting; Scratch removal
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/593—Depth or shape recovery from multiple images from stereo images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/13—Satellite images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G06V20/17—Terrestrial scenes taken from planes or by drones
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B25/00—Alarm systems in which the location of the alarm condition is signalled to a central station, e.g. fire or police telegraphic systems
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G5/00—Traffic control systems for aircraft
    • G08G5/20—Arrangements for acquiring, generating, sharing or displaying traffic information
    • G08G5/21—Arrangements for acquiring, generating, sharing or displaying traffic information located onboard the aircraft
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G5/00—Traffic control systems for aircraft
    • G08G5/20—Arrangements for acquiring, generating, sharing or displaying traffic information
    • G08G5/22—Arrangements for acquiring, generating, sharing or displaying traffic information located on the ground
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G5/00—Traffic control systems for aircraft
    • G08G5/20—Arrangements for acquiring, generating, sharing or displaying traffic information
    • G08G5/25—Transmission of traffic-related information between aircraft
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G5/00—Traffic control systems for aircraft
    • G08G5/20—Arrangements for acquiring, generating, sharing or displaying traffic information
    • G08G5/26—Transmission of traffic-related information between aircraft and ground stations
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G5/00—Traffic control systems for aircraft
    • G08G5/70—Arrangements for monitoring traffic-related situations or conditions
    • G08G5/72—Arrangements for monitoring traffic-related situations or conditions for monitoring traffic
    • G08G5/723—Arrangements for monitoring traffic-related situations or conditions for monitoring traffic from the aircraft
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G5/00—Traffic control systems for aircraft
    • G08G5/70—Arrangements for monitoring traffic-related situations or conditions
    • G08G5/72—Arrangements for monitoring traffic-related situations or conditions for monitoring traffic
    • G08G5/727—Arrangements for monitoring traffic-related situations or conditions for monitoring traffic from a ground station
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G5/00—Traffic control systems for aircraft
    • G08G5/80—Anti-collision systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U2101/00—UAVs specially adapted for particular uses or applications
    • B64U2101/30—UAVs specially adapted for particular uses or applications for imaging, photography or videography
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U2201/00—UAVs characterised by their flight controls
    • B64U2201/20—Remote controls
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G06T2207/10012—Stereo images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G06T2207/10021—Stereoscopic video; Stereoscopic image sequence
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10032—Satellite or aerial image; Remote sensing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/08—Detecting or categorising vehicles
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/10—Recognition assisted with metadata

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Theoretical Computer Science (AREA)
  • Remote Sensing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Astronomy & Astrophysics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Signal Processing (AREA)
  • Automation & Control Theory (AREA)
  • Mechanical Engineering (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Business, Economics & Management (AREA)
  • Emergency Management (AREA)
  • General Engineering & Computer Science (AREA)
  • Traffic Control Systems (AREA)
  • Navigation (AREA)
  • Alarm Systems (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
KR1020207026536A 2018-02-20 2019-01-30 화상 처리 장치, 비행체 및 프로그램 Active KR102266223B1 (ko)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JPJP-P-2018-027905 2018-02-20
JP2018027905A JP6652979B2 (ja) 2018-02-20 2018-02-20 画像処理装置、飛行体及びプログラム
PCT/JP2019/003219 WO2019163454A1 (ja) 2018-02-20 2019-01-30 画像処理装置、飛行体及びプログラム

Publications (2)

Publication Number Publication Date
KR20200115648A KR20200115648A (ko) 2020-10-07
KR102266223B1 true KR102266223B1 (ko) 2021-06-18

Family

ID=67687232

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020207026536A Active KR102266223B1 (ko) 2018-02-20 2019-01-30 화상 처리 장치, 비행체 및 프로그램

Country Status (8)

Country Link
US (1) US11042740B2 (cg-RX-API-DMAC7.html)
EP (1) EP3757941B1 (cg-RX-API-DMAC7.html)
JP (1) JP6652979B2 (cg-RX-API-DMAC7.html)
KR (1) KR102266223B1 (cg-RX-API-DMAC7.html)
CN (1) CN111742348B (cg-RX-API-DMAC7.html)
CA (1) CA3091897C (cg-RX-API-DMAC7.html)
IL (1) IL276818B (cg-RX-API-DMAC7.html)
WO (1) WO2019163454A1 (cg-RX-API-DMAC7.html)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA3107374C (en) * 2018-07-24 2022-09-27 Tg-17, Llc Systems and methods for autonomous machine tracking and localization of mobile objects
US11532155B1 (en) * 2019-07-09 2022-12-20 ACME Atronomatic, LLC Methods and devices for earth remote sensing using stereoscopic hyperspectral imaging in the visible (VIS) and infrared (IR) bands
GB2588893A (en) * 2019-10-23 2021-05-19 Airbus Operations Ltd Aerial vehicle detection
EP4287162A4 (en) * 2021-01-29 2024-07-17 NEC Corporation CONTROL SYSTEM, FLYING BODY IDENTIFICATION METHOD, COMPUTER READABLE MEDIUM AND FLYING BODY
KR102473353B1 (ko) * 2021-04-01 2022-12-06 주식회사 켐에쎈 드론을 이용한 전신주 새집 생성 방지 시스템 및 그 방법

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008298631A (ja) * 2007-05-31 2008-12-11 Nec System Technologies Ltd 地図変化検出装置、地図変化検出方法およびプログラム
KR101252680B1 (ko) * 2012-11-28 2013-04-08 네이버시스템(주) 디지털 항공이미지를 이용한 영상도화시스템

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3652741B2 (ja) * 1995-08-31 2005-05-25 東京瓦斯株式会社 地図データ作成方法
JPH11257931A (ja) * 1998-03-13 1999-09-24 Matsushita Electric Ind Co Ltd 物体認識装置
JP2000283753A (ja) * 1999-03-31 2000-10-13 Fuji Heavy Ind Ltd ステレオ画像による測距装置
JP2001333416A (ja) * 2000-05-19 2001-11-30 Fujitsu General Ltd ネットワーク監視カメラシステム
US8837774B2 (en) * 2010-05-04 2014-09-16 Bae Systems Information Solutions Inc. Inverse stereo image matching for change detection
US8923558B2 (en) * 2012-05-15 2014-12-30 Raytheon Company Moving object detection using stereo rectified images acquired from a moving aircraft
CN102779347B (zh) * 2012-06-14 2014-08-06 清华大学 一种用于飞行器的目标跟踪与定位方法和装置
EP3121675B1 (en) * 2015-07-23 2019-10-02 The Boeing Company Method for positioning aircrafts based on analyzing images of mobile targets
WO2017073310A1 (ja) * 2015-10-27 2017-05-04 三菱電機株式会社 構造物の形状測定用の画像撮影システム、構造物の形状測定に使用する構造物の画像を撮影する方法、機上制御装置、遠隔制御装置、プログラム、および記録媒体
JP2017167870A (ja) 2016-03-17 2017-09-21 セコム株式会社 飛行物体監視システム及び飛行物体監視装置
CN205524962U (zh) * 2016-03-30 2016-08-31 深圳市高巨创新科技开发有限公司 一种组合式无人机
IL244838A0 (en) * 2016-03-30 2016-07-31 Itai Orr System and method for autonomous driving of vehicles
US11137776B2 (en) * 2016-05-16 2021-10-05 SZ DJI Technology Co., Ltd. Systems and methods for coordinating device actions
CN106598075A (zh) * 2016-07-21 2017-04-26 深圳曼塔智能科技有限公司 一种基于发光目标识别的无人机跟踪控制系统及方法
EP3444688B1 (en) * 2016-08-11 2024-10-02 Autel Robotics Co., Ltd. Method and system for tracking and identification, and aircraft
CN106444837A (zh) * 2016-10-17 2017-02-22 北京理工大学 一种无人机避障方法及系统
CN107016690B (zh) * 2017-03-06 2019-12-17 浙江大学 基于视觉的无人机入侵检测与识别系统及方法

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008298631A (ja) * 2007-05-31 2008-12-11 Nec System Technologies Ltd 地図変化検出装置、地図変化検出方法およびプログラム
KR101252680B1 (ko) * 2012-11-28 2013-04-08 네이버시스템(주) 디지털 항공이미지를 이용한 영상도화시스템

Also Published As

Publication number Publication date
KR20200115648A (ko) 2020-10-07
JP6652979B2 (ja) 2020-02-26
JP2019144804A (ja) 2019-08-29
EP3757941C0 (en) 2023-06-07
US11042740B2 (en) 2021-06-22
WO2019163454A1 (ja) 2019-08-29
US20200372238A1 (en) 2020-11-26
EP3757941B1 (en) 2023-06-07
EP3757941A4 (en) 2022-03-30
CN111742348B (zh) 2022-02-15
IL276818B (en) 2021-04-29
EP3757941A1 (en) 2020-12-30
CA3091897A1 (en) 2019-08-29
CA3091897C (en) 2021-03-09
CN111742348A (zh) 2020-10-02
IL276818A (en) 2020-10-29

Similar Documents

Publication Publication Date Title
US11042740B2 (en) Image processing device, flight vehicle, and computer-readable storage medium
EP1998139B1 (en) Building shape change detecting method, and building shape change detecting system
KR102195179B1 (ko) 항공사진을 이용한 정사영상 구축방법
KR101793509B1 (ko) 작물 모니터링을 위하여 무인 비행체의 자동 경로 산정을 통한 원격 관측 방법 및 그 시스템
KR101684293B1 (ko) 무인 비행체의 비상 착륙 지점 검출 시스템 및 그 방법
EP3283842B1 (en) Method for processing images of a ground surface
US20170221241A1 (en) System, method and apparatus for generating building maps
CN111123964B (zh) 一种无人机降落方法、设备以及计算机可读介质
CN112462791A (zh) 一种车载无人机机场全自动高精度飞行着陆系统及方法
US20140078146A1 (en) Flight obstacle extraction device, flight obstacle extraction method, and recording medium
US20190227573A1 (en) Re-search method during uav landing process
JP2020138681A (ja) 無人飛行体の制御システム
KR102679721B1 (ko) 포인트클라우드를 이용한 드론의 비행 제어 방법
CN106647785B (zh) 无人机停机坪控制方法及装置
CN109194284A (zh) 光伏电站巡检系统、地面终端及航拍设备
US9571801B2 (en) Photographing plan creation device and program and method for the same
JP2019144804A5 (cg-RX-API-DMAC7.html)
KR102349818B1 (ko) 개선된 cnn 기반으로 한 도로 균열 감지 자율 비행 드론
CN118170162A (zh) 一种配对方法
CN104735423A (zh) 位于无人机上的输电设备辨认平台
CN109163718A (zh) 一种面向建筑群的无人机自主导航方法
US7171028B2 (en) Method and apparatus for covertly determining the rate of relative motion between two objects
Tahar et al. A new approach on production of slope map using autonomous unmanned aerial vehicle
EP4481353A1 (en) Gas monitoring method, gas monitoring device, gas monitoring system, and gas monitoring program
JPH0757045B2 (ja) 送電線監視システム

Legal Events

Date Code Title Description
A201 Request for examination
A302 Request for accelerated examination
E13-X000 Pre-grant limitation requested

St.27 status event code: A-2-3-E10-E13-lim-X000

P11-X000 Amendment of application requested

St.27 status event code: A-2-2-P10-P11-nap-X000

P13-X000 Application amended

St.27 status event code: A-2-2-P10-P13-nap-X000

PA0105 International application

St.27 status event code: A-0-1-A10-A15-nap-PA0105

PA0201 Request for examination

St.27 status event code: A-1-2-D10-D11-exm-PA0201

PA0302 Request for accelerated examination

St.27 status event code: A-1-2-D10-D17-exm-PA0302

St.27 status event code: A-1-2-D10-D16-exm-PA0302

PG1501 Laying open of application

St.27 status event code: A-1-1-Q10-Q12-nap-PG1501

E902 Notification of reason for refusal
PE0902 Notice of grounds for rejection

St.27 status event code: A-1-2-D10-D21-exm-PE0902

R18-X000 Changes to party contact information recorded

St.27 status event code: A-3-3-R10-R18-oth-X000

E13-X000 Pre-grant limitation requested

St.27 status event code: A-2-3-E10-E13-lim-X000

P11-X000 Amendment of application requested

St.27 status event code: A-2-2-P10-P11-nap-X000

P13-X000 Application amended

St.27 status event code: A-2-2-P10-P13-nap-X000

E701 Decision to grant or registration of patent right
PE0701 Decision of registration

St.27 status event code: A-1-2-D10-D22-exm-PE0701

GRNT Written decision to grant
PR0701 Registration of establishment

St.27 status event code: A-2-4-F10-F11-exm-PR0701

PR1002 Payment of registration fee

St.27 status event code: A-2-2-U10-U12-oth-PR1002

Fee payment year number: 1

PG1601 Publication of registration

St.27 status event code: A-4-4-Q10-Q13-nap-PG1601

PR1001 Payment of annual fee

St.27 status event code: A-4-4-U10-U11-oth-PR1001

Fee payment year number: 4

P22-X000 Classification modified

St.27 status event code: A-4-4-P10-P22-nap-X000

P22-X000 Classification modified

St.27 status event code: A-4-4-P10-P22-nap-X000

PR1001 Payment of annual fee

St.27 status event code: A-4-4-U10-U11-oth-PR1001

Fee payment year number: 5

U11 Full renewal or maintenance fee paid

Free format text: ST27 STATUS EVENT CODE: A-4-4-U10-U11-OTH-PR1001 (AS PROVIDED BY THE NATIONAL OFFICE)

Year of fee payment: 5