CN104756155B - 合并多个地图以用于基于计算机视觉的跟踪的系统及方法 - Google Patents

合并多个地图以用于基于计算机视觉的跟踪的系统及方法 Download PDF

Info

Publication number
CN104756155B
CN104756155B CN201380056317.0A CN201380056317A CN104756155B CN 104756155 B CN104756155 B CN 104756155B CN 201380056317 A CN201380056317 A CN 201380056317A CN 104756155 B CN104756155 B CN 104756155B
Authority
CN
China
Prior art keywords
key frame
scene
diversity
module
map
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201380056317.0A
Other languages
English (en)
Chinese (zh)
Other versions
CN104756155A (zh
Inventor
M·拉马钱德兰
A·狄亚吉
S·迪亚兹斯平多拉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qualcomm Inc
Original Assignee
Qualcomm Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qualcomm Inc filed Critical Qualcomm Inc
Publication of CN104756155A publication Critical patent/CN104756155A/zh
Application granted granted Critical
Publication of CN104756155B publication Critical patent/CN104756155B/zh
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
    • G01C21/3804—Creation or updating of map data
    • G01C21/3833—Creation or updating of map data characterised by the source of data
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/20—Instruments for performing navigational calculations
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
    • G01C21/3863—Structures of map data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—Two-dimensional [2D] image generation
    • G06T11/60—Creating or editing images; Combining images with text
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/579—Depth or shape recovery from multiple images from motion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/255—Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/28—Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Theoretical Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
CN201380056317.0A 2012-10-31 2013-10-08 合并多个地图以用于基于计算机视觉的跟踪的系统及方法 Expired - Fee Related CN104756155B (zh)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US201261720804P 2012-10-31 2012-10-31
US61/720,804 2012-10-31
US13/830,664 2013-03-14
US13/830,664 US9177404B2 (en) 2012-10-31 2013-03-14 Systems and methods of merging multiple maps for computer vision based tracking
PCT/US2013/063876 WO2014070390A1 (en) 2012-10-31 2013-10-08 Systems and methods of merging multiple maps for computer vision based tracking

Publications (2)

Publication Number Publication Date
CN104756155A CN104756155A (zh) 2015-07-01
CN104756155B true CN104756155B (zh) 2017-09-19

Family

ID=50547228

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201380056317.0A Expired - Fee Related CN104756155B (zh) 2012-10-31 2013-10-08 合并多个地图以用于基于计算机视觉的跟踪的系统及方法

Country Status (6)

Country Link
US (1) US9177404B2 (enExample)
EP (1) EP2915138B1 (enExample)
JP (1) JP6339579B2 (enExample)
KR (1) KR20150079730A (enExample)
CN (1) CN104756155B (enExample)
WO (1) WO2014070390A1 (enExample)

Families Citing this family (84)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010045271A1 (en) * 2008-10-14 2010-04-22 Joshua Victor Aller Target and method of detecting, identifying, and determining 3-d pose of the target
US9367811B2 (en) * 2013-03-15 2016-06-14 Qualcomm Incorporated Context aware localization, mapping, and tracking
US10262462B2 (en) 2014-04-18 2019-04-16 Magic Leap, Inc. Systems and methods for augmented and virtual reality
KR101574876B1 (ko) * 2014-02-13 2015-12-04 영남대학교 산학협력단 비전 센서 데이터베이스를 이용한 거리 측정 방법
US9478029B2 (en) * 2014-10-23 2016-10-25 Qualcomm Incorporated Selection strategy for exchanging map information in collaborative multi-user SLAM systems
CA2970985C (en) * 2014-12-18 2021-10-12 Innerspace Technology Inc. Wayfinding system for interior spaces using an auto-generated navigational map
US10185775B2 (en) * 2014-12-19 2019-01-22 Qualcomm Technologies, Inc. Scalable 3D mapping system
EP3062142B1 (en) 2015-02-26 2018-10-03 Nokia Technologies OY Apparatus for a near-eye display
US10838207B2 (en) 2015-03-05 2020-11-17 Magic Leap, Inc. Systems and methods for augmented reality
CN112764536B (zh) 2015-03-05 2025-04-01 奇跃公司 用于增强现实的系统和方法
US9811734B2 (en) * 2015-05-11 2017-11-07 Google Inc. Crowd-sourced creation and updating of area description file for mobile device localization
US9940542B2 (en) * 2015-08-11 2018-04-10 Google Llc Managing feature data for environment mapping on an electronic device
JP2018536244A (ja) 2015-12-04 2018-12-06 マジック リープ， インコーポレイテッドＭａｇｉｃ Ｌｅａｐ，Ｉｎｃ． 再位置特定システムおよび方法
JP6842618B2 (ja) * 2015-12-14 2021-03-17 インターナショナル・ビジネス・マシーンズ・コーポレーションＩｎｔｅｒｎａｔｉｏｎａｌ Ｂｕｓｉｎｅｓｓ Ｍａｃｈｉｎｅｓ Ｃｏｒｐｏｒａｔｉｏｎ ３ｄマップの作成
CN105913459B (zh) * 2016-05-10 2019-07-12 中国科学院自动化研究所 基于高分辨率连拍图像的运动目标检测方法
US10890600B2 (en) 2016-05-18 2021-01-12 Google Llc Real-time visual-inertial motion tracking fault detection
US10802147B2 (en) 2016-05-18 2020-10-13 Google Llc System and method for concurrent odometry and mapping
US11017610B2 (en) * 2016-05-18 2021-05-25 Google Llc System and method for fault detection and recovery for concurrent odometry and mapping
US10217231B2 (en) * 2016-05-31 2019-02-26 Microsoft Technology Licensing, Llc Systems and methods for utilizing anchor graphs in mixed reality environments
IL264489B (en) 2016-08-02 2022-07-01 Magic Leap Inc Virtual and augmented reality systems at a fixed distance and methods
US11430070B1 (en) 2017-07-31 2022-08-30 Block, Inc. Intelligent application of reserves to transactions
US10650552B2 (en) 2016-12-29 2020-05-12 Magic Leap, Inc. Systems and methods for augmented reality
EP4300160A3 (en) 2016-12-30 2024-05-29 Magic Leap, Inc. Polychromatic light out-coupling apparatus, near-eye displays comprising the same, and method of out-coupling polychromatic light
US10812936B2 (en) * 2017-01-23 2020-10-20 Magic Leap, Inc. Localization determination for mixed reality systems
US10534964B2 (en) * 2017-01-30 2020-01-14 Blackberry Limited Persistent feature descriptors for video
US10217232B2 (en) 2017-02-08 2019-02-26 Toyota Motor Engineering & Manufacturing North America, Inc. Systems and methods for locally aligning map data
CN111309955B (zh) * 2017-02-13 2022-06-24 哈尔滨理工大学 一种面向图像检索的融合方法
US10769752B2 (en) 2017-03-17 2020-09-08 Magic Leap, Inc. Mixed reality system with virtual content warping and method of generating virtual content using same
IL303275B2 (en) 2017-03-17 2024-08-01 Magic Leap Inc A mixed reality system with the assembly of multi-source virtual content and a method for creating virtual content using it
KR102366781B1 (ko) 2017-03-17 2022-02-22 매직 립, 인코포레이티드 컬러 가상 콘텐츠 와핑을 갖는 혼합 현실 시스템 및 이를 사용하여 가상 콘텐츠를 생성하는 방법
US10990829B2 (en) * 2017-04-28 2021-04-27 Micro Focus Llc Stitching maps generated using simultaneous localization and mapping
US10578870B2 (en) 2017-07-26 2020-03-03 Magic Leap, Inc. Exit pupil expander
KR20250138816A (ko) 2017-12-10 2025-09-22 매직 립, 인코포레이티드 광학 도파관들 상의 반사―방지 코팅들
KR102761610B1 (ko) 2017-12-20 2025-01-31 매직 립, 인코포레이티드 증강 현실 뷰잉 디바이스용 인서트
CN110099237B (zh) * 2018-01-31 2021-08-17 腾讯科技（深圳）有限公司 图像处理方法、电子装置及计算机可读存储介质
US10755676B2 (en) 2018-03-15 2020-08-25 Magic Leap, Inc. Image correction due to deformation of components of a viewing device
EP3803488A4 (en) 2018-05-30 2021-07-28 Magic Leap, Inc. COMPACT VARIABLE FOCUS CONFIGURATIONS
US11885871B2 (en) 2018-05-31 2024-01-30 Magic Leap, Inc. Radar head pose localization
EP3804306B1 (en) 2018-06-05 2023-12-27 Magic Leap, Inc. Homography transformation matrices based temperature calibration of a viewing system
EP3803545B1 (en) 2018-06-08 2025-02-26 Magic Leap, Inc. Augmented reality viewer with automated surface selection placement and content orientation placement
US10549186B2 (en) * 2018-06-26 2020-02-04 Sony Interactive Entertainment Inc. Multipoint SLAM capture
WO2020010097A1 (en) 2018-07-02 2020-01-09 Magic Leap, Inc. Pixel intensity modulation using modifying gain values
CN109074757B (zh) * 2018-07-03 2021-11-09 达闼机器人有限公司 一种建立地图的方法、终端和计算机可读存储介质
US11856479B2 (en) 2018-07-03 2023-12-26 Magic Leap, Inc. Systems and methods for virtual and augmented reality along a route with markers
WO2020010226A1 (en) 2018-07-03 2020-01-09 Magic Leap, Inc. Systems and methods for virtual and augmented reality
JP7374981B2 (ja) 2018-07-10 2023-11-07 マジック リープ， インコーポレイテッド クロス命令セットアーキテクチャプロシージャ呼出のためのスレッドウィービング
US11307049B2 (en) 2018-07-19 2022-04-19 Uisee Technologies (Beijing) Co., Ltd Methods, apparatuses, systems, and storage media for storing and loading visual localization maps
EP3827299B1 (en) 2018-07-23 2025-07-09 Magic Leap, Inc. Mixed reality system with virtual content warping and method of generating virtual content using same
US11598651B2 (en) 2018-07-24 2023-03-07 Magic Leap, Inc. Temperature dependent calibration of movement detection devices
US11624929B2 (en) 2018-07-24 2023-04-11 Magic Leap, Inc. Viewing device with dust seal integration
JP7139762B2 (ja) * 2018-07-31 2022-09-21 カシオ計算機株式会社 自律移動装置、自律移動方法及びプログラム
EP4650919A3 (en) 2018-08-02 2026-01-21 Magic Leap, Inc. A viewing system with interpupillary distance compensation based on head motion
EP3830631A4 (en) 2018-08-03 2021-10-27 Magic Leap, Inc. DRIFT CORRECTION BASED ON UNMERGED LAYING OF MELTED LAYING OF A TOTEM IN A USER INTERACTION SYSTEM
CN112955073B (zh) 2018-08-22 2025-01-28 奇跃公司 患者观察系统
JP7182976B2 (ja) * 2018-09-27 2022-12-05 キヤノン株式会社 情報処理装置、情報処理方法、およびプログラム
WO2020076822A1 (en) 2018-10-12 2020-04-16 Magic Leap, Inc. Staging system to verify accuracy of a motion tracking system
CN113196138B (zh) 2018-11-16 2023-08-25 奇跃公司 用于保持图像清晰度的图像尺寸触发的澄清
CN113454507B (zh) 2018-12-21 2024-05-07 奇跃公司 用于促进波导内的全内反射的气穴结构
CN109754385A (zh) * 2019-01-11 2019-05-14 中南大学 未配准多聚焦图像的快速融合方法
JP7515489B2 (ja) 2019-02-06 2024-07-12 マジック リープ， インコーポレイテッド 複数のプロセッサによって発生される総熱を限定するための標的意図ベースのクロック速度の決定および調節
EP3939030B1 (en) 2019-03-12 2025-10-29 Magic Leap, Inc. Registration of local content between first and second augmented reality viewers
CN109949412B (zh) * 2019-03-26 2021-03-02 腾讯科技（深圳）有限公司 一种三维对象重建方法和装置
WO2020223636A1 (en) 2019-05-01 2020-11-05 Magic Leap, Inc. Content provisioning system and method
WO2021021670A1 (en) 2019-07-26 2021-02-04 Magic Leap, Inc. Systems and methods for augmented reality
WO2021097318A1 (en) 2019-11-14 2021-05-20 Magic Leap, Inc. Systems and methods for virtual and augmented reality
EP4058979A4 (en) 2019-11-15 2023-01-11 Magic Leap, Inc. VISUALIZATION SYSTEM FOR USE IN A SURGICAL ENVIRONMENT
CN111241944B (zh) * 2019-12-31 2023-05-26 浙江大学 基于背景目标与背景特征匹配的场景识别与回环检测方法
CN111402288A (zh) * 2020-03-26 2020-07-10 杭州博雅鸿图视频技术有限公司 目标检测跟踪方法及装置
WO2022049615A1 (ja) * 2020-09-01 2022-03-10 株式会社ネイン 情報処理システム、情報処理方法およびコンピュータプログラム
CN112541970A (zh) * 2020-11-30 2021-03-23 北京华捷艾米科技有限公司 一种集中式协同SlAM中的重定位方法及装置
TWI779592B (zh) * 2021-05-05 2022-10-01 萬潤科技股份有限公司 地圖編輯方法及裝置
KR102431122B1 (ko) 2021-12-30 2022-08-10 주식회사 버넥트 맵 타겟 추적 방법 및 그 시스템
US12260566B2 (en) * 2022-04-07 2025-03-25 Htc Corporation Method for managing tracking devices, device manager, and computer readable storage medium
US12535336B2 (en) 2022-05-05 2026-01-27 Here Global B.V. Method, apparatus, and computer program product for map geometry generation based on data aggregation and conflation
US12657933B2 (en) 2022-05-05 2026-06-16 Here Global B.V. Method, apparatus, and computer program product for map data generation from probe data imagery
US12281916B2 (en) 2022-05-05 2025-04-22 Here Global B.V. Method, apparatus, and computer program product for map geometry generation based on data aggregation and conflation with statistical analysis
US12292308B2 (en) 2022-05-05 2025-05-06 Here Global B.V. Method, apparatus, and computer program product for map geometry generation based on object detection
US12287225B2 (en) 2022-05-05 2025-04-29 Here Global B.V. Method, apparatus, and computer program product for lane geometry generation based on graph estimation
US12546626B2 (en) 2022-05-05 2026-02-10 Here Global B.V. Method, apparatus, and computer program product for probe data-based geometry generation
CN117555473A (zh) * 2022-08-03 2024-02-13 追觅创新科技(苏州)有限公司 图像存储方法、设备及存储介质
CN115376051B (zh) * 2022-10-25 2023-03-24 杭州华橙软件技术有限公司 关键帧的管理方法及管理装置、slam方法、电子设备
US12675904B2 (en) 2023-12-14 2026-07-07 Htc Corporation Spatial map information generating apparatus, method, and coordinate system integrating apparatus
CN117930279B (zh) * 2024-03-11 2025-03-07 南京森思科技有限责任公司 一种多尺度自适应激光雷达slam方法和装置
US12620129B2 (en) * 2024-05-21 2026-05-05 Snap Inc Pose quantization-based keyframe pruning for simultaneous localization and mapping

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090190857A1 (en) * 2008-01-28 2009-07-30 Microsoft Corporation Importance guided image transformation
US20090310851A1 (en) * 2008-06-12 2009-12-17 Microsoft Corporation 3d content aggregation built into devices
WO2011067713A2 (en) * 2009-12-01 2011-06-09 Rafael Advanced Defense Systems Ltd. Method and system of generating a three-dimensional view of a real scene for military planning and operations
US20110173565A1 (en) * 2010-01-12 2011-07-14 Microsoft Corporation Viewing media in the context of street-level images
CN102541418A (zh) * 2010-12-30 2012-07-04 上海博泰悦臻电子设备制造有限公司 一种实现地图显示的处理方法及装置

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1167120B1 (en) 2000-06-30 2014-08-27 Panasonic Corporation Rendering device for parking aid
EP1924981B1 (en) 2005-07-26 2012-09-12 MacDonald Dettwiler & Associates Inc. Traffic management system for a passageway environment
JP4400659B2 (ja) 2007-09-03 2010-01-20 トヨタ自動車株式会社 車載表示装置
JP4803155B2 (ja) * 2007-10-15 2011-10-26 セイコーエプソン株式会社 複数の画像データのパノラマ合成
CN102308320B (zh) * 2009-02-06 2013-05-29 香港科技大学 从图像生成三维模型
US20120195491A1 (en) 2010-07-21 2012-08-02 Palo Alto Research Center Incorporated System And Method For Real-Time Mapping Of An Indoor Environment Using Mobile Robots With Limited Sensing
KR101732076B1 (ko) 2010-12-22 2017-05-02 엘지전자 주식회사 전자 기기 및 전자 기기의 제어 방법
US9247133B2 (en) * 2011-06-01 2016-01-26 Apple Inc. Image registration using sliding registration windows
US20130293671A1 (en) * 2012-05-01 2013-11-07 Tourwrist, Inc. Systems and methods for stitching and sharing panoramas

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090190857A1 (en) * 2008-01-28 2009-07-30 Microsoft Corporation Importance guided image transformation
US20090310851A1 (en) * 2008-06-12 2009-12-17 Microsoft Corporation 3d content aggregation built into devices
WO2011067713A2 (en) * 2009-12-01 2011-06-09 Rafael Advanced Defense Systems Ltd. Method and system of generating a three-dimensional view of a real scene for military planning and operations
US20110173565A1 (en) * 2010-01-12 2011-07-14 Microsoft Corporation Viewing media in the context of street-level images
CN102541418A (zh) * 2010-12-30 2012-07-04 上海博泰悦臻电子设备制造有限公司 一种实现地图显示的处理方法及装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Collaborative Structure from Motion;Christian Forster;《Master Thesis of Swiss Federal Institute of Technology Zurich》;20120430;第28页第3-6步 *

Also Published As

Publication number Publication date
KR20150079730A (ko) 2015-07-08
CN104756155A (zh) 2015-07-01
EP2915138B1 (en) 2019-07-31
JP6339579B2 (ja) 2018-06-06
JP2016500885A (ja) 2016-01-14
WO2014070390A1 (en) 2014-05-08
US20140119598A1 (en) 2014-05-01
EP2915138A1 (en) 2015-09-09
US9177404B2 (en) 2015-11-03

Similar Documents

Publication Publication Date Title
US9177404B2 (en) Systems and methods of merging multiple maps for computer vision based tracking
US9330471B2 (en) Camera aided motion direction and speed estimation
CN104487915B (zh) 维持扩增的连续性
US11727661B2 (en) Method and system for determining at least one property related to at least part of a real environment
JP5940453B2 (ja) 画像のシーケンス内のオブジェクトのリアルタイム表現のハイブリッド型追跡のための方法、コンピュータプログラム、および装置
JP6430064B2 (ja) データを位置合わせする方法及びシステム
JP5722502B2 (ja) モバイルデバイスのための平面マッピングおよびトラッキング
WO2011061709A1 (en) Method and apparatus for tracking and recognition with rotation invariant feature descriptors
CN109272577B (zh) 一种基于Kinect的视觉SLAM方法
WO2016054779A1 (en) Spatial pyramid pooling networks for image processing
KR20150082417A (ko) 병렬화가능한 구조에서의 이미지들을 사용하여 표면 엘리먼트들의 로컬 지오메트리 또는 표면 법선들을 초기화 및 솔빙하는 방법
Ekekrantz et al. Adaptive iterative closest keypoint
US20160066150A1 (en) Dynamic Configuration of a Positioning System
CN115131691A (zh) 对象匹配方法、装置、电子设备及计算机可读存储介质
Klüssendorff et al. Graph-based visual SLAM and visual odometry using an RGB-D camera
JP2014127068A (ja) 対応点探索装置およびカメラ姿勢推定装置ならびにこれらのプログラム
Knoblauch et al. Non-parametric sequential frame decimation for scene reconstruction in low-memory streaming environments
Bao et al. Fast 2D-to-3D matching with camera pose voting for 3D object identification
CA3251377A1 (en) SYSTEM AND METHOD FOR UNDERSTANDING OBJECTS
CN120374718A (zh) 基于特征点匹配的双目相机目标定位与测距方法及系统
CN112927291A (zh) 三维物体的位姿确定方法、装置及电子设备和存储介质
Donate et al. Three dimensional Information Extraction and applications to video analysis

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20170919

Termination date: 20211008

CF01 Termination of patent right due to non-payment of annual fee