CA3184001A1 - Multi-agent map generation - Google Patents

Multi-agent map generation Download PDF

Info

Publication number
CA3184001A1
CA3184001A1 CA3184001A CA3184001A CA3184001A1 CA 3184001 A1 CA3184001 A1 CA 3184001A1 CA 3184001 A CA3184001 A CA 3184001A CA 3184001 A CA3184001 A CA 3184001A CA 3184001 A1 CA3184001 A1 CA 3184001A1
Authority
CA
Canada
Prior art keywords
nodes
frames
edges
graph
processing devices
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CA3184001A
Other languages
English (en)
French (fr)
Inventor
Gavin CATT
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Commonwealth Scientific and Industrial Research Organization CSIRO
Original Assignee
Commonwealth Scientific and Industrial Research Organization CSIRO
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from AU2020903029A external-priority patent/AU2020903029A0/en
Application filed by Commonwealth Scientific and Industrial Research Organization CSIRO filed Critical Commonwealth Scientific and Industrial Research Organization CSIRO
Publication of CA3184001A1 publication Critical patent/CA3184001A1/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G06T17/05—Geographic models
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
    • G01C21/3804—Creation or updating of map data
    • G01C21/3833—Creation or updating of map data characterised by the source of data
    • G01C21/3841—Data obtained from two or more sources, e.g. probe vehicles
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/005—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 with correlation of navigation data from several sources, e.g. map or contour matching
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/20—Instruments for performing navigational calculations
    • G01C21/206—Instruments for performing navigational calculations specially adapted for indoor navigation
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/28—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network with correlation of data from several navigational instruments
    • G01C21/30—Map- or contour-matching
    • G01C21/32—Structuring or formatting of map data
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
    • G01C21/3804—Creation or updating of map data
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
    • G01C21/3804—Creation or updating of map data
    • G01C21/3833—Creation or updating of map data characterised by the source of data
    • G01C21/3848—Data obtained from both position sensors and additional sensors
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/38—Electronic maps specially adapted for navigation; Updating thereof
    • G01C21/3863—Structures of map data
    • G01C21/3867—Geometry of map features, e.g. shape points, polygons or for simplified maps
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/86—Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G01S17/894—Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/20—Control system inputs
    • G05D1/24—Arrangements for determining position or orientation
    • G05D1/246—Arrangements for determining position or orientation using environment maps, e.g. simultaneous localisation and mapping [SLAM]
    • G05D1/2465—Arrangements for determining position or orientation using environment maps, e.g. simultaneous localisation and mapping [SLAM] using a 3D model of the environment
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/40—Control within particular dimensions
    • G05D1/43—Control of position or course in two dimensions [2D]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—Two-dimensional [2D] image generation
    • G06T11/20—Drawing from basic elements
    • G06T11/26—Drawing of charts or graphs
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
    • G06V10/7625—Hierarchical techniques, i.e. dividing or merging patterns to obtain a tree-like representation; Dendograms
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
    • B64U2101/00—UAVs specially adapted for particular uses or applications
    • B64U2101/30—UAVs specially adapted for particular uses or applications for imaging, photography or videography
    • B64U2101/32—UAVs specially adapted for particular uses or applications for imaging, photography or videography for cartography or topography
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D2107/00—Specific environments of the controlled vehicles
    • G05D2107/50—Confined spaces, e.g. tanks, pipelines, tunnels or containers
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D2107/00—Specific environments of the controlled vehicles
    • G05D2107/70—Industrial sites, e.g. warehouses or factories
    • G05D2107/73—Mining
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D2111/00—Details of signals used for control of position, course, altitude or attitude of land, water, air or space vehicles
    • G05D2111/10—Optical signals
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/29—Graphical models, e.g. Bayesian networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—Two-dimensional [2D] image generation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Landscapes

  • Engineering & Computer Science (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • Geometry (AREA)
  • Electromagnetism (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computer Graphics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Artificial Intelligence (AREA)
  • Image Analysis (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
  • Instructional Devices (AREA)
  • Navigation (AREA)
  • User Interface Of Digital Computer (AREA)
  • Traffic Control Systems (AREA)
CA3184001A 2020-08-25 2021-08-09 Multi-agent map generation Pending CA3184001A1 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
AU2020903029 2020-08-25
AU2020903029A AU2020903029A0 (en) 2020-08-25 Multi-agent map generation
PCT/AU2021/050871 WO2022040723A1 (en) 2020-08-25 2021-08-09 Multi-agent map generation

Publications (1)

Publication Number Publication Date
CA3184001A1 true CA3184001A1 (en) 2022-03-03

Family

ID=80352211

Family Applications (1)

Application Number Title Priority Date Filing Date
CA3184001A Pending CA3184001A1 (en) 2020-08-25 2021-08-09 Multi-agent map generation

Country Status (8)

Country Link
US (1) US11898869B2 (https=)
EP (1) EP4205084A4 (https=)
JP (1) JP2023538946A (https=)
KR (1) KR20230056690A (https=)
CN (1) CN115836325A (https=)
AU (1) AU2021273605B2 (https=)
CA (1) CA3184001A1 (https=)
WO (1) WO2022040723A1 (https=)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116630901B (zh) * 2023-07-24 2023-09-22 南京师范大学 一种基于潜在图预测无监督学习框架的视觉里程计方法
WO2026000065A1 (en) * 2024-06-28 2026-01-02 Leddartech Inc. Vehicle environment map methods and systems
KR102816670B1 (ko) * 2025-01-22 2025-06-05 주식회사 큐팁 에이전트 그래프의 사용을 지원하기 위한 방법 및 시스템

Family Cites Families (38)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007156016A (ja) * 2005-12-02 2007-06-21 Sony Corp 通信装置、データ処理方法、プログラムおよび通信方法
JP2009193240A (ja) * 2008-02-13 2009-08-27 Toyota Motor Corp 移動ロボット及び環境地図の生成方法
CN102270305B (zh) * 2011-08-11 2013-04-24 西北工业大学 一种基于msbn的多智能体协同目标识别方法
US8798840B2 (en) * 2011-09-30 2014-08-05 Irobot Corporation Adaptive mapping with spatial summaries of sensor data
KR101913332B1 (ko) * 2011-12-23 2018-10-31 삼성전자주식회사 이동 장치 및 이동 장치의 위치 인식 방법
WO2014076844A1 (ja) * 2012-11-19 2014-05-22 株式会社日立製作所 自律移動システムおよび管制装置
US9075415B2 (en) * 2013-03-11 2015-07-07 Airphrame, Inc. Unmanned aerial vehicle and methods for controlling same
US20150091909A1 (en) * 2013-09-30 2015-04-02 Alcatel Lucent Systems And Methods For Distance Approximation In Graphs
US20150193971A1 (en) * 2014-01-03 2015-07-09 Motorola Mobility Llc Methods and Systems for Generating a Map including Sparse and Dense Mapping Information
US9342888B2 (en) * 2014-02-08 2016-05-17 Honda Motor Co., Ltd. System and method for mapping, localization and pose correction of a vehicle based on images
CN104374395A (zh) * 2014-03-31 2015-02-25 南京邮电大学 基于图的视觉slam方法
CN104732518B (zh) * 2015-01-19 2017-09-01 北京工业大学 一种基于智能机器人地面特征的ptam改进方法
AU2016275179B2 (en) * 2015-06-11 2021-10-07 Queen's University At Kingston Automated mobile geotechnical mapping
JP6649743B2 (ja) * 2015-10-28 2020-02-19 シャープ株式会社 一致性評価装置および一致性評価方法
US10788836B2 (en) 2016-02-29 2020-09-29 AI Incorporated Obstacle recognition method for autonomous robots
CN106097304B (zh) * 2016-05-31 2019-04-23 西北工业大学 一种无人机实时在线地图生成方法
EP3497405B1 (en) * 2016-08-09 2022-06-15 Nauto, Inc. System and method for precision localization and mapping
US10274325B2 (en) * 2016-11-01 2019-04-30 Brain Corporation Systems and methods for robotic mapping
CN106371445B (zh) * 2016-11-17 2019-05-14 浙江大学 一种基于拓扑地图的无人车规划控制方法
EP3324209A1 (en) * 2016-11-18 2018-05-23 Dibotics Methods and systems for vehicle environment map generation and updating
SG10201700299QA (en) * 2017-01-13 2018-08-30 Otsaw Digital Pte Ltd Three-dimensional mapping of an environment
US10467799B2 (en) * 2017-03-09 2019-11-05 Houzz, Inc. Dynamically modeling an object in an environment from different perspectives
GB201706129D0 (en) * 2017-04-18 2017-05-31 Blue Vision Labs Uk Ltd Distributed device mapping
US10222215B2 (en) * 2017-04-21 2019-03-05 X Development Llc Methods and systems for map generation and alignment
JP2018189457A (ja) * 2017-05-01 2018-11-29 パイオニア株式会社 情報処理装置
WO2019032684A1 (en) * 2017-08-08 2019-02-14 Diversey, Inc. CREATING AND LOADING MAPPING DATA ON AUTONOMOUS ROBOTIC DEVICES
US20190332114A1 (en) * 2018-04-26 2019-10-31 Maidbot, Inc. Robot Contextualization of Map Regions
JP7047576B2 (ja) * 2018-04-27 2022-04-05 株式会社豊田中央研究所 地図作成装置
JP2020004342A (ja) * 2018-07-02 2020-01-09 株式会社Ｓｏｋｅｎ 移動体制御装置
US10948297B2 (en) * 2018-07-09 2021-03-16 Samsung Electronics Co., Ltd. Simultaneous location and mapping (SLAM) using dual event cameras
GB2576322B (en) * 2018-08-13 2022-11-09 Imperial College Innovations Ltd Mapping object instances using video data
CN109559277B (zh) 2018-11-28 2023-02-28 中国人民解放军国防科技大学 一种面向数据共享的多无人机协同地图构建方法
JP6674572B1 (ja) * 2019-03-01 2020-04-01 三菱ロジスネクスト株式会社 Ｓｌａｍ誘導方式の無人作業車および無人作業システム
CN110531759B (zh) * 2019-08-02 2020-09-22 深圳大学 机器人探索路径生成方法、装置、计算机设备和存储介质
CN110580740B (zh) * 2019-08-27 2021-08-20 清华大学 多智能体协同三维建模方法及装置
CN111340870B (zh) * 2020-01-15 2022-04-01 西安交通大学 基于视觉的拓扑地图生成方法
CN111368759B (zh) * 2020-03-09 2022-08-30 河海大学常州校区 基于单目视觉的移动机器人语义地图构建系统
US11657719B2 (en) * 2020-12-18 2023-05-23 Zoox, Inc. System for sparsely representing and storing geographic and map data

Also Published As

Publication number Publication date
AU2021273605A1 (en) 2022-03-17
AU2021273605B2 (en) 2022-03-31
JP2023538946A (ja) 2023-09-12
WO2022040723A1 (en) 2022-03-03
EP4205084A1 (en) 2023-07-05
CN115836325A (zh) 2023-03-21
US11898869B2 (en) 2024-02-13
EP4205084A4 (en) 2025-05-21
US20230175860A1 (en) 2023-06-08
KR20230056690A (ko) 2023-04-27

Similar Documents

Publication Publication Date Title
EP3660618B1 (en) Map building and positioning of robot
CN110531759B (zh) 机器人探索路径生成方法、装置、计算机设备和存储介质
JP6987797B2 (ja) リアルタイムオンラインエゴモーション推定を有するレーザスキャナ
AU2021273605B2 (en) Multi-agent map generation
TWI772743B (zh) 資訊處理裝置以及移動機器人
KR20190131402A (ko) 카메라와 라이다를 이용한 융합 센서 및 이동체
Bosse ATLAS: a framework for large scale automated mapping and localization
Valencia et al. Mapping, planning and exploration with Pose SLAM
Indelman et al. Incremental distributed inference from arbitrary poses and unknown data association: Using collaborating robots to establish a common reference
Nieto et al. Denseslam: Simultaneous localization and dense mapping
US20230071794A1 (en) Method and system for building lane-level map by using 3D point cloud map
CN120558230A (zh) 工业视觉辅助下的救援机器人路径规划方法及系统
CN120252755A (zh) 一种巡检路径动态规划方法、介质及设备
CN119414846A (zh) 清洁机器人环境障碍物标记方法及系统、设备及介质
CN119124128A (zh) 面向复杂环境的智能服务机器人定位导航系统
CN116126022A (zh) 巡检无人机的控制方法、控制终端以及可读存储介质
CN113841068B (zh) 信息处理装置、信息处理方法及存储介质
US20240027224A1 (en) Method for recognizing an erroneous map of an environment
Burger et al. Map-aware slam with sparse map features
Tamjidi et al. 6-DOF pose estimation of an autonomous car by visual feature correspondence and tracking
Tungadi et al. Loop exploration for SLAM with fusion of advanced sonar features and laser polar scan matching
Chen et al. Navigation method of fire-fighting robot based on a commercial electronic map
Chai et al. Topological semantic map construction and localization for intelligent vehicles in underground parking lots
Zhan et al. Semantic Exploration and Dense Mapping of Complex Environments using Ground Robot with Panoramic LiDAR-Camera Fusion
Kleiner et al. Mapping for the support of first responders in critical domains

Legal Events

Date Code Title Description
MFA Maintenance fee for application paid

Free format text: FEE DESCRIPTION TEXT: MF (APPLICATION, 3RD ANNIV.) - STANDARD

Year of fee payment: 3

U00 Fee paid

Free format text: ST27 STATUS EVENT CODE: A-1-1-U10-U00-U101 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE REQUEST RECEIVED

Effective date: 20240729

U11 Full renewal or maintenance fee paid

Free format text: ST27 STATUS EVENT CODE: A-1-1-U10-U11-U102 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE FEE PAYMENT DETERMINED COMPLIANT

Effective date: 20240729

Free format text: ST27 STATUS EVENT CODE: A-1-1-U10-U11-U102 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE FEE PAYMENT PAID IN FULL

Effective date: 20240729

R00 Party data change recorded

Free format text: ST27 STATUS EVENT CODE: A-1-1-R10-R00-R116 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: APPOINTMENT OF AGENT REQUEST

Effective date: 20250116

Free format text: ST27 STATUS EVENT CODE: A-1-1-R10-R00-R119 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: REVOCATION OF AGENT REQUEST

Effective date: 20250116

W00 Other event occurred

Free format text: ST27 STATUS EVENT CODE: A-1-1-W10-W00-W111 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: CORRESPONDENT DETERMINED COMPLIANT

Effective date: 20250116

MFA Maintenance fee for application paid

Free format text: FEE DESCRIPTION TEXT: MF (APPLICATION, 4TH ANNIV.) - STANDARD

Year of fee payment: 4

U00 Fee paid

Free format text: ST27 STATUS EVENT CODE: A-1-1-U10-U00-U101 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE REQUEST RECEIVED

Effective date: 20250729

R17 Change to representative recorded

Free format text: ST27 STATUS EVENT CODE: A-1-1-R10-R17-R121 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: REVOCATION OF AGENT REQUIREMENTS DETERMINED COMPLIANT

Effective date: 20250804

Free format text: ST27 STATUS EVENT CODE: A-1-1-R10-R17-R117 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: APPOINTMENT OF AGENT REQUIREMENTS DETERMINED COMPLIANT

Effective date: 20250804

W00 Other event occurred

Free format text: ST27 STATUS EVENT CODE: A-1-1-W10-W00-W100 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: LETTER SENT

Effective date: 20250804

U11 Full renewal or maintenance fee paid

Free format text: ST27 STATUS EVENT CODE: A-1-1-U10-U11-U102 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE FEE PAYMENT PAID IN FULL

Effective date: 20250828