CA3168799A1 - Methods and systems for augmenting depth data from a depth sensor, such as with data from a multiview camera system - Google Patents

Methods and systems for augmenting depth data from a depth sensor, such as with data from a multiview camera system

Info

Publication number
CA3168799A1
CA3168799A1 CA3168799A CA3168799A CA3168799A1 CA 3168799 A1 CA3168799 A1 CA 3168799A1 CA 3168799 A CA3168799 A CA 3168799A CA 3168799 A CA3168799 A CA 3168799A CA 3168799 A1 CA3168799 A1 CA 3168799A1
Authority
CA
Canada
Prior art keywords
data
depth
scene
point cloud
region
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CA3168799A
Other languages
English (en)
French (fr)
Inventor
Thomas Ivan Nonn
David Julio COLMENARES
James Andrew YOUNGQUIST
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Proprio Inc
Original Assignee
Proprio Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Proprio Inc filed Critical Proprio Inc
Publication of CA3168799A1 publication Critical patent/CA3168799A1/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/77—Retouching; Inpainting; Scratch removal
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/557—Depth or shape recovery from multiple images from light fields, e.g. from plenoptic cameras
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional [3D] objects
    • G06V20/647—Three-dimensional [3D] objects by matching two-dimensional images to three-dimensional objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/111—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
    • H04N13/117—Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/243—Image signal generators using stereoscopic image cameras using three or more two-dimensional [2D] image sensors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/282—Image signal generators for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/24—Indexing scheme for image data processing or generation, in general involving graphical user interfaces [GUIs]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10052—Images from lightfield camera
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20092—Interactive image processing based on input by user
    • G06T2207/20104—Interactive definition of region of interest [ROI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30008—Bone
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30168—Image quality inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00—Indexing scheme for image generation or computer graphics
    • G06T2210/41—Medical
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00—Indexing scheme for image generation or computer graphics
    • G06T2210/56—Particle system, point based geometry or rendering
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N2013/0074—Stereoscopic image analysis
    • H04N2013/0081—Depth or disparity estimation from stereoscopic image signals

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Signal Processing (AREA)
  • Geometry (AREA)
  • Computer Graphics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Processing Or Creating Images (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
  • Measurement Of Optical Distance (AREA)
  • Length Measuring Devices By Optical Means (AREA)
CA3168799A 2020-01-21 2021-01-21 Methods and systems for augmenting depth data from a depth sensor, such as with data from a multiview camera system Pending CA3168799A1 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US202062963717P 2020-01-21 2020-01-21
US62/963,717 2020-01-21
PCT/US2021/014397 WO2021150741A1 (en) 2020-01-21 2021-01-21 Methods and systems for augmenting depth data from a depth sensor, such as with data from a multiview camera system

Publications (1)

Publication Number Publication Date
CA3168799A1 true CA3168799A1 (en) 2021-07-29

Family

ID=76857295

Family Applications (1)

Application Number Title Priority Date Filing Date
CA3168799A Pending CA3168799A1 (en) 2020-01-21 2021-01-21 Methods and systems for augmenting depth data from a depth sensor, such as with data from a multiview camera system

Country Status (6)

Country Link
US (3) US11682165B2 (cg-RX-API-DMAC7.html)
EP (1) EP4094186A4 (cg-RX-API-DMAC7.html)
JP (2) JP7879808B2 (cg-RX-API-DMAC7.html)
AU (2) AU2021211677B2 (cg-RX-API-DMAC7.html)
CA (1) CA3168799A1 (cg-RX-API-DMAC7.html)
WO (1) WO2021150741A1 (cg-RX-API-DMAC7.html)

Families Citing this family (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11682165B2 (en) * 2020-01-21 2023-06-20 Proprio, Inc. Methods and systems for augmenting depth data from a depth sensor, such as with data from a multiview camera system
US11836940B2 (en) * 2020-06-15 2023-12-05 Zebra Technologies Corporation Three-dimensional sensor acuity recovery assistance
EP4381735A4 (en) 2021-08-03 2025-05-28 LEIA Inc. Depth-map view synthesis system and method
US12016642B2 (en) 2021-09-08 2024-06-25 Proprio, Inc. Constellations for tracking instruments, such as surgical instruments, and associated systems and methods
US12357397B2 (en) 2022-05-09 2025-07-15 Proprio, Inc. Methods and systems for calibrating instruments within an imaging system, such as a surgical imaging system
CN114782438B (zh) * 2022-06-20 2022-09-16 深圳市信润富联数字科技有限公司 物体点云修正方法、装置、电子设备和存储介质
CN115100385A (zh) * 2022-06-22 2022-09-23 众趣(北京)科技有限公司 一种三维场景中家具物品替换方法
GB2624621A (en) * 2022-11-15 2024-05-29 Continental Autonomous Mobility Germany GmbH 3-Dimensional (3D) map generation system and method for creating 3D map of surroundings of a vehicle
US20250134476A1 (en) * 2023-06-28 2025-05-01 Abbott Diabetes Care Inc. Systems, methods, and features for analyte monitoring
US11935209B1 (en) 2023-07-24 2024-03-19 Illuscio, Inc. Systems and methods for dynamic backfilling of a three-dimensional object
US20250104323A1 (en) * 2023-09-22 2025-03-27 Proprio, Inc. METHODS AND SYSTEMS FOR GENERATING THREE-DIMENSIONAL RENDERINGS OF A SCENE USING A MOBILE SENSOR ARRAY, SUCH AS NEURAL RADIANCE FIELD (NeRF) RENDERINGS
WO2025171548A1 (en) * 2024-02-15 2025-08-21 Dental Imaging Technologies Corporation Point cloud registration for intraoral scanning applications
CN120910119B (zh) * 2025-10-09 2025-12-09 兰州交通大学 一种基于cbr的山区铁路隧道绿色施工决策方法

Family Cites Families (40)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001231019A (ja) 2000-02-16 2001-08-24 Sony Corp 撮像画像付き表示装置、モニタ装置、テレビ電話装置、プロンプタ付きカメラ装置、テレビ会議装置、並びに、反射エネルギ・モデル化装置
US7389041B2 (en) 2005-02-01 2008-06-17 Eastman Kodak Company Determining scene distance in digital camera images
JP5057734B2 (ja) * 2006-09-25 2012-10-24 株式会社トプコン 測量方法及び測量システム及び測量データ処理プログラム
DE102006055641B4 (de) * 2006-11-22 2013-01-31 Visumotion Gmbh Anordnung und Verfahren zur Aufnahme und Wiedergabe von Bildern einer Szene und/oder eines Objektes
JP2009300268A (ja) 2008-06-13 2009-12-24 Nippon Hoso Kyokai <Nhk> 3次元情報検出装置
JP5329677B2 (ja) * 2009-01-27 2013-10-30 テレフオンアクチーボラゲット エル エム エリクソン(パブル) 奥行き及びビデオのコプロセッシング
GB2490872B (en) * 2011-05-09 2015-07-29 Toshiba Res Europ Ltd Methods and systems for capturing 3d surface geometry
US8734328B2 (en) * 2011-08-12 2014-05-27 Intuitive Surgical Operations, Inc. Increased resolution and dynamic range image capture unit in a surgical instrument and method
US8684914B2 (en) * 2011-08-12 2014-04-01 Intuitive Surgical Operations, Inc. Image capture unit and an imaging pipeline with enhanced color performance in a surgical instrument and method
US8784301B2 (en) * 2011-08-12 2014-07-22 Intuitive Surgical Operations, Inc. Image capture unit and method with an extended depth of field
US8764633B2 (en) * 2011-08-12 2014-07-01 Intuitive Surgical Operations, Inc. Feature differentiation image capture unit and method in a surgical instrument
US8672838B2 (en) * 2011-08-12 2014-03-18 Intuitive Surgical Operations, Inc. Image capture unit in a surgical instrument
US9191646B2 (en) * 2011-08-29 2015-11-17 Nokia Technologies Oy Apparatus, a method and a computer program for video coding and decoding
US10848731B2 (en) * 2012-02-24 2020-11-24 Matterport, Inc. Capturing and aligning panoramic image and depth data
US20140092281A1 (en) * 2012-09-28 2014-04-03 Pelican Imaging Corporation Generating Images from Light Fields Utilizing Virtual Viewpoints
CA2925620C (en) * 2012-10-29 2020-07-21 7D Surgical Inc. Integrated illumination and optical surface topology detection system and methods of use thereof
WO2014200016A1 (ja) 2013-06-11 2014-12-18 Tanji Atsushi 外科手術支援システム、外科手術支援装置、外科手術支援方法、外科手術支援プログラムおよび情報処理装置
CA2931529C (en) 2013-11-27 2022-08-23 Children's National Medical Center 3d corrected imaging
CA2964512C (en) * 2014-10-14 2018-04-24 Synaptive Medical (Barbados) Inc. Patient reference tool
US10121064B2 (en) * 2015-04-16 2018-11-06 California Institute Of Technology Systems and methods for behavior detection using 3D tracking and machine learning
US11045257B2 (en) * 2015-06-22 2021-06-29 Synaptive Medical Inc. System and method for mapping navigation space to patient space in a medical procedure
CA2973479C (en) * 2015-07-21 2019-02-26 Synaptive Medical (Barbados) Inc. System and method for mapping navigation space to patient space in a medical procedure
CA2994024C (en) * 2015-07-29 2019-03-05 Synaptive Medical (Barbados) Inc. Handheld scanner for rapid registration in a medical navigation system
US10699476B2 (en) 2015-08-06 2020-06-30 Ams Sensors Singapore Pte. Ltd. Generating a merged, fused three-dimensional point cloud based on captured images of a scene
WO2017043331A1 (ja) * 2015-09-09 2017-03-16 ソニー株式会社 画像処理装置、及び、画像処理方法
JP6639276B2 (ja) 2016-03-10 2020-02-05 キヤノン株式会社 画像処理装置およびその制御方法、撮像装置、プログラム
EP3520414B1 (en) 2016-09-29 2026-04-22 Koninklijke Philips N.V. Image processing
US11423562B2 (en) * 2016-10-18 2022-08-23 Photonic Sensors & Algorithms, S.L. Device and method for obtaining distance information from views
EP3545675A4 (en) 2016-11-24 2020-07-01 The University of Washington CAPTURE AND RESTITUTION OF LIGHT FIELD FOR HEADSETS
WO2018112875A1 (en) * 2016-12-23 2018-06-28 Intel Corporation Efficient sub-pixel disparity estimation for all sub-aperture images from densely sampled light field cameras
US20200057778A1 (en) * 2017-04-11 2020-02-20 Siemens Mobility GmbH Depth image pose search with a bootstrapped-created database
US10395418B2 (en) * 2017-08-18 2019-08-27 Microsoft Technology Licensing, Llc Techniques for predictive prioritization of image portions in processing graphics
JP6623199B2 (ja) 2017-09-28 2019-12-18 株式会社コロプラ ヘッドマウントデバイスを用いて仮想現実を提供するためにコンピュータで実行されるプログラムおよび情報処理装置
WO2019089039A1 (en) * 2017-11-03 2019-05-09 Google Llc Aperture supervision for single-view depth prediction
US10460509B2 (en) * 2017-11-07 2019-10-29 Dolby Laboratories Licensing Corporation Parameterizing 3D scenes for volumetric viewing
WO2020006519A1 (en) * 2018-06-29 2020-01-02 Eloupes, Inc. Synthesizing an image from a virtual perspective using pixels from a physical imager array
US10627901B2 (en) * 2018-09-14 2020-04-21 Facebook Technologies, Llc Vergence determination
US10958887B2 (en) * 2019-01-14 2021-03-23 Fyusion, Inc. Free-viewpoint photorealistic view synthesis from casually captured video
US11100664B2 (en) * 2019-08-09 2021-08-24 Google Llc Depth-aware photo editing
US11682165B2 (en) 2020-01-21 2023-06-20 Proprio, Inc. Methods and systems for augmenting depth data from a depth sensor, such as with data from a multiview camera system

Also Published As

Publication number Publication date
AU2024202120A1 (en) 2024-05-02
AU2021211677B2 (en) 2024-01-04
US20210225020A1 (en) 2021-07-22
WO2021150741A1 (en) 2021-07-29
JP2026047369A (ja) 2026-03-13
US11682165B2 (en) 2023-06-20
EP4094186A4 (en) 2024-01-24
EP4094186A1 (en) 2022-11-30
US12243162B2 (en) 2025-03-04
AU2021211677A1 (en) 2022-08-25
US20250252664A1 (en) 2025-08-07
JP2023511670A (ja) 2023-03-22
US20240005596A1 (en) 2024-01-04
JP7879808B2 (ja) 2026-06-24

Similar Documents

Publication Publication Date Title
US12243162B2 (en) Methods and systems for augmenting depth data from a depth sensor, such as with data from a multiview camera system
AU2021414195B2 (en) Methods and systems for registering preoperative image data to intraoperative image data of a scene, such as a surgical scene
US11354810B2 (en) Methods and systems for imaging a scene, such as a medical scene, and tracking objects within the scene
US20250104323A1 (en) METHODS AND SYSTEMS FOR GENERATING THREE-DIMENSIONAL RENDERINGS OF A SCENE USING A MOBILE SENSOR ARRAY, SUCH AS NEURAL RADIANCE FIELD (NeRF) RENDERINGS
US20120242795A1 (en) Digital 3d camera using periodic illumination
CN101729920B (zh) 一种自由视角立体视频显示方法
US20150287203A1 (en) Method Of Estimating Imaging Device Parameters
US20110187827A1 (en) Method and apparatus for creating a stereoscopic image
US12357397B2 (en) Methods and systems for calibrating instruments within an imaging system, such as a surgical imaging system
US12261988B2 (en) Methods for generating stereoscopic views in multicamera systems, and associated devices and systems
CN103968829B (zh) 基于虚拟标志物的三维空间定位追踪方法及系统
US12450777B2 (en) Methods and systems for calibrating and/or verifying a calibration of an imaging system such as a surgical imaging system
KR20120056668A (ko) 3차원 정보 복원 장치 및 그 방법
Conen et al. Rectification and robust matching using oriented image triplets for minimally invasive surgery
US12383350B2 (en) Constellations for tracking instruments, such as surgical instruments, and associated systems and methods
KR20130015773A (ko) 패턴 빔을 이용하는 3차원 이미지 촬영 장치 및 방법
HK1182248A1 (zh) 利用立体成像生成自由视点视频

Legal Events

Date Code Title Description
MFA Maintenance fee for application paid

Free format text: FEE DESCRIPTION TEXT: MF (APPLICATION, 4TH ANNIV.) - STANDARD

Year of fee payment: 4

U00 Fee paid

Free format text: ST27 STATUS EVENT CODE: A-1-1-U10-U00-U101 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE REQUEST RECEIVED

Effective date: 20241218

U11 Full renewal or maintenance fee paid

Free format text: ST27 STATUS EVENT CODE: A-1-1-U10-U11-U102 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE FEE PAYMENT DETERMINED COMPLIANT

Effective date: 20241218

Free format text: ST27 STATUS EVENT CODE: A-1-1-U10-U11-U102 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE FEE PAYMENT PAID IN FULL

Effective date: 20241218

D11 Substantive examination requested

Free format text: ST27 STATUS EVENT CODE: A-1-1-D10-D11-D117 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: REQUEST FOR EXAMINATION RECEIVED

Effective date: 20250116

P11 Amendment of application requested

Free format text: ST27 STATUS EVENT CODE: A-1-1-P10-P11-P101 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: AMENDMENT RECEIVED - VOLUNTARY AMENDMENT

Effective date: 20250116

W00 Other event occurred

Free format text: ST27 STATUS EVENT CODE: A-1-1-W10-W00-W111 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: CORRESPONDENT DETERMINED COMPLIANT

Effective date: 20250116

D00 Search and/or examination requested or commenced

Free format text: ST27 STATUS EVENT CODE: A-1-1-D10-D00-D118 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: REQUEST FOR EXAMINATION REQUIREMENTS DETERMINED COMPLIANT

Effective date: 20250117

P11 Amendment of application requested

Free format text: ST27 STATUS EVENT CODE: A-1-1-P10-P11-P102 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: AMENDMENT DETERMINED COMPLIANT

Effective date: 20250117

P13 Application amended

Free format text: ST27 STATUS EVENT CODE: A-1-1-P10-P13-X000 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: APPLICATION AMENDED

Effective date: 20250117

W00 Other event occurred

Free format text: ST27 STATUS EVENT CODE: A-1-1-W10-W00-W111 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: CORRESPONDENT DETERMINED COMPLIANT

Effective date: 20250117

D11 Substantive examination requested

Free format text: ST27 STATUS EVENT CODE: A-1-2-D10-D11-D155 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: ALL REQUIREMENTS FOR EXAMINATION DETERMINED COMPLIANT

Effective date: 20250501

W00 Other event occurred

Free format text: ST27 STATUS EVENT CODE: A-2-2-W10-W00-W100 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: LETTER SENT

Effective date: 20250505

MFA Maintenance fee for application paid

Free format text: FEE DESCRIPTION TEXT: MF (APPLICATION, 5TH ANNIV.) - STANDARD

Year of fee payment: 5

U00 Fee paid

Free format text: ST27 STATUS EVENT CODE: A-2-2-U10-U00-U101 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE REQUEST RECEIVED

Effective date: 20251205

U11 Full renewal or maintenance fee paid

Free format text: ST27 STATUS EVENT CODE: A-2-2-U10-U11-U102 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: MAINTENANCE FEE PAYMENT PAID IN FULL

Effective date: 20251205

D15 Examination report completed

Free format text: ST27 STATUS EVENT CODE: A-2-2-D10-D15-D126 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: EXAMINER'S REPORT

Effective date: 20260128

P11 Amendment of application requested

Free format text: ST27 STATUS EVENT CODE: A-2-2-P10-P11-P100 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: AMENDMENT RECEIVED - RESPONSE TO EXAMINER'S REQUISITION

Effective date: 20260528

P11 Amendment of application requested

Free format text: ST27 STATUS EVENT CODE: A-2-2-P10-P11-P102 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: AMENDMENT DETERMINED COMPLIANT

Effective date: 20260730

P13 Application amended

Free format text: ST27 STATUS EVENT CODE: A-2-2-P10-P13-X000 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: APPLICATION AMENDED

Effective date: 20260730

W00 Other event occurred

Free format text: ST27 STATUS EVENT CODE: A-2-2-W10-W00-W111 (AS PROVIDED BY THE NATIONAL OFFICE); EVENT TEXT: CORRESPONDENT DETERMINED COMPLIANT

Effective date: 20260730