EP3475917A1 - Rgb-d-kamerabasiertes verfolgungssystem und verfahren dafür - Google Patents

Rgb-d-kamerabasiertes verfolgungssystem und verfahren dafür

Info

Publication number
EP3475917A1
EP3475917A1 EP17733432.3A EP17733432A EP3475917A1 EP 3475917 A1 EP3475917 A1 EP 3475917A1 EP 17733432 A EP17733432 A EP 17733432A EP 3475917 A1 EP3475917 A1 EP 3475917A1
Authority
EP
European Patent Office
Prior art keywords
keyframe
graph
constraint
loop
generating
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP17733432.3A
Other languages
English (en)
French (fr)
Other versions
EP3475917B1 (de
Inventor
Soohwan Kim
Benzun Pious Wisely BABU
Zhixin Yan
Liu Ren
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Robert Bosch GmbH
Original Assignee
Robert Bosch GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Robert Bosch GmbH filed Critical Robert Bosch GmbH
Publication of EP3475917A1 publication Critical patent/EP3475917A1/de
Application granted granted Critical
Publication of EP3475917B1 publication Critical patent/EP3475917B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/7837—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using objects detected or recognised in the video content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/579—Depth or shape recovery from multiple images from motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20072—Graph-based image processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20076—Probabilistic image processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30241—Trajectory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose
EP17733432.3A 2016-06-24 2017-06-26 Rgb-d-kamerabasiertes slam-system und verfahren dafür Active EP3475917B1 (de)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201662354251P 2016-06-24 2016-06-24
PCT/EP2017/065677 WO2017220815A1 (en) 2016-06-24 2017-06-26 Rgb-d camera based tracking system and method thereof

Publications (2)

Publication Number Publication Date
EP3475917A1 true EP3475917A1 (de) 2019-05-01
EP3475917B1 EP3475917B1 (de) 2022-01-26

Family

ID=59227732

Family Applications (1)

Application Number Title Priority Date Filing Date
EP17733432.3A Active EP3475917B1 (de) 2016-06-24 2017-06-26 Rgb-d-kamerabasiertes slam-system und verfahren dafür

Country Status (7)

Country Link
US (1) US11093753B2 (de)
EP (1) EP3475917B1 (de)
JP (1) JP2019522288A (de)
KR (1) KR102471301B1 (de)
CN (1) CN109478330B (de)
ES (1) ES2911400T3 (de)
WO (1) WO2017220815A1 (de)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3474230B1 (de) * 2017-10-18 2020-07-22 Tata Consultancy Services Limited Systeme und verfahren für monokularen visuellen slam auf basis von kantenpunkten
CN108564625B (zh) * 2018-04-27 2019-08-23 百度在线网络技术（北京）有限公司 图优化方法、装置、电子设备及存储介质
CN108648274B (zh) * 2018-05-10 2020-05-22 华南理工大学 一种视觉slam的认知点云地图创建系统
TWI679511B (zh) * 2018-08-22 2019-12-11 和碩聯合科技股份有限公司 軌跡規劃方法與系統
CA3028708A1 (en) * 2018-12-28 2020-06-28 Zih Corp. Method, system and apparatus for dynamic loop closure in mapping trajectories
CN110516527B (zh) * 2019-07-08 2023-05-23 广东工业大学 一种基于实例分割的视觉slam回环检测改进方法
CN110956651B (zh) * 2019-12-16 2021-02-19 哈尔滨工业大学 一种基于视觉和振动触觉融合的地形语义感知方法
CN111292420B (zh) * 2020-02-28 2023-04-28 北京百度网讯科技有限公司 用于构建地图的方法和装置
CN112016612A (zh) * 2020-08-26 2020-12-01 四川阿泰因机器人智能装备有限公司 一种基于单目深度估计的多传感器融合slam方法
US11899469B2 (en) 2021-08-24 2024-02-13 Honeywell International Inc. Method and system of integrity monitoring for visual odometry

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2619742B1 (de) * 2010-09-24 2018-02-28 iRobot Corporation Systeme und verfahren zur vslam-optimierung
US8913055B2 (en) 2011-05-31 2014-12-16 Honda Motor Co., Ltd. Online environment mapping
GB201202344D0 (en) * 2012-02-10 2012-03-28 Isis Innovation Method of locating a sensor and related apparatus
US9269003B2 (en) * 2013-04-30 2016-02-23 Qualcomm Incorporated Diminished and mediated reality effects from reconstruction
US9607401B2 (en) 2013-05-08 2017-03-28 Regents Of The University Of Minnesota Constrained key frame localization and mapping for vision-aided inertial navigation
CN104374395A (zh) * 2014-03-31 2015-02-25 南京邮电大学 基于图的视觉slam方法
WO2015194864A1 (ko) * 2014-06-17 2015-12-23 (주)유진로봇 이동 로봇의 맵을 업데이트하기 위한 장치 및 그 방법

Also Published As

Publication number Publication date
US20190377952A1 (en) 2019-12-12
JP2019522288A (ja) 2019-08-08
US11093753B2 (en) 2021-08-17
ES2911400T3 (es) 2022-05-19
CN109478330A (zh) 2019-03-15
WO2017220815A1 (en) 2017-12-28
KR102471301B1 (ko) 2022-11-29
CN109478330B (zh) 2022-03-29
KR20190021257A (ko) 2019-03-05
EP3475917B1 (de) 2022-01-26

Similar Documents

Publication Publication Date Title
CN109478330B (zh) 基于rgb-d照相机的跟踪系统及其方法
CN109084746B (zh) 用于具有辅助传感器的自主平台引导系统的单目模式
US10553026B2 (en) Dense visual SLAM with probabilistic surfel map
JP6745328B2 (ja) 点群データを復旧するための方法及び装置
Wang et al. CamShift guided particle filter for visual tracking
CN107438866B (zh) 深度立体：学习从现实世界形象预测新视图
RU2713611C2 (ru) Способ моделирования трехмерного пространства
Comport et al. Real-time quadrifocal visual odometry
CN103123727B (zh) 即时定位与地图构建方法和设备
EP3688718A1 (de) Unüberwachtes lernen der bildtiefe und neuronale netze zur vorhersage der ego-bewegung
EP3385917A1 (de) Verfahren und vorrichtung zur erzeugung eines dreidimensionalen modells unter verwendung eines annäherungsverfahrens an den volumetrischen nächstgelegenen punkt
US8209143B1 (en) Accurate alignment of multiple laser scans using a template surface
Cheng et al. Bi-pointflownet: Bidirectional learning for point cloud based scene flow estimation
CN105654492A (zh) 基于消费级摄像头的鲁棒实时三维重建方法
US10019801B2 (en) Image analysis system and method
Tao et al. Automated localisation of Mars rovers using co-registered HiRISE-CTX-HRSC orthorectified images and wide baseline Navcam orthorectified mosaics
CA3202821A1 (en) System and method of hybrid scene representation for visual simultaneous localization and mapping
WO2018214086A1 (zh) 场景的三维重建方法、装置及终端设备
US10121259B2 (en) System and method for determining motion and structure from optical flow
Mehralian et al. EKFPnP: extended Kalman filter for camera pose estimation in a sequence of images
CN105527968A (zh) 信息处理方法和信息处理装置
Han et al. DiLO: Direct light detection and ranging odometry based on spherical range images for autonomous driving
Qu et al. Robust local stereo matching under varying radiometric conditions
Aravkin et al. Student's t robust bundle adjustment algorithm
Indelman Bundle adjustment without iterative structure estimation and its application to navigation

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20190124

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: ROBERT BOSCH GMBH

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20201109

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602017052763

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: G06T0007200000

Ipc: G06T0007579000

RIC1 Information provided on ipc code assigned before grant

Ipc: G06T 7/579 20170101AFI20210804BHEP

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20210910

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1465871

Country of ref document: AT

Kind code of ref document: T

Effective date: 20220215

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602017052763

Country of ref document: DE

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2911400

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20220519

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20220126

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1465871

Country of ref document: AT

Kind code of ref document: T

Effective date: 20220126

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220126

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220126

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220126

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220526

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220426

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220126

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220126

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220426

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220126

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220126

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220427

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220126

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220126

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220526

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602017052763

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220126

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220126

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220126

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220126

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220126

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220126

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220126

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20221027

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220126

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20220630

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220126

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220626

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220630

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220626

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220630

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220630

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220630

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220126

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20230622

Year of fee payment: 7

Ref country code: ES

Payment date: 20230719

Year of fee payment: 7

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20230817

Year of fee payment: 7

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20170626

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220126

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20220126