JPWO2020234678A5 - - Google Patents

Download PDF

Info

Publication number
JPWO2020234678A5
JPWO2020234678A5 JP2021568784A JP2021568784A JPWO2020234678A5 JP WO2020234678 A5 JPWO2020234678 A5 JP WO2020234678A5 JP 2021568784 A JP2021568784 A JP 2021568784A JP 2021568784 A JP2021568784 A JP 2021568784A JP WO2020234678 A5 JPWO2020234678 A5 JP WO2020234678A5
Authority
JP
Japan
Prior art keywords
point
probability
tracking
background
model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
JP2021568784A
Other languages
Japanese (ja)
Other versions
JP2022533393A (en
Publication date
Priority claimed from US16/417,675 external-priority patent/US10930012B2/en
Application filed filed Critical
Publication of JP2022533393A publication Critical patent/JP2022533393A/en
Publication of JPWO2020234678A5 publication Critical patent/JPWO2020234678A5/ja
Pending legal-status Critical Current

Links

Claims (9)

３Ｄ点群を、オブジェクトおよび背景のセグメントにセグメント化する方法であって、
プロセッサが、
映像ストリーム内のシーン内のオブジェクトの３Ｄモデルを、カメラが前記シーン内を動き回るのに従って追跡することであって、前記追跡することが、前記映像ストリーム内のフレームごとの点ごとに、前記点が前記オブジェクトに対応するのか、それとも前記点が前記背景に対応するのかを決定することを含む、前記追跡することと、
複数の点の各々を前記オブジェクトに対応する点または前記背景に対応する点のどちらかに、前記点が前記オブジェクトに対応する確率および前記点が前記背景に対応する確率に基づいてセグメント化することであって、点ごとに、前記点が前記オブジェクトに対応する前記確率がより高いことがより多い場合には、前記追跡することが前記点をインライアとして識別し、点ごとに、前記点が前記背景に対応する前記確率がより高いことがより多い場合には、前記追跡することが前記点をアウトライアとして識別する、前記セグメント化することと、
前記オブジェクトのセグメント化された３Ｄモデルに基づいて拡張現実コンテンツを生成することと、
を実行する、方法。 A method for segmenting a 3D point cloud into object and background segments, comprising:
the processor
tracking a 3D model of an object in a scene in a video stream as a camera moves about the scene, said tracking being a point by frame in said video stream; corresponds to the object or the point corresponds to the background;
segmenting each of a plurality of points into either a point corresponding to the object or a point corresponding to the background based on a probability that the point corresponds to the object and a probability that the point corresponds to the background; wherein for each point, if the probability that the point corresponds to the object is higher, the tracking identifies the point as an inlier; the segmenting, wherein the tracking identifies the point as an outlier if more often the probability corresponding to the background is higher;
generating augmented reality content based on the segmented 3D model of the object;
how to run 前記追跡することが、前記オブジェクトの前記セグメント化された３Ｄモデルを用いて、前記オブジェクトの前記３Ｄモデルに対する前記カメラの姿勢を推定することを含む、請求項１に記載の方法。 2. The method of claim 1, wherein tracking comprises using the segmented 3D model of the object to estimate a pose of the camera relative to the 3D model of the object. 前記オブジェクトの前記３Ｄモデルが、点ごとの前記確率のデフォルト設定を用いて最初にセグメント化され、フレームごとに、前記追跡することが、前記点が前記オブジェクトに対応すると決定したのか、それとも前記背景に対応すると決定したのかに基づいて、点ごとの前記確率が更新される、請求項２に記載の方法。 The 3D model of the object is first segmented using the default setting of the probability per point, and for each frame, whether the tracking determined that the point corresponds to the object or the background 3. The method of claim 2, wherein the probability for each point is updated based on whether it is determined to correspond to . 点ごとの前記確率の前記デフォルト設定が．５であり、前記追跡することが、前記点が前記オブジェクトに対応すると決定したときには、前記確率が高められ、前記追跡することが、前記点が前記背景に対応すると決定したときには、前記確率が下げられる、請求項３に記載の方法。 If the default setting for the probability per point is . 5, wherein the probability is increased when the tracking determines that the point corresponds to the object, and the probability is decreased when the tracking determines that the point corresponds to the background; 4. The method of claim 3, wherein: 前記シーン内の前記オブジェクトの前記３Ｄモデルが、前記オブジェクトに対応する少なくともいくつかの点および前記シーンの背景に対応する少なくともいくつかの点を含む複数の点を含む３Ｄ点群を含む、請求項１に記載の方法。 3. The 3D model of the object in the scene comprises a 3D point cloud comprising a plurality of points including at least some points corresponding to the object and at least some points corresponding to the background of the scene. 1. The method according to 1. 前記映像ストリームが、前記シーン内を動き回る前記カメラから取得された、請求項５に記載の方法。 6. The method of claim 5, wherein the video stream is obtained from the camera moving around the scene. 請求項１ないし６の何れか一項に記載の方法を実行するプロセッサを備えたシステム。 A system comprising a processor for performing the method of any one of claims 1-6. プロセッサに、請求項１ないし６の何れか一項に記載の方法を実行させるためのプログラム。A program for causing a processor to execute the method according to any one of claims 1 to 6. 請求項８に記載のプログラムを記憶する記憶媒体。 A storage medium for storing the program according to claim 8 .
JP2021568784A 2019-05-21 2020-05-06 Progressive 3D point cloud segmentation into objects and background from tracking sessions Pending JP2022533393A (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US16/417,675 2019-05-21
US16/417,675 US10930012B2 (en) 2019-05-21 2019-05-21 Progressive 3D point cloud segmentation into object and background from tracking sessions
PCT/IB2020/054284 WO2020234678A1 (en) 2019-05-21 2020-05-06 Progressive 3d point cloud segmentation into object and background from tracking sessions

Publications (2)

Publication Number Publication Date
JP2022533393A JP2022533393A (en) 2022-07-22
JPWO2020234678A5 true JPWO2020234678A5 (en) 2022-09-14

Family

ID=73457066

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2021568784A Pending JP2022533393A (en) 2019-05-21 2020-05-06 Progressive 3D point cloud segmentation into objects and background from tracking sessions

Country Status (6)

Country Link
US (1) US10930012B2 (en)
JP (1) JP2022533393A (en)
CN (1) CN113811919A (en)
DE (1) DE112020000906B4 (en)
GB (1) GB2598512B (en)
WO (1) WO2020234678A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210279967A1 (en) * 2020-03-06 2021-09-09 Apple Inc. Object centric scanning
US11250626B1 (en) * 2020-08-13 2022-02-15 Beijing Institute Of Technology Virtual stage based on parallel simulation
CN113744298B (en) * 2021-01-05 2024-07-19 北京京东乾石科技有限公司 Data processing method, device and storage medium

Family Cites Families (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1297691A2 (en) * 2000-03-07 2003-04-02 Sarnoff Corporation Camera pose estimation
US20030012410A1 (en) * 2001-07-10 2003-01-16 Nassir Navab Tracking and pose estimation for augmented reality using real features
US7706603B2 (en) * 2005-04-19 2010-04-27 Siemens Corporation Fast object detection for augmented reality systems
US7957548B2 (en) * 2006-05-16 2011-06-07 Phonak Ag Hearing device with transfer function adjusted according to predetermined acoustic environments
US9189886B2 (en) * 2008-08-15 2015-11-17 Brown University Method and apparatus for estimating body shape
WO2010088840A1 (en) 2009-02-06 2010-08-12 The Hong Kong University Of Science And Technology Generating three-dimensional models from images
US8401225B2 (en) * 2011-01-31 2013-03-19 Microsoft Corporation Moving object segmentation using depth images
WO2013086678A1 (en) * 2011-12-12 2013-06-20 北京航空航天大学 Point matching and pose synchronization determining method for planar models and computer program product
US9154773B2 (en) * 2013-03-15 2015-10-06 Seiko Epson Corporation 2D/3D localization and pose estimation of harness cables using a configurable structure representation for robot operations
CN103366158A (en) * 2013-06-27 2013-10-23 东南大学 Three dimensional structure and color model-based monocular visual road face detection method
US9536321B2 (en) * 2014-03-21 2017-01-03 Intel Corporation Apparatus and method for foreground object segmentation
EP3192057A4 (en) * 2014-09-10 2018-03-21 Vangogh Imaging Inc. Real-time dynamic three-dimensional adaptive object recognition and model reconstruction
US9710712B2 (en) * 2015-01-16 2017-07-18 Avigilon Fortress Corporation System and method for detecting, tracking, and classifiying objects
US10373366B2 (en) * 2015-05-14 2019-08-06 Qualcomm Incorporated Three-dimensional model generation
KR101703515B1 (en) * 2015-05-19 2017-02-07 국방과학연구소 Apparatus and method for target tracking of image
US9818043B2 (en) * 2015-06-24 2017-11-14 Microsoft Technology Licensing, Llc Real-time, model-based object detection and pose estimation
JP6607630B2 (en) * 2015-06-30 2019-11-20 Ｋｄｄｉ株式会社 Moving object extraction apparatus, method and program
US9836673B2 (en) * 2015-12-30 2017-12-05 International Business Machines Corporation System, method and computer program product for training a three dimensional object indentification system and identifying three dimensional objects using semantic segments
US9904867B2 (en) * 2016-01-29 2018-02-27 Pointivo, Inc. Systems and methods for extracting information about objects from scene information
US10482681B2 (en) * 2016-02-09 2019-11-19 Intel Corporation Recognition-based object segmentation of a 3-dimensional image
US9868212B1 (en) * 2016-02-18 2018-01-16 X Development Llc Methods and apparatus for determining the pose of an object based on point cloud data
US10373380B2 (en) 2016-02-18 2019-08-06 Intel Corporation 3-dimensional scene analysis for augmented reality operations
CN106056614A (en) * 2016-06-03 2016-10-26 武汉大学 Building segmentation and contour line extraction method of ground laser point cloud data
CN107945198B (en) * 2016-10-13 2021-02-23 北京百度网讯科技有限公司 Method and device for marking point cloud data
US10109055B2 (en) * 2016-11-21 2018-10-23 Seiko Epson Corporation Multiple hypotheses segmentation-guided 3D object detection and pose estimation
CN107133966B (en) * 2017-03-30 2020-04-14 浙江大学 Three-dimensional sonar image background segmentation method based on sampling consistency algorithm
US10467756B2 (en) * 2017-05-14 2019-11-05 International Business Machines Corporation Systems and methods for determining a camera pose of an image
US10719987B1 (en) * 2017-06-28 2020-07-21 Kilburn Live, Llc Augmented reality in a virtual reality environment
CN107424166B (en) * 2017-07-18 2020-05-19 深圳市速腾聚创科技有限公司 Point cloud segmentation method and device
US10535160B2 (en) * 2017-07-24 2020-01-14 Visom Technology, Inc. Markerless augmented reality (AR) system
CN107560592B (en) * 2017-08-21 2020-08-18 河南中光学集团有限公司 Precise distance measurement method for photoelectric tracker linkage target

Similar Documents

Publication Publication Date Title
US10306180B2 (en) Predictive virtual reality content streaming techniques
CN107624243B (en) Real-time hyper-lapse video creation through frame selection
US20180204381A1 (en) Image processing apparatus for generating virtual viewpoint image and method therefor
US20160286138A1 (en) Apparatus and method for stitching panoramaic video
JP2019526133A5 (en)
EP3469788A1 (en) Multi-view scene segmentation and propagation
US9525821B2 (en) Video stabilization
KR102464944B1 (en) Method and apparatus of reproducing camera work
US11070706B2 (en) Notifications for deviations in depiction of different objects in filmed shots of video content
US10154228B1 (en) Smoothing video panning
US20200304755A1 (en) Aspect ratio conversion with machine learning
CN106412669A (en) Method and device for rendering panoramic video
JP2020515091A (en) Method, system and apparatus for configuring a virtual camera
US20190392621A1 (en) Banner image generation
GB2598512A (en) Progressive 3D point cloud segmentation into object and background from tracking sessions
US10994202B2 (en) Simulated previews of dynamic virtual cameras
JP6298521B2 (en) Program, system, apparatus, and method
US20210304495A1 (en) Techniques for improving mesh accuracy using labeled inputs
JP6472279B2 (en) Image processing apparatus and image processing method
EP4322115A1 (en) Finding the semantic region of interest in images
JPWO2020234678A5 (en)
JP2005223487A (en) Digital camera work apparatus, digital camera work method, and digital camera work program
CN115187851A (en) Method, apparatus, device and medium for processing sensitive information in visual data
JP6226050B2 (en) Image composition method and apparatus
KR101831138B1 (en) Method and apparatus for manufacturing animation sticker using video