WO2024007478A1 - 基于单手机的人体三维建模数据采集与重建方法及系统 - Google Patents

基于单手机的人体三维建模数据采集与重建方法及系统 Download PDF

Info

Publication number
WO2024007478A1
WO2024007478A1 PCT/CN2022/125581 CN2022125581W WO2024007478A1 WO 2024007478 A1 WO2024007478 A1 WO 2024007478A1 CN 2022125581 W CN2022125581 W CN 2022125581W WO 2024007478 A1 WO2024007478 A1 WO 2024007478A1
Authority
WO
WIPO (PCT)
Prior art keywords
human body
implicit
dimensional
patch
mobile phone
Prior art date
Application number
PCT/CN2022/125581
Other languages
English (en)
French (fr)
Inventor
鲍虎军
孙佳明
罗运升
余致远
赵洪城
周晓巍
Original Assignee
杭州像衍科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 杭州像衍科技有限公司 filed Critical 杭州像衍科技有限公司
Priority to US18/542,825 priority Critical patent/US12014463B2/en
Publication of WO2024007478A1 publication Critical patent/WO2024007478A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—3D [Three Dimensional] animation
    • G06T13/40—3D [Three Dimensional] animation of characters, e.g. humans, animals or virtual beings
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/08—Indexing scheme for image data processing or generation, in general involving all processing steps from image acquisition to 3D model generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Definitions

  • the invention belongs to the field of computer vision, and in particular relates to a method and system for collecting and reconstructing three-dimensional human body modeling data based on a single mobile phone.
  • Human body reconstruction is the basis for interactive and immersive applications such as virtual reality and augmented reality content creation, film and television creation, and virtual dressing.
  • High-quality human body reconstruction is the prerequisite for many application scenarios related to digital humans in virtual reality and augmented reality.
  • the current human body acquisition and reconstruction solutions are mainly divided into two types: using a professional multi-camera acquisition system to scan the human body to collect data. This acquisition system is very expensive and occupies a large area, which limits the large-scale use and use of high-precision human body reconstruction. commercialize.
  • Another type of method uses a single portable device such as a smartphone instead of professional equipment for image acquisition, and uses a multi-view stereo reconstruction method for human body reconstruction. This type of method has weak processing capabilities for the sparse parts of the human body texture, and cannot model the tiny movements of the human body during the acquisition process, which easily leads to low integrity of the reconstruction results and is completely unable to meet the requirements for high-precision human body reconstruction.
  • the purpose of the present invention is to provide a method and system for collecting and reconstructing three-dimensional human body modeling data based on a single mobile phone, so as to solve the problems existing in the above-mentioned traditional static human body model reconstruction scheme.
  • a method for collecting and reconstructing three-dimensional human body modeling data based on a single mobile phone including the following steps:
  • S1 data collection based on augmented reality technology, including:
  • the subject is standing in the scene.
  • the user uses the mobile phone to circle the subject, fits the human body parameterized template grid according to the body shape and posture of the subject captured from multiple perspectives, and uses augmented reality technology to combine the human body with the human body.
  • the parametric template grid is rendered at the scene position where the subject is standing, approximately achieving the visual effect of the parametric template grid of the human body coinciding with the subject;
  • S1.2 use the human body parameterized template grid obtained by fitting to guide the user's data collection process. During the data collection process, it is judged whether the observation of a single patch on the human body parameterized template grid is valid under the current perspective. When a single patch is When a patch is effectively observed by a sufficient number of viewing angles, the color mark of this patch is changed to indicate to the user that the position of this patch has been collected, and the user moves the viewing angle to the part of the human body parameterized template grid that has not been sufficiently observed;
  • step S1.1 the subject is required to stand in the center of the scene and keep the human body surface in a posture that is conducive to reconstruction, and the user uses the mobile phone to circle the subject.
  • step S1.1 is specifically:
  • the human body parameterized template grid is fitted to the subject's position to achieve the visual effect of the human body parameterized template grid and the subject being visually coincident.
  • the user will use the human body parameterized template network according to the obtained Complete data collection according to the specific guidelines.
  • step S1.2 determining whether the observation of a single patch at the current perspective is valid includes:
  • the line connecting the camera optical center and the center point of the patch is calculated. If the angle between this line and the normal vector of the patch is less than the set line of sight angle threshold, the patch is deemed to be valid under the current viewing angle.
  • the sight angle standard for observation is used.
  • step S1.2 if the patch satisfies both the distance standard and the line of sight angle standard for effective observation when observed from a certain viewing angle, then the effective observation count of the patch is increased by one. If the effective observation count of the patch is When the set quantity threshold is reached, it is considered that this patch has a sufficient number of observations. The color mark of this patch is changed to indicate to the user that the location of this patch has been collected. The camera is moved to collect areas that have not yet been sufficiently observed. When the human body parameters After all the patches on the template grid change color, the data collection process is completed.
  • step S2.1 is specifically: perform sparse reconstruction of the image sequence through the motion recovery structure method.
  • the input of the motion recovery structure method is a series of image frames captured with a mobile phone around the human body, and the output is the camera pose and pose corresponding to these images. Camera intrinsics and sparse point clouds reconstructed from these images.
  • step S2.2 is specifically:
  • the implicit spatial deformation field estimation model, implicit signed distance field estimation model and implicit color estimation model are performed on the input image set using volume rendering. Optimize and obtain an implicit three-dimensional human body model
  • the implicit signed distance field of the deformable implicit neural radiation field is post-processed through the isosurface extraction method to obtain an explicit three-dimensional human body model.
  • step S2.2
  • the input of the implicit spatial deformation field estimation model is the three-dimensional point coordinates in the observation frame coordinate system, and the output is the three-dimensional point coordinates in the reference coordinate system;
  • the input of the implicit signed distance field estimation model is the three-dimensional point coordinates in the reference space, and the output is the signed distance and geometric characteristics of the three-dimensional point, which represents the distance from the three-dimensional point to the human body surface;
  • the input of the implicit color estimation model is the geometric characteristics of the three-dimensional points output by the implicit signed distance field estimation model and the vector representing the line of sight, and the output is the color of each sampling point along the specific line of sight direction estimated by the model;
  • sampling point density is calculated based on the signed distance of each sampling point, and the rendering result is obtained through volume rendering technology based on the sampling point density and color.
  • step S2.2 according to the image realism loss function and the signed distance field regularization loss function, back propagation updates the deformation coding and implicit spatial deformation field estimation model and implicit signed distance of each observation frame.
  • a three-dimensional human body modeling data acquisition and reconstruction system based on a single mobile phone.
  • the system includes a data acquisition module and a reconstruction module;
  • Data collection module Using augmented reality technology, the human body parameterized template grid is virtually placed in the collection scene. The user follows the visual guidance on the human body parameterized template grid to collect video data, and extracts image frames from the video data and sends them to reconstruction. module;
  • Reconstruction module Estimate the camera pose and camera intrinsic parameters corresponding to all image frames, use the deformable implicit neural radiation field to perform three-dimensional modeling of the human body, and use volume rendering to estimate the implicit spatial deformation field model and implicitly sign
  • the distance field estimation model and implicit color estimation model are optimized to obtain a three-dimensional human body model.
  • the present invention can only use a single smartphone and use augmented reality technology to guide the user to collect high-quality video data input for the reconstruction algorithm to ensure that the subsequent human body reconstruction algorithm can robustly obtain a high-quality three-dimensional human body model. .
  • the present invention designs a deformable implicit neural radiation field.
  • the use of the implicit spatial deformation field estimation model solves the problem of small movements of the subjects during data collection with a single mobile phone; the use of implicit signed distance fields to represent human body geometry has rich expressive capabilities and improves the accuracy of three-dimensional human model reconstruction.
  • the present invention realizes reliable collection and reconstruction of human body high-quality three-dimensional modeling data based on a single mobile phone.
  • Figure 1 is a flow chart of a method for collecting and reconstructing three-dimensional human body modeling data based on a single mobile phone provided by an embodiment of the present invention
  • Figure 2 is a partial flow chart and effect diagram of data collection provided by the embodiment of the present invention.
  • Figure 3 is a rendering of the resting human body reconstruction results provided by the embodiment of the present invention.
  • Figure 4 is a structural diagram of a human body three-dimensional modeling data acquisition and reconstruction system based on a single mobile phone provided by an embodiment of the present invention.
  • 3D human model reconstruction In the field of 3D human model reconstruction, traditional image-based methods either require the construction of complex acquisition equipment and environments, or are limited by the reconstruction capabilities of traditional multi-view geometric methods. They are unable to reconstruct high-definition images using only a single portable device. Quality 3D human body model.
  • the present invention provides a high-quality three-dimensional human body model reconstruction method based on a deformable implicit neural radiation field, and optimizes the data collection process for the specific task of human body reconstruction, and provides a high-quality human body model using augmented reality technology.
  • Three-dimensional modeling data collection method designed data collection application to guide users to efficiently collect high-quality data for human body reconstruction.
  • Embodiments of the present invention provide a method for collecting and reconstructing three-dimensional human body modeling data based on a single mobile phone, which mainly includes two parts: data collection based on augmented reality technology and high-quality three-dimensional human body model reconstruction based on deformable implicit neural radiation fields.
  • the process is shown in Figure 1, and the specific implementation steps are as follows:
  • the subject is required to stand in the center of the scene and keep the human body surface in a posture that is conducive to reconstruction, such as maintaining an A-shaped posture.
  • the user opens the data collection application in this embodiment through the mobile phone.
  • the data collection application will run real-time positioning and mapping algorithms and human body key point detection algorithms in the background to obtain the camera pose and human body key points on the captured image in real time.
  • this embodiment will automatically fit the human body parameterized template grid according to the body shape and posture of the subject, and use augmented reality technology to render the human body parameterized template grid where the subject is standing.
  • the scene position approximates the visual effect of the human body parameterized template grid coinciding with the subject.
  • the human body parameterized template grid can use any existing human body parameterized template grid model, and the method of fitting the human body parameterized template grid can use any existing human body parameterized template grid fitting from continuous image frames. Grid method.
  • S1.2 based on the human body parameterized template grid fitted by S1.1, guides the user in the data collection process, with the purpose of ensuring that each patch on the human body parameterized template grid is sufficiently observed.
  • the distance between the camera's optical center and the center point of the patch can be calculated.
  • this distance is less than the set distance threshold (set to one meter in this embodiment)
  • the current Under the viewing angle this patch meets the distance standard for effective observation.
  • the line connecting the camera optical center to the center point of the patch can be calculated. If the angle between this line and the normal vector of the patch is less than the set line of sight angle threshold (this implementation In this example, it is set to 60°), then this patch is considered to meet the line of sight angle standard for effective observation under the current viewing angle.
  • the set line of sight angle threshold this implementation In this example, it is set to 60°
  • the effective observation count of the patch is increased by one. If the effective observation count of the patch reaches the set quantity threshold (this embodiment is set to 5), it is considered that this patch has a sufficient number of observations, and the color mark of this patch is changed. In this embodiment, the color of the patch is changed from white to green, indicating to the user that the position collection of this patch has been completed. Move the camera to collect areas that have not been sufficiently observed. When the color of all patches on the human body parameterized template grid has turned green, the data collection process is completed, and the video will be automatically exported to the subsequent reconstruction process.
  • the set quantity threshold this embodiment is set to 5
  • This step can be based on any existing structural approach to motion recovery.
  • This step can use the real-time camera positioning results obtained in S1 as a priori and further optimize based on the motion recovery structure method.
  • This deformable implicit neural radiation field includes the implicit spatial deformation field estimation model R, the implicit signed distance field estimation model S c and the implicit neural radiation field.
  • Formula color estimation model C c Formula color estimation model C c .
  • the input of the implicit spatial deformation field estimation model R is the three-dimensional point coordinates in the observation frame coordinate system, and the output is the three-dimensional point coordinates in the reference coordinate system.
  • the input of the implicit signed distance field estimation model S c is the coordinates of the three-dimensional point in the reference space, and the output is the signed distance and geometric characteristics of the three-dimensional point, which represents the distance from the three-dimensional point to the human body surface.
  • the input of the implicit color estimation model C c is the geometric characteristics of the three-dimensional point output by S c and the vector representing the line of sight, and the output is the color of each sampling point along the specific line of sight direction estimated by the model.
  • the implicit spatial deformation field estimation model R, the implicit signed distance field estimation model S c and the implicit color estimation model C c can all adopt ordinary residual neural network models. After obtaining the signed distance and color of the three-dimensional point from the above S c and C c , the pixel value of the two-dimensional image can be rendered using volume rendering technology. The following details:
  • N c 64 in this embodiment.
  • the implicit spatial deformation field estimation model R is used to obtain the three-dimensional point coordinates corresponding to the coordinates in the observation frame coordinate system and the reference coordinate system.
  • the neural network is optimized during backpropagation.
  • the implicit signed distance field estimation model S c learns the signed distance d sdf and the geometric feature z geo of a three-dimensional point.
  • t is the step size of sampling along the line of sight
  • x c (t) represents the coordinates of the corresponding three-dimensional point when the sampling step length along the line of sight is t
  • S c (x c (t)) is the three-dimensional point x c (t )
  • ⁇ s ( ⁇ ) is the Sigmoid function.
  • this method inputs the line of sight direction and the geometric feature z geo output by the implicit signed distance field estimation model S c into the implicit color estimation model C c , and outputs the color of each sampling point along the line of sight direction v.
  • v represents the line of sight direction calculated from the camera pose coefficient
  • z geo represents the geometric feature output by the implicit signed distance field estimation model at x c (t)
  • n represents the normal vector direction at x c (t)
  • the direction of this normal vector can be obtained by derivation of the estimated signed distance field.
  • RGB represents the three-channel color of red, green and blue of the three-dimensional point.
  • the rendering result C(w,h) can be obtained using the following integration method:
  • C(w,h) represents the rendering color value at the two-dimensional image (w,h)
  • t f and t n represent the farthest and nearest sampling steps along the line of sight direction
  • C c (v,x c ( t)) is the color value of x c (t) in the sight direction v
  • T (t) represents the transmittance at x c (t), which is obtained by integrating ⁇ (t):
  • W and H represent the width and height of the input image respectively
  • (w, h) represents the image pixel coordinates.
  • the present invention also adds a signed distance field regularization loss function
  • a is the number of sight lines
  • N c is the number of sampling points on a single sight line. This formula constrains that the normal vector modulus of all sampling points should be 1.
  • This loss function value is used in the back propagation of the neural network to update each parameter and the deformation encoding of each observation frame.
  • Figure 3 is a rendering of the resting human body reconstruction results provided by an embodiment of the present invention.
  • the present invention also provides embodiments of a human body three-dimensional modeling data collection and reconstruction system based on a single mobile phone.
  • an embodiment of the present invention provides a human body three-dimensional modeling data acquisition and reconstruction system based on a single mobile phone.
  • the system includes a data acquisition module and a reconstruction module;
  • Data collection module Using augmented reality technology, the human body parameterized template grid is virtually placed in the collection scene. The user follows the visual guidance on the human body parameterized template grid to collect video data, and extracts image frames from the video data and sends them to reconstruction. module; the implementation of this module can refer to the above step S1.
  • Reconstruction module Estimate the camera pose and camera intrinsic parameters corresponding to all image frames, use the deformable implicit neural radiation field to perform three-dimensional modeling of the human body, and use volume rendering to estimate the implicit spatial deformation field model and implicitly sign
  • the distance field estimation model and implicit color estimation model are optimized to obtain a three-dimensional human body model.
  • the implementation of this module can refer to the above step S2.
  • a computer device including a memory and a processor.
  • Computer-readable instructions are stored in the memory.
  • the computer-readable instructions When executed by the processor, they cause the processor to execute the above-mentioned embodiments based on a single mobile phone.
  • the steps in the human body three-dimensional modeling data acquisition and reconstruction method.
  • a storage medium that stores computer-readable instructions.
  • the computer-readable instructions are executed by one or more processors, the one or more processors execute the above-mentioned embodiments based on a single mobile phone.
  • the steps in the human body three-dimensional modeling data acquisition and reconstruction method.
  • the storage medium may be a non-volatile storage medium.
  • the program can be stored in a computer-readable storage medium.
  • the storage medium can include: Read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc.
  • first, second, third, etc. may use the terms first, second, third, etc. to describe various information, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other.
  • first information may also be called second information, and similarly, the second information may also be called first information.
  • word “if” as used herein may be interpreted as "when” or “when” or “in response to determining.”

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Graphics (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Geometry (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Processing Or Creating Images (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

本发明公开了一种基于单手机的人体三维建模数据采集与重建方法及系统，本发明在数据采集方面可仅使用单个智能手机，利用增强现实技术引导用户采集对重建算法而言高质量的视频数据输入，以确保后续的人体重建算法能够稳健地得到高质量三维人体模型。在重建算法方面，本发明设计了可变形隐式神经辐射场。隐式空间变形场估计模型的使用解决了单手机采集数据过程中被摄存在微小运动的问题；使用隐式带符号距离场来表示人体几何，表达能力丰富，提高了三维人体模型重建的精度。综合数据采集和重建算法，本发明实现了可靠的基于单手机的人体高质量三维建模数据采集与重建。

Description

基于单手机的人体三维建模数据采集与重建方法及系统 技术领域
本发明属于计算机视觉领域，尤其涉及一种基于单手机的人体三维建模数据采集与重建方法及系统。
背景技术
人体重建是虚拟现实与增强现实内容创作、影视创作和虚拟穿衣等交互沉浸式应用的基础。高质量的人体重建是虚拟现实与增强现实中数字人相关的众多应用场景的前提。目前的人体采集与重建方案主要分为两种：借助专业多相机采集系统对人体进行扫描来采集数据，此种采集系统非常昂贵且占地面积大，限制了高精度人体重建的大规模使用和商业化。另一类方法使用单个便携设备例如智能手机代替专业设备进行图像采集，使用基于多视图立体重建方法进行人体重建。这一类方法对于人体纹理稀疏部分的处理能力弱，且无法建模人体在采集过程中的微小动作，容易导致重建结果完整度低，完全无法满足高精度人体重建的要求。
发明内容
本发明的目的是提供一种基于单手机的人体三维建模数据采集与重建方法及系统，以解决上述传统静止人体模型重建方案中所存在的问题。
本发明的目的是通过以下技术方案实现的：
根据本发明的第一方面，提供一种基于单手机的人体三维建模数据采集与重建方法，包括以下步骤：
S1，基于增强现实技术的数据采集，包括：
S1.1，被摄站立于场景中，用户通过手机环绕被摄一周，根据多视角下拍摄到的被摄的体型和姿态拟合人体参数化模板网格，并利用增强现实技术将所述 人体参数化模板网格渲染在被摄所站立的场景位置，近似达到人体参数化模板网格与被摄重合的视觉效果；
S1.2，利用拟合得到的人体参数化模板网格指引用户数据采集过程，数据采集过程中，判断所述人体参数化模板网格上单个面片在当前视角下的观测是否有效，当单个面片被足够数量的视角有效观测到时，更改此面片的颜色标记，指示用户此面片位置已采集完成，用户将视角移动到还未足够观测的人体参数化模板网格部分；
S2，基于可变形隐式神经辐射场的三维人体模型重建，包括：
S2.1，将S1.2采集的视频抽帧为一系列环绕人体拍摄的图像序列，根据图像间特征点的匹配关系，估计出拍摄图像所对应的相机位姿和相机内参；
S2.2，使用可变形隐式神经辐射场对人体进行三维建模，利用体渲染的方式对隐式空间变形场估计模型、隐式带符号距离场估计模型和隐式颜色估计模型进行优化，得到三维人体模型。
进一步地，步骤S1.1中，要求被摄处于场景中心站立，保持一展开人体表面的利于重建的姿态不动，用户通过手机环绕被摄一周。
进一步地，步骤S1.1具体为：
运行定位与建图算法，实时得到数据采集时的相机位姿；
运行人体关键点检测算法，实时得到所摄图像上的人体关键点位置；
根据得到的相机位姿和人体关键点位置，拟合人体参数化模板网格至被摄位置，达到视觉上人体参数化模板网格与被摄重合的视觉效果，用户将根据人体参数化模板网格的指引完成数据采集。
进一步地，步骤S1.2中，判断单个面片在当前视角下的观测是否有效包括：
基于对相机的实时定位结果，计算相机光心与面片中心点的距离，若此距离小于设定距离阈值，则认为当前视角下此面片符合有效观测的距离标准；
基于对相机的实时定位结果，计算相机光心与面片中心点的连线，若此连线与面片法向量的夹角小于设定视线角度阈值，则认为当前视角下此面片符合有效观测的视线角度标准。
进一步地，步骤S1.2中，若面片在某一视角的观测下，同时满足有效观测的距离标准和视线角度标准，则此面片的有效观测计数加一，若面片的有效观测计数达到设定数量阈值，则认为此面片已经有足够数量的观测，更改此面片的颜色标记，指示用户此面片位置已采集完成，移动相机向还未足够观测的区域采集，当人体参数化模板网格上的全部面片变色后，数据采集过程完成。
进一步地，步骤S2.1具体为：通过运动恢复结构方法对图像序列进行稀疏重建，运动恢复结构方法的输入是围绕人体使用手机拍摄的一系列图像帧，输出是这些图像对应的相机位姿和相机内参以及根据这些图像重建得到的稀疏点云。
进一步地，步骤S2.2具体为：
利用神经网络建立基准空间中表达基准形状的隐式带符号距离场估计模型；
利用神经网络建立基准空间中从特定方向观测三维点的颜色的隐式颜色估计模型；
利用神经网络建立从每一个图像帧对应的观测帧坐标系到基准空间的隐式空间变形场估计模型；
基于S2.1得到的图像对应的相机位姿和相机内参，在输入图像集合上利用体渲染的方式对隐式空间变形场估计模型、隐式带符号距离场估计模型和隐式颜色估计模型进行优化，得到隐式三维人体模型；
通过等值面提取方法对可变形隐式神经辐射场的隐式带符号距离场进行后处理，得到显式三维人体模型。
进一步地，步骤S2.2中，
所述隐式空间变形场估计模型的输入为观测帧坐标系下的三维点坐标，输出为基准坐标系下的三维点坐标；
所述隐式带符号距离场估计模型的输入为基准空间中三维点坐标，输出为三维点的带符号距离和几何特征，表征此三维点到人体表面的距离；
所述隐式颜色估计模型的输入为隐式带符号距离场估计模型输出的三维点的几何特征以及表征视线的向量，输出为模型估计的沿特定视线方向每个采样点的颜色；
根据每个采样点的带符号距离计算得到采样点密度，根据采样点密度和颜色通过体渲染技术得到渲染结果。
进一步地，步骤S2.2中，根据图像真实度损失函数和带符号距离场正则化损失函数，反向传播更新每一个观测帧的形变编码和隐式空间变形场估计模型、隐式带符号距离场估计模型、隐式颜色估计模型。
根据本发明的第二方面，提供一种基于单手机的人体三维建模数据采集与重建系统，该系统包括数据采集模块和重建模块；
数据采集模块：通过增强现实技术在采集场景中虚拟放置人体参数化模板网格，用户跟随所述人体参数化模板网格上的视觉指引采集视频数据，并从视频数据抽取图像帧后送入重建模块；
重建模块：估计所有图像帧所对应的相机位姿和相机内参，使用可变形隐式神经辐射场对人体进行三维建模，利用体渲染的方式对隐式空间变形场估计模型、隐式带符号距离场估计模型和隐式颜色估计模型进行优化，得到三维人体模型。
本发明的有益效果是：
1)本发明在数据采集方面可仅使用单个智能手机，利用增强现实技术引导用户采集对重建算法而言高质量的视频数据输入，以确保后续的人体重建算法能够稳健地得到高质量三维人体模型。
2)在重建算法方面，本发明设计了可变形隐式神经辐射场。隐式空间变形场估计模型的使用解决了单手机采集数据过程中被摄存在微小运动的问题；使用隐式带符号距离场来表示人体几何，表达能力丰富，提高了三维人体模型重建的精度。
综合数据采集和重建算法，本发明实现了可靠的基于单手机的人体高质量三维建模数据采集与重建。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本发明实施例提供的基于单手机的人体三维建模数据采集与重建方法流程图；
图2是本发明实施例提供的数据采集部分流程及效果图；
图3是本发明实施例提供的静止人体重建结果效果图；
图4为本发明实施例提供的基于单手机的人体三维建模数据采集与重建系统结构图。
具体实施方式
下面结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整的描述，显然，所描述的实施例仅仅是本发明的一部分实施例，而不是全部的实施例。基于本发明的实施例，本领域的普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明的保护范围。
在三维人体模型重建领域，传统的基于图像的方法或者是需要复杂的采集设备和环境的构建，或者是受限于传统多视角几何方法的重建能力，都不能够仅使用单个便携设备重建出高质量的三维人体模型。本发明提供了一种基于可变形隐式神经辐射场的高质量三维人体模型重建方法，并针对人体重建这一特定任务，优化了数据采集流程，提供了一种利用增强现实技术的人体高质量三维建模数据采集方法，设计数据采集应用指引用户高效采集高质量数据用于人体重建。
本发明实施例提供一种基于单手机的人体三维建模数据采集与重建方法，主要包括基于增强现实技术的数据采集与基于可变形隐式神经辐射场的高质量三维人体模型重建两部分，方法流程如图1所示，具体实现步骤如下：
S1，基于增强现实技术的数据采集，流程如图2所示。
S1.1，要求被摄在视频采集过程中，处于场景中心站立，保持一展开人体表面的利于重建的姿态不动，例如保持A字形姿态，用户通过手机打开本实施例中的数据采集应用，环绕被摄一周，此过程中，数据采集应用会在后台运行实时定位与建图算法和人体关键点检测算法，实时得到相机位姿和所摄图像上的人体关键点。根据相机定位和人体关键点结果，本实施例将根据被摄的体型和姿态自动拟合人体参数化模板网格，并利用增强现实技术，将此人体参数化模板网格渲染在被摄所站立的场景位置，近似达到人体参数化模板网格与被摄重合的视觉效果。
具体地，人体参数化模板网格可采用任何现有的人体参数化模板网格模型，拟合人体参数化模板网格的方法可采用任何现有的从连续图像帧拟合人体参数化模板网格的方法。
S1.2，基于S1.1拟合得到的人体参数化模板网格指导用户数据采集过程，目的是确保人体参数化模板网格上的每个面片都得到了足够的观测。
在人体参数化模板网格与被摄近似重合的情况下，人体参数化模板网格上的每个面片得到足够观测即意味着被摄被采集数据充分地观测到了。衡量观测是否有效是根据相机光心与面片间的距离和相机视线与面片法向量之间的角度。下面详细介绍具体标准和实践方法：
对于单个面片，基于对相机的实时定位结果，可计算出相机光心与此面片中心点的距离，当此距离小于设定距离阈值(本实施例中设置为一米)时，认为当前视角下此面片符合有效观测的距离标准。
对于单个面片，基于对相机的实时定位结果，可计算出相机光心与此面片中心点的连线，若此连线与面片法向量的夹角小于设定视线角度阈值(本实施例中设置为60°)，则认为当前视角下此面片符合有效观测的视线角度标准。
若面片在某一视角的观测下，同时满足有效观测的距离标准和视线角度标准，则此面片的有效观测计数加一，若面片的有效观测计数达到设定数量阈值(本实施例中设置为5)，则认为此面片已经有足够数量的观测，更改此面片的颜色标记，本实施例中将面片颜色由白变绿，指示用户此面片位置采集已完成， 可移动相机向还未足够观测的区域采集。当人体参数化模板网格上的全部面片颜色均已变绿后，数据采集过程完成，视频将自动导出进入后续重建流程。
S2，基于可变形隐式神经辐射场的高质量三维人体模型重建。
S2.1，将S1.2采集完成的视频抽帧为一系列环绕人体拍摄的图像序列，根据图像间特征点的匹配关系，估计出拍摄图像所对应的相机位姿及相机内参和稀疏点云。
该步骤可基于任何现有的运动恢复结构方法。该步骤可将S1中得到的相机实时定位结果作为先验，基于运动恢复结构方法进行进一步优化。
S2.2，使用可变形隐式神经辐射场对人体进行高精度建模，此可变形隐式神经辐射场包括隐式空间变形场估计模型R、隐式带符号距离场估计模型S c和隐式颜色估计模型C c。
具体地，隐式空间变形场估计模型R的输入为观测帧坐标系下的三维点坐标，输出为基准坐标系下的三维点坐标。隐式带符号距离场估计模型S c的输入为基准空间中三维点坐标，输出为三维点的带符号距离和几何特征，表征此三维点到人体表面的距离。隐式颜色估计模型C c的输入为S c输出的三维点的几何特征以及表征视线的向量，输出为模型估计的沿特定视线方向每个采样点的颜色。隐式空间变形场估计模型R、隐式带符号距离场估计模型S c和隐式颜色估计模型C c均可采用普通残差神经网络模型。由以上S c和C c得到三维点的带符号距离和颜色之后，使用体渲染技术即可渲染得到二维图像的像素值。下面具体说明：
应用体渲染技术需在空间中沿着视线方向在观测帧坐标系下采样N c个三维点x(本实施例中N c＝64)。首先利用隐式空间变形场估计模型R得到观测帧坐标系下的坐标在基准坐标系下对应的三维点坐标。
R：(x，d I)→x c
其中，x代表观测帧坐标系下的三维点，x c代表基准空间下的三维点，d I表示观测帧特定的形变编码，用以处理每一帧下不同的人体运动，此形变编码可在神经网络反向传播过程中被优化。
隐式带符号距离场估计模型S c学习三维点的带符号距离d sdf和几何特征z geo。
S c：(x c)→{d saf，z geo}
根据每个三维点的带符号距离d sdf，可以计算得到其对应的密度ρ(t)。
Figure PCTCN2022125581-appb-000001
其中，t为沿视线方向采样的步长，x c(t)表示沿视线方向采样步长为t时所对应的三维点坐标，S c(x c(t))为三维点x c(t)的带符号距离值，Φ s(·)为Sigmoid函数。
然后，本方法将视线方向和隐式带符号距离场估计模型S c输出的几何特征z geo输入隐式颜色估计模型C c，输出沿视线方向v每个采样点的颜色。
C c(v，x c(t))→C c(v，z geo，n)→RGB
其中，v代表由相机位姿系数计算得到的视线方向，z geo代表x c(t)处隐式带符号距离场估计模型输出的几何特征，n代表x c(t)处的法向量方向，此法向量方向可由对估计出的带符号距离场求导得到，RGB表示三维点的红绿蓝三通道颜色。
在得到每个采样点处的估计出的密度和颜色后，采用如下积分方式即可得到渲染结果C(w,h)：
Figure PCTCN2022125581-appb-000002
其中，C(w,h)代表二维图像(w,h)处的渲染颜色值，t f和t n分别表示沿视线方向的最远和最近采样步长，C c(v,x c(t))为x c(t)在视线方向v下的颜色值，T(t)代表x c(t)处的可透过性，由对ρ(t)积分得到：
Figure PCTCN2022125581-appb-000003
最后，将经过可变形隐式神经辐射场渲染得到的图像C和原图像I一起计算图像真实度损失函数
Figure PCTCN2022125581-appb-000004
Figure PCTCN2022125581-appb-000005
其中，W和H分别表示输入图像的宽和高，(w,h)表示图像像素点坐标。
除图像真实度损失函数外，本发明还添加了带符号距离场正则化损失函数
Figure PCTCN2022125581-appb-000006
用以约束估计出的带符号距离场保持带符号距离场中点的法向量模量为1的数学性质：
Figure PCTCN2022125581-appb-000007
其中，a为视线数量，N c为单个视线上的采样点数量，此式约束所有采样点的法向量模量应为1。
Figure PCTCN2022125581-appb-000008
为遍历到的第k条视线第i个采样点的三维点坐标，
Figure PCTCN2022125581-appb-000009
为三维点
Figure PCTCN2022125581-appb-000010
的带符号距离值。
综合图像真实度损失函数
Figure PCTCN2022125581-appb-000011
和带符号距离场正则化损失函数
Figure PCTCN2022125581-appb-000012
得到完整的损失函数
Figure PCTCN2022125581-appb-000013
Figure PCTCN2022125581-appb-000014
此损失函数值用于神经网络反向传播更新各参数和每一个观测帧的形变编码。
S2.3，通过等值面提取方法对可变形隐式神经辐射场的隐式带符号距离场进行后处理，得到高质量显式三维人体模型。
图3是本发明实施例提供的静止人体重建结果效果图。
与前述基于单手机的人体三维建模数据采集与重建方法的实施例相对应，本发明还提供了基于单手机的人体三维建模数据采集与重建系统的实施例。
参见图4，本发明实施例提供的基于单手机的人体三维建模数据采集与重建系统，该系统包括数据采集模块和重建模块；
数据采集模块：通过增强现实技术在采集场景中虚拟放置人体参数化模板网格，用户跟随所述人体参数化模板网格上的视觉指引采集视频数据，并从视频数据抽取图像帧后送入重建模块；该模块的实现可以参考上述步骤S1。
重建模块：估计所有图像帧所对应的相机位姿和相机内参，使用可变形隐式神经辐射场对人体进行三维建模，利用体渲染的方式对隐式空间变形场估计模型、隐式带符号距离场估计模型和隐式颜色估计模型进行优化，得到三维人体模型。该模块的实现可以参考上述步骤S2。
在一个实施例中，提出了一种计算机设备，包括存储器和处理器，存储器中存储有计算机可读指令，计算机可读指令被处理器执行时，使得处理器执行上述各实施例中基于单手机的人体三维建模数据采集与重建方法中的步骤。
在一个实施例中，提出了一种存储有计算机可读指令的存储介质，计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行上述各实施例中基于单手机的人体三维建模数据采集与重建方法中的步骤。其中，存储介质可以为非易失性存储介质。
本领域普通技术人员可以理解上述实施例的各种方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成，该程序可以存储于一计算机可读存储介质中，存储介质可以包括：只读存储器(ROM，Read Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁盘或光盘等。
还需要说明的是，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、商品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、商品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、商品或者设备中还存在另外的相同要素。
上述对本说明书特定实施例进行了描述。其它实施例在所附权利要求书的范围内。在一些情况下，在权利要求书中记载的动作或步骤可以按照不同于实施例中的顺序来执行并且仍然可以实现期望的结果。另外，在附图中描绘的过程不一定要求示出的特定顺序或者连续顺序才能实现期望的结果。在某些实施方式中，多任务处理和并行处理也是可以的或者可能是有利的。
在本说明书一个或多个实施例使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本说明书一个或多个实施例。在本说明书一个或多个实施例和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。还应当理解，本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。
应当理解，尽管在本说明书一个或多个实施例可能采用术语第一、第二、第三等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本说明书一个或多个实施例范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。取决于语境，如在此所使用的词语“如果”可以被解释成为“在……时”或“当……时”或“响应于确定”。
以上所述仅为本说明书一个或多个实施例的较佳实施例而已，并不用以限制本说明书一个或多个实施例，凡在本说明书一个或多个实施例的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本说明书一个或多个实施例保护的范围之内。

Claims (10)

  1. 一种基于单手机的人体三维建模数据采集与重建方法，其特征在于，包括以下步骤：
    S1，基于增强现实技术的数据采集，包括：
    S1.1，被摄站立于场景中，用户通过手机环绕被摄一周，根据多视角下拍摄到的被摄的体型和姿态拟合人体参数化模板网格，并利用增强现实技术将所述人体参数化模板网格渲染在被摄所站立的场景位置，近似达到人体参数化模板网格与被摄重合的视觉效果；
    S1.2，利用拟合得到的人体参数化模板网格指引用户数据采集过程，数据采集过程中，判断所述人体参数化模板网格上单个面片在当前视角下的观测是否有效，当单个面片被足够数量的视角有效观测到时，更改此面片的颜色标记，指示用户此面片位置已采集完成，用户将视角移动到还未足够观测的人体参数化模板网格部分；
    S2，基于可变形隐式神经辐射场的三维人体模型重建，包括：
    S2.1，将S1.2采集的视频抽帧为一系列环绕人体拍摄的图像序列，根据图像间特征点的匹配关系，估计出拍摄图像所对应的相机位姿和相机内参；
    S2.2，使用可变形隐式神经辐射场对人体进行三维建模，利用体渲染的方式对隐式空间变形场估计模型、隐式带符号距离场估计模型和隐式颜色估计模型进行优化，得到三维人体模型。
  2. 根据权利要求1所述的基于单手机的人体三维建模数据采集与重建方法，其特征在于，步骤S1.1中，要求被摄处于场景中心站立，保持一展开人体表面的利于重建的姿态不动，用户通过手机环绕被摄一周。
  3. 根据权利要求1所述的基于单手机的人体三维建模数据采集与重建方法，其特征在于，步骤S1.1具体为：
    运行定位与建图算法，实时得到数据采集时的相机位姿；
    运行人体关键点检测算法，实时得到所摄图像上的人体关键点位置；
    根据得到的相机位姿和人体关键点位置，拟合人体参数化模板网格至被摄位置，达到视觉上人体参数化模板网格与被摄重合的视觉效果，用户将根据人体参数化模板网格的指引完成数据采集。
  4. 根据权利要求1所述的基于单手机的人体三维建模数据采集与重建方法，其特征在于，步骤S1.2中，判断单个面片在当前视角下的观测是否有效包括：
    基于对相机的实时定位结果，计算相机光心与面片中心点的距离，若此距离小于设定距离阈值，则认为当前视角下此面片符合有效观测的距离标准；
    基于对相机的实时定位结果，计算相机光心与面片中心点的连线，若此连线与面片法向量的夹角小于设定视线角度阈值，则认为当前视角下此面片符合有效观测的视线角度标准。
  5. 根据权利要求4所述的基于单手机的人体三维建模数据采集与重建方法，其特征在于，步骤S1.2中，若面片在某一视角的观测下，同时满足有效观测的距离标准和视线角度标准，则此面片的有效观测计数加一，若面片的有效观测计数达到设定数量阈值，则认为此面片已经有足够数量的观测，更改此面片的颜色标记，指示用户此面片位置已采集完成，移动相机向还未足够观测的区域采集，当人体参数化模板网格上的全部面片变色后，数据采集过程完成。
  6. 根据权利要求1所述的基于单手机的人体三维建模数据采集与重建方法，其特征在于，步骤S2.1具体为：通过运动恢复结构方法对图像序列进行稀疏重建，运动恢复结构方法的输入是围绕人体使用手机拍摄的一系列图像帧，输出是这些图像对应的相机位姿和相机内参以及根据这些图像重建得到的稀疏点云。
  7. 根据权利要求1所述的基于单手机的人体三维建模数据采集与重建方法，其特征在于，步骤S2.2具体为：
    利用神经网络建立基准空间中表达基准形状的隐式带符号距离场估计模型；
    利用神经网络建立基准空间中从特定方向观测三维点的颜色的隐式颜色估计模型；
    利用神经网络建立从每一个图像帧对应的观测帧坐标系到基准空间的隐式空间变形场估计模型；
    基于S2.1得到的图像对应的相机位姿和相机内参，在输入图像集合上利用体渲染的方式对隐式空间变形场估计模型、隐式带符号距离场估计模型和隐式颜色估计模型进行优化，得到隐式三维人体模型；
    通过等值面提取方法对可变形隐式神经辐射场的隐式带符号距离场进行后处理，得到显式三维人体模型。
  8. 根据权利要求1所述的基于单手机的人体三维建模数据采集与重建方法，其特征在于，步骤S2.2中，
    所述隐式空间变形场估计模型的输入为观测帧坐标系下的三维点坐标，输出为基准坐标系下的三维点坐标；
    所述隐式带符号距离场估计模型的输入为基准空间中三维点坐标，输出为三维点的带符号距离和几何特征；
    所述隐式颜色估计模型的输入为隐式带符号距离场估计模型输出的三维点的几何特征以及表征视线的向量，输出为模型估计的沿特定视线方向每个采样点的颜色；
    根据每个采样点的带符号距离计算得到采样点密度，根据采样点密度和颜色通过体渲染技术得到渲染结果。
  9. 根据权利要求1所述的基于单手机的人体三维建模数据采集与重建方法，其特征在于，步骤S2.2中，根据图像真实度损失函数和带符号距离场正则化损失函数，反向传播更新每一个观测帧的形变编码和隐式空间变形场估计模型、隐式带符号距离场估计模型、隐式颜色估计模型。
  10. 一种基于单手机的人体三维建模数据采集与重建系统，其特征在于，该系统包括数据采集模块和重建模块；
    数据采集模块：通过增强现实技术在采集场景中虚拟放置人体参数化模板网格，用户跟随所述人体参数化模板网格上的视觉指引采集视频数据，并从视频数据抽取图像帧后送入重建模块；
    重建模块：估计所有图像帧所对应的相机位姿和相机内参，使用可变形隐式神经辐射场对人体进行三维建模，利用体渲染的方式对隐式空间变形场估计 模型、隐式带符号距离场估计模型和隐式颜色估计模型进行优化，得到三维人体模型。
PCT/CN2022/125581 2022-07-06 2022-10-17 基于单手机的人体三维建模数据采集与重建方法及系统 WO2024007478A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/542,825 US12014463B2 (en) 2022-07-06 2023-12-18 Data acquisition and reconstruction method and system for human body three-dimensional modeling based on single mobile phone

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210788579.7 2022-07-06
CN202210788579.7A CN114863037B (zh) 2022-07-06 2022-07-06 基于单手机的人体三维建模数据采集与重建方法及系统

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/542,825 Continuation US12014463B2 (en) 2022-07-06 2023-12-18 Data acquisition and reconstruction method and system for human body three-dimensional modeling based on single mobile phone

Publications (1)

Publication Number Publication Date
WO2024007478A1 true WO2024007478A1 (zh) 2024-01-11

Family

ID=82626064

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/125581 WO2024007478A1 (zh) 2022-07-06 2022-10-17 基于单手机的人体三维建模数据采集与重建方法及系统

Country Status (2)

Country Link
CN (1) CN114863037B (zh)
WO (1) WO2024007478A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117765187A (zh) * 2024-02-22 2024-03-26 成都信息工程大学 基于多模态深度估计引导的单目隐神经的建图方法
CN117953544A (zh) * 2024-03-26 2024-04-30 安徽农业大学 一种目标行为监测方法及系统

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114863037B (zh) * 2022-07-06 2022-10-11 杭州像衍科技有限公司 基于单手机的人体三维建模数据采集与重建方法及系统
CN116703995B (zh) * 2022-10-31 2024-05-14 荣耀终端有限公司 视频虚化处理方法和装置
CN116468767B (zh) * 2023-03-28 2023-10-13 南京航空航天大学 基于局部几何特征和隐式距离场的飞机表面重建方法
CN117333637B (zh) * 2023-12-01 2024-03-08 北京渲光科技有限公司 三维场景的建模及渲染方法、装置及设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180253895A1 (en) * 2017-03-03 2018-09-06 Augray Pvt. Ltd. System and method for creating a full head 3d morphable model
CN114245000A (zh) * 2020-09-09 2022-03-25 北京小米移动软件有限公司 拍摄方法和装置、电子设备、存储介质
US20220101614A1 (en) * 2020-09-30 2022-03-31 Jean Luo Procedurally generating augmented reality content generators
CN114581571A (zh) * 2022-03-04 2022-06-03 杭州像衍科技有限公司 基于imu和前向变形场的单目人体重建方法及装置
CN114648613A (zh) * 2022-05-18 2022-06-21 杭州像衍科技有限公司 基于可变形神经辐射场的三维头部模型重建方法及装置
CN114863037A (zh) * 2022-07-06 2022-08-05 杭州像衍科技有限公司 基于单手机的人体三维建模数据采集与重建方法及系统

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
ITTO20111150A1 (it) * 2011-12-14 2013-06-15 Univ Degli Studi Genova Rappresentazione stereoscopica tridimensionale perfezionata di oggetti virtuali per un osservatore in movimento
CN112446961A (zh) * 2019-08-30 2021-03-05 中兴通讯股份有限公司 场景重建系统及方法
KR20230062864A (ko) * 2020-11-16 2023-05-09 구글 엘엘씨 변형 가능한 신경 방사 필드
CN112465955B (zh) * 2020-12-10 2023-04-07 浙江大学 一种动态人体三维重建和视角合成方法
CN113421328B (zh) * 2021-05-27 2022-03-11 中国人民解放军军事科学院国防科技创新研究院 一种三维人体虚拟化重建方法及装置
CN114118367B (zh) * 2021-11-16 2024-03-29 上海脉衍人工智能科技有限公司 增量式神经辐射场构建的方法及设备
CN114241113A (zh) * 2021-11-26 2022-03-25 浙江大学 一种基于深度引导采样的高效神经辐射场渲染方法
CN114004941B (zh) * 2022-01-04 2022-08-16 苏州浪潮智能科技有限公司 一种基于神经辐射场的室内场景三维重建系统及方法
CN114119839B (zh) * 2022-01-24 2022-07-01 阿里巴巴（中国）有限公司 三维模型重建与图像生成方法、设备以及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180253895A1 (en) * 2017-03-03 2018-09-06 Augray Pvt. Ltd. System and method for creating a full head 3d morphable model
CN114245000A (zh) * 2020-09-09 2022-03-25 北京小米移动软件有限公司 拍摄方法和装置、电子设备、存储介质
US20220101614A1 (en) * 2020-09-30 2022-03-31 Jean Luo Procedurally generating augmented reality content generators
CN114581571A (zh) * 2022-03-04 2022-06-03 杭州像衍科技有限公司 基于imu和前向变形场的单目人体重建方法及装置
CN114648613A (zh) * 2022-05-18 2022-06-21 杭州像衍科技有限公司 基于可变形神经辐射场的三维头部模型重建方法及装置
CN114863037A (zh) * 2022-07-06 2022-08-05 杭州像衍科技有限公司 基于单手机的人体三维建模数据采集与重建方法及系统

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117765187A (zh) * 2024-02-22 2024-03-26 成都信息工程大学 基于多模态深度估计引导的单目隐神经的建图方法
CN117765187B (zh) * 2024-02-22 2024-04-26 成都信息工程大学 基于多模态深度估计引导的单目隐神经的建图方法
CN117953544A (zh) * 2024-03-26 2024-04-30 安徽农业大学 一种目标行为监测方法及系统

Also Published As

Publication number Publication date
CN114863037B (zh) 2022-10-11
CN114863037A (zh) 2022-08-05
US20240153213A1 (en) 2024-05-09

Similar Documents

Publication Publication Date Title
WO2024007478A1 (zh) 基于单手机的人体三维建模数据采集与重建方法及系统
CN111598998B (zh) 三维虚拟模型重建方法、装置、计算机设备和存储介质
CN109285215B (zh) 一种人体三维模型重建方法、装置和存储介质
WO2021174939A1 (zh) 人脸图像的获取方法与系统
CN111932678B (zh) 多视点实时人体运动、手势、表情、纹理重建系统
WO2022001236A1 (zh) 三维模型生成方法、装置、计算机设备及存储介质
WO2022012192A1 (zh) 三维人脸模型的构建方法、装置、设备及存储介质
US20200334842A1 (en) Methods, devices and computer program products for global bundle adjustment of 3d images
EP3992919A1 (en) Three-dimensional facial model generation method and apparatus, device, and medium
CN108921926A (zh) 一种基于单张图像的端到端三维人脸重建方法
US20240046557A1 (en) Method, device, and non-transitory computer-readable storage medium for reconstructing a three-dimensional model
WO2023284713A1 (zh) 一种三维动态跟踪方法、装置、电子设备和存储介质
CN113689578A (zh) 一种人体数据集生成方法及装置
CN114450719A (zh) 人体模型重建方法、重建系统及存储介质
WO2023066120A1 (zh) 图像处理方法、装置、电子设备及存储介质
CN111754622B (zh) 脸部三维图像生成方法及相关设备
CN115222917A (zh) 三维重建模型的训练方法、装置、设备及存储介质
CN113989434A (zh) 一种人体三维重建方法及设备
CN116681839B (zh) 一种基于改进NeRF的实景三维目标重建与单体化方法
WO2019042028A1 (zh) 全视向的球体光场渲染方法
Xie et al. Hmdo: Markerless multi-view hand manipulation capture with deformable objects
Gava et al. Dense scene reconstruction from spherical light fields
CN109166176B (zh) 三维人脸图像的生成方法与装置
CN115375816A (zh) 可微渲染方法、装置、电子设备及存储介质
US12014463B2 (en) Data acquisition and reconstruction method and system for human body three-dimensional modeling based on single mobile phone

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22950038

Country of ref document: EP

Kind code of ref document: A1