WO2020034785A1 - 三维模型处理方法和装置 - Google Patents

三维模型处理方法和装置 Download PDF

Info

Publication number
WO2020034785A1
WO2020034785A1 PCT/CN2019/095127 CN2019095127W WO2020034785A1 WO 2020034785 A1 WO2020034785 A1 WO 2020034785A1 CN 2019095127 W CN2019095127 W CN 2019095127W WO 2020034785 A1 WO2020034785 A1 WO 2020034785A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
target area
dimensional model
determining
density
Prior art date
Application number
PCT/CN2019/095127
Other languages
English (en)
French (fr)
Inventor
杜成鹏
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2020034785A1 publication Critical patent/WO2020034785A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/005—General purpose rendering architectures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20092—Interactive image processing based on input by user
    • G06T2207/20104—Interactive definition of region of interest [ROI]

Definitions

  • the present application relates to the field of image processing technology, and in particular, to a method and device for processing a three-dimensional model.
  • 3D model reconstruction is a mathematical model suitable for computer representation and processing. It is the basis for processing, manipulating, and analyzing its properties in a computer environment. It is also a key technology for establishing a virtual reality that expresses the objective world in a computer. In related technologies, the key points in the three-dimensional model are processed to realize the reconstruction of the model.
  • the density of key points is high and a more detailed 3D model of the face is generated, a large number of key points will need to be generated, which will not only occupy a large amount of memory space, but also slow the processing speed of the 3D model.
  • fewer key points are used, it will affect the fineness of the 3D model.
  • This application is intended to solve at least one of the technical problems in the related technology.
  • An embodiment of the first aspect of the present application proposes a three-dimensional model processing method.
  • the method includes the following steps: acquiring a three-dimensional model, wherein the three-dimensional model includes multiple key points; and determining a target area in the three-dimensional model that needs to be modified. And determine a target keypoint density corresponding to the target region; obtain a current keypoint density of the target region, compare the current keypoint density with the target keypoint density, and if the current keypoint is known The density is less than the target keypoint density, and a new keypoint is added to the target region so that the current keypoint density of the target region is greater than or equal to the target keypoint density.
  • the embodiment of the second aspect of the present application provides a processing device for a three-dimensional model, including: an acquisition module for acquiring a three-dimensional model, wherein the three-dimensional model includes a plurality of key points; and a determination module for determining the three-dimensional model The target area that needs to be modified and determine the target keypoint density corresponding to the target area; a processing module for obtaining the current keypoint density of the target area, and comparing the current keypoint density with the target keypoint Compare the density. If it is known that the density of the current keypoints is less than the density of the target keypoints, add a keypoint to the target area so that the current keypoint density of the target area is greater than or equal to the target keypoint density. .
  • An embodiment of the third aspect of the present application provides an electronic device including a memory, a processor, and a computer program stored on the memory and executable on the processor.
  • the processor executes the computer program, the processor implements the first
  • the method for processing a three-dimensional model according to the embodiment is not limited to the embodiment.
  • the embodiment of the fourth aspect of the present application proposes a computer-readable storage medium on which a computer program is stored.
  • the computer program is executed by a processor, the method for processing a three-dimensional model according to the foregoing embodiment of the first aspect is implemented.
  • Another embodiment of the present application provides a computer program product.
  • instructions in the computer program product are executed by a processor, the method for processing a three-dimensional model according to the foregoing first embodiment is performed.
  • FIG. 1 is a schematic flowchart of a three-dimensional model processing method according to an embodiment of the present application
  • FIG. 2 is a flowchart of a method for acquiring depth information according to an embodiment of the present application
  • FIG. 3 is a schematic structural diagram of a depth image acquisition component according to an embodiment of the present application.
  • FIG. 4 is a schematic flowchart of another three-dimensional model processing method according to an embodiment of the present application.
  • FIG. 5 is a schematic diagram of an application scenario of a three-dimensional model processing method according to an embodiment of the present application.
  • FIG. 6 is a schematic flowchart of still another three-dimensional model processing method according to an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a split plane according to an embodiment of the present application.
  • FIG. 8 is a schematic flowchart of still another three-dimensional model processing method according to an embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of a three-dimensional model processing device according to an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of another three-dimensional model processing apparatus according to an embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of still another three-dimensional model processing device according to an embodiment of the present application.
  • FIG. 12 is a schematic diagram of an internal structure of an electronic device according to an embodiment
  • FIG. 13 is a schematic diagram of an image processing circuit as a possible implementation manner
  • FIG. 14 is a schematic diagram of an image processing circuit as another possible implementation manner.
  • FIG. 1 is a schematic flowchart of a three-dimensional model processing method according to an embodiment of the present application.
  • the electronic device may be a hardware device such as a mobile phone, a tablet computer, a personal digital assistant, or a wearable device having various operating systems, a touch screen, and / or a display screen.
  • a hardware device such as a mobile phone, a tablet computer, a personal digital assistant, or a wearable device having various operating systems, a touch screen, and / or a display screen.
  • the three-dimensional model processing method includes the following steps:
  • Step 101 Obtain a three-dimensional model, where the three-dimensional model includes multiple key points.
  • the three-dimensional model of the face is actually constructed by key points and a triangle network formed by the key point connections.
  • the establishment of a three-dimensional model of a face requires obtaining two-dimensional images of the face and depth information of the face, which is obtained by aligning the two-dimensional image of the face and the depth information of the face. Specifically, a two-dimensional face image with multiple angles and depth information corresponding to each two-dimensional face image are acquired, so as to be fused with the two-dimensional face image information and depth information based on the multiple angles to be a true two-dimensional person. Face images are more consistent 3D models of faces.
  • the hardware device for obtaining two-dimensional face image information is a visible light RGB image sensor, and a two-dimensional two-dimensional face image can be obtained based on an RGB visible light image sensor in a computer device.
  • the visible light RGB image sensor may include a visible light camera. The visible light camera may capture visible light reflected by the imaging object for imaging, and obtain a two-dimensional face image corresponding to the imaging object.
  • the way to obtain the depth information is through a structured light sensor. Specifically, as shown in FIG. 2, the way to obtain the depth information corresponding to each two-dimensional face image includes the following steps:
  • Step 201 Project structured light onto the face of the current user.
  • Step 202 Take a structured light image modulated by the current user's face.
  • Step 203 Demodulate phase information corresponding to each pixel of the structured light image to obtain depth information corresponding to a two-dimensional face image.
  • the depth image acquisition component 12 includes a structured light projector 121 and a structured light camera 122.
  • Step 201 may be implemented by a structured light projector 121
  • steps 202 and 203 may be implemented by a structured light camera 122.
  • the structured light projector 121 can be used to project structured light onto the face of the current user; the structured light camera 122 can be used to capture the structured light image modulated by the face of the current user, and each pixel corresponding to the demodulated structured light image corresponds to Phase information to get depth information.
  • the structured light projector 121 projects structured light of a certain pattern onto the face of the current user
  • a structured light image modulated by the face of the current user is formed on the surface of the face of the current user.
  • the structured light camera 122 captures the modulated structured light image, and then demodulates the structured light image to obtain depth information.
  • the mode of structured light may be laser fringe, Gray code, sine fringe, non-uniform speckle, and the like.
  • the structured light camera 122 may further be used to demodulate phase information corresponding to each pixel in the structured light image, convert the phase information into depth information, and generate a depth image according to the depth information.
  • the phase information of the modulated structured light has changed.
  • the structured light shown in the structured light image is structured light that has been distorted, and the changed phase information You can characterize the depth information of the object. Therefore, the structured light camera 122 first demodulates phase information corresponding to each pixel in the structured light image, and then calculates depth information according to the phase information.
  • the three-dimensional reconstruction is performed according to the depth information and the two-dimensional face image, and the relevant point depth information and two-dimensional information are given to reconstruct and obtain a three-dimensional model of the face.
  • the three-dimensional model of the face is a three-dimensional stereo model, which can fully restore the human
  • the face, a relatively two-dimensional face model, also includes information such as the three-dimensional angles of the facial features.
  • the three-dimensional reconstruction methods based on depth information and two-dimensional face images to obtain three-dimensional models of human faces include, but are not limited to, the following methods:
  • keypoint recognition is performed on each two-dimensional face image.
  • the plane distance on the face image including the x-axis distance and the y-axis distance in the two-dimensional space, determines the relative position of the positioning key point in the three-dimensional space, and connects adjacent positioning according to the relative position of the positioning key point in the three-dimensional space.
  • the key point is to generate a three-dimensional frame of the face.
  • the key points are characteristic points on the human face, which may include points on the eyes, nose, forehead, mouth corners, cheeks, etc.
  • the positioning key points include points more related to the contours of the user's face, and the positioning key points correspond to the human face Site points where the depth information changes significantly, for example, points on the tip of the nose, points on the nose, points on the corners of the eyes, points on the corners of the mouth, etc. Based on the positioning key points, a three-dimensional frame of the face can be constructed.
  • a two-dimensional face image with two angles and two dimensions is obtained, and a two-dimensional face image with high definition is selected as the original data to locate feature points and use the feature location results to roughly estimate Face angle, according to the angle and contour of the face, a rough three-dimensional deformation model of the face is established, and the facial feature points are adjusted to the same scale as the three-dimensional deformation model of the face by panning and zooming operations, and extracted from the face.
  • the coordinate information of the corresponding points of the feature points forms a three-dimensional deformation model of the sparse face.
  • a particle swarm algorithm is used to iteratively reconstruct the 3D face of the face to obtain a 3D geometric model of the face.
  • the method of texture posting is used to The face texture information in the input two-dimensional image is mapped to the three-dimensional geometric model of the face to obtain a complete three-dimensional model of the face.
  • Step 102 Determine a target area in the three-dimensional model that needs to be modified, and determine a target keypoint density corresponding to the target area.
  • a target region to be modified in the three-dimensional model is determined, and the target region may correspond to a region where key parts related to the current user expression are located, and then the number of target keypoints corresponding to the target region is determined, so that The quantity determines whether the part corresponding to the target area is a refined model.
  • determining the target area in the 3D model that needs to be modified includes:
  • Step 301 Obtain each face area corresponding to the user's different expressions.
  • the three-dimensional model may be divided into multiple facial regions according to a preset radius, where different combinations of facial regions correspond to different parts of the face.
  • Step 302 Obtain the angular distribution of each area, and determine the target area that meets the screening conditions according to the angular distribution.
  • the user's facial expressions are different, and the angles of the positions of the corresponding parts are different.
  • the angle of the area where the eyes are located is relatively curved (reflected in the split surface composed of key points)
  • the slope of the edge is large, etc.)
  • the angle of the mouth area is small, and the degree of bending is small, etc. (the slope of the side of the split surface composed of the key points is small, etc.). Therefore, based on the angular distribution of each region, the target region that meets the screening conditions can be determined according to the angular distribution. For example, when the user laughs, the area where the mouth is more curved and the area where the eyes are located are selected as the target area. For example, when the user makes a grimace, the area where the mouth is relatively curved is selected. The area on the cheek is the target area.
  • the 3D model initially created is displayed to the user in a preview form, and a region selected by the user is received as a target region.
  • a region selected by the user is received as a target region.
  • the area included in the user input track is used as the target area.
  • Step 103 Obtain the current keypoint density of the target area, compare the current keypoint density with the target keypoint density, and if it is known that the current keypoint density is less than the target keypoint density, add keypoints in the target area to make the target area The current keypoint density is greater than or equal to the target keypoint density.
  • the target area is a key area that reflects the authenticity of the current user's expression. Therefore, the degree of refinement of the target area is high.
  • the number of key points in the target area needs to be rich enough to express the user's authenticity. emotion.
  • Take the target area as the area where the mouth is located as an example. As shown in the left figure of Figure 5, when the number of target key points is small and the density is sparse, the posture of the positioned mouth is more distorted, as shown in the right figure of Figure 5, After the number of target key points in the area where the mouth is located, the user's mouth smile gesture can be truly reflected. Among them, the solid black points in FIG. 5 indicate the key points.
  • the current key point density is compared with the target key point density.
  • the target key point density is calibrated based on a large amount of experimental data to ensure that the modeling of the target area is refined. If it is known that the current key point density is less than the target key point Density, adding key points to the target area so that the current key point density in the target area is greater than or equal to the target key point density.
  • the manner of determining the density of target keypoints corresponding to the target region is different. As a possible implementation manner, as shown in FIG. 6, the target keypoints corresponding to the target region are determined. Density, including:
  • Step 401 Obtain angle information of multiple split planes obtained by connecting adjacent key points in a target area as vertices.
  • the three-dimensional model includes a plurality of key points and a plurality of division planes obtained by connecting adjacent key points as vertices.
  • the key points and the cutting plane can be expressed in the form of three-dimensional coordinates.
  • the angle information of each splitting plane may be an angle with an adjacent splitting plane. After obtaining each splitting plane, the angle between the neighboring splitting planes may be used. Obtain the angle information of each split plane.
  • the angle information of the split plane in the target region there is a certain correspondence between the angle information of the split plane in the target region and the flatness of each region.
  • the angle of the split plane in the target region is larger, it means that the flatness of the region is lower.
  • the smaller the angle of the split plane the flatter the area. If the difference between the flatness of two adjacent regions is lower than the difference threshold, the two adjacent regions are merged, where the difference threshold is preset according to the overall structure of the three-dimensional model.
  • the degree of flatness of each area of the face can be determined by calculating the angle between two adjacent split planes in the target area. For example, when two adjacent split planes in the target area are located in the facial area of the face, the included angle between the adjacent split planes may be 2 degrees, indicating that the facial area of the face is relatively flat; When two adjacent split planes are located in the face area of the human face and the other is located in the area where the nose is located, the included angle between the adjacent split planes may be 60 degrees. The flatness is relatively low.
  • Step 402 Determine the target keypoint density of the target area according to the angle information.
  • the target keypoint density of the target region is determined according to the angle information, for example, the target keypoint density corresponding to the target region is further determined according to the flatness of the target region. Specifically, when it is determined that the target area is relatively flat, the corresponding target keypoints in the target area can be set relatively small; when the determination area is relatively low, the corresponding target keypoints in the target area can be set More key points.
  • determining the flatness of the target area according to the angle information includes:
  • Step 501 Determine a normal vector of each split plane in the target area.
  • the three-dimensional model may be divided into a plurality of regions according to a preset radius, and adjacent key points are connected as vertices in each region to obtain a plurality of split planes.
  • the normal vector of each division plane is further determined, wherein the normal vector of the plane is an important vector for determining the position of the plane and refers to a non-zero vector perpendicular to the plane.
  • Step 502 Determine the normal vector of the same vertex according to the normal vector of the split plane containing the same vertex.
  • the normal vectors of the plurality of division planes including the same vertex are summed, and then the normal vectors obtained by the sum are the normal vectors of the vertices.
  • any vertex X in a three-dimensional model there are three split planes A, B, and C in the model that also contain vertex X. Then, after determining the normal vectors of the split planes A, B, and C, the The normal vector is summed, and the vector obtained by the sum is the vector of vertex X.
  • the reflection of light depends on the setting of the vertex normal vector. If the vertex normal vectors are calculated correctly, the displayed three-dimensional model is smooth and shiny, otherwise, the displayed three-dimensional model will be There are sharp edges and ambiguities.
  • Step 503 Determine the flatness of the target region according to the included angle between the normal vectors of adjacent vertices in each region.
  • the normal vector of each vertex in the three-dimensional model is determined. For each vertex in each region in the three-dimensional model, determine the angle between the normal vector of each vertex and the normal vector of adjacent vertices. Further, the normal vector of each vertex determined in the same target region The angle between the normal vectors of the vertices, and the average of the angles is calculated. Finally, determine whether the average value of the included angle of each target area is greater than a preset angle threshold, and then determine whether the target area is flat.
  • the angle threshold is a value set in advance according to the overall structure of the three-dimensional model.
  • the target area is not flat.
  • the target region is flat.
  • the three-dimensional model processing method of the embodiment of the present application obtains a three-dimensional model.
  • the three-dimensional model includes a plurality of key points, determines a target area in the three-dimensional model that needs to be modified, and determines a target key point density corresponding to the target area.
  • To obtain the current keypoint density of the target area compare the current keypoint density with the target keypoint density, and if you know that the current keypoint density is less than the target keypoint density, add a keypoint to the target area to make the current keypoint of the target area
  • the point density is greater than or equal to the target key point density. Therefore, this method maintains the accuracy of the details of the 3D model by increasing the density of key points in the relevant area in the 3D model, and only increases the key points in the target area, avoiding causing significant pressure on the memory and balancing the processing speed .
  • a three-dimensional model processing device is also provided in the present application.
  • FIG. 9 is a schematic structural diagram of a three-dimensional model processing apparatus according to an embodiment of the present application.
  • the three-dimensional model processing apparatus includes: an obtaining module 10, a determining module 20, and a processing module 30.
  • the obtaining module 10 is configured to obtain a three-dimensional model, where the three-dimensional model includes a plurality of key points.
  • a determining module 20 is configured to determine a target area in the three-dimensional model that needs to be modified, and determine a target keypoint density corresponding to the target area.
  • the processing module 30 is configured to obtain the current key point density of the target area, compare the current key point density with the target key point density, and if it is learned that the current key point density is less than the target key point density, Adding key points to the target area, so that the current key point density of the target area is greater than or equal to the target key point density.
  • the obtaining module 10 is specifically configured to:
  • a three-dimensional face model is obtained based on the two-dimensional face images at multiple angles and the depth information corresponding to each two-dimensional face image.
  • the obtaining module 10 is further configured to:
  • the determining module 20 includes a first obtaining unit 21 and a second obtaining unit 22.
  • the first acquiring unit 21 is configured to acquire regions of a face corresponding to different expressions of the user.
  • the second obtaining unit 22 is configured to obtain the angular distribution of each region, and determine a target region that meets the screening condition according to the angular distribution.
  • the processing module 30 includes a third obtaining unit 31 and a determining unit 32, where:
  • the third obtaining unit 31 is configured to obtain angle information of multiple split planes obtained by connecting adjacent key points in the target area as vertices.
  • a determining unit 32 is configured to determine a target keypoint density of a target area according to the angle information.
  • the determining unit 32 includes:
  • a first determining subunit configured to determine the flatness of the target area according to the angle information
  • a second determining subunit is configured to determine a target keypoint density of the target area according to the flatness.
  • the angle information includes a normal vector
  • the first determining subunit is specifically configured to:
  • the degree of flatness of the target region is determined according to the included angle between the normal vectors of adjacent vertices in each region.
  • the first determining subunit is further configured to:
  • the first determining subunit is further configured to:
  • the three-dimensional model processing device in the embodiment of the present application acquires a three-dimensional model, where the three-dimensional model includes multiple key points, determining a target area in the three-dimensional model that needs to be modified, and determining a target key point density corresponding to the target area, and further To obtain the current keypoint density of the target area, compare the current keypoint density with the target keypoint density, and if you know that the current keypoint density is less than the target keypoint density, add a keypoint to the target area to make the current keypoint of the target area The point density is greater than or equal to the target key point density. Therefore, this method maintains the accuracy of the details of the 3D model by increasing the density of key points in the relevant area in the 3D model, and only increases the key points in the target area, avoiding causing significant pressure on the memory and balancing the processing speed .
  • the present application also proposes an electronic device, which is characterized in that it includes: a memory, a processor, and a computer program stored on the memory and executable on the processor.
  • the processor executes the program, it implements The three-dimensional model processing method according to the foregoing embodiment.
  • FIG. 12 is a schematic diagram of the internal structure of the electronic device 200 in an embodiment.
  • the electronic device 200 includes a processor 220, a memory 230, a display 240, and an input device 250 connected through a system bus 210.
  • the memory 230 of the electronic device 200 stores an operating system and computer-readable instructions.
  • the computer-readable instructions can be executed by the processor 220 to implement the three-dimensional model processing method in the embodiment of the present application.
  • the processor 220 is used to provide computing and control capabilities to support the operation of the entire electronic device 200.
  • the display 240 of the electronic device 200 may be a liquid crystal display or an electronic ink display, and the input device 250 may be a touch layer covered on the display 240, or a button, a trackball, or a touchpad provided on the housing of the electronic device 200. It can also be an external keyboard, trackpad, or mouse.
  • the electronic device 200 may be a mobile phone, a tablet computer, a notebook computer, a personal digital assistant, or a wearable device (for example, a smart bracelet, a smart watch, a smart helmet, or smart glasses).
  • FIG. 12 is only a schematic diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the electronic device 200 to which the solution of the present application is applied.
  • the specific electronic device 200 may include more or fewer components than shown in the figure, or some components may be combined, or have different component arrangements.
  • an image processing circuit according to an embodiment of the present application is provided.
  • the image processing circuit may be implemented by using hardware and / or software components.
  • the image processing circuit specifically includes an image unit 310, a depth information unit 320, and a processing unit 330. among them,
  • the image unit 310 is configured to output a two-dimensional image.
  • the depth information unit 320 is configured to output depth information.
  • a two-dimensional image may be acquired through the image unit 310, and depth information corresponding to the image may be acquired through the depth information unit 320.
  • the processing unit 330 is electrically connected to the image unit 310 and the depth information unit 320, respectively, and is configured to construct a three-dimensional model according to the two-dimensional image acquired by the image unit, determine a target area in the three-dimensional model that needs to be modified, and determine The target keypoint density corresponding to the target region, obtaining the current keypoint density of the target region, comparing the current keypoint density with the target keypoint density, if it is learned that the current keypoint density is less than the target Key point density, adding key points to the target area so that the current key point density of the target area is greater than or equal to the target key point density.
  • the two-dimensional image obtained by the image unit 310 may be sent to the processing unit 330, and the depth information corresponding to the image obtained by the depth information unit 320 may be sent to the processing unit 330, and the processing unit 330 may determine the three-dimensional model.
  • the target area that needs to be modified, and the target keypoint density corresponding to the target area determine the current keypoint density of the target area, and compare the current keypoint density with the target keypoint density, if known
  • the current keypoint density is smaller than the target keypoint density, and keypoints are added to the target region so that the current keypoint density of the target region is greater than or equal to the target keypoint density.
  • the image processing circuit may further include:
  • the image unit 310 may specifically include: an electrically connected image sensor 311 and an image signal processing (Image Signal Processing, ISP) processor 312. among them,
  • ISP Image Signal Processing
  • the image sensor 311 is configured to output original image data.
  • the ISP processor 312 is configured to output an image according to the original image data.
  • the original image data captured by the image sensor 311 is first processed by the ISP processor 312.
  • the ISP processor 312 analyzes the original image data to capture image statistics that can be used to determine one or more control parameters of the image sensor 311.
  • Information including images in YUV or RGB format.
  • the image sensor 311 may include a color filter array (such as a Bayer filter), and a corresponding photosensitive unit.
  • the image sensor 311 may obtain light intensity and wavelength information captured by each photosensitive unit, and provide information that can be processed by the ISP processor 312. A set of raw image data.
  • an image in a YUV format or an RGB format is obtained and sent to the processing unit 330.
  • the ISP processor 312 when the ISP processor 312 processes the original image data, it can process the original image data pixel by pixel in a variety of formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and the ISP processor 312 may perform one or more image processing operations on the original image data and collect statistical information about the image data. The image processing operations may be performed with the same or different bit depth accuracy.
  • the depth information unit 320 includes an electrically connected structured light sensor 321 and a depth map generation chip 322. among them,
  • the structured light sensor 321 is configured to generate an infrared speckle pattern.
  • the depth map generation chip 322 is configured to output depth information according to the infrared speckle map; the depth information includes a depth map.
  • the structured light sensor 321 projects speckle structured light onto a subject, obtains the structured light reflected by the subject, and forms an infrared speckle pattern by imaging the reflected structured light.
  • the structured light sensor 321 sends the infrared speckle pattern to the depth map generation chip 322, so that the depth map generation chip 322 determines the morphological change of the structured light according to the infrared speckle pattern, and then determines the depth of the object to obtain a depth map. (Depth map), the depth map indicates the depth of each pixel in the infrared speckle map.
  • the depth map generation chip 322 sends the depth map to the processing unit 330.
  • the processing unit 330 includes a CPU 331 and a GPU (Graphics Processing Unit) 332 which are electrically connected. among them,
  • the CPU 331 is configured to align the image and the depth map according to the calibration data, and output a three-dimensional model according to the aligned image and the depth map.
  • the GPU 332 is configured to determine a target area in the three-dimensional model that needs to be modified, determine a target keypoint density corresponding to the target area, obtain a current keypoint density of the target area, and compare the current keypoint density with all The target keypoint density is compared. If it is known that the current keypoint density is less than the target keypoint density, a keypoint is added to the target area so that the current keypoint density of the target area is greater than or equal to the Target keypoint density.
  • the CPU 331 obtains an image from the ISP processor 312, and obtains a depth map from the depth map generation chip 322. In combination with the calibration data obtained in advance, the two-dimensional image can be aligned with the depth map, thereby determining each Depth information corresponding to pixels. Furthermore, the CPU 331 performs three-dimensional reconstruction based on the depth information and the image to obtain a three-dimensional model.
  • the CPU 331 sends the three-dimensional model to the GPU 332, so that the GPU 332 executes the three-dimensional model processing method as described in the foregoing embodiment according to the three-dimensional model, realizes key point simplification, and obtains a refined three-dimensional model.
  • the GPU 332 may determine a target area in the three-dimensional model that needs to be modified, and determine a target keypoint density corresponding to the target area; obtain a current keypoint density of the target area, and compare the current keypoint density with The target keypoint density is compared. If it is known that the current keypoint density is less than the target keypoint density, a keypoint is added to the target area so that the current keypoint density of the target area is greater than or equal to the target keypoint density. Describe the target key point density.
  • the image processing circuit may further include a display 340.
  • the display 340 is electrically connected to the GPU 332 and is used for displaying a three-dimensional model.
  • the refined three-dimensional model processed by the GPU 332 may be displayed on the display 340.
  • the image processing circuit may further include: an encoder 350 and a memory 360.
  • the refined three-dimensional model processed by the GPU 332 may also be encoded by the encoder 350 and stored in the memory 360, where the encoder 350 may be implemented by a coprocessor.
  • the memory 360 may be multiple, or divided into multiple storage spaces.
  • the image data processed by the storage GPU312 may be stored in a dedicated memory or a dedicated storage space, and may include DMA (Direct Memory Access, direct and direct). Memory access) feature.
  • the memory 360 may be configured to implement one or more frame buffers.
  • the original image data captured by the image sensor 311 is first processed by the ISP processor 312.
  • the ISP processor 312 analyzes the original image data to capture image statistics that can be used to determine one or more control parameters of the image sensor 311.
  • the information including images in YUV format or RGB format, is sent to the CPU 331.
  • the structured light sensor 321 projects speckle structured light onto a subject, acquires the structured light reflected by the subject, and forms an infrared speckle pattern by imaging the reflected structured light.
  • the structured light sensor 321 sends the infrared speckle pattern to the depth map generation chip 322, so that the depth map generation chip 322 determines the morphological change of the structured light according to the infrared speckle pattern, and then determines the depth of the object to obtain a depth map. (Depth Map).
  • the depth map generation chip 322 sends the depth map to the CPU 331.
  • the CPU 331 obtains a two-dimensional image from the ISP processor 312, and obtains a depth map from the depth map generation chip 322. Combined with the calibration data obtained in advance, the face image can be aligned with the depth map, thereby determining the corresponding pixel points in the image. Depth information. Furthermore, the CPU 331 performs three-dimensional reconstruction based on the depth information and the two-dimensional image to obtain a simplified three-dimensional model.
  • the CPU 331 sends the three-dimensional model to the GPU 332, so that the GPU 332 executes the three-dimensional model processing method as described in the foregoing embodiment according to the three-dimensional model, realizes refinement of the three-dimensional model, and obtains a refined three-dimensional model.
  • the simplified three-dimensional model processed by the GPU 332 may be displayed on the display 340, and / or stored in the memory 360 after being encoded by the encoder 350.
  • the present application also proposes a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the three-dimensional model processing method provided by the foregoing embodiment of the application is implemented.
  • the present application also proposes a computer program product, and when the instructions in the computer program product are executed by a processor, the three-dimensional model processing method proposed in the foregoing embodiment is executed.
  • a person of ordinary skill in the art can understand that all or part of the steps carried by the methods in the foregoing embodiments can be implemented by a program instructing related hardware.
  • the program can be stored in a computer-readable storage medium.
  • the program is When executed, one or a combination of the steps of the method embodiment is included.
  • each functional unit in each embodiment of the present application may be integrated into one processing module, or each unit may exist separately physically, or two or more units may be integrated into one module.
  • the above integrated modules may be implemented in the form of hardware or software functional modules. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
  • the aforementioned storage medium may be a read-only memory, a magnetic disk, or an optical disk.

Abstract

本申请提出了一种三维模型处理方法和装置，其中，方法包括：获取三维模型，其中，三维模型包括多个关键点；确定三维模型中需要修正的目标区域，以及确定与目标区域对应的目标关键点密度；获取目标区域的当前关键点密度，将当前关键点密度与目标关键点密度进行比较，若获知当前关键点密度小于目标关键点密度，在目标区域新增关键点，以使目标区域的当前关键点密度大于或者等于目标关键点密度。由此，该方法通过对三维模型中有关区域关键点密度的增加，保持了三维模型的细节精确度，且仅仅在目标区域进行关键点的增加，避免对内存造成较大压力，平衡了处理速度。

Description

三维模型处理方法和装置
相关申请的交叉引用
本申请要求OPPO广东移动通信有限公司于2018年08月16日提交的、发明名称为“三维模型处理方法和装置”的、中国专利申请号“201810935074.2”的优先权。
技术领域
本申请涉及图像处理技术领域，尤其涉及一种三维模型处理方法和装置。
背景技术
三维模型重建是建立适合计算机表示和处理的数学模型，是在计算机环境下对其进行处理、操作和分析其性质的基础，也是在计算机中建立表达客观世界的虚拟现实的关键技术。相关技术中，通过对三维模型中关键点进行处理，实现模型的重建。
申请人发现在实际操作中，对三维模型中各处均采用的是相同的关键点密度进行处理，关键点密度的设置对三维模型的呈现具有较大影响。一方面，如果关键点密度较高，生成较为精细的人脸三维模型，就需要生成大量的关键点，不仅会占用大量的内存空间，而且三维模型处理速度较慢。另一方面，如果采用较少的关键点，则会影响三维模型的精细度。
发明内容
本申请旨在至少在一定程度上解决相关技术中的技术问题之一。
本申请第一方面实施例提出了一种三维模型处理方法，所述方法包括以下步骤：获取三维模型，其中，所述三维模型包括多个关键点；确定所述三维模型中需要修正的目标区域，以及确定与所述目标区域对应的目标关键点密度；获取所述目标区域的当前关键点密度，将所述当前关键点密度与所述目标关键点密度进行比较，若获知所述当前关键点密度小于所述目标关键点密度，在所述目标区域新增关键点，以使所述目标区域的当前关键点密度大于或者等于所述目标关键点密度。
本申请第二方面实施例提出了一种三维模型的处理装置，包括：获取模块，用于获取三维模型，其中，所述三维模型包括多个关键点；确定模块，用于确定所述三维模型中需要修正的目标区域，以及确定与所述目标区域对应的目标关键点密度；处理模块，用于获取所述目标区域的当前关键点密度，将所述当前关键点密度与所述目标关键点密度进行比较，若获知所述当前关键点密度小于所述目标关键点密度，在所述目标区域新增关键点，以使所述目 标区域的当前关键点密度大于或者等于所述目标关键点密度。
本申请第三方面实施例提出了一种电子设备，包括存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述计算机程序时，实现如前述第一方面实施例所述的三维模型的处理方法。
本申请第四方面实施例提出了一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现如前述第一方面实施例所述的三维模型的处理方法。
本申请又一方面实施例提出了一种计算机程序产品，当所述计算机程序产品中的指令由处理器执行时，执行如前述第一方面实施例所述的三维模型的处理方法。
本申请提供的技术方案，至少包括如下有益效果：
获取三维模型，其中，三维模型包括多个关键点，确定三维模型中需要修正的目标区域，以及确定与目标区域对应的目标关键点密度，进而，获取目标区域的当前关键点密度，将当前关键点密度与目标关键点密度进行比较，若获知当前关键点密度小于目标关键点密度，在目标区域新增关键点，以使目标区域的当前关键点密度大于或者等于目标关键点密度。由此，该方法通过对三维模型中有关区域关键点密度的增加，保持了三维模型的细节精确度，且仅仅在目标区域进行关键点的增加，避免对内存造成较大压力，平衡了处理速度。
本申请附加的方面和优点将在下面的描述中部分给出，部分将从下面的描述中变得明显，或通过本申请的实践了解到。
附图说明
本申请上述的和/或附加的方面和优点从下面结合附图对实施例的描述中将变得明显和容易理解，其中：
图1为本申请实施例所提供的一种三维模型处理方法的流程示意图；
图2是根据本申请一个实施例的深度信息的获取方式的流程图；
图3为本申请一个实施例所提供的深度图像采集组件的结构示意图；
图4为本申请实施例所提供的另一种三维模型处理方法的流程示意图；
图5是根据本申请一个实施例的三维模型处理方法的应用场景示意图；
图6为本申请实施例所提供的又一种三维模型处理方法的流程示意图；
图7是根据本申请一个实施例的剖分平面的示意图；
图8为本申请实施例所提供的还一种三维模型处理方法的流程示意图；
图9为本申请实施例提供的一种三维模型处理装置的结构示意图；
图10为本申请实施例提供的另一种三维模型处理装置的结构示意图；
图11为本申请实施例提供的又一种三维模型处理装置的结构示意图；
图12为一个实施例中电子设备的内部结构示意图；
图13为作为一种可能的实现方式的图像处理电路的示意图；
图14为作为另一种可能的实现方式的图像处理电路的示意图。
具体实施方式
下面详细描述本申请的实施例，所述实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的，旨在用于解释本申请，而不能理解为对本申请的限制。
图1为本申请实施例所提供的一种三维模型处理方法的流程示意图。
本申请实施例中，电子设备可以为手机、平板电脑、个人数字助理、穿戴式设备等具有各种操作系统、触摸屏和/或显示屏的硬件设备。
如图1所示，该三维模型处理方法包括以下步骤：
步骤101，获取三维模型，其中，三维模型包括多个关键点。
应当理解的是，人脸三维模型实际上是由关键点以及关键点连接形成的三角网络搭建的。
人脸的三维模型的建立需要获取多张人脸二维图像和人脸的深度信息，通过将人脸的二维图像和人脸的深度信息对齐获取。具体地，获取多个角度的二维人脸图像以及与每个二维人脸图像对应的深度信息，以便与基于多个角度的二维人脸图像信息和深度信息融合为与真实二维人脸图像较为一致的人脸三维模型。
需要说明的是，根据应用场景的不同，本申请的实施例中，获取深度信息和二维的二维人脸图像信息的硬件装置不同：
作为一种可能的实现方式，获取二维人脸图像信息的硬件装置为可见光RGB图像传感器，可以基于计算机设备中的RGB可见光图像传感器获取二维的二维人脸图像。具体地，可见光RGB图像传感器可以包括可见光摄像头，可见光摄像头可以捕获由成像对象反射的可见光进行成像，得到成像对象对应的二维人脸图像。
作为一种可能的实现方式，获取深度信息的方式为通过结构光传感器获取，具体地，如图2所示，获取每个二维人脸图像对应的深度信息的方式包括如下步骤:
步骤201，向当前用户人脸投射结构光。
步骤202，拍摄经当前用户人脸调制的结构光图像。
步骤203，解调结构光图像的各个像素对应的相位信息以得到二维人脸图像对应的深度信息。
在本示例中，参见图3，计算机设备为智能手机1000时，深度图像采集组件12包括结构光投射器121和结构光摄像头122。步骤201可以由结构光投射器121实现，步骤202和步骤203可以由结构光摄像头122实现。
也即是说，结构光投射器121可用于向当前用户人脸投射结构光；结构光摄像头122可用于拍摄经当前用户人脸调制的结构光图像，以及解调结构光图像的各个像素对应的相位信息以得到深度信息。
具体地，结构光投射器121将一定模式的结构光投射到当前用户的人脸上后，在当前用户的人脸的表面会形成由当前用户人脸调制后的结构光图像。结构光摄像头122拍摄经调制后的结构光图像，再对结构光图像进行解调以得到深度信息。其中，结构光的模式可以是激光条纹、格雷码、正弦条纹、非均匀散斑等。
其中，结构光摄像头122可进一步用于解调结构光图像中各个像素对应的相位信息，将相位信息转化为深度信息，以及根据深度信息生成深度图像。
具体地，与未经调制的结构光相比，调制后的结构光的相位信息发生了变化，在结构光图像中呈现出的结构光是产生了畸变之后的结构光，其中，变化的相位信息即可表征物体的深度信息。因此，结构光摄像头122首先解调出结构光图像中各个像素对应的相位信息，再根据相位信息计算出深度信息。
进一步地，根据深度信息和二维人脸图像进行三维重构，赋予相关点深度信息和二维信息，重构获取人脸三维模型，该人脸三维模型为三维立体模型，可以充分还原出人脸，相对二维人脸模型，还包括了人脸的五官的立体角度等信息。
根据应用场景的不同，根据深度信息和二维人脸图像进行三维重构获取人脸三维模型的方式包括但是不限于以下方式：
作为一种可能的实现方式，对每一张二维人脸图像进行关键点识别，对每一张二维人脸图像，根据像素点匹配等技术，根据定位关键点的深度信息和定位关键点在二维人脸图像上的平面距离，包括二维空间上的x轴距离和y轴距离，确定定位关键点在三维空间中的相对位置，根据定位关键点在三维空间中的相对位置，连接相邻的定位关键点，生成人脸三维框架。其中，关键点为人脸上的特征点，可包括眼睛、鼻尖、额头、嘴角上、脸颊上的点等，定位关键点包括与用户人脸轮廓较相关的点，该定位关键点对应于人脸上深度信息明显发生变化的部位点，比如，鼻尖上的点、鼻翼上面的点、眼角上的点、嘴角上的点等，从而，基于该定位关键点可以构建出人脸三维框架。
作为另一种可能的实现方式，获取多个角度二维的二维人脸图像，并筛选出清晰度较高的二维人脸图像作为原始数据，进行特征点定位，利用特征定位结果粗略估计人脸角度，根 据人脸的角度和轮廓建立粗糙的人脸三维形变模型，并将人脸特征点通过平移、缩放操作调整到与人脸三维形变模型在同一尺度上，并抽取出与人脸特征点对应点的坐标信息形成稀疏人脸三维形变模型。
进而，根据人脸角度粗略估计值和稀疏人脸三维形变模型，进行微粒群算法迭代人脸三维重构，得到人脸三维几何模型，在得到人脸三维几何模型后，采用纹理张贴的方法将输入二维图像中的人脸纹理信息映射到人脸三维几何模型，得到完整的人脸三维模型。
步骤102，确定三维模型中需要修正的目标区域，以及确定与目标区域对应的目标关键点密度。
应当理解的是，在本申请的实施例中，为了使得构建的人脸三维模型可以更加真实的反映出人脸表情信息，针对对人脸表情最相关的一些部位进行关键点数量的增加，从而提高了这些部位的模型精细度。
具体地，确定三维模型中需要修正的目标区域，该目标区域可以对应于与当前用户表情相关的关键部位所在区域，进而确定与目标区域对应的目标关键点的数量，以便于根据目标关键点的数量确定目标区域对应的部位是否为精细化的建模。
需要说明的是，根据应用场景的不同，确定三维模型中要修正的目标区域的方式不同，示例说明如下：
第一种示例：
如图4所示，确定三维模型中需要修正的目标区域，包括：
步骤301，获取用户不同表情时对应的面部各区域。
其中，本申请实施例中，可以根据预设的半径，将三维模型划分为多个面部区域，其中不同的面部区域组合对应于面部的不同部位。
步骤302，获取各区域的角度分布情况，根据角度分布情况确定符合筛选条件的目标区域。
应当理解的是，用户的面部表情不一样，其对应的部位所在位置的角度具有不同，比如用户在大笑时，眼睛所在区域的角度相对弯曲程度较大(反映在关键点组成的剖分面的边的斜率较大等)，用户沮丧时，嘴巴所在区域角度弯曲程度小等(反映在关键点组成的剖分面的边的斜率较小等)。由此，可以基于各区域的角度分布情况，根据角度分布情况确定符合筛选条件的目标区域。比如，在用户大笑时，筛选出的是弯曲程度较大的嘴巴所在区域和眼睛所在区域作为目标区域，又比如，在用户做鬼脸时，筛选出的是弯曲程度较大的嘴巴所在区域和脸颊所在区域作为目标区域。
第二种示例：
在本示例中，将最初建立的三维模型以预览的形式展示给用户，接收用户选择的区域作为目标区域。比如，将用户输入轨迹包括的区域作为目标区域等。
步骤103，获取目标区域的当前关键点密度，将当前关键点密度与目标关键点密度进行比较，若获知当前关键点密度小于目标关键点密度，在目标区域新增关键点，以使目标区域的当前关键点密度大于或者等于目标关键点密度。
不难理解的是，目标区域是反应当前用户表情的真实度的关键区域，因而，对目标区域的精细化程度要求较高，需要该目标区域的关键点数量足够丰富，以表达出用户的真实情感。以目标区域为嘴巴所在区域为例，如图5左图所示，当目标关键点的数量较少，密度稀疏时，定位出的嘴巴的姿态较为失真，如图5右图所示，当提高嘴巴所在区域的目标关键点的数量后，可以真实反映出用户嘴巴微笑的姿态，其中，图5中的实心黑点表示关键点。
具体地，将当前关键点密度与目标关键点密度进行比较，该目标关键点密度是根据大量实验数据标定的，用以确保目标区域的建模精细化，若获知当前关键点密度小于目标关键点密度，在目标区域新增关键点，以使目标区域的当前关键点密度大于或者等于目标关键点密度。
需要说明的是，根据应用场景的不同，确定与所述目标区域对应的目标关键点密度的方式不同，作为一种可能的实现方式，如图6所示，确定与目标区域对应的目标关键点密度，包括：
步骤401，获取目标区域中相邻关键点作为顶点进行连线得到的多个剖分平面的角度信息。
正如以上描述的，三维模型，如图7所示，包括多个关键点以及将相邻关键点作为顶点进行连线得到的多个剖分平面。其中，关键点以及剖分平面可以采用三维坐标的形式表示出来。
作为一种可能的实现方式，各剖分平面的角度信息可以是与相邻剖分平面之间的夹角，获得各剖分平面后，通过相邻剖分平面之间的夹角，即可得到各剖分平面的角度信息。
进一步的说明，目标区域内剖分平面的角度信息与各区域的平坦程度存在一定的对应关系，当目标区域内剖分平面的角度越大时，说明该区域平坦程度越低；当目标区域内剖分平面的角度越小时，说明该区域越平坦。若相邻两个区域的平坦程度之间的差异低于差异阈值，则将该相邻两个区域进行合并，其中，差异阈值，是根据三维模型整体的结构预先设定的。
作为一种示例，在人脸三维模型中，通过计算目标区域内两个相邻剖分平面之间的夹角，可判断脸部各区域的平坦程度。例如，当目标区域内两个相邻剖分平面位于人脸的面部区域时，该相邻剖分平面之间的夹角可能为2度，说明人脸的面部区域比较平坦；当目标区域内 的两个相邻剖分平面，一个位于人脸的面部区域时，另一个位于鼻子所在区域时，该相邻剖分平面之间的夹角可能为60度，此时说明该目标区域内的平坦程度比较低。
步骤402，根据角度信息确定目标区域的目标关键点密度。
具体地，根据角度信息确定目标区域的目标关键点密度，比如，根据目标区域的平坦程度，进一步的确定目标区域内对应的目标关键点密度。具体地，当判断目标区域内比较平坦时，该目标区域内对应的目标关键点可以设定相对少一些；当判断区域内平坦程度比较低时，该目标区域内对应的目标关键点可以设定较多的关键点。
需要说明的是，在不同的应用场景下，根据角度信息确定目标区域的平坦程度的方式不同，示例如下：
作为一种可能的实现方式，如图8所示，根据角度信息确定目标区域的平坦程度包括：
步骤501，确定目标区域内各剖分平面的法向量。
本申请实施例中，可以根据预设的半径，将三维模型划分为多个区域，在各个区域内均将相邻关键点作为顶点进行连接，进而得到多个剖分平面。
进一步的，得到各区域的剖分平面后，进一步的确定各剖分平面的法向量，其中，平面的法向量是确定平面位置的重要向量，是指与平面垂直的非零向量。
步骤502，根据包含同一顶点的剖分平面的法向量，确定同一顶点的法向量。
具体地，当三维模型中多个剖分平面包含同一顶点时，对包含同一顶点的多个剖分平面的法向量进行求和，进而求和得到的法向量，即为该顶点的法向量。
例如，对于三维模型中的任意顶点X，在该模型中有三个剖分平面A、B、C同时包含顶点X，则确定剖分平面A、B、C的法向量后，对三个平面的法向量进行求和，求和得到的向量即为顶点X的向量。
需要说明的是，在三维模型中，对光照的反射取决于顶点法向量的设置，如果各顶点法向量计算正确，则显示出的三维模型比较光滑，而且有光泽，否则，显示的三维模型会出现棱角分明，而且模糊不清的情况。
步骤503，根据各区域内相邻顶点的法向量之间的夹角，确定目标区域的平坦程度。
具体地，通过步骤502中确定顶点法向量的方法，确定三维模型中各顶点的法向量。对于三维模型中每一区域内的各顶点，确定各顶点的法向量与相邻顶点的法向量之间的夹角，进一步的，对于在同一目标区域内确定的各顶点的法向量与相邻顶点的法向量之间的夹角，计算夹角的平均值。最后判断得到的每一目标区域的夹角平均值是否大于预设的角度阈值，进而判断该目标区域是否平坦。其中，角度阈值是根据三维模型的整体结构提前设定的值。
当得到三维模型中某一目标区域内各顶点的法向量与相邻顶点的法向量的夹角平均值 大于预设的角度阈值时，则说明该目标区域不平坦。当得到三维模型中某一区域内各顶点的法向量与相邻顶点的法向量的夹角平均值小于预设的角度阈值时，则说明该目标区域平坦。
综上，本申请实施例的三维模型处理方法，获取三维模型，其中，三维模型包括多个关键点，确定三维模型中需要修正的目标区域，以及确定与目标区域对应的目标关键点密度，进而，获取目标区域的当前关键点密度，将当前关键点密度与目标关键点密度进行比较，若获知当前关键点密度小于目标关键点密度，在目标区域新增关键点，以使目标区域的当前关键点密度大于或者等于目标关键点密度。由此，该方法通过对三维模型中有关区域关键点密度的增加，保持了三维模型的细节精确度，且仅仅在目标区域进行关键点的增加，避免对内存造成较大压力，平衡了处理速度。
为了实现上述实施例，本申请还提出一种三维模型处理装置。
图9为本申请实施例提供的一种三维模型处理装置的结构示意图。
如图9所示，该三维模型处理装置包括：获取模块10、确定模块20和处理模块30，其中，
获取模块10，用于获取三维模型，其中，所述三维模型包括多个关键点。
确定模块20，用于确定所述三维模型中需要修正的目标区域，以及确定与所述目标区域对应的目标关键点密度。
处理模块30，用于获取所述目标区域的当前关键点密度，将所述当前关键点密度与所述目标关键点密度进行比较，若获知所述当前关键点密度小于所述目标关键点密度，在所述目标区域新增关键点，以使所述目标区域的当前关键点密度大于或者等于所述目标关键点密度。
作为一种可能的实现方式，所述获取模块10，具体用于：
获取多个角度的二维人脸图像以及与每个二维人脸图像对应的深度信息；
根据所述多个角度的二维人脸图像及所述与每个二维人脸图像对应的深度信息，获取人脸三维模型。
作为另一种可能的实现方式，所述获取模块10，还用于：
向当前用户人脸投射结构光；
拍摄经当前用户人脸调制的结构光图像；
解调所述结构光图像的各个像素对应的相位信息，以得到所述二维人脸图像对应的深度信息。
作为另一种可能的实现方式，如图10所示，在如图9所示的基础上，确定模块20包括 第一获取单元21和第二获取单元22。
其中，第一获取单元21，用于获取用户不同表情时对应的面部各区域。
第二获取单元22，用于获取各区域的角度分布情况，根据角度分布情况确定符合筛选条件的目标区域。
作为另一种可能的实现方式，如图11所示，在如图9所示的基础上，处理模块30包括第三获取单元31和确定单元32，其中，
第三获取单元31，用于获取目标区域中相邻关键点作为顶点进行连线得到的多个剖分平面的角度信息。
确定单元32，用于根据角度信息确定目标区域的目标关键点密度。
作为另一种可能的实现方式，所述确定单元32，包括：
第一确定子单元，用于根据所述角度信息确定所述目标区域的平坦程度；
第二确定子单元，用于根据所述平坦程度确定所述目标区域的目标关键点密度。
作为另一种可能的实现方式，所述角度信息包括法向量，所述第一确定子单元，具体用于：
确定所述目标区域内各剖分平面的法向量；
根据包含同一顶点的剖分平面的法向量，确定所述同一顶点的法向量；
根据各区域内相邻顶点的法向量之间的夹角，确定所述目标区域的平坦程度。
作为另一种可能的实现方式，所述第一确定子单元，还用于：
对所述包含同一顶点的剖分平面的法向量进行求和，以确定所述同一顶点的法向量。
作为另一种可能的实现方式，所述第一确定子单元，还用于：
确定所述目标区域内各顶点的法向量与相邻顶点的法向量之间的夹角的平均值；
判断所述平均值是否大于预设的角度阈值；
若是，则确定所述目标区域不平坦；
若否，则确定所述目标区域平坦。
需要说明的是，前述对三维模型处理方法实施例的解释说明也适用于该实施例的三维模型处理装置，此处不再赘述。
综上，本申请实施例的三维模型处理装置，获取三维模型，其中，三维模型包括多个关键点，确定三维模型中需要修正的目标区域，以及确定与目标区域对应的目标关键点密度，进而，获取目标区域的当前关键点密度，将当前关键点密度与目标关键点密度进行比较，若获知当前关键点密度小于目标关键点密度，在目标区域新增关键点，以使目标区域的当前关键点密度大于或者等于目标关键点密度。由此，该方法通过对三维模型中有关区域关键点密 度的增加，保持了三维模型的细节精确度，且仅仅在目标区域进行关键点的增加，避免对内存造成较大压力，平衡了处理速度。
为了实现上述实施例，本申请还提出电子设备，其特征在于，包括：存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述程序时，实现如前述实施例所述的三维模型处理方法。
图12为一个实施例中电子设备200的内部结构示意图。该电子设备200包括通过系统总线210连接的处理器220、存储器230、显示器240和输入装置250。其中，电子设备200的存储器230存储有操作系统和计算机可读指令。该计算机可读指令可被处理器220执行，以实现本申请实施方式的三维模型处理方法。该处理器220用于提供计算和控制能力，支撑整个电子设备200的运行。电子设备200的显示器240可以是液晶显示屏或者电子墨水显示屏等，输入装置250可以是显示器240上覆盖的触摸层，也可以是电子设备200外壳上设置的按键、轨迹球或触控板，也可以是外接的键盘、触控板或鼠标等。该电子设备200可以是手机、平板电脑、笔记本电脑、个人数字助理或穿戴式设备(例如智能手环、智能手表、智能头盔、智能眼镜)等。
本领域技术人员可以理解，图12中示出的结构，仅仅是与本申请方案相关的部分结构的示意图，并不构成对本申请方案所应用于其上的电子设备200的限定，具体的电子设备200可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
作为一种可能的实现方式，请参阅图13，提供了本申请实施例的图像处理电路，图像处理电路可利用硬件和/或软件组件实现。
如图13，该图像处理电路具体包括：图像单元310、深度信息单元320和处理单元330。其中，
图像单元310，用于输出二维的图像。
深度信息单元320，用于输出深度信息。
本申请实施例中，可以通过图像单元310，获取二维的图像，以及通过深度信息单元320，获取图像对应的深度信息。
处理单元330，分别与图像单元310和深度信息单元320电性连接，用于根据图像单元获取的二维的图像，构建三维模型，确定所述三维模型中需要修正的目标区域，以及确定与所述目标区域对应的目标关键点密度，获取所述目标区域的当前关键点密度，将所述当前关键点密度与所述目标关键点密度进行比较，若获知所述当前关键点密度小于所述目标关键点 密度，在所述目标区域新增关键点，以使所述目标区域的当前关键点密度大于或者等于所述目标关键点密度。
本申请实施例中，图像单元310获取的二维图像可以发送至处理单元330，以及深度信息单元320获取的图像对应的深度信息可以发送至处理单元330，处理单元330可以确定所述三维模型中需要修正的目标区域，以及确定与所述目标区域对应的目标关键点密度，获取所述目标区域的当前关键点密度，将所述当前关键点密度与所述目标关键点密度进行比较，若获知所述当前关键点密度小于所述目标关键点密度，在所述目标区域新增关键点，以使所述目标区域的当前关键点密度大于或者等于所述目标关键点密度。具体的实现过程，可以参见上述实施例中对三维模型处理的方法的解释说明，此处不做赘述。
进一步地，作为本申请一种可能的实现方式，参见图14，在图13所示实施例的基础上，该图像处理电路还可以包括：
作为一种可能的实现方式，图像单元310具体可以包括：电性连接的图像传感器311和图像信号处理(Image Signal Processing，简称ISP)处理器312。其中，
图像传感器311，用于输出原始图像数据。
ISP处理器312，用于根据原始图像数据，输出图像。
本申请实施例中，图像传感器311捕捉的原始图像数据首先由ISP处理器312处理，ISP处理器312对原始图像数据进行分析以捕捉可用于确定图像传感器311的一个或多个控制参数的图像统计信息，包括YUV格式或者RGB格式的图像。其中，图像传感器311可包括色彩滤镜阵列(如Bayer滤镜)，以及对应的感光单元，图像传感器311可获取每个感光单元捕捉的光强度和波长信息，并提供可由ISP处理器312处理的一组原始图像数据。ISP处理器312对原始图像数据进行处理后，得到YUV格式或者RGB格式的图像，并发送至处理单元330。
其中，ISP处理器312在对原始图像数据进行处理时，可以按多种格式逐个像素地处理原始图像数据。例如，每个图像像素可具有8、10、12或14比特的位深度，ISP处理器312可对原始图像数据进行一个或多个图像处理操作、收集关于图像数据的统计信息。其中，图像处理操作可按相同或不同的位深度精度进行。
作为一种可能的实现方式，深度信息单元320，包括电性连接的结构光传感器321和深度图生成芯片322。其中，
结构光传感器321，用于生成红外散斑图。
深度图生成芯片322，用于根据红外散斑图，输出深度信息；深度信息包括深度图。
本申请实施例中，结构光传感器321向被摄物投射散斑结构光，并获取被摄物反射的结 构光，根据反射的结构光成像，得到红外散斑图。结构光传感器321将该红外散斑图发送至深度图生成芯片322，以便深度图生成芯片322根据红外散斑图确定结构光的形态变化情况，进而据此确定被摄物的深度，得到深度图(Depth Map)，该深度图指示了红外散斑图中各像素点的深度。深度图生成芯片322将深度图发送至处理单元330。
作为一种可能的实现方式，处理单元330，包括：电性连接的CPU331和GPU(Graphics Processing Unit，图形处理器)332。其中，
CPU331，用于根据标定数据，对齐图像与深度图，根据对齐后的图像与深度图，输出三维模型。
GPU332，用于确定所述三维模型中需要修正的目标区域，以及确定与所述目标区域对应的目标关键点密度，获取所述目标区域的当前关键点密度，将所述当前关键点密度与所述目标关键点密度进行比较，若获知所述当前关键点密度小于所述目标关键点密度，在所述目标区域新增关键点，以使所述目标区域的当前关键点密度大于或者等于所述目标关键点密度。
本申请实施例中，CPU331从ISP处理器312获取到图像，从深度图生成芯片322获取到深度图，结合预先得到的标定数据，可以将二维图像与深度图对齐，从而确定出图像中各像素点对应的深度信息。进而，CPU331根据深度信息和图像，进行三维重构，得到三维模型。
CPU331将三维模型发送至GPU332，以便GPU332根据三维模型执行如前述实施例中描述的三维模型处理方法，实现关键点简化，得到精细化后的三维模型。
具体地，GPU332可以确定所述三维模型中需要修正的目标区域，以及确定与所述目标区域对应的目标关键点密度；获取所述目标区域的当前关键点密度，将所述当前关键点密度与所述目标关键点密度进行比较，若获知所述当前关键点密度小于所述目标关键点密度，在所述目标区域新增关键点，以使所述目标区域的当前关键点密度大于或者等于所述目标关键点密度。
进一步地，图像处理电路还可以包括：显示器340。
显示器340，与GPU332电性连接，用于对三维模型进行显示。
具体地，GPU332处理得到的精细化后的三维模型，可以由显示器340显示。
可选地，图像处理电路还可以包括：编码器350和存储器360。
本申请实施例中，GPU332处理得到的精细化后的三维模型，还可以由编码器350编码后存储至存储器360，其中，编码器350可由协处理器实现。
在一个实施例中，存储器360可以为多个，或者划分为多个存储空间，存储GPU312处理后的图像数据可存储至专用存储器，或者专用存储空间，并可包括DMA(Direct Memory  Access，直接直接存储器存取)特征。存储器360可被配置为实现一个或多个帧缓冲器。
下面结合图14，对上述过程进行详细说明。
如图14所示，图像传感器311捕捉的原始图像数据首先由ISP处理器312处理，ISP处理器312对原始图像数据进行分析以捕捉可用于确定图像传感器311的一个或多个控制参数的图像统计信息，包括YUV格式或者RGB格式的图像，并发送至CPU331。
如图14所示，结构光传感器321向被摄物投射散斑结构光，并获取被摄物反射的结构光，根据反射的结构光成像，得到红外散斑图。结构光传感器321将该红外散斑图发送至深度图生成芯片322，以便深度图生成芯片322根据红外散斑图确定结构光的形态变化情况，进而据此确定被摄物的深度，得到深度图(Depth Map)。深度图生成芯片322将深度图发送至CPU331。
CPU331从ISP处理器312获取到二维图像，从深度图生成芯片322获取到深度图，结合预先得到的标定数据，可以将人脸图像与深度图对齐，从而确定出图像中各像素点对应的深度信息。进而，CPU331根据深度信息和二维图像，进行三维重构，得到简化的三维模型。
CPU331将三维模型发送至GPU332，以便GPU332根据三维模型执行如前述实施例中描述的三维模型处理方法，实现三维模型的精细化，得到精细化后的三维模型。GPU332处理得到的简化后的三维模型，可以由显示器340显示，和/或，由编码器350编码后存储至存储器360。
为了实现上述实施例，本申请还提出一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现如本申请前述实施例提出的三维模型处理方法。
为了实现上述实施例，本申请还提出一种计算机程序产品，当所述计算机程序产品中的指令由处理器执行时，执行如前述实施例提出的三维模型处理方法。
本技术领域的普通技术人员可以理解实现上述实施例方法携带的全部或部分步骤是可以通过程序来指令相关的硬件完成，所述的程序可以存储于一种计算机可读存储介质中，该程序在执行时，包括方法实施例的步骤之一或其组合。
此外，在本申请各个实施例中的各功能单元可以集成在一个处理模块中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。所述集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。
上述提到的存储介质可以是只读存储器，磁盘或光盘等。
以上所述仅是本申请的优选实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本申请原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本申请的保护范围。

Claims (21)

  1. 一种三维模型处理方法，其特征在于，所述方法包括以下步骤：
    获取三维模型，其中，所述三维模型包括多个关键点；
    确定所述三维模型中需要修正的目标区域，以及确定与所述目标区域对应的目标关键点密度；
    获取所述目标区域的当前关键点密度，将所述当前关键点密度与所述目标关键点密度进行比较，若获知所述当前关键点密度小于所述目标关键点密度，在所述目标区域新增关键点，以使所述目标区域的当前关键点密度大于或者等于所述目标关键点密度。
  2. 如权利要求1所述的方法，其特征在于，所述获取三维模型，包括：
    获取多个角度的二维人脸图像以及与每个二维人脸图像对应的深度信息；
    根据所述多个角度的二维人脸图像及所述与每个二维人脸图像对应的深度信息，获取人脸三维模型。
  3. 如权利要求2所述的方法，其特征在于，获取所述每个二维人脸图像对应的深度信息，包括：
    向当前用户人脸投射结构光；
    拍摄经当前用户人脸调制的结构光图像；
    解调所述结构光图像的各个像素对应的相位信息，以得到所述二维人脸图像对应的深度信息。
  4. 如权利要求1所述的方法，其特征在于，所述确定所述三维模型中需要修正的目标区域，包括：
    获取用户不同表情时对应的面部各区域；
    获取各区域的角度分布情况，根据所述角度分布情况确定符合筛选条件的目标区域。
  5. 如权利要求1所述的方法，其特征在于，所述确定与所述目标区域对应的目标关键点密度，包括：
    获取所述目标区域中相邻关键点作为顶点进行连线得到的多个剖分平面的角度信息；
    根据所述角度信息确定所述目标区域的目标关键点密度。
  6. 如权利要求5所述的方法，其特征在于，所述根据所述角度信息确定所述目标区域的目标关键点密度，包括：
    根据所述角度信息确定所述目标区域的平坦程度；
    根据所述平坦程度确定所述目标区域的目标关键点密度。
  7. 如权利要求6所述的方法，其特征在于，所述角度信息包括法向量，所述根据所述角度信息确定所述目标区域的平坦程度，包括：
    确定所述目标区域内各剖分平面的法向量；
    根据包含同一顶点的剖分平面的法向量，确定所述同一顶点的法向量；
    根据各区域内相邻顶点的法向量之间的夹角，确定所述目标区域的平坦程度。
  8. 如权利要求7所述的方法，其特征在于，所述确定所述同一顶点的法向量，包括：
    对所述包含同一顶点的剖分平面的法向量进行求和，以确定所述同一顶点的法向量。
  9. 如权利要求7所述的方法，其特征在于，所述根据各区域内相邻顶点的法向量之间的夹角，确定所述目标区域的平坦程度，包括：
    确定所述目标区域内各顶点的法向量与相邻顶点的法向量之间的夹角的平均值；
    判断所述平均值是否大于预设的角度阈值；
    若是，则确定所述目标区域不平坦；
    若否，则确定所述目标区域平坦。
  10. 一种三维模型的处理装置，其特征在于，包括：
    获取模块，用于获取三维模型，其中，所述三维模型包括多个关键点；
    确定模块，用于确定所述三维模型中需要修正的目标区域，以及确定与所述目标区域对应的目标关键点密度；
    处理模块，用于获取所述目标区域的当前关键点密度，将所述当前关键点密度与所述目标关键点密度进行比较，若获知所述当前关键点密度小于所述目标关键点密度，在所述目标区域新增关键点，以使所述目标区域的当前关键点密度大于或者等于所述目标关键点密度。
  11. 如权利要求10所述的装置，其特征在于，所述获取模块，具体用于：
    获取多个角度的二维人脸图像以及与每个二维人脸图像对应的深度信息；
    根据所述多个角度的二维人脸图像及所述与每个二维人脸图像对应的深度信息，获取人脸三维模型。
  12. 如权利要求11所述的装置，其特征在于，所述获取模块，还用于：
    向当前用户人脸投射结构光；
    拍摄经当前用户人脸调制的结构光图像；
    解调所述结构光图像的各个像素对应的相位信息，以得到所述二维人脸图像对应的深度信息。
  13. 如权利要求10所述的装置，其特征在于，所述确定模块包括：
    第一获取单元，用于获取用户不同表情时对应的面部各区域；
    第二获取单元，用于获取各区域的角度分布情况，根据所述角度分布情况确定符合筛选条件的目标区域。
  14. 如权利要求10所述的装置，其特征在于，所述处理模块，包括：
    第三获取单元，用于获取所述目标区域中相邻关键点作为顶点进行连线得到的多个剖分平面的角度信息；
    确定单元，用于根据所述角度信息确定所述目标区域的目标关键点密度。
  15. 如权利要求14所述的装置，其特征在于，所述确定单元，包括：
    第一确定子单元，用于根据所述角度信息确定所述目标区域的平坦程度；
    第二确定子单元，用于根据所述平坦程度确定所述目标区域的目标关键点密度。
  16. 如权利要求15所述的装置，其特征在于，所述角度信息包括法向量，所述第一确定子单元，具体用于：
    确定所述目标区域内各剖分平面的法向量；
    根据包含同一顶点的剖分平面的法向量，确定所述同一顶点的法向量；
    根据各区域内相邻顶点的法向量之间的夹角，确定所述目标区域的平坦程度。
  17. 如权利要求16所述的装置，其特征在于，所述第一确定子单元，还用于：
    对所述包含同一顶点的剖分平面的法向量进行求和，以确定所述同一顶点的法向量。
  18. 如权利要求16所述的装置，其特征在于，所述第一确定子单元，还用于：
    确定所述目标区域内各顶点的法向量与相邻顶点的法向量之间的夹角的平均值；
    判断所述平均值是否大于预设的角度阈值；
    若是，则确定所述目标区域不平坦；
    若否，则确定所述目标区域平坦。
  19. 一种电子设备，其特征在于，包括：存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述处理器执行所述计算机程序时，实现如权利要求1-9中任一所述的三维模型处理方法。
  20. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1-9中任一所述的三维模型处理方法。
  21. 一种计算机程序产品，其特征在于，当所述计算机程序产品中的指令由处理器执行时，执行如权利要求1-9中任一所述的三维模型处理方法。
PCT/CN2019/095127 2018-08-16 2019-07-08 三维模型处理方法和装置 WO2020034785A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810935074.2A CN109102559B (zh) 2018-08-16 2018-08-16 三维模型处理方法和装置
CN201810935074.2 2018-08-16

Publications (1)

Publication Number Publication Date
WO2020034785A1 true WO2020034785A1 (zh) 2020-02-20

Family

ID=64849920

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/095127 WO2020034785A1 (zh) 2018-08-16 2019-07-08 三维模型处理方法和装置

Country Status (2)

Country Link
CN (1) CN109102559B (zh)
WO (1) WO2020034785A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111402401A (zh) * 2020-03-13 2020-07-10 北京华捷艾米科技有限公司 一种获取3d人脸数据方法、人脸识别方法及装置
CN112766215A (zh) * 2021-01-29 2021-05-07 北京字跳网络技术有限公司 人脸融合方法、装置、电子设备及存储介质
CN112884870A (zh) * 2021-02-26 2021-06-01 深圳市商汤科技有限公司 一种三维模型的展开方法、电子设备以及计算机存储介质
CN117252991A (zh) * 2023-10-25 2023-12-19 北京华科软科技有限公司 一种体素构造与边界表示的融合方法及三维图形引擎

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109191584B (zh) * 2018-08-16 2020-09-18 Oppo广东移动通信有限公司 三维模型处理方法、装置、电子设备及可读存储介质
CN109102559B (zh) * 2018-08-16 2021-03-23 Oppo广东移动通信有限公司 三维模型处理方法和装置
CN110008873B (zh) * 2019-04-25 2021-06-22 北京华捷艾米科技有限公司 面部表情捕捉方法、系统及设备
CN111489440B (zh) * 2020-04-16 2023-08-29 无锡荣恩科技有限公司 对非标零件的三维扫描建模方法
CN113554741B (zh) * 2020-04-24 2023-08-08 北京达佳互联信息技术有限公司 一种对象三维重建的方法、装置、电子设备及存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1764238A (zh) * 2004-10-18 2006-04-26 欧姆龙株式会社 摄像装置
CN104205826A (zh) * 2012-04-03 2014-12-10 三星泰科威株式会社 用于重建高密度三维图像的设备和方法
CN105513114A (zh) * 2015-12-01 2016-04-20 深圳奥比中光科技有限公司 三维动画生成的方法和装置
US20180082421A1 (en) * 2016-09-22 2018-03-22 Sony Corporation Image processing system and method to reconstruct a three-dimensional (3d) anatomical surface
CN108122250A (zh) * 2016-11-29 2018-06-05 三星电子株式会社 用于立体匹配的方法和设备
CN109102559A (zh) * 2018-08-16 2018-12-28 Oppo广东移动通信有限公司 三维模型处理方法和装置
CN109191584A (zh) * 2018-08-16 2019-01-11 Oppo广东移动通信有限公司 三维模型处理方法、装置、电子设备及可读存储介质

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101188588B1 (ko) * 2008-03-27 2012-10-08 주식회사 만도 모노큘러 모션 스테레오 기반의 주차 공간 검출 장치 및방법
CN101299267A (zh) * 2008-07-02 2008-11-05 北京中星微电子有限公司 一种人脸图像处理方法及装置
CN103971405A (zh) * 2014-05-06 2014-08-06 重庆大学 一种激光散斑结构光及深度信息的三维重建方法
CN107305622B (zh) * 2016-04-15 2021-08-31 北京市商汤科技开发有限公司 一种人脸五官识别方法、装置及系统
CN106952217B (zh) * 2017-02-23 2020-11-17 北京光年无限科技有限公司 面向智能机器人的面部表情增强方法和装置
CN107451560B (zh) * 2017-07-31 2020-05-19 Oppo广东移动通信有限公司 用户表情识别方法、装置及终端
CN108090960B (zh) * 2017-12-25 2019-03-05 北京航空航天大学 一种基于几何约束的目标重建方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1764238A (zh) * 2004-10-18 2006-04-26 欧姆龙株式会社 摄像装置
CN104205826A (zh) * 2012-04-03 2014-12-10 三星泰科威株式会社 用于重建高密度三维图像的设备和方法
CN105513114A (zh) * 2015-12-01 2016-04-20 深圳奥比中光科技有限公司 三维动画生成的方法和装置
US20180082421A1 (en) * 2016-09-22 2018-03-22 Sony Corporation Image processing system and method to reconstruct a three-dimensional (3d) anatomical surface
CN108122250A (zh) * 2016-11-29 2018-06-05 三星电子株式会社 用于立体匹配的方法和设备
CN109102559A (zh) * 2018-08-16 2018-12-28 Oppo广东移动通信有限公司 三维模型处理方法和装置
CN109191584A (zh) * 2018-08-16 2019-01-11 Oppo广东移动通信有限公司 三维模型处理方法、装置、电子设备及可读存储介质

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111402401A (zh) * 2020-03-13 2020-07-10 北京华捷艾米科技有限公司 一种获取3d人脸数据方法、人脸识别方法及装置
CN111402401B (zh) * 2020-03-13 2023-08-18 北京华捷艾米科技有限公司 一种获取3d人脸数据方法、人脸识别方法及装置
CN112766215A (zh) * 2021-01-29 2021-05-07 北京字跳网络技术有限公司 人脸融合方法、装置、电子设备及存储介质
CN112884870A (zh) * 2021-02-26 2021-06-01 深圳市商汤科技有限公司 一种三维模型的展开方法、电子设备以及计算机存储介质
CN117252991A (zh) * 2023-10-25 2023-12-19 北京华科软科技有限公司 一种体素构造与边界表示的融合方法及三维图形引擎
CN117252991B (zh) * 2023-10-25 2024-03-29 北京华科软科技有限公司 一种体素构造与边界表示的融合方法及三维图形引擎

Also Published As

Publication number Publication date
CN109102559A (zh) 2018-12-28
CN109102559B (zh) 2021-03-23

Similar Documents

Publication Publication Date Title
WO2020034785A1 (zh) 三维模型处理方法和装置
EP3614340B1 (en) Methods and devices for acquiring 3d face, and computer readable storage media
WO2020207191A1 (zh) 虚拟物体被遮挡的区域确定方法、装置及终端设备
WO2019228473A1 (zh) 人脸图像的美化方法和装置
WO2020034743A1 (zh) 三维模型处理方法、装置、电子设备及可读存储介质
WO2018119889A1 (zh) 三维场景定位方法和装置
CN104380338B (zh) 信息处理器以及信息处理方法
US11069151B2 (en) Methods and devices for replacing expression, and computer readable storage media
AU2018214005A1 (en) Systems and methods for generating a 3-D model of a virtual try-on product
TW202030697A (zh) 電子裝置及恢復深度圖的方法
EP3992919B1 (en) Three-dimensional facial model generation method and apparatus, device, and medium
US8917317B1 (en) System and method for camera calibration
CN109937434B (zh) 图像处理方法、装置、终端和存储介质
WO2022012085A1 (zh) 人脸图像处理方法、装置、存储介质及电子设备
CN108682050B (zh) 基于三维模型的美颜方法和装置
WO2023093739A1 (zh) 一种多视图三维重建的方法
WO2021027585A1 (zh) 人脸图像处理方法及电子设备
CN109191393B (zh) 基于三维模型的美颜方法
WO2020034738A1 (zh) 三维模型的处理方法、装置、电子设备及可读存储介质
Hernandez et al. Near laser-scan quality 3-D face reconstruction from a low-quality depth stream
CN109242760B (zh) 人脸图像的处理方法、装置和电子设备
US8633926B2 (en) Mesoscopic geometry modulation
CN113793387A (zh) 单目散斑结构光系统的标定方法、装置及终端
Castelán et al. Acquiring height data from a single image of a face using local shape indicators
Anasosalu et al. Compact and accurate 3-D face modeling using an RGB-D camera: let's open the door to 3-D video conference

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19850549

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19850549

Country of ref document: EP

Kind code of ref document: A1