CN113436348A

CN113436348A - Three-dimensional model processing method and device, electronic equipment and storage medium

Info

Publication number: CN113436348A
Application number: CN202110709960.5A
Authority: CN
Inventors: 施侃乐; 李雅子; 郑文
Original assignee: Beijing Dajia Internet Information Technology Co Ltd
Current assignee: Beijing Dajia Internet Information Technology Co Ltd
Priority date: 2021-06-25
Filing date: 2021-06-25
Publication date: 2021-09-24
Anticipated expiration: 2041-06-25
Also published as: CN113436348B

Abstract

The disclosure relates to a three-dimensional model processing method, a three-dimensional model processing device, electronic equipment and a storage medium, and belongs to the technical field of computers. The method comprises the following steps: acquiring light field data and a three-dimensional model, and performing the following iterative process: determining a two-dimensional model view for mapping the three-dimensional model to any viewing angle, and determining a two-dimensional light field view for mapping the light field data to any viewing angle; adjusting the position of the three-dimensional model according to a first difference parameter between the two-dimensional model view and the two-dimensional light field view, so that the coincidence degree of the two-dimensional model view and the two-dimensional light field view of the three-dimensional model adjusted at any visual angle is highest; and stopping the iterative process in response to the iterative process meeting the iterative end condition to obtain the registered three-dimensional model. The three-dimensional model and the light field data are converted into a two-dimensional view, the registration problem in a three-dimensional space can be converted into the registration problem in a two-dimensional space, the registration process is simpler, and the registration precision can be gradually improved by adopting an iterative mode for registration.

Description

Three-dimensional model processing method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a three-dimensional model processing method and apparatus, an electronic device, and a storage medium.

Background

A Light-Field (Light-Field) refers to a data set that records Light information in each direction in a space, and creates a three-dimensional scene in various fields such as virtual reality and games, and acquires Light Field data created three-dimensionally, which records Light information in the three-dimensional scene. Since geometric information is not included in the light field data, the light field data cannot be directly used for processing. A three-dimensional model is typically created for the three-dimensional scene, which is used to describe geometric information in the three-dimensional scene, thereby providing the geometric information as a reference for the processing of the light field data.

However, since the three-dimensional model is created separately from the light field data, the three-dimensional model has a low degree of matching with the light field data. Therefore, it is desirable to provide a method for registering a three-dimensional model with light field data.

Disclosure of Invention

The disclosure provides a three-dimensional model processing method, a three-dimensional model processing device, electronic equipment and a storage medium, which can improve the registration precision of a three-dimensional model and light field data.

According to a first aspect of the embodiments of the present disclosure, there is provided a three-dimensional model processing method, including:

acquiring light field data and a three-dimensional model corresponding to a three-dimensional scene;

performing the following iterative process on the light field data and the three-dimensional model:

determining a two-dimensional model view mapping the three-dimensional model to any view angle, and determining a two-dimensional light field view mapping the light field data to the any view angle;

adjusting the position of the three-dimensional model according to a first difference parameter between the two-dimensional model view and the two-dimensional light field view at any view angle, so that the coincidence degree of the two-dimensional model view of the three-dimensional model adjusted at any view angle and the two-dimensional light field view is highest;

and stopping the iterative process to obtain the registered three-dimensional model in response to the iterative process meeting the iterative end condition.

Optionally, the iteration end condition is: there are a consecutive target number of first difference parameters each being less than the target threshold parameter.

Optionally, the adjusting, according to a first difference parameter between the two-dimensional model view and the two-dimensional light field view at any view angle, a position of the three-dimensional model to maximize a degree of coincidence between the two-dimensional model view of the three-dimensional model adjusted at any view angle and the two-dimensional light field view includes:

determining a first registration parameter of the three-dimensional model according to the first difference parameter, wherein the first registration parameter can enable the coincidence degree of the two-dimensional model view of the three-dimensional model adjusted at any visual angle and the two-dimensional light field view to be highest, and the first registration parameter comprises a translation parameter and a rotation parameter;

in accordance with the first registration parameter, performing at least one of:

controlling the three-dimensional model to translate according to the translation parameters;

and controlling the three-dimensional model to rotate according to the rotation parameters.

Optionally, the determining to map the three-dimensional model to a two-dimensional model view at any viewing angle and determining to map the light field data to a two-dimensional light field view at the any viewing angle comprises:

determining an original two-dimensional model view mapping the three-dimensional model to the any view angle, and determining an original two-dimensional light field view mapping the light field data to the any view angle;

background information in the original two-dimensional model view is removed, and a two-dimensional model view under any view angle is obtained;

and removing background information in the original two-dimensional light field view to obtain the two-dimensional light field view under any view angle.

Optionally, before performing the following iterative process on the light field data and the three-dimensional model, the three-dimensional model processing method further includes:

performing three-dimensional reconstruction on the light field data to obtain first point cloud data corresponding to the light field data, wherein the first point cloud data is used for describing geometric information in the three-dimensional scene;

extracting vertexes in the three-dimensional model, and forming second point cloud data corresponding to the three-dimensional model by using the extracted vertexes;

determining a second registration parameter of the three-dimensional model according to a second difference parameter between the first point cloud data and the second point cloud data, wherein the second registration parameter can enable the contact ratio of the second point cloud data corresponding to the adjusted three-dimensional model and the first point cloud data to be highest;

and adjusting the position of the three-dimensional model according to the second registration parameter.

According to a second aspect of the embodiments of the present disclosure, there is provided a three-dimensional model processing apparatus including:

an acquisition unit configured to perform acquisition of light field data and a three-dimensional model corresponding to a three-dimensional scene;

a registration unit configured to perform the following iterative process on the light field data and the three-dimensional model:

Optionally, the registration unit comprises:

a parameter determining subunit configured to perform determining, according to the first difference parameter, a first registration parameter of the three-dimensional model, the first registration parameter being capable of maximizing a degree of coincidence between a two-dimensional model view of the three-dimensional model adjusted at the any one viewing angle and the two-dimensional light field view, the first registration parameter including a translation parameter and a rotation parameter;

a position adjustment subunit configured to perform, in accordance with the first registration parameter, at least one of:

Optionally, the registration unit comprises:

a view determination subunit configured to perform determining to map the three-dimensional model to an original two-dimensional model view at the any one viewing angle and determining to map the light field data to an original two-dimensional light field view at the any one viewing angle;

a background removing subunit, configured to remove background information in the original two-dimensional model view to obtain a two-dimensional model view under any view angle;

the background removing subunit is further configured to remove background information in the original two-dimensional light field view to obtain the two-dimensional light field view at any viewing angle.

Optionally, the three-dimensional model processing apparatus further includes:

a three-dimensional reconstruction unit configured to perform three-dimensional reconstruction on the light field data to obtain first point cloud data corresponding to the light field data, where the first point cloud data is used to describe geometric information in the three-dimensional scene;

a vertex extraction unit configured to extract vertices in the three-dimensional model, and make the extracted vertices into second point cloud data corresponding to the three-dimensional model;

a registration parameter determination unit configured to determine a second registration parameter of the three-dimensional model according to a second difference parameter between the first point cloud data and the second point cloud data, wherein the second registration parameter can enable the adjusted second point cloud data corresponding to the three-dimensional model to have the highest coincidence degree with the first point cloud data;

a position adjustment unit configured to perform an adjustment of the position of the three-dimensional model in accordance with the second registration parameter.

According to a third aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the three-dimensional model processing method according to the first aspect.

According to a fourth aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium, wherein instructions, when executed by a processor of an electronic device, enable the electronic device to perform the three-dimensional model processing method according to the first aspect.

According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the three-dimensional model processing method as described in the first aspect above.

In the three-dimensional model processing method, the three-dimensional model processing device, the electronic device and the storage medium provided by the embodiment of the disclosure, considering that the light field data itself does not have the geometric information in the three-dimensional scene, the three-dimensional model and the light field data can not be directly registered, thus converting three-dimensional light field data into two-dimensional light field views, converting three-dimensional models into two-dimensional model views, by adjusting the position of the three-dimensional model, the coincidence degree of the two-dimensional light field view and the two-dimensional model view is highest, thereby improving the contact ratio of the three-dimensional model and the light field data, namely converting the registration problem in the three-dimensional space into the registration problem in the two-dimensional space, simplifying the registration process of the three-dimensional model and the light field data, and the registration is carried out in an iterative mode, so that the registration precision of the three-dimensional model and the light field data can be gradually improved in the iterative process.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.

FIG. 1 is a schematic diagram illustrating one implementation environment in accordance with an example embodiment.

FIG. 2 is a flow diagram illustrating a method of processing a three-dimensional model in accordance with an exemplary embodiment.

FIG. 3 is a flow diagram illustrating another method of processing a three-dimensional model in accordance with an exemplary embodiment.

FIG. 4 is a flow diagram illustrating another method of processing a three-dimensional model in accordance with an exemplary embodiment.

Fig. 5 is a block diagram illustrating a three-dimensional model processing apparatus according to an exemplary embodiment.

FIG. 6 is a block diagram illustrating another three-dimensional model processing apparatus according to an exemplary embodiment.

Fig. 7 is a block diagram illustrating a terminal according to an example embodiment.

FIG. 8 is a block diagram illustrating a server in accordance with an exemplary embodiment.

Detailed Description

In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the description of the above-described figures are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

It should be noted that, the user information (including but not limited to user device information, user personal information, etc.) and data referred to in the present disclosure are information and data authorized by the user or sufficiently authorized by each party.

Before the embodiments of the present disclosure are explained in detail, the following explanation of the related concepts is made:

(1) CG (Computer Graphics) is a science that converts two-dimensional or three-dimensional Graphics into a grid form for Computer displays. The research content of computer graphics is to research how graphics are represented in a computer, and the computation, processing and display of the graphics are performed by the computer.

(2) LF (Light Field) describes Light information at any point in space along any direction, and all directional Light sets in the space constitute Light Field data, where the Light information described here is a vector including components of each color model. The light field is a scene representation form different from other forms in the field of computer graphics. The light field represents a three-dimensional scene by recording light ray information scattered in space. The light field has the advantage that the light ray information is directly recorded without the attributes of geometric information, texture information and the like in a three-dimensional scene.

The embodiment of the disclosure provides a three-dimensional model processing method, and an execution main body is electronic equipment. Illustratively, the electronic device is a terminal, and the terminal is a computer, a mobile phone, a tablet computer or other terminals. Illustratively, the electronic device is a server, and the server is a background server or a cloud server providing services such as cloud computing and cloud storage. The electronic equipment acquires light field data and a three-dimensional model corresponding to a three-dimensional scene, and processes the three-dimensional model to register the three-dimensional model and the light field data.

FIG. 1 is a schematic diagram of an implementation environment provided by embodiments of the present disclosure. Referring to fig. 1, the implementation environment includes a terminal 101 and a server 102. The terminal 101 and the server 102 are connected via a wireless or wired network.

Illustratively, the terminal 101 has installed thereon a target application served by the server 102, through which the terminal 101 can implement functions such as creating light field data, creating a three-dimensional model, or data transmission. Illustratively, the target application is a target application in an operating system of the terminal 101 or a target application provided by a third party. The target application has an information display function, for example, the target application has a function of displaying light field data, a three-dimensional model, image information, video information. Of course, the target application can also have other functions, such as a game function, a shopping function, a chat function, and the like. Illustratively, the target application is a short video application, a gaming application, a graphics processing application, a three-dimensional model processing application, or other applications, which are not limited by the embodiments of the present disclosure.

In the embodiment of the present disclosure, the terminal 101 acquires the three-dimensional model and the light field data, sends the three-dimensional model and the light field data to the server 102, registers the three-dimensional model and the light field data by the server 102, and returns the registered three-dimensional model and the light field data to the terminal 101.

The three-dimensional model processing method provided by the embodiment of the disclosure is applied to various scenes.

For example, in the field of games, a virtual three-dimensional scene includes a virtual object and a virtual object, and clicking on the virtual object using a mouse can control the virtual object to pick up the virtual object. The three-dimensional scene is displayed by displaying the light field data, the light field data only comprises light ray information and does not comprise geometric information, and whether the position clicked by the mouse is the position of the virtual object cannot be judged only according to the light field data, so that the virtual object cannot be picked up. Therefore, a three-dimensional model corresponding to the three-dimensional scene needs to be created.

Because the three-dimensional model is created without the light field data and the matching degree of the three-dimensional model and the light field data is low, the method provided by the embodiment of the disclosure is adopted to register the three-dimensional model and the light field data, so that the effect of overlapping the three-dimensional model and the light field data is achieved. And subsequently, when the click operation is executed on the light field data, mapping the click position of the click operation in the light field data to be the click position in the three-dimensional model according to the registration result, and then judging whether the click position is the position of the virtual object according to the geometric information in the three-dimensional model, thereby providing the geometric information for the processing of the light field data as a reference.

FIG. 2 is a flow diagram illustrating a method of processing a three-dimensional model, see FIG. 2, including the steps of:

201. and acquiring light field data and a three-dimensional model corresponding to the three-dimensional scene.

It should be noted that, in the embodiments of the present disclosure, an execution subject is taken as an example of an electronic device, for example, the electronic device is a portable, pocket, handheld, and other types of terminals, such as a mobile phone, a computer, a tablet computer, and the like, or the electronic device is a server and the like. In another embodiment, the execution subject of the three-dimensional model processing method may also be other devices.

The electronic device obtains light field data and a three-dimensional model corresponding to the same three-dimensional scene. The light field data is used for describing light ray information in the three-dimensional scene, and the light ray information comprises component information such as luminance information and chrominance information. A three-dimensional model is used to describe the geometric information in the three-dimensional scene, the three-dimensional model being represented by a plurality of planes of points and lines.

After the electronic device obtains the light field data and the three-dimensional model, the iterative process in the following

step

202 and 203 is performed on the light field data and the three-dimensional model to perform registration on the three-dimensional model and the light field data.

202. Determining a two-dimensional model view that maps the three-dimensional model to any one view angle, and determining a two-dimensional light field view that maps the light field data to any one view angle.

Registering the three-dimensional model and the light field data means that a three-dimensional scene represented by the three-dimensional model is overlapped with a three-dimensional scene represented by the light field data, but the light field data does not include geometric information, so that the three-dimensional model and the light field data cannot be directly registered. The electronic device thus determines, respectively, a two-dimensional model view mapping the three-dimensional model to any viewing angle and a two-dimensional light field view mapping the light field data to any viewing angle, thereby converting the three-dimensional model to a two-dimensional model view and converting the three-dimensional light field data to a two-dimensional light field view.

203. And adjusting the position of the three-dimensional model according to the first difference parameter between the two-dimensional model view and the two-dimensional light field view at any visual angle, so that the coincidence degree of the two-dimensional model view and the two-dimensional light field view of the three-dimensional model adjusted at any visual angle is the highest.

The electronic equipment determines a first difference parameter between the two-dimensional model view and the two-dimensional light field view at any view angle, and adjusts the position of the three-dimensional model according to the determined first difference parameter, so that the coincidence degree of the two-dimensional model view of the adjusted three-dimensional model and the two-dimensional light field view is the highest at any view angle.

204. And stopping the iterative process in response to the iterative process meeting the iterative end condition to obtain the registered three-dimensional model.

The electronic device performs the iterative process in the step 202-203 at least once until the iterative process meets the iteration ending condition, stops the iterative process to obtain the registered three-dimensional model, and then completes the registration of the three-dimensional model and the light field data.

In the method provided by the embodiment of the disclosure, it is considered that the light field data does not have geometric information in a three-dimensional scene, and the three-dimensional model and the light field data cannot be directly registered, so that the three-dimensional light field data is converted into a two-dimensional light field view, the three-dimensional model is converted into a two-dimensional model view, and the coincidence degree of the two-dimensional light field view and the two-dimensional model view is highest by adjusting the position of the three-dimensional model, so that the coincidence degree of the three-dimensional model and the light field data is improved, that is, the registration problem in a three-dimensional space is converted into the registration problem in a two-dimensional space, so that the registration process of the three-dimensional model and the light field data is simpler, and the registration precision of the three-dimensional model and the light field data can be gradually improved in the iterative process.

FIG. 3 is a flow chart illustrating another method of processing a three-dimensional model, see FIG. 3, including the steps of:

301. and acquiring light field data and a three-dimensional model corresponding to the three-dimensional scene.

The electronic equipment acquires light field data and a three-dimensional model corresponding to any one three-dimensional scene, wherein the three-dimensional scene is a real three-dimensional scene or a virtual three-dimensional scene. The light field data is used for describing light ray information in the three-dimensional scene, and the light ray information comprises component information such as luminance information and chrominance information. Three-dimensional models are used to describe the geometric information in the three-dimensional scene, which are composed of vertices and line segments, for example, the three-dimensional models are three-dimensional mesh models, and the meshes are composed of triangles, quadrangles or other simple convex polygons.

In the embodiment of the present disclosure, the light field data is used to represent the three-dimensional scene, but since the light field data does not record geometric information in the three-dimensional scene, such as the outline or the position of an object in the three-dimensional scene, the processing related to the geometric information, such as the position of the object, cannot be performed only based on the light field data. Therefore, a three-dimensional model for describing geometric information is created for the three-dimensional scene, which can roughly describe the outline, position and the like of an object in the three-dimensional scene, thereby providing geometric data for reference for processing light field data. The three-dimensional model is a low-precision three-dimensional model which is created independently from the light field data, so that the matching degree of the three-dimensional model and the light field data is low, and therefore the three-dimensional model and the light field data need to be registered. In the embodiment of the present disclosure, the three-dimensional scene represented by the three-dimensional model is made to coincide with the three-dimensional scene represented by the light field data by adjusting the position of the three-dimensional model.

302. And performing three-dimensional reconstruction on the light field data to obtain first point cloud data corresponding to the light field data.

In the field of computer graphics, when two different types of data are registered, both types of data usually include geometric information, for example, two types of data are a triangular mesh model, voxel construction data, point cloud data or volume data, and the like, and since the data record the geometric information, the registration is possible by the geometric information.

In the embodiment of the present disclosure, registering the three-dimensional model and the light field data means that a three-dimensional scene represented by the three-dimensional model is overlapped with a three-dimensional scene represented by the light field data, but the light field data does not include geometric information, so that the three-dimensional model and the light field data cannot be directly registered. Therefore, the electronic equipment carries out three-dimensional reconstruction on the light field data to obtain first point cloud data corresponding to the light field data. The first point cloud data is a set of a plurality of points, each point has respective position information in the three-dimensional scene, and therefore the first point cloud data can describe geometric information in the three-dimensional scene, so that the electronic device converts light field data which does not include the geometric information into point cloud data which includes the geometric information.

In some embodiments, the electronic device performs feature extraction on two-dimensional light field views at multiple viewing angles to obtain feature points in each two-dimensional light field view; first point cloud data is created from the feature points in each two-dimensional light field view.

The light field data comprises light ray information at each view angle in the three-dimensional scene, so that the electronic equipment can acquire the two-dimensional light field view of the light field data at a plurality of view angles according to the light field data, the two-dimensional light field view of the light field data at a certain view angle refers to an image obtained by observing the three-dimensional scene represented by the light field data at the view angle, and the two-dimensional light field view comprises the light ray information in the three-dimensional scene at the view angle. Therefore, the electronic device can perform feature extraction on the multiple two-dimensional light field views based on a Multi-View reconstruction point cloud technology (MVS) to obtain feature points in each two-dimensional light field View, and create first point cloud data according to the feature points in each two-dimensional light field View, wherein the first point cloud data can roughly represent geometric information in the three-dimensional scene.

The embodiment of the disclosure provides a scheme for reconstructing point cloud data by using multi-view ray information, which performs three-dimensional reconstruction on light field data by means of a two-dimensional light field view under multi-view angles in the light field data, thereby obtaining first point cloud data, realizing conversion of three-dimensional ray information into three-dimensional geometric information, and further providing the geometric information as a reference for a subsequent registration process.

Illustratively, the first point cloud data obtained by the method has an error, and the first point cloud data includes an interference point, where the interference point is an erroneous point in the first point cloud data. And manually correcting the first point cloud data to remove the interference points in the first point cloud data, so that the accuracy of the first point cloud data is improved.

303. And extracting vertexes in the three-dimensional model, and forming second point cloud data corresponding to the three-dimensional model by using the extracted vertexes.

The electronic equipment extracts the vertexes in the three-dimensional model, and the extracted vertexes form second point cloud data corresponding to the three-dimensional model, and the second point cloud data can describe the geometric information in the three-dimensional scene.

304. And determining a second registration parameter of the three-dimensional model according to a second difference parameter between the first point cloud data and the second point cloud data, and adjusting the position of the three-dimensional model according to the second registration parameter.

Because the first point cloud data can describe geometric information in a three-dimensional scene represented by the light field data, and the second point cloud data can describe geometric information in the three-dimensional scene represented by the three-dimensional model, the higher the coincidence degree of the first point cloud data corresponding to the light field data and the second point cloud data corresponding to the three-dimensional model is, the higher the coincidence degree of the three-dimensional scene represented by the light field data and the three-dimensional scene represented by the three-dimensional model is. And under the condition that the second point cloud data corresponding to the adjusted three-dimensional model is superposed with the first point cloud data, the possibility that the three-dimensional scene represented by the light field data is superposed with the three-dimensional scene represented by the three-dimensional model is the largest. Therefore, the electronic device determines a second registration parameter of the three-dimensional model according to a second difference parameter between the first point cloud data and the second point cloud data, and adjusts the position of the three-dimensional model according to the second registration parameter, so that the contact ratio between the second point cloud data corresponding to the adjusted three-dimensional model and the first point cloud data is the highest.

The electronic equipment determines a second difference parameter between the first point cloud data and the second point cloud data, adjusts the second point cloud data according to the second difference parameter so that the coincidence degree of the adjusted second point cloud data and the first point cloud data is the highest, determines a second registration parameter according to the second point cloud data before adjustment and the adjusted second point cloud data, so that the coincidence degree of the second point cloud data corresponding to the adjusted three-dimensional model and the first point cloud data is the highest through the second registration parameter, and adjusts the position of the three-dimensional model according to the second registration parameter.

Wherein the second difference parameter is used to represent the degree of difference between the first point cloud data and the second point cloud data, for example, the second difference parameter may be represented by a one-way hausdorff distance or a two-way hausdorff distance. And adjusting the second point cloud data according to the second difference parameter, so that the process that the contact ratio of the adjusted second point cloud data and the first point cloud data is the highest is the process of adjusting the second point cloud data so that the second difference parameter between the second point cloud data and the first point cloud data is smaller and smaller.

For example, the electronic device first determines an initial adjustment information, which is also called an initial registration parameter, by using a principal component analysis method, for example, the initial adjustment information includes parameters of 6 degrees of freedom, which are a translation amount in an x-axis direction, a translation amount in a y-axis direction, a translation amount in a z-axis direction, a rotation amount of Roll (Roll angle), a rotation amount of Yaw (Yaw angle), and a rotation amount of Pitch (Pitch angle) in a three-dimensional space. The electronic equipment adjusts the second point cloud data according to the initial adjustment information so as to reduce a second difference parameter between the adjusted second point cloud data and the first point cloud data, then searches next adjustment information by adopting an optimization algorithm such as a steepest descent method, a gradient descent method or a simulated annealing method, adjusts the second point cloud data according to the next adjustment information, and reduces the second difference parameter between the second point cloud data and the first point cloud data through multiple iterations. And when the iteration times reach the target times or the second difference parameter is smaller than the target parameter, the second point cloud data is considered to be overlapped with the first point cloud data, and the adjustment of the second point cloud data is completed.

The contact ratio of the adjusted second point cloud data and the first point cloud data is the highest, but the second point cloud data corresponding to the current three-dimensional model is the second point cloud data before adjustment, and if the contact ratio of the second point cloud data corresponding to the three-dimensional model and the first point cloud data is to be the highest, the three-dimensional model is adjusted only according to the adjustment mode of the second point cloud data. Therefore, the electronic device determines a second registration parameter according to the second point cloud data before adjustment and the second point cloud data after adjustment, wherein the second registration parameter is information to be adjusted, and the second point cloud data before adjustment is converted into the second point cloud data after adjustment. The electronic equipment adjusts the three-dimensional model according to the second registration parameter, so that the coincidence degree of the second point cloud data and the first point cloud data corresponding to the three-dimensional model is the highest.

Illustratively, the second registration parameters include a translation parameter and a rotation parameter. The translation parameters include a direction and a distance, and the electronic device controls the three-dimensional model to move the distance along the direction, for example, the translation parameters include a translation amount in an x-axis direction, a translation amount in a y-axis direction, and a translation amount in a z-axis direction. The rotation parameters include a direction and an angle, and the electronic device controls the three-dimensional model to rotate by the angle around the rotation axis indicated by the direction. For example, the rotation parameters include a Roll rotation amount, which is an angle of rotation around the x-axis, a Yaw rotation amount, which is an angle of rotation around the z-axis, and a Pitch rotation amount, which is an angle of rotation around the y-axis.

In the embodiment of the disclosure, because the light field data corresponds to the first point cloud data, and the three-dimensional model corresponds to the second point cloud data, the second point cloud data is directly adjusted to make the adjusted second point cloud data have the highest contact ratio with the first point cloud data, and then the three-dimensional model is adjusted according to the adjustment mode of the second point cloud data to improve the contact ratio between the second point cloud data corresponding to the three-dimensional model and the first point cloud data, so as to improve the contact ratio between the three-dimensional model and the light field data.

It should be noted that, because the second point cloud data corresponding to the light field data can only roughly describe the geometric information of the three-dimensional scene, and the error of the second point cloud data is large, the process of performing registration by using the point cloud data in the

step

302 and 304 is only a process of performing preliminary processing in the three-dimensional model processing method provided in the embodiment of the present disclosure, and in order to improve the accuracy of performing registration on the three-dimensional model and the light field data, the following

step

305 and 307 need to be performed. Alternatively, in another embodiment, the electronic device may not perform the

step

302 and 304, and directly perform the

following step

305 and 307.

It should be noted that, in another embodiment, when performing the preliminary processing, a method of manually controlling an electronic device to perform registration may be further adopted instead of the method of performing registration by using point cloud data in the above-mentioned

step

302 and 304. Wherein, the three-dimensional model is displayed in an editing interface, and the method for manually controlling the electronic equipment to carry out registration comprises at least one of the following steps:

(1) and the electronic equipment controls the three-dimensional model to move in response to the dragging operation of the three-dimensional model after the movement option is triggered in the editing interface.

The electronic equipment displays the three-dimensional model and the light field data in an editing interface, a user views the displayed three-dimensional model and the light field data, if the three-dimensional model is required to be overlapped with the light field data and to be moved, the user triggers a moving option in the editing interface and then executes a dragging operation on the three-dimensional model, and therefore the electronic equipment controls the three-dimensional model to move in response to the dragging operation on the three-dimensional model after the moving option is triggered in the editing interface.

(2) And the electronic equipment controls the three-dimensional model to rotate in response to the dragging operation of the three-dimensional model after the rotation option is triggered in the editing interface.

And if the three-dimensional model is required to be rotated to enable the three-dimensional model to be overlapped with the light field data, triggering a rotation option in the editing interface by a user, and then executing a dragging operation on the three-dimensional model, so that the electronic equipment controls the three-dimensional model to rotate in response to the dragging operation on the three-dimensional model after triggering the rotation option in the editing interface.

(3) The electronic equipment responds to the adjustment information input in the editing interface, and adjusts the three-dimensional model according to the adjustment information.

In addition to performing a drag operation on the three-dimensional model in the editing interface to move or rotate the three-dimensional model, the user may also input a third registration parameter in the editing interface, so that the electronic device adjusts the position of the three-dimensional model according to the third registration parameter in response to the third registration parameter input in the editing interface. The third registration parameter has the same principle as the second registration parameter, and is not described herein again.

In the embodiment of the disclosure, the three-dimensional model is adjusted by adopting a manual intervention mode, so that the contact ratio of the three-dimensional model and the light field data is improved, and the flexibility of registering the three-dimensional model and the light field data is improved. Moreover, the accuracy of registration of the three-dimensional model and the light field data can be improved by means of manual intervention.

In another embodiment, in consideration of the fact that the automatic registration by using the point cloud data can save labor and time, and the registration by using the manual intervention mode can improve the registration accuracy, the two modes are combined, for example, the automatic registration by using the point cloud data is performed first, and if the user considers that the deviation of the result after the registration is large, the registration result is corrected by using the manual intervention mode.

After the electronic device obtains the preliminarily registered light field data and the three-dimensional model, the iterative process in the following step 305 and step 306 is performed on the light field data and the three-dimensional model to continue to register the three-dimensional model and the light field data.

305. Determining a two-dimensional model view that maps the three-dimensional model to any one view angle, and determining a two-dimensional light field view that maps the light field data to any one view angle.

Since the result of the preliminary registration performed by the steps 302-304 has an error, the three-dimensional model needs to be adjusted continuously on the basis of the registration result, so that the three-dimensional model and the light field data coincide. However, since the light field data does not include geometric information, the three-dimensional model and the light field data cannot be directly registered. The electronic device thus determines, respectively, a two-dimensional model view mapping the three-dimensional model to any viewing angle and a two-dimensional light field view mapping the light field data to any viewing angle, thereby converting the three-dimensional model to a two-dimensional model view and converting the three-dimensional light field data to a two-dimensional light field view. For example, the any one perspective is a randomly determined perspective by the electronic device.

The mapping of the three-dimensional model to the two-dimensional model view at any view angle refers to a view obtained by rendering the three-dimensional model according to the view angle, and can be understood as an image obtained by observing a three-dimensional scene represented by the three-dimensional model at the view angle, and the two-dimensional model view includes geometric information in the three-dimensional scene at the view angle. Mapping the light field data to a two-dimensional light field view at any viewing angle refers to an image obtained by observing a three-dimensional scene represented by the light field data at the viewing angle, and the two-dimensional light field view includes light ray information in the three-dimensional scene at the viewing angle.

In some embodiments, an original two-dimensional model view for mapping the three-dimensional model to any view angle is determined, light field data is determined to be mapped to the original two-dimensional light field view at any view angle, background information in the original two-dimensional model view is removed to obtain the two-dimensional model view at any view angle, and background information in the original two-dimensional light field view is removed to obtain the two-dimensional light field view at any view angle.

The original two-dimensional model view comprises foreground information and background information, the foreground information in the original two-dimensional model view refers to geometric information corresponding to a foreground in the original two-dimensional model view, the foreground refers to an object relatively close to a lens, and the background information in the original two-dimensional model view refers to geometric information corresponding to an object behind the foreground in the original two-dimensional model view. The original two-dimensional light field view comprises foreground information and background information, the foreground information in the original two-dimensional light field view refers to light ray information corresponding to a foreground in the original two-dimensional light field view, and the background information in the original two-dimensional light field view refers to light field information corresponding to an object behind the foreground in the original two-dimensional light field view.

The electronic device removes background information in the original two-dimensional model view to obtain a two-dimensional model view only including foreground information, for example, the electronic device directly sets the background of the original two-dimensional model view to be transparent to obtain the two-dimensional model view. The electronic device removes background information in the original two-dimensional light field view to obtain a two-dimensional light field view only including foreground information, for example, the electronic device processes the two-dimensional light field view through a background removal algorithm or a foreground-background segmentation algorithm to obtain the two-dimensional light field view.

In the embodiment of the disclosure, in consideration of the fact that the amount of information of the original two-dimensional light field view and the original two-dimensional model view is large, the subsequent processing complexity is also high, and therefore the two-dimensional light field view and the two-dimensional model view only including the foreground information are obtained, and therefore, the amount of data to be processed can be reduced when the two-dimensional light field view and the two-dimensional model view are processed subsequently, and the processing complexity is reduced.

306. And adjusting the position of the three-dimensional model according to the first difference parameter between the two-dimensional model view and the two-dimensional light field view at any visual angle, so that the coincidence degree of the two-dimensional model view and the two-dimensional light field view of the three-dimensional model adjusted at any visual angle is the highest.

The first difference parameter is used for representing the difference degree between the two-dimensional model view and the two-dimensional light field view, the larger the first difference parameter is, the larger the difference degree between the two-dimensional model view and the two-dimensional light field view is, and the smaller the first difference parameter is, the smaller the difference degree between the two-dimensional model view and the two-dimensional light field view is. For example, the electronic device determines a square value of a difference between pixel values corresponding to the same pixel point in the two-dimensional model view and the two-dimensional light field view, and uses a sum of the determined plurality of square values as a first difference parameter between the two-dimensional model view and the two-dimensional light field view.

In the embodiment of the disclosure, three-dimensional light field data is converted into a two-dimensional light field view, a three-dimensional model is converted into a two-dimensional model view, and the two-dimensional light field view and the two-dimensional model view are overlapped by adjusting the position of the three-dimensional model, so that the overlap ratio of the three-dimensional model and the light field data is improved, and the three-dimensional model and the light field data are registered. The method for indirectly registering based on the two-dimensional light field view and the two-dimensional model view avoids the problem that light field data does not have geometric information.

In some embodiments, a first registration parameter of the three-dimensional model is determined based on the first difference parameter, and the position of the three-dimensional model is adjusted according to the first registration parameter. The first registration parameter can enable the coincidence degree of the two-dimensional model view and the two-dimensional light field view of the three-dimensional model after adjustment under any view angle to be the highest, and the first registration parameter comprises a translation parameter and a rotation parameter.

The electronic equipment adjusts the two-dimensional model view according to the first difference parameter so that the coincidence degree of the adjusted two-dimensional model view and the two-dimensional light field view is the highest, and determines a first registration parameter according to the two-dimensional model view before adjustment and the two-dimensional model view after adjustment; and adjusting the position of the three-dimensional model according to the first registration parameter. Illustratively, the electronic device adjusts the two-dimensional model view according to the first difference parameter by using an optimization algorithm such as a gradient descent method or a simulated annealing method, so that the coincidence degree of the adjusted two-dimensional model view and the two-dimensional light field view is highest. Or, because the two-dimensional model view and the two-dimensional light field view are formed by discrete pixel points, the electronic device adjusts the two-dimensional model view by using an enumeration method according to the first difference parameter, so that the coincidence degree of the adjusted two-dimensional model view and the two-dimensional light field view is the highest.

The coincidence degree of the adjusted two-dimensional model view and the two-dimensional light field view is the highest, but the two-dimensional model view of the current three-dimensional model at the view angle is the two-dimensional model view before adjustment, and if the coincidence degree of the two-dimensional model view of the three-dimensional model at the view angle and the two-dimensional light field view is to be the highest, the position of the three-dimensional model is adjusted only according to the adjustment mode of the two-dimensional model view. Therefore, the electronic device determines a first registration parameter according to the two-dimensional model view before adjustment and the two-dimensional model view after adjustment, wherein the first registration parameter is information to be adjusted by converting the two-dimensional model view before adjustment into the two-dimensional model view after adjustment. The electronic device adjusts the position of the three-dimensional model according to the first registration parameter, so that the coincidence degree of the two-dimensional model view and the two-dimensional light field view of the three-dimensional model under the view angle is the highest.

In the embodiment of the disclosure, since the three-dimensional model corresponds to the two-dimensional model view, the two-dimensional model view is directly adjusted, so that the coincidence degree of the adjusted two-dimensional model view and the two-dimensional light field view is the highest, and then the three-dimensional model is adjusted according to the adjustment mode of the two-dimensional model view, so that the coincidence degree of the two-dimensional model view corresponding to the adjusted three-dimensional model and the two-dimensional light field view is the highest at the view angle, that is, the problem of adjusting the three-dimensional model is converted into the problem of adjusting the two-dimensional view, thereby reducing the complexity of the registration process.

Illustratively, the electronic device performs at least one of the following in accordance with the first registration parameter:

(1) and the electronic equipment controls the three-dimensional model to translate according to the translation parameters in the first registration parameters.

If the two-dimensional model view needs to be moved to coincide with the two-dimensional light field view, the first registration parameters comprise translation parameters, and the translation parameters are used for representing the translation mode. Therefore, the electronic equipment can control the three-dimensional model to translate according to the translation parameters. In the embodiment of the disclosure, because the translation parameter indicates the translation mode of the two-dimensional model view, the electronic device controls the three-dimensional model to translate according to the translation parameter, so that the three-dimensional model translates according to the movement mode of the two-dimensional model view.

For example, the translation parameters include a first direction and a distance, the first direction refers to a direction of translation, and the distance refers to a distance of translation, and since the translation parameters are translation parameters obtained by translating the two-dimensional model view, the two-dimensional model view can only translate in a plane in which the two-dimensional model view is located, and therefore the first direction is parallel to the plane in which the two-dimensional model view is located. The electronics control the three-dimensional model to translate along the first direction. In the embodiment of the present disclosure, because the first direction is a plane parallel to the two-dimensional model view, the translation parameter only includes the translation amount in the x-axis direction and the translation amount in the y-axis direction in the two-dimensional space, but not the translation amount in the x-axis direction, the translation amount in the y-axis direction, and the translation amount in the z-axis direction in the three-dimensional space, and the dimension reduction of the translation parameter is realized.

(2) And the electronic equipment controls the three-dimensional model to rotate according to the rotation parameters in the first registration parameters.

If the two-dimensional model view needs to be rotated to coincide the two-dimensional model view with the two-dimensional light field view, a rotation parameter is included in the first registration parameter, and the rotation parameter is used for representing the rotation mode. Therefore, the electronic equipment can control the three-dimensional model to rotate according to the rotation parameters. In the embodiment of the present disclosure, since the rotation parameter indicates a manner of rotating the two-dimensional model view, the electronic device controls the three-dimensional model to rotate according to the rotation parameter, so that the three-dimensional model rotates according to the manner of rotating the two-dimensional model view.

For example, the rotation parameters include a second direction and an angle, the second direction refers to a direction corresponding to the rotation axis, the angle refers to an angle of rotation, and since the rotation parameters are rotation parameters obtained by rotating the two-dimensional model view, the two-dimensional model view can only rotate around the rotation axis indicated by a direction perpendicular to a plane where the two-dimensional model view is located, and thus the second direction is perpendicular to the plane where the two-dimensional model view is located. The electronic device controls the rotation angle of the three-dimensional model around the rotation axis indicated by the second direction. In the embodiment of the present disclosure, since the two-dimensional model view is rotated in the two-dimensional space, the rotation parameter only includes an angle corresponding to the second direction perpendicular to the plane where the two-dimensional model view is located, and not an angle around the x-axis direction, an angle around the y-axis direction, and an angle around the z-axis direction in the three-dimensional space, so that the dimension reduction of the rotation parameter is realized.

307. And stopping the iterative process in response to the iterative process meeting the iterative end condition to obtain the registered three-dimensional model.

After the light field data and the three-dimensional model are registered, the two-dimensional light field view and the two-dimensional model view at any view angle should be overlapped, so that if the overlap ratio of the two-dimensional light field view and the two-dimensional model view at any view angle is the highest, the overlap ratio of the light field data and the three-dimensional model corresponding to the three-dimensional scene is considered to be the highest. Therefore, if the light field data and the three-dimensional model are to be registered, only the coincidence degree of the two-dimensional light field view and the two-dimensional model view under the visual angles is required to be ensured to be the highest, so that the three-dimensional registration process is reduced into the two-dimensional registration process.

Therefore, the electronic device performs the iterative process in step 305 and step 306 at least once, until the iterative process meets the iteration ending condition, the iterative process is stopped, the three-dimensional model after registration is obtained, and then registration of the three-dimensional model and the light field data is completed.

In the embodiment of the disclosure, it is considered that the light field data itself does not have geometric information in a three-dimensional scene, and the three-dimensional model and the light field data cannot be directly registered, so that the three-dimensional light field data is converted into a two-dimensional light field view, the three-dimensional model is converted into a two-dimensional model view, and the coincidence degree of the two-dimensional light field view and the two-dimensional model view is highest by adjusting the three-dimensional model, so that the coincidence degree of the three-dimensional model and the light field data is improved, that is, the registration problem in a three-dimensional space is converted into the registration problem in a two-dimensional space, so that the registration process of the three-dimensional model and the light field data is simpler, and the registration is performed in an iterative manner, so that the registration accuracy of the three-dimensional model and the light field data can be gradually improved in the iterative process.

In some embodiments, the iteration end condition is: there are a consecutive target number of first difference parameters each being less than the target threshold parameter.

After the step 306 is executed, in response to that there is no consecutive number of first difference parameters smaller than the target threshold parameter, the electronic device continues to adjust the position of the three-dimensional model according to the first difference parameters between the two-dimensional model view and the two-dimensional light field view at other viewing angles, so that the coincidence degree between the two-dimensional model view of the three-dimensional model adjusted at other viewing angles and the two-dimensional light field view of the light field data is the highest. After the electronic device adjusts the three-dimensional model according to the first difference parameter between the two-dimensional model view and the two-dimensional light field view at any view angle, the electronic device determines the first difference parameter between the two-dimensional model view and the two-dimensional light field view at other view angles again. The first difference parameter between the two-dimensional model view and the two-dimensional light field view is smaller than the target threshold parameter, and the overlap ratio of the two-dimensional model view and the two-dimensional light field view can be approximately considered to reach the maximum value. If the number of the first difference parameters of the continuous targets is not smaller than the target threshold parameter, the coincidence degree of the two-dimensional model view and the two-dimensional light field view under the condition of insufficient visual angles is considered to reach the maximum value, so that the electronic equipment needs to continuously adjust the three-dimensional model to enable the coincidence degree of the two-dimensional model view and the two-dimensional light field view under more visual angles to reach the maximum value, and the registration precision of the three-dimensional model and the light field data is ensured. Illustratively, the target number and the target threshold parameter are preset by the electronic device, for example, the target number is 3.

After the step 306 is executed, in response to that the first difference parameters of the continuous target number are all smaller than the target threshold parameter, the electronic device considers that the iterative process meets the iteration end condition, and stops the iterative process to obtain the registered three-dimensional model. The first difference parameters with the number of continuous targets are smaller than the target threshold parameter, which indicates that the coincidence degree between the two-dimensional model view and the two-dimensional light field view under the randomly determined number of viewing angles reaches the maximum value, and if the coincidence degree between the two-dimensional model view and the two-dimensional light field view under enough viewing angles reaches the maximum value, the electronic equipment does not need to adjust the three-dimensional model. In the embodiment of the disclosure, because the light field data does not have geometric information, it is impossible to directly determine whether the three-dimensional model coincides with the light field data according to the light field data itself, and therefore the problem of determining whether the three-dimensional model coincides with the light field data in the three-dimensional scene is converted into the problem of determining whether the three-dimensional model coincides with the light field data in the two-dimensional scene, so that the coincidence degree of the three-dimensional model and the light field data is determined to be the highest under the condition that the coincidence degree of the two-dimensional model view and the two-dimensional light field view.

Fig. 4 is a flowchart illustrating a three-dimensional model processing method according to an exemplary embodiment, and as shown in fig. 4, taking the number of targets as 3 as an example, an electronic device acquires light field data and a three-dimensional model, determines first point cloud data corresponding to the light field data and second point cloud data corresponding to the three-dimensional model, and performs preliminary registration on the three-dimensional model and the light field data according to the first point cloud data and the second point cloud data. And then the electronic equipment randomly determines a visual angle, acquires an original two-dimensional light field view and an original two-dimensional model view under the visual angle, and respectively performs foreground extraction on the original two-dimensional light field view and the original two-dimensional model view to obtain the two-dimensional light field view and the two-dimensional model view which only comprise foreground information. The electronic device registers the two-dimensional model view and the two-dimensional light field view according to the first difference parameter between the two-dimensional model view and the two-dimensional light field view, and the registration between the two-dimensional model view and the two-dimensional light field view is reflected on the three-dimensional model and the light field data. The electronic equipment judges whether 3 continuous first difference parameters are smaller than a target threshold parameter, if so, the electronic equipment determines that the registration between the three-dimensional model and the light field data is completed, if not, the electronic equipment randomly changes one view angle and continues to register the three-dimensional model and the light field data until 3 continuous first difference parameters are smaller than the target threshold parameter.

Fig. 5 is a block diagram illustrating a three-dimensional model processing apparatus according to an exemplary embodiment. Referring to fig. 5, the apparatus includes an acquisition unit 501 and a registration unit 502.

An acquisition unit 501 configured to perform acquisition of light field data and a three-dimensional model corresponding to a three-dimensional scene;

a registration unit 502 configured to perform the following iterative process on the light field data and the three-dimensional model:

determining a two-dimensional model view for mapping the three-dimensional model to any viewing angle, and determining a two-dimensional light field view for mapping the light field data to any viewing angle;

adjusting the position of the three-dimensional model according to a first difference parameter between the two-dimensional model view and the two-dimensional light field view at any visual angle, so that the coincidence degree of the two-dimensional model view and the two-dimensional light field view of the three-dimensional model adjusted at any visual angle is highest;

and stopping the iterative process in response to the iterative process meeting the iterative end condition to obtain the registered three-dimensional model.

The device provided by the embodiment of the disclosure, considering that the light field data itself does not have geometric information in a three-dimensional scene, and the three-dimensional model and the light field data cannot be directly registered, converts the three-dimensional light field data into a two-dimensional light field view, converts the three-dimensional model into a two-dimensional model view, and adjusts the position of the three-dimensional model to make the coincidence degree of the two-dimensional light field view and the two-dimensional model view highest, so as to improve the coincidence degree of the three-dimensional model and the light field data, that is, convert the registration problem in a three-dimensional space into the registration problem in a two-dimensional space, so that the registration process of the three-dimensional model and the light field data is simpler, and the registration precision of the three-dimensional model and the light field data can be gradually improved in the iterative process.

In some embodiments, referring to fig. 6, the registration unit 501, comprises:

a parameter determining subunit 511, configured to perform determining, according to the first difference parameter, a first registration parameter of the three-dimensional model, where the first registration parameter is capable of maximizing a coincidence degree between a two-dimensional model view and a two-dimensional light field view of the adjusted three-dimensional model at any viewing angle, and the first registration parameter includes a translation parameter and a rotation parameter;

a position adjustment subunit 521 configured to perform, in accordance with the first registration parameter, at least one of:

Optionally, the registration unit 501 comprises:

a view determination subunit 531 configured to perform determining an original two-dimensional model view mapping the three-dimensional model to any one viewing angle, and determining an original two-dimensional light field view mapping the light field data to any one viewing angle;

a background removing subunit 541 configured to remove background information in the original two-dimensional model view to obtain a two-dimensional model view at any viewing angle;

the background removing subunit 541 is further configured to perform removing the background information in the original two-dimensional light field view, so as to obtain a two-dimensional light field view at any viewing angle.

Optionally, the three-dimensional model processing apparatus further includes:

a three-dimensional reconstruction unit 503 configured to perform three-dimensional reconstruction on the light field data to obtain first point cloud data corresponding to the light field data, where the first point cloud data is used to describe geometric information in a three-dimensional scene;

a vertex extraction unit 504 configured to perform extraction of vertices in the three-dimensional model, and configure the extracted vertices into second point cloud data corresponding to the three-dimensional model;

a registration parameter determining unit 505 configured to determine a second registration parameter of the three-dimensional model according to a second difference parameter between the first point cloud data and the second point cloud data, wherein the second registration parameter can enable the contact ratio of the second point cloud data corresponding to the adjusted three-dimensional model and the first point cloud data to be highest;

a position adjusting unit 506 configured to perform adjusting the position of the three-dimensional model according to the second registration parameter.

With regard to the apparatus in the above-described embodiment, the specific manner in which each unit performs the operation has been described in detail in the embodiment related to the method, and will not be described in detail here.

In an exemplary embodiment, there is provided an electronic device including: a processor, and a memory for storing instructions executable by the processor. Wherein the processor is configured to execute the instructions to implement the three-dimensional model processing method as described above.

In some embodiments, the electronic device is a terminal. Fig. 7 is a block diagram illustrating a structure of a terminal 700 according to an example embodiment. The terminal 700 may be a portable mobile terminal such as: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, motion video Experts compression standard Audio Layer 3), an MP4 player (Moving Picture Experts Group Audio Layer IV, motion video Experts compression standard Audio Layer 4), a notebook computer, or a desktop computer. Terminal 700 may also be referred to by other names such as user equipment, portable terminal, laptop terminal, desktop terminal, and so on.

The terminal 700 includes: a processor 701 and a memory 702.

The processor 701 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so on. The processor 701 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 701 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 701 may be integrated with a GPU (Graphics Processing Unit) which is responsible for rendering and drawing the content required to be displayed by the display screen. In some embodiments, the processor 701 may further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.

Memory 702 may include one or more computer-readable storage media, which may be non-transitory. Memory 702 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in the memory 702 is used to store at least one program code for execution by the processor 701 to implement the three-dimensional model processing method provided by the method embodiments of the present disclosure.

In some embodiments, the terminal 700 may further optionally include: a peripheral interface 703 and at least one peripheral. The processor 701, the memory 702, and the peripheral interface 703 may be connected by buses or signal lines. Various peripheral devices may be connected to peripheral interface 703 via a bus, signal line, or circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 704, a display screen 705, a camera assembly 706, an audio circuit 707, a positioning component 708, and a power source 709.

The peripheral interface 703 may be used to connect at least one peripheral related to I/O (Input/Output) to the processor 701 and the memory 702. In some embodiments, processor 701, memory 702, and peripheral interface 703 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 701, the memory 702, and the peripheral interface 703 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

The Radio Frequency circuit 704 is used for receiving and transmitting RF (Radio Frequency) signals, also called electromagnetic signals. The radio frequency circuitry 704 communicates with communication networks and other communication devices via electromagnetic signals. The rf circuit 704 converts an electrical signal into an electromagnetic signal to transmit, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 704 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and so forth. The radio frequency circuitry 704 may communicate with other terminals via at least one wireless communication protocol. The wireless communication protocols include, but are not limited to: the world wide web, metropolitan area networks, intranets, generations of mobile communication networks (2G, 3G, 4G, and 5G), Wireless local area networks, and/or WiFi (Wireless Fidelity) networks. In some embodiments, the radio frequency circuit 704 may also include NFC (Near Field Communication) related circuits, which are not limited by this disclosure.

The display screen 705 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display screen 705 is a touch display screen, the display screen 705 also has the ability to capture touch signals on or over the surface of the display screen 705. The touch signal may be input to the processor 701 as a control signal for processing. At this point, the display 705 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, the display 705 may be one, disposed on a front panel of the terminal 700; in other embodiments, the display 705 can be at least two, respectively disposed on different surfaces of the terminal 700 or in a folded design; in other embodiments, the display 705 may be a flexible display disposed on a curved surface or on a folded surface of the terminal 700. Even more, the display 705 may be arranged in a non-rectangular irregular pattern, i.e. a shaped screen. The Display 705 may be made of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), or the like.

The camera assembly 706 is used to capture images or video. Optionally, camera assembly 706 includes a front camera and a rear camera. The front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, the number of the rear cameras is at least two, and each rear camera is any one of a main camera, a depth-of-field camera, a wide-angle camera and a telephoto camera, so that the main camera and the depth-of-field camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize panoramic shooting and VR (Virtual Reality) shooting functions or other fusion shooting functions. In some embodiments, camera assembly 706 may also include a flash. The flash lamp can be a monochrome temperature flash lamp or a bicolor temperature flash lamp. The double-color-temperature flash lamp is a combination of a warm-light flash lamp and a cold-light flash lamp, and can be used for light compensation at different color temperatures.

The audio circuitry 707 may include a microphone and a speaker. The microphone is used for collecting sound waves of a user and the environment, converting the sound waves into electric signals, and inputting the electric signals to the processor 701 for processing or inputting the electric signals to the radio frequency circuit 704 to realize voice communication. For the purpose of stereo sound collection or noise reduction, a plurality of microphones may be provided at different portions of the terminal 700. The microphone may also be an array microphone or an omni-directional pick-up microphone. The speaker is used to convert electrical signals from the processor 701 or the radio frequency circuit 704 into sound waves. The loudspeaker can be a traditional film loudspeaker or a piezoelectric ceramic loudspeaker. When the speaker is a piezoelectric ceramic speaker, the speaker can be used for purposes such as converting an electric signal into a sound wave audible to a human being, or converting an electric signal into a sound wave inaudible to a human being to measure a distance. In some embodiments, the audio circuitry 707 may also include a headphone jack.

The positioning component 708 is used to locate the current geographic Location of the terminal 700 for navigation or LBS (Location Based Service). The Positioning component 708 can be a Positioning component based on the united states GPS (Global Positioning System), the chinese beidou System, the russian glonass Positioning System, or the european union galileo Positioning System.

Power supply 709 is provided to supply power to various components of terminal 700. The power source 709 may be alternating current, direct current, disposable batteries, or rechargeable batteries. When the power source 709 includes a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery may also be used to support fast charge technology.

Those skilled in the art will appreciate that the configuration shown in fig. 7 is not intended to be limiting of terminal 700 and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components may be used.

In some embodiments, the electronic device is a server. Fig. 8 is a schematic structural diagram of a server according to an exemplary embodiment, where the server 800 may generate a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 801 and one or more memories 802, where the memory 802 stores at least one computer program, and the at least one computer program is loaded and executed by the processors 801 to implement the methods provided by the method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input/output interface, so as to perform input/output, and the server may also include other components for implementing the functions of the device, which are not described herein again.

In an exemplary embodiment, there is also provided a computer-readable storage medium, in which instructions, when executed by a processor of an electronic device, enable the electronic device to perform the steps of the above-described three-dimensional model processing method. For example, the computer-readable storage medium may be a ROM (Read Only Memory), a RAM (Random Access Memory), a CD-ROM (Compact Disc Read-Only Memory), a magnetic tape, a floppy disk, an optical data storage device, and the like.

In an exemplary embodiment, a computer program product is also provided, which comprises a computer program, which when executed by a processor of an electronic device, implements the steps in the above-described three-dimensional model processing method.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure herein. This disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A three-dimensional model processing method, characterized by comprising:

2. The three-dimensional model processing method according to claim 1, wherein the iteration end condition is: there are a consecutive target number of first difference parameters each being less than the target threshold parameter.

3. The three-dimensional model processing method according to claim 1, wherein the adjusting the position of the three-dimensional model according to the first difference parameter between the two-dimensional model view and the two-dimensional light field view at any one viewing angle to maximize the coincidence ratio between the two-dimensional model view and the two-dimensional light field view of the three-dimensional model after being adjusted at any one viewing angle comprises:

4. The three-dimensional model processing method of claim 1, wherein said determining maps the three-dimensional model to a two-dimensional model view at any viewing angle and determining maps the light field data to a two-dimensional light field view at the any viewing angle comprises:

5. The three-dimensional model processing method according to claim 1, wherein before performing the following iterative process on the light field data and the three-dimensional model, the three-dimensional model processing method further comprises:

6. A three-dimensional model processing apparatus, characterized by comprising:

7. The three-dimensional model processing apparatus according to claim 6, wherein the iteration end condition is: there are a consecutive target number of first difference parameters each being less than the target threshold parameter.

8. An electronic device, characterized in that the electronic device comprises:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the three-dimensional model processing method of any one of claims 1 to 5.

9. A computer-readable storage medium in which instructions, when executed by a processor of an electronic device, enable the electronic device to perform the three-dimensional model processing method of any one of claims 1 to 5.

10. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, implements the three-dimensional model processing method of any one of claims 1 to 5.