CN115953503B

CN115953503B - Hole texture filling method, device, apparatus and storage medium

Info

Publication number: CN115953503B
Application number: CN202211326542.9A
Authority: CN
Inventors: 请求不公布姓名
Original assignee: Beijing Chengshi Wanglin Information Technology Co Ltd
Current assignee: Beijing Chengshi Wanglin Information Technology Co Ltd
Priority date: 2022-10-27
Filing date: 2022-10-27
Publication date: 2023-06-27
Anticipated expiration: 2042-10-27
Also published as: CN115953503A

Abstract

The embodiment of the application provides a cavity texture filling method, equipment, device and storage medium. In the embodiment of the application, according to the connection relation of the edge vertex sequences in the edge vertex sequences contained in the cavity communication blocks in the three-dimensional grid model, determining the annular sequences formed by other edge vertices connected by the edge vertices in the edge vertex sequences; and then traversing the edge vertex sequence according to the sequence of edge vertices in the annular sequence and the connection relation of the edge vertex sequence in the three-dimensional grid model, traversing all edge vertices contained in the edge vertex sequence, relaxing the constraint on the arrangement form of the cavity boundary, supporting the hole filling of multiple sections of edges and embedded texture blocks in the cavity communicating block, enabling the hole texture filling mode to be suitable for various complex use scenes, and improving the universality and texture filling effect of the hole texture filling.

Description

Hole texture filling method, device, apparatus and storage medium

Technical Field

The present application relates to the field of internet technologies, and in particular, to a method, an apparatus, a device, and a storage medium for filling a hole texture.

Background

In the existing three-dimensional space model of the target object, some positions are blocked by other objects in the shooting process, so that data cannot be acquired, and a live-action image cannot be shot. In the process of constructing a three-dimensional space model of an object, a void can be generated on the surface of the model of the object in a region without data, a black void can be displayed from the view of model display, and the display effect of the model is seriously affected.

For example, VR looks at room systems, because some locations of the house are blocked by other objects during the shooting process, live-action images of those locations cannot be acquired. In constructing a three-dimensional house model, these locations are texture filled with no texture data, resulting in areas without texture data that create voids in the surface of the house model. Therefore, how to fill the cavity on the surface of the model is a problem to be solved.

Disclosure of Invention

Aspects of the present application provide a method, apparatus, device, and storage medium for filling a texture into a cavity on a surface of a three-dimensional model of a target object, which are helpful for improving a display effect of the three-dimensional model.

The embodiment of the application provides a cavity texture filling method, which comprises the following steps:

According to the mapping relation between the three-dimensional grid model of the target object and the live-action image, determining a cavity communicating block formed by a first triangular surface patch which is not filled by the texture of the live-action image from the three-dimensional grid model;

determining an internal vertex set and a communicated edge vertex sequence contained in the cavity communicating block according to the connection relation of the cavity communicating block in the three-dimensional grid model; the edge vertex sequence refers to a sequence formed by vertices of edges shared with a second triangular patch in the first triangular patch, and the internal vertex set refers to a set of vertices of edges not shared with the second triangular patch in the first triangular patch; the second triangular patch refers to a triangular patch filled by textures of the live-action image in the three-dimensional network model;

determining a ring-shaped sequence formed by other edge vertexes connected with any edge vertex according to the connection relation of the edge vertex sequence in the three-dimensional grid model aiming at any edge vertex in the edge vertex sequence;

traversing the edge vertex sequence according to the connection relation of the edge vertex sequence in the three-dimensional grid model and the sequence of edge vertices in the annular sequence to obtain a target edge vertex sequence;

According to the distribution of the target edge vertex sequence in the three-dimensional grid model, determining texture coordinates of the target edge vertex sequence;

calculating texture coordinates of the internal vertex set according to texture coordinates of the target edge vertex sequence and the connection relation between the target edge vertex sequence and the internal vertex set in the three-dimensional grid model;

and filling the texture of the cavity communicating block according to the texture of the edge where the target edge vertex sequence is located and the texture coordinates of the internal vertex set.

The embodiment of the application also provides a cavity texture filling device, which comprises:

the determining module is used for determining a cavity communicating block formed by a first triangular patch which is not filled by the texture of the live-action image from the three-dimensional grid model according to the mapping relation between the three-dimensional grid model of the target object and the live-action image; determining an internal vertex set and a communicated edge vertex sequence contained in the communicated block according to the connection relation of the hollow communicated block in the three-dimensional grid model; the edge vertex sequence refers to a sequence formed by vertices of an edge shared with a second triangular patch in the first triangular patch; the internal vertex set refers to a set of vertices of edges of the first triangular patch that are not shared with the second triangular patch; the second triangular patch refers to a triangular patch filled by textures of the live-action image in the three-dimensional network model; and determining a ring-shaped sequence formed by other edge vertexes connected with any edge vertex according to the connection relation of the edge vertex sequence in the three-dimensional grid model aiming at any edge vertex in the edge vertex sequence;

The traversing module is used for traversing the edge vertex sequence according to the connection relation of the edge vertex sequence in the three-dimensional grid model and the sequence of edge vertices in the annular sequence so as to obtain a target edge vertex sequence;

the determining module is further configured to determine texture coordinates of the target edge vertex sequence according to distribution of the target edge vertex sequence in the three-dimensional mesh model;

the calculation module is used for calculating the texture coordinates of the internal vertex set according to the texture coordinates of the target edge vertex sequence and the connection relation between the target edge vertex sequence and the internal vertex set in the three-dimensional grid model;

and the filling module is used for carrying out texture filling on the cavity communicating block according to the texture of the edge where the target edge vertex sequence is located and the texture coordinates of the internal vertex set.

Embodiments of the present application also provide a computing device, comprising: a memory and a processor; wherein the memory is used for storing a computer program;

the processor is coupled to the memory for executing the computer program for performing the steps in the above-described hole texture filling method.

Embodiments also provide a computer-readable storage medium storing computer instructions that, when executed by one or more processors, cause the one or more processors to perform the steps in the above-described hole texture filling method.

In the embodiment of the application, according to the connection relation of the edge vertex sequences in the edge vertex sequences contained in the cavity communication blocks in the three-dimensional grid model, determining the annular sequences formed by other edge vertices connected by the edge vertices in the edge vertex sequences; and then traversing the edge vertex sequence according to the sequence of edge vertices in the annular sequence and the connection relation of the edge vertex sequence in the three-dimensional grid model, traversing all edge vertices contained in the edge vertex sequence, relaxing the constraint on the arrangement form of the cavity boundary, supporting the hole filling of multiple sections of edges and embedded texture blocks in the cavity communicating block, enabling the hole texture filling mode to be suitable for various complex use scenes, and improving the universality and texture filling effect of the hole texture filling.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute an undue limitation to the application. In the drawings:

FIGS. 1 a-1 c are schematic diagrams of cavity communication blocks according to embodiments of the present application;

fig. 2 is a flow chart of a method for filling a hole texture according to an embodiment of the present application;

FIG. 3 is a schematic diagram of another cavity communication block according to an embodiment of the present disclosure;

fig. 4 is a schematic diagram of a further cavity communication block according to an embodiment of the present application;

fig. 5 is a schematic structural diagram of a void texture filling device according to an embodiment of the present disclosure;

fig. 6 is a schematic structural diagram of a computing device according to an embodiment of the present application.

Detailed Description

For the purposes, technical solutions and advantages of the present application, the technical solutions of the present application will be clearly and completely described below with reference to specific embodiments of the present application and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.

In the embodiment of the application, in order to improve the display effect of the target object, a three-dimensional space model of the target object can be constructed. The target object may be any physical object in nature. For example, the target object may be a building (e.g., a house, a mall, an office building, etc.), a vehicle (e.g., an automobile, etc.), a city, etc. Each texture surface of the three-dimensional space model includes a plurality of triangular patches (faces), with vertices of each triangular patch having corresponding texture coordinates. During rendering, the texture of each triangular patch can be rendered according to the texture coordinates of the triangular patches in the texture surface, and finally the three-dimensional space model of the whole target object is obtained. The three-dimensional space model is a three-dimensional real model of the target object, and is the restoration of the target object in the data world.

The three-dimensional space model of the target object is formed, the data of the target object can be collected by various sensors, and the target object is reconstructed based on the collected data, so that the three-dimensional space model is obtained. For example, in some conventional approaches, point cloud data of a target object may be acquired by a lidar; and constructing a three-dimensional grid model O of the target object based on the point cloud data. Optionally, the point cloud data of the target object may be processed by registration fusion and poisson reconstruction, so as to obtain a three-dimensional grid model O of the target object. The three-dimensional mesh model O includes: a three-dimensional vertex set and a triangular patch set S joining the vertices.

In order to perform texture filling on the three-dimensional grid model of the target object to obtain a three-dimensional space model, an image acquisition device (such as a camera, a camera and the like) can be used for acquiring a live-action image of the target object. The camera in the image acquisition device can be a panoramic camera and the like. Based on the live-action image of the target object, the texture of the live-action image can be mapped to a corresponding triangular patch in the three-dimensional grid model, so that a three-dimensional space model of the target object is obtained. Alternatively, the real image of the target object may be mapped to a triangular patch corresponding to the three-dimensional mesh model according to a mapping relationship between the three-dimensional mesh model and the real image of the target object. The mapping relation between the three-dimensional grid model of the target object and the live-action image can be determined by the coordinate conversion relation between the coordinate system where the point cloud data are located and the coordinate system where the live-action image is located.

However, in practical application, some positions are blocked by other objects in the data acquisition process, so that data cannot be acquired and a live-action image cannot be shot. Therefore, in the process of constructing the three-dimensional space model of the target object, the triangular patches corresponding to the areas without data have no texture of the corresponding live-action image, so that a cavity is generated on the surface of the three-dimensional space model of the target object, a black hole or a cavity is displayed from the view of model display, and the display effect of the model is seriously affected.

In order to fill the cavities of the surface of the three-dimensional spatial model of the target object, in some conventional schemes, for example, the monocular three-dimensional reconstruction scheme MVE performs texture filling on the cavities of the surface of the three-dimensional spatial model of the target object in the following main ways:

step 1, according to a three-dimensional grid model O of a target object and panoramic images acquired by image acquisition equipment at each point position, mapping textures of the panoramic images shot by the image acquisition equipment at all the point positions onto the three-dimensional grid model O to obtain a triangular patch set P which is mapped by the textures and a triangular patch set H which is not mapped by the textures in the three-dimensional grid model. The triangular surface patch which is not mapped by the texture is the texture hole.

And 2, determining a cavity communicating block set U formed by the triangular patch set H according to the connection relation of the triangular patch set H which is not subjected to texture mapping in the three-dimensional grid model O. Each cavity communicating block u is composed of communicating triangular patches which are not mapped by textures. The cavity communicating block set U formed by the triangular patch set H may be determined according to the connection relation of the triangular patch set H which is not texture mapped in the three-dimensional mesh model O by a Breadth First Search (BFS) or Union Find (Union Find) method, but is not limited thereto.

And 3, for each cavity communicating block u, obtaining vertex sets included in all triangular patches in the cavity communicating block u according to the connection relation of the cavity communicating blocks u in the three-dimensional grid model O, and dividing the vertex sets included in the cavity communicating block u into an internal vertex set I and an edge annular vertex sequence B according to whether the vertices are on the edges shared by the cavity communicating blocks and the triangular patches mapped with textures. The order of any sequence B in the edge ring vertex sequence B satisfies that the vertex at any position is adjacent to the vertex at the previous position or the next position on the three-dimensional grid model O.

And 3, the vertexes in the edge ring vertex sequence B in the step are edge vertexes, wherein the edge vertexes refer to vertexes on the edge shared by the cavity communicating block and the triangular surface patch mapped with the texture. The internal vertex set refers to a set of vertices on the side that the hole-communicating block does not share with the texture-mapped triangular patch.

And 4, generating a virtual texture block t to be filled for each cavity communicating block u. The dummy texture block t is empty without filling texture.

And 5, assigning a texture coordinate to each vertex of the edge ring vertex sequence B, and designating the texture coordinates to be distributed on a concentric circle. Wherein the size of the concentric circles and the texture coordinate distribution interval can be determined by the distribution of the edge ring vertex sequence B in the three-dimensional grid model O.

Step 6, for each cavity communicating block u, after obtaining texture coordinates of the edge ring vertex sequence B, the texture coordinates of the internal vertex set I on the virtual texture block t can be set as an unknown item set, and according to the connection relation between the edge ring vertex sequence B and the internal vertex set I in the three-dimensional grid model O, the weight of the texture coordinates of the edge ring vertex sequence B to the texture coordinates of the vertices in the internal vertex set I is determined; and (2) then, taking the texture coordinates of the edge annular vertex sequence B calculated in the previous step (5) as boundary conditions, and establishing a global Laplace equation to solve the texture coordinates of the internal vertex set I.

And 7, performing texture rendering on the cavity formed by the triangular patch set H which is not subjected to texture mapping according to the texture coordinates of the edge annular vertex sequence B, the texture coordinates of the internal vertex set I and the texture of the edge where the edge annular vertex sequence B is positioned, and thus completing hole filling operation.

Although the above scheme can complete the rendering of the texture of the hole, the feasible precondition of the scheme is that the texture hole property is required to satisfy a certain assumption: (1) The degree of the cavity edge vertexes is required to be 2, namely each vertex is only connected with the front vertex and the rear vertex which are adjacent to the vertex; (2) The cavity communicating block does not contain other texture blocks, namely all edge vertexes are connected to form only one end-to-end ring. However, when the condition is not satisfied, the above-described hole texture filling operation may be exited, and hole texture filling may not be achieved. For example, the texture hole communicating block u is shaped as an end-to-end ring or the like. In practical application, the conditions of complex object profile of a target object, large multi-point registration error and the like are often caused by insufficient acquisition points, object shielding and the like of the image acquisition equipment, so that a cavity communicating block in a three-dimensional grid model does not meet the precondition of the assumption, and the air conditioner texture filling fails.

For example, as shown in fig. 1 a-1 c, the white areas are void communication blocks of non-texture mapped triangular patches, and the gray areas are texture filled triangular patch areas. The above-mentioned cavity texture filling scheme is only suitable for the cavity communicating block shown in fig. 1a, i.e. the degree of the edge vertices of the cavity communicating block is 2, and the edge vertices are connected to form only one end-to-end ring. However, the above-described hole texture filling scheme cannot complete the texture filling of the hole communicating block shown in fig. 1b and 1c, and the degree of the edge vertex v0 in fig. 1b is 4, and the above-described assumption condition (1) is not satisfied. In fig. 1c, the hollow communicating block contains other textured communicating blocks, and the edge vertex sequence of the hollow communicating block is multi-segment, i.e. a plurality of end-to-end rings, which does not satisfy the above assumption condition (2). Therefore, the holes shown in fig. 1b and 1c cannot be texture filled using the hole texture filling scheme described above.

Aiming at the technical problem that the use condition of the cavity texture filling scheme is limited, so that the cavity texture filling effect is poor, some embodiments of the application provide a novel cavity texture filling scheme, and the main thought is as follows: determining an annular sequence formed by other edge vertexes connected with edge vertexes in the edge vertex sequence according to the connection relation of the edge vertex sequence in the edge vertex sequence contained in the cavity communication block in the three-dimensional grid model; and then traversing the edge vertex sequence according to the sequence of edge vertices in the annular sequence and the connection relation of the edge vertex sequence in the three-dimensional grid model, traversing all edge vertices contained in the edge vertex sequence, relaxing the constraint on the arrangement form of the cavity boundary, supporting the hole filling of multiple sections of edges and embedded texture blocks in the cavity communicating block, enabling the hole texture filling mode to be suitable for various complex use scenes, and improving the universality and texture filling effect of the hole texture filling.

The following describes in detail the technical solutions provided by the embodiments of the present application with reference to the accompanying drawings.

It should be noted that: like reference numerals denote like objects in the following figures and embodiments, and thus once an object is defined in one figure or embodiment, further discussion thereof is not necessary in the subsequent figures and embodiments.

Fig. 2 is a flow chart of a method for filling a hole texture according to an embodiment of the present application. As shown in fig. 2, the method mainly comprises the following steps:

201. and determining a cavity communicating block formed by the first triangular surface patch which is not filled by the texture of the live-action image from the three-dimensional grid model according to the mapping relation between the three-dimensional grid model of the target object and the live-action image.

202. Determining an internal vertex set contained in the communicating block and a communicated edge vertex sequence according to the connection relation of the cavity communicating block in the three-dimensional grid model; the edge vertex sequence refers to a sequence formed by the vertices of the edges shared with the second triangular patch in the first triangular patch, and the internal vertex set refers to a set of the vertices of the edges not shared with the second triangular patch in the first triangular patch; the second triangular patch refers to a triangular patch filled with the texture of the live-action image in the three-dimensional network model.

203. And determining a ring-shaped sequence formed by other edge vertexes connected with any edge vertex according to the connection relation of the edge vertex sequence in the three-dimensional grid model aiming at any edge vertex in the edge vertex sequence.

204. Traversing the edge vertex sequence according to the connection relation of the edge vertex sequence in the three-dimensional grid model and the sequence of the edge vertices in the annular sequence to obtain a target edge vertex sequence.

205. And determining texture coordinates of the target edge vertex sequence according to the distribution of the target edge vertex sequence in the three-dimensional grid model.

206. And calculating the texture coordinates of the internal vertex set according to the texture coordinates of the target edge vertex sequence and the joint relation between the target edge vertex sequence and the internal vertex set in the three-dimensional grid model.

207. And filling the texture of the empty communication block according to the texture of the edge where the target edge vertex sequence is located and the texture coordinates of the internal vertex set.

In the embodiment of the present application, regarding the method for acquiring the three-dimensional grid model and the live-action image of the target object, reference may be made to the related content of the above-mentioned conventional scheme, which is not described herein again. Based on the three-dimensional grid model of the target object and the live-action image, a cavity communicating block formed by the triangular surface patch which is not filled by the texture of the live-action image can be determined from the three-dimensional grid model according to the mapping relation between the three-dimensional grid model of the target object and the live-action image.

Specifically, according to the mapping relation between the three-dimensional grid model of the target object and the live-action image, the texture of the live-action image can be mapped to the triangular patch of the three-dimensional grid model, so as to obtain the triangular patch which is not mapped by the texture. Wherein the number of triangular patches that are not texture mapped may be 1 or more. In the embodiments of the present application, a plurality means 2 or more. The specific number of triangular patches that are not texture mapped may be determined by the area with the acquired live image coverage.

Further, the cavity communicating block formed by the triangular patches which are not subjected to texture mapping can be determined according to the connection relation of the triangular patches which are not subjected to texture mapping in the three-dimensional grid model. The number of the cavity communicating blocks can be 1 or more, and is specifically determined by the distribution position of the triangular patches which are not mapped by the texture in the three-dimensional grid model. Optionally, searching in the three-dimensional grid model by adopting a breadth-first mode according to the connection relation of the triangular patches which are not mapped by the texture in the three-dimensional grid model, and determining the cavity communication blocks formed by the triangular patches which are not mapped. Each cavity communicating block consists of at least one triangular surface patch which is not mapped by textures, and the triangular surface patches forming the cavity communicating block are mutually communicated. For example, as shown in fig. 3, triangular patches which are not texture mapped are h1-h5, and a communicating block (i.e., a white area portion in fig. 3) formed by the triangular patches h1-h5 is a cavity communicating block.

Based on the determined hole communicating block in the three-dimensional mesh model, in step 202, an internal vertex set and a communicated edge vertex sequence contained in the hole communicating block may be determined according to a connection relationship of the hole communicating block in the three-dimensional mesh model.

In the embodiment of the application, for convenience of description and distinction, a triangular patch in the three-dimensional grid model, which is not filled by textures of a live-action image, is defined as a first triangular patch; and defining a triangular patch filled with textures of the live-action image in the three-dimensional grid model as a second triangular patch. Accordingly, in step 202, the edge vertex sequence refers to a sequence formed by vertices of an edge shared by the first triangular surface patch and the second triangular surface patch in the three-dimensional grid model; an internal vertex set refers to a set of vertices of an edge of the first triangular patch that is not collinear with the second triangular patch.

For example, as shown in fig. 3, the communicating blocks (i.e., white area portions in fig. 3) formed by triangular patches h1-h5 are hollow communicating blocks. According to the connection relation of the cavity communicating blocks formed by the triangular patches h1-h5 in the three-dimensional grid model, the internal vertex set contained in the cavity communicating blocks can be determined to be vertex v0; the connected edge vertex sequence is the sequence formed by edge vertices v1-v 5.

Specifically, according to the connection relation of the cavity communication block in the three-dimensional grid model, an internal vertex set and an edge vertex set included in the cavity communication block can be determined. Wherein, the edge vertex set refers to a set of vertices of an edge shared by the first triangular surface patch and the second triangular surface patch in the three-dimensional grid model. Further, a sequence of connected edge vertices may be determined based on a join relationship of the set of edge vertices in the three-dimensional mesh model. The number of edge vertex sequences is 1 or more. The plural means 2 or more than 2. The front and back vertices in each edge vertex sequence are joined in a three-dimensional mesh model.

For example, as shown in fig. 1c, the edge vertex set is vertices v0-v25, and according to the connection relationship of the edge vertex set v0-v25 in the three-dimensional grid model, the connected edge vertex sequences can be determined to be 3, namely, the edge vertex sequence formed by the edge vertices v0-v15, the edge vertex sequence formed by the edge vertices v16-v20 and the edge vertex sequence formed by the edge vertices v21-v 25. In FIG. 3, the edge vertex set is vertices v1-v5, and the connected edge vertex sequence can be determined to be the edge vertex sequence formed by vertices v1-v 5.

In the embodiment of the present application, in order to improve universality of hole texture filling and texture filling effect, the hole communicating block shown in fig. 1b is prevented from being unable to implement texture filling, and in step 203, for any edge vertex v in the edge vertex sequence _i The edge vertex v can be determined according to the connection relation of the edge vertex sequence in the three-dimensional grid model _i The joined other edge vertices form a circular sequence.

The circular sequence means that edge vertices included in the circular sequence form a circle around edge vertex vi. Wherein the annular sequence comprises 2 or more edge vertexes, specifically edge vertex v _i In a three-dimensional mesh model, the edge vertices that are actually connected are determined. For example, in FIG. 1b above, edge vertices v0 joinIs v2, v16, v20 and v17; and the number of edge vertices joined by other edge vertices is 2. Edge vertices v0 and v3 joined by edge vertex v 2; edge vertices v3 join edge vertices v2 and v4, and so on.

In the embodiment of the present application, the distribution order of edge vertices included in the annular sequence in the three-dimensional mesh model is not limited. In some embodiments, to facilitate subsequent traversals, other edge vertices to which any edge vertex vi joins may be determined from the join relationship of the sequence of edge vertices in the three-dimensional mesh model; and sequencing other edge vertexes connected with the edge vertexes vi according to the set arrangement sequence to obtain a ring sequence. Alternatively, the other edge vertices joined by edge vertex vi may be ordered in a counter-clockwise or clockwise order to obtain a circular sequence. For example, in fig. 1b, the circular sequence formed by the other edge vertices that can be joined by the edge vertex v0 in the counterclockwise arrangement is: edge vertices v2, v16, v20, and v17.

After determining the ring sequence formed by the other edge vertices that join all edge vertices in the edge vertex sequence, in step 204, the edge vertex sequence may be traversed according to the join relationship of the edge vertex sequence in the three-dimensional mesh model and the arrangement order of the edge vertices in the ring sequence, to obtain the target edge vertex sequence.

Optionally, according to the connection relation of the edge vertex sequence in the three-dimensional grid model, performing depth-first traversal on the edge vertex sequence according to the sequence of edge vertices in the annular sequence; and when the number of other edge vertexes connected with any edge vertex vi is greater than 2, repeatedly traversing any edge vertex vi according to the arrangement sequence of the edge vertexes in the annular sequence until the edge vertex traversing in the annular sequence is finished. And traversing each edge vertex in the edge vertex sequence according to the mode to obtain a target edge vertex sequence.

For example, in fig. 1b, for the edge vertex v0, the annular sequence formed by the other edge vertices joined by the edge vertex v0 is the annular sequences v2, v16, v20 and v17 arranged counterclockwise, and when traversing to the annular sequences, the edge vertex v0 may be repeatedly traversed, that is, the traversing order may be: v2- > v0- > v0- > v16- > v0-v20- > v0- > v17. And performing depth-first traversal on the edge vertexes with other degrees of 2 according to the arrangement sequence of the annular sequences until the edge vertex sequence traversal is completed, so as to obtain a target edge vertex sequence.

The number of edge vertex sequences described above may be 1 or more. Each edge vertex sequence is a circular sequence formed by edge vertices. In the case that there are a plurality of edge vertex sequences, each edge vertex sequence may be traversed according to the

steps

203 and 204, to obtain a target edge vertex sequence corresponding to each edge vertex sequence.

After the target edge vertex sequence is obtained, in step 205, texture coordinates of the target edge vertex sequence may be determined according to a distribution of the target edge vertex sequence in the three-dimensional mesh model.

Specifically, a blank virtual texture block corresponding to the hole communicating block may be generated. In the embodiment of the present application, the shape and size of the blank virtual texture block are not limited. In some embodiments, the virtual texture block may be a rectangle that is a blank texture block. Optionally, determining the projection length of the edge of the cavity communicating block in the live-action image; and determining the size of the virtual texture block according to the projection length of the edge of the cavity communication block in the live-action image. Optionally, calculating the sum of projection lengths of edges of the cavity communication blocks in the live-action image; and calculating the sum of projection lengths of edges of the cavity communicating blocks in the live-action image, dividing the sum by the quotient of the circumferential rate, and taking the sum as the length and the width of the virtual texture blocks. Further, a blank virtual texture block may be generated according to the size of the virtual texture block.

After generating a blank virtual texture block corresponding to the hollow texture block, the target edge vertex sequence can be projected onto a manifold with a specified contour in the virtual texture block according to the distribution of the target edge vertex sequence in the three-dimensional grid model, and the projected texture coordinates of the target edge vertex sequence in the virtual texture block are determined as the texture coordinates of the target edge vertex sequence.

The contoured manifold may be a one-dimensional manifold. Alternatively, the target edge vertex sequence can be orthogonally projected onto a fitting plane where the virtual texture block is located, and a contour formed by projection of the target edge vertex sequence on the fitting plane is determined to be a manifold of the specified contour. Alternatively, the manifold of the specified profile may be concentric circles, or the like.

After obtaining the texture coordinates of the target edge vertex sequence, in step 206, the texture coordinates of the internal vertex set may be calculated according to the texture coordinates of the target edge vertex sequence and the join relationship between the target edge vertex sequence and the internal vertex set in the three-dimensional mesh model.

Specifically, the target internal vertices to which the target edge vertex sequence is joined may be determined according to a joining relationship between the target edge vertex sequence and the internal vertex set in the three-dimensional mesh model. For example, in FIG. 3, the target internal vertex to which the target edge vertex sequence joins is determined to be v0 based on the join relationship of the target edge vertex sequences v1-v5 and the internal vertex set v0 in the three-dimensional mesh model.

Further, texture coordinates of the internal vertexes of the target can be used as the to-be-solved quantity, and the weight of the texture coordinates of the edge vertex sequence of the target to-be-solved quantity is determined according to the triangular geometric relationship between the internal vertexes of the target and the edge vertex sequence of the target in the three-dimensional grid model; then, the texture coordinates of the target edge vertex sequence can be weighted and summed according to the weight to be calculated of the texture coordinates of the target edge vertex sequence; and (5) constructing a mathematical model by using the result of weighted summation of texture coordinates of the target edge vertex sequence to be equal to the quantity to be solved. Alternatively, the mathematical model may be a system of Laplace equations.

Further, the mathematical model can be solved to obtain texture coordinates of the internal vertices of the target; and determining texture coordinates of the internal vertex set according to the texture coordinates of the internal vertices of the target. In this embodiment, the texture coordinates of all the internal vertices of the object can be obtained in the same manner, and the texture coordinates of all the internal vertices of the object constitute the texture coordinates of the internal vertex set.

In the embodiment of the present application, the specific implementation manner of determining the weight to be calculated of the texture coordinates of the target edge vertex sequence according to the triangle geometry relationship between the target internal vertex and the target edge vertex sequence in the three-dimensional grid model is not limited. In some embodiments, for a first edge vertex in the target edge vertex sequence, calculating a distance between the first edge vertex and the target internal vertex with texture coordinates of the target internal vertex as a quantity to be solved; the distance comprises a quantity to be calculated; further, determining the weight to be calculated of texture coordinates of the first edge vertex according to the included angles between the first edge vertex and the first edge where the target internal vertex is located and the second edge and the third edge where the target internal vertex is located and the distance between the first edge vertex and the target internal vertex; the weight of the texture coordinates of the first edge vertex comprises a to-be-solved quantity; the second edge and the third edge are edges where front and back edge vertexes of the first edge vertex in the target edge vertex sequence and the target internal vertex are located respectively.

For example, in FIG. 4, the target edge vertex sequence is v _i-1 、v _i And v _i+1 Edge vertex v _i With the internal vertex v of the target ₀ The first edge is edge v ₀ v _i The second side and the third side are respectively the side v ₀ v _i-1 Sum of edges v ₀ v _i+1 The included angles formed by the first edge, the second edge and the third edge are respectively alpha _i-1 And alpha _i . Correspondingly, edge vertex v _i Texture coordinates of (2) to the internal vertices v of the object ₀ The weights of the texture coordinates of (c) can be expressed as:

in formula (1), w _i Representing edge vertex v _i Texture coordinates of (2) to the internal vertices v of the object ₀ Is a texture coordinate weight, lambda _i Representing edge vertex v _i Texture coordinates of (2) to the internal vertices v of the object ₀ Is included in the weight of the sample. k represents the total number of edge vertices in the sequence of target edge vertices,

texture coordinates representing a sequence of vertices of the edge of the object, for vertices v inside the object ₀ Is added to the texture coordinates.

Edge vertex v in determining a sequence of target edge vertices _i Texture coordinates of (2) to the internal vertices v of the object ₀ After the normalized weights of the texture coordinates of (2), the edge vertex v in the target edge vertex sequence can be used _i Texture coordinates of (c), and edge vertices v in the target edge vertex sequence _i Texture coordinates of (2) to the internal vertices v of the object ₀ And constructing a mathematical model. Wherein the mathematical model can be expressed as the following formula (2) and formula (3):

Further, the above equations (2) and (3) can be solved to obtain the target internal vertex v ₀ Is defined in the texture coordinates of the block. And calculating the texture coordinates of each target internal vertex in the internal vertex set in the same mode to obtain the texture coordinates of the internal vertex set.

After determining the texture coordinates of the internal vertex set and the target edge vertex sequence contained in the cavity communication block, in step 207, the cavity communication block may be texture-filled according to the texture of the edge where the target edge vertex sequence is located and the texture coordinates of the internal vertex set, so as to implement cavity texture filling on the surface of the three-dimensional grid model of the target object.

Optionally, taking the texture of the edge where the vertex sequence of the target edge is located as a boundary condition, and adopting a poisson fusion mode to perform inward texture filling on the cavity communication block until the texture is filled to the texture coordinates of the internal vertex set of the cavity communication block, so as to realize cavity texture filling on the surface of the three-dimensional grid model of the target object.

According to the cavity texture filling mode provided by the embodiment, the annular sequence formed by other edge vertexes connected with the edge vertexes in the edge vertex sequence can be determined according to the connection relation of the edge vertex sequence in the edge vertex sequence contained in the cavity communicating block in the three-dimensional grid model; and then traversing the edge vertex sequence according to the sequence of edge vertices in the annular sequence and the connection relation of the edge vertex sequence in the three-dimensional grid model, traversing all edge vertices contained in the edge vertex sequence, relaxing the constraint on the arrangement form of the cavity boundary, supporting the hole filling of multiple sections of edges and embedded texture blocks in the cavity communicating block, enabling the hole texture filling mode to be suitable for various complex use scenes, and improving the universality and texture filling effect of the hole texture filling.

It should be noted that, the execution subjects of each step of the method provided in the above embodiment may be the same device, or the method may also be executed by different devices. For example, the execution subject of steps 401 and 402 may be device a; for another example, the execution body of step 401 may be device a, and the execution body of step 402 may be device B; etc.

In addition, in some of the flows described in the above embodiments and the drawings, a plurality of operations appearing in a specific order are included, but it should be clearly understood that the operations may be performed out of the order in which they appear herein or performed in parallel, the sequence numbers of the operations, such as 401, 402, etc., are merely used to distinguish between the various operations, and the sequence numbers themselves do not represent any order of execution. In addition, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel.

Accordingly, embodiments of the present application also provide a computer-readable storage medium storing computer instructions that, when executed by one or more processors, cause the one or more processors to perform the steps in the above-described hole texture filling method.

Fig. 5 is a schematic structural diagram of a void texture filling device according to an embodiment of the present application. As shown in fig. 5, the void texture filling device includes: a determination module 50a, a traversal module 50b, a calculation module 50c, and a population module 50d.

The determining module 50a is configured to determine, from the three-dimensional mesh model of the target object according to a mapping relationship between the three-dimensional mesh model and the live-action image, a cavity communication block formed by a first triangular patch that is not filled with textures of the live-action image; determining an internal vertex set and a communicated edge vertex sequence contained in the communicated block according to the connection relation of the hollow communicated block in the three-dimensional grid model; the edge vertex sequence refers to a sequence formed by vertices of an edge shared with a second triangular patch in the first triangular patch; the internal vertex set refers to a set of vertices of edges of the first triangular patch that are not shared with the second triangular patch; the second triangular patch refers to a triangular patch filled by textures of the live-action image in the three-dimensional network model; and determining a ring-shaped sequence formed by other edge vertexes connected with any edge vertex according to the connection relation of the edge vertex sequence in the three-dimensional grid model aiming at any edge vertex in the edge vertex sequence.

The traversing module 50b is configured to traverse the edge vertex sequence according to the join relationship of the edge vertex sequence in the three-dimensional mesh model and the order of edge vertices in the ring sequence, so as to obtain a target edge vertex sequence.

The determining module 50a is further configured to determine texture coordinates of the target edge vertex sequence according to a distribution of the target edge vertex sequence in the three-dimensional mesh model.

A calculation module 50c, configured to calculate texture coordinates of the internal vertex set according to texture coordinates of the target edge vertex sequence and a join relationship between the target edge vertex sequence and the internal vertex set in the three-dimensional mesh model.

And a filling module 50d, configured to perform texture filling on the hole communication block according to the texture of the edge where the target edge vertex sequence is located and the texture coordinates of the internal vertex set.

In some embodiments, the determining module 50a is specifically configured to, when determining the cavity communication block formed by the first triangular patch not filled by the texture of the live-action image: according to the mapping relation between the three-dimensional grid model of the target object and the live-action image, mapping the texture of the live-action image to a triangular patch of the three-dimensional grid model so as to obtain the first triangular patch; and determining a cavity communicating block formed by the first triangular surface patch according to the connection relation of the first triangular surface patch in the three-dimensional grid model.

Optionally, the determining module 50a is specifically configured to, when determining the internal vertex set and the connected edge vertex sequence contained in the hole connected block: determining an internal vertex set and an edge vertex set contained in the cavity communicating block according to the connection relation of the cavity communicating block in the three-dimensional grid model; and determining a communicated edge vertex sequence according to the connection relation of the edge vertex set in the three-dimensional grid model.

Optionally, the determining module 50a is specifically configured to, when determining the annular sequence formed by the other edge vertices associated with any edge vertex: determining other edge vertexes connected with any edge vertex according to the connection relation of the edge vertex sequence in the three-dimensional grid model; and sequencing the other edge vertexes according to a set arrangement order to obtain the annular sequence.

Optionally, the traversing module 50b traverses the edge vertex sequence, specifically for: according to the connection relation of the edge vertex sequence in the three-dimensional grid model, performing depth-first traversal on the edge vertex sequence according to the sequence of edge vertices in the annular sequence; and when the number of other edge vertexes connected with any edge vertex is larger than 2, repeatedly traversing any edge vertex according to the sequence of the edge vertexes in the annular sequence when traversing to the edge vertexes in the annular sequence, and ending the edge vertex traversal in the annular sequence.

In some embodiments, as shown in fig. 5, the hole texture filling apparatus may further include: and a generation module 50e. The generating module 50c is configured to generate a blank virtual texture block corresponding to the hole communicating block before determining texture coordinates of the target edge vertex sequence according to the distribution of the target edge vertex sequence in the three-dimensional grid model.

Optionally, when generating the blank virtual texture block corresponding to the hole communication block, the generating module 50c is specifically configured to: determining the projection length of the edge of the cavity communicating block in the live-action image; determining the size of the virtual texture block according to the projection length of the edge of the cavity communication block in the live-action image; and generating the blank virtual texture block according to the size of the virtual texture block.

Accordingly, the determining module 50a is specifically configured to, when determining texture coordinates of the target edge vertex sequence: projecting the target edge vertex sequence to a manifold with a specified contour in the virtual texture block according to the distribution of the target edge vertex sequence in the three-dimensional grid model; and determining the texture coordinates of the projection of the target edge vertex sequence on the virtual texture block as the texture coordinates of the target edge vertex sequence.

Optionally, the hole texture filling device may further include: a projection module 50f. A projection module 50f, configured to orthographically project the target edge vertex sequence to a fitting plane in which the virtual texture block is located; and determining a contour formed by projection of the target edge vertex sequence on a fitting plane, and taking the contour as a manifold of the specified contour.

In some embodiments, calculation module 50c, when calculating texture coordinates for the set of internal vertices, is specifically configured to: determining target internal vertexes connected with the target edge vertex sequence according to the connection relation between the target edge vertex sequence and the internal vertex set in the three-dimensional grid model; taking texture coordinates of the internal vertexes of the target as the quantity to be solved, and determining the weight of the texture coordinates of the edge vertex sequence of the target on the quantity to be solved according to the triangle geometric relationship between the internal vertexes of the target and the edge vertex sequence of the target in the three-dimensional grid model; according to the weight of the to-be-solved quantity of the texture coordinates of the target edge vertex sequence, carrying out weighted summation on the texture coordinates of the target edge vertex sequence; a result of weighted summation by using texture coordinates of the target edge vertex sequence is equal to the quantity to be solved, and a mathematical model is constructed; solving the mathematical model to obtain texture coordinates of the internal vertexes of the target; and determining the texture coordinates of the internal vertex set according to the texture coordinates of the internal vertices of the target.

Further, the calculation module 50c is specifically configured to, when determining the weight of the texture coordinates of the target edge vertex sequence to the to-be-solved quantity: aiming at a first edge vertex in a target edge vertex sequence, calculating the distance between the first edge vertex and the target internal vertex by taking texture coordinates of the target internal vertex as a quantity to be solved; the distance comprises the to-be-solved quantity; determining the weight of the texture coordinates of the first edge vertex to the to-be-calculated amount according to the included angles between the first edge vertex and the first edge where the target internal vertex is located and the second edge and the third edge where the target internal vertex is located, and the distance between the first edge vertex and the target internal vertex; the weight of the texture coordinates of the first edge vertex comprises the to-be-solved quantity; the second edge and the third edge are edges where the front edge vertex and the rear edge vertex of the first edge vertex in the target variable vertex sequence and the target internal vertex are respectively located.

Optionally, the filling module 50d is specifically configured to, when performing texture filling on the hole communication block: and taking the texture of the edge where the vertex sequence of the target edge is located as a boundary condition, and adopting a poisson fusion mode to fill the inward texture of the cavity communicating block until the texture is filled to the texture coordinates of the internal vertex set of the cavity communicating block.

According to the void texture filling device provided by the embodiment, the annular sequence formed by other edge vertexes connected with the edge vertexes in the edge vertex sequence can be determined according to the connection relation of the edge vertex sequence in the edge vertex sequence contained in the void communication block in the three-dimensional grid model; and then traversing the edge vertex sequence according to the sequence of edge vertices in the annular sequence and the connection relation of the edge vertex sequence in the three-dimensional grid model, traversing all edge vertices contained in the edge vertex sequence, relaxing the constraint on the arrangement form of the cavity boundary, supporting the hole filling of multiple sections of edges and embedded texture blocks in the cavity communicating block, enabling the hole texture filling mode to be suitable for various complex use scenes, and improving the universality and texture filling effect of the hole texture filling.

Fig. 6 is a schematic structural diagram of a computing device according to an embodiment of the present application. As shown in fig. 6, the computing device includes: a memory 60a and a processor 60b. Wherein the memory 60a is for storing a computer program.

The processor 60b is coupled to the memory 60a for executing a computer program for: according to the mapping relation between the three-dimensional grid model of the target object and the real scene image, determining a cavity communicating block formed by a first triangular patch which is not filled by the texture of the real scene image from the three-dimensional grid model; determining an internal vertex set and a communicated edge vertex sequence contained in the cavity communicating block according to the connection relation of the cavity communicating block in the three-dimensional grid model; the edge vertex sequence refers to a sequence formed by the vertices of the edges shared with the second triangular patch in the first triangular patch, and the internal vertex set refers to a set of the vertices of the edges not shared with the second triangular patch in the first triangular patch; the second triangular patch refers to a triangular patch filled by textures of the live-action image in the three-dimensional network model; determining a ring sequence formed by other edge vertexes connected with any edge vertex according to the connection relation of the edge vertex sequence in the three-dimensional grid model aiming at any edge vertex in the edge vertex sequence; traversing the edge vertex sequence according to the connection relation of the edge vertex sequence in the three-dimensional grid model and the sequence of edge vertices in the annular sequence to obtain a target edge vertex sequence; according to the distribution of the target edge vertex sequence in the three-dimensional grid model, determining texture coordinates of the target edge vertex sequence; calculating texture coordinates of the internal vertex set according to texture coordinates of the target edge vertex sequence and the connection relation between the target edge vertex sequence and the internal vertex set in the three-dimensional grid model; and filling the texture of the empty communication block according to the texture of the edge where the target edge vertex sequence is located and the texture coordinates of the internal vertex set.

Optionally, the processor 60b is specifically configured to, when determining the cavity communicating block formed by the first triangular patch not filled by the texture of the live-action image: according to the mapping relation between the three-dimensional grid model of the target object and the live-action image, mapping the texture of the live-action image to a triangular patch of the three-dimensional grid model to obtain a first triangular patch; and determining a cavity communicating block formed by the first triangular surface patch according to the connection relation of the first triangular surface patch in the three-dimensional grid model.

Optionally, the processor 60b is specifically configured to, when determining the internal vertex set and the connected edge vertex sequence contained in the hole connected block: determining an internal vertex set and an edge vertex set contained in the cavity communicating block according to the connection relation of the cavity communicating block in the three-dimensional grid model; and determining a communicated edge vertex sequence according to the joint relation of the edge vertex set in the three-dimensional grid model.

Optionally, the processor 60b is specifically configured to, when determining the annular sequence formed by the other edge vertices joined by any edge vertex: determining other edge vertexes connected with any edge vertex according to the connection relation of the edge vertex sequence in the three-dimensional grid model; and sequencing other edge vertexes according to the set arrangement order to obtain a ring sequence.

Optionally, the processor 60b traverses the sequence of edge vertices, in particular: according to the joint relation of the edge vertex sequence in the three-dimensional grid model, performing depth-first traversal on the edge vertex sequence according to the sequence of edge vertices in the annular sequence; when the number of other edge vertexes connected with any edge vertex is larger than 2, repeatedly traversing any edge vertex according to the sequence of the edge vertexes in the annular sequence when traversing to the edge vertexes in the annular sequence, and ending the edge vertex traversal in the annular sequence.

Optionally, the processor 60b is further configured to: and generating blank virtual texture blocks corresponding to the cavity communicating blocks before determining texture coordinates of the target edge vertex sequences according to the distribution of the target edge vertex sequences in the three-dimensional grid model.

Optionally, when generating the empty virtual texture block corresponding to the hole communication block, the processor 60b is specifically configured to: determining the projection length of the edge of the cavity communicating block in the live-action image; determining the size of a virtual texture block according to the projection length of the edge of the cavity communication block in the live-action image; and generating blank virtual texture blocks according to the sizes of the virtual texture blocks.

Accordingly, the processor 60b is specifically configured to, when determining texture coordinates of the target edge vertex sequence according to the distribution of the target edge vertex sequence in the three-dimensional mesh model: projecting the target edge vertex sequence to a manifold with a specified contour in the virtual texture block according to the distribution of the target edge vertex sequence in the three-dimensional grid model; and determining the projected texture coordinates of the target edge vertex sequence in the virtual texture block as the texture coordinates of the target edge vertex sequence.

Optionally, the processor 60b is further configured to: orthogonally projecting the target edge vertex sequence to a fitting plane where the virtual texture block is located; and determining the outline formed by projection of the vertex sequence of the target edge on the fitting plane, and taking the outline as the manifold of the appointed outline.

In some embodiments, processor 60b is specifically configured to, when calculating texture coordinates for the internal vertex set: determining target internal vertexes connected with the target edge vertex sequence according to the connection relation between the target edge vertex sequence and the internal vertex set in the three-dimensional grid model; taking texture coordinates of the internal vertexes of the target as the to-be-solved weight of the texture coordinates of the edge vertex sequences of the target according to the triangle geometric relationship between the internal vertexes of the target and the edge vertex sequences of the target in the three-dimensional grid model; according to the weight to be calculated of the texture coordinates of the target edge vertex sequence, carrying out weighted summation on the texture coordinates of the target edge vertex sequence; the result of weighted summation by using texture coordinates of the target edge vertex sequence is equal to the quantity to be solved, and a mathematical model is constructed; solving the mathematical model to obtain texture coordinates of the internal vertexes of the target; and determining the texture coordinates of the internal vertex set according to the texture coordinates of the internal vertices of the target.

Further, the processor 60b is specifically configured to, when determining the weights to be solved for by the texture coordinates of the target edge vertex sequence: aiming at a first edge vertex in the target edge vertex sequence, calculating the distance between the first edge vertex and the target internal vertex by taking texture coordinates of the target internal vertex as a to-be-calculated quantity; the distance contains the quantity to be calculated; determining the weight to be calculated of texture coordinates of the first edge vertex according to the included angles between the first edge vertex and the first edge of the target internal vertex and the second edge and the third edge of the target internal vertex respectively and the distance between the first edge vertex and the target internal vertex; the weight of the texture coordinates of the first edge vertex comprises a to-be-solved quantity; the second edge and the third edge are edges where front and back edge vertexes of the first edge vertex in the target variable vertex sequence and the target internal vertex are respectively located.

In some embodiments, the processor 60b is specifically configured to, when texture filling the hole-communicating block: and taking the texture of the edge where the vertex sequence of the target edge is located as a boundary condition, and adopting a poisson fusion mode to fill the inward texture of the cavity communication block until the texture is filled to the texture coordinates of the internal vertex set of the cavity communication block.

In some alternative implementations, as shown in fig. 6, the computing device may further include: optional components such as a communication component 60c, a display component 60d, a power component 60e, and an audio component 60 f. Only a portion of the components are schematically shown in fig. 6, which does not mean that the computing device must contain all of the components shown in fig. 6, nor that the computing device can only include the components shown in fig. 6.

The computing device provided by the embodiment can determine a ring sequence formed by other edge vertexes connected by edge vertexes in the edge vertex sequence according to the connection relation of the edge vertex sequence in the edge vertex sequence contained in the cavity communication block in the three-dimensional grid model; and then traversing the edge vertex sequence according to the sequence of edge vertices in the annular sequence and the connection relation of the edge vertex sequence in the three-dimensional grid model, traversing all edge vertices contained in the edge vertex sequence, relaxing the constraint on the arrangement form of the cavity boundary, supporting the hole filling of multiple sections of edges and embedded texture blocks in the cavity communicating block, enabling the hole texture filling mode to be suitable for various complex use scenes, and improving the universality and texture filling effect of the hole texture filling.

In embodiments of the present application, the memory is used to store a computer program and may be configured to store various other data to support operations on the device on which it resides. Wherein the processor may execute a computer program stored in the memory to implement the corresponding control logic. The memory may be implemented by any type of volatile or nonvolatile memory device or combination thereof, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

In the embodiments of the present application, the processor may be any hardware processing device that may execute the above-described method logic. Alternatively, the processor may be a central processing unit (Central Processing Unit, CPU), a graphics processor (Graphics Processing Unit, GPU) or a micro control unit (Microcontroller Unit, MCU); programmable devices such as Field programmable gate arrays (Field-Programmable Gate Array, FPGA), programmable array logic devices (Programmable Array Logic, PAL), general array logic devices (General Array Logic, GAL), complex programmable logic devices (Complex Programmable Logic Device, CPLD), and the like; or an advanced Reduced Instruction Set (RISC) processor (Advanced RISC Machines, ARM) or System On Chip (SOC), etc., but is not limited thereto.

In embodiments of the present application, the communication component is configured to facilitate wired or wireless communication between the device in which it resides and other devices. The device in which the communication component is located may access a wireless network based on a communication standard, such as WiFi,2G or 3G,4G,5G or a combination thereof. In one exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component may also be implemented based on Near Field Communication (NFC) technology, radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, or other technologies.

In embodiments of the present application, the display assembly may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the display assembly includes a touch panel, the display assembly may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or slide action, but also the duration and pressure associated with the touch or slide operation.

In embodiments of the present application, the power supply assembly is configured to provide power to the various components of the device in which it is located. The power components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the devices in which the power components are located.

In embodiments of the present application, the audio component may be configured to output and/or input audio signals. For example, the audio component includes a Microphone (MIC) configured to receive external audio signals when the device in which the audio component is located is in an operational mode, such as a call mode, a recording mode, and a speech recognition mode. The received audio signal may be further stored in a memory or transmitted via a communication component. In some embodiments, the audio assembly further comprises a speaker for outputting audio signals. For example, for a device with language interaction functionality, voice interaction with a user, etc., may be accomplished through an audio component.

It should be noted that, the descriptions of "first" and "second" herein are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, and are not limited to the "first" and the "second" being different types.

It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

The storage medium of the computer is a readable storage medium, which may also be referred to as a readable medium. Readable storage media, including both permanent and non-permanent, removable and non-removable media, may be implemented in any method or technology for information storage. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.

The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and changes may be made to the present application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. which are within the spirit and principles of the present application are intended to be included within the scope of the claims of the present application.

Claims

1. A method of filling a void texture, comprising:

2. The method according to claim 1, wherein determining, from the three-dimensional mesh model, a cavity communication block formed by a first triangular patch not filled by a texture of a live-action image according to a mapping relationship between the three-dimensional mesh model of the target object and the live-action image, includes:

According to the mapping relation between the three-dimensional grid model of the target object and the live-action image, mapping the texture of the live-action image to a triangular patch of the three-dimensional grid model so as to obtain the first triangular patch;

and determining a cavity communicating block formed by the first triangular surface patch according to the connection relation of the first triangular surface patch in the three-dimensional grid model.

3. The method according to claim 1, wherein the determining the internal vertex set and the connected edge vertex sequence contained by the hole connected block according to the connection relation of the hole connected block in the three-dimensional grid model comprises:

determining an internal vertex set and an edge vertex set contained in the cavity communicating block according to the connection relation of the cavity communicating block in the three-dimensional grid model;

and determining a communicated edge vertex sequence according to the connection relation of the edge vertex set in the three-dimensional grid model.

4. The method according to claim 1, wherein determining the ring sequence formed by the other edge vertices of the edge vertex junction according to the junction relation of the edge vertex sequence in the three-dimensional grid model comprises:

Determining other edge vertexes connected with any edge vertex according to the connection relation of the edge vertex sequence in the three-dimensional grid model;

and sequencing the other edge vertexes according to a set arrangement order to obtain the annular sequence.

5. The method of claim 1, wherein traversing the sequence of edge vertices to obtain the sequence of target edge vertices based on a join relationship of the sequence of edge vertices in the three-dimensional mesh model and an order of edge vertices in the circular sequence comprises:

according to the connection relation of the edge vertex sequence in the three-dimensional grid model, performing depth-first traversal on the edge vertex sequence according to the sequence of edge vertices in the annular sequence;

and when the number of other edge vertexes connected with any edge vertex is larger than 2, repeatedly traversing any edge vertex according to the sequence of the edge vertexes in the annular sequence when traversing to the edge vertexes in the annular sequence, and ending the edge vertex traversal in the annular sequence.

6. The method of claim 1, further comprising, prior to determining texture coordinates of the sequence of target edge vertices based on a distribution of the sequence of target edge vertices in the three-dimensional mesh model:

Generating a blank virtual texture block corresponding to the cavity communication block;

the determining texture coordinates of the target edge vertex sequence according to the distribution of the target edge vertex sequence in the three-dimensional grid model comprises the following steps:

projecting the target edge vertex sequence to a manifold with a specified contour in the virtual texture block according to the distribution of the target edge vertex sequence in the three-dimensional grid model;

and determining the texture coordinates of the projection of the target edge vertex sequence on the virtual texture block as the texture coordinates of the target edge vertex sequence.

7. The method as recited in claim 6, further comprising:

orthogonally projecting the target edge vertex sequence to a fitting plane where the virtual texture block is located;

and determining the outline formed by projection of the target edge vertex sequence on a fitting plane, and taking the outline as the manifold of the appointed outline.

8. The method of claim 6, wherein generating the empty virtual texture block corresponding to the hole communication block comprises:

determining the projection length of the edge of the cavity communicating block in the live-action image;

determining the size of the virtual texture block according to the projection length of the edge of the cavity communication block in the live-action image;

And generating the blank virtual texture block according to the size of the virtual texture block.

9. The method of claim 1, wherein the computing the texture coordinates of the set of internal vertices based on the texture coordinates of the sequence of target edge vertices and the join relationship of the sequence of target edge vertices and the set of internal vertices in the three-dimensional mesh model comprises:

determining target internal vertexes connected with the target edge vertex sequence according to the connection relation between the target edge vertex sequence and the internal vertex set in the three-dimensional grid model;

taking texture coordinates of the internal vertexes of the target as the quantity to be solved, and determining the weight of the texture coordinates of the edge vertex sequence of the target on the quantity to be solved according to the triangle geometric relationship between the internal vertexes of the target and the edge vertex sequence of the target in the three-dimensional grid model;

according to the weight of the to-be-solved quantity of the texture coordinates of the target edge vertex sequence, carrying out weighted summation on the texture coordinates of the target edge vertex sequence;

a result of weighted summation by using texture coordinates of the target edge vertex sequence is equal to the quantity to be solved, and a mathematical model is constructed;

Solving the mathematical model to obtain texture coordinates of the internal vertexes of the target;

and determining the texture coordinates of the internal vertex set according to the texture coordinates of the internal vertices of the target.

10. The method of claim 9, wherein determining the weight of the texture coordinates of the target edge vertex sequence to the quantity to be solved according to the triangle geometry relationship between the target interior vertex and the target edge vertex sequence in the three-dimensional mesh model comprises:

aiming at a first edge vertex in a target edge vertex sequence, calculating the distance between the first edge vertex and the target internal vertex by taking texture coordinates of the target internal vertex as a quantity to be solved; the distance comprises the to-be-solved quantity;

determining the weight of the texture coordinates of the first edge vertex to the to-be-calculated amount according to the included angles between the first edge vertex and the first edge where the target internal vertex is located and the second edge and the third edge where the target internal vertex is located, and the distance between the first edge vertex and the target internal vertex; the weight of the texture coordinates of the first edge vertex comprises the to-be-solved quantity;

The second edge and the third edge are edges where the front edge vertex and the rear edge vertex of the first edge vertex in the target variable vertex sequence and the target internal vertex are respectively located.

11. The method according to any one of claims 1-10, wherein said texture filling the hole-communicating block according to the texture of the edge where the target edge vertex sequence is located and the texture coordinates of the internal vertex set, comprises:

and taking the texture of the edge where the vertex sequence of the target edge is located as a boundary condition, and adopting a poisson fusion mode to fill the inward texture of the cavity communicating block until the texture is filled to the texture coordinates of the internal vertex set of the cavity communicating block.

12. A void texture filling apparatus, comprising:

13. A computing device, comprising: a memory and a processor; wherein the memory is used for storing a computer program;

the processor is coupled to the memory for executing the computer program for performing the steps in the method of any of claims 1-11.

14. A computer-readable storage medium storing computer instructions that, when executed by one or more processors, cause the one or more processors to perform the steps in the method of any of claims 1-11.