CN117853641A

CN117853641A - Texture coding and decoding method, device, equipment and storage medium

Info

Publication number: CN117853641A
Application number: CN202410129428.XA
Authority: CN
Inventors: 王钦佳
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2024-01-29
Filing date: 2024-01-29
Publication date: 2024-04-09

Abstract

The application provides a texture coding and decoding method, a device, equipment and a storage medium, which can be applied to the fields of image processing, texture compression and the like and comprises the following steps: acquiring a first texture map, and performing grid division on the first texture map to obtain a first grid map; optimizing a first texture coordinate of each grid vertex in the first grid graph to obtain a second texture coordinate of the grid vertex; resampling the first texture map based on the second texture coordinates of each grid vertex to obtain a second texture map; and obtaining a compressed file of the first texture map based on the second texture map. The first texture map is subjected to grid subdivision, the texture coordinates are adjusted and optimized, and a second texture map with finer texture is generated based on the adjusted and optimized texture coordinates and the first texture map, so that the coding and decoding effects of the texture map can be improved when the second texture map is subjected to texture decoding.

Description

Texture coding and decoding method, device, equipment and storage medium

Technical Field

The embodiment of the application relates to the technical field of computers, in particular to a texture coding and decoding method, a device, equipment and a storage medium.

Background

Texture coding, also known as texture compression, is a technique for reducing the size of texture data, and is widely used in the field of computer graphics. Texture compression can reduce memory footprint of a graphics processing unit (Graphics Processing Unit, GPU for short), reduce texture loading time, improve rendering performance, and reduce bandwidth consumption while maintaining acceptable image quality.

The texture compression algorithms that are common at present are: s3 texture compression (S3 Texture Compression, abbreviated S3 TC), ericsson texture compression (Ericsson Texture Compression, abbreviated ETC), adaptive scalable texture compression (Adaptive Scalable Texture Compression, abbreviated ASTC), and the like. These texture compression techniques divide textures into blocks and compress the textures in units of blocks. To support random fast access of textures, the compressed size of the block is fixed, i.e. the amount of data of the information is consistent. This makes the degree of signal loss of different regions in the image at the final reconstruction inconsistent, e.g. the low frequency signal region can be closer to the original signal than the high frequency signal region, resulting in an undesirable texture compression effect.

Disclosure of Invention

The application provides a texture coding and decoding method, a device, equipment and a storage medium, which can improve the performance of texture coding and decoding and further improve the rendering effect of a map.

In a first aspect, the present application provides a texture coding method, including:

acquiring a first texture map to be encoded, and performing grid division on the first texture map to obtain a first grid map;

optimizing a first texture coordinate of each grid vertex in the first grid graph to obtain a second texture coordinate of the grid vertex;

resampling is carried out in the first texture map based on the second texture coordinates of each grid vertex to obtain a second texture map;

and obtaining a compressed file of the first texture map based on the second texture map, wherein the compressed file comprises the second texture map.

In a second aspect, the present application provides a texture decoding method, including:

obtaining a compressed file of a first texture map, wherein the compressed file comprises a second texture map, the second texture map is obtained by resampling the first texture map based on second texture coordinates of each network vertex in a first grid map, the first grid map is obtained by grid division of the first texture map, and the second texture coordinates of the grid vertices are obtained by optimizing the first texture coordinates of the grid vertices;

And reconstructing the first texture map or determining a pixel value corresponding to the target texture coordinate in the first texture map based on the second texture map.

In a third aspect, the present application provides a texture coding apparatus comprising:

the grid dividing unit is used for obtaining a first texture map to be encoded and carrying out grid division on the first texture map to obtain a first grid map;

the optimization unit is used for optimizing the first texture coordinates of each grid vertex in the first grid graph to obtain the second texture coordinates of the grid vertex;

the resampling unit is used for resampling in the first texture mapping based on the second texture coordinates of each grid vertex to obtain a second texture mapping;

and the encoding unit is used for obtaining a compressed file of the first texture mapping based on the second texture mapping, wherein the compressed file comprises the second texture mapping.

In some embodiments, the meshing unit is specifically configured to map the first texture map into a unit rectangular space to obtain a unitized first texture map; and carrying out grid division on the unitized first texture map to obtain the first grid map.

In some embodiments, the resampling unit is specifically configured to generate a blank unit rectangular area; based on the topological structure of each grid vertex in the first grid map and the second texture coordinates, carrying out grid division on the unit rectangular area to obtain a second grid map; searching an ith pixel value in the unitized first texture map based on a second texture coordinate of an ith grid vertex in the second grid map, wherein i is a positive integer; and determining a pixel value corresponding to the ith grid vertex in the second grid map based on the ith pixel value to obtain the second texture map.

In some embodiments, the resampling unit is specifically configured to determine the ith pixel value as a pixel value corresponding to the ith mesh vertex in the second mesh map, so as to obtain a third texture map; and interpolating the third texture map to obtain the second texture map.

In some embodiments, the resampling unit is specifically configured to optimize, for a jth interpolation point in the third texture map, a first texture coordinate of the jth interpolation point to obtain a second texture coordinate of the jth interpolation point, where j is a positive integer; searching a j pixel value in the unitized first texture map based on the second texture coordinate of the j interpolation point; determining a pixel value corresponding to the j interpolation point in the third texture map based on the j pixel value to obtain a third texture map after interpolation; and obtaining the second texture map based on the interpolated third texture map.

In some embodiments, the resampling unit is specifically configured to determine a first bijective transformation function from a preset plurality of bijective transformation functions; and optimizing the first texture coordinates by adopting the first bijective transformation function to obtain the second texture coordinates.

In some embodiments, the compressed file further includes at least one of: the first texture coordinates and the second texture coordinates of each grid vertex in the first grid graph, the topological structure of each grid vertex and the indication information of the first bijective transformation function.

In a fourth aspect, the present application provides a texture decoding apparatus, comprising:

the device comprises an acquisition unit, a storage unit and a processing unit, wherein the acquisition unit is used for acquiring a compressed file of a first texture map, the compressed file comprises a second texture map, the second texture map is obtained by resampling the first texture map based on second texture coordinates of each network vertex in a first grid map, the first grid map is obtained by grid division of the first texture map, and the second texture coordinates of the grid vertices are obtained by optimizing the first texture coordinates of the grid vertices;

and the decoding unit is used for reconstructing the first texture map or determining a pixel value corresponding to the target texture coordinate in the first texture map based on the second texture map.

In some embodiments, if the compressed file further includes a first texture coordinate and a second texture coordinate of each mesh vertex in the first mesh map, the decoding unit is specifically configured to generate a blank unit rectangular area; determining the topological structure of each grid vertex in the first grid graph, and carrying out grid division on the unit rectangular area based on the topological structure of each grid vertex in the first grid graph and the first texture coordinates to obtain a third grid graph; for an ith grid vertex in the third grid map, searching an ith pixel value corresponding to the ith grid vertex in the second texture map based on a second texture coordinate of the ith grid vertex, wherein i is a positive integer; and determining a pixel value corresponding to the ith network vertex in the third grid graph based on the ith pixel value to obtain a reconstructed first texture map.

In some embodiments, if the compressed file includes a topology structure of each mesh vertex in the first mesh map, the decoding unit is specifically configured to decode the compressed file to obtain the topology structure of each mesh vertex in the first mesh map.

In some embodiments, the decoding unit is specifically configured to determine the ith pixel value as a pixel value corresponding to the ith mesh vertex in the third mesh map, to obtain a fourth texture map; and interpolating the fourth texture map to obtain the reconstructed first texture map.

In some embodiments, the decoding unit is specifically configured to determine, for a jth interpolation point in the fourth texture map, a second texture coordinate corresponding to a first texture coordinate of the jth interpolation point, where j is a positive integer; searching a j pixel value in the second texture map based on the second texture coordinate of the j interpolation point; determining a pixel value corresponding to a j interpolation point in the fourth texture map based on the j pixel value to obtain an interpolated fourth texture map; and obtaining the reconstructed first texture map based on the interpolated fourth texture map.

In some embodiments, the decoding unit is specifically configured to determine a first bijective transformation function; and optimizing the first texture coordinate of the jth interpolation point based on the first bijection transformation function to obtain the second texture coordinate of the jth interpolation point.

In some embodiments, the decoding unit is specifically configured to determine a second texture coordinate corresponding to the target texture coordinate; a first pixel value searched in the second texture map based on a second texture coordinate corresponding to the target texture coordinate; and determining a pixel value corresponding to the target texture coordinate based on the first pixel value.

In some embodiments, if the compressed file further includes a first texture coordinate and a second texture coordinate of each mesh vertex in the first mesh map, the decoding unit is specifically configured to determine, if the target texture coordinate is the first texture coordinate of the first mesh vertex in the first mesh map, the second texture coordinate of the first mesh vertex as the second texture coordinate corresponding to the target texture coordinate; if the target texture coordinate is not the first texture coordinate of the grid vertex in the first grid graph, determining a first bijection transformation function, optimizing the target texture coordinate based on the first bijection transformation function to obtain a second texture coordinate corresponding to the target texture coordinate, or determining a corresponding relation table between the first texture coordinate and the second texture coordinate based on the first bijection transformation function, and searching the second texture coordinate corresponding to the target texture coordinate in the corresponding relation table.

In some embodiments, if the compressed file includes the indication information of the first bijective transformation function, the decoding unit is specifically configured to decode the compressed file to obtain the indication information of the first bijective transformation function; and determining the first bijective transformation function based on the indication information.

In a fifth aspect, an electronic device is provided that includes a processor and a memory. The memory is used for storing a computer program, and the processor is used for calling and running the computer program stored in the memory to execute the method in any one of the first aspect to the second aspect or each implementation manner thereof.

In a sixth aspect, a chip is provided for implementing the method in any one of the first to second aspects or each implementation thereof. Specifically, the chip includes: a processor for calling and running a computer program from a memory, causing a device on which the chip is mounted to perform the method as in any one of the first to second aspects or implementations thereof described above.

In a seventh aspect, a computer-readable storage medium is provided for storing a computer program that causes a computer to perform the method of any one of the above first to second aspects or implementations thereof.

In an eighth aspect, there is provided a computer program product comprising computer program instructions for causing a computer to perform the method of any one of the first to second aspects or implementations thereof.

A ninth aspect provides a computer program which, when run on a computer, causes the computer to perform the method of any one of the first to second aspects or implementations thereof.

In summary, in the present application, during texture coding, a coding device first obtains a first texture map to be coded, and performs meshing on the first texture map to obtain a first mesh map; optimizing a first texture coordinate of each grid vertex in the first grid graph to obtain a second texture coordinate of the grid vertex; resampling the first texture map based on the second texture coordinates of each grid vertex to obtain a second texture map; and finally, based on the second texture map, obtaining a compressed file of the first texture map, wherein the compressed file comprises the second texture map. That is, in the embodiment of the present application, after grid subdivision is performed on the first texture map, adjustment and optimization are performed on the texture coordinates, and based on the adjusted and optimized texture coordinates and the first texture map, a second texture map with finer texture is generated, so as to obtain a new two-dimensional texture compression format of the GPU. Thus, when the new GPU two-dimensional texture compression format is transmitted to the GPU, the GPU can select two modes of direct rendering or random sampling to finish texture sampling according to specific rendering requirements. For example, the GPU may directly render directly based on the second texture map to obtain the first texture map or directly randomly sample any pixel point, so that the performance cost of the GPU is smaller, decoding is completed without additional cost, and since the second texture map includes finer texture expansion, when performing texture reconstruction or random sampling based on the second texture map, the reconstruction or random sampling effect of the texture can be improved, and further the encoding and decoding effects of the texture map are improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic block diagram of a texture codec system according to an embodiment of the present application;

FIG. 2 is a schematic diagram of texture encoding and decoding according to an embodiment of the present application;

FIG. 3 is a flow chart illustrating a texture encoding method according to an embodiment of the present disclosure;

FIG. 4 is a schematic view of a unit rectangle;

FIG. 5 is a schematic illustration of projecting a first texture map into a unit rectangular space;

FIG. 6A is a diagram of meshing;

FIG. 6B is another meshing schematic illustration;

FIG. 7A is a schematic illustration of a first grid pattern;

FIG. 7B is a schematic diagram of a second grid pattern;

FIG. 7C is a schematic diagram of a second texture map;

FIG. 8 is a flow chart of a texture decoding method according to an embodiment of the present disclosure;

FIG. 9 is a schematic illustration of a third grid pattern;

FIG. 10 is a schematic diagram of a reconstructed first texture map;

FIG. 11 is a schematic block diagram of a texture encoding apparatus provided in an embodiment of the present application;

FIG. 12 is a schematic block diagram of a texture decoding apparatus provided in an embodiment of the present application;

fig. 13 is a schematic block diagram of an electronic device provided in an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present application based on the embodiments herein.

It should be noted that the terms "first," "second," and the like in the description and claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that embodiments of the present application described herein may be implemented in sequences other than those illustrated or otherwise described herein. In the embodiment of the invention, "B corresponding to A" means that B is associated with A. In one implementation, B may be determined from a. It should also be understood that determining B from a does not mean determining B from a alone, but may also determine B from a and/or other information. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or server that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. In the description of the present application, unless otherwise indicated, "a plurality" means two or more than two.

The technical scheme provided by the application can be applied to the technical fields of image processing, texture compression, artificial intelligence, game making and the like, is used for guaranteeing the texture compression quality and simultaneously reducing texture data, and further achieving balance between the texture compression quality and the compressed data volume, improving the texture compression performance and improving the map rendering effect.

Related concepts related to the embodiments of the present application are described below.

Graphics processing unit (Graphics Processing Unit, GPU for short): a special chip for processing graphics and images is disclosed.

Texture: when the model is constructed, an artist expands textures wrapped on the 3D model into a 2D map, each vertex corresponds to one (u, v) coordinate of the 2D map at the moment and is finally exported along with the model, the texture coordinates of the vertices are normalized, the texture coordinate systems of OpenGL and DirectX are different in the [0,1] interval, the OpenGL sets an origin at the lower left corner, and the Y axis is positive; directX sets the origin at the upper left corner and Y-axis is positive downward.

Texture compression: is a technique for reducing the size of texture data, which is widely used in the field of computer graphics. Texture compression can reduce GPU memory footprint, reduce texture loading time, improve rendering performance, and reduce bandwidth consumption while maintaining acceptable image quality.

Texture coordinates: that is, UV, which means coordinates mapped into a two-dimensional plane after two-dimensional expansion of the surface of a three-dimensional model, is generally generated by a modeling software texture expansion function.

Texture mapping: the process of mapping texture coordinates to a texture map is typically done automatically by the GPU, and for each pixel that is ultimately rendered, the texture coordinates are linearly interpolated, and the texture of the particular region is sampled in the texture map, filtered, and returned.

Linear interpolation: also known as linear filtering in GPU texture, the pixel colors used to generate textures stretched or compressed from the texture map may produce smoother, more realistic effects than nearest point filtering.

Downsampling: a process of sampling the image from a larger resolution to a smaller resolution.

For ease of understanding, a texture codec system according to an embodiment of the present application will be described first with reference to fig. 1.

Fig. 1 is a schematic block diagram of a texture codec system according to an embodiment of the present application. It should be noted that fig. 1 is only an example, and the texture codec system of the embodiment of the present application includes, but is not limited to, the one shown in fig. 1. As shown in fig. 1, the texture codec system 100 includes an encoding device 110 and a decoding device 120. Wherein the encoding device is arranged to encode (which may be understood as compressing) the texture map to generate a compressed file of textures and to transmit the compressed file to the decoding device. The decoding device decodes the compressed file generated by the encoding device to obtain a reconstructed value of the texture map or obtain a pixel point at any position in the texture map.

The encoding device 110 of the present embodiment may be understood as a device having a texture encoding function, and the decoding device 120 may be understood as a device having a texture decoding function, i.e., the present embodiment includes a broader means for the encoding device 110 and the decoding device 120, such as including a smart phone, a desktop computer, a mobile computing device, a notebook (e.g., laptop) computer, a tablet computer, a set-top box, a television, a camera, a playing device, a digital media player, a game console, a vehicle-mounted computer, and the like.

In some embodiments, the encoding device 110 may transmit the encoded compressed file to the decoding device 120 via the channel 130. Channel 130 may include one or more media and/or devices capable of transmitting compressed files from encoding device 110 to decoding device 120.

In one example, channel 130 includes one or more communication media that enable encoding device 110 to transmit the encoded compressed file directly to decoding device 120 in real-time. In this example, encoding apparatus 110 may modulate the compressed file according to a communication standard and transmit the compressed file to decoding apparatus 120. Where the communication medium comprises a wireless communication medium, such as a radio frequency spectrum, the communication medium may optionally also comprise a wired communication medium, such as one or more physical transmission lines.

In another example, channel 130 includes a storage medium that may store the compressed file of encoding device 110. Storage media include a variety of locally accessed data storage media such as compact discs, DVDs, flash memory, and the like. In this example, the decoding device 120 may obtain the compressed file from the storage medium.

In another example, the channel 130 may comprise a storage server that may store the compressed file of the encoding device 110. In this example, the decoding device 120 may download the stored compressed file from the storage server. Alternatively, the storage server may store the compressed file and may transmit the compressed file to the decoding device 120, such as a web server (e.g., for a web site), a File Transfer Protocol (FTP) server, or the like.

In some embodiments, the encoding apparatus 110 includes a texture encoder 112 and an output interface 113. Wherein the output interface 113 may comprise a modulator/demodulator (modem) and/or a transmitter.

In some embodiments, the encoding device 110 may include a texture map source 111 in addition to the texture encoder 112 and the input interface 113.

The texture map source 111 may comprise at least one of a texture acquisition device, a texture archive, a texture input interface for receiving texture map raw data from a texture map provider, and a computer system for generating texture maps.

Illustratively, the texture encoder 112 encodes the first texture map from the texture map source 111, producing a compressed file of the first texture map. The compressed file contains the encoded information of the first texture map. The encoding information may include a second texture map and associated data. The associated data may include first and second texture coordinates for each mesh vertex of the first mesh map after meshing, other syntax structures, and the like. A syntax structure refers to a collection of zero or more syntax elements arranged in a specified order in a compressed file.

The texture encoder 112 directly transmits the encoded compressed file to the decoding apparatus 120 via the output interface 113. The compressed file may also be stored on a storage medium or storage server for subsequent reading by the decoding device 120.

In some embodiments, decoding apparatus 120 includes an input interface 121 and a texture decoder 122. Alternatively, texture decoder 122 may include a GPU.

In some embodiments, the decoding apparatus 120 may further include a display device 123 in addition to the input interface 121 and the texture decoder 122.

Wherein the input interface 121 comprises a receiver and/or a modem. The input interface 121 may receive the compressed file through the channel 130.

In one example, in a direct rendering scene, the texture decoder 122 is configured to decode the compressed file to obtain a reconstructed texture map, and transmit the reconstructed texture map to the display device 123 for display.

The display device 123 displays the reconstructed texture map. The display device 123 may be integral with the decoding apparatus 120 or external to the decoding apparatus 120.

Furthermore, fig. 1 is merely an example, and the technical solution of the embodiment of the present application is not limited to fig. 1, for example, the technology of the present application may also be applied to single-sided texture coding or single-sided texture decoding.

Texture compression is an image compression technique used specifically for storing textures in computer graphics rendering systems to address memory, bandwidth, and the like.

The texture coding and decoding method provided by the embodiment of the application can be accessed into the GPU in a texture coding standard mode, needs to be realized in a GPU driving layer, and is opened to a product side through an interface for developers and end users to use. After generating a compressed file of the first texture map, a developer transmits the compressed file to the GPU through an interface and renders the compressed file. The interfaces involved include the following:

Transmitting to the GPU: calling an interface command to transmit a certain coded texture to the GPU;

texture random access support is turned on or off. After the starting, the random access of textures can be supported, but the sampling efficiency is reduced, and a certain memory occupation is increased;

invoking direct rendering in the GPU: and directly rendering the texture serving as a plane texture. In the method, the texture rendering process cannot be randomly accessed;

activating into texture in GPU, and performing random access: and taking the texture as a randomly accessible texture, and embedding the texture into a common rendering flow for sampling.

After the interface is called, the rendering or sampling of the texture can be completed by matching with other rendering instructions.

Fig. 2 is a schematic diagram of texture encoding and decoding according to an embodiment of the present application. The picture compression format is based on the whole picture for compression, and the decoding process between pixels has a dependency relationship. The resolution of a single pixel level cannot be directly realized, as shown in fig. 2, the image in the JPG format needs to be decompressed into the RGBA format by the CPU and then can be read by the GPU, thereby increasing the time, the calculation amount and the bandwidth consumption of the CPU. And the texture compression is based on block compression, so that byte blocks to which pixels belong can be read faster for decompression so as to support random access.

As shown in fig. 2, the current texture compression technique mainly includes: s3TC, ETC, ASTC and PVRTC (PowerVR Texture Compression, a compression format designed specifically for PowerVR graphics cards), etc.

Among them, S3TC is also called DXTn or BCn (Block Compression), and is a texture compression algorithm widely applied to DirectX (Direct eXtension, DX for short) and OpenGL (Open Graphics Library, called open graphics library or open graphics library). S3TC includes multiple variants, such as DXT1, DXT3, DXT5 (or BC1, BC2, BC 3), for different color and transparency formats, respectively. S3TC uses a block compression technique to divide a texture into blocks of 4x4 pixels, and then stores two values representing colors and one interpolation control value for each block, thereby achieving compression.

ETC is a texture compression algorithm designed for OpenGL ES (a cross-platform, well-functioning 2D and 3D graphics application program interface API) and Vulkan (a cross-platform 2D and 3D graphics Application Program Interface (API)). ETC also employs block compression techniques and provides variants of ETC1, ETC2 and EAC (Enhanced Alpha Compression), ETC. ETC1 supports only RGB formats, whereas ETC2 and EAC support RGBA formats. ETC is widely supported on mobile devices.

ASTC is a highly configurable texture compression algorithm suitable for OpenGL, openGL ES and Vulkan. ASTC allows developers to freely choose different compression ratios and texture block sizes, making trade-offs between performance and image quality. ASTC supports multiple color and transparency formats including LDR (Low Dynamic Range ) and HDR (High Dynamic Range, high dynamic range).

PVRTC is a texture compression algorithm designed specifically for PowerVR GPUs. The PVRTC employs a compression technique called vector quantization (Vector Quantization) that divides the texture into different color spaces and interpolates in these color spaces. The PVRTC provides two versions of PVRTC1 and PVRTC2, supporting RGB and RGBA formats. PVRTC is widely supported on iOS devices.

These texture compression algorithms described above may have different support scenarios across different graphics APIs, GPU hardware, and platforms.

The texture compression techniques are all to divide textures into blocks and compress the blocks. To support random fast access of textures, the compressed size of the block is fixed, i.e. the amount of data of the information is consistent. This makes the degree of signal loss of different regions in the image at the final reconstruction inconsistent, e.g. the low frequency signal region can be closer to the original signal than the high frequency signal region, resulting in an undesirable texture compression effect.

In order to solve the above technical problems, an embodiment of the present application provides a texture encoding and decoding method, and when encoding textures, an encoding device firstly obtains a first texture map to be encoded, and performs grid division on the first texture map; for each grid vertex in the first texture map, optimizing the first texture coordinates of the grid vertex to obtain the second texture coordinates of the grid vertex; resampling the first texture map based on the second texture coordinates of each grid vertex to obtain a second texture map; and finally, based on the second texture map, obtaining a compressed file of the first texture map, wherein the compressed file comprises the second texture map. That is, in the embodiment of the present application, after grid subdivision is performed on the first texture map, adjustment and optimization are performed on the texture coordinates, and based on the adjusted and optimized texture coordinates and the first texture map, a finer second texture map is generated, so as to obtain a new two-dimensional texture compression format of the GPU. Thus, when the new GPU two-dimensional texture compression format is transmitted to the GPU, the GPU can select two modes of direct rendering or random sampling to finish texture sampling according to specific rendering requirements. For example, the GPU may directly render directly based on the second texture map to obtain the first texture map or directly randomly sample any pixel point, so that the performance cost of the GPU is smaller, decoding is completed without additional cost, and since the second texture map includes finer texture expansion, when performing texture reconstruction or random sampling based on the second texture map, the reconstruction or random sampling effect of the texture can be improved, and further the encoding and decoding effects of the texture map are improved.

The following describes the technical solutions of the embodiments of the present application in detail through some embodiments. The following embodiments may be combined with each other, and some embodiments may not be repeated for the same or similar concepts or processes.

Firstly, taking an encoding end as an example, a texture encoding method provided in the embodiment of the present application is described.

Fig. 3 is a flowchart illustrating a texture encoding method according to an embodiment of the present application. The execution body of the embodiment of the present application is a device having a texture coding function, for example, a texture coding device. In some embodiments, the texture encoding apparatus may be the encoding device of fig. 1. For convenience of description, the embodiment of the present application will be described taking an execution body as an encoding device as an example.

As shown in fig. 3, the texture coding method in the embodiment of the present application includes:

s101, acquiring a first texture map to be encoded, and performing grid division on the first texture map to obtain a first grid map.

In computer graphics, texture mapping is a technique that uses images, functions, or other data sources to alter the appearance of an object's surface. For example, a color image of a brick wall may be applied to a polygon without an accurate representation of the geometry of the brick wall.

In order to reduce the memory occupation of the GPU, reduce the texture loading time and improve the rendering performance, the texture map is compressed. That is, texture encoding of embodiments of the present application may be understood as texture compression.

The first texture map may be understood as any texture map that needs to be compressed.

The embodiment of the application does not limit the specific manner of acquiring the first texture map.

In one possible implementation, the encoding device has a texture map generation function, such that the adaptation encoding device may generate the first texture map using a texture map generation tool based on user requirements.

In one possible implementation manner, the first texture map may be downloaded from a network end to the encoding device or uploaded to the encoding device by a user.

In this embodiment of the present application, after obtaining a first texture map to be encoded, an encoding device performs meshing on the first texture map to obtain a first mesh map.

The embodiment of the application does not limit the specific division mode of the encoding device for carrying out network division on the first texture map. Illustratively, the encoding apparatus includes a plurality of mesh division modes preset.

In one example, the encoding device may select a meshing scheme matching the first texture map from among a plurality of meshing schemes based on feature information of the first texture map to mesh the first texture map. For example, different meshing schemes are provided for different types of texture maps in the embodiments of the present application. Thus, the meshing mode corresponding to the first texture map can be determined from meshing modes corresponding to the texture maps of different types based on the type of the first texture map.

In another example, in meshing the first texture map, the user may select one of a plurality of meshing modes displayed by the encoding device. In this way, the encoding device determines the mesh division mode as the mesh division mode corresponding to the first texture map in response to the user selection operation of the mesh division mode.

In one example, the encoding device may also randomly select a meshing scheme as the meshing scheme corresponding to the first texture map.

Next, the encoding apparatus performs meshing on the first texture map using a meshing scheme corresponding to the first texture map. For example, the first texture map is uniformly divided into a plurality of meshes, such as a plurality of triangle meshes or quadrilateral meshes.

In some embodiments, the user interface of the GPU two-dimensional texture sampling is to initiate a command to the texture sampling unit in the shader, specify a normalized two-dimensional texture coordinate uv, and specify a texture object Tex, and randomly sample with the sampler. Illustratively, "random sampling" is performed by the following equation (1):

color＝Tex(uv) (1)

wherein Tex (u, v) represents a pixel value corresponding to the texture coordinate (u, v) in the texture map. In some embodiments, pixels in the texture map may also be referred to as Texels (Texels) as distinguished from pixels on the screen.

Based on this, the meshing of the first texture map in S101 includes the following steps:

S101-A, mapping the first texture map into a unit rectangular space to obtain a unitized first texture map;

S101-B, performing grid division on the unitized first texture map to obtain a first grid map.

That is, before meshing the first texture map, the embodiment of the present application first projects the first texture map into a unit rectangular space to obtain a unitized first texture map.

In the GPU map sampling pipeline, the texture coordinates of the input are normalized, i.e., the top left corner is the (0, 0) point and the bottom right corner is the (1, 1) point. Thus, in the embodiment of the present application, the first texture map is projected into a unit rectangular space, specifically, a unit rectangular space as shown in fig. 4.

In one example, the encoding device may project the first texture map into the unit rectangular space based on the following equation (2):

wherein Width is the Width of the first texture map, height is the height of the first texture map, (uv) _tex .x,uv _tex Y) is a texel coordinate in the first texture map, and (uv.x, uv.y) is a texture coordinate normalized by the texel coordinate.

For example, as shown in fig. 5, assuming that the first texture map is an image with a size of 2048×2048, the first texture map is projected into a unit rectangular space to obtain a unitized first texture map. At this time, the pixel point with the texture coordinates (1023.5 ) in the first texture map corresponding to the pixel point with the texture coordinates (0.5 ) in the unitized first texture map.

The encoding device maps the first texture map into the unit rectangular space based on the above steps, and performs mesh division on the unitized first texture map after obtaining the unitized first texture map. For example, the encoding device performs meshing on the unitized first texture map using a meshing scheme corresponding to the first texture map.

In one example, as shown in fig. 6A, the unit rectangle described above is composed of two triangles and four vertices. Each vertex has a texture coordinate, and corresponds to four corner points of the map. After the two triangles are rasterized and interpolated to a plurality of pixels, each pixel i has an independent first texture coordinate uv_i. Here uv_i would discretely cover the entire [0,1] interval. Since the rendered model is rectangular, the model can be subdivided to obtain a uniform mesh model.

For example, assuming that the mesh division manner corresponding to the first texture map is to uniformly divide the first texture map into a plurality of triangle meshes, as shown in fig. 6B, the encoding apparatus uniformly divides the unitized first texture map into a plurality of triangle meshes, to obtain the first mesh map.

It should be noted that, when the present application performs grid division on the unitized first texture map, uniform grid division may be performed, or non-uniform grid division may be performed, for example, some areas in the unitized first texture map are divided by using a first grid, and other areas are divided by using a second grid, where the sizes of the first grid and the second grid are inconsistent. In one example, to further improve the performance of texture compression, finer meshing is used for the high frequency signal region in the first texture map to ensure the decoding quality of the high frequency signal region. The lower frequency signal region in the first texture map is partitioned using a coarser grid to reduce the amount of data for texture compression.

The encoding device performs mesh division on the first texture map based on the above steps, and after obtaining the first mesh map, performs the following step S102.

S102, optimizing first texture coordinates of grid vertices to obtain second texture coordinates of the grid vertices for each grid vertex in the first grid graph.

Based on the steps, the encoding end performs grid division on the first texture map, and then corresponds to a texture coordinate uv for each vertex of each grid. The texture is described herein for convenience, wherein the vertices of the mesh are designated as mesh vertices, and the texture coordinates uv of the mesh vertices are designated as first texture coordinates. It should be noted that, the local spatial coordinates of each mesh vertex in the first mesh map and the first texture coordinates may be obtained by interpolation.

In this embodiment of the present application, in order to improve the effect of texture compression, for each grid vertex in the first grid graph, the first texture coordinate of the grid vertex is optimized, so as to obtain the second texture coordinate of the grid vertex.

In the embodiment of the application, the first texture coordinates of the grid vertices are optimized, and the specific mode of obtaining the second texture coordinates of the grid vertices is not limited.

In one possible implementation, a first texture coordinate of a mesh vertex is determined as a second texture coordinate of the mesh vertex.

In one possible implementation, the first texture coordinates of the grid vertices are optimized based on a texture optimization technique of metric tensor integration or a micro-texture optimization method, so as to obtain the second texture coordinates of the grid vertices.

In the embodiment of the present application, the optimization mode used when optimizing the first texture coordinates of the mesh vertex is a bijective transformation function. At this time, the step of optimizing the first texture coordinates of the mesh vertex in the above step S102 to obtain the second texture coordinates of the mesh vertex includes the following steps S102-A1 and S102-A2:

S102-A1, determining a first bijective transformation function from a plurality of preset bijective transformation functions;

S102-A2, optimizing a first texture coordinate of the grid vertex by adopting a first bijective transformation function to obtain a second texture coordinate of the grid vertex.

In this embodiment of the present application, the first bijection transformation function is a bidirectional transformation function, that is, the first texture coordinates of the grid vertices are optimized based on the first bijection transformation function, so as to obtain the second texture coordinates of the grid vertices. The first bijection transformation function can also be used, and the second texture coordinates of the grid vertices are subjected to the steganography to obtain first texture coordinates corresponding to the second texture coordinates.

In one example, for ease of description, embodiments of the present application note this first bijective transformation function as G (). In some embodiments, the first bijective transformation function G () may be any bijective transformation function, such as any one of a texture optimization function based on metric tensor integration and a microtexture optimization function. In some embodiments, the first bijective transformation function G () may include a combination of 2 or more bijective transformation functions, e.g., the first bijective transformation function G () may include a texture optimization function based on metric tensor integration and a microtexture optimization function.

It should be noted that, the bijective transformation function in the embodiment of the present application may be a discrete bijective transformation function, or may be a non-discrete bijective transformation function.

The embodiment of the application does not limit the specific way of determining the first bijective transformation function by the encoding device.

In one possible implementation, the encoding device randomly selects a bijective transformation function from a plurality of preset bijective transformation functions as the first bijective transformation function.

In one possible implementation manner, for an ith bijective transformation function in the preset multiple bijective transformation functions, the encoding device uses the ith bijective transformation function to optimize the first texture coordinates of each grid vertex in the first grid graph, so as to obtain the second texture coordinates of each grid vertex in the first grid graph. Resampling is carried out in the first texture mapping based on the second texture coordinates of the grid vertexes, and a second texture mapping corresponding to the ith bijective transformation function is obtained. And then, inputting the second texture map into a GPU for decoding to obtain a reconstructed first texture map, comparing the reconstructed first texture map with the initial first texture map, and determining the coding cost corresponding to the ith bijective transformation function. Based on the steps, the coding cost corresponding to each of the preset multiple bijective transformation functions can be determined, and then at least one bijective transformation function is selected from the preset multiple bijective transformation functions based on the coding cost and used as a first bijective transformation function. For example, a bijective transformation function with the minimum coding cost is selected from a plurality of preset bijective transformation functions and is used as a first bijective transformation function.

The encoding device determines a first bijective transformation function G () based on the above steps, and then optimizes the first texture coordinates of each grid vertex in the first grid graph by using the first bijective transformation function G () to obtain the second texture coordinates of each grid vertex.

For example, the encoding device optimizes the first texture coordinates based on the following formula (3), resulting in second texture coordinates:

uv′ _i ＝G(uv _i ) (3)

wherein uv _i For the first texture coordinates, uv ', of the ith mesh vertex in the first mesh map' _i For the second texture coordinates of the ith mesh vertex, G () is the first bijective transformation function.

In some embodiments, if the encoding apparatus first maps the first texture map into the unit rectangular space before meshing the first texture map, and further meshing the unitized first texture map, then in this step, the first texture coordinates of the mesh vertices refer to the first texture coordinates of the mesh vertices in the unitized first texture map, and the second texture coordinates of the mesh vertices refer to the second texture coordinates of the mesh vertices in the unitized first texture map. That is, when the first texture coordinate uv of the mesh vertex is in the [0,1] interval, it can be transformed into the warped space, which uniquely corresponds to a determined second texture coordinate uv' of the space.

After optimizing the first texture coordinates of each mesh vertex in the first mesh map to the second texture coordinates, the encoding apparatus performs the following step S103.

S103, resampling is carried out in the first texture mapping based on the second texture coordinates of each grid vertex, and the second texture mapping is obtained.

In the embodiment of the present application, in order to improve the compression quality of textures, when the first texture map is compressed, the first texture map is grid-divided, and the first texture coordinates of each grid vertex are optimized, so as to obtain the second texture coordinates of each grid vertex. Then, resampling is performed in the first texture map based on the second texture coordinates of each mesh vertex, resulting in a second texture map. Since the second texture map is obtained by resampling the first texture map based on the optimized second texture coordinates, the texture details of the second texture map are better than those of the first texture map. In this way, in the scene of reconstructing the texture map at the decoding end, the decoding device (for example, GPU) decodes the second texture map, and when generating the reconstructed first texture map, the reconstruction quality of the reconstructed first texture map can be improved. In a random sampling scenario, the effect of random sampling may be enhanced when a decoding device (e.g., GPU) samples a point in the second texture map. Therefore, according to the embodiment of the application, the first texture map is subjected to grid division, the first texture coordinates of each grid vertex are optimized, the second texture coordinates of each grid vertex are obtained, resampling is further performed in the first texture map based on the second texture coordinates of each grid vertex, and the second texture map is obtained, so that when the decoding device (for example, GPU) performs direct rendering or random sampling based on the second texture map, the performance cost of the decoding device can be reduced, decoding can be completed under the condition of no additional cost, and as the second texture map provides finer texture expansion, the texture signal quantity after rendering reconstruction greatly exceeds the texture signal quality before optimization, and the effect of compressing the texture map is further improved.

The specific process by which the encoding device resamples the first texture map to obtain the second texture map based on the second texture coordinates of each mesh vertex is described below.

As can be seen from the above description, each mesh vertex in the first mesh map includes 2 texture coordinates, one being a first texture coordinate (which may be understood as an original texture coordinate uv), and one being a second texture coordinate (i.e. a texture coordinate uv') obtained by optimizing the first texture coordinate. In one example, for an ith mesh vertex in the first mesh map, a pair of texture coordinates for the ith mesh vertex may be represented as (uv _i ，uv _i ') where uv _i Uv, first texture coordinates for the ith mesh vertex _i ' is the second texture coordinates of the ith mesh vertex.

In this embodiment of the present application, for each mesh vertex in the first mesh map, based on the second texture coordinate of each mesh vertex, a pixel value (the pixel value may be understood as a texel value here) corresponding to the second texture coordinate of each mesh vertex is searched for in the first texture map, and then, based on the second texture coordinate of each mesh vertex and the pixel value corresponding to the second texture coordinate of each mesh vertex, the second texture map is generated. Since the second texture map is derived based on the optimized second texture coordinates, the second texture map can be understood as an added texture map of the first texture map, or an optimized texture map, etc.

In some embodiments, since the step is known, the encoding apparatus first maps the first texture map into the unit rectangular space before meshing the first texture map, to obtain a unitized first texture map, and further meshing the unitized first texture map. In this way, when the encoding device resamples in the first texture map based on the second texture coordinates of each mesh vertex to obtain the second texture map, the encoding device searches, in the unitized first texture map, for example, the second texture coordinates of the mesh vertex are (a, b), for a pixel value c of a point with the first texture coordinates of (a, b), and then uses the pixel value c as a pixel value of a point with the texture coordinates of (a, b) in the second texture map. Based on this method, a second texture map may be obtained.

In some embodiments, the step S103 includes the steps of S103-A through S103-D as follows:

S103-A, generating a blank unit rectangular area;

S103-B, carrying out grid division on the unit rectangular area based on the topological structure of each grid vertex in the first grid graph and the second texture coordinates to obtain a second grid graph;

S103-C, searching an ith pixel value in the unitized first texture map based on a second texture coordinate of an ith grid vertex in the second grid map, wherein i is a positive integer;

S103-D, determining a pixel value corresponding to the ith grid vertex in the second grid map based on the ith pixel value to obtain a second texture map.

In this implementation manner, when the first texture map is gridded, the first texture map is mapped into the unit rectangular space, and gridding and texture coordinate optimization are performed on the unitized first texture map, so as to obtain a first gridding map and second texture coordinates of each grid vertex in the first gridding map. The resampling is performed in the unitized first texture map based on the second texture coordinates of each mesh vertex in the first mesh map, and the obtained second texture map is also a unit rectangle, for example, the unit rectangle of the second texture map with the size of 1×1. Based on this, in this implementation, when generating the second texture map, the encoding apparatus first generates a blank unit rectangular region as shown in fig. 4.

Next, the blank unit rectangular region shown in fig. 4 is gridded based on the topology of each grid vertex in the first gridding diagram and the second texture coordinates. Illustratively, fig. 7A is a first mesh map obtained by mesh-dividing the unitized first texture map. The encoding device performs the mesh division on the unit rectangular area shown in fig. 4 based on the second texture coordinate of each mesh vertex in the first mesh map in fig. 7A and the topology structure of each mesh vertex, specifically, for each mesh vertex in fig. 7A, for example, the ith mesh vertex, the second texture coordinate of the ith mesh vertex is taken as the texture coordinate (the texture coordinate may be understood as the first texture coordinate) of the ith mesh vertex in fig. 7B, and at the same time, performs the mesh division on the basis of the topology structure of the ith mesh vertex, that is, the second texture coordinate of each mesh vertex connected with the ith mesh vertex, and repeating the above steps, and may perform the mesh division on the blank unit rectangular area shown in fig. 4 based on the second texture coordinate and the topology structure of each mesh vertex in fig. 7A, to obtain the second mesh map shown in fig. 7B.

The encoding device performs step S103-C after obtaining the second mesh map based on the above steps, and searches, for each mesh vertex in the second mesh map, a pixel value corresponding to the second texture coordinate of each mesh vertex in the unitized first texture map based on the second texture coordinate of each mesh vertex, thereby obtaining the second texture map.

In this embodiment of the present application, the specific manner of searching for the pixel value corresponding to the second texture coordinate of each mesh vertex in the unitized first texture map by the encoding device is consistent, and for convenience of description, an i-th mesh vertex in the second mesh map is taken as an example.

Specifically, for the ith mesh vertex in the second mesh map, a pixel value corresponding to a point having the second texture coordinate of the ith mesh vertex as a texture coordinate, for example, the ith pixel value, is searched for in the unitized first texture map.

Illustratively, the ith pixel value may be found by the following equation (4):

color _i ＝Tex(uv′ _i ) (4)

wherein uv' _i The second texture coordinates of the ith grid vertex in the second grid map, tex is the unitized first texture map and color _i The pixel value is the ith pixel value corresponding to the second texture coordinate of the ith grid vertex in the unitized first texture map.

And further, determining a pixel value corresponding to the ith mesh vertex in the second mesh map based on the ith pixel value, for example, determining the ith pixel value as the pixel value corresponding to the ith mesh vertex in the second mesh map. Or, the ith pixel value is corrected, and the corrected ith pixel value is determined as the pixel value corresponding to the ith grid vertex in the second grid graph. And repeating the steps to determine the pixel value corresponding to each grid vertex in the second grid graph.

And further determining a second texture map based on the pixel values corresponding to each mesh vertex in the second mesh map.

In some embodiments, a texture map generated based on pixel values corresponding to each mesh vertex in the second mesh map is determined as the second texture map.

In some embodiments, the step S103-D includes the steps of S103-D1 and S103-D2 as follows:

S103-D1, determining the ith pixel value as a pixel value corresponding to the ith grid vertex in the second grid map, and obtaining a third texture map;

S103-D2, interpolation is carried out on the third texture mapping to obtain a second texture mapping.

In this implementation manner, the encoding device determines, for an ith mesh vertex in the second mesh map, an ith pixel value corresponding to the second texture coordinate of the ith mesh vertex in the unitized first texture map as the pixel value corresponding to the ith mesh vertex in the second mesh map. With reference to the method, a pixel value corresponding to each grid vertex in the second grid graph can be determined, and a third texture map is obtained, wherein the third texture map is sparse and contains fewer pixels, for example, the pixels corresponding to each grid vertex in the second grid graph are included. To further enhance the texture signal in the second texture map, the third texture map is interpolated to obtain the second texture map shown in FIG. 7C.

In the embodiment of the present application, the interpolation is performed on the third texture map, and a specific interpolation mode for obtaining the second texture map is not limited.

In one possible implementation, the step S103-D2 includes the following steps S103-D21 to S103-D24:

S103-D21, optimizing a first texture coordinate of a jth interpolation point in the third texture map to obtain a second texture coordinate of the jth interpolation point, wherein j is a positive integer;

S103-D22, searching a jth pixel value in the unitized first texture map based on the second texture coordinate of the jth interpolation point;

S103-D23, determining a pixel value corresponding to a j-th interpolation point in the third texture map based on the j-th pixel value, and obtaining the interpolated third texture map;

S103-D24, obtaining a second texture map based on the interpolated third texture map.

In this manner, when the third texture map is interpolated to obtain the second texture map, first the first texture coordinates of each point in the third texture map are interpolated to obtain the first texture coordinates of each interpolation point, and then the first texture coordinates of each interpolation point are optimized to obtain the second texture coordinates of each interpolation point. Thus, the pixel value corresponding to the second texture coordinate of each interpolation point can be searched in the unitized first texture map. In the embodiment of the present application, the operation procedure for each interpolation point in the third texture map is substantially identical, and for convenience of description, the jth interpolation point in the third texture map is described herein as an example.

Specifically, the encoding device first optimizes the first texture coordinate of the jth interpolation point to obtain the second texture coordinate of the jth interpolation point.

In the embodiment of the present application, the specific manner of obtaining the second texture coordinate of the jth interpolation point is not limited by optimizing the first texture coordinate of the jth interpolation point.

In one possible implementation, the first texture coordinate of the jth interpolation point is determined as the second texture coordinate of the jth interpolation point.

In one possible implementation manner, the first texture coordinate of the jth interpolation point is optimized based on a texture optimization technology of measuring tensor integration or a micro-texture optimization method, so as to obtain the second texture coordinate of the jth interpolation point.

In one possible implementation, the encoding device optimizes the first texture coordinate of the jth interpolation point in the same optimization manner as the first texture coordinate of the optimized mesh vertex.

In one possible implementation manner, the encoding device determines a first bijective transformation function from a plurality of preset bijective transformation functions, and optimizes the first texture coordinates of the jth interpolation point by adopting the first bijective transformation function to obtain the second texture coordinates of the jth interpolation point.

In one possible implementation manner, for an ith bijective transformation function in the preset multiple bijective transformation functions, the encoding device uses the ith bijective transformation function to optimize the first texture coordinates of each interpolation point in the third texture map, so as to obtain the second texture coordinates of each interpolation point in the third texture map. Resampling is carried out in the first texture mapping based on the second texture coordinates of each interpolation point, and a second texture mapping corresponding to the ith bijective transformation function is obtained. And then, inputting the second texture map into a GPU for decoding to obtain a reconstructed first texture map, comparing the reconstructed first texture map with the initial first texture map, and determining the coding cost corresponding to the ith bijective transformation function. Based on the steps, the coding cost corresponding to each of the preset multiple bijective transformation functions can be determined, and then at least one bijective transformation function is selected from the preset multiple bijective transformation functions based on the coding cost and used as a first bijective transformation function. For example, a bijective transformation function with the minimum coding cost is selected from a plurality of preset bijective transformation functions and is used as a first bijective transformation function.

The encoding device optimizes the first texture coordinate of the j-th interpolation point by using the first bijective transformation function G () after determining the first bijective transformation function G () based on the steps, and obtains the second texture coordinate of the j-th interpolation point.

Next, the encoding apparatus searches for the jth pixel value in the unitized first texture map based on the second texture coordinates of the jth interpolation point. For example, assuming that the second texture coordinate of the jth interpolation point is (b, c), a pixel value d of the pixel point whose texture coordinate (i.e., the first texture coordinate) is (b, c) is found in the unitized first texture map, and the pixel value d is the jth pixel value.

Thus, the pixel value corresponding to the jth interpolation point in the third texture map may be determined based on the jth pixel value, e.g., the jth pixel value may be determined as the pixel value corresponding to the jth interpolation point in the third texture map. Or the jth pixel value is corrected, and the corrected jth pixel value is determined as the pixel value corresponding to the jth interpolation point in the third texture map. And executing the steps for each interpolation point in the third texture map, and determining a pixel value corresponding to each interpolation point in the third texture map, thereby obtaining the interpolated third texture map.

Finally, a second texture map is determined based on the interpolated third texture map.

For example, the interpolated third texture map is determined as the second texture map.

For another example, the interpolated third texture map is subjected to a preset process, such as a process of color correction, redundant information removal, and the like, to obtain the second texture map.

The encoding apparatus performs mesh division and texture coordinate optimization on the first texture map based on the above steps to obtain a second texture map, and then performs the following step S104.

S104, obtaining a compressed file of the first texture map based on the second texture map.

Wherein the compressed file includes a second texture map.

In some embodiments, the compressed file further includes at least one of: the first texture coordinates and the second texture coordinates of each grid vertex in the first grid graph, the topological structure of each grid vertex, and the indication information of the first bijective transformation function.

In the embodiment of the application, the two-dimensional texture compression coding mainly comprises a second texture mapping and discrete bijective texture transformation information. The second texture map includes a general two-dimensional texture that is completely recognizable by the GPU, such as ASTC texture or ETC. The discrete bijective texture transformation information is mainly a control point grid of textures, which is a two-dimensional grid structure capable of being defined arbitrarily, such as evenly dividing grids, quadtree grids and the like. Such a mesh needs to store information before and after transformation of each pair of texture control points (i.e. mesh vertices), i.e. a first texture coordinate (uv) and a second texture coordinate (uv'), the data structure is a pair of UNorm16x2, and the topology of the mesh vertices needs to be specified, e.g. triangle model mesh is organized by triangle index, etc.

In some embodiments, the encoding device may encode the second texture map obtained above, and the first texture coordinate (uv) and the second texture coordinate (uv') of each mesh vertex in the first mesh map, into the GPU.

In the texture coding method provided by the embodiment of the application, during texture coding, a coding device firstly acquires a first texture map to be coded, and performs grid division on the first texture map to obtain a first grid map; optimizing a first texture coordinate of each grid vertex in the first grid graph to obtain a second texture coordinate of the grid vertex; resampling the first texture map based on the second texture coordinates of each grid vertex to obtain a second texture map; and finally, based on the second texture map, obtaining a compressed file of the first texture map, wherein the compressed file comprises the second texture map. That is, in the embodiment of the present application, after grid subdivision is performed on the first texture map, adjustment and optimization are performed on the texture coordinates, and based on the adjusted and optimized texture coordinates and the first texture map, a second texture map with finer texture is generated, so as to obtain a new two-dimensional texture compression format of the GPU. Thus, when the new GPU two-dimensional texture compression format is transmitted to the GPU, the GPU can select two modes of direct rendering or random sampling to finish texture sampling according to specific rendering requirements. For example, the GPU may directly render directly based on the second texture map to obtain the first texture map or directly randomly sample any pixel point, so that the performance cost of the GPU is smaller, decoding is completed without additional cost, and since the second texture map includes finer texture expansion, when performing texture reconstruction or random sampling based on the second texture map, the reconstruction or random sampling effect of the texture can be improved, and further the encoding and decoding effects of the texture map are improved.

The texture coding method according to the embodiments of the present application is described above. The following describes a texture decoding method provided in the embodiments of the present application, taking a decoding end as an example.

Fig. 8 is a flowchart of a texture decoding method according to an embodiment of the present application. The execution body of the embodiments of the present application may be a device for a specific texture decoding function, for example, a texture decoding device. In some embodiments, the texture decoding apparatus may be the decoding device of fig. 1. For convenience of description, the embodiment of the present application will be described taking the execution body as a decoding device as an example.

As shown in fig. 8, the texture decoding method in the embodiment of the present application includes the following steps:

s201, obtaining a compressed file of the first texture map.

The compressed file includes a second texture map, the second texture map is obtained by resampling the first texture map based on second texture coordinates of each network vertex in the first mesh map, the first mesh map is obtained by mesh division of the first texture map, and the second texture coordinates of the mesh vertices are obtained by optimization of the first texture coordinates of the mesh vertices.

The above-mentioned texture coding embodiment describes the compression process of the first texture map, and the specific process refers to the description of the above-mentioned embodiment, which is not repeated here.

In some embodiments, the decoding device of embodiments of the present application may be a GPU.

S202, reconstructing the first texture map or determining a pixel value corresponding to the target texture coordinate in the first texture map based on the second texture map.

In this embodiment of the present application, the decoding device obtains a compressed file of the first texture map, decodes a second texture map from the compressed file, and further performs texture decoding based on the second texture map.

In this embodiment of the present application, the texture decoding mode includes at least two modes:

the first is a direct rendering mode. In fact, many times the map is fully accessed, for example, when rendering a rectangle that fits the texture in its entirety, the texture is accessed from left to right and top to bottom. Under this requirement, it is defined herein as a direct rendering mode. In the direct rendering mode, the decoding device may reconstruct the first texture map directly based on the second texture map.

In the direct rendering mode, the initial need of the user is to render a flat rectangular surface, such as bulletin board technology, impinger, flat ground or wall surface, which are commonly used in the industry. At this time, the unit rectangle is usually input to the GPU, after the texture coding mentioned in this embodiment is applied, the GPU will take out the first grid pattern to replace the unit rectangle model, and directly let the GPU sample the second texture map to complete rendering.

The embodiment of the application does not limit the specific way of reconstructing the first texture map by the decoding device based on the second texture map.

In some embodiments, the second texture map may be determined directly as the reconstructed first texture map.

In some embodiments, if the compressed file further includes the first texture coordinate and the second texture coordinate of each mesh vertex in the first mesh map, the decoding apparatus obtains the reconstructed first texture map through the following steps S202-A1 to S202-A4:

S202-A1, generating a blank unit rectangular area;

S202-A2, determining the topological structure of each grid vertex in the first grid graph, and carrying out grid division on a unit rectangular area based on the topological structure of each grid vertex in the first grid graph and the first texture coordinates to obtain a second grid;

S202-A3, searching an ith pixel value corresponding to an ith grid vertex in the second texture mapping based on a second texture coordinate of the ith grid vertex in the second grid mapping, wherein i is a positive integer;

S202-A4, determining a pixel value corresponding to an ith network vertex in the second grid graph based on the ith pixel value, and obtaining the reconstructed first texture map.

In this embodiment, since the encoding apparatus first maps the first texture map into the unit rectangular space before meshing the first texture map, meshing and optimizing the texture coordinates of the unitized first texture map, a first mesh map and a second texture coordinate of each mesh vertex in the first mesh map are obtained. The resampling is performed in the unitized first texture map based on the second texture coordinates of each mesh vertex in the first mesh map, and the obtained second texture map is also a unit rectangle, for example, the unit rectangle of the second texture map with the size of 1×1. Based on this, in this implementation, the decoding device first generates a blank unit rectangular region when reconstructing the first texture map based on the second texture map.

Meanwhile, the decoding device also needs to determine the topology structure of each grid vertex in the first grid graph. The embodiment of the application does not limit the specific way for the decoding device to determine the topological structure of each grid vertex in the first grid graph.

In one possible implementation manner, the decoding device obtains a dividing manner of the encoding device for performing mesh division on the first texture map, and further determines a topology structure of each mesh vertex in the first mesh map based on the mesh dividing manner.

In one possible implementation, the encoding device writes the topology of each mesh vertex in the first mesh map into a compressed file. Thus, the decoding device can obtain the topological relation of each grid vertex in the first grid graph by decoding the compressed file.

In this way, the decoding apparatus may mesh the blank unit rectangular region based on the topology of each mesh vertex in the first mesh map and the first texture coordinates. Illustratively, the decoding apparatus performs mesh division on the blank unit rectangular area shown in fig. 4 based on the topology structure and the first texture coordinates of each mesh vertex in the first mesh map, to obtain a third mesh map shown in fig. 9. In some embodiments, the third grid pattern shown in fig. 9 is identical to the grid structure of the first grid pattern shown in fig. 7A.

For each grid vertex in the third grid graph, based on the second texture coordinates of each grid vertex, searching a pixel value corresponding to the second texture coordinates of each grid vertex in the second texture map. Taking an ith grid vertex in the third grid chart as an example, obtaining a second texture coordinate of the ith grid vertex from the compressed file, and then searching a pixel point with the texture coordinate being the second texture coordinate of the ith grid vertex in the second texture map included in the compressed file, and recording a pixel value of the pixel point as an ith pixel value.

Illustratively, the ith pixel value may be found by the following equation (5):

color _i ＝Tex′(uv′ _i ) (5)

wherein uv' _i The second texture coordinate of the ith grid vertex in the third grid map, tex' is the second texture map and color _i And the pixel value is the ith pixel value corresponding to the second texture coordinate of which the texture coordinate is the ith grid vertex in the second texture map.

Next, the decoding apparatus determines a pixel value corresponding to the ith mesh vertex in the third mesh map based on the ith pixel value.

The embodiment of the application does not limit the specific manner of determining the pixel value corresponding to the ith grid vertex in the third grid graph based on the ith pixel value.

For example, the ith pixel value is determined as the pixel value corresponding to the ith mesh vertex in the third mesh map.

For another example, the ith pixel value is corrected, and the corrected ith pixel value is determined as the pixel value corresponding to the ith mesh vertex in the third mesh map.

And repeating the steps to determine the pixel value corresponding to each grid vertex in the third grid graph. And finally, determining the reconstructed first texture map based on the pixel value corresponding to each grid vertex in the third grid map.

In some embodiments, a texture map generated based on pixel values corresponding to each mesh vertex in the third mesh map is determined as the reconstructed first texture map.

In some embodiments, the step S202-A4 includes the steps of S202-A41 and S202-A42 as follows:

S202-A41, determining the ith pixel value as a pixel value corresponding to the ith grid vertex in the third grid map, and obtaining a fourth texture map;

S202-A42, interpolation is carried out on the fourth texture map, and the reconstructed first texture map is obtained.

In this implementation manner, the decoding device determines, for an ith mesh vertex in the third mesh map, an ith pixel value corresponding to the second texture coordinate of the ith mesh vertex in the second texture map as the pixel value corresponding to the ith mesh vertex in the third mesh map. With reference to the method, a pixel value corresponding to each grid vertex in the third grid graph can be determined, and a fourth texture map is obtained, wherein the fourth texture map is sparse and contains fewer pixels, for example, the pixels corresponding to each grid vertex in the third grid graph are included. In order to further enhance the texture signal quantity in the reconstructed first texture map, interpolation is performed on the fourth texture map, so as to obtain the reconstructed first texture map.

In the embodiment of the present application, interpolation is performed on the fourth texture map, and a specific interpolation mode of the reconstructed first texture map is not limited.

In one possible implementation, the step S202-a42 includes the following steps S202-a421 to S202-a 424:

S202-A421, optimizing a first texture coordinate of a j-th interpolation point in the fourth texture map to obtain a second texture coordinate of the j-th interpolation point, wherein j is a positive integer;

S202-A422, searching a j pixel value in the second texture map based on the second texture coordinate of the j interpolation point;

S202-A423, determining a pixel value corresponding to a j interpolation point in the fourth texture map based on the j pixel value, and obtaining the fourth texture map after interpolation;

S202-A424, obtaining the reconstructed first texture map based on the interpolated fourth texture map.

In this manner, when the fourth texture map is interpolated to obtain the reconstructed first texture map, first the first texture coordinates of each point in the fourth texture map are interpolated to obtain the first texture coordinates of each interpolation point, and then the first texture coordinates of each interpolation point are optimized to obtain the second texture coordinates of each interpolation point. Thus, the pixel value corresponding to the second texture coordinate of each interpolation point can be searched in the second texture map. In the embodiment of the present application, the operation procedure for each interpolation point in the fourth texture map is substantially identical, and for convenience of description, the jth interpolation point in the fourth texture map is described herein as an example.

Specifically, the decoding device optimizes the first texture coordinate of the jth interpolation point to obtain the second texture coordinate of the jth interpolation point.

In one possible implementation, the decoding device optimizes the first texture coordinate of the jth interpolation point in the same optimization manner as the first texture coordinate of the optimized mesh vertex.

In one possible implementation, the decoding device determines a first bijective transformation function that is one or more of a preset plurality of bijective transformation functions. And then, optimizing the first texture coordinate of the jth interpolation point by adopting a first bijection transformation function to obtain the second texture coordinate of the jth interpolation point.

In one possible implementation, the decoding device randomly selects a bijective transformation function from a plurality of preset bijective transformation functions as the first bijective transformation function.

In one possible implementation, the compressed file includes indication information of the first bijective transformation function. The decoding device obtains the indication information of the first bijective transformation function by decoding the compressed file, and further determines the first bijective transformation function based on the indication information of the first bijective transformation function. For example, the indication information of the first bijective transformation function is that the first bijective transformation function is indexed in a preset plurality of bijective transformation functions, so that the first bijective transformation function can be determined from the preset plurality of bijective transformation functions based on the index.

And the decoding equipment optimizes the first texture coordinate of the j interpolation point by using the first bijective transformation function G () after determining the first bijective transformation function G () based on the steps, so as to obtain the second texture coordinate of the j interpolation point.

Next, the decoding apparatus searches for a j-th pixel value in the second texture map based on the second texture coordinates of the j-th interpolation point. For example, assuming that the second texture coordinate of the jth interpolation point is (b, c), a pixel value d of the pixel point with the texture coordinate of (b, c) is found in the second texture map, and the pixel value d is the jth pixel value.

The decoding device determines, based on the above steps, a pixel value corresponding to a jth interpolation point in the fourth texture map, based on the jth pixel value, after determining the jth pixel value corresponding to the jth interpolation point in the second texture map. For example, the j-th pixel value is determined as the pixel value corresponding to the j-th interpolation point in the fourth texture map. Or the jth pixel value is corrected, and the corrected jth pixel value is determined as the pixel value corresponding to the jth interpolation point in the fourth texture map. And executing the steps for each interpolation point in the fourth texture map, and determining a pixel value corresponding to each interpolation point in the fourth texture map, thereby obtaining the fourth texture map after interpolation.

Finally, a reconstructed first texture map as shown in fig. 10 is obtained based on the interpolated fourth texture map.

For example, the interpolated fourth texture map is determined as the reconstructed first texture map.

For another example, the interpolated fourth texture map is subjected to a preset process, such as a process of color correction, redundant information removal, and the like, to obtain the reconstructed first texture map.

As can be seen from the above, in the direct rendering mode, only the first texture coordinates and the second texture coordinates of each mesh vertex in the second texture map Tex' and the first mesh map need to be given, so that the first texture map can be sampled and reconstructed in the decoding device (for example, GPU).

Illustratively, generating a reconstructed first texture map based on the second texture map and the first and second texture coordinates of each mesh vertex in the first mesh map may be represented as formula (6) below

Tex(uv)＝Tex′(uv′) (6)

Wherein Tex (uv) is the reconstructed first texture map and Tex '(uv') is the second texture map.

In the direct rendering mode, the selection and calculation process of the first bijective transformation function (namely G function) is irrelevant to the GPU, and any bijective function can be supported.

In the direct rendering mode, the decoding device can adopt smaller decoding cost, decoding can be completed under the condition of no additional cost, and when the second texture map comprises finer texture unfolding information, the texture signal quantity of the first texture map after rendering reconstruction can greatly exceed the original first texture map, so that the performance of texture compression is improved, and the rendering effect of the texture map is improved.

The texture rendering method in the embodiment of the application includes, in addition to the above-mentioned direct rendering mode, the following second mode:

the second is a random sampling pattern. The random sampling pattern refers to sampling the texture color (i.e., pixel value) of a particular point in an image based on any one given texture coordinate point. For example, the shader may need to sample the texture color of a specific point in the image according to a specified uv value, where uv may be programmatically generated or specified by a user. At this time, the texture decoding method of the embodiment of the present application includes: and determining pixel values corresponding to the target texture coordinates in the first texture map based on the second texture map.

The specific manner of determining the pixel value corresponding to the target texture coordinate in the first texture map based on the second texture map by the decoding device is not limited in the embodiments of the present application.

In some embodiments, the pixel value corresponding to the target texture coordinate in the second texture map is determined as the pixel value corresponding to the target texture coordinate in the first texture map.

In some embodiments, since the second texture map is generated based on the second texture coordinates of each mesh vertex in the first texture map. Therefore, the target texture coordinates first need to be converted before the pixel values corresponding to the target texture coordinates are determined. At this time, the step of determining the pixel value corresponding to the target texture coordinate in the first texture map based on the second texture map in the above step S202 includes the following steps S202-B1 to S202-B3:

S202-B1, determining a second texture coordinate corresponding to the target texture coordinate;

S202-B2, based on a second texture coordinate corresponding to the target texture coordinate, searching a first pixel value in the second texture map;

S202-B3, determining a pixel value corresponding to the target texture position based on the first pixel value.

In this implementation, the decoding device first determines the second texture coordinates to which the target texture coordinates correspond before determining the pixel values to which the target texture coordinates correspond.

In this embodiment of the present application, the determining the second texture coordinate mode corresponding to the target texture coordinate at least includes the following several modes:

in mode 1, if the compressed file includes the first texture coordinates and the second texture coordinates of each mesh vertex in the first mesh map. At this time, if the target texture coordinate is the first texture coordinate of the first mesh vertex in the first mesh map, determining the second texture coordinate of the first mesh vertex as the second texture coordinate corresponding to the target texture coordinate.

And 2, if the target texture coordinate is not the first texture coordinate of the grid vertex in the first grid graph, determining a first bijection transformation function, and optimizing the target texture coordinate based on the first bijection transformation function to obtain a second texture coordinate corresponding to the target texture coordinate.

The embodiment of the application does not limit the specific way of determining the first bijective transformation function by the decoding device.

And the decoding equipment optimizes the target texture coordinates by using the first bijective transformation function G () after determining the first bijective transformation function G () based on the steps, and obtains second texture coordinates corresponding to the target texture coordinates.

And 3, if the target texture coordinate is not the first texture coordinate of the grid vertex in the first grid graph, determining a first bijection transformation function, determining a corresponding relation table between the first texture coordinate and the second texture coordinate based on the first bijection transformation function, and searching the second texture coordinate corresponding to the target texture coordinate in the corresponding relation table.

In some embodiments, the correspondence table, also referred to as a lookup table of G functions, includes second texture coordinates corresponding to different first texture coordinates. That is, for the random sampling mode, the model does not have rectangular characteristics, the number of vertexes and the number of uv_i' have no corresponding relation, and the original uv of the model cannot be directly replaced. It is therefore necessary to generate a map tex_g in the GPU using a look-up table of G functions. In one example, the generation of the map tex_g is as shown in equation (7):

Tex _G (uv)＝G(uv) (7)

this process may be done directly by rendering, i.e., interpolating uv 'of the mesh vertices in a "direct rendering" manner to obtain uv' for each pixel. And finally, using Tex_G as a static map, and executing rendering shown in a formula (8):

color _pixel′ ＝Tex′(G(uv _pixel )) (8)

Wherein, color _pixel’ Is the rendered map.

After determining the second texture coordinate corresponding to the target texture coordinate, the decoding device searches the first pixel value in the second texture map based on the second texture coordinate corresponding to the target texture coordinate. Specifically, a pixel value corresponding to a pixel point with a texture coordinate being the second texture coordinate in the second texture map is determined as a first pixel value.

Then, a pixel value corresponding to the target texture coordinate is determined based on the first pixel value.

In one example, the first pixel value is determined as a pixel value corresponding to the target texture coordinates.

In one example, the first pixel value is modified, and the modified first pixel value is determined as the pixel value corresponding to the target texture coordinate.

In some embodiments, in the random sampling mode, the target texture coordinates are converted to second texture coordinates and the second texture map is sampled based on the second texture coordinates, at which time the second texture map needs to be returned to the normal map. When a random sampling mode is started, the GPU creates a two-dimensional texture in a temporary storage space, a mapping from uv to Tex' (G (uv)) is generated by using a direct rendering mode, and then the two-dimensional texture can be used as an actual mapping when random sampling is performed, and uv texture coordinates are used for sampling, so that rendering is completed.

The invention provides a GPU two-dimensional texture compression coding, in the coding, a program can finely divide a two-dimensional texture in a unit rectangular space in a plurality of modes and adjust the expansion of the texture, and after the adjustment, the distorted texture and a discrete bijection function of the finely divided grid are coded into a new GPU two-dimensional texture compression format. After the format is transmitted to the GPU, two modes of direct rendering or random sampling can be selected to finish texture sampling according to specific rendering requirements. Wherein the performance overhead of the direct rendering mode is smaller, decoding can be completed without additional overhead, and when the pre-calculation process provides finer texture expansion, the texture signal quantity after rendering reconstruction of the method greatly exceeds the mode before optimization.

In the texture decoding method provided by the embodiment of the application, during texture decoding, a decoding device obtains a compressed file of a first texture map, wherein the compressed file comprises a second texture map, the second texture map is obtained by resampling the first texture map based on second texture coordinates of each network vertex in a first grid map, the first grid map is obtained by grid division of the first texture map, and the second texture coordinates of the grid vertices are obtained by optimizing the first texture coordinates of the grid vertices. And reconstructing the first texture map based on the second texture map or determining a pixel value corresponding to the target texture coordinate in the first texture map. That is, in the embodiment of the present application, after grid subdivision is performed on the first texture map, adjustment and optimization are performed on the texture coordinates, and based on the adjusted and optimized texture coordinates and the first texture map, a second texture map with finer texture is generated, so as to obtain a new two-dimensional texture compression format of the GPU. Thus, when the new GPU two-dimensional texture compression format is transmitted to the GPU, the GPU can select two modes of direct rendering or random sampling to finish texture sampling according to specific rendering requirements. For example, the GPU may directly render directly based on the second texture map to obtain the first texture map or directly randomly sample any pixel point, so that the performance cost of the GPU is smaller, decoding is completed without additional cost, and since the second texture map includes finer texture expansion, when performing texture reconstruction or random sampling based on the second texture map, the reconstruction or random sampling effect of the texture can be improved, and further the decoding effect of the texture map is improved.

In order to further explain the technical effects of the embodiments of the present application, the parking method of the embodiments of the present application is tested in the test data.

The texture coding method and the existing downsampling coding method provided by the sample application embodiment compress a plurality of textures with 2048 resolution of initial signals to 1024 and 512 hours, and the compression effect is shown in table 1:

TABLE 1

In table 1, parameters 32x32 or 64x64 before texture compression encoding refer to the size of the grid in the first grid map, and the map a, the map B, and the map C can be understood as the first texture map to be encoded in the above embodiment. The value 48.08 is the peak signal to noise ratio (db).

As can be seen from the peak signal-to-noise ratio shown in table 1, the texture coding method provided in the embodiment of the present application can obtain a compression quality much higher than that of the downsampling compression method on the premise of increasing a small amount of storage space. The second texture map in the texture coding method provided by the embodiment of the application can be further combined with the existing GPU image compression technologies such as ASTC, ETC and the like, so that a higher compression rate is achieved.

According to the texture coding and decoding method, the application platform can be a GPU driving layer, for example, a mobile terminal platform sensitive to the image bandwidth can achieve higher texture definition on the premise of unchanged bandwidth, or the bandwidth is reduced under the condition of unchanged texture definition, and the comprehensive rendering performance is improved.

In some embodiments, the texture codec method of the embodiments of the present application may also be applied in non-graphics rendering fields, such as general purpose computing methods.

The texture codec method embodiments of the present application are described above in detail with reference to fig. 3 to 10, and the device embodiments of the present application are described below in detail with reference to fig. 11 to 12.

Fig. 11 is a schematic block diagram of a texture coding apparatus according to an embodiment of the present application. The apparatus 10 may be applied to an encoding device.

As shown in fig. 11, the texture coding apparatus 10 includes:

a meshing unit 11, configured to obtain a first texture map to be encoded, and mesh-divide the first texture map to obtain a first mesh map;

an optimizing unit 12, configured to optimize, for each mesh vertex in the first mesh map, a first texture coordinate of the mesh vertex, to obtain a second texture coordinate of the mesh vertex;

a resampling unit 13, configured to resample in the first texture map based on the second texture coordinates of each mesh vertex, to obtain a second texture map;

the encoding unit 14 is configured to obtain a compressed file of the first texture map based on the second texture map, where the compressed file includes the second texture map.

In some embodiments, the meshing unit 11 is specifically configured to map the first texture map into a unit rectangular space, so as to obtain a unitized first texture map; and carrying out grid division on the unitized first texture map to obtain the first grid map.

In some embodiments, the resampling unit 13 is specifically configured to generate a blank unit rectangular area; based on the topological structure of each grid vertex in the first grid map and the second texture coordinates, carrying out grid division on the unit rectangular area to obtain a second grid map; searching an ith pixel value in the unitized first texture map based on a second texture coordinate of an ith grid vertex in the second grid map, wherein i is a positive integer; and determining a pixel value corresponding to the ith grid vertex in the second grid map based on the ith pixel value to obtain the second texture map.

In some embodiments, the resampling unit 13 is specifically configured to determine the ith pixel value as a pixel value corresponding to the ith mesh vertex in the second mesh map, so as to obtain a third texture map; and interpolating the third texture map to obtain the second texture map.

In some embodiments, the resampling unit 13 is specifically configured to optimize, for a jth interpolation point in the third texture map, a first texture coordinate of the jth interpolation point to obtain a second texture coordinate of the jth interpolation point, where j is a positive integer; searching a j pixel value in the unitized first texture map based on the second texture coordinate of the j interpolation point; determining a pixel value corresponding to the j interpolation point in the third texture map based on the j pixel value to obtain a third texture map after interpolation; and obtaining the second texture map based on the interpolated third texture map.

In some embodiments, the resampling unit 13 is specifically configured to determine a first bijective transformation function from a preset plurality of bijective transformation functions; and optimizing the first texture coordinates by adopting the first bijective transformation function to obtain the second texture coordinates.

It should be understood that apparatus embodiments and method embodiments may correspond with each other and that similar descriptions may refer to the method embodiments. To avoid repetition, no further description is provided here. Specifically, the apparatus shown in fig. 11 may perform the above-described embodiment of the texture coding method, and the foregoing and other operations and/or functions of each module in the apparatus are respectively for implementing the above-described embodiment of the method, which is not repeated herein for brevity.

Fig. 12 is a schematic block diagram of a texture decoding apparatus provided in an embodiment of the present application. The apparatus 20 may be applied to a decoding device.

As shown in fig. 12, the texture decoding apparatus 20 includes:

an obtaining unit 21, configured to obtain a compressed file of a first texture map, where the compressed file includes a second texture map, the second texture map is obtained by resampling the first texture map based on a second texture coordinate of each network vertex in a first mesh map, the first mesh map is obtained by meshing the first texture map, and the second texture coordinate of the mesh vertex is obtained by optimizing a first texture coordinate of the mesh vertex;

the decoding unit 22 is configured to reconstruct the first texture map based on the second texture map or determine a pixel value corresponding to the target texture coordinate in the first texture map.

In some embodiments, if the compressed file further includes the first texture coordinate and the second texture coordinate of each mesh vertex in the first mesh map, the decoding unit 22 is specifically configured to generate a blank unit rectangular area; determining the topological structure of each grid vertex in the first grid graph, and carrying out grid division on the unit rectangular area based on the topological structure of each grid vertex in the first grid graph and the first texture coordinates to obtain a third grid graph; for an ith grid vertex in the third grid map, searching an ith pixel value corresponding to the ith grid vertex in the second texture map based on a second texture coordinate of the ith grid vertex, wherein i is a positive integer; and determining a pixel value corresponding to the ith network vertex in the third grid graph based on the ith pixel value to obtain a reconstructed first texture map.

In some embodiments, if the compressed file includes the topology of each mesh vertex in the first mesh map, the decoding unit 22 is specifically configured to decode the compressed file to obtain the topology of each mesh vertex in the first mesh map.

In some embodiments, the decoding unit 22 is specifically configured to determine the ith pixel value as a pixel value corresponding to the ith mesh vertex in the third mesh map, so as to obtain a fourth texture map; and interpolating the fourth texture map to obtain the reconstructed first texture map.

In some embodiments, the decoding unit 22 is specifically configured to determine, for a jth interpolation point in the fourth texture map, a second texture coordinate corresponding to a first texture coordinate of the jth interpolation point, where j is a positive integer; searching a j pixel value in the second texture map based on the second texture coordinate of the j interpolation point; determining a pixel value corresponding to a j interpolation point in the fourth texture map based on the j pixel value to obtain an interpolated fourth texture map; and obtaining the reconstructed first texture map based on the interpolated fourth texture map.

In some embodiments, the decoding unit 22 is specifically configured to determine a first bijective transformation function; and optimizing the first texture coordinate of the jth interpolation point based on the first bijection transformation function to obtain the second texture coordinate of the jth interpolation point.

In some embodiments, the decoding unit 22 is specifically configured to determine a second texture coordinate corresponding to the target texture coordinate; a first pixel value searched in the second texture map based on a second texture coordinate corresponding to the target texture coordinate; and determining a pixel value corresponding to the target texture coordinate based on the first pixel value.

In some embodiments, if the compressed file further includes a first texture coordinate and a second texture coordinate of each mesh vertex in the first mesh map, the decoding unit 22 is specifically configured to determine, if the target texture coordinate is the first texture coordinate of the first mesh vertex in the first mesh map, the second texture coordinate of the first mesh vertex as the second texture coordinate corresponding to the target texture coordinate; if the target texture coordinate is not the first texture coordinate of the grid vertex in the first grid graph, determining a first bijection transformation function, optimizing the target texture coordinate based on the first bijection transformation function to obtain a second texture coordinate corresponding to the target texture coordinate, or determining a corresponding relation table between the first texture coordinate and the second texture coordinate based on the first bijection transformation function, and searching the second texture coordinate corresponding to the target texture coordinate in the corresponding relation table.

In some embodiments, if the compressed file includes the indication information of the first bijective transformation function, the decoding unit 22 is specifically configured to decode the compressed file to obtain the indication information of the first bijective transformation function; and determining the first bijective transformation function based on the indication information.

It should be understood that apparatus embodiments and method embodiments may correspond with each other and that similar descriptions may refer to the method embodiments. To avoid repetition, no further description is provided here. Specifically, the apparatus shown in fig. 12 may perform the above-described embodiment of the texture decoding method, and the foregoing and other operations and/or functions of each module in the apparatus are respectively for implementing the above-described embodiment of the method, which is not repeated herein for brevity.

The apparatus of the embodiments of the present application are described above in terms of functional modules in conjunction with the accompanying drawings. It should be understood that the functional module may be implemented in hardware, or may be implemented by instructions in software, or may be implemented by a combination of hardware and software modules. Specifically, each step of the method embodiments in the embodiments of the present application may be implemented by an integrated logic circuit of hardware in a processor and/or an instruction in software form, and the steps of the method disclosed in connection with the embodiments of the present application may be directly implemented as a hardware decoding processor or implemented by a combination of hardware and software modules in the decoding processor. Alternatively, the software modules may be located in a well-established storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, and the like. The storage medium is located in a memory, and the processor reads information in the memory, and in combination with hardware, performs the steps in the above method embodiments.

Fig. 13 is a schematic block diagram of an electronic device provided in an embodiment of the present application, where the electronic device of fig. 13 may be the encoding device or the decoding device described above.

As shown in fig. 13, the electronic device 30 may include:

a memory 31 and a processor 32, the memory 31 being arranged to store a computer program 33 and to transmit the program code 33 to the processor 32. In other words, the processor 32 may call and run the computer program 33 from the memory 31 to implement the methods in the embodiments of the present application.

For example, the processor 32 may be configured to perform the steps of the method 200 described above in accordance with instructions in the computer program 33.

In some embodiments of the present application, the processor 32 may include, but is not limited to:

a general purpose processor, digital signal processor (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like.

In some embodiments of the present application, the memory 31 includes, but is not limited to:

volatile memory and/or nonvolatile memory. The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable EPROM (EEPROM), or a flash Memory. The volatile memory may be random access memory (Random Access Memory, RAM) which acts as an external cache. By way of example, and not limitation, many forms of RAM are available, such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (Double Data Rate SDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), and Direct memory bus RAM (DR RAM).

In some embodiments of the present application, the computer program 33 may be partitioned into one or more modules that are stored in the memory 31 and executed by the processor 32 to perform the methods of recording pages provided herein. The one or more modules may be a series of computer program instruction segments capable of performing the specified functions, which instruction segments describe the execution of the computer program 33 in the electronic device.

As shown in fig. 13, the electronic device 30 may further include:

a transceiver 34, the transceiver 34 being connectable to the processor 32 or the memory 31.

The processor 32 may control the transceiver 34 to communicate with other devices, and in particular, may send information or data to other devices or receive information or data sent by other devices. The transceiver 34 may include a transmitter and a receiver. The transceiver 34 may further include antennas, the number of which may be one or more.

It should be appreciated that the various components in the computing device 30 are connected by a bus system that includes a power bus, a control bus, and a status signal bus in addition to a data bus.

According to an aspect of the present application, there is provided a computer storage medium having stored thereon a computer program which, when executed by a computer, enables the computer to perform the method of the above-described method embodiments. Alternatively, embodiments of the present application also provide a computer program product comprising instructions which, when executed by a computer, cause the computer to perform the method of the method embodiments described above.

According to another aspect of the present application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium and executes the computer instructions to cause the computer device to perform the method of the above-described method embodiments.

In other words, when implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces, in whole or in part, a flow or function consistent with embodiments of the present application. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital video disc (digital video disc, DVD)), or a semiconductor medium (e.g., a Solid State Disk (SSD)), or the like.

Those of ordinary skill in the art will appreciate that the various illustrative modules and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, and for example, the division of the modules is merely a logical function division, and there may be additional divisions when actually implemented, for example, multiple modules or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or modules, which may be in electrical, mechanical, or other forms.

The modules illustrated as separate components may or may not be physically separate, and components shown as modules may or may not be physical modules, i.e., may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. For example, functional modules in the embodiments of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module.

The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A texture coding method, comprising:

2. The method of claim 1, wherein meshing the first texture map to obtain a first mesh map comprises:

mapping the first texture map into a unit rectangular space to obtain a unitized first texture map;

and carrying out grid division on the unitized first texture map to obtain the first grid map.

3. The method of claim 2, wherein resampling in the first texture map based on the second texture coordinates of each mesh vertex to obtain a second texture map comprises:

generating a blank unit rectangular area;

based on the topological structure of each grid vertex in the first grid map and the second texture coordinates, carrying out grid division on the unit rectangular area to obtain a second grid map;

Searching an ith pixel value in the unitized first texture map based on a second texture coordinate of an ith grid vertex in the second grid map, wherein i is a positive integer;

and determining a pixel value corresponding to the ith grid vertex in the second grid map based on the ith pixel value to obtain the second texture map.

4. A method according to claim 3, wherein determining, based on the ith pixel value, a pixel value corresponding to the ith mesh vertex in the second mesh map, to obtain the second texture map, includes:

determining the ith pixel value as a pixel value corresponding to the ith grid vertex in the second grid map to obtain a third texture map;

and interpolating the third texture map to obtain the second texture map.

5. The method of claim 4, wherein interpolating the third texture map to obtain the second texture map comprises:

optimizing a first texture coordinate of a jth interpolation point in the third texture map to obtain a second texture coordinate of the jth interpolation point, wherein j is a positive integer;

Searching a j pixel value in the unitized first texture map based on the second texture coordinate of the j interpolation point;

determining a pixel value corresponding to the j interpolation point in the third texture map based on the j pixel value to obtain a third texture map after interpolation;

and obtaining the second texture map based on the interpolated third texture map.

6. The method of claim 1 or 5, wherein optimizing the first texture coordinates to obtain the second texture coordinates comprises:

determining a first bijective transformation function from a plurality of preset bijective transformation functions;

and optimizing the first texture coordinates by adopting the first bijective transformation function to obtain the second texture coordinates.

7. The method of claim 6, wherein the compressed file further comprises at least one of: the first texture coordinates and the second texture coordinates of each grid vertex in the first grid graph, the topological structure of each grid vertex and the indication information of the first bijective transformation function.

8. A texture decoding method, comprising:

9. The method of claim 8, wherein if the compressed file further includes a first texture coordinate and a second texture coordinate for each mesh vertex in the first mesh map, reconstructing the first texture map based on the second texture map comprises:

generating a blank unit rectangular area;

determining the topological structure of each grid vertex in the first grid graph, and carrying out grid division on the unit rectangular area based on the topological structure of each grid vertex in the first grid graph and the first texture coordinates to obtain a third grid graph;

for an ith grid vertex in the third grid map, searching an ith pixel value corresponding to the ith grid vertex in the second texture map based on a second texture coordinate of the ith grid vertex, wherein i is a positive integer;

and determining a pixel value corresponding to the ith network vertex in the third grid graph based on the ith pixel value to obtain a reconstructed first texture map.

10. The method of claim 9, wherein if the compressed file includes a topology of each mesh vertex in the first mesh map, the determining the topology of each mesh vertex in the first mesh map comprises:

and decoding the compressed file to obtain the topological structure of each grid vertex in the first grid graph.

11. The method of claim 9, wherein determining, based on the ith pixel value, a pixel value corresponding to the ith network vertex in the third mesh map, to obtain the reconstructed first texture map, comprises:

determining the ith pixel value as a pixel value corresponding to the ith grid vertex in the third grid map to obtain a fourth texture map;

and interpolating the fourth texture map to obtain the reconstructed first texture map.

12. The method of claim 11, wherein interpolating the fourth texture map to obtain the reconstructed first texture map comprises:

for a j-th interpolation point in the fourth texture map, determining a second texture coordinate corresponding to a first texture coordinate of the j-th interpolation point, wherein j is a positive integer;

Searching a j pixel value in the second texture map based on the second texture coordinate of the j interpolation point;

determining a pixel value corresponding to a j interpolation point in the fourth texture map based on the j pixel value to obtain an interpolated fourth texture map;

and obtaining the reconstructed first texture map based on the interpolated fourth texture map.

13. The method of claim 12, wherein determining the second texture coordinate corresponding to the first texture coordinate of the jth interpolation point comprises:

determining a first bijective transformation function;

and optimizing the first texture coordinate of the jth interpolation point based on the first bijection transformation function to obtain the second texture coordinate of the jth interpolation point.

14. The method of claim 8, wherein determining a pixel value corresponding to a target texture coordinate in the first texture map based on the second texture map comprises:

determining a second texture coordinate corresponding to the target texture coordinate;

a first pixel value searched in the second texture map based on a second texture coordinate corresponding to the target texture coordinate;

And determining a pixel value corresponding to the target texture coordinate based on the first pixel value.

15. The method of claim 14, wherein if the compressed file further includes a first texture coordinate and a second texture coordinate of each mesh vertex in the first mesh map, determining the second texture coordinate corresponding to the target texture coordinate includes:

if the target texture coordinate is a first texture coordinate of a first grid vertex in the first grid graph, determining a second texture coordinate of the first grid vertex as a second texture coordinate corresponding to the target texture coordinate;

if the target texture coordinate is not the first texture coordinate of the grid vertex in the first grid graph, determining a first bijection transformation function, optimizing the target texture coordinate based on the first bijection transformation function to obtain a second texture coordinate corresponding to the target texture coordinate, or determining a corresponding relation table between the first texture coordinate and the second texture coordinate based on the first bijection transformation function, and searching the second texture coordinate corresponding to the target texture coordinate in the corresponding relation table.

16. The method according to claim 13 or 15, wherein if the compressed file includes the indication information of the first bijective transformation function, the determining the first bijective transformation function includes:

decoding the compressed file to obtain indication information of the first bijective transformation function;

and determining the first bijective transformation function based on the indication information.

17. A texture coding apparatus, comprising:

18. A texture decoding apparatus, comprising:

19. An electronic device includes a processor and a memory;

the memory is used for storing a computer program;

the processor for executing the computer program to implement the method of any of the preceding claims 1 to 7 or 8 to 16.

20. A computer-readable storage medium storing a computer program;

the computer program causes a computer to perform the method of any of the preceding claims 1 to 7 or 8 to 16.