CN113822955B

CN113822955B - Image data processing method, image data processing device, computer equipment and storage medium

Info

Publication number: CN113822955B
Application number: CN202111369444.9A
Authority: CN
Inventors: 项进喜; 张军; 韩骁
Original assignee: Tencent Healthcare Shenzhen Co Ltd
Current assignee: Tencent Healthcare Shenzhen Co Ltd
Priority date: 2021-11-18
Filing date: 2021-11-18
Publication date: 2022-02-22
Anticipated expiration: 2041-11-18
Also published as: CN113822955A

Abstract

The embodiment of the application discloses an image data processing method and device, computer equipment and a storage medium, and belongs to the technical field of computers. The method comprises the following steps: performing first compression on the first image data to obtain a first characteristic diagram; and performing n times of second compression on the first feature map to obtain n second feature maps, acquiring first distribution information corresponding to the first feature map based on the n second feature maps, performing third compression on the first feature map based on the distribution parameter of each feature value in the first feature map to obtain a third feature map corresponding to the first image data, and determining the third feature map and decompression information associated with the first distribution information as compression information of the first image data. According to the method provided by the embodiment of the application, the first image data is compressed in a multi-time compression mode, and the redundant information contained in the third feature diagram obtained by compression is weakened according to the distribution situation of the feature values in the feature diagram corresponding to the first image data, so that the compression effect is ensured.

Description

Image data processing method, image data processing device, computer equipment and storage medium

Technical Field

The embodiment of the application relates to the technical field of computers, in particular to an image data processing method and device, computer equipment and a storage medium.

Background

With the development of computer technology, the application of image data is more and more extensive. Generally, the image data has a large data volume, and occupies a large amount of storage resources or transmission resources when the image data is stored or transmitted, so that the image data is usually compressed first, but the compression effect of the current image data compression method is still not good.

Disclosure of Invention

The embodiment of the application provides an image data processing method and device, a computer device and a storage medium, which can improve the compression effect. The technical scheme is as follows.

In one aspect, an image data processing method is provided, and the method includes:

performing first compression on the first image data to obtain a first characteristic diagram;

performing n times of second compression on the first characteristic diagram to obtain n second characteristic diagrams, wherein n is a positive integer;

acquiring first distribution information corresponding to the first feature map based on the n second feature maps, wherein the first distribution information comprises a distribution parameter of each feature value in the first feature map;

performing third compression on the first feature map based on the distribution parameter of each feature value in the first feature map to obtain a third feature map corresponding to the first image data, wherein the number of feature values contained in the third feature map is smaller than the number of feature values contained in the first feature map;

and determining the third feature map and decompression information associated with the first distribution information as compression information of the first image data, wherein the decompression information is used for decompressing the third feature map to obtain the first image data.

In another aspect, there is provided an image data processing apparatus, the apparatus comprising:

the compression module is used for carrying out first compression on the first image data to obtain a first characteristic diagram;

the compression module is further configured to perform n times of second compression on the first feature map to obtain n second feature maps, where n is a positive integer;

an obtaining module, configured to obtain first distribution information corresponding to the first feature map based on the n second feature maps, where the first distribution information includes a distribution parameter of each feature value in the first feature map;

the compression module is further configured to perform third compression on the first feature map based on a distribution parameter of each feature value in the first feature map to obtain a third feature map corresponding to the first image data, where the number of feature values included in the third feature map is smaller than the number of feature values included in the first feature map;

and the determining module is used for determining the third feature map and the decompression information associated with the first distribution information as the compression information of the first image data, wherein the decompression information is used for decompressing the third feature map to obtain the first image data.

In a possible implementation manner, n is an integer greater than 1, and the obtaining module is configured to obtain distribution information corresponding to an nth second feature map; acquiring distribution information corresponding to the (n-1) th second feature map based on the nth second feature map and the corresponding distribution information until the distribution information corresponding to the 1 st second feature map is obtained; and acquiring the first distribution information based on the 1 st second feature map and the corresponding distribution information.

In another possible implementation manner, the obtaining module is configured to perform third compression on the nth second feature map based on a distribution parameter of each feature value in the nth second feature map to obtain a fourth feature map, where a feature value quantity included in the fourth feature map is smaller than a feature value quantity included in the second feature map; performing first decompression on the fourth feature map based on the distribution parameter of each feature value in the nth second feature map to obtain a fifth feature map; and determining distribution information corresponding to the (n-1) th second feature map based on the number of each feature value in the fifth feature map.

In another possible implementation manner, the decompressed information includes n fourth feature maps corresponding to the n second feature maps; the device further comprises:

the obtaining module is further configured to obtain the first distribution information based on the n fourth feature maps;

the decompression module is used for carrying out second decompression on the third feature map based on the distribution parameter of each feature value in the first feature map to obtain a sixth feature map;

the decompression module is further configured to perform third decompression on the sixth feature map to obtain second image data, where the second image data is the same as the first image data.

In another possible implementation manner, the distribution information corresponding to the nth second feature map includes an occurrence probability of each feature value in the nth second feature map; the obtaining module is configured to obtain a probability distribution function, where the probability distribution function indicates a distribution of occurrence probabilities of a plurality of feature values in the nth second feature map; and acquiring the occurrence probability of each characteristic value in the nth second characteristic diagram based on a plurality of characteristic values in the nth second characteristic diagram and the probability distribution function.

In another possible implementation manner, the compression module is configured to perform dimension reduction processing on the first feature map to obtain a seventh feature map, where the seventh feature map includes a plurality of feature areas; and updating the feature value in each feature region based on the similarity among the plurality of feature regions in the seventh feature map, and forming the plurality of updated feature regions into a 1 st second feature map.

In another possible implementation manner, the compression module includes:

an obtaining unit configured to obtain a target scaling factor of the first image data, the target scaling factor being used for scaling the image data;

the processing unit is used for performing dimensionality reduction processing on the first image data to obtain an eighth feature map;

and the scaling unit is used for scaling each characteristic value in the eighth characteristic diagram based on the target scaling factor and forming the scaled characteristic values into the first characteristic diagram.

In another possible implementation manner, the obtaining unit is configured to obtain a target compression rate of the first image data; and inquiring the corresponding relation between the compression ratio and the scaling factor based on the target compression ratio to obtain the target scaling factor corresponding to the target compression ratio.

In another possible implementation, the decompression information includes the first distribution information; the device further comprises:

In another possible implementation manner, the decompression module is configured to obtain a target scaling factor of the first image data, where the target scaling factor is used for scaling the image data; scaling each feature value in the sixth feature map based on the target scaling factor, and forming a ninth feature map by the scaled feature values; and performing dimensionality-raising processing on the ninth feature map to obtain the second image data.

In another possible implementation manner, the first compression is performed on the first image data to obtain a first feature map; performing n times of second compression on the first characteristic diagram to obtain n second characteristic diagrams; acquiring first distribution information corresponding to the first feature map based on the n second feature maps; performing third compression on the first feature map based on the distribution parameter of each feature value in the first feature map to obtain a third feature map corresponding to the first image data; the step of determining the third feature map and the decompressed information associated with the first distribution information as the compressed information of the first image data is performed based on an image compression model.

In another possible implementation manner, the apparatus further includes:

the decompression module is used for carrying out second decompression on the third characteristic diagram based on an image decompression model and the distribution parameters of each characteristic value in the first characteristic diagram to obtain a sixth characteristic diagram;

and the decompression module is further configured to perform third decompression on the sixth feature map based on the image decompression model to obtain second image data, where the second image data is the same as the first image data.

In another possible implementation manner, the apparatus further includes:

the acquisition module is further used for acquiring sample image data;

the compression module is further configured to compress the sample image data based on the image compression model to obtain sample compression information corresponding to the sample image data;

the decompression module is further used for decompressing the sample compression information based on the image decompression model to obtain decompressed image data;

and the training module is used for training the image compression model and the image decompression model based on the sample image data and the decompressed image data.

In another aspect, a computer device is provided, the computer device comprising a processor and a memory, the memory having stored therein at least one computer program, the at least one computer program being loaded and executed by the processor to perform operations performed by the image data processing method according to the above aspect.

In another aspect, a computer-readable storage medium is provided, in which at least one computer program is stored, the at least one computer program being loaded and executed by a processor to perform the operations performed by the image data processing method according to the above aspect.

In a further aspect, a computer program product is provided, comprising a computer program which, when executed by a processor, performs the operations performed by the image data processing method according to the above aspect.

The beneficial effects brought by the technical scheme provided by the embodiment of the application at least comprise:

the method, the device, the computer device and the storage medium provided by the embodiment of the application compress the first image data in a multi-compression mode, and in the compression process, the distribution situation of the feature values in the feature map corresponding to the first image data is considered, and the third feature map corresponding to the first image data is obtained by compression based on the distribution parameters of the feature values in the feature map, so that the first feature map is compressed as much as possible, redundant information contained in the third feature map is weakened, the number of the feature values contained in the third feature map obtained by compression is smaller than that contained in the first feature map, even if the data volume of the third feature map obtained by compression is as small as possible, and the compression effect is ensured.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application;

fig. 2 is a flowchart of an image data processing method provided in an embodiment of the present application;

FIG. 3 is a flow chart of another image data processing method provided in the embodiments of the present application;

FIG. 4 is a flowchart of another image data processing method provided in the embodiments of the present application;

FIG. 5 is a schematic structural diagram of an image processing model provided in an embodiment of the present application;

FIG. 6 is a schematic structural diagram of an input conversion layer according to an embodiment of the present disclosure;

fig. 7 is a schematic structural diagram of a super-first-order coding layer according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a residual convolution layer according to an embodiment of the present application;

fig. 9 is a schematic diagram of first image data provided in an embodiment of the present application;

FIG. 10 is a schematic structural diagram of a non-local attention layer provided in an embodiment of the present application;

FIG. 11 is a schematic diagram of a comparison between a feature map and a probability acquisition layer provided in an embodiment of the present application;

FIG. 12 is a schematic structural diagram of an output conversion layer according to an embodiment of the present disclosure;

fig. 13 is a flowchart of still another image data processing method provided in an embodiment of the present application;

FIG. 14 is a flow chart of model training provided by an embodiment of the present application;

fig. 15 is a flowchart of still another image data processing method provided in an embodiment of the present application;

FIG. 16 is a comparative illustration of compression performance provided by embodiments of the present application;

FIG. 17 is a comparative schematic of another compression performance provided by embodiments of the present application;

fig. 18 is a schematic structural diagram of an image data processing apparatus according to an embodiment of the present application;

fig. 19 is a schematic structural diagram of another image data processing apparatus according to an embodiment of the present application;

fig. 20 is a schematic structural diagram of a terminal according to an embodiment of the present application;

fig. 21 is a schematic structural diagram of a server according to an embodiment of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the embodiments of the present application more clear, the embodiments of the present application will be further described in detail with reference to the accompanying drawings.

As used herein, the terms "first," "second," "third," "fourth," "fifth," "sixth," and the like may be used herein to describe various concepts, but these concepts are not limited by these terms unless otherwise specified. These terms are only used to distinguish one concept from another. For example, a first feature may be termed a second feature, and, similarly, a second feature may be termed a first feature, without departing from the scope of the present application.

As used herein, the terms "at least one," "a plurality," "each," and "any," at least one of which includes one, two, or more than two, and a plurality of which includes two or more than two, each of which refers to each of the corresponding plurality, and any of which refers to any of the plurality. For example, the plurality of feature values includes 3 feature values, each of the 3 feature values is referred to, and any one of the 3 feature values can be a first feature value, a second feature value, or a third feature value.

Artificial Intelligence (AI) is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.

The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.

Machine Learning (ML) is a multi-domain cross discipline, and relates to a plurality of disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and the like. The special research on how a computer simulates or realizes the learning behavior of human beings so as to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve the performance of the computer. Machine learning is the core of artificial intelligence, is the fundamental approach for computers to have intelligence, and is applied to all fields of artificial intelligence. Machine learning and deep learning generally include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

According to the scheme, the image compression model and the image decompression model can be trained based on the machine learning technology of artificial intelligence, the image data can be compressed based on the trained image compression model, and the compressed information corresponding to the image data can be decompressed based on the trained image decompression model to restore the image data.

The image data processing method provided by the embodiment of the application can be executed by computer equipment. Optionally, the computer device is a terminal or a server. Optionally, the server is an independent physical server, or a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a web service, cloud communication, a middleware service, a domain name service, a security service, a CDN (Content Delivery Network), a big data and artificial intelligence platform, and the like. Optionally, the terminal is a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a smart voice interaction device, a smart home appliance, a vehicle-mounted terminal, and the like, but is not limited thereto.

In some embodiments, the computer program according to the embodiments of the present application may be deployed to be executed on one computer device or on multiple computer devices located at one site, or may be executed on multiple computer devices distributed at multiple sites and interconnected by a communication network, and the multiple computer devices distributed at the multiple sites and interconnected by the communication network can form a block chain system.

In some embodiments, the computer device is provided as a first device. Fig. 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application. Referring to fig. 1, the implementation environment includes a first device 101 and a second device 102, and the first device 101 and the second device 102 are connected through a wireless or wired network.

The first device 101 is configured to compress first image data to be compressed to obtain compression information of the first image data, send the compression information of the first image data to the second device 102 through a network connection with the second device 102, and the second device 102 receives the compression information sent by the first device 101 and decompresses the compression information to obtain second image data, where the second image data is the same as the first image data.

In one possible implementation, an image compression model is deployed in the first device 101, and an image decompression model matching the image compression model is deployed in the second device 102, and the image decompression model can decompress compressed information output by the image compression model. The first device 101 compresses the first image data based on the image compression model, and obtains compression information of the first image data. After receiving the compressed information sent by the first device 101, the second device 102 decompresses the decompressed information based on the image decompression model to obtain second image data identical to the first image data.

Optionally, an image compression model and an image decompression model are deployed on the first device 101, the first device 101 can further store compression information of the first image data, and the first device 101 decompresses the decompression information based on the image decompression model to obtain second image data identical to the first image data.

Fig. 2 is a flowchart of an image data processing method provided by an embodiment of the present application, which is executed by a computer device, and as shown in fig. 2, the method includes the following steps.

201. The computer equipment carries out first compression on the first image data to obtain a first characteristic diagram.

The first image data is image data to be compressed, and the first image data can be any image data, such as medical image data, map image data, and the like. The first feature map is used to represent features of the first image data, and the first compression is a compression form of compressing the first image data so that the data amount of the compressed first feature map is smaller than the data amount of the first image data. For example, the image file size of the first feature map obtained by first compressing the first image data is smaller than the image file size of the first image data.

202. And the computer equipment carries out n times of second compression on the first characteristic diagram to obtain n second characteristic diagrams, wherein n is a positive integer.

The second compression is a compression form for compressing the first characteristic diagram, and the second compression is different from the first compression, and the data volume of the second characteristic diagram is smaller than that of the first characteristic diagram. In the process of performing n times of second compression on the first characteristic diagram, a second characteristic diagram can be obtained after each time of second compression is performed. That is, performing the 1 st second compression on the first feature map to obtain a 1 st second feature map, and performing the 2 nd second compression on the 1 st second feature map to obtain a 2 nd second feature map; and repeating the second compression n times to obtain n second feature maps.

203. And the computer equipment acquires first distribution information corresponding to the first characteristic diagram based on the n second characteristic diagrams, wherein the first distribution information comprises the distribution parameters of each characteristic value in the first characteristic diagram.

In the embodiment of the present application, the first feature map includes at least one feature value, and the distribution parameter of each feature value indicates the distribution of the corresponding feature value in the first feature map. Through the first distribution information corresponding to the first feature map, the distribution situation of at least one feature value in the first feature map can be obtained. The n second feature maps are obtained by performing second compression on the first feature map, redundant information contained in the obtained second feature map is reduced after each second compression, and the first distribution information corresponding to the first feature map can be accurately estimated based on the n second feature maps, so that the influence caused by the redundant information is weakened, and the accuracy of the first distribution information is ensured.

204. And the computer equipment performs third compression on the first feature map based on the distribution parameter of each feature value in the first feature map to obtain a third feature map corresponding to the first image data, wherein the number of the feature values contained in the third feature map is smaller than that contained in the first feature map.

Wherein the third compression is a compression form of compressing the first feature map, and the third compression is different from both the first compression and the second compression. And performing third compression on the first feature map according to the distribution parameter of each feature value in the first feature map so as to compress the first feature map as much as possible, so that redundant information contained in the third feature map can be weakened, the number of feature values contained in the compressed third feature map is smaller than that contained in the first feature map, and the compression effect is ensured even if the data volume of the compressed third feature map is as small as possible.

205. The computer device determines the third feature map and the decompression information associated with the first distribution information as compression information of the first image data, and the decompression information is used for decompressing the third feature map to obtain the first image data.

The third feature map is obtained by compressing the first image data, and in the compression process, after the first feature map corresponding to the first image data is obtained, the third feature map is obtained by compressing the first feature map through the first distribution information, so that the decompressed information associated with the first distribution information and the third feature map are used as the compressed information of the first image data to ensure that the third feature map can be decompressed based on the decompressed information subsequently to restore the first image data.

The method provided by the embodiment of the application compresses the first image data in a multi-compression mode, and in the compression process, the distribution situation of the characteristic values in the characteristic diagram corresponding to the first image data is considered, and the third characteristic diagram corresponding to the first image data is obtained by compression based on the distribution parameters of the characteristic values in the characteristic diagram, so that the first characteristic diagram is compressed as much as possible, redundant information contained in the third characteristic diagram is weakened, the quantity of the characteristic values contained in the compressed third characteristic diagram is smaller than that contained in the first characteristic diagram, even if the data volume of the compressed third characteristic diagram is as small as possible, the compression effect is ensured.

On the basis of the embodiment shown in fig. 2, when acquiring the first distribution information corresponding to the first feature map based on the n second feature maps, the distribution information corresponding to the n second feature maps is acquired first, and the first distribution information is acquired based on the n second feature maps and the distribution information corresponding to each second feature map, and the image compression process is described in detail in the following embodiments.

Fig. 3 is a flowchart of an image data processing method provided by an embodiment of the present application, which is executed by a computer device, and as shown in fig. 3, the method includes the following steps.

301. The computer device obtains a target scaling factor for the first image data, the target scaling factor being used to scale the image data.

The target scaling factor is used for scaling the image data, and the target scaling factor is a specified target scaling factor, that is, the first image data is expected to be scaled based on the target scaling factor. For example, the target Scaling Factor is SF (Scaling Factor). Optionally, the target zoom factor is specified by a user.

In the embodiment of the present application, the first image data is any type of image data, and optionally, the first image data is medical image data, map image data, or the like. For example, the first Image data is medical Image data, for example, the first Image data is WSI (white Slide Image), in a medical scene, the WSI is equivalent to an Image observed under a microscope, and a Whole Slide is subjected to full-information and all-direction fast scanning to obtain a digitized pathological section corresponding to the Slide, that is, the WSI corresponding to the Slide.

In one possible implementation, this step 301 includes: and acquiring a target compression ratio of the first image data, and inquiring a corresponding relation between the compression ratio and the scaling factor based on the target compression ratio to obtain a target scaling factor corresponding to the target compression ratio.

Wherein the target compression rate is used to indicate a desired degree of compression of the first image data. The target compression rate is specified, i.e., it is desirable to compress the first image data at the target compression rate. The corresponding relation between the compression rate and the scaling factor comprises a plurality of compression rates, each compression rate corresponds to one scaling factor, and the target scaling factor corresponding to the target compression rate can be determined by inquiring the corresponding relation between the compression rate and the scaling factor based on the specified target scaling rate. In the embodiment of the application, a plurality of compression ratios are provided, the image data can be compressed according to different compression ratios, the adjustability of the plurality of compression ratios is realized, and the method and the device can be suitable for various compression requirements.

Optionally, each compression rate corresponds to a code rate, and the code rate is used for representing the number of bits corresponding to each feature value in the feature map. For example, the code rate is bpp (bit per pixel). When the image data is compressed by the scaling factor corresponding to each compression ratio, the bit number of each characteristic value in the obtained characteristic graph is matched with the code rate corresponding to the compression ratio, so that the compression effect is ensured.

Alternatively, the computer device includes a plurality of compression rates, and the process of obtaining the target compression rate of the first image data includes: a target compression ratio is selected from the plurality of compression ratios. The plurality of compression rates are compression rates that can be achieved when image data is compressed.

In one possible implementation, the computer device is installed with an image data processing application for compressing or decompressing image data, and step 301 includes: the method includes displaying a plurality of compression ratios based on the image processing application, selecting a target compression ratio from the displayed plurality of compression ratios, and subsequently compressing the first image data based on the image processing application and the target compression ratio.

302. And the computer equipment performs dimensionality reduction on the first image data to obtain an eighth feature map.

The eighth feature map comprises at least one feature value, and the feature dimension of the eighth feature map is smaller than the dimension of the first image data. For example, the first image data and the eighth feature map are both represented in a three-dimensional matrix, the dimension of the first image data is 3 × 512, and the dimension of the eighth feature map is 384 × 32. Optionally, a nonlinear transformation mode is adopted to perform dimensionality reduction processing on the first image data, so that the feature values in the obtained eighth feature map are sparser, redundant information in the image data can be eliminated, and accuracy of subsequent compression is guaranteed.

In one possible implementation, this step 302 includes: and the computer equipment performs multiple dimensionality reduction on the first image data to obtain an eighth feature map. Through carrying out multiple dimensionality reduction processing on the first image data, the smoothness of a dimensionality reduction process is ensured, the poor accuracy of the obtained characteristic diagram caused by a dimensionality reduction amplitude process is avoided, and the accuracy of the eighth characteristic diagram is ensured.

Optionally, in the process of performing dimension reduction processing on the first image data for multiple times, after performing dimension reduction processing on the first image data every time, normalization processing is performed on the obtained feature map, and then, next dimension reduction processing is performed on the feature map after the normalization processing.

For example, the multiple dimensionality reduction processes include 3 times, the 1 st dimensionality reduction process is carried out on the first image data to obtain a 1 st feature map, and the 1 st feature map is subjected to normalization process; performing 2-time dimensionality reduction on the normalized 1 st feature map to obtain a 2 nd feature map, and performing normalization on the 2 nd feature map; and performing 3 rd dimensionality reduction on the normalized 2 nd feature map to obtain a 3 rd feature map, and performing normalization on the 3 rd feature map to obtain an eighth feature map.

303. And the computer equipment scales each characteristic value in the eighth characteristic diagram based on the target scaling factor, and forms the scaled characteristic values into the first characteristic diagram.

In this embodiment of the present application, the eighth feature map includes at least one feature value, each feature value in the eighth feature map is scaled based on the target scaling factor, a scaled feature value corresponding to each feature value is obtained, and the scaled feature value corresponding to each feature value constitutes the first feature map. And scaling each characteristic value in the eighth characteristic diagram by the target scaling factor so that the data volume of the scaled characteristic value is smaller than that of the characteristic value before scaling, and the data volume of the first characteristic diagram is smaller than that of the eighth characteristic diagram, thereby realizing the compression of the eighth characteristic diagram.

In one possible implementation, the target scaling factor, the eighth feature map, and the first feature map satisfy the following relationship:

wherein the content of the first and second substances,

for the purpose of representing a first characteristic diagram,

for representing a target scaling factor for the image data,

for showing an eighth characteristic diagram.

It should be noted that, in the embodiment of the present application, the first feature map is obtained by using the obtained target scaling factor, that is, a process of performing the dimensionality reduction on the first image data and then performing the scaling based on the target scaling factor is equivalent to a process of performing the first compression on the first image data, and in another embodiment, the step 301 and the step 303 need not be performed, and the first feature map can be obtained by performing the first compression on the first image data in other manners.

304. And the computer equipment performs the 1 st second compression on the first characteristic diagram to obtain a 1 st second characteristic diagram, performs the ith second compression on the i-1 st second characteristic diagram to obtain an ith second characteristic diagram until an nth second characteristic diagram is obtained.

Wherein i is an integer greater than 1 and not greater than n, and n is an integer greater than 1. And performing 1 st second compression on the first feature diagram to obtain a 1 st second feature diagram, performing 2 nd second compression on the 1 st second feature diagram to obtain a 2 nd second feature diagram, and so on, performing n-th second compression on the n-1 st second feature diagram to obtain n second feature diagrams, namely performing one second compression each time to obtain one second feature diagram, namely obtaining n second feature diagrams.

In one possible implementation, feature sizes of different ones of the n second feature maps are different. In the embodiment of the present application, for any feature map, after performing the second compression on the feature map, the feature size of the feature map obtained through the second compression is smaller than the feature size of the feature map before compression, that is, the feature sizes of the n second feature maps are gradually reduced according to the arrangement order of the n second feature maps.

In one possible implementation manner, the process of obtaining the 1 st second feature map includes: and performing dimension reduction processing on the first feature map to obtain a seventh feature map, updating the feature value in each feature region based on the similarity among the plurality of feature regions in the seventh feature map, and forming the plurality of updated feature regions into a 1 st second feature map.

The seventh feature map comprises a plurality of feature areas, the obtained 1 st second feature map also comprises a plurality of feature areas, and the plurality of feature areas in the seventh feature map are in one-to-one correspondence with the plurality of features in the 1 st second feature map. By considering the similarity among different feature areas in the seventh feature map, each feature area in the seventh feature map is updated, the seventh feature map is updated at the global intersection angle of the seventh feature map, so as to enrich the global information contained in the obtained 1 st second feature map, and the similar local features in the seventh feature map can be captured, so that redundant information is reduced in the 1 st second feature map.

Optionally, the process of updating any feature region in the second feature map includes: and performing weighted fusion on the plurality of characteristic regions based on the similarity between the characteristic region and the plurality of characteristic regions to obtain the updated characteristic region of the characteristic region.

Optionally, the process of obtaining the 1 st second feature map based on the seventh feature map includes: zooming the seventh feature map to obtain a first zoomed feature map, reconstructing the first zoomed feature map to obtain a second zoomed feature map, wherein the feature size of the second zoomed feature map is different from the feature size of the first zoomed feature map, for any feature region in the seventh feature map, obtaining the similarity between the feature region and a plurality of feature regions in the second zoomed feature map, and forming the similarity feature by the plurality of similarities; reconstructing the seventh feature map to obtain a third scaled feature map, wherein the feature sizes of the third scaled feature map are similar to those of the second scaled feature map, performing weighted fusion on a plurality of feature areas in the third feature map based on the similarity feature to obtain updated feature areas corresponding to the feature areas, obtaining updated feature areas corresponding to each feature area according to the above steps, and forming the updated feature areas into the 1 st second feature map.

For example, the seventh feature map has a feature size of

The feature size of the first scaled feature map is

，

For the scaling factor, the feature size of the second scaling feature map obtained by reconstructing the first scaling feature map is

The feature size of a third scaled feature map obtained by reconstructing the seventh feature map is

The feature size of the 1 st second feature map formed by the updated plurality of feature areas is

。

It should be noted that, the manner of obtaining the other second feature diagrams, except for the 1 st second feature diagram, in the n second feature diagrams is the same as the manner of obtaining the 1 st second feature diagram, and is not described herein again.

It should be noted that, in this embodiment of the present application, n is an integer greater than 1 as an example, n times of second compression are sequentially performed on the first feature map to obtain n second feature maps, and in another embodiment, step 304 is not required to be performed, and other manners can be adopted to perform n times of second compression on the first feature map to obtain n second feature maps, where n is a positive integer.

305. And the computer equipment acquires the distribution information corresponding to the nth second feature map.

The distribution information corresponding to the nth second feature map includes a distribution parameter of each feature value in the nth second feature map, where the distribution parameter of each feature value indicates a distribution situation of the corresponding feature value in the nth second feature map, that is, the distribution information corresponding to the nth second feature map can embody a distribution situation of each feature value in the nth second feature map.

In a possible implementation manner, the distribution information corresponding to the nth second feature map includes an occurrence probability of each feature value in the nth second feature map; this step 305 includes: and acquiring a probability distribution function, and acquiring the occurrence probability of each characteristic value in the nth second characteristic diagram based on the plurality of characteristic values in the nth second characteristic diagram and the probability distribution function.

Wherein the probability distribution function indicates the distribution of the occurrence probability of the plurality of feature values in the nth second feature map, and is referred to as the first probability distribution function. In the embodiment of the present application, the nth second feature map is the second feature map with the smallest feature size, and the first probability distribution function is the second feature map with the smallest feature size, for example, the first probability distribution function is a zero-mean Gaussian distribution function, such as

，

Is a constant. And sequentially substituting each characteristic value in the nth second characteristic diagram into the first probability distribution function to obtain the occurrence probability of each characteristic value, and forming the occurrence probability of each characteristic value into the distribution information corresponding to the nth second characteristic diagram.

Optionally, any one of the feature values in the nth second feature map, the first probability distribution function, and the probability of occurrence of the feature value satisfies the following relationship:

wherein the content of the first and second substances,

for indicating the probability of occurrence of the ith feature value in the nth second feature map,

used for representing the ith characteristic value in the nth second characteristic diagram;

for representing the first probability distribution function.

306. And the computer equipment acquires the distribution information corresponding to the (n-1) th second feature map based on the nth second feature map and the corresponding distribution information until the distribution information corresponding to the 1 st second feature map is obtained.

In the embodiment of the present application, since the nth second feature map is obtained by compressing the nth-1 second feature map, the spatial redundancy information in the nth second feature map is reduced relative to the nth-1 second feature map, and based on the nth second feature map and the corresponding distribution information, the distribution parameter of the feature value in the nth-1 second feature map can be estimated, so as to obtain the distribution information corresponding to the nth-1 second feature map. Based on the arrangement sequence from the nth second feature diagram to the 1 st second feature diagram, according to the above manner, the distribution information of each second feature diagram is sequentially acquired from the (n-1) th feature diagram until the distribution information corresponding to the 1 st second feature diagram is obtained.

For example, if n is 3, the distribution information corresponding to the 2 nd second feature map is obtained based on the 3 rd second feature map and the corresponding distribution information; and acquiring the distribution information corresponding to the 1 st second feature map based on the 2 nd second feature map and the corresponding distribution information.

In a possible implementation manner, the process of obtaining the distribution information corresponding to the (n-1) th second feature map includes the following steps 3061-3063.

3061. And the computer equipment performs third compression on the nth second feature map based on the distribution parameter of each feature value in the nth second feature map to obtain a fourth feature map corresponding to the nth second feature map.

And the number of the characteristic values contained in the fourth characteristic diagram is less than that contained in the second characteristic diagram. The distribution information corresponding to the nth second feature map comprises the distribution parameter of each feature value in the nth second feature map, and the distribution parameter of each feature value can indicate the distribution situation of the corresponding feature value in the nth second feature map.

In one possible implementation, the distribution parameter of each feature value in the nth second feature map is the occurrence probability of each feature value, and then step 3061 includes: and performing third compression on the nth second feature map based on the occurrence probability of each feature value in the nth second feature map to obtain a fourth feature map corresponding to the nth second feature map.

In this embodiment of the application, the distribution information corresponding to the nth second feature map includes an occurrence probability corresponding to each feature value, and the nth second feature map may be subjected to third compression based on the occurrence probability of each feature value in the nth second feature map by using an arithmetic coding method to obtain the fourth feature map. For example, the arithmetic coding is ae (arithmetric encoding). Optionally, the fourth feature map corresponding to the nth second feature map includes one feature value.

Optionally, the nth second feature map includes a plurality of feature values, and the process of obtaining the fourth feature map corresponding to the nth second feature map includes: and determining a probability interval corresponding to each characteristic value based on the occurrence probability corresponding to the characteristic values, compressing the characteristic values based on the probability intervals corresponding to the characteristic values, and forming the fourth characteristic diagram by using the compressed characteristic values.

For example, the plurality of feature values included in the nth second feature map are respectively a feature value 1, a feature value 2, a feature value 3, and a feature value 3, the occurrence probability corresponding to the feature value 1 is 0.2, the occurrence probability corresponding to the feature value 2 is 0.2, and the occurrence probability corresponding to the feature value 3 is 0.6; based on the occurrence probability corresponding to the feature value 1, the occurrence probability corresponding to the feature value 2, and the occurrence probability corresponding to the feature value 3, it is determined that the probability interval corresponding to the feature value 1 is [0, 0.2 ], the probability interval corresponding to the feature value 2 is [0.2, 0.4 ], and the probability interval corresponding to the feature value 3 is [0.4, 1). According to the arrangement sequence of a plurality of characteristic values in the nth second characteristic diagram, if the first characteristic value is a characteristic value 1, the compressed value belongs to a probability interval [0, 0.2 ] corresponding to the characteristic value 1; the second eigenvalue is eigenvalue 2, the occurrence probability corresponding to the eigenvalue 2 is 0.2, and the compressed value belongs to the first 20% of the probability interval [0, 0.2), that is, the compressed value belongs to the probability interval [0, 0.04); the third eigenvalue is eigenvalue 3, the probability corresponding to the eigenvalue 3 is 0.6, and the compressed value belongs to the first 60% of the probability interval [0, 0.04), that is, the compressed value belongs to the probability interval [0, 0.024); the fourth eigenvalue is eigenvalue 3, and the probability corresponding to the eigenvalue 3 is 0.6, then the compressed value belongs to the first 60% of the probability interval [0, 0.024), i.e., the compressed value belongs to the probability interval [0, 0.0144), and any value from the probability interval [0, 0.0144) is selected as the compressed value, and a fourth eigenmap is constructed based on the compressed values, i.e., the fourth eigenmap includes one eigenvalue.

In a possible implementation manner, before performing the third compression on the nth second feature map, the process of obtaining a fourth feature map corresponding to the nth second feature map includes: and quantizing the feature values in the nth second feature map, and performing third compression on the quantized nth second feature map based on the distribution parameter of each feature value in the nth second feature map to obtain a fourth feature map corresponding to the nth second feature map.

Before the third compression is carried out on the nth second feature map, the feature values in the nth second feature map are quantized, the feature values in the nth second feature map are mapped into a target range, the feature values in the quantized nth second feature map all belong to the target range, and the bit number of each feature value is adjusted to realize the adjustment of the compression rate. For example, the target range is {0, 1, 2.,. 255}, the feature values in the nth second feature map include 0.2, 0.5, and the like, and the quantization processing is performed on the nth second feature map, so that the feature values in the nth second feature map after quantization are integers between 0 and 255.

3062. And the computer equipment performs first decompression on the fourth feature map corresponding to the nth second feature map based on the distribution parameter of each feature value in the nth second feature map to obtain a fifth feature map corresponding to the nth second feature map.

Since the fourth feature map is obtained by compressing the nth second feature map based on the distribution parameter of each feature value in the nth second feature map, the fourth feature map is subjected to first decompression based on the distribution parameter of each feature value in the nth second feature map, so as to obtain a fifth feature map similar to the (n-1) th second feature map, for example, the feature size of the (n-1) th second feature map is the same as the feature size of the fifth feature map.

In one possible implementation, step 3062 includes: and performing fourth decompression on a fourth feature map corresponding to the nth second feature map based on the distribution parameter of each feature value in the nth second feature map to obtain a tenth feature map, and performing fifth decompression on the tenth feature map to obtain a fifth feature map.

In the embodiment of the present application, the fourth decompression process is combined with the fifth decompression process, and corresponds to the first decompression process. The fourth decompression process corresponds to the third compression process, that is, a tenth feature map obtained by fourth decompressing the fourth feature map corresponding to the nth second feature map is similar to the nth second feature map, for example, the feature size of the tenth feature map is the same as the feature size of the nth second feature map. The fifth decompression process corresponds to the second compression process, i.e. the fifth characteristic diagram corresponding to the nth second characteristic diagram is similar to the (n-1) th second characteristic diagram. And performing fourth decompression and fifth decompression on the fourth characteristic diagram corresponding to the nth second characteristic diagram to ensure that the compression process corresponds to the decompression process and ensure the accuracy of the subsequently obtained distribution information.

Optionally, the distribution parameter of each feature value in the nth second feature map is the occurrence probability of each feature value, and the fourth feature map is subjected to fourth decompression by using a numerical decoding manner based on the occurrence probability of each feature value in the nth second feature map, so as to obtain a tenth feature map. Wherein, the arithmetic decoding is AD (Arithmetric decoding).

Optionally, the nth second feature map includes a plurality of feature values, a probability interval corresponding to each feature value is determined based on the occurrence probability corresponding to the plurality of feature values, fourth decompression is performed on the feature values in the fourth feature map based on the probability intervals corresponding to the plurality of feature values, and the feature values obtained through the fourth decompression constitute the tenth feature map.

After the probability intervals corresponding to the feature values are determined, the feature values compressed into the fourth feature map can be decompressed based on the probability intervals to which the probability values included in the fourth feature map belong, and the tenth feature map is formed by the decompressed feature values. For example, the fourth feature map corresponding to the nth second feature map includes one feature value, and the feature values corresponding to the feature value are decompressed according to the probability and the probability interval corresponding to the feature value included in the nth second feature map, and the decompressed feature values form the tenth feature map.

In a possible implementation manner, when the fourth feature map corresponding to the nth second feature map is obtained, the nth second feature map is quantized first, and then the quantized nth second feature map is subjected to third compression based on a distribution parameter of each feature value in the nth second feature map to obtain the fourth feature map corresponding to the nth second feature map, and the fifth feature map obtained according to the step 3062 is similar to the quantized nth second feature map, that is, the feature size of the fifth feature map is the same as the feature size of the quantized nth second feature map. The distribution of the feature values in the fifth feature map is similar to the distribution of the feature values included in the quantized nth second feature map.

3063. And the computer equipment determines the distribution information corresponding to the (n-1) th second feature map based on the number of each feature value in the fifth feature map corresponding to the nth second feature map.

And the distribution information corresponding to the (n-1) th second feature map comprises the distribution parameters of each feature value in the (n-1) th second feature map. Since the fifth feature map is similar to the (n-1) th second feature map, the distribution of the feature values in the fifth feature map is similar to the distribution of the feature values in the (n-1) th second feature map, the distribution parameter of each feature value is determined based on the number of each feature value in the fifth feature map, and the distribution parameter of each feature value constitutes the distribution information corresponding to the (n-1) th second feature map.

In one possible implementation, the distribution parameter of each feature value is the occurrence probability of the corresponding feature value, and step 3063 includes: and determining a second probability distribution function corresponding to the fifth feature map based on the number of each feature value in the fifth feature map corresponding to the nth second feature map, and determining distribution information corresponding to the (n-1) th second feature map based on each feature value in the fifth feature map and the probability distribution corresponding to the fifth feature map.

And because the feature values in the fifth feature map are similar to the distribution of the feature values in the (n-1) th second feature map, the occurrence probability of each feature value in the fifth feature map forms the distribution information corresponding to the (n-1) th second feature map.

Optionally, each second feature map corresponds to a fifth feature map, the second probability distribution function corresponding to each fifth feature map is represented in the form of a gaussian distribution function, and the second gaussian distribution function corresponding to each fifth feature map is obtained by weighted fusion of at least one gaussian distribution function.

For the fifth feature map corresponding to each second feature map, the second probability distribution function corresponding to the fifth feature map represents the occurrence probability distribution condition of each feature value in the second feature map in a mode of mixing a Gaussian distribution function, so that the accuracy of the obtained distribution information is ensured.

For example, n is 3, the mixture gaussian distribution function corresponding to the 1 st fifth image feature is obtained by weighted fusion of 3 independent gaussian distribution functions, for example, the mixture gaussian distribution function corresponding to the 1 st fifth image feature is

，

Indicating the order of the gaussian distribution function in the mixed gaussian distribution function,

representing the weight corresponding to each gaussian distribution function in the mixed gaussian distribution function,

expressing the second in the mixed Gaussian distribution function

A function of a gaussian distribution of the intensity of the light,

is shown as

The mean of the individual gaussian distribution functions,

is shown as

The variance of the gaussian distribution function. The Gaussian mixture distribution function corresponding to the 2 nd fifth feature map is obtained by weighted fusion of 2 independent Gaussian distribution functions, for example, the Gaussian mixture distribution function corresponding to the 2 nd fifth image feature is

，

expressing the second in the mixed Gaussian distribution function

A function of a gaussian distribution of the intensity of the light,

is shown as

The mean of the individual gaussian distribution functions,

is shown as

The variance of the gaussian distribution function. The second Gaussian distribution function corresponding to the 3 rd fifth feature map is a zero mean Gaussian distribution function

It is shown that,

is a constant. Wherein the probability distribution is a gaussian probability distribution function.

It should be noted that, in the embodiment of the present application, the process of obtaining the distribution information corresponding to the n-1 th second feature map based on the nth second feature map and the corresponding distribution information is described as an example, and the process of obtaining the distribution information corresponding to the n-1 th second feature map and obtaining the distribution information corresponding to other second feature maps is similar to the step 3061 and 3063, in the process of obtaining the distribution information corresponding to each second feature map, the fourth feature map corresponding to each second feature map can be obtained, and this process is not described herein again.

307. And the computer equipment acquires first distribution information corresponding to the first characteristic diagram based on the 1 st second characteristic diagram and the corresponding distribution information.

The first distribution information comprises a distribution parameter of each characteristic value in the first characteristic diagram. Optionally, the first distribution information includes an occurrence probability of each feature value in the first feature map. In the embodiment of the present application, since the 1 st second feature map is obtained by compressing the first feature map, spatial redundant information in the 1 st second feature map is reduced compared to the first feature map, and based on the 1 st second feature map and corresponding distribution information, the distribution parameter of the feature values in the first feature map can be estimated to obtain the distribution information corresponding to the first feature map.

In the embodiment of the application, n times of second compression are performed on the first feature map to obtain n second feature maps, and then, distribution information corresponding to each second feature map is sequentially obtained from the nth second feature map until the first distribution information corresponding to the first feature map is obtained based on the 1 st second feature map and the corresponding distribution information, so that the distribution situation of feature values in the first feature map is determined by using a local-global multi-level feature map, and thus redundant information in the feature map is reduced.

It should be noted that the step 307 is similar to the step 306, and is not described herein again.

It should be noted that, in the embodiment of the present application, the distribution information corresponding to each second feature map is sequentially obtained from the nth second feature map in the order of the n second feature maps until the distribution information corresponding to the first feature map is obtained based on the 1 st second feature map and the corresponding distribution information, and in another embodiment, the step 305 and the corresponding step 307 do not need to be executed, and the first distribution information corresponding to the first feature map can be obtained based on the n second feature maps in other manners.

308. And the computer equipment performs third compression on the first feature map based on the distribution parameter of each feature value in the first feature map to obtain a third feature map corresponding to the first image data, wherein the number of the feature values contained in the third feature map is smaller than that contained in the first feature map.

The first characteristic diagram is compressed through the distribution parameters of each characteristic value in the first characteristic diagram so as to compress the first characteristic diagram as much as possible, redundant information contained in the third characteristic diagram can be weakened, and therefore the compressed third characteristic diagram is as small as possible, and the compression effect is guaranteed.

It should be noted that the step 308 is similar to the step 3061, and is not described herein again.

309. The computer device determines the third feature map and the decompressed information associated with the first distribution information as compressed information of the first image data.

And the decompression information is used for decompressing the third feature map to obtain the first image data. The third feature map is obtained by compressing the first image data, and the first distribution information is used when the third feature map is obtained by compression, and the decompression information is associated with the first distribution information and used for decompressing the third feature map to obtain the first image data.

In one possible implementation, the decompression information includes the first distribution information.

In one possible implementation manner, the decompressed information includes n fourth feature maps corresponding to the n second feature maps. In this embodiment, in the process of acquiring the first distribution information based on the n second feature maps according to the above-mentioned manner of steps 3061-3063, the fourth feature map corresponding to each second feature map can be acquired, and the n fourth feature maps corresponding to the n second feature maps are taken as the decompression information.

Optionally, the third feature map accounts for more than 90% of the entire compressed information size, and the n fourth feature maps account for less than 10% of the entire compressed information size.

Optionally, the decompressed information includes n fourth feature maps and distribution information corresponding to the nth fourth feature map.

In addition to the embodiment shown in fig. 3, after the compressed information of the first image data is acquired, the compressed information can be decompressed and restored to obtain the first image data, and the decompression process includes the following two modes.

In the first way, the decompression information in the compressed information includes the first distribution information, which includes the following steps 310 and 311.

310. And the computer equipment carries out second decompression on the third feature map based on the distribution parameter of each feature value in the first feature map to obtain a sixth feature map.

The second decompression process corresponds to the third compression process, and the sixth characteristic diagram is similar to the first characteristic diagram, that is, the characteristic size of the sixth characteristic diagram is the same as the characteristic size of the first characteristic diagram. And decompressing the characteristic values in the third characteristic diagram through the distribution parameters of each characteristic value in the first characteristic diagram, and forming a sixth characteristic diagram by the decompressed characteristic values.

311. And thirdly decompressing the sixth feature map by the computer equipment to obtain second image data, wherein the second image data is the same as the first image data.

Wherein the third decompression process corresponds to the first compression process. And carrying out second decompression on the third characteristic diagram through the distribution parameter of each characteristic value in the first characteristic diagram, and then carrying out third decompression on the sixth characteristic diagram obtained by decompression to restore to obtain second image data, wherein the second image data is the same as the first image data, so that decompression of the compressed information is realized.

In one possible implementation, this step 311 includes: acquiring a target scaling factor of the first image data, scaling each characteristic value in the sixth characteristic diagram based on the target scaling factor, and forming a ninth characteristic diagram by the scaled characteristic values; and performing dimension-raising processing on the ninth feature map to obtain second image data.

Wherein the target scaling factor is used for scaling the image data. The process of scaling each feature value in the sixth feature map is the reverse of the process of scaling each feature value in the eighth feature map in step 303 described above. For example, in step 303, each feature value in the eighth feature map is enlarged based on the target scaling factor, and in step 311, each feature value in the sixth feature map is reduced based on the target scaling factor. This ninth feature map corresponds to the eighth feature map described above, and the process of performing the dimension-up processing on the ninth feature map corresponds to the dimension-down processing in step 303 described above. For example, the first image data has a dimension of 3 × 512, and the eighth feature map has a dimension of 384 × 32; the ninth signature has dimensions 384 x 32. The dimension of the second image data is 3 × 512.

Optionally, the target scaling factor, the sixth feature map, and the ninth feature map satisfy the following relationship:

wherein the content of the first and second substances,

for the purpose of representing a first characteristic diagram,

for representing a target scaling factor for the image data,

for showing an eighth characteristic diagram.

In the second way, the decompressed information in the compressed information includes n fourth feature maps corresponding to the n second feature maps, which includes the following steps 312 and 314.

312. The computer device acquires the first distribution information based on the n fourth feature maps.

In a possible implementation manner, the decompressed information further includes distribution information corresponding to the nth second feature map, and the process of obtaining the first distribution information includes: based on the distribution parameter of each feature value in the nth second feature graph, performing first decompression on a fourth feature graph corresponding to the nth second feature graph to obtain a fifth feature graph corresponding to the nth second feature graph, based on the number of each feature value in the fifth feature graph, determining distribution information corresponding to the (n-1) th second feature graph until the distribution information corresponding to the first feature graph is obtained, and based on the 1 st second feature graph and the corresponding distribution information, obtaining first distribution information corresponding to the first feature graph.

It should be noted that the process of obtaining the first distribution information based on the n fourth feature maps is similar to the steps 3062-3063 and 307, which is not described herein again.

313. And the computer equipment carries out second decompression on the third feature map based on the distribution parameter of each feature value in the first feature map to obtain a sixth feature map.

314. And performing third decompression on the sixth feature map to obtain second image data, wherein the second image data is the same as the first image data.

The steps 313 and 314 are similar to the steps 310 and 311, and are not described herein again.

The compressed information of the first image data is decompressed to obtain second image data which is the same as the first image data, and the image data is restored.

In addition to the embodiment shown in fig. 3, the compression of the first image data can be realized based on the image compression model, and the compression information of the first image data can be acquired, and the compression process is described in detail in the following embodiments.

Fig. 4 is a flowchart of an image data processing method provided by an embodiment of the present application, which is executed by a computer device, and as shown in fig. 4, the method includes the following steps.

401. The computer device obtains a target scaling factor for the first image data based on the image compression model, the target scaling factor being used to scale the image data.

402. And the computer equipment performs dimension reduction processing on the first image data based on the image compression model to obtain an eighth feature map.

403. And the computer equipment scales each characteristic value in the eighth characteristic diagram based on the image compression model and the target scaling factor, and forms the scaled characteristic values into the first characteristic diagram.

404. And the computer equipment performs 1 st second compression on the first characteristic diagram based on the image compression model to obtain a 1 st second characteristic diagram, performs ith second compression on the i-1 st second characteristic diagram to obtain an ith second characteristic diagram until an nth second characteristic diagram is obtained.

405. And the computer equipment acquires the distribution information corresponding to the nth second characteristic diagram based on the image compression model.

406. And the computer equipment acquires the distribution information corresponding to the (n-1) th second feature map based on the image compression model, the nth second feature map and the corresponding distribution information until the distribution information corresponding to the 1 st second feature map is obtained.

407. And the computer equipment acquires first distribution information corresponding to the first characteristic diagram based on the image compression model, the 1 st second characteristic diagram and the corresponding distribution information.

408. And the computer equipment performs third compression on the first characteristic diagram based on the image compression model and the distribution parameters of each characteristic value in the first characteristic diagram to obtain a third characteristic diagram corresponding to the first image data, wherein the quantity of the characteristic values contained in the third characteristic diagram is less than that contained in the first characteristic diagram.

409. The computer device determines the third feature map and the decompressed information associated with the first distribution information as compressed information of the first image data based on the image compression model.

It should be noted that the steps 401-409 are the same as the steps 301-309, and are not described herein again.

In addition to the embodiment shown in fig. 4, after the compressed information of the first image data is acquired, the first image data can be obtained by decompressing the compressed information based on the image decompression model, that is, the decompression process includes the following two modes.

In the first way, the decompressed information in the compressed information includes the first distribution information, and the decompressing process includes the following steps 410-411.

410. And the computer equipment carries out second decompression on the third characteristic diagram based on the image decompression model and the distribution parameters of each characteristic value in the first characteristic diagram to obtain a sixth characteristic diagram.

411. And the computer equipment performs third decompression on the sixth feature map based on the image decompression model to obtain second image data, wherein the second image data is the same as the first image data.

It should be noted that the steps 410-411 are the same as the steps 310-311, and are not described herein again.

In the second way, the decompressing information in the compressed information includes n fourth feature maps corresponding to the n second feature maps, including the following steps 412 and 414.

412. And the computer equipment acquires the first distribution information based on the image decompression model and the n fourth feature maps.

413. And the computer equipment carries out second decompression on the third characteristic diagram based on the image decompression model and the distribution parameters of each characteristic value in the first characteristic diagram to obtain a sixth characteristic diagram.

414. And the computer equipment performs third decompression on the sixth feature map based on the image decompression model to obtain second image data, wherein the second image data is the same as the first image data.

It should be noted that the steps 412-414 are similar to the steps 312-314, and are not described herein again.

Based on the embodiment shown in fig. 4, a schematic structural diagram of an image processing model composed of an image compression model and an image decompression model is provided, as shown in fig. 5, taking n as 3 as an example, the image compression model includes an input transformation layer, a first code rate regulation layer, 3 quantization layers, 3 arithmetic coding layers, 2 super-prior coding layers, 2 arithmetic decoding layers, 2 super-prior decoding layers and 3 probability acquisition layers; the image decompression model comprises 3 arithmetic decoding layers, 2 super-prior decoding layers, 2 probability acquisition layers, a second code rate regulation layer and an output conversion layer.

Wherein, the 2-arithmetic decompression layer in the image compression model is the same as the arithmetic decompression layer in the image decompression model; 2 super-prior decoding layers in the image compression model are the same as 2 super-prior decoding layers in the image decompression model; the 2 probability acquisition layers in the image decompression model are the same as the 3 probability acquisition layers in the image compression model. In the embodiment of the present application, the image compression model includes a correspondence between compression ratios and scaling factors, where the correspondence includes multiple compression ratios and scaling factors corresponding to each compression ratio.

The process of obtaining compression information of the first image data based on the image compression model includes the following steps 1-5.

Step 1, performing dimensionality reduction processing on first image data based on an input conversion layer to obtain an eighth feature map; and inquiring the corresponding relation between the compression ratio and the scaling factor based on the target compression ratio to obtain a target scaling factor corresponding to the target compression ratio, scaling each characteristic value in the eighth characteristic graph based on the target scaling factor, and forming the scaled characteristic values into a first characteristic graph.

The input transform layer includes a plurality of convolution layers and a plurality of Normalization layers, for example, the Normalization layer is GDN (Generalized Normalization transform). By using the plurality of convolution layers and the plurality of normalization layers as input conversion layers and performing nonlinear conversion on the first image data, the expression capability of the input conversion layers can be improved, and the accuracy of the obtained eighth feature map can be ensured. As shown in fig. 6, the input conversion layer includes 3 convolutional layers and 3 normalization layers, the number of channels of the first convolutional layer is 3, and the number of channels of the 3 rd convolutional layer is 384. The size of the eighth feature map obtained based on the input transform layer is 384 × 32. The increase of the number of channels can improve the subsequent image data recovery quality, but can reduce the compression rate of the image data, and the number of channels of each convolution layer is determined based on experiments, so that the subsequent image data recovery quality can be ensured, and the compression rate of the image data can also be ensured.

And 2, performing 1 st second compression on the first characteristic diagram based on the 1 st super-first coding layer to obtain a 1 st second characteristic diagram.

As shown in fig. 7, the super-a coding layer includes a plurality of residual convolutional layers and non-local attention layers, and step 2 includes: performing dimensionality reduction processing on the first feature map based on the residual convolution layers to obtain a seventh feature map; and acquiring the similarity among a plurality of characteristic regions in the seventh characteristic diagram based on the non-local attention layer, updating the characteristic value in each characteristic region based on the similarity among the plurality of characteristic regions in the seventh characteristic diagram, and forming the plurality of updated characteristic regions into a 1 st second characteristic diagram.

The residual convolution layer is used for extracting the features of the first feature graph, and the super-first-check coding layer is used for solving the problem that the convolution kernel has a small receptive field, so that the global information modeling capability is improved, similar local features in the feature graph can be captured, and redundant information is reduced. As shown in FIG. 8, each residual convolutional layer includes two convolutional layers and one linear adjustment layer. As shown in fig. 9, the first image data 901 includes a plurality of similar image regions 902, and the feature map corresponding to the first image data includes a plurality of similar feature regions.

As shown in fig. 10, the process of acquiring the 1 st second feature map based on the non-local attention layer includes: based on the non-local attention layer, scaling the seventh feature map to obtain a first scaled feature map, reconstructing the first scaled feature map to obtain a second scaled feature map, wherein the feature size of the second scaled feature map is different from the feature size of the first scaled feature map, for any feature region in the seventh feature map, obtaining the similarity between the feature region and a plurality of feature regions in the second scaled feature map, and forming a similarity feature by the plurality of similarities; reconstructing the seventh feature map to obtain a third scaled feature map, wherein the feature sizes of the third scaled feature map are similar to those of the second scaled feature map, performing weighted fusion on a plurality of feature areas in the third feature map based on the similarity feature to obtain updated feature areas corresponding to the feature areas, obtaining updated feature areas corresponding to each feature area according to the above steps, and forming the updated feature areas into the 1 st second feature map.

Step 3, based on the 2 nd super-first coding layer, carrying out 2 nd second compression on the 1 st second feature map to obtain a 2 nd second feature map; based on the 2 nd quantizer, performing quantization processing on the feature values in the 2 nd second feature map; acquiring the occurrence probability of each characteristic value in the 2 nd second characteristic diagram based on the 3 rd probability acquisition layer, a plurality of characteristic values in the quantized 2 nd second characteristic diagram and the first probability distribution function, so as to obtain the distribution information corresponding to the 2 nd second characteristic diagram; and performing third compression on the quantized 2 nd second feature map based on the 3 rd arithmetic coding layer and the distribution information corresponding to the 2 nd second feature map to obtain a fourth feature map corresponding to the 2 nd second feature map. Performing fourth decompression on a fourth feature map corresponding to the 2 nd second feature map based on the 2 nd arithmetic decompression layer and the distribution information corresponding to the 2 nd second feature map to obtain a tenth feature map corresponding to the 2 nd second feature map, and performing fifth decompression on the tenth feature map corresponding to the 2 nd second feature map based on the 2 nd super-prior decoding layer to obtain a fifth feature map corresponding to the 2 nd second feature map; and determining the distribution information corresponding to the 1 st second feature map based on the 2 nd probability acquisition layer and the number of each feature value in the fifth feature map.

Step 4, based on the 2 nd quantizer, carrying out quantization processing on the 1 st second feature map; and performing third compression on the quantized 1 st second feature map based on the 2 nd arithmetic coding layer and the distribution information corresponding to the 1 st second feature map to obtain a fourth feature map corresponding to the 1 st second feature map. Performing fourth decompression on a fourth feature map corresponding to the 1 st second feature map based on the 1 st arithmetic decompression layer and the distribution information corresponding to the 1 st second feature map to obtain a tenth feature map corresponding to the 1 st second feature map, and performing fifth decompression on the tenth feature map based on the 1 st super-first decoding layer to obtain a fifth feature map corresponding to the 1 st second feature map; and determining distribution information corresponding to the first characteristic diagram based on the number of each characteristic value in the 1 st probability acquisition layer and the fifth characteristic diagram.

Step 5, based on the 1 st quantizer, carrying out quantization processing on the first characteristic diagram; and performing third compression on the quantized first feature map based on the 1 st arithmetic coding layer and the distribution information corresponding to the first feature map to obtain a third feature map, and determining the third feature map, a fourth feature map corresponding to the 2 second feature maps and the distribution information corresponding to the 2 nd second feature map as the compression information of the first image data.

As shown in fig. 11, when the feature sizes of the first feature map, the 1 st second feature map, and the 2 nd second feature map are gradually decreased, the complexity of the 3 probability acquisition layers is gradually decreased from the 1 st probability acquisition layer, the feature sizes of the obtained third feature map, the 1 st fourth feature map, and the 2 nd fourth feature map are gradually decreased, and the redundant information in the third feature map, the 1 st fourth feature map, and the 2 nd fourth feature map is gradually decreased.

The process of decompressing the compressed information of the first image data based on the image decompression model includes the following steps 1-3.

Step 1, performing fourth decompression on a fourth feature map corresponding to a 2 nd second feature map based on a 3 rd arithmetic coding layer and distribution information corresponding to the 2 nd second feature map to obtain a tenth feature map corresponding to the 2 nd second feature map, and performing fifth decompression on the tenth feature map corresponding to the 2 nd second feature map based on a 2 nd super-first decoding layer to obtain a fifth feature map corresponding to the 2 nd second feature map; and determining the distribution information corresponding to the 1 st second feature map based on the 2 nd probability acquisition layer and the number of each feature value in the fifth feature map.

Step 2, performing fourth decompression on a fourth feature map corresponding to the 1 st second feature map based on the 2 nd arithmetic decompression layer and distribution information corresponding to the 1 st second feature map to obtain a tenth feature map corresponding to the 1 st second feature map, and performing fifth decompression on the tenth feature map based on the 1 st super-first decoding layer to obtain a fifth feature map corresponding to the 1 st second feature map; and determining distribution information corresponding to the first characteristic diagram based on the number of each characteristic value in the 1 st probability acquisition layer and the fifth characteristic diagram.

Step 3, performing second decompression on the third characteristic diagram based on the 1 st arithmetic decompression layer and the distribution information corresponding to the first characteristic diagram to obtain a sixth characteristic diagram; scaling each characteristic value in the sixth characteristic diagram based on the second code rate regulation and control layer, and forming a ninth characteristic diagram by the scaled characteristic values; and performing dimensionality-up processing on the ninth feature map based on the output conversion layer to obtain second image data, wherein the second image data is the same as the first image data.

The output conversion layer comprises a plurality of deconvolution layers and a plurality of denormalization layers, wherein the plurality of deconvolution layers correspond to the plurality of convolution layers in the input conversion layer, and the plurality of denormalization layers correspond to the plurality of normalization layers in the input conversion layer. As shown in fig. 12, the output conversion layer includes 3 deconvolution layers and 3 denormalization layers.

In the above description, the independent image compression model and the independent image decompression model are taken as an example, but in another embodiment, the image compression model and the image decompression model constitute an image processing model, and as for the same layer included in the image compression model and the image decompression model, only the same layer may be included in the image processing model, and the layer may be called in both the image compression process and the image decompression process. For example, if 2 super-a-priori decoding layers in the image compression model are the same as 2 super-a-priori decoding layers in the image decompression model, the image processing model includes only 2 super-a-priori decoding layers, and the 2 super-a-priori decoding layers can be called during image compression and the 2 super-a-priori decoding layers can also be called during image decompression.

The image processing model comprises 2 layers of super-prior structures, wherein the first layer of super-prior structure consists of a 1 st super-prior coding layer and a 1 st super-prior decoding layer, the first layer of super-prior structure is a global layer, the probability estimation is carried out on the characteristic values in the first characteristic diagram, the occurrence probability of each characteristic value is determined, and the redundant information in the characteristic diagram is reduced. The second layer of the super-first structure is composed of a 2 nd super-first coding layer and a 2 nd super-first decoding layer, the second layer of the super-first structure is a local layer, the 1 st second feature diagram is further compressed, the correlation of local pixels is reduced, the size of the feature diagram is further reduced, the local detail optimization of the feature diagram is realized, and therefore redundant information in the feature diagram is further weakened.

Based on the embodiment shown in fig. 4, before processing the image data based on the image compression model and the image decompression model, the image compression model and the image decompression model need to be trained, and the training process is described in detail in the following embodiments.

Fig. 13 is a flowchart of an image data processing method provided in an embodiment of the present application, which is executed by a computer device, and as shown in fig. 13, the method includes the following steps.

1301. A computer device acquires sample image data.

Wherein the sample image data is any type of image data, for example, the sample image data is medical image data. For example, the sample image data is image data in png (one image format) format with a size of 1024 × 1024.

In one possible implementation, this step 1301 includes: and acquiring original sample image data, and preprocessing the original sample image data.

Wherein the pretreatment comprises normalization treatment or random block cutting. Because the size of the original sample image data may not be the same as the size adapted by the image compression model, the original sample image data needs to be randomly diced, so that the size of the obtained image data block is the same as the size adapted by the image compression model, and the image compression model and the image decompression model can be conveniently trained by taking the image data block as the sample image data.

1302. And the computer equipment compresses the sample image data based on the image compression model to obtain sample compression information corresponding to the sample image data.

1303. And decompressing the sample compression information by the computer equipment based on the image decompression model to obtain decompressed image data.

The

steps

1302 and 1303 are similar to the

steps

401 and 404, and are not described herein again.

1304. The computer device trains the image compression model and the image decompression model based on the sample image data and the decompressed image data.

The recovery quality of the image data can be determined through the difference between the sample image data and the decompressed image data, and the image compression model and the image decompression model are trained according to the recovery quality so as to ensure the compression performance of the image compression model and the image decompression model.

In one possible implementation, the sample compression information includes a plurality of sample feature maps corresponding to the sample image data, and then the step 1304 includes: the method comprises the steps of obtaining the compression code rate of sample image data based on a plurality of sample characteristic graphs, determining a loss value based on the difference between the sample image data and decompressed image data and the compression code rate, and adjusting an image compression model and an image decompression model based on the loss value.

The process of obtaining the sample third feature map is the same as the process of obtaining the third feature map in the embodiment shown in fig. 3, and the process of obtaining the sample fourth feature map is the same as the process of obtaining the fourth feature map in the embodiment shown in fig. 3, and therefore, the description thereof is omitted here.

Optionally, the loss value satisfies the following relationship:

wherein the content of the first and second substances,

the value of the loss is represented by,

indicating that the first image data corresponds to a compression code rate,

indicating the quality of the restored image of the decompressed image,

represents a regulatory factor;

a third feature map representing a sample corresponding to the sample image data,

a compression code rate representing the sample third feature map,

a 1 st sample fourth feature map representing the sample image data,

the compression code rate of the fourth feature map of the 1 st sample is shown,

a fourth feature map of the 2 nd sample corresponding to the sample image data,

the compression code rate of the fourth feature map of the 2 nd sample is represented;

representing the image data of the sample or samples,

which represents the decompression of the image data,

the root mean square error is indicated.

It should be noted that, in the above formula, n is only 2 for example, but in another embodiment, n can also be other values, which is not described herein again.

It should be noted that the above embodiment only describes one iteration of training the image compression model and the image decompression model, but in another embodiment, the image compression model and the image decompression model can be iteratively trained multiple times according to the above steps 501 and 504. In one possible implementation, in response to the loss value of the current iteration turn being less than the loss value threshold, stopping training the image compression model and the image decompression model; or stopping training the image compression model and the image decompression model in response to the iteration number reaching the number threshold.

For example, 2000 times of iterative training are performed on the image compression model and the image decompression model, Adam (an optimizer) is adopted as an optimizer for optimizing the image compression model and the image decompression model, the initial learning rate is 0.0001, after 1000 times of iterative training are performed on the image compression model and the image decompression model, the learning rate is reduced by 0.5 per iteration for 100 times, and the training of the image compression model and the image decompression model is stopped until the learning rate is 0.000001 which is the minimum value; or stopping training the image compression model and the image decompression model when the iteration number reaches 2000 times.

In a possible implementation manner, the image compression model and the image decompression model constitute the image processing model, and when performing iterative training on the image processing model, the image processing model is subjected to iterative training for a plurality of times on a partial image processing model except for a local layer accident in the image processing model, and then the image processing model is subjected to iterative training.

As shown in fig. 14, a part of the image processing model is iteratively trained for 1000 times, and then parameters of layers included in the part of the model are retained, and the whole image processing model is iteratively trained for multiple times by combining with a local layer, that is, the local layer is equivalent to the whole image processing model.

As shown in fig. 15, according to the image data processing method provided in the embodiment of the present application, the first image data is compressed based on the image compression model to obtain the compressed information of the first image data, and then the obtained compressed information can be stored or transmitted, so as to save storage resources or transmission resources. When the first image data needs to be viewed, the compressed information of the first image data is decompressed based on the image decompression model, and second image data which is the same as the first image data is obtained, so that the second image data can be viewed. For example, in a medical scene, the first image data and the second image data are medical image data, the first medical image data is compressed to obtain compressed information of the first medical image data, and then the compressed information of the first medical image data is obtained; when the first medical image data needs to be viewed, the compressed information of the first medical image data is decompressed, and second image data which is the same as the first medical image data is obtained. In the process, the storage resources can be saved, the corresponding image data can be checked, and the recovery quality of the image data is ensured.

Based on the image data processing method provided by the embodiment of the present application, compared with the image processing method in the related art, as shown in fig. 16 and 17, the horizontal axis represents the code rate, and the vertical axis represents the quality of the restored image, and is represented by PSNR (peak signal-to-noise ratio). The VVC (a compression method) in the related art and the method provided in the present application are significantly superior to JPEG (a compression method) in the related art. Under the condition of the same image quality, the file size of compressed information obtained by compression based on the method provided by the application is only 33 kB, and the file size of compressed information obtained by JPEG compression based on the related technology is 94 kB, namely, the method provided by the application can obtain smaller compressed information. As shown in table 1, the encoding and decoding time of the method provided in the embodiment of the present application is shorter, that is, the performance of the method provided in the present application is better.

TABLE 1

Compression method	Compressing picture size	Coding time (seconds)	Decoding time (seconds)
				This application	38300	1.27	3.16
VVC	48995	19.09	1.09

Fig. 18 is a schematic structural diagram of an image data processing apparatus according to an embodiment of the present application, and as shown in fig. 18, the apparatus includes:

a compression module 1801, configured to perform first compression on the first image data to obtain a first feature map;

the compressing module 1801 is further configured to perform n times of second compression on the first feature map to obtain n second feature maps, where n is a positive integer;

an obtaining module 1802, configured to obtain, based on the n second feature maps, first distribution information corresponding to the first feature map, where the first distribution information includes a distribution parameter of each feature value in the first feature map;

the compressing module 1801 is further configured to perform third compression on the first feature map based on the distribution parameter of each feature value in the first feature map to obtain a third feature map corresponding to the first image data, where the number of feature values included in the third feature map is smaller than the number of feature values included in the first feature map;

a determining module 1803, configured to determine the third feature map and the decompressed information associated with the first distribution information as compressed information of the first image data, where the decompressed information is used to decompress the third feature map to obtain the first image data.

In a possible implementation manner, n is an integer greater than 1, and the obtaining module 1802 is configured to obtain distribution information corresponding to the nth second feature map; acquiring distribution information corresponding to the (n-1) th second feature map based on the nth second feature map and the corresponding distribution information until the distribution information corresponding to the 1 st second feature map is obtained; and acquiring first distribution information based on the 1 st second feature map and the corresponding distribution information.

In another possible implementation manner, the obtaining module 1802 is configured to perform third compression on the nth second feature map based on a distribution parameter of each feature value in the nth second feature map to obtain a fourth feature map, where a feature value quantity included in the fourth feature map is smaller than a feature value quantity included in the second feature map; performing first decompression on the fourth feature map based on the distribution parameter of each feature value in the nth second feature map to obtain a fifth feature map; and determining the distribution information corresponding to the (n-1) th second feature map based on the number of each feature value in the fifth feature map.

In another possible implementation manner, the decompressed information includes n fourth feature maps corresponding to the n second feature maps; as shown in fig. 19, the apparatus further includes:

an obtaining module 1802, further configured to obtain first distribution information based on the n fourth feature maps;

a decompression module 1804, configured to perform second decompression on the third feature map based on the distribution parameter of each feature value in the first feature map, to obtain a sixth feature map;

the decompression module 1804 is further configured to perform third decompression on the sixth feature map to obtain second image data, where the second image data is the same as the first image data.

In another possible implementation manner, the distribution information corresponding to the nth second feature map includes an occurrence probability of each feature value in the nth second feature map; an obtaining module 1802, configured to obtain a probability distribution function, where the probability distribution function indicates a distribution of occurrence probabilities of a plurality of feature values in the nth second feature map; and acquiring the occurrence probability of each characteristic value in the nth second characteristic diagram based on the plurality of characteristic values in the nth second characteristic diagram and the probability distribution function.

In another possible implementation manner, the compressing module 1801 is configured to perform dimension reduction processing on the first feature map to obtain a seventh feature map, where the seventh feature map includes a plurality of feature areas; and updating the feature value in each feature region based on the similarity among the plurality of feature regions in the seventh feature map, and forming the plurality of updated feature regions into the 1 st second feature map.

In another possible implementation, as shown in fig. 19, the compressing module 1801 includes:

an obtaining unit 1811, configured to obtain a target scaling factor of the first image data, the target scaling factor being used for scaling the image data;

a processing unit 1812, configured to perform dimension reduction processing on the first image data to obtain an eighth feature map;

a scaling unit 1813, configured to scale each feature value in the eighth feature map based on the target scaling factor, and configure the scaled feature values into the first feature map.

In another possible implementation, the obtaining unit 1811 is configured to obtain a target compression rate of the first image data; and inquiring the corresponding relation between the compression ratio and the scaling factor based on the target compression ratio to obtain the target scaling factor corresponding to the target compression ratio.

In another possible implementation, the decompression information includes first distribution information; as shown in fig. 19, the apparatus further includes:

In another possible implementation, the decompression module 1804 is configured to obtain a target scaling factor of the first image data, where the target scaling factor is used for scaling the image data; scaling each characteristic value in the sixth characteristic diagram based on the target scaling factor, and forming a ninth characteristic diagram by the scaled characteristic values; and performing dimension-raising processing on the ninth feature map to obtain second image data.

In another possible implementation manner, first compression is performed on first image data to obtain a first feature map; performing n times of second compression on the first characteristic diagram to obtain n second characteristic diagrams; acquiring first distribution information corresponding to the first feature map based on the n second feature maps; performing third compression on the first feature map based on the distribution parameter of each feature value in the first feature map to obtain a third feature map corresponding to the first image data; the step of determining the third feature map and the decompressed information associated with the first distribution information as the compressed information of the first image data is performed based on an image compression model.

In another possible implementation manner, as shown in fig. 19, the apparatus further includes:

the decompression module 1804 is configured to perform second decompression on the third feature map based on the image decompression model and the distribution parameter of each feature value in the first feature map to obtain a sixth feature map;

the decompression module 1804 is further configured to perform third decompression on the sixth feature map based on the image decompression model to obtain second image data, where the second image data is the same as the first image data.

an obtaining module 1802 further configured to obtain sample image data;

the compression module 1801 is further configured to compress the sample image data based on the image compression model, so as to obtain sample compression information corresponding to the sample image data;

the decompression module 1804 is further configured to decompress the sample compression information based on the image decompression model to obtain decompressed image data;

a training module 1805, configured to train an image compression model and an image decompression model based on the sample image data and the decompressed image data.

It should be noted that: the image data processing apparatus provided in the above embodiment is only exemplified by the division of the above functional modules, and in practical applications, the above functions can be distributed to different functional modules according to needs, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the above described functions. In addition, the image data processing apparatus and the image data processing method provided by the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments, and are not described herein again.

The embodiment of the present application further provides a computer device, which includes a processor and a memory, where the memory stores at least one computer program, and the at least one computer program is loaded by the processor and executed to implement the operations performed by the image data processing method of the foregoing embodiment.

Optionally, the computer device is provided as a terminal. Fig. 20 is a block diagram illustrating a structure of a terminal 2000 according to an exemplary embodiment of the present application. The terminal 2000 may be a portable mobile terminal such as: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, motion video Experts compression standard Audio Layer 3), an MP4 player (Moving Picture Experts Group Audio Layer IV, motion video Experts compression standard Audio Layer 4), a notebook computer, or a desktop computer. Terminal 2000 may also be referred to by other names such as user equipment, portable terminal, laptop terminal, desktop terminal, and the like.

The terminal 2000 includes: a processor 2001 and a memory 2002.

The processor 2001 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so on. The processor 2001 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 2001 may also include a main processor and a coprocessor, the main processor being a processor for Processing data in an awake state, also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 2001 may be integrated with a GPU (Graphics Processing Unit) that is responsible for rendering and drawing the content that the display screen needs to display. In some embodiments, the processor 2001 may further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.

The memory 2002 may include one or more computer-readable storage media, which may be non-transitory. The memory 2002 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in the memory 2002 is used to store at least one computer program for execution by the processor 2001 to implement the image data processing methods provided by the method embodiments herein.

In some embodiments, terminal 2000 may further optionally include: a peripheral interface 2003 and at least one peripheral. The processor 2001, memory 2002 and peripheral interface 2003 may be connected by buses or signal lines. Various peripheral devices may be connected to peripheral interface 2003 through a bus, signal line, or circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 2004, a display 2005, a camera assembly 2006, an audio circuit 2007, a positioning assembly 2008, and a power supply 2009.

The peripheral interface 2003 may be used to connect at least one peripheral related to I/O (Input/Output) to the processor 2001 and the memory 2002. In some embodiments, the processor 2001, memory 2002 and peripheral interface 2003 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 2001, the memory 2002, and the peripheral interface 2003 may be implemented on separate chips or circuit boards, which is not limited by this embodiment.

The Radio Frequency circuit 2004 is used for receiving and transmitting RF (Radio Frequency) signals, also called electromagnetic signals. The radio frequency circuit 2004 communicates with a communication network and other communication devices via electromagnetic signals. The radio frequency circuit 2004 converts an electric signal into an electromagnetic signal to transmit, or converts a received electromagnetic signal into an electric signal. Optionally, the radio frequency circuit 2004 comprises: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and so forth. The radio frequency circuit 2004 may communicate with other terminals via at least one wireless communication protocol. The wireless communication protocols include, but are not limited to: the world wide web, metropolitan area networks, intranets, generations of mobile communication networks (2G, 3G, 4G, and 5G), Wireless local area networks, and/or WiFi (Wireless Fidelity) networks. In some embodiments, the rf circuit 2004 may further include NFC (Near Field Communication) related circuits, which are not limited in this application.

The display screen 2005 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display screen 2005 is a touch display screen, the display screen 2005 also has the ability to capture touch signals on or over the surface of the display screen 2005. The touch signal may be input to the processor 2001 as a control signal for processing. At this point, the display 2005 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, display 2005 may be one, provided on the front panel of terminal 2000; in other embodiments, the display screens 2005 can be at least two, respectively disposed on different surfaces of the terminal 2000 or in a folded design; in other embodiments, display 2005 may be a flexible display disposed on a curved surface or a folded surface of terminal 2000. Even more, the display screen 2005 can be arranged in a non-rectangular irregular figure, i.e. a shaped screen. The Display screen 2005 can be made of a material such as an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), and the like.

Camera assembly 2006 is used to capture images or video. Optionally, camera assembly 2006 includes a front camera and a rear camera. The front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, the number of the rear cameras is at least two, and each rear camera is any one of a main camera, a depth-of-field camera, a wide-angle camera and a telephoto camera, so that the main camera and the depth-of-field camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize panoramic shooting and VR (Virtual Reality) shooting functions or other fusion shooting functions. In some embodiments, camera assembly 2006 may also include a flash. The flash lamp can be a monochrome temperature flash lamp or a bicolor temperature flash lamp. The double-color-temperature flash lamp is a combination of a warm-light flash lamp and a cold-light flash lamp, and can be used for light compensation at different color temperatures.

The audio circuitry 2007 may include a microphone and a speaker. The microphone is used for collecting sound waves of a user and the environment, converting the sound waves into electric signals, and inputting the electric signals to the processor 2001 for processing or inputting the electric signals to the radio frequency circuit 2004 so as to realize voice communication. For the purpose of stereo sound collection or noise reduction, a plurality of microphones may be provided at different positions of the terminal 2000. The microphone may also be an array microphone or an omni-directional pick-up microphone. The speaker is used to convert electrical signals from the processor 2001 or the radio frequency circuit 2004 into sound waves. The loudspeaker can be a traditional film loudspeaker or a piezoelectric ceramic loudspeaker. When the speaker is a piezoelectric ceramic speaker, the speaker can be used for purposes such as converting an electric signal into a sound wave audible to a human being, or converting an electric signal into a sound wave inaudible to a human being to measure a distance. In some embodiments, the audio circuitry 2007 may also include a headphone jack.

The positioning component 2008 is configured to locate a current geographic Location of the terminal 2000 to implement navigation or LBS (Location Based Service). The Positioning component 2008 may be a Positioning component based on a Global Positioning System (GPS) in the united states, a beidou System in china, or a galileo System in russia.

Power supply 2009 is used to power the various components in terminal 2000. The power supply 2009 may be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 2009 includes a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery may also be used to support fast charge technology.

In some embodiments, terminal 2000 also includes one or more sensors 2010. The one or more sensors 2010 include, but are not limited to: acceleration sensor 2011, gyro sensor 2012, pressure sensor 2013, fingerprint sensor 2014, optical sensor 2015, and proximity sensor 2016.

The acceleration sensor 2011 can detect the magnitude of acceleration on three coordinate axes of the coordinate system established with the terminal 2000. For example, the acceleration sensor 2011 may be used to detect components of the gravitational acceleration in three coordinate axes. The processor 2001 may control the display screen 2005 to display a user interface in a landscape view or a portrait view according to the gravitational acceleration signal acquired by the acceleration sensor 2011. The acceleration sensor 2011 may also be used for acquisition of motion data of a game or a user.

The gyroscope sensor 2012 can detect the body direction and the rotation angle of the terminal 2000, and the gyroscope sensor 2012 and the acceleration sensor 2011 can cooperate to acquire the 3D motion of the user on the terminal 2000. The processor 2001 may implement the following functions according to the data collected by the gyro sensor 2012: motion sensing (such as changing the UI according to a user's tilting operation), image stabilization at the time of photographing, game control, and inertial navigation.

Pressure sensors 2013 may be disposed on the side frames of terminal 2000 and/or underlying display screen 2005. When the pressure sensor 2013 is disposed on the side frame of the terminal 2000, the holding signal of the user to the terminal 2000 can be detected, and the processor 2001 performs left-right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 2013. When the pressure sensor 2013 is disposed at the lower layer of the display screen 2005, the processor 2001 controls the operability control on the UI interface according to the pressure operation of the user on the display screen 2005. The operability control comprises at least one of a button control, a scroll bar control, an icon control and a menu control.

The fingerprint sensor 2014 is used for collecting fingerprints of the user, and the processor 2001 identifies the identity of the user according to the fingerprints collected by the fingerprint sensor 2014, or the fingerprint sensor 2014 identifies the identity of the user according to the collected fingerprints. Upon identifying that the user's identity is a trusted identity, the processor 2001 authorizes the user to perform relevant sensitive operations including unlocking the screen, viewing encrypted information, downloading software, paying for and changing settings, etc. The fingerprint sensor 2014 may be disposed at a front, rear, or side of the terminal 2000. When a physical key or vendor Logo is provided on the terminal 2000, the fingerprint sensor 2014 may be integrated with the physical key or vendor Logo.

The optical sensor 2015 is used to collect ambient light intensity. In one embodiment, the processor 2001 may control the display brightness of the display screen 2005 according to the ambient light intensity collected by the optical sensor 2015. Specifically, when the ambient light intensity is high, the display luminance of the display screen 2005 is increased; when the ambient light intensity is low, the display luminance of the display screen 2005 is adjusted down. In another embodiment, the processor 2001 may also dynamically adjust the shooting parameters of the camera assembly 2006 according to the ambient light intensity collected by the optical sensor 2015.

A proximity sensor 2016, also known as a distance sensor, is disposed on a front panel of the terminal 2000. The proximity sensor 2016 is used to collect a distance between a user and a front surface of the terminal 2000. In one embodiment, the processor 2001 controls the display 2005 to switch from the bright screen state to the dark screen state when the proximity sensor 2016 detects that the distance between the user and the front surface of the terminal 2000 is gradually reduced; when the proximity sensor 2016 detects that the distance between the user and the front surface of the terminal 2000 is gradually increasing, the display screen 2005 is controlled by the processor 2001 to switch from a rest screen state to a light screen state.

Those skilled in the art will appreciate that the configuration shown in fig. 20 is not intended to be limiting of terminal 2000 and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components may be used.

Optionally, the computer device is provided as a server. Fig. 21 is a schematic structural diagram of a server 2100 according to an embodiment of the present application, where the server 2100 may generate a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 2101 and one or more memories 2102, where the memory 2102 stores at least one computer program, and the at least one computer program is loaded and executed by the processors 2101 to implement the methods provided by the foregoing method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input/output interface, so as to perform input/output, and the server may also include other components for implementing the functions of the device, which are not described herein again.

The embodiment of the present application further provides a computer-readable storage medium, in which at least one computer program is stored, and the at least one computer program is loaded and executed by a processor to implement the operations performed by the image data processing method of the above embodiment.

Embodiments of the present application further provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the computer program implements the operations performed by the image data processing method according to the foregoing embodiments.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The above description is only an alternative embodiment of the present application and should not be construed as limiting the present application, and any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method of image data processing, the method comprising:

2. The method according to claim 1, wherein n is an integer greater than 1, and the obtaining first distribution information corresponding to the first feature map based on the n second feature maps comprises:

acquiring distribution information corresponding to the nth second characteristic diagram;

acquiring distribution information corresponding to the (n-1) th second feature map based on the nth second feature map and the corresponding distribution information until the distribution information corresponding to the 1 st second feature map is obtained;

and acquiring the first distribution information based on the 1 st second feature map and the corresponding distribution information.

3. The method according to claim 2, wherein the obtaining the distribution information corresponding to the (n-1) th second feature map based on the nth second feature map and the corresponding distribution information comprises:

performing third compression on the nth second feature map based on the distribution parameter of each feature value in the nth second feature map to obtain a fourth feature map, wherein the number of feature values contained in the fourth feature map is smaller than the number of feature values contained in the second feature map;

performing first decompression on the fourth feature map based on the distribution parameter of each feature value in the nth second feature map to obtain a fifth feature map;

and determining distribution information corresponding to the (n-1) th second feature map based on the number of each feature value in the fifth feature map.

4. The method according to claim 3, wherein the decompressed information includes n fourth feature maps corresponding to the n second feature maps; after determining the third feature map and the decompression information associated with the first distribution information as compression information of the first image data, the method further includes:

acquiring the first distribution information based on the n fourth feature maps;

performing second decompression on the third feature map based on the distribution parameter of each feature value in the first feature map to obtain a sixth feature map;

and performing third decompression on the sixth feature map to obtain second image data, wherein the second image data is the same as the first image data.

5. The method according to claim 2, wherein the distribution information corresponding to the nth second feature map comprises a probability of occurrence of each feature value in the nth second feature map; the obtaining of the distribution information corresponding to the nth second feature map includes:

obtaining a probability distribution function, wherein the probability distribution function indicates the distribution situation of the occurrence probability of a plurality of characteristic values in the nth second characteristic diagram;

and acquiring the occurrence probability of each characteristic value in the nth second characteristic diagram based on each characteristic value in the nth second characteristic diagram and the probability distribution function.

6. The method according to claim 1, wherein the second compressing the first feature map n times to obtain n second feature maps comprises:

performing dimension reduction processing on the first feature map to obtain a seventh feature map, wherein the seventh feature map comprises a plurality of feature areas;

and updating the feature value in each feature region based on the similarity among the plurality of feature regions in the seventh feature map, and forming the plurality of updated feature regions into a 1 st second feature map.

7. The method of claim 1, wherein the first compressing the first image data to obtain a first feature map comprises:

acquiring a target scaling factor of the first image data, wherein the target scaling factor is used for scaling the image data;

performing dimensionality reduction on the first image data to obtain an eighth feature map;

and scaling each characteristic value in the eighth characteristic diagram based on the target scaling factor, and forming the scaled characteristic values into the first characteristic diagram.

8. The method of claim 7, wherein the obtaining the target scaling factor for the first image data comprises:

acquiring a target compression rate of the first image data;

and inquiring the corresponding relation between the compression ratio and the scaling factor based on the target compression ratio to obtain the target scaling factor corresponding to the target compression ratio.

9. The method of any of claims 1-3 and 5-8, wherein the decompression information comprises the first distribution information; after determining the third feature map and the decompression information associated with the first distribution information as compression information of the first image data, the method further includes:

10. The method according to claim 9, wherein the third decompressing the sixth feature map to obtain the second image data comprises:

scaling each feature value in the sixth feature map based on the target scaling factor, and forming a ninth feature map by the scaled feature values;

and performing dimensionality-raising processing on the ninth feature map to obtain the second image data.

11. The method according to claim 1, wherein the first compression is performed on the first image data to obtain a first feature map; performing n times of second compression on the first characteristic diagram to obtain n second characteristic diagrams; acquiring first distribution information corresponding to the first feature map based on the n second feature maps; performing third compression on the first feature map based on the distribution parameter of each feature value in the first feature map to obtain a third feature map corresponding to the first image data; the step of determining the third feature map and the decompressed information associated with the first distribution information as the compressed information of the first image data is performed based on an image compression model.

12. The method according to claim 11, wherein after determining the third feature map and the decompressed information associated with the first distribution information as compressed information of the first image data, the method further comprises:

performing second decompression on the third feature map based on an image decompression model and the distribution parameters of each feature value in the first feature map to obtain a sixth feature map;

and performing third decompression on the sixth feature map based on the image decompression model to obtain second image data, wherein the second image data is the same as the first image data.

13. The method of claim 12, further comprising:

acquiring sample image data;

compressing the sample image data based on the image compression model to obtain sample compression information corresponding to the sample image data;

decompressing the sample compression information based on the image decompression model to obtain decompressed image data;

and training the image compression model and the image decompression model based on the sample image data and the decompressed image data.

14. An image data processing apparatus, characterized in that the apparatus comprises:

15. A computer device, characterized in that the computer device comprises a processor and a memory, in which at least one computer program is stored, which is loaded and executed by the processor to perform the operations performed by the image data processing method according to any one of claims 1 to 13.

16. A computer-readable storage medium, having stored therein at least one computer program, which is loaded and executed by a processor to perform the operations performed by the image data processing method of any one of claims 1 to 13.