CN114519750A - Face image compression method and system - Google Patents

Abstract

The embodiment of the application discloses an image compression method and system, wherein the method comprises the following steps: inputting a style encoder and a content encoder from an original face image to extract style features and structural features; respectively carrying out probability estimation and entropy coding to obtain a style coding bit stream corresponding to the style characteristics and a structure coding bit stream corresponding to the structure characteristics, and inputting the style coding bit stream and the structure coding bit stream into a decoder and a multitask analysis network; the decoder reconstructs the images of the style coding bit stream and the structure coding bit stream and outputs a reconstructed image; and the multitask analysis network carries out semantic understanding analysis on the style coding bit stream and the structure coding bit stream and outputs semantic information of the image. Under the condition of extremely high compression efficiency, the high subjective visual evaluation quality of the reconstructed image is maintained, and the decoding time and the resource overhead are saved.

Description

A face image compression method and system

technical field

The embodiments of the present application relate to the technical field of digital signal processing, and in particular, to a method and system for compressing a face image.

Background technique

The image/video compression method based on neural network has developed rapidly in recent years, and the quality of its compressed and reconstructed image has surpassed the new-generation video coding standard VVC in objective indicators such as PSNR and MS-SSIM. The compression framework based on the generative model can greatly improve the compression ratio without affecting the relevant evaluation indicators that directly reflect the viewing effect of the human eye.

At present, in various researches, the end-to-end image coding based on neural network faces two major problems: one is the limited representation mechanism for the input original image signal, and the lack of support for the currently widely used computer vision processing tasks; the other is the signal reception. The terminal resources are limited, which is not enough to support the neural network model with a large number of parameters.

SUMMARY OF THE INVENTION

To this end, the embodiments of the present application provide an image compression method and system, which can maintain high subjective visual evaluation quality of reconstructed images and save decoding time and resource overhead under the condition of extremely high compression efficiency.

In order to achieve the above purpose, the embodiments of the present application provide the following technical solutions:

According to a first aspect of the embodiments of the present application, a method for compressing a face image is provided, the method comprising:

Input style encoder and content encoder from raw face images to extract style features and structural features;

Probability estimation and entropy coding are performed respectively to obtain the style coding bitstream corresponding to the style feature and the structure coding bitstream corresponding to the structure feature, which are input to the decoder and the multi-task analysis network;

The decoder reconstructs the images of the style-encoded bitstream and the structure-encoded bitstream, and outputs a reconstructed image; the multi-task analysis network performs semantic understanding and analysis on the style-encoded bitstream and the structure-encoded bitstream, and outputs the semantic information of the image.

Optionally, probability estimation and entropy coding are respectively performed to obtain a style coded bitstream corresponding to the style feature and a structure coded bitstream corresponding to the structural feature, including:

Quantize the style features and structural features respectively to obtain the quantized style features and structural features;

Entropy coding is performed on the quantized style features and structural features according to the probability estimation results calculated by the probability estimation model, respectively, to obtain a style coded bitstream corresponding to the style feature and a structure coded bitstream corresponding to the structural feature.

Optionally, the decoder reconstructs the images of the style-encoded bitstream and the structure-encoded bitstream, and outputs a reconstructed image, including:

The style-encoded bitstream and the structure-encoded bitstream are fused through the fusion module in the decoder, and processed through a multi-layer perceptual MLP to learn the mean and variance of the convolutional layers in the residual block;

Perform the image compression task on the fused encoded bit stream through the generator in the decoder to obtain the compressed reconstructed image;

The compressed reconstructed image is discriminated by the discriminator to obtain a loss optimization function; the generator is trained according to the loss optimization function.

Optionally, the loss optimization function is according to the following formula:

为重建图像，

为量化后的风格特征和结构特征，p为概率估计结果，λ、β为超参数。Among them, D is the discriminator, E is the content encoder and style encoder, G is the generator, P is the probability estimation model, x is the original face image,

To reconstruct the image,

are the quantized style features and structural features, p is the probability estimation result, and λ and β are hyperparameters.

Optionally, the multi-task analysis network performs semantic understanding analysis on the style-coded bitstream and the structure-coded bitstream, and outputs semantic information of the image, including:

Input the style coding bit stream and the structure coding bit stream into the multi-task analysis network, fuse the encoded bit stream through the fusion module, and then train the multi-task analysis network according to the multi-task analysis loss function to obtain the corresponding task results, As the output of the semantic information of the image.

Optionally, the multi-task analysis loss function L _multi is calculated according to the following formula:

L _multi =λ _cls l _cls +λ _seg l _seg

Among them, l _cls and l _seg are the loss functions of the classification task and the segmentation task, respectively, and λ _cls and λ _seg are the corresponding weight hyperparameters.

Optionally, the method further includes: by optimizing the multi-task analysis loss function, training parameters in the multi-task analysis model to obtain a global optimal solution; wherein the multi-task analysis model is applied in training. The total loss function follows the formula:

L=L _EGP +L _D +γL _multi

where γ is a hyperparameter.

According to a second aspect of the embodiments of the present application, a system for compressing a face image is provided, and the system includes:

Feature extraction module for inputting style encoder and content encoder from raw face images to extract style features and structural features;

The encoding module is used to perform probability estimation and entropy encoding respectively to obtain the style encoding bitstream corresponding to the style feature and the structure encoding bitstream corresponding to the structure feature, and input the decoder and the multi-task analysis network;

a compression decoding module, used for the decoder to reconstruct the images of the style-encoded bitstream and the structure-encoded bitstream, and output a reconstructed image;

The multi-task analysis module is used for the multi-task analysis network to perform semantic understanding and analysis on the style-coded bit stream and the structure-coded bit stream, and output the semantic information of the image.

According to a third aspect of the embodiments of the present application, an electronic device is provided, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor running the A computer program is executed to implement the method described in the first aspect above.

According to a fourth aspect of the embodiments of the present application, there is provided a computer-readable storage medium having computer-readable instructions stored thereon, where the computer-readable instructions can be executed by a processor to implement the method described in the first aspect above .

To sum up, the embodiments of the present application provide an image compression method and system, by inputting a style encoder and a content encoder from an original face image to extract style features and structural features; respectively performing probability estimation and entropy encoding, The style encoding bitstream corresponding to the style feature and the structure encoding bitstream corresponding to the structure feature are obtained, and input to the decoder and the multi-task analysis network; the decoder reconstructs the images of the style encoding bitstream and the structure encoding bitstream, and outputs the reconstructed image The multi-task analysis network performs semantic understanding analysis on the style-coded bit stream and the structure-coded bit stream, and outputs the semantic information of the image. In the case of extremely high compression efficiency, the high subjective visual evaluation quality of the reconstructed image is maintained, and decoding time and resource overhead are saved.

Description of drawings

In order to illustrate the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that are required to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only exemplary, and for those of ordinary skill in the art, other implementation drawings can also be obtained according to the extension of the drawings provided without creative efforts.

The structures, proportions, sizes, etc. shown in this specification are only used to cooperate with the contents disclosed in the specification, so as to be understood and read by those who are familiar with the technology, and are not used to limit the conditions for the implementation of the present invention, so there is no technical The substantive meaning, any modification of the structure, the change of the proportional relationship or the adjustment of the size, without affecting the effect that the present invention can produce and the purpose that can be achieved, should still fall within the technical content disclosed in the present invention. within the scope of coverage.

1 is a schematic flowchart of a method for compressing a face image according to an embodiment of the present application;

FIG. 2 is a flow chart of the technical solution provided by the embodiment of the present application;

3 is a multi-task analysis network structure diagram provided by an embodiment of the present application;

FIG. 4 is a schematic diagram showing a quality indicator of a compressed and reconstructed image provided by an embodiment of the present application;

FIG. 5 provides a face image compression system according to an embodiment of the present application;

FIG. 6 shows a schematic structural diagram of an electronic device provided by an embodiment of the present application;

FIG. 7 shows a schematic diagram of a computer-readable storage medium provided by an embodiment of the present application.

Detailed ways

The embodiments of the present invention are described below by specific specific embodiments. Those who are familiar with the technology can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the present invention. , not all examples. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Under the premise of the same bit rate, the image compression method provided by the embodiment of the present application is ahead of the prior art in terms of various evaluation indicators that simulate the visual perception of the human eye; Compared with the original image, it loses little analysis accuracy, but brings a large amount of storage resource overhead and network transmission bandwidth at the decoding end.

FIG. 1 shows an image compression method provided by an embodiment of the present application, and the method includes the following steps:

Step 101: input a style encoder and a content encoder from the original face image to extract style features and structural features;

Step 102: respectively perform probability estimation and entropy coding to obtain the style coding bitstream corresponding to the style feature and the structure coding bitstream corresponding to the structure feature, and input the decoder and the multi-task analysis network;

Step 103: The decoder reconstructs the images of the style-encoded bitstream and the structure-encoded bitstream, and outputs a reconstructed image; the multi-task analysis network performs semantic understanding analysis on the style-encoded bitstream and the structure-encoded bitstream, and outputs the image semantic information.

In a possible implementation, in step 102, probability estimation and entropy coding are performed respectively to obtain a style-coded bitstream corresponding to the style feature and a structure-coded bitstream corresponding to the structural feature, including:

Quantize the style features and structural features respectively to obtain the quantized style features and structural features; entropy encode the quantized style features and structural features according to the probability estimation results calculated by the probability estimation model, and obtain the style codes corresponding to the style features. The bitstream and the structure-encoded bitstream corresponding to the structure feature.

In a possible implementation manner, in step 103, the decoder reconstructs the images of the style-encoded bitstream and the structure-encoded bitstream, and outputs a reconstructed image, including:

The style-encoded bitstream and the structure-encoded bitstream are fused by the fusion module in the decoder, and processed through a multi-layer perceptual MLP to learn the mean and variance of the convolutional layers in the residual block; the generator in the decoder is used to fuse the fusion The encoded bit stream performs an image compression task to obtain a compressed reconstructed image; the compressed reconstructed image is discriminated by the discriminator to obtain a loss optimization function; the generator is trained according to the loss optimization function.

In a possible implementation, the loss optimization function is according to the following formulas (1) and (2):

为重建图像，

为量化后的风格特征和结构特征，p为概率估计结果，λ、β为超参数。Among them, D is the discriminator, E is the content encoder and style encoder, G is the generator, P is the probability estimation model, x is the original face image,

To reconstruct the image,

are the quantized style features and structural features, p is the probability estimation result, and λ and β are hyperparameters.

In a possible implementation manner, in step 103, the multi-task analysis network performs semantic understanding analysis on the style-coded bitstream and the structure-coded bitstream, and outputs semantic information of the image, including:

Input the style coding bit stream and the structure coding bit stream into the multi-task analysis network, fuse the encoded bit stream through the fusion module, and then train the multi-task analysis network according to the multi-task analysis loss function to obtain the corresponding task results, As the output of the semantic information of the image.

In the compression framework, the intermediate compression results are used directly as input for various analysis tasks without decoding to obtain the semantic information of the original image signal. Multitasking here refers to a variety of visual-semantic-related tasks, such as recognition, detection, segmentation, etc.

In a possible implementation manner, the multi-task analysis loss function L _multi is calculated according to the following formula (3):

L _multi =λ _cls l _cls +λ _seg l _seg Formula (3)

Among them, l _cls and l _seg are the loss functions of the classification task and the segmentation task, respectively, and λ _cls and λ _seg are the corresponding weight hyperparameters.

In a possible implementation manner, the method further includes: by optimizing the multi-task analysis loss function, training parameters in the multi-task analysis model to obtain a global optimal solution; wherein the multi-task analysis model The total loss function applied in training follows the formula (4):

L=L _EGP +L _D + _γL multiformula (4)

where γ is a hyperparameter.

The face image compression method provided by the embodiment of the present application will apply the current construction idea of the generation model for the hierarchical representation of the picture signal, and map the original input picture into style features and structural features, so as to further carry out the corresponding feature expression distribution. Quantization and entropy coding. In addition, the embodiment of the present application will directly take compressed data in the compressed domain as the input of various subsequent visual tasks. Since compressed data is an efficient and compact data expression form, the embodiment of the present application proposes a multi-task analysis network model to obtain semantic information of the original image from it at a low computational cost without decompression. At the same time, the rate-distortion loss function and the machine analysis objective loss function are jointly optimized to obtain common solutions for image compression tasks and various machine vision analysis tasks.

FIG. 2 shows a model architecture diagram applicable to the face image compression method for multi-visual analysis tasks proposed by the embodiment of the present application, which mainly includes a compression model and a multi-task analysis model.

The compression model mainly consists of four main parts: encoder, generator, discriminator and probability estimation model. Encoders include content encoders and style encoders.

其中Q为量化函数。在此之后，根据概率估计模型所给定的概率估计结果将

使用熵编码方法无损编码为比特流。在解码器端，有

其中

为重构图像。Given an original image x, the encoder first encodes it as y=E(x), and then quantizes it

where Q is the quantization function. After that, the probability estimation result given by the probability estimation model will be

Lossless encoding into a bitstream using an entropy encoding method. On the decoder side, there is

in

to reconstruct the image.

The original image to be compressed is mapped to the visual semantic feature domain, and then decomposed into style features and structural features. The style features and structural features obtained by decomposing are used for probability distribution fitting using independent probability estimation methods, and the entropy value of the probability obtained by fitting is used as the code rate value obtained by actual coding.

和

The original image signal input is decomposed into content features and style features at the semantic feature level. Two independent encoders are employed to encode the input image x into a content representation and _a style representation, E1 and _E2 , respectively. The content feature and style feature are y ₁ =E ₁ (x) and y ₂ =E ₂ (x), respectively. Then use the quantization function Q for quantization to get

and

和

使用熵编码方法无损编码为编码比特流，将拟合得到概率的熵值作为实际编码得到的码率值。Since the features obtained by decoupling have independent data distributions, a probability estimation model P is set for each layer, namely p ₁ (y ₁ |z ₁ ) and p ₂ (y ₂ |z ₂ ). According to the probability estimation results p ₁ (y ₁ |z ₁ ) and p ₂ (y ₂ |z ₂ ) given by the probability estimation model, the

and

The entropy encoding method is used to encode the bit stream losslessly, and the entropy value of the probability obtained by fitting is used as the code rate value obtained by the actual encoding.

The entropy coding method is used to further compress the features. The input of the entropy coding requires the probability distribution of the elements in the feature, so the probability estimation model is used to estimate the probability distribution of each element that appears. Entropy coding methods include, but are not limited to, Huffman coding, arithmetic coding, and context-based binarization coding.

The abbreviation of the encoded bit stream here is the code stream. The feature is a binary file obtained by compressing the feature by the entropy encoder. The code stream 1 and the code stream 2 in Figure 2 are the entropy coding results of the content feature and the style feature, respectively.

The embodiment of the present application also proposes an encoder based on semantic layers, and the decoder part mainly includes a generator and a discriminator. In this embodiment of the present application, a fusion module is designed at the decoding end. The fusion module is based on the Adaptive Instance Normalization (AdaIN) residual block, in which the content features are directly input as the AdaIN module; the style features are processed by multi-layer perceptron (MLP) as the residual in the AdaIN module. The mean and variance of the convolutional base layer of the difference network to learn the mean and variance of the convolutional layers in the residual block.

其中

为重构图像。On the decoder side, there is also

in

to reconstruct the image.

The embodiments of the present application optimize the compression model according to the rate-distortion theory. There are two reference metrics for distortion loss metrics: pixel-level mean average error (MAE) loss d _MAE and SSIM loss d _SSIM for overall structure evaluation. At the same time, considering the subjective perceptual quality, a perceptual distortion loss _dp is adopted to simulate the human eye perceptual characteristics through the high-order features extracted from the pre-trained convolutional neural network VGG16.

The method for calculating the total distortion loss by the compression model proposed in the embodiment of the present application is according to the following formula (5):

d=λ _MAE d _MAE +λ _SSIM d _SSIM +λ _p d _p Formula (5)

where λ _MAE , λ _SSIM and λ _p are hyperparameters. d _MAE is the pixel-level mean average error (MAE) loss, d _SSIM is the SSIM loss for overall structure evaluation, and d _p is the perceptual distortion loss.

Based on the definition of this loss function, the modules corresponding to the embodiments of the present application (refer to the respective partial modules in FIG. 2 corresponding to the L subscript, E refers to the content and style encoder, G refers to the generator, and P refers to the probability estimation model, D refers to the discriminator) The loss optimization function can be defined as Equation (1) and Equation (2).

In the embodiment of the present application, the expected code stream transmission code rate is changed by changing the number of feature channels when training the compression model, so as to obtain a model with a very high compression ratio more effectively.

During the training process, the method of adding uniform noise is used to avoid that the gradient of the quantization operation is not steerable during backpropagation.

The probability distribution fitting methods used during training include, but are not limited to, Gaussian models and Gaussian mixture models.

FIG. 3 shows a schematic diagram of a multi-task analysis network structure provided by an embodiment of the present application. In terms of the multi-task analysis network model, the compressed-domain multi-task analysis network refers to the classification task and the semantic segmentation task, and adopts the corresponding network structure design and corresponding loss function. ASPP stands for Atrous Spatial Pyramid Pooling.

Input the style features and structural features into the multi-task analysis network, first through the fusion module, and then design task network branches for different task characteristics, and output corresponding results according to different task settings.

In the examples of this application, classification tasks and segmentation tasks, which have been widely studied, are used as examples, because they are the most representative tasks in visual analysis.

The balance between tasks is controlled by setting hyperparameters, so the multi-task analysis loss L _multi can be expressed as formula (6):

L _multi =λ _cls l _cls +λ _seg l _seg Formula (6)

Among them, l _cls and l _seg are the loss functions of the classification task and the segmentation task, respectively, and λ _cls and λ _seg are their corresponding weight hyperparameters.

To further explore the relationship between compression and visual analysis, two methods of training the analysis network are validated: individual training and joint training.

The training process of separate training involves a small amount of parameters and is easier to train. In the separate training setting, the embodiment of the present application fixes the compression model, and only trains the multi-task analysis model.

The training process of joint training is to directly train the relationship between compression, reconstruction and analysis, which makes it easier to find the global optimum and achieve better analysis results.

For the joint training method, which jointly optimizes the compression model and the multi-task analysis model, the sum of the total loss functions is defined as formula (4), where the hyperparameter γ is used to balance the weight between the compression task and the visual analysis task.

In the embodiments of the present application, the rate-distortion loss function and the machine analysis target loss function are jointly optimized to obtain common solutions for image compression tasks and various machine vision analysis tasks.

The effects of the reconstruction map corresponding to the compression model in the embodiment of the present application on the four perception-based image evaluation indicators are shown in Figure 4. It can be seen that various image quality indicators greatly exceed the existing traditional encoding. Methods and Deep Learning-Based End-to-End Compression Methods.

In the face image compression method provided by the embodiment of the present application, the technology decouples the original image signal into style features and structural features in the visual feature domain, applies independent probability estimation models to respectively perform distribution fitting on the features, and then uses entropy The encoder obtains the encoded bit stream; in order to solve the problem of directly analyzing the compressed data to obtain the semantic information, a multi-task analysis model is proposed.

The method proposed in the embodiment of the present application can maintain the high subjective visual evaluation quality of the reconstructed image under the condition of extremely high compression efficiency, and at the decoding end, the code stream can be received without decoding, and the multi-task analysis network acting on the compressed data can be used. Obtaining the semantic information of the original image saves decoding time and resource overhead.

To sum up, the embodiments of the present application provide an image compression method, by inputting a style encoder and a content encoder from an original face image to extract style features and structural features; respectively performing probability estimation and entropy encoding to obtain style The style coding bitstream corresponding to the feature and the structure coding bitstream corresponding to the structure feature are input to the decoder and the multi-task analysis network; the decoder reconstructs the images of the style coding bitstream and the structure coding bitstream, and outputs the reconstructed image; The task analysis network performs semantic understanding analysis on the style-encoded bitstream and the structure-encoded bitstream, and outputs the semantic information of the image. In the case of extremely high compression efficiency, the high subjective visual evaluation quality of the reconstructed image is maintained, and decoding time and resource overhead are saved.

Based on the same technical concept, an embodiment of the present application also provides a face image compression system, as shown in FIG. 5 , the system includes:

A feature extraction module 501, for inputting a style encoder and a content encoder from the original face image to extract style features and structural features;

The coding module 502 is used to perform probability estimation and entropy coding respectively, obtain the style coding bitstream corresponding to the style feature and the structure coding bitstream corresponding to the structural feature, and input the decoder and the multi-task analysis network;

Compression and decoding module 503, used for the decoder to reconstruct the images of the style coded bit stream and the structure coded bit stream, and output the reconstructed image;

The multi-task analysis module 504 is used for the multi-task analysis network to perform semantic understanding and analysis on the style-coded bit stream and the structure-coded bit stream, and output the semantic information of the image.

The embodiments of the present application further provide an electronic device corresponding to the method provided by the foregoing embodiments. Please refer to FIG. 6 , which shows a schematic diagram of an electronic device provided by some embodiments of the present application. The electronic device 20 may include: a processor 200, a memory 201, a bus 202 and a communication interface 203, the processor 200, the communication interface 203 and the memory 201 are connected through the bus 202; A computer program running on the processor 200, when the processor 200 runs the computer program, the method provided by any of the foregoing embodiments of the present application is executed.

The memory 201 may include a high-speed random access memory (RAM: Random Access Memory), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory. The communication connection between the network element of the system and at least one other network element is realized through at least one physical port 203 (which may be wired or wireless), and the Internet, a wide area network, a local network, a metropolitan area network, etc. may be used.

The bus 202 may be an ISA bus, a PCI bus, an EISA bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, and the like. The memory 201 is used to store a program, and the processor 200 executes the program after receiving the execution instruction. The method disclosed in any of the foregoing embodiments of the present application may be applied to the processor 200, or the The processor 200 is implemented.

The processor 200 may be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above-mentioned method can be completed by an integrated logic circuit of hardware in the processor 200 or an instruction in the form of software. The above-mentioned processor 200 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied as executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other storage media mature in the art. The storage medium is located in the memory 201, and the processor 200 reads the information in the memory 201, and completes the steps of the above method in combination with its hardware.

The electronic device provided by the embodiments of the present application and the methods provided by the embodiments of the present application are based on the same inventive concept, and have the same beneficial effects as the methods adopted, operated or realized.

Embodiments of the present application further provide a computer-readable storage medium corresponding to the method provided by the foregoing embodiments, please refer to FIG. 7 , the computer-readable storage medium shown is an optical disc 30 on which a computer program (ie, a computer program) is stored. program product), the computer program, when executed by the processor, executes the method provided by any of the foregoing embodiments.

It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random Access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media will not be repeated here.

The computer-readable storage medium provided by the above-mentioned embodiments of the present application and the methods provided by the embodiments of the present application are based on the same inventive concept, and have the same beneficial effects as the methods adopted, executed or implemented by the application programs stored therein.

It should be noted:

The algorithms and displays provided herein are not inherently related to any particular computer, virtual appliance, or other device. Various general-purpose devices can also be used with the teachings based on this. The structure required to construct such a device is apparent from the above description. Furthermore, this application is not directed to any particular programming language. It should be understood that the content of the application described herein can be implemented using a variety of programming languages and that the descriptions of specific languages above are intended to disclose the best mode of the application.

In the description provided herein, numerous specific details are set forth. It will be understood, however, that the embodiments of the present application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it is to be understood that in the above description of exemplary embodiments of the present application, various features of the present application are sometimes grouped together into a single embodiment, figure, or its description. This disclosure, however, should not be interpreted as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this application.

Those skilled in the art will understand that the modules in the device in the embodiment can be adaptively changed and arranged in one or more devices different from the embodiment. The modules or units or components in the embodiments may be combined into one module or unit or component, and further they may be divided into multiple sub-modules or sub-units or sub-assemblies. All features disclosed in this specification (including accompanying claims, abstract and drawings) and any method so disclosed may be employed in any combination, unless at least some of such features and/or procedures or elements are mutually exclusive. All processes or units of equipment are combined. Each feature disclosed in this specification (including accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will appreciate that although some of the embodiments described herein include certain features, but not others, included in other embodiments, that combinations of features of different embodiments are intended to be within the scope of the present application within and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.

Various component embodiments of the present application may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that, in practice, a microprocessor or a digital signal processor (DSP) may be used to implement some or all functions of some or all components in the apparatus for creating a virtual machine according to the embodiments of the present application. The present application can also be implemented as an apparatus or apparatus program (eg, computer programs and computer program products) for performing part or all of the methods described herein. Such a program implementing the present application may be stored on a computer-readable medium, or may be in the form of one or more signals. Such signals may be downloaded from Internet sites, or provided on carrier signals, or in any other form.

It should be noted that the above-described embodiments illustrate rather than limit the application, and alternative embodiments may be devised by those skilled in the art without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. do not denote any order. These words can be interpreted as names.

The above are only the preferred specific embodiments of the present application, but the protection scope of the present application is not limited to this. Substitutions should be covered within the protection scope of this application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims (10)

1. A method for compressing a face image, the method comprising:

inputting a style encoder and a content encoder from an original face image to extract style features and structural features;

respectively carrying out probability estimation and entropy coding to obtain a style coding bit stream corresponding to the style characteristics and a structure coding bit stream corresponding to the structure characteristics, and inputting the style coding bit stream and the structure coding bit stream into a decoder and a multitask analysis network;

the decoder reconstructs the images of the style coding bit stream and the structure coding bit stream and outputs a reconstructed image; and the multitask analysis network carries out semantic understanding analysis on the style coding bit stream and the structure coding bit stream and outputs semantic information of the image.

2. The method of claim 1, wherein the performing probability estimation and entropy coding to obtain a stylized coded bitstream corresponding to the stylistic characteristic and a structurally coded bitstream corresponding to the structurally characteristic, respectively, comprises:

quantizing the style characteristic and the structural characteristic respectively to obtain quantized style characteristic and structural characteristic;

and entropy coding the quantized style features and the structure features according to probability estimation results calculated by the probability estimation model respectively to obtain style coding bit streams corresponding to the style features and structure coding bit streams corresponding to the structure features.

3. The method of claim 1, wherein the decoder reconstructs the images of the style and structure coded bitstreams comprising:

fusing the style coding bit stream and the structure coding bit stream through a fusion module in a decoder, and learning the mean value and the variance of convolution layers in a residual block through multi-layer perception MLP (maximum likelihood prediction) processing;

executing an image compression task on the fused coded bit stream through a generator in a decoder to obtain a compressed reconstructed image;

judging the compressed reconstructed image through a discriminator to obtain a loss optimization function; the generator is trained according to a loss optimization function.

4. The method of claim 3, wherein the loss optimization function is in accordance with the following equation:

wherein D is a discriminator, E is a content encoder and a style encoder, G is a generator, P is a probability estimation model, x is an original face image,

in order to reconstruct the image,

and p is a probability estimation result, and lambda and beta are hyper-parameters for the quantized style characteristics and structure characteristics.

5. The method of claim 1, wherein the multitask analysis network performs semantic parsing on the style coded bitstream and the structure coded bitstream to output semantic information for an image, comprising:

inputting the style coding bit stream and the structure coding bit stream into the multitask analysis network, fusing the coding bit streams through a fusion module, and training the multitask analysis network according to a multitask analysis loss function to obtain a corresponding task result which is used as the output of the semantic information of the image.

6. The method of claim 5, wherein the multitasking analysis loss function L_multiCalculated according to the following formula:

L_multi＝λ_clsl_cls+λ_segl_seg

wherein l_cls、l_segIs a loss function, lambda, of the classification task and the segmentation task, respectively_cls、λ_segIs the corresponding weight hyperparameter.

7. The method of any of claims 1 to 6, further comprising: training parameters in the multi-task analysis model through optimization of the multi-task analysis loss function to obtain a global optimal solution; wherein the total loss function applied in the training of the multi-tasking analysis model is according to the following formula:

L＝L_EGP+L_D+γL_multi

wherein gamma is a hyperparameter.

8. A face image compression system, the system comprising:

the characteristic extraction module is used for inputting a style encoder and a content encoder from an original face image so as to extract style characteristics and structural characteristics;

the coding module is used for respectively carrying out probability estimation and entropy coding to obtain a style coding bit stream corresponding to the style characteristics and a structure coding bit stream corresponding to the structure characteristics, and inputting the style coding bit stream and the structure coding bit stream into the decoder and the multitask analysis network;

the compression decoding module is used for reconstructing the images of the style coding bit stream and the structure coding bit stream by a decoder and outputting a reconstructed image;

and the multitask analysis module is used for carrying out semantic understanding analysis on the style coding bit stream and the structure coding bit stream by the multitask analysis network and outputting semantic information of the image.

9. An electronic device, comprising: memory, processor and computer program stored on the memory and executable on the processor, characterized in that the processor executes when executing the computer program to implement the method according to any of claims 1-7.

10. A computer-readable storage medium having computer-readable instructions stored thereon, the computer-readable instructions being executable by a processor to implement the method of any one of claims 1-7.

Images