CN114071164A

CN114071164A - Training method and device of image compression model and image compression method and device

Info

Publication number: CN114071164A
Application number: CN202010784856.8A
Authority: CN
Inventors: 孙振鉷; 孙修宇; 谭志羽; 李�昊; 钱一琛
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2020-08-06
Filing date: 2020-08-06
Publication date: 2022-02-18

Abstract

The present specification provides a training method and apparatus for an image compression model, and an image compression method and apparatus, wherein the training method for the image compression model includes: acquiring a sample image, a target code rate parameter and a target full-connection parameter; inputting the sample image, the target code rate parameter and the target full-link parameter into an image compression model to be trained to obtain a target image corresponding to the sample image; calculating a loss value from the sample image and the target image; and training the image compression model according to the loss value until a training stop condition is reached, wherein the training method of the image compression model provided by the specification trains the image compression model through a target code rate parameter and a target full-connection parameter, so that the training time is saved, and the robustness of algorithm training is improved.

Description

Training method and device of image compression model and image compression method and device

Technical Field

The present specification relates to the field of computer technology, and in particular, to a training method for an image compression model. The present specification relates to two image compression model training methods, three image compression model training apparatuses, three image compression methods and apparatuses, a computing device, and a computer-readable storage medium.

Background

In the scheme of compressing the image based on the deep learning, the image compression is realized by controlling the code rate, a corresponding model is required for each code rate control, 100 models are required for realizing the control of 100 code rates, and for a terminal device with small storage capacity, too many models do not have practical application value, so the deep learning model with variable code rate is appeared for compressing the image.

The currently realized deep learning model with variable code rate performs image compression by inputting a model and a target code rate into an image compression model, but the deep learning model based on the variable code rate needs two training stages of coarse multi-code rate training and fine multi-code rate training, is complex in operation and long in training time, and reduces the training robustness in overfitting of longer training time and the quality of images obtained by compression.

Therefore, there is a need for simpler and more convenient compression of images.

Disclosure of Invention

In view of this, embodiments of the present specification provide a method for training an image compression model. The present specification also relates to an image compression model training device, three image compression methods and devices, a computing device, and a computer-readable storage medium, so as to solve the technical defects in the prior art.

According to a first aspect of embodiments of the present specification, there is provided a method for training an image compression model, including:

acquiring a sample image, a target code rate parameter and a target full-connection parameter;

inputting the sample image, the target code rate parameter and the target full-link parameter into an image compression model to be trained to obtain a target image corresponding to the sample image;

calculating a loss value from the sample image and the target image;

and training the image compression model according to the loss value until a training stop condition is reached.

According to a second aspect of embodiments of the present specification, there is provided a training method of an image compression model, including:

displaying an image input interface for a user based on a model training request of the user;

receiving a sample image, a target code rate parameter and a target full-connection parameter which are input by the user based on the image input interface;

calculating a loss value from the sample image and the target image;

According to a third aspect of embodiments herein, there is provided a method for training an image compression model, including:

receiving a model training request sent by a user, wherein the model training request carries a sample image, a target code rate parameter and a target full-connection parameter;

calculating a loss value from the sample image and the target image;

According to a fourth aspect of embodiments of the present specification, there is provided an image compression method including:

acquiring an image to be compressed and an image compression requirement of the image to be compressed;

determining a target code rate parameter and a target full-connection parameter according to the image compression requirement;

and inputting the image to be compressed, the target code rate parameter and the target full-link parameter into an image compression model to obtain a compressed image corresponding to the image to be compressed.

According to a fifth aspect of embodiments of the present specification, there is provided an image compression method including:

displaying an image input interface for a user based on a call request of the user;

receiving a picture to be compressed input by the user based on the image input interface and a picture compression requirement of the picture to be compressed;

According to a sixth aspect of embodiments of the present specification, there is provided an image compression method including:

receiving a calling request sent by a user, wherein the calling request carries a picture to be compressed and a picture compression requirement of the picture to be compressed;

According to a seventh aspect of the embodiments of the present specification, there is provided an apparatus for training an image compression model, including:

the acquisition module is configured to acquire a sample image, a target code rate parameter and a target full-connection parameter;

the input module is configured to input the sample image, the target code rate parameter and the target full-link parameter into an image compression model to be trained to obtain a target image corresponding to the sample image;

a calculation module configured to calculate a loss value from the sample image and the target image;

a training module configured to train the image compression model according to the loss value until a training stop condition is reached.

According to an eighth aspect of embodiments herein, there is provided an apparatus for training an image compression model, including:

an interface presentation module configured to present an image input interface for a user based on a model training request of the user;

a receiving module configured to receive a sample image, a target code rate parameter and a target full-link parameter input by the user based on the image input interface;

According to a ninth aspect of embodiments herein, there is provided an apparatus for training an image compression model, including:

the device comprises a request receiving module, a model training module and a target full-connection module, wherein the request receiving module is configured to receive a model training request sent by a user, and the model training request carries a sample image, a target code rate parameter and a target full-connection parameter;

According to a tenth aspect of embodiments herein, there is provided an image compression apparatus comprising:

the device comprises an acquisition module, a compression module and a compression module, wherein the acquisition module is configured to acquire an image to be compressed and an image compression requirement of the image to be compressed;

a determining module configured to determine a target code rate parameter and a target full-link parameter according to the image compression requirement;

and the compression module is configured to input the image to be compressed, the target code rate parameter and the target full-link parameter into an image compression model to obtain a compressed image corresponding to the image to be compressed.

According to an eleventh aspect of embodiments herein, there is provided an image compression apparatus comprising:

the interface display module is configured to display an image input interface for a user based on a call request of the user;

the receiving module is configured to receive a picture to be compressed input by the user based on the image input interface and a picture compression requirement of the picture to be compressed;

According to a twelfth aspect of embodiments of the present specification, there is provided an image compression apparatus comprising:

the device comprises a request receiving module, a processing module and a processing module, wherein the request receiving module is configured to receive a calling request sent by a user, and the calling request carries a picture to be compressed and a picture compression requirement of the picture to be compressed;

According to a thirteenth aspect of embodiments herein, there is provided a computing device comprising:

a memory and a processor;

the memory is configured to store computer-executable instructions and the processor is configured to execute the computer-executable instructions, wherein the processor implements the steps of the image compression model training method or the image compression method when executing the computer-executable instructions.

According to a fourteenth aspect of embodiments herein, there is provided a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the training method or the steps of the image compression method of any of the image compression models.

The training method of the image compression model provided by the specification comprises the steps of obtaining a sample image, a target code rate parameter and a target full-connection parameter; inputting the sample image, the target code rate parameter and the target full-link parameter into an image compression model to be trained to obtain a target image corresponding to the sample image; calculating a loss value from the sample image and the target image; and training the image compression model according to the loss value until a training stop condition is reached, and training the image compression model by the training method of the image compression model through a target code rate parameter and a target full-connection parameter, so that the training time is saved, and the robustness of algorithm training is improved.

Drawings

Fig. 1 is a diagram illustrating an exemplary image compression method applied to a communication tool for sending pictures according to an embodiment of the present disclosure;

FIG. 2 is a flowchart of a first method for training an image compression model according to an embodiment of the present disclosure;

FIG. 3 is a schematic structural diagram of an image compression model provided in an embodiment of the present specification;

FIG. 4 is a flowchart illustrating encoding a sample image in a training method of an image compression model according to an embodiment of the present disclosure;

fig. 5 is a structural diagram of an interpolation variable rate control subunit in an image compression model provided in an embodiment of the present specification;

fig. 6 is a flowchart illustrating decoding of image feature vectors in a training method of an image compression model according to an embodiment of the present disclosure;

fig. 7 is a flowchart of a first image compression method provided in an embodiment of the present specification;

FIG. 8 is a flowchart illustrating a second method for compressing an image according to an embodiment of the present disclosure;

FIG. 9 is a flowchart illustrating a third method for compressing an image according to an embodiment of the present disclosure;

FIG. 10 is a schematic structural diagram of a training apparatus for a first image compression model according to an embodiment of the present disclosure;

fig. 11 is a schematic structural diagram of a first image compression apparatus provided in an embodiment of the present specification;

fig. 12 is a schematic structural diagram of a second image compression apparatus provided in an embodiment of the present specification;

fig. 13 is a schematic structural diagram of a third image compression apparatus provided in an embodiment of the present specification;

FIG. 14 is a flowchart of a second method for training an image compression model according to an embodiment of the present disclosure;

FIG. 15 is a flowchart of a third method for training an image compression model according to an embodiment of the present disclosure;

FIG. 16 is a schematic structural diagram of a training apparatus for a second image compression model according to an embodiment of the present disclosure;

FIG. 17 is a schematic structural diagram of a training apparatus for a third image compression model according to an embodiment of the present disclosure;

fig. 18 is a block diagram of a computing device according to an embodiment of the present disclosure.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present description. This description may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, as those skilled in the art will be able to make and use the present disclosure without departing from the spirit and scope of the present disclosure.

The terminology used in the description of the one or more embodiments is for the purpose of describing the particular embodiments only and is not intended to be limiting of the description of the one or more embodiments. As used in one or more embodiments of the present specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in one or more embodiments of the present specification refers to and encompasses any and all possible combinations of one or more of the associated listed items.

It will be understood that, although the terms first, second, etc. may be used herein in one or more embodiments to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, a first can also be referred to as a second and, similarly, a second can also be referred to as a first without departing from the scope of one or more embodiments of the present description. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.

First, the noun terms to which one or more embodiments of the present specification relate are explained.

Code rate: corresponding to the compression ratio in the image compression algorithm, but opposite to the compression ratio. The higher the code rate is, the smaller the compression ratio is, the larger the compressed picture file is, the higher the picture quality is, and the more similar the picture is to the original picture. The lower the code rate, the larger the compression ratio, the smaller the compressed picture file, the lower the picture quality, and the more dissimilar with the original picture.

Picture quality: measure the similarity with the original. The commonly used picture quality metrics include peak signal-to-noise ratio (PSNR) and multi-level structural similarity (MS-SSIM).

In the present specification, a method of training an image compression model is provided. The present specification also relates to an image compression model training apparatus, three image compression methods and apparatuses, a computing device, and a computer-readable storage medium, which are described in detail in the following embodiments one by one.

In real life, images are often compressed, for example, pictures are sent through a communication tool, pictures with the size not more than 50k are uploaded according to the requirements of a website, the pictures are inserted into a document, and the like. Based on this, one or more embodiments of the present specification provide a training method of an image compression model and an image compression method, so as to solve the above technical problems. The training method and the image compression method of the image compression model provided by the specification are suitable for any scene needing to compress images, and the specification does not limit the application scene.

For ease of understanding, the following description will take an example in which an image compression method is applied to a communication tool sending a picture.

Referring to fig. 1, fig. 1 is a diagram illustrating a specific example of an image compression method applied to a communication tool sending picture according to an embodiment of the present disclosure.

The application scene of fig. 1 includes a terminal 102 and a terminal 104, the terminal 102 sends a picture to the terminal 104, the size of the picture is 2M, in order to improve the transmission rate and save the flow, the picture needs to be compressed, the picture is a picture to be compressed, after the picture is sent by firing at a midpoint of the terminal 102, according to the picture compression requirement corresponding to the determination of the current picture sending scene, according to the picture compression requirement, the corresponding target code rate parameter and the target full connection parameter are determined, the picture to be compressed, the target code rate parameter and the target full connection parameter are input into a pre-trained image compression model to be image compressed, a corresponding compressed picture is obtained, the size of the compressed picture is 950K, the compressed picture is then sent to the terminal 104, and at this time, the terminal 104 receives the compressed picture with the capacity of 950K.

According to the image compression method provided by the embodiment of the specification, the target code rate parameter and the target full-link parameter are determined according to the image compression requirement, the image to be compressed, the target code rate parameter and the target full-link parameter are input into the image compression model for image compression, and the code rate can be finely controlled according to the target code rate parameter and the target full-link parameter, so that the image compression model can realize fine code rate change control, obtain a finer compressed image, and improve the quality of the compressed image.

Referring to fig. 2, fig. 2 is a flowchart illustrating a training method of a first image compression model provided in an embodiment of the present specification, which specifically includes the following steps:

step 202: and acquiring a sample image, a target code rate parameter and a target full-connection parameter.

Specifically, in practical application, a sample image is obtained from a preset sample image set; acquiring a target code rate parameter from a preset code rate parameter set; and acquiring target full-connection parameters from a preset initial full-connection parameter set.

The sample image is an image compressed in the training process of the image compression model, and the sample image is obtained from a preset sample image set.

If the image compression model has N code rates and the code rate is represented by λ, a code rate list λ _ list corresponding to the image compression model is (λ 1, λ 2, … … λ N), and a value range of a target code rate parameter i is an integer of [0, N-2], that is, when the target code rate parameter i is 0, the target code rate parameter corresponds to λ 1 in the code rate list; and when the target code rate parameter i takes 1, corresponding to lambda 2 in the code rate list, and the like, wherein [0, N-2] is a preset code rate parameter set.

The target full-link parameter is marked by a, in the specification, the target full-link parameter is a parameter for determining an interpolation code rate, and the value of a is adjusted to enable two adjacent code rates to be associated, so that in the model training process, model training between the two adjacent code rates is not completely independent any more. The target full-connection parameter is selected from an initial full-connection parameter set, the value range of the initial full-connection parameter set is [0, 1], in order to save training time of the model in a training stage, a fixed value can be set for the value in the full-connection parameter set according to actual conditions, for example, the initial full-connection parameter set is set to be a set of three numbers (0, 0.5, 1), or to be a set of four numbers (0, 0.3, 0.6, 1). This is not a limitation in the present specification.

In an embodiment provided in this specification, the model training method provided in this specification is further explained by taking the sample image x, the target code rate parameter i, and the target full link parameter a as an example.

Step 204: and inputting the sample image, the target code rate parameter and the target full-link parameter into an image compression model to be trained to obtain a target image corresponding to the sample image.

Referring to fig. 3, fig. 3 shows a schematic structural diagram of an image compression model according to an embodiment of the present specification. The image compression model is a multi-code-rate image compression model and comprises an encoder and a decoder, wherein the encoder comprises a plurality of encoding convolutional layers which are sequentially connected, each encoding convolutional layer comprises a convolution subunit and an interpolation variable code rate control subunit, the decoder comprises a plurality of decoding convolutional layers which are sequentially connected, each decoding convolutional layer comprises a convolution subunit and an interpolation variable code rate control subunit, and fig. 3 shows a schematic diagram of the encoding convolutional layers and the decoding convolutional layers.

Inputting the sample image, the target code rate parameter and the target full-link parameter into the image compression model for image compression processing, and obtaining a target image corresponding to the sample image, wherein the method comprises the following steps:

determining a target code rate according to the target code rate parameter and the target full-connection parameter;

inputting the sample image and the target code rate into the encoder to obtain an image feature vector corresponding to the sample image;

and inputting the image feature vector and the target code rate into the decoder to obtain a target image corresponding to the sample image.

In practical application, under the condition that the code rate list has N code rates, an identity matrix index of N x N is constructed, after a target code rate parameter i and a target full-connection parameter a are determined, a corresponding target code rate lambda can be determined, the target code rate lambda is an interpolation variable code rate corresponding to the target code rate parameter i and the target full-connection parameter a, and the target code rate is multiplied by a convolution result in each interpolation variable code rate control subunit to obtain a corresponding interpolation result. The specific calculation formula of the target code rate λ is shown in the following formula (1):

λ＝a*(λ_list[i])+(1-a)*(λ_list[i+1]) (1)

wherein, λ is target code rate, a is target full-link parameter, i is target code rate parameter, λ _ list [ i ] and λ _ list [ i +1] are two adjacent code rates in the code rate set of the model.

In one embodiment provided in the present specification, when i is 3 and a is 0.5, λ is 0.5 × λ 4+0.5 × λ 5.

In another embodiment provided in the present specification, when i is 5 and a is 0, λ is λ 7.

In another embodiment provided in the present specification, i ═ 0, a ═ 1, and λ ═ λ 1 are taken as examples.

And after a target code rate lambda is determined according to a target code rate parameter i and a target full-link parameter a, inputting the sample image and the target code rate into the encoder to obtain an image feature vector corresponding to the sample image.

In an embodiment provided in this specification, the above example is used, the obtained sample image x, the target code rate parameter i, and the target full link parameter a are input to the image compression model to be trained, and the target code rate λ is determined according to the target code rate parameter i and the target full link parameter a.

Specifically, the encoder comprises n sequentially connected encoding convolutional layers, wherein n is greater than 1;

referring to fig. 4, fig. 4 shows a flowchart for encoding a sample image provided by an embodiment of the present specification, which includes steps 402 to 410.

Step 402: and inputting the sample image and the target code rate into a first coding convolutional layer to obtain a first image characteristic output by the first coding convolutional layer.

Specifically, the sample image and the target code rate are input to a first encoding convolutional layer for encoding, and it is assumed that the size of the sample image is H × W, where H is the height of the sample image and W is the width of the sample image, and in practical applications, the image features are usually four-dimensional representations (batch, H, W, channel), where batch is the number of images, H is the height of the images, W is the width of the images, and channel is the number of channels.

And performing convolution downsampling on the sample image to extract the image features of the sample image, wherein in the process of extracting the image features of the sample image through convolution downsampling, the feature size is gradually reduced, for example, the original size is 1024 × 128, the feature size is 512 × 64 after one downsampling, the feature size is 256 × 32 after the second downsampling, and the feature size is 128 × 16 after the third downsampling.

Obtaining a first image feature output by a first coding convolutional layer according to the image feature of the sample image and the target code rate, referring to fig. 5, where fig. 5 shows a structure diagram of an interpolation variable code rate control subunit provided in the embodiment of the present specification, as shown in fig. 5, a target code rate λ may be determined according to a target code rate parameter i and a target full connection parameter a, a parameter entering Full Connection (FCN) is associated with different code rates, N is the number of code rates in a code rate list, CH is channel, i.e., the number of channels, N × CH refers to the number of parameters to be learned in the FCN full connection layer, and the output of the full connection layer is multiplied by the image feature output by the convolutional subunit after being processed by a Softplus activation function, so as to obtain the first image feature output by the first coding convolutional layer.

In one embodiment provided in the present specificationContinuing with the above example, the sample image x and the target code rate λ are input to the first encoded convolutional layer to obtain the first image feature M output by the first encoded convolutional layer₁。

Step 404: inputting the t-1 image characteristics output by the t-1 coding convolutional layer and the target code rate into the t coding convolutional layer to obtain the t image characteristics output by the t coding convolutional layer, wherein t is more than or equal to 2 and less than or equal to n.

In practical application, t is taken from 2, that is, from the 2 nd encoding convolutional layer, the image features output by the last encoding convolutional layer and the target code rate are input to the current encoding convolutional layer, and the image features output by the current encoding convolutional layer are obtained.

Specifically, obtaining the tth image feature output by the tth encoded convolutional layer includes:

carrying out convolution downsampling on the t-1 image features, and extracting the image features of the t-1 image features;

and obtaining the t image characteristic output by the t coding convolutional layer according to the image characteristic of the t-1 image characteristic and the target code rate.

Regarding the operation in the t-th encoded convolutional layer, it is similar to the operation in the first encoded convolutional layer, and regarding the processing in the t-th encoded convolutional layer, see the processing of the 1 st encoded convolutional layer, and will not be described again here.

In an embodiment provided in this specification, the above example is used to receive the image feature M output by the last encoded convolutional layer_t-1And target code rate lambda, obtaining image characteristic M output by current coding convolution layer_t。

Step 406: increasing t by 1, judging whether t is larger than n, if not, executing the step 404, and if so, executing the step 408.

And increasing t by 1, judging whether t is greater than n, if t is less than or equal to n, indicating that the last coding convolutional layer is not reached, continuing to execute the step 404, and if t is greater than n, indicating that the last coding convolutional layer is reached, executing the step 408.

In an embodiment provided in this specification, the above example is used, where n is 4, that is, 4 coding convolutional layers are included in the current encoder, when t is 2, t is incremented by 1 to 3, and 3 is smaller than 4, step 404 is continuously performed, and when t is 4, t is incremented by 1 to 5, and 5 is larger than 4, step 408 is performed.

Step 408: and determining the image characteristics output by the last encoding convolutional layer as the image characteristics corresponding to the sample image.

And taking the image characteristics output by the last coding convolution layer as the image characteristics corresponding to the sample image.

In an embodiment provided in the present specification, following the above example, when the 4 th encoded convolutional layer outputs the image feature M_tAs the image feature M corresponding to the sample image x.

Step 410: and determining an image feature vector corresponding to the image feature according to the image feature.

The image features corresponding to the sample image are usually represented in a multi-dimensional manner, and feature stretching needs to be performed on the image features to obtain image feature vectors corresponding to the image features, and there are many ways of determining the image feature vectors corresponding to the image features according to the image features, for example, the image feature vectors are converted according to the product of the height, the width and the number of channels of the image features, and the like.

In an embodiment provided in the present specification, following the above example, the image feature vector E is converted according to the product of the height, width and number of channels of the image feature M.

Specifically, the decoder comprises m decoding convolution layers which are connected in sequence, wherein m is more than 1; the decoder is used for decoding according to the image feature vector obtained by the encoder to obtain a decoded compressed image.

Referring to fig. 6, fig. 6 shows a flowchart for decoding an image feature vector provided by an embodiment of the present specification, which includes steps 602 to 610.

Step 602: and inputting the image characteristic vector and the target code rate into a first decoding convolutional layer to obtain a first decoding vector output by the first decoding convolutional layer.

Specifically, obtaining a first decoded vector output by a first decoded convolutional layer comprises:

performing convolution upsampling on the image feature vector to obtain an initial decoding vector of the image feature vector;

and obtaining a first decoding vector output by the first decoding convolutional layer according to the initial decoding vector of the image feature vector and the target code rate.

Convolution upsampling is to perform convolution operation on image feature vectors to restore an image, and an upsampling method adopts an interpolation method, namely, a proper interpolation algorithm is adopted to insert new elements among pixel points on the basis of original image pixels.

In an embodiment provided in this specification, following the above example, the image feature vector E and the target code rate λ are input to the first decoding convolutional layer for decoding processing, the convolution subunit performs convolution upsampling on the image feature vector E to obtain an initial decoding vector of the image feature vector, and the initial decoding vector is multiplied by the corresponding target code rate λ to obtain a first decoding vector output by the first decoding convolutional layer.

Step 604: inputting the j-1 decoding vector output by the j-1 coding convolutional layer and the target code rate into the j decoding convolutional layer to obtain the j decoding vector output by the j decoding convolutional layer, wherein j is more than or equal to 2 and less than or equal to m.

Specifically, obtaining the jth decoded vector output by the jth decoded convolutional layer includes:

performing convolution upsampling on the j-1 decoding vector to obtain an initial decoding vector of the j-1 decoding vector;

and obtaining a j decoding vector output by the j decoding convolutional layer according to the initial decoding vector of the j-1 decoding vector and the target code rate.

For other decoding convolutional layers of the first decoding convolutional layer, receiving the decoding vector output by the last decoding convolutional layer, performing convolutional upsampling on the decoding vector output by the last decoding convolutional layer in the current decoding convolutional layer to obtain a corresponding initial decoding vector, and multiplying the initial decoding vector corresponding to the decoding vector output by the last decoding convolutional layer by the corresponding target code rate lambda to obtain the decoding vector output by the current decoding convolutional layer.

Step 606: incrementing j by 1, determining whether j is greater than m, if not, executing step 604, and if so, executing step 608.

Incrementing j by 1, determining whether j is greater than m, if j is less than or equal to m, indicating that the last decoded convolutional layer is not reached, then continue to execute step 604, and if j is greater than m, indicating that the last decoded convolutional layer is reached, then execute step 608.

In an embodiment provided in this specification, following the above example, where m is 4, that is, the current decoder includes 4 decoded convolutional layers, when j is 2, j is incremented by 1 to 3, and 3 is smaller than 4, the step 604 is continuously executed, and when j is 4, j is incremented by 1 to 5, and 5 is larger than 4, the step 608 is executed.

Step 608: and determining the decoding vector output by the last decoding convolution layer as the decoding vector corresponding to the sample image.

And taking the decoding vector output by the last decoding convolutional layer as the decoding vector output by the decoder, namely the decoding vector corresponding to the sample image.

In an embodiment provided in this specification, following the above example, the decoding vector output by the 4 th decoded convolutional layer is used as the decoding vector corresponding to the sample image.

Step 610: and normalizing the decoding vector to obtain a target image corresponding to the sample image.

And performing full connection and normalization processing on the decoding vector to finally obtain a target image corresponding to the sample image output by the image compression model, wherein the target image is an image obtained after the sample image is subjected to compression processing.

Step 206: a loss value is calculated from the sample image and the target image.

Optionally, step 206 includes calculating a loss value according to the target bitrate, the sample image and the target image.

The image compression model provided by the description is an unsupervised training, loss values are calculated through a target code rate, a sample image and a target image output by the image compression model, and a specific loss value calculation method is shown in the following formula 2-4.

Loss＝λD+R (2)

D＝(Sum((x-x_hat)²))/(H*W) (3)

R＝Sum(-log₂Prob(y_hat)) (4)

Wherein λ is the target code rate, and is determined by the target code rate parameter i and the target full-link parameter a, see the above formula (1). D represents the difference between the target picture and the sample picture. R represents the size of a storage space which needs to be stored at last for each pixel, x is a sample image, x _ hat is a target image, H is the height of the sample image x, and W is the width of the sample image x. y _ hat is an image feature vector obtained after the sample image is processed by the encoder, Prob (y _ hat) represents the probability distribution of y _ hat, and each value of y _ hat follows the laplace distribution (u, sigma).

In the embodiment provided in the present specification, the loss value is calculated according to the target code rate λ, the sample image x, and the target image x _ hat output by the image compression model.

Step 208: and training the image compression model according to the loss value until a training stop condition is reached.

And adjusting the parameters of the image compression model according to the back propagation of the loss value until the number of times of model training reaches a preset training turn or the loss value is smaller than a preset threshold value, and stopping training.

The training stopping condition is a preset training turn, or the loss value is smaller than a preset threshold value.

In the training method for the image compression model provided in the embodiment of the present specification, the sample image, the target code rate parameter and the target full link parameter are obtained, the sample image, the target code rate parameter and the target full link parameter are input to the image compression model to be trained to be processed, the target image is obtained, the loss value is calculated according to the sample image and the target image, and the image compression model is trained according to the loss value. The multi-code-rate image compression model is trained by adjusting the target code rate parameter and the target full-connection parameter without the need of a multi-stage training process, an interpolation variable code rate control subunit is directly added into an encoder and a decoder in the training process, a training method is simplified, the model is finely adjusted by controlling the input parameter to complete the training, and the quality of the compressed target image is improved.

Referring to fig. 7, fig. 7 is a flowchart illustrating a first image compression method provided in an embodiment of the present specification, which specifically includes the following steps:

step 702: and acquiring an image to be compressed and an image compression requirement of the image to be compressed.

The image to be compressed is an image which needs to be compressed, such as a photo which is transferred between two terminal devices, a photo thumbnail in news, a photo which needs to be uploaded to a website, and the like, the image compression requirement is a requirement on how much the image to be compressed needs to be compressed, such as transferring the photo between two terminal devices, the original photo size is 9M, in order to facilitate the transfer, the image to be compressed is compressed to 1M, the image to be compressed of 9M is compressed to 1M, which is an image compression requirement, for example, a head portrait within 50K needs to be uploaded to some websites, and the selected photo size is 2M, the photo size needs to be compressed to within 50K, and the image to be compressed to within 50K is an image compression requirement.

In one embodiment provided in the present specification. Taking the example of uploading a personal avatar on a certain website as an explanation, an image x to be compressed is obtained, and the image compression requirement is to compress the image x to be compressed to 50K.

Step 704: and determining a target code rate parameter and a target full-connection parameter according to the image compression requirement.

Optionally, determining a target code rate parameter and a target full link parameter according to the image compression requirement includes: and determining a target code rate parameter in a preset code rate parameter set according to the image compression requirement, and determining a target full-connection parameter in a preset full-connection parameter set.

In practical application, the target code rate parameter is still determined from a preset code rate parameter set, and the preset code rate parameter set is the same as the code rate parameter set in the training process. The target full-link parameter is determined from the full-link parameter set, in the process of training the image model, the initial full-link parameter set is a relatively coarse value, such as selected from three values of 0, 0.5 and 1, in the process of applying the image model, the full-link parameter set is selected from a full-link parameter set with a fixed interval between 0 and 1, such as the target full-link parameter a is selected from a full-link parameter set with 100 values (value interval 1/100) in total of [1, 0.99, 0.98, … … 0.02 and 0.01], or from a more precise full-link parameter set (with a numerical interval of 1/200), in a word, the value of a is determined to be a value in the interval of [0, 1 ]. Therefore, the value of the interpolation variable code rate can be finer, and the image compression model can realize more than 1000 code rate change controls. And compressing the compressed picture with finer code rate.

In the embodiment provided in this specification, according to the image compression requirement, an image x to be compressed is compressed to 50K, and a corresponding target code rate parameter is determined to be i and a target full link parameter a.

Step 706: and inputting the image to be compressed, the target code rate parameter and the target full-link parameter into an image compression model to obtain a compressed image corresponding to the image to be compressed.

Optionally, the image compression model comprises an encoder and a decoder,

inputting the image to be compressed, the target code rate parameter and the target full-link parameter into an image compression model, and obtaining a compressed image corresponding to the image to be compressed, wherein the steps from S7062 to S7066 are as follows:

s7062, determining a target code rate according to the target code rate parameter and the target full-link parameter.

S7064, inputting the image to be compressed and the target code rate into the encoder, and obtaining an image feature vector corresponding to the image to be compressed.

Specifically, the encoder comprises n sequentially connected encoding convolutional layers, wherein n is more than 1; step S7064 includes:

inputting the image to be compressed and the target code rate into a first coding convolutional layer to obtain a first image characteristic output by the first coding convolutional layer;

inputting the t-1 image characteristics output by the t-1 coding convolutional layer and the target code rate into the t coding convolutional layer to obtain the t image characteristics output by the t coding convolutional layer, wherein t is more than or equal to 2 and less than or equal to n;

increasing t by 1, and judging whether t is greater than n;

if not, continuing to input the t-1 image characteristics output by the t-1 coding convolutional layer and the target code rate into the t coding convolutional layer to obtain the t image characteristics output by the t coding convolutional layer;

if so, determining the image feature output by the last encoding convolutional layer as the image feature corresponding to the image to be compressed, and determining the image feature vector corresponding to the image feature according to the image feature.

Specifically, obtaining the first image feature output by the first encoded convolutional layer comprises: performing convolution downsampling on the image to be compressed, and extracting image characteristics of the image to be compressed; and obtaining a first image characteristic output by the first coding convolutional layer according to the image characteristic of the image to be compressed and the target code rate.

Obtaining the t image feature output by the t coding convolution layer, comprising: carrying out convolution downsampling on the t-1 image features, and extracting the image features of the t-1 image features; and obtaining the t image characteristic output by the t coding convolutional layer according to the image characteristic of the t-1 image characteristic and the target code rate.

S7066, inputting the image feature vector and the target code rate to the decoder to obtain a target image corresponding to the image to be compressed.

Specifically, the decoder comprises m decoding convolution layers which are sequentially connected, wherein m is more than or equal to 1; step S7066 includes:

inputting the image characteristic vector and the target code rate into a first decoding convolutional layer to obtain a first decoding vector output by the first decoding convolutional layer;

inputting a j-1 decoding vector output by a j-1 coding convolutional layer and the target code rate into a j decoding convolutional layer to obtain a j decoding vector output by the j decoding convolutional layer, wherein j is more than or equal to 2 and less than or equal to m;

increasing j by 1, and judging whether j is greater than m;

if not, continuing to input the j-1 decoding vector output by the j-1 coding convolutional layer and the target code rate into the j decoding convolutional layer to obtain the j decoding vector output by the j decoding convolutional layer;

if so, determining a decoding vector output by the last decoding convolution layer as a decoding vector corresponding to the image to be compressed, and performing normalization processing on the decoding vector to obtain a target image corresponding to the image to be compressed.

Specifically, obtaining a first decoded vector output by a first decoded convolutional layer comprises: performing convolution upsampling on the image feature vector to obtain an initial decoding vector of the image feature vector; and obtaining a first decoding vector output by the first decoding convolutional layer according to the initial decoding vector of the image feature vector and the target code rate.

Obtaining a jth decoded vector output by a jth decoded convolutional layer, comprising: performing convolution upsampling on the j-1 decoding vector to obtain an initial decoding vector of the j-1 decoding vector; and obtaining a j decoding vector output by the j decoding convolutional layer according to the initial decoding vector of the j-1 decoding vector and the target code rate.

Step 706 is identical to the method of step 204, and for the specific explanation of step 706, refer to the details of step 204 in the foregoing embodiment, which are not repeated herein.

In the embodiment provided in this specification, an image x to be compressed, the target code rate parameter i, and the target all-connection parameter a are input to a pre-trained image compression model, and a target code rate λ is determined according to the target code rate parameter i and the target all-connection parameter a.

Inputting the image x to be compressed and the target code rate lambda into the encoder, wherein the encoder comprises a plurality of sequentially connected encoding convolutional layers, each encoding convolutional layer comprises a convolution subunit and an interpolation variable code rate control subunit, convolution processing is performed on the image to be compressed in each encoding convolutional layer, and a final encoding vector E output by the encoder is obtained by multiplying the target code rate (namely the plug-in variable code rate) by a convolution result.

And inputting the encoding vector E and the target code rate lambda into a decoder for decoding, wherein the decoder comprises a plurality of decoding convolutional layers which are connected in sequence, each decoding convolutional layer also comprises a convolutional subunit and an interpolation variable code rate control subunit, the encoding vector is subjected to convolutional upsampling in each decoding convolutional layer, and the convolutional upsampling is multiplied by a convolution result according to the target code rate (namely the plug-in variable code rate) to obtain a compressed image x _ hat corresponding to the image to be compressed, which is finally output by the decoder.

The image compression method provided by the embodiment of the specification comprises the steps of obtaining an image to be compressed and an image compression requirement of the image to be compressed; determining a target code rate parameter and a target full-connection parameter according to the image compression requirement; and inputting the image to be compressed, the target code rate parameter and the target full-link parameter into an image compression model to obtain a compressed image corresponding to the image to be compressed. The image compression model can realize more precise code rate control, compressed pictures with different code rates are obtained in one multi-code-rate image compression model through parameter control, and the quality of the compressed pictures obtained by the image compression model can be effectively improved through interpolation variable code rate control.

Referring to fig. 8, fig. 8 is a flowchart illustrating a second image compression method provided in an embodiment of the present specification, including the following steps:

step 802: and displaying an image input interface for the user based on the call request of the user.

Specifically, under the condition of receiving a call request of a user, an image input interface is determined according to the call request, the image input interface is displayed to the user, and the user can input a picture to be compressed and a picture compression requirement of the picture to be compressed through the image input interface.

Step 804: and receiving the picture to be compressed input by the user based on the image input interface and the picture compression requirement of the picture to be compressed.

Step 806: and determining a target code rate parameter and a target full-connection parameter according to the image compression requirement.

Step 808: and inputting the image to be compressed, the target code rate parameter and the target full-link parameter into an image compression model to obtain a compressed image corresponding to the image to be compressed.

It should be noted that, for a portion of the image compression method provided in this embodiment of the present specification corresponding to the embodiment of the first image compression method provided in the foregoing embodiment, reference may be made to the detailed description of the first image compression method in the foregoing embodiment, and details are not repeated here.

The image compression method provided by the embodiment of the specification provides an interactive interface for a user, and obtains an image to be compressed and an image compression requirement of the image to be compressed through the interactive interface; determining a target code rate parameter and a target full-connection parameter according to the image compression requirement; and inputting the image to be compressed, the target code rate parameter and the target full-link parameter into an image compression model to obtain a compressed image corresponding to the image to be compressed. The image compression model can realize more precise code rate control, compressed pictures with different code rates are obtained in one multi-code-rate image compression model through parameter control, and the quality of the compressed pictures obtained by the image compression model can be effectively improved through interpolation variable code rate control.

Referring to fig. 9, fig. 9 is a flowchart illustrating a third image compression method provided in an embodiment of the present specification, including the following steps:

step 902: receiving a call request sent by a user, wherein the call request carries a picture to be compressed and a picture compression requirement of the picture to be compressed.

Step 904: and determining a target code rate parameter and a target full-connection parameter according to the image compression requirement.

Step 906: and inputting the image to be compressed, the target code rate parameter and the target full-link parameter into an image compression model to obtain a compressed image corresponding to the image to be compressed.

It should be noted that, for a portion of the image compression method provided in this specification, which corresponds to the embodiment of the first image compression method provided in the foregoing embodiment, reference may be made to the detailed description of the first image compression method in the foregoing embodiment, and details are not repeated here.

In practical application, for users who have a requirement for compressing pictures in large batches, such as website editing and the like, and need to compress pictures in batches, a call request sent by the user can be received through a corresponding API interface, the call request carries the pictures to be compressed and the picture compression requirement, the pictures are compressed in batches by the image compression method provided by the embodiment of the specification, and the compressed target images are returned to the user, so that user experience is improved.

The image compression method provided by the embodiment of the specification receives a call request of a user, wherein the call request carries a picture to be compressed and a picture compression requirement, and a target code rate parameter and a target full-connection parameter are determined according to the picture compression requirement; and inputting the image to be compressed, the target code rate parameter and the target full-link parameter into an image compression model to obtain a compressed image corresponding to the image to be compressed. The image compression model can realize more precise code rate control, compressed pictures with different code rates are obtained in one multi-code-rate image compression model through parameter control, and the quality of the compressed pictures obtained by the image compression model can be effectively improved through interpolation variable code rate control.

Corresponding to the above embodiment of the training method of the image compression model, the present specification further provides an embodiment of a training apparatus of the image compression model, and fig. 10 shows a schematic structural diagram of a training apparatus of a first image compression model provided in an embodiment of the present specification. As shown in fig. 10, the apparatus includes:

an obtaining module 1002 configured to obtain a sample image, a target code rate parameter, and a target full connection parameter.

An input module 1004 configured to input the sample image, the target code rate parameter, and the target full-link parameter into an image compression model to be trained, so as to obtain a target image corresponding to the sample image.

A calculation module 1006 configured to calculate a loss value from the sample image and the target image.

A training module 1008 configured to train the image compression model according to the loss value until a training stop condition is reached.

Optionally, the image compression model comprises an encoder and a decoder;

the input module 1004 includes: :

a determining submodule configured to determine a target code rate according to the target code rate parameter and the target full-connection parameter;

the encoding submodule is configured to input the sample image and the target code rate into the encoder, and obtain an image feature vector corresponding to the sample image;

and the decoding submodule is configured to input the image feature vector and the target code rate into the decoder to obtain a target image corresponding to the sample image.

Optionally, the encoder includes n sequentially connected encoding convolutional layers, where n > 1;

the encoding submodule, further configured to:

inputting the sample image and the target code rate into a first coding convolutional layer to obtain a first image characteristic output by the first coding convolutional layer;

increasing t by 1, and judging whether t is greater than n;

if not, continuing to input the t-1 image characteristic output by the t-1 coding convolutional layer and the target code rate into the t-coding convolutional layer to obtain the t image characteristic output by the t-coding convolutional layer;

if so, determining the image feature output by the last encoding convolutional layer as the image feature corresponding to the sample image, and determining the image feature vector corresponding to the image feature according to the image feature.

Optionally, the encoding sub-module is further configured to:

performing convolution downsampling on the sample image, and extracting the image characteristics of the sample image;

and obtaining a first image characteristic output by the first coding convolutional layer according to the image characteristic of the sample image and the target code rate.

Optionally, the encoding sub-module is further configured to:

Optionally, the decoder includes m decoding convolutional layers connected in sequence, where m > 1;

the decoding sub-module further configured to:

increasing j by 1, and judging whether j is greater than m;

if not, continuing to execute the operation of inputting the j-1 decoding vector output by the j-1 coding convolutional layer and the target code rate into the j decoding convolutional layer to obtain the j decoding vector output by the j decoding convolutional layer;

if so, determining a decoding vector output by the last decoding convolution layer as a decoding vector corresponding to the sample image, and performing normalization processing on the decoding vector to obtain a target image corresponding to the sample image.

Optionally, the decoding sub-module is further configured to:

Optionally, the calculating module 1006 is further configured to calculate a loss value according to the target code rate, the sample image and the target image.

Optionally, the obtaining module 1002 is further configured to:

acquiring a sample image from a preset sample image set;

acquiring a target code rate parameter from a preset code rate parameter set;

and acquiring target full-connection parameters from a preset initial full-connection parameter set.

In the training device for the image compression model provided in the embodiment of the present specification, the sample image, the target code rate parameter, and the target full link parameter are obtained, the sample image, the target code rate parameter, and the target full link parameter are input to the image compression model to be trained to be processed, the target image is obtained, the loss value is calculated according to the sample image and the target image, and the image compression model is trained according to the loss value. The multi-code-rate image compression model is trained by adjusting the target code rate parameter and the target full-connection parameter without the need of a multi-stage training process, an interpolation variable code rate control subunit is directly added into an encoder and a decoder in the training process, a training method is simplified, the model is finely adjusted by controlling the input parameter to complete the training, and the quality of the compressed target image is improved.

The above is a schematic scheme of a training apparatus for an image compression model according to this embodiment. It should be noted that the technical solution of the training apparatus for the image compression model and the technical solution of the training method for the image compression model belong to the same concept, and details not described in detail in the technical solution of the training apparatus for the image compression model can be referred to the description of the technical solution of the training method for the image compression model.

Corresponding to the embodiment of the first image compression method, the present specification further provides an embodiment of an image compression apparatus, and fig. 11 shows a schematic structural diagram of the first image compression apparatus provided in an embodiment of the present specification. As shown in fig. 11, the apparatus includes:

an obtaining module 1102 configured to obtain an image to be compressed and an image compression requirement of the image to be compressed.

A determining module 1104 configured to determine a target bitrate parameter and a target full connection parameter according to the image compression requirement.

A compression module 1106, configured to input the image to be compressed, the target code rate parameter, and the target full link parameter into an image compression model, so as to obtain a compressed image corresponding to the image to be compressed.

Optionally, the image compression model comprises an encoder and a decoder;

the compression module 1106 includes:

the coding submodule is configured to input the image to be compressed and the target code rate into the coder, and obtain an image feature vector corresponding to the image to be compressed;

and the decoding submodule is configured to input the image feature vector and the target code rate into the decoder to obtain a target image corresponding to the image to be compressed.

the encoding submodule, further configured to:

increasing t by 1, and judging whether t is greater than n;

Optionally, the encoding sub-module is further configured to:

performing convolution downsampling on the image to be compressed, and extracting image characteristics of the image to be compressed;

and obtaining a first image characteristic output by the first coding convolutional layer according to the image characteristic of the image to be compressed and the target code rate.

Optionally, the encoding sub-module is further configured to:

the decoding sub-module further configured to:

increasing j by 1, and judging whether j is greater than m;

Optionally, the decoding sub-module is further configured to:

Optionally, the determining module 1104 is further configured to determine a target bitrate parameter in a preset bitrate parameter set according to the image compression requirement, and determine a target full-connection parameter in a preset full-connection parameter set.

The image compression device provided by the embodiment of the specification acquires an image to be compressed and an image compression requirement of the image to be compressed; determining a target code rate parameter and a target full-connection parameter according to the image compression requirement; and inputting the image to be compressed, the target code rate parameter and the target full-link parameter into an image compression model to obtain a compressed image corresponding to the image to be compressed. The image compression model can realize more precise code rate control, compressed pictures with different code rates are obtained in one multi-code-rate image compression model through parameter control, and the quality of the compressed pictures obtained by the image compression model can be effectively improved through interpolation variable code rate control.

The above is a schematic arrangement of an image compression apparatus of the present embodiment. It should be noted that the technical solution of the image compression apparatus belongs to the same concept as the technical solution of the first image compression method of the foregoing embodiment, and details that are not described in detail in the technical solution of the image compression apparatus can be referred to the description of the technical solution of the first image compression method of the foregoing embodiment.

Corresponding to the embodiment of the second image compression method, the present specification also provides an embodiment of a second image compression apparatus, and fig. 12 shows a schematic structural diagram of the second image compression apparatus provided in an embodiment of the present specification. As shown in fig. 12, the apparatus includes:

an interface presentation module 1202 configured to present an image input interface for a user based on a call request of the user.

A receiving module 1204, configured to receive the picture to be compressed input by the user based on the image input interface and the picture compression requirement of the picture to be compressed.

A determining module 1206 configured to determine a target bitrate parameter and a target full connection parameter according to the image compression requirement.

A compression module 1208, configured to input the image to be compressed, the target code rate parameter, and the target full-link parameter into an image compression model, and obtain a compressed image corresponding to the image to be compressed.

The image compression device provided by the embodiment of the specification provides an interactive interface for a user, and obtains an image to be compressed and an image compression requirement of the image to be compressed through the interactive interface; determining a target code rate parameter and a target full-connection parameter according to the image compression requirement; and inputting the image to be compressed, the target code rate parameter and the target full-link parameter into an image compression model to obtain a compressed image corresponding to the image to be compressed. The image compression model can realize more precise code rate control, compressed pictures with different code rates are obtained in one multi-code-rate image compression model through parameter control, and the quality of the compressed pictures obtained by the image compression model can be effectively improved through interpolation variable code rate control.

The above is a schematic arrangement of an image compression apparatus of the present embodiment. It should be noted that the technical solution of the image compression apparatus belongs to the same concept as the technical solution of the second image compression method of the above embodiment, and details that are not described in detail in the technical solution of the image compression apparatus can be referred to the description of the technical solution of the second image compression method of the above embodiment.

Corresponding to the embodiment of the third image compression method, the present specification also provides an embodiment of a third image compression apparatus, and fig. 13 shows a schematic structural diagram of the third image compression apparatus provided in an embodiment of the present specification. As shown in fig. 13, the apparatus includes:

the request receiving module 1302 is configured to receive an invoking request sent by a user, where the invoking request carries a picture to be compressed and a picture compression requirement of the picture to be compressed.

A determining module 1304 configured to determine a target bitrate parameter and a target full connection parameter according to the image compression requirement.

And the compression module 1306 is configured to input the image to be compressed, the target code rate parameter and the target full-link parameter into an image compression model, and obtain a compressed image corresponding to the image to be compressed.

The image compression device provided by the embodiment of the description receives a call request of a user, wherein the call request carries a picture to be compressed and a picture compression requirement, and a target code rate parameter and a target full-connection parameter are determined according to the picture compression requirement; and inputting the image to be compressed, the target code rate parameter and the target full-link parameter into an image compression model to obtain a compressed image corresponding to the image to be compressed. The image compression model can realize more precise code rate control, compressed pictures with different code rates are obtained in one multi-code-rate image compression model through parameter control, and the quality of the compressed pictures obtained by the image compression model can be effectively improved through interpolation variable code rate control.

The above is a schematic arrangement of an image compression apparatus of the present embodiment. It should be noted that the technical solution of the image compression apparatus belongs to the same concept as that of the third image compression method of the above-mentioned embodiment, and details that are not described in detail in the technical solution of the image compression apparatus can be referred to the description of the technical solution of the third image compression method of the above-mentioned embodiment.

Referring to fig. 14, fig. 14 is a flowchart illustrating a training method of a second image compression model provided in an embodiment of the present specification, including the following steps:

step 1402: an image input interface is presented to a user based on a model training request of the user.

Step 1404: and receiving a sample image, a target code rate parameter and a target full-connection parameter which are input by the user based on the image input interface.

Step 1406: and inputting the sample image, the target code rate parameter and the target full-link parameter into an image compression model to be trained to obtain a target image corresponding to the sample image.

Step 1408: a loss value is calculated from the sample image and the target image.

Step 1410: and training the image compression model according to the loss value until a training stop condition is reached.

It should be noted that, for a part of the training method of the image compression model provided in the embodiment of this specification, which corresponds to the embodiment of the training method of the first image compression model provided in the foregoing embodiment, reference may be made to the detailed description of the training method of the first image compression model in the foregoing embodiment, and details are not repeated here.

Referring to fig. 15, fig. 15 is a flowchart illustrating a training method of a third image compression model provided in an embodiment of the present specification, including the following steps:

step 1502: receiving a model training request sent by a user, wherein the model training request carries a sample image, a target code rate parameter and a target full-link parameter.

Step 1504: and inputting the sample image, the target code rate parameter and the target full-link parameter into an image compression model to be trained to obtain a target image corresponding to the sample image.

Step 1506: a loss value is calculated from the sample image and the target image.

Step 1508: and training the image compression model according to the loss value until a training stop condition is reached.

Corresponding to the above embodiment of the method for training the second image compression model, the present specification further provides an embodiment of a training apparatus for the second image compression model, and fig. 16 shows a schematic structural diagram of the training apparatus for the second image compression model provided in an embodiment of the present specification. As shown in fig. 16, the apparatus includes:

an interface presentation module 1602 configured to present an image input interface for a user based on the user's model training request.

A receiving module 1604 configured to receive the sample image, the target bitrate parameter and the target full-link parameter input by the user based on the image input interface.

An input module 1606, configured to input the sample image, the target bitrate parameter, and the target full connection parameter to an image compression model to be trained, so as to obtain a target image corresponding to the sample image.

A calculation module 1608 configured to calculate a loss value from the sample image and the target image.

A training module 1610 configured to train the image compression model according to the loss value until a training stop condition is reached.

The above is a schematic scheme of a training apparatus for an image compression model according to this embodiment. It should be noted that the technical solution of the training apparatus for image compression models and the technical solution of the training method for second image compression models in the above embodiments belong to the same concept, and details of the technical solution of the training apparatus for image compression models, which are not described in detail, can be referred to the description of the technical solution of the training method for second image compression models in the above embodiments.

Corresponding to the embodiment of the training method of the third image compression model, the present specification further provides an embodiment of a training apparatus of the third image compression model, and fig. 17 shows a schematic structural diagram of the training apparatus of the third image compression model provided in an embodiment of the present specification. As shown in fig. 17, the apparatus includes:

a request receiving module 1702, configured to receive a model training request sent by a user, where the model training request carries a sample image, a target bitrate parameter, and a target full connection parameter.

An input module 1704, configured to input the sample image, the target bitrate parameter, and the target full-link parameter to an image compression model to be trained, so as to obtain a target image corresponding to the sample image.

A calculation module 1706 configured to calculate a loss value from the sample image and the target image.

A training module 1708 configured to train the image compression model according to the loss value until a training stop condition is reached.

The above is a schematic scheme of a training apparatus for an image compression model according to this embodiment. It should be noted that the technical solution of the training apparatus for image compression models and the technical solution of the training method for image compression models in the above-mentioned embodiments belong to the same concept, and details of the technical solution of the training apparatus for image compression models, which are not described in detail, can be referred to the description of the technical solution of the training method for image compression models in the above-mentioned embodiments.

FIG. 18 illustrates a block diagram of a computing device 1800, according to an embodiment of the present specification. Components of the computing device 1800 include, but are not limited to, the memory 1810 and the processor 1820. The processor 1820 is coupled to the memory 1810 via the bus 1830, and the database 1850 is used for storing data.

Computing device 1800 also includes access device(s) 1840, which access device(s) 1840 enable computing device 1800 to communicate via one or more networks 1860. Examples of such networks include the Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the internet. The access device 1840 may include one or more of any type of network interface (e.g., a Network Interface Card (NIC)) whether wired or wireless, such as an IEEE802.11 Wireless Local Area Network (WLAN) wireless interface, a global microwave interconnect access (Wi-MAX) interface, an ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a bluetooth interface, a Near Field Communication (NFC) interface, and so forth.

In one embodiment of the specification, the above-described components of computing device 1800, as well as other components not shown in FIG. 18, may also be connected to each other, such as by a bus. It should be understood that the block diagram of the computing device structure shown in FIG. 18 is for purposes of example only and is not limiting as to the scope of the description. Those skilled in the art may add or replace other components as desired.

The computing device 1800 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., tablet, personal digital assistant, laptop, notebook, netbook, etc.), mobile phone (e.g., smartphone), wearable computing device (e.g., smartwatch, smartglasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. Computing device 1800 may also be a mobile or stationary server.

Wherein the processor 1820 is configured to execute computer-executable instructions, wherein the processor when executing the computer-executable instructions implements the method for training the image compression model or the steps of the image compression method.

The above is an illustrative scheme of a computing device of the present embodiment. It should be noted that the technical solution of the computing device belongs to the same concept as the above-mentioned training method of the image compression model or the above-mentioned technical solution of the image compression method, and details that are not described in detail in the technical solution of the computing device can be referred to the above-mentioned description of the training method of the image compression model or the above-mentioned technical solution of the image compression method.

An embodiment of the present specification further provides a computer readable storage medium storing computer instructions, which when executed by a processor, implement the method for training the image compression model or the steps of the image compression method.

The above is an illustrative scheme of a computer-readable storage medium of the present embodiment. It should be noted that the technical solution of the storage medium belongs to the same concept as the above-mentioned technical solution of the image compression model training method or the image compression method, and details that are not described in detail in the technical solution of the storage medium can be referred to the above-mentioned description of the technical solution of the image compression model training method or the image compression method.

The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

The computer instructions comprise computer program code which may be in the form of source code, object code, an executable file or some intermediate form, or the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.

It should be noted that, for the sake of simplicity, the foregoing method embodiments are described as a series of acts or combinations, but those skilled in the art should understand that the present disclosure is not limited by the described order of acts, as some steps may be performed in other orders or simultaneously according to the present disclosure. Further, those skilled in the art should also appreciate that the embodiments described in this specification are preferred embodiments and that acts and modules referred to are not necessarily required for this description.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

The preferred embodiments of the present specification disclosed above are intended only to aid in the description of the specification. Alternative embodiments are not exhaustive and do not limit the invention to the precise embodiments described. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the specification and its practical application, to thereby enable others skilled in the art to best understand the specification and its practical application. The specification is limited only by the claims and their full scope and equivalents.

Claims

1. A training method of an image compression model comprises the following steps:

calculating a loss value from the sample image and the target image;

2. The method of training an image compression model of claim 1, the image compression model comprising an encoder and a decoder;

inputting the sample image, the target code rate parameter and the target full-link parameter into an image compression model to be trained, and obtaining a target image corresponding to the sample image, wherein the target image comprises:

3. The method of claim 2, wherein the encoder comprises n sequentially connected encoded convolutional layers, wherein n > 1;

inputting the sample image and the target code rate into the encoder, and obtaining an image feature vector corresponding to the sample image, including:

increasing t by 1, and judging whether t is greater than n;

4. The method of claim 3, wherein obtaining the first image feature of the first encoded convolutional layer output comprises:

5. The method for training an image compression model according to claim 3, obtaining the tth image feature of the tth encoded convolutional layer output, comprising:

6. The method of claim 2, wherein the decoder comprises m sequentially connected decoding convolutional layers, wherein m > 1;

inputting the image feature vector and the target code rate into the decoder to obtain a target image corresponding to the sample image, wherein the method comprises the following steps:

increasing j by 1, and judging whether j is greater than m;

7. The method of claim 6, wherein obtaining the first decoded vector output by the first decoded convolutional layer comprises:

8. The method of claim 6, wherein obtaining the jth decoded vector of the jth decoded convolutional layer output comprises:

9. The method for training an image compression model according to claim 2, wherein calculating a loss value according to the sample image and the target image comprises:

and calculating a loss value according to the target code rate, the sample image and the target image.

10. The method for training an image compression model according to claim 1, wherein the obtaining of the sample image, the target code rate parameter and the target full-link parameter comprises:

acquiring a sample image from a preset sample image set;

acquiring a target code rate parameter from a preset code rate parameter set;

11. A training method of an image compression model comprises the following steps:

calculating a loss value from the sample image and the target image;

12. A training method of an image compression model comprises the following steps:

calculating a loss value from the sample image and the target image;

13. An image compression method comprising:

14. The image compression method of claim 13, the image compression model comprising an encoder and a decoder;

inputting the image to be compressed, the target code rate parameter and the target full-link parameter into an image compression model to obtain a compressed image corresponding to the image to be compressed, wherein the method comprises the following steps:

inputting the image to be compressed and the target code rate into the encoder to obtain an image characteristic vector corresponding to the image to be compressed;

and inputting the image characteristic vector and the target code rate into the decoder to obtain a target image corresponding to the image to be compressed.

15. The image compression method of claim 14, the encoder comprising n sequentially connected encoded convolutional layers, wherein n > 1;

inputting the image to be compressed and the target code rate into the encoder, and obtaining an image feature vector corresponding to the image to be compressed, including:

increasing t by 1, and judging whether t is greater than n;

16. The image compression method of claim 15, obtaining a first image feature of the first encoded convolutional layer output, comprising:

17. The image compression method of claim 15, obtaining the tth image feature of the tth encoded convolutional layer output, comprising:

18. The image compression method of claim 14, wherein the decoder comprises m sequentially connected decoding convolutional layers, wherein m ≧ 1;

inputting the image feature vector and the target code rate into the decoder to obtain a target image corresponding to the image to be compressed, wherein the method comprises the following steps:

increasing j by 1, and judging whether j is greater than m;

19. The image compression method of claim 18, obtaining the first decoded vector output by the first decoded convolutional layer, comprising:

20. The image compression method of claim 18, obtaining a jth decoded vector of a jth decoded convolutional layer output, comprising:

21. The image compression method of claim 13, wherein determining the target bitrate parameter and the target full connection parameter according to the image compression requirement comprises:

and determining a target code rate parameter in a preset code rate parameter set according to the image compression requirement, and determining a target full-connection parameter in a preset full-connection parameter set.

22. An image compression method comprising:

23. An image compression method comprising:

24. An apparatus for training an image compression model, comprising:

25. An apparatus for training an image compression model, comprising:

26. An apparatus for training an image compression model, comprising:

27. An image compression apparatus comprising:

28. An image compression apparatus comprising:

29. An image compression apparatus comprising:

30. A computing device, comprising:

a memory and a processor;

the memory is configured to store computer-executable instructions, and the processor is configured to execute the computer-executable instructions, wherein the processor when executing the computer-executable instructions implements the steps of the method for training an image compression model according to any one of claims 1 to 10 or 11 or 12 or the method for image compression according to any one of claims 13 to 21 or 22 or 23.

31. A computer readable storage medium storing computer instructions which, when executed by a processor, carry out the steps of a training method of an image compression model according to any one of claims 1 to 10 or 11 or 12 or an image compression method according to any one of claims 13 to 21 or 22 or 23.