CN116248897A - Image processing method, system, electronic device and computer readable storage medium - Google Patents
Image processing method, system, electronic device and computer readable storage medium Download PDFInfo
- Publication number
- CN116248897A CN116248897A CN202310144875.8A CN202310144875A CN116248897A CN 116248897 A CN116248897 A CN 116248897A CN 202310144875 A CN202310144875 A CN 202310144875A CN 116248897 A CN116248897 A CN 116248897A
- Authority
- CN
- China
- Prior art keywords
- image
- jnd
- residual
- original image
- distorted
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/124—Quantisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
- H04N19/91—Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The application discloses an image processing method, an image processing system, electronic equipment and a computer readable storage medium, wherein the method comprises the following steps: dividing an original image into a JND point image and a residual image; inputting the JND point image and the residual image into a preset multi-level residual compensation network model, and outputting to obtain potential representation parameters; and sequentially quantizing and entropy coding the potential representation parameters to obtain a target compressed image corresponding to the original image. The method and the device improve the compression effect of image compression.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image processing method, an image processing system, an electronic device, and a computer readable storage medium.
Background
When image compression is performed, the traditional HEVC-SCC (High Efficiency Video Coding-Screen Content Coding) based mode of high efficiency video coding-screen content coding relies on the characteristics of the screen content of manual production and the traditional optimization interest rate to minimize the rate distortion loss, but the compression efficiency is severely limited due to the manual production operation, so that the compression effect on image compression is poor.
Disclosure of Invention
The invention mainly aims to provide an image processing method, an image processing system, electronic equipment and a computer readable storage medium, and aims to solve the technical problem of how to improve the compression effect of image compression.
To achieve the above object, the present application provides an image processing method, including:
dividing an original image into a JND point image and a residual image;
inputting the JND point image and the residual image into a preset multi-level residual compensation network model, and outputting to obtain potential representation parameters;
and sequentially quantizing and entropy coding the potential representation parameters to obtain a target compressed image corresponding to the original image.
Optionally, the multi-level residual compensation network model includes two sub-encoder models sharing the same network structure, and the network model formulas corresponding to the two sub-encoder moduli respectively include:
wherein said x m Representing a residual image, said x p Represents a JND dot image, where i represents the number of processing times, where
Representing a previously processed residual image of a processing input by a multi-level residual compensation network model, said +.>
The JND point image of the previous processing which represents the processing input of the multistage residual compensation network model, and the ≡indicates the multiplication by element.
Optionally, the step of inputting the JND point image and the residual image into a preset multi-level residual compensation network model and outputting to obtain the potential representation parameters includes:
inputting the residual image into a sub-encoder model for forward transformation processing to obtain a first transformation processing result;
forward transforming the JND point image and the first transforming result through another sub-encoder to obtain a second transforming result;
and determining potential representation parameters according to the preset processing times and the second transformation processing result.
Optionally, the step of dividing the original image into the JND point image and the residual image includes:
determining a distorted image corresponding to the original image in the JND data set;
determining a JND point image based on the original image and the distorted image;
and determining a residual image according to the JND point image and the original image.
Optionally, before the step of determining a distorted image in the JND dataset corresponding to the original image, the method further includes:
constructing a data table with a corresponding relation between an original image and a distorted image, and taking the data table as a JND data set, wherein the data table comprises a corresponding relation between at least one original image and at least one distorted image, and the distorted image is obtained by compressing the original image.
Optionally, the step of determining a JND point image based on the original image and the distorted image includes:
sequentially comparing a plurality of distorted images with the original image;
and if a distorted image with visual difference with the original image exists in the plurality of distorted images, taking the distorted image with visual difference as a JND point image, wherein the visual difference comprises image difference pixels which can be obviously recognized by human eyes of a user.
Optionally, determining a residual image according to the JND point image and the original image includes:
and carrying out pixel value subtraction on the same position points between the original image and the JND point image, and taking the original image with the pixel values of all the position points subjected to pixel value subtraction as a residual image.
In addition, in order to achieve the above object, the present application further provides an image processing system, which includes a preprocessing module, an analysis transformation module, and a quantization encoding module;
the preprocessing module is used for dividing an original image into a JND point image and a residual image;
the analysis transformation module is used for inputting the JND point image and the residual image into a preset multi-level residual compensation network model, and outputting to obtain potential representation parameters;
and the quantization coding module is used for sequentially quantizing and entropy coding the potential representation parameters to obtain a target compressed image corresponding to the original image.
In addition, in order to achieve the above object, the present application further provides an electronic device, including: the image processing device includes a memory, a processor, and an image processing program stored in the memory and executable on the processor, wherein the image processing program realizes the steps of the image processing method when executed by the processor.
In addition, in order to achieve the above object, the present application further provides a computer-readable storage medium having stored thereon an image processing program which, when executed by a processor, implements the steps of the image processing method as described above.
According to the method, the original image is divided into the JND point image and the residual image through just observing distortion guidance, a manual feature extraction mode is not relied on, the divided JND point image is enabled to be as close to the original image as possible, in image compression, the JND point image and the residual image are input into a multi-level residual compensation network model together, potential representation parameters are obtained through output, then quantization and entropy coding are carried out on the potential representation parameters, a target compression image is obtained, namely transformation processing is carried out through combining the JND point image and the residual image, so that bits can be distributed in a self-adaptive mode when the target compression image is generated through image compression according to the residual image, and the compression effect of image compression is improved.
Drawings
FIG. 1 is a flowchart of a first embodiment of an image processing method according to the present application;
FIG. 2 is a flowchart of a second embodiment of an image processing method according to the present application;
FIG. 3 is a schematic block diagram of an image processing system of the present application;
FIG. 4 is a schematic overall flow chart of the image processing method of the present application;
FIG. 5 is a schematic diagram of a preprocessing module in the image processing method of the present application;
FIG. 6 is a schematic diagram corresponding to a multi-level residual error compensation network model in the image processing method of the present application;
fig. 7 is a schematic device structure diagram of a hardware running environment related to an image processing method in an embodiment of the present application.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Because of low compression efficiency and poor compression effect on image compression based on HEVC-SCC (High Efficiency Video Coding-Screen Content Coding, high-efficiency video coding-screen content coding), in the embodiment, the end-to-end screen content image compression is guided by just observing distortion, bits can be adaptively allocated by using human eyes to perceive priori information, and the phenomenon of poor compression performance on the screen content image is avoided. And the reconstructed image can more accord with human eye perception rules when the right-to-visible distortion is used for guiding end-to-end screen content image compression, bits can be distributed in a self-adaptive mode by utilizing human eye perception priori information, and compression performance is remarkably improved.
Embodiments of the present application are further described below with reference to the accompanying drawings.
Referring to fig. 1, the present application provides an image processing method, in a first embodiment of the image processing method, the image processing method is applied to an image processing system, including:
step S10, dividing an original image into a JND point image and a residual image;
step S20, inputting the JND point image and the residual image into a preset multi-level residual compensation network model, and outputting to obtain potential representation parameters;
and step S30, sequentially quantizing and entropy coding the potential representation parameters to obtain a target compressed image corresponding to the original image.
In this embodiment, image compression may be implemented by encoding an image, and when one frame of image is compressed, since the one frame of image is first divided into a plurality of image blocks, each image block or one frame of image is subtracted from a prediction block obtained by prediction in a prediction mode to obtain a residual block, and then transformed and quantized, and encoded by an entropy encoder, to form an encoded bitstream, and transmitted to a decoding end, where the decoding end reconstructs the image using a synthetic transform after decoding. Alternatively, the image compression or image encoding and decoding process may include analyzing a context model of transformation, synthesis transformation, quantization and entropy encoding, on the basis of which the present embodiment adds a just-in-sight distortion-based preprocessing module (JND-ESN) and a multi-level residual compensation structure. It may thus be that the input raw image x is fed to a pre-processing module to eliminate human visual redundancy, and then the output of the pre-processing module is converted into a potential representation y by an analysis transformation module in which a multi-stage residual compensation structure is provided. And then, using additive noise approximation to replace quantization on the potential representation y to obtain compressed data, coding the compressed data through entropy coding to obtain a bit stream, and transmitting the bit stream to a decoding end. After decoding the bit stream at the decoding end, the decoded data is reconstructed into a reconstructed image using a synthetic transform. For example, as shown in fig. 4, an original image is input to a JND-ESN, a residual image and a JND point image are output, the JND point image and the residual image are input to an analysis transformation module, the JND point image and the residual image are processed by a multi-level residual compensation network model in the analysis transformation module, potential representation parameters are output, and then the potential representation parameters are processed by two coding branches, wherein the first branch is processed by a quantization module and an entropy coding module to obtain a bar code stream, and the bar code stream is decoded by an entropy decoder at a decoding end, and entropy estimation is performed to obtain an entropy estimation result. The other branch is processed by the quantization module and the entropy coding module, the processing result and the entropy estimation result are coded together to obtain another bar code stream, and the bar code stream is decoded according to the entropy decoder in the decoding end to obtain a reconstructed image.
Further, for step S10, the original image is divided into a JND point image and a residual image;
in this embodiment, when image compression is required, an original image to be subjected to image compression is acquired first, and then the original image is subjected to segmentation processing to obtain a JND point image and a residual image. The original image may be a frame image, or may be a certain image block after dividing the image, which is not limited herein. The JND point image may be an image generated by inputting an original image to a JND model set in advance for processing. The residual image may be a JND image generated by the original processing module after processing the original image, where the JND image includes energy removed by the preprocessing module.
The minimum perceptible error (JND, just Noticeable Distortion) is used to represent the maximum image distortion that cannot be perceived by human eyes, and represents the tolerance of human eyes to image changes.
In this embodiment, the original image may be subjected to a segmentation process by the preprocessing module to segment the original image into the JND point image and the residual image. Alternatively, the preprocessing module may be set to JND-ESN, and may generate a low-energy image by reducing perceptual redundancy, that is, an image after removing the signal components that are difficult to perceive, as shown in fig. 5, which is a JND-ESN framework of the preprocessing module, including the GT dataset, the original image, the preprocessing module, the JND dot image, and the residual image. And the preprocessing module can enable the image to be as close to the original image as possible, so that the JND-ESN can realize the perception lossless image preprocessing.
And it should be noted that the residual image includes energy removed by the preprocessing module. The higher the pixel value at a position in the residual image, the higher the distortion level at that position. Therefore, in the subsequent process, the compression distortion of the areas with the same code rate has larger influence, and the areas need to be compensated. At this time, pixel value compensation can be performed according to the multi-level residual compensation network model.
For example, if there is an original image x ori If the compression processing is needed, the original image x can be processed by the preprocessing module ori Segmentation into predicted JND point images x p And residual image x m . Wherein, JND point image x p And residual image x m Expressed as:
x p ＝E(x ori )；
x m ＝x ori -x p ；
where E () represents the ESN-JND network process. E (x) ori ) Representing the original image being network processed by the ESN-JND.
Further, for step S20, the JND point image and the residual image are input into a preset multi-level residual compensation network model, and potential representation parameters are obtained through output;
in this embodiment, after the JND point image and the residual image are obtained by the preprocessing module, the JND point image and the residual image may be synthesized to perform analysis transformation processing, so as to obtain the potential representation parameters. Wherein the potential representation parameter may be a compressed hidden vector that the encoder converts the image. The JND point image and the residual image are required to be subjected to network processing through a multi-level residual compensation network model in the process of analysis transformation processing.
Further, the multi-level residual compensation network model includes two sub-encoder models sharing the same network structure, and the network model formulas corresponding to the two sub-encoder modules include:
wherein said x m Representing a residual image, said x p Represents a JND dot image, where i represents the number of processing times, where
Representing a previously processed residual image of a processing input by a multi-level residual compensation network model, said +.>
The JND point image of the previous processing which represents the processing input of the multistage residual compensation network model, and the ≡indicates the multiplication by element.
For example, as shown in fig. 6, the multi-level residual compensation network model includes two sub-encoder models sharing the same network structure, and a multi-level network architecture may be set in the sub-encoder models, and the number of layers of the network architecture is related to i in the network model formula, that is, i in the network model formula may be set equal to the number of layers of the network architecture. As shown in fig. 3, with a four-tier network architecture, i may be 4. Wherein, the first layer network architecture is RBS, the second layer network architecture is RB and RBS, the third layer network architecture is CBAM and RBS, and the fourth layer network architecture is RB and RBS. Wherein, RB includes Conv (convolutional layer), GDN (normalized layer), LRELU (activation function layer), conv, GDN, and LRELU. RBS includes Conv With Stride, GDN, LRELU, conv With Stride, GDN and LRELU. And RB denotes a residual block, RBs denotes one downsampling on the basis of RB.
That is, in this embodiment, when multiple transformations are required for the residual image and the JND point image, a transformation process may be performed on the pixel points in the residual image to obtain a transformation process result, and when the transformation process is performed on the same pixel point of the JND point image, the transformation process is performed on the same pixel point of the JND point image at this time in combination with the transformation process result obtained when the transformation process is performed on the same pixel point of the JND point image at the previous time, so as to compensate the region with a higher distortion level in the JND point image, and achieve the purpose of adaptively allocating bits according to the residual image.
Further, step S20, inputting the JND point image and the residual image into a preset multi-level residual compensation network model, and outputting to obtain potential representation parameters, includes:
step a, inputting the residual image into a sub-encoder model for forward transformation processing to obtain a first transformation processing result;
b, performing forward transformation processing on the JND point image and the first transformation processing result through another sub-encoder to obtain a second transformation processing result;
and c, determining potential representation parameters according to the preset processing times and the second transformation processing result.
In this embodiment, when performing the analysis conversion processing on the JND point image and the residual image, the number of processing times that the analysis conversion processing needs to be performed may be determined first, and then the analysis conversion processing of the number of processing times may be performed on the JND point image and the residual image. And when analysis transformation processing is performed, the residual image can be input into a sub-encoder to perform first forward transformation processing, so as to obtain a first residual transformation result. And (3) inputting the JND point image to another sub-encoder for performing first forward conversion processing to obtain a first JND conversion result. And when the second transformation is performed, the first residual transformation result is directly subjected to the second forward transformation by a sub-encoder until the number of times of the processing is reached. Wherein, the network model formula corresponding to one sub-encoder can be
When the first JND conversion result is subjected to the second conversion process, the first residual conversion result and the first JND conversion result need to be subjected to element-by-element multiplication, and then the processing result of the element-by-element multiplication and the first JND conversion result are combined by another sub-encoder to perform the second forward conversion process until the processing times are reached. Wherein, the network model formula corresponding to another sub-encoder may be:
and the result of the last transform process of the two sub-encoders may be used as a potential representation parameter.
Further, for step S30, the potential representation parameters are quantized and entropy encoded sequentially, so as to obtain a target compressed image corresponding to the original image.
In this embodiment, after the latent representation parameter is obtained, quantization processing may be performed on the latent representation parameter, for example, using additive noise approximation instead of quantization, and entropy encoding processing may be performed after quantization is completed, so as to achieve a target compressed image corresponding to the original image. I.e. the compression of the original image has been completed at this time.
In this embodiment, the original image is split into the JND point image and the residual image through just noticeable distortion guidance, and the split JND point image is made to be as close as possible to the original image without relying on a manual feature extraction mode, and when the image is compressed, the JND point image and the residual image are input into a multi-level residual compensation network model together, the latent representation parameters are output to obtain, and then quantization and entropy coding are performed on the latent representation parameters to obtain a target compressed image, that is, transformation processing is performed by combining the JND point image and the residual image, so that bits can be allocated adaptively when the target compressed image is generated by performing image compression according to the residual image, and the compression effect of image compression is improved.
Further, on the basis of the first embodiment described above, a second embodiment of the image processing method of the present application is proposed, and referring to fig. 2, in the second embodiment, step S10, a step of dividing an original image into a JND point image and a residual image includes:
step d, determining a distorted image corresponding to the original image in the JND data set;
step e, determining a JND point image based on the original image and the distorted image;
and f, determining a residual image according to the JND point image and the original image.
In this embodiment, when the original image is divided into the JND point image and the residual image, all the distorted images in the JND dataset corresponding to the original image in a matching manner may be determined first, then the original image and all the distorted images are input to the preprocessing module, and each distorted image is compared with the original image by the preprocessing module, so as to select a proper distorted image as the JND point image. The residual image is then determined by comparing the original image with the JND dot image. Wherein the original image can be obtained by combining the residual image and the JND dot image.
In this embodiment, the original image is split into the JND point image and the residual image by determining the distorted image corresponding to the original image in the JND dataset, determining the JND point image according to the original image and the distorted image, and determining the residual image according to the JND point image and the original image.
Further, step d, before the step of determining a distorted image in the JND dataset corresponding to the original image, includes:
and step x, constructing a data table with the corresponding relation between an original image and a distorted image, and taking the data table as a JND data set, wherein the data table comprises the corresponding relation between at least one original image and at least one distorted image, and the distorted image is obtained by compressing the original image.
In this embodiment, before the original image is divided into the JND point image and the residual image by the preprocessing module, a JND data set corresponding to the original image needs to be constructed. The method can acquire a plurality of original images in advance, acquire a compressed image corresponding to each original image, and take the compressed image as a distorted image. Wherein, the distortion degree of each distorted image corresponding to the original image is different. At this time, a blank data table can be constructed, all original images are filled into the blank data table, the blank data table is sequentially filled according to the corresponding relation between the original images and the distorted images, the data table with the corresponding relation between the original images and the distorted images is obtained, and the data table is used as a JND data set.
In the present embodiment, by constructing a data table including correspondence relation between original image and distorted image, and taking the data table as the JND dataset, it is possible to facilitate subsequent determination of JND dot images from the JND dataset.
Further, step e, determining a JND point image based on the original image and the distorted image, includes:
step e1, comparing a plurality of distorted images with the original image in sequence;
and e2, if a distorted image with visual difference with the original image exists in the plurality of distorted images, taking the distorted image with visual difference as a JND point image, wherein the visual difference comprises image difference pixels which can be obviously recognized by human eyes of a user.
In this embodiment, a plurality of distorted images may be compared with an original image in sequence, whether there is a large visual difference between the distorted image and the original image is determined, and when it is determined that there is a distorted image having a large visual difference with the original image, the distorted image may be directly used as a JND point image. The method for judging whether the distorted image with the visual difference from the original image exists in the plurality of distorted images may include at least one of the following:
mode one: and simultaneously placing each distorted image and the original image in a screen for comparison, and after detecting a comparison consistent instruction or a comparison inconsistent instruction input by a user, replacing the distorted images in the screen until all the distorted images are compared. And counting all the distorted images corresponding to the inconsistent comparison instruction, and selecting one distorted image closest to the original image from all the distorted images corresponding to the inconsistent comparison instruction as a JND point image.
Mode two: determining the distortion degree of each distorted image relative to the original image, taking the distorted image corresponding to the distortion degree within a preset threshold value range as a target distorted image, and selecting one of the target distorted images as a JND point image.
Mode three: and placing the distorted images in the screen in turn according to the distortion degree of each distorted image relative to the original image so as to compare with the original image in the screen, and taking the distorted images displayed on the screen as JND point images when the user inputs the inconsistent comparison instruction after detecting the inconsistent comparison instruction input by the user.
In this embodiment, by comparing the plurality of distorted images with the original image in sequence and using the distorted image having a visual difference with the original image as the JND point image, it is ensured that the acquired JND point image is attached to the original image as much as possible, and bits can be adaptively allocated during subsequent image compression according to human eye perception learning.
Further, step f, determining a residual image according to the JND point image and the original image, includes:
and f1, subtracting the pixel values of the same position points between the original image and the JND point image, and taking the original image with the pixel values of all the position points subtracted as a residual image.
In this embodiment, after the JND point image is determined, the original image and the JND point image may be input to the preprocessing module for processing, for example, for each pixel point in the original image, subtraction processing may be performed on the pixel points in the JND point image, and the original image after the pixel values of all the position points are subtracted is used as the residual image. In the residual image, the higher the pixel value is, the higher the distortion level at the position where the pixel value is located is.
In this embodiment, by subtracting all the pixel points in the original image from all the pixel points in the JND point image, a residual image is obtained, so that the validity of the obtained residual image is ensured.
In addition, referring to fig. 3, the present application further provides an image processing system, where the image processing system includes a preprocessing module a10, an analysis transformation module a20, and a quantization coding module a30;
the preprocessing module a10 is configured to divide an original image into a JND point image and a residual image;
the analysis transformation module a20 is configured to input the JND point image and the residual image into a preset multi-level residual compensation network model, and output the JND point image and the residual image to obtain potential representation parameters;
and the quantization coding module A30 is used for sequentially quantizing and entropy coding the potential representation parameters to obtain a target compressed image corresponding to the original image.
Optionally, the multi-level residual compensation network model includes two sub-encoder models sharing the same network structure, and the network model formulas corresponding to the two sub-encoder moduli respectively include:
wherein said x m Representing a residual image, said x p Represents a JND dot image, where i represents the number of processing times, where
Representing a previously processed residual image of a processing input by a multi-level residual compensation network model, said +.>
The JND point image of the previous processing which represents the processing input of the multistage residual compensation network model, and the ≡indicates the multiplication by element.
Optionally, the analysis transformation module a20 is configured to:
inputting the residual image into a sub-encoder model for forward transformation processing to obtain a first transformation processing result;
forward transforming the JND point image and the first transforming result through another sub-encoder to obtain a second transforming result;
and determining potential representation parameters according to the preset processing times and the second transformation processing result.
Optionally, the preprocessing module a10 is configured to:
determining a distorted image corresponding to the original image in the JND data set;
determining a JND point image based on the original image and the distorted image;
and determining a residual image according to the JND point image and the original image.
Optionally, the preprocessing module a10 is configured to:
constructing a data table with a corresponding relation between an original image and a distorted image, and taking the data table as a JND data set, wherein the data table comprises a corresponding relation between at least one original image and at least one distorted image, and the distorted image is obtained by compressing the original image.
Optionally, the preprocessing module a10 is configured to:
sequentially comparing a plurality of distorted images with the original image;
and if a distorted image with visual difference with the original image exists in the plurality of distorted images, taking the distorted image with visual difference as a JND point image, wherein the visual difference comprises image difference pixels which can be obviously recognized by human eyes of a user.
Optionally, the preprocessing module a10 is configured to:
and carrying out pixel value subtraction on the same position points between the original image and the JND point image, and taking the original image with the pixel values of all the position points subjected to pixel value subtraction as a residual image.
In addition, the application also provides an electronic device, which comprises a memory, a processor and an image processing program stored on the memory and capable of running on the processor, wherein the image processing program realizes the steps of the image processing method when being executed by the processor.
In addition, in an embodiment, fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, where, as shown in fig. 7, the electronic device includes a processor, and optionally further includes an internal bus, a network interface, and a memory at a hardware level. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory (non-volatile Memory), such as at least 1 disk Memory. Of course, the electronic device may also include hardware required for other services. The processor, network interface, and memory may be interconnected by an internal bus, which may be an ISA (Ind ustry Standa rd Architecture ) bus, a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus, or EISA (Extended Industry Standard Architecture ) bus, among others. The buses may be classified as address buses, data buses, control buses, etc. For ease of illustration, only one bi-directional arrow is shown in FIG. 7, but not only one bus or type of bus. And the memory is used for storing programs. In particular, the program may include program code including computer-operating instructions. The processor reads the corresponding computer program from the nonvolatile memory into the memory and then runs, forming the shared resource access control device on a logic level. And a processor for executing the program stored in the memory, and specifically for executing the steps of the image processing method.
The specific implementation manner of the electronic device in the present application is substantially the same as the embodiments of the image processing method described above, and will not be repeated here.
In addition, in order to achieve the above object, the present application further provides a computer-readable storage medium having stored thereon an image processing program which, when executed by a processor, implements the steps of the image processing method as described above.
The specific embodiments of the computer readable storage medium are basically the same as the embodiments of the image processing method described above, and are not repeated here.
Those of ordinary skill in the art will appreciate that all or some of the steps, systems, functional modules/units in the apparatus, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between the functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed cooperatively by several physical components. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
Claims (10)
1. An image processing method, characterized in that the image processing method comprises:
dividing an original image into an just noticeable distortion JND point image and a residual image;
inputting the JND point image and the residual image into a preset multi-level residual compensation network model, and outputting to obtain potential representation parameters;
and sequentially quantizing and entropy coding the potential representation parameters to obtain a target compressed image corresponding to the original image.
2. The image processing method according to claim 1, wherein the multi-level residual compensation network model includes two sub-encoder models sharing the same network structure, and the network model formulas corresponding to the two sub-encoder modules include:
wherein said x m Representing a residual image, said x p Represents a JND dot image, where i represents the number of processing times, where
Representing a previously processed residual image of a processing input by a multi-level residual compensation network model, said +.>
The JND point image of the previous processing which represents the processing input of the multistage residual compensation network model, and the ≡indicates the multiplication by element.
3. The image processing method according to claim 2, wherein the step of inputting the JND point image and the residual image into a preset multi-level residual compensation network model and outputting the resulting potential representation parameters includes:
inputting the residual image into a sub-encoder model for forward transformation processing to obtain a first transformation processing result;
forward transforming the JND point image and the first transforming result through another sub-encoder to obtain a second transforming result;
and determining potential representation parameters according to the preset processing times and the second transformation processing result.
4. The image processing method according to claim 1, wherein the step of dividing the original image into the JND point image and the residual image comprises:
determining a distorted image corresponding to the original image in the JND data set;
determining a JND point image based on the original image and the distorted image;
and determining a residual image according to the JND point image and the original image.
5. The image processing method according to claim 4, wherein the step of determining a distorted image in the JND dataset corresponding to the original image, comprises:
constructing a data table with a corresponding relation between an original image and a distorted image, and taking the data table as a JND data set, wherein the data table comprises a corresponding relation between at least one original image and at least one distorted image, and the distorted image is obtained by compressing the original image.
6. The image processing method according to claim 4, wherein the step of determining a JND dot image based on the original image and the distorted image comprises:
sequentially comparing a plurality of distorted images with the original image;
and if a distorted image with visual difference with the original image exists in the plurality of distorted images, taking the distorted image with visual difference as a JND point image, wherein the visual difference comprises image difference pixels which can be obviously recognized by human eyes of a user.
7. The image processing method of claim 4, wherein the determining a residual image from the JND dot image and the original image comprises:
and carrying out pixel value subtraction on the same position points between the original image and the JND point image, and taking the original image with the pixel values of all the position points subjected to pixel value subtraction as a residual image.
8. An image processing system is characterized by comprising a preprocessing module, an analysis conversion module and a quantization coding module;
the preprocessing module is used for dividing an original image into a JND point image and a residual image;
the analysis transformation module is used for inputting the JND point image and the residual image into a preset multi-level residual compensation network model, and outputting to obtain potential representation parameters;
and the quantization coding module is used for sequentially quantizing and entropy coding the potential representation parameters to obtain a target compressed image corresponding to the original image.
9. An electronic device, the electronic device comprising: memory, a processor and an image processing program stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the image processing method according to any one of claims 1 to 7.
10. A computer-readable storage medium, on which an image processing program is stored, which when executed by a processor implements the steps of the image processing method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310144875.8A CN116248897A (en) | 2023-02-16 | 2023-02-16 | Image processing method, system, electronic device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310144875.8A CN116248897A (en) | 2023-02-16 | 2023-02-16 | Image processing method, system, electronic device and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116248897A true CN116248897A (en) | 2023-06-09 |
Family
ID=86629221
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310144875.8A Pending CN116248897A (en) | 2023-02-16 | 2023-02-16 | Image processing method, system, electronic device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116248897A (en) |
-
2023
- 2023-02-16 CN CN202310144875.8A patent/CN116248897A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111641832B (en) | Encoding method, decoding method, device, electronic device and storage medium | |
| JP2021521517A (en) | HDR image representation using neural network mapping | |
| US11335034B2 (en) | Systems and methods for image compression at multiple, different bitrates | |
| CN111641826B (en) | Method, device and system for encoding and decoding data | |
| US11955994B2 (en) | Data processing apparatuses, methods, computer programs and computer-readable media | |
| CN110753225A (en) | Video compression method and device and terminal equipment | |
| KR20070053098A (en) | Decoding apparatus, inverse quantization method, and computer readable medium | |
| US7778468B2 (en) | Decoding apparatus, dequantizing method, and program thereof | |
| CN115988215A (en) | Variable bit rate image compression method, system, device, terminal and storage medium | |
| US10129565B2 (en) | Method for processing high dynamic range video in order to improve perceived visual quality of encoded content | |
| CN115880381A (en) | Image processing method, image processing apparatus, and model training method | |
| CN112715029A (en) | AI encoding apparatus and operating method thereof, and AI decoding apparatus and operating method thereof | |
| CN116248897A (en) | Image processing method, system, electronic device and computer readable storage medium | |
| EP4315866A1 (en) | Multi-distribution entropy modeling of latent features in image and video coding using neural networks | |
| CN115460415A (en) | Video compression method for man-machine mixed vision | |
| EP3154023A1 (en) | Method and apparatus for de-noising an image using video epitome | |
| CN111050167B (en) | Method and apparatus for recovering degraded frames resulting from reconstruction of source frames | |
| JP2016082395A (en) | Encoder, coding method and program | |
| US20240121414A1 (en) | Hybrid inter-frame coding using an autoregressive model | |
| CN117425013B (en) | Video transmission method and system based on reversible architecture | |
| KR102471796B1 (en) | Method and system for preprocessing cognitive video using saliency map | |
| CN114359418A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| WO2022229495A1 (en) | A method, an apparatus and a computer program product for video encoding and video decoding | |
| WO2022187026A2 (en) | Dependent quantization state transition method for video coding | |
| CN115880379A (en) | Semantic scalable image coding method, system, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |