CN116325752A

CN116325752A - Image compression method and device

Info

Publication number: CN116325752A
Application number: CN202080105452.XA
Authority: CN
Inventors: 陈绍林; 付洋
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2020-09-29
Filing date: 2020-09-29
Publication date: 2023-06-23
Also published as: WO2022067526A1

Abstract

The embodiment of the application provides an image compression method and device, relates to the field of image processing, and aims to realize quantization of different image blocks in a frame of image according to different quantization step sizes, meet different visual demands of users on different image areas in different scenes, and reduce code rate so as to reduce storage space of pictures. The method specifically comprises the following steps: performing first transformation on an image block in an image to be compressed to obtain a frequency domain coefficient set of the image block; performing second quantization operation on the frequency domain coefficient set to obtain a second quantized coefficient set; the second quantization operation is used for removing high-frequency components in the frequency domain coefficient set, and maintaining or reducing the amplitude of low-frequency components in the frequency domain coefficient set; the number of high frequency components removed by the second quantization operation is related to the image block; performing first quantization operation on the second quantized coefficient set based on the first quantization matrix to obtain a first quantized coefficient set; and carrying out entropy coding on the first quantized coefficient set to obtain compressed data of the image block.

Description

Image compression method and device

Technical Field

The embodiment of the application relates to the field of image processing, in particular to an image compression method and device.

Background

The joint photographic experts group (joint photographic expert group, JPEG) is a standard for image compression, and is a mainstream compression format for image storage and transmission and is widely used because of its simplicity and wide compatibility.

In the JPEG compression process, a frame of image is first divided into a series of 16×16 image blocks, each 16×16 image block is further divided into 4 8×8 image blocks, and then the JPEG compression process as illustrated in fig. 1 is used for each 8×8 image block.

Each of the luminance component and the chrominance component in an 8 x 8 image block is compressed as illustrated in fig. 1 to complete the compression of the 8 x 8 image block. As illustrated in fig. 1, one component (luminance or chrominance) of an 8×8 image block is first discrete cosine transformed (discrete cosine transform, DCT), the spatial data block is transformed into DCT coefficients represented as 8×8 in the frequency domain, and different frequency components contained in the spatial data block are separated in the frequency domain to obtain a direct current component (DC) in the upper left corner and an alternating current component (AC) in other positions in the DCT coefficients. Then, according to an externally input 8×8 quantization matrix, the DCT coefficients are quantized by dividing the DCT coefficients by the element values (called quantization step sizes) at the corresponding positions in the quantization matrix, and the quantized values are truncated or rounded to the nearest integer. The data volume in the frequency domain DCT coefficient is reduced through quantization, so that the purpose of compression is realized. In general, according to the characteristic that human eyes are insensitive to high-frequency coefficients, quantization step sizes are gradually increased from low to high in a quantization matrix along with the increase of frequency, so that a better compression ratio is achieved. And finally, carrying out entropy coding on the quantized data to obtain compressed data.

The key link of JPEG compression is a quantization process, but all image blocks in a frame of image can only be quantized according to the same set of quantization matrix (the same components of different image blocks can only be quantized according to the same quantization matrix), and the compression rate is fixed, so that different visual demands of users on different image areas in a image cannot be met, and meanwhile, the compressed data code rate obtained by the current compression scheme is high, and the storage space is occupied.

Disclosure of Invention

The embodiment of the application provides an image compression method and device, which can realize that different image blocks in one frame of image are quantized according to different quantization step sizes, so that different visual demands of users on different image areas in different scenes can be met; for areas which are insensitive to vision or are not focused by users, a larger quantization step can be adopted to further improve the compression rate and reduce the code rate so as to reduce the storage space of pictures.

In order to achieve the above purpose, the embodiment of the application adopts the following technical scheme:

in a first aspect, there is provided an image compression method, which may include: performing first transformation on an image block in an image to be compressed to obtain a frequency domain coefficient set of the image block; the image block is a continuous area comprising a plurality of pixel points in the image to be compressed; performing second quantization operation on the frequency domain coefficient set to obtain a second quantized coefficient set; the second quantization operation is used for removing high-frequency components in the frequency domain coefficient set, and maintaining or reducing the amplitude of low-frequency components in the frequency domain coefficient set; the number of high frequency components removed by the second quantization operation is related to the image block; the number of high-frequency components removed by the second quantization operation is greater than or equal to the number of high-frequency components removed by the first quantization operation based on the first quantization matrix; performing first quantization operation on the second quantized coefficient set based on the first quantization matrix to obtain a first quantized coefficient set; and carrying out entropy coding on the first quantized coefficient set to obtain compressed data of the image block.

According to the image compression method provided by the embodiment of the application, the second quantization operation related to the image block is performed on the frequency coefficient set of the data block once, the high-frequency component is removed, the low-frequency component is kept or reduced in amplitude, the first quantization operation is performed on the basis of the configured first quantization matrix, and then entropy coding is performed to complete compression. In this way, the quantization of the correlation of different image blocks is different, and different image blocks in one frame of image can be quantized according to different quantization step sizes, so that different visual demands of users on different image areas in different scenes can be met; the size of the encoded JPEG format image is adjusted according to actual requirements; meanwhile, the region of interest can be flexibly encoded on the premise of meeting the JPEG standard, the quality of the region of interest is ensured, the code rate consumed by other regions is reduced, the code rate (storage space) is further saved, and the compression rate is further improved.

In one possible implementation manner, performing a second quantization operation on the frequency domain coefficient set to obtain a second quantized coefficient set, including: acquiring a region of interest (Region Of Interest, ROI) in an image to be compressed; if the image block is in the ROI of the image to be compressed, reserving the first N1 elements arranged in the frequency domain coefficient set of the image block according to the ZigZag sequence, and setting the rest elements to zero to obtain a second quantization coefficient set of the image block; if the image block is positioned in the non-ROI of the image to be compressed, the first N2 elements which are arranged in the frequency domain coefficient set according to the ZigZag sequence are reserved, and the rest elements are set to zero, so that a second quantization coefficient set of the image block is obtained. N1 is greater than N2. The second quantization operation is performed through the implementation mode, the implementation process is simple, the calculation complexity is low, and the cost is saved.

In another possible implementation, N1 is 63 and N2 is 32.

In one possible implementation manner, the image compression method provided by the application may further include: acquiring a characteristic value of the image block, wherein the characteristic value is used for indicating any one of the following characteristics of the image block: spatial domain features, frequency domain features, or texture features. Performing a second quantization operation on the frequency domain coefficient set to obtain a second quantized coefficient set may include: and determining a second quantization matrix of the image block according to the quantization information of the first quantization matrix of the image block and the characteristic value of the image block, and performing a third operation on the frequency coefficient set based on the second quantization matrix to obtain a second quantization coefficient set of the image block.

The feature value of the image block is used for reflecting the complexity degree of the image block for the visual system, and the higher the feature value is, the more complex and the less sensitive the image block is for the visual system, the larger quantization step size can be adopted for quantization. In the implementation mode, after the first quantization matrix is determined according to the quantization information of the first quantization matrix of the image block, the second quantization matrix of the image block is determined by combining the characteristic value of the image block, so that the determined second quantization matrix is related to the characteristics of the image block, and further, the second quantization operation is related to the characteristics of the image block. In this way, the second quantization matrixes of the same component set of the image blocks with different features are different, and different image blocks in one frame of image can be quantized according to different quantization step sizes through the second quantization operation.

The first quantization matrix is a quantization matrix for performing a first quantization operation, and the first quantization matrix of the same component set of different image blocks in one frame of image is the same. The quantization information of the first quantization matrix is used to uniquely determine the first quantization matrix. The quantization information of the first quantization matrix may be the first quantization matrix itself, or the quantization information of the first quantization matrix may be a quality factor that determines the first quantization matrix, and the first quantization matrix may be calculated according to the quality factor and the first relational expression. The first relational expression is quality factor and standard quantization matrix QM _S Is a relational expression of (2).

Wherein the first relational expression satisfies the following relationship: QM (quality control model) _qf ＝floor(S×QM _s +50)。

QM _qf For the calculated quantization matrix, floor (·) is a rounding operation, S satisfies the following expression:

in one possible implementation, the third operation may be an inverse quantization operation that performs the first quantization operation and the first quantization operation sequentially.

In another possible implementation manner, the third operation may include comparing an element in the frequency domain coefficient set of the image block with an element in the second quantization matrix of the image block at a position corresponding to the element, and determining the second quantization coefficient set of the image block based on a preset rule. The preset rule may include: when the element of the first position in the frequency domain coefficient set is larger than the element of the first position in the second quantization matrix, reserving the element of the first position in the frequency domain coefficient set; the first position is any position in the frequency domain coefficient set; when the element of the first position in the frequency domain coefficient set is smaller than the element of the first position in the second quantization matrix, setting the element of the first position in the frequency domain coefficient set to zero; when the element of the first position in the frequency domain coefficient set is equal to the element of the first position in the second quantization matrix, the element of the first position in the frequency domain coefficient set is reserved or zeroed.

In another possible implementation manner, the image compression method provided by the application may further include: acquiring a characteristic value of the image block, wherein the characteristic value is used for indicating any one of the following characteristics of the image block: spatial domain features, frequency domain features, or texture features. Performing a second quantization operation on the frequency domain coefficient set to obtain a second quantized coefficient set may include: determining a second quantization matrix of the image block according to quantization information of the first quantization matrix of the image block and characteristic values of the image block, and sequentially performing first quantization operation and inverse quantization operation on the frequency domain coefficient set of the image block based on the second quantization matrix of the image block to obtain a second quantization coefficient set of the image block.

In this implementation, the high frequency components in the set of frequency domain coefficients are removed and the magnitude of the low frequency components in the set of frequency domain coefficients is maintained or reduced by the determined second quantization matrix associated with the image block. And performing a first quantization operation through a determined second quantization matrix related to the image block, removing high-frequency components in the frequency domain coefficient set, and maintaining or reducing the amplitude of low-frequency components in the frequency domain coefficient set through performing an inverse quantization operation after the first quantization operation on the result after the first quantization operation, so as to realize compatibility with the JPEG compression standard.

In another possible implementation manner, the image compression method provided by the application may further include: acquiring a characteristic value of the image block, wherein the characteristic value is used for indicating any one of the following characteristics of the image block: spatial domain features, frequency domain features, or texture features. Performing a second quantization operation on the frequency domain coefficient set to obtain a second quantized coefficient set, which may be specifically implemented as: determining a second quantization matrix of the image block according to quantization information of the first quantization matrix of the image block and characteristic values of the image block; comparing the element in the frequency domain coefficient set of the image block with the element in the position corresponding to the element in the second quantization matrix of the image block, and determining the second quantization coefficient set of the image block based on a preset rule. The preset rule may include: when the element of the first position in the frequency domain coefficient set is larger than the element of the first position in the second quantization matrix, reserving the element of the first position in the frequency domain coefficient set; the first position is any position in the frequency domain coefficient set; when the element of the first position in the frequency domain coefficient set is smaller than the element of the first position in the second quantization matrix, setting the element of the first position in the frequency domain coefficient set to zero; when the element of the first position in the frequency domain coefficient set is equal to the element of the first position in the second quantization matrix, the element of the first position in the frequency domain coefficient set is reserved or zeroed.

The comparison operation replaces the first quantization operation and the inverse quantization operation, so that multiplication and division operations are avoided, and the implementation cost of hardware logic is reduced.

In another possible implementation manner, determining the second quantization matrix of the image block according to the quantization information and the eigenvalue of the first quantization matrix includes: according to a second quantization matrix QM ₂ And a first quantization matrix QM ₁ Relationship between feature value X, and QM is determined ₂ . Wherein QM ₂ And QM ₁ The relationship of the feature value X satisfies the following expression: QM (quality control model) ₂ ＝QM ₁ +F ₁ (X). Wherein F is ₁ (. Cndot.) is a first predetermined function.

In the implementation manner, the expression of the second quantization matrix can be configured and determined according to actual experience, so that the determined quantization matrix can better reflect the characteristics of the image block, further the second quantization operation matched with the characteristics of the image block is realized, and the purposes of giving consideration to visual experience, user experience and image compression rate in the image compression process are achieved.

In another possible implementation, the quantization information of the first quantization matrix may include determining a first quality factor QF of the first quantization matrix ₁ ，QF ₁ For according to the firstThe relational expression calculates a first quantization matrix. Wherein the first relational expression is a quality factor and a standard quantization matrix QM _S Is a relational expression of (2). Correspondingly, according to the quantization information of the first quantization matrix and the eigenvalue, determining the second quantization matrix of the image block may be specifically implemented as: determining a quality factor offset value delta QF according to the characteristic value X; according to delta QF and QF ₁ Calculating a second quality factor QF ₂ The method comprises the steps of carrying out a first treatment on the surface of the According to QF ₂ And a first relational expression determining a second quantization matrix. Wherein Δqf satisfies the following expression: Δqf=f ₂ (X)；F ₂ (. Cndot.) is a second predetermined function; QF (quad Flat No lead) ₂ The following expression is satisfied: QF (quad Flat No lead) ₂ ＝|QF ₁ |-F ₃ (ΔQF)，F ₃ (. Cndot.) is a third predetermined function.

In a scene of determining a quantization matrix through quality factors, an expression of a second quality factor is determined according to actual experience configuration, and then the second quantization matrix is determined according to the second quality factor, so that the determined quantization matrix can better reflect the characteristics of an image block, further a second quantization operation matched with the characteristics of the image block is realized, and the purposes of giving consideration to visual experience, user experience and image compression rate in an image compression process are achieved.

In another possible implementation, the feature value may be used to indicate a texture feature of an image block, and the feature value X may be calculated according to an expression of the feature value X and a pixel value in the image block. Wherein, the expression of the characteristic value X and the pixel value in the image block satisfies the following relation:

Wherein,

Pix _i for the pixel value of the ith pixel point in the image block to be processed, M is the imageThe block width, L, is the height of the image block.

In another possible implementation manner, the image compression method provided by the application may further include: acquiring an ROI in an image to be compressed; if the image block is outside the ROI, acquiring the characteristic value of the image block and determining a second quantization matrix of the image block according to the quantization information and the characteristic value of the first quantization matrix. Or if the image block is in the ROI, performing a first quantization operation on the frequency domain coefficient set based on the first quantization matrix to obtain a first quantization coefficient set, and performing entropy coding on the first quantization coefficient set to obtain compressed data of the image block.

The image blocks of the non-ROI area are quantized twice, the purposes of increasing the quantization step length, improving the compression rate and reducing the storage space are achieved, the image blocks of the ROI area are quantized according to the JPEG compression protocol, the region of interest is flexibly encoded, the quality of the region of interest is ensured, the code rate consumed by other regions is reduced, the code rate (storage space) is further saved, and the compression rate is further improved.

In another possible implementation manner, the ROI in the image to be compressed is obtained, which may be specifically implemented as: and receiving the input ROI region information, and determining the region indicated by the ROI region information in the image to be compressed as the ROI. The ROI area information is used to indicate the coordinate position of the ROI in the image to be compressed. In the implementation manner, the user can better meet the user requirements by inputting the ROI area information by the user and determining the position of the ROI in the image to be compressed.

In another possible implementation manner, the ROI in the image to be compressed is obtained, which may be specifically implemented as: and receiving Map image information of the input image to be compressed, and determining a region of interest (ROI) in the image to be compressed based on the Map image information, wherein the Map image information is used for indicating whether each image block in the image to be compressed is in the region of the ROI. In the implementation mode, the user can better meet the user requirements by inputting Map image information of the image to be compressed and determining the position of the ROI in the image to be compressed.

In another possible implementation manner, the ROI in the image to be compressed is obtained, which may be specifically implemented as: the ROI in the image to be compressed is acquired by artificial intelligence (artificial intelligence, AI) identification techniques. The machine automatically acquires the ROI by adopting an AI identification technology, so that the efficiency is high and the implementation is easy.

In a second aspect, there is provided an image compression apparatus, which may include: the device comprises a transformation unit, a second quantization unit, a first quantization unit and a coding unit. Wherein: the transformation unit is used for carrying out first transformation on the image blocks in the image to be compressed to obtain a frequency domain coefficient set of the image blocks; the image block is a continuous area comprising a plurality of pixel points in the image to be compressed. And the second quantization unit is used for performing a second quantization operation on the frequency domain coefficient set obtained by the transformation unit to obtain a second quantized coefficient set. The second quantization operation is used for removing high-frequency components in the frequency domain coefficient set, and maintaining or reducing the amplitude of low-frequency components in the frequency domain coefficient set; the number of high frequency components removed by the second quantization operation is related to the image block; the number of high-frequency components removed by the second quantization operation is greater than or equal to the number of high-frequency components removed by the first quantization operation based on the first quantization matrix; and the first quantization unit is used for carrying out first quantization operation on the second quantization coefficient set based on the first quantization matrix to obtain a first quantization coefficient set. And the encoding unit is used for entropy encoding the first quantized coefficient set obtained by the first quantization unit to obtain compressed data of the image block.

By the image compression device provided by the embodiment of the application, the second quantization operation related to the image block is performed on the frequency coefficient set of the data block once, the high-frequency component is removed, the low-frequency component is kept or reduced in amplitude, the first quantization operation is performed on the basis of the configured first quantization matrix, and then entropy coding is performed to complete compression. In this way, the quantization of the correlation of different image blocks is different, and different image blocks in one frame of image can be quantized according to different quantization step sizes, so that different visual demands of users on different image areas in different scenes can be met; the size of the encoded JPEG format image is adjusted according to actual requirements; meanwhile, the region of interest can be flexibly encoded on the premise of meeting the JPEG standard, the quality of the region of interest is ensured, the code rate consumed by other regions is reduced, the code rate (storage space) is further saved, and the compression rate is further improved.

In one possible implementation manner, the image compression apparatus provided in the present application further includes a first acquiring unit, configured to acquire an ROI in an image to be compressed. The second quantization unit is specifically configured to: if the image block is in the ROI of the image to be compressed, reserving the first N1 elements arranged in the frequency domain coefficient set according to the ZigZag sequence, and setting the rest elements to zero to obtain a second quantized coefficient set; and if the image block is positioned in the non-ROI of the image to be compressed, reserving the first N2 elements arranged in the frequency domain coefficient set according to the ZigZag sequence, and setting the rest elements to zero to obtain a second quantized coefficient set. N1 is greater than N2. The second quantization operation is performed through the implementation mode, the implementation process is simple, the calculation complexity is low, and the cost is saved.

In another possible implementation, N1 is 63 and N2 is 32.

In a possible implementation manner, the image compression apparatus provided in the embodiment of the present application may further include a second obtaining unit, configured to obtain a feature value of the image block, where the feature value is used to indicate any one of the following features of the image block: spatial domain features, frequency domain features, or texture features. The second quantization unit is specifically configured to: and determining a second quantization matrix of the image block according to the quantization information of the first quantization matrix of the image block and the characteristic value of the image block, and performing a third operation on the frequency coefficient set based on the second quantization matrix to obtain a second quantization coefficient set of the image block.

In another possible implementation manner, the image compression apparatus provided in the embodiment of the present application may further include a second obtaining unit, configured to obtain a feature value of the image block, where the feature value is used to indicate any one of the following features of the image block: spatial domain features, frequency domain features, or texture features. The second quantization unit is specifically configured to: determining a second quantization matrix of the image block according to quantization information of the first quantization matrix of the image block and characteristic values of the image block, and sequentially performing first quantization operation and inverse quantization operation on the frequency domain coefficient set of the image block based on the second quantization matrix of the image block to obtain a second quantization coefficient set of the image block.

In another possible implementation manner, the image compression apparatus provided in the embodiment of the present application may further include a second obtaining unit, configured to obtain a feature value of the image block, where the feature value is used to indicate any one of the following features of the image block: spatial domain features, frequency domain features, or texture features. The second quantization unit is specifically configured to: determining a second quantization matrix of the image block according to quantization information of the first quantization matrix of the image block and characteristic values of the image block; comparing the element in the frequency domain coefficient set of the image block with the element in the position corresponding to the element in the second quantization matrix of the image block, and determining the second quantization coefficient set of the image block based on a preset rule. The preset rule may include: when the element of the first position in the frequency domain coefficient set is larger than the element of the first position in the second quantization matrix, reserving the element of the first position in the frequency domain coefficient set; the first position is any position in the frequency domain coefficient set; when the element of the first position in the frequency domain coefficient set is smaller than the element of the first position in the second quantization matrix, setting the element of the first position in the frequency domain coefficient set to zero; when the element of the first position in the frequency domain coefficient set is equal to the element of the first position in the second quantization matrix, the element of the first position in the frequency domain coefficient set is reserved or zeroed.

In another possible implementation manner, the second quantization unit determines a second quantization matrix of the image block according to quantization information and feature values of the first quantization matrix, including: according to a second quantization matrix QM ₂ And a first quantization matrix QM ₁ Relationship between feature value X, and QM is determined ₂ . Wherein QM ₂ And QM ₁ The relationship of the feature value X satisfies the following expression: QM (quality control model) ₂ ＝QM ₁ +F ₁ (X). Wherein F is ₁ (. Cndot.) is a first predetermined function.

In another possible implementation, the quantization information of the first quantization matrix may include determining a first quality factor QF of the first quantization matrix ₁ ，QF ₁ For calculating a first quantization matrix from the first relational expression. Wherein the first relational expression is a quality factor and a standard quantization matrix QM _S Is a relational expression of (2). First, theThe second quantization unit determines a second quantization matrix of the image block according to quantization information and characteristic values of the first quantization matrix, and the second quantization unit comprises: determining a quality factor offset value delta QF according to the characteristic value X; according to delta QF and QF ₁ Calculating a second quality factor QF ₂ The method comprises the steps of carrying out a first treatment on the surface of the According to QF ₂ And a first relational expression determining a second quantization matrix. Wherein Δqf satisfies the following expression: Δqf=f ₂ (X)；F ₂ (. Cndot.) is a second predetermined function; QF (quad Flat No lead) ₂ The following expression is satisfied: QF (quad Flat No lead) ₂ ＝|QF ₁ |-F ₃ (ΔQF)，F ₃ (. Cndot.) is a third predetermined function.

In another possible implementation manner, the image compression apparatus provided in the present application may further include a first acquiring unit, configured to acquire the ROI in the image to be compressed. The second quantization unit is specifically configured to: if the image block is outside the ROI, executing to acquire the characteristic value of the image block and determining a second quantization matrix of the image block according to the quantization information and the characteristic value of the first quantization matrix; or if the image block is in the ROI, the first quantization unit is further configured to perform a first quantization operation on the frequency domain coefficient set based on the first quantization matrix to obtain a first quantization coefficient set.

In another possible implementation manner, the first obtaining unit is specifically configured to: and receiving the input ROI region information, and determining the region indicated by the ROI region information in the image to be compressed as the ROI. The ROI area information is used to indicate the coordinate position of the ROI in the image to be compressed. In the implementation manner, the user can better meet the user requirements by inputting the ROI area information by the user and determining the position of the ROI in the image to be compressed.

In another possible implementation manner, the first obtaining unit is specifically configured to: and receiving Map image information of the input image to be compressed, and determining a region of interest (ROI) in the image to be compressed based on the Map image information, wherein the Map image information is used for indicating whether each image block in the image to be compressed is in the region of the ROI. In the implementation mode, the user can better meet the user requirements by inputting Map image information of the image to be compressed and determining the position of the ROI in the image to be compressed.

It should be noted that, the image compression apparatus provided in the second aspect is configured to perform the image compression method provided in the first aspect or any possible implementation manner of the first aspect, and the specific implementation manner may refer to the first aspect or any possible implementation manner of the first aspect.

In a third aspect, the present application provides an image compression apparatus, which may implement the functions in the method examples described in the first aspect, where the functions may be implemented by hardware, or may be implemented by executing corresponding software by hardware. The hardware or software comprises one or more modules corresponding to the functions. The image compression device may exist in the form of a product of a chip.

In one possible implementation, the image compression apparatus may include a processor and a transmission interface. Wherein the transmission interface is used for receiving and transmitting data. The processor is configured to invoke the program instructions stored in the memory to cause the image compression device to perform the functions in the method examples described in the first aspect above.

In a fourth aspect, there is provided a computer readable storage medium having stored therein program instructions which when run on a computer or processor cause the computer or processor to perform the image compression method provided by the first aspect or the second aspect or any one of the possible implementations thereof.

In a fifth aspect, there is provided a computer program product comprising program instructions which, when run on a computer or processor, cause the computer or processor to perform the image compression method provided by the first aspect or any one of its possible implementations.

In a sixth aspect, a chip system is provided, where the chip system includes a processor and may further include a memory, to implement the corresponding functions in the above method. The chip system may be formed of a chip or may include a chip and other discrete devices.

In a seventh aspect, there is provided an image compression system comprising the image compression apparatus of the second or third aspect.

The various possible implementations of any of the foregoing aspects may be combined without contradiction between the schemes.

Drawings

Fig. 1 is a schematic diagram of an exemplary JPEG compression flow provided in an embodiment of the present application;

FIG. 2 is a schematic diagram of an exemplary matrix provided in an embodiment of the present application;

fig. 3 is a schematic view of a scenario of an exemplary image compression transmission according to an embodiment of the present application;

Fig. 4 is a schematic structural diagram of an exemplary image compression apparatus according to an embodiment of the present application;

fig. 5 is a schematic structural diagram of an exemplary image block according to an embodiment of the present application;

fig. 6 is a flowchart of an exemplary image compression method according to an embodiment of the present application;

FIG. 7 is a schematic diagram of an exemplary compressed image block process provided in an embodiment of the present application;

FIG. 8 is a schematic diagram of another exemplary compressed image block process provided by embodiments of the present application;

FIG. 9 is a schematic diagram of a process for compressing image blocks according to yet another exemplary embodiment of the present application;

FIG. 10 is a schematic diagram of a process for compressing image blocks according to yet another exemplary embodiment of the present application;

FIG. 11 is a schematic diagram of a process for compressing image blocks according to yet another exemplary embodiment of the present application;

FIG. 12 is a schematic diagram of a process for compressing image blocks according to yet another exemplary embodiment of the present application;

FIG. 13 is a schematic diagram of a process for compressing image blocks according to yet another exemplary embodiment of the present application;

FIG. 14 is a schematic diagram of a process for compressing image blocks according to yet another exemplary embodiment of the present application;

FIG. 15 is a schematic diagram of a process for compressing image blocks according to yet another exemplary embodiment of the present application;

Fig. 16 is a schematic structural diagram of another exemplary image compression apparatus according to an embodiment of the present application;

fig. 17 is a schematic structural diagram of still another exemplary image compression apparatus according to an embodiment of the present application;

fig. 18 is a schematic structural diagram of still another exemplary image compression apparatus according to an embodiment of the present application.

Detailed Description

In the embodiments of the present application, in order to facilitate the clear description of the technical solutions of the embodiments of the present application, the words "first", "second", etc. are used to distinguish the same item or similar items having substantially the same function and effect. It will be appreciated by those of skill in the art that the words "first," "second," and the like do not limit the amount and order of execution, and that the words "first," "second," and the like do not necessarily differ. The technical features described in the first and second descriptions are not sequential or in order of magnitude.

In the embodiments of the present application, words such as "exemplary" or "such as" are used to mean serving as examples, illustrations, or descriptions. Any embodiment or design described herein as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion that may be readily understood.

In the description of the present application, unless otherwise indicated, "/" means that the associated object is an "or" relationship, e.g., a/B may represent a or B; the term "and/or" in this application is merely an association relation describing an association object, and means that three kinds of relations may exist, for example, a and/or B may mean: there are three cases, a alone, a and B together, and B alone, wherein a, B may be singular or plural. Also, in the description of the present application, unless otherwise indicated, "a plurality" means two or more than two. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b, or c may represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, c may be single or plural.

In the embodiments of the present application, at least one may also be described as one or more, and a plurality may be two, three, four or more, which is not limited in this application.

In addition, the network architecture and the scenario described in the embodiments of the present application are for more clearly describing the technical solution of the embodiments of the present application, and do not constitute a limitation on the technical solution provided in the embodiments of the present application, and those skilled in the art can know that, with the evolution of the network architecture and the appearance of a new service scenario, the technical solution provided in the embodiments of the present application is also applicable to similar technical problems.

Before describing the embodiments of the present application, the terms related to the present application are explained in detail, and will not be explained in detail.

An image block (also referred to as a data block) is a contiguous region of an image to be compressed that includes a plurality of pixels. The image to be compressed may be divided into a plurality of image blocks by means of division. In the JPEG image compression standard, one image block may include 8×8 pixels. One pixel may include multiple components of interest to the vision system, such as a luminance component, a chrominance component, and the like. The components included in the pixel point may be represented by component values.

The component set of an image block refers to a set of component values of the component included in a plurality of pixel points in the image block. The component set may be embodied in the form of a matrix. For example, a set of values of pixel points in an image block on a luminance component is referred to as a set of luminance components of the image block; the set of values of the pixels in an image block on the chrominance components is referred to as the set of chrominance components of the image block. The size of a set of image block components is the same as the size of the image block. For example, an image block in YUV format may include one set of luminance components for the image block and two sets of chrominance components (a set of chrominance U components and a set of chrominance V components) for the image block.

It should be appreciated that an image block typically comprises a plurality of component sets of the image block, e.g. an RGB image comprises an R component set, a G component set and a B component set of the image block; the YUV image comprises a Y component set, a U component set and a V component set of an image block, and the image block is subjected to first transformation, including respectively performing first transformation on a plurality of component sets of the image block to obtain a plurality of frequency domain coefficient sets of the image block.

The frequency domain coefficient set of the component set of the image block refers to a set of frequency domain coefficients obtained after the component set of the image block is converted from a spatial domain to a frequency domain. Each set of components of an image block may have its corresponding set of frequency domain coefficients.

The high-frequency component in the frequency domain coefficient set refers to an element preset in the frequency domain coefficient set and representing a position insensitive to the human eye vision system. For example, all coefficients of the frequency domain coefficient set with a position number greater than Z are defined as high frequency components based on the ZigZag sequential scan approach. Z can be configured according to actual requirements.

The low-frequency component in the frequency-domain coefficient set refers to an element in the frequency-domain coefficient set that indicates a position other than the position of the high-frequency component. The human eye vision system is sensitive to low frequency components in the set of frequency domain coefficients.

Quantization refers to the discretization of the amplitude. In image processing, the magnitudes in the component set of the image block are quantized, the non-important (vision system insensitive) high frequency components in the component set of the image block are removed, and the important (vision system sensitive) low frequency component magnitudes are maintained or reduced.

Quantization step size refers to a step size used for reducing amplitude in the quantization process. For example, the component set point of the image block is quantized by dividing the quantization matrix, and the value of the element in the quantization matrix is the quantization step.

The first quantization operation includes dividing the quantization matrix by a point to obtain an integer result for the object to be processed. When the object to be processed is integer data, dividing the quantization matrix by the object to be processed by the first quantization operation to obtain an integer result; when the object to be processed is floating point data, the first quantization operation divides the quantization matrix by the object to be processed and then rounds or cuts off the object to be processed. For example, in the image compression process, a first quantization operation is performed on the frequency domain coefficient set a based on the first quantization matrix B, and the elements in the ith row and jth column in the obtained result C are: c (i, j) =a (i, j)/B (i, j), where/is a dot-division operation.

The second quantization operation includes removing high frequency components from the set of frequency domain coefficients, maintaining or reducing the magnitude of the low frequency components. The number of high-frequency components removed by the second quantization operation is related to the image block, the sensitivity degree of the vision system to the image block can be reflected through the characteristics (spatial domain characteristics, frequency domain characteristics, texture characteristics or other characteristics) of the image block, or whether the user is interested in the image block is reflected through the area where the image block is located, in the second quantization operation, the image block in the area which is insensitive to vision or is not interested in the user is removed, the compression rate is increased to reduce the size of the compressed image by removing more high-frequency components, and the vision effect is improved by removing fewer high-frequency components from the image block which is sensitive to vision or the image block in the area which is interested in the user. I.e. the image block dependent quantization is achieved in different image blocks by a second quantization operation. For the specific procedure of the second quantization operation, the following embodiment content is described in detail, and will not be described herein.

The ZigZag order may refer to a scan traversal order from the top left corner to the bottom right corner of a set in rectangular form. For example, the order from small to large of the numbers 0 to 63, i.e., the order shown by the arrows in fig. 2, may be as in the matrix illustrated in fig. 2.

Before describing the scheme of the application, a simple description is given of JPEG image compression.

A normal picture of 800×800 size, which is about 1.7 Megabits (MB) if not compressed, occupies a large space during storage or transmission, and therefore, image compression is important. Currently, the pictures mostly use JPEG compression technology, i.e. common JPEG image files. The JPEG compression technique can achieve a compression ratio of 1/8 with respect to the original image because of using a lossy compression technique. Lossy compression is the removal of insignificant parts of the original data in order to reduce the space taken up by the data.

In the process of JPEG compression, a frame (picture) is first divided into a series of 16 x 16 image blocks. The image blocks are divided in such a manner that the first 16 lines of the image are scanned in the horizontal direction from the upper left corner of the image, and each 16 columns are divided into 1 image block during the scanning. After the first 16 lines are scanned, the next 16 lines of data in the first 16 lines in the image are scanned continuously from left to right, and every 16 columns are divided into 1 image block in the scanning process. The segmentation is scanned every 16 lines in this order until one frame of image is completed. The image blocks for each 16 x 16 image block are further divided into 4 8 x 8 image blocks and then compressed for each component set of each 8 x 8 image block using the JPEG compression scheme as illustrated in fig. 1. The image segmentation process is not described in detail in this application. The size of each image block after image segmentation can be adjusted according to actual requirements.

As shown in fig. 1, an 8×8 image block (each component set in the image block) is subjected to discrete cosine transform (discrete cosine transform, DCT), quantization, and entropy encoding, and compressed data of the image block is obtained.

It should be noted that, the operations performed on the image block described in the embodiments of the present application may be understood as performing operations on each component set in the image block, which is not described in detail later.

For example, after DCT, quantization, and entropy encoding are performed on an image block in 8×8 YUV format, compressed data of the image block is obtained, which means that: the compressed data of the luminance component set of the image block is obtained by DCT, quantization and entropy coding of the luminance component set of the image block 8×8, the compressed data of the chrominance U component set of the image block is obtained by DCT, quantization and entropy coding of the chrominance U component set of the image block 8×8, and the compressed data of the chrominance V component set of the image block is obtained by DCT, quantization and entropy coding of the chrominance V component set of the image block 8×8.

The following describes each step in the JPEG compression flow illustrated in fig. 1.

The DCT is a matrix multiplication of 8×8 row transforms and 8×8 column transforms on 8×8 image blocks, respectively, to obtain 8×8 DCT coefficients. Since an image block has strong correlation in a spatial domain, contains limited frequency components and is mainly distributed in a middle-low frequency region, DCT transforms a spatial data block into a frequency domain in image compression, and is represented by DCT coefficients (a set of frequency domain coefficients), separates different frequency components contained in the spatial data block in the frequency domain, and the coefficients of the DCT after transformation are often 0 or less at a high frequency. For example, the corresponding high frequency coefficient for a flat region with less texture is often 0. The set of frequency coefficients may be embodied in a matrix or table or other form, which is not limited in this application. The transformed set of frequency domain coefficients may be divided into a Direct Current (DC) component in the upper left corner (first element of the set of frequency domain coefficients in the zig-zag scanning mode) and an alternating current (alternating current, AC) component in other positions (second element of the set of frequency domain coefficients in the zig-zag scanning mode and all elements thereafter).

And quantizing, namely quantizing the DCT coefficients according to the quantization matrix input from the outside to obtain quantized coefficients. The quantization is performed by dividing the DCT coefficients by an integer other than 0 (quantization step), and the quantized value is truncated or rounded to the nearest integer. The quantization step is implemented using the aforementioned first quantization operation. In practice, according to the characteristic that human eyes are insensitive to high-frequency coefficients, quantization step sizes are gradually increased in a quantization matrix along with the increase of frequencies, so that a better compression ratio is achieved. The luminance component set and the chrominance component set may employ different quantization matrices to achieve compression. The luminance component contains more texture details and contour information, and on the transform domain of the DCT, the frequency distribution is wider, and more high-frequency information is contained, so that the luminance component set is quantized by using a larger quantization step length, and the effect of reducing the code rate is realized. While the chrominance components contain less detail and contour information, but the human eye is more sensitive to chrominance, the frequency distribution is wider in the transform domain of the DCT, containing less high frequency information, and typically the quantization step size used by the set of chrominance components is smaller than the quantization step size used by the set of luminance components during quantization.

It should be noted that, the fixed quantization matrix may be configured according to actual requirements, which is not specifically limited in the embodiments of the present application.

Illustratively, a standard quantization matrix for the luminance component employed for JPEG compression may be illustrated as follows:

illustratively, a standard quantization matrix for the chrominance components employed for JPEG compression may be illustrated as follows:

in the standard quantization matrix of the luminance component and the chrominance component, quantization step sizes are gradually increased from the upper left corner to the lower right corner according to the ZigZag sequence, so that high-frequency components are discarded through a large quantization step size, the compression rate is improved, and the volume of an image after compression is reduced. In addition, as can be seen from the standard quantization matrices of the luminance component and the chrominance component, the quantization step size used by the chrominance component is smaller than that of the luminance component, so as to meet the requirement that human eyes are more sensitive to chrominance.

Entropy coding is a process of coding quantized coefficients obtained by quantization to obtain compressed data. In entropy encoding, one-dimensional differential pulse code modulation (differential pulse coding modulation, DPCM) may be used on the DC component, followed by entropy encoding. The AC quantized coefficients are ZigZag scanned and entropy encoded using Huffman coding for the number of consecutive zero coefficients and non-zero coefficient magnitudes.

When compressing an image, the quantization matrix used in the compression process of the encoding end and the Huffman table used in the corresponding entropy encoding also need to be compressed into the code stream to be transmitted to the decoding end so as to be convenient for the decoding end to restore the image.

As is clear from the above-described JPEG compression process, only one set of quantization matrices (quantization matrix for luminance component, quantization matrix for chrominance component) is used in one frame (whole image), and only one unique quantization matrix is used for a single component, and the compression rate is fixed.

In practice, in the security monitoring, the application scenes such as capturing data and photographing by mobile phones need to pay special attention to certain target areas, finer quantization step sizes are hoped to be used, and other insensitive areas, such as background areas, can utilize the visual characteristics that the human eyes have different perceptions of quantization degrees of areas with different texture characteristics, and coarser quantization precision is hoped to be used, so that the purpose of reducing the code rate is achieved. Current JPEG compression cannot meet different visual demands of users on different image areas in different scenes.

Based on this, the embodiment of the application provides an image compression method, before quantization operation is performed in the existing JPEG compression standard, the frequency coefficient set of the component set of the image block is quantized once in relation to the image block, the high-frequency component is removed, the low-frequency component is kept or reduced in amplitude, quantization is performed in the JPEG compression standard based on the configured fixed quantization matrix, and then entropy coding is performed to complete compression. In this way, when the image blocks are different, the quantization related to the image blocks is different, so that the same component of different image blocks in one frame of image can be quantized according to different quantization step sizes, and different visual demands of users on different image areas in different scenes can be met; the method realizes the adjustment of the size of the encoded JPEG format image according to the actual requirement; meanwhile, the function of coding of interest can be flexibly realized on JPEG, the quality of the region of interest is ensured, the code rate consumed by other regions is reduced, the code rate (storage space) is further saved, and the compression rate can be further improved.

The image compression method provided by the embodiment of the application can be applied to an image compression storage scene or a scene of image compression transmission.

Fig. 3 illustrates a scenario of image compression transmission, where the scenario includes an encoding end device 301 and a decoding end device 302.

The encoding end device 301 may compress the image and transmit the compressed image to the decoding end device 302, and the compressed image is decompressed by the decoding end device 202 and presented or stored.

The encoding end device 301 or the decoding end device 302 may be a product form such as a terminal device, a mobile phone, a palm computer, etc., which is not limited in the embodiment of the present application.

In the image compression storage scene, the encoding end device can acquire an image and store the image in a memory corresponding to the encoding end device after compressing the image.

Embodiments of the present application are specifically described below with reference to the accompanying drawings.

In one aspect, an embodiment of the present application provides an image compression apparatus for performing the image compression method provided by the present application.

Fig. 4 shows an image compression apparatus 40 relating to various embodiments of the present application. As shown in fig. 4, the image compression apparatus 40 may include a processor 401, a memory 402, and a transceiver 403.

The following describes the respective constituent elements of the image compression apparatus 40 in detail with reference to fig. 4:

Wherein the memory 402 may be a volatile memory (RAM), such as a random-access memory (RAM); or a nonvolatile memory (non-volatile memory), such as a read-only memory (ROM), a flash memory (flash memory), a hard disk (HDD) or a Solid State Drive (SSD); or a combination of the above, for storing program code, configuration files, or other content that may be used to implement the methods of the present application.

The processor 401 is a control center of the image compression apparatus 40. For example, processor 401 may be a central processing unit (central processing unit, CPU), may be an integrated circuit (application specific integrated circuit, ASIC), or may be one or more integrated circuits configured to implement embodiments of the present application, such as: one or more microprocessors (digital singnal processor, DSPs), or one or more field programmable gate arrays (field programmable gate array, FPGAs).

The transceiver 403 is used to communicate with other devices. The transceiver 403 may be a communication port or otherwise.

The processor 401 performs the following functions by running or executing software programs and/or modules stored in the memory 402 and invoking data stored in the memory 402:

Performing first transformation on an image block in an image to be compressed to obtain a frequency domain coefficient set of the data block component; performing second quantization operation on the frequency domain coefficient set to obtain a second quantized coefficient set; the second quantization operation is used for removing high-frequency components in the frequency domain coefficient set, and maintaining or reducing the amplitude of low-frequency components in the frequency domain coefficient set; the number of high frequency components removed by the second quantization operation is related to the image block; the number of high-frequency components removed by the second quantization operation is greater than or equal to the number of high-frequency components removed by the first quantization operation based on the first quantization matrix; performing first quantization operation on the second quantized coefficient set based on the first quantization matrix to obtain a first quantized coefficient set; and carrying out entropy coding on the first quantized coefficient set to obtain compressed data of the image block.

On the other hand, the embodiment of the application provides an image compression method, which is executed by an image compression device, and the image to be compressed is compressed to obtain compressed data of the image to be compressed. The image compression device can divide an image to be compressed into a plurality of image blocks, and the image compression method provided by the application is carried out on each component set of each image block. Wherein the compression process of the image compression means is the same for each component set of each image block. The compression process of the image block by the image compression device described in the following embodiments of the present application should be understood that the image compression device performs the following compression process on each component set of the image block, and will not be described in detail. It should also be understood that "image block" described in the embodiments of the present application may be equivalently replaced with "a set of chrominance components of the image block" or "a set of luminance components of the image block" or "a set of components of the image block" to describe the scheme protected by the present application.

It should be noted that, for the specific implementation of dividing the image to be compressed into image blocks, the foregoing JPEG compression process illustrated in fig. 1 has been described, which is not repeated here.

It should be noted that, the image to be compressed may be a YUV image, or other image formats including a chrominance component and a luminance component, which is not limited in the present application. When the format of the image to be compressed is an image format which is not supported by the image compression device, the image to be compressed can be subjected to format conversion to obtain the image format supported by the image compression device, and then the image compression method provided by the application is executed.

By way of example, it is assumed that an image block of YUV420 format, which includes a 16×16 luma component set, an 8×8 chroma U component set, and an 8×8 chroma V component set, after segmentation of an image to be compressed, may include components as shown in fig. 5. The image compression apparatus may divide the 16×16 luminance component set into 4 8×8 luminance component sets, and then perform the image compression method provided herein on each of the 4 8×8 luminance component sets, one 8×8 chrominance U component set, and one 8×8 chrominance V component set to obtain compressed data of each component set as the compressed data of the image block.

As shown in fig. 6, the image compression method provided in the embodiment of the present application may include:

s601, the image compression device performs first transformation on an image block in an image to be compressed to obtain a frequency domain coefficient set of the image block.

The image block is any image block obtained by dividing an image to be compressed, and is hereinafter also referred to as an image block to be processed.

Wherein the first transformation is used to convert the spatial data value representation of the image block into a frequency domain representation. Each pixel point in the image block corresponds to an element in the same position in the set of frequency domain coefficients. The arrangement of elements and the number of elements in the set of frequency domain coefficients for an image block are the same as for the image block. For example, the first transform may be a DCT transform or other type of transform, which is not limited in this application and the process thereof is not repeated.

Specifically, the set of frequency domain coefficients of the image block obtained by the first transformation may be represented by a matrix or other form, which is not specifically limited in the embodiment of the present application. In the embodiment of the present application, the set of frequency domain coefficients is represented in a matrix form, which is merely an example and not particularly limited.

It should be noted that, the specific implementation of S601 may refer to the JPEG compression standard, which is not described herein.

S602, the image compression device performs a second quantization operation on the frequency domain coefficient set to obtain a second quantized coefficient set.

The second quantization operation is used for removing high-frequency components in the frequency domain coefficient set, and maintaining or reducing the amplitude of low-frequency components in the frequency domain coefficient set; the number of the high-frequency components removed by the second quantization operation is related to the image block, and the different image blocks in the image to be compressed are subjected to self-adaptive quantization by the second quantization operation, so that the quantization effect of each image block is matched with the characteristics of the image block, and different quantization effects of different areas in the image to be compressed are achieved.

For example, the sensitivity of the vision system to the image block may be reflected by the characteristics of the image block (spatial domain characteristics, frequency domain characteristics, texture characteristics, or other characteristics), or whether the user is interested in the image block may be reflected by the region in which the image block is located, the image block in the region that is not sensitive to vision or is not interested by the user may be removed in the second quantization operation, the compression rate may be increased by removing more high frequency components to reduce the size of the compressed image, and the vision effect may be increased by removing fewer high frequency components for the image block that is sensitive to vision or is interested by the user.

In practical application, the specific content of the second quantization operation can be configured according to practical requirements, so that the purposes of removing high-frequency components in the frequency domain coefficient set, maintaining or reducing the amplitude of low-frequency components in the frequency domain coefficient set, and correlating the number of the removed high-frequency components with the image block are achieved.

Wherein the number of high frequency components removed by the second quantization operation is greater than or equal to the number of high frequency components removed by the first quantization operation based on the first quantization matrix.

In a possible implementation, for image blocks in the area of interest of the user, or for image blocks in the area of sensitivity of the visual system, the number of high frequency components removed by the second quantization operation may be equal to the number of high frequency components removed by the first quantization operation based on the first quantization matrix.

In another possible implementation, the number of high frequency components removed by the second quantization operation may be greater for image blocks in areas where the user's attention is not high, or for image blocks in areas where the visual system is insensitive than the number of high frequency components removed by the first quantization operation based on the first quantization matrix.

In an alternative case, the first quantization matrix is a fixed quantization matrix input in the JPEG compression standard. The first quantization matrices of the same component of different image blocks in a frame of an image to be compressed are identical.

S603, the image compression device performs a first quantization operation on the second quantization coefficient set based on the first quantization matrix to obtain a first quantization coefficient set.

Specifically, the image compression apparatus in S603 performs a first quantization operation based on the first quantization matrix, which specifically includes dividing the second quantization coefficient set by the first quantization matrix to obtain a first quantization coefficient set of the integer result.

For example, the element in the ith row and jth column in the first quantized coefficient set may be the product of the quotient of the element in the ith row and jth column in the second quantized coefficient set divided by the element in the ith row and jth column in the first quantized matrix.

S604, the image compression device performs entropy coding on the first quantization coefficient set to obtain compressed data of the image block.

Specifically, in S604, the first quantized coefficient set is entropy encoded to obtain compressed data of the image block, which may refer to an entropy encoding process in the JPEG compression standard, and will not be described herein.

For example, the procedure of compressing an image block by the image compression method provided in S601 to S604 described above may be shown in fig. 7.

In a possible implementation manner, in practical application, the image compression device may perform the above processes S601 to S604 on each image block in the image to be compressed to obtain compressed data of the image to be compressed

In a possible implementation manner, in an actual application, the image compression device may execute the processes of S601 to S604 described above on a portion of image blocks in an image to be compressed, and compress the remaining image blocks according to a JPEG compression standard, to obtain compressed data of the image to be compressed.

For example, the image compression apparatus may compress the image blocks in the non-region of interest (region of interest, ROI) in the image to be compressed by performing the above-described processes S601 to S604, and compress the image blocks of the ROI region according to the JPEG compression standard.

The ROI is a region with high user attention in the image to be compressed, and specific implementation of the ROI in the image to be compressed can be determined according to actual requirement configuration, which is not limited in the embodiment of the present application.

Alternatively, the ROI in the image to be compressed may be obtained by an AI identification technique, or the ROI in the image to be compressed may be obtained according to the indication information of the ROI area input by the user. The embodiments of the present application are not limited in this regard.

In one possible implementation, the ROI in the image to be compressed may be acquired through an AI identification technique, and the indication information of the ROI area is output. The type of AI technique for acquiring the ROI is not limited in the embodiments of the present application.

For example, the targets in the image to be compressed, such as a face, a person shape, a license plate, a car, etc., can be detected in real time based on the target detection of the AI, the region where the identified target is located is determined as the ROI, the other regions are non-ROIs, and the rectangular frame coordinate information of each ROI region is used as the indication information of the ROI region.

In one possible implementation manner, the indication information of the ROI area output by the AI identification may be directly input into the image compression device, and the image compression device may determine, in real time, whether the image block to be processed is in the ROI area according to the indication information of the ROI area during the compression process.

Another possible way of handling is: the indication information of the ROI area output by AI identification may be generated by using the size of the image block (which may be processed according to a larger size) as a unit according to the position of the indication information in the image to be compressed, for example, 16×16, a binary image is generated for the whole frame of the image to be compressed (the binary image generating process may be generated according to the AI analysis result, for example, the AI object detection result), whether each image block is the ROI area is identified by using a different identifier, and the binary image is used as an input of the image compression device, and the image compression device determines whether the image block to be processed is in the ROI area according to the specific value of the image block to be processed in the binary image during the compression process. For example, it may be identified by 0 and 1, 1 identifying that the tile is inside the ROI area, 0 identifying that the tile is in a non-ROI area.

In another possible implementation manner, the image compression device may receive the input ROI area information, determine an area indicated by the ROI area information in the image to be compressed as an ROI, and determine other areas as non-ROIs. The ROI area information is used for indicating coordinate positions of the ROI in the image to be compressed. For example, the ROI area information may be input by a user of the image compression apparatus at a human-computer interaction interface of the image compression apparatus. Of course, the ROI area information may be input by other subjects in other manners, which is not limited in the embodiments of the present application.

In still another possible implementation manner, the image compression device may receive Map image information of an input image to be compressed, determine a region of interest ROI in the image to be compressed based on the Map image information, and the other region is a non-ROI. The Map image information is used for indicating whether each image block in the image to be compressed is in the region of the ROI.

As described above, in the present application S602, the specific implementation of performing the second quantization operation on the frequency domain coefficient set to obtain the second quantized coefficient set may be configured according to actual needs, and the embodiments of the present application provide two types of performing the second quantization operation on the frequency domain coefficient set to obtain the specific implementation of the second quantized coefficient set, including the first implementation and the second implementation described below, but are not limited to the specific implementation.

The first implementation:

the image compression device acquires an ROI in an image to be compressed; if the image block to be processed is positioned in the region of interest (ROI) of the image to be compressed, reserving the first N1 elements in the frequency domain coefficient set, which are arranged according to the ZigZag sequence, and setting the rest elements to zero to obtain a second quantized coefficient set; and if the image block to be processed is in the non-ROI of the image to be compressed, reserving the first N2 elements arranged in the frequency domain coefficient set according to the ZigZag sequence, and setting the rest elements to zero to obtain a second quantized coefficient set.

Specifically, in the first implementation, in combination with the characteristics of the human visual system, the image compression device is sensitive to low-frequency information and insensitive to high-frequency information, and the frequency components gradually rise from the upper left corner to the lower right corner according to the zigbee sequence, so that the image compression device can perform the second quantization operation by adopting the first implementation to obtain the second quantization coefficient.

Wherein N1 is greater than N2. The values of N1 and N2 can be configured according to actual requirements. N1 can be determined according to the attention degree of the region of interest, and the larger the N1 value is, the lower the quantization degree of the region of interest is, and the closer the compressed image is to the original image. N2 can be determined according to the attention degree of the region outside the region of interest, the smaller the N2 value is, the higher the quantization degree of the region outside the region of interest is, the larger the compressed image loss is, the higher the compression rate is, and the smaller the compressed image volume is.

In one possible implementation, N1 may be much greater than N2. For example, N1 is 63 and N2 is 32.

In another possible implementation, the values of N1 may be different and the values of N2 may be different in different component sets of the image block.

The image compression device performs the second quantization operation on the frequency domain coefficient set TC based on the zigbee sequential scanning mode, and in the process of obtaining the second quantized coefficient set TC', the process of performing the second quantization operation on the idx-th coefficient in the zigbee sequential scanning can be implemented by the following expression:

n corresponds to N1 of the ROI and N2 of the non-ROI. TC (i, j) is the value of the idx-th coefficient of the frequency domain coefficient set TC according to the ZigZag sequential scanning mode.

In a first implementation, the process of compressing an image block by using the image compression method provided in the embodiment of the present application may be shown in fig. 8. As shown in fig. 8, the image block is subjected to a first transformation to obtain a set of frequency domain coefficients for the image block. Then carrying out zero setting treatment on the frequency domain coefficient set, wherein the zero setting treatment refers to carrying out second quantization operation through the first implementation, namely if the data block component to be treated is positioned in a region of interest (ROI) of the image to be compressed, reserving the first N1 elements arranged in the frequency domain coefficient set according to the ZigZag sequence, and setting the rest elements to zero to obtain a second quantization coefficient set; and if the data block component to be processed is in the non-ROI of the image to be compressed, reserving the first N2 elements in the frequency domain coefficient set, which are arranged according to the ZigZag sequence, and setting the rest elements to zero to obtain a second quantization coefficient set. Next, a first quantization operation is performed on the second quantized coefficient set based on the first quantization matrix as described in S603 above, resulting in a first quantized coefficient set. And finally, carrying out entropy coding on the first quantized coefficient set to obtain compressed data of the image block.

The second implementation:

the image compression device firstly acquires a second quantization matrix of the image block to be processed, and completes second quantization operation on the frequency domain coefficient set of the image block according to the second quantization matrix.

The first quantization matrix, the second quantization matrix and the row and column of the image block to be processed are the same in size, and the amplitude of an element in the second quantization matrix of a component set of an image block is greater than or equal to the amplitude of an element in the same position in the first quantization matrix of the component set of the image block.

It should be noted that, the method for obtaining the second quantization matrix of the image block may be configured according to actual requirements, which is not limited in the embodiment of the present application.

In a second implementation, the process of compressing the data block component by the image compression method provided in the embodiment of the present application may be shown in fig. 9. As shown in fig. 9, the image block is subjected to a first transformation to obtain a set of frequency domain coefficients for the image block. And obtaining a second quantization matrix of the image block, and then completing second quantization operation on the frequency domain coefficient set based on the second quantization matrix to obtain a second quantization coefficient set. Next, a first quantization operation is performed on the second quantized coefficient set based on the first quantization matrix as described in S603 above, resulting in a first quantized coefficient set. And finally, carrying out entropy coding on the first quantized coefficient set to obtain compressed data of the image block.

In one possible implementation, the second quantization matrix of the image block may be specified by the user.

In another possible implementation, the image compression device may obtain the second quantization matrix of the image block according to the characteristics of the image block.

For example, the image compression apparatus may acquire the feature value of the image block, and determine the second quantization matrix of the image block according to the quantization information of the first quantization matrix of the image block and the feature value of the image block. It should be appreciated that the image compression apparatus may obtain a characteristic value of a certain component set (a luminance component set or a chrominance component set) of an image block, and determine a second quantization matrix of the component set of the image block based on quantization information of a first quantization matrix of the component set of the image block and the characteristic value of the component set of the image block.

In a second implementation, when the image compression device performs the second quantization operation, the image compression device obtains the feature value of the image block, determines the second quantization matrix of the image block according to the quantization information of the first quantization matrix of the image block and the feature value of the image block, and then completes the second quantization operation, a process of compressing the image block by using the image compression method provided in the embodiment of the present application may be shown in fig. 10. As shown in fig. 10, the image block is subjected to a first transformation to obtain a set of frequency domain coefficients for the image block. And acquiring a characteristic value X of the image block through image characteristic analysis. And then determining a second quantization matrix of the image block according to the characteristic value X and the first quantization matrix of the image block. And then, completing a second quantization operation on the frequency domain coefficient set based on the second quantization matrix to obtain a second quantization coefficient set. Next, a first quantization operation is performed on the second quantized coefficient set based on the first quantization matrix as described in S603 above, resulting in a first quantized coefficient set. And finally, carrying out entropy coding on the first quantized coefficient set to obtain compressed data of the image block.

Wherein the feature value of the image block is used to indicate any one of the following features of the image block: spatial domain features, frequency domain features, or texture features. Of course, the feature value of the image block may also indicate other features of the image block, which are not limited in the embodiments of the present application.

Optionally, if the image block to be processed is outside the ROI, the image compression device may acquire the feature value of the image block, and determine the second quantization matrix of the image block according to the quantization information of the first quantization matrix of the image block and the feature value of the image block, so as to complete the second quantization operation.

Optionally, if the image block to be processed is in the ROI, after S601, the image compression device may perform a first quantization operation on the frequency domain coefficient set of the image block based on the first quantization matrix to obtain a first quantization coefficient set, and perform entropy encoding on the first quantization coefficient set to obtain compressed data of the image block.

In a possible implementation manner, the image compression device may acquire spatial domain characteristics of the image block to be processed through gradient or variance and edge detection means. Specifically, the spatial feature of the image block may be a spatial feature value, specifically, the edge intensity of each point in the image block in the horizontal direction and the vertical direction may be detected by a Sobel edge detection operator, the edge intensities of all points in the horizontal direction and the vertical direction are accumulated and then averaged to obtain an edge intensity index in the image block, the edge intensity index may be used to indicate the spatial feature of the image block, and the edge intensity index may be used as the feature value of the image block.

The larger the value of the edge strength index, the stronger the texture or edge strength in the image block, and in practice, the larger quantization step size can be used for quantization without significantly reducing the quality of the compressed image. Conversely, if the value of the edge strength index is smaller, a smaller quantization step is required, and subjective quality of the image by human eyes is affected as little as possible.

In another possible implementation manner, the image compression device may also analyze the frequency domain coefficient set of the image block to be processed and the frequency domain characteristics of the image block. Specifically, the frequency domain feature of the image block may be a frequency domain feature value, which is used to indicate the feature of the image block in the frequency domain. Specifically, the intensity of the frequency domain component can be obtained by summing and averaging all the AC coefficients or the medium-high frequency coefficients (for example, all the coefficients with sequence numbers greater than 16 based on the ZigZag sequential scanning mode) in the frequency domain coefficient set obtained by performing DCT on the image block. The intensity of the frequency domain component is taken as the characteristic value of the image block. The greater the intensity of the frequency domain component, the more complex the image is represented, and in practice a larger quantization step may be used for quantization without significantly degrading the quality of the compressed image. Conversely, if the intensity of the frequency domain component is smaller, a smaller quantization step is required, and subjective quality of the image by human eyes is affected as little as possible.

In yet another possible implementation, the feature value is used to indicate a texture feature of the image block, and the image compression device may calculate the feature value X according to an expression of the feature value X and a pixel value in the image block.

Wherein, the expression of the characteristic value X and the pixel value in the image block satisfies the following relation:

wherein,

Pix _i for the pixel value of the ith pixel point in the image block to be processed, M is the width of the image block, and L is the height of the image block.

It should be noted that, the feature values of the image blocks may also be obtained by other manners, which are not described in detail in the embodiments of the present application.

After the image compression device obtains the characteristic value of the image block, the image compression device may then determine a second quantization matrix of the image block according to quantization information of the first quantization matrix of the image block and the characteristic value of the image block.

Since the human visual system (human vision system, HVS) is sensitive to different areas of different features (frequency features, spatial features, texture features) of the image, for example, areas are flat, for example, skin areas of a wall surface and a face, HVS is sensitive, quantization is required by using smaller quantization step sizes, and areas are complex, for example, grasslands, HVS is not sensitive and contains more high frequency component information, quantization can be performed by using larger quantization step sizes, the effect of reducing the code rate is achieved, and therefore the second quantization matrix can be determined based on the principle.

Optionally, the image compression apparatus provided in this embodiment of the present application may determine, according to quantization information of a first quantization matrix of an image block and a feature value of the image block, a second quantization matrix of the image block by, but not limited to, a first determination manner or a second determination manner described below.

The first determination mode is as follows: the quantization information of the first quantization matrix is the first quantization matrix, or the quantization information of the first quantization matrix includes determining a first quality factor QF of the first quantization matrix ₁ The image compression apparatus determines a first quantization matrix based on quantization information of the first quantization matrix, and then determines a second quantization matrix QM based on the first quantization matrix ₂ And a first quantization matrix QM ₁ Relationship between feature value X, and QM is determined ₂ . Wherein QM ₂ And QM ₁ The relationship of the feature value X satisfies the following expression: QM (quality control model) ₂ ＝QM ₁ +F ₁ (X)；F ₁ (. Cndot.) is a first predetermined function.

The content of the first preset function may be configured according to actual requirements, which is not limited in the embodiment of the present application. The first preset function may be a log function, for example. For example, the first predetermined function may be a log function log based on 2 ₂ (X)。

It should be noted that, when calculating the second quantization matrix of a certain component set in an image block, the feature values of other component sets of the image block may be used for calculation.

For example, in calculating the second quantization matrix of the set of chrominance components of an image block, the feature value of the set of luminance components of the image block may be calculated from the second quantization matrix QM ₂ And a first quantization matrix QM ₁ And calculating a second quantization matrix of the chroma component set of the image block according to the relation of the characteristic value X.

Alternatively, when the quantization information of the first quantization matrix may be QM ₁ Image compression apparatusThen the QM can be determined directly ₁ 。

Alternatively, the quantization information of the first quantization matrix may include determining a first quality factor QF of the first quantization matrix ₁ ，QF ₁ For calculating a first quantization matrix QM based on a first relational expression ₁ . The image compression device can calculate QM according to the first relation expression ₁ 。

Wherein the first relational expression is a quality factor QF and a standard quantization matrix QM _S The first relational expression satisfies the following relationship: QM (quality control model) _qf ＝floor(S×QM _s +50)。

Wherein QM _qf For the calculated quantization matrix, floor (·) is a rounding operation, S satisfies the following expression:

the first determination mode is as follows: the quantization information of the first quantization matrix includes determining a first quality factor QF of the first quantization matrix ₁ The image compression device calculates a second quality factor QF ₂ And then according to the first relational expression and QF ₂ A second quantization matrix is determined.

In the first determination mode, the image compression apparatus determines the second quantization matrix of the image block specifically by the following steps 1 to 3:

and step 1, the image compression device determines a quality factor offset value delta QF according to the characteristic value X of the image block.

Wherein Δqf satisfies the following expression: Δqf=f ₂ (X)；F ₂ (. Cndot.) is a second predetermined function.

The content of the second preset function may be configured according to actual requirements, which is not limited in the embodiment of the present application.

The second predetermined function may be, for example, alpha log ₂ (·)，Δqf satisfies the following expression: Δqf=α log ₂ (X), wherein α is an adjustment coefficient for controlling the range of Δqf.

Step 2, the image compression device uses delta QF and QF as basis ₁ Calculating a second quality factor QF ₂ 。

Wherein QF (quad Flat No lead) ₂ The following expression is satisfied: QF (quad Flat No lead) ₂ ＝|QF ₁ |-F ₃ (ΔQF)；F ₃ (. Cndot.) is a third predetermined function.

The content of the third preset function may be configured according to actual requirements, which is not limited in the embodiment of the present application.

The third preset function may be, for example, null, QF ₂ The following expression may be satisfied: QF (quad Flat No lead) ₂ ＝|QF ₁ |-ΔQF。

The third preset function may be, for example, a clamping operation, QF ₂ The following expression may be satisfied: QF (quad Flat No lead) ₂ ＝|QF ₁ |-clip(0，MaxΔQF,ΔQF)。

Step 3, the image compression device is according to QF ₂ And a first relational expression determining a second quantization matrix.

Through the processes from the step 1 to the step 3, for the region with complex texture, the larger the texture feature X obtained by calculation is, the larger the corresponding delta QF is, and QF is ₂ The smaller the quantization step size of the determined second quantization matrix, the larger more high frequency details can be quantized out by the second quantization operation.

Optionally, implementing S602 in the second implementation may include, but is not limited to, the following two specific operations:

the first specific operation is as follows: the image compression device sequentially carries out first quantization operation and inverse quantization operation of the first quantization operation on the frequency domain coefficient set based on the second quantization matrix to obtain a second quantization coefficient set.

The foregoing description of the term interpretation section has been given for the first quantization operation, where the first quantization operation is performed on the frequency domain coefficient set based on the second quantization matrix, that is, the frequency domain coefficient set is divided by the second quantization matrix to obtain an integer result, and then the integer result is obtained by multiplying the integer result by the second quantization matrix as the second quantization coefficient set.

By way of example, in the first determination manner, the image compression apparatus determines the second quantization matrix of the image block according to the quantization information of the first quantization matrix of the image block and the feature value of the image block, and completes the second quantization operation according to the first specific operation, and the process of compressing the image block by using the image compression method provided in the embodiment of the present application may be shown in fig. 11. As shown in fig. 11, the image block is subjected to a first transformation to obtain a set of frequency domain coefficients for the image block. And acquiring a characteristic value X of the image block through image characteristic analysis. And then determining a second quantization matrix of the image block according to the characteristic value X and the first quantization matrix of the image block. And then, successively performing a second quantization operation and an inverse quantization operation of the first quantization operation on the frequency domain coefficient set based on the second quantization matrix to obtain a second quantization coefficient set. Next, a first quantization operation is performed on the second quantized coefficient set based on the first quantization matrix as described in S603 above, resulting in a first quantized coefficient set. And finally, carrying out entropy coding on the first quantized coefficient set to obtain compressed data of the image block.

By way of example, in the second determining manner, the image compression apparatus determines the second quantization matrix of the image block according to the quantization information of the first quantization matrix of the image block and the feature value of the image block, and completes the second quantization operation according to the first specific operation, and the process of compressing the image block by using the image compression method provided in the embodiment of the present application may be shown in fig. 12. As shown in fig. 12, the image block is subjected to a first transformation to obtain a set of frequency domain coefficients for the image block. And acquiring a characteristic value X of the image block through image characteristic analysis. And determining a second quality factor according to the first quality factor and the steps 1 to 3, and determining a second quantization matrix of the image block according to the characteristic value X and the second quality factor. And then, successively performing a first quantization operation and an inverse quantization operation of the first quantization operation on the frequency domain coefficient set based on the second quantization matrix to obtain a second quantization coefficient set. Next, a first quantization matrix of the image block is calculated according to the first quality factor, and a first quantization operation is performed on the second quantization coefficient set based on the first quantization matrix according to the description of S603, so as to obtain the first quantization coefficient set. And finally, carrying out entropy coding on the first quantized coefficient set to obtain compressed data of the image block.

Exemplary, the image compression apparatus is based on the second quantization matrix QM ₂ The process of sequentially performing the first quantization operation and the inverse quantization operation of the first quantization operation on the frequency domain coefficient set TC to obtain the second quantized coefficient set TC' may be represented by the following formula (1) and formula (2), where formula (1) is based on the second quantization matrix QM ₂ Performing first quantization operation on the frequency domain coefficient set TC to obtain a result QC ₁ Equation (2) is based on the second quantization matrix QM ₂ For the result QC of formula (1) ₂ And performing inverse quantization operation of the first quantization operation to obtain a second quantized coefficient set TC' with the high-frequency coefficients removed.

QC ₁ ＝TC./QM ₂ Formula (1).

TC′＝QM ₂ .*QC ₁ Formula (2).

Where "/" is the matrix dot division operation and ".# is the dot multiplication operation of the matrix.

For example, in practical applications, if the image compression device performs the above-described processes of S601 to S604 on each image block in the image to be compressed, the second quantization matrix of the image block of the ROI area may be the first quantization matrix of the image block, and the second quantization matrix of the image block of the non-ROI area may be obtained according to the method described in the second implementation above. If the second quantization operation is completed according to the first specific operation, the process of compressing the image block by the image compression method provided in the embodiment of the present application may be shown in fig. 13. As shown in fig. 13, the image block is subjected to a first transformation to obtain a set of frequency domain coefficients for the image block. The image compression apparatus acquires quantization matrices (first quantization matrices of ROI areas, second quantization matrices of non-ROI areas) that complete the second quantization operation. Then, based on the quantization matrix with the second quantization operation, the first quantization operation and the inverse quantization operation of the first quantization operation are sequentially performed on the frequency domain coefficient set, so as to obtain a second quantization coefficient set. Next, a first quantization operation is performed on the second quantized coefficient set based on the first quantization matrix as described in S603 above, resulting in a first quantized coefficient set. And finally, carrying out entropy coding on the first quantized coefficient set to obtain compressed data of the image block.

For example, in practical application, if the image compression device performs the above-mentioned processes of S601 to S604 on each image block in the image to be compressed, the second quantization matrix of the image block of the ROI area may be the first quantization matrix of the image block, the second quantization matrix of the image block of the non-ROI area may be obtained according to the method described in the second implementation, and by determining the ROI based on the AI identification of the object detection, the process of compressing the data block component by the image compression method provided in the embodiment of the present application may be shown in fig. 14. As shown in fig. 14, the input image is divided into image blocks, and the image blocks are subjected to a first transformation to obtain a set of frequency domain coefficients for the image blocks. The image compression device obtains quantization matrix (first quantization matrix of ROI area, second quantization matrix of non-ROI area) for completing the second quantization operation based on AI target detection of the input image. And then, based on the quantization matrix with the second quantization operation, combining with the ROI indication information, sequentially performing a first quantization operation and an inverse quantization operation of the first quantization operation on the frequency domain coefficient set to obtain a second quantization coefficient set. Next, a first quantization operation is performed on the second quantized coefficient set based on the first quantization matrix as described in S603 above, resulting in a first quantized coefficient set. And finally, carrying out entropy coding on the first quantized coefficient set to obtain compressed data of the image block.

Further, for the first quantization operation and the inverse quantization operation of the first quantization operation in the first specific operation, which correspond to the division and multiplication operations, respectively, in order to save the hardware implementation cost, the first quantization operation and the inverse quantization operation of the first quantization operation may be implemented by a comparison operation, which avoids the multiplication and division operations, and hardly increases the cost of hardware logic implementation, the comparison operation is the second specific operation as follows.

The second specific operation: the image compression device determines a second quantization matrix of the image block according to quantization information of the first quantization matrix of the image block and characteristic values of the image block; comparing the element in the frequency domain coefficient set with the element in the position corresponding to the element in the second quantization matrix of the image block, and determining the second quantization coefficient set of the image block based on a preset rule.

The preset rule may include: when the element of the first position in the frequency domain coefficient set of the image block is larger than the element of the first position in the second quantization matrix of the image block, reserving the element of the first position in the frequency domain coefficient set of the image block; the first position is any position in the frequency domain coefficient set; when the element of the first position in the frequency domain coefficient set of the image block is smaller than the element of the first position in the second quantization matrix of the image block, setting the element of the first position in the frequency domain coefficient set to zero; when the element of the first position in the frequency domain coefficient set of the image block is equal to the element of the first position in the second quantization matrix of the image block, the element of the first position in the frequency domain coefficient set is reserved or set to zero.

In the second specific operation, in the process of performing the second quantization operation on the frequency domain coefficient set TC based on the second quantization matrix to obtain the second quantization coefficient set, an element TC' (i, j) of an ith row and a jth column in the second quantization coefficient set TC, an element TC (i, j) of an ith row and a jth column in the frequency domain coefficient set TC, and an element QM of an ith row and a jth column in the second quantization matrix ₂ The relationship of (i, j) satisfies the following expression:

in a second specific operation, the process of compressing an image block according to the image compression method provided in the embodiment of the present application may be shown in fig. 15. As shown in fig. 15, the image block is subjected to a first transformation to obtain a set of frequency domain coefficients for the image block. And comparing the frequency coefficient set based on the second quantization matrix, namely completing the second quantization operation in the second specific operation to obtain a second quantization coefficient set. Next, a first quantization operation is performed on the second quantized coefficient set based on the first quantization matrix as described in S603 above, resulting in a first quantized coefficient set. And finally, carrying out entropy coding on the first quantized coefficient set to obtain compressed data of the image block.

The image compression method provided in the embodiment of the present application is described below by way of a specific example.

When the image compression device processes a certain component set of a certain image block to be processed, according to the second implementation, calculating the texture feature x=1426 of the component set of the image block, taking the adjustment coefficient α=2, according to the above steps 1 to 3, by the formula Δqf=α×log ₂ (X) Δqf=20 can be calculated assuming user entered QF ₁ =90, according to the formula QF ₂ ＝|QF ₁ The I-clip (0, max. DELTA. QF, DELTA. QF) can be calculated to obtain QF ₂ =70, according to the first relational expression and QF ₁ 、QF ₂ And a standard quantization matrix, obtaining a first quantization matrix QM of the component set of the image block ₁ Second quantization matrix QM ₂ The following are provided:

the component set of the image block is assumed as follows:

the frequency coefficient set TC of the image block after DCT transformation is as follows:

a second quantization matrix QM based on the set of components of the image block ₂ Result QC of first quantization operation on frequency coefficient set TC of component set of image block ₁ QC is as follows ₁ ＝TC./QM ₂ 。

A second quantization matrix QM based on the set of components of the image block ₂ QC for the set of components of the image block ₁ The result TC 'of the inverse quantization operation of the first quantization operation is as follows, TC' =qc ₁ ./*QM ₂ . TC 'is essentially a set of frequency domain coefficients that have undergone a second quantization operation, with more elements in TC' quantized to 0 than TC.

First quantization matrix QM based on the component set of the image block ₁ Performing a first quantization operation on TC' of the component set of the image block to obtain a final quantization result QC ₂ (second quantization coefficient set) QC is as follows ₂ ＝TC′./QM ₁ 。

If the image compression device is based on the first quantization matrix QM of the component set of the image block ₁ Directly performing a first quantization operation on a frequency coefficient set TC of a component set of the image block, namely performing quantized result QC in a JPEG compression standard ₃ QC is as follows ₃ ＝TC./QM ₁ 。

Comparison of QC ₃ And QC ₂ In QC ₂ More coefficients are quantized to 0 and most of the low frequency coefficients are smaller in magnitude, QC is known from the entropy-encoded properties ₂ The number of bits required for encoding will be significantly smaller than the encoding QC ₃ Is a bit number of (c).

The above description has been made mainly in terms of the working principle of the image compression apparatus. It will be appreciated that the image compression apparatus includes, in order to implement the above functions, corresponding hardware structures and/or software modules that perform the respective functions. Those of skill in the art will readily appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be implemented as hardware or combinations of hardware and computer software. Whether a function is implemented as hardware or computer software driven hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

The embodiment of the present application may divide the functional modules of the image compression apparatus executing the present application according to the above method example, for example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. The integrated modules may be implemented in hardware or in software functional modules. It should be noted that, in the embodiment of the present application, the division of the modules is schematic, which is merely a logic function division, and other division manners may be implemented in actual implementation.

Fig. 16 shows a possible configuration diagram of the image compression apparatus 160 involved in the above-described embodiment in the case where respective functional blocks are divided with corresponding respective functions. The image compression device 160 may be a functional module or a chip. As shown in fig. 16, the image compression apparatus 160 may include: transform section 1601, second quantization section 1602, first quantization section 1603, and coding section 1604. Wherein the transformation unit 1601 is configured to perform a process S601 in fig. 6; the second quantization unit 1602 is for performing the process S602 in fig. 6; the first quantization unit 1603 is for performing a process S603 in fig. 6; the encoding unit 1604 is used to perform the process S604 in fig. 6. All relevant contents of each step related to the above method embodiment may be cited to the functional description of the corresponding functional module, which is not described herein.

Further, as shown in fig. 17, the image compression apparatus 160 may further include a first acquisition unit 1605. The first acquisition unit 1605 is used to acquire an ROI in an image to be compressed.

Further, as shown in fig. 17, the image compression apparatus 160 may further include a second acquisition unit 1606. The second obtaining unit 1606 is used for obtaining the feature value of the image block.

In the case of using an integrated unit, fig. 18 shows another possible structural schematic diagram of the image compression apparatus involved in the above-described embodiment. As shown in fig. 18, the image compression apparatus 180 may include: a processing module 1801, a communication module 1802. The processing module 1801 is used for controlling and managing the operation of the image compression apparatus 180, and the communication module 1802 is used for communicating with other devices. For example, the processing module 1801 is used to perform any one of the processes S601 to S604 in fig. 3. The image compression apparatus 180 may further include a storage module 1803 for storing program code and data of the image compression apparatus 180.

The processing module 1801 may be the processor 401 in the physical structure of the image compression apparatus 40 shown in fig. 4, and may be a processor or a controller. For example, it may be a CPU, general purpose processor, DSP, ASIC, FPGA or other programmable logic device, transistor logic device, hardware component, or any combination thereof. Which may implement or perform the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. The processing module 801 may also be a combination implementing computing functionality, e.g., including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc. The communication module 1802 may be a transceiver 403 in the physical structure of the image compression apparatus 40 shown in fig. 4, and the communication module 1802 may be a communication port, or may be a transceiver, a transceiver circuit, a communication interface, or the like. Alternatively, the communication interface may implement communication with other devices through the element having the transceiver function. The above-mentioned elements with transceiving functions may be realized by antennas and/or radio frequency devices. The storage module 1803 may be the memory 402 in the physical structure of the image compression apparatus 40 shown in fig. 4.

When the processing module 1801 is a processor, the communication module 1802 is a transceiver, and the storage module 1803 is a memory, the image compression apparatus 40 according to fig. 18 of the embodiment of the present application may be the image compression apparatus 40 shown in fig. 4.

As mentioned above, the image compression apparatus 160 or the image compression apparatus 180 provided in the embodiments of the present application may be used to implement the corresponding functions in the methods implemented in the embodiments of the present application, and for convenience of explanation, only the portions relevant to the embodiments of the present application are shown, and specific technical details are not disclosed, please refer to the embodiments of the present application.

As another form of the present embodiment, there is provided a computer-readable storage medium having stored thereon instructions that, when executed, perform the image compression method in the above-described method embodiment.

As another form of the present embodiment, there is provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the image compression method in the method embodiment described above.

The embodiment of the application further provides a chip system, which comprises a processor and is used for realizing the technical method of the embodiment of the invention. In one possible design, the system on a chip also includes memory to hold the program instructions and/or data necessary for embodiments of the present invention. In one possible design, the system-on-chip further includes a memory for the processor to invoke application code stored in the memory. The chip system may be formed by one or more chips, or may include chips and other discrete devices, which are not specifically limited in this embodiment.

The steps of a method or algorithm described in connection with the disclosure herein may be embodied in hardware, or may be embodied in software instructions executed by a processor. The software instructions may be comprised of corresponding software modules that may be stored in RAM, flash memory, ROM, erasable programmable read-only memory (erasable programmable ROM, EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disk, a removable disk, a compact disc read-only memory (CD-ROM), or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. In addition, the ASIC may be located in a core network interface device. The processor and the storage medium may reside as discrete components in a core network interface device. Alternatively, the memory may be coupled to the processor, e.g., the memory may be separate and coupled to the processor via a bus. The memory may also be integrated with the processor. The memory may be used for storing application program codes for executing the technical solutions provided in the embodiments of the present application, and the processor may control the execution. The processor is configured to execute the application program code stored in the memory, thereby implementing the technical solution provided in the embodiments of the present application.

From the foregoing description of the embodiments, it will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of functional modules is illustrated, and in practical application, the above-described functional allocation may be implemented by different functional modules according to needs, i.e. the internal structure of the apparatus is divided into different functional modules to implement all or part of the functions described above.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical functional division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another apparatus, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and the parts displayed as units may be one physical unit or a plurality of physical units, may be located in one place, or may be distributed in a plurality of different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a readable storage medium. Based on such understanding, the technical solution of the embodiments of the present application may be essentially or a part contributing to the prior art or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, including several instructions for causing a device (may be a single-chip microcomputer, a chip or the like) or a processor (processor) to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk, etc.

The foregoing is merely a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any changes or substitutions within the technical scope of the present disclosure should be covered in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

An image compression method, the method comprising:

performing first transformation on an image block in an image to be compressed to obtain a frequency domain coefficient set of the image block; the image block is a continuous area comprising a plurality of pixel points in the image to be compressed;

performing second quantization operation on the frequency domain coefficient set to obtain a second quantized coefficient set; the second quantization operation is used for removing high-frequency components in the frequency domain coefficient set, and maintaining or reducing the amplitude of low-frequency components in the frequency domain coefficient set; the number of high frequency components removed by the second quantization operation is related to the image block; the number of the high-frequency components removed by the second quantization operation is greater than or equal to the number of the high-frequency components removed by the first quantization operation based on the first quantization matrix;

performing first quantization operation on the second quantized coefficient set based on a first quantization matrix to obtain a first quantized coefficient set;

And carrying out entropy coding on the first quantization coefficient set to obtain compressed data of the image block.
The method of claim 1, wherein performing a second quantization operation on the set of frequency domain coefficients to obtain a second set of quantized coefficients comprises:

acquiring a region of interest (ROI) in the image to be compressed;

if the image block is positioned in the region of interest (ROI) of the image to be compressed, reserving the first N1 elements which are arranged in the frequency domain coefficient set according to the ZigZag sequence, and setting the rest elements to zero to obtain the second quantized coefficient set; if the image block is positioned in the non-ROI of the image to be compressed, reserving the first N2 elements which are arranged in the frequency domain coefficient set according to the ZigZag sequence, and setting the rest elements to zero to obtain the second quantized coefficient set; the N1 is greater than the N2.
The method according to claim 1, wherein the method further comprises:

acquiring a characteristic value of the image block, wherein the characteristic value is used for indicating any one of the following characteristics of the image block: spatial domain features, frequency domain features, or texture features;

the performing a second quantization operation on the frequency domain coefficient set to obtain a second quantized coefficient set, including:

Determining a second quantization matrix of the image block according to the quantization information of the first quantization matrix and the characteristic value; the first quantization matrix, the second quantization matrix and the image block have the same row and column size, and the amplitude of one element in the second quantization matrix is larger than that of the element in the same position in the first quantization matrix;

and successively performing the first quantization operation and the inverse quantization operation of the first quantization operation on the frequency domain coefficient set based on the second quantization matrix to obtain the second quantization coefficient set.
The method according to claim 1, wherein the method further comprises:

acquiring a characteristic value of the image block, wherein the characteristic value is used for indicating any one of the following characteristics of the image block: spatial domain features, frequency domain features, or texture features;

the performing a second quantization operation on the frequency domain coefficient set to obtain a second quantized coefficient set, including:

determining a second quantization matrix of the image block according to the quantization information of the first quantization matrix and the characteristic value; the first quantization matrix, the second quantization matrix and the image block have the same row and column size, and the amplitude of one element in the second quantization matrix is larger than or equal to the amplitude of the element at the same position in the first quantization matrix;

Comparing the size of the elements in the frequency domain coefficient set with the size of the elements at the positions corresponding to the elements in the second quantization matrix, and determining the second quantization coefficient set based on a preset rule;

wherein, the preset rule comprises: when the element of the first position in the frequency domain coefficient set is larger than the element of the first position in the second quantization matrix, reserving the element of the first position in the frequency domain coefficient set; the first position is any position in the frequency domain coefficient set; when the element of the first position in the frequency domain coefficient set is smaller than the element of the first position in the second quantization matrix, setting zero for the element of the first position in the frequency domain coefficient set; when the element of the first position in the frequency domain coefficient set is equal to the element of the first position in the second quantization matrix, reserving or setting zero the element of the first position in the frequency domain coefficient set.
The method according to claim 3 or 4, wherein the determining the second quantization matrix of the image block according to the quantization information of the first quantization matrix and the eigenvalue comprises:

according to the second quantization matrix QM ₂ And the first quantization matrix QM ₁ The relation of the characteristic value X, and the QM is determined ₂ ；

Wherein the QM ₂ And the QM ₁ The relation of the characteristic value X satisfies the following expression: QM (quality control model) ₂ ＝QM ₁ +F ₁ (X); wherein the F is ₁ (. Cndot.) is a first predetermined function.
The method according to claim 3 or 4, wherein the quantization information of the first quantization matrix comprises determining a first quality factor QF of the first quantization matrix ₁ The Q isF ₁ For calculating the first quantization matrix according to a first relational expression; wherein the first relational expression is a quality factor and a standard quantization matrix QM _S Is a relational expression of (2);

the determining a second quantization matrix of the image block according to the quantization information of the first quantization matrix and the eigenvalue includes:

determining a quality factor offset value delta QF according to the characteristic value X; wherein the Δqf satisfies the following expression: Δqf=f ₂ (X); the F is ₂ (. Cndot.) is a second predetermined function;

according to the delta QF and the QF ₁ Calculating a second quality factor QF ₂ The method comprises the steps of carrying out a first treatment on the surface of the Wherein the QF ₂ The following expression is satisfied: QF (quad Flat No lead) ₂ ＝|QF ₁ |-F ₃ (Δqf); the F is ₃ (. Cndot.) is a third predetermined function;

according to the QF ₂ And the first relational expression, determining the second quantization matrix.
The method according to any one of claims 3-6, further comprising:

acquiring a region of interest (ROI) in the image to be compressed;

if the image block is outside the ROI, executing the acquisition of the characteristic value of the image block and the determination of a second quantization matrix of the image block according to the quantization information of the first quantization matrix and the characteristic value;

or,

and if the image block is positioned in the ROI, performing the first quantization operation on the frequency domain coefficient set based on the first quantization matrix to obtain the first quantization coefficient set.
The method according to claim 2 or 7, wherein said acquiring a region of interest, ROI, in said image to be compressed comprises:

receiving input ROI (region of interest) region information, and determining a region indicated by the ROI region information in the image to be compressed as the ROI; the ROI area information is used for indicating the coordinate position of the ROI in the image to be compressed;

or,

and receiving input Map image information of the image to be compressed, and determining the region of interest (ROI) in the image to be compressed based on the Map image information, wherein the Map image information is used for indicating whether each image block in the image to be compressed is in the ROI region or not.
The method of claim 2, wherein N1 is 63 and N2 is 32.
An image compression apparatus, the apparatus comprising:

the transformation unit is used for carrying out first transformation on the image block in the image to be compressed to obtain a frequency domain coefficient set of the image block; the image block is a continuous area comprising a plurality of pixel points in the image to be compressed;

the second quantization unit is used for performing second quantization operation on the frequency domain coefficient set obtained by the transformation unit to obtain a second quantized coefficient set; the second quantization operation is used for removing high-frequency components in the frequency domain coefficient set, and maintaining or reducing the amplitude of low-frequency components in the frequency domain coefficient set; the number of high frequency components removed by the second quantization operation is related to the image block; the number of the high-frequency components removed by the second quantization operation is greater than or equal to the number of the high-frequency components removed by the first quantization operation based on the first quantization matrix;

the first quantization unit is used for carrying out first quantization operation on the second quantization coefficient set based on the first quantization matrix to obtain a first quantization coefficient set;

and the encoding unit is used for entropy encoding the first quantization coefficient set obtained by the first quantization unit to obtain compressed data of the image block.
The apparatus of claim 10, wherein the device comprises a plurality of sensors,

the device further comprises a first acquisition unit for acquiring a region of interest (ROI) in the image to be compressed;

the second quantization unit is specifically configured to: if the image block is positioned in the region of interest (ROI) of the image to be compressed, reserving the first N1 elements which are arranged in the frequency domain coefficient set according to the ZigZag sequence, and setting the rest elements to zero to obtain the second quantized coefficient set; if the image block is positioned in the non-ROI of the image to be compressed, reserving the first N2 elements which are arranged in the frequency domain coefficient set according to the ZigZag sequence, and setting the rest elements to zero to obtain the second quantized coefficient set; the N1 is greater than the N2.
The apparatus of claim 10, wherein the apparatus further comprises:

a second obtaining unit, configured to obtain a feature value of the image block, where the feature value is used to indicate any one of the following features of the image block: spatial domain features, frequency domain features, or texture features;

the second quantization unit is specifically configured to:

determining a second quantization matrix of the image block according to the quantization information of the first quantization matrix and the characteristic value; the first quantization matrix, the second quantization matrix and the image block have the same row and column size, and the amplitude of one element in the second quantization matrix is larger than that of the element in the same position in the first quantization matrix;

And successively performing the first quantization operation and the inverse quantization operation of the first quantization operation on the frequency domain coefficient set based on the second quantization matrix to obtain the second quantization coefficient set.
The apparatus of claim 10, wherein the apparatus further comprises:

a second obtaining unit, configured to obtain a feature value of the image block, where the feature value is used to indicate any one of the following features of the image block: spatial domain features, frequency domain features, or texture features;

the second quantization unit is specifically configured to:

determining a second quantization matrix of the image block according to the quantization information of the first quantization matrix and the characteristic value; the first quantization matrix, the second quantization matrix and the image block have the same row and column size, and the amplitude of one element in the second quantization matrix is larger than or equal to the amplitude of the element at the same position in the first quantization matrix;

comparing the size of the elements in the frequency domain coefficient set with the size of the elements at the positions corresponding to the elements in the second quantization matrix, and determining the second quantization coefficient set based on a preset rule;

wherein, the preset rule comprises: when the element at the first position in the frequency domain coefficient set is larger than the element at the first position in the second quantization matrix, reserving the element at the first position in the frequency domain coefficient set; the first position is any position in the frequency domain coefficient set; when the element of the first position in the frequency domain coefficient set is smaller than the element of the first position in the second quantization matrix, setting zero for the element of the first position in the frequency domain coefficient set; when the element of the first position in the frequency domain coefficient set is equal to the element of the first position in the second quantization matrix, reserving or setting zero the element of the first position in the frequency domain coefficient set.
The apparatus according to claim 12 or 13, wherein the second quantization unit determines a second quantization matrix of the image block based on quantization information of the first quantization matrix and the eigenvalue, comprising:

according to the second quantization matrix QM ₂ And the first quantization matrix QM ₁ The relation of the characteristic value X, and the QM is determined ₂ ；

Wherein the QM ₂ And the QM ₁ The relation of the characteristic value X satisfies the following expression: QM (quality control model) ₂ ＝QM ₁ +F ₁ (X); wherein the F is ₁ (. Cndot.) is a first predetermined function.
The apparatus according to claim 12 or 13, wherein the quantization information of the first quantization matrix comprises determining a first quality factor QF of the first quantization matrix ₁ The QF is ₁ For calculating the first quantization matrix according to a first relational expression; wherein the first relational expression is a quality factor and a standard quantization matrix QM _S Is a relational expression of (2);

the second quantization unit determines a second quantization matrix of the image block according to quantization information of the first quantization matrix and the eigenvalue, including:

determining a quality factor offset value delta QF according to the characteristic value X; wherein the Δqf satisfies the following expression: Δqf=f ₂ (X); the F is ₂ (. Cndot.) is a second predetermined function;

according to the delta QF and the QF ₁ Calculating a second quality factor QF ₂ The method comprises the steps of carrying out a first treatment on the surface of the Wherein the QF ₂ The following expression is satisfied: QF (quad Flat No lead) ₂ ＝|QF ₁ |-F ₃ (Δqf); the F is ₃ (. Cndot.) is a third predetermined function;

according to the QF ₂ And the first relational expression, determining the second quantization matrix.
The apparatus according to any one of claims 12-15, wherein the apparatus further comprises:

a first acquiring unit, configured to acquire a region of interest ROI in the image to be compressed;

the second quantization unit is specifically configured to: if the image block is outside the ROI, executing the acquisition of the characteristic value of the image block and the determination of a second quantization matrix of the image block according to the quantization information of the first quantization matrix and the characteristic value;

or,

and if the image block is in the ROI, the first quantization unit is further configured to perform the first quantization operation on the frequency domain coefficient set based on the first quantization matrix to obtain the first quantization coefficient set.
The apparatus according to claim 11 or 16, wherein the first acquisition unit is specifically configured to:

receiving input ROI (region of interest) region information, and determining a region indicated by the ROI region information in the image to be compressed as the ROI; the ROI area information is used for indicating the coordinate position of the ROI in the image to be compressed;

Or,

and receiving input Map image information of the image to be compressed, and determining the region of interest (ROI) in the image to be compressed based on the Map image information, wherein the Map image information is used for indicating whether each image block in the image to be compressed is in the ROI region or not.
The apparatus of claim 11, wherein N1 is 63 and N2 is 32.
An image compression apparatus, characterized in that the image compression apparatus comprises: a processor and a transmission interface;

the transmission interface is used for receiving and transmitting data;

the processor is configured to invoke program instructions stored in a memory to cause the image compression apparatus to perform the image compression method of any of claims 1 to 9.
A computer readable storage medium having stored therein program instructions which, when run on a computer or a processor, cause the computer or the processor to perform the image compression method of any of claims 1 to 9.
A computer program product comprising program instructions which, when run on a computer or processor, cause the computer or processor to perform the image compression method of any one of claims 1 to 9.