WO2022205094A1 - 数据处理方法、数据传输系统、设备及存储介质 - Google Patents
数据处理方法、数据传输系统、设备及存储介质 Download PDFInfo
- Publication number
- WO2022205094A1 WO2022205094A1 PCT/CN2021/084471 CN2021084471W WO2022205094A1 WO 2022205094 A1 WO2022205094 A1 WO 2022205094A1 CN 2021084471 W CN2021084471 W CN 2021084471W WO 2022205094 A1 WO2022205094 A1 WO 2022205094A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- frequency coefficient
- coefficient matrix
- compressed
- frequency
- Prior art date
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 17
- 238000003672 processing method Methods 0.000 title claims abstract description 17
- 239000011159 matrix material Substances 0.000 claims abstract description 198
- 238000000034 method Methods 0.000 claims abstract description 43
- 238000007906 compression Methods 0.000 claims abstract description 24
- 230000006835 compression Effects 0.000 claims abstract description 23
- 230000009466 transformation Effects 0.000 claims description 12
- 238000005070 sampling Methods 0.000 claims description 10
- 230000006870 function Effects 0.000 claims description 7
- 238000012986 modification Methods 0.000 claims description 2
- 230000004048 modification Effects 0.000 claims description 2
- 230000008569 process Effects 0.000 description 11
- 238000005516 engineering process Methods 0.000 description 10
- 238000007781 pre-processing Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 4
- 238000001514 detection method Methods 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000001149 cognitive effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 230000003313 weakening effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
Definitions
- the embodiments of the present application relate to the technical field of image processing, and in particular, to a data processing method, a data transmission system, a device, and a storage medium.
- an encoder is used to compress data
- a decoder is used to restore the compressed data
- the original image or original video has a large amount of data, and the transmission channel is limited. Therefore, the original image or original video needs to be effectively compressed before transmission.
- the compression adopted by the encoder is relatively large, the image or video information decoded by the decoder is seriously lost, so that effective image or video information cannot be extracted.
- the embodiments of the present application provide a data processing method, a data transmission system, a device and a storage medium, which can reduce the degree of image detail loss caused by compression, thereby improving the quality of the restored image.
- a first aspect of the embodiments of the present application provides a data processing method, including:
- the to-be-compressed image is compressed to obtain a compressed image.
- a second aspect of the embodiments of the present application provides a data processing apparatus, including:
- an acquisition module used for acquiring the frequency coefficient matrix corresponding to the image to be compressed
- a processing module configured to perform enhancement processing on the frequency coefficients in the frequency coefficient matrix corresponding to the details of the to-be-compressed image to obtain a processed frequency coefficient matrix
- a compression module configured to compress the to-be-compressed image according to the processed frequency coefficient matrix to obtain a compressed image.
- a third aspect of the embodiments of the present application provides a data transmission system, including: an encoding end device and a decoding end device;
- the encoding end device is used to obtain a frequency coefficient matrix corresponding to the image to be compressed; perform enhancement processing on the frequency coefficients in the frequency coefficient matrix corresponding to the details of the to-be-compressed image to obtain a processed frequency coefficient matrix; the processed frequency coefficient matrix, compress the to-be-compressed image to obtain a compressed image; and send the compressed image to the decoding end device;
- the decoding end device is configured to decode the compressed image to obtain a decoded image.
- a fourth aspect of the embodiments of the present application provides an electronic device, including: a memory and a processor;
- the memory for storing programs
- the processor coupled with the memory, is configured to execute the program stored in the memory, so as to execute the data processing method described in any one of the above.
- a fifth aspect of the embodiments of the present application provides a computer-readable storage medium, including instructions, which, when executed on a computer, cause the computer to implement any of the data processing methods described above.
- the frequency coefficients corresponding to the details of the image to be compressed in the frequency coefficient matrix corresponding to the image to be compressed are enhanced, and then the image to be compressed is compressed according to the frequency coefficient matrix obtained after the enhancement processing.
- the frequency coefficient matrix corresponding to the to-be-compressed image is the representation of the to-be-compressed image in the frequency domain. That is to say, before compressing the to-be-compressed image, the image details in the to-be-compressed image will be enhanced in the frequency domain. In this way, during subsequent compression, more image details can be preserved, reducing the loss of image details caused by compression, thereby improving the quality of the restored image.
- FIG. 1 is a schematic flowchart of an embodiment of a data processing method provided by an embodiment of the present application
- FIG. 2 is a structural block diagram of an embodiment of a data transmission system provided by an embodiment of the present application.
- FIG. 3 is a schematic flowchart of another embodiment of a data processing method provided by an embodiment of the present application.
- FIG. 4 is a structural block diagram of an embodiment of a data processing apparatus provided by an embodiment of the present application.
- FIG. 5 is a structural block diagram of an embodiment of an electronic device according to an embodiment of the present application.
- the preprocessing techniques used mainly include the following:
- Noise is unavoidable for any actual image or video acquisition. If unnecessary noise is not removed before encoding, it will not only affect the quality of the decoded image or video, but also need to be adjusted in the subsequent encoding process. Coding with noise reduces coding efficiency. Of course, because images and noise are often intertwined, we need to compromise the strength of denoising to avoid the loss of image or video details caused by excessive denoising.
- This technology mainly determines which information is not important or interesting to the human eye from the perspective of human vision; and removes the information from the original image or video signal through preprocessing technology to improve its compression efficiency.
- the above-mentioned image and video preprocessing technologies often start from the encoder’s point of view, and reduce the redundant information and noise information of the human eye in the image and video through a series of processing methods, thereby improving the compression performance of the encoder. More image content can be transmitted than the lower channel. However, in the case of ultra-high compression, the redundant signals removed by the previous preprocessing will also be discarded during the encoding process. quality yields significant benefits.
- the embodiments of the present application provide a new preprocessing technology, which enhances image details to avoid being discarded during the compression process, thereby obtaining relatively high image quality.
- FIG. 1 shows a schematic flowchart of a data processing method provided by an embodiment of the present application.
- the execution body of this method can be the client or the server.
- the client can be hardware with embedded programs integrated on the terminal, or application software installed in the terminal, or tool software embedded in the terminal operating system, etc.
- This application implements The example does not limit this.
- the terminal may be any terminal device including a mobile phone, a tablet computer, a camera, and a drone.
- the server may be a common server, a cloud or a virtual server, etc., which is not specifically limited in this embodiment of the present application.
- the method includes:
- the above image to be compressed may be a photo taken by a camera, an artificially synthesized animation picture, or the like.
- the above image to be compressed may be an image captured by an aerial photography drone when performing an aerial photography task, or may be a video frame in a video captured by an aerial photography drone when performing an aerial photography task.
- the image to be compressed can be transformed from the spatial domain to the frequency domain to obtain a frequency coefficient matrix corresponding to the image to be compressed.
- the transformation algorithm used in the above-mentioned transformation processing may be discrete cosine transform (Discrete Cosine Transform, DCT), that is, discrete cosine transform is performed on the image to be compressed to obtain a frequency coefficient matrix corresponding to the image to be compressed.
- DCT Discrete Cosine Transform
- the image to be compressed as a whole can be transformed from the spatial domain to the frequency domain to obtain a frequency coefficient matrix corresponding to the image to be compressed.
- the to-be-compressed image may be divided to obtain a plurality of image blocks; each image block is transformed from a spatial domain to a frequency domain to obtain a frequency coefficient matrix corresponding to each image block.
- the frequency coefficient matrix corresponding to the image to be compressed includes frequency coefficient matrices corresponding to multiple image blocks of the image to be compressed.
- the size of the above image block may be set according to actual needs, which is not specifically limited in this embodiment of the present application. For example, the size of the above image block may be 8*8.
- the high-frequency components in the exemplary to-be-compressed image correspond to the details of the to-be-compressed image
- the low-frequency components (low-frequency signals) and high-frequency components (high-frequency signals) in the to-be-compressed image are distributed in the frequency coefficient matrix.
- different regions Using different transformation algorithms, the high-frequency components in the image to be compressed are located in different regions in the frequency coefficient matrix. That is to say, according to the transformation algorithm used in the above transformation processing, the region where the high-frequency components in the image to be compressed are located in the frequency coefficient matrix can be predetermined, that is, the details of the image to be compressed corresponding to the frequency coefficient matrix can be predetermined.
- the size and shape of the region where the frequency coefficients corresponding to the details of the image to be compressed in the frequency coefficient matrix are located may be set according to actual needs, which is not limited in this application.
- the upper left corner area of the frequency coefficient matrix corresponds to the low frequency component in the image to be compressed
- the lower right corner area corresponds to the high frequency component in the image to be compressed, that is, corresponds to the details of the image to be compressed.
- the size and shape of the upper left corner area and the lower right corner area may be set according to actual needs.
- the above-mentioned enhancement processing may be: performing enhancement processing on only the frequency coefficients in the frequency coefficient matrix corresponding to the details of the image to be compressed, while other frequency coefficients in the frequency coefficient matrix remain unchanged; or, enhancing all the frequency coefficients in the frequency coefficient matrix processing, but the enhancement amplitude of the frequency coefficients corresponding to the details of the image to be compressed is greater than that of other frequency coefficients; or, all the frequency coefficients in the frequency coefficient matrix are weakened, but the frequency coefficients corresponding to the details of the image to be compressed are weakened.
- the weakening amplitude is smaller than that of other frequency coefficients.
- the above-mentioned other frequency coefficients refer to frequency coefficients other than the frequency coefficients corresponding to the details of the image to be compressed in the frequency coefficient matrix. That is, a relative enhancement process is performed on the frequency coefficients in the frequency coefficient matrix corresponding to the details of the original image relative to other frequency coefficients in the frequency coefficient matrix to obtain a processed frequency coefficient matrix.
- a gain matrix may be preset, each element in the gain matrix is a gain factor, and the number of rows and columns of the gain matrix is consistent with the number of rows and columns of the frequency coefficient matrix.
- the gain factor of the first region in the gain matrix is greater than the gain factor of the second region; the second region refers to a region other than the first region in the gain matrix.
- the position and size of the above-mentioned first area are determined according to the actual situation.
- the frequency coefficients in the area corresponding to the first area in the frequency coefficient matrix are the frequency coefficients corresponding to the details of the image to be compressed.
- the gain matrix and the frequency coefficient matrix are dot-multiplied to implement enhancement processing on the frequency coefficients in the frequency coefficient matrix corresponding to the details of the image to be compressed.
- the processed frequency coefficient matrix can be directly quantized, arranged and encoded in sequence, and finally a compressed image can be obtained.
- the inverse transformation can adopt a two-dimensional IDCT (Inverse Discrete Cosine Transform, inverse discrete cosine transform).
- the processed frequency coefficient matrices are also multiple;
- the processed frequency coefficient matrices are inversely transformed from the frequency domain to the spatial domain to obtain multiple processed image blocks; the multiple processed image blocks are spliced to obtain a processed image.
- the processed image may be input into the encoder for compression for encoding to obtain a compressed image.
- the processed frequency coefficients are inversely transformed to obtain the processed image, and the encoder acts on the processed image.
- the existing encoder and decoder can be directly used without redesigning the encoder and decoder, which not only reduces the cost, but also improves the applicability of the technical solutions provided by the embodiments of the present application.
- the frequency coefficients corresponding to the details of the image to be compressed in the frequency coefficient matrix corresponding to the image to be compressed are enhanced, and then the image to be compressed is compressed according to the frequency coefficient matrix obtained after the enhancement processing.
- the frequency coefficient matrix corresponding to the to-be-compressed image is the representation of the to-be-compressed image in the frequency domain. That is to say, before compressing the to-be-compressed image, the image details in the to-be-compressed image will be enhanced in the frequency domain. In this way, during subsequent compression, more image details can be preserved, reducing the degree of loss of image details caused by compression, thereby ensuring that the final restored image has better picture quality.
- the technical solutions provided by the embodiments of the present application can enable the decoding end to display more effective information.
- multiple frequency coefficient matrices there are multiple frequency coefficient matrices; multiple frequency coefficient matrices correspond to multiple image blocks in the image to be compressed one-to-one.
- "enhancing the frequency coefficients in the frequency coefficient matrix corresponding to the details of the image to be compressed to obtain a processed frequency coefficient matrix" can be realized by adopting the following steps:
- the detail richness of an image block can be determined in one or more of the following ways:
- Manner 1 Calculate the distribution variance of pixel values in the image block corresponding to the frequency coefficient matrix; determine the detail richness according to the distribution variance.
- the detail richness may be determined according to the range of the distribution variance.
- multiple distribution variance ranges can be divided according to actual needs in advance, and each distribution variance range corresponds to a different detail richness value.
- the detail richness value corresponding to the range of the distribution variance is subsequently used as the detail richness of the corresponding image block.
- Manner 2 Calculate the sum of the absolute values of the frequency coefficients in the frequency coefficient matrix; determine the detail richness of the image block corresponding to the frequency coefficient matrix according to the sum of the absolute values.
- the richness of details may be determined according to the range in which the sum of the absolute values is located.
- a plurality of numerical ranges can be divided in advance according to actual needs, and each numerical range corresponds to a different detail richness value.
- the detail richness value corresponding to the range in which the sum of the absolute values is located is subsequently used as the detail richness of the corresponding image block.
- the detail richness of different image blocks in the image to be compressed is different, and the enhancement processing is performed in combination with the detail richness of the image blocks, so that the enhancement processing is more targeted and helps to improve the image obtained by the final restoration. Picture quality.
- the number of rows and columns of the gain matrix is consistent with the number of rows and columns of the frequency coefficient matrix.
- gain matrices corresponding to different detail richnesses can be obtained through continuous optimization according to a plurality of experimental image blocks in different scenarios in advance, and a corresponding relationship between different detail richnesses and their corresponding gain matrices can be established to facilitate subsequent acquisition. .
- each element in the gain matrix is a gain factor.
- the gain factor of the first region in the gain matrix is greater than the gain factor of the second region; the second region refers to a region other than the first region in the gain matrix.
- the position and size of the above-mentioned first area are determined according to the actual situation.
- the frequency coefficients in the area corresponding to the first area in the frequency coefficient matrix are the frequency coefficients corresponding to the details of the image to be compressed.
- the point multiplication of the gain matrix and the frequency coefficient matrix realizes the enhancement processing of the frequency coefficients in the frequency coefficient matrix corresponding to the details of the image to be compressed to obtain the processed frequency coefficient matrix.
- the dot product is the multiplication of the elements at the corresponding positions in the two matrices.
- the above method before performing the dot product on the gain matrix and the frequency coefficient matrix, the above method further includes:
- the user feels that the gain matrix obtained according to the above method does not meet his needs, he can modify it according to the actual situation to improve the user experience. If the encoder needs to retain more image details, the user can choose to increase the gain factor of the first region in the gain matrix, where the first region corresponds to the region of the frequency coefficient corresponding to the image details to be compressed in the frequency coefficient matrix.
- a configuration interface can be provided for the user, and the user can input configuration information in the configuration interface.
- the number of rows and columns of the gain matrix is consistent with the number of rows and columns of the frequency coefficient matrix.
- the foregoing gain matrix may be determined according to the actual situation, and the manner of determination thereof is not specifically limited in this embodiment of the present application.
- Each element in the gain matrix is a gain factor, and the number of rows and columns of the gain matrix is consistent with the number of rows and columns of the frequency coefficient matrix.
- the gain factor of the first region in the gain matrix is greater than the gain factor of the second region; the second region refers to a region other than the first region in the gain matrix.
- the position and size of the above-mentioned first area are determined according to the actual situation.
- the frequency coefficients in the area corresponding to the first area in the frequency coefficient matrix are the frequency coefficients corresponding to the details of the image to be compressed.
- the gain matrix and the frequency coefficient matrix are dot-multiplied to implement enhancement processing on the frequency coefficients in the frequency coefficient matrix corresponding to the details of the image to be compressed.
- the gain matrix is modified according to the detail richness to obtain a modified gain matrix. Specifically, if the detail richness is greater than or equal to a preset threshold, the gain factor of the first region in the gain matrix is modified. Increase processing. If the detail richness is less than the preset threshold, the gain matrix is not modified.
- adjustment coefficients corresponding to different detail richnesses can be obtained through continuous optimization according to a plurality of experimental image blocks in different scenarios in advance, and a corresponding relationship between different detail richnesses and their corresponding adjustment coefficients can be established to facilitate subsequent acquisition. .
- the above method further includes: performing a two-dimensional discrete cosine transform on the image block to obtain the frequency coefficient matrix.
- the matrix of frequency coefficients includes first frequency coefficients.
- the position corresponding to the first frequency coefficient includes the row and column of the first frequency coefficient in the frequency coefficient matrix.
- the exponent of the preset power function is the adjustment coefficient.
- gamma is the above adjustment coefficient
- i and j are the row and column of the first frequency coefficient in the frequency coefficient matrix, respectively
- N is the sum of the total number of rows and columns of the frequency coefficient matrix.
- the amount of original data is reduced by down-sampling the image. This is equivalent to reducing the compression rate of the encoder to a certain extent, resulting in higher image quality.
- the size of the up-sampled image and the original to-be-compressed image is the same, and the residual image indicates the difference between the up-sampled image and the original to-be-compressed image.
- the up-sampled image and the initial to-be-compressed image can be regarded as two matrices, and the above-mentioned residual image can be obtained by subtracting the two matrices.
- the residual image may be superimposed on the down-sampled image to obtain a modified down-sampled image.
- the corrected down-sampled image can be directly used as the target down-sampled image.
- the modified down-sampled image can be regarded as a new down-sampled image, and the above steps 107, 108 and 109 can be iteratively executed until the number of iterations reaches a preset number of times. Take the corrected downsampled image obtained at the end of the iteration as the target downsampled image.
- the preset number of times may be set according to actual needs, which is not specifically limited in this embodiment of the present application.
- the target down-sampled image can be directly used as the to-be-compressed image.
- target down-sampled image is a dark-light image
- contrast enhancement processing on the target down-sampled image to obtain an enhanced target down-sampled image
- the corresponding histogram distribution is mainly concentrated in the low-brightness interval. If the histogram distribution of the target down-sampled image is centered in the low-brightness range, the target down-sampled image is determined to be a dark-light image.
- the noise of the original image is further amplified.
- simple denoising can be selected after contrast enhancement to ensure picture quality. That is, performing denoising processing on the enhanced target down-sampled image to obtain the to-be-compressed image.
- efficient denoising algorithms such as Gaussian filtering, non-local mean filtering, and DnCNN (feedforward noise reduction convolutional neural network) can be selected.
- the to-be-compressed image is a frame image in the to-be-compressed video.
- "compress the image to be compressed to obtain a compressed image according to the processed frequency coefficient matrix” specifically: according to the processed frequency coefficient matrix, compress the video. Compress to get the compressed video.
- the processed images corresponding to each frame of the image in the video to be compressed can be obtained according to the methods provided in the above embodiments; the processed image corresponding to each frame of the image in the video to be compressed is input into the encoder for encoding to form a code stream file, That is, the compressed video.
- the encoder can have h264, h265, etc.
- FIG. 2 shows a structural block diagram of a data transmission system provided by another example of the present application. As shown in FIG. 2 , the system includes: an encoding end device 201 and a decoding end device 202 . in,
- the encoding end device 201 is configured to obtain a frequency coefficient matrix corresponding to the image to be compressed; perform enhancement processing on the frequency coefficients in the frequency coefficient matrix corresponding to the details of the to-be-compressed image to obtain a processed frequency coefficient matrix; For the processed frequency coefficient matrix, compress the to-be-compressed image to obtain a compressed image; and send the compressed image to the decoding end device 202;
- the decoding end device 202 is configured to decode the compressed image to obtain a decoded image.
- the frequency coefficients corresponding to the details of the image to be compressed in the frequency coefficient matrix corresponding to the image to be compressed are enhanced, and then the image to be compressed is compressed according to the frequency coefficient matrix obtained after the enhancement processing.
- the frequency coefficient matrix corresponding to the to-be-compressed image is the representation of the to-be-compressed image in the frequency domain. That is to say, before compressing the to-be-compressed image, the image details in the to-be-compressed image will be enhanced in the frequency domain. In this way, during subsequent compression, more image details can be preserved, the degree of loss of image details caused by compression is reduced, and the quality of the restored image is improved.
- the to-be-compressed image is a frame image in the to-be-compressed video.
- the encoding end device 201 obtains a stream file through encoding, that is, after the compressed video, can transmit the compressed video through a transmission channel such as wireless.
- the decoding end device 202 receives the compressed video and uses the corresponding decoder to perform decoding operations. Then, the decoded image is upsampled to obtain the original resolution image. Specifically, the upsampling algorithm here is consistent with the previous downsampling algorithm.
- the decoding end device 202 can display the decoded image after obtaining the decoded image.
- the downsampling of the initial to-be-compressed image may be performed according to a preset downsampling ratio.
- a preset downsampling ratio For the specific down-sampling process, reference may be made to the corresponding content in the foregoing embodiments.
- Dark light detection is to judge whether the down-sampled image of the target is a dark light image. Not a dark light image, that is, a normal image.
- step 306 If it is a dark-light image, execute the following steps 304, 305 and 306 in sequence; otherwise, execute step 306 directly.
- Contrast enhancement processing can enhance its image details.
- the final image can be output for display or processing.
- the embodiment of the present application proposes a video preprocessing technology and framework under ultra-low bandwidth transmission. Under the condition of the same compression rate, the detailed characteristics of the image obtained by the decoding end are effectively improved, and the sensory quality of the image is effectively improved. Contributes to the processing of the back-end edge extraction algorithm.
- the detail enhancement method provided by the embodiments of the present application has a strong degree of fit with the compression scene.
- contrast enhancement algorithms or sharpening algorithms operate in the spatial domain of pixels, while encoder coding generally operates in the frequency domain. Therefore, the detail enhancement solution provided by the embodiment of the present application performs enhancement processing on the frequency coefficients in the frequency domain, and thus has a higher degree of fit with the encoder, so that the enhancement effect on details is better.
- the detail enhancement solution provided by the embodiments of the present application is simple to handle, and achieves a better image quality effect.
- FIG. 4 shows a structural block diagram of a data processing apparatus provided by an embodiment of the present application. As shown in Figure 4, the device includes:
- an acquisition module 401 configured to acquire a frequency coefficient matrix corresponding to the image to be compressed
- a processing module 402 configured to perform enhancement processing on the frequency coefficients in the frequency coefficient matrix corresponding to the details of the image to be compressed, to obtain a processed frequency coefficient matrix
- the compression module 403 is configured to compress the to-be-compressed image according to the processed frequency coefficient matrix to obtain a compressed image.
- the data processing apparatus shown in FIG. 4 can also execute the data processing methods in the foregoing embodiments.
- the parts not described in detail in this embodiment reference may be made to the relevant content of the foregoing embodiments.
- the execution process and technical effects of the technical solution reference may be made to the relevant content of the foregoing embodiments, which will not be repeated here.
- FIG. 5 shows a schematic structural diagram of an electronic device provided by an embodiment of the present application.
- the electronic device includes a memory 1101 and a processor 1102 .
- the memory 1101 may be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method to operate on the electronic device.
- Memory 1101 may be implemented by any type of volatile or non-volatile storage device or combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EEPROM), Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.
- SRAM Static Random Access Memory
- EEPROM Electrically Erasable Programmable Read Only Memory
- EEPROM Erasable Programmable Read Only Memory
- EPROM Programmable Read Only Memory
- PROM Programmable Read Only Memory
- ROM Read Only Memory
- Magnetic Memory Flash Memory
- the memory 1101 is used to store programs
- the processor 1102, coupled with the memory 1101, is configured to execute the program stored in the memory 1101, so as to implement the data processing methods provided by the foregoing method embodiments.
- the embodiments of the present application further provide a computer-readable storage medium, including instructions, which, when executed on a computer, enable the computer to implement the steps or functions of the data processing methods provided by the above method embodiments.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
Abstract
一种数据处理方法、数据传输系统、设备及存储介质，其中，获取待压缩图像对应的频率系数矩阵（步骤101）；对频率系数矩阵中对应于待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵（步骤102）；根据处理后的频率系数矩阵，对待压缩图像进行压缩得到压缩后的图像（步骤103）。本技术方案能够降低压缩所导致的图像细节丢失的程度，从而提高还原得到的图像的质量。
Description
本申请实施例涉及图像处理技术领域，尤其涉及一种数据处理方法、数据传输系统、设备及存储介质。
为了方便存储和传输，需要对数据进行压缩。通常，采用编码器对数据进行压缩，采用解码器对压缩后的数据进行还原。
以图像或视频的传输系统为例，原始图像或原始视频的数据量较大，而传输信道受限，因此，需要对原始图像或原始视频进行有效的压缩才可以进行传输。当编码器采用的压缩比较大时，解码器解码得到的图像或视频信息丢失严重，以至于无法提取有效的图像或视频信息。
发明内容
本申请实施例提供了一种数据处理方法、数据传输系统、设备及存储介质，可以降低压缩所导致的图像细节丢失的程度，从而提高还原得到的图像的质量。
本申请实施例的第一方面提供了一种数据处理方法，包括：
获取待压缩图像对应的频率系数矩阵；
对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵；
根据所述处理后的频率系数矩阵，对所述待压缩图像进行压缩得到压缩后的图像。
本申请实施例的第二方面提供了一种数据处理装置，包括：
获取模块，用于获取待压缩图像对应的频率系数矩阵；
处理模块，用于对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵；
压缩模块，用于根据所述处理后的频率系数矩阵，对所述待压缩图像进行压缩得到压缩后的图像。
本申请实施例的第三方面提供了一种数据传输系统，包括：编码端设备和解码端设备；
所述编码端设备，用于获取待压缩图像对应的频率系数矩阵；对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵；根据所述处理后的频率系数矩阵，对所述待压缩图像进行压缩得到压缩后的图像；并将所述压缩后的图像发送给所述解码端设备；
所述解码端设备，用于对所述压缩后的图像进行解码，得到解码后图像。
本申请实施例的第四方面提供了一种电子设备，包括：存储器与处理器；
所述存储器，用于存储程序；
所述处理器，与所述存储器耦合，用于执行所述存储器中存储的所述程序，以用于执行上述任一项所述的数据处理方法。
本申请实施例的第五方面提供了一种计算机可读存储介质，包括指令，当其在计算机上运行时，使得所述计算机实现上述任一项所述的数据处理方法。
本申请实施例提供的技术方案中，对待压缩图像对应的频率系数矩阵中对应于待压缩图像细节的频率系数进行增强处理，然后再根据增强处理后得到的频率系数矩阵对待压缩图像进行压缩。其中，待压缩图像对应的频率系数矩阵为待压缩图像在频域中的表示。也就是说，在对待压缩图像进行压缩之前，会先在频域针对待压缩图像中的图像细节进行增强。这样后续压缩时，可以更多地保留图像细节，降低了压缩所导致的图像细节丢失的程度，从而 提高还原得到的图像质量。
图1为本申请实施例提供的一种数据处理方法一个实施例的流程示意图；
图2为本申请实施例提供的一种数据传输系统的一个实施例的结构框图；
图3为本申请实施例提供的一种数据处理方法的又一个实施例的流程示意图；
图4为本申请实施例提供的一种数据处理装置的一个实施例的结构框图；
图5为本申请实施例提供的一种电子设备的一个实施例的结构框图。
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述。
除非另有定义，本文所使用的所有的技术和科学术语与属于本申请的技术领域的技术人员通常理解的含义相同。本文中在本申请的说明书中所使用的术语只是为了描述具体的实施例的目的，不是旨在于限制本申请。
目前，在进行图像或视频压缩之前，所采用的预处理技术主要有以下几点：
1)图像及视频去噪及平滑技术
噪声对于任何实际的图像或视频采集来说均是不可避免的，如果在编码前未将不必要的噪声去除，不仅会影响解码得到的图像或视频的质量，而且后面的编码过程中还需对噪声进行编码，降低了编码效率。当然，因为往往图像和噪声是交织在一起的，我们需要对去噪的强度进行折中，避免过度去噪导致的图像或视频的细节丢失。
2)基于人眼视觉模型的预处理技术
该技术主要从人眼视觉角度出发，确定哪些信息对人眼是不重要或者不感兴趣的；并通过预处理技术将这些信息从原始的图像或者视频信号中移除， 提高其压缩效率。
上述图像及视频预处理技术往往从编码器的角度出发，通过一系列的处理手段降低图像和视频中的人眼冗余信息、噪声信息等，进而提高编码器的压缩性能，使得在相同的压缩比下信道可以传输更多的图像内容。然而，在超高压缩的情况下，通过前面预处理移除的冗余信号在编码过程中也会丢弃掉，也就是说，这些预处理在带宽极度受限的情况下并无法对视频的画质产生明显的收益。
本申请实施例提供一种新的预处理技术，通过增强图像细节，以避免其在压缩过程中被丢弃掉，从而获得相对较高的图像质量。
图1示出了本申请一实施例提供的数据处理方法的流程示意图。该方法的执行主体可以为客户端，也可以为服务端。其中，所述客户端可以是集成在终端上的一个具有嵌入式程序的硬件，也可以是安装在终端中的一个应用软件，还可以是嵌入在终端操作系统中的工具软件等，本申请实施例对此不作限定。该终端可以为包括手机、平板电脑、摄像头、无人机等任意终端设备。其中，服务端可以是常用服务器、云端或虚拟服务器等，本申请实施例对此不作具体限定。如图1所示，该方法包括：
101、获取待压缩图像对应的频率系数矩阵。
102、对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵。
103、根据所述处理后的频率系数矩阵，对所述待压缩图像进行压缩得到压缩后的图像。
上述101中，上述待压缩图像可以是相机拍摄得到的照片、人工合成的动漫图片，等等。以航拍场景为例，上述待压缩图像可以是航拍无人机在执行航拍任务时拍摄的图像，还可以是航拍无人机在执行航拍任务时拍摄的视频中的视频帧。
实际应用时，可对待压缩图像进行从空间域到频域的变换处理，得到待压缩图像对应的频率系数矩阵。在一实例中，上述变换处理所采用的变换算法可以是离散余弦变换(Discrete Cosine Transform，DCT)，也即对待压缩图像进行离散余弦变换，得到待压缩图像对应的频率系数矩阵。
在一实例中，可将待压缩图像作为整体进行从空间域到频域的变换处理，得到待压缩图像对应的频率系数矩阵。
在另一实例中，可对待压缩图像进行分割，得到多个图像块；分别对各图像块进行从空间域到频域的变换处理，得到各图像块对应的频率系数矩阵。其中，待压缩图像对应的频率系数矩阵包括待压缩图像的多个图像块各自对应的频率系数矩阵。上述图像块的大小可根据实际需要来设定，本申请实施例对此不做具体限定。例如：上述图像块的大小可以是8*8。
上述102中，示例性的待压缩图像中的高频分量对应于待压缩图像细节，且待压缩图像中的低频分量(低频信号)和高频分量(高频信号)分布在频率系数矩阵中的不同区域。采用不同的变换算法，待压缩图像中的高频分量在其频率系数矩阵中的所处区域不同。也就是说，根据上述变换处理所采用的变换算法，可以预先确定待压缩图像中的高频分量在频率系数矩阵中的所处区域，也即可以预先确定频率系数矩阵中对应于待压缩图像细节的区域，该区域也即对应于待压缩图像细节的频率系数所在区域。此外，频率系数矩阵中对应于待压缩图像细节的频率系数所在区域的大小、形状可以根据实际需要来设置，本申请对此不作限定。
以上述变换为二维DCT为例，频率系数矩阵中左上角区域对应于待压缩图像中的低频分量，其右下角区域对应于待压缩图像中的高频分量，也即对应于待压缩图像细节。其中，左上角区域和右下角区域的大小、形状可根据实际需要来设置。
上述增强处理可以是：仅对频率系数矩阵中对应于待压缩图像细节的频率系数进行增强处理，频率系数矩阵中其他频率系数保持不变；或者，对频率系数矩阵中所有的频率系数都进行增强处理，但是对应于待压缩图像细节 的频率系数的增强幅度大于其他频率系数的增强幅度；或者，对频率系数矩阵中所有的频率系数都进行弱化处理，但是对应于待压缩图像细节的频率系数的弱化幅度小于其他频率系数的弱化幅度。上述其他频率系数指的是频率系数矩阵中除对应于待压缩图像细节的频率系数以外的频率系数。也即是，对所述频率系数矩阵中对应于所述原始图像细节的频率系数进行相对于所述频率系数矩阵中其他频率系数的相对增强处理，得到处理后的频率系数矩阵。
在一种可实现的方案中，可预先设置一个增益矩阵，该增益矩阵中每一个元素为一个增益因子，且该增益矩阵的行列数与所述频率系数矩阵的行列数一致。增益矩阵中第一区域的增益因子大于第二区域的增益因子；第二区域指的是增益矩阵中除第一区域以外的区域。上述第一区域的位置和大小根据实际情况来确定。所述频率系数矩阵中与第一区域相对应的区域内的频率系数即为对应于待压缩图像细节的频率系数。将增益矩阵与频率系数矩阵进行点乘，以实现对频率系数矩阵中对应于待压缩图像细节的频率系数进行增强处理。
上述103中，在一种可实现的方案中，可直接对处理后的频率系数矩阵依次进行量化、编排和编码，最终得到压缩后的图像。
在本实施例中，相当于对编码器的编码方式进行了更新。而解码器与编码器的处理流程通常需要严格对称，才能够成功解码。因此，若采用本实施例提供的方案，还需要对解码器的解码方式进行相应更新。
在另一种可实现的方案中，上述103中“根据所述处理后的频率系数矩阵，对所述待压缩图像进行压缩得到压缩后的图像”，具体可采用如下步骤来实现：
1031、对所述处理后的频率系数矩阵进行从频域到空间域的逆变换处理，得到处理后图像。
1032、对所述处理后图像进行压缩，得到压缩后的图像。
上述1031中，在一具体实例中，若上述变换采用的是二维DCT，那么，逆变换就可采用二维IDCT(Inverse Discrete Cosine Transform，逆离散余弦变换)。
当所述频率系数矩阵为多个，且多个所述频率系数矩阵与所述待压缩图像中的多个图像块一一对应时，处理后的频率系数矩阵也就为多个；分别对多个处理后的频率系数矩阵进行从频域到空间域的逆变换处理，得到多个处理后图像块；将多个处理后图像块进行拼接，得到处理后图像。
上述1032中，可将处理后图像输入到用于压缩的编码器中进行编码，得到压缩后的图像。
在本实施例中，对处理后的频率系数进行逆变换得到处理后图像，让编码器作用于处理后图像。这样一来，可以直接使用现有的编码器和解码器，无需重新设计编码器和解码器，不仅可降低成本，还可提高本申请实施例提供的技术方案的适用性。
本申请实施例提供的技术方案中，对待压缩图像对应的频率系数矩阵中对应于待压缩图像细节的频率系数进行增强处理，然后再根据增强处理后得到的频率系数矩阵对待压缩图像进行压缩。其中，待压缩图像对应的频率系数矩阵为待压缩图像在频域中的表示。也就是说，在对待压缩图像进行压缩之前，会先在频域针对待压缩图像中的图像细节进行增强。这样后续压缩时，可以更多地保留图像细节，降低了压缩所导致的图像细节丢失的程度，从而确保了最终还原得到的图像具有较好的画面质量。本申请实施例提供的技术方案能够让解码端展现更多有效的信息。
可选的，所述频率系数矩阵为多个；多个所述频率系数矩阵与所述待压缩图像中的多个图像块一一对应。上述102中“对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵”，可采用如下步骤来实现：
1021、确定所述频率系数矩阵对应的图像块的细节丰富度。
1022、根据所述细节丰富度，对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵。
上述1021中，图像块的纹理越丰富，其细节丰富度就越高。可采用如下多种方式中的一种或多种来确定图像块的细节丰富度：
方式一：计算所述频率系数矩阵对应的图像块中像素值的分布方差；根据所述分布方差，确定所述细节丰富度。
其中，分布方差越大，细节丰富度越高。
示例性的，可根据所述分布方差所处范围，确定所述细节丰富度。实际应用时，可事先根据实际需要划分出多个分布方差范围，并且每个分布方差范围对应不同的细节丰富度值。这样，后续将分布方差所处范围对应的细节丰富度值作为相应图像块的细节丰富度。
方式二：计算所述频率系数矩阵中频率系数的绝对值之和；根据所述绝对值之和，确定所述频率系数矩阵对应的图像块的细节丰富度。
其中，绝对值之和越大，细节丰富度越高。
具体地，可根据所述绝对值之和所处范围，确定所述细节丰富度。实际应用时，可事先根据实际需要划分出多个数值范围，并且每个数值范围对应不同的细节丰富度值。这样，后续将绝对值之和所处范围对应的细节丰富度值作为相应图像块的细节丰富度。
上述1022，根据所述细节丰富度，对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理。例如：细节丰富度越大，所述频率系数矩阵中对应于所述待压缩图像细节的频率系数的增强幅度就越大。
在本实施例中，待压缩图像中不同的图像块的细节丰富度是不同的，结合图像块的细节丰富度进行增强处理，使得增强处理更具针对性，有助于提高最终还原得到的图像画面质量。
在一实例中，上述1022中“根据所述细节丰富度，对所述频率系数矩阵 中对应于所述待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵”，具体可采用如下步骤来实现：
S11、根据所述细节丰富度，获取相应的增益矩阵。
其中，所述增益矩阵的行列数与所述频率系数矩阵的行列数一致。
S12、将所述增益矩阵与所述频率系数矩阵进行点乘，以实现对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵。
上述S11中，可事先根据多个不同场景下的实验图像块，通过不断优化得到不同细节丰富度对应的增益矩阵，并建立不同细节丰富度与其对应的增益矩阵之间的对应关系，方便后续获取。
上述S12中，增益矩阵中每一个元素为一个增益因子。增益矩阵中第一区域的增益因子大于第二区域的增益因子；第二区域指的是增益矩阵中除第一区域以外的区域。上述第一区域的位置和大小根据实际情况来确定。所述频率系数矩阵中与第一区域相对应的区域内的频率系数即为对应于待压缩图像细节的频率系数。这样，将所述增益矩阵与所述频率系数矩阵进行点乘，就实现了对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵。其中，点乘就是两个矩阵中对应位置处的元素相乘。
可选的，在将所述增益矩阵与所述频率系数矩阵进行点乘之前，上述方法，还包括：
104、获取用户的配置信息。
105、根据所述配置信息，对所述增益矩阵中相应元素进行修正。
这样，用户若觉得按照上述方法获取的增益矩阵不符合自己的需求时，可以根据实际情况进行修改，提高用户体验。如果编码端需要保留更多的图像细节，用户可以选择对增益矩阵中上述第一区域的增益因子进行增大处理， 第一区域与频率系数矩阵中待压缩图像细节对应的频率系数所在区域对应。
上述104中，可为用户提高一个配置界面，用户可在该配置界面输入配置信息。
在又一实例中，上述1022中“根据所述细节丰富度，对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵”，具体可采用如下步骤来实现：
S21、获取增益矩阵。
其中，所述增益矩阵的行列数与所述频率系数矩阵的行列数一致。
S22、根据所述细节丰富度，对所述增益矩阵进行修正，得到修正后增益矩阵。
S23、将所述修正后增益矩阵与所述频率系数矩阵进行点乘，以实现对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵。
上述S21中，上述增益矩阵可以根据实际情况来确定，本申请实施例对其确定方式不作具体限定。该增益矩阵中每一个元素为一个增益因子，且该增益矩阵的行列数与所述频率系数矩阵的行列数一致。增益矩阵中第一区域的增益因子大于第二区域的增益因子；第二区域指的是增益矩阵中除第一区域以外的区域。上述第一区域的位置和大小根据实际情况来确定。所述频率系数矩阵中与第一区域相对应的区域内的频率系数即为对应于待压缩图像细节的频率系数。将增益矩阵与频率系数矩阵进行点乘，以实现对频率系数矩阵中对应于待压缩图像细节的频率系数进行增强处理。
上述S22中，根据所述细节丰富度，对所述增益矩阵进行修正，得到修正后增益矩阵，具体地，若细节丰富度大于或等于预设阈值，则对增益矩阵中第一区域的增益因子进行增大处理。若细节丰富度小于预设阈值，则不对增益矩阵进行修正。
在又一实例中，上述1022中“根据所述细节丰富度，对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵”，具体可采用如下步骤来实现：
S31、根据所述图像块的细节丰富度，获取对应的调整系数。
S32、根据所述调整系数以及所述频率系数矩阵中每一个频率系数对应的位置，确定每一个频率系数对应的增益因子。
S33、将所述频率系数矩阵中每一个频率系数乘以各自对应的增益因子，以实现对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到所述处理后的频率系数矩阵。
上述S31中，可事先根据多个不同场景下的实验图像块，通过不断优化得到不同细节丰富度对应的调整系数，并建立不同细节丰富度与其对应的调整系数之间的对应关系，方便后续获取。
在一种可实现的方案中，上述方法，还包括：对所述图像块进行二维离散余弦变换，得到所述频率系数矩阵。所述频率系数矩阵中包括第一频率系数。相应的，上述S32中“根据所述调整系数以及所述第一频率系数对应的位置，确定所述第一频率系数对应的增益因子”，可采用如下步骤来实现：
S321、根据所述第一频率系数对应的位置，确定所述第一频率系数所在行和所在列的数值之和。
其中，第一频率系数对应的位置包括第一频率系数在频率系数矩阵中的所在行和所在列。
S322、对所述数值之和进行归一化，得到预设幂函数的输入数据。
S323、将所述输入数据输入至所述预设幂函数中，得到所述第一频率系数对应的增益因子。
其中，所述预设幂函数的指数为所述调整系数。
具体地，可采用下述公式(1)来实现上述步骤S321、S322和S323：
gain＝((i+j)/N)
gamma (1)
其中，gamma即为上述调整系数；i、j分别为第一频率系数在频率系数矩阵中所在行和所在列；N为频率系数矩阵的总行数和总列数之和。
为了节省带宽，从降低原始数据量的角度出发，通过将图像进行下采样，降低原始的数据量。这等同于一定程度上降低了编码器的压缩率，从而得到较高的图像画质。
在超低带宽的压缩中，因为对压缩比的要求很高，经常达到400倍甚至更高，同时又希望比较好的恢复图像质量。因此，我们通过下采样的方式对图像的数据量进行降低，使得在相同的压缩率下可以得到更好的画质。申请人通过实验也验证了这一结论的有效性。并且，为了有效降低下采样过程对信息的过度丢失，提出了一种基于残差的下采样方式。具体地，上述方法，还可包括：
106、对初始待压缩图像执行下采样操作，得到下采样图像。
107、对所述下采样图像进行上采样操作，得到上采样图像。
108、确定所述上采样图像与所述初始待压缩图像之间的残差图像。
109、根据所述残差图像，对所述下采样图像进行修正，得到修正后下采样图像。
110、根据所述修正后下采样图像，确定目标下采样图像。
111、根据所述目标下采样图像，确定所述待压缩图像。
上述106中，下采样的具体实现原理和过程可参见现有技术，在此不再详述。
上述107中，上采样的具体实现原理和过程也可参见现有技术，在此不再详述。
上述108中，上采样图像和初始待压缩图像的尺寸相同，残差图像指示 上采样图像与初始待压缩图像之间的差别。
具体地，可将上采样图像和初始待压缩图像看成是两个矩阵，两个矩阵相减，即可得到上述残差图像。
上述109中，具体地，可将所述残差图像叠加到所述下采样图像上，得到修正后下采样图像。
上述110中，在一实例中，可直接将所述修正后下采样图像作为目标下采样图像。
在另一实例中，可将修正后下采样图像作为新的下采样图像，继续迭代执行上述步骤107、108和109，直到迭代次数到达预设次数结束。将迭代结束时得到的修正后下采样图像作为目标下采样图像。预设次数可根据实际需要来设定，本申请实施例对此不做具体限定。
上述111中，在一实例中，可将所述目标下采样图像直接作为所述待压缩图像。
在另一实例中，上述111中“根据所述目标下采样图像，确定所述待压缩图像”，
1111、根据所述目标下采样图像的直方图分布情况，确定所述目标下采样图像是否为暗光图像。
1112、所述目标下采样图像为暗光图像时，对所述目标下采样图像进行对比度增强处理，得到增强后的目标下采样图像。
1113、根据所述增强后的目标下采样图像，确定所述待压缩图像。
上述1111中，对于一个处在暗光条件下的图像，其所对应的直方图分布主要集中在低亮度的区间。若目标下采样图像的直方图分布居中在低亮度区间，则确定目标下采样图像为暗光图像。
上述1112中，为了解码端的显示效果及其后续对应的算法处理，我们需 要将其暗光场景做对应的增强。对于夜晚的场景，往往存在对比度不足的情况。所述目标下采样图像为暗光图像时，对所述目标下采样图像进行对比度增强处理，得到增强后的目标下采样图像。本申请实施例对对比度增强所采用的算法不做任何限制，可选用常用的直方图均衡、gamma变换、基于retinex的图像增强技术，等等。对比度增强后，图像的亮度也会相应增大。
上述1113中，随着对比度增强，原始图像的噪声也被进一步放大。为此，在对比度增强后可选择进行简单的去噪以保证画面质量。也即：对所述增强后的目标下采样图像进行去噪处理，得到所述待压缩图像。本申请实施例对去噪的算法不做限制，可以选择高斯滤波、非局部均值滤波、DnCNN(前馈降噪卷积神经网络)等高效的去噪算法。
在一实例中，所述待压缩图像为待压缩视频中的帧图像。相应的，上述103中“根据所述处理后的频率系数矩阵，对所述待压缩图像进行压缩得到压缩后的图像”，具体为：根据所述处理后的频率系数矩阵，对所述视频进行压缩，得到压缩后的视频。可按照上述各实施例中提供的方法，得到待压缩视频中各帧图像对应的处理后图像；将待压缩视频中各帧图像对应的处理后图像输入到编码器进行编码，形成码流文件，也即压缩后的视频。这里编码器可以有h264、h265等。
图2示出了本申请又一实例提供的数据传输系统的结构框图。如图2所述，该系统，包括：编码端设备201和解码端设备202。其中，
所述编码端设备201，用于获取待压缩图像对应的频率系数矩阵；对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵；根据所述处理后的频率系数矩阵，对所述待压缩图像进行压缩得到压缩后的图像；并将所述压缩后的图像发送给所述解码端设备202；
所述解码端设备202，用于对所述压缩后的图像进行解码，得到解码后图 像。
本申请实施例提供的技术方案中，对待压缩图像对应的频率系数矩阵中对应于待压缩图像细节的频率系数进行增强处理，然后再根据增强处理后得到的频率系数矩阵对待压缩图像进行压缩。其中，待压缩图像对应的频率系数矩阵为待压缩图像在频域中的表示。也就是说，在对待压缩图像进行压缩之前，会先在频域针对待压缩图像中的图像细节进行增强。这样后续压缩时，可以更多地保留图像细节，降低了压缩所导致的图像细节丢失的程度，从而提高还原得到的图像的质量。
在一实例中，所述待压缩图像为待压缩视频中的帧图像。上述编码端设备201通过编码得到码流文件，也即压缩后的视频后，可通过无线等传输信道将压缩后的视频进行传输。解码端设备202接收到压缩后的视频并使用对应的解码器进行解码操作。然后，将解码得到的图像进行上采样得到原始分辨率的图像。具体地，本处的上采样算法与前面的下采样的算法保持一致。
在一实例中，所述解码端设备202，得到解码后图像后可进行展示。
这里需要说明的是：本申请实施例提供的所述方法中各步骤未尽详述的内容可参见上述实施例中的相应内容，此处不再赘述。此外，本申请实施例提供的所述方法中除了上述各步骤以外，还可包括上述各实施例中其他部分或全部步骤，具体可参见上述各实施例相应内容，在此不再赘述。
下面将结合图3对本申请实施例提供的技术方案进行详细介绍：
301、输入初始待压缩图像。
302、对初始待压缩图像进行下采样处理，得到目标下采样图像。
可按照预设的下采样比例进行初始待压缩图像的下采样。具体的下采样过程可参见上述各实施例中相应内容。
303、对目标下采样图像进行暗光检测。
暗光检测也即是判断目标下采样图像是否为暗光图像。不是暗光图像，也即是正常图像。
如果是暗光图像，则依次执行下述步骤304、305和306；否则，直接执行步骤306。
304、对目标下采样图像进行对比度增强处理，得到增强后的目标下采样图像。
对比度增强处理可以提升其图像细节特征。
305、对增强后的目标下采样图像进行去噪处理，得到待压缩图像。
306、对待压缩图像进行图像细节增强处理，得到处理后图像。
具体的增强过程可参见上述各实施例中相应内容，在此不再赘述。
307、对处理后图像进行编码，得到码流，也即压缩后的图像。
308、对码流进行解码，得到解码后图像。
309、对解码后图像进行上采样，得到最终图像。
310、输出最终图像。
实际应用时，可输出最终图像进行显示或者处理。
本申请实施例提出了一种超低带宽传输下的视频预处理技术及框架，在压缩率相同的情况下，有效提高了解码端所获取的图像细节特征，有效提升了图像的感官质量，也有助于后端边缘提取算法的处理。
本申请实施例提供的细节增强方法和压缩场景具有很强的契合度。通常，对比度增强算法或锐化算法都是在像素的空间域操作，而编码器编码一般是在频域操作。因此，本申请实施例提供的细节增强方案通过对频域的频率系数进行增强处理，与编码器的契合度更高，使得对细节的增强效果更好。并且，本申请实施例提供的细节增强方案处理简单，并达到了较好的画质效果。
此外，通过基于残差以及迭代的下采样技术，有效提升了解码端上采样端画面质量的表现。针对不同的图像场景区分处理，有效提升了夜晚图像画面质量的表现。
本申请实施例提供的技术方案可以应用于图像及视频传输的场景中，包括无人机图传、广播图传等。
图4示出了本申请一实施例提供的数据处理装置的结构框图。如图4所示，该装置，包括：
获取模块401，用于获取待压缩图像对应的频率系数矩阵；
处理模块402，用于对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵；
压缩模块403，用于根据所述处理后的频率系数矩阵，对所述待压缩图像进行压缩得到压缩后的图像。
需要注意的是，图4所示的数据处理装置还可以执行上述各实施例中的数据处理方法，本实施例未详细描述的部分，可参考上述实施例的相关内容。该技术方案的执行过程和技术效果参见上述实施例的相关内容，在此不再赘述。
图5示出了本申请一实施例提供的电子设备的结构示意图。如图5所示，所述电子设备包括存储器1101以及处理器1102。存储器1101可被配置为存储其它各种数据以支持在电子设备上的操作。这些数据的示例包括用于在电子设备上操作的任何应用程序或方法的指令。存储器1101可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。
所述存储器1101，用于存储程序；
所述处理器1102，与所述存储器1101耦合，用于执行所述存储器1101中存储的所述程序，以实现上述各方法实施例提供的数据处理方法。
相应地，本申请实施例还提供一种计算机可读存储介质，包括指令，当其在计算机上运行时，使得所述计算机实现上述各方法实施例提供的数据处理方法的步骤或功能。
以上各个实施例中的技术方案、技术特征在与本相冲突的情况下均可以单独，或者进行组合，只要未超出本领域技术人员的认知范围，均属于本申请保护范围内的等同实施例。
以上所述仅为本申请的实施例，并非因此限制本申请的专利范围，凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本申请的专利保护范围内。
最后应说明的是：以上各实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述各实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的范围。
Claims (19)
- 一种数据处理方法，其特征在于，包括：获取待压缩图像对应的频率系数矩阵；对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵；根据所述处理后的频率系数矩阵，对所述待压缩图像进行压缩得到压缩后的图像。
- 根据权利要求1所述的方法，其特征在于，所述频率系数矩阵为多个；多个所述频率系数矩阵与所述待压缩图像中的多个图像块一一对应；对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵，包括：确定所述频率系数矩阵对应的图像块的细节丰富度；根据所述细节丰富度，对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵。
- 根据权利要求2所述的方法，其特征在于，根据所述细节丰富度，对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵，包括：根据所述细节丰富度，获取相应的增益矩阵；其中，所述增益矩阵的行列数与所述频率系数矩阵的行列数一致；将所述增益矩阵与所述频率系数矩阵进行点乘，以实现对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵。
- 根据权利要求3所述的方法，其特征在于，在将所述增益矩阵与所述频率系数矩阵进行点乘之前，还包括：获取用户的配置信息；根据所述配置信息，对所述增益矩阵中相应元素进行修正。
- 根据权利要求2所述的方法，其特征在于，根据所述细节丰富度，对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵，包括：获取增益矩阵；其中，所述增益矩阵的行列数与所述频率系数矩阵的行列数一致；根据所述细节丰富度，对所述增益矩阵进行修正，得到修正后增益矩阵；将所述修正后增益矩阵与所述频率系数矩阵进行点乘，以实现对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵。
- 根据权利要求2所述的方法，其特征在于，根据所述细节丰富度，对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵，包括：根据所述图像块的细节丰富度，获取对应的调整系数；根据所述调整系数以及所述频率系数矩阵中每一个频率系数对应的位置，确定每一个频率系数对应的增益因子；将所述频率系数矩阵中每一个频率系数乘以各自对应的增益因子，以实现对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到所述处理后的频率系数矩阵。
- 根据权利要求6所述的方法，其特征在于，还包括：对所述图像块进行二维离散余弦变换，得到所述频率系数矩阵。
- 根据权利要求7所述的方法，其特征在于，所述频率系数矩阵中包括第一频率系数；根据所述调整系数以及所述第一频率系数对应的位置，确定所述第一频率系数对应的增益因子，包括：根据所述第一频率系数对应的位置，确定所述第一频率系数在所述频率系数矩阵矩阵中所在行和所在列的数值之和；对所述数值之和进行归一化，得到预设幂函数的输入数据；将所述输入数据输入至所述预设幂函数中，得到所述第一频率系数对应的增益因子；其中，所述预设幂函数的指数为所述调整系数。
- 根据权利要求2至8中任一项所述的方法，其特征在于，确定所述频率系数矩阵对应的图像块的细节丰富度，包括：计算所述频率系数矩阵对应的图像块中像素值的分布方差；根据所述分布方差，确定所述细节丰富度；其中，分布方差越大，细节丰富度越高。
- 根据权利要求2至8中任一项所述的方法，其特征在于，确定所述频率系数矩阵对应的图像块的细节丰富度，包括：计算所述频率系数矩阵中频率系数的绝对值之和；根据所述绝对值之和，确定所述频率系数矩阵对应的图像块的细节丰富度；其中，绝对值之和越大，细节丰富度越高。
- 根据权利要求1至8中任一项所述的方法，其特征在于，根据所述处理后的频率系数矩阵，对所述待压缩图像进行压缩得到压缩后的图像，包括：对所述处理后的频率系数矩阵进行从频域到空间域的逆变换处理，得到 处理后图像；对所述处理后图像进行压缩，得到压缩后的图像。
- 根据权利要求1至8中任一项所述的方法，其特征在于，还包括：对初始待压缩图像执行下采样操作，得到下采样图像；对所述下采样图像进行上采样操作，得到上采样图像；确定所述上采样图像与所述初始待压缩图像之间的残差图像；根据所述残差图像，对所述下采样图像进行修正，得到修正后下采样图像；根据所述修正后下采样图像，确定目标下采样图像；根据所述目标下采样图像，确定所述待压缩图像。
- 根据权利要求12所述的方法，其特征在于，根据所述目标下采样图像，确定所述待压缩图像，包括：根据所述目标下采样图像的直方图分布情况，确定所述目标下采样图像是否为暗光图像；所述目标下采样图像为暗光图像时，对所述目标下采样图像进行对比度增强处理，得到增强后的目标下采样图像；根据所述增强后的目标下采样图像，确定所述待压缩图像。
- 根据权利要求13所述的方法，其特征在于，根据所述增强后的目标下采样图像，确定所述待压缩图像，对所述增强后的目标下采样图像进行去噪处理，得到所述待压缩图像。
- 根据权利要求1至8中任一项所述的方法，其特征在于，所述待压缩图像为待压缩视频中的帧图像；根据所述处理后的频率系数矩阵，对所述待压缩图像进行压缩得到压缩 后的图像，包括：根据所述处理后的频率系数矩阵，对所述视频进行压缩，得到压缩后的视频。
- 一种数据处理装置，其特征在于，包括：获取模块，用于获取待压缩图像对应的频率系数矩阵；处理模块，用于对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵；压缩模块，用于根据所述处理后的频率系数矩阵，对所述待压缩图像进行压缩得到压缩后的图像。
- 一种数据传输系统，其特征在于，包括：编码端设备和解码端设备；所述编码端设备，用于获取待压缩图像对应的频率系数矩阵；对所述频率系数矩阵中对应于所述待压缩图像细节的频率系数进行增强处理，得到处理后的频率系数矩阵；根据所述处理后的频率系数矩阵，对所述待压缩图像进行压缩得到压缩后的图像；并将所述压缩后的图像发送给所述解码端设备；所述解码端设备，用于对所述压缩后的图像进行解码，得到解码后图像。
- 一种电子设备，其特征在于，包括：存储器与处理器；所述存储器，用于存储程序；所述处理器，与所述存储器耦合，用于执行所述存储器中存储的所述程序，以用于执行权利要求1-15中任一项所述的数据处理方法。
- 一种计算机可读存储介质，其特征在于，包括指令，当其在计算机上运行时，使得所述计算机实现权利要求1-15中任一项所述的数据处理方法。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2021/084471 WO2022205094A1 (zh) | 2021-03-31 | 2021-03-31 | 数据处理方法、数据传输系统、设备及存储介质 |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2021/084471 WO2022205094A1 (zh) | 2021-03-31 | 2021-03-31 | 数据处理方法、数据传输系统、设备及存储介质 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2022205094A1 true WO2022205094A1 (zh) | 2022-10-06 |
Family
ID=83455362
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2021/084471 WO2022205094A1 (zh) | 2021-03-31 | 2021-03-31 | 数据处理方法、数据传输系统、设备及存储介质 |
Country Status (1)
| Country | Link |
|---|---|
| WO (1) | WO2022205094A1 (zh) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2004172657A (ja) * | 2002-11-15 | 2004-06-17 | Nec Viewtechnology Ltd | 画像伸長縮小装置、方法及びプログラム |
| US20050271284A1 (en) * | 2004-06-07 | 2005-12-08 | Chen-Chung Chen | Printing quality enhancement via graphic/text detection method in compression (JPEG) image |
| CN102281446A (zh) * | 2011-09-20 | 2011-12-14 | 西南交通大学 | 一种分布式视频编码中基于视觉感知特性的量化方法 |
| CN103745443A (zh) * | 2014-01-10 | 2014-04-23 | 北京优纳科技有限公司 | 提高图像质量的方法和设备 |
| CN104883579A (zh) * | 2015-06-08 | 2015-09-02 | 电子科技大学 | 一种基于空-频域的联合视频图像的上采样方法 |
| CN105144681A (zh) * | 2013-03-15 | 2015-12-09 | 三星电子株式会社 | 用频率提高来创建图像中的细节 |
| CN106612436A (zh) * | 2016-01-28 | 2017-05-03 | 四川用联信息技术有限公司 | 一种基于dct变换下的视觉感知修正图像压缩方法 |
| CN111225216A (zh) * | 2020-01-10 | 2020-06-02 | Tcl华星光电技术有限公司 | 显示器补偿表压缩方法、装置、系统及显示器 |
-
2021
- 2021-03-31 WO PCT/CN2021/084471 patent/WO2022205094A1/zh active Application Filing
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2004172657A (ja) * | 2002-11-15 | 2004-06-17 | Nec Viewtechnology Ltd | 画像伸長縮小装置、方法及びプログラム |
| US20050271284A1 (en) * | 2004-06-07 | 2005-12-08 | Chen-Chung Chen | Printing quality enhancement via graphic/text detection method in compression (JPEG) image |
| CN102281446A (zh) * | 2011-09-20 | 2011-12-14 | 西南交通大学 | 一种分布式视频编码中基于视觉感知特性的量化方法 |
| CN105144681A (zh) * | 2013-03-15 | 2015-12-09 | 三星电子株式会社 | 用频率提高来创建图像中的细节 |
| CN103745443A (zh) * | 2014-01-10 | 2014-04-23 | 北京优纳科技有限公司 | 提高图像质量的方法和设备 |
| CN104883579A (zh) * | 2015-06-08 | 2015-09-02 | 电子科技大学 | 一种基于空-频域的联合视频图像的上采样方法 |
| CN106612436A (zh) * | 2016-01-28 | 2017-05-03 | 四川用联信息技术有限公司 | 一种基于dct变换下的视觉感知修正图像压缩方法 |
| CN111225216A (zh) * | 2020-01-10 | 2020-06-02 | Tcl华星光电技术有限公司 | 显示器补偿表压缩方法、装置、系统及显示器 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zhang et al. | CONCOLOR: Constrained non-convex low-rank model for image deblocking | |
| Zhang et al. | DMCNN: Dual-domain multi-scale convolutional neural network for compression artifacts removal | |
| Zhang et al. | Compression artifact reduction by overlapped-block transform coefficient estimation with block similarity | |
| JP4920599B2 (ja) | ハイブリッドビデオ圧縮の量子化雑音除去用の非線形ループ内デノイズフィルタ | |
| US7570832B2 (en) | Image clean-up and pre-coding | |
| Liu et al. | Data-driven soft decoding of compressed images in dual transform-pixel domain | |
| WO2021169408A1 (zh) | 图像处理方法、装置、电子设备及存储介质 | |
| US20030035586A1 (en) | Decoding compressed image data | |
| Zhao et al. | Learning a virtual codec based on deep convolutional neural network to compress image | |
| CN111602402B (zh) | 一种产生滤波重构帧的方法及装置 | |
| EP2145476B1 (en) | Image compression and decompression using the pixon method | |
| CN113068034B (zh) | 视频编码方法及装置、编码器、设备、存储介质 | |
| EP3203438A1 (en) | Method and apparatus for locally sharpening a video image using a spatial indication of blurring | |
| CN115409716B (zh) | 视频处理方法、装置、存储介质及设备 | |
| CN112868228A (zh) | 自适应dct锐化器 | |
| Song et al. | Compressed image restoration via artifacts-free PCA basis learning and adaptive sparse modeling | |
| Zhang et al. | Image deblocking using group-based sparse representation and quantization constraint prior | |
| CN110378860A (zh) | 修复视频的方法、装置、计算机设备和存储介质 | |
| WO2022205094A1 (zh) | 数据处理方法、数据传输系统、设备及存储介质 | |
| CN116847087A (zh) | 视频处理方法、装置、存储介质及电子设备 | |
| CN110730347A (zh) | 图像压缩方法、装置及电子设备 | |
| Najgebauer et al. | Fully convolutional network for removing dct artefacts from images | |
| Jiang et al. | Oriented total variation l1/2 regularization | |
| US8768089B2 (en) | System and method for filtering an image | |
| CN116760983B (zh) | 用于视频编码的环路滤波方法及装置 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21933765 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 21933765 Country of ref document: EP Kind code of ref document: A1 |