CN116668691A - Picture compression transmission method and device and terminal equipment - Google Patents

Picture compression transmission method and device and terminal equipment Download PDF

Info

Publication number
CN116668691A
CN116668691A CN202310479114.8A CN202310479114A CN116668691A CN 116668691 A CN116668691 A CN 116668691A CN 202310479114 A CN202310479114 A CN 202310479114A CN 116668691 A CN116668691 A CN 116668691A
Authority
CN
China
Prior art keywords
image data
sub
target
encoder
determining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310479114.8A
Other languages
Chinese (zh)
Inventor
王荣刚
徐陈缘
高文
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Peking University Shenzhen Graduate School
Original Assignee
Peking University Shenzhen Graduate School
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Peking University Shenzhen Graduate School filed Critical Peking University Shenzhen Graduate School
Priority to CN202310479114.8A priority Critical patent/CN116668691A/en
Publication of CN116668691A publication Critical patent/CN116668691A/en
Pending legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/119Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103Selection of coding mode or of prediction mode

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The application is applicable to the technical field of picture compression, and provides a picture compression transmission method, a device and terminal equipment, wherein the method comprises the following steps: the method comprises the steps of obtaining original image data to be transmitted, dividing the original image data to obtain a plurality of sub-image data, determining a target encoder corresponding to each sub-image data according to a pre-trained classification model, processing the corresponding sub-image data through the target encoder to obtain a bit stream, and generating a code stream of the original image according to bit streams of all the sub-image data. According to the application, the target encoder corresponding to the content of the image data is selected from a plurality of encoders quickly and adaptively according to the image content of the image data by the pre-trained classification model, and the image data is encoded by the target encoder, so that the encoding and decoding rates of the image data can be improved under the condition of ensuring the quality of the reconstructed image.

Description

Picture compression transmission method and device and terminal equipment
Technical Field
The application belongs to the technical field of picture compression, and particularly relates to a picture compression transmission method, a picture compression transmission device and terminal equipment.
Background
In the technical field of data transmission, in order to improve data transmission efficiency and accuracy, data is compressed and then transmitted under normal conditions. Among them, image compression processing is an important component of signal processing and transmission.
The related image compression method based on deep learning generally adopts a single codec structure, has higher rate distortion performance, but has too long encoding and decoding time, so that the transmission efficiency is affected, and the real-time requirement in the actual application scene can not be met.
Disclosure of Invention
The embodiment of the application provides a picture compression transmission method, a picture compression transmission device and terminal equipment, which can solve the problem that the encoding and decoding speeds of the related picture compression transmission method are not high.
In a first aspect, an embodiment of the present application provides a method for compressing and transmitting a picture, including:
acquiring original image data to be transmitted;
dividing the original image data to obtain a plurality of sub-image data;
determining a target encoder corresponding to each sub-image data according to a pre-trained classification model;
processing the corresponding sub-image data by the target encoder to obtain a bit stream;
and generating a code stream of the original image according to the bit streams of all the sub-image data.
According to the embodiment, the target encoder corresponding to the content of the image data is selected from a plurality of encoders quickly and adaptively according to the image content of the image data through the pre-trained classification model, and the image data is encoded through the target encoder, so that the encoding and decoding rates of the image data can be improved under the condition that the quality of a reconstructed image is ensured.
In one embodiment, said determining a target encoder corresponding to each of said sub-image data according to a pre-trained classification model comprises:
respectively determining each piece of sub-image data as target sub-image data;
inputting the target sub-image data into the pre-trained classification model to obtain a class confidence coefficient set of the target sub-image data; the class confidence coefficient set comprises matching probabilities between the target sub-image data and each encoder respectively;
and determining a target encoder corresponding to the target sub-image data according to the category confidence coefficient set.
In one embodiment, the determining a target encoder corresponding to the target sub-image data according to the set of category confidence levels includes:
selecting the matching probability with the largest numerical value in the category confidence coefficient set as a target confidence coefficient;
And determining the encoder corresponding to the target confidence as a target encoder.
In one embodiment, the processing, by the target encoder, the corresponding sub-image data to obtain a bit stream further includes:
extracting the characteristics of the target sub-image data through the target encoder to obtain target characteristic sub-image data;
carrying out quantization processing on the target characteristic sub-image data to obtain target quantized sub-image data;
and carrying out entropy coding processing on the target quantized sub-image data to obtain a corresponding bit stream.
In one embodiment, after determining the target encoder corresponding to each of the sub-image data according to the pre-trained classification model, the method further includes:
determining an index code of each of the target sub-image data, and an encoder ID of a corresponding target encoder;
and establishing an association relationship between the index codes and the corresponding encoder IDs, and generating a corresponding preset relationship table.
In a second aspect, an embodiment of the present application provides a picture compression transmission apparatus, including:
acquiring a code stream of original image data to be decoded;
decoding the code stream to obtain a characteristic sub-image set; the feature sub-image set includes a plurality of feature sub-image data;
Determining a target decoder corresponding to each of the characteristic sub-image data;
analyzing the corresponding characteristic sub-image data according to the target decoder to obtain corresponding sub-image data;
and splicing all the sub-image data to obtain the original image data.
In one embodiment, the determining a target decoder corresponding to each of the feature sub-image data includes:
determining a target encoder corresponding to each characteristic sub-image data according to the code stream; the target encoder is obtained by screening after processing sub-image data through a pre-trained classification model; the sub-image data are image data which are in one-to-one correspondence with the characteristic sub-image data and are not subjected to characteristic extraction processing;
and determining a target decoder corresponding to each characteristic sub-image data according to the target encoder.
In one embodiment, the determining, according to the target encoder, a target decoder corresponding to each of the characteristic sub-image data includes:
identifying an index encoding for each of the feature sub-image data;
determining a target encoder associated with each index code according to a preset relation table; wherein the preset relation table comprises index codes of a plurality of characteristic sub-images and encoder IDs associated with each index code;
And determining a decoder corresponding to the target encoder as a target decoder.
According to the embodiment, the target decoder corresponding to each characteristic sub-image is respectively determined, and the corresponding characteristic sub-images are decoded according to the target decoder, so that the corresponding decoding operation is finished for different image contents through different decoders, and the decoding efficiency of the image data is further improved.
In a third aspect, an embodiment of the present application provides a picture compression transmission apparatus, including:
the data receiving module is used for acquiring original image data to be transmitted;
the image segmentation module is used for segmenting the original image data to obtain a plurality of sub-image data;
the computing module is used for determining a target encoder corresponding to each piece of sub-image data according to the pre-trained classification model;
the data processing module is used for processing the corresponding sub-image data through the target encoder to obtain a bit stream;
and the transmission module is used for generating the code stream of the original image according to the bit streams of all the sub-image data.
In one embodiment, a computing module includes:
the target determining submodule is used for respectively determining each piece of sub-image data as target sub-image data;
The model processing sub-module is used for inputting the target sub-image data into the pre-trained classification model to obtain a class confidence coefficient set of the target sub-image data; the class confidence coefficient set comprises matching probabilities between the target sub-image data and each encoder respectively;
and the encoder screening submodule is used for determining a target encoder corresponding to the target sub-image data according to the category confidence coefficient set.
In one embodiment, the encoder screening sub-module includes:
the selecting unit is used for selecting the matching probability with the largest numerical value in the category confidence coefficient set as the target confidence coefficient;
and the encoder screening unit is used for determining the encoder corresponding to the target confidence coefficient as a target encoder.
In one embodiment, the data processing module further comprises:
the feature extraction sub-module is used for carrying out feature extraction on the target sub-image data through the target encoder to obtain target feature sub-image data;
the quantization processing sub-module is used for carrying out quantization processing on the target characteristic sub-image data to obtain target quantized sub-image data;
and the entropy coding processing sub-module is used for performing entropy coding processing on the target quantized sub-image data to obtain a corresponding bit stream.
In one embodiment, the image compression transmission device further includes:
the coding determining module is used for determining index coding of each target sub-image data and coder ID of a corresponding target coder;
and the relation establishing module is used for establishing the association relation between the index codes and the corresponding encoder IDs and generating a corresponding preset relation table.
In a fourth aspect, an embodiment of the present application provides a picture compression transmission apparatus, including:
the code stream receiving module is used for obtaining the code stream of the original image data to be decoded;
the decoding module is used for decoding the code stream to obtain a characteristic sub-image set; the feature sub-image set includes a plurality of feature sub-image data;
a decoder determining module for determining a target decoder corresponding to each of the characteristic sub-image data;
the data conversion module is used for analyzing the corresponding characteristic sub-image data according to the target decoder to obtain corresponding sub-image data;
and the image splicing module is used for splicing all the sub-image data to obtain the original image data.
In one embodiment, the decoder determination module includes:
An encoder determining unit configured to determine a target encoder corresponding to each of the characteristic sub-image data according to the code stream; the target encoder is obtained by screening after processing sub-image data through a pre-trained classification model; the sub-image data are image data which are in one-to-one correspondence with the characteristic sub-image data and are not subjected to characteristic extraction processing;
and the decoder determining unit is used for determining a target decoder corresponding to each characteristic sub-image data according to the target encoder.
In one embodiment, the decoder determining unit includes:
an identification subunit for identifying an index encoding of each of the feature sub-image data;
an encoder determining subunit, configured to determine a target encoder associated with each index code according to a preset relationship table; wherein the preset relation table comprises index codes of a plurality of characteristic sub-images and encoder IDs associated with each index code;
and the decoder determining subunit is used for determining the decoder corresponding to the target encoder as the target decoder.
In a fifth aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the picture compression transmission method according to any one of the first or second aspects when the processor executes the computer program.
In a sixth aspect, an embodiment of the present application provides a computer readable storage medium storing a computer program, which when executed by a processor implements the picture compression transmission method according to any one of the first or second aspects above.
In a seventh aspect, an embodiment of the present application provides a computer program product, which when run on a terminal device, causes the terminal device to perform the picture compression transmission method of any one of the first or second aspects above.
It will be appreciated that the advantages of the third to seventh aspects may be found in the relevant description of the first or second aspects, and are not described here again.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments or the description of the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic diagram of a picture compression transmission system according to an embodiment of the present application;
fig. 2 is a flowchart of a picture compression transmission method according to an embodiment of the present application;
fig. 3 is a flowchart of step S103 of a picture compression transmission method according to an embodiment of the present application;
fig. 4 is another flow chart of a picture compression transmission method according to a second embodiment of the present application;
fig. 5 is a schematic structural diagram of a picture compression transmission device according to a third embodiment of the present application;
fig. 6 is another schematic structural diagram of a picture compression transmission device according to a fourth embodiment of the present application;
fig. 7 is a schematic structural diagram of a terminal device according to a fifth embodiment of the present application.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
As used in the present description and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".
Furthermore, the terms "first," "second," "third," and the like in the description of the present specification and in the appended claims, are used for distinguishing between descriptions and not necessarily for indicating or implying a relative importance.
Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.
The picture compression transmission method provided by the embodiment of the application can be applied to terminal equipment such as mobile phones, tablet computers, notebook computers, ultra-Mobile Personal Computer (UMPC) and the like, and the embodiment of the application does not limit the specific type of the terminal equipment.
In recent years, although image compression methods have been developed to some extent, the compression process of the related methods is complex and the data volume is large, and the encoding and decoding efficiency of the image is not high. In order to solve the problem, the application provides a picture compression transmission method, a picture compression transmission device, a terminal device and a computer readable storage medium, which can divide original image data to obtain a plurality of sub-image data when the original image data is obtained, determine a target encoder corresponding to each sub-image data according to a pre-trained classification model, and process the sub-image data corresponding to the target encoder to generate a code stream of the original image data; when the code stream of the original image data to be decoded is acquired, the code stream is decoded to obtain a characteristic sub-image set, a target decoder corresponding to each characteristic sub-image data is determined, the corresponding characteristic sub-image data is analyzed according to the target decoder to obtain corresponding sub-image data, and all the sub-image data are spliced to obtain the original image data. The encoding and decoding rate of the image data is improved while ensuring the quality of the reconstructed image.
In order to realize the technical scheme provided by the application, a picture compression transmission system can be constructed first. Referring to fig. 1, the picture compression transmission system is composed of more than one terminal device (only 4 are shown in fig. 1, such as terminal device a, terminal device b, terminal device c, and terminal device d), and the terminal devices are in communication connection with each other.
The terminal equipment is equipment capable of supporting multiple encoder pairs, identifying the category confidence of the image data through a pre-trained classification model, selecting a target encoder-decoder pair, and performing encoding, decoding and transmission operations on the image data according to the target encoder-decoder pair.
When the terminal equipment acquires the original image data, the original image data is segmented to obtain a plurality of sub-image data, a target encoder corresponding to each sub-image data is determined according to a pre-trained classification model, the sub-image data corresponding to the target encoder is processed, and a code stream is generated.
When receiving the code stream of the original image data to be decoded, the terminal equipment decodes the received code stream to obtain a characteristic sub-image set, analyzes the characteristic sub-image data according to a target decoder corresponding to the characteristic sub-image data in the characteristic sub-image set to obtain corresponding sub-image data, and splices the sub-image data to obtain the original image data.
The method and the device realize the rapid and self-adaptive selection of the corresponding target codec pair according to the content of the sub-image data through the classification model, and perform the encoding and decoding operations on the corresponding sub-image data according to the target codec pair, thereby improving the encoding and decoding rates of the image data under the condition of ensuring the quality of the reconstructed image.
In order to illustrate the technical scheme provided by the application, the following description is made by specific embodiments.
Example 1
Fig. 2 shows a schematic flowchart of a picture compression transmission method provided by the present application, which can be applied to the above-mentioned notebook computer by way of example and not limitation.
S101, acquiring original image data to be transmitted.
Specifically, a storage position of original image data is determined, and corresponding original image data is acquired.
S102, dividing the original image data to obtain a plurality of sub-image data.
Specifically, the original image data is segmented according to a preset segmentation method or a user-set segmentation method, so as to obtain a plurality of sub-image data. The preset dividing method can be specifically set according to actual situations, for example: the preset division method is a method of dividing according to a preset division size, or the preset division method is a method of dividing according to a preset division number.
The preset segmentation size and the preset segmentation number can be specifically set according to user requirements or actual requirements. For example, the preset division size is set to 256×256px, and the original image is required to be divided into a plurality of sub-image data according to the 256×256px. For example, the preset division number is set to 4*4. Correspondingly, the original image needs to be divided into 4*4 sub-image data with the same size.
By way of example and not limitation, when the size of the original image cannot be divided by the preset division size or the preset division number, the size of the boundary of the original image may be added by a padding (padding) method, so that the original image can be divided by the preset division size or the preset division number to obtain a plurality of sub-image data. For example, the original image is 1000×1024px, and the preset division size is 256×256px. Correspondingly, the original image is required to be processed into image data with 1024 x 1024px size through padding, so as to obtain corresponding 4*4 sub-image data. Or, the original image is 999 x 999px, and the preset division number is 4*4. Correspondingly, the original image is required to be processed into image data with 1000 x 1000px size through padding, so as to obtain corresponding 4*4 sub-image data.
S103, determining a target encoder corresponding to each piece of sub-image data according to the pre-trained classification model.
Specifically, a plurality of image data are pre-trained on the classification model (classification token, cls) to obtain a pre-trained classification model, and each sub-image data is respectively input into the pre-trained classification model to obtain a target encoder corresponding to each sub-image data output by the pre-trained classification model.
In one embodiment, the target encoder is the encoder with the highest probability of matching with the sub-image data calculated by the pre-trained classification model.
S104, processing the corresponding sub-image data through the target encoder to obtain a bit stream.
Specifically, each target encoder encodes the corresponding sub-image data to obtain a bit stream of the sub-image data, thereby obtaining a bit stream of each sub-image data in the original image data.
S105, generating a code stream of the original image according to bit streams of all the sub-image data.
Specifically, the code stream for generating the original image data is written according to the bit streams of all the sub-image data in the original image data.
As shown in fig. 3, in one embodiment, the step S103 of determining a target encoder corresponding to each of the sub-image data according to a pre-trained classification model includes:
s1031, respectively determining each piece of sub-image data as target sub-image data;
s1032, inputting the target sub-image data into the pre-trained classification model to obtain a class confidence coefficient set of the target sub-image data; the class confidence coefficient set comprises matching probabilities between the target sub-image data and each encoder respectively;
s1033, determining a target encoder corresponding to the target sub-image data according to the category confidence coefficient set.
Specifically, each piece of sub-image data is determined to be the target sub-image data, and each piece of target sub-image data is input into a pre-trained classification model in parallel to be processed, so that a class confidence coefficient set of the target sub-image data calculated by each piece of target sub-image data is obtained. The class confidence is the matching probability between the target sub-image data and the encoders, and the corresponding class confidence set comprises the matching probability (namely a plurality of class confidence) between the target sub-image data and each encoder. And a target encoder corresponding to the target sub-image data is determined according to the class confidence screening.
In the encoding process of the conventional image compression method, an image is required to be divided into encoding units of different sizes for encoding. The different coding units are in turn divided into one or more prediction units and also into one or more transform units. In the prediction unit, prediction is performed using an intra mode or an inter mode to obtain a corresponding prediction block. In the transformation unit, the residual image block is obtained by subtracting the corresponding prediction block from the original image block. And transforming and quantizing the residual image block to obtain a quantized block, wherein the code stream is required to be written with the partition information of the prediction unit and the transformation unit, the mode adopted by the prediction and the quantized block. In the decoding process of the traditional image compression method, all the coding information of each coding unit, the dividing information of a prediction unit and a transformation unit, the mode adopted by prediction and a quantization block are obtained from a code stream in a resolving mode. And carrying out corresponding intra-frame or inter-frame prediction according to the information to obtain a predicted image block corresponding to the prediction unit. And performing inverse quantization and inverse transformation on the quantized block to obtain a corresponding residual image block, and adding the residual image block and the corresponding predicted image block to obtain a reconstructed image block. The method has complex encoding and decoding processes, large transmission data quantity and poor rate distortion performance. In contrast, related image compression methods based on deep learning have been greatly improved in terms of rate-distortion performance. However, the above-described image compression method based on the deep learning is generally a single codec structure in which only one and only one pair of encoder and decoder are used in the codec process. The encoding and decoding time required for processing the original image data is too long to meet the real-time requirement in the actual application scene. According to the method, each piece of sub-image data is determined to be the target sub-image data, the matching probability between the target sub-image data and each encoder is calculated according to the pre-trained classification model, so that the target encoder meeting the requirements is screened, the rate distortion performance and the image data quality are guaranteed through the deep learning model, meanwhile, the encoder with the higher encoding speed is selected correspondingly according to the image content classification of the sub-image data, and the encoding speed of the sub-image data is improved.
In one embodiment, the determining a target encoder corresponding to the target sub-image data according to the set of category confidence levels includes:
selecting the matching probability with the largest numerical value in the category confidence coefficient set as a target confidence coefficient;
and determining the encoder corresponding to the target confidence as a target encoder.
Specifically, a plurality of matching probabilities in a category confidence coefficient set of target sub-image data are arranged in a sequence from large to small (or from small to large), the matching probability meeting a preset condition in a queue is selected as target confidence coefficient, and an encoder corresponding to the target confidence coefficient is used as a target encoder corresponding to the target sub-image data. The preset condition is that the matching probability with the largest numerical value is selected.
And selecting the target confidence coefficient meeting the preset condition from the matching probabilities between the target sub-image data and each encoder respectively so as to determine the target encoder, realizing adaptability, correspondingly selecting the target encoder with optimal coding efficiency according to the content of the target sub-image data, and improving the coding efficiency of the sub-image data aiming at different contents, thereby improving the coding rate aiming at the original image data.
In one embodiment, the processing, by the target encoder, the corresponding sub-image data to obtain a bit stream further includes:
extracting the characteristics of the target sub-image data through the target encoder to obtain target characteristic sub-image data;
carrying out quantization processing on the target characteristic sub-image data to obtain target quantized sub-image data;
and carrying out entropy coding processing on the target quantized sub-image data to obtain a corresponding bit stream.
Specifically, shallow layer feature extraction is performed on target sub-image data of the target encoder to obtain target feature sub-image data, then quantization processing is performed on the feature map to obtain target quantized sub-image data, and the target quantized sub-image data is processed by using an arithmetic coding isentropic coding mode to obtain a corresponding bit stream.
In one embodiment, after determining the target encoder corresponding to each of the sub-image data according to the pre-trained classification model, the method further includes:
determining an index code of each of the target sub-image data, and an encoder ID of a corresponding target encoder;
and establishing an association relationship between the index codes and the corresponding encoder IDs, and generating a corresponding preset relationship table.
Specifically, after the original image data is divided, a corresponding index code is assigned to each target sub-image data, and after a target encoder corresponding to each target sub-image data is determined by a pre-trained classification model, an encoder ID of the target encoder of each target sub-image data is determined. And establishing an association relationship between the index codes of each target sub-image data and the corresponding encoder IDs, so that the receiving end can determine the encoder ID bound with each target sub-image data through the index codes of each target sub-image data after receiving the code stream, thereby determining the corresponding target encoder.
Through pre-establishing the association relation between the index codes of each target sub-image data and the corresponding encoder IDs, the receiving end can conveniently and quickly identify the target encoder corresponding to each sub-image data through the association relation, so that the corresponding target decoder is determined, the sub-image data is encoded and decoded through the selected target encoder pair with the optimal encoding and decoding efficiency, and the encoding and decoding rate of the original image data is correspondingly improved.
In one possible implementation, the method for determining the target encoder corresponding to the characteristic sub-image data includes: the size of each characteristic sub-image data is determined, and a corresponding target encoder is determined according to the size of the characteristic sub-image data.
In one possible implementation, generating a bitstream of the original image from bitstreams of all sub-image data includes: and adding a corresponding 'target encoder ID' tag for the bit stream of each sub-image data, and writing the bit stream of each sub-image data after the tag addition into a code stream for generating a corresponding original image.
According to the embodiment, the target encoder corresponding to the content of the image data is selected from a plurality of encoders quickly and adaptively according to the image content of the image data through the pre-trained classification model, and the image data is encoded through the target encoder, so that the encoding and decoding rates of the image data can be improved under the condition that the quality of a reconstructed image is ensured.
Example two
Fig. 4 shows another schematic flowchart of a picture compression transmission method provided by the present application, which can be applied to the above-mentioned notebook computer by way of example and not limitation.
S201, acquiring a code stream of original image data to be decoded.
Specifically, a code stream of original image data to be decoded is acquired.
In one embodiment, the code stream carries an association table (for storing the index encoding of each sub-image data and the encoder ID of the target encoder associated therewith).
S202, decoding the code stream to obtain a characteristic sub-image set; the feature sub-image set includes a plurality of feature sub-image data.
Specifically, in the decoding stage, after receiving the code stream of the original image, the current terminal device parses the code stream by an entropy decoding mode corresponding to the entropy encoding mode used in the encoding process, and restores the reconstructed characteristic sub-image data (the characteristics of the preliminary atomic image data).
S203, determining a target decoder corresponding to each characteristic sub-image data.
Specifically, the target encoder corresponding to each feature sub-image data is first determined, and then the decoder corresponding to each target encoder is respectively determined as the target decoder corresponding to each feature sub-image data by the correspondence between the encoder pairs.
S204, analyzing the corresponding characteristic sub-image data according to the target decoder to obtain corresponding sub-image data;
and S205, splicing all the sub-image data to obtain the original image data.
Specifically, the corresponding characteristic sub-image data is analyzed through each target decoder to obtain the corresponding sub-image data (the characteristic sub-image data is restored to the pixel domain image data), so that each sub-image data of the original image data is obtained, and the sub-image data is spliced according to the index coding sequence of the sub-image data to obtain the original image data.
In one embodiment, the determining a target decoder corresponding to each of the feature sub-image data includes:
determining a target encoder corresponding to each characteristic sub-image data according to the code stream; the target encoder is obtained by screening after processing sub-image data through a pre-trained classification model; the sub-image data are image data which are in one-to-one correspondence with the characteristic sub-image data and are not subjected to characteristic extraction processing;
and determining a target decoder corresponding to each characteristic sub-image data according to the target encoder.
Specifically, the transmitting end processes the sub-image data which is not subjected to feature extraction processing, quantization, entropy coding processing and other processing operations (namely, the sub-image data corresponds to the feature sub-image data obtained after decoding by the current terminal equipment one by one) through a pre-trained classification model to obtain a bit stream, screens to obtain a target encoder corresponding to the content of the sub-image data, and writes the bit stream of the sub-image data and the encoder ID of the target encoder into the bit stream through the modes of adding a label or establishing a preset relation table and the like. Correspondingly, the current terminal equipment determines a target encoder corresponding to each characteristic sub-image data carried in the code stream by analyzing the code stream, and determines a decoder corresponding to each target encoder according to the corresponding relation between the codec pairs as a target decoder corresponding to the characteristic sub-image data.
In one embodiment, the determining, according to the target encoder, a target decoder corresponding to each of the characteristic sub-image data includes:
identifying an index encoding for each of the feature sub-image data;
determining a target encoder associated with each index code according to a preset relation table; wherein the preset relation table comprises index codes of a plurality of characteristic sub-images and encoder IDs associated with each index code;
and determining a decoder corresponding to the target encoder as a target decoder.
Specifically, the index codes of each characteristic sub-image data carried in the code stream are identified, and the encoder ID matched with the index codes of the characteristic sub-image data is searched in a preset relation table, so that the target encoder with the association relation with the index codes of the characteristic sub-image data is determined, and the decoder corresponding to each target encoder is determined as the target decoder corresponding to the characteristic sub-image data according to the corresponding relation between the encoder-decoder pairs.
By pre-establishing the association relation between the index codes of each target sub-image data and the corresponding encoder IDs, a preset relation table is obtained, and the target decoder corresponding to the target encoder of each sub-image data can be quickly identified according to the association relation in the preset relation table, so that the efficiency of decoding the sub-image data through the decoder is improved.
In one possible implementation, since different codec pairs differ in size of the processed image data, the corresponding method of determining the target decoder corresponding to the feature sub-image data includes: the size of each characteristic sub-image data is directly determined, and a corresponding target decoder is determined according to the size of the characteristic sub-image data.
According to the embodiment, the target decoder corresponding to each characteristic sub-image is respectively determined, and the corresponding characteristic sub-images are decoded according to the target decoder, so that the corresponding decoding operation is finished for different image contents through different decoders, and the decoding rate of the image data is further improved.
It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present application.
Example III
Corresponding to the picture compression transmission method described in the first embodiment, fig. 5 shows a block diagram of a picture compression transmission device according to an embodiment of the present application, and for convenience of explanation, only the parts related to the embodiment of the present application are shown.
Referring to fig. 5, the picture compression transmission apparatus 100 includes:
a data receiving module 101, configured to acquire original image data to be transmitted;
an image segmentation module 102, configured to segment the original image data to obtain a plurality of sub-image data;
a calculation module 103, configured to determine a target encoder corresponding to each sub-image data according to a pre-trained classification model;
a data processing module 104, configured to process the corresponding sub-image data by using the target encoder to obtain a bit stream;
and the transmission module 105 is used for generating a code stream of the original image according to the bit streams of all the sub-image data.
In one embodiment, a computing module includes:
the target determining submodule is used for respectively determining each piece of sub-image data as target sub-image data;
the model processing sub-module is used for inputting the target sub-image data into the pre-trained classification model to obtain a class confidence coefficient set of the target sub-image data; the class confidence coefficient set comprises matching probabilities between the target sub-image data and each encoder respectively;
and the encoder screening submodule is used for determining a target encoder corresponding to the target sub-image data according to the category confidence coefficient set.
In one embodiment, the encoder screening sub-module includes:
the selecting unit is used for selecting the matching probability with the largest numerical value in the category confidence coefficient set as the target confidence coefficient;
and the encoder screening unit is used for determining the encoder corresponding to the target confidence coefficient as a target encoder.
In one embodiment, the data processing module further comprises:
the feature extraction sub-module is used for carrying out feature extraction on the target sub-image data through the target encoder to obtain target feature sub-image data;
the quantization processing sub-module is used for carrying out quantization processing on the target characteristic sub-image data to obtain target quantized sub-image data;
and the entropy coding processing sub-module is used for performing entropy coding processing on the target quantized sub-image data to obtain a corresponding bit stream.
In one embodiment, the image compression transmission device further includes:
the coding determining module is used for determining index coding of each target sub-image data and coder ID of a corresponding target coder;
and the relation establishing module is used for establishing the association relation between the index codes and the corresponding encoder IDs and generating a corresponding preset relation table.
According to the embodiment, the target encoder corresponding to the content of the image data is selected from a plurality of encoders quickly and adaptively according to the image content of the image data through the pre-trained classification model, and the image data is encoded through the target encoder, so that the encoding and decoding rates of the image data can be improved under the condition that the quality of a reconstructed image is ensured.
Example IV
Corresponding to the picture compression transmission method described in the second embodiment, fig. 6 shows a block diagram of a picture compression transmission device according to an embodiment of the present application, and for convenience of explanation, only the parts related to the embodiment of the present application are shown.
Referring to fig. 6, the picture compression transmission apparatus 200 includes:
a code stream receiving module 201, configured to obtain a code stream of original image data to be decoded;
a decoding module 202, configured to decode the code stream to obtain a feature sub-image set; the feature sub-image set includes a plurality of feature sub-image data;
a decoder determining module 203, configured to determine a target decoder corresponding to each of the feature sub-image data;
the data conversion module 204 is configured to parse the corresponding characteristic sub-image data according to the target decoder to obtain corresponding sub-image data;
And the image stitching module 205 is configured to stitch all the sub-image data to obtain the original image data.
In one embodiment, the decoder determination module includes:
an encoder determining unit configured to determine a target encoder corresponding to each of the characteristic sub-image data according to the code stream; the target encoder is obtained by screening after processing sub-image data through a pre-trained classification model; the sub-image data are image data which are in one-to-one correspondence with the characteristic sub-image data and are not subjected to characteristic extraction processing;
and the decoder determining unit is used for determining a target decoder corresponding to each characteristic sub-image data according to the target encoder.
In one embodiment, the decoder determining unit includes:
an identification subunit for identifying an index encoding of each of the feature sub-image data;
an encoder determining subunit, configured to determine a target encoder associated with each index code according to a preset relationship table; wherein the preset relation table comprises index codes of a plurality of characteristic sub-images and encoder IDs associated with each index code;
and the decoder determining subunit is used for determining the decoder corresponding to the target encoder as the target decoder.
According to the embodiment, the target decoder corresponding to each characteristic sub-image is respectively determined, and the corresponding characteristic sub-images are decoded according to the target decoder, so that the self-adaptive decoding operation aiming at different image contents is realized, and the decoding rate of the image data is further improved.
It should be noted that, because the content of information interaction and execution process between the above devices/units is based on the same concept as the method embodiment of the present application, specific functions and technical effects thereof may be referred to in the method embodiment section, and will not be described herein.
Example five
Fig. 7 is a schematic structural diagram of a terminal device according to an embodiment of the present application. As shown in fig. 7, the terminal device 7 of this embodiment includes: at least one processor 70 (only one is shown in fig. 7), a memory 71 and a computer program 72 stored in the memory 71 and executable on the at least one processor 70, the processor 70 implementing the steps in any of the various picture compression transmission method embodiments described above when executing the computer program 72.
The terminal device 7 may be a computing device such as a desktop computer, a notebook computer, a palm computer, a cloud server, etc. The terminal device may include, but is not limited to, a processor 70, a memory 71. It will be appreciated by those skilled in the art that fig. 7 is merely an example of the terminal device 7 and is not limiting of the terminal device 7, and may include more or fewer components than shown, or may combine certain components, or different components, such as may also include input-output devices, network access devices, etc.
The processor 70 may be a central processing unit (Central Processing Unit, CPU) and the processor 70 may be other general purpose processors, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), an off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 71 may in some embodiments be an internal storage unit of the terminal device 7, such as a hard disk or a memory of the terminal device 7. The memory 71 may in other embodiments also be an external storage device of the terminal device 7, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital Card (SD), a Flash memory Card (Flash Card) or the like, which are provided on the terminal device 7. Further, the memory 71 may also include both an internal storage unit and an external storage device of the terminal device 7. The memory 71 is used for storing an operating system, application programs, boot loader (BootLoader), data, other programs, etc., such as program codes of the computer program. The memory 71 may also be used for temporarily storing data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, the specific names of the functional units and modules are only for distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working process of the units and modules in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.
The embodiment of the application also provides a network device, which comprises: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, which when executed by the processor performs the steps of any of the various method embodiments described above.
Embodiments of the present application also provide a computer readable storage medium storing a computer program which, when executed by a processor, implements steps for implementing the various method embodiments described above.
Embodiments of the present application provide a computer program product which, when run on a mobile terminal, causes the mobile terminal to perform steps that enable the implementation of the method embodiments described above.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the present application may implement all or part of the flow of the method of the above embodiments, and may be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps of each of the method embodiments described above. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code to a photographing device/terminal apparatus, recording medium, computer Memory, read-Only Memory (ROM), random access Memory (RAM, random Access Memory), electrical carrier signals, telecommunications signals, and software distribution media. Such as a U-disk, removable hard disk, magnetic or optical disk, etc. In some jurisdictions, computer readable media may not be electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus/network device and method may be implemented in other manners. For example, the apparatus/network device embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical functional division, and there may be additional divisions in actual implementation, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
The above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application, and are intended to be included in the scope of the present application.

Claims (12)

1. A picture compression transmission method, comprising:
acquiring original image data to be transmitted;
dividing the original image data to obtain a plurality of sub-image data;
determining a target encoder corresponding to each sub-image data according to a pre-trained classification model;
Processing the corresponding sub-image data by the target encoder to obtain a bit stream;
and generating a code stream of the original image according to the bit streams of all the sub-image data.
2. The picture compression transmission method according to claim 1, wherein the determining a target encoder corresponding to each of the sub-image data according to a pre-trained classification model comprises:
respectively determining each piece of sub-image data as target sub-image data;
inputting the target sub-image data into the pre-trained classification model to obtain a class confidence coefficient set of the target sub-image data; the class confidence coefficient set comprises matching probabilities between the target sub-image data and each encoder respectively;
and determining a target encoder corresponding to the target sub-image data according to the category confidence coefficient set.
3. The picture compression transmission method according to claim 2, wherein the determining a target encoder corresponding to the target sub-image data according to the category confidence set includes:
selecting the matching probability with the largest numerical value in the category confidence coefficient set as a target confidence coefficient;
And determining the encoder corresponding to the target confidence as a target encoder.
4. The picture compression transmission method according to claim 2, wherein the processing of the corresponding sub-image data by the target encoder to obtain a bitstream further comprises:
extracting the characteristics of the target sub-image data through the target encoder to obtain target characteristic sub-image data;
carrying out quantization processing on the target characteristic sub-image data to obtain target quantized sub-image data;
and carrying out entropy coding processing on the target quantized sub-image data to obtain a corresponding bit stream.
5. The picture compression transmission method according to claim 2, wherein after determining the target encoder corresponding to each of the sub-image data according to the pre-trained classification model, further comprising:
determining an index code of each of the target sub-image data, and an encoder ID of a corresponding target encoder;
and establishing an association relationship between the index codes and the corresponding encoder IDs, and generating a corresponding preset relationship table.
6. A picture compression transmission method, comprising:
acquiring a code stream of original image data to be decoded;
Decoding the code stream to obtain a characteristic sub-image set; the feature sub-image set includes a plurality of feature sub-image data;
determining a target decoder corresponding to each of the characteristic sub-image data;
analyzing the corresponding characteristic sub-image data according to the target decoder to obtain corresponding sub-image data;
and splicing all the sub-image data to obtain the original image data.
7. The picture compression transmission method according to claim 6, wherein the determining a target decoder corresponding to each of the characteristic sub-image data comprises:
determining a target encoder corresponding to each characteristic sub-image data according to the code stream; the target encoder is obtained by screening after processing sub-image data through a pre-trained classification model; the sub-image data are image data which are in one-to-one correspondence with the characteristic sub-image data and are not subjected to characteristic extraction processing;
and determining a target decoder corresponding to each characteristic sub-image data according to the target encoder.
8. The picture compression transmission method according to claim 7, wherein the determining a target decoder corresponding to each of the characteristic sub-image data according to the target encoder comprises:
Identifying an index encoding for each of the feature sub-image data;
determining a target encoder associated with each index code according to a preset relation table; wherein the preset relation table comprises index codes of a plurality of characteristic sub-images and encoder IDs associated with each index code;
and determining a decoder corresponding to the target encoder as a target decoder.
9. A picture compression transmission apparatus, comprising:
the data receiving module is used for acquiring original image data to be transmitted;
the image segmentation module is used for segmenting the original image data to obtain a plurality of sub-image data;
the computing module is used for determining a target encoder corresponding to each piece of sub-image data according to the pre-trained classification model;
the data processing module is used for processing the corresponding sub-image data through the target encoder to obtain a bit stream;
and the transmission module is used for generating the code stream of the original image according to the bit streams of all the sub-image data.
10. A picture compression transmission apparatus, the apparatus comprising:
the code stream receiving module is used for obtaining the code stream of the original image data to be decoded;
The decoding module is used for decoding the code stream to obtain a characteristic sub-image set; the feature sub-image set includes a plurality of feature sub-image data;
a decoder determining module for determining a target decoder corresponding to each of the characteristic sub-image data;
the data conversion module is used for analyzing the corresponding characteristic sub-image data according to the target decoder to obtain corresponding sub-image data;
and the image splicing module is used for splicing all the sub-image data to obtain the original image data.
11. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the method according to any one of claims 1 to 5 or any one of claims 6 to 8 when executing the computer program.
12. A computer readable storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the method of any one of claims 1 to 5 or any one of claims 6 to 8.
CN202310479114.8A 2023-04-26 2023-04-26 Picture compression transmission method and device and terminal equipment Pending CN116668691A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310479114.8A CN116668691A (en) 2023-04-26 2023-04-26 Picture compression transmission method and device and terminal equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310479114.8A CN116668691A (en) 2023-04-26 2023-04-26 Picture compression transmission method and device and terminal equipment

Publications (1)

Publication Number Publication Date
CN116668691A true CN116668691A (en) 2023-08-29

Family

ID=87710264

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310479114.8A Pending CN116668691A (en) 2023-04-26 2023-04-26 Picture compression transmission method and device and terminal equipment

Country Status (1)

Country Link
CN (1) CN116668691A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117456016A (en) * 2023-10-27 2024-01-26 四川中屹互联信息技术有限公司 Canvas-based image block compression and reconstruction method, system and application

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117456016A (en) * 2023-10-27 2024-01-26 四川中屹互联信息技术有限公司 Canvas-based image block compression and reconstruction method, system and application

Similar Documents

Publication Publication Date Title
JP2020174374A (en) Digital image recompression
US11257252B2 (en) Image coding apparatus, probability model generating apparatus and image compression system
US10764588B2 (en) Deep quality enhancement of adaptive downscaled coding for image compression
CN111246209A (en) Adaptive encoding method, apparatus, electronic device, and computer storage medium
CN113473126A (en) Video stream processing method and device, electronic equipment and computer readable medium
CN116668691A (en) Picture compression transmission method and device and terminal equipment
CN113965751A (en) Screen content coding method, device, equipment and storage medium
CN111432213B (en) Method and apparatus for tile data size coding for video and image compression
CN113395523B (en) Image decoding method, device, equipment and storage medium based on parallel threads
US11051080B2 (en) Method for improving video resolution and video quality, encoder, and decoder
CN108765503B (en) Skin color detection method, device and terminal
CN116566397A (en) Encoding method, decoding method, encoder, decoder, electronic device, and storage medium
CN114495081A (en) Text recognition method and device, readable medium and electronic equipment
CN112954357A (en) Dynamic efficient self-adaptive video stream intelligent coding and decoding method and system
CN113473154B (en) Video encoding method, video decoding method, video encoding device, video decoding device and storage medium
WO2023082773A1 (en) Video encoding method and apparatus, video decoding method and apparatus, and device, storage medium and computer program
CN115462090B (en) Advanced video coding apparatus, encoder, method, storage medium, and device
US11729395B2 (en) Methods and devices for extracting motion vector data from compressed video data
CN113727112B (en) Video transcoding method, device, electronic equipment and medium
WO2024188073A1 (en) Encoding method, decoding method and electronic device
US20240129537A1 (en) Method and apparatus for signaling cmaf switching sets in isobmff
CN117880593A (en) Network video stream real-time display system and display method based on RK3588 processing chip
WO2023137003A1 (en) Systems and methods for privacy protection in video communication systems
KR20160105848A (en) Method and apparatus for performing an arithmetic coding for data symbols
CN118696315A (en) System and method for privacy protection in video communication systems

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination