CN110875906A - Image recovery method and encoding end - Google Patents

Image recovery method and encoding end Download PDF

Info

Publication number
CN110875906A
CN110875906A CN201811012695.XA CN201811012695A CN110875906A CN 110875906 A CN110875906 A CN 110875906A CN 201811012695 A CN201811012695 A CN 201811012695A CN 110875906 A CN110875906 A CN 110875906A
Authority
CN
China
Prior art keywords
frame
image quality
reconstructed image
inter
determined
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201811012695.XA
Other languages
Chinese (zh)
Inventor
朱存望
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ZTE Corp
Original Assignee
ZTE Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ZTE Corp filed Critical ZTE Corp
Priority to CN201811012695.XA priority Critical patent/CN110875906A/en
Publication of CN110875906A publication Critical patent/CN110875906A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/80—Responding to QoS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/184—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being bits, e.g. of the compressed video stream

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The embodiment of the invention discloses an image recovery method and a device, wherein the method comprises the following steps: when the image quality needs to be adjusted, obtaining a reference candidate frame according to the inter-frame similarity and the reconstructed image quality; and allocating bits for the target frame according to the reference candidate frame, and performing video coding on the target frame based on the bits allocated for the target frame. Therefore, the reference candidate frame is adaptively adjusted through the inter-frame similarity and the reconstructed image quality, so that the bit allocation of the target frame is adjusted, the network packet loss of the image frame is reduced, the packet loss resistance of the image is effectively improved, and the better image quality and fluency are obtained while the image is quickly recovered at the decoding end.

Description

Image recovery method and encoding end
Technical Field
The present invention relates to, but not limited to, video encoding and decoding technologies, and in particular, to an image restoration method and an encoding end.
Background
Image restoration is a key technology for real-time video coding and decoding. During transmission under the limited bandwidth constraint, since the transmission state of the network is dynamically changed, bandwidth overflow easily occurs to cause data loss.
Fig. 1 is a schematic structural diagram of a related art video codec system. As shown in fig. 1, the video coding and decoding system includes an encoding end and a decoding end, wherein the encoding end includes a video coding device, and the decoding end includes a video decoding device. The video encoding device is used for encoding input video data, the encoded video sequence is transmitted to a decoding end through a video transmission channel, and the video decoding device is used for decoding the encoded video data for playing. In order to avoid the loss of image quality caused by the fluctuation of network transmission rate, there have been related technical researches from three aspects of source, channel and sink, wherein the source mainly relates to the layered coding technology of the coding side, such as: SVC (Scaled Video Coding, scalable Video Coding), and the like; the channel mainly involves various retransmission mechanisms, such as: FEC (Forward error correction); the signal sink mainly relates to the error concealment technology, and belongs to the technology of no midlife. Under a signal source, channel and signal sink image recovery mechanism, when an image still can not be recovered, the adopted technology is to apply a space domain I frame, and the space domain I frame is not referred to a previous image frame during encoding and decoding, so that the loss of the current image frame can not be caused even if the previous image frame is lost; however, the image code of the airspace I frame is relatively large, and under the severe network environment with limited bandwidth, the larger the image code is, the vicious circle of network transmission is naturally caused, thereby accelerating the loss of image frames under the limited bandwidth, and further failing to ensure the ideal image quality.
In the related art, no reasonable image restoration mechanism exists, and ideal image quality is difficult to guarantee.
Disclosure of Invention
In view of this, an embodiment of the present invention provides an image recovery method, including:
when the image quality needs to be adjusted, obtaining a reference candidate frame according to the inter-frame similarity and the reconstructed image quality;
and allocating bits for the target frame according to the reference candidate frame, and performing video coding on the target frame based on the bits allocated for the target frame.
The embodiment of the present invention further provides an encoding end, including:
the computing unit is used for obtaining a reference candidate frame according to the inter-frame similarity and the reconstructed image quality when the image quality needs to be adjusted;
a distribution unit for distributing bits for the target frame according to the reference candidate frame;
an encoding unit configured to perform video encoding on the target frame based on the bits allocated for the target frame.
The embodiment of the present invention further provides an encoding end, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and when the computer program is executed by the processor, the image restoration method described above is implemented.
An embodiment of the present invention further provides a computer-readable storage medium, where an information processing program is stored on the computer-readable storage medium, and when the information processing program is executed by a processor, the steps of the image restoration method are implemented.
Compared with the related art, the embodiment of the invention provides an image recovery method and a coding end, wherein the method comprises the following steps: when the image quality needs to be adjusted, obtaining a reference candidate frame according to the inter-frame similarity and the reconstructed image quality; and allocating bits for the target frame according to the reference candidate frame, and performing video coding on the target frame based on the bits allocated for the target frame. Therefore, the reference candidate frame is adaptively adjusted through the inter-frame similarity and the reconstructed image quality, so that the bit allocation of the target frame is adjusted, the network packet loss of the image frame is reduced, the packet loss resistance of the image is effectively improved, and the better image quality and fluency are obtained while the image is quickly recovered at the decoding end.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the example serve to explain the principles of the invention and not to limit the invention.
FIG. 1 is a schematic diagram of a related art video codec system;
fig. 2 is a schematic flowchart of an image restoration method according to an embodiment of the present invention;
fig. 3 is a schematic flowchart of an image restoration method according to a second embodiment of the present invention;
fig. 4 is a schematic structural diagram of a video encoding and decoding system according to a third embodiment of the present invention;
fig. 5 is a schematic flowchart of an image restoration method according to a third embodiment of the present invention;
fig. 6 is a schematic structural diagram of an encoding end according to a fourth embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that the embodiments and features of the embodiments in the present application may be arbitrarily combined with each other without conflict.
The steps illustrated in the flow charts of the figures may be performed in a computer system such as a set of computer-executable instructions. Also, while a logical order is shown in the flow diagrams, in some cases, the steps shown or described may be performed in an order different than here.
By analyzing the correlation technique, the redundancy of the spatial domain I frame is generally large, and the redundancy of the time domain P frame is generally small; if the normal recovery of the image can be ensured and the redundancy is small, the loss of the image frame under the limited bandwidth can be avoided. Therefore, an embodiment of the present application provides an image recovery method and apparatus, including: when the image quality needs to be adjusted, obtaining a reference candidate frame according to the inter-frame similarity and the reconstructed image quality; and allocating bits for the target frame according to the reference candidate frame, and performing video coding on the target frame based on the bits allocated for the target frame. Therefore, the reference candidate frame is adaptively adjusted through the inter-frame similarity and the reconstructed image quality, so that the bit allocation of the target frame is adjusted, the network packet loss of the image frame is reduced, the packet loss resistance of the image is effectively improved, and the better image quality and fluency are obtained while the image is quickly recovered at the decoding end.
Example one
Fig. 2 is a schematic flowchart of an image restoration method according to an embodiment of the present invention. As shown in fig. 2, the image restoration method includes:
step 201, when determining that the image quality needs to be adjusted, obtaining a reference candidate frame according to the interframe similarity and the reconstructed image quality;
step 202, allocating bits for a target frame according to the reference candidate frame, and performing video coding on the target frame based on the bits allocated for the target frame.
Wherein the determining that image quality needs to be adjusted comprises:
determining the similarity measurement between frames and the quality of a reconstructed image;
and determining that the image quality needs to be adjusted according to the determined inter-frame similarity measure, the inter-frame similarity measure threshold, the reconstructed image quality threshold and the decoding feedback information.
Wherein the determining the inter-frame similarity measure comprises:
an inter-frame similarity measure is determined based on the motion vectors or motion complexity of the pixels.
Wherein the determining of the reconstructed image quality comprises:
and determining the quality of the reconstructed image according to the peak signal-to-noise ratio (PSNR) or the Quantization Parameter (QP) of the reconstructed image.
Obtaining a reference candidate frame according to the inter-frame similarity and the quality of the reconstructed image, wherein the obtaining of the reference candidate frame comprises:
and comparing the determined inter-frame similarity measure with a set inter-frame similarity measure threshold, comparing the determined reconstructed image quality with a set reconstructed image quality threshold, and obtaining a reference candidate frame according to the comparison result.
Obtaining a reference candidate frame according to the inter-frame similarity and the quality of the reconstructed image, wherein the obtaining of the reference candidate frame comprises:
four interframe similarity measurement thresholds are set, which are respectively: SThr1, SThr2, SThr3 and SThr4, wherein SThr1> SThr2> SThr3> SThr4, the larger the threshold value of the interframe similarity measure indicates that the images are more similar, and the lower the time domain complexity of the image sequence; and setting four reconstructed image quality thresholds which are respectively as follows: QPThr1, QPThr2, QPThr3 and QPThr4, wherein QPThr1< QPThr2< QPThr3< QPThr4, and a smaller reconstructed image quality threshold indicates a better reconstructed image quality;
when the determined inter-frame similarity measure is greater than or equal to SThr1 and the determined reconstructed image quality is less than or equal to QPThr1, which indicates that the image sequence has low time domain complexity and good reconstructed image quality, the frame is taken as a reference candidate frame Ref 1;
when the determined inter-frame similarity measure is greater than or equal to SThr1 and the determined reconstructed image quality is greater than or equal to QPThr4, which indicates that the image sequence has low time-domain complexity and poor reconstructed image quality, the frame is taken as a reference candidate frame Ref 2;
when the determined inter-frame similarity measure is greater than or equal to SThr1, and the determined reconstructed image quality is greater than or equal to QPThr3 and less than QPThr4, which indicates that the image sequence has low time-domain complexity and poor reconstructed image quality, the frame is taken as a reference candidate frame Ref 3;
when the determined inter-frame similarity measure is greater than or equal to SThr1 and the determined reconstructed image quality is greater than or equal to QPThr2 and less than QPThr3, which indicates that the image sequence has low time-domain complexity and general image quality, the frame is taken as a reference candidate frame Ref 4;
when the determined inter-frame similarity measure is larger than SThr4 and smaller than SThr1 and fluctuates between SThr2 and SThr3 and the determined image quality is larger than QPThr2 and smaller than QPThr3, which indicates that the image sequence has low or medium time-domain complexity and general image quality, the frame is taken as a reference candidate frame Ref 5;
and when the determined inter-frame similarity measure is less than or equal to SThr4 and the determined reconstructed image quality is greater than or equal to QPThr4, indicating that the image sequence has high time-domain complexity and poor image quality, not adopting the frame as a reference candidate frame.
The technical solution provided by the first embodiment is described in detail by two specific embodiments.
Example two
Fig. 3 is a flowchart illustrating an image restoration method according to a second embodiment of the present invention. As shown in fig. 3, the image restoration method includes:
step 301, determining inter-frame similarity;
the determining the inter-frame similarity specifically determines an inter-frame similarity measure. In determining the inter-frame similarity measure, a motion vector may be selected or the inter-frame similarity measure may be determined based on the motion complexity of the pixels; the inter-frame similarity measure threshold can be set according to requirements, and is used for setting the level of inter-frame similarity;
step 302, determining the quality of a reconstructed image;
when the quality of the reconstructed image is determined, the peak signal to noise Ratio (PSNR) or Quantization Parameter (QP) of the reconstructed image is mainly considered; the reconstructed image quality threshold can be set according to requirements, and the reconstructed image quality threshold is used for setting the quality level of the reconstructed image.
There is no fixed sequence between the steps 301 and 302, and the sequence is only an exemplary one.
Step 303, detecting decoding feedback information, and determining whether image quality needs to be adjusted according to the determined inter-frame similarity measure, the inter-frame similarity measure threshold, the reconstructed image quality threshold and the decoding feedback information;
wherein, when it is determined that the image quality needs to be adjusted, step 304 is performed, and when it is determined that the image quality does not need to be adjusted, video encoding is directly performed according to the existing method, for example, the step 306 is performed.
When the image is lost during decoding and the like, the image quality needs to be adjusted, and the purpose of adjusting the image quality is to reduce the image quality loss as much as possible without decoding the lost image. For example, when fewer codewords are found and are below a codeword threshold, it may be determined that image quality needs to be adjusted; if more codewords are found and are higher than the codeword threshold, the existing standard process can be considered. Wherein, the codeword threshold value < ═ is (current actual code rate (1-packet loss rate))/frame rate.
Step 304, obtaining a reference candidate frame according to the inter-frame similarity and the quality of the reconstructed image;
and comparing the determined inter-frame similarity measure with a set inter-frame similarity measure threshold, comparing the determined reconstructed image quality with a set reconstructed image quality threshold, and obtaining a reference candidate frame according to a comparison result.
Specifically, four inter-frame similarity measure thresholds may be set, which are: SThr1, SThr2, SThr3 and SThr4, and setting four reconstructed image quality thresholds, respectively: QPThr1, QPThr2, QPThr3, QPThr 4; wherein, SThr1> SThr2> SThr3> SThr4, the larger the threshold value of the interframe similarity measure is, the more similar the images are; QPThr1< QPThr2< QPThr3< QPThr4, a smaller reconstructed image quality threshold indicates better image quality;
when the determined inter-frame similarity measure is greater than or equal to SThr1 and the determined reconstructed image quality is less than or equal to QPThr1, which indicates that the image sequence has low time-domain complexity and good image quality, the frame can be used as a reference candidate frame Ref 1;
when the determined inter-frame similarity measure is greater than or equal to SThr1 and the determined reconstructed image quality is greater than or equal to QPThr4, which indicates that the image sequence has low time-domain complexity and poor image quality, the frame can be used as a reference candidate frame Ref 2;
when the determined inter-frame similarity measure is greater than or equal to SThr1 and the determined reconstructed image quality is greater than or equal to QPThr3 and less than QPThr4, the image sequence is low in time-domain complexity and poor in image quality, and the frame can be used as a reference candidate frame Ref 3;
when the determined inter-frame similarity measure is greater than or equal to SThr1 and the determined reconstructed image quality is greater than or equal to QPThr2 and less than QPThr3, the image sequence is low in time-domain complexity and general in image quality, and the frame can be used as a reference candidate frame Ref 4;
when the determined inter-frame similarity measure is greater than SThr4 and less than SThr1 and fluctuates between SThr2 and SThr3 and the determined image quality is greater than QPThr2 and less than QPThr3, indicating that the image sequence has low or medium time-domain complexity and general image quality, the frame can be taken as a reference candidate frame Ref 5;
when the determined inter-frame similarity measure is less than or equal to SThr4 and the determined image quality calculation value is greater than or equal to QPThr4, which indicates that the image sequence has high time-domain complexity and poor image quality, the frame may not be used as the reference candidate frame.
305, allocating bits for a target frame according to the reference candidate frame, and performing code rate control on the target frame based on the bits allocated for the target frame;
and step 306, performing video coding processing on the target frame based on the code rate controlled by the target frame.
Wherein the video encoding process comprises: deciding a coding type, namely a time domain P frame or a space domain I frame; video encoding is performed.
The image recovery method provided by the embodiment of the invention determines the coding frame suitable for the current network state, namely a temporal P frame or a spatial I frame, and obtains a reference candidate frame according to the inter-frame similarity and the quality of the reconstructed image; and distributing bits for the target frame according to the reference candidate frame, performing code rate control on the target frame based on the bits distributed for the target frame, and distributing a target bit number to the target frame according to the reference candidate frame when the target frame data is coded, namely correcting the bit number of the target frame, so that better image quality playing can be ensured under the condition of poor network.
EXAMPLE III
Fig. 4 is a schematic structural diagram of a video encoding and decoding system according to a third embodiment of the present invention, and fig. 5 is a schematic flow chart of an image restoration method according to the third embodiment of the present invention.
As shown in fig. 4, the video coding and decoding system includes a coding end and a decoding end, wherein the coding end includes a video coding apparatus 401, an image restoration apparatus 402, and an image detection apparatus 403. The decoding side includes an image feedback device 405 and a video decoding device 404.
The image restoration device 402 at the encoding end is used for adaptively adjusting the target bits of the image to ensure the image quality; the image detection device 403 at the encoding end is configured to calculate inter-frame similarity and reconstructed image quality, receive decoding feedback information fed back by the decoding end, and determine whether to execute a subsequent encoding strategy according to the inter-frame similarity, the reconstructed image quality, and the decoding feedback information. The video decoding apparatus 403 at the decoding end performs decoding according to the encoding policy information at the encoding end.
As shown in fig. 5, the image restoration method applied to the video codec system shown in fig. 4 includes:
step 501, an encoding end calculates the similarity between frames;
wherein, the similarity between frames is calculated, a motion vector can be selected or a similarity measure between frames can be calculated based on the motion complexity of pixels, and a threshold value is used for setting the similarity level between frames.
The image quality is calculated, mainly considering PSNR or quantization parameter QP of a reconstructed image, and setting the quality level of the image quality by using a threshold.
Wherein step 501 may be performed by the image restoration apparatus 402.
Step 502, the encoding end calculates the quality of the reconstructed image;
the image quality is calculated, mainly considering PSNR or quantization parameter QP of a reconstructed image, and setting the quality level of the image quality by using a threshold.
Wherein step 502 may be performed by the image restoration apparatus 402.
Step 503, the encoding end analyzes the feedback information state of the decoder in real time, dynamically monitors the decoding state, and acquires information such as the frame number of the complete key reference frame decoding;
wherein step 503 may be performed by the image detection apparatus 403.
Step 504, the encoding end determines whether to execute the encoding strategy, if the execution is shifted to step 505, if the execution is not shifted to step 507;
wherein, the encoding end adaptively determines whether an encoding strategy needs to be executed according to the inter-frame similarity measure and the threshold of step 501, the reconstructed image quality and the threshold of step 502 and the decoder feedback information of step 503, that is, determines whether the image quality needs to be adjusted;
wherein, whether the image quality needs to be adjusted is determined according to the determined inter-frame similarity measure, the inter-frame similarity measure threshold, the reconstructed image quality threshold and the decoding feedback information.
When it is determined that the encoding strategy needs to be executed, step 505 is executed to determine a reference candidate frame.
Wherein step 504 may be performed by the image restoration device 402.
Step 505, the encoding end adjusts the reference frame strategy;
wherein, the step 506 is executed after the execution is finished;
and adjusting the reference frame strategy to obtain a reference candidate frame, and allocating bits for the target frame according to the reference candidate frame.
And comparing the determined inter-frame similarity measure with a set inter-frame similarity measure threshold, comparing the determined reconstructed image quality with a set reconstructed image quality threshold, and obtaining a reference candidate frame according to a comparison result.
Specifically, four inter-frame similarity measure thresholds are set, which are respectively: SThr1, SThr2, SThr3 and SThr4, and setting four reconstructed image quality thresholds, respectively: QPThr1, QPThr2, QPThr3, QPThr 4; wherein, SThr1> SThr2> SThr3> SThr4, the larger the threshold value of the interframe similarity measure is, the more similar the images are; QPThr1< QPThr2< QPThr3< QPThr4, a smaller reconstructed image quality threshold indicates better image quality;
when the determined inter-frame similarity measure is greater than or equal to SThr1 and the determined reconstructed image quality is less than or equal to QPThr1, which indicates that the image sequence has low time-domain complexity and good image quality, the frame can be used as a reference candidate frame Ref 1;
when the determined inter-frame similarity measure is greater than or equal to SThr1 and the determined reconstructed image quality is greater than or equal to QPThr4, which indicates that the image sequence has low time-domain complexity and poor image quality, the frame can be used as a reference candidate frame Ref 2;
when the determined inter-frame similarity measure is greater than or equal to SThr1 and the determined reconstructed image quality is greater than or equal to QPThr3 and less than QPThr4, the image sequence is low in time-domain complexity and poor in image quality, and the frame can be used as a reference candidate frame Ref 3;
when the determined inter-frame similarity measure is greater than or equal to SThr1 and the determined reconstructed image quality is greater than or equal to QPThr2 and less than QPThr3, the image sequence is low in time-domain complexity and general in image quality, and the frame can be used as a reference candidate frame Ref 4;
when the determined inter-frame similarity measure is greater than SThr4 and less than SThr1 and fluctuates between SThr2 and SThr3 and the determined image quality is greater than QPThr2 and less than QPThr3, indicating that the image sequence has low or medium time-domain complexity and general image quality, the frame can be taken as a reference candidate frame Ref 5;
when the determined inter-frame similarity measure is less than or equal to SThr4 and the determined image quality calculation value is greater than or equal to QPThr4, which indicates that the image sequence has high time-domain complexity and poor image quality, the frame may not be used as the reference candidate frame.
Wherein step 505 may be performed by the image restoration apparatus 402.
Step 506, the coding end decides the coding type;
wherein, deciding the space domain I frame or the time domain P frame, and turning to the step 507 after the execution is finished;
step 507, the encoding end executes video encoding;
wherein steps 506, 507 may be performed by the video encoding device 402.
Wherein performing video encoding comprises: and performing code rate control on the target frame based on the bits distributed to the target frame, and performing video coding processing on the target frame based on the code rate for controlling the target frame.
And step 508, the decoding end decodes according to the coding strategy of the coding end.
Wherein step 508 may be performed by video decoding apparatus 403.
The image recovery method provided by the third embodiment of the invention has the following advantages and positive effects:
(1) the reference frame and the coding type are adaptively adjusted through the image similarity, the image quality and the decoding feedback information, and the target bit allocation of the frame is adjusted, so that a proper key frame is coded to reduce the network packet loss of the frame, the image is quickly recovered at a decoding end, and meanwhile, the better image quality and fluency are obtained.
(2) By the method, when the complexity of the image is low or moderate, the reference frame selection is adjusted, so that the anti-packet loss capability of the image can be effectively improved, and the image quality and the fluency of image playing are improved.
(3) By the method, when the image complexity is constantly changed, the reference frame selection and the decision coding type can be reasonably adjusted according to the image complexity, and the image quality and the fluency of image playing are improved.
Example four
Fig. 6 is a schematic structural diagram of an encoding end according to a fourth embodiment of the present invention. As shown in fig. 6, the encoding end includes:
the computing unit is used for obtaining a reference candidate frame according to the inter-frame similarity and the reconstructed image quality when the image quality needs to be adjusted;
a distribution unit for distributing bits for the target frame according to the reference candidate frame;
an encoding unit configured to perform video encoding on the target frame based on the bits allocated for the target frame.
Wherein the determining that image quality needs to be adjusted comprises:
determining the similarity measurement between frames and the quality of a reconstructed image;
and determining that the image quality needs to be adjusted according to the determined inter-frame similarity measure, the inter-frame similarity measure threshold, the reconstructed image quality threshold and the decoding feedback information.
Wherein the determining the inter-frame similarity measure comprises:
an inter-frame similarity measure is determined based on the motion vectors or motion complexity of the pixels.
Wherein the determining of the reconstructed image quality comprises:
and determining the quality of the reconstructed image according to the peak signal-to-noise ratio (PSNR) or the Quantization Parameter (QP) of the reconstructed image.
Obtaining a reference candidate frame according to the inter-frame similarity and the quality of the reconstructed image, wherein the obtaining of the reference candidate frame comprises:
and comparing the determined inter-frame similarity measure with a set inter-frame similarity measure threshold, comparing the determined reconstructed image quality with a set reconstructed image quality threshold, and obtaining a reference candidate frame according to the comparison result.
Obtaining a reference candidate frame according to the inter-frame similarity and the quality of the reconstructed image, wherein the obtaining of the reference candidate frame comprises:
four interframe similarity measurement thresholds are set, which are respectively: SThr1, SThr2, SThr3 and SThr4, wherein SThr1> SThr2> SThr3> SThr4, the larger the threshold value of the interframe similarity measure indicates that the images are more similar, and the lower the time domain complexity of the image sequence; and setting four reconstructed image quality thresholds which are respectively as follows: QPThr1, QPThr2, QPThr3 and QPThr4, wherein QPThr1< QPThr2< QPThr3< QPThr4, and a smaller reconstructed image quality threshold indicates a better reconstructed image quality;
when the determined inter-frame similarity measure is greater than or equal to SThr1 and the determined reconstructed image quality is less than or equal to QPThr1, which indicates that the image sequence has low time domain complexity and good reconstructed image quality, the frame is taken as a reference candidate frame Ref 1;
when the determined inter-frame similarity measure is greater than or equal to SThr1 and the determined reconstructed image quality is greater than or equal to QPThr4, which indicates that the image sequence has low time-domain complexity and poor reconstructed image quality, the frame is taken as a reference candidate frame Ref 2;
when the determined inter-frame similarity measure is greater than or equal to SThr1, and the determined reconstructed image quality is greater than or equal to QPThr3 and less than QPThr4, which indicates that the image sequence has low time-domain complexity and poor reconstructed image quality, the frame is taken as a reference candidate frame Ref 3;
when the determined inter-frame similarity measure is greater than or equal to SThr1 and the determined reconstructed image quality is greater than or equal to QPThr2 and less than QPThr3, which indicates that the image sequence has low time-domain complexity and general image quality, the frame is taken as a reference candidate frame Ref 4;
when the determined inter-frame similarity measure is larger than SThr4 and smaller than SThr1 and fluctuates between SThr2 and SThr3 and the determined image quality is larger than QPThr2 and smaller than QPThr3, which indicates that the image sequence has low or medium time-domain complexity and general image quality, the frame is taken as a reference candidate frame Ref 5;
and when the determined inter-frame similarity measure is less than or equal to SThr4 and the determined reconstructed image quality is greater than or equal to QPThr4, indicating that the image sequence has high time-domain complexity and poor image quality, not adopting the frame as a reference candidate frame.
Wherein said encoding the target frame based on the bits allocated for the target frame comprises:
performing rate control on the target frame based on bits allocated to the target frame;
and performing video coding on the target frame based on the code rate controlled by the target frame.
The function of the calculation unit corresponds to the function of determining reference candidate frames by the image monitoring apparatus 403, the function of the allocation unit corresponds to the bit allocation function by the image restoration apparatus 402, and the function of the encoding unit corresponds to the video encoding function by the video encoding apparatus 401.
The embodiment of the present invention further provides an encoding end, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and when the computer program is executed by the processor, the image restoration method described in any one of the above is implemented.
An embodiment of the present invention further provides a computer-readable storage medium, where an information processing program is stored on the computer-readable storage medium, and when the information processing program is executed by a processor, the information processing program implements the steps of any one of the image restoration methods described above.
According to the rapid image restoration method and the encoding end provided by the embodiment, the reference candidate frame is obtained according to the inter-frame similarity and the quality of the reconstructed image; allocating bits for a target frame according to the reference candidate frame, and performing video coding on the target frame based on the bits allocated for the target frame; namely, when the target frame data is coded, the target bit number is distributed to the target frame according to the reference candidate frame, namely, the bit number of the target frame is corrected, so that better image quality playing can be ensured under the condition of poorer network.
It will be understood by those of ordinary skill in the art that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation. Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.
Although the embodiments of the present invention have been described above, the above description is only for the convenience of understanding the present invention, and is not intended to limit the present invention. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims (10)

1. An image restoration method comprising:
when the image quality needs to be adjusted, obtaining a reference candidate frame according to the inter-frame similarity and the reconstructed image quality;
and allocating bits for the target frame according to the reference candidate frame, and performing video coding on the target frame based on the bits allocated for the target frame.
2. The method of claim 1, wherein determining that image quality needs to be adjusted comprises:
determining the similarity measurement between frames and the quality of a reconstructed image;
and determining that the image quality needs to be adjusted according to the determined inter-frame similarity measure, the inter-frame similarity measure threshold, the reconstructed image quality threshold and the decoding feedback information.
3. The method of claim 2, wherein determining the inter-frame similarity measure comprises:
an inter-frame similarity measure is determined based on the motion vectors or motion complexity of the pixels.
4. The method of claim 2, wherein determining the quality of the reconstructed image comprises:
and determining the quality of the reconstructed image according to the peak signal-to-noise ratio (PSNR) or the Quantization Parameter (QP) of the reconstructed image.
5. The method of claim 1, wherein obtaining the reference candidate frame according to the inter-frame similarity and the reconstructed image quality comprises:
and comparing the determined inter-frame similarity measure with a set inter-frame similarity measure threshold, comparing the determined reconstructed image quality with a set reconstructed image quality threshold, and obtaining a reference candidate frame according to the comparison result.
6. The method of claim 5, wherein obtaining the reference candidate frame according to the inter-frame similarity and the reconstructed image quality comprises:
four interframe similarity measurement thresholds are set, which are respectively: SThr1, SThr2, SThr3 and SThr4, wherein SThr1> SThr2> SThr3> SThr4, the larger the threshold value of the interframe similarity measure indicates that the images are more similar, and the lower the time domain complexity of the image sequence; and setting four reconstructed image quality thresholds which are respectively as follows: QPThr1, QPThr2, QPThr3 and QPThr4, wherein QPThr1< QPThr2< QPThr3< QPThr4, and a smaller reconstructed image quality threshold indicates a better reconstructed image quality;
when the determined inter-frame similarity measure is greater than or equal to SThr1 and the determined reconstructed image quality is less than or equal to QPThr1, which indicates that the image sequence has low time domain complexity and good reconstructed image quality, the frame is taken as a reference candidate frame Ref 1;
when the determined inter-frame similarity measure is greater than or equal to SThr1 and the determined reconstructed image quality is greater than or equal to QPThr4, which indicates that the image sequence has low time-domain complexity and poor reconstructed image quality, the frame is taken as a reference candidate frame Ref 2;
when the determined inter-frame similarity measure is greater than or equal to SThr1, and the determined reconstructed image quality is greater than or equal to QPThr3 and less than QPThr4, which indicates that the image sequence has low time-domain complexity and poor reconstructed image quality, the frame is taken as a reference candidate frame Ref 3;
when the determined inter-frame similarity measure is greater than or equal to SThr1 and the determined reconstructed image quality is greater than or equal to QPThr2 and less than QPThr3, which indicates that the image sequence has low time-domain complexity and general image quality, the frame is taken as a reference candidate frame Ref 4;
when the determined inter-frame similarity measure is larger than SThr4 and smaller than SThr1 and fluctuates between SThr2 and SThr3 and the determined image quality is larger than QPThr2 and smaller than QPThr3, which indicates that the image sequence has low or medium time-domain complexity and general image quality, the frame is taken as a reference candidate frame Ref 5;
and when the determined inter-frame similarity measure is less than or equal to SThr4 and the determined reconstructed image quality is greater than or equal to QPThr4, indicating that the image sequence has high time-domain complexity and poor image quality, not adopting the frame as a reference candidate frame.
7. The method of claim 1, wherein encoding the target frame based on the bits allocated for the target frame comprises:
performing rate control on the target frame based on bits allocated to the target frame;
and performing video coding on the target frame based on the code rate controlled by the target frame.
8. An encoding end, comprising:
the computing unit is used for obtaining a reference candidate frame according to the inter-frame similarity and the reconstructed image quality when the image quality needs to be adjusted;
a distribution unit for distributing bits for the target frame according to the reference candidate frame;
an encoding unit configured to perform video encoding on the target frame based on the bits allocated for the target frame.
9. An encoding end, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the image restoration method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that an information processing program is stored thereon, which when executed by a processor, implements the steps of the image restoration method according to any one of claims 1 to 7.
CN201811012695.XA 2018-08-31 2018-08-31 Image recovery method and encoding end Pending CN110875906A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811012695.XA CN110875906A (en) 2018-08-31 2018-08-31 Image recovery method and encoding end

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811012695.XA CN110875906A (en) 2018-08-31 2018-08-31 Image recovery method and encoding end

Publications (1)

Publication Number Publication Date
CN110875906A true CN110875906A (en) 2020-03-10

Family

ID=69715822

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811012695.XA Pending CN110875906A (en) 2018-08-31 2018-08-31 Image recovery method and encoding end

Country Status (1)

Country Link
CN (1) CN110875906A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113438551A (en) * 2020-03-23 2021-09-24 未来穿戴技术有限公司 Data transmission method, data transmission device, computer equipment and computer readable storage medium
CN113920010A (en) * 2020-07-10 2022-01-11 华为技术有限公司 Super-resolution implementation method and device for image frame

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101127919A (en) * 2007-09-28 2008-02-20 中兴通讯股份有限公司 A video sequence coding method
US20090161982A1 (en) * 2007-12-19 2009-06-25 Nokia Corporation Restoring images
CN102938840A (en) * 2012-11-26 2013-02-20 南京邮电大学 Key frame quantization parameter selecting method applied to multi-viewpoint video coding system
CN105323592A (en) * 2014-07-11 2016-02-10 中兴通讯股份有限公司 Code rate control method and device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101127919A (en) * 2007-09-28 2008-02-20 中兴通讯股份有限公司 A video sequence coding method
US20090161982A1 (en) * 2007-12-19 2009-06-25 Nokia Corporation Restoring images
CN102938840A (en) * 2012-11-26 2013-02-20 南京邮电大学 Key frame quantization parameter selecting method applied to multi-viewpoint video coding system
CN105323592A (en) * 2014-07-11 2016-02-10 中兴通讯股份有限公司 Code rate control method and device

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113438551A (en) * 2020-03-23 2021-09-24 未来穿戴技术有限公司 Data transmission method, data transmission device, computer equipment and computer readable storage medium
CN113438551B (en) * 2020-03-23 2023-06-27 未来穿戴技术有限公司 Data transmission method, device, computer equipment and computer readable storage medium
CN113920010A (en) * 2020-07-10 2022-01-11 华为技术有限公司 Super-resolution implementation method and device for image frame
WO2022007895A1 (en) * 2020-07-10 2022-01-13 华为技术有限公司 Ultra resolution implementation method and device for image frames

Similar Documents

Publication Publication Date Title
US10735751B2 (en) Multi-pass video encoding
US10911763B2 (en) System and method for electronic data communication
CN102036061B (en) Video data transmission and sending processing method, device and network system
US8856624B1 (en) Method and apparatus for dynamically generating error correction
KR101125846B1 (en) Method for transmitting image frame data based on packet system and apparatus thereof
RU2377737C2 (en) Method and apparatus for encoder assisted frame rate up conversion (ea-fruc) for video compression
US20070009039A1 (en) Video encoding and decoding methods and apparatuses
US8638851B2 (en) Joint bandwidth detection algorithm for real-time communication
US20060188025A1 (en) Error concealment
US11889122B2 (en) Quality aware error concealment technique for streaming media
KR102473678B1 (en) Method for transmitting video and data transmitter
CN110876066A (en) Adaptive forward error correction method, apparatus, medium and device
US10432946B2 (en) De-juddering techniques for coded video
CN110875906A (en) Image recovery method and encoding end
EP3796652B1 (en) Video encoding method and method for reducing file size of encoded video
CN111416978B (en) Video encoding and decoding method and system, and computer readable storage medium
CN118158418A (en) Video coding method and related device
CN105323592A (en) Code rate control method and device
WO2021004420A1 (en) Encoding and decoding methods, transmission method, and encoding and decoding devices and systems
JP2000138930A (en) Image decoder
US8971393B2 (en) Encoder
Dissanayake et al. Error resilience for multi-view video using redundant macroblock coding
EP4369709A1 (en) Methods and systems for quality of experience-centric adaptive bitrate encoding for live video streaming
US20150350688A1 (en) I-frame flashing fix in video encoding and decoding
Gao et al. A Low-Latency Video Transmission Scheme Based on Sliding-Window FEC

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20200310