WO2023169426A1 - 编解码方法及电子设备 - Google Patents

编解码方法及电子设备 Download PDF

Info

Publication number
WO2023169426A1
WO2023169426A1 PCT/CN2023/080113 CN2023080113W WO2023169426A1 WO 2023169426 A1 WO2023169426 A1 WO 2023169426A1 CN 2023080113 W CN2023080113 W CN 2023080113W WO 2023169426 A1 WO2023169426 A1 WO 2023169426A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame
current frame
reconstructed image
encoder
code stream
Prior art date
Application number
PCT/CN2023/080113
Other languages
English (en)
French (fr)
Inventor
胡巩
卢宇峰
汪澄
邸佩云
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2023169426A1 publication Critical patent/WO2023169426A1/zh

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding

Definitions

  • the embodiments of the present application relate to the field of coding and decoding technology, and in particular, to a coding and decoding method and an electronic device.
  • this application provides a coding and decoding method and electronic equipment.
  • the reference frame of the codec can be flexibly managed, thereby increasing the encoder's ability to withstand weak networks, thereby ensuring the smoothness of the video when network conditions deteriorate in RTC scenarios.
  • embodiments of the present application provide an encoding method.
  • the method includes: first, obtaining the current frame; then, obtaining the reconstructed image corresponding to the reference frame of the current frame from the external reference list of the encoder.
  • the reference frame is the frame that has been encoded by the encoder, and the external reference list is independent of the encoder; then, the encoder performs intra-frame coding on the reconstructed image; based on the result of intra-frame coding, the encoder performs inter-frame coding on the current frame to Get the code stream corresponding to the current frame.
  • the encoder can flexibly select reference frames from the external reference list for encoding, thereby enabling cross-frame reference or cross-resolution reference, which increases the flexibility of the encoder's reference frame management. This can increase the encoder's ability to withstand weak networks and ensure the smoothness of the video when network conditions deteriorate in RTC scenarios.
  • the external reference list is independent of the encoder, which means that the external reference list is located outside the encoder; it is used to distinguish the reference list managed internally by the encoder; both the encoder and other modules can access the external reference list.
  • the code stream corresponding to the current frame can be sent to other devices or stored locally. This application does not limit this.
  • the encoder may perform intra-frame encoding on the reconstructed image corresponding to the reference frame of the current frame with a quantization parameter smaller than the first preset threshold.
  • the first preset threshold can be set according to requirements, for example, the first preset threshold The value is a number less than 10, such as 2, 3, 4, 5, 6, 7, 8, 9. This application does not limit this. In this way, high-quality intra-frame coding can be achieved for the reconstructed image corresponding to the reference frame of the current frame, thereby improving the quality of the encoded image of the current frame.
  • the encoder may perform intra-frame encoding on the reconstructed image corresponding to the reference frame of the current frame at a code rate greater than the second preset threshold.
  • the second preset threshold can be set according to requirements, and this application does not limit this. In this way, high-quality intra-frame coding can also be achieved for the reconstructed image corresponding to the reference frame of the current frame, thereby improving the coding quality of the current frame.
  • the encoding and decoding scenarios corresponding to the encoding method provided by this application may include multiple encoding and decoding scenarios, such as scenarios using long-term reference frame coding, time-domain hierarchical coding scenarios, and reference frame resampling scenarios, etc. This application does not limit this.
  • the encoding method of this application may refer to a method for encoding P frames (Predictive frames, forward predictive frames).
  • the method further includes: when it is determined that the current frame needs to be used as a reference frame for subsequent encoding, generating a reconstructed image of the current frame based on the code stream corresponding to the current frame; adding the reconstructed image of the current frame to the external reference list . In this way, the frame after the current frame can obtain the reconstructed image of the reference frame from the external reference list.
  • the code stream corresponding to the current frame includes: a first identifier and first indication information; the first identifier is used to identify whether the current frame is encoded based on the external reference list, and the An indication information is used to indicate the reconstructed image corresponding to the reference frame of the current frame in the external reference list.
  • the first identifier may also be used to identify whether the current frame refers to the long-term reference frame/time-domain hierarchical reference frame/resampling reference frame coding.
  • the code stream corresponding to the current frame may also include only the first identifier.
  • the first identifier is used to indicate that the current frame is not encoded based on an external reference list.
  • the code stream corresponding to the current frame includes: a second identifier and second indication information; the second identifier is used to identify whether the current frame is used as a reference frame for subsequent encoding, The second indication information is used to indicate the reconstructed image of the current frame in the external reference list.
  • the code stream corresponding to the current frame may also include only the third identifier.
  • the third identifier is used to identify that the current frame will not be resampled.
  • the standard syntax element used to identify the reference frame of the current frame in the code stream corresponding to the current frame; wherein the standard syntax element refers to the standard encoding Syntax elements specified in the protocol.
  • the standard syntax element refers to the standard encoding Syntax elements specified in the protocol.
  • the standard syntax elements used to identify the reference frame related to the current frame include at least one of the following:
  • Syntax element used to identify whether the forward reference frame is used for the current frame reference is used for the current frame reference.
  • the syntax element used to identify the number of long-term reference frames can be: num_long_term_pics
  • the syntax element used to identify the long-term reference frame POC can be poc_lsb_lt[i] (poc_lsb_lt[i], which refers to the i-th POC of long-term reference frames)
  • the syntax element used to identify whether the long-term reference frame is used for the current frame reference can be: used_by_curr_pic_lt_flag[i] (used_by_curr_pic_lt_flag[i], refers to whether the i-th long-term reference frame is used for the current frame reference ).
  • the syntax element used to identify whether to use a forward reference frame can be short_term_ref_pic_set_sps_flag; the syntax element used to identify the number of forward reference frames can be num_negative_pics; used to identify the current frame and the forward reference frame
  • the syntax element of the distance can be: delta_poc_s0_minus1[i] (delta_poc_s0_minus1[i], refers to the distance between the current frame and the i-th forward reference frame); the syntax element used to identify whether the forward reference frame is used for the current frame reference can It is used_by_curr_pic_s0_flag[i] (used_by_curr_pic_s0_flag[i] refers to whether the i-th forward reference frame is used for the current frame reference).
  • standard syntax element used to identify the reference frame related to the current frame may also include the syntax element pic_order_cnt_lsb used to identify the POC value of the current frame.
  • obtaining the reconstructed image corresponding to the reference frame of the current frame from the external reference list of the encoder includes: determining whether the preset reference rule is satisfied; when the When presetting reference rules, the reference indication information is determined according to the preset reference rules; according to the reference indication information, the encoding Obtain the reconstructed image corresponding to the reference frame of the current frame from the external reference list of the processor.
  • determining whether the preset reference rule is satisfied may refer to determining whether the current frame satisfies the preset reference rule.
  • the preset reference rules include any one of the following: long-term reference rules, time domain hierarchical reference rules and resampling reference rules.
  • the long-term reference rule includes at least one of the following:
  • the reference period of the long-term reference frame is reached
  • reaching the reference period of the long-term reference frame may refer to the distance difference between the encoded frame that is closest to the current frame and referring to the long-term reference frame and the current frame, and reaching the reference period of the long-term reference frame.
  • time domain hierarchical reference rules can be set according to the time domain hierarchical structure.
  • the resampling reference rule may include at least one of the following: the resampling reference period is reached; the previous frame of the current frame is resampled. Reaching the reference period for resampling may refer to the coded frame that is closest to the current frame and refers to the resampling reference frame. The distance difference to the current frame is reaching the reference period for resampling.
  • the intra-frame coding result includes the intra-frame coding reconstructed image of the reconstructed image corresponding to the reference frame of the current frame; based on the intra-frame coding result, the encoder Performing inter-frame coding on the current frame to obtain the code stream corresponding to the current frame includes: using the intra-frame coding reconstructed image as a reference by the encoder to perform inter-frame coding on the current frame to obtain the code stream corresponding to the current frame.
  • the reference frame of the current frame is: a long-term reference frame or a time domain hierarchical reference frame or a resampling reference frame.
  • the encoder is a hardware encoder.
  • the decoder can also flexibly select reference frames from the external reference list for decoding, allowing the decoder to implement cross-frame reference encoding or cross-resolution reference decoding, which improves the flexibility of the reference frame management of the hardware decoder. It then cooperates with the encoder to ensure the smoothness of the video when network conditions deteriorate in RTC scenarios.
  • the decoder may include a decoder with flexible reference frame management, or may include a decoder with inefficient reference frame management. Live decoder, this application does not limit this.
  • the reconstructed image corresponding to the reference frame of the current frame may be intra-frame decoded with a quantization parameter smaller than the first preset threshold.
  • the first preset threshold can be set according to requirements.
  • the first preset threshold is a number less than 10, such as 2, 3, 4, 5, 6, 7, 8, and 9. This application does not limit this. In this way, high-quality intra-frame decoding of the reconstructed image corresponding to the reference frame of the current frame can be achieved, thereby improving the decoded image quality of the current frame.
  • the reconstructed image corresponding to the reference frame of the current frame may be intra-frame decoded at a code rate greater than the second preset threshold.
  • the second preset threshold can be set according to requirements, and this application does not limit this. In this way, high-quality intra-frame decoding of the reconstructed image corresponding to the reference frame of the current frame can also be achieved, thereby improving the decoding quality of the current frame.
  • the decoding method of this application may refer to a method for decoding P frames (Predictive frames, forward predictive frames).
  • the method further includes: when it is determined that the current frame needs to be used as a reference frame for subsequent decoding, adding the reconstructed image of the current frame to the external reference list. In this way, the frame after the current frame can obtain the reconstructed image of the reference frame from the external reference list.
  • adding the reconstructed image of the current frame to the external reference list includes: resampling the reconstructed image of the current frame; adding the resampled reconstructed image to List of external references. In this way, the reconstructed image corresponding to the resampled reference frame can be added to the external reference list to facilitate subsequent coding of the resampled reference frame.
  • determining the reconstructed image corresponding to the reference frame of the current frame from the external reference list of the decoder includes: parsing the code stream corresponding to the current frame to obtain Reference frame parameters; based on the reference frame parameters, determine the reconstructed image corresponding to the reference frame of the current frame from the external reference list of the decoder.
  • the reference frame parameters include: a first identifier and first indication information; the first identifier is used to identify whether the current frame is decoded based on the external reference list, and the first indication information Used to indicate the reconstructed image corresponding to the reference frame of the current frame in the external reference list.
  • the reference frame parameter may also include only the first identifier.
  • the first identifier is used to indicate that the current frame is not decoded based on the external reference list.
  • the reference frame parameters include: a second identifier and second indication information; the second identifier is used to identify whether the current frame is used as a reference frame for subsequent decoding, and the first indication Information used to indicate the reconstructed image for the current frame in the external reference list.
  • the reference frame parameter may also include only the second identifier.
  • the second identifier is used to indicate that the current frame is not used as a reference frame for subsequent decoding.
  • the code stream corresponding to the current frame includes: a third identifier, third indication information and resampling resolution; the third identifier is used to identify whether the current frame is resampled. Sampling, the third indication information is used to indicate the resampled reconstructed image of the current frame in the external reference list, and the resampling resolution includes the resolution corresponding to the reconstructed image of the current frame and the resolution corresponding to the resampled reconstructed image.
  • the reference frame parameters may also include only the third identifier.
  • the third identifier is used to indicate that the current frame will not be resampled.
  • the reference frame of the current frame is: a long-term reference frame or a time domain hierarchical reference frame or a resampling reference frame.
  • the second aspect and any implementation manner of the second aspect respectively correspond to the first aspect and any implementation manner of the first aspect.
  • the technical effects corresponding to the second aspect and any implementation manner of the second aspect may be referred to the technical effects corresponding to the above-mentioned first aspect and any implementation manner of the first aspect, which will not be described again here.
  • the external reference list is independent of the second encoder; when it is determined that encoding by the second encoder is required,
  • the reconstructed image corresponding to the reference frame of the current frame is determined from the external reference list, and based on the reconstructed image corresponding to the reference frame of the current frame, the second encoder encodes the current frame to obtain the code stream corresponding to the current frame;
  • the first encoder is a hardware encoder
  • the second encoder is a software encoder.
  • the method further includes: determining whether the current frame is a T0 layer in the time domain hierarchical structure; when the current frame is a T0 layer in the time domain hierarchical structure, determining that it needs to be encoded by the first encoder; when When the current frame is not the T0 layer in the time domain hierarchical structure, it is determined that it needs to be encoded by the second encoder.
  • the T0 layer is the lowest layer of the time domain hierarchical structure, which can also be called the base layer; other layers can be called enhancement layers.
  • the code stream corresponding to the current frame includes: a first identifier and first indication information; the first identifier is used to identify whether the current frame is encoded based on the external reference list, and the An indication information is used to indicate the reconstructed image corresponding to the reference frame of the current frame in the external reference list.
  • the code stream corresponding to the current frame may also include only the first identifier.
  • the first identifier is used to indicate that the current frame is not encoded based on an external reference list.
  • embodiments of the present application provide a decoding method.
  • the method includes: first, receiving a code stream corresponding to the current frame.
  • the first decoder decodes the code stream corresponding to the current frame to obtain the reconstructed image of the current frame; and adds the reconstructed image of the current frame to the external part of the second decoder.
  • the method further includes: parsing the code stream corresponding to the current frame to obtain reference frame parameters; and determining whether decoding by the first decoder is required based on the reference frame parameters.
  • the reference frame parameters include: a first identifier and first indication information; the first identifier is used to identify whether the current frame is decoded based on the external reference list, and the first indication information Used to indicate the reconstructed image corresponding to the reference frame of the current frame in the external reference list.
  • determining the reconstructed image corresponding to the reference frame of the current frame from the external reference list includes: when the value of the first identifier is the first preset information, The reconstructed image matching the first indication information in the external reference list is determined as the reconstructed image corresponding to the reference frame of the current frame.
  • the first decoder and the second decoder are both software decoders.
  • embodiments of the present application provide an electronic device, including: a memory and a processor, the memory is coupled to the processor; the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the third aspect or The encoding method in any possible implementation of the third aspect.
  • the eighth aspect and any implementation manner of the eighth aspect respectively correspond to the fourth aspect and any implementation manner of the fourth aspect.
  • the technical effects corresponding to the eighth aspect and any implementation manner of the eighth aspect can be found in the technical effects corresponding to the above-mentioned fourth aspect and any implementation manner of the fourth aspect, which will not be described again here.
  • embodiments of the present application provide a chip, including one or more interface circuits and one or more processors; the interface circuit is used to receive signals from the memory of the electronic device and send signals to the processor, and the signals include the memory computer instructions stored in; when the processor executes the computer instructions, the electronic device is caused to execute the encoding method in the first aspect or any possible implementation of the first aspect.
  • the tenth aspect and any implementation manner of the tenth aspect respectively correspond to the second aspect and any implementation manner of the second aspect.
  • the technical effects corresponding to the tenth aspect and any implementation manner of the tenth aspect please refer to the technical effects corresponding to the above-mentioned second aspect and any implementation manner of the second aspect, which will not be described again here.
  • the eleventh aspect and any implementation manner of the eleventh aspect respectively correspond to the third aspect and any implementation manner of the third aspect.
  • the technical effects corresponding to the eleventh aspect and any implementation manner of the eleventh aspect please refer to the technical effects corresponding to the above third aspect and any implementation manner of the third aspect, which will not be described again here.
  • embodiments of the present application provide a computer-readable storage medium.
  • the computer-readable storage medium stores a computer program.
  • the computer program When the computer program is run on a computer or processor, it causes the computer or processor to execute the first aspect or Encoding method in any possible implementation of the first aspect.
  • the thirteenth aspect and any implementation manner of the thirteenth aspect respectively correspond to the first aspect and any implementation manner of the first aspect.
  • the technical effects corresponding to the thirteenth aspect and any implementation manner of the thirteenth aspect may be referred to the technical effects corresponding to the above-mentioned first aspect and any implementation manner of the first aspect, and will not be described again here.
  • the fourteenth aspect and any implementation manner of the fourteenth aspect respectively correspond to the second aspect and any implementation manner of the second aspect.
  • the technical effects corresponding to the fourteenth aspect and any implementation manner of the fourteenth aspect please refer to the technical effects corresponding to the above-mentioned second aspect and any implementation manner of the second aspect, which will not be described again here.
  • embodiments of the present application provide a computer-readable storage medium.
  • the computer-readable storage medium stores a computer program.
  • the computer program When the computer program is run on a computer or processor, it causes the computer or processor to execute the third aspect or The encoding method in any possible implementation of the third aspect.
  • the fifteenth aspect and any implementation manner of the fifteenth aspect respectively correspond to the third aspect and any implementation manner of the third aspect.
  • the technical effects corresponding to the fifteenth aspect and any implementation method of the fifteenth aspect please refer to the technical effects corresponding to the above-mentioned third aspect and any implementation method of the third aspect, which will not be described again here.
  • embodiments of the present application provide a computer-readable storage medium.
  • the computer-readable storage medium stores a computer program.
  • the computer program When the computer program is run on a computer or processor, it causes the computer or processor to execute the fourth aspect or The decoding method in any possible implementation of the fourth aspect.
  • the sixteenth aspect and any implementation manner of the sixteenth aspect respectively correspond to the fourth aspect and any implementation manner of the fourth aspect.
  • the technical effects corresponding to the sixteenth aspect and any one implementation manner of the sixteenth aspect may be referred to the technical effects corresponding to the above-mentioned fourth aspect and any one implementation manner of the fourth aspect, which will not be described again here.
  • inventions of the present application provide a computer program product.
  • the computer program product includes a software program.
  • the software program When executed by a computer or a processor, it causes the computer or processor to execute the first aspect or any possibility of the first aspect. Encoding method in the implementation.
  • the seventeenth aspect and any implementation manner of the seventeenth aspect respectively correspond to the first aspect and any implementation manner of the first aspect.
  • the technical effects corresponding to the seventeenth aspect and any one of the implementation methods of the seventeenth aspect can be found in the technical effects corresponding to the above-mentioned first aspect and any one implementation method of the first aspect, which will not be described again here.
  • inventions of the present application provide a computer program product.
  • the computer program product includes a software program.
  • the software program When executed by a computer or a processor, it causes the computer or processor to execute the second aspect or any possibility of the second aspect.
  • the decoding method in the implementation is not limited to:
  • the eighteenth aspect and any implementation manner of the eighteenth aspect respectively correspond to the second aspect and any implementation manner of the second aspect.
  • the technical effects corresponding to the eighteenth aspect and any implementation manner of the eighteenth aspect please refer to the technical effects corresponding to the above-mentioned second aspect and any implementation manner of the second aspect, which will not be described again here.
  • inventions of the present application provide a computer program product.
  • the computer program product includes a software program.
  • the software program When executed by a computer or processor, it causes the computer or processor to execute the third aspect or any possibility of the third aspect. Encoding method in the implementation.
  • inventions of the present application provide a computer program product.
  • the computer program product includes a software program.
  • the software program When executed by a computer or a processor, it causes the computer or processor to execute the fourth aspect or any possibility of the fourth aspect.
  • the decoding method in the implementation is not limited to, but not limited to, but not limited to, but not limited to,
  • the twentieth aspect and any implementation manner of the twentieth aspect respectively correspond to the fourth aspect and any implementation manner of the fourth aspect.
  • the twentieth aspect and the technology corresponding to any implementation method of the twentieth aspect please refer to the above-mentioned fourth aspect and the technical effect corresponding to any implementation manner of the fourth aspect, which will not be described again here.
  • inventions of the present application provide an electronic device.
  • the electronic device is used for encoding and may include an acquisition module and a processing module.
  • the acquisition module may be used to acquire the current frame; the acquisition module may also be used to obtain the current frame from the encoder.
  • the external reference list of the current frame obtain the reconstructed image corresponding to the reference frame of the current frame.
  • the reference frame of the current frame is the frame that has been encoded by the encoder.
  • the external reference list is independent of the encoder; the processing module is used to call the encoder to perform processing on the reconstructed image.
  • Intra-frame coding the processing module is also used to call the encoder to perform inter-frame coding on the current frame based on the results of intra-frame coding to obtain the code stream corresponding to the current frame.
  • the processing module may perform, for example, the functions of the video encoding module in FIG. 1 .
  • the electronic device may also include a communication module, which may be used to communicate with other electronic devices and to communicate between various modules within the electronic device. For example, it may be used to send a code stream corresponding to the current frame.
  • a communication module which may be used to communicate with other electronic devices and to communicate between various modules within the electronic device. For example, it may be used to send a code stream corresponding to the current frame.
  • the above-mentioned electronic device may be used to perform the encoding method in the first aspect or any possible implementation of the first aspect.
  • the twenty-first aspect and any implementation manner of the twenty-first aspect respectively correspond to the first aspect and any implementation manner of the first aspect.
  • the technical effects corresponding to the twenty-first aspect and any implementation manner of the twenty-first aspect may be referred to the technical effects corresponding to the first aspect and any implementation manner of the first aspect, and will not be described again here.
  • inventions of the present application provide an electronic device.
  • the electronic device is used for decoding and may include a communication module and a processing module.
  • the communication module may be used to receive the code stream corresponding to the current frame; the processing module is used to receive the code stream from the current frame.
  • the external reference list of the decoder determine the reconstructed image corresponding to the reference frame of the current frame.
  • the reference frame of the current frame is the frame that has been decoded by the decoder.
  • the external reference list is independent of the decoder; the processing module is also used to perform processing on the reconstructed image.
  • Intra-frame encoding to obtain the corresponding intra-frame encoding code stream the processing module is also used to call the decoder to perform intra-frame decoding on the intra-frame encoding code stream; call the decoder to perform intra-frame decoding on the code stream corresponding to the current frame based on the result of intra-frame decoding. Perform inter-frame decoding to obtain the reconstructed image of the current frame.
  • the processing module may perform, for example, the functions of the video decoding module in FIG. 1 .
  • the above-mentioned electronic device may be used to perform the decoding method in the second aspect or any possible implementation of the second aspect.
  • inventions of the present application provide an electronic device.
  • the electronic device is used for encoding and may include an acquisition module and a processing module.
  • the acquisition module may be used to acquire the current frame; the processing module is used to determine whether the first frame needs to be processed.
  • the first encoder When the first encoder is encoding, the first encoder is called to encode the current frame to obtain the code stream corresponding to the current frame; and the reconstructed image generated based on the code stream corresponding to the current frame is added to the external reference list of the second encoder,
  • the external reference list is independent of the second encoder; the processing module is also used to determine the reconstructed image corresponding to the reference frame of the current frame from the external reference list when it is determined that it needs to be encoded by the second encoder, and call the second encoder based on the current
  • the reference frame of the frame Correspondingly reconstruct the image, and encode the current frame to obtain the code stream corresponding to the current frame; wherein, the first encoder is a hardware encoder, and the second encoder is a software encoder.
  • the processing module may perform, for example, the functions of the video encoding module in FIG. 1 .
  • the electronic device may also include a communication module, which may be used to communicate with other electronic devices and to communicate between various modules within the electronic device. For example, it may be used to send a code stream corresponding to the current frame.
  • a communication module which may be used to communicate with other electronic devices and to communicate between various modules within the electronic device. For example, it may be used to send a code stream corresponding to the current frame.
  • the above-mentioned electronic device may be used to perform the encoding method in the third aspect or any possible implementation of the third aspect.
  • the twenty-third aspect and any implementation manner of the twenty-third aspect respectively correspond to the third aspect and any implementation manner of the third aspect.
  • the technical effects corresponding to the twenty-third aspect and any one of the implementation methods of the twenty-third aspect please refer to the technical effects corresponding to the above-mentioned third aspect and any one of the implementation methods of the third aspect, which will not be described again here.
  • the above-mentioned electronic device may be used to perform the decoding method in the fourth aspect or any possible implementation of the fourth aspect.
  • the twenty-fourth aspect and any implementation manner of the twenty-fourth aspect respectively correspond to the fourth aspect and any implementation manner of the fourth aspect.
  • the technical effects corresponding to the twenty-fourth aspect and any implementation method of the twenty-fourth aspect please refer to the technical effects corresponding to the above-mentioned fourth aspect and any implementation method of the fourth aspect, which will not be described again here.
  • the present application provides a code stream, wherein the code stream is obtained by the encoding method in the aforementioned first aspect or any possible implementation of the first aspect.
  • the code stream can be stored on a computer-readable storage medium, or transmitted in the form of electromagnetic signals.
  • the present application provides a code stream, wherein the code stream is obtained by the encoding method in the aforementioned third aspect or any possible implementation of the third aspect.
  • the code stream can be stored on a computer-readable storage medium, or transmitted in the form of electromagnetic signals.
  • the twenty-sixth aspect and any implementation manner of the twenty-sixth aspect respectively correspond to the third aspect and any implementation manner of the third aspect.
  • the technical effects corresponding to the twenty-sixth aspect and any implementation method of the twenty-sixth aspect please refer to the technical effects corresponding to the above-mentioned third aspect and any implementation method of the third aspect, which will not be described again here.
  • Figure 2 is a schematic diagram of an exemplary encoding process
  • Figure 3 is a schematic diagram of an exemplary decoding process
  • Figure 5b is a schematic diagram of the encoding and decoding process
  • Figure 5c is a schematic diagram of the encoding and decoding process
  • Figure 6 is a schematic diagram of an exemplary encoding process
  • Figure 7b is a schematic diagram of the encoding and decoding process
  • Figure 7c is a schematic diagram of the encoding and decoding process
  • Figure 8 is a schematic diagram of an exemplary encoding process
  • Figure 9a is a schematic diagram of an exemplary decoding process
  • Figure 9b is a schematic diagram of the encoding and decoding process
  • Figure 10 is an exemplary encoding flow chart
  • Figure 11a is a schematic diagram of an exemplary decoding process
  • Figure 12 is a schematic structural diagram of an exemplary device.
  • first and second in the description and claims of the embodiments of this application are used to distinguish different objects, rather than to describe a specific order of objects.
  • first target object, the second target object, etc. are used to distinguish different target objects, rather than to describe a specific order of the target objects.
  • the first device and the second device include but are not limited to: PC (Personal Computer, personal computer), notebook computer, tablet computer, mobile phone, etc.
  • PC Personal Computer
  • notebook computer tablet computer
  • mobile phone etc.
  • the server can be a single server or a server cluster, which is not limited in this application.
  • the server After receiving the code stream corresponding to the current frame, the server forwards the code stream corresponding to the current frame to the corresponding second device.
  • the receiving module of the second device receives the code stream, and then sends the code stream to the video decoding module, which decodes the code stream to obtain the corresponding reconstructed image. Subsequently, the video decoding module sends the reconstructed image to the video display module, and the video display module displays the reconstructed image.
  • the video encoding module in Figure 1 is an encoder with poor ability to withstand weak networks (such as a hardware encoder, or other encoders produced with the development of technology), then when the network conditions deteriorate in the RTC scenario , the video on the second device may freeze, drop frames, etc.
  • weak networks such as a hardware encoder, or other encoders produced with the development of technology
  • This application considers that one of the reasons for the poor ability of hardware encoders to withstand weak networks is that hardware encoders do not provide an external interface for reference frame management, and most hardware encoders encode frame-by-frame reference, that is, the hardware encoder Reference frame management is inflexible.
  • this application can enhance the hardware encoder's ability to withstand weak networks by increasing the flexibility of the reference frame management of the hardware encoder, thereby ensuring the smoothness of the video when network conditions deteriorate in RTC scenarios.
  • the first device can create an external reference list that is independent of the encoder (that is, the external reference list is located outside the encoder to distinguish it from the reference list managed internally by the encoder); subsequently, when the encoder encodes During the process, the reconstructed image corresponding to the reference frame determined to be subsequently encoded in the encoded frame is added to the external reference list.
  • the current frame acquired by the first device may be an image of a person and/or an object.
  • a preset reference frame setting rule may be configured in advance, and the preset reference frame setting rule may be used to determine whether the current frame needs to be set as a reference frame for subsequent encoding; details will be described later. Furthermore, after each frame is encoded by the encoder, it can be determined according to the preset reference frame setting rules whether the encoded frame needs to be used as a reference frame for subsequent encoding. When it is determined that the encoded frame needs to be used as a reference frame for subsequent encoding, the reconstructed image corresponding to the encoded frame can be added to the external reference list for reference in subsequent encoding of other frames. In this way, the external reference list may include reconstructed images corresponding to one or more encoded frames.
  • the reference indication information may be determined according to the reference frame determination rule in the preset reference rule; where the reference indication information is used to indicate the reference frame corresponding to the current frame in the external reference list. Reconstruct the image. In this way, the reconstructed image corresponding to the reference frame of the current frame can be obtained from the external reference list according to the reference indication information. Then, S203 to S204 can be executed to implement encoding of the current frame.
  • S203 The encoder performs intra-frame coding on the reconstructed image.
  • the encoder performs inter-frame coding on the current frame to obtain the corresponding code stream.
  • an encoder with inflexible reference frame management it usually performs inter-frame coding with frame-by-frame reference. Therefore, the reconstructed image corresponding to the reference frame of the current frame can be input into the encoder, and the encoder performs intra-frame coding on the reconstructed image corresponding to the reference frame of the current frame to obtain a corresponding intra-frame coding result.
  • the result of intra-frame coding may include: an intra-coded code stream of the reconstructed image corresponding to the reference frame of the current frame, and an intra-coded reconstructed image of the reconstructed image corresponding to the reference frame of the current frame.
  • the encoder may perform intra-frame coding on the reconstructed image corresponding to the reference frame of the current frame with a quantization parameter smaller than the first preset threshold.
  • the first preset threshold can be set according to requirements.
  • the first preset threshold is a number less than 10, such as 2, 3, 4, 5, 6, 7, 8, and 9. This application does not limit this. In this way, high-quality intra-frame coding can be achieved for the reconstructed image corresponding to the reference frame of the current frame.
  • the encoder may perform intra-frame coding on the reconstructed image corresponding to the reference frame of the current frame at a code rate greater than the second preset threshold.
  • the second preset threshold can be set according to requirements, and this application does not limit this. In this way, high-quality intra-frame coding of the reconstructed image corresponding to the reference frame of the current frame can also be achieved.
  • the first device does not send the intra-frame coding stream to the server/second device.
  • the encoder can flexibly select reference frames from the external reference list for encoding, thereby enabling cross-frame reference or cross-resolution reference, which increases the flexibility of the encoder's reference frame management.
  • the second device can also establish an external reference list that is independent of the decoder (that is, the external reference list is located outside the decoder to distinguish it from the reference list managed internally by the decoder); subsequently, in During the decoding process of the decoder, the reconstructed image corresponding to the reference frame determined to be subsequently decoded in the decoded frame is added to the external reference list.
  • the reconstructed image corresponding to the matching reference frame can be selected from the external reference list and used as a reference for decoding; this enables the decoder to implement cross-frame reference decoding or cross-resolution reference decoding, which improves Decoder reference frame management flexibility. It then cooperates with the encoder to ensure the smoothness of the video when network conditions deteriorate in RTC scenarios.
  • Figure 3 is a schematic diagram of an exemplary decoding process.
  • the decoding process in the embodiment of FIG. 3 corresponds to the encoding process in the embodiment of FIG. 2 .
  • S302 Determine the reconstructed image corresponding to the reference frame of the current frame from the external reference list of the decoder.
  • the reference frame of the current frame is the frame that has been decoded by the decoder.
  • the external reference list is independent of the decoder.
  • the reference frame parameters of the current frame can be added to the code stream corresponding to the current frame; where the reference frame parameters are used to describe The current frame is associated with the reconstructed image corresponding to the reference frame in the external reference list.
  • the reference frame parameters include but are not limited to: the relevant description information of whether the current frame is used as a reference frame for subsequent encoding, the relevant description information of whether the current frame refers to the external reference list encoding, the relevant description information of whether the current frame is resampled, etc., This application does not limit this.
  • the code stream with the reference frame parameters added is sent to the second device.
  • the decoder in the second device can determine whether the decoded frame needs to be used as a reference frame based on the reference frame parameters parsed from the code stream corresponding to the decoded frame.
  • the reconstructed image corresponding to the decoded frame can be added to the external reference list; in this way, the external reference list contains the reconstruction of the reference frame image.
  • the reconstructed image corresponding to the decoded frame can be input to the encoder for intra-frame encoding to obtain the intra-frame encoding code stream; then, the frame of the reconstructed image corresponding to the decoded frame can be The intra-coded code stream is added to the external reference list; in this way, the external reference list contains the intra-coded code stream of the reference frame.
  • This application takes an external reference list containing a reconstructed image of a reference frame as an example for explanation.
  • the encoder can perform intra-frame coding on the reconstructed image corresponding to the decoded frame with a quantization parameter smaller than the first preset threshold to obtain the intra-frame coding corresponding to the reconstructed image of the decoded frame. code stream.
  • the first preset threshold can be set according to requirements.
  • the first preset threshold is a number less than 10, such as 2, 3, 4, 5, 6, 7, 8, and 9. This application does not limit this. In this way, high-quality intra-frame coding of the reconstructed image corresponding to the decoded frame can be achieved.
  • the encoder can perform intra-frame encoding on the reconstructed image corresponding to the decoded frame at a code rate greater than the second preset threshold, to obtain the intra-frame corresponding to the reconstructed image corresponding to the decoded frame.
  • the second preset threshold can be set according to requirements, and this application does not limit this. In this way, high-quality intra-frame coding can also be achieved for the reconstructed image corresponding to the decoded frame.
  • the quantization parameter used by the encoder for intra-frame encoding of the reconstructed image corresponding to the decoded frame may be smaller than the quantization parameter used by the encoder for intra-frame encoding of the I frame.
  • the code rate at which the encoder performs intra-frame coding on the reconstructed image corresponding to the decoded frame may be greater than the code rate at which the encoder performs intra-frame coding on the I frame.
  • the code stream corresponding to the current frame can be parsed to obtain the reference frame parameters; and then based on the reference frame parameters, the reference of the current frame is determined from the external reference list of the decoder. The reconstructed image corresponding to the frame.
  • the current frame can be determined according to the relevant description information of whether the current frame is encoded with reference to the external reference list in the reference frame parameter (for the decoding end, the description information is the relevant description information of whether the current frame is decoded with reference to the external reference list). Whether it is necessary to refer to an external reference list for decoding.
  • the code stream corresponding to the current frame can be directly input to the decoder, and the decoder decodes the code stream corresponding to the current frame to obtain the corresponding reconstructed image.
  • the decoder can perform intra-frame decoding on the code stream corresponding to the current frame.
  • the decoder can perform inter-frame decoding on the code stream corresponding to the current frame; among them, the decoder can perform inter-frame decoding on the code stream corresponding to the current frame based on the reconstructed image corresponding to the previous frame of the current frame. , to obtain the corresponding reconstructed image.
  • the reconstruction corresponding to the reference frame of the current frame in the external reference list can be determined again based on the relevant description information of whether the current frame is decoded with reference to the external reference list in the reference frame parameter. image; then execute S303 ⁇ S305:
  • S303 Perform intra-frame coding on the reconstructed image to obtain the corresponding intra-frame coding stream.
  • S304 The decoder performs intra-frame decoding on the intra-frame coded stream.
  • the decoder Based on the result of intra-frame decoding, the decoder performs inter-frame decoding on the code stream corresponding to the current frame to obtain a reconstructed image of the current frame.
  • the reconstructed image corresponding to the reference frame of the current frame determined from the external reference list can be input to the encoder for intra-frame coding to obtain the intra-frame coding of the reconstructed image corresponding to the reference frame of the current frame. code stream. Then, the intra-frame coded stream of the reconstructed image corresponding to the reference frame of the current frame can be input to the decoder, and the decoder decodes the intra-frame coded stream to obtain the reconstructed image corresponding to the reference frame of the current frame. Intra-frame coding of images reconstructs images.
  • the quantization parameter used by the encoder to perform intra-frame coding on the reconstructed image corresponding to the reference frame of the current frame can be less than the first preset threshold; or, the encoder performs intra-frame coding on the reconstructed image of the current frame at a code rate greater than the second preset threshold.
  • the reconstructed image corresponding to the reference frame is intra-frame encoded; for details, please refer to the above description and will not be described again here.
  • the second device after obtaining the intra-coded reconstructed image of the reconstructed image corresponding to the reference frame of the current frame, the second device does not display the intra-coded reconstructed image of the reconstructed image corresponding to the reference frame of the current frame; instead, After obtaining the reconstructed image corresponding to the current frame, the reconstructed image corresponding to the current frame is displayed.
  • the decoder in this embodiment may include a decoder with inflexible reference frame management (such as a hardware decoder, or other decoders produced with the development of technology), or may include a decoder with flexible reference frame management. decoder (such as a software decoder, or other decoders produced with the development of technology), this application does not limit this.
  • the current frame does not need to refer to the reference frame coding in the external reference list and does not need to be used as a reference frame for subsequent coding, there is no need to generate reference frame parameters, and there is no need to modify the code stream corresponding to the current frame. Subsequently, the code stream to be encoded corresponding to the current frame will be directly sent to the server or the second device.
  • the relevant description information of whether the current frame is encoded with reference to an external reference list may include: a first identifier and first indication information; the first identifier is used to identify whether the current frame is encoded based on an external reference list, or It can be used to identify whether the current frame refers to a long-term reference frame or a temporal hierarchical reference frame or a resampling reference frame encoding.
  • the first indication information is used to indicate the reconstructed image corresponding to the reference frame of the current frame in the external reference list.
  • the relevant description information of whether the current frame is resampled may include: a third identifier, a third indication information and a resampling resolution; the third identifier is used to identify whether the current frame is resampled.
  • the third indication information is used to indicate the resampled reconstructed image of the current frame in the external reference list, and the resampling resolution includes the resolution corresponding to the reconstructed image of the current frame and the resolution corresponding to the resampled reconstructed image.
  • reference frame parameters may also include other parameters, which are specifically determined according to requirements, and this application does not limit this.
  • the first identifier and the first indication information in the reference frame parameters can be added to the code stream corresponding to the current frame.
  • the second identification and the second indication information in the reference frame parameters can be added to the code stream corresponding to the current frame.
  • the third identifier, the third indication information and the resampling resolution in the reference frame parameters can be added to the code stream corresponding to the current frame.
  • the first identifier, the first indication information, the second identifier and the second indication information in the reference frame parameters can be added to the code stream corresponding to the current frame.
  • the first identifier, the first indication information, the third identifier, the third indication information and the resampling resolution in the reference frame parameters can be added to the code stream corresponding to the current frame.
  • At least one of the first identifier, the second identifier, and the third identifier in the reference frame parameters can be added to the code stream corresponding to the current frame.
  • the first identifier is used to identify that the current frame is not encoded based on the external reference list
  • the second identifier is used to identify that the current frame is not used as a reference frame for subsequent encoding
  • the third identifier is used to identify that the current frame is not resampled.
  • the reference frame parameters of the current frame can be added to the parameter set in the code stream corresponding to the current frame (such as VPS (Video Parameter Set, video parameter set), SPS (Sequence Parameter Set, sequence parameter set) ), PPS (Picture Parameter Set, image parameter set)).
  • VPS Video Parameter Set, video parameter set
  • SPS Sequence Parameter Set, sequence parameter set
  • PPS Picture Parameter Set, image parameter set
  • the reference frame parameters of the current frame are encapsulated in the header of the transmission protocol packet.
  • the reference frame parameters of the current frame are encapsulated in the extension header of RTP (Real-time Transport Protocol).
  • all the reference frame parameters can be encapsulated as payload data.
  • the code stream and parameter frame parameters corresponding to the current frame can be packaged into a file format (file format), and then encapsulate the file format according to the transmission protocol.
  • the file format may include the file format specified by the standard ISO14496-12.
  • the file format is the file format specified by the standard ISO14496-12, you can add the reference frame parameters to track hearer(), or add the box() corresponding to the reference frame parameters.
  • the file may also be in MPEG2TS file format. It should be understood that this application imposes no restrictions on file format.
  • the coding and decoding scenarios provided by this application may include a variety of coding and decoding scenarios, such as long-term reference frame coding and decoding scenarios, time-domain hierarchical coding and decoding scenarios, and reference frame resampling scenarios.
  • This application will No restrictions. The following describes the encoding and decoding methods in scenarios using long-term reference frame encoding and decoding, time domain hierarchical encoding and decoding scenarios, and reference frame resampling scenarios respectively.
  • Figure 4 is a schematic diagram of an exemplary encoding process.
  • the preset reference rule is a long-term reference rule
  • the preset reference frame setting rule is a long-term reference frame setting rule.
  • long-term reference rules can be set according to the reference period of the long-term reference frame, packet loss feedback information, application scenarios, etc., and this application does not limit this.
  • Exemplary long-term reference rules may also include: reference judgment rules and reference frame determination rules. When it is determined that the reference judgment rule in the long-term reference rule is satisfied, it can be determined that the long-term reference rule is satisfied; at this time, it can be explained that the current frame needs to be encoded with reference to the long-term reference frame, and S403 to S406 can be executed.
  • long-term reference rules are set according to the reference period of the long-term reference frame.
  • the reference judgment rule in the long-term reference rule can be set as: reaching the reference period of the long-term reference frame (which can refer to the coded frame closest to the current frame and referring to the long-term reference frame, the distance difference to the current frame, reaches the reference period of the long-term reference frame); set the reference frame determination rule in the long-term reference rule to: the long-term reference frame closest to the current frame in the external reference list.
  • the reference period of the long-term reference frame can be set as required, such as 5 frames, and this application does not limit this. Assume that a GOP (Group Of Picture, picture group) is used as an example.
  • GOP Picture Order Count, picture sequence number
  • long-term reference rules can be set based on packet loss feedback information. For example, you can set the reference judgment rule in the long-term reference rule to: the code stream corresponding to the preset reference frame of the current frame is lost; set the reference frame determination rule in the long-term reference rule to: the closest distance to the current frame in the external reference list long-term reference frame.
  • the preset reference frame may refer to a reference frame pre-specified for the current frame before encoding the current frame.
  • long-term parameters can be The reference judgment rules in the test rules are set to: the current application scene is a desktop sharing scene, and there is a reference frame in the external reference list that is similar to the current frame higher than the similarity threshold; the reference frame determination rules in the long-term reference rules are set is: the long-term reference frame in the external reference list that is most similar to the current frame.
  • the similarity threshold can be set according to requirements, and this application does not limit this.
  • the reconstructed image corresponding to the encoded frame when it is determined to be a reference frame for subsequent encoding, can be added to the external reference list, and the reconstructed image corresponding to the encoded frame can be added to the external reference list.
  • the corresponding reconstructed image is assigned a frame identifier.
  • the frame identifier can be used to uniquely identify a reconstructed image in the external reference list, and can be determined according to requirements, such as frame serial number, POC, PTS (Presentation Time Stamp), etc. This application does not limit this.
  • an identifier that uniquely specifies the reconstructed image corresponding to the encoded frame may also be calculated according to preset calculation rules (such as the setting period of the long-term reference frame and the distance to the nearest I frame, etc.).
  • the frame identifier of the reconstructed image corresponding to the long-term reference frame of the current frame can be determined according to the reference frame determination rule in the long-term reference rule, and then the frame identifier is determined as the reference indication information.
  • the frame identifier of the reconstructed image corresponding to the long-term reference frame closest to the current frame in the external reference list can be , as reference information.
  • the reference frame determination rule in the long-term reference rule is: the long-term reference frame with the highest similarity to the current frame in the external reference list
  • the long-term reference frame corresponding to the long-term reference frame with the highest similarity to the current frame in the external reference list can be reconstructed.
  • the frame identifier of the image used as reference indication information.
  • S405 The encoder performs intra-frame coding on the reconstructed image corresponding to the long-term reference frame of the current frame.
  • S405 to S406 may refer to the description of S302 to S304 above, which will not be described again here.
  • the encoder When it is determined that the long-term reference rule is not satisfied, the encoder performs inter-frame coding on the current frame based on the reconstructed image corresponding to the previous frame of the current frame to obtain a code stream corresponding to the current frame.
  • the current frame can be directly input to the encoder. Then, the encoder can perform inter-frame coding on the current frame based on the reconstructed image corresponding to the previous frame of the current frame to obtain the code stream corresponding to the current frame.
  • long-term reference frame setting rules can be set according to the application scenario.
  • the long-term reference frame setting rule can be set as follows: the application scene is a desktop sharing scene, and the difference from the previous frame is greater than the difference threshold; the difference threshold can be set according to requirements, and this application does not limit this.
  • the difference threshold is 0.6.
  • the difference between the current frame and the previous frame is greater than or equal to 0.6, it can be determined that the long-term reference frame setting rules are met, that is, it is determined that the current frame needs to be used as a long-term reference frame for subsequent encoding. .
  • the reconstructed image of the current frame can be added to the external reference list, and a corresponding frame identifier can be set for the reconstructed image of the current frame.
  • the reference frame parameters can be added to the code stream corresponding to the current frame.
  • the first identifier in the reference frame parameter may be used to identify whether the current frame is encoded with reference to the long-term reference frame, and the first indication information may be used to indicate the long-term reference frame corresponding to the current frame in the external reference list.
  • the second identification may be used to identify whether the current frame is used as a long-term reference frame for subsequent encoding, and the second indication information is used to indicate the reconstructed image of the current frame in the external reference list.
  • the first identifier can be assigned to the first preset information (such as "A1"), and the first An indication information is assigned a frame identifier corresponding to the reconstructed image of the current frame in the external reference list.
  • the first identifier may be configured as second preset information (such as "A0"), and the first indication information may be assigned a value of "NULL".
  • Figure 5a is a schematic diagram of an exemplary decoding process. Described in the embodiment of Figure 5a is the scene of long-term reference frame decoding, which corresponds to the encoding process in the embodiment of Figure 4.
  • S503 Based on the reference frame parameters, determine whether the current frame needs to be decoded with reference to the long-term reference frame.
  • the reconstructed image corresponding to the long-term reference frame of the current frame is determined from the external reference list of the decoder based on the reference frame parameters.
  • the long-term reference frame in the external reference list is the Decoded frame.
  • the reconstructed image matching the first indication information in the reference frame parameter is determined from the external reference list of the decoder as the long-term reference frame corresponding to the current frame. reconstructed image.
  • S505 Perform intra-frame coding on the reconstructed image to obtain the corresponding intra-frame coding stream.
  • S506 The decoder performs intra-frame decoding on the intra-frame encoded code stream.
  • the decoder Based on the intra-frame coded reconstructed image obtained by decoding the intra-frame coded code stream, the decoder performs inter-frame decoding on the code stream corresponding to the current frame to obtain the reconstructed image of the current frame.
  • S505 to S507 may refer to the description of S303 to S305 above, which will not be described again here.
  • the code stream corresponding to the current frame can be directly input to the decoder, and then the decoder can reconstruct the image based on the previous frame of the current frame.
  • the code stream corresponding to the current frame is inter-frame decoded to obtain the reconstructed image of the current frame.
  • the current frame needs to be used as a long-term reference frame for subsequent decoding. For example, it can be determined whether the reference frame parameter includes the second identifier; when the reference frame parameter includes the second identifier, it can be determined whether the value of the second identifier is the third preset information. When the value of the second identifier is the third preset information, it can be determined that the current frame needs to be used as a long-term reference frame for subsequent decoding, and S509 can be performed at this time. When the value of the second identifier is the fourth preset information, or the reference frame parameters do not include the second identifier, it can be determined that the current frame does not need to be used as The long-term reference frame for subsequent decoding can end the decoding process.
  • the reconstructed image of the current frame can be added to the external reference list, and a corresponding frame identifier can be set for the reconstructed image of the current frame.
  • Figure 5b is a schematic diagram of an exemplary encoding and decoding process.
  • the encoder in the first device is a hardware encoder
  • the decoder in the second device is a hardware decoder.
  • the gray and black frames in the left area are frames sent by the first device
  • the gray and black frames in the right area are frames received by the second device; among them, the black frames are long-term reference frames. .
  • the long-term reference frame setting rule is: the setting period of the long-term reference frame is reached, the setting period of the long-term reference frame is 4; the long-term reference rule is : The reference period of the long-term reference frame is reached, and the reference period of the long-term reference frame is 5.
  • the I frame can be intra-frame encoded to obtain an I frame code stream, and then the I frame code stream is sent to the second device.
  • the current frame is the first P frame.
  • the hardware encoder can encode the first P frame with reference to the I frame reconstructed image to obtain the code stream of the first P frame, and then convert the code stream of the first P frame sent to the second device.
  • the current frame is the second P frame.
  • the hardware encoder can encode the second P frame with reference to the reconstructed image of the first P frame to obtain the code stream of the second P frame, and then encode the second P frame.
  • the code stream of the frame is sent to the second device.
  • the current frame is the third P frame.
  • the hardware encoder can encode the third P frame with reference to the reconstructed image of the second P frame to obtain the code stream of the third P frame.
  • the code stream of the third P frame can be input to the decoder to obtain the reconstructed image of the third P frame output by the decoder, and then the third P frame can be The reconstructed image is stored in the external reference list. Then, a second identifier can be added to the code stream of the third P frame, and the second identifier can be assigned as the third preset information, and the second indication information (such as 3) can be added, and then the third reference frame parameter can be added.
  • the code stream of P frames is sent to the second device. At time t4, the current frame is the 4th P frame.
  • the code stream of P frames is sent to the second device.
  • the current frame is the fifth P frame.
  • the hardware encoder can encode the fifth P frame with reference to the reconstructed image of the fourth P frame to obtain the code stream of the fifth P frame, and then encode the fifth P frame.
  • the code stream of the frame is sent to the second device.
  • the code stream of the I frame can be intra-frame decoded to obtain the reconstructed image of the I frame, and then the reconstructed image of the I frame is sent to the display module for display.
  • the current frame is the first P frame.
  • the hardware decoder can refer to the I frame reconstructed image to decode the code stream of the first P frame to obtain the reconstructed image of the first P frame, and then convert the first P frame into The reconstructed image is sent to the display module.
  • the current frame is the second P frame.
  • the hardware decoder can refer to the reconstructed image of the first P frame to decode the code stream of the second P frame to obtain the reconstructed image of the second P frame, and then convert the The reconstructed images of 2 P frames are sent to the display module.
  • the current frame is the third P frame.
  • the hardware decoder can refer to the reconstructed image of the second P frame to decode the code stream of the third P frame to obtain the reconstructed image of the third P frame, and then convert the The reconstructed images of 3 P frames are sent to the display module.
  • the reconstructed image of the third P frame can be stored in the external reference list according to the second identification and the second indication information.
  • the current frame is the 4th P frame.
  • the reconstructed image of the 3rd P frame can be obtained from the external reference list according to the first identification and the first indication information; then the reconstructed image of the 3rd P frame is input
  • the encoder can perform intra-frame coding on the reconstructed image of the third P frame, and can obtain the intra-frame coding code stream of the reconstructed image of the third P frame.
  • the encoder sends the intra-coded code stream of the reconstructed image of the third P frame to the hardware decoder, and the hardware decoder can decode and obtain the intra-frame decoded reconstructed image of the reconstructed image of the third P frame (not sent).
  • the code stream of the fourth P frame is input to the hardware decoder, and the hardware decoder reconstructs the image based on the intra-frame decoding of the reconstructed image of the third P frame, and decodes the code stream of the fourth P frame.
  • the hardware decoder reconstructs the image based on the intra-frame decoding of the reconstructed image of the third P frame, and decodes the code stream of the fourth P frame.
  • Obtain the reconstructed image of the 4th P frame and then send the reconstructed image of the 4th P frame to the display module.
  • the current frame is the fifth P frame.
  • the hardware decoder can refer to the reconstructed image of the fourth P frame to decode the code stream of the fifth P frame to obtain the reconstructed image of the fifth P frame, and then convert the The reconstructed images of 5 P frames are sent to the display module.
  • the first device can modify the code stream corresponding to the current frame to identify the standard syntax elements related to the reference frame of the current frame; where the standard syntax elements refer to Syntax elements specified in standard encoding protocols.
  • the standard syntax elements used to identify the reference frame of the current frame that need to be modified include at least one of the following: syntax used to identify the POC value of the current frame. Element, a syntax element used to identify the number of long-term reference frames; a syntax element used to identify the long-term reference frame image sequence number POC; a syntax element used to identify whether the long-term reference frame is used for the current frame reference.
  • syntax used to identify the POC value of the current frame a syntax element used to identify the number of long-term reference frames
  • a syntax element used to identify the long-term reference frame image sequence number POC a syntax element used to identify whether the long-term reference frame is used for the current frame reference.
  • other standard syntax elements used to identify the reference frame related to the current frame may also be included, and this application does not limit this.
  • the standard syntax elements used in the code stream corresponding to the current frame and related to the reference frame of the current frame can be as shown in Table 1:
  • the encoder performs intra-frame encoding on the reconstructed image corresponding to the reference frame of the current frame, the POC in the obtained intra-frame encoding code stream is 0; therefore, the encoder encodes the current frame to obtain The POC in the code stream is 1.
  • the image sequence number POC in the code stream corresponding to the current frame can be modified to 5, that is, the value of pic_order_cnt_lsb is modified from 0 to 5; so that the POC in the code stream corresponding to the current frame is the same as the The POC in the code stream corresponding to the previous frame of the current frame is continuous.
  • the long-term reference frame of the current frame is frame 4.
  • the decoding process in Figure 5c is similar to the decoding process in Figure 5b except for the decoding process of the fourth P frame.
  • the decoding process of the other frames is similar and will not be described again here.
  • the second device can refer to the reconstructed image of the second P frame for decoding to obtain the reconstructed image of the fourth P frame.
  • Figure 6 is a schematic diagram of an exemplary encoding process.
  • the preset reference rules in the embodiment of FIG. 6 are time domain hierarchical reference rules
  • the preset reference frame setting rules are time domain hierarchical reference frame setting rules.
  • S602 Determine whether the time domain hierarchical reference rules are met.
  • the number of layers is 2.
  • the corresponding reference mode is reference mode 1: the third frame refers to the coding of the first frame, the fifth frame refers to the coding of the third frame, the seventh frame refers to the coding of the fifth frame; the second frame refers to the coding of the first frame, and the fourth frame refers to the coding of the first frame.
  • 3-frame encoding the 6th frame refers to the 5th frame encoding, and the 8th frame refers to the 7th frame encoding.
  • you can set the reference judgment rule in the time domain hierarchical reference rule as: time domain hierarchical structure 1, and the POC value is an integer other than 8 ⁇ (n-1) (the value of n is 0, 1,2,...);
  • the reference frame determination condition is: reference mode 1.
  • the corresponding reference mode is reference mode 2: the second frame refers to the coding of the first frame, the third frame refers to the coding of the first frame, the fourth frame refers to the coding of the third frame; the fifth frame refers to the coding of the first frame, and the sixth frame refers to the coding of the first frame.
  • 5 frame coding the 7th frame refers to the 5th frame coding, the 8th frame refers to the 7th frame coding;
  • the 9th frame refers to the 5th frame coding, the 10th frame refers to the 9th frame coding, the 11th frame refers to the 9th frame coding, 12 frames are encoded with reference to frame 11.
  • the reference judgment rule in the time domain hierarchical reference rule is: time domain hierarchical structure 2, and the POC value is an integer other than 12 ⁇ (n-1) (the value of n is 0, 1,2,...); the reference frame determination condition is: reference mode 2.
  • the time domain hierarchical structure can be preset, that is, the time domain hierarchical structure is a fixed value.
  • the time domain hierarchical structure can be determined based on channel feedback information; for example, when the channel quality is determined to be better based on the channel feedback information, the number of time domain layers can be increased; when the channel quality is determined to be better based on the channel feedback information, When the difference is different, the number of time domain layers can be reduced, and this application does not limit this.
  • the temporal hierarchical structure is dynamic.
  • time domain hierarchical reference rule when the time domain hierarchical reference rule is satisfied, it is determined that the current frame needs to be encoded with reference to the time domain hierarchical reference frame; at this time, S603 to S606 can be performed.
  • S607 When the time domain hierarchical reference rules are not satisfied, it is determined that the current frame does not need to be coded with reference to the time domain hierarchical reference frame, and S607 can be performed.
  • the frame identifier of the reconstructed image corresponding to the temporal hierarchical reference frame of the current frame can be determined according to the reference frame determination condition in the temporal hierarchical reference rule, and then The frame identification is determined as reference indication information.
  • the reference frame determination condition in the time domain hierarchical reference rule is reference mode 1
  • the first frame in the GOP can be determined as its reference frame; at this time, the external In the reference list, the frame identifier of the reconstructed image of the first frame serves as the reference indication information.
  • the reference frame determination condition in the time domain hierarchical reference rule is reference mode 2
  • the 3rd frame in the GOP can be determined as its reference frame; at this time, the external In the reference list, the frame identifier of the reconstructed image of the third frame serves as the reference indication information.
  • the reference indication information can be combined with the frame of the reconstructed image corresponding to each reference frame in the external reference list. Identify matching to obtain the reconstructed image corresponding to the time domain hierarchical reference frame of the current frame from the external reference list of the encoder.
  • the encoder performs intra-frame coding on the reconstructed image corresponding to the temporal hierarchical reference frame of the current frame.
  • S606 Based on the intra-frame coding of the reconstructed image corresponding to the time-domain hierarchical reference frame of the current frame, the encoder performs inter-frame coding on the current frame to obtain a code stream corresponding to the current frame.
  • S605 to S606 may refer to the description of S302 to S304 above, which will not be described again here.
  • the encoder When it is determined that the temporal hierarchical reference rule is not satisfied, the encoder performs inter-frame coding on the current frame based on the reconstructed image corresponding to the previous frame of the current frame to obtain a code stream corresponding to the current frame.
  • S607 may refer to the description of S407 above, which will not be described again here.
  • time domain hierarchical setting rules can be set according to the time domain hierarchical structure. Furthermore, after the encoding of the current frame is completed, it can be determined according to the time domain hierarchical setting rules whether the current frame needs to be used as a time domain hierarchical reference frame for subsequent encoding. When it is determined that the time domain hierarchical setting rules are met, it is determined that the current frame needs to be used as a time domain hierarchical reference frame for subsequent encoding, and the reconstructed image of the current frame can be added to the external reference list of the encoder. When it is determined that the temporal hierarchical setting rules are not satisfied, it is determined that the current frame does not need to be used as a temporal hierarchical reference frame for subsequent encoding. At this time, there is no need to add the reconstructed image of the current frame to the external reference list of the encoder, and it can be ended. Coding process.
  • the time domain hierarchical setting rule may be: time domain hierarchical structure 1, and the POC value is an integer other than 8 ⁇ (n-1).
  • the time domain hierarchical structure is the above-mentioned time domain hierarchical structure 1
  • the current frame is the 1st frame or the 3rd frame or the 5th frame or the 7th frame
  • it can be determined that the time domain hierarchical setting rules are satisfied that is, The current frame needs to be used as a time domain hierarchical reference frame for subsequent coding.
  • the reference frame parameters can be added to the code stream corresponding to the current frame.
  • the first identifier in the reference frame parameter can be used to identify whether the current frame refers to the time domain hierarchical reference frame frame encoding, and the first indication information can be used to indicate the time domain of the current frame in the external reference list.
  • the second identification may be used to identify whether the current frame is used as a temporal hierarchical reference frame for subsequent encoding, and the second indication information is used to indicate the reconstructed image of the current frame in the external reference list.
  • the first identifier when it is determined that the time domain hierarchical reference rules are met and the current frame has been encoded based on the time domain hierarchical reference frame in the external reference list, the first identifier can be assigned to the fifth preset information (such as "C1 ”), and assign the first indication information to the frame identifier corresponding to the reconstructed image of the current frame in the external reference list.
  • the first identifier can be configured as the sixth preset information (such as "C0"), and the first indication information can be assigned a value of "NULL".
  • the second identifier can be configured as the seventh preset information (such as "D1"), and assign the second indication information as the frame identifier corresponding to the reconstructed image corresponding to the time domain hierarchical reference frame of the current frame in the external reference list.
  • the second label can be The identification is configured as the eighth preset information (such as "D0"), and the second indication information is assigned a value of "NULL".
  • the value of the first identifier is the sixth preset information, or the reference frame parameter does not include the first identifier, it is determined that the current frame does not need to be decoded with reference to the time domain hierarchical reference frame, and S708 can be performed at this time.
  • S705 Perform intra-frame coding on the reconstructed image to obtain the corresponding intra-frame coding stream.
  • the decoder Based on the intra-frame coded reconstructed image obtained by decoding the intra-frame coded code stream, the decoder performs inter-frame decoding on the code stream corresponding to the current frame to obtain the reconstructed image of the current frame.
  • S705 to S707 may refer to the description of S303 to S305 above, which will not be described again here.
  • the decoder when it is determined that the current frame does not need to be decoded with reference to the time domain hierarchical reference frame, based on the reconstructed image corresponding to the previous frame of the current frame, the decoder performs inter-frame decoding on the code stream corresponding to the current frame to obtain the current frame. Reconstruct the image.
  • the code stream corresponding to the current frame can be directly input to the decoder, and then the decoder reconstructs the image corresponding to the previous frame of the current frame. , decode the code stream corresponding to the current frame to obtain the reconstructed image of the current frame.
  • the current frame needs to be used as a time domain hierarchical reference frame for subsequent decoding. For example, it can be determined whether the reference frame parameter includes the second identifier; when the reference frame parameter includes the second identifier, it can be determined whether the value of the second identifier is the seventh preset information. When the value of the second identifier is the seventh preset information, it can be determined that the current frame needs to be used as a time domain hierarchical reference frame for subsequent decoding, and S709 can be performed at this time.
  • the decoding process can be ended.
  • the reconstructed image of the current frame can be added to the external reference list, and a corresponding frame identifier can be set for the reconstructed image of the current frame.
  • the reconstructed image of the current frame can also be intra-frame coded to obtain the intra-frame code stream corresponding to the reconstructed image of the current frame; and then the intra-frame code stream corresponding to the reconstructed image of the current frame can be obtained.
  • the code stream is added to the external reference list; for details, please refer to the above description and will not be repeated here.
  • Figure 7b is a schematic diagram of an exemplary encoding and decoding process.
  • the encoder in the first device is a hardware encoder
  • the decoder in the second device is a hardware decoder.
  • the gray and black frames in the left area are frames sent by the first device
  • the gray and black frames in the right area are frames received by the second device; among them, the black frames are time domain division Layer reference frame.
  • a second identifier can be added to the code stream of the I frame, and the second identifier can be assigned as third preset information, and second indication information (such as 0) can be added, and then the reference frame parameter can be added to the code stream of the I frame. sent to the second device.
  • the current frame is the first P frame.
  • the hardware encoder can encode the first P frame with reference to the reconstructed image of the I frame to obtain the code stream of the first P frame, and then convert the code stream of the first P frame into The stream is sent to the second device.
  • the current frame is the second P frame.
  • the hardware encoder can obtain the reconstructed image of the I frame from the external reference list, and then perform intra-frame coding on the reconstructed image of the I frame, and obtain the reconstructed image of the I frame. Intra-coded code stream (not sent), and intra-coded reconstructed image of I-frame reconstructed image. Then the second P frame is input to the hardware encoder.
  • the hardware encoder refers to the intra-frame coding of the reconstructed image of the I frame to reconstruct the image, and encodes the second P frame to obtain the code stream of the second P frame. .
  • the identifier is assigned the third preset information, and the second indication information (such as 2) is added; and then the code stream of the second P frame with the reference frame parameter added is sent to the second device.
  • the current frame is the third P frame.
  • the hardware encoder can encode the third P frame with reference to the reconstructed image of the second P frame to obtain the code stream of the third P frame, and then encode the third P frame.
  • the code stream of the frame is sent to the second device.
  • the current frame is the 4th P frame.
  • the hardware encoder can obtain the reconstructed image of the 3rd P frame from the external reference list, and then the hardware encoder can perform intra-frame encoding on the reconstructed image of the 3rd P frame. , the intra-coded code stream (not sent) of the reconstructed image of the third P frame can be obtained, as well as the intra-coded reconstructed image of the reconstructed image of the third P frame. Then the fourth P frame is input to the hardware encoder.
  • the hardware encoder refers to the intra-frame coding of the reconstructed image of the third P frame to reconstruct the image, and encodes the fourth P frame to obtain the fourth P frame. code stream.
  • the current frame is the 5th P frame.
  • the hardware encoder can encode the 5th P frame with reference to the reconstructed image of the 4th P frame to obtain the 5th P frame. code stream, and then sends the code stream of the fifth P frame to the second device.
  • the hardware decoder can perform intra-frame decoding on the code stream of the I frame to obtain I Then, on the one hand, the reconstructed image of the I frame is sent to the display module for display; on the other hand, the reconstructed image of the I frame can be stored in the external reference list according to the second identification and the second instruction information.
  • the current frame is the first P frame.
  • the hardware decoder can refer to the I frame reconstructed image to decode the code stream of the first P frame to obtain the reconstructed image of the first P frame, and then convert the first P frame into The reconstructed image is sent to the display module.
  • the current frame is the second P frame.
  • the reconstructed image of the I frame can be obtained from the external reference list according to the first identification and the first indication information; then the reconstructed image of the I frame is input to the encoder.
  • the encoder The reconstructed image of the I frame can be intra-frame encoded, and the intra-frame encoding code stream of the reconstructed image of the I frame can be obtained.
  • the encoder sends the intra-frame coded stream of the reconstructed image of the I frame to the hardware decoder, and the hardware decoder can decode the intra-frame decoded reconstructed image of the reconstructed image of the I frame (not sent). Then the code stream of the second P frame is input to the hardware decoder.
  • the hardware decoder reconstructs the image based on the intra-frame decoding of the reconstructed image of the I frame, and decodes the code stream of the second P frame to obtain the second P frame. Then, on the one hand, the reconstructed image of the second P frame is sent to the display module; on the other hand, the reconstructed image of the second P frame can be stored according to the second identification and the second instruction information. to the external reference list. At time t3, the current frame is the third P frame.
  • the hardware decoder can refer to the reconstructed image of the second P frame to decode the code stream of the third P frame to obtain the reconstructed image of the third P frame, and then convert the The reconstructed images of 3 P frames are sent to the display module.
  • the code stream of the fourth P frame is input to the hardware decoder, and the hardware decoder reconstructs the image based on the intra-frame decoding of the reconstructed image of the third P frame, and decodes the code stream of the fourth P frame.
  • the hardware decoder reconstructs the image based on the intra-frame decoding of the reconstructed image of the third P frame, and decodes the code stream of the fourth P frame.
  • Obtain the reconstructed image of the 4th P frame and then, on the one hand, send the reconstructed image of the 4th P frame to the display module; on the other hand, according to the second identification and the second instruction information, the 4th P frame can be
  • the reconstructed image is stored in the external reference list.
  • the current frame is the fifth P frame.
  • the hardware decoder can refer to the reconstructed image of the fourth P frame to decode the code stream of the fifth P frame to obtain the reconstructed image of the fifth P frame, and then convert the The reconstructed images of 5 P frames are sent
  • the first device can modify the standard syntax elements in the code stream corresponding to the current frame that are used to identify the reference frame of the current frame; where the standard syntax elements refer to Syntax elements specified in standard encoding protocols.
  • the standard syntax elements for identifying the reference frame of the current frame that need to be modified include at least one of the following: syntax for identifying the POC value of the current frame.
  • a syntax element used to identify whether to use a forward reference frame a syntax element used to identify the number of forward reference frames; a syntax element used to identify the distance between the current frame and the forward reference frame; a syntax element used to identify whether the forward reference frame Syntax element used for the current frame reference.
  • other standard syntax elements used to identify the reference frame related to the current frame may also be included, and this application does not limit this.
  • the corresponding reference mode is reference mode 1.
  • the third frame refers to the coding of the first frame
  • the fifth frame refers to the coding of the third frame
  • the seventh frame refers to the coding of the fifth frame
  • the second frame refers to the coding of the first frame
  • the fourth frame refers to the coding of the first frame.
  • 3-frame encoding the 6th frame refers to the 5th frame encoding
  • the 8th frame refers to the 7th frame encoding.
  • the encoder performs intra-frame coding on the reconstructed image corresponding to the reference frame of the current frame (i.e., the 5th frame), the resulting intra-frame coding stream
  • the POC is 0; therefore, the code stream obtained by the encoder encoding the current frame has a POC of 1.
  • the image sequence number POC in the code stream corresponding to the current frame can be modified to 5, that is, the value of pic_order_cnt_lsb is modified from 0 to 5; so that the POC in the code stream corresponding to the current frame is the same as the The POC in the code stream corresponding to the previous frame of the current frame is continuous.
  • Figure 7c is a schematic diagram of an exemplary encoding and decoding process.
  • the encoder in the first device of Figure 7c(1) is hardware
  • the encoder and the decoder in the second device are hardware codecs or software decoders.
  • the gray and black frames in the left area are frames sent by the first device
  • the gray and black frames in the right area are frames received by the second device; among them, the black frames represent the time domain division Layer reference frame.
  • the encoding process of Figure 7c is similar to the encoding process of Figure 7b, except that the encoding process of the I frame, the second P frame and the fourth P frame is different.
  • the encoding process of the other frames is similar and will not be repeated here.
  • Figure 7c there is no need to add reference frame parameters in the code streams of the I frame, the 2nd P frame, and the 4th P frame. Instead, it is necessary to modify the code streams of the 2nd P frame and the 4th P frame. grammatical elements.
  • the decoding process in Figure 7c is similar to the decoding process in Figure 5c except for the decoding process of the second P frame and the fourth P frame.
  • the decoding process of the other frames is similar and will not be described again here.
  • the second device After receiving the code stream of the second P frame, the second device can refer to the reconstructed image of the I frame for decoding to obtain the reconstructed image of the second P frame.
  • the second device After receiving the code stream of the fourth P frame, the second device can refer to the reconstructed image of the second P frame for decoding to obtain the reconstructed image of the fourth P frame.
  • FIG. 8 is a schematic diagram of an exemplary encoding process.
  • the preset reference rule in the embodiment of FIG. 8 is a resampling reference rule
  • the preset reference frame setting rule is a resampling setting rule.
  • the resampling reference rule can be set according to the resampling reference period, the resampling situation of the previous frame, etc., and this application does not limit this.
  • Exemplary resampling reference rules may also include: reference judgment rules and reference frame determination rules.
  • the resampling reference rule is set according to the resampling reference period.
  • the reference judgment rule in the resampling reference rule can be set as follows: reaching the resampling reference period (which can refer to the coded frame closest to the current frame and referring to the resampling reference frame, the distance difference to the current frame, The reference period of resampling is reached); the reference frame determination rule in the resampling reference rule can be set to: the resampling reference frame closest to the current frame in the external reference list.
  • the reference period of the resampled reference frame can be set to 10 frames as required, and this application does not limit this.
  • the reference judgment rule in the resampling reference rule can be set to: the previous frame of the current frame has been resampled; the reference frame determination rule in the resampling reference rule can be set to: the closest distance to the current frame in the external reference list Resample reference frame.
  • the resampling reference rule when the resampling reference rule is satisfied, it is determined that the current frame needs to be encoded with reference to the resampling reference frame; at this time, S803 to S806 can be performed.
  • the resampling reference rule is not satisfied, it is determined that the current frame does not need to be coded with reference to the resampling reference frame, and S807 can be executed.
  • the frame identifier of the reconstructed image corresponding to the resampling reference frame of the current frame can be determined according to the reference frame determination rule in the resampling reference rule, and then the frame identifier is determined as Refer to the instructions.
  • the reference frame determination rule in the resampling reference rule is: the resampling reference frame closest to the current frame in the external reference list
  • the image can be reconstructed corresponding to the resampling reference frame closest to the current frame in the external reference list
  • the frame identification is used as reference indication information.
  • the reference indication information can be matched with the frame identifier of the reconstructed image corresponding to each reference frame in the external reference list to obtain the resampled reference frame corresponding to the current frame from the external reference list of the encoder. Reconstruct the image.
  • the encoder performs intra-frame coding on the reconstructed image corresponding to the resampled reference frame of the current frame.
  • S805 to S806 may refer to the description of S302 to S304 above, which will not be described again here.
  • S807 may refer to the description of S407 above, which will not be described again here.
  • the resampling setting rules may be set according to the setting period of the resampling reference frame (which may also be called the resampling period), channel quality, etc., and this application does not limit this. Furthermore, after the encoding of the current frame is completed, it can be determined whether the current frame needs to be used as a resampling reference frame for subsequent encoding (that is, whether the current frame needs to be resampled) by determining whether the resampling setting rules are met.
  • the current frame needs to be used as a resampling reference frame for subsequent encoding (or the current frame needs to be resampled), and S809 can be performed.
  • the current frame does not need to be used as a resampling reference frame for subsequent encoding (or the current frame does not need to be resampled).
  • the encoding process can be ended.
  • the setting rule of the resampling reference frame is set according to the setting period of the resampling reference frame.
  • the resampling setting rule may be: reaching the setting period of the resampling reference frame (which may refer to the resampling reference frame closest to the current frame, the distance difference to the current frame, reaching the setting period of the resampling).
  • the setting period of the resampling reference frame may be a preset fixed value such as 9; this application does not limit this. Assume that the setting period of the long-term reference frame is 9. If the current frame is the 9th frame, it can be determined that the setting period of the resampling reference frame has been reached. At this time, it can be determined that the resampling setting rules are met, that is, it is determined that the current frame needs to be used as the subsequent The encoded resampled reference frame (or the current frame needs to be resampled).
  • the setting rules for resampling reference frames can be set according to channel quality.
  • the long-term reference frame setting rules may include: the channel quality change degree is greater than the first threshold, or the channel quality change degree is lower than the second threshold; where the first threshold is a positive number and the second threshold is a negative number, which can be set according to requirements. , this application There are no restrictions on this. For example, if the detected channel quality change degree is greater than the first threshold, it means that the channel quality has become better. At this time, the channel can carry a larger code rate, so the resolution of the current frame can be increased, and then the settings that satisfy the resampling reference frame can be determined.
  • Rule that is, determining that the current frame needs to be resampled (or that the current frame needs to be resampled). If the detected channel quality change degree is less than the second threshold, it means that the channel quality has deteriorated. At this time, the channel can carry a smaller code rate, so the resolution of the current frame can be reduced, and it can be determined that the resampling setting rules are met. Otherwise, it is determined that the resampling setting rules are met, that is, the current frame does not need to be used as a resampling reference frame for subsequent encoding (or the current frame does not need to be resampled).
  • the corresponding resampling target resolution can also be set.
  • the resampling setting rule is: when the setting period of the resampling reference frame is reached, the corresponding resampling target resolution can be set to: 1720*920.
  • the resampling setting rule is: if the channel quality change degree is greater than the first threshold, the corresponding resampling target resolution can be set to: 1720*1080.
  • the resampling setting rule is: the channel quality change degree is greater than the first threshold, and the corresponding resampling target resolution can be set to: 960*480.
  • the reference frame parameters can be added to the code stream corresponding to the current frame.
  • the first identifier in the reference frame parameter may be used to identify whether the current frame is coded with reference to the resampling reference frame, and the first indication information may be used to indicate where the resampling reference frame of the current frame in the external reference list is.
  • the third identifier is used to identify whether the current frame is resampled, and the third indication information is used to indicate the resampled reconstructed image of the current frame in the external reference list.
  • the resampling resolution includes the reconstructed image corresponding to the current frame. The resolution and the resolution corresponding to the resampled reconstructed image.
  • the first identifier when it is determined that the resampling reference rules are met and the current frame has been coded based on the reference resampling reference frame, the first identifier can be assigned to the ninth preset information (such as "E1"), and the first indication can be The information assignment is the frame identifier corresponding to the reconstructed image of the current frame in the external reference list.
  • the first identifier can be configured as the tenth preset information (such as "E0"), and the first indication information can be assigned a value of "NULL".
  • Figure 9a is a schematic diagram of an exemplary decoding process. What is described in the embodiment of Figure 9a is the scene of reference frame resampling decoding, which corresponds to the encoding process in the embodiment of Figure 8.
  • the reference frame parameters After parsing the reference frame parameters from the code stream corresponding to the current frame, it can be determined whether the reference frame parameters Whether the first identifier exists. When the first identifier exists in the reference frame parameter, it is determined whether the value of the first identifier is ninth preset information. If the value of the first identifier is the ninth preset information, it is determined that the current frame needs to be decoded with reference to the resampling reference frame; at this time, S904 to S907 can be performed.
  • the value of the first identifier is the tenth preset information, or the reference frame parameter does not include the first identifier, it is determined that the current frame does not need to be decoded with reference to the resampled reference frame, and S908 can be performed at this time.
  • the reconstructed image matching the second indication information in the reference frame parameter is determined from the external reference list of the decoder as the resampled reference frame of the current frame. the corresponding reconstructed image.
  • S905 Perform intra-frame coding on the reconstructed image to obtain the corresponding intra-frame coding stream.
  • S906 The decoder performs intra-frame decoding on the intra-frame encoded code stream.
  • the decoder Based on the intra-frame coding reconstructed image obtained by decoding the intra-frame coding code stream, the decoder performs inter-frame decoding on the code stream corresponding to the current frame to obtain the reconstructed image of the current frame.
  • S905 to S907 may refer to the description of S303 to S305 above, which will not be described again.
  • the decoder when it is determined that the current frame does not need to be decoded with reference to the resampling reference frame, based on the reconstructed image corresponding to the previous frame of the current frame, the decoder performs inter-frame decoding on the code stream corresponding to the current frame to obtain the reconstructed image of the current frame.
  • the code stream corresponding to the current frame can be directly input to the decoder, and then the decoder can reconstruct the image based on the reconstructed image corresponding to the previous frame of the current frame.
  • the code stream corresponding to the current frame is decoded to obtain the reconstructed image of the current frame.
  • the current frame needs to be used as a resampling reference frame for subsequent decoding. For example, it can be determined whether the reference frame parameter includes the third identifier; when the reference frame parameter includes the third identifier, it can be determined whether the value of the third identifier is the eleventh preset information. When the value of the third identifier is the eleventh preset information, it can be determined that the current frame needs to be used as a resampling reference frame for subsequent decoding, and S909 can be performed at this time.
  • the decoding process can be ended.
  • Figure 9b is a schematic diagram of an exemplary encoding and decoding process.
  • the encoder in the first device is a hardware encoder
  • the decoder in the second device is a hardware decoder.
  • the gray and black frames in the left area are frames sent by the first device
  • the gray and black frames in the right area are frames received by the second device; among them, the black frames are resampling references. frame.
  • the resampling setting rule is: when the resampling setting period is reached, the resampling setting period is 4; the resampling reference rule is: when the resampling reference period is reached, the resampling reference period is reached. is 5.
  • Figure 9b(1) and Figure 9b(2) the encoding and decoding process is similar to Figure 5b(1) and Figure 5b(2), and will not be described again here.
  • the reconstruction image of the third P frame can be resampled first, and then the resampled reconstruction image of the third P frame The image is saved to an external reference list.
  • a third identifier can be added to the code stream of the third P frame and assigned as the eleventh preset information, and third indication information (such as 3) and corresponding to the reconstructed image of the third frame can be added.
  • the resolution corresponds to the reconstructed image of the third frame after resampling.
  • the reconstructed image of the third P frame can be reconstructed according to the third identifier, the resolution corresponding to the reconstructed image of the third frame and the resolution corresponding to the reconstructed image of the third frame after resampling. sampling, and then according to the third instruction information, the reconstructed image of the third P frame after resampling is stored in the external reference list.
  • a hardware encoder can be combined with software encoding to achieve flexible reference frame management.
  • FIG. 10 is an exemplary encoding flow chart. In the embodiment of FIG. 10, it can be applied in the scenario of time domain hierarchical coding.
  • the hardware encoder can be combined with the software encoder to implement time domain hierarchical coding.
  • the first encoder is a hardware encoder
  • the second encoder is a software encoder.
  • the frame is encoded by the first encoder or the second encoder. For example, since the hardware encoder encodes frame by frame reference, and the frames of the T0 layer in the time domain hierarchical structure are encoded frame by frame, it can be determined whether the current frame is the T0 layer in the time domain hierarchical structure. . When the current frame is the T0 layer in the time domain hierarchical structure, it is determined to be encoded by the first encoder; when the current frame is not the T0 layer in the time domain hierarchical structure, it is determined to be encoded by the second encoder.
  • the time domain hierarchical structure is the time domain hierarchical structure 1 in the embodiment with reference to FIG. 6 .
  • the GOP in which the current frame is located has 8 frames, and the current frame is the 1st frame, the 3rd frame, the 5th frame, or the 7th frame, it can be determined that the current frame needs to be encoded by the first encoder.
  • the current frame is the 2nd frame, the 4th frame, the 6th frame, or the 8th frame, it can be determined that the current frame needs to be encoded by the second encoder.
  • the first encoder encodes the current frame to obtain a code stream corresponding to the current frame.
  • the current frame may be input to the first encoder.
  • Encoder encoding the first encoder encodes the current frame to obtain a code stream corresponding to the current frame.
  • the first encoder may perform intra-frame coding on the current frame to obtain an intra-frame coding stream of the current frame.
  • the first encoder may use the reconstructed image of the previous frame of the current frame as a reference to perform inter-frame coding on the current frame to obtain an intra-frame coding stream of the current frame.
  • the first encoder encodes the T0 layer
  • frames in other time domain layers (such as T1, T2...) are usually encoded with the T0 layer frame as a reference; therefore, after obtaining the code stream corresponding to the current frame, you can Decoding is performed based on the code stream corresponding to the current frame to obtain the reconstructed image of the current frame. Then the reconstructed image of the current frame is added to the external reference list of the second encoder. In this way, when the second decoder encodes frames of other time domain layers except the T0 layer, it can obtain the reconstructed image corresponding to the reference frame from its external reference list.
  • the reconstructed image corresponding to the reference frame of the current frame is determined from the external reference list. Based on the reconstructed image corresponding to the reference frame of the current frame, the second encoder The frame is encoded to obtain the code stream corresponding to the current frame.
  • the second encoder also has a corresponding internal reference list. After each frame is encoded by the second encoder, the reconstructed image of the frame can be added to the internal reference list.
  • time domain hierarchical reference rules can be set in the manner described in the embodiment in Figure 6 above, which will not be described again here.
  • the reference judgment rule in the time domain hierarchical reference rule is: time domain hierarchical structure 1
  • the POC value is an integer other than 8 ⁇ (n-1);
  • the reference frame determination condition is: reference mode 1. If the current frame is the 8th frame, it is determined that the 8th frame refers to the 7th frame according to the reference frame determination conditions in the time domain hierarchical reference rules. Since the 7th frame is encoded by the first encoder, it can be determined that encoding based on the external reference list is required. Then, the reconstructed image corresponding to the reference frame of the current frame can be determined from the external reference list according to the reference frame determination condition in the time domain hierarchical reference rule.
  • the second encoder can select the reconstructed image corresponding to the reference frame of the current frame from the external reference list, put the reconstructed image corresponding to the reference frame of the current frame into the internal reference list, and then select the reconstructed image based on the reference frame of the current frame. Correspondingly reconstruct the image and encode the current frame to obtain the code stream corresponding to the current frame.
  • the reference judgment rule in the time domain hierarchical reference rule is: time domain hierarchical structure 2
  • the POC value is an integer other than 12 ⁇ (n-1);
  • the reference frame determination condition is: reference mode 2.
  • the current frame is the 8th frame
  • the 8th frame refers to the 7th frame according to the reference frame determination conditions in the time domain hierarchical reference rule. Since the 7th frame is encoded by the second encoder, it can be determined that it needs to be encoded with reference to the internal reference list.
  • the reconstructed image corresponding to the reference frame of the current frame can be determined from the internal reference list according to the reference frame determination condition in the time domain hierarchical reference rule.
  • the second encoder encodes the current frame to obtain a code stream corresponding to the current frame.
  • the first identifier may be used to identify whether the current frame is encoded based on an external reference list, and may be an identifier (such as a stream ID) of a code stream corresponding to the reference frame of the current frame.
  • the first indication information is used to indicate the time domain layering of the current frame in the external reference list The reconstructed image corresponding to the reference frame.
  • the second identification is used to identify whether the current frame is used as a time domain hierarchical reference frame for subsequent encoding, and the second indication information is used to indicate the reconstructed image of the current frame in the external reference list.
  • the first identifier can be assigned to the first preset information (such as "G1"); and the first indication information can be assigned to, in the external reference list
  • the first identifier can be assigned the value of the second preset information (such as "G0"), and the first indication information can be assigned the value of "NULL".
  • the second identifier can be configured as third preset information (such as " H1"), and assign the second indication information as the frame identifier corresponding to the reconstructed image corresponding to the time domain hierarchical reference frame of the current frame in the external reference list.
  • the second identifier can be configured as third preset information (such as "H0"), and the second indication information is assigned a value of "NULL".
  • Figure 11a is a schematic diagram of an exemplary decoding process. What is described in the embodiment of FIG. 11a is a time-domain hierarchical decoding scenario, which corresponds to the encoding process in the embodiment of FIG. 10 .
  • both the first decoder and the second decoder are software decoders.
  • the first decoder is a hardware decoder
  • the second decoder is a software decoder.
  • the code stream corresponding to the current frame can be parsed to obtain the reference frame parameters; and then, based on the reference frame parameters, it is determined whether to be decoded by the first decoder.
  • the first identifier exists in the reference frame parameters.
  • the value of the first identifier is the first preset information. If the value of the first identifier is the first preset information, it is determined that the current frame needs to be decoded with reference to the external reference list. At this time, it can be determined that it needs to be decoded by the second decoder; at this time, S1104 can be performed.
  • the value of the first identifier is the second preset information, or the reference frame parameter does not include the first identifier, it is determined that the current frame does not need to be decoded with reference to the external reference list. At this time, it can be determined that it needs to be decoded by the first decoder; at this time, S1102 can be executed.
  • the code stream corresponding to the current frame can be input to the first decoder, and the first decoder decodes the code stream corresponding to the current frame to obtain the current frame. reconstructed image.
  • the second identifier exists in the reference frame parameters.
  • the value of the second identifier is the third preset information. If the value of the second identifier is the third preset information, it is determined that the current frame needs to be used as a time domain hierarchical reference frame for subsequent decoding. At this time, the reconstructed image of the current frame can be added to the external reference list of the second decoder. If the value of the second identifier is the fourth preset information, or the reference frame parameter does not include the second identifier, it is determined that the current frame does not need to be used as a time domain hierarchical reference frame for subsequent decoding.
  • the reconstructed image corresponding to the reference frame of the current frame is determined from the external reference list; based on the reconstructed image corresponding to the reference frame, the second decoder determines the reconstructed image corresponding to the current frame.
  • the code stream is decoded to obtain the reconstructed image of the current frame.
  • the reconstructed image corresponding to the reference frame of the current frame may be determined from the external reference list according to the first indication information in the reference frame parameter. Then, the reconstructed image corresponding to the reference frame of the current frame is taken out from the external reference list and stored in the internal reference list of the second decoder. Then, the second decoder decodes the code stream corresponding to the current frame based on the reconstructed image corresponding to the reference frame of the current frame to obtain the reconstructed image of the current frame.
  • the corresponding reference method is: the third frame is encoded with reference to the first frame, the second frame is encoded with reference to the first frame, and the fourth frame is encoded with reference to the third frame.
  • the current frame is an I frame.
  • the I frame can be input to the first encoder, and the first encoder performs intra-frame encoding on the I frame to obtain an I frame code stream. .
  • the code stream of the I frame can be input to the decoder to obtain a reconstructed image of the I frame output by the decoder, and then the reconstructed image of the I frame can be stored in the external reference list of the second encoder.
  • a second identifier can be added to the code stream of the I frame, and the second identifier can be assigned as third preset information, and second indication information (such as 0) can be added, and then the reference frame parameter can be added to the code stream of the I frame. sent to the second device.
  • the current frame is the first P frame.
  • the first P frame can be input to the second encoder, and the second encoder reads the reconstructed image of the I frame from the external reference list into the internal reference list; Then based on the reconstructed image of the I frame, the first P frame is encoded to obtain the code stream of the first P frame.
  • a first identifier can be added to the code stream of a P frame and the first identifier can be assigned as the first preset information, and the first indication information (such as 0) can be added, and then the first reference frame parameter can be added.
  • the code stream of the P frame is sent to the second device.
  • the current frame is the second P frame.
  • the second P frame can be input to the first encoder.
  • the first encoder encodes the second P frame based on the reconstructed image of the I frame to obtain the second P frame.
  • a code stream of P frames is
  • the code stream of the second P frame can be input to the decoder to obtain the reconstructed image of the second P frame output by the decoder, and then the reconstructed image of the second P frame can be stored in the external reference list of the second encoder. middle.
  • the identifier is assigned the third preset information, and the second indication information (such as 2) is added, and then the code stream of the second P frame with the reference frame parameter added is sent to the second device.
  • the current frame is the third P frame.
  • the third P frame can be input to the second encoder, and the second encoder reads the reconstructed image of the second P frame from the external reference list to the internal reference. in the list; then based on the reconstructed image of the second P frame, the third P frame is encoded to obtain the code stream of the third P frame. Then, you can add a first identifier to the code stream of the three P frames and assign the first identifier as the first preset information, and add the first indication information (such as 2), and then add the third reference frame parameter.
  • the code stream of the P frame is sent to the second device.
  • the code stream of the first P frame can be input to the second decoder, and the second decoder obtains the code from the external reference list based on the first identification and the first indication information.
  • the current frame is the second P frame.
  • the code stream of the second P frame can be input to the first decoder.
  • the first decoder will process the code stream of the second P frame based on the reconstructed image of the I frame.
  • the reconstructed image of the second P frame can be stored in the external reference list of the second decoder according to the second identification and the second indication information.
  • the reconstructed image of the second P frame can be stored in the external reference list of the second decoder.
  • the image is sent to the display module for display.
  • the current frame is the third P frame.
  • the code stream of the third P frame can be input to the second decoder, and the second decoder obtains the code from the external reference list based on the first identification and the first indication information.
  • the reconstructed image of the second P frame is read into the internal reference list; then based on the reconstructed image of the second P frame, the code stream of the third P frame is decoded to obtain the reconstructed image of the third P frame.
  • the reconstructed image of the third P frame can be sent to the display module for display.
  • the memory 1203 may store instructions used in the foregoing method embodiments.
  • the processor 1201 can be used to execute instructions in the memory 1203, and control the receiving pin to receive signals, and control the transmitting pin to send signals.
  • the device 1200 may be the electronic device or a chip of the electronic device in the above method embodiment.
  • This embodiment also provides a computer-readable storage medium.
  • Computer instructions are stored in the computer-readable storage medium.
  • the electronic device causes the electronic device to execute the above-mentioned related method steps to implement the above-mentioned embodiments. Encoding and decoding methods.
  • This embodiment also provides a computer program product.
  • the computer program product When the computer program product is run on a computer, it causes the computer to perform the above related steps to implement the encoding and decoding method in the above embodiment.
  • inventions of the present application also provide a device.
  • This device may be a chip, a component or a module.
  • the device may include a connected processor and a memory.
  • the memory is used to store computer execution instructions.
  • the processor can execute computer execution instructions stored in the memory, so that the chip executes the encoding and decoding methods in the above method embodiments.
  • the electronic equipment, computer-readable storage media, computer program products or chips provided by this embodiment are all It is used to execute the corresponding method provided above. Therefore, the beneficial effects it can achieve can be referred to the beneficial effects in the corresponding method provided above, and will not be described again here.
  • the disclosed devices and methods can be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of modules or units is only a logical function division.
  • there may be other division methods for example, multiple units or components may be combined or can be integrated into another device, or some features can be ignored, or not implemented.
  • the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, and the indirect coupling or communication connection of the devices or units may be in electrical, mechanical or other forms.
  • a unit described as a separate component may or may not be physically separate.
  • a component shown as a unit may be one physical unit or multiple physical units, that is, it may be located in one place, or it may be distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application can be integrated into one processing unit, each unit can exist physically alone, or two or more units can be integrated into one unit.
  • the above integrated units can be implemented in the form of hardware or software functional units.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

本申请实施例提供了一种编解码方法及电子设备。该方法包括：首先，获取当前帧；接着，从编码器的外部参考列表中，获取当前帧的参考帧所对应的重建图像，当前帧的参考帧为编码器已编码的帧，外部参考列表独立于编码器；随后，由编码器对重建图像进行帧内编码；基于帧内编码的结果，由编码器对当前帧进行帧间编码，以得到当前帧对应的码流。这样，编码器可以从外部参考列表中灵活选取参考帧进行编码，进而能够实现跨帧参考或跨分辨率参考，增加了对编码器参考帧管理的灵活性。从而能够增加编码器的抗弱网能力，可以在RTC场景中网络条件变差时，保证视频的流畅性。

Description

编解码方法及电子设备
本申请要求于2022年03月11日提交中国国家知识产权局、申请号为202210243272.9、申请名称为“编解码方法及电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及编解码技术领域，尤其涉及一种编解码方法及电子设备。
背景技术
随着视频逐渐向4K(即分辨率为4096×2160)、8K(即分辨率为7680×4320)等超高清分辨率发展，具有高规格、低时延、低功耗等优点的编码器(如硬件编码器)逐渐成为RTC(Real Time Communication，实时通信)视频场景中终端设备上的主流，以保证用户具有良好的RTC视频体验。
然而，这些编码器的抗弱网能力差，导致在RTC场景中网络条件变差时，无法保证视频的流畅性。
发明内容
为了解决上述技术问题，本申请提供一种编解码方法及电子设备。在该方法中，能够实现编解码器的参考帧灵活管理，进而增加编码器的抗弱网能力，从而可以在RTC场景中网络条件变差时，保证视频的流畅性。
第一方面，本申请实施例提供一种编码方法，该方法包括：首先，获取当前帧；接着，从编码器的外部参考列表中，获取当前帧的参考帧所对应的重建图像，当前帧的参考帧为编码器已编码的帧，外部参考列表独立于编码器；然后，由编码器对重建图像进行帧内编码；基于帧内编码的结果，由编码器对当前帧进行帧间编码，以得到当前帧对应的码流。这样，编码器可以从外部参考列表中灵活选取参考帧进行编码，进而能够实现跨帧参考或跨分辨率参考，增加了对编码器参考帧管理的灵活性。从而能够增加编码器的抗弱网能力，可以在RTC场景中网络条件变差时，保证视频的流畅性。
示例性的，外部参考列表独立于编码器，是指外部参考列表位于编码器的外部；用于区别编码器内部管理的参考列表；其中，编码器和其它模块，均可以访问该外部参考列表。
示例性的，编码器可以包括参考帧管理灵活的编码器(例如，软件编码器)，也可以包括参考帧管理不灵活的编码器(例如，硬件编码器)，本申请对此不作限制。
示例性的，在得到当前帧对应的码流后，可以将当前帧对应的码流发送至其他设备，也可以存储在本地，本申请对此不作限制。
示例性的，编码器可以以小于第一预设阈值的量化参数，对当前帧的参考帧所对应的重建图像进行帧内编码。其中，第一预设阈值可以按照需求设置，例如，第一预设阈 值为小于10的数，如2，3，4，5，6，7，8，9，本申请对此不作限制。这样，能够实现对当前帧的参考帧所对应的重建图像进行高质量的帧内编码，进而能够提高当前帧的编码图像质量。
示例性的，编码器可以按照大于第二预设阈值的码率，对当前帧的参考帧所对应的重建图像进行帧内编码。其中，第二预设阈值可以按照需求设置，本申请对此不作限制。这样，也能够实现对当前帧的参考帧所对应的重建图像进行高质量的帧内编码，进而能够提高当前帧的编码质量。
示例性的，本申请提供的编码方法对应的编解码场景可以包括多种，如使用长期参考帧编码的场景，时域分层编码场景以及参考帧重采样场景等，本申请对此不作限制。
需要说明的是，本申请的编码方法可以是指用于对P帧(Predictive frame，前向预测帧)进行编码的方法。
根据第一方面，该方法还包括：当确定需要将当前帧作为后续编码的参考帧时，基于当前帧对应的码流，生成当前帧的重建图像；将当前帧的重建图像，加入外部参考列表。这样，当前帧之后的帧能够从外部参考列表中获取参考帧的重建图像。
根据第一方面，或者以上第一方面的任意一种实现方式，将当前帧的重建图像，加入外部参考列表，包括：对当前帧的重建图像进行重采样；将重采样后的重建图像，加入外部参考列表。这样，可以实现将重采样后的参考帧所对应的重建图像加入外部参考列表中，便于后续参考重采样参考帧编码。
根据第一方面，或者以上第一方面的任意一种实现方式，当前帧对应的码流包括：第一标识和第一指示信息；第一标识用于标识当前帧是否基于外部参考列表编码，第一指示信息用于指示外部参考列表中当前帧的参考帧所对应的重建图像。
示例性的，第一标识还可以用于标识当前帧是否参考长期参考帧/时域分层参考帧/重采样参考帧编码。
示例性的，当前帧对应的码流也可以仅包括第一标识，此时，第一标识用于指示当前帧不基于外部参考列表编码。
根据第一方面，或者以上第一方面的任意一种实现方式，当前帧对应的码流包括：第二标识和第二指示信息；第二标识用于标识当前帧是否作为后续编码的参考帧，第二指示信息用于指示外部参考列表中当前帧的重建图像。
示例性的，当前帧对应的码流也可以仅包括第二标识，此时，第二标识用于指示当前帧不作为后续编码的参考帧。
根据第一方面，或者以上第一方面的任意一种实现方式，当前帧对应的码流包括：第三标识、第三指示信息和重采样分辨率；第三标识用于标识当前帧是否进行重采样，第三指示信息用于指示外部参考列表中当前帧的重采样后的重建图像，重采样分辨率包 括当前帧的重建图像对应的分辨率和重采样后的重建图像对应的分辨率。
示例性的，当前帧对应的码流也可以仅包括第三标识，此时，第三标识用于标识当前帧不进行重采样。
根据第一方面，或者以上第一方面的任意一种实现方式，修改当前帧对应的码流中，用于标识与当前帧的参考帧相关的标准语法元素；其中，标准语法元素是指标准编码协议中所规定的语法元素。这样，无需在当前帧对应的码流中添加参考帧参数(包括第一标识、第二标识、第一指示信息、第二指示信息、第三标识、第三指示信息和重采样分辨率)；解码器接收到当前帧对应的码流后，可以按照现有的解码方式进行解码。
示例性的，标准编码协议可以包括但不限于H265、H264、H266、AV1、MPEG、AVS系列编码协议等，本申请对此不作限制。
根据第一方面，或者以上第一方面的任意一种实现方式，用于标识与当前帧的参考帧相关的标准语法元素包括以下至少一种：
用于标识长期参考帧数量的语法元素；
用于标识长期参考帧POC(Picture Order Count，图像顺序号)的语法元素；
用于标识长期参考帧是否用于当前帧参考的语法元素；
用于标识是否使用前向参考帧的语法元素；
用于标识前向参考帧数量的语法元素；
用于标识当前帧与前向参考帧距离的语法元素；
用于标识前向参考帧是否用于当前帧参考的语法元素。
示例性的，以H265为例，用于标识长期参考帧数量的语法元素可以为：num_long_term_pics，用于标识长期参考帧POC的语法元素可以为poc_lsb_lt[i](poc_lsb_lt[i]，是指第i个长期参考帧的POC)，用于标识长期参考帧是否用于当前帧参考的语法元素可以为：used_by_curr_pic_lt_flag[i](used_by_curr_pic_lt_flag[i]，是指第i个长期参考帧是否用于当前帧参考)。
示例性的，以H265为例，用于标识是否使用前向参考帧的语法元素可以为short_term_ref_pic_set_sps_flag；用于标识前向参考帧数量的语法元素可以为num_negative_pics；用于标识当前帧与前向参考帧距离的语法元素可以为：delta_poc_s0_minus1[i](delta_poc_s0_minus1[i]，是指当前帧与第i个前向参考帧的距离)；用于标识前向参考帧是否用于当前帧参考的语法元素可以为used_by_curr_pic_s0_flag[i](used_by_curr_pic_s0_flag[i]，是指第i个前向参考帧是否用于当前帧参考)。
此外，用于标识与当前帧的参考帧相关的标准语法元素，还可以包括用于标识当前帧的POC值的语法元素pic_order_cnt_lsb。
根据第一方面，或者以上第一方面的任意一种实现方式，从编码器的外部参考列表中，获取当前帧的参考帧所对应的重建图像，包括：判断是否满足预设参考规则；当满足预设参考规则时，根据预设参考规则确定参考指示信息；根据参考指示信息，从编码 器的外部参考列表中，获取当前帧的参考帧所对应的重建图像。
示例性的，判断是否满足预设参考规则，可以是指判断当前帧是否满足预设参考规则。
根据第一方面，或者以上第一方面的任意一种实现方式，预设参考规则包括以下任意一种：长期参考规则，时域分层参考规则和重采样参考规则。
根据第一方面，或者以上第一方面的任意一种实现方式，长期参考规则包括以下至少一种：
达到长期参考帧的参考周期；
当前帧的预设参考帧对应的码流丢失；
外部参考列表中，存在与当前帧的相似度高于相似度阈值的重建图像。
示例性的，达到长期参考帧的参考周期，可以是指与当前帧距离最近的且参考长期参考帧的已编码的帧，至当前帧的距离差，达到长期参考帧的参考周期。
示例性的，可以根据时域分层结构，设置时域分层参考规则。
示例性的，重采样参考规则可以包括以下至少一种：达到重采样的参考周期；当前帧的上一帧进行了重采样。其中，达到重采样的参考周期，可以是指与当前帧距离最近的且参考重采样参考帧的已编码的帧，至当前帧的距离差，达到重采样的参考周期。
根据第一方面，或者以上第一方面的任意一种实现方式，帧内编码的结果包括当前帧的参考帧所对应的重建图像的帧内编码重建图像；基于帧内编码的结果，由编码器对当前帧进行帧间编码，以得到当前帧对应的码流，包括：由编码器以帧内编码重建图像为参考，对当前帧进行帧间编码，以得到当前帧对应的码流。
根据第一方面，或者以上第一方面的任意一种实现方式，当前帧的参考帧为：长期参考帧或时域分层参考帧或重采样参考帧。
根据第一方面，或者以上第一方面的任意一种实现方式，编码器为硬件编码器。
第二方面，本申请实施例提供一种解码方法，该方法包括：首先，接收当前帧对应的码流；接着，从解码器的外部参考列表中，确定当前帧的参考帧所对应的重建图像，当前帧的参考帧为解码器已解码的帧，外部参考列表独立于解码器；随后，对重建图像进行帧内编码，以得到对应的帧内编码码流；以及由解码器对帧内编码码流进行帧内解码；然后，基于帧内解码的结果，由解码器对当前帧对应的码流进行帧间解码，以得到当前帧的重建图像。这样，解码器也可以灵活从外部参考列表中选取参考帧进行解码，使得解码器能够实现跨帧参考编码或跨分辨率参考解码，提高了硬件解码器的参考帧管理的灵活性。进而与编码器协同，保证在RTC场景中网络条件变差时视频的流畅性。
示例性的，解码器可以包括参考帧管理灵活的解码器，也可以包括参考帧管理不灵 活的解码器，本申请对此不作限制。
示例性的，可以以小于第一预设阈值的量化参数，对当前帧的参考帧所对应的重建图像进行帧内解码。其中，第一预设阈值可以按照需求设置，例如，第一预设阈值为小于10的数，如2，3，4，5，6，7，8，9，本申请对此不作限制。这样，能够实现对当前帧的参考帧所对应的重建图像进行高质量的帧内解码，进而能够提高当前帧的解码图像质量。
示例性的，可以按照大于第二预设阈值的码率，对当前帧的参考帧所对应的重建图像进行帧内解码。其中，第二预设阈值可以按照需求设置，本申请对此不作限制。这样，也能够实现对当前帧的参考帧所对应的重建图像进行高质量的帧内解码，进而能够提高当前帧的解码质量。
示例性的，本申请提供的解码方法对应的编解码场景可以包括多种，如使用长期参考帧解码的场景，时域分层解码场景以及参考帧重采样场景等，本申请对此不作限制。
需要说明的是，本申请的解码方法可以是指用于对P帧(Predictive frame，前向预测帧)进行解码的方法。
根据第二方面，该方法还包括：当确定需要将当前帧作为后续解码的参考帧时，将当前帧的重建图像，加入外部参考列表。这样，当前帧之后的帧能够从外部参考列表中获取参考帧的重建图像。
根据第二方面，或者以上第二方面的任意一种实现方式，将当前帧的重建图像，加入外部参考列表，包括：对当前帧的重建图像进行重采样；将重采样后的重建图像，加入外部参考列表。这样，可以实现将重采样后的参考帧所对应的重建图像加入外部参考列表中，便于后续参考重采样参考帧编码。
根据第二方面，或者以上第二方面的任意一种实现方式，从解码器的外部参考列表中，确定当前帧的参考帧所对应的重建图像，包括：解析当前帧对应的码流，以得到参考帧参数；基于参考帧参数，从解码器的外部参考列表中，确定当前帧的参考帧所对应的重建图像。
根据第二方面，或者以上第二方面的任意一种实现方式，参考帧参数包括：第一标识和第一指示信息；第一标识用于标识当前帧是否基于外部参考列表解码，第一指示信息用于指示外部参考列表中当前帧的参考帧所对应的重建图像。
示例性的，参考帧参数也可以仅包括第一标识，此时，第一标识用于指示当前帧不基于外部参考列表解码。
根据第二方面，或者以上第二方面的任意一种实现方式，参考帧参数包括：第二标识和第二指示信息；第二标识用于标识当前帧是否作为后续解码的参考帧，第一指示信息用于指示外部参考列表中当前帧的重建图像。
示例性的，参考帧参数也可以仅包括第二标识，此时，第二标识用于指示当前帧不作为后续解码的参考帧。
根据第二方面，或者以上第二方面的任意一种实现方式，当前帧对应的码流包括：第三标识、第三指示信息和重采样分辨率；第三标识用于标识当前帧是否进行重采样，第三指示信息用于指示外部参考列表中当前帧的重采样后的重建图像，重采样分辨率包括当前帧的重建图像对应的分辨率和重采样后的重建图像对应的分辨率。
示例性的，参考帧参数也可以仅包括第三标识，此时，第三标识用于指示当前帧不进行重采样。
根据第二方面，或者以上第二方面的任意一种实现方式，当前帧的参考帧为：长期参考帧或时域分层参考帧或重采样参考帧。
第二方面以及第二方面的任意一种实现方式分别与第一方面以及第一方面的任意一种实现方式相对应。第二方面以及第二方面的任意一种实现方式所对应的技术效果可参见上述第一方面以及第一方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第三方面，本申请实施例提供一种编码方法，该方法包括：首先，获取当前帧；当确定需要由第一编码器编码时，由第一编码器对当前帧进行编码，以得到当前帧对应的码流；以及将基于当前帧对应的码流生成的重建图像，加入第二编码器的外部参考列表，外部参考列表独立于第二编码器；当确定需要由第二编码器编码时，从外部参考列表中确定当前帧的参考帧所对应的重建图像，基于当前帧的参考帧所对应的重建图像，由第二编码器对当前帧进行编码，以得到当前帧对应的码流；其中，第一编码器为硬件编码器，第二编码器为软件编码器。这样，通过硬件编码器和软件编码器结合的方式，提高硬件编码器的参考帧管理能力。从而能够实现跨帧参考或跨分辨率参考，增加了对编码器参考帧管理的灵活性。进而能够增加编码器的抗弱网能力，可以在RTC场景中网络条件变差时，保证视频的流畅性。
根据第三方面，该方法还包括：判断当前帧是否是时域分层结构中的T0层；当当前帧是时域分层结构中的T0层时，确定需要由第一编码器编码；当当前帧不是时域分层结构中的T0层时，确定需要由第二编码器编码。
示例性的，T0层为时域分层结构的最低层，也可以叫做基础层；其他层可以叫做增强层。
根据第三方面，或者以上第三方面的任意一种实现方式，当前帧对应的码流包括：第一标识和第一指示信息；第一标识用于标识当前帧是否基于外部参考列表编码，第一指示信息用于指示外部参考列表中当前帧的参考帧所对应的重建图像。
示例性的，当前帧对应的码流也可以仅包括第一标识，此时，第一标识用于指示当前帧不基于外部参考列表编码。
第四方面，本申请实施例提供一种解码方法，该方法包括：首先，接收当前帧对应的码流。当确定需要由第一解码器进行解码时，由第一解码器对当前帧对应的码流进行解码，以得到当前帧的重建图像；以及将当前帧的重建图像，加入第二解码器的外部参考列表，外部参考列表位于第二编码器的外部；当确定需要由第二解码器进行解码时，从外部参考列表中确定当前帧的参考帧所对应的重建图像；基于参考帧所对应的重建图像，由第二解码器对当前帧对应的码流进行解码，以得到当前帧的重建图像。这样，通过第一解码器和第一解码器结合的方式，实现对码流进行解码。当第一解码器为硬件解码器，第二解码器为软件解码器时，可以增强硬件解码器的参考帧管理的灵活性。
根据第四方面，该方法还包括：解析当前帧对应的码流，以得到参考帧参数；根据参考帧参数，确定是否需要由第一解码器进行解码。
根据第四方面，或者以上第四方面的任意一种实现方式，参考帧参数包括：第一标识和第一指示信息；第一标识用于标识当前帧是否基于外部参考列表解码，第一指示信息用于指示外部参考列表中当前帧的参考帧所对应的重建图像。
示例性的，参考帧参数也可以仅包括第一标识，此时，第一标识用于指示当前帧不基于外部参考列表解码。
根据第四方面，或者以上第四方面的任意一种实现方式，从外部参考列表中确定当前帧的参考帧所对应的重建图像，包括：当第一标识的值为第一预设信息时，将外部参考列表中，与第一指示信息匹配的重建图像，确定为当前帧的参考帧所对应的重建图像。
根据第四方面，或者以上第四方面的任意一种实现方式，第一解码器和第二解码器均为软件解码器。
根据第四方面，或者以上第四方面的任意一种实现方式，第一解码器为硬件解码器，第二解码器为软件解码器。
第五方面，本申请实施例提供一种电子设备，包括：存储器和处理器，存储器与处理器耦合；存储器存储有程序指令，当程序指令由处理器执行时，使得电子设备执行第一方面或第一方面的任意可能的实现方式中的编码方法。
第五方面以及第五方面的任意一种实现方式分别与第一方面以及第一方面的任意一种实现方式相对应。第五方面以及第五方面的任意一种实现方式所对应的技术效果可参见上述第一方面以及第一方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第六方面，本申请实施例提供一种电子设备，包括：存储器和处理器，存储器与处理器耦合；存储器存储有程序指令，当程序指令由处理器执行时，使得电子设备执行第 二方面或第二方面的任意可能的实现方式中的解码方法。
第六方面以及第六方面的任意一种实现方式分别与第二方面以及第二方面的任意一种实现方式相对应。第六方面以及第六方面的任意一种实现方式所对应的技术效果可参见上述第二方面以及第二方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第七方面，本申请实施例提供一种电子设备，包括：存储器和处理器，存储器与处理器耦合；存储器存储有程序指令，当程序指令由处理器执行时，使得电子设备执行第三方面或第三方面的任意可能的实现方式中的编码方法。
第七方面以及第七方面的任意一种实现方式分别与第三方面以及第三方面的任意一种实现方式相对应。第七方面以及第七方面的任意一种实现方式所对应的技术效果可参见上述第三方面以及第三方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第八方面，本申请实施例提供一种电子设备，包括：存储器和处理器，存储器与处理器耦合；存储器存储有程序指令，当程序指令由处理器执行时，使得电子设备执行第四方面或第四方面的任意可能的实现方式中的解码方法。
第八方面以及第八方面的任意一种实现方式分别与第四方面以及第四方面的任意一种实现方式相对应。第八方面以及第八方面的任意一种实现方式所对应的技术效果可参见上述第四方面以及第四方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第九方面，本申请实施例提供一种芯片，包括一个或多个接口电路和一个或多个处理器；接口电路用于从电子设备的存储器接收信号，并向处理器发送信号，信号包括存储器中存储的计算机指令；当处理器执行计算机指令时，使得电子设备执行第一方面或第一方面的任意可能的实现方式中的编码方法。
第九方面以及第九方面的任意一种实现方式分别与第一方面以及第一方面的任意一种实现方式相对应。第九方面以及第九方面的任意一种实现方式所对应的技术效果可参见上述第一方面以及第一方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第十方面，本申请实施例提供一种芯片，包括一个或多个接口电路和一个或多个处理器；接口电路用于从电子设备的存储器接收信号，并向处理器发送信号，信号包括存储器中存储的计算机指令；当处理器执行计算机指令时，使得电子设备执行第二方面或第二方面的任意可能的实现方式中的解码方法。
第十方面以及第十方面的任意一种实现方式分别与第二方面以及第二方面的任意一种实现方式相对应。第十方面以及第十方面的任意一种实现方式所对应的技术效果可参见上述第二方面以及第二方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第十一方面，本申请实施例提供一种芯片，包括一个或多个接口电路和一个或多个处理器；接口电路用于从电子设备的存储器接收信号，并向处理器发送信号，信号包括存储器中存储的计算机指令；当处理器执行计算机指令时，使得电子设备执行第三方面 或第三方面的任意可能的实现方式中的编码方法。
第十一方面以及第十一方面的任意一种实现方式分别与第三方面以及第三方面的任意一种实现方式相对应。第十一方面以及第十一方面的任意一种实现方式所对应的技术效果可参见上述第三方面以及第三方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第十二方面，本申请实施例提供一种芯片，包括一个或多个接口电路和一个或多个处理器；接口电路用于从电子设备的存储器接收信号，并向处理器发送信号，信号包括存储器中存储的计算机指令；当处理器执行计算机指令时，使得电子设备执行第四方面或第四方面的任意可能的实现方式中的解码方法。
第十二方面以及第十二方面的任意一种实现方式分别与第四方面以及第四方面的任意一种实现方式相对应。第十二方面以及第十二方面的任意一种实现方式所对应的技术效果可参见上述第四方面以及第四方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第十三方面，本申请实施例提供一种计算机可读存储介质，计算机可读存储介质存储有计算机程序，当计算机程序运行在计算机或处理器上时，使得计算机或处理器执行第一方面或第一方面的任意可能的实现方式中的编码方法。
第十三方面以及第十三方面的任意一种实现方式分别与第一方面以及第一方面的任意一种实现方式相对应。第十三方面以及第十三方面的任意一种实现方式所对应的技术效果可参见上述第一方面以及第一方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第十四方面，本申请实施例提供一种计算机可读存储介质，计算机可读存储介质存储有计算机程序，当计算机程序运行在计算机或处理器上时，使得计算机或处理器执行第二方面或第二方面的任意可能的实现方式中的解码方法。
第十四方面以及第十四方面的任意一种实现方式分别与第二方面以及第二方面的任意一种实现方式相对应。第十四方面以及第十四方面的任意一种实现方式所对应的技术效果可参见上述第二方面以及第二方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第十五方面，本申请实施例提供一种计算机可读存储介质，计算机可读存储介质存储有计算机程序，当计算机程序运行在计算机或处理器上时，使得计算机或处理器执行第三方面或第三方面的任意可能的实现方式中的编码方法。
第十五方面以及第十五方面的任意一种实现方式分别与第三方面以及第三方面的任意一种实现方式相对应。第十五方面以及第十五方面的任意一种实现方式所对应的技术效果可参见上述第三方面以及第三方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第十六方面，本申请实施例提供一种计算机可读存储介质，计算机可读存储介质存储有计算机程序，当计算机程序运行在计算机或处理器上时，使得计算机或处理器执行第四方面或第四方面的任意可能的实现方式中的解码方法。
第十六方面以及第十六方面的任意一种实现方式分别与第四方面以及第四方面的任意一种实现方式相对应。第十六方面以及第十六方面的任意一种实现方式所对应的技术效果可参见上述第四方面以及第四方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第十七方面，本申请实施例提供一种计算机程序产品，计算机程序产品包括软件程序，当软件程序被计算机或处理器执行时，使得计算机或处理器执行第一方面或第一方面的任意可能的实现方式中的编码方法。
第十七方面以及第十七方面的任意一种实现方式分别与第一方面以及第一方面的任意一种实现方式相对应。第十七方面以及第十七方面的任意一种实现方式所对应的技术效果可参见上述第一方面以及第一方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第十八方面，本申请实施例提供一种计算机程序产品，计算机程序产品包括软件程序，当软件程序被计算机或处理器执行时，使得计算机或处理器执行第二方面或第二方面的任意可能的实现方式中的解码方法。
第十八方面以及第十八方面的任意一种实现方式分别与第二方面以及第二方面的任意一种实现方式相对应。第十八方面以及第十八方面的任意一种实现方式所对应的技术效果可参见上述第二方面以及第二方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第十九方面，本申请实施例提供一种计算机程序产品，计算机程序产品包括软件程序，当软件程序被计算机或处理器执行时，使得计算机或处理器执行第三方面或第三方面的任意可能的实现方式中的编码方法。
第十九方面以及第十九方面的任意一种实现方式分别与第三方面以及第三方面的任意一种实现方式相对应。第十九方面以及第十九方面的任意一种实现方式所对应的技术效果可参见上述第三方面以及第三方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第二十方面，本申请实施例提供一种计算机程序产品，计算机程序产品包括软件程序，当软件程序被计算机或处理器执行时，使得计算机或处理器执行第四方面或第四方面的任意可能的实现方式中的解码方法。
第二十方面以及第二十方面的任意一种实现方式分别与第四方面以及第四方面的任意一种实现方式相对应。第二十方面以及第二十方面的任意一种实现方式所对应的技术 效果可参见上述第四方面以及第四方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第二十一方面，本申请实施例提供了一种电子设备，该电子设备用于编码，可以包括获取模块和处理模块，获取模块可以用于获取当前帧；获取模块还可以用于从编码器的外部参考列表中，获取当前帧的参考帧所对应的重建图像，当前帧的参考帧为编码器已编码的帧，外部参考列表独立于编码器；处理模块用于调用编码器对重建图像进行帧内编码；处理模块还用于调用编码器基于帧内编码的结果，对当前帧进行帧间编码，以得到当前帧对应的码流。处理模块可以执行例如图1中视频编码模块的功能。
示例性的，该电子设备还可以包括通信模块，该通信模块可以用于与其它电子设备通信以及该电子设备内部各个模块之间的通信，例如可以用于发送当前帧对应的码流
上述电子设备可以用于执行第一方面或第一方面的任意可能的实现方式中的编码方法。
第二十一方面以及第二十一方面的任意一种实现方式分别与第一方面以及第一方面的任意一种实现方式相对应。第二十一方面以及第二十一方面的任意一种实现方式所对应的技术效果可参见上述第一方面以及第一方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第二十二方面，本申请实施例提供了一种电子设备，该电子设备用于解码，可以包括通信模块和处理模块，通信模块可以用于接收当前帧对应的码流；处理模块用于从解码器的外部参考列表中，确定当前帧的参考帧所对应的重建图像，当前帧的参考帧为解码器已解码的帧，外部参考列表独立于解码器；处理模块还用于对重建图像进行帧内编码，以得到对应的帧内编码码流；处理模块还用于调用解码器对帧内编码码流进行帧内解码；调用解码器基于帧内解码的结果，对当前帧对应的码流进行帧间解码，以得到当前帧的重建图像。处理模块可以执行例如图1中视频解码模块的功能。
上述电子设备可以用于执行第二方面或第二方面的任意可能的实现方式中的解码方法。
第二十二方面以及第二十二方面的任意一种实现方式分别与第二方面以及第二方面的任意一种实现方式相对应。第二十二方面以及第二十二方面的任意一种实现方式所对应的技术效果可参见上述第二方面以及第二方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第二十三方面，本申请实施例提供了一种电子设备，该电子设备用于编码，可以包括获取模块和处理模块，获取模块可以用于获取当前帧；处理模块用于当确定需要由第一编码器编码时，调用第一编码器对当前帧进行编码，以得到当前帧对应的码流；以及将基于当前帧对应的码流生成的重建图像，加入第二编码器的外部参考列表，外部参考列表独立于第二编码器；处理模块还用于当确定需要由第二编码器编码时，从外部参考列表中确定当前帧的参考帧所对应的重建图像，调用第二编码器基于当前帧的参考帧所 对应的重建图像，对当前帧进行编码，以得到当前帧对应的码流；其中，第一编码器为硬件编码器，第二编码器为软件编码器。处理模块可以执行例如图1中视频编码模块的功能。
示例性的，该电子设备还可以包括通信模块，该通信模块可以用于与其它电子设备通信以及该电子设备内部各个模块之间的通信，例如可以用于发送当前帧对应的码流
上述电子设备可以用于执行第三方面或第三方面的任意可能的实现方式中的编码方法。
第二十三方面以及第二十三方面的任意一种实现方式分别与第三方面以及第三方面的任意一种实现方式相对应。第二十三方面以及第二十三方面的任意一种实现方式所对应的技术效果可参见上述第三方面以及第三方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第二十四方面，本申请实施例提供了一种电子设备，该电子设备用于解码，可以包括通信模块和处理模块，通信模块可以用于接收当前帧对应的码流；处理模块用于当确定需要由第一解码器进行解码时，调用第一解码器对当前帧对应的码流进行解码，以得到当前帧的重建图像；以及将当前帧的重建图像，加入第二解码器的外部参考列表，外部参考列表位于第二编码器的外部；处理模块还用于当确定需要由第二解码器进行解码时，从外部参考列表中确定当前帧的参考帧所对应的重建图像；调用第二解码器基于参考帧所对应的重建图像，对当前帧对应的码流进行解码，以得到当前帧的重建图像。处理模块可以执行例如图1中视频解码模块的功能。
上述电子设备可以用于执行第四方面或第四方面的任意可能的实现方式中的解码方法。
第二十四方面以及第二十四方面的任意一种实现方式分别与第四方面以及第四方面的任意一种实现方式相对应。第二十四方面以及第二十四方面的任意一种实现方式所对应的技术效果可参见上述第四方面以及第四方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第二十五方面，本申请提供一种码流，其中，该码流由前述第一方面或第一方面的任意可能的实现方式中的编码方法得到。可选的，该码流可以存储在计算机可读存储介质上，或者以电磁信号的形式进行传输。
第二十五方面以及第二十五方面的任意一种实现方式分别与第一方面以及第一方面的任意一种实现方式相对应。第二十五方面以及第二十五方面的任意一种实现方式所对应的技术效果可参见上述第一方面以及第一方面的任意一种实现方式所对应的技术效果，此处不再赘述。
第二十六方面，本申请提供一种码流，其中，该码流由前述第三方面或第三方面的任意可能的实现方式中的编码方法得到。可选的，该码流可以存储在计算机可读存储介质上，或者以电磁信号的形式进行传输。
第二十六方面以及第二十六方面的任意一种实现方式分别与第三方面以及第三方面的任意一种实现方式相对应。第二十六方面以及第二十六方面的任意一种实现方式所对应的技术效果可参见上述第三方面以及第三方面的任意一种实现方式所对应的技术效果，此处不再赘述。
附图说明
图1为示例性示应用场景示意图；
图2为示例性示出的编码流程示意图；
图3为示例性示出的解码流程示意图；
图4为示例性示出的编码流程示意图；
图5a为示例性示出的解码流程示意图；
图5b为示例性示出的编解码过程示意图；
图5c为示例性示出的编解码过程示意图；
图6为示例性示出的编码流程示意图；
图7a为示例性示出的解码流程示意图；
图7b为示例性示出的编解码过程示意图；
图7c为示例性示出的编解码过程示意图；
图8为示例性示出的编码流程示意图；
图9a为示例性示出的解码流程示意图；
图9b为示例性示出的编解码过程示意图；
图10为示例性示出的编码流程图；
图11a为示例性示出的解码流程示意图；
图11b为示例性示出的编解码过程示意图；
图12为示例性示出的装置的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。
本申请实施例的说明书和权利要求书中的术语“第一”和“第二”等是用于区别不同的对象，而不是用于描述对象的特定顺序。例如，第一目标对象和第二目标对象等是用于区别不同的目标对象，而不是用于描述目标对象的特定顺序。
在本申请实施例中，“示例性的”或者“例如”等词用于表示作例子、例证或说明。本申请实施例中被描述为“示例性的”或者“例如”的任何实施例或设计方案不应被解释为比其它实施例或设计方案更优选或更具优势。确切而言，使用“示例性的”或者“例如”等词 旨在以具体方式呈现相关概念。
在本申请实施例的描述中，除非另有说明，“多个”的含义是指两个或两个以上。例如，多个处理单元是指两个或两个以上的处理单元；多个系统是指两个或两个以上的系统。
图1为示例性示出的应用场景示意图。图1的应用场景包括但不限于：视频通话、屏幕内容共享(如文档共享、视频共享等)、手写体内容共享、视频会议、短距投屏、直播等各种RTC视频场景。
参照图1，示例性的，第一设备、第二设备均包括但不限于：PC(Personal Computer，个人计算机)、笔记本电脑、平板电脑、手机等，本申请对此不作限制。示例性的，服务器可以是单个服务器，也可以是服务器集群，本申请对此不作限制。
参照图1，示例性的，编码过程中，可以由第一设备的视频采集模块进行视频采集，得到当前帧。示例性的，视频采集模块包括但不限于：摄像头、显卡、录屏工具等。接着，视频采集模块可以将采集到的当前帧发送至视频编码模块，由视频编码模块对当前帧进行视频编码，得到当前帧对应的码流。随后，视频编码模块将当前帧对应的码流发送至发送模块，由发送模块将当前帧对应的码流发送至服务器。
示例性的，服务器接收到当前帧对应的码流后，将当前帧对应的码流转发至对应的第二设备。
继续参照图1，示例性的，解码过程中，第二设备的接收模块接收码流，然后将码流发送至视频解码模块，由视频解码模块对该码流进行解码，得到对应的重建图像。随后，视频解码模块将重建图像发送至视频显示模块，由视频显示模块对重建图像进行显示。
应该理解的是，第一设备编码得到码流后，也可以直接将码流发送至第二设备，无需经过服务器的转发，本申请对此不作限制。本申请以第一设备将编码得到的码流发送至服务器，由服务器将码流转发给第二设备为例进行示例性说明。
应该理解的是，图1所示第一设备以及第二设备仅是一个范例，并且第一设备以及第二设备可以具有比图中所示的更多的或者更少的部件，可以组合两个或多个的部件，或者可以具有不同的部件配置。图1中所示出的各种模块可以在包括一个或多个信号处理和/或专用集成电路在内的硬件、软件、或硬件和软件的组合中实现。
示例性的，若图1中的视频编码模块是抗弱网能力差的编码器(如硬件编码器，或者是随着技术发展产生的其他编码器)，则当RTC场景中网络条件变差时，第二设备的视频会出现卡顿、丢帧等情况。本申请考虑到导致硬件编码器抗弱网能力差的原因之一在于，硬件编码器没有提供参考帧管理的对外接口，且大多数硬件编码器是逐帧参考进行编码的，即硬件编码器的参考帧管理不灵活。进而本申请可以通过增加硬件编码器的参考帧管理的灵活性，来增强硬件编码器的抗弱网能力，从而保证在RTC场景中网络条件变差时视频的流畅性。
示例性的，第一设备可以创建独立于编码器的外部参考列表(也就是说，外部参考列表位于编码器的外部，以区别于编码器内部管理的参考列表)；随后，在编码器编码 过程中，将已编码的帧中被确定后续编码的参考帧所对应的重建图像添加至外部参考列表。这样，在编码器后续编码过程中，可以从外部参考列表中选取合适的参考帧所对应的重建图像，作为参考进行编码；使得编码器能够实现跨帧参考编码或跨分辨率参考编码，提高了编码器的参考帧管理的灵活性，进而能够增加编码器的抗弱网能力，保证在RTC场景中网络条件变差时视频的流畅性。
以下对编码过程进行说明。
图2为示例性示出的编码流程示意图。
S201，获取当前帧。
示例性的，若当前的应用场景为视频通话/视频会议场景，则第一设备获取到的当前帧可以是人和/或物体的图像。
示例性的，若当前的应用场景为屏幕内容共享/短距投屏场景，则第一设备获取到的当前帧可以是桌面展示的文档图像或者视频图像。
示例性的，若当前的应用场景为手写体内容共享场景，则第一设备获取到的当前帧可以是手写文本/手绘图形的图像。
S202，从编码器的外部参考列表中，获取当前帧的参考帧所对应的重建图像，当前帧的参考帧为编码器已编码的帧，外部参考列表独立于编码器。
示例性的，可以预先配置预设参考帧设置规则，该预设参考帧设置规则可以用于判断是否需要将当前帧设置为后续编码的参考帧；具体在后续进行说明。进而，在编码器每编码一帧后，可以根据预设参考帧设置规则，判断是否需要将该已编码的帧作为后续编码的参考帧。当确定需要将该已编码的帧作为后续编码的参考帧时，可以将该已编码的帧对应的重建图像，加入至外部参考列表，以供后续编码其他帧的参考。这样，外部参考列表中可以包括一个或多个已编码的帧所对应的重建图像。
示例性的，可以预先配置预设参考规则，该预设参考规则可以包括参考判断规则和参考帧确定规则；其中，参考判断规则用于判断当前帧是否需要参考外部参考列表编码，参考帧确定规则用于确定外部参考列表中当前帧的参考帧所对应的重建图像。进而，在获取当前帧后，判断是否满足预设参考规则中的参考判断规则。
示例性的，当确定不满足参考判断规则时，则可以确定当前帧无需参考外部参考列表编码，此时可以直接将当前帧输入至编码器中，由编码器对当前帧进行编码，以得到当前帧对应的码流。示例性的，当当前帧为I帧(Intra frame，帧内编码帧)时，编码器可以对当前帧进行帧内编码。当当前帧为P帧(Predictive frame，前向预测帧)时，编码器可以对当前帧进行帧间编码；其中，编码器可以根据当前帧的上一帧所对应的重建图像，对当前帧进行帧间编码，以得到当前帧对应的码流。
示例性的，当确定满足参考判断规则时，可以根据预设参考规则中的参考帧确定规则，确定参考指示信息；其中，参考指示信息用于指示外部参考列表中当前帧的参考帧所对应的重建图像。这样，可以根据参考指示信息，从外部参考列表中，获取当前帧的参考帧所对应的重建图像。接着，可以执行S203～S204，实现对当前帧的编码。
S203，由编码器对重建图像进行帧内编码。
S204，基于帧内编码的结果，由编码器对当前帧进行帧间编码，以得到当前帧对应 的码流。
示例性的，对于参考帧管理不灵活的编码器而言，其通常逐帧参考进行帧间编码的。因此，可以将当前帧的参考帧所对应的重建图像，输入至编码器中，由编码器对当前帧的参考帧所对应的重建图像进行帧内编码，以得到对应的帧内编码的结果。其中，帧内编码的结果可以包括：当前帧的参考帧所对应的重建图像的帧内编码码流，以及当前帧的参考帧所对应的重建图像的帧内编码重建图像。
一种可能的方式中，编码器可以以小于第一预设阈值的量化参数，对当前帧的参考帧所对应的重建图像进行帧内编码。其中，第一预设阈值可以按照需求设置，例如，第一预设阈值为小于10的数，如2，3，4，5，6，7，8，9，本申请对此不作限制。这样，能够实现对当前帧的参考帧所对应的重建图像进行高质量的帧内编码。
一种可能的方式中，编码器可以按照大于第二预设阈值的码率，对当前帧的参考帧所对应的重建图像进行帧内编码。其中，第二预设阈值可以按照需求设置，本申请对此不作限制。这样，也能够实现对当前帧的参考帧所对应的重建图像进行高质量的帧内编码。
需要说明的是，编码器对当前帧的参考帧所对应的重建图像，进行帧内编码所使用的量化参数，可以小于编码器对I帧进行帧内编码所使用的量化参数。示例性的，编码器对当前帧的参考帧所对应的重建图像进行帧内编码的码率，可以大于编码器对I帧进行帧内编码的码率。
需要说明的是，第一设备不将该帧内编码码流发送至服务器/第二设备。
接着，在编码器完成对当前帧的参考帧所对应的重建图像的帧内编码后，可以将当前帧输入至硬件编码器；这样，编码器可以以帧内编码重建图像为参考，对当前帧进行帧间编码，以得到当前帧对应的码流。示例性的，第一设备可以将当前帧对应的码流发送至服务器/第二设备。
这样，编码器可以从外部参考列表中灵活选取参考帧进行编码，进而能够实现跨帧参考或跨分辨率参考，增加了对编码器参考帧管理的灵活性。
需要说明的是，本实施例中的编码器可以包括参考帧管理不灵活的编码器(如硬件编码器，或者是随着技术发展产生的其他编码器)，也可以包括考帧管理灵活的编码器(如软件编码器，或者是随着技术发展产生的其他编码器)，本申请对此不作限制。
一种可能的方式中，第二设备也可以建立独立于解码器的外部参考列表(也就是说，外部参考列表位于解码器的外部，以区别于解码器内部管理的参考列表)；随后，在解码器解码过程中，将已解码的帧中被确定后续解码的参考帧所对应的重建图像添加至外部参考列表。这样，在解码器后续解码过程中，可以从外部参考列表中选取匹配的参考帧所对应的重建图像，作为参考进行解码；使得解码器能够实现跨帧参考解码或跨分辨率参考解码，提高了解码器的参考帧管理的灵活性。进而与编码器协同，保证在RTC场景中网络条件变差时视频的流畅性。
图3为示例性示出的解码流程示意图。在图3的实施例中的解码过程，与图2的实施例中编码过程对应。
S301，接收当前帧对应的码流。
S302，从解码器的外部参考列表中，确定当前帧的参考帧所对应的重建图像，当前帧的参考帧为解码器已解码的帧，外部参考列表独立于解码器。
示例性的，第一设备在得到当前帧对应的码流后，一种可能的方式中，可以将当前帧的参考帧参数添加至当前帧对应的码流中；其中，参考帧参数用于描述当前帧与外部参考列表中参考帧所对应的重建图像之间关联。其中，参考帧参数包括但不限于：当前帧是否作为后续编码的参考帧的相关描述信息，当前帧是否参考外部参考列表编码的相关描述信息，当前帧是否作进行重采样的相关描述信息等，本申请对此不作限制。然后再将添加参考帧参数的码流发送至第二设备。
这样，第二设备中的解码器在每解码一帧后，可以根据从该已解码的帧对应的码流所解析出的参考帧参数，判断是否需要将该已解码的帧作为参考帧。当确定需要将该已解码的帧作为参考帧时，一种可能的方式中，可以将已解码的帧对应的重建图像加入至外部参考列表中；这样，外部参考列表包含的是参考帧的重建图像。一种可能的方式中，可以将该已解码的帧对应的重建图像输入至编码器中进行帧内编码，以得到帧内编码码流；接着，可以将已解码的帧对应的重建图像的帧内编码码流，加入至外部参考列表中；这样，外部参考列表中包含的是参考帧的帧内编码码流。本申请以包含参考帧的重建图像的外部参考列表为例进行说明。
一种可能的方式中，编码器可以以小于第一预设阈值的量化参数，对该已解码的帧对应的重建图像进行帧内编码，得到该已解码的帧的重建图像对应的帧内编码码流。其中，第一预设阈值可以按照需求设置，例如，第一预设阈值为小于10的数，如2，3，4，5，6，7，8，9，本申请对此不作限制。这样，能够实现对该已解码的帧对应的重建图像进行高质量的帧内编码。
一种可能的方式中，编码器可以按照大于第二预设阈值的码率，对该已解码的帧对应的重建图像进行帧内编码，得到该已解码的帧对应的重建图像对应的帧内编码码流。其中，第二预设阈值可以按照需求设置，本申请对此不作限制。这样，也能够实现对该已解码的帧对应的的重建图像进行高质量的帧内编码。
需要说明的是，编码器对已解码的帧对应的重建图像，进行帧内编码所使用的量化参数，可以小于编码器对I帧进行帧内编码所使用的量化参数。示例性的，编码器对已解码的帧对应的重建图像进行帧内的码率，可以大于编码器对I帧进行帧内编码的码率。
示例性的，在接收到当前帧对应的码流后，可以解析当前帧对应的码流，以得到参考帧参数；然后基于参考帧参数，从解码器的外部参考列表中，确定当前帧的参考帧所对应的重建图像。
示例性的，可以根据参考帧参数中当前帧是否参考外部参考列表编码的相关描述信息(对于解码端而言，该描述信息为当前帧是否参考外部参考列表解码的相关描述信息)，判断当前帧是否需要参考外部参考列表解码。
示例性的，当确定当前帧无需参考外部参考列表解码时，可以直接将当前帧对应的码流输入至解码器中，由解码器对当前帧对应的码流进行解码，以得到对应的重建图像。示例性的，当当前帧为I帧时，解码器可以对当前帧对应的码流进行帧内解码。当当前 帧为P帧时，解码器可以对当前帧对应的码流进行帧间解码；其中，解码器可以根据当前帧的上一帧所对应的重建图像，对当前帧对应的码流进行帧间解码，以得到对应的重建图像。
示例性的，当确定当前帧需要参考外部参考列表解码时，可以再次根据参考帧参数中当前帧是否参考外部参考列表解码的相关描述信息，确定外部参考列表中当前帧的参考帧所对应的重建图像；然后执行S303～S305：
S303，对重建图像进行帧内编码，以得到对应的帧内编码码流。
S304，由解码器对帧内编码码流进行帧内解码。
S305，基于帧内解码的结果，由解码器对当前帧对应的码流进行帧间解码，以得到当前帧的重建图像。
示例性的，可以将从外部参考列表中确定的当前帧的参考帧所对应的重建图像，输入至编码器中进行帧内编码，以得到当前帧的参考帧所对应的重建图像的帧内编码码流。接着，可以将当前帧的参考帧所对应的重建图像的帧内编码码流输入至解码器中，由解码器对该帧内编码码流进行解码，以得到当前帧的参考帧所对应的重建图像的帧内编码重建图像。然后，将当前帧对应的码流输入至解码器中，由解码器以该帧内编码重建图像为参考，对当前帧对应的码流进行帧间解码，以得到当前帧的重建图像。
其中，编码器对当前帧的参考帧所对应的重建图像进行帧内编码的量化参数，可以小于第一预设阈值；或者，编码器以大于第二预设阈值的码率，对当前帧的参考帧所对应的重建图像进行帧内编码；具体可以参照上文中的描述，在此不再赘述。
需要说明的是，在得到当前帧的参考帧所对应的重建图像的帧内编码重建图像后，第二设备不显示该当前帧的参考帧所对应的重建图像的帧内编码重建图像；而在得到当前帧对应的重建图像后，显示该当前帧对应的重建图像。
这样，解码器也可以灵活从外部参考列表中选取参考帧进行解码，使得解码器能够实现跨帧参考编码或跨分辨率参考解码，提高了硬件解码器的参考帧管理的灵活性。
需要说明的是，本实施例中的解码器可以包括参考帧管理不灵活的解码器(如硬件解码器，或者是随着技术发展产生的其他解码器)，也可以包括考帧管理灵活的解码器(如软件解码器，或者是随着技术发展产生的其他解码器)，本申请对此不作限制。
示例性的，在得到当前帧对应的码流后，一种可能的方式中，可以修改当前帧对应的码流中，用于标识与当前帧的参考帧相关的标准语法元素；其中，标准语法元素是指标准编码协议中所规定的语法元素。其中，标准编码协议可以包括：H265、H264、H266、AV1、MPEG、AVS系列编码协议等，本申请对此不作限制。其中，需要修改的标准语法元素在后续进行说明。这样，第二设备的解码器在接收到码流后，按照现有的解码方式对码流的解码即可，在此不再赘述。
需要说明的是，当确定当前帧无需参考外部参考列表中的参考帧编码，且无需作为后续编码的参考帧时，可以无需生成参考帧参数，也无需对当前帧对应的码流进行修改。后续直接将对当前帧进行编码得到的待编码对应的码流，发送至服务器或第二设备即可。
示例性的，参考帧参数中，当前帧是否参考外部参考列表编码的相关描述信息，可以包括：第一标识和第一指示信息；第一标识用于标识当前帧是否基于外部参考列表编码，也可以用于标识当前帧是否参考长期参考帧或时域分层参考帧或重采样参考帧编码，第一指示信息用于指示外部参考列表中当前帧的参考帧所对应的重建图像。
示例性的，参考帧参数中，当前帧是否作为后续编码的参考帧的相关描述信息，可以包括：第二标识和第二指示信息；其中，第二标识和第二指示信息；第二标识用于标识当前帧是否作为后续编码的参考帧，第二指示信息用于指示外部参考列表中当前帧的重建图像。
示例性的，参考帧参数中，当前帧是否作进行重采样的相关描述信息，可以包括：第三标识、第三指示信息和重采样分辨率；第三标识用于标识当前帧是否进行重采样，第三指示信息用于指示外部参考列表中当前帧的重采样后的重建图像，重采样分辨率包括当前帧的重建图像对应的分辨率和重采样后的重建图像对应的分辨率。
应该理解的是，参考帧参数还可以包括其他参数，具体按照需求确定，本申请对此不作限制。
一种可能的方式中，可以将参考帧参数中第一标识和第一指示信息，添加至当前帧对应的码流中。
一种可能的方式中，可以将参考帧参数中第二标识和第二指示信息，添加至当前帧对应的码流中。
一种可能的方式中，可以将参考帧参数中第三标识、第三指示信息和重采样分辨率，添加至当前帧对应的码流中。
一种可能的方式中，可以将参考帧参数中第一标识、第一指示信息、第二标识和第二指示信息，添加至当前帧对应的码流中。
一种可能的方式中，可以将参考帧参数中第一标识、第一指示信息、第三标识、第三指示信息和重采样分辨率，添加至当前帧对应的码流中。
一种可能的方式中，可以将参考帧参数中的第一标识、第二标识和第三标识中的至少一种添加至当前帧对应的码流中。此时，第一标识用于标识当前帧不基于外部参考列表编码，第二标识用于标识当前帧不作为后续编码的参考帧，第三标识用于标识当前帧不进行重采样。
一种可能的方式中，可以将当前帧的参考帧参数，添加至当前帧对应的码流中的参数集(如VPS(Video Parameter Set，视频参数集)、SPS(Sequence Parameter Set，序列参数集)、PPS(Picture Parameter Set，图像参数集))中。
一种可能的方式中，在按照传输协议对当前帧对应的码流进行封装的过程中，将当前帧的参考帧参数，封装传输协议包的包头中。例如将当前帧的参考帧参数，封装在RTP(Real-time Transport Rrotocol，实时传输协议)的扩展头中。
一种可能的方式中，在按照传输协议对当前帧对应的码流进行封装的过程中，可以将参考帧参数均作为负载数据进行封装。
一种可能的方式中，可以将当前帧对应的码流和参数帧参数打包成文件格式(file  format)，然后再按照传输协议封装该文件格式。示例性的，文件格式可以包括标准ISO14496-12制定的文件格式。例如，文件格式为标准ISO14496-12制定的文件格式，则可以在将参考帧参数添加至track hearder()中，也可以增加参考帧参数对应的box()。示例性的，文件可以也可以为MPEG2TS文件格式。应该理解的是，本申请对文件格式不作限制。
示例性的，本申请提供的编解码方法对应的编解码场景可以包括多种，如使用长期参考帧编解码的场景，时域分层编解码场景以及参考帧重采样场景等，本申请对此不作限制。以下分别对使用长期参考帧编解码的场景，时域分层编解码场景以及参考帧重采样场景中的编解码方法进行说明。
图4为示例性示出的编码流程示意图。在图4的实施例中，描述了使用长期参考帧编码的场景。其中，在图4的实施例中，预设参考规则为长期参考规则，预设参考帧设置规则为长期参考帧设置规则。
S401，获取当前帧。
S402，判断是否满足长期参考规则。
示例性的，可以根据长期参考帧的参考周期、丢包反馈信息和应用场景等设置长期参考规则，本申请对此不作限制。示例性的长期参考规则也可以包括：参考判断规则和参考帧确定规则。当确定满足长期参考规则中的参考判断规则时，可以确定满足长期参考规则；此时，可以说明当前帧需要参考长期参考帧编码，可以执行S403～S406。当不满足长期参考规则中的参考判断规则时，可以确定不满足长期参考规则；此时，可以说明确定当前帧无需参考长期参考帧编码，可以执行S407。
一种可能的方式中，根据长期参考帧的参考周期，设置长期参考规则。例如，可以将长期参考规则中的参考判断规则设置为：达到长期参考帧的参考周期(可以是指与当前帧距离最近的且参考长期参考帧的已编码的帧，至当前帧的距离差，达到长期参考帧的参考周期)；将长期参考规则中的参考帧确定规则设置为：外部参考列表中与当前帧距离最近的长期参考帧。示例性的，长期参考帧的参考周期可以按照需求设置如5帧，本申请对此不作限制。假设以一个GOP(Group OfPicture，图像组)为例，若长期参考帧的参考周期为5，则当当前帧为该GOP中的第5帧(POC(Picture Order Count，图像顺序号)＝4)时，则可以确定达到了长期参考帧的参考周期，此时可以确定满足长期参考规则。
一种可能的方式中，可以根据丢包反馈信息，设置长期参考规则。例如，可以将长期参考规则中的参考判断规则设置为：当前帧的预设参考帧对应的码流丢失；将长期参考规则中的参考帧确定规则设置为：外部参考列表中与当前帧距离最近的长期参考帧。其中，预设参考帧可以是指在对当前帧进行编码之前，预先为当前帧指定的参考帧。假设当前帧为该GOP中的第5帧(POC＝4)，当前帧的预设参考帧为该GOP中的第4帧(POC＝3)；若接收到该GOP中第4帧码流丢失的丢包反馈信息，则可以确定满足长期参考规则。
一种可能的方式中，可以根据应用场景，设置长期参考规则。例如，可以将长期参 考规则中的参考判断规则设置为：当前应用场景为桌面共享场景，且外部参考列表中存在与当前帧的相似度高于相似度阈值的参考帧；将长期参考规则中的参考帧确定规则设置为：外部参考列表中与当前帧相似度最高的长期参考帧。其中，相似度阈值可以按照需求设置，本申请对此不作限制。
S403，当确定满足长期参考规则时，根据长期参考规则确定参考指示信息。
示例性的，对于任一已编码的帧而言，在被确定为后续编码的参考帧时，可以将该已编码的帧所对应的重建图像加入外部参考列表，并为该已编码的帧所对应的重建图像分配一帧标识。其中，帧标识可以用于唯一标识外部参考列表中的一个重建图像，可以按照需求确定，如帧序号、POC、PTS(Presentation Time Stamp显示时间戳)等等，本申请对此不作限制。此外，也可以根据预设计算规则(如长期参考帧的设置周期，和与其最近的I帧的距离等)所计算得到的可唯一指定已编码的帧所对应的重建图像的标识。
这样，当确定满足长期参考规则时，可以根据长期参考规则中参考帧确定规则，确定当前帧的长期参考帧所对应的重建图像的帧标识，然后将该帧标识，确定为参考指示信息。
例如，当长期参考规则中参考帧确定规则为：外部参考列表中与当前帧距离最近的长期参考帧时，可以将外部参考列表中与当前帧距离最近的长期参考帧所对应重建图像的帧标识，作为参考指示信息。又例如，当长期参考规则中参考帧确定规则为：外部参考列表中与当前帧相似度最高的长期参考帧时，可以将外部参考列表中与当前帧相似度最高的长期参考帧所对应的重建图像的帧标识，作为参考指示信息。
S404，根据参考指示信息，从编码器的外部参考列表中，获取当前帧的长期参考帧所对应的重建图像。
示例性的，可以将参考指示信息，与外部参考列表中各参考帧所对应重建图像的帧标识匹配，来从编码器的外部参考列表中，获取所述当前帧的长期参考帧所对应的重建图像。
S405，由编码器对当前帧的长期参考帧所对应的重建图像进行帧内编码。
S406，基于当前帧的长期参考帧所对应的重建图像的帧内编码重建图像，由编码器对当前帧进行帧间编码，以得到当前帧对应的码流。
示例性的，S405～S406可以参照上述S302～S304的描述，在此不再赘述。
S407，当确定不满足长期参考规则时，由编码器基于当前帧的上一帧所对应的重建图像对当前帧进行帧间编码，以得到当前帧对应的码流。
示例性的，当确定不满足长期参考规则时，也就是说，确定当前帧无需参考长期参考帧时，可以直接将当前帧输入至编码器。然后，编码器可以基于当前帧的上一帧所对应的重建图像对当前帧进行帧间编码，以得到该当前帧对应的码流。
S408，当确定需要将当前帧作为后续编码的长期参考帧时，基于当前帧对应的码流进行解码，生成当前帧的重建图像。
示例性的，可以根据长期参考帧的设置周期、信道质量和应用场景等设置长期参考帧设置规则，本申请对此不作限制。进而在完成对当前帧的编码后，可以通过判断是否满足长期参考帧设置规则，来判断是否需要将当前帧作为后续编码的长期参考帧。当确 定满足长期参考帧设置规则时，则确定需要将当前帧作为后续编码的长期参考帧，可以将当前帧的重建图像加入至编码器的外部参考列表。当确定不满足长期参考帧设置规则时，则确定无需将当前帧作为后续编码的长期参考帧，此时可以结束编码流程。
一种可能的方式中，可以根据长期参考帧的设置周期，设置长期参考帧设置规则。例如，长期参考帧设置规则可以设置为：达到长期参考帧的设置周期(可以是指与当前帧距离最近的长期参考帧，至当前帧的距离差，达到长期参考帧的设置周期)。一种可能的方式中，长期参考帧的设置周期可以是预先设置的固定值如4；本申请对此不作限制。假设以一个GOP为例，若长期参考帧的设置周期为4，则当当前帧为该GOP中的第4帧(POC＝3)时，则可以确定达到了长期参考帧的设置周期，此时可以确定满足长期参考帧设置规则，即确定需要将当前帧作为后续编码的长期参考帧。
一种可能的方式中，长期参考帧的设置周期可以是根据信道质量调整的动态值，本申请对此不作限制。例如，检测到信道质量较差时，若根据丢包反馈信息确定连续4帧的码流被成功接收时，可以将长期参考帧的设置周期调整为4；若根据丢包反馈信息确定连续3帧的码流被成功接收时，可以将长期参考帧的设置周期调整为3。
一种可能的方式中，可以根据信道质量，设置长期参考帧设置规则。例如，长期参考帧设置规则可以设置为：信道质量低于质量阈值；其中，质量阈值可以按照需求设置，本申请对此不作限制。例如，若检测到信道质量低于质量阈值，则可以确定满足长期参考帧设置规则，即确定需要将当前帧作为后续编码的长期参考帧。若检测到信道质量高于或等于质量阈值，则可以确定不满足长期参考帧设置规则，即确定无需将当前帧作为后续编码的长期参考帧。
一种可能的方式中，可以根据应用场景，设置长期参考帧设置规则。例如，长期参考帧设置规则可以设置为：应用场景为桌面共享场景，且与上一帧的差异大于差异阈值；其中，差异阈值可以按照需求设置，本申请对此不作限制。例如，差异阈值为0.6，在桌面共享场景中，当前帧与上一帧的差异大于或等于0.6时，则可以确定满足长期参考帧设置规则，即确定需要将当前帧作为后续编码的长期参考帧。当前帧与上一帧的差异小于0.6时，则可以确定不满足长期参考帧设置规则，即确定无需将当前帧作为后续编码的长期参考帧。
S409，将当前帧的重建图像，加入外部参考列表。
示例性的，可以将当前帧的重建图像，加入至外部参考列表，并为该当前帧的重建图像设置对应的帧标识。
示例性的，在得到当前帧对应的码流后，可以在当前帧对应的码流中添加参考帧参数。在图4的示例性中，参考帧参数中的第一标识可以用于标识当前帧是否参考长期参考帧编码，第一指示信息可以用于指示外部参考列表中当前帧的长期参考帧所对应的重建图像，第二标识可以用于标识当前帧是否作为后续编码的长期参考帧，第二指示信息用于指示外部参考列表中当前帧的重建图像。
示例性的，当确定满足长期参考规则，且当前帧已基于外部参考列表中的长期参考帧编码后，则可以将第一标识赋值为第一预设信息(如“A1”)，且将第一指示信息赋值为外部参考列表中当前帧的重建图像对应的帧标识。当不确定满足长期参考规则，则 可以将第一标识配置为第二预设信息(如“A0”)，且将第一指示信息赋值为“NULL”。
示例性的，当确定需要将当前帧作为后续编码的长期参考帧，且将当前帧的重建图像加入外部参考列表后，则可以将第二标识配置为第三预设信息(如“B1”)，且将第二指示信息赋值为，外部参考列表中当前帧的长期参考帧所对应的重建图像对应的帧标识。当确定无需将当前帧作为后续编码的长期参考帧时，则可以将第二标识配置为第四预设信息(如“B0”)，并将第二指示信息赋值为“NULL”。
图5a为示例性示出的解码流程示意图。在图5a的实施例中描述的是长期参考帧解码的场景，与图4实施例中的编码过程对应。
S501，接收当前帧对应的码流。
S502，解析当前帧对应的码流，以得到参考帧参数。
S503，基于参考帧参数，判断是否当前帧是否需要参考长期参考帧解码。
示例性的，在从当前帧对应的码流中解析出参考帧参数后，可以判断参考帧参数中是否存在第一标识。当参考帧参数中存在第一标识时，判断第一标识的值是否为第一预设信息。若第一标识的值为第一预设信息，则确定当前帧需要参考长期参考帧解码；此时，可以执行S504～S507。若第一标识的值为第二预设信息，或者参考帧参数不包含第一标识，则确定当前帧无需参考长期参考帧解码，此时可以执行S508。
S504，当确定当前帧需要参考长期参考帧解码时，基于参考帧参数从解码器的外部参考列表中，确定当前帧的长期参考帧所对应的重建图像，外部参考列表中的长期参考帧为已解码的帧。
示例性的，当确定当前帧需要参考长期参考帧解码时，从解码器的外部参考列表中，将与参考帧参数中第一指示信息匹配的重建图像，确定为当前帧的长期参考帧所对应的重建图像。
S505，对重建图像进行帧内编码，以得到对应的帧内编码码流。
S506，由解码器对帧内编码码流进行帧内解码。
S507，基于对帧内编码码流进行解码得到的帧内编码重建图像，由解码器对当前帧对应的码流进行帧间解码，以得到当前帧的重建图像。
示例性的，S505～S507可以参照上述S303～S305的描述，在此不再赘述。
S508，当确定当前帧无需参考长期参考帧解码时，基于当前帧的上一帧所对应的重建图像，由解码器对当前帧对应的码流进行帧间解码，以得到当前帧的重建图像。
示例性的，当确定当前帧无需参考长期参考帧解码时，可以将当前帧对应的码流直接输入至解码器中，然后由解码器可以基于当前帧的上一帧所对应的重建图像，对当前帧对应的码流进行帧间解码，以得到当前帧的重建图像。
示例性的，可以基于参考帧参数，判断是否需要将当前帧作为后续解码的长期参考帧。示例性的，可以判断参考帧参数中是否包括第二标识；当参考帧参数中包括第二标识时，可以判断第二标识的值是否为第三预设信息。当第二标识的值为第三预设信息时，可以确定需要将当前帧作为后续解码的长期参考帧，此时可以执行S509。当第二标识的值为第四预设信息，或者参考帧参数中不包括第二标识时，可以确定无需将当前帧作为 后续解码的长期参考帧，可以结束解码流程。
S509，当确定需要将当前帧作为后续解码的长期参考帧时，将当前帧的重建图像，加入至外部参考列表中。
示例性的，可以将当前帧的重建图像，加入至外部参考列表，并为该当前帧的重建图像设置对应的帧标识。
当然，也可以对当前帧的重建图像进行帧内编码，以得到当前帧的重建图像所对应的帧内编码码流；然后将当前帧的重建图像所对应的帧内编码码流，加入外部参考列表中；具体可以参照上文的描述，在此不再赘述。
图5b为示例性示出的编解码过程示意图。图5b(1)的第一设备中的编码器为硬件编码器，第二设备中的解码器为硬件解码器。图5b(2)中左边的区域中灰色以及黑色的帧为第一设备发送的帧，右边的区域中灰色以及黑色的帧为第二设备接收到的帧；其中，黑色的帧为长期参考帧。
结合图5b(1)和图5b(2)，假设待编码的视频序列为IPPPPP，长期参考帧设置规则为：达到长期参考帧的设置周期，长期参考帧的设置周期为4；长期参考规则为：达到长期参考帧的参考周期，长期参考帧的参考周期为5。硬件编码器在对该视频序列编码的过程中，t0时刻，当前帧为I帧时，可以对I帧进行帧内编码，得到I帧码流，然后将I帧码流发送给第二设备。t1时刻，当前帧为第1个P帧，硬件编码器可以参考I帧重建图像对第1个P帧进行编码，得到第1个P帧的码流，然后将第1个P帧的码流发送给第二设备。t2时刻，当前帧为第2个P帧，硬件编码器可以参考第1个P帧的重建图像对第2个P帧进行编码，得到第2个P帧的码流，然后将第2个P帧的码流发送给第二设备。t3时刻，当前帧为第3个P帧，硬件编码器可以参考第2个P帧的重建图像对第3个P帧进行编码，得到第3个P帧的码流。由于t3时刻达到了长期参考帧的设置周期4，因此可以将第3个P帧的码流输入至解码器，得到解码器输出的第3个P帧的重建图像，然后将第3个P帧的重建图像存储至外部参考列表中。接着，可以在第3个P帧的码流中添加第二标识并将第二标识赋值为第三预设信息，以及添加第二指示信息(如3)，然后将添加参考帧参数的第3个P帧的码流发送给第二设备。t4时刻，当前帧为第4个P帧，由于此时达到了长期参考帧的参考周期5，因此硬件编码器可以从外部参考列表中，获取第3个P帧的重建图像，然后硬件编码器可以对第3个P帧的重建图像进行帧内编码，可以得到第3个P帧的重建图像的帧内编码码流(不发送)，以及第3个P帧的重建图像的帧内编码重建图像。然后再将第4个P帧输入至硬件编码器中，由硬件编码器参考第3个P帧的重建图像的帧内编码重建图像，对第4个P帧进行编码，得到第4个P帧的码流。接着，可以在第4个P帧的码流中添加第一标识并将第一标识赋值为第一预设信息，以及添加第一指示信息(如3)，然后将添加参考帧参数的第4个P帧的码流发送给第二设备。t5时刻，当前帧为第5个P帧，硬件编码器可以参考第4个P帧的重建图像对第5个P帧进行编码，得到第5个P帧的码流，然后将第5个P帧的码流发送给第二设备。
结合图5b(1)和图5b(2)，硬件解码器在对该视频序列解码的过程中，t0时刻， 当前帧为I帧时，可以对I帧的码流进行帧内解码，得到I帧的重建图像，然后将I帧重建图像发送给显示模块显示。t1时刻，当前帧为第1个P帧，硬件解码器可以参考I帧重建图像对第1个P帧的码流进行解码，得到第1个P帧的重建图像，然后将第1个P帧的重建图像发送给显示模块。t2时刻，当前帧为第2个P帧，硬件解码器可以参考第1个P帧的重建图像对第2个P帧的码流进行解码，得到第2个P帧的重建图像，然后将第2个P帧的重建图像发送给显示模块。t3时刻，当前帧为第3个P帧，硬件解码器可以参考第2个P帧的重建图像对第3个P帧的码流进行解码，得到第3个P帧的重建图像，然后将第3个P帧的重建图像发送给显示模块。在得到硬件解码器输出的第3个P帧的重建图像后，可以根据第二标识和第二指示信息，将第3个P帧的重建图像存储至外部参考列表中。t4时刻，当前帧为第4个P帧，可以根据第一标识和第一指示信息，从外部参考列表中，获取第3个P帧的重建图像；然后将第3个P帧的重建图像输入至编码器，编码器可以对第3个P帧的重建图像进行帧内编码，可以得到第3个P帧的重建图像的帧内编码码流。接着，编码器将第3个P帧的重建图像的帧内编码码流发送给硬件解码器，硬件解码器可以解码得到第3个P帧的重建图像的帧内解码重建图像(不发送)。然后再将第4个P帧的码流输入至硬件解码器中，由硬件解码器根据第3个P帧的重建图像的帧内解码重建图像，对第4个P帧的码流进行解码，得到第4个P帧的重建图像，然后将第4个P帧的重建图像发送给显示模块。t5时刻，当前帧为第5个P帧，硬件解码器可以参考第4个P帧的重建图像对第5个P帧的码流进行解码，得到第5个P帧的重建图像，然后将第5个P帧的重建图像发送给显示模块。
一种可能的方式中，第一设备在执行完S401～S408后，可以修改当前帧对应的码流中，用于标识与当前帧的参考帧相关的标准语法元素；其中，标准语法元素是指标准编码协议中所规定的语法元素。
示例性的，第一设备在执行完S401～S408后，需要修改的用于标识与所述当前帧的参考帧相关的标准语法元素包括以下至少一种：用于标识当前帧的POC值的语法元素，用于标识长期参考帧数量的语法元素；用于标识长期参考帧图像顺序号POC的语法元素；用于标识长期参考帧是否用于当前帧参考的语法元素。当然，还可以包括其他用于标识与所述当前帧的参考帧相关的标准语法元素，本申请对此不作限制。
以编码协议H265为例，当前帧对应的码流中用于与所述当前帧的参考帧相关的标准语法元素，可以如表1所示：
表1
应该理解的是，表1中所示的语法元素，仅是当前帧对应的码流中，用于标识与所述当前帧的参考帧相关的标准语法元素的一些示例，本申请中当前帧对应的码流中用于标识与所述当前帧的参考帧相关的标准语法元素，可以具有比表1中更多或更少的语法元素，在此不再赘述。
假设当前帧为第6帧(POC＝5)，且需要参考长期参考帧编码。由于在对当前帧编码之前，编码器对当前帧的参考帧所对应的重建图像进行了帧内编码，得到的帧内编码码流中的POC为0；因此编码器对当前帧进行编码得到的码流中POC为1。为了使得解码器能够正常解码，可以将当前帧对应的码流中图像顺序号POC修改为5，即将pic_order_cnt_lsb的值由0修改为5；以使所述当前帧对应的码流中POC与所述当前帧的前一帧对应的码流中POC连续。
假设，当前帧的长期参考帧为第4帧。若外部参考列表包括两个长期参考帧，这两个长期参考帧为第1帧(POC＝0)和第4帧(POC＝3)；则可以将num_long_term_pics的值由0修改为2，将poc_lsb_lt[0]的值由“NULL”的修改为0，poc_lsb_lt[1]的值由“NULL”的修改为3；以及将used_by_curr_pic_lt_flag[0]由“NULL”修改为0，将used_by_curr_pic_lt_flag[1]由“NULL”修改为1；其中，used_by_curr_pic_lt_flag[0]＝0表示该第1帧不用于当前帧参考，used_by_curr_pic_lt_flag[1]＝1表示该第4帧用于当前帧参考。
图5c为示例性示出的编解码过程示意图。图5c(1)的第一设备中的编码器为硬件编码器，第二设备中的解码器为硬件编解码或软件解码器。图5c(2)中左边的区域中灰色以及黑色的帧为第一设备发送的帧，右边的区域中灰色以及黑色的帧为第二设备接收到的帧；其中，黑色的帧表示长期参考帧。
图5c的编码过程与图5b中的编码过程，除了对第3个P帧和第4个P帧的编码过程不同之外，其余帧的编码过程类似，在此不再赘。其中，图5c中，无需在第3个P帧的码流中添加参考帧参数，而是需要修改第4个P帧的码流中的语法元素。其中，修改后的第4个P帧的码流的语法元素的值可以如：pic_order_cnt_lsb＝4，num_long_term_pics＝1，poc_lsb_lt[0]＝2，used_by_curr_pic_lt_flag[0]＝1。
图5c的解码过程与图5b中的解码过程除了对第4个P帧的解码过程不同之外，其余帧的解码过程类似，在此不再赘述。第二设备接收到第4个P帧的码流后，可以参考第2个P帧的重建图像进行解码，得到第4个P帧的重建图像。
图6为示例性示出的编码流程示意图。在图6的实施例中，描述了将时域分层编码场景。其中，图6的实施例中的预设参考规则为时域分层参考规则，预设参考帧设置规则为时域分层参考帧设置规则。
S601，获取当前帧。
S602，判断是否满足时域分层参考规则。
示例性的，可以根据时域分层结构，设置时域分层参考规则。示例性的时域分层参考规则也可以包括：参考判断规则和参考帧确定规则。
若时域分层结构为时域分层结构1：第1帧(POC＝0)、第3帧(POC＝2)、第5帧(POC＝4)、第7帧(POC＝6)为T0层(即TL(Temporal Layer，时域层)＝0)；第 2帧(POC＝1)、第4帧(POC＝3)、第6帧(POC＝5)、第8帧(POC＝7)为T1层(即TL＝1)，也就是说，时域层的数量为2。对应的参考方式为参考方式1：第3帧参考第1帧编码，第5帧参考第3帧编码，第7帧参考第5帧编码；第2帧参考第1帧编码，第4帧参考第3帧编码，第6帧参考第5帧编码，第8帧参考第7帧编码。这种情况下，可以设置时域分层参考规则中的参考判断规则为：时域分层结构1，POC值为除8×(n-1)之外的整数(n的取值为0，1,2，......)；参考帧确定条件为：参考方式1。
若时域分层结构为时域分层结构2：第1帧(POC＝0)、第5帧(POC＝4)和第9帧(POC＝8)为T0层，第3帧(POC＝2)、第7帧(POC＝6)和第11帧(POC＝10)为T1层，第2帧(POC＝1)、第4帧(POC＝3)、第6帧(POC＝5)、第8帧(POC＝7)、第10帧(POC＝9)和第12帧(POC＝11)为T2层(即TL＝2)。对应的参考方式是参考方式2：第2帧参考第1帧编码，第3帧参考第1帧编码，第4帧参考第3帧编码；第5帧参考第1帧编码，第6帧参考第5帧编码，第7帧参考第5帧编码，第8帧参考第7帧编码；第9帧参考第5帧编码，第10帧参考第9帧编码，第11帧参考第9帧编码，第12帧参考第11帧编码。这种情况下，可以设置时域分层参考规则中的参考判断规则为：时域分层结构2，POC值为除12×(n-1)之外的整数(n的取值为0，1,2，.....)；参考帧确定条件为：参考方式2。
一种可能的方式中，时域分层结构可以预先设定，也就是说，时域分层结构是固定值。
一种可能的方式中，时域分层结构可以根据信道反馈信息确定；例如，当根据信道反馈信息确定信道质量较好时，可以增加时域层的数量；当根据信道反馈信息确定信道质量较差时，可以减少时域层的数量，本申请对此不作限制。这种情况下，时域分层结构是动态的。
示例性的，当满足时域分层参考规则时，确定当前帧需要参考时域分层参考帧编码；此时可以执行S603～S606。当不满足时域分层参考规则时，确定当前帧无需参考时域分层参考帧编码，可以执行S607。
S603，当确定满足时域分层参考规则时，根据时域分层参考规则确定参考指示信息。
示例性的，当确定满足时域分层参考规则时，可以根据时域分层参考规则中参考帧确定条件，确定当前帧的时域分层参考帧所对应的重建图像的帧标识，然后将该帧标识，确定为参考指示信息。
例如，假设时域分层参考规则中参考帧确定条件为参考方式1，则当前帧为所属GOP的第3帧时，可以确定该GOP中的第1帧为其参考帧；此时可以确定外部参考列表中，第1帧的重建图像的帧标识，作为参考指示信息。
例如，假设时域分层参考规则中参考帧确定条件为参考方式2，则当前帧为所属GOP的第4帧时，可以确定该GOP中的第3帧为其参考帧；此时可以确定外部参考列表中，第3帧的重建图像的帧标识，作为参考指示信息。
S604，根据所述参考指示信息，从编码器的外部参考列表中，获取所述当前帧的时域分层次参考帧所对应的重建图像。
示例性的，可以将参考指示信息，与外部参考列表中各参考帧所对应重建图像的帧 标识匹配，来从编码器的外部参考列表中，获取所述当前帧的时域分层参考帧所对应的重建图像。
S605，由编码器对当前帧的时域分层参考帧所对应的重建图像进行帧内编码。
S606，基于当前帧的时域分层参考帧所对应的重建图像的帧内编码重建图像，由编码器对当前帧进行帧间编码，以得到当前帧对应的码流。
示例性的，S605～S606可以参照上述S302～S304的描述，在此不再赘述。
S607，当确定不满足时域分层参考规则时，由编码器基于当前帧的上一帧所对应的重建图像对当前帧进行帧间编码，以得到当前帧对应的码流。
示例性的，S607可以参照上述S407的描述，在此不再赘述。
S608，当确定需要将当前帧作为后续编码的时域分层参考帧时，基于当前帧对应的码流，生成当前帧的重建图像。
示例性的，可以根据时域分层结构，设置时域分层设置规则。进而在完成对当前帧的编码后，可以根据时域分层设置规则，来确定是否需要将当前帧作为后续编码的时域分层参考帧。当确定满足时域分层设置规则时，则确定需要将当前帧作为后续编码的时域分层参考帧，可以将当前帧的重建图像加入至编码器的外部参考列表中。当确定不满足时域分层设置规则时，则确定无需将当前帧作为后续编码的时域分层参考帧，此时无需将当前帧的重建图像加入至编码器的外部参考列表中，可以结束编码流程。
例如，时域分层设置规则可以是：时域分层结构1，POC值为除8×(n-1)之外的整数。假设时域分层结构为上述的时域分层结构1时，若当前帧为第1帧或第3帧或第5帧或第7帧时，则可以确定满足时域分层设置规则，即需要将当前帧作为后续编码的时域分层参考帧。
S609，将当前帧的重建图像，加入外部参考列表。
示例性的，可以将当前帧的重建图像，加入至外部参考列表中，并为该当前帧的重建图像设置对应的帧标识。
示例性的，在得到当前帧对应的码流后，可以在当前帧对应的码流中添加参考帧参数。在图6的示例性中，参考帧参数中的第一标识可以用于标识当前帧是否参考时域分层参考帧帧编码，第一指示信息可以用于指示外部参考列表中当前帧的时域分层参考帧所对应的重建图像，第二标识可以用于标识当前帧是否作为后续编码的时域分层参考帧，第二指示信息用于指示外部参考列表中当前帧的重建图像。
示例性的，当确定满足时域分层参考规则，且当前帧已基于外部参考列表中的时域分层参考帧编码后，则可以将第一标识赋值为第五预设信息(如“C1”)，且将第一指示信息赋值为外部参考列表中当前帧的重建图像对应的帧标识。当不确定满足时域分层参考规则，则可以将第一标识配置为第六预设信息(如“C0”)，且将第一指示信息赋值为“NULL”。
示例性的，当确定需要将当前帧作为后续编码的时域分层参考帧，且将当前帧的重建图像加入外部参考列表后，则可以将第二标识配置为第七预设信息(如“D1”)，且将第二指示信息赋值为，外部参考列表中当前帧的时域分层参考帧所对应的重建图像对应的帧标识。当确定无需将当前帧作为后续编码的时域分层参考帧时，则可以将第二标 识配置为第八预设信息(如“D0”)，并将第二指示信息赋值为“NULL”。
图7a为示例性示出的解码流程示意图。在图7a的实施例中描述的是时域分层解码的场景，与图6实施例中的编码过程对应。
S701，接收当前帧对应的码流。
S702，解析当前帧对应的码流，以得到参考帧参数。
S703，基于参考帧参数，判断是否当前帧是否需要参考时域分层参考帧解码。
示例性的，在从当前帧对应的码流中解析出参考帧参数后，可以判断参考帧参数中是否存在第一标识。当参考帧参数中存在第一标识时，判断第一标识的值是否为第五预设信息。若第一标识的值为第五预设信息，则确定当前帧需要参考时域分层参考帧解码；此时，可以执行S704～S707。若第一标识的值为第六预设信息，或者参考帧参数不包含第一标识，则确定当前帧无需参考时域分层参考帧解码，此时可以执行S708。
S704，当确定当前帧需要参考时域分层参考帧解码时，基于参考帧参数从解码器的外部参考列表中，确定当前帧的时域分层参考帧所对应的重建图像，外部参考列表中的时域分层参考帧为已解码的帧。
示例性的，当确定当前帧需要参考时域分层参考帧解码时，从解码器的外部参考列表中，将与参考帧参数中第一指示信息匹配的重建图像，确定为当前帧的时域分层参考帧所对应的重建图像。
S705，对重建图像进行帧内编码，以得到对应的帧内编码码流。
S706，由解码器对帧内编码码流进行帧内解码。
S707，基于对帧内编码码流进行解码得到的帧内编码重建图像，由解码器对当前帧对应的码流进行帧间解码，以得到当前帧的重建图像。
示例性的，S705～S707可以参照上述S303～S305的描述，在此不再赘述。
S708，当确定当前帧无需参考时域分层参考帧解码时，基于当前帧的上一帧所对应的重建图像，由解码器对当前帧对应的码流进行帧间解码，以得到当前帧的重建图像。
示例性的，当确定当前帧无需参考时域分层参考帧解码时，可以将当前帧对应的码流直接输入至解码器中，然后由解码器基于当前帧的上一帧所对应的重建图像，对当前帧对应的码流进行解码，以得到当前帧的重建图像。
示例性的，可以基于参考帧参数，判断是否需要将当前帧作为后续解码的时域分层参考帧。示例性的，可以判断参考帧参数中是否包括第二标识；当参考帧参数中包括第二标识时，可以判断第二标识的值是否为第七预设信息。当第二标识的值为第七预设信息时，可以确定需要将当前帧作为后续解码的时域分层参考帧，此时可以执行S709。当第二标识的值为第八预设信息，或者参考帧参数中不包括第二标识时，可以确定无需将当前帧作为后续解码的时域分层参考帧，可以结束解码流程。
S709，当确定需要将当前帧作为后续解码的时域分层参考帧时，将所述当前帧的重建图像，加入至外部参考列表中。
示例性的，可以将当前帧的重建图像，加入至外部参考列表中，并为该当前帧的重建图像设置对应的帧标识。
当然，也可以对所述当前帧的重建图像进行帧内编码，以得到所述当前帧的重建图像所对应的帧内编码码流；然后将所述当前帧的重建图像所对应的帧内编码码流，加入所述外部参考列表中；具体可以参照上文的描述，在此不再赘述。
图7b为示例性示出的编解码过程示意图。图7b(1)的第一设备中的编码器为硬件编码器，第二设备中的解码器为硬件解码器。图7b(2)中左边的区域中灰色以及黑色的帧为第一设备发送的帧，右边的区域中灰色以及黑色的帧为第二设备接收到的帧；其中，黑色的帧为时域分层参考帧。
假设，时域分层结构为：第1帧(POC＝0)、第3帧(POC＝2)、第5帧(POC＝4)、为T0层；第2帧(POC＝1)、第4帧(POC＝3)、第6帧(POC＝5)为T1层；对应的参考方式为：第3帧参照第1帧编码，第5帧参照第3帧编码，第2帧参照第1帧编码，第4帧参照第3帧编码，第6帧参照第5帧编码。
结合图7b(1)和图7b(2)，硬件编码器在对该视频序列编码的过程中，t0时刻，当前帧为I帧时，可以对I帧进行帧内编码，得到I帧码流。由于I帧需要作为后续编码的时域参考帧，因此可以将I帧的码流输入至解码器，得到解码器输出的I帧的重建图像，然后将I帧的重建图像存储至外部参考列表中。接着，可以在I帧的码流中添加第二标识并将第二标识赋值为第三预设信息，以及添加第二指示信息(如0)，然后将添加参考帧参数的I帧的码流发送给第二设备。t1时刻，当前帧为第1个P帧，硬件编码器可以参考I帧的重建图像对第1个P帧进行编码，得到第1个P帧的码流，然后将第1个P帧的码流发送给第二设备。t2时刻，当前帧为第2个P帧，硬件编码器可以从外部参考列表中，获取I帧的重建图像，然后可以对I帧的重建图像进行帧内编码，可以得到I帧的重建图像的帧内编码码流(不发送)，以及I帧的重建图像的帧内编码重建图像。然后再将第2个P帧输入至硬件编码器中，由硬件编码器参考I帧的重建图像的帧内编码重建图像，对第2个P帧进行编码，得到第2个P帧的码流。接着，可以在第2个P帧的码流中添加第一标识并将第一标识赋值为第一预设信息，以及添加第一指示信息(如0)；以及添加第二标识并将第二标识赋值为第三预设信息，以及添加第二指示信息(如2)；然后将添加参考帧参数的第2个P帧的码流发送给第二设备。t3时刻，当前帧为第3个P帧，硬件编码器可以参考第2个P帧的重建图像对第3个P帧进行编码，得到第3个P帧的码流，然后将第3个P帧的码流发送给第二设备。t4时刻，当前帧为第4个P帧，硬件编码器可以从外部参考列表中，获取第3个P帧的重建图像，然后硬件编码器可以对第3个P帧的重建图像进行帧内编码，可以得到第3个P帧的重建图像的帧内编码码流(不发送)，以及第3个P帧的重建图像的帧内编码重建图像。然后再将第4个P帧输入至硬件编码器中，由硬件编码器参考第3个P帧的重建图像的帧内编码重建图像，对第4个P帧进行编码，得到第4个P帧的码流。接着，可以在第4个P帧的码流中添加第一标识并将第一标识赋值为第一预设信息，以及添加第一指示信息(如2)；以及添加第二标识并将第二标识赋值为第三预设信息，以及添加第二指示信息(如4)；然后将添加参考帧参数的第4个P帧的码流发送给第二设备。t5时刻，当前帧为第5个P帧，硬件编码器可以参考第4个P帧的重建图像对第5个P帧进行编码，得到第5个P帧的 码流，然后将第5个P帧的码流发送给第二设备。
结合图7b(1)和图7b(2)，硬件解码器在对该视频序列解码的过程中，t0时刻，当前帧为I帧时，可以对I帧的码流进行帧内解码，得到I帧的重建图像；然后一方面，将I帧重建图像发送给显示模块显示，另一方面，可以根据第二标识和第二指示信息，将I帧的重建图像存储至外部参考列表中。t1时刻，当前帧为第1个P帧，硬件解码器可以参考I帧重建图像对第1个P帧的码流进行解码，得到第1个P帧的重建图像，然后将第1个P帧的重建图像发送给显示模块。t2时刻，当前帧为第2个P帧，可以根据第一标识和第一指示信息，从外部参考列表中，获取I帧的重建图像；然后将I帧的重建图像输入至编码器，编码器可以对I帧的重建图像进行帧内编码，可以得到I帧的重建图像的帧内编码码流。接着，编码器将I帧的重建图像的帧内编码码流发送给硬件解码器，硬件解码器可以解码得到I帧的重建图像的帧内解码重建图像(不发送)。然后再将第2个P帧的码流输入至硬件解码器中，由硬件解码器根据I帧的重建图像的帧内解码重建图像，对第2个P帧的码流进行解码，得到第2个P帧的重建图像，然后一方面，将第2个P帧的重建图像发送给显示模块；另一方面，可以根据第二标识和第二指示信息，将第2个P帧的重建图像存储至外部参考列表中。t3时刻，当前帧为第3个P帧，硬件解码器可以参考第2个P帧的重建图像对第3个P帧的码流进行解码，得到第3个P帧的重建图像，然后将第3个P帧的重建图像发送给显示模块。t4时刻，当前帧为第4个P帧，可以根据第一标识和第一指示信息，从外部参考列表中，获取第2个P帧的重建图像；然后将第2个P帧的重建图像输入至编码器，编码器可以对第2个P帧的重建图像进行帧内编码，可以得到第2个P帧的重建图像的帧内编码码流。接着，编码器将第2个P帧的重建图像的帧内编码码流发送给硬件解码器，硬件解码器可以解码得到第2个P帧的重建图像的帧内解码重建图像(不发送)。然后再将第4个P帧的码流输入至硬件解码器中，由硬件解码器根据第3个P帧的重建图像的帧内解码重建图像，对第4个P帧的码流进行解码，得到第4个P帧的重建图像，然后，一方面，将第4个P帧的重建图像发送给显示模块；另一方面，可以根据第二标识和第二指示信息，将第4个P帧的重建图像存储至外部参考列表中。t5时刻，当前帧为第5个P帧，硬件解码器可以参考第4个P帧的重建图像对第5个P帧的码流进行解码，得到第5个P帧的重建图像，然后将第5个P帧的重建图像发送给显示模块。
一种可能的方式中，第一设备在执行完S601～S608后，可以修改当前帧对应的码流中，用于标识与当前帧的参考帧相关的标准语法元素；其中，标准语法元素是指标准编码协议中所规定的语法元素。
示例性的，第一设备在执行完S601～S608后，需要修改的用于标识与所述当前帧的参考帧相关的标准语法元素包括以下至少一种：用于标识当前帧的POC值的语法元素，用于标识是否使用前向参考帧的语法元素；用于标识前向参考帧数量的语法元素；用于标识当前帧与前向参考帧距离的语法元素；用于标识前向参考帧是否用于当前帧参考的语法元素。当然，还可以包括其他用于标识与所述当前帧的参考帧相关的标准语法元素，本申请对此不作限制。
以编码协议H265为例，当前帧对应的码流中用于与所述当前帧的参考帧相关的标准语法元素，可以如表2所示：
表2
应该理解的是，表2中所示的语法元素，仅是当前帧对应的码流中，用于标识与所述当前帧的参考帧相关的标准语法元素的一些示例，本申请中当前帧对应的码流中用于标识与所述当前帧的参考帧相关的标准语法元素，可以具有比表2中更多或更少的语法元素，在此不再赘述。
假设时域分层结构为时域分层结构1：第1帧(POC＝0)、第3帧(POC＝2)、第5帧(POC＝4)、第7帧(POC＝6)为T0层(即TL(Temporal Layer，时域层)＝0)；第2帧(POC＝1)、第4帧(POC＝3)、第6帧(POC＝5)、第8帧(POC＝7)为T1层(即TL＝1)，也就是说，时域层的数量为2。对应的参考方式为参考方式1，第3帧参考第1帧编码，第5帧参考第3帧编码，第7帧参考第5帧编码；第2帧参考第1帧编码，第4帧参考第3帧编码，第6帧参考第5帧编码，第8帧参考第7帧编码。若当前帧为第6帧，由于在对当前帧编码之前，编码器对当前帧的参考帧(即第5帧)所对应的重建图像进行了帧内编码，得到的帧内编码码流中的POC为0；因此编码器对当前帧进行编码得到的码流中POC为1。为了使得解码器能够正常解码，可以将当前帧对应的码流中图像顺序号POC修改为5，即将pic_order_cnt_lsb的值由0修改为5；以使所述当前帧对应的码流中POC与所述当前帧的前一帧对应的码流中POC连续。以及将当前帧对应的码流中short_term_ref_pic_set_sps_flag的值配置为“1”，表示当前帧参考前向参考帧编码。
假设，当前帧的参考帧为第5帧；若外部参考列表包括3个时域分层参考帧，这3个时域分层参考帧为第1帧(POC＝0)、第3帧(POC＝2)和第5帧(POC＝4)；则可以将num_negative_pics的值由0修改为3，将delta_poc_s0_minus1[0]的值由“NULL”的修改为“0”，将delta_poc_s0_minus1[1]的值由“NULL”的修改为“2”，将delta_poc_s0_minus1[2]的值由“NULL”的修改为“4”。以及将used_by_curr_pic_s0_flag[0]由“NULL”修改为“0”，将used_by_curr_pic_s0_flag[1]由“NULL”修改为“0”，将used_by_curr_pic_s0_flag[2]由“NULL”修改为“1”；其中，used_by_curr_pic_s0_flag[0]＝0，表示第1帧不用于当前帧参考，used_by_curr_pic_s0_flag[1]＝0表示第3帧不用于当前帧参考，used_by_curr_pic_s0_flag[2]＝1表示第5帧用于当前帧参考。
图7c为示例性示出的编解码过程示意图。图7c(1)的第一设备中的编码器为硬件 编码器，第二设备中的解码器为硬件编解码或软件解码器。图7c(2)中左边的区域中灰色以及黑色的帧为第一设备发送的帧，右边的区域中灰色以及黑色的帧为第二设备接收到的帧；其中，黑色的帧表示时域分层参考帧。
图7c的编码过程与图7b中的编码过程，除了对I帧、第2个P帧和第4个P帧的编码过程不同之外，其余帧的编码过程类似，在此不再赘。其中，图7c中，无需在I帧、第2个P帧和第4个P帧的码流中添加参考帧参数，而是需要修改第2个P帧和第4个P帧的码流中的语法元素。其中，修改后的第2个P帧的码流的语法元素的值可以如：pic_order_cnt_lsb＝2，num_long_term_pics＝1，poc_lsb_lt[0]＝0，used_by_curr_pic_lt_flag[0]＝1。修改后的第4个P帧的码流的语法元素的值可以如：pic_order_cnt_lsb＝4，num_long_term_pics＝2，poc_lsb_lt[0]＝0，poc_lsb_lt[1]＝2，used_by_curr_pic_lt_flag[0]＝0，used_by_curr_pic_lt_flag[1]＝1。
图7c的解码过程与图5c中的解码过程除了对第2个P帧和第4个P帧的解码过程不同之外，其余帧的解码过程类似，在此不再赘述。第二设备接收到第2个P帧的码流后，可以参考I帧的重建图像进行解码，得到第2个P帧的重建图像。第二设备接收到第4个P帧的码流后，可以参考第2个P帧的重建图像进行解码，得到第4个P帧的重建图像。
图8为示例性示出的编码流程示意图。在图8的实施例中，描述了使用参考帧重采样编码的场景。其中，图8的实施例中的预设参考规则为重采样参考规则，预设参考帧设置规则为重采样设置规则。
S801，获取当前帧。
S802，判断是否满足重采样参考规则。
示例性的，可以根据重采样的参考周期，上一帧的重采样情况等，来设置重采样参考规则，本申请对此不作限制。示例性的重采样参考规则也可以包括：参考判断规则和参考帧确定规则。
一种可能的方式中，根据重采样的参考周期，设置重采样参考规则。例如，重采样参考规则中的参考判断规则可以设置为：达到重采样的参考周期(可以是指与当前帧距离最近的且参考重采样参考帧的已编码的帧，至当前帧的距离差，达到重采样的参考周期)；重采样参考规则中的参考帧确定规则可以设置为：外部参考列表中与当前帧距离最近的重采样参考帧。示例性的，重采样参考帧的参考周期可以可以按照需求设置如10帧，本申请对此不作限制。
一种可能的方式中，根据上一帧的重采样情况，来设置重采样参考规则。例如，重采样参考规则中的参考判断规则可以设置为：当前帧的上一帧进行了重采样；重采样参考规则中的参考帧确定规则可以设置为：外部参考列表中与当前帧距离最近的重采样参考帧。
示例性的，当满足重采样参考规则时，确定当前帧需要参考重采样参考帧编码；此时可以执行S803～S806。当不满足重采样参考规则时，确定当前帧无需参考重采样参考帧编码，可以执行S807。
S803，当确定满足重采样参考规则时，根据重采样参考规则确定参考指示信息。
示例性的，当确定满足重采样参考规则时，可以根据重采样参考规则中参考帧确定规则，确定当前帧的重采样参考帧所对应的重建图像的帧标识，然后将该帧标识，确定为参考指示信息。
例如，当重采样参考规则中参考帧确定规则为：外部参考列表中与当前帧距离最近的重采样参考帧时，可以将外部参考列表中与当前帧距离最近的重采样参考帧所对应重建图像的帧标识，作为参考指示信息。
S804，根据所述参考指示信息，从编码器的外部参考列表中，获取所述当前帧的重采样参考帧所对应的重建图像。
示例性的，可以将参考指示信息，与外部参考列表中各参考帧所对应重建图像的帧标识匹配，来从编码器的外部参考列表中，获取所述当前帧的重采样参考帧所对应的重建图像。
S805，由编码器对当前帧的重采样参考帧所对应的重建图像进行帧内编码。
S806，基于当前帧的重采样参考帧所对应的重建图像的帧内编码重建图像，由编码器对当前帧进行帧间编码，以得到当前帧对应的码流。
示例性的，S805～S806可以参照上述S302～S304的描述，在此不再赘述。
S807，当确定不满足重采样参考规则时，由编码器基于当前帧的上一帧所对应的重建图像对当前帧进行帧间编码，以得到当前帧对应的码流。
示例性的，S807可以参照上述S407的描述，在此不再赘述。
S808，当确定需要将当前帧作为后续编码的重采样参考帧时，基于当前帧对应的码流，生成当前帧的重建图像。
示例性的，可以根据重采样参考帧的设置周期(也可以称为重采样周期)、信道质量等设置重采样设置规则，本申请对此不作限制。进而在完成对当前帧的编码后，可以通过判断是否满足重采样设置规则，来判断是否需要将当前帧作为后续编码的重采样参考帧(也就是判断是否需要对当前帧进行重采样)。当确定满足重采样设置规则时，则确定需要将当前帧作为后续编码的重采样参考帧(或需要对当前帧进行重采样)，可以执行S809。当确定不满足重采样设置规则时，则无需将当前帧作为后续编码的重采样参考帧(或无需对当前帧进行重采样)，此时，可以结束编码流程。
一种可能的方式中，根据重采样参考帧的设置周期，来设置重采样参考帧的设置规则。重采样设置规则可以是：达到重采样参考帧的设置周期(可以是指与当前帧距离最近的重采样参考帧，至当前帧的距离差，达到重采样的设置周期)。一种可能的方式中，重采样参考帧的设置周期可以是预先设置的固定值如9；本申请对此不作限制。假设长期参考帧的设置周期为9，若当前帧为第9帧时，则可以确定达到了重采样参考帧的设置周期，此时可以确定满足重采样设置规则，即确定需要将当前帧作为后续编码的重采样参考帧(或需要对当前帧进行重采样)。
一种可能的方式中，可以根据信道质量，设置重采样参考帧的设置规则。例如，长期参考帧设置规则可以包括：信道质量变化程度大于第一阈值，或，信道质量变化程度低于第二阈值；其中，第一阈值为正数，第二阈值为负数，可以按照需求设置，本申请 对此不作限制。例如，若检测信道质量变化程度大于第一阈值，则说明信道质量变好，此时信道可以承载更大的码率，因此可以增加当前帧的分辨率，进而可以确定满足重采样参考帧的设置规则，即确定需要对当前帧进行重采样(或需要对当前帧进行重采样)。若检测信道质量变化程度小于第二阈值，则说明信道质量变差，此时信道可承载的码率更小，因此可以降低当前帧的分辨率，进而可以确定满足重采样设置规则。否则，则确定满足重采样设置规则，即无需将当前帧作为后续编码的重采样参考帧(或无需对当前帧进行重采样)。
S809，对当前帧的重建图像进行重采样。
S810，将重采样后的重建图像，加入外部参考列表。
示例性的，在设置重采样设置规则时，还可以设置对应的重采样的目标分辨率。例如，重采样设置规则为：达到重采样参考帧的设置周期，对应的重采样的目标分辨率可以设置为：1720*920。又例如，重采样设置规则为：信道质量变化程度大于第一阈值，对应的重采样的目标分辨率可以设置为：1720*1080。还例如，重采样设置规则为：信道质量变化程度大于第一阈值，对应的重采样的目标分辨率可以设置为：960*480。
进而，可以按照重采样的目标分辨率，对当前帧的重建图像进行重采样。然后再将重采样后的重建图像，加入至外部参考列表中。
示例性的，在得到当前帧对应的码流后，可以在当前帧对应的码流中添加参考帧参数。在图8的示例性中，参考帧参数中的第一标识可以用于标识当前帧是否参考重采样参考帧编码，第一指示信息可以用于指示外部参考列表中当前帧的重采样参考帧所对应的重建图像，第三标识用于标识当前帧是否进行重采样，第三指示信息用于指示外部参考列表中当前帧的重采样后的重建图像，重采样分辨率包括当前帧的重建图像对应的分辨率和重采样后的重建图像对应的分辨率。
示例性的，当确定满足重采样参考规则，且当前帧已基于参考重采样参考帧编码后，则可以将第一标识赋值为第九预设信息(如“E1”)，且将第一指示信息赋值为外部参考列表中当前帧的重建图像对应的帧标识。当不确定满足重采样参考规则，则可以将第一标识配置为第十预设信息(如“E0”)，且将第一指示信息赋值为“NULL”。
示例性的，当确定需要对当前帧进行重采样时，则可以将第三标识配置为第十一预设信息(如“F1”)，且将第三指示信息赋值为，外部参考列表中当前帧的重采样参考帧所对应的重建图像对应的帧标识。当确定无需对当前帧进行重采样时，则可以将第二标识配置为第十二预设信息(如“F0”)，并将第二指示信息赋值为“NULL”。
图9a为示例性示出的解码流程示意图。在图9a的实施例中描述的是参考帧重采样解码的场景，与图8实施例中的编码过程对应。
S901，接收当前帧对应的码流。
S902，解析当前帧对应的码流，以得到参考帧参数。
S903，基于参考帧参数，判断是否当前帧是否需要参考重采样参考帧解码。
示例性的，在从当前帧对应的码流中解析出参考帧参数后，可以判断参考帧参数中 是否存在第一标识。当参考帧参数中存在第一标识时，判断第一标识的值是否为第九预设信息。若第一标识的值为第九预设信息，则确定当前帧需要参考重采样参考帧解码；此时，可以执行S904～S907。若第一标识的值为第十预设信息，或者参考帧参数不包含第一标识，则确定当前帧无需参考重采样参考帧解码，此时可以执行S908。
S904，当确定当前帧需要参考重采样参考帧解码时，基于参考帧参数从解码器的外部参考列表中，确定当前帧的重采样参考帧所对应的重建图像，外部参考列表中的重采样参考帧为已解码的帧。
示例性的，当确定当前帧需要参考重采样参考帧解码时，从解码器的外部参考列表中，将与参考帧参数中第二指示信息匹配的重建图像，确定为当前帧的重采样参考帧所对应的重建图像。
S905，对重建图像进行帧内编码，以得到对应的帧内编码码流。
S906，由解码器对帧内编码码流进行帧内解码。
S907，基于对帧内编码码流进行解码得到的帧内编码重建图像，由解码器对当前帧对应的码流进行帧间解码，以得到当前帧的重建图像。
示例性的，S905～S907可以参照上述S303～S305的描述，在此不再赘述。
S908，当确定当前帧无需参考重采样参考帧解码时，基于当前帧的上一帧所对应的重建图像，由解码器对当前帧对应的码流进行帧间解码，以得到当前帧的重建图像。
示例性的，当确定当前帧无需参考重采样参考帧解码时，可以将当前帧对应的码流直接输入至解码器中，然后由解码器基于当前帧的上一帧所对应的重建图像，对当前帧对应的码流进行解码，以得到当前帧的重建图像。
示例性的，基于参考帧参数，判断是否需要将当前帧作为后续解码的重采样参考帧。示例性的，可以判断参考帧参数中是否包括第三标识；当参考帧参数中包括第三标识时，可以判断第三标识的值是否为第十一预设信息。当第三标识的值为第十一预设信息时，可以确定需要将当前帧作为后续解码的重采样参考帧，此时可以执行S909。当第三标识的值为第十二预设信息，或者参考帧参数中不包括第三标识时，可以确定无需将当前帧作为后续解码的重采样参考帧，可以结束解码流程。
S909，当确定需要将当前帧作为后续解码的重采样参考帧时，对当前帧的重建图像进行重采样。
S910，将重采样后的重建图像，加入至外部参考列表中。
示例性的，S901～S910，可以参照上述S809～S810的描述，在此不再赘述。
当然，也可以对重采样或的重建图像进行帧内编码，以得到重采样后的重建图像所对应的帧内编码码流；然后将重采样后的重建图像所对应的帧内编码码流，加入外部参考列表中；具体可以参照上文的描述，在此不再赘述。
图9b为示例性示出的编解码过程示意图。图9b(1)的第一设备中的编码器为硬件编码器，第二设备中的解码器为硬件解码器。图9b(2)中左边的区域中灰色以及黑色的帧为第一设备发送的帧，右边的区域中灰色以及黑色的帧为第二设备接收到的帧；其中，黑色的帧为重采样参考帧。
假设待编码的视频序列为IPPPPP，重采样设置规则为：达到重采样的设置周期，重采样的设置周期为4；重采样参考规则为：达到重采样的参考周期，重采样参考帧的参考周期为5。
结合图9b(1)和图9b(2)，与图5b(1)和图5b(2)编解码过程类似，在此不再赘述。区别在于，在图9b(1)和图9b(2)中，在编码过程中，可以先对第3个P帧的重建图进行重采样，然后将重采样后的第3个P帧的重建图像存储至外部参考列表。接着，可以在第3个P帧的码流中添加第三标识并将第三标识赋值为第十一预设信息，以及添加第三指示信息(如3)和第3帧的重建图像对应的分辨率和所述重采样后的第3帧的重建图像对应的分辨率。在解码过程中，可以根据第三标识，第3帧的重建图像对应的分辨率和所述重采样后的第3帧的重建图像对应的分辨率，对第3个P帧的重建图进行重采样，然后根据第三指示信息，将重采样后的第3个P帧的重建图像存储至外部参考列表。
示例性的，硬件编码器可以结合软件编码进行编码，从而来实现灵活的参考帧管理。
图10为示例性示出的编码流程图。在图10的实施例中，可以应用在时域分层编码的场景中，硬件编码器可以结合软件编码器，来实现时域分层编码。其中，第一编码器为硬件编码器，第二编码器为软件编码器。
S1001，获取当前帧。
示例性的，在获取到当前帧后，可以判断是由第一编码器编码，还是由第二编码器编码。示例性的，由于硬件编码器是逐帧参考进行编码的，而时域分层结构中T0层的帧是逐帧进行编码的，因此可以判断当前帧是否是时域分层结构中的T0层。当当前帧是时域分层结构中的T0层时，确定由第一编码器编码；当当前帧不是时域分层结构中的T0层时，确定由第二编码器编码。
示例性的，可以根据时域分层结构和当前帧的帧标识，来判断当前帧是否是时域分层结构中的T0层。
例如，假设时域分层结构为参照图6的实施例中的时域分层结构1。若当前帧所在GOP有8帧，当当前帧为第1帧或第3帧或第5帧或第7帧时，可以确定当前帧需要由第一编码器编码。当当前帧为第2帧或第4帧或第6帧或第8帧时，可以确定当前帧需要由第二编码器编码。
又例如，假设时域分层结构为参照图6的实施例中的时域分层结构2。若当前帧所在GOP有12帧，当当前帧为第1帧或第5帧或第9帧，则可以确定当前帧需要由第一编码器编码。若当前帧为第3帧或第7帧或第2帧或第4帧或第6帧或第8帧或第10帧或第11帧或第12帧，则可以确定当前帧需要由第二编码器编码。
S1002，当确定需要由第一编码器进行编码时，由第一编码器对当前帧进行编码，以得到当前帧对应的码流。
S1003，将基于当前帧对应的码流生成的重建图像，加入第二编码器的外部参考列表，外部参考列表位于第二编码器的外部。
示例性的，当确定当前帧需要由第一编码器进行编码时，可以将当前帧输入至第一 编码器编码，由第一编码器对当前帧进行编码，以得到当前帧对应的码流。其中，当当前帧为I帧时，第一编码器可以对当前帧进行帧内编码，以得到当前帧的帧内编码码流。当当前帧为P帧时，第一编码器可以以当前帧的上一帧的重建图像为参考，对当前帧进行帧间编码，以得到当前帧的帧内编码码流。
由于第一编码器编码的是T0层，其他时域层(如T1、T2...)的帧通常会以T0层的帧为参考进行编码；因此在得到当前帧对应的码流后，可以基于当前帧对应的码流进行解码，以得到当前帧的重建图像。然后将当前帧的重建图像，加入第二编码器的外部参考列表。这样，第二解码器在编码除T0层之外的其它时域层的帧时，可以从其外部参考列表获取参考帧所对应的重建图像。
S1004，当确定需要由第二编码器进行编码时，从外部参考列表中确定当前帧的参考帧所对应的重建图像，基于当前帧的参考帧所对应的重建图像，由第二编码器对当前帧进行编码，以得到当前帧对应的码流。
示例性的，第二编码器还存在对应的内部参考列表，第二编码器每编码一帧后，可以将该帧的重建图像加入至内部参考列表中。
进而，当确定需要由第二编码器进行编码时，可以根据时域分层参考规则，来确定是基于第二编码器的外部参考列表编码，还是基于第二编码器的内部参考列表编码。示例性的，时域分层参考规则可以按照上述图6中的实施例描述的方式设置，在此不再赘述。
例如，假设时域分层参考规则中的参考判断规则为：时域分层结构1，POC值为除8×(n-1)之外的整数；参考帧确定条件为：参考方式1。若当前帧为第8帧，则根据时域分层参考规则中的参考帧确定条件，确定第8帧参考第7帧。由于第7帧是由第一编码器编码的，因此可以确定需要基于外部参考列表进行编码。接着，可以继续根据时域分层参考规则中的参考帧确定条件，从外部参考列表中确定当前帧的参考帧所对应的重建图像。随后，第二编码器可以从外部参考列表中选取出当前帧的参考帧所对应的重建图像，将当前帧的参考帧所对应的重建图像放入内部参考列表，然后基于当前帧的参考帧所对应的重建图像，对当前帧进行编码，以得到当前帧对应的码流。
例如，假设时域分层参考规则中的参考判断规则为：时域分层结构2，POC值为除12×(n-1)之外的整数；参考帧确定条件为：参考方式2。若当前帧为第8帧，则可以根据时域分层参考规则中的参考帧确定条件，确定第8帧参考第7帧。由于第7帧是由第二编码器编码的，因此可以确定需要参照内部参考列表进行编码。接着，可以继续根据时域分层参考规则中的参考帧确定条件，从内部参考列表中确定当前帧的参考帧所对应的重建图像。随后，基于当前帧的参考帧所对应的重建图像，由第二编码器对当前帧进行编码，以得到当前帧对应的码流。
示例性的，在第一编码器对当前帧进行编码得到当前帧对应的码流后，可以在当前帧对应的码流中添加第一标识、第一指示信息、第二标识和第二指示信息。其中，第一标识可以用于标识当前帧是否基于外部参考列表编码，可以是当前帧的参考帧所对应的码流的标识(如stream ID)。第一指示信息用于指示外部参考列表中当前帧的时域分层 参考帧所对应的重建图像。第二标识用于标识当前帧是否作为后续编码的时域分层参考帧，第二指示信息用于指示外部参考列表中当前帧的重建图像。
当当前帧的参考帧所对应的重建图像来自于外部参考列表时，可以将第一标识赋值为第一预设信息(如“G1”)；且将第一指示信息赋值为，外部参考列表中当前帧的时域分层参考帧所对应的重建图像对应的帧标识。当前帧的参考帧所对应的重建图像来自于内部参考列表时，可以将第一标识赋值为第二预设信息(如“G0”)，并将第一指示信息赋值为“NULL”。
示例性的，当确定需要将当前帧作为后续编码的时域分层参考帧，且将当前帧的重建图像加入外部参考列表后，则可以将第二标识配置为第三预设信息(如“H1”)，且将第二指示信息赋值为，外部参考列表中当前帧的时域分层参考帧所对应的重建图像对应的帧标识。当确定无需将当前帧作为后续编码的时域分层参考帧时，则可以将第二标识配置为第三预设信息(如“H0”)，并将第二指示信息赋值为“NULL”。
图11a为示例性示出的解码流程示意图。在图11a的实施例中描述的是时域分层解码的场景，与图10实施例中的编码过程对应。一种可能的方式中，第一解码器和第二解码器均为软件解码器。另一种可能的方式中，第一解码器为硬件解码器，第二解码器为软件解码器。
S1101，接收当前帧对应的码流。
示例性的，当接收到当前帧对应的码流后，可以解析当前帧对应的码流，以得到参考帧参数；然后根据参考帧参数，确定是否由第一解码器进行解码。
示例性的，在从当前帧对应的码流中解析出参考帧参数后，可以判断参考帧参数中是否存在第一标识。当参考帧参数中存在第一标识时，判断第一标识的值是否为第一预设信息。若第一标识的值为第一预设信息，则确定当前帧需要参考外部参考列表解码，此时可以确定需要由第二解码器解码；此时，可以执行S1104。若第一标识的值为第二预设信息，或者参考帧参数不包含第一标识，则确定当前帧无需参考外部参考列表解码，此时，可以确定需要由第一解码器解码；此时，可以执行S1102。
S1102，当确定需要由第一解码器进行解码时，由第一解码器对当前帧对应的码流进行解码，以得到当前帧的重建图像。
S1103，将当前帧的重建图像，加入第二解码器的外部参考列表，外部参考列表位于第二解码器的外部。
示例性的，当确定由第一解码器进行解码时，可以将当前帧对应的码流输入至第一解码器中，由第一解码器对当前帧对应的码流进行解码，以得到当前帧的重建图像。
示例性的，在从当前帧对应的码流中解析出参考帧参数后，可以判断参考帧参数中是否存在第二标识。当参考帧参数中存在第二标识时，判断第二标识的值是否为第三预设信息。若第二标识的值为第三预设信息，则确定当前帧需要作为后续解码的时域分层参考帧，此时可以将当前帧的重建图像，加入第二解码器的外部参考列表。若第二标识的值为第四预设信息，或者参考帧参数不包含第二标识，则确定当前帧无需作为后续解码的时域分层参考帧。
S1104，当确定需要由第二解码器进行解码时，从外部参考列表中确定当前帧的参考帧所对应的重建图像；基于参考帧所对应的重建图像，由第二解码器对当前帧对应的码流进行解码，以得到当前帧的重建图像。
示例性的，当确定由第一解码器进行解码时，可以根据参考帧参数中的第一指示信息，从外部参考列表中确定当前帧的参考帧所对应的重建图像。然后将当前帧的参考帧所对应的重建图像从外部参考列表中取出，存储至第二解码器的内部参考列表中。接着，第二解码器基于当前帧的参考帧所对应的重建图像，对当前帧对应的码流进行解码，以得到当前帧的重建图像。
图11b为示例性示出的编解码过程示意图。
假设，时域分层结构为：第1帧(POC＝0)、第3帧(POC＝2)为T0层；第2帧(POC＝1)、第4帧(POC＝3)为T1层；对应的参考方式为：第3帧参照第1帧编码，第2帧参照第1帧编码，第4帧参照第3帧编码。
参照图11b，第一设备的编码过程中，t0时刻，当前帧为I帧，可以将I帧输入至第一编码器，由第一编码器对I帧进行帧内编码，得到I帧码流。然后可以将I帧的码流输入至解码器，得到解码器输出的I帧的重建图像，然后将I帧的重建图像存储至第二编码器的外部参考列表中。接着，可以在I帧的码流中添加第二标识并将第二标识赋值为第三预设信息，以及添加第二指示信息(如0)，然后将添加参考帧参数的I帧的码流发送给第二设备。t1时刻，当前帧为第1个P帧，可以将第1个P帧输入至第二编码器中，由第二编码器从外部参考列表中读取I帧的重建图像至内部参考列表中；然后基于I帧的重建图像，对第1个P帧进行编码，得到第1个P帧的码流。接着，可以在1个P帧的码流中添加第一标识并将第一标识赋值为第一预设信息，以及添加第一指示信息(如0)，然后将添加参考帧参数的第1个P帧的码流发送给第二设备。t2时刻，当前帧为第2个P帧，可以将第2个P帧输入至第一编码器，由第一编码器根据I帧的重建图像，对第2个P帧进行编码，得到第2个P帧的码流。然后可以将第2个P帧的码流输入至解码器，得到解码器输出的第2个P帧的重建图像，然后将第2个P帧的重建图像存储至第二编码器的外部参考列表中。接着，可以在第2个P帧的码流中添加第一标识并将第一标识赋值为第一预设信息，以及添加第一指示信息(如0)；以及添加第二标识并将第二标识赋值为第三预设信息，以及添加第二指示信息(如2)，然后将添加参考帧参数的第2个P帧的码流发送给第二设备。t3时刻，当前帧为第3个P帧，可以将第3个P帧输入至第二编码器中，由第二编码器从外部参考列表中读取第2个P帧的重建图像至内部参考列表中；然后基于第2个P帧的重建图像，对第3个P帧进行编码，得到第3个P帧的码流。接着，可以在3个P帧的码流中添加第一标识并将第一标识赋值为第一预设信息，以及添加第一指示信息(如2)，然后将添加参考帧参数的第3个P帧的码流发送给第二设备。
参照图11b，第二设备的解码过程中，t0时刻，当前帧为I帧，可以将I帧的码流输入至第一解码器，由第一解码器对I帧进行帧内解码，得到I帧的重建图像。然后一方面，可以根据第二标识和第二指示信息，将I帧的重建图像存储至第二解码器的外部参考列 表中，另一方面，可以将I帧的重建图像发送至显示模块进行显示。t1时刻，当前帧为第1个P帧，可以将第1个P帧的码流输入至第二解码器中，由第二解码器根据第一标识和第一指示信息，从外部参考列表中读取I帧的重建图像至内部参考列表中；然后基于I帧的重建图像，对第1个P帧的码流进行解码，得到第1个P帧的重建图像。接着，可以将第1个P帧的重建图像发送至显示模块进行显示。t2时刻，当前帧为第2个P帧，可以将第2个P帧的码流输入至第一解码器，由第一解码器根据I帧的重建图像，对第2个P帧的码流进行解码，得到第2个P帧的重建图像。然后，一方面，可以根据第二标识和第二指示信息，将第2个P帧的重建图像存储至第二解码器的外部参考列表中，另一方面，可以将第2个P帧的重建图像发送至显示模块进行显示。t3时刻，当前帧为第3个P帧，可以将第3个P帧的码流输入至第二解码器中，由第二解码器根据第一标识和第一指示信息，从外部参考列表中读取第2个P帧的重建图像至内部参考列表中；然后基于第2个P帧的重建图像，对第3个P帧的码流进行解码，得到第3个P帧的重建图像。可以将第3个P帧的重建图像发送至显示模块进行显示。
图10和图11a所示的实施例中，解码端/编码端均无需对参考帧的重建图像进行帧内编码；然后基于帧内编码重建图像进行编码/解码，这样能够提高图像质量。
一个示例中，图12示出了本申请实施例的一种示意性框图装置1200可包括：处理器1201和收发器/收发管脚1202，可选地，还包括存储器1203。
装置1200的各个组件通过总线1204耦合在一起，其中总线1204除包括数据总线之外，还包括电源总线、控制总线和状态信号总线。但是为了清楚说明起见，在图中将各种总线都称为总线1204。
可选地，存储器1203可以存储用于前述方法实施例中的指令。该处理器1201可用于执行存储器1203中的指令，并控制接收管脚接收信号，以及控制发送管脚发送信号。
装置1200可以是上述方法实施例中的电子设备或电子设备的芯片。
其中，上述方法实施例涉及的各步骤的所有相关内容均可以援引到对应功能模块的功能描述，在此不再赘述。
本实施例还提供一种计算机可读存储介质，该计算机可读存储介质中存储有计算机指令，当该计算机指令在电子设备上运行时，使得电子设备执行上述相关方法步骤实现上述实施例中的编解码方法。
本实施例还提供了一种计算机程序产品，当该计算机程序产品在计算机上运行时，使得计算机执行上述相关步骤，以实现上述实施例中的编解码方法。
另外，本申请的实施例还提供一种装置，这个装置具体可以是芯片，组件或模块，该装置可包括相连的处理器和存储器；其中，存储器用于存储计算机执行指令，当装置运行时，处理器可执行存储器存储的计算机执行指令，以使芯片执行上述各方法实施例中的编解码方法。
其中，本实施例提供的电子设备、计算机可读存储介质、计算机程序产品或芯片均 用于执行上文所提供的对应的方法，因此，其所能达到的有益效果可参考上文所提供的对应的方法中的有益效果，此处不再赘述。
通过以上实施方式的描述，所属领域的技术人员可以了解到，为描述的方便和简洁，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。
在本申请所提供的几个实施例中，应该理解到，所揭露的装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个装置，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是一个物理单元或多个物理单元，即可以位于一个地方，或者也可以分布到多个不同地方。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
本申请各个实施例的任意内容，以及同一实施例的任意内容，均可以自由组合。对上述内容的任意组合均在本申请的范围之内。
集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个可读取存储介质中。基于这样的理解，本申请实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该软件产品存储在一个存储介质中，包括若干指令用以使得一个设备(可以是单片机，芯片等)或处理器(processor)执行本申请各个实施例方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
上面结合附图对本申请的实施例进行了描述，但是本申请并不局限于上述的具体实施方式，上述的具体实施方式仅仅是示意性的，而不是限制性的，本领域的普通技术人员在本申请的启示下，在不脱离本申请宗旨和权利要求所保护的范围情况下，还可做出很多形式，均属于本申请的保护之内。
结合本申请实施例公开内容所描述的方法或者算法的步骤可以硬件的方式来实现，也可以是由处理器执行软件指令的方式来实现。软件指令可以由相应的软件模块组成，软件模块可以被存放于随机存取存储器(Random Access Memory，RAM)、闪存、只读存储器(Read Only Memory，ROM)、可擦除可编程只读存储器(Erasable Programmable ROM，EPROM)、电可擦可编程只读存储器(Electrically EPROM，EEPROM)、寄存 器、硬盘、移动硬盘、只读光盘(CD-ROM)或者本领域熟知的任何其它形式的存储介质中。一种示例性的存储介质耦合至处理器，从而使处理器能够从该存储介质读取信息，且可向该存储介质写入信息。当然，存储介质也可以是处理器的组成部分。处理器和存储介质可以位于ASIC中。
本领域技术人员应该可以意识到，在上述一个或多个示例中，本申请实施例所描述的功能可以用硬件、软件、固件或它们的任意组合来实现。当使用软件实现时，可以将这些功能存储在计算机可读介质中或者作为计算机可读介质上的一个或多个指令或代码进行传输。计算机可读介质包括计算机可读存储介质和通信介质，其中通信介质包括便于从一个地方向另一个地方传送计算机程序的任何介质。存储介质可以是通用或专用计算机能够存取的任何可用介质。
上面结合附图对本申请的实施例进行了描述，但是本申请并不局限于上述的具体实施方式，上述的具体实施方式仅仅是示意性的，而不是限制性的，本领域的普通技术人员在本申请的启示下，在不脱离本申请宗旨和权利要求所保护的范围情况下，还可做出很多形式，均属于本申请的保护之内。

Claims (36)

  1. 一种编码方法，其特征在于，所述方法包括：
    获取当前帧；
    从编码器的外部参考列表中，获取所述当前帧的参考帧所对应的重建图像，所述当前帧的参考帧为所述编码器已编码的帧，所述外部参考列表独立于所述编码器；
    由所述编码器对所述重建图像进行帧内编码；
    基于所述帧内编码的结果，由所述编码器对所述当前帧进行帧间编码，以得到所述当前帧对应的码流。
  2. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    当确定需要将所述当前帧作为后续编码的参考帧时，基于所述当前帧对应的码流，生成所述当前帧的重建图像；
    将所述当前帧的重建图像，加入所述外部参考列表。
  3. 根据权利要求2所述的方法，其特征在于，所述将所述当前帧的重建图像，加入所述外部参考列表，包括：
    对所述当前帧的重建图像进行重采样；
    将所述重采样后的重建图像，加入所述外部参考列表。
  4. 根据权利要求1至3中任一项所述的方法，其特征在于，
    所述当前帧对应的码流包括：第一标识和第一指示信息；
    所述第一标识用于标识所述当前帧是否基于所述外部参考列表编码，所述第一指示信息用于指示所述外部参考列表中所述当前帧的参考帧所对应的重建图像。
  5. 根据权利要求1至4中任一项所述的方法，其特征在于，
    所述当前帧对应的码流包括：第二标识和第二指示信息；
    所述第二标识用于标识所述当前帧是否作为后续编码的参考帧，所述第二指示信息用于指示所述外部参考列表中所述当前帧的重建图像。
  6. 根据权利要求1至4中任一项所述的方法，其特征在于，
    所述当前帧对应的码流包括：第三标识、第三指示信息和重采样分辨率；所述第三标识用于标识所述当前帧是否进行重采样，所述第三指示信息用于指示所述外部参考列表中所述当前帧的重采样后的重建图像，所述重采样分辨率包括所述当前帧的重建图像对应的分辨率和所述重采样后的重建图像对应的分辨率。
  7. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    修改所述当前帧对应的码流中，用于标识与所述当前帧的参考帧相关的标准语法元 素；
    其中，所述标准语法元素为标准编码协议中所规定的语法元素。
  8. 根据权利要求7所述的方法，其特征在于，所述用于标识与所述当前帧的参考帧相关的标准语法元素包括以下至少一种：
    用于标识长期参考帧数量的语法元素；
    用于标识长期参考帧图像顺序号POC的语法元素；
    用于标识长期参考帧是否用于当前帧参考的语法元素；
    用于标识是否使用前向参考帧的语法元素；
    用于标识前向参考帧数量的语法元素；
    用于标识当前帧与前向参考帧距离的语法元素；
    用于标识前向参考帧是否用于当前帧参考的语法元素。
  9. 根据权利要求1至8中任一项所述的方法，其特征在于，所述从编码器的外部参考列表中，获取所述当前帧的参考帧所对应的重建图像，包括：
    判断是否满足预设参考规则；
    当满足预设参考规则时，根据所述预设参考规则确定参考指示信息；
    根据所述参考指示信息，从编码器的外部参考列表中，获取所述当前帧的参考帧所对应的重建图像。
  10. 根据权利要求9所述的方法，其特征在于，
    所述预设参考规则包括以下任意一种：长期参考规则，时域分层参考规则和重采样参考规则。
  11. 根据权利要求10所述的方法，其特征在于，所述长期参考规则包括以下至少一种：
    达到长期参考帧的参考周期；
    所述当前帧的预设参考帧对应的码流丢失；
    所述外部参考列表中，存在与所述当前帧的相似度高于相似度阈值的重建图像。
  12. 根据权利要求1至11中任一项所述的方法，其特征在于，所述帧内编码的结果包括所述当前帧的参考帧所对应的重建图像的帧内编码重建图像；
    所述基于所述帧内编码的结果，由所述编码器对所述当前帧进行帧间编码，以得到所述当前帧对应的码流，包括：
    由所述编码器以所述帧内编码重建图像为参考，对所述当前帧进行帧间编码，以得到所述当前帧对应的码流。
  13. 根据权利要求1至12中任一项所述的方法，其特征在于，所述当前帧的参考帧为：长期参考帧或时域分层参考帧或重采样参考帧。
  14. 根据权利要求1至13中任一项所述的方法，其特征在于，所述编码器为硬件编码器。
  15. 一种解码方法，其特征在于，所述方法包括：
    接收当前帧对应的码流；
    从解码器的外部参考列表中，确定所述当前帧的参考帧所对应的重建图像，所述当前帧的参考帧为所述解码器已解码的帧，所述外部参考列表独立于所述解码器；
    对所述重建图像进行帧内编码，以得到对应的帧内编码码流；
    由所述解码器对所述帧内编码码流进行帧内解码；
    基于所述帧内解码的结果，由所述解码器对所述当前帧对应的码流进行帧间解码，以得到所述当前帧的重建图像。
  16. 根据权利要求15所述的方法，其特征在于，所述方法还包括：
    当确定需要将所述当前帧作为后续解码的参考帧时，将所述当前帧的重建图像，加入所述外部参考列表。
  17. 根据权利要求16所述的方法，其特征在于，所述将所述当前帧的重建图像，加入所述外部参考列表，包括：
    对所述当前帧的重建图像进行重采样；
    将所述重采样后的重建图像，加入所述外部参考列表。
  18. 根据权利要求15至17中任一项所述的方法，其特征在于，所述从解码器的外部参考列表中，确定所述当前帧的参考帧所对应的重建图像，包括：
    解析所述当前帧对应的码流，以得到参考帧参数；
    基于所述参考帧参数，从解码器的外部参考列表中，确定所述当前帧的参考帧所对应的重建图像。
  19. 根据权利要求18所述的方法，其特征在于，
    所述参考帧参数包括：第一标识和第一指示信息；
    所述第一标识用于标识所述当前帧是否基于所述外部参考列表解码，所述第一指示信息用于指示所述外部参考列表中所述当前帧的参考帧所对应的重建图像。
  20. 根据权利要求18或19所述的方法，其特征在于，
    所述参考帧参数包括：第二标识和第二指示信息；
    所述第二标识用于标识所述当前帧是否作为后续解码的参考帧，所述第一指示信息用于指示所述外部参考列表中所述当前帧的重建图像。
  21. 根据权利要求18或19所述的方法，其特征在于，
    所述当前帧对应的码流包括：第三标识、第三指示信息和重采样分辨率；所述第三标识用于标识所述当前帧是否进行重采样，所述第三指示信息用于指示所述外部参考列表中所述当前帧的重采样后的重建图像，所述重采样分辨率包括所述当前帧的重建图像对应的分辨率和所述重采样后的重建图像对应的分辨率。
  22. 根据权利要求15至21中任一项所述的方法，其特征在于，所述当前帧的参考帧为：长期参考帧或时域分层参考帧或重采样参考帧。
  23. 一种编码方法，其特征在于，所述方法包括：
    获取当前帧；
    当确定需要由第一编码器编码时，由所述第一编码器对所述当前帧进行编码，以得到所述当前帧对应的码流；以及将基于所述当前帧对应的码流生成的重建图像，加入第二编码器的外部参考列表，所述外部参考列表独立于所述第二编码器；
    当确定需要由所述第二编码器编码时，从所述外部参考列表中确定所述当前帧的参考帧所对应的重建图像，基于所述当前帧的参考帧所对应的重建图像，由所述第二编码器对所述当前帧进行编码，以得到所述当前帧对应的码流；
    其中，所述第一编码器为硬件编码器，所述第二编码器为软件编码器。
  24. 根据权利要求23所述的方法，其特征在于，所述方法还包括：
    判断所述当前帧是否是时域分层结构中的T0层；
    当所述当前帧是时域分层结构中的T0层时，确定需要由所述第一编码器编码；
    当所述当前帧不是时域分层结构中的T0层时，确定需要由所述第二编码器编码。
  25. 根据权利要求23或24所述的方法，其特征在于，
    所述当前帧对应的码流包括：第一标识和第一指示信息；
    所述第一标识用于标识当前帧是否基于所述外部参考列表编码，所述第一指示信息用于指示所述外部参考列表中所述当前帧的参考帧所对应的重建图像。
  26. 一种解码方法，其特征在于，所述方法包括：
    接收当前帧对应的码流；
    当确定需要由第一解码器进行解码时，由第一解码器对所述当前帧对应的码流进行解码，以得到所述当前帧的重建图像；以及将所述当前帧的重建图像，加入第二解码器的外部参考列表，所述外部参考列表位于所述第二编码器的外部；
    当确定需要由所述第二解码器进行解码时，从所述外部参考列表中确定所述当前帧的参考帧所对应的重建图像；基于所述参考帧所对应的重建图像，由所述第二解码器对所述当前帧对应的码流进行解码，以得到所述当前帧的重建图像。
  27. 根据权利要求26所述的方法，其特征在于，所述方法还包括：
    解析所述当前帧对应的码流，以得到参考帧参数；
    根据所述参考帧参数，确定是否需要由所述第一解码器进行解码。
  28. 根据权利要求27所述的方法，其特征在于，
    所述参考帧参数包括：第一标识和第一指示信息；
    所述第一标识用于标识当前帧是否基于所述外部参考列表解码，所述第一指示信息用于指示所述外部参考列表中所述当前帧的参考帧所对应的重建图像。
  29. 根据权利要求28所述的方法，其特征在于，
    所述从所述外部参考列表中确定所述当前帧的参考帧所对应的重建图像，包括：
    当所述第一标识的值为第一预设信息时，将所述外部参考列表中，与所述第一指示信息匹配的重建图像，确定为所述当前帧的参考帧所对应的重建图像。
  30. 根据权利要求26至29中任一项所述的方法，其特征在于，
    所述第一解码器和所述第二解码器均为软件解码器。
  31. 根据权利要求26至29中任一项所述的方法，其特征在于，
    所述第一解码器为硬件解码器，所述第二解码器为软件解码器。
  32. 一种电子设备，其特征在于，包括：
    存储器和处理器，所述存储器与所述处理器耦合；
    所述存储器存储有程序指令，当所述程序指令由所述处理器执行时，使得所述电子设备执行权利要求1至权利要求14中任一项所述的编码方法，或，执行权利要求23至权利要求25中任一项所述的编码方法。
  33. 一种电子设备，其特征在于，包括：
    存储器和处理器，所述存储器与所述处理器耦合；
    所述存储器存储有程序指令，当所述程序指令由所述处理器执行时，使得所述电子设备执行权利要求15至权利要求22中任一项所述的解码方法，或执行权利要求26至权利要求31中任一项所述的解码方法。
  34. 一种芯片，其特征在于，包括一个或多个接口电路和一个或多个处理器；所述接口电路用于从电子设备的存储器接收信号，并向所述处理器发送所述信号，所述信号包括存储器中存储的计算机指令；当所述处理器执行所述计算机指令时，使得所述电子设备执行权利要求1至权利要求31中任一项所述的方法。
  35. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有计算机 程序，当所述计算机程序运行在计算机或处理器上时，使得所述计算机或所述处理器执行如权利要求1至权利要求31中任一项所述的方法。
  36. 一种计算机程序产品，其特征在于，所述计算机程序产品包含软件程序，当所述软件程序被计算机或处理器执行时，使得权利要求1至权利要求31中任一项所述的方法的步骤被执行。
PCT/CN2023/080113 2022-03-11 2023-03-07 编解码方法及电子设备 WO2023169426A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210243272.9 2022-03-11
CN202210243272.9A CN116781907A (zh) 2022-03-11 2022-03-11 编解码方法及电子设备

Publications (1)

Publication Number Publication Date
WO2023169426A1 true WO2023169426A1 (zh) 2023-09-14

Family

ID=87936023

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/080113 WO2023169426A1 (zh) 2022-03-11 2023-03-07 编解码方法及电子设备

Country Status (2)

Country Link
CN (1) CN116781907A (zh)
WO (1) WO2023169426A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116684610A (zh) * 2023-05-17 2023-09-01 北京百度网讯科技有限公司 确定长期参考帧的参考状态的方法、装置及电子设备

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB201118117D0 (en) * 2011-10-20 2011-11-30 Skype Ltd Transmission of video data
US20130279589A1 (en) * 2012-04-23 2013-10-24 Google Inc. Managing multi-reference picture buffers for video data coding
WO2013158024A1 (en) * 2012-04-16 2013-10-24 Telefonaktiebolaget L M Ericsson (Publ) Encoder, decoder and methods thereof for video encoding and decoding
CN104067620A (zh) * 2012-01-25 2014-09-24 夏普株式会社 视频解码方法和视频编码方法
US9609341B1 (en) * 2012-04-23 2017-03-28 Google Inc. Video data encoding and decoding using reference picture lists
CN112585974A (zh) * 2018-08-17 2021-03-30 华为技术有限公司 视频译码中的参考图像管理

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB201118117D0 (en) * 2011-10-20 2011-11-30 Skype Ltd Transmission of video data
CN104067620A (zh) * 2012-01-25 2014-09-24 夏普株式会社 视频解码方法和视频编码方法
WO2013158024A1 (en) * 2012-04-16 2013-10-24 Telefonaktiebolaget L M Ericsson (Publ) Encoder, decoder and methods thereof for video encoding and decoding
US20130279589A1 (en) * 2012-04-23 2013-10-24 Google Inc. Managing multi-reference picture buffers for video data coding
US9609341B1 (en) * 2012-04-23 2017-03-28 Google Inc. Video data encoding and decoding using reference picture lists
CN112585974A (zh) * 2018-08-17 2021-03-30 华为技术有限公司 视频译码中的参考图像管理

Also Published As

Publication number Publication date
CN116781907A (zh) 2023-09-19

Similar Documents

Publication Publication Date Title
CN112868184B (zh) 对视频序列进行解码的方法、设备以及存储介质
US10602156B2 (en) Devices and methods for identifying a leading picture
TWI533677B (zh) 用於減少視訊編碼及解碼中之延遲的方法、系統及電腦可讀取媒體
JP2021529460A (ja) 点群符号化のための高レベルシンタックス設計
TW201351964A (zh) 簡化視訊隨機存取之限制及單元類型
JP2015501098A5 (zh)
US11818392B2 (en) Decoder and encoder and methods for coding of a video sequence
JP2023143935A (ja) サブブロック区分モードのためのエンコーダ、デコーダ、および対応する方法
WO2023169426A1 (zh) 编解码方法及电子设备
KR20230087474A (ko) 병렬 코딩을 위한 양방향 데이터 블록들의 조인트 터미네이션
CN109151481A (zh) 图片的传输和接收的方法、装置及其系统
CN103873872A (zh) 参考图像管理方法及装置
WO2023130896A1 (zh) 媒体数据的数据处理方法、装置、计算机设备及存储介质
CN109302574B (zh) 一种处理视频流的方法和装置
EP4380155A1 (en) Encoding and decoding method, encoder, decoder, and electronic device
CN115866297A (zh) 视频处理方法、装置、设备及存储介质
US10356439B2 (en) Flexible frame referencing for display transport
CN114616801B (zh) 视频编码的方法、装置、设备以及存储介质
WO2023169424A1 (zh) 编解码方法及电子设备
EP3989566A1 (en) Motion information list construction method in video encoding and decoding, device, and apparatus
WO2024077806A1 (zh) 一种编解码方法、装置、编码器、解码器及存储介质
WO2023142662A1 (zh) 图像编码方法、实时通信方法、设备、存储介质及程序产品
US20240022765A1 (en) Base Mesh Data and Motion Information Sub-Stream Format for Video-Based Dynamic Mesh Compression
CN117676146A (zh) 一种编解码方法及装置
TW202408245A (zh) 一種編解碼方法、裝置、編碼器、解碼器、儲存媒介及碼流

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23766004

Country of ref document: EP

Kind code of ref document: A1