WO2018090600A1 - 虚拟参考帧的建立方法及设备 - Google Patents

虚拟参考帧的建立方法及设备 Download PDF

Info

Publication number
WO2018090600A1
WO2018090600A1 PCT/CN2017/086899 CN2017086899W WO2018090600A1 WO 2018090600 A1 WO2018090600 A1 WO 2018090600A1 CN 2017086899 W CN2017086899 W CN 2017086899W WO 2018090600 A1 WO2018090600 A1 WO 2018090600A1
Authority
WO
WIPO (PCT)
Prior art keywords
virtual reference
reference frame
block
preset
image
Prior art date
Application number
PCT/CN2017/086899
Other languages
English (en)
French (fr)
Inventor
张怡轩
吴东昇
秘谧
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2018090600A1 publication Critical patent/WO2018090600A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/154—Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/513—Processing of motion vectors
    • H04N19/517—Processing of motion vectors by encoding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/577—Motion compensation with bidirectional frame interpolation, i.e. using B-pictures

Definitions

  • the present application relates to the field of image processing, and in particular, to a method and a device for establishing a virtual reference frame.
  • the encoder encodes each frame of image and transmits it to the decoder.
  • the decoder decodes the image and then plays the video on the display device.
  • some images are repeated, such as fixed.
  • the background will reappear after the foreground object leaves.
  • the repeated images will be used as virtual reference frames.
  • the virtual reference frames will not be displayed during video playback. They are only used for reference during encoding and decoding. And decoding efficiency.
  • the encoder generates a background image according to at least one image frame to be transmitted and a preset modeling algorithm, and the background image may be a fixed background during video playback, and for each image frame in at least one image frame, If a block in the image frame is similar to a corresponding block in the background image, a similar block is used as a block for establishing a virtual reference frame, and a block for establishing a virtual reference frame is stored in a buffer, and then The image frame of the block that has been selected to establish the virtual reference frame is sent to the decoder, so that when enough blocks for establishing the virtual reference frame are stored, the block for establishing the virtual reference frame stored in the buffer is extracted.
  • the image frame to be transmitted after is encoded with reference to the virtual reference frame, and the virtual reference frame is transmitted to the decoder as a separate frame.
  • the encoder does not need to transmit the virtual reference frame as a frame to the decoder alone; after receiving the image frame, the decoder generates a background image based on the same modeling algorithm as the encoder, and according to the Selecting a block for establishing a virtual reference frame in each image frame according to the background image, and storing the block for establishing the virtual reference frame in a buffer area, when storing the same block as the encoder that establishes the virtual reference frame, The block for establishing the virtual reference frame in the buffer area is extracted as a virtual reference frame, and the image frame after the virtual reference frame is decoded.
  • the decoder needs to be based on the same as the encoder.
  • the algorithm establishes a virtual reference frame, and the decoder has higher complexity and higher power consumption.
  • the embodiment of the present application provides a method and a device for establishing a virtual reference frame.
  • the technical solution is as follows:
  • a method for establishing a virtual reference frame for use in a decoder, the method comprising:
  • each image frame of the at least one image frame comprising at least one block
  • Virtual reference frame is established by using a block with a virtual reference frame identifier in at least one image frame, and the virtual reference frame identifier is used The block in which the indication is located is used to create a virtual reference frame.
  • the method before the virtual reference frame is established by using the block with the virtual reference frame identifier in the at least one image frame, the method further includes:
  • Establishing a virtual reference frame using a block with a virtual reference frame identifier in at least one image frame including:
  • each block with a virtual reference frame identification is determined and each block with a virtual reference frame identification is stored in a first virtual reference frame buffer, respectively.
  • determining the location of each block with a virtual reference frame identifier includes:
  • each block with a virtual reference frame identifier is determined according to a preset location mapping relationship.
  • the virtual reference frame identifier is further used to indicate location information of the block in which the location of the block is located, and the location of each block with the virtual reference frame identifier is determined, including:
  • each block with a virtual reference frame identification is determined based on the indication of the virtual reference frame identification.
  • the method further includes:
  • the indication information is used to indicate a preset location mapping relationship.
  • the virtual reference frame identifier is carried on a newly added preset number of bits.
  • the virtual reference frame identifier is carried in a preset format field, and the preset format field is used to indicate a preset quantization step size, a preset block size, a preset encoding mode, and a preset motion vector. At least one of a preset residual and a preset quantization parameter.
  • the method further includes:
  • the decoding instruction signal transmitted by the encoder is received, and is decoded according to the decoding instruction signal by referring to the virtual reference frame, and the decoding indication signal is used to instruct decoding by referring to the virtual reference frame.
  • the second aspect provides a method for establishing a virtual reference frame, which is applied to an encoder, and the method includes:
  • each image frame of the at least one image frame comprising at least one block
  • At least one image frame is transmitted to the decoder.
  • determining a block for establishing a virtual reference frame in at least one image frame includes:
  • the target block Determining the target block as a block for establishing a virtual reference frame when the number of similar pixel pairs in the target block and the target background block is greater than or equal to the first threshold;
  • the target block is any one of the target image frames
  • the target background block is a block corresponding to the target block in the background image determined according to the position of the target block in the target image frame.
  • At least one image frame includes the target image frame; the similar pixel pair includes two pixels respectively corresponding to positions in the target block and the target background block, and the two pixels included in the similar pixel pair satisfy The absolute value of the difference between the at least one component is less than or equal to the second threshold.
  • the method before determining the coded block for establishing the virtual reference frame in the at least one image frame, the method further includes:
  • the background image is generated according to a preset modeling algorithm based on the at least one image frame.
  • the method before determining a block for establishing a virtual reference frame in the at least one image frame, the method further includes:
  • the method further includes:
  • the virtual reference frame identifier is further used to indicate location information of the block in which it is located.
  • the method further includes:
  • the virtual reference frame identifier is added to the block used to establish the virtual reference frame in the at least one image frame, including:
  • the virtual reference frame identifier is added to the block used to establish the virtual reference frame in the at least one image frame, including:
  • a field of a preset format in the block for establishing a virtual reference frame where the virtual reference frame identifier is carried in a field of a preset format, where the preset format field is used to indicate a preset quantization step size At least one of a preset block size, a preset encoding mode, a preset motion vector, a preset residual, and a preset quantization parameter.
  • the method further includes:
  • a decoding indication signal is sent to the decoder, the decoding indication signal being used to indicate decoding with reference to the virtual reference frame.
  • a decoder comprising:
  • a receiving unit configured to receive at least one image frame transmitted by the encoder, where each image frame in the at least one image frame includes at least one block;
  • a processing unit configured to establish a virtual reference frame by using a block with a virtual reference frame identifier in the at least one image frame, where the virtual reference frame identifier is used to indicate that the block is used to establish a virtual reference frame.
  • the processing unit is further configured to allocate a first virtual reference frame buffer; determine a location of each block with a virtual reference frame identifier, and each block with a virtual reference frame identifier They are stored in the first virtual reference frame buffer area, respectively.
  • the processing unit is further configured to determine a location of each block with a virtual reference frame identifier according to a preset location mapping relationship.
  • the virtual reference frame identifier is further used to indicate location information of the block in which the block is located.
  • the processing unit is further configured to determine, according to the indication of the virtual reference frame identifier, the location of each block with the virtual reference frame identifier.
  • the receiving unit receives the indication information sent by the encoder, where the indication information is used to indicate a preset location mapping relationship.
  • the virtual reference frame identifier is carried on a newly added preset number of bits.
  • the virtual reference frame identifier is carried in a preset format field, and the preset format field is used to indicate a preset quantization step size, a preset block size, a preset encoding mode, and a preset motion vector. At least one of a preset residual and a preset quantization parameter.
  • the receiving unit is further configured to receive a decoding indication signal sent by the encoder, where the decoding indication signal is used to indicate decoding by referring to the virtual reference frame;
  • the processing unit is further configured to perform decoding by referring to the virtual reference frame according to the decoding indication signal.
  • an encoder including:
  • a processing unit configured to acquire at least one image frame, determine a block for establishing a virtual reference frame in the at least one image frame, and add a virtual reference frame identifier to the block for establishing the virtual reference frame in the at least one image frame, at least one image
  • Each image frame in the frame includes at least one block, at least one image frame does not include a virtual reference frame, and the virtual reference frame identifier is used to indicate that the block is used to establish a virtual reference frame;
  • a sending unit configured to transmit at least one image frame to the decoder.
  • the processing unit is further configured to: when the number of similar pixel pairs in the target block and the target background block is greater than or equal to the first threshold, determine the target block as a block for establishing a virtual reference frame;
  • the target block is any one of the target image frames
  • the target background block is a corresponding block in the background image determined according to the position of the target block in the target image frame, and at least one image frame includes the target image frame
  • the pair of pixels includes two pixels respectively corresponding to positions in the target block and the target background block, and the absolute values of the difference between the two pixels included in the similar pixel pair satisfying at least one component are less than or equal to the second threshold.
  • the processing unit is further configured to generate a background image according to a preset modeling algorithm based on the at least one image frame.
  • the processing unit is further configured to allocate a second virtual reference frame buffer; determine a location of each block used to establish the virtual reference frame according to a preset location mapping relationship, and use each The blocks for establishing the virtual reference frame are respectively stored in the second virtual reference frame buffer.
  • the virtual reference frame identifier is further used to indicate location information of the block in which it is located.
  • the sending unit is further configured to send indication information to the decoder, where the indication information is used to indicate a preset location mapping relationship.
  • the processing unit is further configured to add a preset number of bits to the block used to establish the virtual reference frame, where the virtual reference frame identifier is carried on the newly added preset number of bits.
  • the processing unit is further configured to: set a field in a preset format in a block used to establish a virtual reference frame, where the virtual reference frame identifier is carried in a preset format field, and the preset format is The field is used to indicate at least one of a preset quantization step size, a preset block size, a preset encoding mode, a preset motion vector, a preset residual, and a preset quantization parameter.
  • the processing unit is further configured to: when the blocks used to establish the virtual reference frame in the at least one image frame are combined into a virtual reference frame, refer to the virtual reference frame for encoding;
  • the sending unit is further configured to send a decoding indication signal to the decoder, where the decoding indication signal is used to indicate decoding by referring to the virtual reference frame.
  • a decoder comprising: a processor, a memory and a receiver, a processor and a memory connected to each other, a processor and a receiver connected to each other, a memory for storing data and a program, and a processor for calling the memory
  • the stored program implements the following functions:
  • a receiver configured to receive at least one image frame transmitted by the encoder, where each image frame in the at least one image frame includes at least one block;
  • a processor configured to establish a virtual reference frame by using a block with a virtual reference frame identifier in the at least one image frame, where the virtual reference frame identifier is used to indicate that the block is used to establish a virtual reference frame.
  • the processor is further configured to allocate a first virtual reference frame buffer; determine a location of each block with a virtual reference frame identifier, and each block with a virtual reference frame identifier They are stored in the first virtual reference frame buffer area, respectively.
  • the processor is further configured to determine a location of each block with a virtual reference frame identifier according to a preset location mapping relationship.
  • the virtual reference frame identifier is further used to indicate location information of the block in which the block is located.
  • the processor is further configured to determine a location of each block with a virtual reference frame identifier according to the indication of the virtual reference frame identifier.
  • the receiver receives the indication information sent by the encoder, where the indication information is used to indicate a preset location mapping relationship.
  • the virtual reference frame identifier is carried on a newly added preset number of bits.
  • the virtual reference frame identifier is carried in a preset format field, and the preset format field is used to indicate a preset quantization step size, a preset block size, a preset encoding mode, and a preset motion vector. At least one of a preset residual and a preset quantization parameter.
  • the receiver is further configured to receive a decoding indication signal sent by the encoder, where the decoding indication signal is used to indicate decoding by referring to the virtual reference frame;
  • the processor is further configured to perform decoding by referring to the virtual reference frame according to the decoding indication signal.
  • an encoder comprising: a processor, a memory and a transmitter, a processor and a memory connected to each other, a processor and a transmitter connected to each other, a memory for storing data and a program, and a processor for calling the memory storage
  • the program implements the following functions:
  • a processor configured to acquire at least one image frame, determine a block for establishing a virtual reference frame in the at least one image frame, and add a virtual reference frame identifier to the block for establishing the virtual reference frame in the at least one image frame, at least one image
  • Each image frame in the frame includes at least one block, at least one image frame does not include a virtual reference frame, and the virtual reference frame identifier is used to indicate that the block is used to establish a virtual reference frame;
  • a transmitter for transmitting at least one image frame to the decoder.
  • the processor is further configured to determine the target block as a block for establishing a virtual reference frame when the number of similar pixel pairs in the target block and the target background block is greater than or equal to the first threshold;
  • the target block is any one of the target image frames
  • the target background block is a corresponding block in the background image determined according to the position of the target block in the target image frame, and at least one image frame includes the target image frame
  • the pair of pixels includes two pixels respectively corresponding to positions in the target block and the target background block, and the absolute values of the difference between the two pixels included in the similar pixel pair satisfying at least one component are less than or equal to the second threshold.
  • the processor is further configured to generate a background image according to a preset modeling algorithm based on the at least one image frame.
  • the processor is further configured to allocate a second virtual reference frame buffer; according to a preset The location mapping relationship determines the location of each block used to establish the virtual reference frame, and stores each block used to establish the virtual reference frame in a second virtual reference frame buffer.
  • the virtual reference frame identifier is further used to indicate location information of the block in which it is located.
  • the transmitter is further configured to send indication information to the decoder, where the indication information is used to indicate a preset location mapping relationship.
  • the processor is further configured to add a preset number of bits to the block used to establish the virtual reference frame, where the virtual reference frame identifier is carried on the newly added preset number of bits.
  • the processor is further configured to: set a field of a preset format in a block for establishing a virtual reference frame, where the virtual reference frame identifier is carried in a field of a preset format, and the preset format is The field is used to indicate at least one of a preset quantization step size, a preset block size, a preset encoding mode, a preset motion vector, a preset residual, and a preset quantization parameter.
  • the processor is further configured to: when the blocks used to establish the virtual reference frame in the at least one image frame are combined into a virtual reference frame, refer to the virtual reference frame for encoding;
  • the transmitter is further configured to send a decoding indication signal to the decoder, where the decoding indication signal is used to indicate decoding by referring to the virtual reference frame.
  • an encoder acquires at least one image frame, and determines, in at least one image frame, a block for establishing a virtual reference frame, and is used for establishing at least one image frame.
  • the block of the virtual reference frame adds a virtual reference frame identifier and transmits at least one image frame to the decoder. Since the virtual reference frame identifier is added to the block used to establish the virtual reference frame, there is no need to transmit the virtual reference frame during transmission, which will not cause the code rate to rise, and the decoder only needs to extract the block with the virtual reference frame identifier. Coming out to form a virtual reference frame does not require the same algorithm as the encoder to create a virtual reference frame based on the image frame, reducing the complexity and power consumption of the decoder.
  • FIG. 1 is a schematic flowchart of a method for establishing a virtual reference frame according to an embodiment of the present application
  • 2a is a schematic diagram of a location mapping relationship provided by an embodiment of the present application.
  • 2b is a schematic diagram of another location mapping relationship provided by an embodiment of the present application.
  • FIG. 3 is a schematic structural diagram of an image frame according to an embodiment of the present disclosure.
  • FIG. 3b is a schematic structural diagram of a field in a preset format according to an embodiment of the present disclosure
  • FIG. 4 is a schematic diagram of a virtual reference frame establishment effect according to an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a block effect of the same position according to an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a reference frame buffer queue according to an embodiment of the present disclosure.
  • FIG. 7 is a schematic flowchart of a method for establishing a virtual reference frame according to another embodiment of the present application.
  • FIG. 8 is a schematic flowchart of a method for establishing a virtual reference frame according to another embodiment of the present disclosure.
  • FIG. 9 is a schematic structural diagram of a decoder according to an embodiment of the present disclosure.
  • FIG. 10 is a schematic structural diagram of an encoder according to an embodiment of the present disclosure.
  • FIG. 11 is a schematic structural diagram of another decoder provided by an embodiment of the present application.
  • FIG. 12 is a schematic structural diagram of another encoder according to an embodiment of the present application.
  • An embodiment of the present application provides a method for establishing a virtual reference frame, which is applied to a decoder. As shown in FIG. 1, the method includes the following steps:
  • Each of the at least one image frame may include one block or a plurality of blocks. It should be noted that one block includes a plurality of pixels, for example, one block may include 16 ⁇ 16 pixels, or 8 ⁇ 8 pixels, and may of course be other numbers, which is not limited in this application.
  • at least one image frame contains the data content of the video.
  • At least one image frame may include a block with a virtual reference frame identifier for each image frame, or a partial image frame may include a block with a virtual reference frame identifier.
  • An image frame may include one or more blocks with virtual reference frame identification.
  • the method further includes: allocating a first virtual reference frame buffer.
  • the first virtual reference frame buffer is used to store a block with a virtual reference frame identifier, and the size of the first virtual reference frame buffer may be an image frame size, when the first virtual reference frame buffer is stored.
  • the block with the virtual reference frame identifier fills the entire first virtual reference frame buffer, the virtual reference frame is established, and the virtual reference frame may be extracted in the first virtual reference frame buffer; or, when the first virtual reference is used
  • the virtual reference frame is established when the number of blocks with the virtual reference frame identifier stored in the frame buffer exceeds the preset number.
  • an initial image may be stored, and the initial image may be a solid color image, such as an all white image or an all black image, or only the first virtual reference frame buffer may be allocated. , do not save images.
  • Establishing a virtual reference frame by using a plurality of blocks with virtual reference frame identifiers in at least one image frame including:
  • Each block with a virtual reference frame identification is stored in the first virtual reference frame buffer to form a virtual reference frame in accordance with each block mapped position with a virtual reference frame identification. It should be noted that when the positions of the two blocks with the virtual reference frame identifier are the same, the previously received block with the virtual reference frame identifier can be replaced by the block with the virtual reference frame identifier received later. Or, the pixel components of each of the two blocks with the same virtual reference frame identification are calculated as weighted averages to form a new block stored in the first virtual reference frame buffer.
  • the preset location mapping relationship may be a location-equivalent mapping relationship, that is, the blocks with the virtual reference frame identifiers have the same position in the frame before and after the mapping.
  • the block a is a band.
  • a block with a virtual reference frame identifier the first image frame is the frame in which the block a is located, the position of the block a in the first image frame is the third row and the fourth column, and the block a is in the created virtual reference frame.
  • the position is also the third row and the fourth column; or the preset positional relationship is a mapping relationship of different positions, that is, the position of the block with the virtual reference frame identifier changes in the frame before and after the mapping, for example,
  • block a is a block with a virtual reference frame identifier
  • the first image frame is the frame in which block a is located
  • the position of block a in the first image frame is the third row and the fourth column
  • the block The position in a virtual reference frame after the establishment is also the fourth row and the fifth column.
  • the preset location mapping relationship may be stored in the decoder in advance, or may receive indication information sent by the encoder, where the indication information is used to indicate a preset location mapping relationship.
  • the virtual reference frame identifier includes location information for indicating the location of the block, and the virtual reference frame is established by using a plurality of blocks with the virtual reference frame identifier in the at least one image frame, including:
  • the location information contained in the virtual reference frame identifier in each block with the virtual reference frame identifier will be virtual
  • a block of reference frame identification is stored in the first virtual reference frame buffer to form a virtual reference frame.
  • the virtual reference frame identifier is used to indicate that the block in which it is located is used to establish a virtual reference frame. It should be noted that the virtual reference frame identifier can exist in multiple ways. Two specific methods are provided here:
  • the virtual reference frame identifier is carried on the newly added preset number of bits. That is, the virtual reference frame identifier is represented by an extra bit.
  • FIG. 3a is a schematic structural diagram of an image frame, and an image frame includes a network abstraction layer unit header. , slice segment header and slice segment data.
  • the frame data includes a coding tree unit, the coding tree unit includes a coding quadtree, the coding quadtree includes a coding unit, and the like, such a level one level division.
  • the field of the preset format may include one of a motion vector difference field, a coding unit quantization parameter delta absolute (cu_qp_delta_abs) field, a split mode (part_mode) field, or the like. Multiple.
  • the preset format field may be used to indicate one or more of a preset quantization step size, a preset block size, a preset encoding mode, a preset motion vector, a preset residual, and a preset quantization parameter.
  • the field of the preset format may be a field directly indicating a parameter, or a plurality of fields may be combined to indicate a parameter. For example, as shown in FIG. 3b, FIG.
  • the field of the preset format may include an initial quantized coefficient minus 26 (Intitial QP minus 26, init_qp_minus26) field, and a coding unit QP delta enabled flag (Coding unit QP delta enabled flag, Cu_qp_delta_enabled_flag) field, a coding unit QD delta depth (diff_cu_qp_delta_depth) field, a coding unit quantized coefficient change absolute value field, and a coding unit QP delta sign flag (cu_qp_delta_sign_flag) field
  • the QP is a Quantization Parameter (QP).
  • the block indicating the field of the preset format is a block for establishing a virtual reference frame. .
  • the decoder may receive a decoding indication signal sent by the encoder, where the decoding indication signal is used to indicate that the block with the virtual reference frame identifier stored in the first virtual reference frame buffer may form a virtual reference frame, and the decoder is at the first
  • the virtual reference frame extracted from the virtual reference frame buffer is added to the first reference frame buffer queue, and the subsequently received image frame is decoded with reference to the virtual reference frame, where the first reference frame buffer queue includes a reference frame for decoding.
  • it is determined according to the information contained in each image frame, which reference frames are used for decoding, which is determined at the time of encoder encoding, and the reference frame number used for decoding is written into the image frame, and decoded.
  • the device selects the corresponding reference frame decoding according to the information contained in each image frame.
  • FIG. 4 Exemplarily, referring to FIG. 4, FIG. 5 and FIG. 6, the specific example of the process of establishing a virtual reference frame is described in detail in this embodiment:
  • the decoder before receiving the decoding indication signal, stores the block with the virtual reference frame identifier in the first virtual reference frame buffer area, and receives an image frame with a virtual reference in the image frame.
  • the block identified by the frame is stored in the first virtual reference frame buffer, and then receives the next image frame, such that one image frame is processed by one image frame, and the block with the virtual reference frame identifier in the image frame is stored in the first
  • the virtual reference frame buffer is the process of establishing a virtual reference frame; at this time, the frame formed by the block with the virtual reference frame identifier buffered in the first virtual reference frame buffer cannot be used for decoding.
  • the decoder Before receiving the indication signal, the decoder stores the block with the virtual reference frame identifier in each image frame in the first virtual reference frame buffer according to the respective location, when two blocks with virtual reference frame identification Position phase At the same time, the block with the virtual reference frame identifier received before can be replaced by the block with the virtual reference frame identifier received later; or each of the two locations with the same virtual reference frame identifier
  • the pixel component of one pixel calculates a weighted average to form a new block that is stored in the first virtual reference frame buffer.
  • the decoder receives five blocks with virtual reference frame identifiers in the first image frame, which are block a, block b, block c, block d, and block e, respectively.
  • the positions of the blocks are respectively stored in the first virtual reference frame buffer area
  • the decoder receives three blocks with virtual reference frame identifiers in the second image frame, which are block f, block g and block h, respectively, according to three
  • the positions of the blocks are respectively stored in the virtual reference frame buffer, but the block a in the first image frame and the block f in the second image frame are in the same position, and the block a can be replaced with the block f, and the block f and the block f can be calculated.
  • the weighted average of the components of each pixel in block a forms a new block i and stores block i at the original location of block f and block a.
  • block 1 and block f respectively include 4 pixels, and block a includes pixel a1, pixel a2, pixel a3, and pixel a4, and block f includes pixel f1, pixel f2, pixel f3, and pixel f4, pixel a1 and
  • the pixel f1 corresponds to the position
  • the pixel a2 and the pixel f2 correspond to each other
  • the pixel a3 and the pixel f3 correspond to each other
  • the pixel a4 and the pixel f4 correspond to each other.
  • Each component of the pixel can be calculated according to the first formula or the second formula, and each pair of corresponding pixels can also calculate a new value of each component according to a preset formula, so that a new pixel i1 and a pixel can be obtained.
  • I2, pixel i3 and pixel i4, can form a new block i, and store block i in the original position of block a.
  • the decoding indication signal indicates that the block with the virtual reference frame identifier buffered in the first virtual reference frame buffer constitutes a virtual reference frame that can be used for decoding, and after receiving the decoding indication signal, the decoder will The virtual reference frame in the first virtual reference frame buffer is extracted and added to the first reference frame buffer queue, and the decoder decodes the image frame received later with reference to the virtual reference frame.
  • the frames that need to be transmitted in this application are all image frames.
  • the blocks used to establish the virtual reference frame are added with identifiers in the image frames.
  • the virtual reference frames are not separately transmitted, which reduces the amount of data transmission and saves network resources.
  • the decoder establishes a background image in the image frame according to the same algorithm as the encoder, selects a block that establishes a virtual reference frame, and establishes a virtual reference frame, as compared with
  • the present application does not need a decoder to establish a background image, and selects a block for establishing a virtual reference frame, and only needs to form a virtual reference frame with a virtual reference frame identifier according to the virtual reference frame identifier, thereby reducing the decoder.
  • the complexity saves processor resources. Moreover, because the decoder only needs to obtain the block for establishing the virtual reference frame according to the virtual reference frame identifier, if the algorithm used by the encoder to establish the virtual reference frame is changed, the decoder is not affected, and the compatibility is better. .
  • another embodiment of the present application provides a method for establishing a virtual reference frame, which is applied to an encoder. As shown in FIG. 7, the method includes the following steps:
  • the at least one image frame may be one image frame or multiple image frames, one image frame may comprise one block or multiple blocks, and one block may contain one or more pixels.
  • the method for establishing the virtual reference frame provided by the embodiment further includes:
  • the second virtual reference frame buffer is allocated, and the second virtual reference frame buffer may buffer a block for establishing a virtual reference frame, and the size of the second virtual reference frame buffer may be the size of one image frame. It should be noted that, while the second virtual reference frame buffer area is allocated, an initial image may be stored, and the initial image may be a solid color image, such as an all white image or a full black image, or may simply allocate a second virtual reference frame buffer. , do not save images.
  • any one of the at least one image frame is used as the target image frame, and any one of the target image frames is used as the target block.
  • the target block and the target background block are similar pixel pairs.
  • the target block is determined as a block for establishing a virtual reference frame, and the first threshold is a preset value, and the first threshold may be obtained through experiments, or may refer to commonly used in related technologies. The value is determined.
  • the target background block is a block corresponding to the target block in the target image frame according to the position of the target block in the target image frame, and the background image may be generated according to a preset modeling algorithm based on at least one image frame, for example, In a video, the fixed background will reappear after the foreground object leaves. The fixed background will not change and will only be obscured by the foreground object. This fixed background is the background image.
  • a pixel pair refers to two pixels corresponding to positions in the target block and the target background block, and two of the similar pixel pairs satisfy the difference that the absolute value of the difference between the at least one identical pixel component is less than or equal to the second threshold, and second
  • the threshold is a preset value, and the second threshold can be obtained through experiments, or can be determined by referring to values commonly used in related art.
  • the position of the target background block in the background image may be the same as the position of the target block in the target image frame.
  • each image frame may include m rows and n columns, and the position of each block may be Determined by the row number and column number in which the target block is located in the x row y column in the target image frame, and the target background block is also located in the x row y column in the background image; or, the position of the target background block in the background image is The position of the target block in the target image frame may also be different.
  • the target block is located in the x-row y column in the target image frame, and the target background block is also located in the x+1 row y+1 column in the background image. No restrictions.
  • one pixel may contain three pixel components, such as a YUV pixel component, where Y is a luminance component and U, V are chrominance components.
  • Y is a luminance component
  • U, V are chrominance components.
  • the pixel pair is determined to be a similar pixel pair, or may be three pixels.
  • the pair of pixels is determined to be a pair of similar pixels, otherwise it is a pair of dissimilar pixels.
  • the method further includes:
  • each image frame may contain blocks of m rows and n columns, taking block a in the first image frame as an example, block a is a block for establishing a virtual reference frame, and block a is in the first image frame.
  • the position of the block a in the first image frame may be represented as (x, y), and the preset position mapping relationship may indicate the storage of the block a in the second virtual reference frame buffer area.
  • the position is (x, y), or the preset position mapping relationship may indicate that the storage location of the block a in the second virtual reference frame buffer is (x+dx, y+dy), where dx and dy are positions.
  • the offset is merely illustrative here and does not represent that the application is limited thereto.
  • the encoder may send the indication information by the decoder, where the indication information is used to indicate a preset location mapping relationship.
  • the virtual reference frame identifier is used to indicate that the block in which it is located is used to establish a virtual reference frame.
  • this embodiment provides two specific ways of adding a virtual reference frame identifier:
  • the virtual reference frame identifier is carried on the newly added preset number of bits. That is, the virtual reference frame identifier is represented by an extra bit.
  • the virtual reference frame identifier is carried in a field in a preset format.
  • the field of the preset format may include one or more of a field of a motion vector difference field, a coding unit quantization coefficient change absolute value field, a split mode field, and the like.
  • the preset format field may be used to indicate one or more of a preset quantization step size, a preset block size, a preset encoding mode, a preset motion vector, a preset residual, and a preset quantization parameter.
  • the location information may be separately added to the virtual reference frame identifier, so that the decoder may directly follow the virtual
  • the reference frame identification determines the location of each block used to create the virtual reference frame and no further calculations are required.
  • the encoder When the encoder adds the virtual reference frame identifier to the block used to create the virtual reference frame in an image frame, the image frame can be sent and the next image frame continues to be processed.
  • the virtual reference frame may be encoded; specifically, when the second virtual reference frame buffer is completely filled or stored.
  • the virtual reference frame is considered to be completed, and the virtual reference frame may be placed in the second reference frame buffer queue, and the image frame after the virtual reference frame is encoded, and the second reference frame buffer queue is encoded.
  • the decoding indication signal may be sent to the decoder, where the decoding indication signal is used to instruct the decoder to perform decoding by referring to the virtual reference frame.
  • the decoder stores the block with the virtual reference frame identifier in the first virtual reference frame buffer before receiving the decoding indication signal, and when the decoding indication signal is received, the first virtual The virtual reference frame in the reference frame buffer is extracted and placed in the first reference frame buffer queue, and the image frame received later is decoded with reference to the virtual reference frame.
  • the encoder may encode at least one image frame.
  • the at least one image frame transmitted to the decoder does not include the virtual reference frame.
  • the decoding indication signal is last used in at least one image frame to establish a virtual reference frame.
  • the decoding indication signal can also directly indicate the frame number or the block number of the last transmitted image frame when the virtual reference frame is established. This application does not limit this.
  • the encoder adds a virtual reference frame identifier to the block used to establish the virtual reference frame, there is no need to transmit the virtual reference frame during transmission, which does not cause the code rate to rise, and the decoder only needs to have the virtual reference frame identifier.
  • the blocks are extracted to form a virtual reference frame, and the same algorithm as the encoder is not needed to establish a background image based on the image frame, and then the block for establishing the virtual reference frame is selected, and then the virtual reference frame is established, which reduces the complexity of the decoder and Power consumption.
  • the encoder and the decoder use the same algorithm to establish a virtual reference frame.
  • the decoder cannot be compatible with different algorithms, or the encoding.
  • the algorithm for establishing a virtual reference frame is updated, and the decoder also needs to be updated.
  • the decoder is often located on the video playback device used by the user, for example, a television, a computer, etc., and the update period of these devices is generally long, and will be given User use is inconvenient.
  • the method for establishing a virtual reference frame provided by the present application, because the block for establishing the virtual reference frame identifier has a virtual reference frame identifier, and the decoder extracts the block with the virtual reference frame identifier to establish a virtual reference frame, without Encoder one
  • the algorithm uses the algorithm to establish a virtual reference frame. Therefore, even if the algorithm for establishing the virtual reference frame on the encoder side does not affect the decoder to establish a virtual reference frame, the decoder has good compatibility with encoders produced by different manufacturers. It will not cause inconvenience to users.
  • the method for establishing the virtual reference frame provided in this embodiment is the coding side method corresponding to the method for establishing the decoding side virtual reference frame described in the foregoing embodiment corresponding to FIG. 1, and the explanation and the diagram in the corresponding embodiment of FIG. The explanations in the corresponding embodiments of 7 apply to each other.
  • another embodiment of the present application provides a method for establishing a virtual reference frame, which is applied to an encoder and a decoder.
  • the method includes the following steps:
  • the encoder acquires N image frames.
  • N is a positive integer greater than one.
  • the encoder determines, in the nth image frame, a block used to establish a virtual reference frame.
  • the nth image frame is the nth image frame among the N image frames acquired by the encoder, and n is an integer in [1, N].
  • the encoder stores, in the second virtual reference frame buffer, a block for establishing a virtual reference frame in the nth image frame, and adds a virtual reference frame to the block used to establish the virtual reference frame in the nth image frame.
  • the encoder transmits the nth image frame to the decoder.
  • Steps 802 - 804 are repeated. After step 804, the method further includes:
  • the encoder When the number of blocks used to establish the virtual reference frame in the second virtual reference frame buffer exceeds a preset number, the encoder establishes a virtual reference frame and sends a decoding indication signal to the decoder.
  • the encoder establishes a virtual reference frame by using a block in the second virtual reference frame buffer for establishing a virtual reference frame, and the encoder encodes the image frame to be transmitted after referring to the virtual reference frame after the virtual reference frame is established.
  • the decoder After receiving the indication signal sent by the encoder, the decoder establishes a virtual reference frame by using a block with a virtual reference frame identifier in the received image frame.
  • the decoder may extract the block with the virtual reference frame identifier stored in the first virtual reference frame buffer to form a virtual reference frame.
  • the decoder adds the virtual reference frame to the first reference frame buffer queue, and performs decoding by referring to the virtual reference frame.
  • the frames that need to be transmitted in this application are all image frames, and no virtual reference frames are transmitted.
  • the related art not only image frames but also virtual reference frames are transmitted.
  • the technical solution of the present application reduces the amount of data transmission. Save network resources.
  • some related technologies do not transmit virtual reference frames, but the decoder establishes a background image in the image frame according to the same algorithm as the encoder, selects a block that establishes a virtual reference frame, and establishes a virtual reference frame, in contrast.
  • the present application does not require a decoder to establish a background image, and selects a block for establishing a virtual reference frame, and only needs to form a virtual reference frame with the virtual reference frame identifier directly according to the virtual reference frame identifier, thereby reducing the decoder. Complexity saves processor resources. Moreover, because the decoder only needs to obtain the block for establishing the virtual reference frame according to the virtual reference frame identifier, if the algorithm used by the encoder to establish the virtual reference frame is changed, the decoder is not affected, and the compatibility is better. .
  • the embodiment of the present application provides a decoder for performing the method for establishing a virtual reference frame described in the foregoing embodiment corresponding to FIG. 1.
  • the decoder 90 is shown.
  • the processing unit 901 and the receiving unit 902 are included.
  • the receiving unit 902 is configured to receive at least one image frame transmitted by the encoder, and each of the at least one image frame The frame contains at least one block;
  • the processing unit 901 is configured to establish a virtual reference frame by using a block with a virtual reference frame identifier in the at least one image frame, where the virtual reference frame identifier is used to indicate that the block is used to establish a virtual reference frame.
  • the processing unit 901 is further configured to allocate a first virtual reference frame buffer; determine a location of each block with a virtual reference frame identifier, and each with a virtual reference frame identifier The blocks are stored in the first virtual reference frame buffer area, respectively.
  • the processing unit 901 is further configured to determine, according to a preset location mapping relationship, a location of each block with a virtual reference frame identifier.
  • the virtual reference frame identifier is further used to indicate location information of the block in which the block is located.
  • the processing unit 901 is further configured to determine, according to the indication of the virtual reference frame identifier, a location of each block with a virtual reference frame identifier.
  • the receiving unit 902 receives the indication information sent by the encoder, where the indication information is used to indicate a preset location mapping relationship.
  • the virtual reference frame identifier is carried on a newly added preset number of bits.
  • the virtual reference frame identifier is carried in a preset format field, and the preset format field is used to indicate a preset quantization step size, a preset block size, a preset encoding mode, and a preset motion vector. At least one of a preset residual and a preset quantization parameter.
  • the receiving unit 902 is further configured to receive a decoding indication signal sent by the encoder, where the decoding indication signal is used to indicate decoding by referring to the virtual reference frame.
  • the processing unit 901 is further configured to perform decoding by referring to the virtual reference frame according to the decoding indication signal.
  • the decoder provided by the embodiment of the present application receives at least one image frame transmitted by the encoder, and establishes a virtual reference frame by using a block with a virtual reference frame identifier in at least one image frame. Since the virtual reference frame identifier is added to the block used to establish the virtual reference frame, there is no need to transmit the virtual reference frame during transmission, which will not cause the code rate to rise, and the decoder only needs to extract the block with the virtual reference frame identifier. Coming out to form a virtual reference frame does not require the same algorithm as the encoder to create a virtual reference frame based on the image frame, reducing the complexity and power consumption of the decoder.
  • the embodiment of the present application provides an encoder for performing the method for establishing a virtual reference frame described in the foregoing embodiment corresponding to FIG. 8.
  • the encoder 100 is shown.
  • the processing unit 1001 and the transmitting unit 1002 are included.
  • the processing unit 1001 is configured to acquire at least one image frame, determine a block for establishing a virtual reference frame in the at least one image frame, and add a virtual reference frame identifier to the block used to establish the virtual reference frame in the at least one image frame, at least one Each image frame in the image frame includes at least one block, at least one image frame does not include a virtual reference frame, and the virtual reference frame identifier is used to indicate that the block is used to establish a virtual reference frame;
  • the sending unit 1002 is configured to transmit at least one image frame to the decoder.
  • the processing unit 1001 is further configured to: when the number of similar pixel pairs in the target block and the target background block is greater than or equal to the first threshold, determine the target block as a block for establishing a virtual reference frame. ;
  • the target block is any one of the target image frames
  • the target background block is a corresponding block in the background image determined according to the position of the target block in the target image frame, and at least one image frame includes the target image frame
  • the pair of pixels includes two pixels respectively corresponding to positions in the target block and the target background block, and the absolute values of the difference between the two pixels included in the similar pixel pair satisfying at least one component are less than or equal to the second threshold.
  • the processing unit 1001 is further configured to generate a background image according to a preset modeling algorithm based on the at least one image frame.
  • the processing unit 1001 is further configured to allocate a second virtual reference frame buffer; determine a location of each block used to establish a virtual reference frame according to a preset location mapping relationship, and each The blocks used to create the virtual reference frame are stored in the second virtual reference frame buffer, respectively.
  • the virtual reference frame identifier is further used to indicate location information of the block in which it is located.
  • the sending unit 1002 is further configured to send indication information to the decoder, where the indication information is used to indicate a preset location mapping relationship.
  • the processing unit 1001 is further configured to add a preset number of bits to the block used to establish the virtual reference frame, where the virtual reference frame identifier is carried on the newly added preset number of bits.
  • the processing unit 1001 is further configured to: set a field in a preset format in a block used to establish a virtual reference frame, where the virtual reference frame identifier is carried in a preset format field, and the preset format is The field is used to indicate at least one of a preset quantization step size, a preset block size, a preset encoding mode, a preset motion vector, a preset residual, and a preset quantization parameter.
  • the processing unit 1001 is further configured to: when the blocks used to establish the virtual reference frame in the at least one image frame are combined into a virtual reference frame, perform coding by referring to the virtual reference frame;
  • the sending unit 1002 is further configured to send a decoding indication signal to the decoder, where the decoding indication signal is used to indicate decoding by referring to the virtual reference frame.
  • the encoder provided by the embodiment of the present application acquires at least one image frame, determines a block for establishing a virtual reference frame in at least one image frame, and adds a virtual reference frame identifier to a block used to establish a virtual reference frame in at least one image frame. Transmitting at least one image frame to the decoder. Since the virtual reference frame identifier is added to the block used to establish the virtual reference frame, there is no need to transmit the virtual reference frame during transmission, which will not cause the code rate to rise, and the decoder only needs to extract the block with the virtual reference frame identifier. Coming out to form a virtual reference frame does not require the same algorithm as the encoder to create a virtual reference frame based on the image frame, reducing the complexity and power consumption of the decoder.
  • the embodiment of the present application provides another decoder for performing the method for establishing a virtual reference frame described in the foregoing embodiment corresponding to FIG. 1, and the decoder is shown in FIG. 110 includes a processor 1101, a memory 1102, and a receiver 1103.
  • the processor 1101 and the memory 1102 are connected to each other, and the processor 1101 and the receiver 1103 are connected to each other.
  • the memory 1102 is used to store data and programs.
  • the processor 1101 is configured to invoke a program stored in the memory to implement a method for establishing a virtual reference frame described in the embodiment corresponding to FIG. 1.
  • the processor 1101 corresponds to the processing unit 301 in the embodiment corresponding to FIG.
  • the receiver 1103 corresponds to the receiving unit 302 in the embodiment corresponding to FIG.
  • Receiver 1103 can be an antenna or a wired network interface.
  • the functions implemented by the processor 1101 and the receiver 1103 are the same as those of the processing unit 301 and the receiving unit 302 in the embodiment corresponding to FIG. 9, and are not described herein again.
  • the decoder provided by the embodiment of the present application receives at least one image frame transmitted by the encoder, and establishes a virtual reference frame by using a block with a virtual reference frame identifier in at least one image frame. Since the virtual reference frame identifier is added to the block used to establish the virtual reference frame, there is no need to transmit the virtual reference frame during transmission, which will not cause the code rate to rise, and the decoder only needs to extract the block with the virtual reference frame identifier. Coming out to form a virtual reference frame, no need to use the same algorithm as the encoder The establishment of virtual reference frames based on image frames reduces the complexity and power consumption of the decoder.
  • the embodiment of the present application provides another encoder for performing the method for establishing a virtual reference frame described in the foregoing embodiment corresponding to FIG. 8.
  • the encoder is shown. 120 includes a processor 1201, a memory 1202, and a transmitter 1203.
  • the processor 1201 and the memory 1202 are connected to each other, and the processor 1201 and the transmitter 1203 are connected to each other.
  • the memory 1202 is used to store data and programs.
  • the processor 1201 is configured to invoke a program stored in the memory to implement a method for establishing a virtual reference frame described in the embodiment corresponding to FIG. 7.
  • the processor 1201 corresponds to the processing unit 401 in the embodiment corresponding to FIG.
  • the transmitter 1203 corresponds to the transmitting unit 402 in the embodiment corresponding to FIG.
  • Transmitter 1203 can be an antenna or a wired network interface.
  • the functions implemented by the processor 1201 and the transmitter 1203 are the same as those of the processing unit 401 and the sending unit 402 in the embodiment corresponding to FIG. 10, and details are not described herein again.
  • the encoder acquires at least one image frame, determines a block for establishing a virtual reference frame in at least one image frame, and adds a virtual reference frame identifier to a block used to establish a virtual reference frame in at least one image frame. Transmitting at least one image frame to the decoder. Since the virtual reference frame identifier is added to the block used to establish the virtual reference frame, there is no need to transmit the virtual reference frame during transmission, which does not cause the code rate to rise, and the decoder only needs to block the block with the virtual reference frame identifier.
  • the virtual reference frames are extracted to form a virtual reference frame based on the image frame without using the same algorithm as the encoder, which reduces the complexity and power consumption of the decoder.
  • a person skilled in the art may understand that all or part of the steps of implementing the above embodiments may be completed by hardware, or may be instructed by a program to execute related hardware, and the program may be stored in a computer readable storage medium.
  • the storage medium mentioned may be a read only memory, a magnetic disk or an optical disk or the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

本申请公开了一种虚拟参考帧的建立方法及设备，涉及图像处理领域，能够解决现有技术中建立虚拟参考帧时造成的解码器复杂度较高，功耗较大或者需要额外码率传输的问题。具体方案为：接收编码器传输的至少一个图像帧，利用至少一个图像帧中带有虚拟参考帧标识的块建立虚拟参考帧。因为用于建立虚拟参考帧的块添加了虚拟参考帧标识，在传输过程中不需要传输虚拟参考帧，不会造成码率上升，而且，解码器只需要将带有虚拟参考帧标识的块提取出来组成虚拟参考帧，不需要用和编码器相同的算法基于图像帧建立虚拟参考帧，减少了解码器的复杂度和功耗。本申请用于建立虚拟参考帧。

Description

虚拟参考帧的建立方法及设备
本申请要求于2016年11月18日提交中国专利局、申请号为201611035422.8、发明名称为“虚拟参考帧的建立方法及设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理领域，特别涉及一种虚拟参考帧的建立方法及设备。
背景技术
在视频文件的传输过程中，编码器将每一帧图像进行编码后传输至解码器，解码器将图像解码然后在显示设备上播放视频，在播放视频时，有一些图像会重复出现，例如固定的背景，会在前景物体离开后重新出现，将这些重复出现的图像作为虚拟参考帧，虚拟参考帧在视频播放的过程中并不会显示，只是用来在编码和解码时进行参考，提高编码和解码效率。
例如，相关技术中，编码器根据待传输的至少一个图像帧以及预设的建模算法生成背景图像，该背景图像可以是视频播放时固定的背景，对于至少一个图像帧中每一个图像帧，如果该图像帧中某个块与背景图像中对应的块相似，则将相似的块作为用于建立虚拟参考帧的块，将用于建立虚拟参考帧的块存储在一个缓存区中，然后将挑选过建立虚拟参考帧的块的图像帧发送至解码器，这样，当存入足够多的用于建立虚拟参考帧的块时，将缓存区中存储的用于建立虚拟参考帧的块提取出来作为虚拟参考帧，参考该虚拟参考帧对之后待传输的图像帧进行编码，并且将该虚拟参考帧作为单独的一帧发送至解码器。
又如，另一相关技术中，编码器不需要将虚拟参考帧单独作为一帧传输至解码器；解码器接收到图像帧后，基于和编码器相同的建模算法生成背景图像，并根据并根据背景图像挑选每一个图像帧中用于建立虚拟参考帧的块，将用于建立虚拟参考帧的块存储在一个缓存区中，当存入与编码器相同的建立虚拟参考帧的块时，将缓存区中用于建立虚拟参考帧的块提取出来作为虚拟参考帧，并参考该虚拟参考帧对之后的图像帧解码。
但是，如果将虚拟参考帧作为单独的一帧传输，增加了数据传输量，增加了额外的码率，占用了过多的网络资源；如果不传输虚拟参考帧，解码器需要基于和编码器相同的算法建立虚拟参考帧，解码器复杂度较高，功耗较大。
发明内容
为了解决现有技术中建立虚拟参考帧时造成的解码器复杂度较高，功耗较大，或者使用额外码率的问题，本申请实施例提供了一种虚拟参考帧的建立方法及设备。所述技术方案如下：
第一方面，提供了一种虚拟参考帧的建立方法，应用于解码器，该方法包括：
接收编码器传输的至少一个图像帧，至少一个图像帧中每一个图像帧包含至少一个块；
利用至少一个图像帧中带有虚拟参考帧标识的块建立虚拟参考帧，虚拟参考帧标识用 于指示所在的块用于建立虚拟参考帧。
在一种可能的实现方式中，利用至少一个图像帧中带有虚拟参考帧标识的块建立虚拟参考帧之前，方法还包括：
分配第一虚拟参考帧缓存区；
利用至少一个图像帧中带有虚拟参考帧标识的块建立虚拟参考帧，包括：
确定每一个带有虚拟参考帧标识的块的位置，并将每一个带有虚拟参考帧标识的块分别存储在第一虚拟参考帧缓存区中。
在一种可能的实现方式中，确定每一个带有虚拟参考帧标识的块的位置，包括：
根据预设的位置映射关系确定每一个带有虚拟参考帧标识的块的位置。
在一种可能的实现方式中，虚拟参考帧标识还用于指示所在的块的位置信息，确定每一个带有虚拟参考帧标识的块的位置，包括：
根据虚拟参考帧标识的指示确定每一个带有虚拟参考帧标识的块的位置。
在一种可能的实现方式中，该方法还包括：
接收编码器发送的指示信息，指示信息用于指示预设的位置映射关系。
在一种可能的实现方式中，虚拟参考帧标识承载在新增的预设数量的比特上。
在一种可能的实现方式中，虚拟参考帧标识承载在预设格式的字段上，预设格式的字段用于指示预设量化步长、预设块大小、预设编码方式、预设运动矢量、预设残差、预设量化参数中的至少一项。
在一种可能的实现方式中，利用至少一个图像帧中带有虚拟参考帧标识的块建立虚拟参考帧之后，方法还包括：
接收编码器发送的解码指示信号，根据解码指示信号，参照虚拟参考帧进行解码，解码指示信号用于指示参照虚拟参考帧进行解码。
第二方面，提供了一种虚拟参考帧的建立方法，应用于编码器，该方法包括：
获取至少一个图像帧，至少一个图像帧中每一个图像帧包含至少一个块；
在至少一个图像帧中确定用于建立虚拟参考帧的块，至少一个图像帧不包含虚拟参考帧；
对至少一个图像帧中用于建立虚拟参考帧的块添加虚拟参考帧标识，虚拟参考帧标识用于指示所在的块用于建立虚拟参考帧；
向解码器传输至少一个图像帧。
在一种可能的实现方式中，在至少一个图像帧中确定用于建立虚拟参考帧的块，包括：
当目标块与目标背景块中相似像素对的数量大于或等于第一阈值时，将目标块确定为用于建立虚拟参考帧的块；
其中，所述目标块为目标图像帧中的任意一个块，所述目标背景块是根据所述目标块在目标图像帧中的位置确定的所述目标块在背景图像中对应的块，所述至少一个图像帧包含所述目标图像帧；所述相似像素对包括分别位于所述目标块与所述目标背景块中位置相对应的两个像素，且所述相似像素对包括的两个像素满足至少一个分量之差的绝对值小于或等于第二阈值。
在一种可能的实现方式中，在至少一个图像帧中确定用于建立虚拟参考帧的编码块之前，该方法还包括：
基于所述至少一个图像帧，按照预设的建模算法生成所述背景图像。
在一种可能的实现方式中，在至少一个图像帧中确定用于建立虚拟参考帧的块之前，该方法还包括：
分配第二虚拟参考帧缓存区；
在所述至少一个图像帧中确定用于建立虚拟参考帧的块之后，所述方法还包括：
根据预设的位置映射关系确定每一个用于建立虚拟参考帧的块的位置，并将每一个用于建立虚拟参考帧的块分别存储在所述第二虚拟参考帧缓存区中。
在一种可能的实现方式中，虚拟参考帧标识还用于指示所在的块的位置信息。
在一种可能的实现方式中，该方法还包括：
向所述解码器发送指示信息，所述指示信息用于指示所述预设的位置映射关系。
在一种可能的实现方式中，对至少一个图像帧中用于建立虚拟参考帧的块添加虚拟参考帧标识，包括：
对所述用于建立虚拟参考帧的块增加预设数量的比特，所述虚拟参考帧标识承载在新增的所述预设数量的比特上。
在一种可能的实现方式中，对至少一个图像帧中用于建立虚拟参考帧的块添加虚拟参考帧标识，包括：
在所述用于建立虚拟参考帧的块中设定预设格式的字段，所述虚拟参考帧标识承载在预设格式的字段上，所述预设格式的字段用于指示预设量化步长、预设块大小、预设编码方式、预设运动矢量、预设残差、预设量化参数中的至少一项。
在一种可能的实现方式中，对至少一个图像帧中用于建立虚拟参考帧的块添加虚拟参考帧标识之后，该方法还包括：
当至少一个图像帧中用于建立虚拟参考帧的块组合成虚拟参考帧时，参照虚拟参考帧进行编码；
向解码器发送解码指示信号，解码指示信号用于指示参照虚拟参考帧进行解码。
第三方面，提供了一种解码器，包括：
接收单元，用于接收编码器传输的至少一个图像帧，至少一个图像帧中每一个图像帧包含至少一个块；
处理单元，用于利用至少一个图像帧中带有虚拟参考帧标识的块建立虚拟参考帧，虚拟参考帧标识用于指示所在的块用于建立虚拟参考帧。
在一种可能的实现方式中，处理单元，还用于分配第一虚拟参考帧缓存区；确定每一个带有虚拟参考帧标识的块的位置，并将每一个带有虚拟参考帧标识的块分别存储在第一虚拟参考帧缓存区中。
在一种可能的实现方式中，处理单元，还用于根据预设的位置映射关系确定每一个带有虚拟参考帧标识的块的位置。
在一种可能的实现方式中，虚拟参考帧标识还用于指示所在的块的位置信息，
处理单元，还用于根据虚拟参考帧标识的指示确定每一个带有虚拟参考帧标识的块的位置。
在一种可能的实现方式中，接收单元，接收编码器发送的指示信息，指示信息用于指示预设的位置映射关系。
在一种可能的实现方式中，虚拟参考帧标识承载在新增的预设数量的比特上。
在一种可能的实现方式中，虚拟参考帧标识承载在预设格式的字段上，预设格式的字段用于指示预设量化步长、预设块大小、预设编码方式、预设运动矢量、预设残差、预设量化参数中的至少一项。
在一种可能的实现方式中，接收单元，还用于接收编码器发送的解码指示信号，解码指示信号用于指示参照虚拟参考帧进行解码；
处理单元，还用于根据解码指示信号，参照虚拟参考帧进行解码。
第四方面，提供了一种编码器，包括：
处理单元，用于获取至少一个图像帧，在至少一个图像帧中确定用于建立虚拟参考帧的块，对至少一个图像帧中用于建立虚拟参考帧的块添加虚拟参考帧标识，至少一个图像帧中每一个图像帧包含至少一个块，至少一个图像帧不包含虚拟参考帧，虚拟参考帧标识用于指示所在的块用于建立虚拟参考帧；
发送单元，用于向解码器传输至少一个图像帧。
在一种可能的实现方式中，处理单元，还用于当目标块与目标背景块中相似像素对的数量大于或等于第一阈值时，将目标块确定为用于建立虚拟参考帧的块；
其中，目标块为目标图像帧中的任意一个块，目标背景块是根据目标块在目标图像帧中的位置确定的目标块在背景图像中对应的块，至少一个图像帧包含目标图像帧；相似像素对包括分别位于目标块与目标背景块中位置相对应的两个像素，且相似像素对包括的两个像素满足至少一个分量之差的绝对值小于或等于第二阈值。
在一种可能的实现方式中，处理单元，还用于基于至少一个图像帧，按照预设的建模算法生成背景图像。
在一种可能的实现方式中，处理单元，还用于分配第二虚拟参考帧缓存区；根据预设的位置映射关系确定每一个用于建立虚拟参考帧的块的位置，并将每一个用于建立虚拟参考帧的块分别存储在第二虚拟参考帧缓存区中。
在一种可能的实现方式中，虚拟参考帧标识还用于指示所在的块的位置信息。
在一种可能的实现方式中，发送单元，还用于向解码器发送指示信息，指示信息用于指示预设的位置映射关系。
在一种可能的实现方式中，处理单元，还用于对用于建立虚拟参考帧的块增加预设数量的比特，虚拟参考帧标识承载在新增的预设数量的比特上。
在一种可能的实现方式中，处理单元，还用于在用于建立虚拟参考帧的块中设定预设格式的字段，虚拟参考帧标识承载在预设格式的字段上，预设格式的字段用于指示预设量化步长、预设块大小、预设编码方式、预设运动矢量、预设残差、预设量化参数中的至少一项。
在一种可能的实现方式中，处理单元，还用于当至少一个图像帧中用于建立虚拟参考帧的块组合成虚拟参考帧时，参照虚拟参考帧进行编码；
发送单元，还用于向解码器发送解码指示信号，解码指示信号用于指示参照虚拟参考帧进行解码。
第五方面，提供了一种解码器，包括：处理器、存储器和接收器，处理器和存储器相互连接，处理器和接收器相互连接，存储器用于存储数据和程序，处理器用于调用存储器 存储的程序实现以下功能：
接收器，用于接收编码器传输的至少一个图像帧，至少一个图像帧中每一个图像帧包含至少一个块；
处理器，用于利用至少一个图像帧中带有虚拟参考帧标识的块建立虚拟参考帧，虚拟参考帧标识用于指示所在的块用于建立虚拟参考帧。
在一种可能的实现方式中，处理器，还用于分配第一虚拟参考帧缓存区；确定每一个带有虚拟参考帧标识的块的位置，并将每一个带有虚拟参考帧标识的块分别存储在第一虚拟参考帧缓存区中。
在一种可能的实现方式中，处理器，还用于根据预设的位置映射关系确定每一个带有虚拟参考帧标识的块的位置。
在一种可能的实现方式中，虚拟参考帧标识还用于指示所在的块的位置信息，
处理器，还用于根据虚拟参考帧标识的指示确定每一个带有虚拟参考帧标识的块的位置。
在一种可能的实现方式中，接收器，接收编码器发送的指示信息，指示信息用于指示预设的位置映射关系。
在一种可能的实现方式中，虚拟参考帧标识承载在新增的预设数量的比特上。
在一种可能的实现方式中，虚拟参考帧标识承载在预设格式的字段上，预设格式的字段用于指示预设量化步长、预设块大小、预设编码方式、预设运动矢量、预设残差、预设量化参数中的至少一项。
在一种可能的实现方式中，接收器，还用于接收编码器发送的解码指示信号，解码指示信号用于指示参照虚拟参考帧进行解码；
处理器，还用于根据解码指示信号，参照虚拟参考帧进行解码。
第六方面，提供了一种编码器，包括：处理器、存储器和发送器，处理器和存储器相互连接，处理器和发送器相互连接，存储器用于存储数据和程序，处理器用于调用存储器存储的程序实现以下功能：
处理器，用于获取至少一个图像帧，在至少一个图像帧中确定用于建立虚拟参考帧的块，对至少一个图像帧中用于建立虚拟参考帧的块添加虚拟参考帧标识，至少一个图像帧中每一个图像帧包含至少一个块，至少一个图像帧不包含虚拟参考帧，虚拟参考帧标识用于指示所在的块用于建立虚拟参考帧；
发送器，用于向解码器传输至少一个图像帧。
在一种可能的实现方式中，处理器，还用于当目标块与目标背景块中相似像素对的数量大于或等于第一阈值时，将目标块确定为用于建立虚拟参考帧的块；
其中，目标块为目标图像帧中的任意一个块，目标背景块是根据目标块在目标图像帧中的位置确定的目标块在背景图像中对应的块，至少一个图像帧包含目标图像帧；相似像素对包括分别位于目标块与目标背景块中位置相对应的两个像素，且相似像素对包括的两个像素满足至少一个分量之差的绝对值小于或等于第二阈值。
在一种可能的实现方式中，处理器，还用于基于至少一个图像帧，按照预设的建模算法生成背景图像。
在一种可能的实现方式中，处理器，还用于分配第二虚拟参考帧缓存区；根据预设的 位置映射关系确定每一个用于建立虚拟参考帧的块的位置，并将每一个用于建立虚拟参考帧的块分别存储在第二虚拟参考帧缓存区中。
在一种可能的实现方式中，虚拟参考帧标识还用于指示所在的块的位置信息。
在一种可能的实现方式中，发送器，还用于向解码器发送指示信息，指示信息用于指示预设的位置映射关系。
在一种可能的实现方式中，处理器，还用于对用于建立虚拟参考帧的块增加预设数量的比特，虚拟参考帧标识承载在新增的预设数量的比特上。
在一种可能的实现方式中，处理器，还用于在用于建立虚拟参考帧的块中设定预设格式的字段，虚拟参考帧标识承载在预设格式的字段上，预设格式的字段用于指示预设量化步长、预设块大小、预设编码方式、预设运动矢量、预设残差、预设量化参数中的至少一项。
在一种可能的实现方式中，处理器，还用于当至少一个图像帧中用于建立虚拟参考帧的块组合成虚拟参考帧时，参照虚拟参考帧进行编码；
发送器，还用于向解码器发送解码指示信号，解码指示信号用于指示参照虚拟参考帧进行解码。
本申请实施例提供的一种虚拟参考帧的建立方法及设备，编码器获取至少一个图像帧，在至少一个图像帧中确定用于建立虚拟参考帧的块，对至少一个图像帧中用于建立虚拟参考帧的块添加虚拟参考帧标识，向解码器传输至少一个图像帧。因为用于建立虚拟参考帧的块添加了虚拟参考帧标识，在传输过程中不需要传输虚拟参考帧，不会造成码率上升，而且，解码器只需要将带有虚拟参考帧标识的块提取出来组成虚拟参考帧，不需要用和编码器相同的算法基于图像帧建立虚拟参考帧，减少了解码器的复杂度和功耗。
附图说明
图1是本申请实施例提供的一种虚拟参考帧的建立方法流程示意图；
图2a是本申请实施例提供的一种位置映射关系示意图；
图2b是本申请实施例提供的另一种位置映射关系示意图；
图3a为本申请实施例提供的一种图像帧的结构示意图；
图3b为本申请实施例提供的一种预设格式的字段结构示意图；
图4为本申请实施例提供的一种虚拟参考帧建立效果示意图；
图5为本申请实施例提供的一种位置相同的块效果示意图；
图6为本申请实施例提供的一种参考帧缓存队列示意图；
图7是本申请另一实施例提供的一种虚拟参考帧的建立方法流程示意图；
图8是本申请又一实施例提供的一种虚拟参考帧的建立方法流程示意图；
图9是本申请实施例提供的一种解码器结构示意图；
图10是本申请实施例提供的一种编码器结构示意图；
图11是本申请实施例提供的另一种解码器结构示意图；
图12是本申请实施例提供的另一种编码器结构示意图。
具体实施方式
本申请实施例提供一种虚拟参考帧的建立方法，应用于解码器，如图1所示，该方法包括以下步骤：
101、接收编码器传输的至少一个图像帧。
至少一个图像帧中每一个图像帧可以包含一个块或多个块。需要说明的是，一个块包含多个像素，例如，一个块可以包含16×16个像素，或者8×8个像素，当然也可以是其他个数，本申请对此不作限制。此处，至少一图像帧包含的是视频的数据内容。
102、利用至少一个图像帧中带有虚拟参考帧标识的块建立虚拟参考帧。
至少一个图像帧中可以是每一个图像帧都包含带有虚拟参考帧标识的块，也可以是部分图像帧包含带有虚拟参考帧标识的块。一个图像帧可以包括一个或多个带有虚拟参考帧标识的块。
可选的，步骤102之前，该方法还包括：分配第一虚拟参考帧缓存区。
其中，该第一虚拟参考帧缓存区用于存储带有虚拟参考帧标识的块，第一虚拟参考帧缓存区的大小可以是一个图像帧的大小，当第一虚拟参考帧缓存区存入的带有虚拟参考帧标识的块占满整个第一虚拟参考帧缓存区时，虚拟参考帧建立完成，可以在第一虚拟参考帧缓存区中将虚拟参考帧提取出来；或者，当第一虚拟参考帧缓存区存入的带有虚拟参考帧标识的块的数量超过预设数量时，虚拟参考帧建立完成。需要说明的是，分配好第一虚拟参考帧缓存区的同时，可以存储一个初始图像，初始图像可以是纯色图像，例如全白图像或全黑图像，也可以只是分配第一虚拟参考帧缓存区，不存入图像。
利用至少一个图像帧中多个带有虚拟参考帧标识的块建立虚拟参考帧，包括：
根据预设的位置映射关系对每一个带有虚拟参考帧标识的块在其所在帧中的位置进行映射以获取每一个带有虚拟参考帧标识的块映射后的位置；
按照每一个带有虚拟参考帧标识的块映射后的位置将每一个带有虚拟参考帧标识的块存储在该第一虚拟参考帧缓存区以形成虚拟参考帧。需要说明的是，当两个带有虚拟参考帧标识的块的位置相同时，可以用在后接收到的带有虚拟参考帧标识的块替换在前接收到的带有虚拟参考帧标识的块；或者，将位置相同的两个带有虚拟参考帧标识的块中每一个像素的像素分量计算加权平均值形成新的块存储在第一虚拟参考帧缓存区中。
其中，预设的位置映射关系可以是位置等同的映射关系，即在映射之前和映射之后，带有虚拟参考帧标识的块在帧中的位置相同，比如，在图2a中，块a为带有虚拟参考帧标识的块，第一个图像帧为块a所在的帧，块a在第一个图像帧中的位置是第3行第4列，则块a在建立后的虚拟参考帧中的位置也是第3行第4列；或者预设的位置关系是位置不同的映射关系，即在映射之前和映射之后，带有虚拟参考帧标识的块在帧中的位置发生了变化，比如，在图2b中，块a为带有虚拟参考帧标识的块，第一个图像帧为块a所在的帧，块a在第一个图像帧中的位置是第3行第4列，则块a在建立后的虚拟参考帧中的位置也是第4行第5列。
需要说明的是，该预设的位置映射关系可以预先存储在解码器，或者，接收编码器发送的指示信息，指示信息用于指示预设的位置映射关系。
或者，所述虚拟参考帧标识包括用于指示所在块的位置信息，则利用至少一个图像帧中多个带有虚拟参考帧标识的块建立虚拟参考帧，包括：
按照每一个带有虚拟参考帧标识的块中虚拟参考帧标识所包含的位置信息将带有虚拟 参考帧标识的块存储在该第一虚拟参考帧缓存区以形成虚拟参考帧。
虚拟参考帧标识用于指示所在的块用于建立虚拟参考帧。需要说明的是，虚拟参考帧标识可以有多种存在方式，此处提供两种具体的方式：
第一种方式、在带有虚拟参考帧标识的块中，虚拟参考帧标识承载在新增的预设数量的比特上。即利用额外的比特表示虚拟参考帧标识。
第二种方式、虚拟参考帧标识承载在预设格式的字段上，如图3a所示，图3a为一个图像帧的结构示意图，一个图像帧包括网络抽象层单位头(network abstraction layer unit header)、帧头(slice segment header)和帧数据(slice segment data)。帧数据包括编码树单元(coding tree unit)，编码树单元包括编码四分树(coding quadtree)，编码四分树包括编码单元(Coding unit)等等，如此一级一级划分。预设格式的字段可以包括：运动矢量差(motion vector difference)字段、编码单元量化系数变化绝对值(coding unit quantization parameter delta absolute，cu_qp_delta_abs)字段、分割模式(part_mode)字段等字段中的一项或多项。预设格式的字段可以用于指示预设量化步长、预设块大小、预设编码方式、预设运动矢量、预设残差、预设量化参数中的一项或多项。此处，预设格式的字段可以是一个字段直接指示一个参数，也可以是多个字段组合起来指示一个参数，例如，如图3b所示，图3b为预设格式的字段结构示意图，预设格式的字段指示预设量化系数时，预设格式的字段可以包括初始量化系数减26的值(Intitial QP minus 26，init_qp_minus26)字段、编码单元量化系数变化使能信号(Coding unit QP delta enabled flag，cu_qp_delta_enabled_flag)字段、编码单元量化系数变化深度(Different coding unit QP delta depth，diff_cu_qp_delta_depth)字段、编码单元量化系数变化绝对值字段和编码单元量化系数变化正负号(Coding unit QP delta sign flag，cu_qp_delta_sign_flag)字段，其中QP为量化系数(Quantization Parameter，QP)，当预设格式的字段所指示的量化系数是预设量化系数时，表示该预设格式的字段所在的块为用于建立虚拟参考帧的块。
进一步，解码器可以接收编码器发送的解码指示信号，该解码指示信号用于指示第一虚拟参考帧缓存区中存储的带有虚拟参考帧标识的块可以形成虚拟参考帧，解码器在第一虚拟参考帧缓存区中提取出虚拟参考帧加入到第一参考帧缓存队列中，参照虚拟参考帧对随后接收到的图像帧进行解码，第一参考帧缓存队列包含用于解码的参考帧。解码过程中是根据每一个图像帧中包含的信息确定使用哪几个参考帧进行解码，这是在编码器编码时确定的，并将解码使用的参考帧序号写入到该图像帧中，解码器根据每个图像帧中包含的信息选择相应的参考帧解码即可。
示例性的，参照图4、图5和图6所示，本实施例举出具体示例对建立虚拟参考帧的过程进行详细说明：
如图4所示，解码器在接收到解码指示信号之前，将带有虚拟参考帧标识的块存入第一虚拟参考帧缓存区中，接收一个图像帧，将该图像帧中带有虚拟参考帧标识的块存入到第一虚拟参考帧缓存区中，然后接收下一个图像帧，这样一个图像帧一个图像帧进行处理，将图像帧中带有虚拟参考帧标识的块存入到第一虚拟参考帧缓存区就是建立虚拟参考帧的过程；此时，第一虚拟参考帧缓存区中缓存的带有虚拟参考帧标识的块所形成的帧还不能用于解码。在接收到指示信号之前，解码器将每一个图像帧中带有虚拟参考帧标识的块根据各自的位置存储在第一虚拟参考帧缓存区中，当两个带有虚拟参考帧标识的块的位置相 同时，可以用在后接收到的带有虚拟参考帧标识的块替换在前接收到的带有虚拟参考帧标识的块；或者，将位置相同的两个带有虚拟参考帧标识的块中每一个像素的像素分量计算加权平均值形成新的块存储在第一虚拟参考帧缓存区中。
如图5所示，例如，解码器接收到第一个图像帧中有5个带有虚拟参考帧标识的块，分别为块a、块b、块c、块d、块e，按照5个块的位置分别存储在第一虚拟参考帧缓存区，解码器接收到第二个图像帧中有3个带有虚拟参考帧标识的块，分别为块f、块g和块h，按照3个块的位置分别存储在虚拟参考帧缓存中，但是，第一个图像帧中的块a和第二个图像帧中的块f位置相同，可以用块f替换块a，也可以计算块f和块a中每个像素的分量的加权平均值形成新的块i，并将块i存储在块f和块a原本的位置上。
示例地，以块a和块f分别包含4个像素为例，块a包含像素a1、像素a2、像素a3和像素a4，块f包含像素f1、像素f2、像素f3和像素f4，像素a1和像素f1位置对应、像素a2和像素f2位置对应、像素a3和像素f3位置对应、像素a4和像素f4位置对应。如果每个像素有三个分量Y、U、V，对于像素a1和像素f1其中的一个分量，以分量Y为例，可以按照第一公式Pi1＝Pa1×W1+Pf1×W2+c计算块i的像素i1的Y分量，其中，Pi1为像素i1的Y分量的值，Pa1为像素a1的Y分量的值，Pf1为像素f1的Y分量的值，W1和W2为加权系数，c为补偿系数；或者，按照第二公式Pi1＝Pa1×W0+Pf1×(1-W0)+c计算块i的像素i1的Y分量，其中，W0为加权系数。像素的每个分量都可以按照第一公式或第二公式进行计算，每一对对应的像素也可以按照预设公式计算出每个分量新的数值，这样，就可以得到新的像素i1、像素i2、像素i3和像素i4，就可以组成新的块i，将块i存储在块a原本的位置上。
如图6所示，解码指示信号指示第一虚拟参考帧缓存区中缓存的带有虚拟参考帧标识的块组成了可以用于解码的虚拟参考帧，解码器在接收到解码指示信号之后，将第一虚拟参考帧缓存区中的虚拟参考帧提取出来加入到第一参考帧缓存队列中，解码器参照虚拟参考帧对之后接收的图像帧进行解码。
本申请中需要传输的帧都是图像帧，将用于建立虚拟参考帧的块在图像帧中添加标识，没有另外单独传送虚拟参考帧，减少了数据传输量，节约了网络资源。另外，有些相关技术中虽然不传输虚拟参考帧，但是，解码器要在图像帧中根据与编码器相同的算法建立背景图像，选择建立虚拟参考帧的块，并建立虚拟参考帧，相比之下，本申请不需要解码器建立背景图像，选择用于建立虚拟参考帧的块，只需要根据虚拟参考帧标识直接将带有虚拟参考帧标识的块组成虚拟参考帧就好，降低了解码器的复杂度，节省了处理器资源。而且，因为解码器只需要根据虚拟参考帧标识就可以获取用于建立虚拟参考帧的块，如果编码器建立虚拟参考帧时所使用的算法有了变更，解码器不受影响，兼容性更好。
结合上述图1对应的实施例，本申请另一实施例提供一种虚拟参考帧的建立方法，应用于编码器，如图7所示，该方法包括以下步骤：
701、获取至少一个图像帧。
至少一个图像帧可以是一个图像帧或多个图像帧，一个图像帧可以包含一个块或多个块，一个块可以包含一个或多个像素。
702、在至少一个图像帧中确定用于建立虚拟参考帧的块。
此处，可选的，步骤702之前，本实施例提供的虚拟参考帧的建立方法还包括：
分配第二虚拟参考帧缓存区，第二虚拟参考帧缓存区可以缓存用于建立虚拟参考帧的块，第二虚拟参考帧缓存区的大小可以是一个图像帧的大小。需要说明的是，分配好第二虚拟参考帧缓存区的同时，可以存储一个初始图像，初始图像可以是纯色图像，例如全白图像或全黑图像，也可以只是分配第二虚拟参考帧缓存区，不存入图像。
可选的，将该至少一个图像帧中任意一个图像帧作为目标图像帧，将目标图像帧中的任意一个块作为目标块进行示例性说明，当该目标块与目标背景块中相似像素对的数量大于或等于第一阈值时，将该目标块确定为用于建立虚拟参考帧的块，第一阈值为预先设定的值，第一阈值可以通过实验获取，也可以参考相关技术中常用的值确定。其中，目标背景块是根据目标块在目标图像帧中的位置确定的目标块在背景图像中对应的块，背景图像可以是基于至少一个图像帧，按照预设的建模算法生成的，例如，一个视频中，固定的背景会在前景物体离开后重新出现，固定的背景不会变化，只会被前景物体遮挡，这个固定的背景就是背景图像。像素对是指位于目标块与目标背景块中位置相对应的两个像素，相似像素对中的的两个像素满足至少一个相同的像素分量之差的绝对值小于或等于第二阈值，第二阈值为预先设定的值，第二阈值可以通过实验获取，也可以参考相关技术中常用的值确定。
需要说明的是，目标背景块在背景图像中的位置与目标块在目标图像帧中的位置可以相同，例如，每一个图像帧包含的块可以是m行，n列，每一个块的位置可以用所处的行序号和列序号确定，目标块在目标图像帧中位于x行y列，目标背景块在背景图像中也位于x行y列；或者，目标背景块在背景图像中的位置与目标块在目标图像帧中的位置也可以不同，例如，目标块在目标图像帧中位于x行y列，目标背景块在背景图像中也位于x+1行y+1列，本申请对此不作限制。
可选的，一个像素可以包含三个像素分量，例如YUV像素分量，其中Y为亮度分量，U，V为色度分量。对于像素对中两个像素，当三个像素分量中，至少一个相同的像素分量之差的绝对值小于或等于第二阈值时，将该像素对确定为相似像素对，也可以是三个像素分量重每个相同的像素分量之差的绝对值均小于或等于第二阈值时，将该像素对确定为相似像素对，否则为不相似像素对。
可选的，步骤702之后，还包括：
根据预设的位置映射关系确定每一个用于建立虚拟参考帧的块的位置，并将每一个用于建立虚拟参考帧的块分别存储在第二虚拟参考帧缓存区中。
例如，每一个图像帧包含的块可以是m行，n列，以第一个图像帧中的块a为例，块a为用于建立虚拟参考帧的块，块a在第一个图像帧中位于第x行第y列，块a在第一个图像帧中的位置可以表示为(x，y)，预设的位置映射关系可以指示块a在第二虚拟参考帧缓存区中的存储位置为(x，y)，或者，预设的位置映射关系可以指示块a在第二虚拟参考帧缓存区中的存储位置为(x+dx，y+dy)，其中，dx和dy是位置偏移量，当然，此处只是举例说明，并不代表本申请局限于此。
可选的，编码器可以解码器发送指示信息，该指示信息用于指示预设的位置映射关系。
703、对至少一个图像帧中用于建立虚拟参考帧的块添加虚拟参考帧标识。
虚拟参考帧标识用于指示所在的块用于建立虚拟参考帧。
可选的，本实施例提供两种具体的添加虚拟参考帧标识的方式：
第一种方式、在带有虚拟参考帧标识的块中，虚拟参考帧标识承载在新增的预设数量的比特上。即利用额外的比特表示虚拟参考帧标识。
第二种方式、虚拟参考帧标识承载在预设格式的字段上。预设格式的字段可以包括：运动矢量差字段、编码单元量化系数变化绝对值字段、分割模式字段等字段中的一项或多项。预设格式的字段可以用于指示预设量化步长、预设块大小、预设编码方式、预设运动矢量、预设残差、预设量化参数中的一项或多项。
可选的，编码器根据预设的位置映射关系确定每一个用于建立虚拟参考帧的块的位置之后，可以将位置信息分别添加入虚拟参考帧标识中，这样，解码器就可以直接根据虚拟参考帧标识确定每一个用于建立虚拟参考帧的块的位置，不需要再做计算。
704、向解码器传输至少一个图像帧。
当编码器将一个图像帧中用于建立虚拟参考帧的块添加了虚拟参考帧标识后即可发送该图像帧，继续处理下一个图像帧。
步骤703之后，当至少一个图像帧中用于建立虚拟参考帧的块组合成虚拟参考帧时，可以参照虚拟参考帧进行编码；具体的，当第二虚拟参考帧缓存区全部填满或者存储的块的数量超过预设数量时，视为虚拟参考帧建立完成，可以将虚拟参考帧放入第二参考帧缓存队列中，参照虚拟参考帧对之后的图像帧进行编码，第二参考帧缓存队列包含用于编码的参考帧。需要说明的是，如果存入的块没有占满第一虚拟参考帧缓存区而需要提取虚拟参考帧时，可以将空白的部分填充纯色的块完成虚拟参考帧的建立，例如，填充纯白的块；或者，不需要填充空白的部分，直接将第一虚拟参考帧缓存区中的块提取出来组成虚拟参考帧。可选的，当虚拟参考帧建立之后，还可以向解码器发送解码指示信号，解码指示信号用于指示解码器参照虚拟参考帧进行解码。参照图1对应的实施例，解码器在接收到解码指示信号之前，将带有虚拟参考帧标识的块存入第一虚拟参考帧缓存区中，当接收到解码指示信号时，将第一虚拟参考帧缓存区中的虚拟参考帧提取出来放入第一参考帧缓存队列中，并参照虚拟参考帧对之后接收的图像帧进行解码。
可选的，步骤704之前，编码器可以将至少一个图像帧进行编码。
此处，向解码器传输的至少一个图像帧不包括虚拟参考帧，可选的，结合步骤703中有关解码指示信号的描述，解码指示信号在至少一个图像帧中最后一个用于建立虚拟参考帧的块之后发送至解码器，当然，此处只是举例说明，解码指示信号也可以直接指示虚拟参考帧建立完成时最后一个发送的图像帧的帧序号或块序号，本申请对此不作限制。
因为编码器对用于建立虚拟参考帧的块添加了虚拟参考帧标识，在传输过程中不需要传输虚拟参考帧，不会造成码率上升，而且，解码器只需要将带有虚拟参考帧标识的块提取出来组成虚拟参考帧，不需要用和编码器相同的算法基于图像帧建立背景图像，再选择用于建立虚拟参考帧的块，然后建立虚拟参考帧，减少了解码器的复杂度和功耗。另外，现有技术中，编码器和解码器利用相同的算法建立虚拟参考帧，当不同厂家生产的编码器使用的建立虚拟参考帧的算法不同时，解码器无法兼容不同的算法，或者，编码器建立虚拟参考帧的算法有了更新，则解码器也需要更新，而解码器往往位于用户使用的视频播放设备上，例如，电视机、电脑等，这些设备的更新周期一般较长，会给用户使用造成不便。而本申请提供的虚拟参考帧的建立方法，因为用于建立虚拟参考帧标识的块带有虚拟参考帧标识，解码器提取带有虚拟参考帧标识的块就可以建立虚拟参考帧，不需要像编码器一 样利用算法建立虚拟参考帧，因此，编码器侧建立虚拟参考帧的算法即便更新，也不影响解码器建立虚拟参考帧，同时，对于不同厂家生产的编码器，解码器有很好的兼容性，也不会造成用户使用不便。
本实施例提供的虚拟参考帧的建立方法，是上述图1对应的实施例中所描述的解码侧虚拟参考帧的建立方法对应的编码侧方法，图1对应的实施例中的解释说明与图7对应的实施例中的解释说明相互适用。
基于上述图1和图7对应的实施例中所描述的虚拟参考帧的建立方法，本申请又一实施例提供一种虚拟参考帧的建立方法，应用于编码器和解码器，参照图8所示，该方法包括以下步骤：
801、编码器获取N个图像帧。
N为大于1的正整数。
802、编码器在第n个图像帧中确定用于建立虚拟参考帧的块。
第n个图像帧为编码器获取的N个图像帧中的第n个图像帧，n为[1，N]内的整数。
803、编码器将第n个图像帧中用于建立虚拟参考帧的块存储在第二虚拟参考帧缓存区中，并对第n个图像帧中用于建立虚拟参考帧的块添加虚拟参考帧标识。
804、编码器向解码器传输第n个图像帧。
重复执行步骤802-步骤804。在步骤804之后还包括：
805、当第二虚拟参考帧缓存区中用于建立虚拟参考帧的块的数量超过预设数量时，编码器建立虚拟参考帧，并向解码器发送解码指示信号。
编码器利用第二虚拟参考帧缓存区中用于建立虚拟参考帧的块建立虚拟参考帧，编码器建立虚拟参考帧之后参照虚拟参考帧对之后待发送的图像帧进行编码。
806、解码器接收到编码器发送的指示信号后，利用接收到的图像帧中带有虚拟参考帧标识的块建立虚拟参考帧。
具体的，解码器可以将第一虚拟参考帧缓存区中存储的带有虚拟参考帧标识的块提取出来组成虚拟参考帧。
807、解码器将虚拟参考帧加入到第一参考帧缓存队列中，并参照虚拟参考帧进行解码。
本申请中需要传输的帧都是图像帧，没有传送虚拟参考帧，相关技术中不仅要传输图像帧，还要传输虚拟参考帧，相比之下，本申请的技术方案减少了数据传输量，节约了网络资源。另外，有些相关技术中不传输虚拟参考帧，但是，解码器要在图像帧中根据与编码器相同的算法建立背景图像，选择建立虚拟参考帧的块，并建立虚拟参考帧，相比之下，本申请不需要解码器建立背景图像，选择用于建立虚拟参考帧的块，只需要根据虚拟参考帧标识直接将带有虚拟参考帧标识的块组成虚拟参考帧就好，降低了解码器的复杂度，节省了处理器资源。而且，因为解码器只需要根据虚拟参考帧标识就可以获取用于建立虚拟参考帧的块，如果编码器建立虚拟参考帧时所使用的算法有了变更，解码器不受影响，兼容性更好。
基于上述图1对应的实施例，本申请实施例提供一种解码器，用于执行上述图1对应的实施例中所描述的虚拟参考帧的建立方法，参照图9所示，该解码器90包括：处理单元901和接收单元902。
接收单元902，用于接收编码器传输的至少一个图像帧，至少一个图像帧中每一个图像 帧包含至少一个块；
处理单元901，用于利用至少一个图像帧中带有虚拟参考帧标识的块建立虚拟参考帧，虚拟参考帧标识用于指示所在的块用于建立虚拟参考帧。
在一种可能的实现方式中，处理单元901，还用于分配第一虚拟参考帧缓存区；确定每一个带有虚拟参考帧标识的块的位置，并将每一个带有虚拟参考帧标识的块分别存储在第一虚拟参考帧缓存区中。
在一种可能的实现方式中，处理单元901，还用于根据预设的位置映射关系确定每一个带有虚拟参考帧标识的块的位置。
在一种可能的实现方式中，虚拟参考帧标识还用于指示所在的块的位置信息，
处理单元901，还用于根据虚拟参考帧标识的指示确定每一个带有虚拟参考帧标识的块的位置。
在一种可能的实现方式中，接收单元902，接收编码器发送的指示信息，指示信息用于指示预设的位置映射关系。
在一种可能的实现方式中，虚拟参考帧标识承载在新增的预设数量的比特上。
在一种可能的实现方式中，虚拟参考帧标识承载在预设格式的字段上，预设格式的字段用于指示预设量化步长、预设块大小、预设编码方式、预设运动矢量、预设残差、预设量化参数中的至少一项。
在一种可能的实现方式中，接收单元902，还用于接收编码器发送的解码指示信号，解码指示信号用于指示参照虚拟参考帧进行解码；
处理单元901，还用于根据解码指示信号，参照虚拟参考帧进行解码。
本申请实施例提供的解码器，接收编码器传输的至少一个图像帧，利用至少一个图像帧中带有虚拟参考帧标识的块建立虚拟参考帧。因为用于建立虚拟参考帧的块添加了虚拟参考帧标识，在传输过程中不需要传输虚拟参考帧，不会造成码率上升，而且，解码器只需要将带有虚拟参考帧标识的块提取出来组成虚拟参考帧，不需要用和编码器相同的算法基于图像帧建立虚拟参考帧，减少了解码器的复杂度和功耗。
基于上述图7对应的实施例，本申请实施例提供一种编码器，用于执行上述图8对应的实施例中所描述的虚拟参考帧的建立方法，参照图10所示，该编码器100包括：处理单元1001和发送单元1002。
处理单元1001，用于获取至少一个图像帧，在至少一个图像帧中确定用于建立虚拟参考帧的块，对至少一个图像帧中用于建立虚拟参考帧的块添加虚拟参考帧标识，至少一个图像帧中每一个图像帧包含至少一个块，至少一个图像帧不包含虚拟参考帧，虚拟参考帧标识用于指示所在的块用于建立虚拟参考帧；
发送单元1002，用于向解码器传输至少一个图像帧。
在一种可能的实现方式中，处理单元1001，还用于当目标块与目标背景块中相似像素对的数量大于或等于第一阈值时，将目标块确定为用于建立虚拟参考帧的块；
其中，目标块为目标图像帧中的任意一个块，目标背景块是根据目标块在目标图像帧中的位置确定的目标块在背景图像中对应的块，至少一个图像帧包含目标图像帧；相似像素对包括分别位于目标块与目标背景块中位置相对应的两个像素，且相似像素对包括的两个像素满足至少一个分量之差的绝对值小于或等于第二阈值。
在一种可能的实现方式中，处理单元1001，还用于基于至少一个图像帧，按照预设的建模算法生成背景图像。
在一种可能的实现方式中，处理单元1001，还用于分配第二虚拟参考帧缓存区；根据预设的位置映射关系确定每一个用于建立虚拟参考帧的块的位置，并将每一个用于建立虚拟参考帧的块分别存储在第二虚拟参考帧缓存区中。
在一种可能的实现方式中，虚拟参考帧标识还用于指示所在的块的位置信息。
在一种可能的实现方式中，发送单元1002，还用于向解码器发送指示信息，指示信息用于指示预设的位置映射关系。
在一种可能的实现方式中，处理单元1001，还用于对用于建立虚拟参考帧的块增加预设数量的比特，虚拟参考帧标识承载在新增的预设数量的比特上。
在一种可能的实现方式中，处理单元1001，还用于在用于建立虚拟参考帧的块中设定预设格式的字段，虚拟参考帧标识承载在预设格式的字段上，预设格式的字段用于指示预设量化步长、预设块大小、预设编码方式、预设运动矢量、预设残差、预设量化参数中的至少一项。
在一种可能的实现方式中，处理单元1001，还用于当至少一个图像帧中用于建立虚拟参考帧的块组合成虚拟参考帧时，参照虚拟参考帧进行编码；
发送单元1002，还用于向解码器发送解码指示信号，解码指示信号用于指示参照虚拟参考帧进行解码。
本申请实施例提供的编码器，获取至少一个图像帧，在至少一个图像帧中确定用于建立虚拟参考帧的块，对至少一个图像帧中用于建立虚拟参考帧的块添加虚拟参考帧标识，向解码器传输至少一个图像帧。因为用于建立虚拟参考帧的块添加了虚拟参考帧标识，在传输过程中不需要传输虚拟参考帧，不会造成码率上升，而且，解码器只需要将带有虚拟参考帧标识的块提取出来组成虚拟参考帧，不需要用和编码器相同的算法基于图像帧建立虚拟参考帧，减少了解码器的复杂度和功耗。
基于上述图1对应的实施例，本申请实施例提供另一种解码器，用于执行上述图1对应的实施例中所描述的虚拟参考帧的建立方法，参照图11所示，该解码器110包括：处理器1101、存储器1102和接收器1103，处理器1101和存储器1102相互连接，处理器1101和接收器1103相互连接。
其中，存储器1102用于存储数据和程序。
处理器1101用于调用存储器中存储的程序实现图1对应的实施例中所描述的虚拟参考帧的建立方法。处理器1101与图9对应的实施例中的处理单元301对应。
接收器1103与图9对应的实施例中的接收单元302对应。接收器1103可以是天线或者有线网络接口。
处理器1101和接收器1103实现的功能与图9对应的实施例中处理单元301和接收单元302的功能相同，此处不再赘述。
本申请实施例提供的解码器，接收编码器传输的至少一个图像帧，利用至少一个图像帧中带有虚拟参考帧标识的块建立虚拟参考帧。因为用于建立虚拟参考帧的块添加了虚拟参考帧标识，在传输过程中不需要传输虚拟参考帧，不会造成码率上升，而且，解码器只需要将带有虚拟参考帧标识的块提取出来组成虚拟参考帧，不需要用和编码器相同的算法 基于图像帧建立虚拟参考帧，减少了解码器的复杂度和功耗。
基于上述图7对应的实施例，本申请实施例提供另一种编码器，用于执行上述图8对应的实施例中所描述的虚拟参考帧的建立方法，参照图12所示，该编码器120包括：处理器1201、存储器1202和发送器1203，处理器1201和存储器1202相互连接，处理器1201和发送器1203相互连接。
其中，存储器1202用于存储数据和程序。
处理器1201用于调用存储器中存储的程序实现图7对应的实施例中所描述的虚拟参考帧的建立方法。处理器1201与图10对应的实施例中的处理单元401对应。
发送器1203与图10对应的实施例中的发送单元402对应。发送器1203可以是天线或者有线网络接口。
处理器1201和发送器1203实现的功能与图10对应的实施例中处理单元401和发送单元402的功能相同，此处不再赘述。
本申请实施例提供的编码器，获取至少一个图像帧，在至少一个图像帧中确定用于建立虚拟参考帧的块，对至少一个图像帧中用于建立虚拟参考帧的块添加虚拟参考帧标识，向解码器传输至少一个图像帧。因为对用于建立虚拟参考帧的块添加了虚拟参考帧标识，在传输过程中不需要传输虚拟参考帧，不会造成码率上升，而且，解码器只需要将带有虚拟参考帧标识的块提取出来组成虚拟参考帧，不需要用和编码器相同的算法基于图像帧建立虚拟参考帧，减少了解码器的复杂度和功耗。
本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可以通过硬件来完成，也可以通过程序来指令相关的硬件完成，所述的程序可以存储于一种计算机可读存储介质中，上述提到的存储介质可以是只读存储器，磁盘或光盘等。
以上所述仅为本申请的可选实施例，并不用以限制本申请，凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims (34)

  1. 一种虚拟参考帧的建立方法，其特征在于，应用于解码器，所述方法包括：
    接收编码器传输的至少一个图像帧，所述至少一个图像帧中每一个图像帧包含至少一个块；
    利用所述至少一个图像帧中带有虚拟参考帧标识的块建立虚拟参考帧，所述虚拟参考帧标识用于指示所在的块用于建立虚拟参考帧。
  2. 根据权利要求1所述的方法，其特征在于，利用所述至少一个图像帧中带有虚拟参考帧标识的块建立虚拟参考帧之前，所述方法还包括：
    分配第一虚拟参考帧缓存区；
    利用所述至少一个图像帧中带有虚拟参考帧标识的块建立虚拟参考帧，包括：
    确定每一个带有虚拟参考帧标识的块的位置，并将每一个带有虚拟参考帧标识的块分别存储在所述第一虚拟参考帧缓存区中。
  3. 根据权利要求2所述的方法，其特征在于，确定每一个带有虚拟参考帧标识的块的位置，包括：
    根据预设的位置映射关系确定每一个带有虚拟参考帧标识的块的位置。
  4. 根据权利要求2所述的方法，其特征在于，所述虚拟参考帧标识还用于指示所在的块的位置信息，确定每一个带有虚拟参考帧标识的块的位置，包括：
    根据所述虚拟参考帧标识的指示确定每一个带有虚拟参考帧标识的块的位置。
  5. 根据权利要求2所述的方法，其特征在于，所述方法还包括：
    接收所述编码器发送的指示信息，所述指示信息用于指示所述预设的位置映射关系。
  6. 根据权利要求1所述的方法，其特征在于，
    所述虚拟参考帧标识承载在新增的预设数量的比特上。
  7. 根据权利要求1所述的方法，其特征在于，
    所述虚拟参考帧标识承载在预设格式的字段上，所述预设格式的字段用于指示预设量化步长、预设块大小、预设编码方式、预设运动矢量、预设残差、预设量化参数中的至少一项。
  8. 根据权利要求1-7任一项所述的方法，其特征在于，利用所述至少一个图像帧中带有虚拟参考帧标识的块建立虚拟参考帧之后，所述方法还包括：
    接收所述编码器发送的解码指示信号，根据所述解码指示信号，参照所述虚拟参考帧进行解码，所述解码指示信号用于指示参照所述虚拟参考帧进行解码。
  9. 一种虚拟参考帧的建立方法，其特征在于，应用于编码器，所述方法包括：
    获取至少一个图像帧，所述至少一个图像帧中每一个图像帧包含至少一个块；
    在所述至少一个图像帧中确定用于建立虚拟参考帧的块，所述至少一个图像帧不包含所述虚拟参考帧；
    对所述至少一个图像帧中用于建立虚拟参考帧的块添加虚拟参考帧标识，所述虚拟参考帧标识用于指示所在的块用于建立虚拟参考帧；
    向解码器传输所述至少一个图像帧。
  10. 根据权利要求9所述的方法，其特征在于，在所述至少一个图像帧中确定用于建立 虚拟参考帧的块，包括：
    当目标块与目标背景块中相似像素对的数量大于或等于第一阈值时，将所述目标块确定为用于建立虚拟参考帧的块；
    其中，所述目标块为目标图像帧中的任意一个块，所述目标背景块是根据所述目标块在目标图像帧中的位置确定的所述目标块在背景图像中对应的块，所述至少一个图像帧包含所述目标图像帧；所述相似像素对包括分别位于所述目标块与所述目标背景块中位置相对应的两个像素，且所述相似像素对包括的两个像素满足至少一个分量之差的绝对值小于或等于第二阈值。
  11. 根据权利要求10所述的方法，其特征在于，在所述至少一个图像帧中确定用于建立虚拟参考帧的编码块之前，所述方法还包括：
    基于所述至少一个图像帧，按照预设的建模算法生成所述背景图像。
  12. 根据权利要求9所述的方法，其特征在于，在所述至少一个图像帧中确定用于建立虚拟参考帧的块之前，所述方法还包括：
    分配第二虚拟参考帧缓存区；
    在所述至少一个图像帧中确定用于建立虚拟参考帧的块之后，所述方法还包括：
    根据预设的位置映射关系确定每一个用于建立虚拟参考帧的块的位置，并将每一个用于建立虚拟参考帧的块分别存储在所述第二虚拟参考帧缓存区中。
  13. 根据权利要求12所述的方法，其特征在于，
    所述虚拟参考帧标识还用于指示所在的块的位置信息。
  14. 根据权利要求12所述的方法，其特征在于，所述方法还包括：
    向所述解码器发送指示信息，所述指示信息用于指示所述预设的位置映射关系。
  15. 根据权利要求9所述的方法，其特征在于，对所述至少一个图像帧中用于建立虚拟参考帧的块添加虚拟参考帧标识，包括：
    对所述用于建立虚拟参考帧的块增加预设数量的比特，所述虚拟参考帧标识承载在新增的所述预设数量的比特上。
  16. 根据权利要求9所述的方法，其特征在于，对所述至少一个图像帧中用于建立虚拟参考帧的块添加虚拟参考帧标识，包括：
    在所述用于建立虚拟参考帧的块中设定预设格式的字段，所述虚拟参考帧标识承载在预设格式的字段上，所述预设格式的字段用于指示预设量化步长、预设块大小、预设编码方式、预设运动矢量、预设残差、预设量化参数中的至少一项。
  17. 根据权利要求9-16任一项所述的方法，其特征在于，对所述至少一个图像帧中用于建立虚拟参考帧的块添加虚拟参考帧标识之后，所述方法还包括：
    当所述至少一个图像帧中用于建立虚拟参考帧的块组合成所述虚拟参考帧时，参照所述虚拟参考帧进行编码；
    向所述解码器发送解码指示信号，所述解码指示信号用于指示参照所述虚拟参考帧进行解码。
  18. 一种解码器，其特征在于，包括：
    接收单元，用于接收编码器传输的至少一个图像帧，所述至少一个图像帧中每一个图像 帧包含至少一个块；
    处理单元，用于利用所述至少一个图像帧中带有虚拟参考帧标识的块建立虚拟参考帧，所述虚拟参考帧标识用于指示所在的块用于建立虚拟参考帧。
  19. 根据权利要求18所述的设备，其特征在于，
    所述处理单元，还用于分配第一虚拟参考帧缓存区；确定每一个带有虚拟参考帧标识的块的位置，并将每一个带有虚拟参考帧标识的块分别存储在所述第一虚拟参考帧缓存区中。
  20. 根据权利要求19所述的设备，其特征在于，
    所述处理单元，还用于根据预设的位置映射关系确定每一个带有虚拟参考帧标识的块的位置。
  21. 根据权利要求19所述的设备，其特征在于，所述虚拟参考帧标识还用于指示所在的块的位置信息，
    所述处理单元，还用于根据所述虚拟参考帧标识的指示确定每一个带有虚拟参考帧标识的块的位置。
  22. 根据权利要求19所述的设备，其特征在于，
    所述接收单元，接收所述编码器发送的指示信息，所述指示信息用于指示所述预设的位置映射关系。
  23. 根据权利要求18所述的设备，其特征在于，
    所述虚拟参考帧标识承载在新增的预设数量的比特上。
  24. 根据权利要求18所述的设备，其特征在于，
    所述虚拟参考帧标识承载在预设格式的字段上，所述预设格式的字段用于指示预设量化步长、预设块大小、预设编码方式、预设运动矢量、预设残差、预设量化参数中的至少一项。
  25. 根据权利要求18-24任一项所述的设备，其特征在于，
    所述接收单元，还用于接收所述编码器发送的解码指示信号，所述解码指示信号用于指示参照所述虚拟参考帧进行解码；
    所述处理单元，还用于根据所述解码指示信号，参照所述虚拟参考帧进行解码。
  26. 一种编码器，其特征在于，包括：
    处理单元，用于获取至少一个图像帧，在所述至少一个图像帧中确定用于建立虚拟参考帧的块，对所述至少一个图像帧中用于建立虚拟参考帧的块添加虚拟参考帧标识，所述至少一个图像帧中每一个图像帧包含至少一个块，所述至少一个图像帧不包含所述虚拟参考帧，所述虚拟参考帧标识用于指示所在的块用于建立虚拟参考帧；
    发送单元，用于向解码器传输所述至少一个图像帧。
  27. 根据权利要求26所述的设备，其特征在于，
    所述处理单元，还用于当目标块与目标背景块中相似像素对的数量大于或等于第一阈值时，将所述目标块确定为用于建立虚拟参考帧的块；
    其中，所述目标块为目标图像帧中的任意一个块，所述目标背景块是根据所述目标块在目标图像帧中的位置确定的所述目标块在背景图像中对应的块，所述至少一个图像帧包含所述目标图像帧；所述相似像素对包括分别位于所述目标块与所述目标背景块中位置相对应的两个像素，且所述相似像素对包括的两个像素满足至少一个分量之差的绝对值小于或等于第 二阈值。
  28. 根据权利要求27所述的设备，其特征在于，
    所述处理单元，还用于基于所述至少一个图像帧，按照预设的建模算法生成所述背景图像。
  29. 根据权利要求26所述的设备，其特征在于，
    所述处理单元，还用于分配第二虚拟参考帧缓存区；根据预设的位置映射关系确定每一个用于建立虚拟参考帧的块的位置，并将每一个用于建立虚拟参考帧的块分别存储在所述第二虚拟参考帧缓存区中。
  30. 根据权利要求29所述的设备，其特征在于，
    所述虚拟参考帧标识还用于指示所在的块的位置信息。
  31. 根据权利要求29所述的设备，其特征在于，
    所述发送单元，还用于向所述解码器发送指示信息，所述指示信息用于指示所述预设的位置映射关系。
  32. 根据权利要求26所述的设备，其特征在于，
    所述处理单元，还用于对所述用于建立虚拟参考帧的块增加预设数量的比特，所述虚拟参考帧标识承载在新增的所述预设数量的比特上。
  33. 根据权利要求26所述的设备，其特征在于，
    所述处理单元，还用于在所述用于建立虚拟参考帧的块中设定预设格式的字段，所述虚拟参考帧标识承载在预设格式的字段上，所述预设格式的字段用于指示预设量化步长、预设块大小、预设编码方式、预设运动矢量、预设残差、预设量化参数中的至少一项。
  34. 根据权利要求26-33任一项所述的设备，其特征在于，
    所述处理单元，还用于当所述至少一个图像帧中用于建立虚拟参考帧的块组合成所述虚拟参考帧时，参照所述虚拟参考帧进行编码；
    所述发送单元，还用于向所述解码器发送解码指示信号，所述解码指示信号用于指示参照所述虚拟参考帧进行解码。
PCT/CN2017/086899 2016-11-18 2017-06-01 虚拟参考帧的建立方法及设备 WO2018090600A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201611035422.8 2016-11-18
CN201611035422.8A CN106791829B (zh) 2016-11-18 2016-11-18 虚拟参考帧的建立方法及设备

Publications (1)

Publication Number Publication Date
WO2018090600A1 true WO2018090600A1 (zh) 2018-05-24

Family

ID=58971089

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/086899 WO2018090600A1 (zh) 2016-11-18 2017-06-01 虚拟参考帧的建立方法及设备

Country Status (2)

Country Link
CN (1) CN106791829B (zh)
WO (1) WO2018090600A1 (zh)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106791829B (zh) * 2016-11-18 2020-01-21 华为技术有限公司 虚拟参考帧的建立方法及设备
CN110235446B (zh) * 2017-06-08 2024-04-12 深圳市大疆创新科技有限公司 视频编码方法、视频解码方法及相关装置
CN112218086A (zh) * 2019-07-11 2021-01-12 中兴通讯股份有限公司 编码、解码方法、传输方法、编码、解码装置及系统
CN110636334B (zh) * 2019-08-23 2022-12-09 西安万像电子科技有限公司 数据传输方法及系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101674483A (zh) * 2009-09-28 2010-03-17 武汉大学 一种基于动态纹理合成的虚拟参考图像选择方法
US20110310968A1 (en) * 2004-02-24 2011-12-22 Winger Lowell L Method and apparatus for determining a second picture for temporal direct-mode block prediction
CN103297778A (zh) * 2013-05-27 2013-09-11 华为技术有限公司 一种对图像进行编、解码的方法及设备
CN104702956A (zh) * 2015-03-24 2015-06-10 武汉大学 一种面向视频编码的背景建模方法
CN106791829A (zh) * 2016-11-18 2017-05-31 华为技术有限公司 虚拟参考帧的建立方法及设备

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101841717B (zh) * 2010-05-07 2012-08-08 华为技术有限公司 一种解码的实现方法、软件解码器和解码设备
US8824560B2 (en) * 2010-07-07 2014-09-02 Netzyn, Inc. Virtual frame buffer system and method
US20130329800A1 (en) * 2012-06-07 2013-12-12 Samsung Electronics Co., Ltd. Method of performing prediction for multiview video processing
CN102790895B (zh) * 2012-07-30 2015-03-25 武汉大学 一种基于最小二乘的多视点视频编码视点合成预测方法
CN103929641A (zh) * 2014-05-12 2014-07-16 北京工商大学 一种基于虚拟参考帧的帧内编码方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110310968A1 (en) * 2004-02-24 2011-12-22 Winger Lowell L Method and apparatus for determining a second picture for temporal direct-mode block prediction
CN101674483A (zh) * 2009-09-28 2010-03-17 武汉大学 一种基于动态纹理合成的虚拟参考图像选择方法
CN103297778A (zh) * 2013-05-27 2013-09-11 华为技术有限公司 一种对图像进行编、解码的方法及设备
CN104702956A (zh) * 2015-03-24 2015-06-10 武汉大学 一种面向视频编码的背景建模方法
CN106791829A (zh) * 2016-11-18 2017-05-31 华为技术有限公司 虚拟参考帧的建立方法及设备

Also Published As

Publication number Publication date
CN106791829A (zh) 2017-05-31
CN106791829B (zh) 2020-01-21

Similar Documents

Publication Publication Date Title
AU2020201708B2 (en) Techniques for encoding, decoding and representing high dynamic range images
WO2017190710A1 (en) Method and apparatus for mapping omnidirectional image to a layout output format
TWI606718B (zh) 規定視覺動態範圍編碼操作及參數
KR101727094B1 (ko) ３ｄ 비디오에서 추가 뷰들을 생성하기 위한 뷰포인트들을 선택하는 방법
JP4575447B2 (ja) パノラマ画像の動き推定及び補償のために仮想画像領域を生成する方法及び装置
WO2018090600A1 (zh) 虚拟参考帧的建立方法及设备
JP6245888B2 (ja) エンコーダおよび符号化方法
US9538239B2 (en) Decoder and method for decoding encoded input data containing a plurality of blocks or packets
RU2693185C2 (ru) Способ кодирования и способ декодирования цветового преобразования и соответствующие устройства
KR20230113665A (ko) 더 높은 계층과 더 낮은 계층 사이의 차이를 이용한비디오 압축
EP3777154A1 (en) Methods and apparatus for depth encoding and decoding
US8165391B2 (en) Image processing apparatus and image processing method to correct color misregistration near a boundary
KR20090052849A (ko) 비디오 색 인핸스먼트 데이터를 인코딩하기 위한 방법 및 장치, 그리고 비디오 색 인핸스먼트 데이터를 디코딩하기 위한 방법 및 장치
KR20220145405A (ko) 피쳐 양자화/역양자화를 수행하는 영상 부호화/복호화 방법, 장치 및 비트스트림을 저장하는 기록 매체
JP2014039219A (ja) 映像復号装置、映像送受信システム、映像復号方法及び映像送受信方法
WO2014166338A1 (en) Method and apparatus for prediction value derivation in intra coding
JP2024012332A (ja) マルチビュービデオのデコードの方法及び装置、並びに画像処理の方法及び装置
JP2022517118A (ja) 点群コーディングにおける効率的なパッチ回転
US20150264345A1 (en) Method for Coding Videos and Pictures Using Independent Uniform Prediction Mode
JP2017017684A (ja) 高ダイナミックレンジフレームおよびインポーズされる低ダイナミックレンジフレームの両方を符号化するための方法およびデバイス
JP2013504258A (ja) 無線ｈｄ１．１におけるグラフィック−モード圧縮のためのビットストリーム構文
TWI559748B (zh) 用於視訊編解碼器的延伸預測模式及效能之方法及系統
CN110087074B (zh) 图像处理装置和用于操作图像处理装置的方法
US20110317758A1 (en) Image processing apparatus and method of processing image and video
CN113228665A (zh) 用于处理配置数据的方法、设备、计算机程序和计算机可读介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17872032

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17872032

Country of ref document: EP

Kind code of ref document: A1