WO2023197932A1 - 视频帧错误隐藏方法、装置、电子设备及介质 - Google Patents

视频帧错误隐藏方法、装置、电子设备及介质 Download PDF

Info

Publication number
WO2023197932A1
WO2023197932A1 PCT/CN2023/086622 CN2023086622W WO2023197932A1 WO 2023197932 A1 WO2023197932 A1 WO 2023197932A1 CN 2023086622 W CN2023086622 W CN 2023086622W WO 2023197932 A1 WO2023197932 A1 WO 2023197932A1
Authority
WO
WIPO (PCT)
Prior art keywords
damaged
macroblock
pixel
video frame
electronic device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/086622
Other languages
English (en)
French (fr)
Inventor
张勇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Vivo Mobile Communication Co Ltd
Original Assignee
Vivo Mobile Communication Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Vivo Mobile Communication Co Ltd filed Critical Vivo Mobile Communication Co Ltd
Publication of WO2023197932A1 publication Critical patent/WO2023197932A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • H04N19/89—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving methods or arrangements for detection of transmission errors at the decoder
    • H04N19/895—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving methods or arrangements for detection of transmission errors at the decoder in combination with error concealment
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques

Definitions

  • the present application belongs to the field of communication technology, and specifically relates to a video frame error hiding method, device, electronic equipment and media.
  • the functions of electronic devices are becoming more and more abundant. For example, when the video frame in the video encoding stream is damaged, the electronic device can use the error concealment algorithm to detect the damaged macro blocks in the damaged video frame. Error concealment is performed to improve the display of decoded video frames.
  • the electronic device can use the bilinear interpolation BI (Bilinear Interpolation, BI) method to error-conceal the damaged macroblocks in the damaged video frame; if the damaged video frame For inter-frame coding, the electronic device can use the Boundary Matching Algorithm (BMA) algorithm to hide the damaged macroblocks in the damaged video frame.
  • BI Bilinear Interpolation
  • BMA Boundary Matching Algorithm
  • the purpose of the embodiments of the present application is to provide a video frame error concealment method, device, electronic equipment and medium, which can solve the problem in the related art that the error concealment effect of damaged macroblocks in damaged video frames is poor.
  • embodiments of the present application provide a video frame error concealment method.
  • the method includes: determining damaged video frames in the video encoding stream; when the damaged video frames are intra-coded, based on the damaged
  • the macroblock type includes: flat block, edge block and texture block; when the damaged video frame is inter-coded, according to the target
  • the macroblock state of a macroblock performs error concealment on damaged macroblocks; target macroblocks include any of the following: macroblocks adjacent to the damaged macroblock in the damaged video frame; adjacent macroblocks within the damaged video frame macroblocks in the reference frame of the corrupted video frame.
  • embodiments of the present application provide a video frame error concealment device, which includes a determination module and a processing module; a determination module for determining damaged video frames in a video encoding stream; and a processing module for When the damaged video frame is intra-coded, error concealment is performed on the damaged macroblock according to the macroblock type of the damaged macroblock in the damaged video frame.
  • the macroblock type includes: flat block, edge block and texture block; Alternatively, it is used to hide the error of the damaged macroblock according to the macroblock status of the target macroblock when the damaged video frame is inter-coded; the target macroblock includes any of the following: damaged in the damaged video frame The macroblocks adjacent to the macroblock; the macroblocks adjacent to the damaged macroblock in the damaged video frame, and the macroblocks in the reference frame of the damaged video frame.
  • inventions of the present application provide an electronic device.
  • the electronic device includes a processor and a memory.
  • the memory stores programs or instructions that can be run on the processor.
  • the programs or instructions are processed by the processor.
  • the processor is executed, the steps of the method described in the first aspect are implemented.
  • embodiments of the present application provide a readable storage medium.
  • Programs or instructions are stored on the readable storage medium.
  • the steps of the method described in the first aspect are implemented. .
  • inventions of the present application provide a chip.
  • the chip includes a processor and a communication interface.
  • the communication interface is coupled to the processor.
  • the processor is used to run programs or instructions to implement the first aspect. the method described.
  • embodiments of the present application provide a computer program product, the program product is stored in a storage medium, and the program product is executed by at least one processor to implement the method as described in the first aspect.
  • the damaged video frame in the video encoding stream can be determined; and the damaged video frame is intra-coded.
  • error concealment is performed on the damaged macroblock according to the macroblock type of the damaged macroblock in the damaged video frame.
  • the macroblock type includes: flat block, edge block and texture block; in the damaged video frame, it is inter-frame coding.
  • error concealment is performed on the damaged macroblock according to the macroblock status of the target macroblock; the target macroblock includes any of the following: macroblocks adjacent to the damaged macroblock in the damaged video frame; macroblocks in the damaged video frame Macroblocks adjacent to the damaged macroblock, and macroblocks in the reference frame of the damaged video frame.
  • the damaged macroblock can be error-hidden based on the macroblocks adjacent to the damaged macroblock in the damaged video frame, or the adjacent macroblocks and the macroblocks in the reference frame of the damaged video frame; therefore, compared with In the related art, a single error concealment algorithm is used to hide the errors of damaged macroblocks in intra-coded or inter-coded damaged video frames.
  • the video frame error concealment method provided by the embodiment of the present application can more accurately conceal the damaged video frames. Error concealment is performed on individual damaged macroblocks. This can improve error concealment of damaged macroblocks in damaged video frames.
  • Figure 1 is a schematic diagram of using the BI method to hide errors in damaged macroblocks in the related art
  • Figure 2 is a flow chart of a video frame error hiding method provided by an embodiment of the present application.
  • Figure 3 is a schematic diagram of a method for determining M pixel adjustment amounts corresponding to damaged pixels in the video frame error concealment method provided by the embodiment of the present application;
  • Figure 4 is a schematic diagram of at least one adjacent area of a damaged macroblock in the video frame error concealment method provided by the embodiment of the present application;
  • Figure 5 is a schematic diagram of a method for determining macroblocks adjacent to the boundary of a damaged macroblock in the video frame error concealment method provided by an embodiment of the present application;
  • Figure 6 is a schematic diagram of a video frame error hiding device provided by an embodiment of the present application.
  • Figure 7 is a schematic diagram of an electronic device provided by an embodiment of the present application.
  • Figure 8 is a hardware schematic diagram of an electronic device provided by an embodiment of the present application.
  • first, second, etc. in the description and claims of this application are used to distinguish similar objects and are not used to describe a specific order or sequence. It is to be understood that the figures so used are interchangeable under appropriate circumstances so that the embodiments of the present application can be practiced in orders other than those illustrated or described herein, and that "first,” “second,” etc. are distinguished Objects are usually of one type, and the number of objects is not limited. For example, the first object can be one or multiple.
  • “and/or” in the description and claims indicates at least one of the connected objects, and the character “/" generally indicates that the related objects are in an "or” relationship.
  • Intra-encoded frame i.e. I frame
  • I frame Also called intra frame, it belongs to intra-frame compression. The picture of the I frame will be completely preserved. When decoding the I frame, only the data of this frame can be completed.
  • Unidirectional predictive coding frame (i.e. P frame): also known as difference frame or inter-frame compression, the P frame after encoding represents the difference information between the current frame and the I frame or the P frame before the current frame; when decoding the P frame, It is necessary to superimpose the coded difference information defined by this frame with the P frame or I frame cache picture before the current frame to reconstruct the picture of the current frame.
  • B frame is a bidirectional difference frame.
  • the encoded B frame records the difference information between this frame (i.e. current frame) and the previous and subsequent frames; in other words, to decode the B frame, not only must the previous
  • the cached picture is also decoded, and the current frame image is reconstructed through the previous and next frames and the encoded data of this frame.
  • the unreliability of the communication channel will cause video data to be damaged or lost during transmission, and the high compression rate of video source file encoding makes the video decoder very sensitive to damaged or lost video data in the encoded video.
  • videos are usually encoded using predictive coding and variable length coding
  • the demand may be based on All video frames predicted by the motion vector of the decoded video frame of this video frame cannot be decoded correctly, resulting in the spread of bit errors in space and time, causing the decoded
  • the playback quality of the video is seriously degraded and even cannot be played, causing the decoded video source to be distorted.
  • the correlation of the video data in the spatial domain and the time domain can be used to correct the erroneous information in the video with the correct information that has been decoded in the video. To hide.
  • Error concealment methods in related technologies include: intra-frame error concealment and inter-frame error concealment.
  • Intra-frame error concealment that is, spatial error concealment
  • Intra-frame error concealment is a method of recovering the data lost in the intra-coded frame in the encoded video. It mainly uses the damaged macroblock to be adjacent to the video frame where the damaged macroblock is located and Based on the correlation of correctly received macroblocks, the BI method is used to estimate the pixel values of damaged pixels in damaged macroblocks, thereby hiding error information in the video.
  • Inter-frame error concealment is a method of recovering lost data in P frames/B frames in encoded videos. It mainly uses the temporal correlation between adjacent video frames of the video to restore (or estimate) the motion vector (Motion Vector, MV) of the damaged macroblock in the damaged video frame; and the estimation of the MV of the damaged macroblock mainly uses the MV of the correctly received macroblocks around the damaged macroblock as candidate MV , then, if the average motion amplitude of the damaged video frame is small, the MV of the macroblock in the previous video frame of the damaged video that has the same position as a damaged macroblock in the damaged video frame can be used , directly restore the MV of the damaged macroblock in the damaged video frame; if the average motion amplitude of the damaged video frame is large, the BMA algorithm or OBMA algorithm can be used to restore the MV of the damaged macroblock in the damaged video frame. In this way, error information in the video can be hidden.
  • MV Motion Vector
  • the electronic device can perform packet loss/bit error detection on the received video encoding stream. , if the packet loss/bit error occurs in the I frame, the electronic device performs error concealment on the damaged macroblock in the corresponding video frame through intra-frame error concealment.
  • AVC Advanced Video Coding
  • Electronic devices can take advantage of the smooth changes in the spatial domain of video images and use the BI method to perform weighted interpolation on the pixel values of damaged pixels in damaged macro blocks based on the boundary pixels of damaged macro blocks, so that the damaged macro blocks can be Blocks perform error concealment; if packet loss/error occurs in P frame or B frame, the electronic device can error conceal the damaged macroblock through inter-frame error concealment.
  • the electronic device can use the BMA algorithm to reconstruct the damaged macro block according to the error concealment.
  • the smoothness of the frame and the MV of adjacent macro blocks are used to determine the corresponding candidate motion compensation macro blocks, and the MV of the damaged macro block is restored based on the MV of the macro block with the smallest boundary distortion, so that the MV along the damaged macro block is restored.
  • the brightness changes at block boundaries are minimal, allowing error concealment of corrupted macroblocks.
  • the electronic device can then send the error-concealed code stream to the H.264/AVC reference software for decoding, so that the video can be
  • the BI method uses the four pixels closest to the damaged pixel in the correctly received adjacent macroblocks around the damaged macroblock to perform linear interpolation to obtain the estimated pixel value of the damaged pixel.
  • Figure 1 shows a schematic diagram of using the BI method to hide errors in damaged macroblocks. As shown in Figure 1, the macroblock indicated by the gray area in the figure is the damaged macroblock. For the damaged pixels in the damaged macroblock 10.
  • the electronic device can determine the pixels 11 and 13 located in the same column as the damaged pixel 10 from the boundary pixels of the correctly received or restored macroblock adjacent to the damaged macroblock, and the pixels 11 and 13 located in the same column as the damaged pixel 10, and the pixels 11 and 13 that are located in the same column as the damaged pixel 10, and the pixels 11 and 13 that are located in the same column as the damaged pixel 10, and the pixels 11 and 13 that are located in the same column as the damaged pixel 10 from the boundary pixels of the correctly received or restored macroblock that are adjacent to the damaged macroblock.
  • the damaged pixel 10 is located in the pixel 12 and pixel 14 in the same row, and the estimated brightness value Pi,j of the damaged pixel 10 is calculated through the following interpolation formula (1):
  • P1 is the brightness value of pixel 11
  • P2 is the brightness value of pixel 12
  • P3 is the brightness value of pixel 13
  • P4 is the brightness value of pixel 14
  • d1 is the distance between pixel 11 and pixel 10
  • d2 is the brightness value of pixel 12
  • d3 is the distance between pixel 13 and pixel 10
  • d4 is the distance between pixel 14 and pixel 10.
  • the electronic device can calculate the average motion vector V of all correctly received macroblocks based on the MV of all correctly received macroblocks in the damaged video frame through the following formula (2):
  • N is the number of correctly received macroblocks in the damaged video frame
  • Vx and Vy are the horizontal and vertical components of MV respectively.
  • V is less than a preset threshold T, it means that the overall motion of the damaged video frame is relatively small, and the electronic device can Through the temporal error concealment mode without motion compensation, all damaged macroblocks in the damaged video frame are replaced with macroblocks at the same position in the previous frame of the damaged video frame; if V is greater than or equal to the threshold T, then the electronic device can determine the candidate MV set including the MV of the macroblock adjacent to the damaged macroblock and the zero motion vector ZMV through the temporal error concealment mode of motion compensation, and select from the candidate MV set as follows (3) The MV with the smallest output value of the cost function DBMA shown is:
  • ⁇ T , ⁇ B , ⁇ L , and ⁇ R respectively indicate whether the macro blocks adjacent to the upper, lower, left and right around the damaged macro block are correctly received macro blocks. If they are correctly received macro blocks, their value is 1, otherwise their value is 1.
  • the value is 0; N is the number of correctly received macroblocks in the damaged video frame; f(,,t) and f(,,t-1) represent the damaged video frame and reference frame respectively; f(x,y, ,) represents the pixel value at the integer pixel position (x, y); mvx and mvy are the horizontal and vertical components of a certain MV respectively, and f(x+mvx+i,y+mvy,t-1) is the subject
  • the pixel value of the damaged video frame at (x, y) position is restored in the reference frame using (mvx, mvy); f (x, y-1, t) is the damaged video frame at (x, y) The pixel value above the vertically adjacent position.
  • the brightness value of each damaged pixel in the damaged macroblock can be updated to the value indicated by the MV.
  • the brightness values of pixels with the same position are the same position.
  • the damaged video frame in the video encoding stream can be determined; and when the damaged video frame is intra-coded, the damaged video frame can be determined according to the damaged video frame.
  • the macroblock type of the damaged macroblock in the frame which performs error concealment on the damaged macroblock.
  • the macroblock type includes: flat block, edge block and texture block; when the damaged video frame is inter-coded, according to the target macro
  • the macroblock state of the block performs error concealment on damaged macroblocks
  • target macroblocks include any of the following: macroblocks adjacent to the damaged macroblock in the damaged video frame; adjacent macroblocks in the damaged video frame Macroblocks, and macroblocks in the reference frame of the corrupted video frame.
  • the damaged macroblock can be error-hidden based on the macroblocks adjacent to the damaged macroblock in the damaged video frame, or the adjacent macroblocks and the macroblocks in the reference frame of the damaged video frame; therefore, compared with In the related art, a single error concealment algorithm is used to hide the errors of damaged macroblocks in intra-coded or inter-coded damaged video frames.
  • the video frame error concealment method provided by the embodiment of the present application can more accurately conceal the damaged video frames. Error concealment is performed on individual damaged macroblocks. This can improve error concealment of damaged macroblocks in damaged video frames.
  • FIG. 2 shows a flow chart of the video frame error hiding method provided by an embodiment of the present application.
  • the video frame error hiding method provided by the embodiment of the present application may include the following steps 201 to 203, or may include the following steps 201, 202 and 204.
  • the following takes an electronic device as an example to illustrate the method.
  • Step 201 The electronic device determines the damaged video frame in the video encoding stream.
  • the electronic device can perform packet loss detection or error detection on the received video encoding stream, and determine video frames with packet loss or error as damaged video frames.
  • the electronic device can perform packet loss detection or bit error detection during the process of receiving the video encoding stream. Detection; alternatively, the electronic device can perform packet loss detection or error detection after receiving the video encoding stream.
  • Step 202 The electronic device determines the encoding type of the damaged video frame.
  • the electronic device can determine the encoding type of the damaged video frame. If the damaged video frame is intra-frame coded, the electronic device can continue to perform the following step 203 (a possible implementation); if the damaged video frame is inter-frame coded, the electronic device can continue to perform the following steps 204 (another possible implementation).
  • Step 203 When the damaged video frame is intra-coded, the electronic device performs error concealment on the damaged macroblock according to the macroblock type of the damaged macroblock in the damaged video frame.
  • macro block types may include: (a) flat blocks, (b) edge blocks, and (c) texture blocks.
  • the macro block type of the damaged macro block is a flat block, it means that the damaged macro block image is relatively flat and has no complex texture characteristics; if the macro block type of the damaged macro block is an edge block, it means that the damaged macro block image The edge information is relatively simple; if the macroblock type of the damaged macroblock is a texture block, it means that the edge information of the damaged macroblock image is relatively complex.
  • the intra-coded damaged video frame may be an I frame.
  • the above step 203 when the macro block type is the above-mentioned (a) flat block, the above step 203 can be specifically implemented through the following step 203a; when the macro block type is the above-mentioned (b) edge block In this case, the above step 203 can be specifically implemented through the following step 203b.
  • Step 203a When the damaged video frame is intra-coded and the macroblock type of the damaged macroblock in the damaged video frame is a flat block, the electronic device uses the BI method to hide the error of the damaged macroblock.
  • the BI method since the computational complexity of the BI method is low, the BI method is used for error concealment on flat blocks, which can achieve a better recovery effect for damaged macroblocks.
  • Step 203b When the damaged video frame is intra-frame coded and the macroblock type of the damaged macroblock in the damaged video frame is an edge block, the electronic device uses a one-way interpolation method to hide the error of the damaged macroblock. .
  • the interpolation direction may be the prediction direction corresponding to the maximum cumulative gradient intensity among the cumulative gradient intensities corresponding to all prediction directions.
  • the electronic device can use different error concealment methods (ie, BI method or one-way interpolation method) to hide the error of the damaged macroblock according to different macroblock types of the damaged macroblock. Therefore, the electronic equipment can be improved. Device flexibility for error concealment of corrupted macroblocks.
  • error concealment methods ie, BI method or one-way interpolation method
  • the above-mentioned step 203 can be specifically implemented through the following steps 203c and 203d.
  • the electronic device completes error concealment of the damaged macroblock after performing the following steps respectively for each damaged pixel in the damaged macroblock whose macroblock type is the texture block.
  • Step 203c When the damaged video frame is intra-coded and the macroblock type of the damaged macroblock in the damaged video frame is a texture block, the electronic device determines The position information of the pixel determines the M pixel adjustment amounts corresponding to the damaged pixel.
  • each prediction direction corresponds to a pixel adjustment amount
  • M is an integer greater than 2.
  • the M prediction directions may be evenly distributed, that is, the angles between every two adjacent prediction directions are the same.
  • the above M prediction directions can be 8 directions evenly distributed in the range of 0- ⁇ , which are: 0, ⁇ /8, ⁇ /4, 3 ⁇ /8, ⁇ /2, 5 ⁇ /8, 3 ⁇ /4, 7 ⁇ /8.
  • the embodiments of the present application do not limit the setting method of the M prediction directions.
  • the electronic device can set the M prediction directions as 6 directions evenly distributed in the range of 0- ⁇ , etc.
  • the electronic device determines a pixel adjustment amount (hereinafter referred to as the target pixel adjustment amount) corresponding to a damaged pixel (hereinafter referred to as the target damaged pixel) in the damaged macroblock as an example.
  • the electronic device determines The specific method of adjusting the M pixels corresponding to the target damaged pixel is exemplarily explained.
  • the electronic device can be affected by the target according to the prediction direction and the damaged macroblock 30 .
  • the position information of the damaged pixel 31 is determined, the intersection point , the pixel 32 closest to the intersection point
  • the distance between pixels 31 is d2
  • the electronic device can use the one-way interpolation method to calculate the target pixel adjustment amount Pi,j corresponding to the target damaged pixel 31 through the following formula (4):
  • the electronic device can determine a pixel adjustment amount of the target damaged pixel through the above method for each of the M prediction directions except the prediction direction indicated by arrow a->b, so that the target can be determined.
  • Step 203d The electronic device updates the pixel value of the damaged pixel based on the M pixel adjustment amounts.
  • the pixel value of the damaged pixel may be the brightness value of the damaged pixel.
  • the updated pixel value of the target damaged pixel is: the initial pixel value of the target damaged pixel + M pixel adjustments corresponding to the target damaged pixel value The sum of quantities.
  • the initial pixel value of the damaged pixel is the remaining pixel value after the damaged pixel is damaged.
  • the electronic device can use an iterative method to update the pixel value of the target damaged pixel based on the M pixel adjustment amounts.
  • the electronic device may first sum the M pixel adjustment amounts, and then add the initial pixel value of the target damaged pixel to the sum of the M pixel adjustment amounts.
  • the electronic device can update the pixel values of all damaged pixels in a damaged macroblock according to the above method, and then the error concealment of the damaged macroblock can be completed.
  • the electronic device can perform the adjustment on the damaged pixel in each prediction direction based on the pixel adjustment amount corresponding to each prediction direction. Pixel value updates, thereby improving the accuracy of electronic devices in updating pixel values of damaged pixels.
  • the video frame error hiding method provided by the embodiment of the present application may also include the following step 203e; then, the above-mentioned step 203d may be specifically implemented through the following step 203d1. .
  • Step 203e The electronic device determines the pixel adjustment weight corresponding to each prediction direction based on each prediction direction.
  • the electronic device can be based on the cumulative gradient intensity corresponding to each prediction direction (method one), or it can be based on each prediction direction and the position information of the damaged pixel (method two), or it can be based on the position information of each prediction direction and the damaged pixel (method two).
  • method one the cumulative gradient intensity corresponding to each prediction direction
  • method two the position information of the damaged pixel
  • method two the position information of each prediction direction and the damaged pixel
  • step 203e may be specifically implemented through the following step 203e1.
  • Step 203e1 The electronic device determines the pixel adjustment weight corresponding to each prediction direction based on the accumulated gradient intensity corresponding to each prediction direction.
  • the electronic device may normalize the accumulated gradient intensity corresponding to each prediction direction to determine the pixel adjustment weight corresponding to each prediction direction. Specifically, assuming that the prediction directions set by the electronic device are 8 directions uniformly distributed in the range of 0- ⁇ , the electronic device can normalize the cumulative gradient intensity corresponding to each prediction direction through the following formula (5) :
  • ⁇ k is one of the 8 prediction directions
  • W k is the pixel adjustment weight corresponding to the prediction direction ⁇ k
  • Stren( ⁇ k ) is the cumulative gradient intensity corresponding to the prediction direction ⁇ k
  • It is the sum of the cumulative gradient strengths corresponding to the 8 prediction directions.
  • the electronic device can determine the pixel adjustment weight corresponding to each prediction direction, and the pixel adjustment weights of different damaged pixels in the damaged macroblock in the same prediction direction are the same.
  • W k can be used to weight the interpolation pixels in each prediction direction, and W k represents the contribution of the prediction value in each prediction direction to the final interpolation pixel.
  • the video frame error hiding method provided by the embodiment of the present application may also include the following steps A to C.
  • Step A The electronic device detects the direction angle and gradient intensity of pixels in at least one adjacent area of the damaged macroblock.
  • the adjacent area of a damaged macroblock is an area adjacent to the boundary of the damaged macroblock (excluding corner adjacent areas).
  • the at least one adjacent area and the area where the damaged macroblock is located are different areas in the same damaged video frame.
  • each adjacent area in the at least one adjacent area may be any of the following: an area adjacent to the upper boundary of the damaged macroblock in the damaged video frame, the damaged The area in the video frame adjacent to the lower boundary of the damaged macroblock, the area in the damaged video frame adjacent to the left boundary of the damaged macroblock, the area in the damaged video frame adjacent to the right boundary of the damaged macroblock adjacent area.
  • the number of adjacent areas in the above-mentioned at least one adjacent area may be 2, 3 or 4, which may be specifically determined according to the position of the damaged macroblock in the damaged video frame.
  • each adjacent area in the at least one adjacent area may be: at least a partial area in a macro block adjacent to the boundary of the damaged macro block, and the at least partial area adjacent to the damaged macroblock.
  • each adjacent area when there are multiple adjacent areas in at least one adjacent area, the size of each adjacent area is the same, that is, the number of pixels in each adjacent area is the same.
  • area 41, area 42, area 43, and area 44 are adjacent areas (ie, at least one adjacent area) of the damaged macroblock, and area 41, area 42, area 43, and area 44.
  • the area where the damaged macroblock is located is a different area in the same damaged video frame 40; and the damaged macroblock includes 16 ⁇ 16 pixels, and the pixels in the area 41 are adjacent to the upper boundary of the damaged macroblock.
  • the pixels in area 42 are the 8 ⁇ 16 pixels adjacent to the lower boundary of the damaged macroblock, and the pixels in area 43 are the 16 ⁇ 16 pixels adjacent to the left boundary of the damaged macroblock. 8 pixels, the pixels in area 44 are 16 ⁇ 8 pixels adjacent to the right boundary of the damaged macroblock.
  • the electronic device can detect the direction angle and gradient intensity of the pixels in at least one adjacent area of the damaged macroblock by calculating the edge vector of the pixels in the at least one adjacent area.
  • the first pixel a pixel in the at least one adjacent area
  • the following is an example of a specific method for an electronic device to detect the direction angle of a pixel in at least one adjacent area of a damaged macroblock. illustrate.
  • the electronic device can define an edge vector of the first pixel for:
  • Fx i,j is The component in the X-axis direction
  • Fy i,j is Components in the Y-axis direction
  • electronic equipment can calculate Fx i,j and Fy i,j through Sobel operator or Prewitt operator, the convolution kernel formula of Sobel operator X-axis and Y-axis as follows:
  • the electronic device calculates the edge vector of the first pixel After Fx i,j and Fy i,j , it can be calculated by the following formula (11) direction angle
  • the electronic device calculates the edge vector of the first pixel through the above method Fx i,j and Fy i,j , then the electronic device can be calculated based on Fx i,j and Fy i,j by the following formula (13) Amplitude
  • the electronic device can detect the gradient intensity G (i, j) of the first pixel as:
  • Step B The electronic device determines the predicted direction corresponding to the direction angle of each pixel.
  • each predicted direction corresponds to a direction angle range.
  • the electronic device can preset the corresponding relationship between the predicted direction and the direction angle range. In this way, after detecting the direction angle of a pixel, the electronic device can determine the predicted direction corresponding to the direction angle of the pixel based on the direction angle range of the direction angle.
  • the electronic device after detecting the direction angle of the pixels in at least one adjacent area, can determine the predicted direction corresponding to the direction angle of each pixel based on the corresponding relationship between the predicted direction and the direction angle range. so ok Based on the direction angle of the pixel, pixels in at least one adjacent area are classified.
  • the electronic device can preset the corresponding relationship between the predicted direction and the direction angle range, as shown in Table 1 below;
  • the electronic device can, according to the corresponding relationship between the predicted direction and the direction angle range in Table 1, and the direction angle range to which the direction angle of each pixel belongs, respectively. Determine the predicted direction corresponding to the direction angle of each pixel.
  • the electronic device can set different corresponding relationships between the predicted direction and the direction angle range according to actual usage requirements.
  • Step C The electronic device adds the gradient intensities of pixels corresponding to the same prediction direction to obtain the cumulative gradient intensity corresponding to each prediction direction.
  • the electronic device may cumulatively add the gradient intensity of each pixel corresponding to each prediction direction to obtain the cumulative gradient intensity corresponding to the prediction direction.
  • G (i, j) is the gradient intensity of the first pixel.
  • the electronic device can obtain the accumulated gradient intensity corresponding to each prediction direction.
  • step 203e may be specifically implemented through the following step 203e2 and step 203e3.
  • Step 203e2 The electronic device determines the first boundary pixel and the second boundary pixel corresponding to each prediction direction based on each prediction direction and the position information of the damaged macroblock.
  • Step 203e3 The electronic device determines the pixel adjustment weight corresponding to each prediction direction based on the pixel value and position information of the first boundary pixel and the second boundary pixel corresponding to each prediction direction.
  • Step 203e3 will be exemplarily explained below with reference to the accompanying drawings, taking the electronic device to determine the pixel adjustment weight corresponding to the target prediction direction as an example.
  • the first boundary pixel determined by the electronic device based on the target prediction direction and the position information of the damaged pixel 31 is the pixel 32 and the second boundary pixel.
  • the pixel is pixel 33
  • the electronic device can set a pixel adjustment weight for the target prediction direction based on the pixel value of pixel 32 and the pixel value of pixel 33 through the following formula (16) (i.e., the similarity factor of edge pixels on both sides of the damaged macroblock):
  • exp(.) represents the exponential function with the natural number e as the base
  • P 1 is the pixel value of pixel 42
  • P 2 is the pixel value of pixel 43
  • ⁇ is a constant.
  • the electronic device can separately determine the pixel adjustment weight corresponding to each prediction direction, and the pixel adjustment weights of different damaged pixels corresponding to the same prediction direction are different.
  • the electronic device can determine the pixel adjustment weight corresponding to each prediction direction through the above method, so that the electronic device determines the damage based on the image edge information of the damaged macroblock in each prediction direction.
  • the degree of adjustment of each damaged pixel in the macroblock on the other hand, the electronic device can determine the pixel adjustment weight corresponding to each prediction direction through the above method 2, and a pixel adjustment weight can indicate the adjustment degree of the damaged macroblock in the corresponding prediction direction.
  • the similarity of the edge pixels on both sides of the image allows the electronic device to restore the damaged image more smoothly by adjusting the weight of the pixel. This increases the flexibility of the electronic device in determining pixel adjustment weights, thereby improving error concealment of damaged macroblocks.
  • Step 203d1 The electronic device updates the pixel value of the damaged pixel based on the M pixel adjustment amounts and M pixel adjustment weights.
  • the above-mentioned M pixel adjustment amounts correspond to the above-mentioned M pixel adjustment weights one-to-one.
  • the specific method for the electronic device to update the pixel value of the damaged pixel will be described in detail below.
  • the electronic device can first adjust the prediction direction based on the M pixel adjustment amounts and M pixel adjustment weights. Update the pixel value P of the target damaged pixel through the following formula (17):
  • W is the target damaged pixel in the prediction direction among the above M pixel adjustment weights. Adjust the weight of the corresponding pixel. It can be understood that W can be any of the following: W k ,
  • the electronic device can then target the next predicted direction Continue to update the pixel value of the target damaged pixel according to the above method.
  • Pi ,j in the above formula (17) is the predicted direction of the electronic device.
  • the pixel value after updating the pixel value of the target damaged pixel, P′ i,j is the predicted direction of the target damaged pixel among the above M pixel adjustment amounts.
  • the corresponding pixel adjustment amount, W is the target damaged pixel in the prediction direction among the above M pixel adjustment weights. Adjust the weight of the corresponding pixel. In this way, until the electronic device cumulatively updates the pixel value of the target damaged pixel for each prediction direction, the update of the pixel value of the target damaged pixel is completed.
  • the electronic device completes updating the pixel value of a damaged pixel in a damaged macroblock. Afterwards, the pixel value of the next damaged pixel in the damaged macroblock that is closest to the damaged pixel and located to the right or below the damaged pixel can be updated until all damaged pixels in the damaged macroblock are After the damaged pixels are updated, the error information of the damaged macroblock is hidden.
  • the electronic device has updated all damaged pixels in all damaged macroblocks through the above method, it can complete error hiding in the entire video, so as to improve the effect of the electronic device playing the video.
  • the electronic device can update the pixel value of the damaged pixel based on M pixel adjustment amounts and M pixel adjustment weights determined according to each prediction direction and corresponding to the M pixel adjustment amounts. , so the pixel value of the damaged pixel can be updated to different degrees in different prediction directions, which can further improve the accuracy of the electronic device in updating the pixel value of the damaged pixel.
  • Step 204 When the damaged video frame is inter-frame coded, the electronic device performs error concealment on the damaged macroblock according to the macroblock status of the target macroblock.
  • the target macroblock includes any of the following: macroblocks adjacent to the damaged macroblock in the damaged video frame; macroblocks adjacent to the damaged macroblock in the damaged video frame, and Macroblock in the reference frame of the corrupted video frame.
  • the macroblock in the reference frame of the damaged video frame may be: the macroblock in the reference frame of the damaged video frame has the same position as the damaged macroblock in the damaged video frame. piece.
  • the macroblock status may include: correctly received or damaged.
  • the damaged video frames of inter-frame coding may be P frames or B frames.
  • the electronic device hides errors in each damaged video frame in the video encoding stream, it can complete error hiding in the entire video encoding stream, thereby improving the video playback effect of the electronic device.
  • the following embodiments do not limit the number of damaged video frames in the video encoding code stream, nor limit the number of damaged macroblocks in each damaged video frame. That is to say, the number of damaged video frames in the video encoding code stream may be one or multiple; correspondingly, the number of damaged macroblocks in each damaged video frame may be one or multiple There are multiple; the details can be determined according to actual usage requirements.
  • step 204 can be specifically implemented through the following steps 204a and 204b.
  • Step 204a The electronic device determines the first candidate motion vector set of the damaged macroblock based on the first number or the second number.
  • the first number is the number of correctly received macroblocks adjacent to the damaged macroblock in the damaged video frame
  • the second number is the number of damaged macroblocks adjacent to the damaged macroblock in the damaged video frame. The number of macroblocks.
  • a macroblock that is adjacent to a damaged macroblock and is received correctly is a macroblock that is adjacent to the boundary of the damaged macroblock and has no missing or erroneous pixels.
  • the first candidate motion vector set when the first number is greater than or equal to the first preset number, or the second number is less than or equal to the second preset number, the first candidate motion vector set includes the second candidate motion Vector set.
  • the first candidate motion vector set when the first number is less than the first preset number, or the second number is greater than the second preset number, the first candidate motion vector set includes the second candidate motion vector set and the second candidate motion vector set. Set of three candidate motion vectors.
  • the first preset number may be: an integer greater than 0 and less than or equal to 4; for example, the first preset number is 2.
  • the second preset number may be: an integer greater than 0 and less than or equal to 4; the second preset number may be the same as or different from the first preset number.
  • the second candidate motion vector set includes: motion vectors of macroblocks adjacent to the damaged macroblock in the damaged video frame and correctly received, zero motion vectors, and motion vectors of the damaged macroblock in the damaged video frame.
  • the third candidate motion vector set includes: the motion vector of the macroblock with the same position as the damaged macroblock in the reference frame of the damaged video frame.
  • the above-mentioned median motion vector may be: the median value of all X-axis direction components of the motion vectors of macroblocks adjacent to the damaged macroblock in the damaged video frame and correctly received. , and the motion vector determined by the median value of all Y-axis direction components.
  • the reference frame of the damaged video frame may be any one of the L video frames in the video encoding stream that is continuous with the damaged video frame and located before the damaged video frame.
  • Frame, L is a positive integer less than or equal to 16.
  • the reference frame of a damaged video frame is the video frame preceding the damaged video frame, in other words, the reference frame is the last video frame received before the damaged video frame.
  • a macroblock whose position in the reference frame is the same as that of the damaged macroblock can be understood as: the position information of the macroblock in the reference frame is the same as the position information of the damaged macroblock in the damaged video frame.
  • the location information in is the same.
  • the motion vector of the correctly received macroblock adjacent to the damaged macroblock in the damaged video frame has the highest correlation with the motion vector of the damaged macroblock, and the selected motion vector included in the first candidate motion vector set The number is moderate, so that the complexity of the operation can be reduced while ensuring the accuracy of error concealment of damaged macroblocks.
  • each macroblock adjacent to the boundary of the damaged macroblock E in the damaged video can be divided into multiple sub-macroblocks. It can be seen that the electronic device will interact with the damaged macroblock E.
  • the top sub-macroblock A is determined as the macroblock adjacent to the left boundary of the damaged macroblock E;
  • the leftmost sub-macroblock B is determined to be the macroblock adjacent to the upper boundary of the damaged macroblock E; among the multiple sub-macroblocks adjacent to the lower boundary of the damaged macroblock E,
  • the leftmost sub-macroblock D is determined to be the macroblock adjacent to the lower boundary of the damaged macroblock E.
  • the electronic device can determine the uppermost sub-macroblock among them as being the same as the damaged macroblock E.
  • the electronic The device may determine the first candidate motion vector set to be the second candidate motion vector set; if the number of correctly received macroblocks among macroblock A, macroblock B, macroblock C and macroblock D is less than 2, the electronic device may determine the first candidate motion vector set.
  • the candidate motion vector set includes a second candidate motion vector set and a third candidate motion vector set.
  • the electronic device can determine different first candidate motion vector sets for the damaged macroblock according to the relationship between the first number or the second number and the corresponding preset number, therefore when the bit error rate is large (That is, when the number of correctly received macroblocks adjacent to the damaged macroblock is small), sufficient candidate motion vectors can be ensured, thereby improving the robustness of the electronic device in concealing errors of the damaged macroblock.
  • Step 204b The electronic device performs error concealment on the damaged macroblock based on the first candidate motion vector set.
  • each motion vector in the first candidate motion vector set indicates a macroblock in the reference video frame.
  • the electronic device may determine the reference macroblock with the same position information in the reference frame based on the position information of the zero motion vector in the first candidate motion vector set, and may determine the reference macroblock based on the position information of the zero motion vector in the first candidate motion vector set.
  • the offset of each other motion vector relative to the zero motion vector determines a macroblock corresponding to each other motion vector in the reference frame.
  • the electronic device when the damaged video frame is inter-frame coded, can calculate the damaged video frame based on the first candidate motion vector set corresponding to the damaged macroblocks determined by the first number or the second number. Macroblocks perform error concealment, so the accuracy of error concealment of damaged macroblocks by electronic devices can be further improved.
  • the above-mentioned step 204b may specifically include the following steps 204b1 to 204b3.
  • Step 204b1 The electronic device determines the candidate motion vector among the first candidate motion vectors that minimizes the loss cost function value as the target motion vector.
  • D BMA is the boundary matching cost function, that is, the above formula (3)
  • D OBMA is the overlapping boundary matching cost function
  • ⁇ 1 and ⁇ 2 are preset weights.
  • D OBMA can be expressed as:
  • the electronic device can substitute each candidate motion vector in the first candidate motion vector set into the above formula (18), and determine a D The target motion vector with the smallest tot .
  • Step 204b2 The electronic device determines the motion compensation block of the target motion vector in the reference frame of the damaged video frame.
  • the electronic device may determine the motion compensation block of the target motion vector in the reference frame of the damaged video frame based on the offset of the target motion vector relative to the above-mentioned zero motion vector.
  • Step 204b3 The electronic device performs error concealment on the damaged macroblock according to the motion compensation block.
  • the electronic device can replace the pixel value of each damaged pixel in the damaged macro block with: a pixel with the same position information as each damaged pixel in the above motion compensation block. pixel value.
  • error concealment of damaged macroblocks can be accomplished.
  • the electronic device can determine a motion compensation block in the reference frame through the loss cost function combining D BMA and D OBMA , and can hide the error of the damaged macroblock based on the motion compensation block, therefore It can solve the problem of mismatch errors when the electronic device restores the object boundary, and can make the pixels on both sides of the restored damaged macroblock smoother, thereby further improving the error hiding effect of the damaged macroblock by the electronic device.
  • the embodiments of the present application for intra-coded damaged video frames, different methods can be used to hide errors for damaged macroblocks with different macroblock types in the damaged video frame. ;
  • the damaged video frame can be modified based on the macroblocks adjacent to the damaged macroblock in the damaged video frame, or the adjacent macroblocks and the macroblocks in the reference frame of the damaged video frame. Therefore, compared with the solution in the related art that uses a single error concealment algorithm to hide the damaged macroblocks in the damaged video frames of intra-frame coding or inter-frame coding, the embodiments of the present application provide The video frame error concealment method can more accurately conceal errors for individual damaged macroblocks. This can improve error concealment of damaged macroblocks in damaged video frames.
  • the video frame error hiding method provided by the embodiment of the present application may also include the following step 205.
  • Step 205 The electronic device performs edge detection on the damaged macroblock, and determines the macroblock type of the damaged macroblock based on the edge information of the edge detection.
  • edge detection may include flatness detection; or edge detection may include flatness detection and texture complexity detection.
  • the edge information of the macro block can indicate the edge texture characteristics of the macro block image, so that the electronic device can determine the macro block type of the damaged macro block based on the edge information.
  • the electronic device can also use any other possible method to determine the macro block type of the damaged macro block, which can be determined according to actual usage requirements.
  • the electronic device can perform edge detection on the damaged macroblock, the flatness and texture complexity of the damaged macroblock image can be obtained, so that the electronic device determines the damaged macroblock based on the edge information of the edge detection.
  • the electronic device determines the damaged macroblock based on the edge information of the edge detection.
  • it can be based on the flatness and texture complexity of the damaged macroblock image, which can improve the accuracy of the electronic device in determining the macroblock type.
  • step 205 can be implemented through the following steps 205a to 204c, or can be implemented through the following steps 205a, 205b, 205d and 205e.
  • Step 205a The electronic device detects the gradient intensity of pixels in at least one adjacent area of the damaged macroblock.
  • step 205a For the specific description of step 205a, reference may be made to the relevant descriptions in the above embodiments, and to avoid repetition, they will not be described again here.
  • Step 205b The electronic device determines the flatness threshold of the damaged macroblock according to the gradient intensity.
  • the electronic device can determine the flatness of the damaged macroblock based on a gradient intensity threshold (hereinafter referred to as the first gradient intensity threshold). threshold.
  • Step 205c When the flatness of the damaged macro block is less than the flatness threshold, the electronic device determines the macro block type of the damaged macro block as a flat block.
  • the electronic device may determine the flatness of the damaged macroblock based on the maximum gradient intensity and/or the second largest gradient intensity among the gradient intensities of the pixels in the at least one adjacent area.
  • the electronic device can determine that the flatness of the damaged macroblock is less than the flatness threshold, That is, the damaged macroblock image is relatively flat and has no complex texture characteristics; if the maximum gradient intensity is greater than or equal to the first gradient intensity threshold, the electronic device can determine that the flatness of the damaged macroblock is greater than or equal to the flatness threshold, that is, it is damaged. There are rich texture areas in macroblock images. Therefore, the electronic device can determine the macroblock type of the damaged macroblock as a flat block when the flatness of the damaged macroblock is less than the flatness threshold.
  • the electronic device determines the macro block type of the damaged macro block based on the detected gradient intensity of the pixels in at least one adjacent area of the damaged macro block. Determining to be a flat block can improve the accuracy of the electronic device in determining the macroblock type of the damaged macroblock.
  • Step 205d When the flatness of the damaged macro block is greater than or equal to the flatness threshold, the electronic device detects the texture complexity of the damaged macro block.
  • step 205d may be specifically implemented through the following steps 205d1 and 205d2.
  • Step 205d1 When the flatness of the damaged macroblock is greater than or equal to the flatness threshold, the electronic device obtains the pixel gradient intensity of at least one pixel prediction direction.
  • each pixel prediction direction in the at least one pixel prediction direction corresponds to a pixel gradient intensity
  • the pixel gradient intensity is the sum of the gradient intensities of pixels within the angle range corresponding to the pixel prediction direction.
  • step 205d1 For other descriptions in step 205d1, specific reference may be made to the relevant descriptions of the prediction direction and gradient strength in the above embodiments. In order to avoid repetition, they will not be described again here.
  • Step 205d2 The electronic device determines the edge texture complexity of the damaged macroblock based on the cumulative gradient intensity and the maximum gradient intensity of at least one pixel gradient intensity.
  • the above-mentioned accumulated gradient intensity is the sum of the above-mentioned gradient intensities of at least one pixel.
  • the electronic device may determine the texture complexity of the damaged macroblock based on the ratio of the maximum gradient intensity and the accumulated gradient intensity.
  • the electronic device may determine that the texture complexity of the damaged macroblock is greater, that is, the damaged macroblock The image edge information is more complex, and there is more edge information in other prediction directions except the prediction direction corresponding to the maximum cumulative gradient intensity; if the above ratio is greater than or equal to the second gradient intensity threshold, the electronic device can determine the damaged macroblock The texture complexity is smaller, that is, the damaged macroblock image edge information is less.
  • a gradient intensity threshold hereinafter referred to as the second gradient intensity threshold
  • the electronic device can also determine the texture complexity of the damaged macroblock based on the ratio of the second largest gradient intensity and the accumulated gradient intensity among the at least one pixel gradient intensity mentioned above, which can be set according to actual usage requirements.
  • the embodiments of this application are not limited.
  • the image edge is one of the basic features of the image, it appears in the form of discontinuous local features, such as sudden changes in gray value, sudden changes in color, sudden changes in texture, etc., which are characterized by The gradient strength is greater in one direction; Macroblocks with richer detail components have larger gradient intensities in many directions or even all directions; therefore, the electronic device can determine the texture complexity of the damaged macroblock based on the pixel gradient intensity in at least one pixel prediction direction.
  • the electronic device can determine the texture complexity of the damaged macroblock based on the ratio of the maximum gradient intensity and the accumulated gradient intensity of at least one pixel gradient intensity, and this ratio can accurately reflect the image edge of the damaged macroblock. information, thereby improving the accuracy of electronic devices in determining the texture complexity of damaged macroblocks.
  • Step 205e The electronic device determines the macroblock type of the damaged macroblock based on texture complexity.
  • the electronic device can determine that the damaged macroblock
  • the macroblock type is an edge block; if the flatness of the damaged macroblock is less than the above-mentioned flatness threshold, and the texture complexity of the damaged macroblock is less than the texture complexity threshold, the electronic device can determine the macroblock of the damaged macroblock.
  • Type is texture block.
  • the texture complexity threshold may be determined by an electronic device based on the second gradient intensity threshold.
  • the electronic device can determine that the macro block type of the damaged macro block is a flat block through the above flatness detection, and can determine that the macro block type of the damaged macro block is an edge block or a texture block through the above texture complexity detection.
  • the electronic device can determine the macroblock type of the damaged macroblock to be an edge block or a texture block based on the detected texture complexity of the damaged macroblock, the electronic device can further improve the ability of the electronic device to determine the damaged macroblock. Macroblock type accuracy.
  • the electronic device encodes a Quarter Common Intermediate Format (QCIF) (176 ⁇ 144) video sequence, in which one frame of image contains 99 macroblocks of 16 ⁇ 16 size; and
  • QCIF Quarter Common Intermediate Format
  • B frame prediction is turned off, and only I frames and P frames are used for encoding, and I frames are inserted periodically, with one I frame inserted every 1 second of video.
  • the electronic device can use the error concealment module to hide errors in damaged macroblocks using the video frame error concealment method provided in the embodiments of the present application.
  • Step 1 The electronic device detects the flatness of the damaged macro block; if it is estimated that the damaged macro block image is relatively flat and has no complex texture characteristics, the electronic device determines that the type of the damaged macro block is a flat block, and uses the BI method to detect the damaged macro block. Macroblocks perform error concealment. Otherwise, the electronics proceeds with texture complexity detection.
  • Step 2 The electronic device detects the texture complexity of the damaged macro block; if the type of the damaged macro block is an edge block, the electronic device uses a one-way interpolation method to hide the error of the damaged macro block; if the type of the damaged macro block is If it is a texture block, the electronic device uses an improved multi-directional interpolation method (that is, a method of updating the pixel value of the damaged pixel based on M pixel adjustment amounts and M pixel adjustment weights) to hide the error of the damaged macro block.
  • an improved multi-directional interpolation method that is, a method of updating the pixel value of the damaged pixel based on M pixel adjustment amounts and M pixel adjustment weights
  • Step 3 Repeat steps 1 to 2.
  • the electronic device can hide errors in the damaged macroblocks in the order of damaged macroblocks on both sides first and then the damaged macroblocks in the middle, until all damaged macroblocks in the current I frame are processed.
  • Step 1 The electronic device determines the status of the four adjacent macroblocks above, below, left, and right of the current damaged macroblock. If at least 2 of the macroblocks are correctly received macroblocks, determine the first candidate motion.
  • the vector set is the second candidate motion vector set, otherwise, it is determined that the first candidate motion vector set includes the second candidate motion vector set and the third candidate motion vector set (ie, the extended candidate motion vector set).
  • Step 2 The electronic device selects the motion vector that minimizes the loss cost function D tot from the first candidate motion vector set as the best motion vector of the damaged macroblock.
  • Step 3 According to the best motion vector determined in step 2, determine the corresponding motion compensation macroblock in the reference frame of the damaged video frame, and insert it into the position of the current damaged macroblock (that is, replace the pixel value of the damaged pixel is the pixel value of the pixel with the same position information in the motion compensated macroblock).
  • Step 4 Repeat steps 1 to 3.
  • the electronic device can hide the errors of the damaged macroblocks in the order of damaged macroblocks on both sides first and then the damaged macroblocks in the middle, until all damaged macroblocks in the current P frame are processed. .
  • the video frame error concealment method provided by the embodiments of the present application fully utilizes the correlation of video signals in the spatial domain and time domain, and effectively improves the performance of the error concealment algorithm in traditional technologies, thereby enabling electronic devices to The error code stream can still be decoded and restored to clear and smooth video images.
  • the execution subject may be a video frame error concealment device.
  • a video frame error concealment device performing a video frame error concealment method is used as an example to illustrate the video frame error concealment device provided by the embodiment of the present application.
  • an embodiment of the present application provides a video frame error concealment device 60 , which may include a determination module 61 and a processing module 62 .
  • the determination module 61 may be used to determine damaged video frames in the video encoding stream.
  • the processing module 62 may be configured to perform error concealment on the damaged macroblocks according to the macroblock type of the damaged macroblock in the damaged video frame when the damaged video frame is intra-coded.
  • the macroblock type includes: flat blocks, edge blocks, and texture blocks; alternatively, can be used to perform error concealment on damaged macroblocks based on the macroblock status of the target macroblock when the damaged video frame is inter-coded;
  • the target macroblock includes any of the following Items: macroblocks adjacent to the damaged macroblock in the damaged video frame; macroblocks adjacent to the damaged macroblock in the damaged video frame, and macroblocks in the reference frame of the damaged video frame.
  • the processing module 62 may also be used to perform edge detection on the damaged macroblock before error concealment of the damaged macroblock according to the macroblock type of the damaged macroblock in the damaged video frame. , and determine the macroblock type of the damaged macroblock based on the edge information of edge detection.
  • the processing module 62 may include a detection sub-module and a determination sub-module.
  • the detection sub-module may be used to detect the gradient intensity of pixels in at least one adjacent area of the damaged macroblock.
  • the determination sub-module can be used to determine the flatness threshold of the damaged macro block based on the gradient intensity detected by the detection sub-module, and when the flatness of the damaged macro block is less than the flatness threshold, the flatness of the damaged macro block is The macroblock type is determined to be flat block.
  • the above detection sub-module can also be used to detect the texture complexity of the damaged macroblock when the flatness of the damaged macroblock is greater than or equal to the flatness threshold.
  • the above determination sub-module may also be used to determine the macro block type of the damaged macro block based on the texture complexity detected by the detection sub module.
  • the above-mentioned detection sub-module may include a first obtaining sub-module and a first determining sub-module.
  • the first acquisition sub-module can be used to obtain the pixel gradient intensity of at least one pixel prediction direction.
  • Each pixel prediction direction corresponds to a pixel gradient intensity.
  • the pixel gradient intensity is the sum of the gradient intensity of pixels within the angle range corresponding to the pixel prediction direction.
  • the first determination sub-module is configured to determine the texture complexity of the damaged macroblock based on the cumulative gradient intensity and the maximum gradient intensity of at least one pixel gradient intensity obtained by the first acquisition sub-module; wherein the cumulative gradient intensity is the at least one The sum of pixel gradient intensities.
  • the processing module 62 may also include an update sub-module.
  • the above determination sub-module can also be used to determine the M pixel adjustment amounts corresponding to the damaged pixels based on the M prediction directions and the position information of the damaged pixels in the damaged macroblock. Each prediction direction corresponds to one pixel adjustment amount.
  • M is an integer greater than 2.
  • the update sub-module may be used to update the pixel value of the damaged pixel based on the M pixel adjustment amounts determined by the determination sub-module.
  • the above-mentioned determination sub-module can also be used to separately determine each prediction based on each prediction direction before the above-mentioned update sub-module updates the pixel value of the damaged pixel based on the above-mentioned M pixel adjustment amounts.
  • the pixel corresponding to the direction adjusts the weight.
  • the update sub-module may be specifically used to update the pixel value of the damaged pixel based on the M pixel adjustment amounts and the M pixel adjustment weights determined by the determination sub-module, the M pixel adjustment amounts and the M pixel adjustment weights One-to-one correspondence.
  • the above-mentioned determination sub-module can be used to determine the pixel adjustment weight corresponding to each prediction direction based on the accumulated gradient intensity corresponding to each prediction direction; the above-mentioned determination sub-module can also be used to specifically determine the pixel adjustment weight corresponding to each prediction direction based on Each prediction direction and the position information of the damaged macroblock determine the first boundary pixel and the second boundary pixel corresponding to each prediction direction, and based on the determined first boundary pixel and the second boundary pixel corresponding to each prediction direction Pixel value and position information determine the pixel adjustment weight corresponding to each prediction direction.
  • the processing module 62 may be configured to use a bilinear interpolation method to hide errors in the damaged macroblock when the macroblock type of the damaged macroblock is a flat block; or, Specifically, it can be used to use a one-way interpolation method to hide errors in the damaged macroblock when the macroblock type of the damaged macroblock is an edge block.
  • the processing module 62 may also include a processing sub-module.
  • the above determination sub-module can also be used to determine the first candidate motion vector set of the damaged macroblock based on the first number or the second number.
  • the first number is the damaged macroblock adjacent to the damaged macroblock in the damaged video frame and correctly received.
  • the number of macroblocks, the second number being the number of damaged macroblocks adjacent to the damaged macroblock in the damaged video frame quantity.
  • the processing submodule may be configured to perform error concealment on the damaged macroblock based on the first candidate motion vector set determined by the determination submodule; wherein each motion vector in the first candidate motion vector set indicates one of the reference video frames. Macroblock.
  • the first candidate motion vector set when the first number is greater than or equal to the first preset number, or the second number is less than or equal to the second preset number, the first candidate motion vector set includes the second candidate motion vector set. ; Or, in the case where the first number is less than the first preset number, or the second number is greater than the second preset number, the first candidate motion vector set includes the second candidate motion vector set and the third candidate motion vector set; wherein , the second candidate motion vector set includes: motion vectors of macroblocks adjacent to the damaged macroblock and correctly received in the damaged video frame, zero motion vectors, motion vectors of macroblocks adjacent to the damaged macroblock in the damaged video frame and correctly received
  • the third candidate motion vector set includes: the motion vector of the macroblock with the same position as the damaged macroblock in the reference frame of the damaged video frame.
  • the above-mentioned processing sub-module may include a second determination sub-module and a first processing sub-module; the second determination sub-module may be used to focus the first candidate motion vectors on candidates that minimize the loss cost function value.
  • the motion vector is determined as the target motion vector, and the motion compensation block of the target motion vector in the reference frame of the damaged video frame is determined.
  • the embodiments of the present application for intra-coded damaged video frames, different methods can be used for error concealment of damaged macroblocks with different macroblock types in the damaged video frame. ;
  • the damaged video frame can be modified based on the macroblocks adjacent to the damaged macroblock in the damaged video frame, or the adjacent macroblocks and the macroblocks in the reference frame of the damaged video frame. Therefore, compared with the solution in the related art that uses a single error concealment algorithm to hide the damaged macroblocks in the damaged video frames of intra-frame coding or inter-frame coding, the embodiments of the present application provide The video frame error concealment device can more accurately conceal errors of each damaged macroblock. This can improve error concealment of damaged macroblocks in damaged video frames.
  • the video frame error hiding device in the embodiment of the present application may be an electronic device or a component in the electronic device, such as an integrated circuit or chip.
  • the electronic device may be a terminal or other devices other than the terminal.
  • the electronic device can be a mobile phone, a tablet computer, a notebook computer, a handheld computer, a vehicle-mounted electronic device, a mobile internet device (Mobile Internet Device, MID), or augmented reality (AR)/virtual reality (VR).
  • the video frame error hiding device in the embodiment of the present application may be a device with an operating system.
  • the operating system can be an Android operating system, an ios operating system, or other possible operating systems, which are not specifically limited in the embodiments of this application.
  • the video frame error concealment device provided by the embodiment of the present application can implement each process implemented by the method embodiments of Figures 1 to 5. To avoid duplication, they will not be described again here.
  • this embodiment of the present application also provides an electronic device 700, including a processor 701 and a memory 702.
  • the memory 702 stores programs or instructions that can be run on the processor 701.
  • each step of the above video error hiding method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, the details will not be described here.
  • the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.
  • FIG. 8 is a schematic diagram of the hardware structure of an electronic device implementing an embodiment of the present application.
  • the electronic device 1000 includes but is not limited to: radio frequency unit 1001, network module 1002, audio output unit 1003, input unit 1004, sensor 1005, display unit 1006, user input unit 1007, interface unit 1008, memory 1009, processor 1010, etc. part.
  • the electronic device 1000 may also include a power supply (such as a battery) that supplies power to various components.
  • the power supply may be logically connected to the processor 1010 through a power management system, thereby managing charging, discharging, and And functions such as power consumption management.
  • the structure of the electronic device shown in Figure 8 does not constitute a limitation on the electronic device.
  • the electronic device may include more or less components than shown in the figure, or combine certain components, or arrange different components, which will not be described again here. .
  • the processor 1010 may be used to determine the damaged video frame in the video encoding stream, and when the damaged video frame is intra-frame coded, according to the macroblock type of the damaged macroblock in the damaged video frame, Perform error concealment on damaged macroblocks.
  • Macroblock types include: flat blocks, edge blocks, and texture blocks; or, if the damaged video frame is inter-coded, hide the damaged macroblocks based on the macroblock status of the target macroblock.
  • blocks for error concealment; target macroblocks include any of the following: macroblocks adjacent to the damaged macroblock in the damaged video frame; macroblocks adjacent to the damaged macroblock in the damaged video frame, and macroblocks of the damaged video frame Macroblock in the reference frame.
  • the processor 1010 may also be configured to perform edge detection on the damaged macroblock before error concealment of the damaged macroblock according to the macroblock type of the damaged macroblock in the damaged video frame. , and determine the macroblock type of the damaged macroblock based on the edge information of edge detection.
  • the processor 1010 may be configured to detect the gradient intensity of pixels in at least one adjacent area of the damaged macroblock, and determine the flatness threshold of the damaged macroblock based on the detected gradient intensity. , and when the flatness of the damaged macroblock is less than the flatness threshold, the macroblock type of the damaged macroblock is determined as a flat block.
  • the processor 1010 may also be configured to detect the texture complexity of the damaged macroblock when the flatness of the damaged macroblock is greater than or equal to the flatness threshold, and based on the detected texture complexity Determines the macroblock type of the damaged macroblock.
  • the processor 1010 may be specifically configured to obtain the pixel gradient intensity of at least one pixel prediction direction.
  • Each pixel prediction direction corresponds to a pixel gradient intensity, and the pixel gradient intensity is within the angle range corresponding to the pixel prediction direction.
  • the sum of the gradient intensities of the pixels and determine the texture complexity of the damaged macroblock based on the cumulative gradient intensity and the maximum gradient intensity among the obtained at least one pixel gradient intensity; where the accumulated gradient intensity is the sum of the gradient intensities of at least one pixel .
  • the processor 1010 may also be configured to determine M pixel adjustment amounts corresponding to the damaged pixels based on the M prediction directions and the position information of the damaged pixels in the damaged macroblock.
  • Each prediction The direction corresponds to a pixel adjustment amount, M is an integer greater than 2, and based on the determined M pixel adjustment amounts, the pixel value of the damaged pixel is updated.
  • the processor 1010 may also be configured to determine the pixel adjustment corresponding to each prediction direction based on each prediction direction before updating the pixel value of the damaged pixel based on the above M pixel adjustment amounts. weight, and update the pixel value of the damaged pixel based on the M pixel adjustment amounts and the determined M pixel adjustment weights.
  • the M pixel adjustment amounts correspond to the M pixel adjustment weights one-to-one.
  • the processor 1010 may be configured to determine the pixel adjustment weight corresponding to each prediction direction based on the cumulative gradient intensity corresponding to each prediction direction; the processor 1010 may be configured to determine the pixel adjustment weight corresponding to each prediction direction based on the accumulated gradient intensity corresponding to each prediction direction.
  • prediction directions and the position information of the damaged macroblock determine the first boundary pixel and the second boundary pixel corresponding to each prediction direction, and based on the determined first boundary pixel and the second boundary pixel corresponding to each prediction direction value and position information to determine the pixel adjustment weight corresponding to each prediction direction.
  • the processor 1010 may be configured to use a bilinear interpolation method to hide errors in the damaged macroblock when the macroblock type of the damaged macroblock is a flat block; or, Specifically, it can be used to use a one-way interpolation method to hide errors in the damaged macroblock when the macroblock type of the damaged macroblock is an edge block.
  • the processor 1010 may also be configured to determine a first candidate motion vector set of the damaged macroblock based on a first number or a second number, where the first number is the number of motion vectors between the damaged video frame and the damaged macroblock. The number of macroblocks adjacent to the macroblock and correctly received, the second number is the number of macroblocks adjacent to the damaged macroblock and damaged in the damaged video frame; and based on the determined first candidate motion vector set, Corrupted macroblocks undergo error concealment; wherein each motion vector in the first set of candidate motion vectors indicates a macroblock in the reference video frame.
  • the first candidate motion vector set when the first number is greater than or equal to the first preset number, or the second number is less than or equal to the second preset number, the first candidate motion vector set includes the second candidate motion vector set. ; Or, in the case where the first number is less than the first preset number, or the second number is greater than the second preset number, the first candidate motion vector set includes the second candidate motion vector set and the third candidate motion vector set; wherein , the second candidate motion vector set includes: motion vectors of macroblocks adjacent to the damaged macroblock and correctly received in the damaged video frame, zero motion vectors, motion vectors of macroblocks adjacent to the damaged macroblock in the damaged video frame and correctly received
  • the third candidate motion vector set includes: the motion vector of the macroblock with the same position as the damaged macroblock in the reference frame of the damaged video frame.
  • the processor 1010 can also be used to focus the first candidate motion vector to make the loss cost function
  • the candidate motion vector with the smallest value is determined as the target motion vector, and the motion compensation block of the target motion vector in the reference frame of the damaged video frame is determined, and the damaged macroblock is error-hidden according to the determined motion compensation block;
  • the electronic device for damaged video frames encoded within frames, different methods can be used to hide errors for damaged macroblocks with different macroblock types in the damaged video frames; for frames For inter-coded damaged video frames, the damaged macroblocks can be calculated based on the macroblocks adjacent to the damaged macroblock in the damaged video frame, or the adjacent macroblocks and the macroblocks in the reference frame of the damaged video frame. Perform error concealment; therefore, compared with the solution in the related art that a single error concealment algorithm is used to conceal the error of the damaged macroblock in the damaged video frames of intra-frame coding or inter-frame coding, the electronic device provided by the embodiment of the present application can More accurate error concealment of individual corrupted macroblocks. This can improve error concealment of damaged macroblocks in damaged video frames.
  • the input unit 1004 may include a graphics processor (Graphics Processing Unit, GPU) 10041 and a microphone 10042.
  • the graphics processor 10041 is responsible for the image capture device (GPU) in the video capture mode or the image capture mode. Process the image data of still pictures or videos obtained by cameras (such as cameras).
  • the display unit 1006 may include a display panel 10061, which may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like.
  • the user input unit 1007 includes at least one of a touch panel 10071 and other input devices 10072 .
  • Touch panel 10071 also known as touch screen.
  • the touch panel 10071 may include two parts: a touch detection device and a touch controller.
  • Other input devices 10072 may include but are not limited to physical keyboards, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, and joysticks, which will not be described again here.
  • Memory 1009 may be used to store software programs as well as various data.
  • the memory 1009 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, Image playback function, etc.) etc.
  • memory 1009 may include volatile memory or nonvolatile memory, or memory 1009 may include both volatile and nonvolatile memory.
  • non-volatile memory can be read-only memory (Read-Only Memory, ROM), programmable read-only memory (Programmable ROM, PROM), erasable programmable read-only memory (Erasable PROM, EPROM), electrically removable memory.
  • Volatile memory can be random access memory (Random Access Memory, RAM), static random access memory (Static RAM, SRAM), dynamic random access memory (Dynamic RAM, DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDRSDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (Synch link DRAM) , SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DRRAM).
  • RAM Random Access Memory
  • SRAM static random access memory
  • DRAM dynamic random access memory
  • synchronous dynamic random access memory Synchronous DRAM, SDRAM
  • Double data rate synchronous dynamic random access memory Double Data Rate SDRAM, DDRSDRAM
  • enhanced SDRAM synchronous dynamic random access memory
  • Synch link DRAM synchronous link dynamic random access memory
  • SLDRAM direct memory bus random access memory
  • Direct Rambus RAM Direct Rambus RAM
  • the processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor and a modem processor, where the application processor mainly handles operations related to the operating system, user interface, application programs, etc., Modem processors mainly process wireless communication signals, such as baseband processors. It can be understood that the above modem processor may not be integrated into the processor 1010.
  • Embodiments of the present application also provide a readable storage medium, which stores a program or instructions.
  • a program or instructions When the program or instructions are executed by a processor, each process of the above video frame error hiding method embodiment is implemented, and can achieve the same technical effect, so to avoid repetition, we will not repeat them here.
  • the processor is the processor in the electronic device described in the above embodiment.
  • the readable storage medium includes computer readable storage media, such as computer read-only memory ROM, random access memory RAM, magnetic disk or optical disk, etc.
  • An embodiment of the present application further provides a chip.
  • the chip includes a processor and a communication interface.
  • the communication interface is coupled to the processor.
  • the processor is used to run programs or instructions to implement the above video frame error hiding method.
  • Each process of the embodiment can achieve the same technical effect, so to avoid repetition, it will not be described again here.
  • chips mentioned in the embodiments of this application may also be called system-on-chip, system-on-a-chip, system-on-a-chip or system-on-chip, etc.
  • Embodiments of the present application provide a computer program product, the program product is stored in a storage medium, and the program product is One less processor is required to implement each process of the above video frame error concealment method embodiment, and the same technical effect can be achieved. To avoid duplication, the details will not be described here.
  • the methods of the above embodiments can be implemented by means of software plus the necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is better. implementation.
  • the technical solution of the present application can be embodied in the form of a computer software product that is essentially or contributes to the existing technology.
  • the computer software product is stored in a storage medium (such as ROM/RAM, disk , optical disk), including several instructions to cause a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of this application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

本申请公开了一种视频帧错误隐藏方法、装置、电子设备及介质，属于通信技术领域。该方法包括：确定视频编码码流中的受损视频帧；在受损视频帧为帧内编码的情况下，根据受损视频帧中受损宏块的宏块类型，对受损宏块进行错误隐藏，宏块类型包括：平坦块、边缘块和纹理块；在受损视频帧为帧间编码的情况下，根据目标宏块的宏块状态对受损宏块进行错误隐藏；目标宏块包括以下任一项：受损视频帧中受损宏块相邻的宏块；受损视频帧中受损宏块相邻的宏块，以及受损视频帧的参考帧中的宏块。

Description

视频帧错误隐藏方法、装置、电子设备及介质
相关申请的交叉引用
本申请主张在2022年04月11日在中国提交的中国专利申请号202210377025.8的优先权，其全部内容通过引用包含于此。
技术领域
本申请属于通信技术领域，具体涉及一种视频帧错误隐藏方法、装置、电子设备及介质。
背景技术
随着通信技术的发展，电子设备的功能越来越丰富，例如，当视频编码码流中的视频帧受损时，电子设备可以通过错误隐藏算法，对受损视频帧中的受损宏块进行错误隐藏，以提高解码后的视频帧的显示效果。
具体的，若受损视频帧为帧内编码，则电子设备可以通过双线性插值BI(Bilinear Interpolation，BI)法对受损视频帧中的受损宏块进行错误隐藏；若受损视频帧为帧间编码，则电子设备可以通过边界匹配BMA(Boundary Matching Algorithm，BMA)算法对受损视频帧中的受损宏块进行错误隐藏。
然而，按照上述方法，一方面，通过BI法对受损视频帧中的受损宏块进行错误隐藏，会使得受损视频帧中的细节丢失，因此导致错误隐藏后的视频帧过于模糊；另一方面，由于BMA算法是基于受损视频帧中的各个宏块的平均运动矢量，对受损视频帧中的受损宏块进行错误隐藏的，因此当受损宏块的运动矢量与平均运动矢量间的差异较大时，通过BMA算法无法对受损宏块进行错误隐藏。如此，相关技术中对受损视频帧中的受损宏块进行错误隐藏的效果较差。
发明内容
本申请实施例的目的是提供一种视频帧错误隐藏方法、装置、电子设备及介质，能够解决相关技术中对受损视频帧中的受损宏块进行错误隐藏的效果较差的问题。
第一方面，本申请实施例提供了一种视频帧错误隐藏方法，该方法包括：确定视频编码码流中的受损视频帧；在受损视频帧为帧内编码的情况下，根据受损视频帧中受损宏块的宏块类型，对受损宏块进行错误隐藏，宏块类型包括：平坦块、边缘块和纹理块；在受损视频帧为帧间编码的情况下，根据目标宏块的宏块状态对受损宏块进行错误隐藏；目标宏块包括以下任一项：受损视频帧中受损宏块相邻的宏块；受损视频帧中受损宏块相邻的宏块，以及受损视频帧的参考帧中的宏块。
第二方面，本申请实施例提供了一种视频帧错误隐藏装置，该装置包括确定模块和处理模块；确定模块，用于确定视频编码码流中的受损视频帧；处理模块，用于在受损视频帧为帧内编码的情况下，根据受损视频帧中受损宏块的宏块类型，对受损宏块进行错误隐藏，宏块类型包括：平坦块、边缘块和纹理块；或者，用于在受损视频帧为帧间编码的情况下，根据目标宏块的宏块状态对受损宏块进行错误隐藏；目标宏块包括以下任一项：受损视频帧中受损宏块相邻的宏块；受损视频帧中受损宏块相邻的宏块，以及受损视频帧的参考帧中的宏块。
第三方面，本申请实施例提供了一种电子设备，该电子设备包括处理器和存储器，所述存储器存储可在所述处理器上运行的程序或指令，所述程序或指令被所述处理器执行时实现如第一方面所述的方法的步骤。
第四方面，本申请实施例提供了一种可读存储介质，所述可读存储介质上存储程序或指令，所述程序或指令被处理器执行时实现如第一方面所述的方法的步骤。
第五方面，本申请实施例提供了一种芯片，所述芯片包括处理器和通信接口，所述通信接口和所述处理器耦合，所述处理器用于运行程序或指令，实现如第一方面所述的方法。
第六方面，本申请实施例提供一种计算机程序产品，该程序产品被存储在存储介质中，该程序产品被至少一个处理器执行以实现如第一方面所述的方法。
在本申请实施例中，可以确定视频编码码流中的受损视频帧；并在受损视频帧为帧内编码的 情况下，根据受损视频帧中受损宏块的宏块类型，对受损宏块进行错误隐藏，宏块类型包括：平坦块、边缘块和纹理块；在受损视频帧为帧间编码的情况下，根据目标宏块的宏块状态对受损宏块进行错误隐藏；目标宏块包括以下任一项：受损视频帧中受损宏块相邻的宏块；受损视频帧中受损宏块相邻的宏块，以及受损视频帧的参考帧中的宏块。通过该方案，由于对于帧内编码的受损视频帧，可以对受损视频帧中宏块类型不同的受损宏块，分别采用不同的方法进行错误隐藏；对于帧间编码的受损视频帧，可以根据受损视频帧中受损宏块相邻的宏块，或该相邻的宏块与受损视频帧的参考帧中的宏块，对受损宏块进行错误隐藏；因此相比于相关技术中对于帧内编码或帧间编码的受损视频帧均采用单一错误隐藏算法对受损宏块进行错误隐藏的方案，本申请实施例提供的视频帧错误隐藏方法可以更加准确地对各个受损宏块进行错误隐藏。从而可以提高对受损视频帧中的受损宏块进行错误隐藏的效果。
附图说明
图1是相关技术中采用BI法对受损宏块进行错误隐藏的示意图；
图2是本申请实施例提供的视频帧错误隐藏方法的流程图；
图3是本申请实施例提供的视频帧错误隐藏方法中确定受损像素对应的M个像素调节量的方法的示意图；
图4是本申请实施例提供的视频帧错误隐藏方法中受损宏块的至少一个相邻区域的示意图；
图5是本申请实施例提供的视频帧错误隐藏方法中确定与受损宏块的边界相邻的宏块的方法的示意图；
图6是本申请实施例提供的视频帧错误隐藏装置的示意图；
图7是本申请实施例提供的电子设备的示意图；
图8是本申请实施例提供的电子设备的硬件示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员获得的所有其他实施例，都属于本申请保护的范围。
本申请的说明书和权利要求书中的术语“第一”、“第二”等是用于区别类似的对象，而不用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便本申请的实施例能够以除了在这里图示或描述的那些以外的顺序实施，且“第一”、“第二”等所区分的对象通常为一类，并不限定对象的个数，例如第一对象可以是一个，也可以是多个。此外，说明书以及权利要求中“和/或”表示所连接对象的至少其中之一，字符“/”，一般表示前后关联对象是一种“或”的关系。
下面首先对本申请的说明书和权利要求书中涉及的一些名词或者术语进行解释说明。
内部编码帧(即I帧)：也称为内部帧，属于帧内压缩，I帧的画面会完整保留，解码I帧时只需要本帧数据就可以完成。
单向预测编码帧(即P帧)：也称为差别帧或帧间压缩，P帧编码后表示的是当前帧与I帧或与当前帧之前的P帧的差别信息；解码P帧时，需要用当前帧之前的P帧或I帧缓存的画面叠加上本帧定义的编码的差别信息，重建当前帧的画面。
双向预测编码帧(即B帧)：B帧是双向差别帧，编码后的B帧记录的是本帧(即当前帧)与前后帧的差别信息；换言之，要解码B帧，不仅要取得之前的缓存画面，还要解码之后的画面，通过前后帧与本帧编码数据重建本帧图像。
下面结合附图，通过具体的实施例及其应用场景对本申请实施例提供的视频帧错误隐藏方法、装置、电子设备及介质进行详细地说明。
通信信道的不可靠性会导致视频数据在传输过程中受损或丢失，而视频源文件编码的高压缩率使得视频解码器对编码后的视频中受损或丢失的视频数据十分敏感。
示例性地，由于通常是采用预测编码和可变长编码对视频(视频文件)编码的，因此若编码后的视频的某个视频帧中的宏块受损或丢包，则可能使得需求基于该视频帧的解码视频帧的运动矢量进行预测的所有视频帧均无法正确解码，从而造成误码在空间和时间上的扩散，导致解码后 的视频的播放质量严重下降，甚至无法播放，使得解码后的视频源失真。
为了减少错误码流(即视频数据受损或丢失)引起的视频源失真问题，可以利用视频数据在空域和时域上的相关性，用视频中已经解码出的正确信息对视频中的错误信息进行隐藏。
相关技术中的错误隐藏方式包括：帧内错误隐藏和帧间错误隐藏。
1，帧内错误隐藏即空域错误隐藏，是恢复编码后的视频中的内部编码帧中丢失的数据的方法，其主要利用受损宏块与该受损宏块所在的视频帧内相邻且正确接收的宏块的相关性，采用BI法来估计受损宏块中的受损像素的像素值，从而对视频中的错误信息进行隐藏。
2，帧间错误隐藏即时域错误隐藏，是恢复编码后的视频中的P帧/B帧中丢失的数据的方法，其主要利用视频的相邻视频帧之间在时间上的相关性来恢复(或估计)受损视频帧中受损宏块的运动矢量(Motion Vector，MV)；而受损宏块的MV的估计主要是利用受损宏块周围正确接收的宏块的MV作为候选MV，然后，若受损视频帧的平均运动幅度较小，则可以根据位于该受损视频的前一视频帧中与该受损视频帧中的一个受损宏块具有相同位置的宏块的MV，直接恢复该受损视频帧中受损宏块的MV；若受损视频帧的平均运动幅度较大，则可以采用BMA算法或OBMA算法恢复该受损视频帧中受损宏块的MV。如此，可以对视频中的错误信息进行隐藏。
示例性地，以通过H.264/高级视频编码(Advanced Video Coding，AVC)参考软件对视频进行错误隐藏为例，首先，电子设备可以对接收到的视频编码码流进行丢包/误码检测，若丢包/误码发生在I帧，则电子设备通过帧内错误隐藏对相应视频帧中的受损宏块进行错误隐藏。电子设备可以利用视频图像在空域上变化平稳的特点，采用BI法，基于受损宏块的边界像素，对受损宏块中的受损像素的像素值进行加权插值，从而可以对受损宏块进行错误隐藏；若丢包/误码发生在P帧或B帧，则电子设备可以通过帧间错误隐藏对受损宏块进行错误隐藏，电子设备可以采用BMA算法，根据错误隐藏后重构帧的平滑性及相邻宏块的MV，确定出相应的候选运动补偿宏块，并根据其中具有最小边界失真的宏块的MV对受损宏块的MV进行恢复，从而使沿受损宏块边界的亮度变化最小，如此可以对受损宏块进行错误隐藏。然后，电子设备可以将错误隐藏后的码流发送至H.264/AVC参考软件进行解码，从而可以正常播放视频。
下面分别对采用BI法和BMA算法，对受损宏块进行错误隐藏的原理进行详细说明。
BI法是利用受损宏块周围被正确接收的相邻宏块中四个距离受损像素最近的像素进行线性插值，从而得到估计的受损像素的像素值。图1示出了采用BI法对受损宏块进行错误隐藏的示意图，如图1所示，图中灰色区域指示的宏块为受损宏块，对于该受损宏块中的受损像素10，电子设备可以从与该受损宏块相邻的，且正确接收的或者已经恢复的宏块的边界像素中，确定与受损像素10位于同一列的像素11和像素13，及与受损像素10位于同一行的像素12和像素14，并通过下述的插值公式(1)计算出估计的受损像素10的亮度值Pi,j：
其中，P1为像素11的亮度值，P2为像素12的亮度值，P3为像素13的亮度值，P4为像素14的亮度值，d1为像素11与像素10之间的距离，d2为像素12与像素10之间的距离，d3为像素13与像素10之间的距离，d4为像素14与像素10之间的距离。
可以理解，电子设备在通过BI法，对上述受损宏块中的每个受损像素进行恢复后，可以完成对受损宏块的错误隐藏。
而在BMA算法中，电子设备可以根据受损视频帧中所有正确接收的宏块的MV，并通过下述的公式(2)计算出所有正确接收的宏块的平均运动矢量V：
其中，N为受损视频帧中正确接收的宏块的数量，Vx和Vy分别为MV的水平分量和垂直分量。
若V小于一个预先设定的阈值T，说明上述受损视频帧的总体运动比较小，则电子设备可以 通过无运动补偿的时域错误隐藏模式，将该受损视频帧中的受损宏块，全部用该受损视频帧的前一帧同样位置的宏块来代替；若V大于或等于该阈值T，则电子设备可以通过运动补偿的时域错误隐藏模式，确定包括与受损宏块相邻的宏块的MV，及零运动矢量ZMV的候选MV集，并在该候选MV集中选择使如下(3)所示的代价函数DBMA的输出值最小的MV：
其中，ωT、ωB、ωL、ωR分别表示受损宏块周围上下左右相邻的宏块是否为正确接收的宏块，若为正确接收的宏块则其值为1，否则其值为0；N为受损视频帧中正确接收的宏块的数量；f(,,t)和f(,,t-1)分别表示受损视频帧和参考帧；f(x,y,,)表示处于整数像素位置(x,y)处的像素值；mvx和mvy分别为某一个MV的水平分量和垂直分量，f(x+mvx+i,y+mvy,t-1)是受损视频帧处于(x,y)位置的像素利用(mvx,mvy)在参考帧中恢复出来的像素值；f(x,y-1,t)是受损视频帧中处于(x,y)位置垂直相邻的上方的像素值。
在确定出使DBMA的输出值最小的MV之后，由于该MV可以指示参考帧中的一个宏块，因此可以将受损宏块中的每个受损像素的亮度值，更新为该MV指示的宏块内，具有相同位置的像素的亮度值。从而完成对该受损宏块的错误隐藏。
然而，当受损宏块的纹理较为丰富，或者受损宏块的位置位于受损视频帧的边缘时，通过BI法重建的图像将产生较大的块效应且模糊不清；而BMA算法对无物体边界的视频帧恢复较好，但通过视频帧的平均运动矢量来估计受损视频中每个受损宏块的运动程度，只有当整帧运动程度比较一致时才是准确的，且若受损宏块的外边界像素为物体边界，则BMA算法将不能选中合适的宏块。如此，相关技术中的错误隐藏方法存在对视频帧的边缘部分以及纹理细节丰富的区域重建效果差、对物体边界造成失配错误，以及高误码率时恢复效果差等问题。
为了解决上述问题，在本申请实施例提供的视频帧错误隐藏方法中，可以确定视频编码码流中的受损视频帧；并在受损视频帧为帧内编码的情况下，根据受损视频帧中受损宏块的宏块类型，对受损宏块进行错误隐藏，宏块类型包括：平坦块、边缘块和纹理块；在受损视频帧为帧间编码的情况下，根据目标宏块的宏块状态对受损宏块进行错误隐藏；目标宏块包括以下任一项：受损视频帧中受损宏块相邻的宏块；受损视频帧中受损宏块相邻的宏块，以及受损视频帧的参考帧中的宏块。通过该方案，由于对于帧内编码的受损视频帧，可以对受损视频帧中宏块类型不同的受损宏块，分别采用不同的方法进行错误隐藏；对于帧间编码的受损视频帧，可以根据受损视频帧中受损宏块相邻的宏块，或该相邻的宏块与受损视频帧的参考帧中的宏块，对受损宏块进行错误隐藏；因此相比于相关技术中对于帧内编码或帧间编码的受损视频帧均采用单一错误隐藏算法对受损宏块进行错误隐藏的方案，本申请实施例提供的视频帧错误隐藏方法可以更加准确地对各个受损宏块进行错误隐藏。从而可以提高对受损视频帧中的受损宏块进行错误隐藏的效果。
本申请实施例提供一种视频帧错误隐藏方法，图2示出了本申请实施例提供的视频帧错误隐藏方法的流程图。如图2所示，本申请实施例提供的视频帧错误隐藏方法可以包括下述的步骤201至步骤203，或者可以包括下述的步骤201、步骤202和步骤204。下面以电子设备执行该方法为例进行示例性地说明。
步骤201、电子设备确定视频编码码流中的受损视频帧。
可选地，本申请实施例中，电子设备可以对接收的视频编码码流进行丢包检测或误码检测，并将存在丢包或误码的视频帧确定为受损视频帧。
可选地，本申请实施例中，电子设备可以在接收视频编码码流的过程中进行丢包检测或误码 检测；或者，电子设备可以在接收视频编码码流之后，再进行丢包检测或误码检测。
步骤202、电子设备判断受损视频帧的编码类型。
对于电子设备判断受损视频帧的编码类型的方法，可以参照相关技术中的具体描述，为了避免重复，此处不再赘述。
本申请实施例中，电子设备可以判断受损视频帧的编码类型。若受损视频帧为帧内编码，则电子设备可以继续执行下述的步骤203(一种可能的实现方式)；若受损视频帧为帧间编码，则电子设备可以继续执行下述的步骤204(另一种可能的实现方式)。
一种可能的实现方式
步骤203、在受损视频帧为帧内编码的情况下，电子设备根据受损视频帧中受损宏块的宏块类型，对受损宏块进行错误隐藏。
可选地，本申请实施例中，宏块类型可以包括：(a)平坦块、(b)边缘块和(c)纹理块。
其中，若受损宏块的宏块类型为平坦块，则表示受损宏块图像比较平坦，无复杂纹理特性；若受损宏块的宏块类型为边缘块，则表示受损宏块图像边缘信息较单一；若受损宏块的宏块类型为纹理块，则表示受损宏块图像边缘信息较复杂。
对于电子设备确定受损宏块的宏块类型的具体方法，将在下述的实施例中进行详细说明，为了避免重复，此处不予赘述。
可选地，本申请实施例中，帧内编码的受损视频帧可以为I帧。
可选地，本申请实施例中，在宏块类型为上述(a)平坦块的情况下，上述步骤203具体可以通过下述的步骤203a实现；在宏块类型为上述(b)边缘块的情况下，上述步骤203具体可以通过下述的步骤203b实现。
步骤203a、在受损视频帧为帧内编码，且受损视频帧中受损宏块的宏块类型为平坦块的情况下，电子设备采用BI法，对受损宏块进行错误隐藏。
对于BI法的具体描述，可以参照上述实施例中对公式(1)的相关描述，为了避免重复，此处不再赘述。
本申请实施例中，由于BI法的运算复杂度较低，因此对平坦快采用BI法进行错误隐藏，可以使受损宏块有较好的恢复效果。
步骤203b、在受损视频帧为帧内编码，且受损视频帧中受损宏块的宏块类型为边缘块的情况下，电子设备采用单向插值法，对受损宏块进行错误隐藏。
可选地，本申请实施例中，电子设备在采用单向插值法对受损宏块进行错误隐藏时，插值方向可以为所有预测方向对应的累计梯度强度中的最大累计梯度强度对应的预测方向。
对于单向插值法、预测方向及累计梯度强度的具体描述，可以参照下述的实施例中的相关描述，为了避免重复，此处不予赘述。
本申请实施例中，电子设备可以根据受损宏块的不同宏块类型，分别采用不同的错误隐藏方法(即BI法或单向插值法)对受损宏块进行错误隐藏，因此可以提高电子设备对受损宏块进行错误隐藏的灵活性。
可选地，本申请实施例中，在宏块类型为上述(c)纹理块的情况下，上述步骤203具体可以通过下述的步骤203c和步骤203d实现。
可以理解，电子设备在对宏块类型为纹理块的受损宏块中的每个受损像素，分别执行以下步骤之后，完成对受损宏块的错误隐藏。
步骤203c、在受损视频帧为帧内编码，且受损视频帧中受损宏块的宏块类型为纹理块的情况下，电子设备根据M个预测方向和受损宏块中的受损像素的位置信息，确定受损像素对应的M个像素调节量。
本申请实施例中，每个预测方向与一个像素调节量对应，M为大于2的整数。
可选地，本申请实施例中，上述M个预测方向可以均匀分布，即每两个相邻预测方向之间的夹角均相同。
例如，上述M个预测方向可以为0-π范围内均匀分布的8个方向，分别为：0、π/8、π/4、3π/8、π/2、5π/8、3π/4、7π/8。
需要说明的是，本申请实施例不限定上述M个预测方向的设置方法，例如电子设备可以设置该M个预测方向为0-π范围内均匀分布的6个方向等。
下面结合附图，以电子设备确定受损宏块中的一个受损像素(以下称为目标受损像素)对应的一个像素调节量(以下称为目标像素调节量)为例，对电子设备确定该目标受损像素对应的M个像素调节量的具体方法进行示例性地说明。
示例性地，如图3所示，假设箭头a—>b所示的方向为上述M个预测方向中的一个预测方向，那么电子设备可以根据该预测方向和受损宏块30中的目标受损像素31的位置信息，确定目标受损像素31在该预测方向上的延长线l与受损宏块30的边界的交点X和交点Y，并分别确定受损宏块30的周围宏块中，距离交点X最近的像素32，及距离交点Y最近的像素33；若像素32的像素值为P1，像素33的像素值为P2，像素32与像素31之间的距离为d1，像素33与像素31之间的距离为d2，则电子设备可以采用单向插值法，通过下述的公式(4)计算出目标受损像素31对应的目标像素调节量Pi,j：
可以看出，箭头a->b所示的预测方向与目标像素调节量对应。
可以理解，电子设备可以对上述M个预测方向中除箭头a->b所示的预测方向外的每个预测方向，分别通过上述方法确定目标受损像素的一个像素调节量，从而可以确定目标受损像素对应的M个像素调节量。
步骤203d、电子设备基于M个像素调节量，更新受损像素的像素值。
可选地，本申请实施例中，受损像素的像素值可以为受损像素的亮度值。
可选地，本申请实施例中，以上述目标受损像素为例：目标受损像素更新后的像素值为：目标受损像素的初始像素值+目标受损像素值对应的M个像素调节量之和。
本申请实施例中，受损像素的初始像素值为受损像素受损后剩余的像素值。
具体的，电子设备可以基于M个像素调节量，采用迭代法更新目标受损像素的像素值。或者，电子设备可以先将M个像素调节量求和，然后再将目标受损像素的初始像素值与M个像素调节量之和相加。
可以理解，对于每个受损宏块，电子设备可以将一个受损宏块中的所有受损像素的像素值按照上述方法更新后，即可完成对受损宏块的错误隐藏。
本申请实施例中，由于在受损宏块的宏块类型为纹理块类型的情况下，电子设备可以基于每个预测方向对应的像素调节量，在每个预测方向上均对受损像素进行像素值更新，因此可以提高电子设备更新受损像素的像素值的准确性。
可选地，本申请实施例中，在上述步骤203d之前，本申请实施例提供的视频帧错误隐藏方法还可以包括下述的步骤203e；那么，上述步骤203d具体可以通过下述的步骤203d1实现。
步骤203e、电子设备基于每个预测方向，分别确定每个预测方向对应的像素调节权重。
可选地，本申请实施例中，电子设备可以基于上述每个预测方向对应的累计梯度强度(方式一)，也可以基于该每个预测方向和受损像素的位置信息(方式二)，还可以同时包括上述方式一和方式二，分别确定该每个预测方向对应的像素调节权重。
下面分别对上述方式一和方式二进行详细说明。
方式一
可选地，本申请实施例中，上述步骤203e具体可以通过下述的步骤203e1实现。
步骤203e1、电子设备基于每个预测方向对应的累计梯度强度，分别确定每个预测方向对应的像素调节权重。
可选地，本申请实施例中，电子设备可以对每个预测方向对应的累计梯度强度进行归一化处理，以确定每个预测方向对应的像素调节权重。具体的，假设电子设备设置的预测方向为0-π范围内均匀分布的8个方向，那么电子设备可以通过下述的公式(5)对每个预测方向对应的累计梯度强度进行归一化处理：
其中，φk为8个预测方向中的一个预测方向，Wk为预测方向φk对应的像素调节权重，Stren(φk)为预测方向φk对应的累计梯度强度，为8个预测方向对应的累计梯度强度之和。
可以看出，通过上述公式(5)，电子设备可以分别确定每个预测方向对应的像素调节权重，且受损宏块中的不同受损像素在相同预测方向上的像素调节权重均相同。
本申请实施例中，Wk可以用于对每一个预测方向的插值像素进行加权，Wk表征了每一个预测方向上的预测值对最终的插值像素的贡献。
可选地，本申请实施例中，在上述步骤203e1之前，本申请实施例提供的视频帧错误隐藏方法还可以包括下述的步骤A至步骤C。
步骤A、电子设备检测受损宏块的至少一个相邻区域内的像素的方向角度和梯度强度。
本申请实施例中，受损宏块的相邻区域为与该受损宏块的边界相邻(不包括角相邻)的区域。
本申请实施例中，上述至少一个相邻区域与受损宏块所在的区域为相同受损视频帧中的不同区域。
可选地，本申请实施例中，上述至少一个相邻区域中的每个相邻区域可以为以下任一项：受损视频帧中与受损宏块的上边界相邻的区域、受损视频帧中与受损宏块的下边界相邻的区域、受损视频帧中与受损宏块的左侧边界相邻的区域、受损视频帧中与受损宏块的右侧边界相邻的区域。
可以理解，上述至少一个相邻区域中的相邻区域的数量可以为2个、3个或4个，具体可以根据受损宏块在受损视频帧中所处的位置确定。
可选地，本申请实施例中，上述至少一个相邻区域中的每个相邻区域可以为：与受损宏块的边界相邻的一个宏块中的至少部分区域，且该至少部分区域与受损宏块相邻。
需要说明的是，本申请实施例中，至少一个相邻区域中的相邻区域的数量为多个时，每个相邻区域的尺寸相同，即每个相邻区域中的像素的数量相同。
示例性地，如图4所示，区域41、区域42、区域43和区域44为受损宏块的相邻区域(即至少一个相邻区域)，且区域41、区域42、区域43和区域44，与受损宏块所在的区域为同一受损视频帧40中的不同区域；且受损宏块中包括16×16个像素，区域41内的像素为与受损宏块的上边界相邻的8×16个像素，区域42内的像素为与受损宏块的下边界相邻的8×16个像素，区域43内的像素为与受损宏块的左边界相邻的16×8个像素，区域44内的像素为与受损宏块的右边界相邻的16×8个像素。
可选地，本申请实施例中，电子设备可以通过计算上述至少一个相邻区域内的像素的边缘矢量，检测受损宏块的至少一个相邻区域内的像素的方向角度和梯度强度。
下面以上述至少一个相邻区域内的一个像素(以下称为第一像素)为例，对电子设备检测受损宏块的至少一个相邻区域内的像素的方向角度的具体方法进行示例性地说明。
示例性地，假设第一像素为在受损视频帧中的位置为(i,j)处的像素，且第一像素的亮度值为Pi,j，那么电子设备可以定义第一像素的边缘矢量为：
其中，Fxi,j为在X轴方向上的分量，Fyi,j为在Y轴方向上的分量，且电子设备可以通过索贝尔Sobel算子或普鲁伊特Prewitt算子计算Fxi,j和Fyi,j，Sobel算子X轴和Y轴的卷积核公式如下：
Prewitt算子X轴和Y轴的卷积核公式如下：
以电子设备通过Sobel算子计算Fxi,j和Fyi,j为例，电子设备可以通过上述公式(7)计算Fxi,j和Fyi,j如下：
Fxi,j＝(pi+1,j-1-pi-1,j-1)+2(pi+1,j-pi-1,j)+(pi+1,j+1-pi-1,j+1)；    (9)
Fyi,j＝(pi-1,j+1-pi-1,j-1)+2(pi,j+1-pi,j-1)+(pi+1,j+1-pi+1,j-1)；    (10)
从而电子设备在计算出第一像素的边缘矢量的Fxi,j和Fyi,j之后，可以通过下述的公式(11)计算的方向角
如此电子设备可以根据检测到第一像素的方向角度θ(i,j)为：
对于电子设备检测上述至少一个相邻区域内的其它像素的方向角度的描述，具体可以参见上述对检测第一像素的方向角度的相关描述，为了避免重复，此处不再赘述。
下面仍然以上述至少一个相邻区域内的第一像素为例，对电子设备检测受损宏块的至少一个相邻区域内的像素的梯度强度的具体方法进行示例性地说明。
示例性地，若电子设备通过上述方法计算出第一像素的边缘矢量的Fxi,j和Fyi,j，则电子设备可以基于Fxi,j和Fyi,j，并通过下述的公式(13)计算出的幅度
从而电子设备可以检测到第一像素的梯度强度G(i,j)为：
对于电子设备检测上述至少一个相邻区域内的其它像素的梯度强度的描述，具体可以参见上述对检测第一像素的梯度强度的相关描述，为了避免重复，此处不再赘述。
步骤B、电子设备分别确定每个像素的方向角度对应的预测方向。
本申请实施例中，每个预测方向对应一个方向角度范围。
可以理解，电子设备可以预先设置预测方向与方向角度范围间的对应关系。如此，电子设备在检测到一个像素的方向角度之后，可以基于该方向角度所处的方向角度范围，确定该像素的方向角度对应的预测方向。
本申请实施例中，电子设备在检测到上述至少一个相邻区域内的像素的方向角度之后，可以根据预测方向与方向角度范围的对应关系，确定每个像素的方向角度对应的预测方向。如此可以 基于像素的方向角度，实现对至少一个相邻区域内的像素的归类。
示例性地，电子设备可以预先设置预测方向与方向角度范围的对应关系，如下表1所示；
表1
其中，根据上述表1可知，预测方向对应方向角度范围[0,π/8)；预测方向对应方向角度范围[π/8,π/4)；预测方向对应方向角度范围[π/4,3π/8)；预测方向对应方向角度范围[3π/8,π/2)；预测方向对应方向角度范围[π/2,5π/8)；预测方向对应方向角度范围[5π/8,3π/4)；预测方向对应方向角度范围[3π/8,7π/8)；预测方向对应方向角度范围[7π/8,π)。
从上表1可以看出，对于每个预测方向均有一个方向角度范围与之对应。从而，电子设备在检测到上述至少一个相邻区域内的像素的方向角度之后，可以根据表1中预测方向与方向角度范围的对应关系，以及每个像素的方向角度所属的方向角度范围，分别确定该每个像素的方向角度对应的预测方向。
可以理解，电子设备可以根据实际使用需求，设置不同的预测方向与方向角度范围的对应关系。
步骤C、电子设备将相同预测方向对应的像素的梯度强度相加，得到每个预测方向对应的累计梯度强度。
可选地，本申请实施例中，电子设备可以对每个预测方向对应的每个像素的梯度强度进行累计相加，得到该预测方向对应的累计梯度强度。
示例性地，以上述第一像素为例，对于预测方向若电子设备检测到第一像素的方向角度对应预测方向则电子设备可以通过下述公式(15)对预测方向对应的累计梯度强度进行更新：
其中，G(i,j)为第一像素的梯度强度。
如此，按照上述方法，电子设备可以得到每个预测方向对应的累计梯度强度。
方式二
可选地，本申请实施例中，上述步骤203e具体可以通过下述的步骤203e2和步骤203e3实现。
步骤203e2、电子设备根据每个预测方向和受损宏块的位置信息，确定每个预测方向对应的第一边界像素和第二边界像素。
对于电子设备确定每个预测方向对应的第一边界像素和第二边界像素的方法，具体可以参照上述图3的方法实施例中的相关描述，为了避免重复，此处不再赘述。
步骤203e3、电子设备基于每个预测方向对应的第一边界像素和第二边界像素的像素值和位置信息，确定每个预测方向对应的像素调节权重。
下面结合附图，以电子设备确定目标预测方向对应的像素调节权重为例，对步骤203e3进行示例性地说明。
示例性地，结合图3，假设目标预测方向为斜线(a->b)方向，电子设备基于目标预测方向和受损像素31的位置信息确定的第一边界像素为像素32、第二边界像素为像素33，那么电子设备可以基于像素32的像素值和像素33的像素值，通过下述的公式(16)为目标预测方向设置一个像素调节权重(即受损宏块两侧边缘像素的相似度因子)：
其中，exp(.)代表以自然数e为底的指数函数，P1为像素42的像素值，P2为像素43的像素值，γ为常数。
进一步地，通过上述公式(16)，电子设备可以分别确定每个预测方向对应的像素调节权重，且相同预测方向对应的不同受损像素的像素调节权重不同。
本申请实施例中，采用对受损宏块进行错误隐藏，可以使恢复后的受损视频帧图像更加平滑。
可以理解，若电子设备通过上述方式一和方式二，确定每个预测方向对应的像素调节权重，则一个像素调节权重为一个预测方向对应的
本申请实施例中，由于一方面电子设备可以通过上述方式一确定每个预测方向对应的像素调节权重，以使电子设备根据受损宏块在每个预测方向上的图像边缘信息，确定受损宏块中的每个受损像素的调节程度；另一方面电子设备可以通过上述方式二确定每个预测方向对应的像素调节权重，且一个像素调节权重可以指示受损宏块在相应预测方向上的两侧边缘像素的相似度，以使电子设备通过该像素调节权重可以使受损图像恢复的更平滑。如此可以提高电子设备确定像素调节权重的灵活性，进而可以提升对受损宏块进行错误隐藏的效果。
步骤203d1、电子设备基于M个像素调节量及M个像素调节权重，更新受损像素的像素值。
本申请实施例中，上述M个像素调节量与上述M个像素调节权重一一对应。
下面以目标受损像素为例，对电子设备更新受损像素的像素值的具体方法进行详细说明。
可选地，本申请实施例中，电子设备可以基于上述M个像素调节量及M个像素调节权重，首先针对预测方向通过下述的公式(17)更新目标受损像素的像素值P：
P＝Pi,j+W×P′i,j；     (17)
其中，Pi,j为目标受损像素的初始像素值，P′i,j为上述M个像素调节量中目标受损像素在预测方向上对应的像素调节量，W为上述M个像素调节权重中目标受损像素在预测方向上对应的像素调节权重。可以理解，W可以为以下任一项：Wk、
然后，电子设备可以再针对下一个预测方向按照上述方法对目标受损像素的像素值继续进行更新，此时上述公式(17)中的Pi,j为电子设备针对预测方向对目标受损像素的像素值更新后的像素值，P′i,j为上述M个像素调节量中目标受损像素在预测方向上对应的像素调节量，W为上述M个像素调节权重中目标受损像素在预测方向上对应的像素调节权重。如此，直至电子设备针对每个预测方向均对目标受损像素的像素值进行累计更新后，完成对目标受损像素的像素值的更新。
可选地，本申请实施例中，电子设备在对一个受损宏块中的一个受损像素的像素值更新完毕 之后，可以对该受损宏块中距离该受损像素最近，且位于该受损像素右方或下方的下一个受损像素的像素值进行更新，直至对该受损宏块中的所有受损像素更新完毕后，完成对该受损宏块的错误信息的隐藏。
进一步地，电子设备可以通过上述方法对所有受损宏块中的所有受损像素更新完毕后，完成对整个视频的错误隐藏，以提高电子设备播放该视频的效果。
本申请实施例中，由于电子设备可以基于M个像素调节量，及根据每个预测方向确定的且与该M个像素调节量一一对应的M个像素调节权重，更新受损像素的像素值，因此可以对受损像素的像素值在不同预测方向上进行不同程度的更新，从而可以进一步提高电子设备更新受损像素的像素值的准确性。
另一种可能的实现方式
步骤204、在受损视频帧为帧间编码的情况下，电子设备根据目标宏块的宏块状态对受损宏块进行错误隐藏。
可选地，本申请实施例中，目标宏块包括以下任一项：受损视频帧中受损宏块相邻的宏块；受损视频帧中受损宏块相邻的宏块，以及受损视频帧的参考帧中的宏块。
可选地，本申请实施例中，受损视频帧的参考帧中的宏块可以为：受损视频帧的参考帧中，与该受损视频帧中的受损宏块的位置相同的宏块。
可选地，本申请实施例中，宏块状态可以包括：正确接收或受损。
可选地，本申请实施例中，帧间编码的受损视频帧可以为P帧或B帧。
可以理解，电子设备在对受损视频帧中的所有受损宏块进行错误隐藏之后，即可完成对受损视频帧的错误隐藏。
进一步地，电子设备在对视频编码码流中的每个受损视频帧进行错误隐藏之后，即可完成对整个视频编码码流的错误隐藏，从而可以提高电子设备播放视频的效果。
需要说明的是，为了便于描述，下述实施例中不限定视频编码码流中的受损视频帧的数量，也不限定每个受损视频帧中的受损宏块的数量。也就是说，对于视频编码码流中的受损视频帧的数量可以为一个，也可以为多个；相应地，每个受损视频帧中的受损宏块的数量可以为一个，也可以为多个；具体可以根据实际使用需求确定。
可选地，本申请实施例中，上述步骤204具体可以通过下述的步骤204a和步骤204b实现。
步骤204a、电子设备基于第一数量或第二数量，确定受损宏块的第一候选运动矢量集。
本申请实施例中，第一数量为受损视频帧中与受损宏块相邻且正确接收的宏块的数量，第二数量为受损视频帧中与受损宏块相邻且受损的宏块的数量。
可以理解，与受损宏块相邻且正确接收的宏块为：与受损宏块的边界相邻，且不存在丢失或出错的像素的宏块。
可选地，本申请实施例中，在第一数量大于或等于第一预设数量，或者第二数量小于或等于第二预设数量的情况下，第一候选运动矢量集包括第二候选运动矢量集。
可选地，本申请实施例中，在第一数量小于第一预设数量，或者第二数量大于第二预设数量的情况下，第一候选运动矢量集包括第二候选运动矢量集和第三候选运动矢量集。
可选地，本申请实施例中，第一预设数量可以为：大于0，且小于或等于4的整数；例如，第一预设数量为2。
可选地，本申请实施例中，第二预设数量可以为：大于0，且小于或等于4的整数；第二预设数量可以与第一预设数量相同或不同。
本申请实施例中，第二候选运动矢量集包括：受损视频帧中与受损宏块相邻且正确接收的宏块的运动矢量、零运动矢量、受损视频帧中与受损宏块相邻且正确接收的宏块的运动矢量的中值运动矢量；第三候选运动矢量集包括：受损视频帧的参考帧中与受损宏块的位置相同的宏块的运动矢量。
可选地，本申请实施例中，上述中值运动矢量可以为：根据受损视频帧中与受损宏块相邻且正确接收的宏块的运动矢量的所有X轴方向的分量的中值，及所有Y轴方向的分量的中值确定的运动矢量。
可选地，本申请实施例中，受损视频帧的参考帧可以为视频编码码流中与受损视频帧相连续，且位于该受损视频帧之前的L个视频帧中的任一视频帧，L为小于或等于16的正整数。
例如，受损视频帧的参考帧为受损视频帧的前一视频帧，换句话说，参考帧为在受损视频帧之前接收的最后一个视频帧。
可选地，本申请实施例中，参考帧中位置与受损宏块的位置相同的宏块可以理解为：该宏块在参考帧中的位置信息，与受损宏块在受损视频帧中的位置信息相同。
可以理解，受损视频帧中与受损宏块相邻且正确接收的宏块的运动矢量，与受损宏块的运动矢量的相关性最高，且第一候选运动矢量集中包括的选运动矢量的数量适中，从而可以在确保对受损宏块进行错误隐藏的准确性的同时，可以降低运算的复杂度。
下面结合附图，对本申请实施例提供的视频帧错误隐藏方法进行示例性地说明。
示例性地，如图5所示，受损视频中与受损宏块E的边界相邻的每个宏块可以分割为多个子宏块，可以看出，电子设备将与受损宏块E的左边界相邻的多个子宏块中，位于最上方的子宏块A确定为与受损宏块E的左边界相邻的宏块；将与受损宏块E的上边界相邻的多个子宏块中，位于最左边的子宏块B确定为与受损宏块E的上边界相邻的宏块；将与受损宏块E的下边界相邻的多个子宏块中，位于最左边的子宏块D确定为与受损宏块E的下边界相邻的宏块。可以理解，若与受损宏块E的右边界相邻的宏块也包括被分割后的多个子宏块，则电子设备可以将其中位于最上方的子宏块确定为与受损宏块E的右边界相邻的宏块。
进一步地，假设上述第一预设数量为2，那么若宏块A、宏块B、宏块C和宏块D中正确接收的宏块的数量(即第一数量)大于或等于2，则电子设备可以确定第一候选运动矢量集为第二候选运动矢量集；若宏块A、宏块B、宏块C和宏块D中正确接收的宏块的数量小于2，则电子设备可以确定第一候选运动矢量集包括第二候选运动矢量集和第三候选运动矢量集。
本申请实施例中，由于电子设备可以根据第一数量或第二数量分别与对应的预设数量的关系，确定受损宏块不同的第一候选运动矢量集，因此在误码率较大(即与受损宏块相邻且正确接收的宏块的数量较小)的情况下，可以确保有足够的候选运动矢量，从而可以提高电子设备对受损宏块进行错误隐藏的鲁棒性。
步骤204b、电子设备基于第一候选运动矢量集，对受损宏块进行错误隐藏。
本申请实施例中，第一候选运动矢量集中的每个运动矢量指示参考视频帧中的一个宏块。
可选地，本申请实施例中，电子设备可以根据第一候选运动矢量集中的零运动矢量的位置信息，确定参考帧中具有相同位置信息的参考宏块，并可以根据第一候选运动矢量集中其它的每个运动矢量相对于该零运动矢量的偏移量，确定该参考帧中与该其它的每个运动矢量对应的一个宏块。
本申请实施例中，由于在受损视频帧为帧间编码的情况下，电子设备可以基于由第一数量或第二数量确定的受损宏块对应的第一候选运动矢量集，对受损宏块进行错误隐藏，因此可以进一步提高电子设备对受损宏块进行错误隐藏的准确性。
下面对电子设备基于第一候选运动矢量集，对受损宏块进行错误隐藏的具体方法进行详细说明。
可选地，本申请实施例中，上述步骤204b具体可以通过下述的步骤204b1至步骤204b3。
步骤204b1、电子设备将第一候选运动矢量集中使损失代价函数值最小的候选运动矢量确定为目标运动矢量。
本申请实施例中，上述损失代价函数Dtot表示为：
Dtot＝α1·DBMA+α2·DOBMA；    (18)
其中，DBMA为边界匹配代价函数，即上述公式(3)，DOBMA为重叠边界匹配代价函数，α1和α2为预设权重。
本申请实施例中，DOBMA可以表示为：

对于公式(19)的具体描述，可以参照上述公式(3)中的相关描述，为了避免重复，此处不再赘述。
可选地，本申请实施例中，电子设备在确定上述第一候选运动矢量集之后，可以将第一候选运动矢量集中的每个候选运动矢量代入上述公式(18)，并从中确定一个令Dtot最小的目标运动矢量。
可选地，本申请实施例中，若上述α1＝1，α2＝0，则Dtot＝DBMA，即电子设备可以采用DBMA算法对受损宏块进行错误隐藏；若α1＝0，α2＝1，则Dtot＝DOBMA，即电子设备可以采用DOBMA算法对受损宏块进行错误隐藏；若α1和α2均为大于0的正实数，则电子设备可以通过DBMA与DOBMA相结合的损失代价函数，对受损宏块进行错误隐藏。如此，可以提高电子设备对受损宏块进行错误隐藏的灵活性。
步骤204b2、电子设备确定目标运动矢量在受损视频帧的参考帧中的运动补偿块。
可选地，本申请实施例中，电子设备可以根据目标运动矢量相对于上述零运动矢量的偏移量，确定目标运动矢量在受损视频帧的参考帧中的运动补偿块。
步骤204b3、电子设备根据运动补偿块对受损宏块进行错误隐藏。
可选地，本申请实施例中，电子设备可以将受损宏块中的每个受损像素的像素值替换为：上述运动补偿块中，与该每个受损像素具有相同位置信息的像素的像素值。从而可以完成对受损宏块的错误隐藏。
本申请实施例中，由于电子设备可以通过DBMA与DOBMA相结合的损失代价函数，确定参考帧中的一个运动补偿块，并可以根据该运动补偿块对受损宏块进行错误隐藏，因此可以解决电子设备对物体边界恢复时失配错误的问题，且可以使恢复后的受损宏块两边的像素更加平滑，从而可以进一步提高电子设备对受损宏块进行错误隐藏的效果。
在本申请实施例提供的视频帧错误隐藏方法中，由于对于帧内编码的受损视频帧，可以对受损视频帧中宏块类型不同的受损宏块，分别采用不同的方法进行错误隐藏；对于帧间编码的受损视频帧，可以根据受损视频帧中受损宏块相邻的宏块，或该相邻的宏块与受损视频帧的参考帧中的宏块，对受损宏块进行错误隐藏；因此相比于相关技术中对于帧内编码或帧间编码的受损视频帧均采用单一错误隐藏算法对受损宏块进行错误隐藏的方案，本申请实施例提供的视频帧错误隐藏方法可以更加准确地对各个受损宏块进行错误隐藏。从而可以提高对受损视频帧中的受损宏块进行错误隐藏的效果。
可选地，本申请实施例中，在上述一种可能的实现方式中，在上述步骤203之前，本申请实施例提供的视频帧错误隐藏方法还可以包括下述的步骤205。
步骤205、电子设备对受损宏块进行边缘检测，并根据边缘检测的边缘信息确定受损宏块的宏块类型。
可选地，本申请实施例中，边缘检测可以包括平坦度检测；或者，边缘检测可以包括平坦度检测和纹理复杂度检测。
可以理解，宏块的边缘信息能够指示宏块图像的边缘纹理特性，从而可以使电子设备根据该边缘信息确定受损宏块的宏块类型。
需要说明的是，实际实现中，电子设备还可以采用其他任意可能的方式，确定受损宏块的宏块类型，具体可以根据实际使用需求确定。
本申请实施例中，由于电子设备可以对受损宏块进行边缘检测，因此可以获取受损宏块图像的平坦度和纹理复杂度，从而在电子设备根据边缘检测的边缘信息确定受损宏块的宏块类型时，可以基于受损宏块图像的平坦度和纹理复杂度，如此可以提高电子设备确定宏块类型的准确性。
可选地，本申请实施例中，上述步骤205具体可以通过下述的步骤205a至步骤204c实现，或者可以通过下述的步骤205a、步骤205b、步骤205d和步骤205e实现。
步骤205a、电子设备检测受损宏块的至少一个相邻区域内的像素的梯度强度。
对于步骤205a的具体描述，可以参照上述实施例中的相关描述，为了避免重复，此处不再赘述。
步骤205b、电子设备根据梯度强度，确定受损宏块的平坦度阈值。
可选地，本申请实施例中，由于像素的梯度强度可以指示图像的平坦度，因此电子设备可以根据一个梯度强度阈值(以下称为第一梯度强度阈值)，确定受损宏块的平坦度阈值。
步骤205c、电子设备在受损宏块的平坦度小于平坦度阈值的情况下，将受损宏块的宏块类型确定为平坦块。
可选地，本申请实施例中，电子设备可以根据上述至少一个相邻区域内的像素的梯度强度中的最大梯度强度和/或次大梯度强度，确定受损宏块的平坦度。
例如，以电子设备基于上述最大梯度强度确定受损宏块的平坦度为例，若该最大梯度强度小于第一梯度强度阈值，则电子设备可以确定受损宏块的平坦度小于平坦度阈值，即受损宏块图像比较平坦且无复杂纹理特性；若该最大梯度强度大于或等于第一梯度强度阈值，则电子设备可以确定受损宏块的平坦度大于或等于平坦度阈值，即受损宏块图像中有较丰富的纹理区域。从而电子设备可以在受损宏块的平坦度小于平坦度阈值的情况下，将受损宏块的宏块类型确定为平坦块。
本申请实施例中，由于像素的梯度强度可以反映图像的边缘特征，因此电子设备基于检测的受损宏块的至少一个相邻区域内的像素的梯度强度，将受损宏块的宏块类型确定为平坦块，可以提高电子设备确定受损宏块的宏块类型的准确性。
步骤205d、电子设备在受损宏块的平坦度大于或等于平坦度阈值的情况下，检测受损宏块的纹理复杂度。
可选地，本申请实施例中，上述步骤205d具体可以通过下述的步骤205d1和步骤205d2实现。
步骤205d1、电子设备在受损宏块的平坦度大于或等于平坦度阈值的情况下，获取至少一个像素预测方向的像素梯度强度。
本申请实施例中，上述至少一个像素预测方向中的每一像素预测方向对应一个像素梯度强度，像素梯度强度为像素预测方向对应的角度范围内像素的梯度强度之和。
对于步骤205d1中的其它描述，具体可以参照上述实施例中对预测方向和梯度强度的相关描述，为了避免重复，此处不再赘述。
步骤205d2、电子设备根据累计梯度强度和至少一个像素梯度强度中的最大梯度强度，确定受损宏块的边缘纹理复杂度。
本申请实施例中，上述累计梯度强度为上述至少一个像素梯度强度之和。
可选地，本申请实施例中，电子设备可以根据上述最大梯度强度与累计梯度强度的比值，确定受损宏块的纹理复杂度。
可选地，本申请实施例中，若上述比值小于一个梯度强度阈值(以下称为第二梯度强度阈值)，则电子设备可以确定受损宏块的纹理复杂度较大，即受损宏块图像边缘信息较复杂，且除上述最大累计梯度强度对应的预测方向之外的其它预测方向的边缘信息较多；若上述比值大于或等于第二梯度强度阈值，则电子设备可以确定受损宏块的纹理复杂度较小，即受损宏块图像边缘信息较少。
需要说明的是，实际实现中，电子设备还可以根据上述至少一个像素梯度强度中的次大梯度强度与累计梯度强度的比值，确定受损宏块的纹理复杂度，具体可以根据实际使用需求设置，本申请实施例不作限定。
本申请实施例中，由于图像边缘是图像的基本特征之一，其是以局部特征不连续的形式出现的，例如灰度值的突变、颜色的突变、纹理的突变等，其特征为在某一个方向上梯度强度较大； 而细节成分较丰富的宏块在很多方向甚至所有方向的梯度强度均较大；因此电子设备可以基于上述至少一个像素预测方向的像素梯度强度，确定受损宏块的纹理复杂度。
本申请实施例中，由于电子设备可以基于至少一个像素梯度强度中的最大梯度强度与累计梯度强度的比值，确定受损宏块的纹理复杂度，而该比值可以准确反映受损宏块图像边缘信息，因此可以提高电子设备确定受损宏块的纹理复杂度的准确性。
步骤205e、电子设备基于纹理复杂度确定受损宏块的宏块类型。
可选地，本申请实施例中，若受损宏块的平坦度小于上述平坦度阈值，且受损宏块的纹理复杂度大于或等于纹理复杂度阈值，则电子设备可以确定受损宏块的宏块类型为边缘块；若受损宏块的平坦度小于上述平坦度阈值，且受损宏块的纹理复杂度小于该纹理复杂度阈值，则电子设备可以确定受损宏块的宏块类型为纹理块。
可选地，本申请实施例中，上述纹理复杂度阈值可以由电子设备根据上述第二梯度强度阈值确定。
可以理解，电子设备可以通过上述平坦度检测，确定受损宏块的宏块类型为平坦块，并可以通过上述纹理复杂度检测，确定受损宏块的宏块类型为边缘块或纹理块。
本申请实施例中，由于电子设备可以基于检测的受损宏块的纹理复杂度，确定受损宏块的宏块类型为边缘块或纹理块，因此可以进一步提高电子设备确定受损宏块的宏块类型的准确性。
示例性地，假设电子设备对一个四分之一通用中间格式(Quarter Common Intermediate Format，QCIF)(176×144)视频序列进行编码，其中一帧图像包含99个16×16大小的宏块；且电子设备在进行视频编码时，关闭B帧预测，只采用I帧和P帧编码，并且I帧是周期性插入，每1秒钟视频插入一个I帧。设定电子设备中的编码器为帧编码模式(说明：若采用场编码模式，则相关配置参数需要乘以2)，帧率为24帧/秒，运动估计搜索范围为W＝16，参考视频帧的个数设定为1，开启率失真优化编码，量化参数QP＝28；并且编码后的码流通过信道进行传输，信道的丢包率为5％；那么，电子设备在接收码流后可以开启错误检测，若电子设备未检测到错误，则正常解码，若电子设备检测到码流错误，则调用错误隐藏模块对错误码流进行恢复。电子设备可以通过错误隐藏模块采用本申请实施例提供的视频帧错误隐藏方法对受损宏块进行错误隐藏。
情形1：I帧中受损宏块的错误隐藏
步骤1：电子设备检测受损宏块的平坦度；若估计受损宏块图像比较平坦，无复杂纹理特性，则电子设备确定受损宏块的类型为平坦块，并采用BI法对受损宏块进行错误隐藏。否则，电子设备继续进行纹理复杂度检测。
步骤2：电子设备检测受损宏块的纹理复杂度；若受损宏块的类型为边缘块，则电子设备采用单向插值法对受损宏块进行错误隐藏；若受损宏块的类型为纹理块，则电子设备采用改进的多向插值法(即基于M个像素调节量及M个像素调节权重，更新受损像素的像素值的方法)对受损宏块进行错误隐藏。
步骤3：重复步骤1至步骤2，电子设备可以按照先两侧受损宏块再中间受损宏块的次序对受损宏块进行错误隐藏，直至当前I帧所有受损宏块处理完毕。
情形2：P帧或B帧中的受损宏块的错误隐藏
步骤1：电子设备判断当前受损宏块的上、下、左、右相邻的四个宏块的状态，若其中至少有2个宏块是正确接收的宏块，则确定第一候选运动矢量集为第二候选运动矢量集，否则，确定第一候选运动矢量集包括第二候选运动矢量集和第三候选运动矢量集(即扩展的候选运动矢量集)。
步骤2：电子设备在第一候选运动矢量集中选择使损失代价函数Dtot最小的运动矢量作为受损宏块的最佳运动矢量。
步骤3：根据步骤2确定的最佳运动矢量，确定受损视频帧的参考帧中相应的运动补偿宏块，并将其插入到当前受损宏块的位置(即将受损像素的像素值替换为运动补偿宏块中具有相同位置信息像素的像素值)。
步骤4：重复步骤1至步骤3，电子设备可以按照先两侧受损宏块再中间受损宏块的次序对受损宏块的进行错误隐藏，直至当前P帧所有受损宏块处理完毕。
可以看出，本申请实施例提供的视频帧错误隐藏方法，充分利用了视频信号在空域和时域上的相关性，有效提升了传统技术中的错误隐藏算法的性能，从而可以使电子设备对错误码流仍然可以解码恢复出清晰流畅的视频图像。
本申请实施例提供的视频帧错误隐藏方法，执行主体可以为视频帧错误隐藏装置。本申请实施例中以视频帧错误隐藏装置执行视频帧错误隐藏方法为例，说明本申请实施例提供的视频帧错误隐藏装置。
结合图6，本申请实施例提供一种视频帧错误隐藏装置60，该视频帧错误隐藏装置60可以包括确定模块61和处理模块62。确定模块61，可以用于确定视频编码码流中的受损视频帧。处理模块62，可以用于在受损视频帧为帧内编码的情况下，根据受损视频帧中受损宏块的宏块类型，对受损宏块进行错误隐藏，宏块类型包括：平坦块、边缘块和纹理块；或者，可以用于在受损视频帧为帧间编码的情况下，根据目标宏块的宏块状态对受损宏块进行错误隐藏；目标宏块包括以下任一项：受损视频帧中受损宏块相邻的宏块；受损视频帧中受损宏块相邻的宏块，以及受损视频帧的参考帧中的宏块。
一种可能的实现方式中，处理模块62，还可以用于在根据受损视频帧中受损宏块的宏块类型，对受损宏块进行错误隐藏之前，对受损宏块进行边缘检测，并根据边缘检测的边缘信息确定受损宏块的宏块类型。
一种可能的实现方式中，处理模块62可以包括检测子模块和确定子模块。检测子模块，可以用于检测受损宏块的至少一个相邻区域内的像素的梯度强度。确定子模块，可以用于根据该检测子模块检测的梯度强度，确定受损宏块的平坦度阈值，并在受损宏块的平坦度小于平坦度阈值的情况下，将受损宏块的宏块类型确定为平坦块。
一种可能的实现方式中，上述检测子模块，还可以用于在受损宏块的平坦度大于或等于平坦度阈值的情况下，检测受损宏块的纹理复杂度。上述确定子模块，还可以用于基于检测子模块检测的纹理复杂度确定受损宏块的宏块类型。
一种可能的实现方式中，上述检测子模块可以包括第一获取子模块和第一确定子模块。第一获取子模块，可以用于获取至少一个像素预测方向的像素梯度强度，每一像素预测方向对应一个像素梯度强度，像素梯度强度为像素预测方向对应的角度范围内像素的梯度强度之和。第一确定子模块，用于根据累计梯度强度和第一获取子模块获取的至少一个像素梯度强度中的最大梯度强度，确定受损宏块的纹理复杂度；其中，累计梯度强度为该至少一个像素梯度强度之和。
一种可能的实现方式中，处理模块62还可以包括更新子模块。上述确定子模块，还可以用于根据M个预测方向和受损宏块中的受损像素的位置信息，确定受损像素对应的M个像素调节量，每个预测方向与一个像素调节量对应，M为大于2的整数。更新子模块，可以用于基于该确定子模块确定的M个像素调节量，更新受损像素的像素值。
一种可能的实现方式中，上述确定子模块，还可以用于在上述更新子模块基于上述M个像素调节量，更新受损像素的像素值之前，基于每个预测方向，分别确定每个预测方向对应的像素调节权重。该更新子模块，具体可以用于基于该M个像素调节量及该确定子模块确定的M个像素调节权重，更新受损像素的像素值，该M个像素调节量与该M个像素调节权重一一对应。
一种可能的实现方式中，上述确定子模块，具体可以用于基于每个预测方向对应的累计梯度强度，分别确定每个预测方向对应的像素调节权重；上述确定子模块具体还可以用于根据每个预测方向和受损宏块的位置信息，确定每个预测方向对应的第一边界像素和第二边界像素，并基于确定的每个预测方向对应的第一边界像素和第二边界像素的像素值和位置信息，确定每个预测方向对应的像素调节权重。
一种可能的实现方式中，处理模块62，具体可以用于在受损宏块的宏块类型为平坦块的情况下，采用双线性插值法，对受损宏块进行错误隐藏；或者，具体可以用于在受损宏块的宏块类型为边缘块的情况下，采用单向插值法，对受损宏块进行错误隐藏。
一种可能的实现方式中，处理模块62还可以包括处理子模块。上述确定子模块，还可以用于基于第一数量或第二数量，确定受损宏块的第一候选运动矢量集，第一数量为受损视频帧中与受损宏块相邻且正确接收的宏块的数量，第二数量为受损视频帧中与受损宏块相邻且受损的宏块的 数量。处理子模块，可以用于基于该确定子模块确定的第一候选运动矢量集，对受损宏块进行错误隐藏；其中，第一候选运动矢量集中的每个运动矢量指示参考视频帧中的一个宏块。
一种可能的实现方式中，在第一数量大于或等于第一预设数量，或者第二数量小于或等于第二预设数量的情况下，第一候选运动矢量集包括第二候选运动矢量集；或者，在第一数量小于第一预设数量，或者第二数量大于第二预设数量的情况下，第一候选运动矢量集包括第二候选运动矢量集和第三候选运动矢量集；其中，第二候选运动矢量集包括：受损视频帧中与受损宏块相邻且正确接收的宏块的运动矢量、零运动矢量、受损视频帧中与受损宏块相邻且正确接收的宏块的运动矢量的中值运动矢量；第三候选运动矢量集包括：受损视频帧的参考帧中与受损宏块的位置相同的宏块的运动矢量。
一种可能的实现方式中，上述处理子模块可以包括第二确定子模块和第一处理子模块；第二确定子模块，可以用于将第一候选运动矢量集中使损失代价函数值最小的候选运动矢量确定为目标运动矢量，并确定目标运动矢量在受损视频帧的参考帧中的运动补偿块。第一处理子模块，可以用于根据第二确定子模块确定的运动补偿块对受损宏块进行错误隐藏；其中，损失代价函数Dtot表示为：Dtot＝α1·DBMA+α2·DOBMA；其中，DBMA为边界匹配代价函数，DOBMA为重叠边界匹配代价函数，α1和α2为预设权重。
在本申请实施例提供的视频帧错误隐藏装置中，由于对于帧内编码的受损视频帧，可以对受损视频帧中宏块类型不同的受损宏块，分别采用不同的方法进行错误隐藏；对于帧间编码的受损视频帧，可以根据受损视频帧中受损宏块相邻的宏块，或该相邻的宏块与受损视频帧的参考帧中的宏块，对受损宏块进行错误隐藏；因此相比于相关技术中对于帧内编码或帧间编码的受损视频帧均采用单一错误隐藏算法对受损宏块进行错误隐藏的方案，本申请实施例提供的视频帧错误隐藏装置可以更加准确地对各个受损宏块进行错误隐藏。从而可以提高对受损视频帧中的受损宏块进行错误隐藏的效果。
本申请实施例中的视频帧错误隐藏装置可以是电子设备，也可以是电子设备中的部件，例如集成电路或芯片。该电子设备可以是终端，也可以为除终端之外的其他设备。示例性的，电子设备可以为手机、平板电脑、笔记本电脑、掌上电脑、车载电子设备、移动上网装置(Mobile Internet Device，MID)、增强现实(augmented reality，AR)/虚拟现实(virtual reality，VR)设备、机器人、可穿戴设备、超级移动个人计算机(ultra-mobile personal computer，UMPC)、上网本或者个人数字助理(personal digital assistant，PDA)等，还可以为服务器、网络附属存储器(Network Attached Storage，NAS)、个人计算机(personal computer，PC)、电视机(television，TV)、柜员机或者自助机等，本申请实施例不作具体限定。
本申请实施例中的视频帧错误隐藏装置可以为具有操作系统的装置。该操作系统可以为安卓(Android)操作系统，可以为ios操作系统，还可以为其他可能的操作系统，本申请实施例不作具体限定。
本申请实施例提供的视频帧错误隐藏装置能够实现图1至图5的方法实施例实现的各个过程，为避免重复，这里不再赘述。
可选地，如图7所示，本申请实施例还提供一种电子设备700，包括处理器701和存储器702，存储器702上存储有可在所述处理器701上运行的程序或指令，该程序或指令被处理器701执行时实现如上述视频错误隐藏方法实施例的各个步骤，且能达到相同的技术效果，为避免重复，这里不再赘述。
需要说明的是，本申请实施例中的电子设备包括上述所述的移动电子设备和非移动电子设备。
图8为实现本申请实施例的一种电子设备的硬件结构示意图。
该电子设备1000包括但不限于：射频单元1001、网络模块1002、音频输出单元1003、输入单元1004、传感器1005、显示单元1006、用户输入单元1007、接口单元1008、存储器1009、以及处理器1010等部件。
本领域技术人员可以理解，电子设备1000还可以包括给各个部件供电的电源(比如电池)，电源可以通过电源管理系统与处理器1010逻辑相连，从而通过电源管理系统实现管理充电、放电、 以及功耗管理等功能。图8中示出的电子设备结构并不构成对电子设备的限定，电子设备可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置，在此不再赘述。
其中，处理器1010，可以用于确定视频编码码流中的受损视频帧，并在受损视频帧为帧内编码的情况下，根据受损视频帧中受损宏块的宏块类型，对受损宏块进行错误隐藏，宏块类型包括：平坦块、边缘块和纹理块；或者，在受损视频帧为帧间编码的情况下，根据目标宏块的宏块状态对受损宏块进行错误隐藏；目标宏块包括以下任一项：受损视频帧中受损宏块相邻的宏块；受损视频帧中受损宏块相邻的宏块，以及受损视频帧的参考帧中的宏块。
一种可能的实现方式中，处理器1010，还可以用于在根据受损视频帧中受损宏块的宏块类型，对受损宏块进行错误隐藏之前，对受损宏块进行边缘检测，并根据边缘检测的边缘信息确定受损宏块的宏块类型。
一种可能的实现方式中，处理器1010，具体可以用于检测受损宏块的至少一个相邻区域内的像素的梯度强度，且根据检测的梯度强度，确定受损宏块的平坦度阈值，并在受损宏块的平坦度小于平坦度阈值的情况下，将受损宏块的宏块类型确定为平坦块。
一种可能的实现方式中，处理器1010，还可以用于在受损宏块的平坦度大于或等于平坦度阈值的情况下，检测受损宏块的纹理复杂度，并基于检测的纹理复杂度确定受损宏块的宏块类型。
一种可能的实现方式中，处理器1010，具体可以用于获取至少一个像素预测方向的像素梯度强度，每一像素预测方向对应一个像素梯度强度，像素梯度强度为像素预测方向对应的角度范围内像素的梯度强度之和；并根据累计梯度强度和获取的至少一个像素梯度强度中的最大梯度强度，确定受损宏块的纹理复杂度；其中，累计梯度强度为该至少一个像素梯度强度之和。
一种可能的实现方式中，处理器1010，还可以用于根据M个预测方向和受损宏块中的受损像素的位置信息，确定受损像素对应的M个像素调节量，每个预测方向与一个像素调节量对应，M为大于2的整数，并基于确定的该M个像素调节量，更新受损像素的像素值。
一种可能的实现方式中，处理器1010，还可以用于在基于上述M个像素调节量，更新受损像素的像素值之前，基于每个预测方向，分别确定每个预测方向对应的像素调节权重，并基于该M个像素调节量及确定的M个像素调节权重，更新受损像素的像素值，该M个像素调节量与该M个像素调节权重一一对应。
一种可能的实现方式中，处理器1010，具体可以用于基于每个预测方向对应的累计梯度强度，分别确定每个预测方向对应的像素调节权重；处理器1010，具体还可以用于根据每个预测方向和受损宏块的位置信息，确定每个预测方向对应的第一边界像素和第二边界像素，并基于确定的每个预测方向对应的第一边界像素和第二边界像素的像素值和位置信息，确定每个预测方向对应的像素调节权重。
一种可能的实现方式中，处理器1010，具体可以用于在受损宏块的宏块类型为平坦块的情况下，采用双线性插值法，对受损宏块进行错误隐藏；或者，具体可以用于在受损宏块的宏块类型为边缘块的情况下，采用单向插值法，对受损宏块进行错误隐藏。
一种可能的实现方式中，处理器1010，还可以用于基于第一数量或第二数量，确定受损宏块的第一候选运动矢量集，第一数量为受损视频帧中与受损宏块相邻且正确接收的宏块的数量，第二数量为受损视频帧中与受损宏块相邻且受损的宏块的数量；并基于确定的第一候选运动矢量集，对受损宏块进行错误隐藏；其中，第一候选运动矢量集中的每个运动矢量指示参考视频帧中的一个宏块。
一种可能的实现方式中，在第一数量大于或等于第一预设数量，或者第二数量小于或等于第二预设数量的情况下，第一候选运动矢量集包括第二候选运动矢量集；或者，在第一数量小于第一预设数量，或者第二数量大于第二预设数量的情况下，第一候选运动矢量集包括第二候选运动矢量集和第三候选运动矢量集；其中，第二候选运动矢量集包括：受损视频帧中与受损宏块相邻且正确接收的宏块的运动矢量、零运动矢量、受损视频帧中与受损宏块相邻且正确接收的宏块的运动矢量的中值运动矢量；第三候选运动矢量集包括：受损视频帧的参考帧中与受损宏块的位置相同的宏块的运动矢量。
一种可能的实现方式中，处理器1010，还可以用于将第一候选运动矢量集中使损失代价函数 值最小的候选运动矢量确定为目标运动矢量，且确定目标运动矢量在受损视频帧的参考帧中的运动补偿块，并根据确定的运动补偿块对受损宏块进行错误隐藏；其中，损失代价函数Dtot表示为：Dtot＝α1·DBMA+α2·DOBMA；其中，DBMA为边界匹配代价函数，DOBMA为重叠边界匹配代价函数，α1和α2为预设权重。
在本申请实施例提供的电子设备中，由于对于帧内编码的受损视频帧，可以对受损视频帧中宏块类型不同的受损宏块，分别采用不同的方法进行错误隐藏；对于帧间编码的受损视频帧，可以根据受损视频帧中受损宏块相邻的宏块，或该相邻的宏块与受损视频帧的参考帧中的宏块，对受损宏块进行错误隐藏；因此相比于相关技术中对于帧内编码或帧间编码的受损视频帧均采用单一错误隐藏算法对受损宏块进行错误隐藏的方案，本申请实施例提供的电子设备可以更加准确地对各个受损宏块进行错误隐藏。从而可以提高对受损视频帧中的受损宏块进行错误隐藏的效果。
应理解的是，本申请实施例中，输入单元1004可以包括图形处理器(Graphics Processing Unit，GPU)10041和麦克风10042，图形处理器10041对在视频捕获模式或图像捕获模式中由图像捕获装置(如摄像头)获得的静态图片或视频的图像数据进行处理。显示单元1006可包括显示面板10061，可以采用液晶显示器、有机发光二极管等形式来配置显示面板10061。用户输入单元1007包括触控面板10071以及其他输入设备10072中的至少一种。触控面板10071，也称为触摸屏。触控面板10071可包括触摸检测装置和触摸控制器两个部分。其他输入设备10072可以包括但不限于物理键盘、功能键(比如音量控制按键、开关按键等)、轨迹球、鼠标、操作杆，在此不再赘述。
存储器1009可用于存储软件程序以及各种数据。存储器1009可主要包括存储程序或指令的第一存储区和存储数据的第二存储区，其中，第一存储区可存储操作系统、至少一个功能所需的应用程序或指令(比如声音播放功能、图像播放功能等)等。此外，存储器1009可以包括易失性存储器或非易失性存储器，或者，存储器1009可以包括易失性和非易失性存储器两者。其中，非易失性存储器可以是只读存储器(Read-Only Memory，ROM)、可编程只读存储器(Programmable ROM，PROM)、可擦除可编程只读存储器(Erasable PROM，EPROM)、电可擦除可编程只读存储器(Electrically EPROM，EEPROM)或闪存。易失性存储器可以是随机存取存储器(Random Access Memory，RAM)，静态随机存取存储器(Static RAM，SRAM)、动态随机存取存储器(Dynamic RAM，DRAM)、同步动态随机存取存储器(Synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(Double Data Rate SDRAM，DDRSDRAM)、增强型同步动态随机存取存储器(Enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(Synch link DRAM，SLDRAM)和直接内存总线随机存取存储器(Direct Rambus RAM，DRRAM)。本申请实施例中的存储器1009包括但不限于这些和任意其它适合类型的存储器。
处理器1010可包括一个或多个处理单元；可选的，处理器1010集成应用处理器和调制解调处理器，其中，应用处理器主要处理涉及操作系统、用户界面和应用程序等的操作，调制解调处理器主要处理无线通信信号，如基带处理器。可以理解的是，上述调制解调处理器也可以不集成到处理器1010中。
本申请实施例还提供一种可读存储介质，所述可读存储介质上存储有程序或指令，该程序或指令被处理器执行时实现如上述视频帧错误隐藏方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
其中，所述处理器为上述实施例中所述的电子设备中的处理器。所述可读存储介质，包括计算机可读存储介质，如计算机只读存储器ROM、随机存取存储器RAM、磁碟或者光盘等。
本申请实施例另提供了一种芯片，所述芯片包括处理器和通信接口，所述通信接口和所述处理器耦合，所述处理器用于运行程序或指令，实现如上述视频帧错误隐藏方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
应理解，本申请实施例提到的芯片还可以称为系统级芯片、系统芯片、芯片系统或片上系统芯片等。
本申请实施例提供一种计算机程序产品，该程序产品被存储在存储介质中，该程序产品被至 少一个处理器执行以实现如上述视频帧错误隐藏方法实施例的各个过程，且能达到相同的技术效果，为避免重复，这里不再赘述。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。此外，需要指出的是，本申请实施方式中的方法和装置的范围不限按示出或讨论的顺序来执行功能，还可包括根据所涉及的功能按基本同时的方式或按相反的顺序来执行功能，例如，可以按不同于所描述的次序来执行所描述的方法，并且还可以添加、省去、或组合各种步骤。另外，参照某些示例所描述的特征可在其他示例中被组合。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以计算机软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端(可以是手机，计算机，服务器，或者网络设备等)执行本申请各个实施例所述的方法。
上面结合附图对本申请的实施例进行了描述，但是本申请并不局限于上述的具体实施方式，上述的具体实施方式仅仅是示意性的，而不是限制性的，本领域的普通技术人员在本申请的启示下，在不脱离本申请宗旨和权利要求所保护的范围情况下，还可做出很多形式，均属于本申请的保护之内。

Claims (20)

  1. 一种视频帧错误隐藏方法，所述方法包括：
    确定视频编码码流中的受损视频帧；
    在所述受损视频帧为帧内编码的情况下，根据所述受损视频帧中受损宏块的宏块类型，对所述受损宏块进行错误隐藏，所述宏块类型包括：平坦块、边缘块和纹理块；
    在所述受损视频帧为帧间编码的情况下，根据目标宏块的宏块状态对所述受损宏块进行错误隐藏；
    所述目标宏块包括以下任一项：
    所述受损视频帧中所述受损宏块相邻的宏块；
    所述受损视频帧中所述受损宏块相邻的宏块，以及所述受损视频帧的参考帧中的宏块。
  2. 根据权利要求1所述的方法，其中，所述根据所述受损视频帧中受损宏块的宏块类型，对所述受损宏块进行错误隐藏之前，所述方法还包括：
    对所述受损宏块进行边缘检测，并根据所述边缘检测的边缘信息确定所述受损宏块的宏块类型。
  3. 根据权利要求2所述的方法，其中，所述对所述受损宏块进行边缘检测，并根据所述边缘检测的边缘信息确定所述受损宏块的宏块类型，包括：
    检测所述受损宏块的至少一个相邻区域内的像素的梯度强度；
    根据所述梯度强度，确定所述受损宏块的平坦度阈值；
    在所述受损宏块的平坦度小于所述平坦度阈值的情况下，将所述受损宏块的宏块类型确定为平坦块。
  4. 根据权利要求3所述的方法，其中，在所述受损宏块的平坦度大于或等于所述平坦度阈值的情况下，检测所述受损宏块的纹理复杂度；
    基于所述纹理复杂度确定所述受损宏块的宏块类型。
  5. 根据权利要求4所述的方法，其中，所述检测所述受损宏块的纹理复杂度，包括：
    获取至少一个像素预测方向的像素梯度强度，每一所述像素预测方向对应一个所述像素梯度强度，所述像素梯度强度为所述像素预测方向对应的角度范围内像素的梯度强度之和；
    根据累计梯度强度和至少一个所述像素梯度强度中的最大梯度强度，确定所述受损宏块的纹理复杂度；
    其中，所述累计梯度强度为至少一个所述像素梯度强度之和。
  6. 根据权利要求1所述的方法，其中，在所述受损宏块的宏块类型为所述纹理块的情况下，所述根据所述受损视频帧中所述受损宏块的宏块类型，对所述受损宏块进行错误隐藏，包括：
    根据M个预测方向和所述受损宏块中的受损像素的位置信息，确定所述受损像素对应的M个像素调节量，每个所述预测方向与一个所述像素调节量对应，M为大于2的整数；
    基于所述M个像素调节量，更新所述受损像素的像素值。
  7. 根据权利要求6所述的方法，其中，所述基于所述M个像素调节量，更新所述受损像素的像素值之前，所述方法还包括：
    基于每个所述预测方向，分别确定每个所述预测方向对应的像素调节权重；
    所述基于所述M个像素调节量，更新所述受损像素的像素值，包括：
    基于所述M个像素调节量及M个所述像素调节权重，更新所述受损像素的像素值，所述M个像素调节量与所述M个所述像素调节权重一一对应。
  8. 根据权利要求7所述的方法，其中，所述基于每个所述预测方向，分别确定每个所述预测方向对应的像素调节权重，包括以下至少一项：
    基于每个所述预测方向对应的累计梯度强度，分别确定每个所述预测方向对应的像素调节权重；
    根据每个所述预测方向和所述受损宏块的位置信息，确定每个所述预测方向对应的第一边界像素和第二边界像素；
    基于每个所述预测方向对应的所述第一边界像素和所述第二边界像素的像素值和位置信息， 确定每个所述预测方向对应的像素调节权重。
  9. 根据权利要求1所述的方法，其中，所述根据所述受损视频帧中受损宏块的宏块类型，对所述受损宏块进行错误隐藏，包括：
    在所述受损宏块的宏块类型为所述平坦块的情况下，采用双线性插值法，对所述受损宏块进行错误隐藏；
    在所述受损宏块的宏块类型为所述边缘块的情况下，采用单向插值法，对所述受损宏块进行错误隐藏。
  10. 根据权利要求1所述的方法，其中，所述根据目标宏块的宏块状态对所述受损宏块进行错误隐藏，包括：
    基于第一数量或第二数量，确定所述受损宏块的第一候选运动矢量集，所述第一数量为所述受损视频帧中与所述受损宏块相邻且正确接收的宏块的数量，所述第二数量为所述受损视频帧中与所述受损宏块相邻且受损的宏块的数量；
    基于所述第一候选运动矢量集，对所述受损宏块进行错误隐藏；
    其中，所述第一候选运动矢量集中的每个运动矢量指示参考视频帧中的一个宏块。
  11. 根据权利要求10所述的方法，其中，在所述第一数量大于或等于第一预设数量，或者所述第二数量小于或等于第二预设数量的情况下，所述第一候选运动矢量集包括第二候选运动矢量集；
    或者，
    在所述第一数量小于第一预设数量，或者所述第二数量大于第二预设数量的情况下，所述第一候选运动矢量集包括第二候选运动矢量集和第三候选运动矢量集；
    其中，所述第二候选运动矢量集包括：所述受损视频帧中与所述受损宏块相邻且正确接收的宏块的运动矢量、零运动矢量、所述受损视频帧中与所述受损宏块相邻且正确接收的宏块的运动矢量的中值运动矢量；
    所述第三候选运动矢量集包括：所述受损视频帧的参考帧中与所述受损宏块的位置相同的宏块的运动矢量。
  12. 根据权利要求10或11所述的方法，其中，所述基于所述第一候选运动矢量集，对所述受损宏块进行错误隐藏，包括：
    将所述第一候选运动矢量集中使损失代价函数值最小的候选运动矢量确定为目标运动矢量；
    确定所述目标运动矢量在所述受损视频帧的参考帧中的运动补偿块；
    根据所述运动补偿块对所述受损宏块进行错误隐藏；
    其中，所述损失代价函数Dtot表示为：
    Dtot＝α1·DBMA+α2·DOBMA；
    其中，DBMA为边界匹配代价函数，DOBMA为重叠边界匹配代价函数，α1和α2为预设权重。
  13. 一种视频帧错误隐藏装置，所述装置包括确定模块和处理模块；
    所述确定模块，用于确定视频编码码流中的受损视频帧；
    所述处理模块，用于在所述受损视频帧为帧内编码的情况下，根据所述受损视频帧中受损宏块的宏块类型，对所述受损宏块进行错误隐藏，所述宏块类型包括：平坦块、边缘块和纹理块；或者，
    用于在所述受损视频帧为帧间编码的情况下，根据目标宏块的宏块状态对所述受损宏块进行错误隐藏；
    所述目标宏块包括以下任一项：
    所述受损视频帧中所述受损宏块相邻的宏块；
    所述受损视频帧中所述受损宏块相邻的宏块，以及所述受损视频帧的参考帧中的宏块。
  14. 根据权利要求13所述的装置，其中，
    所述处理模块，还用于在根据所述受损视频帧中受损宏块的宏块类型，对所述受损宏块进行错误隐藏之前，对所述受损宏块进行边缘检测，并根据所述边缘检测的边缘信息确定所述受损宏 块的宏块类型。
  15. 根据权利要求14所述的装置，其中，所述处理模块包括检测子模块和确定子模块；
    所述检测子模块，用于检测所述受损宏块的至少一个相邻区域内的像素的梯度强度；
    所述确定子模块，用于根据所述检测子模块检测的所述梯度强度，确定所述受损宏块的平坦度阈值，并在所述受损宏块的平坦度小于所述平坦度阈值的情况下，将所述受损宏块的宏块类型确定为平坦块。
  16. 一种电子设备，包括处理器和存储器，所述存储器存储可在所述处理器上运行的程序或指令，所述程序或指令被所述处理器执行时实现如权利要求1-12中任一项所述的视频帧错误隐藏方法的步骤。
  17. 一种可读存储介质，所述可读存储介质上存储程序或指令，所述程序或指令被处理器执行时实现如权利要求1-12中任一项所述的视频帧错误隐藏方法的步骤。
  18. 一种计算机程序产品，所述计算机程序产品被至少一个处理器执行以实现如权利要求1-12中任一项所述的视频帧错误隐藏方法。
  19. 一种电子设备，包括所述电子设备被配置成用于执行如权利要求1-12中任一项所述的视频帧错误隐藏方法。
  20. 一种芯片，所述芯片包括处理器和通信接口，所述通信接口和所述处理器耦合，所述处理器用于运行程序或指令，实现如权利要求1-12中任一项所述的视频帧错误隐藏方法。
PCT/CN2023/086622 2022-04-11 2023-04-06 视频帧错误隐藏方法、装置、电子设备及介质 Ceased WO2023197932A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210377025.8 2022-04-11
CN202210377025.8A CN114827632B (zh) 2022-04-11 2022-04-11 视频帧错误隐藏方法、装置、电子设备及介质

Publications (1)

Publication Number Publication Date
WO2023197932A1 true WO2023197932A1 (zh) 2023-10-19

Family

ID=82535127

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/086622 Ceased WO2023197932A1 (zh) 2022-04-11 2023-04-06 视频帧错误隐藏方法、装置、电子设备及介质

Country Status (2)

Country Link
CN (1) CN114827632B (zh)
WO (1) WO2023197932A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114827632B (zh) * 2022-04-11 2025-10-31 维沃移动通信有限公司 视频帧错误隐藏方法、装置、电子设备及介质
CN116016934B (zh) * 2023-01-06 2025-07-29 广东中星电子有限公司 视频编码方法、装置、电子设备和计算机可读介质
CN118337962B (zh) * 2024-06-12 2024-09-03 湖南中泓汇智智能科技有限公司 一种用于超视距远程驾驶平台的5g网络数据传输方法

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20060085003A (ko) * 2005-01-21 2006-07-26 한양대학교 산학협력단 Ｈ．２６４／ａｖｃ에서의 시방향 에러은닉 방법
CN101163250A (zh) * 2006-10-09 2008-04-16 北京航空航天大学 一种基于边界梯度的视频流容错方法
US20100002775A1 (en) * 2008-07-03 2010-01-07 Huang Shih-Chia Low-Complexity And High-Quality Error Concealment Techniques For Video Sequence Transmissions
US20100150253A1 (en) * 2008-12-11 2010-06-17 Sy-Yen Kuo Efficient Adaptive Mode Selection Technique For H.264/AVC-Coded Video Delivery In Burst-Packet-Loss Networks
CN102665082A (zh) * 2012-04-26 2012-09-12 中山大学 一种适用于视频监控系统的错误隐藏的方法
CN103856781A (zh) * 2014-03-18 2014-06-11 江西理工大学 一种自适应阈值的视频流多纹理方向错误隐藏方法
CN104486633A (zh) * 2014-11-11 2015-04-01 三星电子（中国）研发中心 视频错误掩藏方法及装置
CN105763882A (zh) * 2016-03-04 2016-07-13 广东顺德中山大学卡内基梅隆大学国际联合研究院 用于解码端的帧内错误隐藏方法及其系统
CN106101726A (zh) * 2016-07-22 2016-11-09 福建师范大学 一种时空域相结合的自适应错误隐藏修复方法及系统
CN114827632A (zh) * 2022-04-11 2022-07-29 维沃移动通信有限公司 视频帧错误隐藏方法、装置、电子设备及介质

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7606313B2 (en) * 2004-01-15 2009-10-20 Ittiam Systems (P) Ltd. System, method, and apparatus for error concealment in coded video signals
KR101144283B1 (ko) * 2010-07-27 2012-05-11 중앙대학교 산학협력단 디코딩된 비디오에 포함된 에러를 은닉하는 장치 및 방법

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20060085003A (ko) * 2005-01-21 2006-07-26 한양대학교 산학협력단 Ｈ．２６４／ａｖｃ에서의 시방향 에러은닉 방법
CN101163250A (zh) * 2006-10-09 2008-04-16 北京航空航天大学 一种基于边界梯度的视频流容错方法
US20100002775A1 (en) * 2008-07-03 2010-01-07 Huang Shih-Chia Low-Complexity And High-Quality Error Concealment Techniques For Video Sequence Transmissions
US20100150253A1 (en) * 2008-12-11 2010-06-17 Sy-Yen Kuo Efficient Adaptive Mode Selection Technique For H.264/AVC-Coded Video Delivery In Burst-Packet-Loss Networks
CN102665082A (zh) * 2012-04-26 2012-09-12 中山大学 一种适用于视频监控系统的错误隐藏的方法
CN103856781A (zh) * 2014-03-18 2014-06-11 江西理工大学 一种自适应阈值的视频流多纹理方向错误隐藏方法
CN104486633A (zh) * 2014-11-11 2015-04-01 三星电子（中国）研发中心 视频错误掩藏方法及装置
CN105763882A (zh) * 2016-03-04 2016-07-13 广东顺德中山大学卡内基梅隆大学国际联合研究院 用于解码端的帧内错误隐藏方法及其系统
CN106101726A (zh) * 2016-07-22 2016-11-09 福建师范大学 一种时空域相结合的自适应错误隐藏修复方法及系统
CN114827632A (zh) * 2022-04-11 2022-07-29 维沃移动通信有限公司 视频帧错误隐藏方法、装置、电子设备及介质

Also Published As

Publication number Publication date
CN114827632B (zh) 2025-10-31
CN114827632A (zh) 2022-07-29

Similar Documents

Publication Publication Date Title
WO2023197932A1 (zh) 视频帧错误隐藏方法、装置、电子设备及介质
CN110741640B (zh) 用于视频代码化中的运动补偿预测的光流估计
CN106331703B (zh) 视频编码和解码方法、视频编码和解码装置
US8964829B2 (en) Techniques to perform fast motion estimation
CN112073735B (zh) 视频信息处理方法、装置、电子设备及存储介质
CN110166771B (zh) 视频编码方法、装置、计算机设备和存储介质
CN107852500B (zh) 运动矢量场编码方法和解码方法、编码和解码装置
CN108012155A (zh) 预拼接图像的视频编码方法、视频解码方法和相关的装置
CN104053001A (zh) 对视频进行稳定化的方法和包括后处理电路的视频解码器
US20150264383A1 (en) Block Copy Modes for Image and Video Coding
WO2023131059A1 (zh) 图像编码方法、图像编码装置、电子设备和可读存储介质
CN109089121B (zh) 一种基于视频编码的运动估计方法、装置及电子设备
WO2010109564A1 (ja) 画像符号化装置及び画像符号化方法
US20260095580A1 (en) Video Encoding Method, Electronic Device and Non-Transitory Readable Storage Medium
CN117643053A (zh) 基于网络的视频编解码图像滤波
Yang et al. Efficient panoramic video coding for immersive metaverse experience
CN101326833B (zh) 利用参考帧选择规则进行视频错误隐藏的方法与装置
CN107483940A (zh) 一种基于屏幕变化检测的屏幕视频编码方法
CN118077203A (zh) 具有明确信示的扩展旋转的经扭曲的运动补偿
CN115086665A (zh) 误码掩盖方法、装置、系统、存储介质和计算机设备
CN115412727B (zh) 编码方法、解码方法及其装置
Lei et al. High performance and low complexity decoding light-weight video coding with motion estimation and mode decision at decoder
WO2023198144A1 (zh) 帧间预测方法及终端
WO2023116510A1 (zh) 帧间预测方法及终端
CN119211529A (zh) 视频帧间编码中的运动向量修正处理方法、装置及设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23787577

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23787577

Country of ref document: EP

Kind code of ref document: A1