CN111010575A - Code stream fault tolerance method and device and readable storage medium - Google Patents
Code stream fault tolerance method and device and readable storage medium Download PDFInfo
- Publication number
- CN111010575A CN111010575A CN201911307289.0A CN201911307289A CN111010575A CN 111010575 A CN111010575 A CN 111010575A CN 201911307289 A CN201911307289 A CN 201911307289A CN 111010575 A CN111010575 A CN 111010575A
- Authority
- CN
- China
- Prior art keywords
- code stream
- dpb
- image frame
- fault
- target image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/177—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a group of pictures [GOP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/577—Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
- H04N19/89—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving methods or arrangements for detection of transmission errors at the decoder
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The application provides a code stream fault tolerance method and device and a readable storage medium. The method comprises the following steps: acquiring a code stream to be processed; the code stream to be processed comprises header information of a target image frame; reading the header information in the code stream to be processed; if the head information in the code stream to be processed is incomplete, storing the previously stored head information or preset head information into the code stream to be processed to obtain a processed code stream; and if the wrong head information is read, replacing the wrong head information with the previously stored head information or preset head information to obtain a processed code stream. The method can improve the stability of decoding.
Description
Technical Field
The present application relates to the field of video decoding technologies, and in particular, to a method and an apparatus for code stream fault tolerance, and a readable storage medium.
Background
In the process of video decoding, the situation of error code and error code is easy to occur, most of the prior art focuses on how to eliminate the influence of the encoding end on the error code and error code, so corresponding measures are designed in an encoder to prevent errors from occurring.
However, for practical application scenarios, the error probability of the encoder is very small, and for reasons of universality and cost, the economic value of processing the code stream at the encoding transmitting end is low and it cannot be guaranteed that the receiving end is completely error-free.
Therefore, the fault-tolerant mode of the prior art is not simple and practical enough, and the stability of video decoding is not high.
Disclosure of Invention
An object of the present invention is to provide a method and an apparatus for code stream fault tolerance, and a readable storage medium, so as to improve stability of video decoding.
In a first aspect, an embodiment of the present application provides a code stream fault tolerance method, including: acquiring a code stream to be processed; the code stream to be processed comprises header information of a target image frame; reading the header information in the code stream to be processed; if the head information in the code stream to be processed is incomplete, storing the previously stored head information or preset head information into the code stream to be processed to obtain a processed code stream; and if the wrong head information is read, replacing the wrong head information with the previously stored head information or preset head information to obtain a processed code stream.
In the embodiment of the application, after the code stream to be processed is obtained, fault-tolerant processing is performed on the header information in the code stream, incomplete header information is supplemented or wrong header information is replaced through the previously stored header information or preset header information, and simple and practical code stream fault tolerance is achieved. Compared with the prior art, the error-tolerant processing of the head information is simply, quickly and practically carried out by utilizing the head information stored in advance or the preset head information aiming at the head information which is easy to generate error codes or error codes in the code stream, the error rate of the code stream subjected to the error-tolerant processing of the head information is reduced, and the decoding stability can be further improved.
As a possible implementation, the method further includes: pushing the processed code stream into a DPB, and carrying out DPB fault-tolerant processing on the processed code stream according to the frame type of the target image frame; and decoding and reconstructing the target image frame based on the code stream subjected to the DPB fault-tolerant processing.
In the embodiment of the application, after the code stream to be processed is subjected to the header information fault-tolerant processing, the code stream is pushed into the DPB, the DPB fault-tolerant processing is performed according to the frame type, the decoding and reconstruction of the target image frame are performed on the basis of the code stream subjected to the DPB fault-tolerant processing, and the code stream subjected to the header information fault-tolerant processing and the DPB fault-tolerant processing can further reduce the error rate of the code stream and further improve the decoding stability.
As a possible implementation manner, performing DPB fault-tolerant processing on the processed codestream according to the frame type of the target image frame includes: if the target image frame is a P frame or a B frame, searching a reference frame of the target image frame in the DPB; if the reference frame of the target image frame is found in the DPB, acquiring the reference frame information of the target image frame; if the reference frame corresponding to the target image frame is not found in the DPB, taking a frame which is closest to the playing sequence of the target image frame in the DPB as a reference frame of the target image frame; acquiring information of a frame closest to the playing sequence of the target image frame as reference frame information of the target image frame; correspondingly, decoding and reconstructing the target image frame based on the code stream subjected to the DPB fault-tolerant processing comprises the following steps: suspending pushing reference frames of the target image frame out of the DPB; and decoding and reconstructing the target image frame based on the reference frame information of the target image frame and the code stream subjected to the DPB fault-tolerant processing.
In the embodiment of the application, because the information of the reference frame needs to be utilized for decoding the P frame and the B frame, when the fault-tolerant processing of the P frame or the B frame is performed, the reference frame of the target image frame is firstly searched in the DPB, and if the reference frame is searched, the corresponding reference frame information is directly obtained; if the frame is not found, the frame closest to the playing sequence of the target image frame can be used as a reference frame of the target image frame, and the information of the frame is obtained; and then the reference frame of the target image frame is temporarily pushed out of the DPB, and the information of the reference frame can be utilized during decoding and reconstruction, so that the decoding stability is further ensured.
As a possible implementation manner, performing DPB fault-tolerant processing on the processed codestream according to the frame type of the target image frame includes: if the target image frame is an I frame, determining the playing sequence in the DPB; correspondingly, decoding and reconstructing the target image frame based on the code stream subjected to the DPB fault-tolerant processing comprises the following steps: and decoding and reconstructing the target image frame based on the code stream subjected to the DPB fault-tolerant processing and the playing sequence in the DPB.
In the embodiment of the present application, if the target image frame is an I frame, because the fault-tolerant processing of the header information is performed, the playing sequence needs to be determined again, and then decoding and reconstruction are performed, so as to ensure the stability of decoding.
As a possible implementation manner, the target image frame further includes image information; decoding and reconstructing the target image frame based on the code stream subjected to the DPB fault-tolerant processing, wherein the decoding and reconstructing process comprises the following steps: decoding the image information according to the code stream subjected to the DPB fault-tolerant processing; judging whether the code stream subjected to the DPB fault-tolerant processing has wrong image information or not; if the code stream subjected to the DPB fault-tolerant processing has wrong image information, carrying out image information fault-tolerant processing according to the error level of the wrong image information; and reconstructing the target image frame according to the code stream subjected to the image fault-tolerant processing.
In the embodiment of the application, when the target image frame is decoded, whether the code stream has the image information with errors is judged, if so, fault-tolerant processing of the image information is required, and then the target image frame is reconstructed based on the code stream subjected to the fault-tolerant processing, so that the target image frame obtained by final reconstruction is the correct image frame, and the decoding stability is improved.
As a possible implementation manner, after decoding and reconstructing the target image frame based on the code stream subjected to the DPB fault-tolerant processing, the method further includes: judging whether the current image frame to be pushed out in the DPB meets the pushing-out condition or not; if the current image frame to be pushed out meets the pushing-out condition, pushing out the current image frame to be pushed out, and detecting whether the memory occupation in the DPB is abnormal or not; if the memory occupation in the DPB is abnormal, recovering an image frame related to the memory occupied abnormally; and if the current image frame to be pushed does not accord with the pushing condition, rearranging the image frames stored in the DPB.
In the embodiment of the application, when the image frame is pushed out, whether the image frame to be pushed out meets the pushing-out condition is judged, and the DPB is subjected to related memory processing according to the judgment result, so that the size of the buffer area is limited, the situation that the memory is too large or the image data is blocked is avoided, and the decoding stability is improved.
In a second aspect, an embodiment of the present application provides another method for code stream fault tolerance, including: acquiring a code stream to be processed; the code stream to be processed corresponds to a target image frame; pushing the code stream to be processed into a DPB; and if the code stream to be processed is subjected to header information fault-tolerant processing, carrying out DPB fault-tolerant processing on the code stream to be processed according to the frame type of the target image frame.
In the embodiment of the application, for the fault-tolerant processing of the DPB, for the code stream subjected to the fault-tolerant processing of the header information, different DPB fault-tolerant processing strategies can be selected according to the frame type, so that the code stream error rate is reduced, and the decoding stability is improved.
In a third aspect, an embodiment of the present application further provides a device for code stream fault tolerance, where the device includes a functional module configured to implement the method in the first aspect and any one of possible implementation manners of the first aspect.
In a fourth aspect, an embodiment of the present application further provides another code stream fault-tolerant apparatus, where the apparatus includes a functional module for implementing the method in the second aspect.
In a fifth aspect, the present application provides a readable storage medium, on which a computer program is stored, where the computer program is executed by a computer to perform the method described in the first aspect and any one of the possible implementation manners of the first aspect and the second aspect.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.
Fig. 1 is a schematic diagram illustrating a relationship between a decoding order and a playing order according to an embodiment of the present application;
fig. 2 is a flowchart of a code stream fault tolerance method according to an embodiment of the present application;
fig. 3 is a flowchart illustrating an implementation of header fault tolerance according to an embodiment of the present application;
fig. 4 is a flowchart illustrating an implementation of a method for code stream fault tolerance according to an embodiment of the present application;
fig. 5 is a functional structure block diagram of a code stream fault-tolerant apparatus according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.
In order to more clearly introduce the code stream fault-tolerant method provided by the embodiment of the present application, before introducing the code stream fault-tolerant method, an application scenario and a related technology related to the code stream fault-tolerant method are introduced.
The code stream fault-tolerant method is applied to video decoding, and particularly can be applied to various video decoders.
Regarding video decoding, the source data amount of currently popular high definition and ultra high definition videos is quite huge, so that under the condition of limited bandwidth, a binary file obtained by encoding a video source and removing redundant information is required to be an ES (Elementary Stream), and the ES Stream reaches a decoding end after transmission and is reconstructed by video information to obtain video information which is almost the same as that of the source video. The code stream fault-tolerant method can be applied to the fault tolerance of the ES stream.
Regarding the division of video frame types, in order to remove the spatial redundancy of video images, video frames are classified during encoding, and are roughly divided into three types, I (intra frame), P (predicted frame), and B (bi-directionally coded frame), wherein the image information of P, B frames can be obtained by transforming images of adjacent frames (referred frames) in the playing order, which can be done without using a large amount of information description for each frame, but if the referred frames are in error, the images of the following P, B frames will be in error. Therefore, in order to prevent error diffusion, the video source is divided into several GOPs (Group of Picture) during encoding, so that if an error occurs in one GOP, the next GOP can continue normal decoding without being affected. Each picture is divided into several slices or regions, which follow different rules according to different video coding standards, one of the purposes of which is to prevent error diffusion.
Regarding the composition of the ES stream, the ES stream contains all information of one video, and in the video images composed of a plurality of GOPs, the images of each GOP share a group of image set parameters, including the information of the resolution, the frame rate and the like of the video images; meanwhile, each frame of image shares a group of image parameters including information such as the quantization parameter of the frame; each image divided into different strips (areas) also has a group of common parameters, including information such as the range of the determined area; besides these parameter information, the code stream also includes image specific numerical information, which is read by the decoder and reconstructed into an image.
Regarding decoding order rearrangement, the decoding order of a video should correspond to the playback order in terms of time, i.e., pictures that are played earlier should also be preferentially decoded. However, because P, B frames exist, the reference relationship of adjacent frames in the playing order determines that the referenced frame should be decoded first to decode the current frame, so for the decoding working environment, the frame decoded first is not necessarily played first, and needs to be rearranged. Specifically, as shown in fig. 1, the decoding order is different from the actual playing order, and different arrangement is provided depending on the I, P, B frame division in a GOP, so a buffer queue is required inside the decoder to store decoded pictures. The DPB (Dcode Picture Buffer, decoded Picture Buffer) is an internal Buffer queue for storing decoded pictures in the decoder.
Based on the above application scenario and introduction of related technologies, referring to fig. 2, a flowchart of a method for code stream fault tolerance provided in an embodiment of the present application is shown, where the method includes:
step 101: and acquiring a code stream to be processed. The code stream to be processed comprises the header information of the target image frame.
Step 102: and reading the header information in the code stream to be processed.
Step 103: and if the head information in the code stream to be processed is incomplete, storing the previously stored head information or preset head information into the code stream to be processed to obtain the processed code stream.
Step 104: and if the wrong head information is read, replacing the wrong head information by utilizing the previously stored head information or preset head information. And obtaining the processed code stream.
In the embodiment of the application, after the code stream to be processed is obtained, fault-tolerant processing is performed on the header information in the code stream, incomplete header information is supplemented or wrong header information is replaced through the previously stored header information or preset header information, and simple and practical code stream fault tolerance is achieved. Compared with the prior art, the error-tolerant processing of the head information is simply, quickly and practically carried out by utilizing the head information stored in advance or the preset head information aiming at the head information which is easy to generate error codes or error codes in the code stream, the error rate of the code stream subjected to the error-tolerant processing of the head information is reduced, and the decoding stability can be further improved.
The detailed implementation flow of steps 101-104 will be described next.
In step 101, the header information of the target image frame includes various types of header information, such as sequence header information, image header information, slice header information, and the like. In addition, the target image frame is a frame to be currently decoded, but is not necessarily a frame to be currently played.
It should be noted that the process of steps 101 to 104 generally follows the whole decoding process, i.e. the decoding process is performed with the fault-tolerant processing, and all that is required for decoding is to read the information in the stream bit by bit according to different standards, so in step 102, only the various header information in the stream bit by bit according to different standards is required. If the header information in the code stream is incomplete or a certain header information is wrong, fault-tolerant processing can be performed.
Further, in step 103, if the header information in the code stream to be processed is incomplete, the previously stored header information or the preset header information is stored in the code stream to be processed. The previously stored header information refers to information read and stored in the code stream last time, and the preset header information refers to preset information, which is irrelevant to the last reading and storing. In step 103, the information is incomplete, so that the fault-tolerant processing of the header information can be completed only by storing the previously stored header information or the preset header information to the incomplete position in the code stream.
In step 104, if the wrong header information is read, the wrong header information is replaced with the previously stored header information or the preset header information. In this case, the information reading is incorrect, and the previously stored header information or the preset header information needs to be replaced by the incorrect header information to complete the fault-tolerant processing of the header information.
Referring to fig. 3, it is a flow chart of implementing header information fault tolerance, which reads information according to the standard for sequence header information, image header information and stripe header information, and then determines whether reading is erroneous, where the erroneous condition includes erroneous reading and incomplete condition, and when an error occurs, the last read value is used, and if not, the preset value is used. Finally, the finally determined information needs to be stored for subsequent decoding operations, regardless of whether errors occur or not. It should be noted that, in the code stream, each kind of information is represented by a value, so that the stored information is represented by a stored value and the read information is represented by a read value in fig. 3.
After the header information is subjected to the fault-tolerant processing, the subsequent decoding process can be continued, and different fault-tolerant processing can be performed on the subsequent decoding process and different processes. Therefore, after step 103 or step 104, the method further comprises: and pushing the processed code stream into the DPB, and carrying out DPB fault-tolerant processing on the processed code stream according to the frame type of the target image frame.
When the processed code stream is pushed into the DPB, the processed code stream can be pushed into a corresponding cache position according to the cache position of the target image frame in the DPB.
In addition, if the header information has no error, the DPB fault-tolerant processing can be performed, so that in the above process, when the code stream (which may be processed by an error or may not be processed by an error) is pushed into the DPB, it can be further determined whether the code stream to be pushed into the DPB is subjected to the fault-tolerant processing of the header information, and if not, the DPB can be directly pushed; if the fault-tolerant processing of the header information is carried out, the duplication removing processing can be carried out on the information in the code stream, the header information and the image information are also contained in the code stream, and partial repeated information is modified according to other information in the code stream aiming at the header information after the fault-tolerant processing, so that a large amount of repeated information in the code stream is avoided. An alternative embodiment is: if the frame with error header information is I, P frames, the playing order of the frame is modified according to the general decoding logic when the DPB is pushed, if it is B frame, the frame is discarded, and the next frame decoding is continued.
In the embodiment of the present application, when the target image frame is a P frame or a B frame, it is described that the target image frame has a reference frame, and therefore, DPB fault-tolerant processing needs to be performed, so that any situation that fault-tolerant processing may need to be performed can be guaranteed not to be missed, and the decoding stability is further improved.
For a P frame and a B frame, performing DPB fault-tolerant processing according to a preset DPB fault-tolerant policy may include: if the target image frame is a P frame or a B frame, searching a reference frame of the target image frame in the DPB; if the reference frame of the target image frame is found in the DPB, acquiring the reference frame information of the target image frame; if the reference frame corresponding to the target image frame is not found in the DPB, taking the frame closest to the playing sequence of the target image frame in the DPB as the reference frame of the target image frame; and acquiring information of a frame closest to the playing sequence of the target image frame as reference frame information of the target image frame. Correspondingly, decoding and reconstructing the target image frame based on the code stream subjected to the DPB fault-tolerant processing comprises the following steps: suspending pushing reference frames of a target image frame out of the DPB; and decoding and reconstructing the target image frame based on the reference frame information of the target image frame and the code stream subjected to the DPB fault-tolerant processing.
After the code stream is pushed into the DPB, the reference frame of the target image frame is searched, the reference frame of the target image frame is obtained, and the process of pushing the reference frame out of the DPB is suspended, namely the DPB is regarded as DPB fault-tolerant processing of the code stream.
When the reference frame of the target image frame is searched, the reference frame can be positioned according to the information obtained by decoding in the header information, if the reference frame of the target image frame is not searched, all the decoded frames are traversed, and the frame with the playing sequence closest to the target image frame is used as the reference frame. Whether the reference frame is found or not is found, the frame with the nearest playing sequence is taken as the reference frame, and corresponding information is acquired to perform the subsequent processing process. In addition, when the frame with the latest playing sequence is taken as the reference frame of the target image frame, the frame with the latest playing sequence can also be taken as a forward or backward reference frame according to the relationship between the playing sequences of the frame with the latest playing sequence and the target image frame. For example: the playing sequence of the frame with the latest playing sequence is ahead of the target image frame, so that the frame with the latest playing sequence can be used as a forward reference frame; the frame with the latest playing order is played after the target image frame, and then the frame with the latest playing order can be used as a backward reference frame.
For an I frame, performing DPB fault-tolerant processing on the processed code stream according to the frame type of the target image frame, wherein the DPB fault-tolerant processing comprises the following steps: and if the target image frame is an I frame, determining the playing sequence in the DPB. Correspondingly, decoding and reconstructing the target image frame based on the code stream subjected to the DPB fault-tolerant processing comprises the following steps: and decoding and reconstructing the target image frame based on the code stream subjected to the DPB fault-tolerant processing and the playing sequence in the DPB. It can be understood that, for an I frame, if the header is processed in a fault-tolerant manner, part of the header may be deleted in the fault-tolerant processing, and the playing order of each frame in the DPB needs to be confirmed again. Of course, if header information processing is not performed and the target image frame is an I frame, DPB fault-tolerant processing may not be performed.
Furthermore, in the decoding and reconstruction process, if the image information in the target image frame is wrong, fault-tolerant processing can be performed on the image information. Thus, the decoding and reconstruction process may include: decoding a target image frame according to the code stream subjected to the DPB fault-tolerant processing; judging whether the code stream subjected to the DPB fault-tolerant processing has wrong image information or not; if the code stream subjected to the DPB fault-tolerant processing has wrong image information, carrying out the fault-tolerant processing on the image information according to the error level of the wrong image information; and reconstructing a target image frame according to the code stream subjected to the image fault-tolerant processing.
However, when decoding the target image frame, since the header information is already decoded at the time of the error-tolerant processing of the header information, the decoding here can be understood as decoding the image information, and similarly, the reconstruction of the target image frame can also be understood as reconstructing the image information.
When the image information is wrong, different fault-tolerant processes are selected according to the error level, and an optional implementation mode is as follows: if only the parameter in the block is wrong, limiting according to the standard parameter range; if the code stream information is not complete, determining how to complement according to the information quantity of the target image frame contained in the code stream: if the information amount exceeds half of the total information amount of the target image frame, filling the residual partial information into the information of the adjacent frame; and if the information amount is less than half of the total information amount of the target image frame, decoding is abandoned, and the reference frame or the adjacent frame reconstruction information is directly copied. For example, if the intra-block parameter is erroneous and the standard parameter range is 0-10, when the standard range is not too much out, the value of the parameter may be modified so that the modified value is limited to the range of 0-10; if the data is beyond the standard range too much, the decoding may not be successful, the data can be discarded, and the storage space corresponding to the discarded data is filled with the correction data corresponding to the data.
In the embodiment of the application, when the target image frame is decoded, whether the code stream has the image information with errors is judged, if so, fault-tolerant processing of the image information is required, and then the target image frame is reconstructed based on the code stream subjected to the fault-tolerant processing, so that the target image frame obtained by final reconstruction is the correct image frame, and the decoding stability is improved.
Further, after the decoding and reconstructing of the image information are completed, the determination of the DPB removal logic may be performed, and therefore, the method further includes: judging whether the current image frame to be pushed out in the DPB meets the pushing-out condition or not; if the current image frame to be pushed out accords with the pushing-out condition, pushing out the current image frame to be pushed out, and detecting whether the memory occupation in the DPB is abnormal or not; if the memory occupation in the DPB is abnormal, recovering the image frame related to the memory occupied abnormally; and if the current image frame to be pushed does not accord with the pushing condition, rearranging the image frames stored in the DPB.
The determination of whether the current image frame to be pushed out in the DPB meets the pushing condition may be performed by determining whether the image frame to be pushed out is consistent with the corresponding playing sequence, and if the frame to be played in the current playing sequence is not the image frame to be pushed out, it indicates that a frame lack or a multi-frame error may occur in the DPB, at this time, the DPB needs to be rearranged, for example, the corresponding playing sequence is re-determined according to information in a code stream of the frame to be pushed out, then the frame to be pushed out is arranged at a position of the corresponding playing sequence in the DPB, and the arrangement of other frames is adjusted. When the problem that the playing sequence is wrong cannot be solved, the DPB cannot play the current frame, and a forced exit can be adopted.
If there is no problem in the playing sequence, the detection of memory abnormality may also be performed, and when there is no abnormality in the memory, the playing sequence in the DPB may also be continuously checked to perform the arrangement of each playing frame. When the memory is abnormal, the error memory can be forcibly recovered, and specifically, the image frame related to the memory occupied by the abnormality can be deleted. For example, if a DPB stores frames that have been played but have not been pushed out, the part of the memory should be empty when detecting, and the image frame is detected to be still present at this time, which may be determined as a memory abnormality, then the part of the image frame may be forcibly recycled to solve the problem of abnormal occupation of the memory in order to successfully push out the next image frame.
In the embodiment of the application, when the image frame is pushed out, whether the image frame to be pushed out meets the pushing-out condition is judged, and the DPB is subjected to related memory processing according to the judgment result, so that the size of the buffer area is limited, the situation that the memory is too large or the image data is blocked is avoided, and the decoding stability is improved.
In order to make the code stream fault-tolerant method provided in the embodiment of the present application easier to understand, referring to fig. 4, an optional implementation flow of the code stream fault-tolerant method provided in the embodiment of the present application in practical application is shown in fig. 4, for image information of a code stream, it is first determined whether to adopt header information fault-tolerance, if adopting header information fault-tolerance, it is necessary to modify part of repeated information first and then put the repeated information into a DPB, and if not adopting header information fault-tolerance, it may be directly put into the DPB. Further, after the DPB is placed, different processing methods are adopted according to the frame type: when the frame is a P/B frame, the DPB fault-tolerant processing needs to be performed, which specifically includes: firstly, searching a reference frame in a DPB (differential motion bus), and if the reference frame is searched, acquiring reference frame information; if the reference frame is not found, finding the frame with the closest playing sequence of the current frame (target image frame) in the DPB as the reference frame, acquiring corresponding information, and then decoding. When the frame is an I frame, if the fault-tolerant processing of the header information is carried out, the playing sequence needs to be confirmed again, and then decoding is carried out; if the header information fault-tolerant processing is not carried out, decoding is directly carried out. After decoding, carrying out fault-tolerant processing on the image information according to the error condition of the image information, and directly reconstructing without carrying out fault-tolerant processing if no error exists. If there is error, it is divided into reading the information has error, limit the error message with the specified range; and the code stream does not contain all information, completes the information and then carries out reconstruction based on the fault-tolerant result. Finally, judging whether the DPB pushing logic is met or not, and if not, rearranging the DPB; if yes, detecting whether the memory is abnormal, and recycling part of the memory.
In the code stream fault-tolerant method described in the foregoing embodiment, both the DPB fault-tolerant and the image information fault-tolerant may be combined into the header information fault-tolerant, so as to implement a complete decoding fault-tolerant process, but in an actual application process, the DPB fault-tolerant may also be implemented as an independent code stream fault-tolerant method, so that an embodiment of the present application further provides a code stream fault-tolerant method, including: acquiring a code stream to be processed; the code stream to be processed corresponds to the target image frame; pushing the code stream to be processed into the DPB; and if the code stream to be processed is the code stream subjected to the header information fault-tolerant processing, carrying out DPB fault-tolerant processing on the code stream to be processed according to the frame type of the target image frame.
The specific implementation of each step in the method is the same as the implementation of the DPB fault tolerance process described in the foregoing embodiment, and the description is not repeated here.
In the embodiment of the application, for the fault-tolerant processing of the DPB, for the code stream subjected to the fault-tolerant processing of the header information, different DPB fault-tolerant processing strategies can be selected according to the frame type, so that the code stream error rate is reduced, and the decoding stability is improved.
Based on the same inventive concept, please refer to fig. 5, an embodiment of the present application further provides a code stream fault-tolerant apparatus 200, including: an acquisition module 201 and a header information fault-tolerant module 202.
The obtaining module 201 is configured to obtain a code stream to be processed. And the code stream to be processed comprises the header information of the target image frame. The obtaining module 201 is further configured to read header information in the code stream to be processed. A header fault tolerance module 202 configured to: if the head information in the code stream to be processed is incomplete, storing the previously stored head information or preset head information into the code stream to be processed to obtain a processed code stream; and if the wrong head information is read, replacing the wrong head information with the previously stored head information or preset head information to obtain a processed code stream.
Optionally, the code stream fault-tolerant apparatus 200 further includes: and the pushing module is used for pushing the processed code stream into the DPB. A DPB fault tolerant module to: and carrying out DPB fault-tolerant processing on the processed code stream according to the frame type of the target image frame. A decoding module to: and decoding and reconstructing the target image frame based on the code stream subjected to the DPB fault-tolerant processing.
Optionally, the DPB fault tolerance module is specifically configured to: if the target image frame is a P frame or a B frame, searching a reference frame of the target image frame in the DPB; if the reference frame of the target image frame is found in the DPB, acquiring the reference frame information of the target image frame; if the reference frame corresponding to the target image frame is not found in the DPB, taking a frame which is closest to the playing sequence of the target image frame in the DPB as a reference frame of the target image frame; and acquiring information of a frame closest to the playing sequence of the target image frame as reference frame information of the target image frame. The decoding module is specifically configured to: suspending pushing reference frames of the target image frame out of the DPB; and decoding and reconstructing the target image frame based on the reference frame information of the target image frame and the code stream subjected to the DPB fault-tolerant processing.
Optionally, the DPB fault-tolerant processing module is further specifically configured to: and if the target image frame is an I frame, determining the playing sequence in the DPB. The decoding module is specifically further configured to: and decoding and reconstructing the target image frame based on the code stream subjected to the DPB fault-tolerant processing and the playing sequence in the DPB.
Optionally, the target image frame further includes image information; the decoding module is specifically the same as: decoding the target image frame according to the code stream subjected to the DPB fault-tolerant processing; judging whether the code stream subjected to the DPB fault-tolerant processing has wrong image information or not; if the code stream subjected to the DPB fault-tolerant processing has wrong image information, carrying out image information fault-tolerant processing according to the error level of the wrong image information; and reconstructing the target image frame according to the code stream subjected to the image fault-tolerant processing.
Optionally, the DPB processing module is configured to: judging whether the current image frame to be pushed out in the DPB meets the pushing-out condition or not; if the current image frame to be pushed out meets the pushing-out condition, pushing out the current image frame to be pushed out, and detecting whether the memory occupation in the DPB is abnormal or not; if the memory occupation in the DPB is abnormal, deleting the image frame related to the memory occupied abnormally; and if the current image frame to be pushed does not accord with the pushing condition, rearranging the image frames stored in the DPB.
Based on the same inventive concept, the embodiment of the present application further provides another code stream fault-tolerant apparatus, including: the acquisition module is used for acquiring a code stream to be processed; and the code stream to be processed corresponds to the target image frame. And the pushing module is used for pushing the code stream to be processed into the DPB. A DPB fault tolerant module to: and if the code stream to be processed is subjected to header information fault-tolerant processing, carrying out DPB fault-tolerant processing on the code stream to be processed according to the frame type of the target image frame.
Each implementation manner and specific example in the code stream fault-tolerant method in the foregoing embodiments are also applicable to the two code stream fault-tolerant devices provided in the embodiments of the present application, and through the foregoing detailed description of the code stream fault-tolerant method, a person skilled in the art can clearly know an implementation method of the code stream fault-tolerant device, so for brevity of the description, detailed description is not provided here.
Based on the same inventive concept, an embodiment of the present application further provides a readable storage medium, where a computer program is stored on the readable storage medium, and when the computer program is executed by a computer, the method for code stream fault tolerance according to any of the above embodiments is executed.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
In addition, units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
Furthermore, the functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
In this document, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.
The above description is only an example of the present application and is not intended to limit the scope of the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (10)
1. A method for code stream fault tolerance is characterized by comprising the following steps:
acquiring a code stream to be processed; the code stream to be processed comprises header information of a target image frame;
reading the header information in the code stream to be processed;
if the head information in the code stream to be processed is incomplete, storing the previously stored head information or preset head information into the code stream to be processed to obtain a processed code stream;
and if the wrong head information is read, replacing the wrong head information with the previously stored head information or preset head information to obtain a processed code stream.
2. The method of claim 1, further comprising:
pushing the processed code stream into a DPB, and carrying out DPB fault-tolerant processing on the processed code stream according to the frame type of the target image frame;
and decoding and reconstructing the target image frame based on the code stream subjected to the DPB fault-tolerant processing.
3. The method of claim 2, wherein performing DPB fault-tolerant processing on the processed codestream according to the frame type of the target image frame comprises:
if the target image frame is a P frame or a B frame, searching a reference frame of the target image frame in the DPB;
if the reference frame of the target image frame is found in the DPB, acquiring the reference frame information of the target image frame;
if the reference frame corresponding to the target image frame is not found in the DPB, taking a frame which is closest to the playing sequence of the target image frame in the DPB as a reference frame of the target image frame; acquiring information of a frame closest to the playing sequence of the target image frame as reference frame information of the target image frame;
correspondingly, decoding and reconstructing the target image frame based on the code stream subjected to the DPB fault-tolerant processing comprises the following steps:
suspending pushing reference frames of the target image frame out of the DPB;
and decoding and reconstructing the target image frame based on the reference frame information of the target image frame and the code stream subjected to the DPB fault-tolerant processing.
4. The method of claim 2, wherein performing DPB fault-tolerant processing on the processed codestream according to the frame type of the target image frame comprises:
if the target image frame is an I frame, determining the playing sequence in the DPB;
correspondingly, decoding and reconstructing the target image frame based on the code stream subjected to the DPB fault-tolerant processing comprises the following steps:
and decoding and reconstructing the target image frame based on the code stream subjected to the DPB fault-tolerant processing and the playing sequence in the DPB.
5. The method of claim 2, wherein the target image frame further includes image information therein; decoding and reconstructing the target image frame based on the code stream subjected to the DPB fault-tolerant processing, wherein the decoding and reconstructing process comprises the following steps:
decoding the target image frame according to the code stream subjected to the DPB fault-tolerant processing;
judging whether the code stream subjected to the DPB fault-tolerant processing has wrong image information or not;
if the code stream subjected to the DPB fault-tolerant processing has wrong image information, carrying out image information fault-tolerant processing according to the error level of the wrong image information;
and reconstructing the target image frame according to the code stream subjected to the image fault-tolerant processing.
6. The method of claim 1, wherein after decoding and reconstructing the target image frame based on the DPB fault-tolerant code stream, the method further comprises:
judging whether the current image frame to be pushed out in the DPB meets the pushing-out condition or not;
if the current image frame to be pushed out meets the pushing-out condition, pushing out the current image frame to be pushed out, and detecting whether the memory occupation in the DPB is abnormal or not; if the memory occupation in the DPB is abnormal, recovering an image frame related to the memory occupied abnormally;
and if the current image frame to be pushed does not accord with the pushing condition, rearranging the image frames stored in the DPB.
7. A method for code stream fault tolerance is characterized by comprising the following steps:
acquiring a code stream to be processed; the code stream to be processed corresponds to a target image frame;
pushing the code stream to be processed into a DPB;
and if the code stream to be processed is subjected to header information fault-tolerant processing, carrying out DPB fault-tolerant processing on the code stream to be processed according to the frame type of the target image frame.
8. A code stream fault-tolerant apparatus, comprising:
the acquisition module is used for acquiring a code stream to be processed; the code stream to be processed comprises header information of a target image frame;
the acquisition module is also used for reading the header information in the code stream to be processed;
a header information fault tolerance module for:
if the head information in the code stream to be processed is incomplete, storing the head information stored in advance or preset head information into the code stream to be processed;
and if the wrong head information is read, replacing the wrong head information with the previously stored head information or preset head information.
9. A code stream fault-tolerant apparatus, comprising:
the acquisition module is used for acquiring a code stream to be processed; the code stream to be processed corresponds to a target image frame;
the pushing module is used for pushing the code stream to be processed into the DPB;
a DPB fault tolerant module to:
and if the code stream to be processed is subjected to header information fault-tolerant processing, carrying out DPB fault-tolerant processing on the code stream to be processed according to the frame type of the target image frame.
10. A readable storage medium, having stored thereon a computer program which, when executed by a computer, performs the method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911307289.0A CN111010575B (en) | 2019-12-17 | 2019-12-17 | Code stream fault tolerance method and device and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911307289.0A CN111010575B (en) | 2019-12-17 | 2019-12-17 | Code stream fault tolerance method and device and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111010575A true CN111010575A (en) | 2020-04-14 |
| CN111010575B CN111010575B (en) | 2022-08-02 |
Family
ID=70116554
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911307289.0A Active CN111010575B (en) | 2019-12-17 | 2019-12-17 | Code stream fault tolerance method and device and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111010575B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1200223A (en) * | 1996-09-02 | 1998-11-25 | 株式会社东芝 | Information transmitting method, encoder/decoder of information transmitting system using the method, and encoding multiplexer/decoding inverse multiplexer |
| WO2007125458A2 (en) * | 2006-04-27 | 2007-11-08 | Koninklijke Philips Electronics N.V. | Method and apparatus for encoding/transcoding and decoding |
| CN101087431A (en) * | 2006-06-07 | 2007-12-12 | 中国科学院长春光学精密机械与物理研究所 | Error code recognition and processing method of compression image |
| CN101202923A (en) * | 2006-12-15 | 2008-06-18 | 扬智科技股份有限公司 | Method for detecting, recovering and hiding image in decoder |
| CN101242540A (en) * | 2007-02-09 | 2008-08-13 | 安凯（广州）软件技术有限公司 | Error control method for video decoder chip |
| CN101299820A (en) * | 2008-06-05 | 2008-11-05 | 华为技术有限公司 | Video decoding method, apparatus as well as video repairing apparatus |
| CN102223544A (en) * | 2011-06-02 | 2011-10-19 | 福州瑞芯微电子有限公司 | Method for processing error after detecting error in H264 video stream |
| CN102469315A (en) * | 2010-11-16 | 2012-05-23 | 联芯科技有限公司 | Error recovery method and device for Moving Picture Experts Group Audio Layer IV (MPEG-4) video stream |
| CN105681801A (en) * | 2016-03-30 | 2016-06-15 | 成都芯程科技有限责任公司 | Method for realizing H.264CAVLC decoding fault tolerance function and decoder |
-
2019
- 2019-12-17 CN CN201911307289.0A patent/CN111010575B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1200223A (en) * | 1996-09-02 | 1998-11-25 | 株式会社东芝 | Information transmitting method, encoder/decoder of information transmitting system using the method, and encoding multiplexer/decoding inverse multiplexer |
| WO2007125458A2 (en) * | 2006-04-27 | 2007-11-08 | Koninklijke Philips Electronics N.V. | Method and apparatus for encoding/transcoding and decoding |
| CN101087431A (en) * | 2006-06-07 | 2007-12-12 | 中国科学院长春光学精密机械与物理研究所 | Error code recognition and processing method of compression image |
| CN101202923A (en) * | 2006-12-15 | 2008-06-18 | 扬智科技股份有限公司 | Method for detecting, recovering and hiding image in decoder |
| CN101242540A (en) * | 2007-02-09 | 2008-08-13 | 安凯（广州）软件技术有限公司 | Error control method for video decoder chip |
| CN101299820A (en) * | 2008-06-05 | 2008-11-05 | 华为技术有限公司 | Video decoding method, apparatus as well as video repairing apparatus |
| CN102469315A (en) * | 2010-11-16 | 2012-05-23 | 联芯科技有限公司 | Error recovery method and device for Moving Picture Experts Group Audio Layer IV (MPEG-4) video stream |
| CN102223544A (en) * | 2011-06-02 | 2011-10-19 | 福州瑞芯微电子有限公司 | Method for processing error after detecting error in H264 video stream |
| CN105681801A (en) * | 2016-03-30 | 2016-06-15 | 成都芯程科技有限责任公司 | Method for realizing H.264CAVLC decoding fault tolerance function and decoder |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111010575B (en) | 2022-08-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2264625C (en) | Data hiding and extraction methods | |
| US11050902B2 (en) | Video encoding and decoding | |
| JP2019165455A (en) | Network device and error handling method | |
| HUE028853T2 (en) | Reference picture signaling | |
| US20140072038A1 (en) | Reference Picture List Handling | |
| JP2010515400A (en) | Multi-view video encoding and decoding method and apparatus using global difference vector | |
| JP2010506530A (en) | Multi-view video encoding and decoding method and apparatus | |
| CN111630861A (en) | Video processing method and device | |
| EP3796652B1 (en) | Video encoding method and method for reducing file size of encoded video | |
| US20220201318A1 (en) | Arrangements and methods thereof for processing video | |
| US8767840B2 (en) | Method for detecting errors and recovering video data | |
| CN111010575B (en) | Code stream fault tolerance method and device and readable storage medium | |
| CN112004114B (en) | Video processing method and device, readable storage medium and electronic equipment | |
| JPH10164522A (en) | Data-processing method inside teletext page, and decoder | |
| US20140119445A1 (en) | Method of concealing picture header errors in digital video decoding | |
| CN116320448A (en) | Video decoding session multiplexing optimization method based on dynamic self-adaptive resolution | |
| CN104980763B (en) | Video code stream, video coding and decoding method and device | |
| US9436551B2 (en) | Method for codec-based recovery of a video using a cluster search | |
| CN109120943B (en) | Video data recovery method and device | |
| CN113747159A (en) | Method, device and related assembly for generating variable frame rate video media file | |
| CN116248884B (en) | Multi-channel video decoding method based on session multiplexing | |
| KR0152021B1 (en) | Decoding method and apparatus for error correcting of data transmitting | |
| CN115460423A (en) | Video decoding method, device, hardware decoder and storage medium | |
| CN106713916B (en) | Video image storage and reading method, device and system | |
| CN116320619A (en) | Video fault tolerance method, device, electronic equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |