CN108093258B - Code stream data decoding method, computer device and computer readable storage medium - Google Patents
Code stream data decoding method, computer device and computer readable storage medium Download PDFInfo
- Publication number
- CN108093258B CN108093258B CN201810027196.1A CN201810027196A CN108093258B CN 108093258 B CN108093258 B CN 108093258B CN 201810027196 A CN201810027196 A CN 201810027196A CN 108093258 B CN108093258 B CN 108093258B
- Authority
- CN
- China
- Prior art keywords
- code stream
- stream data
- data
- frame
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
- H04N19/423—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
- H04N19/436—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using parallelised computational arrangements
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computing Systems (AREA)
- Theoretical Computer Science (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses a decoding method of code stream data, a computer device and a computer readable storage medium, wherein the method comprises the steps that a code stream data manager establishes a code stream detection thread; the code stream data manager receives code stream data, a code stream detection thread detects the code stream data, and the starting and stopping positions of the code stream data of one frame of image in the code stream data are detected; the decoder acquires code stream data of a frame of image and decodes the received code stream data of the frame of image. The invention also provides a computer device and a computer readable storage medium for realizing the method. The invention simultaneously uses the two threads to respectively execute the detection of the code stream data and the decoding operation of the code stream data, thereby avoiding simultaneously executing the two threads on the decoder, improving the decoding efficiency of the decoder and ensuring that the video is more smoothly played.
Description
Technical Field
The present invention relates to the field of decoding of code stream data, and in particular, to a method for decoding video code stream data, a computer apparatus for implementing the method, and a computer-readable storage medium.
Background
With the development of electronic technology and network technology, video data spread via network has been widely used in industries such as network television, remote education, digital cinema, and broadcast television. Since the amount of video data is large, in order to efficiently transmit the video data, it is often necessary to encode the video data at the transmitting end of the video signal and decrypt the data at the receiving end of the video data. Therefore, the video encoding and decoding technology, as a technology for processing image information data, plays an important role in the dissemination of video data, especially in the popularization of internet televisions.
Moreover, thanks to the promotion and maturity of the video coding and decoding technology, 4K videos have gradually upgraded to become the mainstream of the set top box field, so that the promotion of the video coding and decoding technology competitiveness becomes the primary task of each set top box chip manufacturer. At present, chip manufacturers of set top boxes generally adopt a hardware acceleration engine mode to improve the performance of video coding and decoding, namely, a special hardware circuit is designed to process tasks related to coding and decoding.
For video encoding and decoding technology, the improvement of hardware technology is of fundamental importance, but complementary software technology, such as hardware driver software, is also a non-negligible part. The perfect driving software can often exert the performance of the hardware to the maximum extent, and if the design of the driving software cannot be well adapted to the characteristics of the hardware, the efficiency of video encoding and decoding is often reduced, and the playing efficiency of the video is further influenced.
For decoding technology, there are two formats of code stream data to be processed, namely frame format and stream format. The frame format data is that each data transmitted from the parsing end of the video data, such as the parser end to the decoding end, can decode a frame image, so the driver program can decode the code stream data of the frame format by directly sending the data to the decoding chip without processing the data. The parser end referred to herein is generally a module for parsing a video file to obtain compressed data of audio and video, i.e. code stream data.
Since the start position and the end position of the code stream data of each frame of image are not marked, it is unclear for the decoding chip to decode the start position and the end position of the code stream data of each frame of image, and therefore it is necessary to detect the start position and the end position of one frame of data in the code stream data by using a driver and mark the start position and the end position of the code stream data of one frame of image.
At present, in stream format code stream data, there are two methods for detecting the start and stop positions of a frame of image, one method is to detect by hardware, but the hardware detection has the defects of inaccuracy and insufficient compatibility with the scene of an error code stream; the other method is to use a driving software mode for detection, the driving software mode for detection is flexible, various code stream scenes can be compatible, and the starting position and the ending position of frame code stream data can be accurately detected.
Currently, the flow of detecting the start-stop position of the code stream data of one frame of image in the code stream data and decoding by the driver software is shown in fig. 1. After receiving the code stream data in the stream format at the decoding end, step S1 is executed first to detect the start-stop position of the code stream data of one frame of image in the code stream data, that is, detect the code stream data in the frame format.
Since the first byte of the Code stream data of a frame of image is a start mark, i.e. start Code, and the value of the start mark is usually 00000001, the Code stream data of a frame format can be determined by detecting two adjacent start marks in the Code stream data. When detecting the frame format code stream data, as shown in fig. 2, the data of each byte is detected, and whether the current byte is the start mark is determined. When the code stream data of one frame of image is detected, step S2 is executed to decode the code stream data in the frame format by using the decoder.
However, in the existing scheme, the detection of the code stream and the driving of the decoder and other enabling hardware for decoding are executed in the same thread, but the operation of detecting the code stream has the problem of long time consumption, and when the playing scene is a small-size video, the time consumed by detecting the code stream is short, and the influence on the overall performance of the decoder is small. If a large-size video with a complex scene (such as a 4K 60fps video, and the playing rate is 100 Mbps) is played, the time consumed for detecting the code stream can reach the millisecond level, and the time consumption fluctuation is large (under the condition that the load of a CPU is large, the time consumption peak value of the detected code stream reaches more than ten milliseconds), so that the overall performance of the decoder is reduced, the playing performance of the video is directly influenced, and the video playing performance is directly affected, such as the video playing jam phenomenon.
Disclosure of Invention
The invention mainly aims to provide a method for decoding code stream data, which can improve the fluency of video decoding.
Another object of the present invention is to provide a computer apparatus capable of implementing the decoding method of code stream data.
It is still another object of the present invention to provide a computer-readable storage medium capable of implementing the above decoding method of code stream data.
In order to achieve the main purpose, the decoding method of the code stream data provided by the invention comprises the steps that a code stream data manager establishes a code stream detection thread; the code stream data manager receives code stream data, a code stream detection thread detects the code stream data, and the starting and stopping positions of the code stream data of one frame of image in the code stream data are detected; the decoder acquires code stream data of a frame of image and decodes the received code stream data of the frame of image.
According to the scheme, a new thread is created in the code stream data manager to detect the code stream, and after the code stream data in the frame format is detected, the decoder acquires the code stream data in the frame format to decode, namely, the detection and decoding processes of the code stream data are respectively executed through two different threads, and the detection of the code stream data and the decoding of the code stream data can be respectively and simultaneously executed by the two threads. For example, the code stream detection thread depends on CPU resources, and the decoding thread is directly executed on the decoder, that is, depends on hardware resources of the decoder, so that detection and decoding of code stream data are respectively executed by two hardware devices, dependence on hardware resources of the decoder can be reduced, decoding efficiency is improved, and unsmooth phenomena such as jamming and the like in a video playing process are effectively avoided.
In a preferred embodiment, the acquiring, by the decoder, code stream data of one frame of image includes: the start and stop positions of code stream data of one frame of image are obtained, and code stream data between the start and stop positions are read from a buffer of a code stream data manager.
Therefore, after the code stream detection thread detects the code stream data in the frame format, the decoder can decode the data of one frame of image only by acquiring the data between the start and stop positions of the code stream data in the frame format. That is, the decoder does not need to perform a probing operation on the code stream data, thereby improving the decoding efficiency of the decoder.
The further scheme is that the code stream detection thread detecting the start-stop position of code stream data of one frame of image comprises the following steps: two adjacent start marks in the code stream data are detected, and the start-stop position in the code stream data between the two adjacent start marks is used as the start-stop position of the code stream data of one frame of image.
Therefore, the start and stop positions of the code stream data of one frame of image data can be quickly determined by detecting the positions of two adjacent start marks, for example, data between a first start mark and a byte before a next start mark is used as the code stream data of one frame of image.
In a further aspect, after detecting the start-stop position of the code stream data of one frame of image, the code stream detection thread determines whether the number of the code stream data in the frame format that is not read by the decoder reaches a threshold, and if so, suspends the operation of detecting the start-stop position of the code stream data of one frame of image.
Therefore, after the currently detected code stream data in the frame format exceeds the preset threshold, it indicates that the buffer in the code stream data manager may not be able to store a large amount of data at the start and stop positions of the code stream data in the frame format, and therefore, the detection operation of the code stream data needs to be suspended until the decoder reads the code stream data in the frame format.
The code stream detection thread detects whether an error code stream exists in the received code stream data, and if the error code stream exists, the starting and stopping positions of the error code stream are not recorded.
Therefore, once the error code stream exists in the detected code stream data, the received error code stream is discarded, so that the decoding by using the error code stream is avoided, the unnecessary operation of the decoder is avoided, and the decoding efficiency of the decoder is improved.
In a further aspect, the detecting the error code stream by the code stream detection thread includes: and the code stream detection thread determines the code stream data before the first starting mark is detected as an error code stream, or after the code stream detection thread detects the code stream data of one frame of image, the code stream data of the one frame of image is analyzed, whether the value obtained by analysis is an abnormal value is determined, and if so, the code stream data of the currently detected one frame of image is determined as the error code stream.
Therefore, the code stream before the first start mark is used as an error code stream, or after the code stream data of one frame of image is analyzed, the error code stream can be effectively detected by performing abnormity judgment on the value.
The code stream data manager judges whether an idle buffer stores the code stream data to be received before receiving the code stream data, and if the idle buffer does not exist, the code stream data is temporarily stopped from being received.
Therefore, when the code stream data manager does not have an idle buffer to receive new code stream data, the new code stream data is received in a pause mode to avoid the loss of the code stream data caused by continuous receiving of the code stream data, the code stream data can be effectively received, and the video transmission and decoding efficiency is improved.
In order to achieve the above another object, the present invention further provides a computer apparatus including a processor and a memory, wherein the memory stores a computer program, and the computer program implements the steps of the method for decoding code stream data when executed by the processor.
In order to achieve the above-mentioned still another object, the present invention further provides a computer readable storage medium having a computer program stored thereon, where the computer program is executed by a processor to implement the steps of the method for decoding code stream data.
Drawings
Fig. 1 is a flow chart of decoding of existing code stream data.
Fig. 2 is a schematic diagram of a start flag of a frame of image data in existing detection code stream data.
Fig. 3 is a flowchart of an embodiment of a method for decoding code stream data according to the present invention.
FIG. 4 is a schematic diagram illustrating operations of a detection thread and a decoding thread in an embodiment of a method for decoding codestream data according to the present invention.
The invention is further explained with reference to the drawings and the embodiments.
Detailed Description
The decoding method of the code stream data is applied to various electronic equipment with video decoding and playing functions, such as an intelligent mobile phone, a tablet personal computer or an automobile data recorder and the like. The computer device of the present invention may be the electronic device, and the computer readable storage medium may be various storage media having data storage function, including but not limited to non-volatile memories such as FLASH, EEPROM, and the like.
The main idea of the code stream data decoding method of the present invention is to move the operation of detecting the code stream by software out of the decoding thread, that is, to perform the code stream detection operation inside the code stream data manager (stream buffer manager, SBM module for short). Therefore, the method of the present invention needs to create an independent thread inside the code stream data manager, such as a code stream detection thread, which is specially used for detecting the start position and the end position of the code stream data in the frame format, when the decoding thread decodes, an interface of the code stream data manager is called to obtain the code stream data of a frame of image, and then the code stream data information of the frame of image is configured to a register of a decoder for decoding. Thus, the code stream data manager realizes code stream detection through a software program, a code stream detection thread depends on hardware resources of a CPU, a decoding thread depends on a circuit of a decoder, and the two threads run in parallel actually. Therefore, the invention distributes two tasks depending on different hardware resources to two different threads, and executes the two tasks in parallel, thereby replacing the mode of executing the two tasks in series in the prior art and reducing the whole decoding time consumption.
The embodiment of the decoding method of the code stream data comprises the following steps:
the flow of the method for decoding code stream data according to this embodiment is described below with reference to fig. 3. First, step S11 is executed, and the codestream data manager creates a codestream probe thread. In the prior art, no thread is established inside the code stream manager, and only a plurality of interfaces are provided for external calling, for example, for the parsing end of the video data, i.e., the parser module and the decoding module of the decoder, during the decoding process, the parsing end of the video data and the decoder will call the interface of the code stream data manager.
In order to detect the code stream data of the video image inside the code stream data manager, in this embodiment, a code stream detection thread needs to be created at an initialization stage of the code stream data manager, and the code stream detection thread is an internal thread of the code stream data manager.
Then, step S12 is executed to receive the code stream data and detect the start position and the end position of one frame of image in the code stream data. When the code stream data of the video file is analyzed from an analyzing end, i.e. a parser end, the analyzing end calls an interface of a code stream data manager, for example, an sbmRequestBuffer interface, obtains a buffer with a sufficient size from the code stream data manager, and then stores the code stream data to be sent into the allocated buffer, at this time, the code stream data stored into the buffer is the code stream data in a stream format, that is, the code stream data is not marked with the start position and the end position of a frame of image data.
Then, code Stream data stored in the buffer is submitted to the code Stream data manager by calling an sbmSubmitBuffer interface of the code Stream data manager, the code Stream data manager manages the code Stream data in the buffer in a first-in first-out mode, namely, a mechanism of a first-in first-out structure (sbm Stream Fifo) of the code Stream data is designed, and the code Stream data in a Stream format is managed through the mechanism.
In step S12, it is further necessary to perform detection on the stream-formatted code stream data, that is, to find the starting and ending position of a frame of image data from the stream-formatted code stream data, so as to determine the code stream data of a frame of image, so as to transmit the code stream data of a frame of image to the decoder for decoding. When the start-stop position of the code stream data of one frame of image in the code stream data in the stream format is detected, the code stream data in the frame format is obtained, that is, the start-stop position of each frame of image data is marked. Therefore, the decoder acquires the data of one frame of image for decoding each time, and a complete image can be obtained after decoding.
When the code stream data manager uses the code stream detection thread to detect the code stream data in the stream format, the code stream data manager adopts a first-in first-out mode. When detecting the code stream data in the stream format, the start and stop positions of the code stream in the frame format are detected. Since the start position of the Code stream data of each frame image has a start mark Code, and the start mark is usually one byte of data, such as binary number (00000001), the start position and the end position of the Code stream data of one frame image can be determined by searching the Code stream data in the stream format and searching two start marks.
For example, two adjacent start marks are searched, the first start mark is used as the start position of the code stream data of one frame image, the second start mark is used as the start position of the code stream data of the next frame image, that is, the last byte of the second start mark is the end position of the previous frame image. Therefore, only two start marks are searched from the code stream data in the stream format, the start and stop positions of one frame of image can be determined, and the code stream data between the start and stop positions is the code stream data in the frame format.
After a code stream detection thread in the code stream data manager acquires code stream data in a stream format, searching an initial mark in the code stream data in the stream format, if the initial mark does not exist in the current code stream data, acquiring the code stream data in the stream format again, and continuously searching the initial mark until two initial marks are searched.
After the code stream detection thread detects the start-stop position of the code stream data in the frame format, the detected start-stop position needs to be recorded in a buffer, for example, a buffer preset by a code stream data manager. Generally, the buffer is provided with a plurality of bytes for storing the start and stop positions of the code stream data in a plurality of frame formats. When the decoder needs to acquire the code stream data in the frame format, the start and stop positions of the code stream data of one frame of image can be acquired, and then the code stream data between the start and stop positions is directly read from the buffer.
However, since the buffer for recording the start-stop position of the codestream data in the frame format has a limitation of the storage capacity, for example, only 200 bytes, each byte being used from the start position or the end position of storing one frame of image data. And, after the decoder acquires the data of the start and stop positions of a certain frame image, the data of the start and stop positions of the code stream data of the frame image can be covered.
Therefore, if the code stream detection thread continues to perform the detection operation of the start and stop positions of the code stream data in the frame format without limit and detects the start and stop positions of a large amount of code stream data in the frame format, the decoder may not be able to read the code stream data in the frame format and perform the decoding operation in time, which often results in the continuous operation of the code stream detection thread, but the decoder may not read the code stream data in the frame format in time. In addition, in the playing starting stage of the player, the CPU load of the electronic device is large, and if the code stream detection thread is working all the time, the load of the CPU is increased, so that the execution efficiency of other threads is reduced, and finally the running efficiency of the player is reduced, and the playing speed of the video is slowed down.
To avoid this, the present embodiment sets a threshold, for example, set to the argument maxFrameNum, for limiting the operation of the codestream detection thread. For example, when the number of frame code streams detected by the code stream detection thread and not read by the decoder reaches the threshold, that is, the number of start and stop positions of the code stream data in the frame format detected by the code stream detection thread reaches the threshold, the code stream detection thread enters a sleep state, that is, the code stream detection thread suspends the operation of the code stream detection in the frame format until the decoder reads the detected frame code stream data, that is, after the decoder reads the start and stop positions of the detected frame code stream data in the frame format, the code stream detection thread continues to perform the detection operation of the frame code stream data in the frame format.
In addition, in a network streaming media playing scene, if the speed of network data transmission is slow, a data packet may be lost, that is, the data of the code stream transmitted from the parsing end of the video data to the code stream data manager may be incomplete, and if the incomplete data of the code stream is decoded, a complete image cannot be obtained, or a decoder cannot recognize the data of the code stream and cannot complete the decoding operation.
In order to avoid performing a decoding operation on incomplete code stream data, the incomplete code stream data also needs to be identified in this embodiment, and such incomplete code stream data is referred to as an error code stream in this embodiment. Therefore, when the code stream data manager performs the detection operation of the code stream data, it is also necessary to identify the error code stream, and after identifying the error code stream, discard the error code stream, for example, not recording the start and stop position of the error code stream.
Therefore, after step S12 is executed, step S13 is executed to determine whether the code stream data in the frame format is an error code stream, and if so, step S16 is executed to discard the data of the error code stream. The error code stream has two cases, the first case is that there is error code stream before the first start mark, that is, the code stream manager receives a section of code stream data without start mark, in this case, all code stream data before the first start mark can be determined as error code stream, so that the error data is directly discarded.
The second situation is that an error code stream exists before the first start mark and the second start mark, and in this situation, whether the error data exists is judged by analyzing the value of the code stream data, and if the analyzed value is abnormal, the error data is considered to be the error data, and the error data is discarded. Since the data between the two start flags should be the data of one frame of image, the code stream data is required to satisfy the specification of the H265 standard since it is required to comply with a predetermined rule, for example, video data encoded based on the H265 standard, at the time of encoding and decoding of an image. For example, in a frame of image data, some positions are used for storing the data length of the current frame, when analyzing the code stream data, it may be calculated whether the data length in the code stream data in the current frame format is equal to the value stored in the frame, and if not, the value is considered to be abnormal, that is, it is determined that the data in the current frame format is abnormal data.
Or, the data in frame format includes check code, when the data in frame format is analyzed, check calculation is performed on the code stream data, whether the calculated check code is consistent with the check code stored in the data in frame format is judged, and if not, the current data in frame format is considered to be error code stream.
If the code stream data of the current frame format is determined to be an error code stream, directly discarding the code stream data of the current frame format, if the start-stop position of the code stream data of the current frame format is not recorded, so that the decoder cannot acquire the code stream data of the frame format, and the resource waste of the decoder is avoided.
After a certain frame of code stream data is lost, the sending end of the video data needs to resend the frame data, and the code stream data manager detects the code stream data again after acquiring new code stream data. Therefore, the playing of the video is not affected after the error code stream is discarded.
If the current code stream data is determined not to be the error code stream, step S14 is executed to record the start and stop position of the code stream data in the current frame format, for example, record the start and stop position of the code stream data in the current frame format in the buffer. Finally, step S15 is executed, in which the decoder acquires start and stop positions of the code stream data of one frame of image data from the code stream data manager, acquires code stream data between the start and stop positions, and decodes the code stream data of one frame of image, thereby acquiring a decoded image.
It can be seen that, in the prior art, a decoding thread is run on a decoder, the decoding thread calls a requestStreamData interface of a stream data manager to acquire stream data in a stream format, then a start position and an end position of a frame of image data are detected by searching a start mark in the stream data in the stream format, then information of the stream data of the frame of image is configured to a hardware register, and decoding is performed through an enabling device.
In the embodiment, the operation of detecting the code stream data is moved out of the decoding thread, the decoding thread directly calls a requestFrameData interface of the code stream data manager to acquire the code stream data in a frame format, and then the code stream data information in the frame format is configured to the hardware register and decoded by enabling hardware, so that the code stream detecting thread and the decoding thread are two threads which operate independently, and the code stream detecting thread and the decoding thread are prevented from operating on a decoder at the same time.
The following specifically describes the decoding process of the encoded stream data in this embodiment, and it should be noted that, the steps described below do not necessarily need to be executed in time sequence, and some steps may be executed simultaneously.
The first step is as follows: initializing a code stream data manager, and allocating resources required by the code stream data manager, such as applying for sufficient node memory space, applying for a sufficiently large buffer for storing code stream data in a stream format, and the like for a stream format data first-in first-out structure sbmStreamFifo and a frame format data first-in first-out structure sbmFrameFifo. In this embodiment, the node memory space may be a buffer for storing the start-stop position of the frame format data.
The second step is that: and establishing an internal thread of the code stream data manager, namely a code stream detection thread, and starting the code stream detection thread to enable the code stream detection thread to be in a running state. Since the parsing end of the video data, i.e. the parser module and the decoder, need to call the interfaces of the bitstream data manager, and the calls of the interfaces have no limitation, there are actually three threads, i.e. the parser module, the decoder, and the bitstream detection thread, which can manipulate the member variables of the bitstream data manager. If a plurality of threads operate on the variables, the interaction between the threads is possible, and therefore the mutual influence between the threads must be ensured by designing a mutual exclusion lock. The mutex lock may be a hardware lock, and the setting of the hardware lock may be performed by using an existing hardware lock setting technology, which is not described herein again.
The third step: the analysis section of the video data analyzes the data of the video file, separates the code stream data of the video from the video file data, calls an sbmRequestStreamBuffer interface of a code stream data manager according to the size of the code stream data, obtains a buffer with the same size, and copies the code stream data into the buffer. And then, calling an sbmSubmitStreamData interface of the code stream data manager to configure the code stream data in the buffer to a node of the sbmDaturamFifo of the code stream data manager.
When the parsing end of the code stream data calls the sbmRequestStreamBuffer interface of the code stream data manager, there may be a case that an idle buffer cannot be obtained, because the storage space of the buffer for the existence of the code stream data, which is applied by the code stream data manager, is limited, and the management node of sbmstreamfiffo in the code stream data manager is also limited, if the parsing speed of the parsing end is faster, it may be possible that the buffer of the code stream data manager is filled or the node of sbmstreamfiffo is used up.
Therefore, before the code stream data manager receives the code stream data, it needs to determine whether there is an idle buffer currently storing the code stream data to be received, and if there is no idle buffer currently, it needs to suspend receiving the code stream data, that is, the parsing end of the video data needs to be in a sleep state, or wait until the idle buffer can be obtained, and then the parsed code stream data can be continuously sent to the code stream data manager.
When the decoder reads the code stream data from the code stream data buffer, the read code stream data can be covered, at the moment, the read code stream data buffer is changed into an idle buffer immediately, and the analysis end of the video data can write the code stream data into the idle buffer.
The fourth step: the code stream detection thread of the code stream data manager performs operations of detecting code stream data in a frame format, for example, acquiring streamData1 of a code stream in a stream format from an sbmStreamFifo node, acquiring a management node frameDataInfoNode from the sbmFrameFifo node, searching the acquired streamData streamDataInfode 1, thereby searching out the position of a startCode, and recording the position information of the searched start marker in frameDataInfoNode. Then continue to get the next codestream data streamData2 from sbmStreamFifo and search for the second start marker.
And then circularly performing the third step and the fourth step to obtain the positions of a plurality of initial marks.
The fifth step: initializing the decoding module, distributing the resource needed by the decoding module, operating the hardware circuit of the decoder and starting the decoding function.
And a sixth step: and calling an sbmRequestFrameData interface of the code stream data manager to acquire code stream data in a frame format.
The seventh step: and (4) performing syntax analysis on the acquired code stream data, such as analyzing the type of nalu, analyzing reference frame information to update a reference frame queue and the like.
Eighth step: and configuring necessary values such as code stream data information in a frame format, parameters analyzed by the seventh step, address information of an output buffer and the like to a corresponding hardware register, triggering an enabling bit of the hardware to decode the code stream data information, triggering the hardware to enable the hardware, and enabling a decoding thread to enter a dormant state until the hardware sends an interrupt signal for finishing decoding. After the decoding operation of the decoder is completed, it means that the current code stream data is used up, so that the sbmReturnFrameData interface of the code stream data manager needs to be called, and the buffer occupied by the acquired code stream data is set to be in an idle state.
And then, circularly performing the sixth step, the seventh step and the eighth step until all code stream data are decoded.
It can be seen that the first step to the fourth step are actually the threads for executing the code stream detection by the code stream data manager, i.e. the left part in fig. 4, to convert the code stream data in the stream format into the code stream data in the frame format. And the fifth step to the eighth step are that the decoder executes a decoding thread, that is, the right part of fig. 4, that is, executes step S21, acquires code stream data in the frame format, and executes step S22 to decode the code stream data in the frame format by the decoder. It can be seen that the two threads are running independently of each other, i.e. synchronously.
For example, this embodiment is applied to a video with a playback format of H265 and a resolution of 1920 × 1080, where the code stream data manager applies, in an initialization stage, a buffer with a size of 8 MB for storing the code stream data in the stream format, and applies 2048 sbmStreamFifo nodes and 100 sbmFrameFifo nodes for managing information of the code stream data in the stream format and the code stream data in the frame format. And the analysis end of the video data analyzes the code stream data of H265 from the video file, copies the code stream data into a buffer of a code stream data manager, and distributes the buffer information to an sbmStreamFifo node.
And detecting the code stream data in the flow format by an internal thread of the code stream data manager, namely a code stream detection thread, and distributing the detected information of the code stream data in the frame format to the sbmFrameFifo node. Then, the H265 decoder obtains the code stream data of a frame of image from the sbmFrameFifo node of the code stream data manager, configures the information of the buffer of the code stream data of the frame of image to the hardware register, starts the H265 decoding hardware to decode, and returns the information of the sbmFrameFifo node to the code stream data manager after the decoding is completed.
Therefore, through the two threads which are executed synchronously, the decoding efficiency of the decoder can be reduced and improved, and the video can be played more smoothly.
The embodiment of the computer device comprises:
the computer device of the embodiment includes a processor, a memory, and a computer program stored in the memory and executable on the processor, such as an information processing program for implementing the information processing method. When the processor executes the computer program, the steps of the code stream data decoding method are realized.
For example, a computer program may be partitioned into one or more modules that are stored in a memory and executed by a processor to implement the modules of the present invention. One or more of the modules may be a series of computer program instruction segments capable of performing certain functions, which are used to describe the execution of the computer program in the terminal device.
It should be noted that the terminal device may be a desktop computer, a notebook, a palm computer, a cloud server, or other computing devices. The terminal device may include, but is not limited to, a processor, a memory. It will be understood by those skilled in the art that the schematic diagram of the present invention is merely an example of a terminal device, and does not constitute a limitation of the terminal device, and may include more or less components than those shown, or combine some components, or different components, for example, the terminal device may further include an input-output device, a network access device, a bus, etc.
The Processor may be a Central Processing Unit (CPU), or may be other general-purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, a discrete hardware component, or the like. The general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like, the processor being the control center of the terminal device and connecting the various parts of the entire terminal device using various interfaces and lines.
The memory may be used to store computer programs and/or modules, and the processor may implement various functions of the terminal device by running or executing the computer programs and/or modules stored in the memory and invoking data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. In addition, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other volatile solid state storage device.
A computer-readable storage medium:
the module integrated with the terminal device may be stored in a computer-readable storage medium if it is implemented in the form of a software functional unit and sold or used as a separate product. Based on such understanding, all or part of the flow in the method according to the above embodiments may also be implemented by a computer program, which may be stored in a computer readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps of the decoding method for each code stream data.
Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying computer program code, recording medium, U.S. disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution media, and the like. It should be noted that the computer readable medium may contain other components which may be suitably increased or decreased as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, in accordance with legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunications signals.
Of course, the above-mentioned solution is only a preferred embodiment of the present invention, and the practical application may be more varied, for example, the size of the buffer and the number of each node may be adjusted according to the practical situation, and these changes do not affect the implementation of the present invention, and should be included in the protection scope of the present invention.
Claims (9)
1. The decoding method of the code stream data is characterized by comprising the following steps:
a code stream data manager establishes a code stream detection thread;
the code stream data manager receives code stream data, the code stream detection thread detects the code stream data, and the starting and stopping position of the code stream data of one frame of image in the code stream data is detected;
the decoder acquires code stream data of a frame of image, and decodes the received code stream data of the frame of image: the method comprises the steps of obtaining starting and stopping positions of code stream data of one frame of image, and reading the code stream data between the starting and stopping positions from a buffer of a code stream data manager.
2. The method for decoding codestream data according to claim 1, characterized in that:
the code stream detection thread detecting the start-stop position of code stream data of one frame of image comprises the following steps: and detecting two adjacent start marks in the code stream data, and taking the start-stop position in the code stream data between the two adjacent start marks as the start-stop position of the code stream data of one frame of image.
3. The method for decoding codestream data according to claim 1, characterized in that:
after the code stream detection thread detects the start and stop position of the code stream data of one frame of image, whether the number of the code stream data in the frame format which is not read by the decoder reaches a threshold value is judged, and if so, the operation of detecting the start and stop position of the code stream data of one frame of image is suspended.
4. The method for decoding codestream data according to claim 2, characterized in that:
and the code stream detection thread detects whether an error code stream exists in the received code stream data, and if the error code stream exists, the starting and stopping positions of the error code stream are not recorded.
5. The method for decoding codestream data according to claim 4, characterized in that:
the code stream detection thread detecting the error code stream comprises: and the code stream detection thread determines the code stream data before the first starting mark is detected as an error code stream.
6. The method for decoding codestream data according to claim 4, characterized in that:
the code stream detection thread detecting the error code stream comprises: and after the code stream detection thread detects the code stream data of one frame of image, analyzing the code stream data of the one frame of image, and determining whether the value obtained by analysis is an abnormal value, if so, determining that the code stream data of the currently detected one frame of image is an error code stream.
7. The method for decoding codestream data according to claim 1, characterized in that:
before the code stream data manager receives the code stream data, whether an idle buffer stores the code stream data to be received or not is judged, and if no idle buffer exists, the code stream data is temporarily stopped from being received.
8. Computer arrangement, characterized in that it comprises a processor and a memory, said memory storing a computer program that, when executed by the processor, implements the steps of a method for decoding codestream data according to any one of claims 1 to 7.
9. A computer-readable storage medium having stored thereon a computer program, characterized in that: the computer program, when executed by a processor, implements the steps of a method for decoding codestream data according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810027196.1A CN108093258B (en) | 2018-01-11 | 2018-01-11 | Code stream data decoding method, computer device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810027196.1A CN108093258B (en) | 2018-01-11 | 2018-01-11 | Code stream data decoding method, computer device and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108093258A CN108093258A (en) | 2018-05-29 |
| CN108093258B true CN108093258B (en) | 2020-06-16 |
Family
ID=62182839
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810027196.1A Active CN108093258B (en) | 2018-01-11 | 2018-01-11 | Code stream data decoding method, computer device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108093258B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110213650B (en) * | 2019-05-17 | 2022-02-15 | 芯原微电子（成都）有限公司 | Method and device for rapidly searching synchronous words in video code stream by software |
| CN113923507B (en) * | 2021-12-13 | 2022-07-22 | 北京蔚领时代科技有限公司 | Low-delay video rendering method and device for Android terminal |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101409831A (en) * | 2008-07-10 | 2009-04-15 | 浙江师范大学 | Method for processing multimedia video object |
| CN101640805A (en) * | 2008-07-28 | 2010-02-03 | 青岛海信信芯科技有限公司 | Video decoding method and video decoder |
| CN102196268A (en) * | 2011-05-06 | 2011-09-21 | 成都市华为赛门铁克科技有限公司 | Method, device and system for processing multimedia data |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8411734B2 (en) * | 2007-02-06 | 2013-04-02 | Microsoft Corporation | Scalable multi-thread video decoding |
| US7933937B2 (en) * | 2008-02-08 | 2011-04-26 | Oracle America, Inc. | System and method for asynchronous parallel garbage collection |
-
2018
- 2018-01-11 CN CN201810027196.1A patent/CN108093258B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101409831A (en) * | 2008-07-10 | 2009-04-15 | 浙江师范大学 | Method for processing multimedia video object |
| CN101640805A (en) * | 2008-07-28 | 2010-02-03 | 青岛海信信芯科技有限公司 | Video decoding method and video decoder |
| CN102196268A (en) * | 2011-05-06 | 2011-09-21 | 成都市华为赛门铁克科技有限公司 | Method, device and system for processing multimedia data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108093258A (en) | 2018-05-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110312156B (en) | Video caching method and device and readable storage medium | |
| US11336953B2 (en) | Video processing method, electronic device, and computer-readable medium | |
| US10241799B2 (en) | Out-of-order command execution with sliding windows to maintain completion statuses | |
| CN108377394B (en) | Image data reading method of video encoder, computer device and computer-readable storage medium | |
| CN104837052A (en) | Playing method of multimedia data and device | |
| US20080291209A1 (en) | Encoding Multi-media Signals | |
| CN105451071A (en) | Video stream processing method, device and system | |
| CN108093258B (en) | Code stream data decoding method, computer device and computer readable storage medium | |
| US8667080B2 (en) | Computer-implemented method, a computer program product and an embedded system for increasing the speed of data retrieval by a client on the embedded system | |
| CN110225399B (en) | Streaming media processing method and device, computer equipment and storage medium | |
| CN114339378A (en) | Audio and video code stream processing method and device and electronic equipment | |
| US7725905B1 (en) | Media accelerator interface API | |
| US20240143521A1 (en) | Peripheral device with embedded video codec functionality | |
| US8543612B2 (en) | Mobile terminal | |
| US10630809B2 (en) | Information processing apparatus, image processing apparatus and control methods thereof | |
| US10477245B2 (en) | Methods and devices for coding and decoding depth information, and video processing and playing device | |
| CN115567754A (en) | Video playing method, device, equipment and storage medium | |
| CN113747171B (en) | Self-recovery video decoding method | |
| US7154948B2 (en) | Video codec system, method for processing data between system and host system, and encoding/decoding control method in the system | |
| TW201404165A (en) | Coding of video and audio with initialization fragments | |
| CN117478958B (en) | Video playing method, device, electronic equipment and storage medium | |
| CN109743521B (en) | Video data transmission method and device, electronic equipment and storage medium | |
| CN115834949A (en) | Method and device for extracting program information, computer readable storage medium and terminal | |
| CN115942000B (en) | H.264 format video stream transcoding method, device, equipment and medium | |
| CN114666514B (en) | Data processing method, device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220412 Address after: 510000 unit 904, building a, No. 18, science Avenue, Science City, Huangpu District, Guangzhou, Guangdong Patentee after: Guangzhou xinzhilian Technology Co.,Ltd. Address before: 519080 No. 9 science and technology two road, Tangjia Bay, Zhuhai high tech Zone, Guangdong Patentee before: ALLWINNER TECHNOLOGY Co.,Ltd. |
|
| TR01 | Transfer of patent right |