WO2011015096A1

WO2011015096A1 - Method and device for analyzing code streams

Info

Publication number: WO2011015096A1
Application number: PCT/CN2010/074593
Authority: WO
Inventors: 赵新中; 陈家大; 马明刚; 万宜; 陈展; 梁炜新; 张俊; 何东岭; 谢宜昊
Original assignee: 炬力集成电路设计有限公司
Priority date: 2009-08-05
Filing date: 2010-06-28
Publication date: 2011-02-10
Also published as: CN101990094A; CN101990094B

Abstract

A device for analyzing code streams is disclosed by the present invention, comprising a decoder library consisting of more than one decoder, a component module, and a user interface module. The component module is used for selecting corresponding decoders according to inputted code streams, invoking an interface unit corresponding to the decoder library according to an operation instruction from the user interface module, receiving data from the decoder library, and outputting the data to the user interface module; the user interface module is used for providing a human-computer interaction interface which is visible for users, receiving operation instructions from the users, outputting the operation instructions to the component module, and displaying data from the component module on the human-computer interaction interface. A method for analyzing code streams is also disclosed by the present invention, which is implemented based on the device for analyzing code streams. The device for analyzing code streams provided by the present invention can support multimedia code streams with multiple types of coding standards.

Description

Code stream analysis method and device

Technical field

The present invention relates to the field of audio and video codec technology, and in particular, to a method and apparatus for code stream analysis. Background of the invention

In recent years, with the continuous innovation and development of integrated circuit technology, the consumer electronics industry has made considerable progress. One of the major functions of consumer electronics is the compatibility of multimedia content. Therefore, major consumer electronics manufacturers attach great importance to the development of multimedia decoders and encoders. In the relevant research and development process, it is very important to acquire and analyze the coded information in the multimedia code stream. This link is called code stream analysis.

Companies involved in multimedia codecs have released tools for code stream analysis, but the code stream analysis tools released by these companies are generally limited to the company's coding standards, and can only support one type or less. Several encoding format streams. The current popular coding standards belong to several companies, and no single stream analysis tool can support all common coding standards. At present, when performing code stream analysis, the code stream analysis tool is used to analyze the multimedia code stream conforming to the company A coding standard; when the multimedia code stream conforming to the company B coding standard needs to be analyzed, only another code stream can be exchanged. Analysis tools are very inconvenient to use. Summary of the invention

In view of the above, an object of the present invention is to provide a code stream analysis method and apparatus for analyzing a multimedia code stream conforming to various commonly used coding standards. A code stream analyzing apparatus according to an embodiment of the present invention includes:

Decoder library, consisting of more than one decoder and a unified interface unit, each solution The code device is configured to decode an encoded format code stream, and extract data in the code stream; the unified interface unit includes at least an identification interface unit and a parameter set information interface unit, and the identification interface unit is configured to use attribute information according to the code stream. Identifying the format of the code stream; the parameter set information interface unit is configured to output the data extracted from the code stream;

a component module, configured to invoke a unified interface unit of the decoder library according to an operation instruction from the user interface module, select a corresponding decoder by the identification interface unit of the unified interface unit to decode, and receive the decoder to pass The data output by the parameter set information interface unit of the unified interface unit outputs the data to the user interface module;

a user interface module, configured to provide a human-machine interaction interface visible to the user, receive an operation indication from the user, output the operation indication to the component module, and display data from the component module on the human-computer interaction interface.

The component module further includes: an analyzing unit, configured to analyze data from the decoder library, and output the analysis result to the user interface module.

The component module may further include: a statistical unit, configured to perform related statistical calculation on the data from the decoder library according to a predefined or user-defined formula, and output the calculation result to the user interface module.

The unified interface unit further includes:

a main information interface unit, configured to acquire a primary node in the code stream and an upper node of the primary node, and list syntax hierarchy information of the code stream; the primary node is a pre-designated nth node in the code stream syntax level, n Natural number;

The primary node decoding interface unit is configured to decode the currently selected primary node and the lower node of the primary node.

The code stream is a video code stream, and the main node is a grammatical structure node corresponding to the frame. The component module further includes: an editing module, configured to modify, according to a user instruction, header information in data from the decoder library; The unified interface unit further includes: a save file interface unit, configured to save the modified code stream according to the modification implemented by the editing module.

The unified interface unit further includes: a decoder decoder interface unit for releasing software and hardware resources used by the decoder.

The unified interface unit further includes: a decoder information interface unit, configured to provide related information in a decoder decoding process.

The user interface module further includes:

The code stream hierarchy represents an interface, and the tree structure represents the syntax structure of the code stream; and/or the data list interface, the semantics of each parameter of the code stream is given by a table; and/or the decoding result represents an interface, and the decoding result is displayed by the image. .

A code stream analysis method proposed by the embodiment of the present invention includes the following steps:

Determining a decoder matching the attribute information from more than one decoder according to attribute information of the input code stream;

Decoding the code stream with the selected decoder to extract data in the code stream; and displaying the data extracted from the code stream on a human-computer interaction interface visible to the user. The step of displaying the data extracted from the code stream on the human-computer interaction interface visible to the user includes:

The data is analyzed, and the analysis result is displayed on the human-computer interaction interface.

The step of displaying the data extracted from the code stream on the human-computer interaction interface visible to the user includes:

According to a predefined or user-defined formula, the data is subjected to relevant statistical calculation, and the calculation result is displayed on the human-computer interaction interface.

The step of decoding the code stream with the selected decoder and extracting data in the code stream includes:

Get the primary node in the code stream and the upper node of the primary node, and list the syntax of the code stream Hierarchical structure information; the primary node is a pre-designated nth node in the code stream syntax hierarchy, and n is a natural number.

The listing of the syntax hierarchy information of the code stream further includes:

The primary node selected by the user is used as the current primary node, and the lower node of the current primary node is decoded.

After the displaying the data extracted from the code stream on the human-machine interaction interface visible to the user, the method further includes:

The header information in the data is modified, and the modified code stream is saved.

It can be seen from the above technical solution that a plurality of decoders are arranged in the code stream analyzing device, and various decoders are operated by using a unified platform, and the corresponding decoder is automatically recognized for decoding by the input code stream, thereby implementing various supports. Encoding a standard multimedia stream. Furthermore, the invention adopts the on-demand decoding method, can accurately respond to the analysis requirements of the user, and greatly improves the operating efficiency of the tool; the solution of the invention can perform information statistics on the basis of analyzing the code stream data. , code stream editing and other operations, more enriched the function of code stream analysis. BRIEF DESCRIPTION OF THE DRAWINGS

1 is a structural block diagram of a code stream analysis apparatus according to an embodiment of the present invention;

2 is a schematic diagram showing a main interface displayed by the user interface module 103 of the code stream analyzing device shown in FIG. 1;

Figure 3 is a schematic diagram showing the syntax hierarchy of the mpeg4 format code stream;

4 is a flowchart of operation of a code stream analyzing apparatus according to an embodiment of the present invention;

FIG. 5 is a flowchart of code stream analysis according to an embodiment of the present invention. Mode for carrying out the invention

In order to make the objects, technical solutions and advantages of the present invention more clear, the following The invention is further elaborated.

A block diagram of a code stream analyzing apparatus according to an embodiment of the present invention is shown in FIG. 1. The code stream analyzing apparatus mainly includes a decoder library 101, a component module 102, and a user interface module 103.

The decoder library 101 includes more than one decoder, each decoder corresponding to a code stream of one format for decoding the code stream of the format and extracting data in the code stream. The data includes multimedia decoded data extracted from the code stream packet payload, and related information extracted from the code stream header. The code stream format may include, but is not limited to, H.264, mpegl, mpeg2, mpeg4, mp3, mp4, wmv, and the like. Each of the decoders provides a unified interface unit; the interface unit includes at least an identification interface unit and a parameter set information interface unit, and the identification interface unit is configured to identify a format of the code stream according to the attribute information of the code stream; the parameter set information interface unit is used for external Output data extracted from the code stream.

The component module 102 is configured to call the identification interface unit of the decoder library 101 to select a corresponding decoder, and according to an operation instruction from the user interface module 103, call the interface unit corresponding to the decoder library 101, and receive data from the decoder library 101. The data is output to the user interface module 103.

The component module 102 can provide the following functional units for analyzing, counting, and editing operations: an analyzing unit, configured to analyze the decoded data, and display related parameter information of the decoded data, for example, displaying the size of the current frame image, and encoding compression efficiency.

a statistical unit, configured to perform related statistical calculation according to the decoded data and/or the header information according to a predefined or user-defined formula, and display the calculation result in the code stream result representation interface or the data list interface, for example, the specificity in the statistical code stream The ratio of the type of frame to the total frame. Frame types include I type, P type, and B type.

Editing unit, used to modify the header information, such as changing the timestamp of the current frame (time_stamp).

The user interface module 103 is configured to provide a human-machine interaction interface visible to the user, and the code stream data is Visually showing, receiving an operation indication from a user, outputting the operation indication to the component module 102; and displaying data from the component module 102 on the human-computer interaction interface.

The main interface displayed by the user interface module 103 can be divided into three parts, namely, a code stream layer structure structure interface 201, a data list interface 202, and a decoding result display interface 203, as shown in FIG. The code stream hierarchy representation interface 201 represents the syntax structure of the code stream to be analyzed in a tree structure. The data list interface 202 provides the semantics of each parameter of the code stream in a table, and the decoding result indicates that the interface 203 displays the decoding with an image. result.

Multimedia coded data has the characteristics of high compression rate, and decoding is a process that is quite computationally consuming. The following uses the common mpeg4 format as an example to illustrate the structural characteristics of the multimedia stream. Figure 3 graphically shows the syntax hierarchy of the mpeg4 format stream according to the mpeg4 video stream standard - ISO/IEC 14496-2. In the figure, mpeg4 ASP Bitstream represents a code stream conforming to the mpeg4 ASP profile, VOSi represents the i-th visual object sequence (Visual Object Sequence) of this code stream, and VOi represents the i-th visual object in VOSi (Visual Object VOLi represents the i-th visual object layer in VOi, VOPi represents the i-th visual object plane in VOLi, and MBi represents the i-th macroblock in VOPi ( Macro Block ), SB1 to SB6 represent six sub-blocks in the MBi (Sub-Block). The multimedia data has a code stream syntax hierarchy similar to that of Figure 3, which is dictated by various compression standards.

The method adopted in the prior art generally only analyzes a part of content in the code stream, for example, the analysis content is limited to the file header and the data packet header; or, only the bit rate information of the code stream is analyzed. This way the data obtained from the code stream has great limitations. The solution of the invention can analyze all the information in the code stream, but adopts an on-demand analysis method for saving computing resources. The present invention refers to each unit on each type of multimedia data syntax structure as a "node" or "grammar node." The code stream includes nodes of multiple levels, and one of the nodes is defined as a "main node" or a "main node." For example, in a video stream, the node corresponding to the frame is called a "trunk node". Or "primary node", in Figure 3, the node corresponding to the frame is a VOP.

The decoder library contains more than one decoder, which decodes the code stream as required by the component module. The decoder library of the embodiment of the present invention includes an H.264 decoder, an mp3 decoder, an mpegl/2 decoder, and a jpeg decoder. Each decoder library is designed with multiple interface units. For each decoder, it should have the following interface unit:

Identification interface unit: Each code stream belongs to a class of coding standards, so each code stream has a unique start code to identify which coding format it belongs to. The start code is the attribute information described above. For example, the hexadecimal number 起始 is the starting code of the Visual Object Sequence, which identifies the beginning of a VOS for the mpeg4 stream. The decoder will judge whether the decoder supports decoding of the current code stream according to the encoding format supported by itself.

The main information interface unit: obtains the main node in the code stream and the upper node of the main node, and lists the syntax hierarchy information of the code stream.

Main node decoding interface unit: Decode the currently selected main node. If there is a stream syntax level below the node, decode the nodes of these levels, such as the main node - there are two layers of Macro Block and Sub-Block under the frame. node.

Parameter Set Information Interface Unit: Each node has its own different parameter set information, audio data as well as audio pulse data information, video and image data information. This interface of the decoder will provide this information.

Save file interface unit: The decoder saves the modified code stream according to user operations. Decoder information interface unit: Used to provide related information in the decoding process of the decoder, so as to implement functions such as error control and decoding state control. For example, the decoder can output an error status value when an error occurs; when the decoding takes a long time, the decoding progress can be provided through this interface.

Turn off the decoder interface unit: When replacing the decoder or turning off the analysis tool, release the software and hardware resources used by the decoder.

As can be seen from the above description, the hierarchical structure of multimedia data is complicated, and the decoding process consumes resources. In view of this, the present invention designs the operation flow of the code stream analyzing device in an on-demand manner, as shown in FIG. 4, and includes the following steps:

Step 401: Open phase: selecting, according to attribute information of the input code stream, a decoder matching the attribute information from more than one decoder; decoding the code stream by using the selected decoder, and extracting the code The data in the stream.

Step 402: Analysis phase: displaying the data extracted from the code stream on a human-computer interaction interface visible to the user.

Step 403: Close phase: Close the stream file, close the decoder, and release the resources occupied by the program. If the user edits the stream, the modified content is saved.

The process of the on-demand decoding process will be specifically described below through a specific embodiment. The specific process of the embodiment of the present invention is as shown in FIG. 5, and includes the following steps:

Step 501: Select a decoder that matches the attribute information according to the attribute information of the input code stream.

In the embodiment of the present invention, each decoder includes an identification sub-module for determining whether the attribute information of the input code stream matches the decoder. The component module selects the decoder that can support this code stream by calling the identification submodule of each decoder. After finding a decoder capable of supporting the current code stream, the component module will call the decoder to analyze the code stream, and subsequent decoding actions are performed by the decoder; if none of the decoders can support the current code stream, then Indicates that the code stream is not supported, and the corresponding prompt information is displayed through the user interface module.

Step 502: Acquire main stream information by the decoder supporting the current code stream.

The main information of the code stream can be defined by each decoder separately, and can be generally defined as: related information of the primary node and the upper node of the primary node. For example, for a video stream, the main information of the stream is defined at the level of the frame and the frame, in particular, mpeg4 ASP Bitstream The main information of the code stream is defined in four layers of VOP, VOL, VO, and VOS. This definition determines the decoding complexity when the code stream is first turned on, and is therefore one of the factors that affect the efficiency of the tool.

Step 503: Determine the current node according to the user's selection, determine whether the current node is the primary node according to the predefined primary node, and if yes, execute step 504; otherwise, go to step 505.

The specific implementation manner is as follows: The user clicks on the code stream hierarchy to represent a certain node displayed in the interface, and the user interface module sends the information of the user clicking the node to the component module, and after receiving the information of the user interface module, the component module will use The node clicked as the current node, and the decoder supporting the current code stream is called to determine whether the current node is the main node defined by the decoder, and the decoder returns the judgment result to the component module.

Step 504: The component module invokes the decoder to obtain the child node information under the current primary node. This embodiment is shown in Figure 3. The primary node will contain two layers of child nodes.

Step 505: The component module calls the decoder to decode the current code stream, and obtains the header information of the syntax node and the decoded data. Each syntax node of the code stream has corresponding header information. For example, the VOL node of the mpeg4 ASP Bitstream will have a frame type (picture_type), a timestamp (time_stamp), a width (width), a height (height), and the like. Department information. The decoded data, such as the audio code stream, is decoded by pulse code modulation waveform data, and the video code stream is decoded by image data of a certain chromaticity space.

Step 506: Perform analysis, statistics, and/or editing operations on the header information and data. When the header information and the decoded data are acquired, the header information is displayed in the form of a character string on the data list interface, and the decoded data is displayed in the form of a waveform or an image on the code stream result presentation interface.

Step 507: When the user performs the editing operation of the code stream and then requests to save the copy of the modified code stream file, the modified code stream file is saved. This step is only performed when the user performs a stream editing operation, which is an optional step.

Step 508: The user continues to click on the code stream syntax node for analysis, and the process will be transferred. Go to step 503, if the user does not click on other code stream syntax nodes, step 509 is performed. Step 509: After the user opts out, the component module will release the decoder resource, close the decoder, release the runtime resources, and exit the program.

The code stream analysis method and apparatus proposed by the present invention can analyze a plurality of audio files, video files, and image data. The code stream information is visually displayed to the user. For audio files, the sample rate, rate, and other parameters of the file can be analyzed to analyze the value of each sample point. For the video file, the encoder, frame type, and macroblock type can be read. With equal parameters, you can observe the image of each frame, the motion vector of each macroblock, the macroblock chrominance space data, and so on.

In the solution of the present invention, the architecture of the code stream analyzing device includes a user operating window, a component and a decoder library, which solves the problems of multiple decoders and multiple data types running and sharing on the same platform. The solution to this problem can improve the compatibility and functionality of the stream analysis device. The “on-demand decoding” approach to process design accurately responds to the user's analytical needs and greatly improves operational efficiency. The invention designs seven interfaces for the decoder, and unifies the decoders of different code stream formats, and provides the functions of code stream analysis, information statistics and code stream editing. The code stream analysis scheme according to the present invention supports analysis of code stream data, information statistics, and code stream editing, and is characterized by rich functions, good compatibility, and easy expansion.

Through the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus a necessary hardware platform, and of course, all can be implemented by hardware, but in many cases, the former is better. Implementation. Based on such understanding, all or part of the technical solution of the present invention contributing to the background art may be embodied in the form of a software product, which may be stored in a storage medium such as a ROM/RAM, a magnetic disk, an optical disk, or the like. A number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform the methods described in various embodiments of the present invention or portions of the embodiments.

The above is only the preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the invention are intended to be included within the scope of the invention.

Claims

Claim

A code stream analyzing device, comprising:

The decoder library is composed of more than one decoder and a unified interface unit, each decoder is configured to decode an encoded format code stream and extract data in the code stream; the unified interface unit includes at least Identifying an interface unit and a parameter set information interface unit, the identification interface unit is configured to identify a format of the code stream according to the attribute information of the code stream; and the parameter set information interface unit is configured to output the data extracted from the code stream;

The code stream analyzing apparatus according to claim 1, wherein the component module further comprises: an analyzing unit configured to analyze data from the decoder library and output the analysis result to the user interface module.

The code stream analyzing apparatus according to claim 1, wherein the component module further comprises: a statistical unit, configured to perform correlation statistics on data from the decoder library according to a predefined or user-defined formula. Calculate and output the calculation results to the user interface module.

The code stream analyzing device according to claim 1, wherein the unified interface unit further comprises:

The main information interface unit is configured to acquire the main node in the code stream and the main node a layer node, which lists syntax hierarchy information of the code stream; the primary node is a pre-designated nth node in the code stream syntax level, where n is a natural number;

The code stream analyzing apparatus according to claim 4, wherein the code stream is a video code stream, and the main node is a syntax structure node corresponding to a frame.

The code stream analyzing apparatus according to claim 1, wherein the component module further comprises: an editing module, configured to modify header information in the data from the decoder library according to a user instruction;

The unified interface unit further includes: a save file interface unit, configured to save the modified code stream according to the modification implemented by the editing module.

The code stream analyzing apparatus according to claim 1, wherein the unified interface unit further comprises: a decoder decoder interface unit for releasing software and hardware resources used by the decoder.

The code stream analyzing apparatus according to claim 1, wherein the unified interface unit further comprises: a decoder information interface unit, configured to provide related information in a decoder decoding process.

The code stream analyzing device according to any one of claims 1 to 8, wherein the user interface module further comprises:

The code stream hierarchy represents an interface, and the tree structure is used to represent the syntax structure of the code stream; and/or the data list interface, the semantics of each parameter of the code stream is given by a table; and/or

The decoding result represents an interface, and the decoded result is displayed by an image.

10. A code stream analysis method, comprising the steps of:

Determining a decoder matching the attribute information from more than one decoder according to attribute information of the input code stream; Decoding the code stream with the selected decoder to extract data in the code stream; and displaying the data extracted from the code stream on a human-computer interaction interface visible to the user.

The method according to claim 10, wherein the step of displaying the data extracted from the code stream on the human-computer interaction interface visible to the user comprises:

The method according to claim 10, wherein the displaying the data extracted from the code stream on the human-computer interaction interface visible to the user comprises:

The method according to claim 10, wherein the step of decoding the code stream by using the selected decoder to extract data in the code stream comprises:

Obtaining the primary node in the code stream and the upper node of the primary node, and listing the syntax hierarchy information of the code stream; the primary node is a pre-designated nth node in the code stream syntax hierarchy, where n is a natural number.

The method according to claim 13, wherein the listing of the syntax hierarchy information of the code stream further comprises:

The method according to claim 10, wherein after the displaying the data in the code stream is displayed on the human-machine interaction interface visible to the user, the method further includes: