WO2008048010A1

WO2008048010A1 - Device and method for encoding/decoding bit-stream

Info

Publication number: WO2008048010A1
Application number: PCT/KR2007/004861
Authority: WO
Inventors: Euee-S. Jang; Sun-Young Lee; Chung-Ku Lee; Eun-Kyung Kwak
Original assignee: Humax Co., Ltd.
Priority date: 2006-10-18
Filing date: 2007-10-05
Publication date: 2008-04-24
Also published as: KR20080035220A; KR100841558B1

Abstract

A bitstream encoding/decoding method and device is provided. A decoding device includes; a tool box comprising 2 or more of functional units each implemented to perform independently its process according to one or more of decoding standards for decoding a bitstream; a hierarchical information storing unit storing hierarchical information to operate systemically a part of or the whole of the functional units once or more times; and a decoding control unit activating one or more of functional units included in the highest layer by referring to the hierarchical information. Here, when processing data for performing one or more functional units in a lower layer is stored corresponding to the hierarchical information in the information storing unit, the lower layer is called by an upper layer. According to the present invention, it allows scheduling management of each codec using the decoder description and systematic processing of each functional unit.

Description

[DESCRIPTION]

[Invention Title] DEVICE AND METHOD FOR ENCODING/DECODING BIT-STREAM

[Technical Field]

The present invention relates to a unified codec and more particularly, to a method and device for encoding/decoding bitstream.

[Background] Typically, video data is converted into bitstream data by an encoder. At this time, the bitstream is stored depending on an encoding type that satisfies the constraint condition of the encoder.

MPEG, which is the constraint condition of the bitstream, requests syntax and semantics. The syntax, which refers to the structure, format, or length of data, shows the sequence of expressing the data. In other words, the syntax is to meet a rule for encoding/decoding and defines the sequence, length, format, and the like of each element in the bitstream.

The semantics refers to the meaning of each bit forming data. In other words, the semantics shows the meaning of each element in the bitstream. Accordingly, various types of bitstreams can be generated depending on the encoding condition or the applied standard (or codec) of the encoder. Typically, each standard (e.g. MPEG-I, MPEG-2, MPEG-4, and MPEG-4 AVC, etc.) has different bitstream syntax. Therefore, it can be said that the bitstream encoded according to each standard or encoding condition has different types (i.e. syntax and semantics). A decoder corresponding to a pertinent encoder may be used for the deciding of the bitstream.

As described above, the conventional bitstream decoder has a restriction that must satisfy the constraint condition of the encoder. This restriction makes it difficult to implement a united decoder corresponding to a plurality of standards. [Disclosure] [Technical Problem]

Accordingly, the present invention, which is designed to solve the aforementioned problems, provides a method and device for decoding a bitstream that is encoded by various types (syntax and semantics) in accordance with each standard (e.g. MPEG-I, MPEG-2, MPEG-4, MPEG-4 AVC) by using the same information recognizing method.

Further, the present invention provides a method and device for encoding a bitstream that can generate an extended bitstream including decoder descriptions to decode bitstreams encoded by various types (syntax and semantics) in accordance with each standard by using the same information recognizing method.

Further, the present invention provides a method and device for decoding a bitstream that can parse a bitstream compressed by various encoding methods by using the same information analyzing method and systematically control each functional unit for decoding by using the parsed data.

Further, the present invention provides a method and device for encoding a bitstream that can describe decoder descriptions more efficiently by employing hierarchical structure of codec to the syntax parsing and the decoding process.

Further, the present invention provides a method and device for decoding a bitstream that can suggest scheduling management of each codec and systematical processing structure (i.e., parallel combination structure, sequential combination structure, independent processing structure, individual processing structure, etc.) of each functional unit by using the decoder description.

Further, the present invention provides a method and device for encoding a bitstream that can design and build various systems only with the above described decoder descriptions or with hierarchical information.

Further, the present invention provides a method and device for encoding/decoding a bitstream that can universally apply a syntax analysis method to decode various types of bitstreams. Further, the present invention provides a method and device for encoding/decoding a bitstream that can apply a new set of commands to parse various types of bitstreams by a common syntax analysis method.

Further, the present invention provides a method and device for decoding a bitstream that can decode bitstreams even when a syntax element is changed, added, or deleted.

Further, the present invention provides a method and device for decoding a bitstream that can share elements used for the bitstream decoding of the element information (i.e. a result from syntax parsing) of analyzed syntax.

Further, the present invention provides a method and device for decoding a bitstream that can use the element information of analyzed syntax for analyzing the syntax element of a following bitstream.

Further, the present invention provides a method and device for decoding a bitstream that can separate functions, which are included during various decoding processes suggested by several standards (codec), corresponding to each functional unit and provide the result into a tool-box.

Further, the present invention provides a method and device for decoding a bitstream that can selectively use only necessary functional units from the tool box to decode a bitstream encoded by various types.

Further, the present invention provides a method and device for decoding a bitstream that can allow easy changing, inserting, or deleting functional units stored in the tool box.

Further, the present invention is to aim at international standardization of codec unification for the bitstream decoding, generation of decoder descriptions to be processed by the same information analyzing method and establishment of extended bitstream.

Other problems that the present invention solves will become more apparent through the following description.

[Technical solution] To solve the above problems, an aspect of the present invention features a decoder and/or a unified codec device that can be universally used for various standards.

According to an embodiment of the present invention, there is providing a decoding device that includes: a tool box composed with 2 or more of functional units each implemented to independently perform each process according to one or more of decoding standards for the bitstream decoding; a hierarchical information storing unit storing hierarchical information to systemically operate a part or entire of the function units once or more times; and a decoding control unit controlling an activation of one or more functional units belonging to the highest layer by referring to the hierarchical information. Here, a call between layers can be performed corresponding to the hierarchical information. The decoding control unit may control object creation and execution for the highest layer and activate a calling unit to call a layer designated by the hierarchical information and the functional units included in the highest layer by the object execution. The calling unit can activate a pertinent layer when processing data, requested for one or more of functional units included in a correspondingly determined layer to perform a predetermined process, is stored in the information storing unit.

The functional unit included in the activated layer starts operation when the processing data to perform a predetermined process is stored in the information storing unit.

The result data by the execution of each functional unit may be stored in the information storing unit.

The hierarchical information stored in the hierarchical information storing unit can be provided independently from the bitstream. Also, a united bitstream may include data corresponding to the hierarchical information and the bitstream and in this case, the decoding device may further include a dividing unit to extract the data.

The decoding device may further include a description decoder to convert the data to the hierarchical information.

The tool box may include one or more of parsing units for syntax parsing of the bitstream and 2 or more of decoding functional units decoding the encoded video data. The decoding functional units each may start operation when control signal and context information (CSCI) required for performing a predetermined process and data for decoding are stored.

A dedicated storage space may be assigned for each of the decoding functional units.

The information storing unit may include a control signal and context mformation(CSCI) storing unit where the control signal and context information(CSCI) generated by the parsing functional unit is stored; and a data storing unit where data for decoding processing, which is at least one of data corresponding to the encoded video data generated by the parsing functional unit and data processed by the decoding functional unit, is stored.

The hierarchical structure corresponding to the hierarchical information may be composed of 2 or more layers chosen from a sequence layer, a GOP layer, a picturing layer, a slice layer, a macro block layer and a block layer. The hierarchical information stored in the hierarchical information storing unit is information about hierarchical structure to decode the encoded video data, specification of a layer to be called in each layer, and a sequence of call or activation of 2 or more functional units based on the hierarchical structure.

To solve the above problems, another aspect of the present invention features a decoding method that can be universally used for various standards and/or a recording medium recorded with a program for executing the method.

A decoding method according to an embodiment of the invention may include: receiving a bitstream; and activating at least one functional unit belonging to the highest layer by referring to stored hierarchical information for decoding the bitstream. Here, 2 or more of functional units each, implemented to perform independently a process corresponding to one or more of decoding standards, is included to one of 2 or more layers, the hierarchical information is specific information to systemically operate a part or entire of the functional units once or more times, and a call between layers is performed corresponding to the hierarchical information.

The step of activating the one or more functional unit belonging to the highest layer controls object creation and execution of the highest layer, and activates the functional units included in the highest layer by the object execution and a calling unit to call another layer designated by the hierarchical information. The calling unit may activate a pertinent layer when processing data, requested for one or more of functional units included in the determined layer to perform a predetermined process, is stored in the information storing unit.

The functional unit included in the activated layer may start operation when the processing data to perform a predetermined process is stored in the information storing unit. The result data by the performance of the functional units may be stored in the information storing unit.

The hierarchical information stored in the hierarchical information storing unit may be provided independently from the bitstream. Also, a united bitstream may include data corresponding to the hierarchical information and the bitstream, and in this case, extracting the data may be preceded in order to store the hierarchical information.

The decoding method may further include converting the data to the hierarchical information in order to store the hierarchical information.

The functional units for the bitstream decoding may include one or more parsing functional units for syntax parsing of the bitstream and 2 or more of decoding functional units for decoding the encoded video data.

The decoding functional units each may start operation when control signal and context information(CSCI) required for performing a predetermined process and data for decoding are stored. A dedicated storage space may be assigned for each of the decoding functional units.

The information storing unit may include a control signal and context information(CSCI) storing unit where the control signal and context information(CSCI) generated by the parsing functional unit is stored; and a data storing unit where data for decoding processing, which is at least one of data corresponding to the encoded video data generated by the parsing functional unit and data processed by the decoding functional unit, is stored

The hierarchical structure corresponding to the hierarchical information may be composed of 2 or more layers chosen from a sequence layer, a GOP layer, a picturing layer, a slice layer, a macro block layer and a block layer.

The hierarchical information may be information about hierarchical structure to decode the encoded video data, specification of a layer to be called in each layer, and a sequence of call or activation of 2 or more functional units based on the hierarchical structure. According to another embodiment of the present invention, there is providing a recording medium recorded with a program wherein the recording medium is implemented with a program of commands executable in the decoding device and is recorded with a program readable by the decoding device, the recording medium recorded with a program including: receiving a bitstream; and activating at least one functional unit belonging to the highest layer by referring to stored hierarchical information for decoding the bitstream. Here, 2 or more of functional units each, implemented to perform independently a process corresponding to one or more of decoding standards, is included to one of 2 or more layers, the hierarchical information is specific information to systemically operate a part or entire of the functional units once or more times, and when processing data for performing one or more functional units in a first layer, which is a deactivated state, is stored corresponding to the hierarchical information in the information storing unit, the first layer is called by a second layer which is in an activation state.

[ Brief Description of the Drawings ]

FIG. 1 is a schematic block diagram illustrating the structure of a typical decoder.

FIG. 2 is a schematic block diagram illustrating the structure of a typical encoder. FIG. 3 is a schematic block diagram illustrating the structure of a decoder in accordance with an embodiment of the present invention.

FIG. 4 is a schematic block diagram illustrating the structure of an extended bitstream in accordance with an embodiment of the present invention.

FIG. 5 is a schematic block diagram illustrating the structure of a decoding unit in accordance with an embodiment of the present invention.

FIG. 6 is a schematic block diagram illustrating the structure of a syntax(S YN) parser in accordance with an embodiment of the present invention.

FIG. 7 is a schematic block diagram illustrating the structure of a decoding processing unit in accordance with an embodiment of the present invention. FIG. 8 illustrates the hierarchical structure assigning a decoding hierarchy table(DHT) in accordance with an embodiment of the present invention.

FIG. 9 illustrates the calling structure between layers and the connection structure within layers assigned by a syntax rule table(S-RT) in accordance with an embodiment of the present invention. FIG. 10 illustrates an interface set for functional units applied to 2 or more of standards in accordance with an embodiment of the present invention.

FIG. 11 illustrates the connection structure of a dedicated buffer space between nodes per each hierarchy in accordance with an embodiment of the present invention.

FIG. 12 illustrates the inclusion relation of a dedicated buffer space for storing control signal and context information (CSCI) data in accordance with an embodiment of the present invention.

FIG. 13 illustrates the inclusion relation of a dedicated buffer space for storing output data by functional units in accordance with an embodiment of the present invention. FIG. 14 illustrates a parallel processing of syntax parsing and data decoding in accordance with an embodiment of the present invention.

FIG. 15 is a block diagram illustrating an encoder in accordance with an embodiment of the present invention. [Mode for Invention]

The above objects, features and advantages will become more apparent through the below description with reference to the accompanying drawings.

Since there can be a variety of permutations and embodiments of the present invention, certain embodiments will be illustrated and described with reference to the accompanying drawings. This, however, is by no means to restrict the present invention to certain embodiments, and shall be construed as including all permutations, equivalents and substitutes covered by the spirit and scope of the present invention. Throughout the drawings, similar elements are given similar reference numerals. Throughout the description of the present invention, when describing a certain technology is determined to evade the point of the present invention, the pertinent detailed description will be omitted.

Terms such as "first" and "second" can be used in describing various elements, but the above elements shall not be restricted to the above terms. The above terms are used only to distinguish one element from the other. For instance, the first element can be named the second element, and vice versa, without departing the scope of claims of the present invention. The term "and/or" shall include the combination of a plurality of listed items or any of the plurality of listed items.

When one element is described as being "connected" or "accessed" to another element, it shall be construed as being connected or accessed to the other element directly but also as possibly having another element in between. On the other hand, if one element is described as being "directly connected" or "directly accessed" to another element, it shall be construed that there is no other element in between.

The terms used in the description are intended to describe certain embodiments only, and shall by no means restrict the present invention. Unless clearly used otherwise, expressions in the singular number include a plural meaning. In the present description, an expression such as "comprising" or "consisting of is intended to designate a characteristic, a number, a step, an operation, an element, a part or combinations thereof, and shall not be construed to preclude any presence or possibility of one or more other characteristics, numbers, steps, operations, elements, parts or combinations thereof.

Unless otherwise defined, all terms, including technical terms and scientific terms, used herein have the same meaning as how they are generally understood by those of ordinary skill in the art to which the invention pertains. Any term that is defined in a general dictionary shall be construed to have the same meaning in the context of the relevant art, and, unless otherwise defined explicitly, shall not be interpreted to have an idealistic or excessively formalistic meaning.

Hereinafter, preferred embodiments will be described in detail with reference to the accompanying drawings. Identical or corresponding elements will be given the same reference numerals, regardless of the figure number, and any redundant description of the identical or corresponding elements will not be repeated. FIG. 1 is a schematic block diagram illustrating the structure of a typical decoder, and FIG. 2 is a schematic block diagram illustrating the structure of the typical encoder. As illustrated in FIG. 1, an MPEG-4 decoder 100 typically includes a variable length decoding unit 110, an inverse scanning unit 115, an inverse DC/ AC prediction unit 120, an inverse quantization unit 125, an inverse discrete cosine transform unit 130 and a VOP reconstruction unit 135. It shall be evident that the decoder 100 can have a structure changed depending on an applying standard and some elements can be replaced with other elements.

If a transferred bitstream 105 is syntax-parsed and corresponding header information and encoded video data are extracted, the variable length decoding unit 110 forms a quantized discrete cosine transform (DCT) coefficient by using predetermined Huffman table, the inverse scanning unit 115 generates data having the same sequence as pertinent video data 140 by performing inverse scanning. In other words, the inverse scanning unit 115 outputs a value in inverse order of scanning by various methods. In the encoding, after performing the quantization, a scanning direction can be defined depending on the distribution of frequency range. Typically, a zig-zag scanning method can be used. However, various scanning methods per codec can be used. Syntax parsing can be integratedly performed in the variable length decoding unit 110 or in an element for processing the bitstream prior to the variable length decoding unit 110. In this case, since the same standard is applied to the corresponding encoder and decoder, the syntax parsing is processed by a predetermined setting only, to correspond to the pertinent standard. The inverse DC/AC prediction unit 120 determines the direction of a reference block for prediction by using the size of the DCT coefficient at a frequency range.

The inverse quantization unit 125 performs the inverse quantization of inverse- scanned data. In other words, the inverse quantization unit 125 returns DC and AC coefficients by using a quantization parameter (QP) designed in an encoding process. The inverse discrete cosine transform unit 130 calculates an actual video data pixel value to generate a video object plane (VOP) by performing inverse discrete cosine transform.

The VOP reconstruction unit 135 decodes a video signal by using the VOP generated by the inverse discrete cosine transform unit 130 and outputs the decoded signal.

As illustrated in FIG. 2, an MPEG-4 encoder 200 typically includes a discrete cosine transform unit 210, a quantization unit 215, a DC/ AC prediction unit 220, a scanning unit 230 and a variable length encoding unit 235.

Each element included in the encoder 200 performs the inverse functions of the corresponding elements of the decoder 100. This is well-known to those of ordinary skill in the art. Briefly describing, the encoder 200 converts a video signal (i.e. a digital video pixel value) to a frequency value through the DCT and the quantization and performs the encoding. Then, the encoder 200 performs variable length encoding to differentiate bit length according to the frequency number of information and outputs compressed bit stream format.

FIG. 3 is a schematic block diagram illustrating the structure of a decoder in accordance with an embodiment of the present invention, FIG. 4 is a schematic block diagram illustrating the structure of an extended bitstream in accordance with an embodiment of the present invention, FIG. 5 is a schematic block diagram illustrating the structure of a decoding unit in accordance with an embodiment of the present invention, FIG. 6 is a schematic block diagram illustrating the structure of a syntax(SYN) parser in accordance with an embodiment of the present invention, FIG. 7 is a schematic block diagram illustrating the structure of a decoding processing unit in accordance with an embodiment of the present invention, FIG. 8 illustrates the hierarchical structure assigning a decoding hierarchy table(DHT) in accordance with an embodiment of the present invention, FIG. 9 illustrates the calling structure between layers and the connection structure within layers assigned by a syntax rule table(S-RT) in accordance with an embodiment of the present invention, FIG. 10 illustrates an interface set for functional units applied to 2 or more of standards in accordance with an embodiment of the present invention, FIG. 11 illustrates the connection structure of a dedicated buffer space between nodes per each hierarchy in accordance with an embodiment of the present invention, FIG. 12 illustrates the inclusion relation of a dedicated buffer space for storing control signal and context information (CSCI) in accordance with an embodiment of the present invention, FIG. 13 illustrates the inclusion relation of a dedicated buffer space for storing output data by functional units in accordance with an embodiment of the present invention, and FIG. 14 illustrates a parallel processing of syntax parsing and data decoding in accordance with an embodiment of the present invention. In order to perform the encoding/decoding method according to the hierarchical structure of the invention, data are provided to the decoder 300 to generate hierarchical information or to be extracted and used as hierarchical information. Data corresponding to the hierarchical information may be provided to the decoder 300 in an extended bitstream with bitstreams or in an independent data form from bitstreams. If the hierarchical information corresponding to a pertinent data is pre-stored in a certain storing unit of the decoder 300, it is apparent that the providing of the decoder description be omitted. However, hereinafter a case, that pertinent data is provided to the decoder 300 in an extended bitstream form with bitstream, will be described.

As illustrated in FIG. 3, the decoder 300 of the present invention has a different function from the conventional decoder (refer to FIG. 1). The decoder 300 according to an embodiment of the invention includes a dividing unit 310, a decoder description decoder 320, a hierarchical information storing unit 330, and a decoding unit 340.

It is apparent that at least one of elements of the decoder 300(i.e., a dividing unit 310, a decoder description decoder 320, a decoding unit 340, etc.) be implemented to a software program(or a combination of program codes) to perform the functions hereinafter described.

The dividing unit 310 divides an extended bitstream to a decoder description region and a typical bitstream 105 (hereinafter referring to as "conventional bitstream") region. The decoder description including the hierarchical information is inputted to the decoder description decoder 320 and the conventional bitstream is inputted to the decoding unit 340.

The extended bitstream according to an embodiment of the invention may include decoder descriptions 410-470 and a conventional bitstream 105 as shown in FIG. 4. It is apparent that the hierarchical information of the present invention is not limited to the decoder description of FIG. 4 since it is an example only to describe of the present invention. That is, the hierarchical information may be enough if it is information about hierarchical structure to decode the encoded video data, specification of a layer to be called in each layer, and a sequence of call or activation of 2 or more functional units based on the hierarchical structure, and its structure can be varied. Hereinafter, a case that hierarchical information is specified into a plurality of information (referred as to "table information") will be described.

The decoder description includes structure information of the conventional bitstream 105 and information related to a encoding method of the conventional bitstream 105 (or connection information between functional units), to parse a bitstream, encoded by various encoding methods (various encoding standards), and/or a bitstream, encoded by a function that is selected by a user among various functions, by using a common analyzing method.

It is evident and easily understood by those of ordinary skill in the art that the decoder description information can be included in the extended bitstream 305 and provided to the decoder 300 or can be provided to the decoder 300 by additional independent bitstream format or data. The decoder description can be described by a description method which is read and/or interpreted by the decoder description decoder 320, for example it can be descried by a textual description method or a binary description method. However, it is also apparent that if the pertinent hierarchical information can be read by the decoding unit 340 without processing by the decoder description decoder 320, the decoder description decoder 320 may be omitted. The decoder description may include a functional unit list(FL) 410, a functional unit rule table(F-RT) 420, a functional unit control signal and context information table(FU-CSCIT) 430, a control signal and context information table(CSCIT) 440, a decoding hierarchy table(DHT) 445, a syntax element table(SET) 450, a syntax-rule table(S-RT) 460, a default value table(DVT) 470, and the like. The FL 410, the F-RT 420, or the like each generated by using the decoder description can be called as partial decoder description. The tables described above may be used as hierarchical information for the 2 or more of functional units in the decoding device of the present invention, which are activated corresponding to the hierarchical structure of encoded video data, to perform each predetermined process. Further, the tables described above may be used as operation control information to determine whether each functional unit performs a predetermined process by using input data or stores data resulted from performing a process as information with a certain name. It is apparent that the operation control information may be included in the hierarchical information.

The hierarchical information can be specified by one or more chosen from F- RT 420, DHT 445, and S-RT 469, etc. If encoded video data has dependency structure such as "the highest layer, the second highest layer, , the second lowest layer, and the lowest layer", the highest layer is activated (that is, an object in the highest layer is created and executed) by the hierarchical information, the second highest layer is successively activated and the lowest layer is lastly activated. Even though each layer is successively activated, it doesn't mean that the encoded video data is decoded by successive operation of the functional units in each layer. The encoded video data can be decoded by alternative operation of functional units in several layers. When one layer is activated, all decoding functional units included in the object of the corresponding layer and one or more of calling units of the next layer included in the corresponding object are activated. The decoding functional units in the activated layer starts operation when the control signal and context information (CSCI) requested for performing a predetermined process and data for decoding are stored in the storing unit 520. When the control signal and context information (CSCI) requested for performing a process by one or more of the decoding functional units of the same layer and data for decoding are stored in the storing unit 520, the calling unit in the activated next layer creates and executes an object of the corresponding layer, so that all decoding functional units included in the corresponding object and one or more of calling units of the next layer included in the corresponding object are activated. As described above, a lower layer can be called by an object execution of a layer. It is apparent that a sequence of tables (that is, information to generate partial decoder descriptions or partial decoder description itself) to describe decoder description information can be varied.

Here, the functional unit list(FL) 410, the functional unit rule table(F-RT) 420, the functional unit control signal and context information table(FU-CSCIT) 430, the control signal and context information table(CSCIT) 440, and the decoding hierarchy table(DHT) 445 may be used to build a reconfigurable connection of each functional unit(as necessary, a table may be referred to as "a first decoder description").

Among these, the decoding hierarchy table (DHT) 445 may be information describing for hierarchical structure of each codec and a lower layer per each hierarchy (see FIG. 8). The hierarchical structure of each codec may be described with upper layers and lower layers and may be subdivided into a sequence layer, a GOP layer, a VOP layer, a slice layer, a macro block layer, a block layer and the like. Each code is composed of at least one of those layers and at least one object may be included in each layer. A process of decoding encoded video data composed of 2 or more layers is described briefly as follows. However, it is apparent that it is not limited to this method for decoding bitstreams corresponding to the hierarchical structure.

During the decoding, after an object in a first layer, which is the highest layer, is created and executed, all decoding functional units included in the corresponding object and one or more of calling units of a lower layer, which is a second layer, included in the corresponding object are activated. For example, if it is assumed that the hierarchical structure is constructed in order of a first layer, a second layer and a third layer, at least one calling unit (i.e., GOP-I layer, GOP -2 layer, GOP-3 layer) included in the second layer is activated by executing the object of the first layer(i.e., a sequence layer), at least one calling unit (i.e., a picture-1 layer, a picture-2 layer with executing the object of the GOP-I layer, picture-3, picture-4 with executing the object of the GOP -2 layer, etc.) included in the third layer is activated by executing at least one object in the second layer. It is not limited to that only an upper layer calls a lower layer in the activation process by calling in the hierarchical structure and it is apparent that an upper layer can be called by a lower layer if necessary based on table information.

The functional units in the activated layer (the first layer) start operation if control signal and context information (CSCI) for performing a predetermined process and data for decoding are stored in a storing unit 520.

The calling unit in the lower layer (the second layer) of the activated layer (the first layer) creates and executes an object of the corresponding layer (the second layer) if control signal and context information for performing a process by one or more of the decoding functional units of the corresponding layer and data for decoding are stored in a storing unit 520, all decoding functional units in the corresponding object and at least one calling unit of the lower layer (the third layer) included in the corresponding object are activated.

As described, when an object of a certain layer is executed, its lower layer is called and decoding functional units in the called layer perform a predetermined process. Functional units selected by a decoder description (or a partial decoder description) among the functional units included in each layer are performed more than once and the encoded video data therefrom is decoded to video data and output the result data. The control signal and context information table(CSCIT) 440 may be referred to as decoder description information for mapping between element information stored in each functional unit of the decoding processing unit 550 and the CSCI storing unit. In this case, the element information can be a control parameter for a decoding processing unit 550 and/or each functional unit of a Syntax(SYN) parser 540.

The control signal and context information table(CSCIT) 440, the decoding hierarchy table(DHT) 445, the syntax element table(SET) 450, the syntax-rule table(S- RT) 460, the default value table(DVT) 470, and the like may be used for the syntax parsing of the conventional bitstream 105 (as necessary, a table may be referred to as "a second decoder description"). The type and function of the each decoder description information will be described below.

The decoder description decoder 320 stores the decoder description inputted from the dividing unit 310 as 2 or more tables which are the types to be recognized by the decoding unit 340 in the hierarchical information storing unit 330. It is not necessary that each table stored in the hierarchical information storing unit 330 is a typical type of table but enough that it is any information type to be recognized by the decoding unit 340. In other words, the decoder description decoder 320 converts data in a binary data format, etc. in the extended bitstream 305 into information that the syntax(SYN) parser 540 and/or the decoding control unit 530 can analyze, and stores the converted result information in the hierarchical information storing unit 330.

The tables (that is, partial decoder descriptions), which are stored in the hierarchical information storing unit 330 by the decoder description analysis of the decoder description decoder 320, may include a FL 410, a F-RT 420, a FU-CSCIT 430, a CSCIT 440, a DHT 445, a SET 450, a S-RT 460, and a DVT 470, etc. The decoder description decoder 320 may identify each table by referring to a table identifϊer(TI) by each specified table information.

Of course, it is not necessary that all tables (that is, information to generate all partial decoder description or all partial decoder descriptions themselves) should be included in the decoder description but enough if a codec number 820 and a profile and layer number 830 are included, or enough if a codec number 1020 and a profile and layer numberl030 for a part of tables are included. When a codec number and a profile and layer number are included, the decoder description decoder 320 may select a pertinent table among the tables pre-stored in the hierarchical information storing unit 330 to be used during the decoding without generating a new table for a whole table or a part of tables. When a codec number, a profile and layer number, and amendment information are included, the decoder description decoder 320 may extract the table corresponding to an appropriate code from the pre-stored tables in the hierarchical information storing unit 330 and generate a new table reflecting the amendment information. When a codec number and a profile and layer number are not included and description for generating a table is included, the decoder description decoder 320 may generate a new table for a whole table or a part of tables to be used during the decoding.

In addition, the decoder description may further include revision information besides the decoder description(DD-T) for each table. Tables divided by the decoder description decoder 320 are stored in the hierarchical information storing unit 330. When the extended bitstream 305 includes a codec number and a profile and layer number, the hierarchical information storing unit 320 may pre-store one or more pertinent tables to be used by the decoding unit 340. The decoding unit 340 may include a tool box 510, a storing unit 520, and a decoding control unit 530 as shown in FIG. 5. The storing unit 520 may include a CSCI storing unit 522 to store control signal and context information generated by the syntax(SYN) parser 540 and a data storing unit 524 to store data to be decoded by the decoding processing unit 550.

The too-box 510 may include the syntax(SYN) parser 540 and 2 or more of functional units decoding the data stored in the data storing unit 524 and outputting the result data. It is evident that the Syntax(SYN) parser 540 can be realized as an functional unit and the Syntax(S YN) parser 540 and the functional units can be realized as a combination of each program codes.

In other words, the tool box 510 is a region where the functional units can be realized to perform functions, respectively and each functional unit in the decoding processing unit 550 is activated by the decoding control unit 530 and outputs the encoded video data in the conventional bitstream 105 into video data. The decoding control unit 530 may control to activate the functions corresponding to each hierarchical structure in order (for example, a first layer, a second layer, and a third layer, etc. are successively activated, at the time when the second layer is activated, the first layer may be kept in the activation state or be deactivated if the data processing is not needed anymore). However, even though functional units in the same hierarchical structure are simultaneously activated, if a functional unit uses the result data processed by another functional unit as input data, the process is reserved till the result data is generated by the another functional unit and stored in the data storing unit 524.

The decoding functional units each may belong to at least one divided layer according to the characteristics of codec. A called parsing functional unit performs the syntax parsing of a bitstream and the decoding functional units performs the processing if CSCI required for performing a predetermined process and data for decoding are stored. An upper layer may call functional units in a lower layer according to flow of CSCI and data, and video data corresponding to the bitstream is outputted by performing one or more times of the process according to the need of all functional units described in the decoder description. Thus, scheduling management of each codec and systemic processing of each functional unit are provided by using the decoder description. That is, when the conventional bitstream is inputted in the decoding unit 340, the decoding control unit 530 activates the syntax(SYN) parser 540, allows syntax parsing by using the S-RT 460, and controls to activate functional units in the highest layer with reference to the F-RT 420. Even though a part of the first decoder description and the second decoder description is used at an initial state, the first decoder description and the second decoder description are eventually systemically used by mutually referring to each other.

Each activated functional unit stands by till all necessary CSCI and/or data are stored in the storing unit 520 for processing and if they are stored in the storing unit 520, the processing is performed by a functional unit. The decoding unit 340 of the present invention may perform simultaneously the syntax parsing process and the data decoding for outputting video data (see FIG. 22). Further, activation of the functional units in the decoding processing unit 550 is determined by a unit of the hierarchical structure and each functional unit is independently determined for operation reservation or operation starting, so that each functional unit can be operated systemically (for example, operation by forming sequential/parallel combinations between the functional units).

The syntax(SYN) parser 540 included in tool box 510 may be set immediately to interpret the conventional bitstream 105 if the conventional bitstream 105 is inputted, regardless of a connection control of the decoding control unit 530. This is for the functional units in the decoding processing unit 550 to perform post processings since the functional units may use element information which the syntax(SYN) parser 540 interprets and stores in the CSCI storing unit 522, and/or data(hereinafter, referring to as "encoded data") which the Syntax(SYN) parser 540 stores in the data storing unit 524.

The syntax(SYN) parser 540 interprets the conventional bitstream 105 inputted by using the DHT 445, the SET 450, the S-RT 460, the CSCIT 440, and the DVT 470, etc. and stores element information, which is the result of the syntax parsing, in the CSCI storing unit 522. For example, the CSCI storing unit 522 may be a buffer memory. The element information may be control signal context information (CSCI). The element information, which is parsed by the syntax(SYN) parser 540 and stored in the CSCI storing unit 522, may be the parsed result value of a pertinent step, and at the same time, may be an input value for determining the syntax of the following bitstream. Further, the syntax(S YN) parser 540 performs entropy decoding for a header of the syntax parsed conventional bitstream 105 and video data, and stores data encoded in a predetermined size (i.e., 1 pixel unit, 4x4 unit, 8x8 unit, etc.) in the data storing unit 524 to be used by the functional units of the decoding processing unit. The syntax(SYN) parser 540 may further store data to be used in the decoding step such as DC/AC out of syntax in the data storing unit 524. The data storing unit 524 may be a buffer memory and a dedicated buffer space may be predetermined per each functional unit.

Each dedicated buffer space, as shown in FIG. 12 and FIG. 13, may be set to have inclusion relationship per a hierarchy. Thus, when input data is written in the data storing unit 524, the activated functional unit may immediately start processing and if the processed result data is used as input data of another functional unit, may write the corresponding result data in the data storing unit 524.

The syntax(SYN) parser 540 may be realized as a software program (including the combination of program codes). Even though the syntax(SYN) parser 540 is realized to perform 2 or more functions corresponding to 2 or more standards (for example, MPEG-1/2/4/AVC, etc.), respectively, the syntax(SYN) parser 540 can perform an appropriate operation by using the decoding hierarchy table(DHT) 445, the syntax element table(SET) 450, the syntax-rule table(S-RT) 460, the control signal and context information table(CSCIT) 440, and the default value table(DVT) 470, etc. Of course, the syntax(S YN) parser 540 may be realized with subdivision into 2 or more of functional units as shown in FIG. 6. It is apparent that each functional unit be realized as a combination of blocked program codes.

The function of the syntax(SYN) parser 540 will be described below with particular description of each functional unit as shown in FIG. 6.

The syntax(SYN) parser 540, as illustrated in FIG. 6, can include a network abstraction layer parsing functional unit (NALP FU) 610, a syntax parsing functional unit (SYNP FU) 620, a context determination functional unit (CTX FU) 630, a variable length decoding functional unit (VLD FU) 640, a run length decoding functional unit (RLD FU) 650, and a macro block generator functional unit (MBG FU) 660, etc. Of course, it shall be evident that the syntax(SYN) parser 540 can include all functional units for syntax parsing regardless of the applied standard, a functional unit requested for a technology development process can be newly added, a previous functional unit can be amended, and an unnecessary functional unit can be deleted. It is also evident that each functional unit included in the syntax(SYN) parser 540 can be combined as one functional unit in case that they are independently not provided for each standard and can be identically processed regardless of the standards. Since the functions of each of the functional units are well-known to those of ordinary skill in the art, they will be described briefly. The NALP FU 610 is a functional unit parsing the network abstraction layer of

MPEG-4 AVC, and the SYNP FU 620 is a functional unit parsing the syntax of the bitstream. The SYNP FU 620 can be included in the VLD FU 640.

The CTX FU 630 is a functional unit determining the VLC table of MPEG-4 AVC, and the VLD FU 640 is a functional unit performing an entropy decoding. The RLD FU 650 is a functional unit performing the entropy decoding of AC values, and the MBG FU 660 is a functional unit generating macro block data by coupling DC values with AC values. The functions corresponding to all or some of functional units in the aforementioned syntax(SYN) parser 540 can be included in the VLD FU 640 depending on a system realizing method. As described above, the Syntax(SYN) parser 540 can be realized as one software program or 2 or more of software programs (e.g. the VLD FU 640 is realized as an independent software program). A process, that the Syntax(SYN) parser 540 extracts element information by using a first description information (that is, at least one of a decoding hierarchy table(DHT) 445, a syntax element table(SET) 450, a syntax-rule table(S-RT) 460, a control signal and context information table(CSCIT) 440, a default value table(DVT) 470) and stores the element information in the CSCI storing unit 522, will be described in detail when the decoding control unit 530 is described.

The decoding processing unit 550 processes the encoded data stored in the data storing unit 524 in a predetermined processing unit and outputs into video data in a predetermined size. The processing unit of the encoded data can be predetermined to be applied differently for each functional unit or be generalized to be identical.

Functional units may be included in the decoding processing unit 550 to perform the above-described functions according to each standard. Each functional unit can be realized by an independent processing block (e.g. software program, the combination of instruction codes, function, etc.) or the decoding processing unit 550 may be realized as one combined processing block. However, it is apparent that the decoding processing unit 550 perform an appropriate processing by a connection control of the decoding control unit 530 even though it is realized as one combined processing block. As illustrated in FIG. 7, the decoding processing unit 550 includes a de- blocking functional unit (DF FU) 710, a VOP reconstruction functional unit (VR FU) 715, a frame field reordering functional unit (FFR FU) 720, an intra prediction and picture reconstruction functional unit (IPR FU) 730, an inverse transform functional unit (IT FU) 735, an inverse quantization functional unit 745, an inverse AC prediction functional unit (IAP FU) 755, an inverse scan functional unit (IS FU) 760 and a DC reconstruction (DCR FU) 765.

A IT 4 x 4 FU 740, an IQ 4 x 4 FU 750 and DCR 4 x 4 FU 770 process the block having 4 x 4 size. This is because there is a case that MPEG-4 AVC processes the block of 4 x 4 size, while MPEG- 1/2/4 processes the block of 8 x 8 size in transform, quantization.

It shall be evident that the decoding processing group 550 can include all functional units for data decoding function regardless of the applied standard, a functional unit requested for a technology development process can be newly added, a previous functional unit can be amended and an unnecessary functional unit can be deleted. For example, in the case of additionally requesting an IS 4 x 4 FU processing data with 4 x 4 block size, the decoding processing unit 550 can further include an appropriate functional unit. Also, the decoding processing unit 550 can further include a special prediction functional unit (SPR FU) for performing the intra prediction at the MPEG-4 AVC. It is also evident that each functional unit included in the decoding processing unit 550 can be combined as one functional unit in case that they are independently not provided for each standard and can be identically processed regardless of the standards. Since the functions of each of the functional units are well-known to those of ordinary skilled in the art, they will be described briefly. The DF FU 710 is the de-blocking filter of MPEG-4 AVC, and the VR FU 715 is a functional unit storing a decoded pixel.

The FFR FU 720 is a functional unit for an interlaced mode, and the IPR FU 730 is a functional unit performing the intra prediction of MPEG-4 AVC and storing the decoded pixel value. As described above, the intra prediction of MPEG-4 AVC can be performed by the SPR FU.

The IT FU 735 is a functional unit performing the inverse transform of DC values and AC values, and the IQ FU 745 is a functional unit performing the inverse quantization of AC values.

The IAP FU 755 is a functional unit performing the inverse AC prediction of AC values, and the IS FU 760 is a functional unit performing the inverse scan of AC values. The DCR FU 765 is a functional unit performing the inverse prediction and inverse quantization of DC values.

Each operation of the aforementioned Syntax(SYN) parser 540 and the decoding processing unit 550 is independently performed, and each functional unit in the decoding processing unit 550 performs independently its operation when the CSCI and/or data for the processing in the storing unit 520 is(are) stored after the decoding control unit 530 activates by the activation control.

Element information (for example, CSCI), which is the result value from the syntax parsing by using the second decoder description (that is, CSCIT 440, DHT 445, SET 450, S-RT 460, DVT 470) at the Syntax(SYN) parser 540, is stored in the CSCI storing unit 522. The element information is stored to correspond to the CSCIT 440. The CSCI storing unit 522 may be a buffer memory.

The element information stored in the CSCI storing unit 522 may be used as input data for performing the process by the SET 450 or as a control variable determining a connection index which is the following process by the S-RT 460.

The element information stored in the CSCI storing unit 522 may be used for each functional unit, which the decoding control unit 530 activated by the hierarchical structure unit, to perform mapping the input CSCI designated in the FU-CSCIT 430 with the element information stored in the CSCI storing unit 522 by referring the F-RT 420. As described above, if the element information can be used by the functional units in the decoding processing unit 550, the syntax(SYN) parser 540 can also store the element information in the data storing unit 524.

The decoding control unit 530 controls the activation of the syntax(SYN) parser 540 and/or each functional unit included in the decoding processing unit 550 in order to decode the bitstream encoded by various standards. In other words, the decoding control unit 530 controls for each functional unit included in the decoding processing unit 550 to be activated per the hierarchical structure unit by referring to the F-RT 420. This is because input data for the functional unit in the lower layer can be result data processed by the functional unit of the upper layer. The decoding control unit 530 uses FL 410, F-RT 420, FU-CSCIT 430, CSCIT

440, and DHT 445 to process the activation of each functional unit and the data encoded by the each functional unit to video data and output the result. The encoded data can be converted into video data by systematical operation of 2 or more of functional units and the result can be outputted as illustrated in FIG. 7. The order of operations by the functional units to convert into video data may be different according to each encoding standard and this is apparent to those skilled in the art. Therefore, hereinafter a systemic relationship between tables of the first decoder description included in an extended bitstream, which is used as basic information for systemic operation control of 2 or more of functional units, will be described.

The decoding hierarchy table (DHT) 445 is a table describing information for codec hierarchical structure and lower layers per each hierarchy.

Table 1 : DHT(Decoding Hierarchy Table)

The DHT 445, as shown in Table 1, may include an index which is a layer number, a name which is a name of each layer, a children hierarchy which is a lower layer of a corresponding layer, etc. A number of layers or a number of children hierarchies may be different according to codec. For example, when there are further VOP-3, MB-3, etc., it is apparent that layer number be correspondingly added, as illustrated in FIG 16.

The hierarchical structure of the DHT 445 of the present invention may combine a codec which is different from a base code at any position in any layer by any method based on the base codec. In other words, several different codecs may be combined to a unified codec to provide a new codec which further allows video compression and decompression.

A main function of the DHT 445 allows various implementations including sequential/parallel, single/multiple threads, sw/hw hybrid decoding of a decoding solution to compose using decoder descriptions by introducing the hierarchical structure to a given codec.

The hierarchical structure described with the DHT 445 in Table 1 is commonly used in the syntax parsing and decoding process. It is also apparent that another hierarchical structure can exist to be independently applied in the syntax parsing or in the decoding process.

The FU list(FL) 410, as shown in Table 2, is a table describing information for a list of each functional unit in the decoding processing unit 550, volume of input/output data, etc. Table 2 : FL(FU List)

The FL 410 may further include a name of a dedicated buffer region where input data is written (or recording address of a pertinent data or address of buffer memory where a pertinent data is written) and a name of a dedicated buffer region where output data is to be written by a pertinent functional unit (or recording address of a pertinent data or address of buffer memory where a pertinent data is written). Each functional unit can read input data and write processed output data by using the FL 410. However, input data and output data of the Syntax(SYN) parser 540, which generates element information, is not written in the FL 410 because the Syntax(SYN) parser 540 generates element information and writes the generated element information in a determined position by using the SET 450 and the like.

The FL 410 may include FU ID which is an identification number of each functional unit in the tool box 510, FU name which is a name of each functional unit, input CSCI representing volume of input data, output CSCI representing volume of result data (output data), etc. as shown in FIG. 2. A list order of the functional units in the FL 410 corresponds to FU list order of FU-CSCIT 430 or index number. A name of each functional unit, which is described by referring to FIG. 7, may be written in the column of FU name.

A particular functional unit which is activated by the decoding control unit 530 reads necessary data from the data storing unit 524 by using FU-CSCIT 430, CSCIT 440, etc., performs a predetermined process, and generates output data. The generated output data may be stored in the data storing unit 524 and the stored data in the storing unit 524 can be used as input data for a functional unit performing a following process. Here, the functional unit is included in the decoding processing unit 550 and means a series of processings (e.g. ability, algorithm or function, etc.) which performs a predetermined process for input data and generates output data.

When the decoding unit 340 only uses one standard to decode encoded video data included in the conventional bitstream 105, the FL 410 may include only information for the functional units which perform the processing corresponding to that standard. However, when a pertinent video data is encoded by using a plurality of standards (e.g., when encoding standard is applied with a plurality of frame units), information of the functional units according to a plurality of standards is required to decode the corresponding encoded video data. Thus, in this case, the FL 410 should include information of functional units according to a plurality of standards required for the encoded video data decoding from all functional units according to the corresponding plurality of standards.

Even though video data applies different encoding standards by a plurality of frame units, if a plurality of conventional bitstreams 105 and extended bitstreams 305 can be generated and provided by each applied encoding standard, each FL 410 includes only information of functional units corresponding to each standard.

The FL 410 can be described not only by a textual description method or a binary description method (bit converted binary code format) but also by key minimum data of the table by using a pseudo script language.

Then, FU - Rule Table (F-RT) 420 provides connection information of functional units to be used for decoding inputted conventional bitstream 105. F-RT 420, as shown Table 3 below, is composed that an upper layer calls a lower layer by using the hierarchical structure specified in the DHT 445.

Table 3 : F-RT(FU - Rule Table)

F-RT 420, as shown in Table 3, includes decoding hierarchy representing each hierarchical structure, children representing functional units to be activated or lower layers to call, loop representing determination of loop operation, loop condition representing loop conditions when loop operation is required, input representing decoding hierarchy or children as each functional unit or input information of a lower layer(or activation order), etc.

As shown in Table 3, a FHO layer will be activated very first and since a FHl layer uses result data of FUl as input data, it will be then activated. Or the FHl layer and the FHO layer can be activated at the same time and processing operation can be reserved till the result data of FUl is stored in the data storing unit 524.

Each layer is generated as an object with guaranteed independence based on instance/object concept during the calling. In other words, a plurality of FHl is generated such as FH 1 [0] , FH 1 [ 1 ] , FH 1 [2] , ... , FH 1 [n] and this structure may be applied to another layer(e.g., FHO, FH2, FH3, etc.). A dedicated buffer space per each layer can be assigned. As illustrated in FIG. 20 and FIG. 21, CSCI or data for the processing by functional units in a particular hierarchical structure or a pertinent hierarchical structure is stored in each dedicated buffer space.

The F-RT 420 can be described not only by a textual description method or a binary description method (bit converted binary code format) but also by key minimum data of the table by using a pseudo script language.

Then, FU-CSCIT(FU CSCI Table) 430 is a table for connecting element information stored in the CSCI storing unit 522 with element information(input CSCI) requested by each functional unit.

Table 4 : FU-CSCIT(FU CSCI Table)

As shown in Table 4, the FU-CSCIT 430 includes index number mapping 1 : 1 with FU ID of the FL 410, CSCI information, and CSCI table element for mapping. A list order of the functional units of the FL 410 is mapped with a list order of the index number of the FU-CSCIT 430. In other words, FU-A, which is the first functional unit of Table 2, is mapped with index no. 1, which is the first index of the FU-CSCIT 430. The FU-CSCIT 430 further includes information of how many interface sets are per index number. It means that a number of interface sets are a number of mappings. In other words, D0092 of the FL 410 is assigned to have 1 interface set by the FU-CSCIT 430 and thus uses only CH2.C6 and CH2.C1 as element information. D0098 of the FL 410 has 2 interface sets and thus uses CH3.C1 and CH3.C6 as element information in one case and CH4.C2 and CH4.C5 in the other case. Each functional unit performs a predetermined processing by reading element information from the CSCI storing unit 522 or the data storing unit 524 and stores the generated output data in the data storing unit 524.

The reason to assign a number of interface sets per each index number in the FU-CSCIT 430 is that a functional unit can perform a different processing operation according to a different data flow. In other words, as shown in FIG. 18, if FU-A is a function which can be used in both MPEG-2 and MPEG-4, element information for processing encoded data by the MPEG-2 standard can be different from element information for processing encoded data by the MPEG-4 standard. Therefore, 2 or more of interface sets are required in order to classify element information requested for encoded data processing according to each standard.

The FU-CSCIT 430 can be described not only by a textual description method or a binary description method (bit converted binary code form) but also by key minimum data of the table by using a pseudo script language.

The CSCIT 440, as shown in Table 5 below, describes details of the element information (e.g., CSCI) as the result information that the Syntax(SYN) parser 540 processes by using the SET 450, S-RT 460, etc. In other words, the CSCIT 522 has all meaningful data (i.e., element information) that is processed from the conventional bitstream 105, stored in the CSCI storing unit 522 or the data storing unit 524, and used by the decoding processing unit 550.

Table 5 : CSCIT

As shown in Table 5, the CSCIT 440 includes decoding hierarchy of each element information, an index as an identifier which is the identity number of the pertinent element information, a name of the pertinent element information, a type for designating the data structural characteristic of the pertinent element information (e.g., whether the pertinent element information is an integer type or an array type, etc.). The CSCI information composed of layers may be used by a type of several arrays, if necessary, during the decoding.

The CSCIT 440 can be described not only by a textual description method or a binary description method (bit converted binary code format) but also by key minimum data of the table by using a pseudo script language.

Hereinafter, the CSCIT 440, the DHT 445, the SET 450, the S-RT 460, and the DVT 470, which are used when the Syntax(S YN) parser 540 extracts and generates element information from the conventional bitstream 105 and stores the result in the CSCIT storing unit 522, will be described. However, the CSCIT 440 and the DHT 445 have been explained above and their details will be thus omitted. 2 Or more of functional units as shown in Table 6 operate systematically each other to generate element information and store the result in the storing unit 520. A type of element information can be different according to each encoding standard and it is apparent to those skilled in the art. A sysmetic relationship between tables of the second decoder descriptions included in an extended bitstream, which 2 or more of functional units or the Syntax(SYN) parser 540, which is a combined functional unit, use to generate element information and store the result in the storing unit 520, will be explained.

First, the SET 450 is a table that consists of information related to the syntax of the inputted conventional bitstream as shown in Table 6.

The SET 450, as shown in Table 6, includes index of each syntax, element name, input data, and process by SET-PROC. Here, the index is an identifier S identifying each syntax used for the S-RT 460. The element name can be named according to the meaning or function of the syntax. The input data refers to nominal bit length of data inputted at a time in the conventional bitstream. The SET-process describes which processing operation is undergone after receiving each bitstream syntax to generate the element information as the output data. Here, process by SET-PROC "READ 32 B:(IBS==HEX:IBO)»:" corresponding to the index SO means that if "(IBS==HEX:IBO)" is satisfied with reading 32 bits, output(») the corresponding information. Output data is element information (that is, CSCI information (C), e.g., CH0.C1, etc.), which is provided by the S-RT 460 and stored in the CSCI storing unit 522 or the data storing unit 524.

The SET 450 can be described not only by a textual description method or a binary description method (bit converted binary code format) but also by key minimum data of the table by using a pseudo script language.

Then, the S-RT 460 refers to layers and connection information between each syntax in the conventional bitstream 105 as shown in Table 7. In other words, the S-RT 460 has information indicating for an upper layer to call syntax of a lower layer and to move to next syntax. The Syntax(SYN) parser 540 uses the R-ST 460 to read the conventional bitstream or to define the sequence of storing and/or updating the element information in the CSCI storing unit 522. As illustrated in FIG. 9, the Syntax(SYN) parser 540 is composed of several threads and performs syntax parsing per layer by referring to the S-RT 460 since an upper thread calls a lower thread.

Table 7 : S-RT(Syntax Rule Table)

As illustrated in Table 7, the S-RT 460 includes decoding hierarchy of each syntax, index R, syntax #ID, input data (CSCI), and branch information.

The index R identifies each connection information (rule). Since the index S of syntax designates syntax that a particular connection index will process in each layer, the Syntax(SYN) parser 540 performs a designated process of the pertinent syntax by suing the SET 450.

The input data refers to a list of element information to be used for condition determination for controlling the connection of the pertinent connection index. The process result by the SET 450 is stored in the name of input data corresponding to syntax ID as described earlier.

The branch information refers to condition determining algorithm determining which connection index will be processed in next turn. The branch information can directly determine the sequence and contents of data to be read. If the number of branch is l(e.g., in case that syntax #ID is Sl), no input data is provided.

The S-RT 460 can be described not only by a textual description method or a binary description method (bit converted binary code format) but also by key minimum data of the table by using a pseudo script language.

Finally, the DVT 470 is a table describing Huffman table information used by each encoder/decoder as shown in Table 8. Entropy coding is performed during each encoding in MPEG- 1/2/4/ A VC. At this time, the Huffman coding method is mainly used and the used information is the Huffman table. There should be provided Huffman table information to be used in a pertinent decoder during each decoding process to implement a unified codec. Therefore, decoder descriptions of the present invention include Huffman table information corresponding to each syntax during the syntax parsing. If Huffman table information corresponding to each standard is pre-stored in the table storing unit 330, the DVT transmission may be omitted or only codec number 1020 and profile and level number 1030 can be included as illustrated in FIG. 10.

Table 8 : DVT(Default Value Table)

As shown in Table 8, the DVT 470 includes name of each Huffman table, actual value compressed and outputted by the Huffman decoding, and code value used when the compressed actual value is stored in the conventional bitstream 105. For example, if an actual value obtained with the compression of MCBPC value is 3, a code value 011 is written in the conventional bitstream 105 by the Huffman table mapping process (e.g., process by the SET 450). As another example, if VLD[I] is written in the process part of a particular index of the SET 450, a function of VLD is called, a code value can be obtained by reading the conventional bitstream 105 for the length predetermined by the corresponding function, and the corresponding actual value can be obtained by the Huffman table mapping process. Here, the used Huffman table is [1], which is the first table CBPY.

The DVT 470 can be described not only by a textual description method or a binary description method (bit converted binary code format) but also by key minimum data of the table by using a pseudo script language. The storage space for each table can be reduced with binary description, the processing efficiency is improved, and the transmission time of the extended bitstream 305 including the decoder description is further reduced.

Hereinafter, the linking process between each table used by the Syntax(SYN) parser 540 and/or the decoding control unit 530 will be described. First, the decoding control unit 530 calls functional units of the highest layer assigned by the F-RT 420 in the table storing unit 330. The called functional units stands by till the Syntax(SYN) parser 540 stores the CSCI information and/or data requested for the processing in the storing unit 520.

The Syntax(SYN) parser 540 reads syntax corresponding to the highest layer from rules of th S-RT 460, recognizes VS start code by the SET 450, and stores output value (that is, element information, CO) in the name of input data (CSCI) provided in the S-RT 460 in the CSCI storing unit 522 by reading a bit set as input value in the SET 450 (that is, 32 bits determined as input value at the SET 450) corresponding to the conventional bitstream 105 and performing the SET-process. What is element information stored in the CSCI storing unit 522 or in which layer will it be used, etc. is described in the CSCIT 440. The Syntax(SYN) parser 540 replaces the element information CO stored in the CSCI storing unit 522 for pertinent branch information of the S-RT 460 and performs the index processing corresponding to the result. For example, since branch information corresponding to index SH0.SH1 is 'CH0.C0= =1', if satisfied, it proceeds to index SR2 in the SHO layer but if not satisfied, it is determined as error.

However, in the process where the Syntax(S YN) parser 540 generates element information by using the SET 450 and stores the result in the CSCI storing unit 522, if the VLD function is called (e.g., index S74 of the SET 450), the entropy encoding is performed by using the DVT 470. During this process, if element information is generated, it will be stored in the CSCI storing unit 522.

During the Syntax(SYN) parser 540 stores the element information and/or data in the storing unit 520, each activated functional unit in the decoding processing unit 550 verifies if all requested element information and/or data is stored to perform a predetermined process. A pertinent functional unit can verify if all is stored by referring to the FU-CSCIT 430. Further, it can recognize what the corresponding element information is by mapping with the CSCIT 440. The functional unit, that stores the requested element information and/or data, performs a predetermined process and stores the processed result data in the data storing unit 524. As described above, the functional units may be successively activated or called by hierarchical structure unit. The activation of the hierarchical structure unit can be controlled by the decoding control unit 530 which, for example, watches if the element information and/or data processing requested by a certain layer completes. If it is recognized that the element information and/or data requested to perform a predetermined process is stored, the called functional units read this and process. When each functional unit is connected sequentially or in parallel and performs the process, it provides same effect through such processes.

When layer(s) of the functional units reqested for converting into video data is(are) activated, the decoding unit 340 outputs video data corresponding to the inputted conventional bitstream 105.

FIG. 14 illustrates a parallel processing of syntax parsing and data decoding in accordance with an embodiment of the present invention. As illustrated in FIG. 22, the syntax parsing by the Syntax(S YN) parser 540 and the data decoding by the decoding processing unit 550 can be independently performed by using the decoder description information. In other words, when the element information and/or data requested by each functional unit is stored for the data decoding, the corresponding functional unit can perform a predetermined process. Further, each functional unit in the decoding processing unit 550 performing the decoding processing may perform independent function if result data of a different functional unit is not used as input data. Therefore, the decoder 300 of the present invention is not limited to that the decoding processing unit 550 can perform the operation after the syntax parsing is completed, but allows that the SYN parsing 540 and the decoding processing unit 550 can perform independently. It further allows that each functional unit in the decoding processing unit 550 can independently operate. As described above, according to the present invention, existing profiles can be used by using the functional units provided in the conventional standard (that is, codec), a new decoder can be implemented by using the conventional functional units, and a new decoder can be implemented by using new functional units, hi other words, various decoders can be implemented without any limitation. Only, when new functional units are added in the tool box 510, the algorithm

(that is, description of functional units) corresponding to those functional units is added and the corresponding information is added to the FL 410. In this case, a compile process for the algorithm may be needed additionally.

To realize unified codec, each element must be systematically controlled such that a bitstream can be decoded with a decoding method corresponding to a pertinent encoding method by parsing the bitstream compressed by various encoding methods.

In this case, the pertinent bitstream can be a bitstream of various shapes mixed with various standards (codecs) or another bitstream of various shapes generated by various encoding methods in one standard. Also, to support various encoding / decoding methods, it is necessary that various functions can be separated into independent units and desired units can be selected among the independent units to make them one codec (encoder and decoder).

Also, described above, the present invention can systemically connect and control each functional unit by use of the same information analyzing method regardless of an encoding method encoded with a bitstream by providing together decoder description information.

Also, the present invention can deal with the change or new addition of the syntax of a bitstream by only amending pertinent information or inserting the additional information. The present invention also can set the connection relationship of the functional units of the decoding processing unit 550 of the pertinent decoder and functional units by allowing a user to select a desired function of bitstream-level, frame- level and macro block-level (MB level) and forms the F-RT 420.

Hereinafter, commands of each table will be described in detail. Commands used in each table shown above are provided. Information (that is, table) can be composed for the syntax parsing of standards such as MPEG-2/MPEG- 4/MPEG-4 AVC, etc.

Commands used for composing each table can be READ, SEEK, FLUSH, IF, WHILE, UNTIL, DO-WHILE, DO-UNTIL, BREAK, SET, STOP, PUSH, etc. It is not necessary that all commands should be used in each table. It is apparent that certain commands per each table can be selected and used. Each command is described briefly. First, the READ can be used for reading a certain bit in the bitstream. For example, the READ can be described as "READ bits B>CSCI;", in which "bits" is a number of bits to read, "B" is byte-alignment flag, ">CSCI" is CSCI index to store. "B" and ">CSCI" are optional and if ">CSCI" is not designated, it is set to store only in a variable IBS.

The SEEK is a command to read a certain bit in the bitstream but not to move file point. The file point is a standard position during the operation such as reading a certain bit. Parameters of the SEEK command can be the same as the READ.

The FLUSH can be used for moving a file point as much as a certain bit in the bitstream and parameters thereof can be similar to the READ.

The IF can be used in a form of "IF (condition) { ~ } ELSE { ~ }" and is a command to provide branch according to a given condition.

The WHILE can be used in a form of "WHILE (condition) { ~ }" and to repeatedly perform assigned blocks during a given condition is true. The UNTIL can be used in a form of "UNTIL (condition) { ~ } " and to repeatedly perform assigned blocks till a given condition is true.

The DO-WHILE can be used in a form of "DO { ~ } WHILE (condition)" and to perform a block prior to condition determination by modifying the WHILE.

The DO-UNTIL is used in a form of "DO { ~ } UNTIL (condition)" and to perform a block prior to condition determination by modifying the UNTIL. The ( ~ ) (compute) is used in a form of, for example, "(Cl l=(V2+3));". In other words, all equations of the SET-PROC can be described within the parenthesis, operators, such as four arithmetical operations, assignment, comparison, addition/subtraction (++/--), bitwise operation, OR/AND operation, determining the use of CSCI, etc., can be used.

The BREAK can be used to secede from the closest loop structure. The SET can be used to set a flag determining the use of the designated CSCI and the CSCI to designate the flag is arranged and can be sorted with a comma (e.g., SET CO, C2;). The STOP can be used to stop the syntax element processing which is currently in process and to move next.

The PUSH can be used to add data at the last region where data is written in an array CSCI and added value is arranged and sorted by comma(e.g., PUSH C8 8, 16, 32;).

The GO can be used to go a designated position. For example, in case of GO SR#;;, it is an instruction to go SR# and GO RT is an instruction to return to the called position.

The HEX can be used to represent that a value after the HEX command is hexadecimal digits.

The RLD is an interface for a RLD function which is supported in MPEG-4, and can be used in a form of "RLD index, layer, run, islastrun, t#;". Here, index, layer, ran and islastrun can be CSCI storing RLD return values or variables and t# represents Huffman table ID used in the RLD.

The VLD2 is a VLD function for MPEG-2 and can be used in a form of "VLD2 [t#] in > vl, v2, v3;". Here, t# represents Huffman table ID used in the VLD, in represents an inputted index value, and vl~v3 represent output values.

Finally, the VLD4 is a VLD function for MPEG-4 and can be used in a form of "VLD4 [t#] > CSCI;". Here, t# represents Huffman table ID used in the VLD and ">CSCI" represents a CSCI index to be stored. If ">CSCI" is not designated as option, it is stored only in a variable IBS.

FIG. 15 is a block diagram illustrating an encoder in accordance with an embodiment of the present invention.

An encoder 1500 according to the present invention further includes an extended bitstream generating and outputting unit 1510, compared to the conventional encoder 200 as described in FIG. 2. The extended bitstream generating and outputting unit 1510 generates decoder descriptions by using a control information list and reconfigurable connection for the conventional bitstream generation process generated by previous processings, input data for pertinent functional units, syntax information, and syntax connection information, etc. An extended bitstream 305 is generated by using the generated decoder descriptions and the conventional bitstream 105 and transmitted to the decoder 300. Since the method of generating decoder description is easily understood to those skilled in the art, detailed explanation thereof is omitted. It is also apparent to those skilled in the art that the encoder 1500 has a tool box including 2 or more of functional units and one or more of bitstreams are generated according to one or more of encoding standards by sequential combinations or systemical combinations of these functional units.

Further, the variable length encoding unit 230 merely refers to an element (e.g. encoding unit) finally performing an encoding to generate the conventional bitstream 105 in the encoder 1500, but the present invention is not limited to the variable length encoding unit 230. Also this does not restrict the scope of claims of the present invention.

FIG. 15 assumes the case that the extended bitstream 305, generated by using the decodr description information and the conventional bitstream 105, is provided to the decoder. However, as described above, the decoder description information can be provided to the decoder 300 by an additional data format or a bitstream format. In this case, it is evidently possible that the extended bitstream generating and outputting unit 1510 is not provided at the end of the variable length encoding unit 235 and the extended bitstream generating and outputting unit 1510 is independently provided from the conventional encoding unit 200 to provide independently-generated information to the decoder 300.

In the present description, the unified codec device and method in accordance with the present invention is mainly described based on the decoder. Considering that the relationship between the decoder and encoder is well-known to those skilled in the art and the encoder can easily be structured with the description related to the decoder only, it is evident that the present invention is not limited to the decoder.

As described above, the unified codec device and method of the present invention makes it easy to analyze a syntax element in a standard (or codec) or between different standards or to control the connection of pertinent functional units, hi other words, in the present invention, there is no problem when changing the sequence of syntax elements in the bitstream generated according to a particular standard, inserting new elements and deleting a previous syntax element.

Also, in accordance with a prior art, there is a problem that the decoder is not able to properly decode a corresponding bitstream in the manipulation of the syntax element. For example, if a structure of ABC bitstream information is changed to a structure of ACB, the decoder may not recognize the structure of ACB and thus impossible to perform a proper decoding. Similarly, for a structure of ABFC by adding F or another structure of AC by deleting B, the decoder may not recognize the structure and thus impossible to perform the proper decoding. However, in the unified codec device and method of the present invention, since the decoder description included in the extended bitstream or the decoder description as independent data is provided to the decoder 300, the decoder 300 can perform the proper decoding operation.

The syntax analyzing method for the decoding device and the bitstream decoding in accordance with the present invention has been described based on the MPEG-4 AVC. However, it is evident that the present invention can be identically applied to the MPEG-I, MPEG-2, MPEG-4 and other video data encoding/decoding standards without any restriction.

It is also evident that each table can be described not only with information of reconfigurable connection of functional units for performing decoding according to one standard and processing processes requested according to a pertinent functional unit but also with information for performing decoding according to 2 or more standards. For example, it is assumed that the first plurality of frames of encoded video data included in an extended bitstream is encoded by MPEG-2, the following plurality of frames is encoded by MPEG-4, and the last plurality of frames is encoded by MPEG- 1. In this case, it is apparent that table information included in the decoder description to decode the encoded video data is described such way that each frame with a different encoding method allows functional units corresponding to each standard included in the tool box 510 to be systemically combined and performed.

[Industrial Applicability]

The present invention allows decoding a bitstream that is encoded by various types (syntax and semantics) in accordance with each standard (e.g. MPEG-I, MPEG-2, MPEG-4, MPEG-4 AVC) by using the same information recognizing method.

The present invention allows generating an extended bitstream including decoder descriptions for encoding a bitstream that can generate an extended bitstream including decoder description to decode bitstreams encoded by various types (syntax and semantics) in accordance with each standard by using the same information recognizing method.

Further, the present invention allows describing decoder descriptions more efficiently by employing hierarchical structure of codec to syntax parsing and decoding process. Further, the present invention allows suggesting scheduling management of each codec and systematic processing structure (i.e., parallel combination structure, sequential combination structure, independent processing structure, individual processing structure, etc.) of each functional unit by using the decoder description.

Further, the present invention allows designing and building various systems only with the above described decoder descriptions. Further, the present invention allows parsing a bitstream compressed by various encoding methods by using the same information analyzing method and controlling systematically each functional unit for decoding by using the parsed data. Further, the present invention allows universally applying a syntax analysis method to decode various types of bit streams.

Further, the present invention allows applying a new set of commands to parse various types of bitstreams by a common syntax analysis method.

Further, the present invention allows decoding easily bitstreams when a syntax element is changed, inserted, or deleted. Further, the present allows sharing elements used for the bitstream decoding of the element information (i.e. a result from syntax parsing) of analyzed syntax.

Further, the present invention allows using the element information of analyzed syntax for analyzing the syntax element of a following bitstream.

Further, the present invention can be applied when video and still image codecs processing by a block unit besides MPEG- 1 , MPEG-2, MPEG-4 MPEG-4 AVC are unified.

Further, the present invention allows storing functions which are included during various decoding processes suggested by several standards (codec) and divided by each functional unit into a tool box. Further, the present invention can select and use only necessary functional units from the tool box to decode a bitstream encoded by various types.

Further, the present invention allows easy changing, inserting, or deleting functional units stored in the tool box.

The drawings and detailed description are only examples of the present invention, serve only for describing the present invention and by no means limit or restrict the spirit and scope of the present invention. Thus, any person of ordinary skill in the art shall understand that a large number of permutations and other equivalent embodiments are possible. The true scope of the present invention must be defined only by the spirit of the appended claims

Claims

[CLAIMS]

[Claim 1 ] A decoding device comprising: a tool box comprising 2 or more of functional units each implemented to perform independently its process according to one or more of decoding standards for decoding a bitstream; a hierarchical information storing unit storing hierarchical information to operate systemically a part of or the whole of the functional units once or more times; and a decoding control unit activating one or more of functional units included in the highest layer by referring to the hierarchical information, wherein calls between layers are performed corresponding to the hierarchical information.

[Claim 2] The decoding device of Claim 1 , wherein the decoding control unit controls object creation and execution for the highest layer and the functional units included in the highest layer and a calling unit to call a layer determined by the hierarchical information are activated by the object execution.

[Claim 3 ] The decoding device of Claim 2, wherein the calling unit activates a pertinent layer when processing data, requested for one or more of functional units included in the determined layer to perform a predetermined process, is stored in the information storing unit.

[Claim 4] The decoding device of Claim 3, wherein the functional unit included in the activated layer starts operation when the processing data to perform a predetermined process is stored in the information storing unit.

[Claim 5 ] The decoding device of Claim 1 , wherein the result data by the performance of the functional units is stored in the information storing unit.

[Claim 6] The decoding device of Claim 5, wherein the tool box comprises one or more of parsing units syntax parsing of the bitstream and 2 or more of decoding functional units decoding the encoded video data.

[Claim 7] The decoding device of Claim 6, wherein the decoding functional units each performs the processing when control signal and context information(CSCI) required for performing a predetermined process and data for decoding are stored.

[Claim 8 ] The decoding device of Claim 6, wherein a dedicated storing space is assigned for each of the decoding functional units.

[Claim 9] A decoding device comprising: an information storing unit; a control signal and context information(CSCI) storing unit where the control signal and context information(CSCI) generated by the parsing functional unit is stored; and a data storing unit where data for decoding processing which is at least one of data corresponding to the encoded video data generated by the parsing functional unit and data processed by the decoding functional unit is stored.

[Claim 10] A decoding method comprising: receiving a bitstream; and activating at least one functional unit belonging to the highest layer by referring to stored hierarchical information for decoding the bitstream, wherein 2 or more of functional units each, implemented to perform independently a process corresponding to one or more of decoding standards, is included to one of 2 or more layers, the hierarchical information is specific information to systemically operate a part or entire of the functional units once or more times, a call between layers is performed corresponding to the hierarchical information.

[Claim 11 ] The decoding method of Claim 10, wherein the step of activating one or more functional unit belonging to the highest layer controls object creation and execution of the highest layer, and activates the functional units included in the highest layer by the object execution and a calling unit to call another layer designated by the hierarchical information.

[Claim 12] A recording medium recorded with a program wherein the recording medium is implemented with a program of commands executable in the decoding device and is recorded with a program readable by the decoding device, the recording medium recorded with a program comprising: receiving a bitstream; and activating at least one functional unit belonging to the highest layer by referring to stored hierarchical information for decoding the bitstream, wherein 2 or more of functional units each, implemented to perform independently a process corresponding to one or more of decoding standards, is included to one of 2 or more layers, the hierarchical information is specific information to systemically operate a part of or the entire functional units once or more times, when processing data for performing one or more functional units in a first layer, which is a deactivated state, is stored corresponding to the hierarchical in the information storing unit, the first layer is called by a second layer which is in an activation state.