JP2003324700A - Multiplex encoded information separator and decoder - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve a problem at the time of switching a program between a multiplex encoded information separator/distributor for separating appropriate encoded information from a multiplexed stream and an image decoding system having an image decoder for storing image encoded information sequentially in an internal buffer and decoding it while reading out sequentially that incomplete frame data is displayed at the time of switching because the start of new encoded information can not be recognized after switching. <P>SOLUTION: At the time of switching a program, a multiplex encoded information separating/distributing processing 101 adds a marker to the head of new encoded information so that an image decoding processing 102 can recognize the start of new encoded information. When the image decoding processing 102 finds a marker indicative of the head of new encoded information during encoding operation of the encoded information, a control processing 104 interrupts an image signal conversion processing 103 until the image decoding processing 102 can produce correctly decoded image frame data. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a decoding system for receiving and decoding multiplexed coded information, and more particularly to a decoding system using a general-purpose processor as hardware.

[0002]

2. Description of the Related Art In recent years, many digital moving image compression coding systems for handling moving image data have been standardized. MPEG-1 (ISO / IEC11172-2),
MPEG-2 (ITU-T H.262, ISO / IE
C13818-2), MPEG-4 (ISO / IEC1
4496-2) and the like.

In these encoding methods, one frame of image frame data is divided into 16 pixels × 16 pixels in macroblock units, and discrete cosine transform (DC) is performed.
T), etc., orthogonal transform, quantization of transform coefficient, motion compensation prediction,
Compression coding is performed using variable length coding or the like.

The image coded information includes information obtained by compression-coding such image frame data as well as information obtained by coding image configuration information such as the size of an image. These identifications are made by the start synchronization code and the start code following it. Generally, the above-mentioned configuration information is inserted at a certain interval in the image coded information in order to enable random access.

On the other hand, like BS digital broadcasting, digital cable broadcasting and terrestrial digital broadcasting, video, audio and related data are being broadcast as digital data. In an image decoding processing system such as a digital broadcast receiving system that handles image coding information, an ISO / IEC MPEG (Moving Pic) is used.
image compression coding information coded by the above coding technique, such as a program stream (hereinafter abbreviated as PS) or a transport stream (hereinafter abbreviated as TS) standardized by true Experts Group, Data in which audio compression coding information is multiplexed is used.

[0006] For example, a TS in the MPEG-2 system is a transport packet (hereinafter abbreviated as TP).
Is composed of packets called TP, and TP is a packet ID
It has an identification number called (hereinafter abbreviated as PID). In digital broadcasting, video and audio compression coding information of one or a plurality of broadcast programs is stored in TP for each PID.

[0007] Also, a PSI (Program) describing specification information of the broadcast program such as which broadcast program exists in the TS and in which TP the encoded information of the video and audio constituting the broadcast program is stored.
Specific Information) is also PI
It is stored in TP for each D.

[0008] Of the PSI, information indicating the relationship between the broadcast program and the video and audio composing the broadcast program is stored in the PMT (Pro
This is called "gram Map Table", and the information showing the relation between the broadcast program in the currently transmitted TS and its PMT is recorded in PAT (Program Associate).
on Table).

Each broadcast program has a program number (Pr
It is identified by a number called "Ogram Number". The PAT contains the program number of the broadcast program in the TS currently being sent and the PM of that broadcast program.
A table showing the correspondence with the PID of the TP storing T is stored. On the other hand, the PMT stores a table showing the correspondence between the streams such as video and audio forming the broadcast program and the PID of the TP storing the streams.

In order to view the broadcast program, the receiver first detects the PAT from the TS and acquires the PID of the TP storing the PMT corresponding to the broadcast program desired to be viewed. Next, the receiver detects the PMT from the TS based on the PID, and acquires the PID of the TP that stores the encoded information such as video and audio forming the broadcast program. Then, based on these PIDs, a PES (Packetized Elem) in which video and audio compression encoding information and the like are stored from TP
entry stream) packet and PE
Encoding information and PTS (Presenta) from S packet
function time stamp) and DTS (Deco
Ding Time Stamp) and the like. PT
S indicates time information for reproducing and outputting the data obtained by decoding the coding information in the PES packet, and DTS indicates time information for decoding the coding information in the PES packet.

In the image decoding system, the multi-coding information separating device separates the image compression coding information and the audio compression coding information of the broadcast program to be viewed from the TS obtained from the tuner unit and the like, and the separated image code The image decoding device decodes the encoded information. Next, the image signal conversion device converts the decoded image frame data into an image signal and outputs the image signal to the display device. Then, these devices are controlled by the control device.

The multiple coded information separation device acquires an appropriate PID from the PSI by the method described above and separates the coded information forming the program from the TS. The image decoding apparatus first acquires the image configuration information from the image encoding information by the start synchronization code and the start code, and notifies the control apparatus of this. For example, in MPEG-2, the sequence
Image configuration information is stored in a header (Sequence Header).

Next, similarly, a variable length decoding device which obtains compression coding information of image frame data from the coding information by a start synchronization code and a start code and decodes the variable length coded coding information, and a variable length An inverse quantizer that performs inverse quantization on block data obtained by decoding, an inverse orthogonal transform device that performs inverse orthogonal transform on block data after dequantization, and motion compensation from past or future image frame data. Decoding is performed using a motion compensation prediction device that performs prediction, and a frame buffer that accumulates image data in macro block units decoded by these devices in frame units.

[0014]

As described above, when the image decoding system receives a designation of a desired program as a stream such as a TS in which coded information is multiplexed, the image forming the program is input from the input stream. The encoded information of is separated and this is decoded.

A general-purpose processor having a DMA transfer device is used as hardware for realizing the image decoding system,
When the function is realized by the software module, the image decoding device sequentially transfers the encoded information to the internal buffer by the DMA transfer device, and performs the decoding while sequentially reading the encoded information. It is considered that the program in the input stream is switched and decoded by using such an image decoding processing system.

The image decoding apparatus acquires the image configuration information based on the start synchronization code and the start code from the image coding information of the program after switching. Next, the coding information of the image frame data is acquired from the coding information based on the start synchronization code and the start code, and is decoded. However, if the coding information of the image frame data acquired at this time is coded using inter-frame prediction that refers to past image frame data, the image frame data may not be correctly decoded.

For example, the image coding information is MPEG-2.
If the B picture following the first I picture after the sequence header has been coded using inter-frame prediction that refers to a past picture, immediately after switching, the past reference frame Since this does not exist, this B picture cannot be correctly decoded. Therefore, in the image decoding system, the display is started from the image frame data encoded by using the intra-frame prediction such as the above-mentioned I picture which first appears after the switching.

Then, the image decoding system does not display the image frame data encoded using the inter-frame prediction that refers to the past image frame such as the B picture appearing before the I picture. However, since the image encoding information of the program before switching exists in the internal buffer of the image encoding device and it is not possible to recognize where the image encoding information of the program after switching starts, the above-described I The picture cannot be recognized.

[0019]

According to the present invention, at the time of program switching, the multiple coded information demultiplexing device adds a marker indicating a break in the coded information to the beginning of new coded information, and adds the marker to the image. The output to the decoding device enables the image decoding device to recognize the beginning of new encoded information. When the image decoding apparatus finds this marker during the decoding of the image coded information, the image decoding apparatus reads the coded information until before the first image configuration information that appears after this marker, and starts decoding the coded information from here.

Then, the image decoding system suspends the display of the image frame data obtained by decoding the encoded information after the configuration information until the image frame data encoded by using the intra-frame prediction is obtained. It will be configured.

[0021]

FIG. 2 is a diagram showing a hardware configuration of an embodiment of an image decoding system to which the present invention is applied. The general-purpose processor in this embodiment has a central processing unit 201, a main memory unit 202, a DMA control unit 203, and a multiple coded information input unit 204, which are connected by a system bus 205.

The main storage device 202 stores a program group for realizing the functions of the image decoding system, and a work area for the program group is secured. The central processing unit 201 controls each device connected to the system bus,
The processing of the program group in the main storage device 202 is performed.
The DMA transfer device 203 receives a control signal from the central processing unit 201 and the multiple coded information input device 204, and centrally transfers data between the main storage device 202 and the coded information input device 204 connected to the system bus 205. This is performed without the intervention of the arithmetic processing unit 201.

FIG. 1 is a diagram showing a processing procedure in an image decoding system which is an embodiment of the present invention. The multiple coded information separation / distribution processing 101 receives the multiplexed coded information 105 such as TS from the multiplex coded information input device by DMA transfer, separates the corresponding coded information from this, and DMAs this. It is output to the image decoding process 102 by transfer.

The image decoding process 102 receives the image coding information from the multiple coding information separation / distribution process 101, decodes the received image coding information, and the frame data of the decoded image and the decoded image are coded by intra-frame prediction. The picture type information indicating whether the picture is encoded or encoded by intra-frame prediction is output to the image signal conversion processing 103. Also,
The image processing information acquired from the encoding information is controlled by the control processing 104.
To notify.

The image signal conversion processing 103 is a control processing 10
The image frame data obtained from the image decoding process 102 is converted based on the image configuration information obtained from No. 4, and the image signal 106 is output. The control process 104 notifies the image signal conversion process 103 of the image configuration information obtained from the image decoding process 102.

When the program is switched, control processing 1
Reference numeral 04 denotes program switching to the program number 107, multiplex coded information separation / distribution processing 101, and image decoding processing 10.
2 and the image signal conversion processing 103. Upon receiving this notification, the multiple coded information separation / distribution processing 101 separates the coded information of the image and the audio which form the program,
The encoded information with the marker indicating the program switching is output.

The image decoding process 102 skips the coded information until a marker is found in the coded information. When the marker is found, the image decoding process 102 notifies the control process 104 of the discovery of the marker, and then the coding information is skipped until new configuration information after program switching is found.

On the other hand, the image signal conversion processing 103 stops the output of the image signal 106 when receiving the notification of the program switching. Then, when the frame data of the image coded by the intra-frame prediction is received, the output of the image signal 106 is restarted.

FIG. 3 is a diagram showing the multiple coded information separation / distribution processing in the embodiment of the present invention. The multiple coded information 307 such as TS is transmitted from the multiple coded information input device to the DM.
By the A transfer, the data is sequentially stored in the input buffer 301. The multiple coded information separation process 302 manages the PSI table based on the program number set by the control process, and sequentially reads the TPs in the input buffer 301 to separate the coded information forming the specified program. Then, this is stored in the image encoding information buffer 303 and the audio encoding information buffer 305.

The image coding information distribution processing 304 is performed by the DMA.
By the transfer, the image coded information 308 is transferred from the image coded information buffer 303 to the image decoding process. On the other hand, the voice coded information distribution processing 305 transfers the voice coded information 30 from the voice coded information buffer 305 by DMA transfer.
9 is transferred to the voice decoding process.

FIG. 4 is a diagram showing the structure of the PSI table in this embodiment. In the PSI table, the program number 401, the PMT 403 corresponding to the program,
An image 404 that constitutes a program and a PID 402 such as a sound 405 are stored.

FIG. 5 is a diagram showing an image decoding process according to this embodiment. In the present embodiment, the image coding information coded in the MPEG-2 format is decoded.
TU-TH. 263 and MPEG-4 (ISO / IEC
It is also applicable to the case of decoding image coding information coded in a format such as 14496-2).

The variable-length decoding processing 502 sequentially reads the image coded information 508 transferred to the coded information buffer 501 by DMA transfer, and performs variable-length decoding on the coded information 508 to macro-code the image configuration information and the image frame data. Information that is divided into blocks and compression-encoded is obtained. When the configuration information is acquired, the control process is notified of the content.

In the inverse quantization processing 503, the macroblock data is inversely quantized with reference to the quantization scale, the quantization matrix and the like. The inverse orthogonal transform process 504 performs inverse orthogonal transform of the quantized macroblock data. Then, when the image frame currently being decoded is a P picture or a B picture, the motion compensation processing 505 refers to the reference image frame data based on information such as the prediction method and the motion vector, and performs motion compensation prediction.

The image frame data storage processing 506 stores the macroblock data obtained by the above processing in the decoding image frame data area in the frame buffer 507. Further, when the decoding process of the macroblock for one image frame is completed, the image frame data storage process 506
Outputs the processed macroblock to the image signal conversion processing.

If the decoded image frame is an I or P picture, the image frame data storage processing 506
The older of the two reference image frame data in the frame buffer 507 is discarded, and the decoded image frame data is newly used as reference image frame data.

The markers in the embodiment of the present invention will be described with reference to FIGS. 6 to 8. The image coding information is coded hierarchically. Then, each layer in the encoded information is identified by a start synchronization code and a start code that follows it. The start sync code indicates a division of the hierarchical structure, and the start code following the start sync code indicates the type of hierarchy. A code for position adjustment is inserted before the start synchronization code so that the start position of the start synchronization code in the encoded information has byte alignment. In this embodiment, image coding information coded in the MPEG-2 format is used.
PEG-1 and ITU-TH. 263, MPEG-4
Image encoding information encoded in a format such as (ISO / IEC 14496-2) may be used.

FIG. 6 is a diagram showing the start synchronization code in MPEG-2, and FIG. 7 is a diagram showing the start code in MPEG-2 and the hierarchy corresponding thereto. In MPEG-2,
A 0 bit is inserted before the start sync code so that the start position of the start sync code is in byte alignment. In the present embodiment, as the marker, a code including a start synchronization code and a sequence error that is one of the start codes is used as shown in FIG. 8, but another code may be used.

The flow of the program switching process in the multiplex coded information separation process of the multiplex coded information separation / distribution process of this embodiment will be described with reference to FIGS. 9 to 12.

FIG. 9 is a diagram showing the flow of processing in the multiple coded information separation processing of this embodiment. When the program number is received from the control process (step 901), the multiple coded information separation process is initialized (step 90).
2). Here, the PSI table is initialized and the PSI
The program number is stored in the table. Also, the respective start flags used in the PAT processing, PMT processing, image processing, and audio processing described later are cleared.

Next, the TP is read out from the input buffer (step 903), and the PID of the TP is compared with the PID in the PSI table (step 904). If the PID is PAT, the PAT processing (step 905) is performed. If the PID is PMT, the PMT processing (step 906) is performed. If the PID is image coding information, the image processing (step 907). However, if the PID is voice coded information, voice processing (step 908) is performed.

FIG. 10 is a diagram showing the flow of PAT processing in the multiple coded information separation processing of this embodiment. P
In the AT process, the PMT PID of the PSI table is the initial value (step 1001), this TP has data starting from the beginning of the PAT (step 1002), and the PAT process start flag is not set (step 1).
003) and set a start flag (step 1004).
The PAT information is extracted from the TP and the PAT is reproduced (step 1005) (step 1006).

If the start flag is set, the PMT PI corresponding to the program number from the reproduced PAT.
D is taken out and stored in the PSI table (step 1007). On the other hand, if the TP does not have data starting from the beginning of the PAT and the start flag has already been set (step 1005), the PAT information is extracted from the TP and the PAT is reproduced (step 1006).

If the PMT PID in the PSI table has already been set (step 1001) or if the start flag has not been set (step 1005), the process is terminated.

FIG. 11 is a diagram showing the flow of the PMT processing in the multiple coded information separation / distribution processing of this embodiment. The PMT processing is performed by the PSI table image and audio PI.
D is the initial value (step 1101), and this TP is PMT.
Has data starting from the beginning of (step 1102),
If the start flag for the PMT process is not set (step 1103), the start flag is set (step 11
04), the PMT information is taken out from the TP and the PMT is reproduced (step 1105) (step 1106).

If the start flag is set, the P and P of the image and audio that make up the program are composed of the reproduced PMT.
The ID is taken out and stored in the PSI table (step 1107). On the other hand, if the TP does not have data starting from the head of the PMT and the start flag has already been set (step 1105), the PMT information is extracted from the TP and the PMT is reproduced (step 1106).

If the image or audio PID in the PSI table has already been set (step 1101) or the start flag has not been set (step 1105), the process ends.

FIG. 12 is a diagram showing the flow of image processing in the multiple coded information separation processing of this embodiment. In the image processing, when the TP finds data starting from the head of the PES (step 1201), the encoded information is extracted from the PES reproduced so far and stored in the encoded information buffer (step 1202). Then, if the start flag is not set (step 1203), the start flag of the image processing is set (step 1204), and the marker indicating the switching to the program is stored in the encoding information buffer (step 1205).

Then, the PES information is extracted from the TP and the PES is reproduced (step 1206, step 1207). If the start flag has already been set, T
The PES information is taken out from P and the PES is reproduced (step 1207). If the TP does not have data starting from the beginning of the PES, the start flag is checked (step 1206), and if the start flag is set, the PES information is taken out from the TP and the PES is reproduced (step 1206). 1207).

FIG. 13 is a diagram showing the flow of the program switching process in the variable length decoding process of the image decoding process of this embodiment. In the variable length decoding process, the start encoding code is searched (step 1302) while sequentially reading the image encoding information stored in the encoding information buffer by DMA transfer (step 1301). When the start synchronization code is detected (step 1302), appropriate processing is performed (steps 1304 to 1309) based on the subsequent start code (step 1303).

When the start code is a marker, the control processing is notified that the marker is found in the encoded information (step 1309). Since the coding information after the marker is an image of a new program, in order to obtain new structure information from the coding information, the coding information is further skipped to the sequence header appearing after the marker (step 1309). When the start code is a sequence, the control process is notified of the configuration information of the image obtained by variable-length decoding the coding information (step 1304).

[0052]

According to the present invention, it is possible to smoothly switch programs without displaying incomplete image frame data generated when switching programs.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of a decoding system according to an embodiment of the present invention.

FIG. 2 is a diagram showing a configuration of hardware that implements a decoding system according to an embodiment of the present invention.

[Fig. 3] Fig. 3 is a diagram illustrating the configuration of multiple encoded information separation / distribution processing.

FIG. 4 is a diagram showing a structure of a PSI table.

FIG. 5 is a diagram showing a configuration of an image decoding process.

FIG. 6 is a diagram showing an MPEG-2 start synchronization code.

FIG. 7 is a diagram showing a start code of MPEG-2.

FIG. 8 is a diagram showing a structure of a marker.

FIG. 9 is a diagram showing a flow of program switching in the multiplex encoded information separation processing in the multiplex encoded information separation / distribution processing.

FIG. 10 is a diagram showing a flow of a PAT process in the multiple coded information separation process.

FIG. 11 is a diagram showing a flow of processing for PMT in the multiple coded information separation processing.

FIG. 12 is a diagram showing a flow of image processing in the multiple coded information separation processing.

[Fig. 13] Fig. 13 is a diagram illustrating the flow of a variable length decoding process of an image decoding process.

[Explanation of symbols]

201 ... Central processing unit, 202 ... Main storage device, 20
3 ... DMA transfer device, 204 ... Multiple encoded information input device, 205 ... System bus, 301 ... Input buffer,
303 ... Image coded information buffer, 305 ... Audio coded information buffer, 501 ... Coded information buffer, 5
07 ... Frame buffer.

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Atsushi Kawaguchi             1099 Ozenji, Aso-ku, Kawasaki City, Kanagawa Prefecture             Ceremony company Hitachi Systems Development Laboratory (72) Inventor Taku Tashiro             3 shares at 6-16 Shinmachi, Ome City, Tokyo             Hitachi Device Development Center F-term (reference) 5C059 MA00 RB02 RB08 RB10 RC32                       SS02 UA02 UA38 UA39                 5C063 AA01 AB03 AB07 CA36 DA07                       DA13 DB10                 5K028 AA14 EE03 EE05 KK01 KK03                       KK12 MM08 SS05 SS15 SS24

Claims (5)

[Claims] 1. A method for separating coded information of images and audios constituting a target program from a stream in which coded information such as images and audios constituting a plurality of programs are multiplexed. The encoded stream is stored in a buffer, and when the program is switched to the target program, the multiplex code forming the target program is separated from the multiplexed stream, and the program is added to the head of the separated coded information. A method for separating encoded information, which is characterized by adding a marker indicating the switching of the. 2. A method for decoding the coded information of images and audios constituting a target program from a stream in which the coded information of images and voices constituting a plurality of programs are multiplexed. The encoded stream is stored in a buffer, and when the program is switched to the target program, the multiplex code forming the target program is separated from the multiplexed stream, and the program is added to the head of the separated coded information. A marker indicating the switching of the marker is added, the coded information to which the marker is added is detected, and after the coded information to which the marker is added, information for controlling the coded information is detected, and the detected A decoding method, characterized in that the encoded information is decoded based on information. 3. A multiple coded information separating device for separating coded information of images and audios constituting a target program from a stream in which coded information such as images and voices constituting a plurality of programs are multiplexed. And a means for storing the multiplexed stream in the buffer, having a buffer, and a multiplexing code forming a target program from the multiplexed stream when switching to the target program. A multiple coded information separation device comprising: a separation means; and a means for adding a marker indicating a program change to the head of the separated encoded information. 4. A coding device for decoding coded information of images and audios constituting a target program from a stream in which coded information such as images and voices constituting a plurality of programs are multiplexed, comprising a buffer. And means for storing the multiplexed stream in the buffer, and means for separating a multiplexing code constituting the target program from the multiplexed stream when switching to the target program. A unit for adding a marker indicating the switching of the program to the head of the separated encoded information; a unit for detecting information that controls the encoded information after the encoded information to which the marker has been added; Decoding means for decoding the coded information based on the decoded information. 5. The decoding apparatus according to claim 4, further comprising means for stopping decoding of the coded information until information for controlling the coded information is retrieved.