KR100835909B1 - Digital image processing device and operating method therefore - Google Patents

Abstract

A digital image signal processor and an operating method thereof are provided to modify only a supplementary information output device when supplementary information is changed in the decoding of A/V data and process the A/V data, thereby decoding the A/V data regardless of a change of the supplementary information. A digital image signal processor comprises a mass storage device(110), a decoding device(360), and an output device(130). The decoding device analyzes plural PES(Packetized Element Stream) packets to generate supplementary information output screen data and decoded A/V(Audio/Video) data. The decoding device comprises a first storage space(330), a second storage space(320), a searching device(310), a decoder(340), and a supplementary information output device(350). The first storage space stores a pair of time information and supplementary information included in each header of the plural PES packets. The searching device reads the PES packets and supplementary information corresponding to the PES packets from the mass storage device to output the read PES packets and the supplementary information to the first storage space and the second storage space. The supplementary information output device generates a supplementary information output screen from the supplementary information matching the time information stored in the first storage space in response to the time information, and outputs the generates supplementary information output screen to the output device.

Description

Digital image signal processing device and operating method thereof

1 is a block diagram of a conventional general digital image signal processing apparatus.

2 is a block diagram of a digital video signal processing apparatus including a conventional decoding device.

3 is an embodiment of a digital video signal processing apparatus including a decoding apparatus according to the present invention.

4A illustrates an embodiment of a presentation time stamp and additional information storage space of video / audio data according to the present invention.

FIG. 4B is an embodiment of a data structure actually stored in the storage space of FIG. 4A.

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a digital video signal processing apparatus and a method of operating the apparatus, and more particularly, to processing of additional information that can be changed.

When the video / audio data is compressed and stored by using the conventional video compression encoding technique, and then reproduced, for example, time information such as "year / month / date hour: minute: second" at the time of playback or a title for each channel Additional information such as may be output so that the user recognizes. At this time, synchronization between the video / audio data being played and the additional information should be performed. In this regard, the international standard (ISO / IEC 13818-1) describes the process of creating a single stream (STREAM) through video / audio data encoding, and then decoding the video and audio data in synchronization with each other. Present a standard.

This standard does not describe the synchronization between side information and video / audio data, but as suggested by the standard, the side information can also be stored in one PES packet stream and processed in one decoding device as well as video / audio data. have.

However, in case of the additional information, it can be variously changed depending on the application, and thus the decoding device must be continuously modified accordingly. Therefore, if the decoding device of the video / audio data is implemented in hardware according to the recent trend, it is not easy to modify it. There was this.

1 is a block diagram of a conventional general digital image signal processing apparatus.

The mass storage device 110 stores compressed and encoded video / audio data including additional information. The decoding device 120 interprets such data, generates additional information output screen data, and decoded video / audio data and transmits the same to the output device 130. The output device 130 is a device that outputs the data received from the decoding device 120 to a screen that can be recognized by the user.

2 is a block diagram of a digital video signal processing apparatus including a conventional decoding device.

The general decoding apparatus 120 includes a searcher 220, a video / audio data storage space 230, a decoder 240, and an additional information output 250.

The searcher 220 reads the additional information to be synchronized and output from the mass storage device 110 with the compressed / encoded video / audio data and the video / audio data, and stores the additional information in the video / audio data storage space 230.

The decoder 240 reads the compressed coded video / audio data and additional information stored in the video / audio data storage space 230 and processes the decoding process. Through this, the decoded video / audio data to be finally output is output to the output device 130 and the additional information is transmitted to the additional information outputter 250.

The additional information output unit 250 configures the actual additional information output screen data from the additional information output by the decoder 240 and outputs the actual additional information output screen data to the output device 130.

The decoder 240 receives the side information and the compression coded video / audio data together to synchronize the side information and the video / audio data together. At this time, the additional information to be output together when the video / audio data is reproduced is transferred to the additional information output unit 250 to generate additional information output screen data.

If the additional information is added or changed, the decoder 240 must be changed together to properly transmit the changed additional information to the additional information outputter 250. Recently, the decoder 240 supporting video compression coding standards such as MPEG1 / MPEG2 / MPEG4 / H.263 / H.264 has the largest amount of computation in the digital image signal processing apparatus, so that it is possible to improve the processing speed. Often implemented in ASIC) hardware. In this case, there is a difficulty in changing the decoder 240 which is hardware whenever the additional information is changed. Accordingly, there is a need for a digital image signal processing apparatus including a decoder capable of processing additional information without changing the decoder for various changes and additions of the additional information.

An object of the present invention is to provide a digital video signal processing apparatus having a decoder that can be used regardless of whether or not the additional information is changed.

Another object of the present invention is to provide a method of operating a digital image signal processing apparatus having a decoder that can be used regardless of whether additional information is changed to achieve the above object.

The digital image signal processing apparatus of the present invention for achieving the above object comprises a decoding device for decoding a plurality of PES packets (PACKETIZED ELEMENT STREAM) stored in the mass storage device and outputs the output to the output device, the decoding device is a PES packet of each A first storage space for storing the time information included in the header and the additional information in pairs, a second storage space for storing the compressed coded video / audio data included in the PES packets, and reading PES packets from the mass storage device. A searcher for outputting the first storage space and the second storage space, a decoder for decoding the compressed coded video / audio data stored in the second storage space, and outputting time information, and a first storage space in response to the time information. The additional information output device for generating the additional information output screen from the additional information matching the stored time information and outputting the additional information output screen to the output device. It is characterized by including.

Time information of the present invention for achieving the above object is PTS (PRESENTATION TIME STAMP) information, or PTS (PRESENTATION TIME STAMP) / DTS (DECODING) when the digital image signal processing apparatus does not use a non-frame (B-FRAME) TIME STAMP) information.

In order to achieve the above object, a searcher of the present invention retrieves compressed / coded video / audio data from PES packets stored in a mass storage device and outputs the compressed / encoded video / audio data to a second storage space. And outputting the data to the first storage space in pairs.

The decoder of the present invention for achieving the above object is characterized in that for outputting the time information of the last decoded video / audio data to the additional information output.

A method of operating a digital image signal processing apparatus for achieving the another object includes searching for PES (PACKETIZED ELEMENT STREAM) packets and additional information, and storing the data generated from the retrieved PES packets and additional information, respectively. Decoding the encoded / encoded video / audio data among the data and outputting time information of the compressed / encoded video / audio data; and generating an additional information screen in response to the time information. It is done.

According to another aspect of the present invention, a data storage step includes storing time information of video / audio data compression-coded from PES packets and additional information retrieved together, and video / compressed from PES packets. And storing the audio data.

The additional information outputting step for achieving the another object may include generating an additional information output screen of the additional information corresponding to the time information in response to the time information.

Hereinafter, a digital image signal processing apparatus and a processing method thereof according to the present invention will be described with reference to the accompanying drawings.

3 is an embodiment of a digital video signal processing apparatus including a decoding apparatus according to the present invention. Description of the same parts as in FIG. 1 will be omitted to avoid duplication.

The decoder 360 according to the present invention includes a searcher 310, a video / audio data storage space 320, PTS (PRESENTATION TIME STAMP) information and additional information storage space 330 of the video / audio data, and a decoder 340. , And the additional information outputter 350.

Video or audio data compressed and encoded according to the international standard (ISO / IEC 13818-1) is composed of PES (PACKETIZED ELEMENT STREAM) packets. One PES packet becomes one output unit and is composed of a PES header, a video, and an audio data body. The PES header stores PTS information for synchronizing video / audio streams stored in each of the PES packets.

In general, the time stamps are time information values required for decoding and reproduction added to a stream based on the MOVING PICTURE EXPERTS GROUP. The time stamp of an MPEG-based stream includes PTS (PRESENTATION TIME STAMP) and DTS (DECODING TIME STAMP) information.

The PTS information is used when the synchronization between video, audio, and additional information needs to be synchronized as time information in reproduction output. For example, when a person appearing on a screen displayed to the user through the output device 130 speaks with open mouth, audio data should be played together. In this way, the PTS information includes time information at the time of reproduction output of the audio data to be reproduced in synchronization with the video data.

In addition, the DTS information is a unit of compressed code as a relative time value referred to during a type of decoding reproduction process. This is because the i-frame, which is the unit of encoding the actual screen, is encoded and output before the non-frame (B-FRAME) during the encoding process, so that the decoding order and the output order to be reproduced differ. Do. Therefore, the actual output order and decoding order are matched with reference to the DTS information stored in the PES packet header.

When the additional information is output to the user through the output device 130 as described above, the video and audio data and the additional information are synchronized with reference to the PTS information. However, when the implemented digital video signal processing apparatus supports the international standard (MPEG4) and does not use a non-frame (B-FRAME), since the decoding order and the output order of the encoding process are the same, PTS information and DTS information are used. Is the same, and DTS information may also be used.

The additional information corresponding to the PES packet may be additionally located before the PES packet header or processed as a separate stream composed of only other additional information according to an actual implementation method. When processed as a separate additional information stream, synchronization with video / audio data may be processed by referring to time information included in the additional information.

The searcher 310 reads video / audio data and additional information compressed and encoded by the mass storage device 110. Since the compressed / encoded video / audio data is composed of PES packets defined according to the video compression encoding standard, the PTS information and additional information of the video / audio data included in the PES packet header are converted into the PTS information and additional information of the video / audio data. The storage space 330 is stored.

As described above, in order to process the additional information, the PTS information of the compressed / encoded video / audio data is stored together in the PTS information and the additional information storage space 330 of the video / audio data. To match motives.

The PTS information and additional information storage space 330 of the video / audio data is a temporary storage space in which the PTS information and additional information of the plurality of video / audio data read by the searcher 310 are paired. Embodiments of the PTS information and the additional information storage space 330 of the video / audio data will be described in detail with reference to FIGS. 4A and 4B.

The searcher 310 outputs the PTS information and the additional information of the video / audio data by pairing the searched additional information together with the PTS information obtained from the PES header of the video or audio data searched for playback read from the mass storage device 110. To store the PTS and additional information storage space 330 of the video / audio data. PES packets of compression-encoded video / audio data are stored in the video / audio data storage 320 for decoding. Since the PES packet header includes the PTS information, the decoder 340 may use the PTS information to synchronize video / audio data.

The PTS information and the additional information storage space 330 of the video / audio data stored in pairs with the PTS information in the header of the PES packets including the played video / audio data and the additional information to be output together with the video / audio data are stored. Include. Compressed coded video / audio data stored in the video / audio data storage space 320 is output to the decoder 340.

The decoder 340 finally delivers the PTS information of the decoded video / audio data to the additional information outputter 350. Lastly, the decoded video / audio data is data to be output on the screen through the actual output device 130, and is to check whether the additional information outputter 350 has additional information with corresponding PTS information.

The decoder 340 implemented with hardware for decoding actual video / audio data does not directly process additional information, which may be changed as necessary, as shown in FIG. 3, but only synchronization between the additional information and video / audio data. Only the last PTS information that is needed for reading is read from the PES packet header and output to the additional information output unit 350. This is different from the conventional decoder 240 directly processing the side information.

In addition, the decoder 340 reads, decodes, and outputs the PES packet of the video / audio data stored in the video / audio data storage space 320 to the output device 130.

The additional information output unit 350 compares the PTS information of the last encoded packet received from the decoder 340 with the PTS information of the video / audio data and the PTS information stored in the additional information storage space 330. Read the information. If the additional information having the same PTS value as the PTS information exists, the additional information output screen data is generated from the corresponding PTS information and output to the output device 130.

An implementation between the additional information output unit 350 and the PTS information of the video / audio data and the additional information storage space 330 is performed by using the PTS information received by the additional information output unit 350 from the decoder 340. Can be read from Alternatively, after temporarily storing the PTS information and the additional information output from the searcher 310 in the corresponding storage space 330, the additional information outputter 350 does not search the storage space 330 separately. The data 310 may be implemented to process the data output first. Embodiments related to this will be described later in more detail with reference to FIG. 4A.

As another embodiment between the decoder 340 and the side information output unit 350, the side information output unit 350 is a PTS input of the PTS information of the video / audio data and the data stored in the side information storage space 330. Information may be output to the decoder 340. In response to this, the decoder 340 compares the PTS information of the PES packet, which finally performed the decoding process, and notifies the additional information output unit 350 of whether there is a match. Upon receiving this information, the additional information output unit 350 confirms that there is additional information to be output in synchronization with the corresponding video / audio data when the PTS information matches, and generates the additional information output screen to the output device 130. Output

The output device 130 finally outputs the decoded video / audio data output from the decoder 340 and the additional information output screen data output from the additional information output unit 350.

Accordingly, the additional information may be processed for the decoder 340 that supports various video compression coding standards such as MPEG1 / MPEG2 / MPEG4 / H.263 / H.264 without modification of the decoder 340. As such, the decoder 340 does not read and process additional information directly from the storage spaces 320 and 330, and outputs the PTS information of the video / audio data to be output to the additional information outputter 350 to be actually output. do. This uses the fact that the additional information output screen must be configured with reference to the PTS information in order for the additional information to be displayed on the screen in synchronization with the video / audio data.

Therefore, when the decoder 340 is configured with hardware to perform decoding of video / audio data, when the additional information is changed, only the additional information outputter 350 may be modified to process the data. The additional information output unit 350 is often implemented in software because the operation amount to be processed is relatively small compared to the decoder 340. Therefore, when the decoder 340 is implemented as a chip (CHIP), which is hardware, and the additional information output unit 350 is implemented in software, the additional information output unit 350 may be modified whenever a change in the additional information occurs. It is relatively easy.

4A illustrates an embodiment of a PTS information and side information storage space of video / audio data according to the present invention.

The PTS information and additional information storage space 330 of the video / audio data may be implemented to have a QUEUE structure as shown in FIG. 4A. When data paired with PTS information and additional information of compressed / encoded video / audio data (ID1_DATA, ID2_DATA, ID3_DATA) are input to the PTS information and additional information storage space 330 of video / audio data, first input is performed. This data has a first-in-first-out (FIRST-IN FIRST-OUT) structure in which the first data is first output to the additional information output unit 350.

In addition, the additional information output unit 350 receives the PTS information, retrieves additional information having the same PTS information from the PTS information of the video / audio data and the additional information storage space 330, and generates an additional information output screen from the additional information. It may be implemented in a structure. As another embodiment, a controller may be separately provided to manage the PTS information and the additional information storage space 330 of the video / audio data, and to search for the PTS information input from the additional information outputter 350.

In addition, the size of the PTS information and the additional information storage space 330 of the video / audio data is determined according to the processing speed of the searcher 310 and the additional information output device 350, the processing speed of the system, and the like.

FIG. 4B is an embodiment of a data structure actually stored in the storage space of FIG. 4A.

An embodiment of each configuration of data included in each of the data ID1_DATA, ID2_DATA, and ID3_DATA stored in the PTS information of the video / audio data and the additional information storage space 330 of FIG. 4A is as follows.

A data structure 410 including data (ID1_DATA, ID2_DATA, ID3_DATA) paired with PTS information and additional information of compression-encoded video / audio data includes PTS information (PTS) and additional information therein. .

The PTS information (PTS) represents a relative time value at which video / audio data and side information are decoded. When the PTS information PTS is 1000 from Fig. 4B, the additional information to be displayed on the screen together is the channel information value ("CHANNEL 1"). If the PTS information (PTS) is 1100, the additional information ("CAMERA 1") including the camera information is used. If the PTS information (PTS) is 1200, the time information ("2006/12/21 15:48:02") ) May be included.

For example, if the PTS information of the last decompressed image output by the decoder 340 is 1000, the PTS information and the additional information storage space 330 of the video / audio data with reference to the stored PTS information, and the corresponding additional information. ("CHANNEL 1") is output to the additional information output unit 350.

Additional information such as a recording position (“APARTMENT 1”) may be added from the embodiment as shown in FIG. 4B, and FIG. 4B is an example in which one additional information exists for the PTS information PTS, but in reality, one PTS is present. A plurality of additional information may be added to the information PTS.

For example, the PTS information PTS has 1000 and includes additional information "CHANNEL 1" and additional information "APARTMENT 1". Also in this case, the additional information output unit 350 configures and outputs a corresponding additional information output screen from the PTS information (PTS) of the last decoded video data having the PTS information (PTS) value of 1000.

When having the data structure as shown in Fig. 4B, the operation of the decoding device 360 of Fig. 3 is as follows. Description of the same parts as in Fig. 3 is omitted to avoid duplication.

The searcher 310 retrieves video / audio data from the mass storage device 110 and analyzes additional information changed from the video / audio data and PES packet header. Thereafter, the changed additional information and the PTS information of the video / audio data are output as a pair ID1_DATA to the PTS information and the additional information storage space 330 of the video / audio data. From FIG. 3A, video / audio data is output to video / audio data storage space 320.

When the PTS information of the image decompressed last by the decoder 340 is 1000, the PTS information of the video / audio data and the PTS information of the data ID1_DATA of the additional information storage space 330 are matched. 350 configures output screen data including additional information ("CHANNEL 1") and outputs it to the output device 130. Alternatively, when other additional information (“APARTMENT 1”) having a corresponding PTS information value is present, output screen data including a plurality of additional information may exist.

From this, the video / audio data is decoded by the decoder 340 regardless of the change of the additional information as described above. Regardless of the change of the additional information, only the PTS information of the decompressed image is output to the additional information outputter 350 so that the decoding process of the video / audio data proceeds regardless of the change of the additional information.

In contrast, the PTS information of the video / audio data and the data structure of the additional information storage space 330 may be changed as necessary to process the added additional information (“APARTMENT 1”). That is, the data structure of FIG. 4B may be changed according to the changed characteristic of the additional information or the change of the output data of the additional information output unit 350 and the searcher 310.

Although the above has been described with reference to a preferred embodiment of the present invention, those skilled in the art will be able to variously modify and change the present invention without departing from the spirit and scope of the invention as set forth in the claims below. It will be appreciated.

Therefore, even if the additional information is changed, the digital image signal processing apparatus of the present invention can use the decoder provided therein without modification.

Claims (10)

A mass storage device for storing a plurality of PES packets (PACKETIZED ELEMENT STREAM); A decoder for interpreting the plurality of PES packets to generate additional information output screen data and decoded video / audio data; An output device for receiving the additional information output screen data and the decoded video / audio data and outputting the screen to a screen recognizable by a user; The decoding device A first storage space for storing time information and additional information included in a header of each of the plurality of PES packets in pairs; A second storage space for storing compressed coded video / audio data included in each of the plurality of PES packets; A searcher for reading the PES packets and additional information corresponding thereto from the mass storage device and outputting the PES packets to the first storage space and the second storage space; A decoder that performs a decoding process of the compression-encoded video / audio data stored in the second storage space and outputs time information; And And an additional information output unit configured to generate an additional information output screen from the additional information corresponding to the time information stored in the first storage space and output the output information to the output device in response to the time information. Device. The method of claim 1, wherein the time information is Digital image signal processing device characterized in that the PTS (PRESENTATION TIME STAMP) information. The method of claim 1, wherein the time information is The digital video signal processing device is PTS (PRESENTATION TIME STAMP) / DTS (DECODING TIME STAMP) information when the digital video signal processing apparatus does not use a non-frame (B-FRAME). The system of claim 1, wherein the searcher Retrieve the compressed coded video / audio data from the PES packets stored in the mass storage device and output the compressed / encoded video / audio data to the second storage space; And outputting the pair of time information included in each of the plurality of PES packets and additional information corresponding to each of the plurality of PES packets to the first storage space in pairs. The method of claim 1, wherein the decoder And outputting the time information of the finally decoded video / audio data to the additional information output device. Searching and reproducing PES (PACKETIZED ELEMENT STREAM) packets and additional information; A data storage step of storing data generated from the found PES packet and additional information, respectively; Decoding compression-coded video / audio data among the data and outputting time information of the compression-coded video / audio data; And And an additional information output step of generating an additional information output screen in response to the time information. The method of claim 6, wherein the time information is Method of operation of a digital video signal processing apparatus, characterized in that the PTS (PRESENTATION TIME STAMP) information. The method of claim 6, wherein the time information is And a PTS (PRESENTATION TIME STAMP) / DTS (DECODING TIME STAMP) information when the digital image signal processing apparatus does not use a non-frame (B-FRAME). The data storage step from claim 6 Storing the time information of the compressed / encoded video / audio data and the additional information retrieved together from the retrieved PES packet in pairs; And And storing the compression-encoded video / audio data from the retrieved PES packet. The method of claim 6, wherein the additional information outputting step And generating the additional information output screen of the additional information corresponding to the time information in response to the time information.

Images