JP2006254352A - Image compositing apparatus, digital tv broadcast receiving apparatus, digital tv broadcast receiving method, system lsi, program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce real-time properties required for a CPU to control image compositing, and to reduce the processing load on the CPU. <P>SOLUTION: An image decoder 104 decodes encoded data and outputs an image frame and its image frame timing information to a shared memory 102, to respectively store these data in respective memories 152, 151. A subtitle decoder 105 decodes the encoded data asynchronously with an image compositing section 101 and the image decoder 104, and outputs a subtitle frame 253 and its subtitle frame timing information in the shared memory 102 to store these data in respective memories 154, 153. The image compositing section 101 performs image compositing processing of the image frame and the subtitle frame, while referring to the shared memory 102 and a time counter 106, in accordance with the instruction stored in a display composition instructing memory 156 and outputted from a control unit 103, and then displays a composited image on a display part 107. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image synthesizing apparatus, a digital TV broadcast receiving apparatus, a digital TV broadcast receiving method, a system LSI, a program, which receives a digital TV broadcast encoded by MPEG2 or the like and synchronizes and synthesizes video data and caption data. The present invention relates to a recording medium.

In recent years, a new broadcast called a digital TV broadcast is becoming widespread in addition to a conventional analog broadcast. In digital TV broadcasting, various types of data such as video, audio, subtitles and other data are multiplexed and broadcast, received by the digital TV broadcast receiver, and the multiplexed and received various data for each data type. Processing such as decryption is performed, and playback is performed at a predetermined time. For example, video and text subtitles usually need to be combined and displayed on the same screen display unit, and a digital TV broadcast receiver is generally equipped with an image combining function. In addition, in order to synchronize with the video / audio, it is necessary to display the start / end of subtitle display at a timing designated by the broadcast station. Usually, this timing is processed by a CPU mounted on the receiver. (For example, refer to Patent Document 1).
JP 2000-331464 A

  However, in the conventional method, in order for the CPU to control the final display timing, it is necessary for the CPU to manage the decoding results of both video and subtitles and the timing information thereof. In recent years, the complexity of image coding has increased for image quality improvement and low bit rate, and this decoding also requires a large amount of processing, and can be decoded by a DSP or dedicated hardware. There are many more. However, for the display composition control, the CPU needs to grasp the video decoding status.

  In addition, since the screen display is normally updated at a frequency around 60 Hz, the display timing is controlled approximately every 1/60 s, and the image is disturbed if the composite content changes during the display. Must be done during the display blanking period. For this reason, the CPU to be controlled is required to have a considerable real-time property, and special measures are required when using a non-real-time OS or the like.

  Therefore, the present invention can greatly reduce the real-time property required for the CPU for image synthesis control, and reduce the processing load on the CPU by performing subtitle decoding and video decoding independently at different parts. An object of the present invention is to provide an image composition device, a digital TV broadcast receiving device, a digital TV broadcast receiving method, a system LSI, a program, and a recording medium.

  In order to solve the above problems, an image synthesizing apparatus according to the present invention includes a video decoding unit that decodes a video signal encoded by MPEG or the like in digital TV broadcasting and outputs video data and video timing information for synchronization thereof. A subtitle decoding unit that decodes subtitle information transmitted as a data broadcast in the digital TV broadcast and outputs subtitle data and subtitle timing information for synchronization thereof, and the video data and the subtitle data, respectively, Based on the video timing information and the caption timing information, an image synthesizing unit that synthesizes and displays in synchronism with the counter from the timer counter is employed.

  The image synthesizing apparatus of the present invention can greatly reduce the real-time property required for the CPU for image synthesizing control by adopting the above-described configuration, and the subtitle decoding and video decoding are different. This can be performed independently at each part, and the processing load on the CPU can be reduced.

  Hereinafter, an image composition device according to an embodiment of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing a specific configuration example of an image composition apparatus according to an embodiment of the present invention.

  As shown in the figure, the image composition apparatus according to the present embodiment includes an image composition unit 101, a shared memory 102, a control unit 103, a video decoding unit 104, a caption decoding unit 105, a timer counter 106, an LCD, and the like. A display unit 107 is provided.

  The image composition unit 101 refers to various data on the shared memory 102 and the timer counter 106 according to instructions such as setting in addition to the screen configuration from the control unit 103 stored in the display composition instruction memory 156 in advance. The synthesized video data is output to the display unit 107 by performing synthesis.

  The shared memory 102 has a readable / writable memory such as a video frame memory 152, a video frame timing memory 151, a subtitle frame memory 154, a subtitle frame timing memory 153, other image frame memory 155, and a display composition instruction memory 156. Video frame data which is video data, video frame timing information which is video timing information, subtitle frame data which is subtitle data, subtitle frame timing information which is subtitle timing information, other subtitle frame data, screen configuration and other Stores display composition instructions such as settings.

  The control unit 103 includes a CPU, a ROM, a RAM, and the like, and controls each unit by program processing to realize a desired image composition. Specifically, the control unit 103 controls the video decoding unit 104 and the caption decoding unit 105 to start / end decoding, while setting the display composition instruction memory 156 in addition to the screen configuration, Command) is output and stored, and the composition start / end control is performed on the image composition unit 101, and the operation start / end control and the timer value setting control are performed on the timer counter 106.

  The video decoding unit 104 includes a CPU or DSP, ROM, RAM, and dedicated hardware, and decodes video data encoded by program processing or hardware processing in accordance with instructions from the control unit 103. is there. A dedicated CPU or DSP, ROM, and RAM are not necessarily required, and some or all of them may be shared with the control unit 103, the caption decoding unit 105, and the like.

  The caption decoding unit 105 includes a CPU, a ROM, a RAM, and the like, and realizes decoding of caption data by program processing. A dedicated CPU, ROM, and RAM are not necessarily required, and some or all of them may be shared with the control unit 103, the video decoding unit 104, and the like.

  The timer counter 106 is a counter that is counted at a constant period. The counter value and the like can be updated by an instruction from the control unit 103, and a counter serving as a reference for image composition timing is output.

  The display unit 107 is an LCD or the like, and displays an output signal from the image composition unit 101.

  Next, an actual image composition operation in the image composition apparatus having such a configuration will be described with reference to FIG.

  When encoded data is input to the image synthesizer of the present embodiment, the video decoding unit 104 decodes the encoded data, and for example, as shown in FIG. 2, the nth video frame 231 and the video frame Timing information 241 such as “start: 0001” is output to the shared memory 102 and stored in the video frame memory 152 and the video frame timing memory 151, respectively.

  When the counter value of the timer counter 106 matches the “start: 0001” of the video frame timing information 241, the image composition unit 101 refers to the instruction from the control unit 103 stored in advance in the display composition instruction memory 156. Then, the synthesized image 201 is synthesized and output to the display unit 107 for display.

  Next, when the counter value of the timer counter 106 reaches “0002”, the image composition unit 101 refers to the instruction from the control unit 103 stored in the display composition instruction memory 156 in advance, and similarly the nth sheet. The synthesized image 202 is synthesized from the video frame 231 and output to the display unit 107 for display.

  By the way, the subtitle decoding unit 105 decodes the encoded data asynchronously with the image synthesizing unit 101 and the video decoding unit 104, and “XXXXXXX” which is the subtitle frame 253 and “start” which is the subtitle frame timing information 263. : 0003, end: 0005 "are output to the shared memory 102 and stored in the caption frame memory 154 and the caption frame timing memory 153, respectively.

  When the counter value of the timer counter 106 becomes “0003”, the image composition unit 101 refers to the instruction from the control unit 103 stored in the display composition instruction memory 156 in advance, and “ Based on “start: 0003, end: 0005”, the synthesized image 203 composed of the nth video frame 231 and the subtitle frame 253 “XXXXXXX” is synthesized, outputted to the display unit 107 and displayed.

  On the other hand, the video decoding unit 104 decodes the encoded data asynchronously with the image synthesizing unit 101 and the subtitle decoding unit 105, and the “starting:” is the (n + 1) th video frame 234 and its video timing information 244. 0004 "is output to the shared memory 102 and stored in the video frame memory 152 and the video frame timing memory 151, respectively.

  When the counter value of the timer counter 106 reaches “0004”, the image composition unit 101 refers to the instruction from the control unit 103 stored in advance in the display composition instruction memory 156 and refers to the (n + 1) th video frame 234. The synthesized image 204 is synthesized from “XXXXXXX” which is the caption frame 253, and is output to the display unit 107 for display.

  Next, when the counter value of the timer counter 106 becomes “0005”, the image composition unit 101 refers to the instruction from the control unit 103 stored in advance in the display composition instruction memory 156 and also displays the caption frame timing information 263. Similarly, based on “start: 0003, end: 0005”, the synthesized image 205 is synthesized from “XXXXXXX” that is the n + 1-th video frame 234 and the caption frame 253, and is output to the display unit 107 for display.

  On the other hand, the subtitle decoding unit 105 decodes the encoded data asynchronously with the image synthesizing unit 101 and the video decoding unit 104, and “YYYYYY” that is the subtitle frame 256 and “start” of the subtitle frame timing information 266. : 0006, end: 0007 "are output to the shared memory 102 and stored in the caption frame memory 154 and the caption frame timing memory 153, respectively.

  When the counter value of the timer counter 106 reaches “0006”, the image composition unit 101 refers to the instruction from the control unit 103 stored in the display composition instruction memory 156 in advance, and “ The synthesis of the subtitle frame 253 is ended based on “start: 0003, end: 0005”, while the (n + 1) th video frame 234 and subtitle frame 256 are based on “start: 0006, end: 0007” of the subtitle frame timing information 266. The combined image 206 with “YYYYYY” is combined and output to the display unit 107 for display.

  On the other hand, the video decoding unit 104 decodes the encoded data asynchronously with the image synthesizing unit 101 and the subtitle decoding unit 105, and the “start: Are output to the shared memory 102 and stored in the video frame memory 152 and the video frame timing memory 151, respectively.

  Next, when the counter value of the timer counter 106 becomes “0007”, the image composition unit 101 refers to the instruction from the control unit 103 stored in the display composition instruction memory 156 in advance, and the video frame 237 and the caption frame. The synthesized image 207 is synthesized from “YYYYYY”, which is 256, and is output to the display unit 107 and displayed.

  When the counter value of the timer counter 106 becomes “0008”, the image composition unit 101 refers to the instruction from the control unit 103 stored in the display composition instruction memory 156 in advance, and the subtitle frame timing information 266 “ Based on “start: 0006, end: 0007”, the synthesized image 208 is synthesized from only the video frame 237 and is output to the display unit 107 for display.

  Next, control of this apparatus by the control unit 103 will be described with reference to the flowchart shown in FIG.

  For example, when a screen update request is received from the higher-level device or the like that controls the image composition apparatus, the control unit 103 starts the control flow illustrated in FIG.

  That is, first, in step 301, the control unit 103 determines the target screen configuration from the screen requested by the higher-level device or the like, that is, the target frame to be used for specifying the video decoding unit 104 or the subtitle decoding unit 105, respectively. Determine memory, synthesis method, etc. These pieces of information may be determined by the program of the control unit 103, but may be directly received from the host.

  Next, in step 302, the control unit 103 delivers necessary information such as the addresses of the video frame memory 152 and the video frame timing memory 151 among the contents determined in step 301 to the video decoding unit 104, and starts operation. Instruct.

  Next, in step 303, the control unit 103 delivers necessary information such as the addresses of the subtitle frame memory 154 and the subtitle frame timing memory 153 among the contents determined in step 301 to the subtitle decoding unit 105, and starts operation. Instruct.

  Next, the control unit 103 waits until output from the video decoding unit 104 and the caption decoding unit 105 becomes possible in step 304.

  When the video decoding unit 104 and the subtitle decoding unit 105 can output (step 304 “YES”), the control unit 103 adds the display synthesis instruction memory 156 to the display synthesis instruction memory 156 for the image synthesis unit 101 in subsequent step 305. The image composition unit 101 is instructed about the screen composition, that is, the video frame memory 152 and the caption frame memory 154 that are target frame memories used for the screen composition, the composition method, and the like.

  Through the above processing flow, the control unit 103 gives a screen composition instruction to the image composition unit 101, and the image composition unit 101 synthesizes a video frame and a subtitle frame, for example, as shown in FIG. Can be set and displayed on the display unit 107. At this time, there is no need to consider timing constraints such as display output timing, and any storage constraints may be used as long as they are stored in advance in the display composition instruction memory 156.

  Thus, according to the present embodiment, the video frame and subtitle frame decoded by the video decoding unit 104 or the subtitle decoding unit 105 and their timing information are stored in the shared memory 102, and the image synthesis unit 101 synthesizes video frames and subtitle frames stored in the shared memory 102 in advance as instructed from the control unit 103 such as a CPU for each display timing based on each timing information with reference to the timer counter 106. Thus, the control unit 103 such as a CPU can set the display composition method such as the display position and composition method of each video frame and subtitle frame without interlocking with the display timing. At the same time, the control unit 103 converts the received encoded data into a video frame and a subtitle frame. Cloning and performs decoding, video data separated is able to perform independently of the control unit 103 by the video decoding unit 104 such as a DSP.

  For this reason, the real-time property of the control unit 103 such as a CPU for controlling the image composition of the video frame and the subtitle frame can be greatly reduced, and the independence between the video decoding unit 104 and the subtitle decoding unit 105 is increased. Therefore, the processing load on the control unit 103 such as a CPU can be reduced, and a simple system configuration can be realized. As a result, when the image synthesizing apparatus according to the present embodiment is employed in a digital TV receiving apparatus, a personal computer, or the like, it is possible to increase the effect of reducing power.

  In the above embodiment, the image synthesizing apparatus has been described as an example. However, the present invention is not limited to this, and the digital TV broadcast receiving apparatus having the image synthesizing apparatus of the above embodiment, or the above embodiment. Digital TV broadcast receiving method for processing the image synthesizing apparatus, system LSI for executing the digital TV broadcast receiving method, program for causing a computer to execute the digital TV broadcast receiving method, and recording for recording the program The same applies to the medium and the like.

  As described above, the image synthesizing apparatus, the digital TV broadcast receiving apparatus, the digital TV broadcast receiving method, the system LSI, the program, and the recording medium according to the present invention have real-time characteristics required for the CPU for image synthesis control. Since it can be greatly reduced and the independence from each data decoding unit can be increased, the processing load on the CPU can be reduced and a simple system configuration can be realized, thereby reducing power consumption as a digital TV receiver. This is effective for image synthesizing apparatuses, digital TV broadcast receiving apparatuses, digital TV broadcast receiving methods, system LSIs, programs, recording media, and the like.

The block diagram which shows the specific structural example in embodiment of this invention Diagram for explaining image composition method Flowchart for explaining control of image composition apparatus

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Image composition part 102 Shared memory 103 Control part 104 Video decoding part 105 Subtitle decoding part 106 Timer counter 107 Display part

Claims (6)

A video decoding unit that decodes a video signal encoded by MPEG or the like in digital TV broadcasting and outputs video data and video timing information for synchronization thereof;
A subtitle decoder that decodes subtitle information transmitted as data broadcast in the digital TV broadcast and outputs subtitle data and subtitle timing information for synchronization thereof;
An image composition unit for synthesizing and displaying the video data and the subtitle data in synchronization with a counter from a timer counter based on the video timing information and the subtitle timing information, respectively;
An image synthesizing apparatus.   A digital TV broadcast receiving apparatus comprising the image composition apparatus according to claim 1. Decoding video data encoded in MPEG or the like in digital TV broadcasting and outputting video data and video timing information for synchronization thereof;
Decoding subtitle information sent as data broadcast in the digital TV broadcast and outputting subtitle data and subtitle timing information for synchronization thereof;
Synthesizing and displaying the video data and the subtitle data in synchronization with a counter from a timer counter based on the video timing information and the subtitle timing information, respectively.
A receiving method for digital TV broadcasting.   A system LSI for executing the digital TV broadcast receiving method according to claim 3.   A program for causing a computer to execute the digital TV broadcast receiving method according to claim 3.   A recording medium on which the program according to claim 5 is recorded.

Images