JP7598685B2 - Broadcast receiving terminal equipment - Google Patents

Description

The present invention relates to a broadcast receiving terminal device, and in particular to a broadcast receiving terminal device adapted to a digital broadcast service system.

In television broadcasting services, broadcasters use various techniques to enhance the programs they provide in order to increase the interest of viewers. For example, in sports programs, a method is adopted in which a commentator provides commentary while synthesizing images handwritten by a commentator using a touch panel computer with real-time images captured by on-site television cameras. More specifically, in golf programs, a commentator will handwrite and superimpose images of the slope of the green, the direction of the grass grain, and the target putting line on the camera image, while providing commentary on the results.

As a technology for synthesizing such camera images and hand-drawn images, for example, the following Patent Document 1 is known. That is, Patent Document 1 discloses an image synthesis device that synthesizes a manually input image and an image captured by a camera in conjunction with the camera parameters of the camera.

On the other hand, in current television broadcasting services, programs are sometimes provided as multiplexed audio broadcasts (e.g., dual audio broadcasts) in which multiple audio streams are multiplexed into one program. For example, in multiplexed audio broadcasts, the audio of a commentator in a program such as the one described above is generally broadcast using a main audio channel, and other commentators or guests provide commentary in dialogue with each other on a switchable secondary audio channel.

JP 2007-323418 A

In a program that uses the above-mentioned technology of combining camera footage and handwritten images input with a pen, even if the viewer selects the secondary audio, the image input with a pen by the commentator on the main audio channel is displayed on the television screen, as shown in FIG. 14, for example. Therefore, for a viewer who has selected the secondary audio channel under multiplex audio broadcasting, the image input with a pen and inserted into the camera footage is meaningless and has no relation to the content of the secondary audio, and can sometimes be perceived as a nuisance.

In view of the above, the present invention aims to provide a broadcasting service that can hide inserted images that are not related to the selected audio channel in the display content of a program provided to a viewer.

More specifically, one of the objects of the present invention is to provide a broadcast receiving terminal device and a display control method that, in a program in which a camera image and another inserted image are superimposed and synthesized and provided to a viewer, allows the inserted image to be displayed or hidden in conjunction with the audio channel (audio mode) selected by the viewer.

Another object of the present invention is to provide a broadcast receiving terminal device and a display control method that allows a viewer to selectively display or hide an insert image, even if the insert image is related to an audio channel, in a program in which camera footage and other insert images are superimposed and synthesized to be provided to the viewer.

The present invention, which aims to solve the above problems, comprises the following invention-specific matters and technical features:

The present invention according to one aspect is a broadcast receiving terminal device adapted for a digital broadcast service using a broadcast signal based on a program stream in which multiple streams based on program material data constituting a program are multiplexed. The broadcast receiving terminal device includes a front-end processing unit that generates the program material data including at least video data, first audio data, second audio data, and character and graphic data based on the broadcast signal of the received program, a display control unit that controls so that a program image is displayed on the screen of a display device based on display data, and an audio control unit that controls so that audio based on either the first audio data or the second audio data is output from a speaker. The display control unit generates the display data by superimposing the character and graphic data on the video data in conjunction with the output of either the first audio data or the second audio data selected according to a viewer's operation.

The front-end processing unit may include a demodulation unit that demodulates the broadcast signal into the program stream, a separation unit that separates the demodulated program stream into at least a video stream, a first audio stream, a second audio stream, and a text and graphics stream, and a decoding unit that decodes the video stream, the first audio stream, the second audio stream, and the text and graphics stream.

The display control unit may control the display so that the character and graphic data is superimposed on the video data when the first audio data is selected, and may control the display so that the character and graphic data is not superimposed on the video data when the second audio data is selected.

The display control unit may also perform control so that, when a further operation is performed by the viewer after the first audio data is selected, the text and graphic data is not superimposed on the video data.

The present invention according to another aspect is a broadcast receiving terminal device adapted for a digital broadcast service. The broadcast receiving terminal device includes a front-end processing unit that generates at least video data, first audio data, second audio data, and character and graphic data based on a broadcast signal of a program to be received, a display control unit that controls so that a program image of either the first form or the second form is displayed on the screen of a display device, and an audio control unit that controls so that audio based on either the first audio data or the second audio data is output from a speaker. Here, the program image of the first form is an image that includes at least a moving image based on the video data and a character and graphic image based on the character and graphic data. Also, the program image of the second form is an image that includes the moving image but does not include the character and graphic image, and the display control unit controls so that the program image of the first form is displayed on the screen when the audio based on the first audio data is controlled to be output from the speaker. Furthermore, when the sound based on the second audio data is controlled to be output from the speaker, the display control unit controls so that the program image in the second form is displayed on the screen.

The display control unit may also include a switching control unit that controls whether or not to output the character and graphic data, and a synthesis unit that synthesizes the video data and the character and graphic data. The switching control unit may control the character and graphic data to be output to the synthesis unit when the sound based on the first audio data is controlled to be output from the speaker. The switching control unit may also control the character and graphic data not to be output to the synthesis unit when the sound based on the second audio data is controlled to be output from the speaker.

The character/graphic data may include at least first character/graphic data and second character/graphic data. The switching control unit may control the first character/graphic data to be output to the synthesis unit when the program control information generated based on the broadcast signal includes information specifying the display of a character/graphic image in a first audio mode. Also, the switching control unit controls the second character/graphic data to be output to the synthesis unit when the program control information includes information specifying the display of a character/graphic image in a second audio mode.

The present invention according to another aspect is a broadcast receiving terminal device adapted for digital broadcasting services. The broadcast receiving terminal device includes a front-end processing unit that generates at least video data, audio data, and character and graphic data based on a broadcast signal of a program to be received, a display control unit that controls so that a program image of either the first form or the second form is displayed on the screen of a display device, and an audio control unit that controls so that audio based on the audio data is output from a speaker. Here, the program image of the first form is an image that includes at least a moving image based on the video data and a character and graphic image based on the character and graphic data, and the program image of the second form is an image that includes the moving image but does not include the character and graphic image. And, when the program is configured in either a monaural broadcasting system or a stereo broadcasting system, the display control unit controls so that the program image of either the first form or the second form is selectively displayed on the screen in response to the operation of an audio switching button of a remote controller associated with the broadcast receiving terminal device.

The display control unit may also include a switching control unit that controls whether or not to output the character and graphic data, and a synthesis unit that synthesizes the video data and the character and graphic data. The switching control unit may control to switch whether or not to output the character and graphic data to the synthesis unit each time the audio switching button is operated.

The present invention according to another aspect is a broadcast receiving terminal device adapted for a digital broadcasting service. The broadcast receiving terminal device includes a front-end processing unit that generates at least video data, first audio data, second audio data, character and graphic data, and program control information based on a broadcast signal of a program to be received, a display control unit that controls so that a program image of either the first form or the second form is displayed on the screen of a display device, and an audio processing unit that controls so that audio based on either the first audio data or the second audio data is output from a speaker according to an audio mode. Here, the program image of the first form is an image that includes at least a moving image based on the video data and a character and graphic image based on the character and graphic data, and the program image of the second form is an image that includes the moving image but does not include the character and graphic image. And, when the program control information includes information specifying the display of the character and graphic image in the first audio mode, the display control unit controls so that the program image of the first form is displayed on the screen, and the audio processing unit controls so that the audio based on the first audio data is output from the speaker. On the other hand, when the program control information includes information specifying the display of a text and graphic image in the second audio mode, the display control unit controls the program image in the first form to be displayed on the screen, and the audio processing unit controls the audio based on the second audio data to be output from the speaker.

The display control unit may also include a switching control unit that controls whether or not to output the character and graphic data, and a synthesis unit that synthesizes the video data and the character and graphic data. When the program control information includes information that specifies the display of a character and graphic image in the first audio mode, the switching control unit may control the character and graphic data to be output to the synthesis unit. On the other hand, when the program control information includes information that specifies the display of a character and graphic image in the second audio mode, the switching control unit may control the character and graphic data not to be output to the synthesis unit.

The present invention can be considered as a method invention, and can also be considered as a computer program for causing a computer to execute the method, and as a recording medium on which the program is recorded.

In this disclosure, "means" does not simply mean physical means, but also includes cases where the functions of the means are realized by software. Also, the functions of one means may be realized by two or more physical means, and the functions of two or more means may be realized by one physical means.

In addition, in this disclosure, a "system" includes a logical collection of multiple devices (or functional modules that realize a specific function), regardless of whether each device or functional module is physically configured as a single entity or as separate entities.

According to the present invention, it becomes possible to hide insert images that are not related to the selected audio channel in the display content of a program provided to a viewer. In particular, in a program in which a camera image and another insert image are superimposed and synthesized to be provided to a viewer, it becomes possible to display or hide the insert image in conjunction with the audio channel (audio mode) selected by the viewer.

In addition, according to the present invention, in a program in which camera footage and other inserted images are superimposed and synthesized to provide the viewer with the image, the viewer can selectively display or hide the inserted image, even if the inserted image is related to an audio channel.

Other technical features, objects, and effects or advantages of the present invention will become apparent from the following embodiments described with reference to the accompanying drawings. The effects described in this disclosure are illustrative and not limiting, and other effects may be possible.

FIG. 1 is a diagram for explaining an example of an overall scheme of a digital broadcasting service system according to a first embodiment of the present invention. FIG. 2 is a block diagram showing an example of a schematic configuration of a broadcasting company system in a digital broadcasting service system according to an embodiment of the present invention. FIG. 3 is a block diagram showing an example of a schematic configuration of a broadcast receiving terminal device in a digital broadcast service system according to a first embodiment of the present invention. FIG. 4 is a flowchart showing an example of processing in the broadcast receiving terminal device of the digital broadcast service system according to the first embodiment of the present invention. FIG. 5 is a diagram for explaining the transition of setting states caused by the audio switch button in the broadcast receiving terminal device of the digital broadcast service system according to the first embodiment of the present invention. FIG. 6 is a flowchart showing an example of processing in a broadcast receiving terminal device of a digital broadcast service system according to the second embodiment of the present invention. FIG. 7 is a flowchart showing an example of processing in a broadcast receiving terminal device of a digital broadcast service system according to the third embodiment of the present invention. FIG. 8 is a flowchart showing an example of processing in a broadcast receiving terminal device of a digital broadcast service system according to the third embodiment of the present invention. FIG. 9 is a diagram for explaining the transition of setting states caused by the audio switch button in a broadcast receiving terminal device of a digital broadcast service system according to a third embodiment of the present invention. FIG. 10 is a flowchart showing an example of processing in a broadcast receiving terminal device of a digital broadcast service system according to the fourth embodiment of the present invention. FIG. 11 is a block diagram showing an example of a schematic configuration of a broadcasting company system in a digital broadcasting service system according to the fifth embodiment of the present invention. FIG. 12 is a block diagram showing an example of a schematic configuration of a broadcast receiving terminal device in a digital broadcast service system according to a fifth embodiment of the present invention. FIG. 13 is a flowchart showing an example of processing in a broadcast receiving terminal device in a digital broadcast service system according to a modified example of the fifth embodiment of the present invention. FIG. 14 is a diagram for explaining an example of a main audio and a sub-audio in a dual audio broadcast for a program image displayed on the screen of a broadcast receiving terminal device.

Below, an embodiment of the present invention will be described with reference to the drawings. However, the embodiment described below is merely an example, and there is no intention to exclude the application of various modifications and techniques not specified below. The present invention can be implemented with various modifications (for example, combining the various embodiments) without departing from the spirit of the invention. In addition, in the description of the drawings below, the same or similar parts are represented by the same or similar reference numerals. The drawings are schematic and do not necessarily correspond to actual dimensions, ratios, etc. The drawings may also include parts with different dimensional relationships and ratios.

[First embodiment]
This embodiment is characterized in that, in a digital broadcasting service using a broadcasting signal based on a transmission stream in which a plurality of digital data streams constituting program content are multiplexed, when the audio mode of a broadcast receiving terminal device is a first audio mode (e.g., main audio mode), a character and graphic image is controlled to be displayed superimposed on a main moving image (e.g., a video image from a television camera) constituting an image (program image) of the program content on a screen, and when the audio mode is a second audio mode (e.g., sub-audio mode), the character and graphic image is controlled not to be displayed superimposed on the moving image. Note that in this disclosure, a program image in which a character and graphic image is superimposed on a moving image may be referred to as a program image of a first form, and a program image in which a character and graphic image is not superimposed on a moving image may be referred to as a program image of a second form.

Figure 1 is a diagram for explaining an example of the overall scheme of a digital broadcasting service system according to one embodiment of the present invention. As shown in the figure, the digital broadcasting service system 1 of this embodiment is configured to include a broadcasting company system 10 and a broadcasting receiving terminal device 20. Note that, in the following, the digital broadcasting service system 1 is explained using a terrestrial digital broadcasting service as an example, but is not limited to this, and may also be a satellite broadcasting service such as a so-called BS broadcasting service using a broadcasting satellite S or a CS broadcasting service using a communication satellite. Alternatively, the digital broadcasting service system 1 may be a so-called cable television broadcasting service via a distribution company C, or an IPTV service using an IP network.

The broadcasting provider system 10 may be configured to include broadcasting facilities for television and radio broadcasting. The broadcasting provider system 10 is typically, but not limited to, managed and operated by a broadcasting provider (e.g., a broadcasting station) that has been granted a broadcasting license. Although not shown, the broadcasting provider system 10 broadcasts (e.g., terrestrial digital broadcasting) by transmitting program content stored in a transmission server over airwaves under the control of an automatic operation device, for example.

Program content is packaged as a "program" that is organized along a time axis and viewed by viewers, and may be used synonymously with "program" depending on the context. Program content may include main program content and secondary data broadcasting program content. Program content may be composed of a collection of various program material data (mono-media data) such as moving images (video), subtitles (and superimposed text), text and graphic images, and/or audio. Program material data is converted and transmitted into a multiplexed digital data stream (hereinafter referred to as a "stream") conforming to the MPEG2 format, which is a high-compression encoding technology, but is not limited to this. An MPEG-2 transport stream (TS) is one form of stream consisting of a series of TS packets. From the perspective of transmission technology, a multiplexed stream may be called a transmission stream. In this disclosure, a stream or data related to program content may be called a "program stream" or "program data". The broadcasting company system 10 generates a program stream by multiplexing each of the multiple streams based on the program material data with the specified service information according to the program schedule along the time axis, and further modulates this into a broadcast signal in a specified frequency band and transmits it as a broadcast wave from a transmitting base station 12 such as a radio tower to the target broadcast area.

The audio broadcasting method of a broadcast program can be classified into monaural broadcasting (including so-called mono-stereo broadcasting) and audio multiplex broadcasting according to the usage of the audio channel, and audio multiplex broadcasting can be further classified into stereo broadcasting and dual audio broadcasting. Dual audio broadcasting is a multiplex broadcast consisting of a main audio and a secondary audio. This embodiment will be described on the premise that the audio broadcasting method of the program is dual audio broadcasting.

The broadcast receiving terminal device 20 is a terminal device that allows a viewer (user) to watch programs provided by a broadcasting company. The broadcast receiving terminal device 20 can typically be a so-called television receiver or a set-top box, but is not limited to this. In other words, the broadcast receiving terminal device 20 may be any device that has the function of allowing a viewer to watch a desired program, and may be, for example, a computing device such as a desktop computer, a notebook computer, a tablet computer, a smartphone, a feature phone, or other intelligent device.

Figure 2 is a block diagram showing an example of a schematic configuration of a broadcasting operator system in a digital broadcasting service system according to the first embodiment of the present invention. As shown in the figure, the broadcasting operator system 10 is composed of components such as a program operation device 110, an encoding unit 120, a text and graphic insertion unit 130, a program information generation unit 140, a multiplexing unit 150, a scrambling unit 160, a modulation unit 170, and a transmission unit 180.

The program operation device 110 outputs program material data constituting the program content stored in a transmission server (not shown) to the encoding unit 120 in accordance with the program schedule. The program operation device 110 also outputs various control information related to the program (hereinafter referred to as "program control information") to the program information generation unit 140. The program control information includes, for example, information on what kind of streams the program content is made up of, what kind of address information is assigned, etc.

The encoding unit 120 generates a stream by encoding (compressing and encoding) various program material data. The encoding unit 120 includes, for example, a video encoding unit 121, a still image encoding unit 122, an audio encoding unit 123, a subtitle encoding unit 124, and a text and graphics encoding unit 125.

The video encoding unit 121 encodes the video (moving image/picture) data related to the program content using, for example, a codec that complies with the MPEG-2 video standard, thereby generating a video stream in PES format. The video data related to the program content is displayed on the screen as the main video (moving images), and includes, for example, video data shot by a television camera and video data to which dramatic elements have been added by the producer (broadcast station). The video encoding unit 121 outputs the generated video stream to the multiplexing unit 150.

Conventionally, for example, a video stream obtained by superimposing text and graphic images handwritten by a commentator using a tablet computer onto video captured by a television camera at a broadcasting company constitutes the display content (main moving image) of a program. In this disclosure, as described below, such text and graphic images are treated as a separate stream (text and graphic stream) from the video stream based on the video captured by the television camera, as necessary.

The still image encoding unit 122, for example, encodes still image data related to data broadcasting program content using a specified codec to generate a still image stream in PES format. The still image encoding unit 122 converts and encodes still image data in JPEG format, for example, into video data in units of several seconds (for example, 30 seconds), thereby generating a pseudo still image stream. The still image encoding unit 122 outputs the generated still image stream to the multiplexing unit 150.

The audio encoding unit 123 encodes audio data related to the program content using a codec that complies with the MPEG-2 audio standard, for example, thereby generating an audio stream in PES format. The audio stream may include a first audio stream and a second audio stream. A primary audio stream corresponding to the primary audio mode (primary audio channel) is one form of the first audio stream, and a secondary audio stream corresponding to the secondary audio mode (secondary audio channel) is one form of the second audio stream. The audio encoding unit 123 outputs the generated audio stream to the multiplexing unit 150.

The subtitle encoding unit 124 encodes the subtitle and/or superimposed text data related to the program content according to, for example, a broadcast subtitle file exchange format, thereby generating a stream in PES format. The subtitle encoding unit 124 outputs the generated subtitle stream to the multiplexing unit 150.

The character and graphic encoding unit 125 accepts character and graphic data input from the character and graphic insertion unit 130 (described later) and encodes it, thereby generating a character and graphic stream in PES format. The character and graphic encoding unit 125 generates a character and graphic stream by converting, for example, JPEG format character and graphic data into video data in units of several seconds (e.g., 30 seconds). The character and graphic encoding unit 125 outputs the generated character and graphic stream to the multiplexing unit 150.

The character and graphic inserting unit 130 is an interface for inserting character and graphic data related to the program content into the program content. The character and graphic data is image data that represents characters and/or graphics, and should not be interpreted in a limited manner. The character and graphic inserting unit 130 is configured to include, for example, a touch panel computer operated by the staff producing the program or the performers of the program. As an example, in the broadcasting of a sports program, the character and graphic inserting unit 130 accepts information input by a commentator using a pen while commenting in the program being broadcast using a touch panel computer. In other words, in this disclosure, the character and graphic image that was previously composed by the broadcasting company side into the original video and formed a part of it is treated independently from the video by the viewer side as necessary. The character and graphic inserting unit 130 outputs the accepted character and graphic stream to the character and graphic encoding unit 125. Note that in this example, the character and graphic inserting unit 130 is configured as a component separate from the character and graphic encoding unit 125, but for example, the character and graphic inserting unit 130 may be configured to have the function of the character and graphic encoding unit 125.

The program information generator 140 generates a section-format stream based on the program control information. The program control information includes, for example, information about program organization. Program sequence information in MPEG-2 is one form of program control information. Program sequence information includes program specific information (PSI) and complementary service information (SI).

The multiplexing unit 150 generates a multiplexed program stream (transmission stream) based on the various streams described above. A multiplexer is one form of the multiplexing unit 150. As described above, an MPEG-2 transport stream (TS) is one form of a transmission stream made up of a series of TS packets. The multiplexing unit 150 outputs the generated program stream to the scrambler unit 160.

The scrambling unit 160 scrambles the generated program stream using an encryption key. Scrambling is mainly performed to realize limited reception for the purpose of protecting the rights of the program content. This type of limited reception method is called CAS (Conditional Access System). The scrambling unit 160 outputs the scrambled program stream to the modulation unit 170.

The modulation unit 170 digitally modulates the scrambled program stream to generate a broadcast signal in a specific frequency band assigned to each broadcaster. The modulation unit 170 outputs the modulated broadcast signal to the transmission unit 180.

The transmitting unit 180 emits the modulated broadcast signal as a broadcast wave, for example, from a radio tower toward a specified broadcast target area in accordance with a specified broadcast output. Alternatively, the broadcast wave is emitted toward a broadcast satellite S or the like, and transmitted from the broadcast satellite S to the ground. Alternatively, the transmitting unit 180 may transmit the broadcast signal to a distribution company C that operates, for example, a cable television broadcasting service.

Although not shown, the broadcaster system 10 may include a component for providing data broadcast program content to viewers. Data related to the data broadcast program content may also be multiplexed into the program stream. The data broadcast program content may be linked content that is linked to the main program content, or it may be independent content that is independent of the main program content.

Figure 3 is a block diagram showing an example of the schematic configuration of a broadcast receiving terminal device in a digital broadcast service system according to the first embodiment of the present invention. As shown in the figure, the broadcast receiving terminal device 20 includes, for example, a tuner/modulation unit 210, a descrambling unit 220, a separation unit 230, a decoding unit 240, an output control unit 250, a data broadcasting engine 260, a control unit 270, and an operation reception unit 280. Also, as shown in the figure, the broadcast receiving terminal device 20 is connected to a display device DSP and a speaker SPK as output devices.

The tuner/modulator 210 tunes (selects) the broadcast waves received by an antenna (not shown) to extract the desired broadcast signal, and then demodulates it to obtain a program stream. The program stream at this stage is scrambled. The tuner/modulator 210 also performs error correction processing when demodulating the program stream. In this example, the tuner/modulator 210 is configured as a single component, but is not limited to this and may be configured as separate components. The tuner/modulator 210 outputs the demodulated program stream to the descramble unit 220.

The descrambling unit 220 descrambles (decrypts) the demodulated program stream using a decryption key, restoring it to the original multiplexed program stream. The decryption key is read, for example, from an IC card (not shown) called a CAS card that is attached to the broadcast receiving terminal device. The descrambling unit 220 outputs the descrambled program stream to the separation unit 230.

The separation unit 230 separates the descrambled multiplexed program stream into each stream. A demultiplexer is one form of the separation unit 230. The separation unit 230 outputs each stream separated from the program stream to the decoding unit 240, the data broadcasting engine 260, and/or the control unit 270 according to the stream. For example, the separation unit 230 outputs the video stream, audio stream, subtitle stream, and text and graphics stream separated from the program stream to the decoding unit 240, while outputting the stream for data broadcasting to the data broadcasting engine 260. The separation unit 230 also outputs the separated program control information to the control unit 270.

The decoding unit 240 decodes each stream to generate program material data that constitutes the program content. The decoding unit 240 includes, for example, a video decoding unit 241, an audio decoding unit 242, a still image decoding unit 243, a subtitle decoding unit 244, and a character and graphic decoding unit 245.

The video decoding unit 241 decodes the video stream using a codec conforming to the MPEG-2 video standard, for example, thereby generating video data. The video decoding unit 241 outputs the decoded video data to the output control unit 250. As described below, the video data input to the output control unit 250 is selectively and superimposed with other data and is converted into a video signal by a display driver (not shown) and output to the display device DSP. The display device displays the video of the program based on the video signal. The display device DSP is, for example, a liquid crystal display (LCD) or an organic electroluminescent display (OELD), but is not limited to these.

The still image decoding unit 243 decodes the still image stream, thereby generating still image data. The still image decoding unit 243 outputs the decoded still image data to the output control unit 250. The decoded still image data is displayed on the screen of the display device, for example, as a data broadcast image.

The audio decoding unit 242 decodes the audio stream, for example, in accordance with the MPEG-2 audio standard, thereby generating audio data. As described above, in this disclosure, the audio stream may include a first audio stream and a second audio stream. Therefore, the audio decoding unit 242 may generate first audio data corresponding to the main audio mode and second audio data corresponding to the secondary audio mode based on the first audio stream and the second audio stream, respectively. At least one of the decoded first audio data and second audio data is output to an audio driver (not shown) via an audio selection unit 252 (described later), and is further converted by the audio driver into an audio signal and output from the speaker SPK as audio (voice) in the audible range.

The subtitle decoding unit 244 decodes the subtitle stream according to, for example, a broadcast subtitle file exchange format, thereby generating subtitle data. The subtitle decoding unit 244 outputs the generated subtitle data to the output control unit 250. The subtitle data output to the output control unit 250 is selectively superimposed on the video data to form part of the display content of the program.

The character and graphic decode unit 245 decodes the character and graphic stream, thereby generating character and graphic data. The character and graphic decode unit 245 outputs the generated character and graphic data to the output control unit 250. The character and graphic data output to the output control unit 250 is selectively synthesized with video data by superimposing it. In this embodiment, the character and graphic data is synthesized with video data by superimposing it only in the main audio mode.

In this disclosure, the tuner/modulator 210, the descramble 220, the separator 230, and the decoder 240 constitute a front-end processor. That is, the front-end processor is understood as a component that generates various program material data (mono-media data) such as video data and audio data, and program control information, etc., based on the received broadcast signal.

The output control unit 250 controls the output to the display device DSP and/or the speaker SPK under the control of the control unit 270. In this disclosure, the output control unit 250 can be seen as a component including a display control unit that controls the output to the display device DSP and an audio control unit that controls the output to the speaker SPK. Under the control of the control unit 270, the output control unit 250 performs control so that either a program image of a first form in which a character and graphic image based on character and graphic data is superimposed on a moving image based on video data, or a program image of a second form in which a character and graphic image is not superimposed on a moving image, is displayed on the screen of the display device DSP.

That is, under the control of the control unit 270, the display control unit in the output control unit 250 selectively and superimposes other program material data related to display onto the video data to generate display data. Furthermore, the display control unit converts the display data into a video signal of an appropriate format using a display driver, and drives the display device DSP based on the video signal so that the program image is displayed on the display device DSP.

The audio control unit in the output control unit 250 selects and outputs at least one of the decoded first audio data and second audio data. Furthermore, the audio control unit converts the output audio data into an audio signal by an audio driver, thereby driving the speaker SPK.

In the figure, the components of the output control unit 250 are illustrated as a video/still image switching unit 251, an audio selection unit 252, switches 253a and 253b, and a synthesis unit 254. The video/still image switching unit 251, switches 253a and 253b, and synthesis unit 254 are examples of the display control unit, and the audio selection unit 252 is an example of the sound control unit.

The video/still image switching unit 251 selects and outputs either video data or still image data. The video/still image switching unit 251 selectively switches under the control of the data broadcasting engine 260, for example. Alternatively, the video/still image switching unit 251 may be configured to select both video data and still image data, and output video data in which they are appropriately laid out on a single screen.

Under the control of the control unit 270, the audio selection unit 252 selects and outputs the first audio data and/or the second audio data. In this embodiment, the audio selection unit 252 selects and outputs either the first audio data or the second audio data. Alternatively, in the case of program content configured as a stereo broadcast, the audio selection unit 252 selects and outputs both the first audio data and the second audio data. As will be described later, in this embodiment, the selection operation of the audio selection unit 252 is controlled so as to be linked to the on/off operation of the switch 253b.

Under the control of the control unit 270, the switch 253a switches between outputting or not outputting subtitle data to the synthesis unit 254. Also, under the control of the control unit 270, the switch 253b switches between outputting or not outputting character and graphic data to the synthesis unit 254. The switches 253a and 253b are one form of a switching control unit.

The synthesizer 254 synthesizes other program material data related to display with the video data (and/or still image data) in a superimposed manner, thereby generating display data. The data is synthesized, for example, by alpha blending using a transparency information coefficient (alpha value). In this example, the synthesizer 254 includes a synthesizer 254a for synthesizing subtitle data with the video data, and a synthesizer 254b for further synthesizing character and graphic data with the output of the synthesizer. When there is no output of subtitle data and/or character and graphic data, null data is synthesized with the video data, and the output is the video data itself. The display data output by the synthesizer 254 is converted into a video signal by a display driver (not shown) and output to a display device.

The data broadcasting engine 260 decodes the separated data broadcasting stream, thereby generating data broadcasting display data and a display control signal. The data broadcasting engine 260 analyzes the display control signal and controls the video/still image switching unit 251 based on the analysis results.

The operation reception unit 280 receives operation instructions from the viewer to the broadcast receiving terminal device 20. For example, the viewer can give instructions to the broadcast receiving terminal device 20 by operating a remote controller in hand. The viewer's operation instructions include, for example, an instruction to switch channels (broadcast stations), an instruction to control the volume, an instruction to display/hide data broadcast program content, an instruction to display/hide subtitles, and an instruction to switch audio modes. For example, the viewer can selectively switch the audio mode between the main audio mode and the secondary audio mode by pressing an audio switch button. The operation reception unit 280 outputs the received operation instructions to the control unit 270.

The control unit 270 provides overall control over the operation of the broadcast receiving terminal device 20. Although not shown, the control unit 270 is configured to include a processor module including a CPU and a memory module including various types of memory. Furthermore, under the control of the CPU, the control unit 270 can control the operation of applications implemented in the broadcast receiving terminal device 20. The applications may be pre-implemented in the broadcast receiving terminal device 20, or may be downloaded and implemented via broadcast waves or a communication network.

For example, the control unit 270 determines the audio broadcasting format of the selected program based on the program control information. In this embodiment, the control unit 270 determines whether the audio broadcasting format of the program is dual audio broadcasting based on the program control information. The control unit 270 also stores in memory the state of the current audio mode selected by the audio switch button.

When the control unit 270 receives an instruction to switch the audio mode via the operation reception unit 280, it controls the audio selection unit 252 and also controls the switch 253b. More specifically, in this embodiment, when the main audio mode is selected by the instruction to switch the audio mode, the control unit 270 controls the audio selection unit 252 to select the first audio data (the second audio data is not selected), and controls the switch 253b to turn on so that the character and graphic data is output to the synthesis unit 254. On the other hand, when the sub-audio mode is selected by the instruction to switch the audio mode, the control unit 270 controls the audio selection unit 252 to select the second audio data (the first audio data is not selected), and controls the switch 253b to turn off so that the character and graphic data is not output to the synthesis unit 254.

Figure 4 is a flowchart showing an example of processing in a broadcast receiving terminal device of a digital broadcast service system according to the first embodiment of the present invention. More specifically, the figure shows an example of the flow of processing until the broadcast receiving terminal device 1 provides a program to a viewer based on the received broadcast waves.

As shown in the figure, the broadcast receiving terminal device 20 tunes (selects a channel) the broadcast waves received by an antenna (not shown) in the tuner/modulator unit 210, thereby extracting the desired broadcast signal, and further demodulates it to obtain a scrambled program stream (S401). The scrambled program stream is output to the descramble unit 220.

Then, in the descrambling section 220, the broadcast receiving terminal device 20 descrambles the demodulated program stream using a decryption key, restoring it to the original multiplexed program stream (S402). Furthermore, in the broadcast receiving terminal device 20, the separation section 230 separates the descrambled program stream into each stream (S403). Each separated stream is output to the decoding section 240. As described above, the separated stream may include, for example, a video stream, a first audio stream, and a second audio stream, as well as at least one of a still image stream, a subtitle stream, and a text and graphics stream. Furthermore, the data broadcasting stream among the separated streams is output to the data broadcasting engine 260.

The broadcast receiving terminal device 20 then decodes each of the separated streams in the decoding section 240, thereby generating each program material data (S404). As described above, the program material data may include, for example, at least one of still image data, subtitle data, and text and graphic data, in addition to the video data, the first audio data, and the second audio data. Note that in this disclosure, the processes from S401 to S404 may be collectively referred to as "front-end processing."

The control unit 270 of the broadcast receiving terminal device 20 determines the audio broadcasting format of the selected program (S405). That is, the control unit 270 determines, based on the program control information, whether the audio broadcasting format of the selected program is a dual audio broadcast consisting of a main audio and a secondary audio, or is another audio broadcast (monaural broadcast/stereo broadcast).

When the broadcast receiving terminal device 20 determines that the audio broadcasting method is dual audio broadcasting, it further determines whether the current audio mode is the main audio mode or the secondary audio mode (S406). When the broadcast receiving terminal device 20 determines that the current audio mode is the main audio mode, it selects and outputs the first audio data and turns on the output of the character and graphic data (S407). On the other hand, when the broadcast receiving terminal device 20 determines that the current audio mode is the secondary audio mode, it selects and outputs the second audio data and turns off the output of the character and graphic data (S408).

Next, the broadcast receiving terminal device 20 superimposes the subtitle data and/or character and graphic data on the video data in the output control unit 250 to generate display data (S409). If there is no output of subtitle data and/or character and graphic data, null data is superimposed on the video data.

The broadcast receiving terminal device 20 controls the display device DSP to display a screen based on the output display data, and also controls the speaker SPK to output audible sound based on the output audio data (S410).

If the broadcast receiving terminal device 20 determines in S405 that the broadcast is an audio broadcast other than a dual audio broadcast (monaural/stereo broadcast), it superimposes subtitle data and/or text and graphic data onto the video data in accordance with the current settings, generates display data (S409), and controls the display and audio output (S410). Alternatively, as described in other embodiments, the broadcast receiving terminal device 20 may perform processing in accordance with monoural/stereo broadcast.

In the main audio mode, the broadcast receiving terminal device 20 of the present embodiment configured as described above selects the first audio data under the control of the control unit 270, outputs the character and graphic data to the synthesis unit 254, and synthesizes it in a superimposed manner on the video data. As a result, audio based on the first audio data is output from the speaker SPK, while the display device DSP displays a program image in a form (first form) in which a character and graphic image (e.g., an image input by handwriting pen) based on the character and graphic data is superimposed on a moving image (e.g., a video image from a television camera) based on the video data. On the other hand, when the viewer selects the sub-audio mode by operating, for example, the audio switching button of the remote controller, the broadcast receiving terminal device 20 selects the second audio data under the control of the control unit 270, and turns off the output of the character and graphic data. As a result, audio based on the second audio data is output from the speaker SPK, while the display device DSP displays a program image in a form (second form) in which a character and graphic image is not superimposed on a moving image. This allows viewers to watch programs without the video being interrupted by text and graphic images that are unrelated to the audio.

Second Embodiment
This embodiment is a variation of the first embodiment, and is characterized in that in a digital broadcasting service using broadcast signals based on a multiplexed transmission stream, when the audio mode of the broadcast receiving terminal device is a specific audio mode (e.g., main audio mode), the viewer is able to selectively switch between displaying or not displaying text and graphic images that should be superimposed on the main moving image of a program on the screen (e.g., a television camera image).

That is, the broadcast receiving terminal device 20 of this embodiment is configured to be able to switch between displaying and hiding character and graphic images as well as switching between different audio modes each time the viewer operates the audio switch button on the remote controller. More specifically, the broadcast receiving terminal device 20 performs control so that each time the audio switch button is pressed, as shown in FIG. 5, the setting state (hereinafter referred to as the "setting state") set by the audio switch button is cyclically switched between a state ST1 in which displaying character and graphic images is permitted in the main audio mode, a state ST2 in which displaying character and graphic images is not permitted in the main audio mode, and a state ST3 in which displaying character and graphic images is not permitted in the secondary audio mode.

For this reason, in this embodiment, the control unit 270 stores the previous audio mode and display/non-display state of the text and graphic image, i.e., the state of the audio selection unit 252 and the switch 253b, in, for example, a non-volatile memory (not shown). The function of storing and reproducing such a previous state is sometimes referred to as a last memory function.

For example, assume that the current setting state is state ST1. That is, the main sound is output from the speaker, and the display device DSP displays character and graphic data superimposed on the moving image. In state ST1, when the viewer presses the sound switch button once, the control unit 270 changes the setting state to state ST2. Therefore, in order to hide the character and graphic images while keeping the sound mode in the main sound mode, the control unit 270 controls the switch 253b to be turned off without controlling the audio selection unit 252. As a result, the main sound is output from the speaker, and the display device DSP displays the moving image without the character and graphic images superimposed.

Next, when the viewer presses the audio switch button once in state ST2, the control unit 270 changes the setting state to state ST3. Therefore, the control unit 270 controls the audio selection unit 252 to switch the audio mode from the main audio mode to the secondary audio mode. At this time, the switch 253b remains off. As a result, the secondary audio is output from the speaker, and a moving image without superimposed text and graphic images is displayed on the display device DSP.

Next, in state ST3, when the viewer presses the audio switch button once, the control unit 270 returns the setting state to state ST1. Therefore, in order to switch the audio mode from the sub-audio mode to the main audio mode, the control unit 270 controls the audio selection unit 252 and turns on the switch 253b. As a result, the main audio is output from the speaker, and a text and graphic image superimposed on the moving image is displayed on the display device DSP.

Figure 6 is a flowchart showing an example of processing in a broadcast receiving terminal device of a digital broadcast service system according to a second embodiment of the present invention.

As shown in the figure, the broadcast receiving terminal device 20 that receives the broadcast wave performs front-end processing (S601). That is, as described above, the broadcast receiving terminal device 20 tunes (selects a channel) the received broadcast to extract the desired broadcast signal, demodulates it, and further descrambles it using a decryption key, thereby restoring the multiplexed program stream. Next, the broadcast receiving terminal device 20 separates the program stream into each stream and decodes each separated stream, thereby generating each program material data. In addition, the program control information is output to the control unit 270.

At this time, the control unit 270 of the broadcast receiving terminal device 20 determines the audio broadcasting format of the selected program (S602). That is, the control unit 270 determines, based on the program control information, whether the audio broadcasting format of the selected program is a dual audio broadcast consisting of a main audio and a secondary audio, or is another audio broadcast (monaural broadcast/stereo broadcast).

If the broadcast receiving terminal device 20 determines that the audio broadcasting method is dual audio broadcasting, it then determines whether the current audio mode is the main audio mode or the secondary audio mode (S603). If the broadcast receiving terminal device 20 determines that the current audio mode is the main audio mode, it then determines the current display state of the text and graphic image (S604).

When the broadcast receiving terminal device 20 determines that the current display state of the character and graphic image is displayed, it selects and outputs the first audio data and turns on the output of the character and graphic data (S605). On the other hand, when the broadcast receiving terminal device 20 determines that the current display state of the character and graphic image is not displayed, it selects and outputs the first audio data and turns off the output of the character and graphic data (S606).

On the other hand, if the broadcast receiving terminal device 20 determines in S603 that the current audio mode is the sub-audio mode, it selects and outputs the second audio data and turns off the output of the character and graphic data (S607).

Next, the broadcast receiving terminal device 20 superimposes the subtitle data and/or character and graphic data on the video data in the output control unit 250 to generate display data (S608). If there is no output of subtitle data and/or character and graphic data, null data is superimposed on the video data.

The broadcast receiving terminal device 20 controls the display device so that a screen based on the output display data is displayed, and also controls the speaker so that audible sound is output based on the output audio data (S609).

If the broadcast receiving terminal device 20 determines in S602 that the broadcast is an audio broadcast other than a dual audio broadcast (monaural/stereo broadcast), it superimposes subtitle data and/or text and graphic data onto the video data in accordance with the current settings, generates display data (S608), and controls the display and audio output (S609). Alternatively, as described in other embodiments, the broadcast receiving terminal device 20 may perform processing in accordance with monoural/stereo broadcast.

The broadcast receiving terminal device 20 of this embodiment configured as described above operates to switch between displaying and hiding the text and graphic images linked to the audio mode each time the audio switch button is operated. Therefore, the viewer can display or hide the text and graphic images even if they are associated with the audio mode.

[Third embodiment]
This embodiment is a variation of the above embodiment, and is characterized in that in a digital broadcasting service using a broadcast signal based on a multiplexed transmission stream, text and graphic images are specified in advance to be superimposed on the main moving image of the program on the screen in either a first audio mode (e.g., main audio mode) or a second audio mode (e.g., secondary audio mode), and the broadcast receiving terminal device accordingly displays the text and graphic images superimposed on the main moving image of the program on the screen.

To realize the digital broadcasting service of this embodiment, the broadcaster system 10 transmits program control information including a descriptor indicating whether text and graphic images are to be displayed in main audio mode or sub-audio mode in a program stream. As an example, a descriptor for specifying whether text and graphic images are to be displayed in main audio mode or sub-audio mode is defined in the EIT (Event Information Table) of the program arrangement information. The program information generation unit 140 of the broadcaster system 10 generates a stream based on the program control information including such a descriptor in the EIT, and outputs it to the multiplexing unit 150. The multiplexing unit 150 generates and outputs a program stream by multiplexing each stream.

The separation unit 230 of the broadcast receiving terminal device 20 outputs the program control information obtained by separating such a multiplexed program stream to the control unit 270. The control unit 270 refers to the EIT in the program control information and controls the audio selection unit 252 and the switch 253b according to the descriptors in the EIT.

Figures 7 and 8 are flowcharts showing an example of processing in a broadcast receiving terminal device of a digital broadcast service system according to the third embodiment of the present invention. Note that Figure 8 shows details of processing S703 shown in Figure 7.

As shown in FIG. 7, the broadcast receiving terminal device 20 that receives the broadcast wave performs front-end processing (S701). That is, as described above, the broadcast receiving terminal device 20 tunes (selects) the received broadcast to extract the desired broadcast signal, demodulates it, and further descrambles it using a decryption key, thereby restoring the multiplexed program stream. Next, the broadcast receiving terminal device 20 separates the program stream into each stream and decodes each separated stream, thereby generating each program material data. In addition, the program control information is output to the control unit 270.

At this time, the control unit 270 of the broadcast receiving terminal device 20 determines the audio broadcasting format of the selected program (S702). That is, based on the program control information, the control unit 270 determines whether the audio broadcasting format of the selected program is a dual audio broadcast consisting of a main audio and a secondary audio, or is another audio broadcast (monaural broadcast/stereo broadcast).

If the broadcast receiving terminal device 20 determines that the audio broadcasting method is dual audio broadcasting, the control unit 270 then performs output control processing of character and graphic data based on the audio mode and program control information (S703).

That is, in the output control process of the character and graphic data, the broadcast receiving terminal device 20 judges which is the current audio mode, as shown in FIG. 8 (S801). If the broadcast receiving terminal device 20 judges that the current audio mode is the main audio mode, it then judges whether or not the descriptor of the program control information specifies that there is display in the main audio mode (S802). If the broadcast receiving terminal device 20 judges that the descriptor of the program control information specifies that there is display in the main audio mode, it selects and outputs the first audio data and turns on the output of the character and graphic data (S803). On the other hand, if the broadcast receiving terminal device 20 judges that the descriptor of the program control information specifies that there is no display in the main audio mode (i.e., specifies that there is display in the secondary audio mode), it selects and outputs the first audio data and turns off the output of the character and graphic data (S804).

On the other hand, if the broadcast receiving terminal device 20 determines in S801 that the current audio mode is not the main audio mode (it is the secondary audio mode), it then determines whether the descriptor in the program control information specifies that there is display in the secondary audio mode (S805). If the broadcast receiving terminal device 20 determines that the descriptor in the program control information specifies that there is display in the secondary audio mode, it selects and outputs the second audio data and turns on the output of the character and graphic data (S806). On the other hand, if the broadcast receiving terminal device 20 determines that the descriptor in the program control information specifies that there is no display in the secondary audio mode (i.e., specifies that there is display in the main audio mode), it selects and outputs the second audio data and turns off the output of the character and graphic data (S807).

Returning to FIG. 7, the broadcast receiving terminal device 20 superimposes the subtitle data and/or character and graphic data onto the video data to generate display data (S704). If no subtitle data and/or character and graphic data is output, null data is superimposed onto the video data. The broadcast receiving terminal device 20 then controls the display device so that a screen based on the output display data is displayed, and also controls the speaker so that audible sound is output based on the output audio data (S705).

If the broadcast receiving terminal device 20 determines in S702 that the broadcast is an audio broadcast other than a dual audio broadcast (monaural broadcast/stereo broadcast), it superimposes subtitle data and/or text and graphic data onto the video data in accordance with the current settings, generates display data (S704), and controls the display and audio output (S705). Alternatively, as described in other embodiments, the broadcast receiving terminal device 20 may perform control in accordance with a dual audio broadcast.

As described above, according to this embodiment, the character and graphic image is displayed on the screen of the broadcast receiving terminal device 20 only in the audio mode specified by the broadcasting company. That is, when the character and graphic data is specified to be displayed in the main audio mode, the broadcast receiving terminal device 20 selects the first audio data and outputs the character and graphic data to the synthesis unit 254 only when the current audio mode is the main audio mode, and synthesizes it in a superimposed manner with the video data. As a result, audio based on the first audio data is output from the speaker SPK, and the display device DSP displays a screen in which, for example, a character and graphic image input by handwriting pen is superimposed on the image captured by the television camera. On the other hand, when the secondary audio mode is selected, the broadcast receiving terminal device 20 selects the second audio data and turns off the output of the character and graphic data. As a result, audio based on the second audio data is output from the speaker SPK, while the display device DSP displays the screen of the image captured by the television camera, and the character and graphic image is no longer displayed.

When the character and graphic data is designated to be displayed in the secondary audio mode, the broadcast receiving terminal device 20 selects the second audio data and outputs the character and graphic data to the synthesis unit 254 to synthesize it on the video data in a superimposed manner only when the current audio mode is the secondary audio mode. As a result, audio based on the second audio data is output from the speaker SPK, while the display device DSP displays a screen in which, for example, a character and graphic image input by handwriting pen is superimposed on the image captured by the television camera. On the other hand, when the broadcast receiving terminal device 20 is in the main audio mode, it selects the second audio data and turns off the output of the character and graphic data. As a result, audio based on the second audio data is output from the speaker SPK, while the display device DSP displays the screen of the image captured by the television camera, and the character and graphic image is no longer displayed.

(Modification)
As a variation of this embodiment, as shown in the second embodiment, the broadcast receiving terminal device 20 may be controlled so that the character and graphic images superimposed on the main moving image of the program on the screen can be selectively displayed or hidden in response to an operation by the viewer.

That is, the broadcast receiving terminal device 20 of this embodiment is configured to switch between displaying and hiding character and graphic images as well as switching between different audio modes each time the viewer operates the audio switch button on the remote controller. More specifically, the broadcast receiving terminal device 20 performs control so that each time the audio switch button is operated, the state is cyclically switched between a state ST31 in which displaying character and graphic images is permitted in the main audio mode, a state ST32 in which displaying character and graphic images is not permitted in the main audio mode, a state ST33 in which displaying character and graphic images is permitted in the secondary audio mode, and a state ST34 in which displaying character and graphic images is not permitted in the secondary audio mode, as shown in FIG. 9.

For this reason, the control unit 270 realizes the last memory function by storing the previous audio mode and display/non-display state of the text and graphic image, i.e., the state of the audio selection unit 252 and the switch 253b, for example in a non-volatile memory (not shown).

For example, suppose that the descriptor of the program control information for the selected program specifies that text and graphic images are to be displayed in the secondary audio mode. Also, suppose that the current setting state is state ST31. Therefore, the main audio is output from the speaker SPK, and a moving image without superimposed text and graphic images is displayed on the display device DSP. In state ST31, when the viewer presses the audio switch button once, the control unit 270 changes the setting state to state ST32. In other words, because the descriptor of the program control information does not permit the display of text and graphic images in the main audio mode, the control unit 270 transitions the setting state by one, but does not control the audio selection unit 252 or the switch 253b. Therefore, there is no change in the output of the speaker SPK or the display of text and graphic images on the display device DSP.

Next, when the viewer presses the audio switch button once in state ST32, the control unit 270 changes the setting state to state ST33. In other words, because the descriptor in the program control information permits the display of text and graphic images in the secondary audio mode, the control unit 270 controls the audio selection unit 252 to switch to secondary audio output and turns on switch 253b in order to switch the audio mode from the main audio mode to the secondary audio mode. As a result, the secondary audio is output from the speaker, and a moving image with a text and graphic image superimposed is displayed on the display device DSP.

Next, when the viewer presses the audio switch button once in state ST33, the control unit 270 changes the setting state to state ST34. Therefore, although the display of text and graphic images in the secondary audio mode is permitted by the program control information descriptor, the control unit 270 controls the switch 253b to be turned off when the viewer selects not to display the text and graphic data. As a result, the secondary audio is output from the speaker SPK, and a moving image without superimposed text and graphic images is displayed on the display device DSP.

Next, in state ST34, when the viewer presses the audio switch button once, the control unit 270 returns the setting state to state ST31. Therefore, the control unit 270 controls the audio selection unit 252 to switch to the secondary audio output in order to switch the audio mode from the primary audio mode to the secondary audio mode. As a result, the primary audio is output from the speaker SPK, and a moving image without superimposed text and graphic images is displayed on the display device DSP.

As described above, the broadcast receiving terminal device 20 operates to switch between displaying and hiding the text and graphic images linked to the audio mode each time the audio switch button is operated. Therefore, the viewer can display or hide the text and graphic images even if they are related to the audio mode.

[Fourth embodiment]
This embodiment is a variation of the above embodiment, and is characterized in that in a digital broadcasting service using broadcast signals based on a multiplexed transmission stream, when a program is configured in either a mono or stereo audio broadcasting format, control is performed so that a character or graphic image is superimposed on the main moving image of the program on the screen (e.g., a television camera image) in response to the operation of the audio switch button on the broadcast receiving terminal device, or control is performed so that the character or graphic image is not displayed on the screen.

That is, in conventional broadcast receiving terminal devices, the audio switching button on the remote controller is disabled (does not respond to operations) for programs that use the monaural or stereo broadcasting format. Therefore, the broadcast receiving terminal device 20 of this embodiment is configured to use the audio mode switching button on the remote controller to switch between displaying and not displaying text and graphic images for programs that use the monaural or stereo broadcasting format.

Figure 10 is a flow chart showing an example of processing in a broadcast receiving terminal device of a digital broadcast service system according to a fourth embodiment of the present invention. This processing is executed when the program selected by the broadcast receiving terminal device 20 is either a monaural broadcast format or a stereo broadcast format. Note that the processing by the broadcast receiving terminal device 20 is the same regardless of whether the program is a monaural broadcast format or a stereo broadcast format, so the following description will be given assuming that the program is a monaural broadcast format. Also, the setting state of the audio switch button on the broadcast receiving terminal device 20 is either a state in which a character and graphic image is displayed or a state in which it is not displayed.

As shown in the figure, the broadcast receiving terminal device 20 that receives the broadcast wave performs front-end processing (S1001). That is, as described above, the broadcast receiving terminal device 20 tunes (selects) the received broadcast to extract the desired broadcast signal, demodulates it, and further descrambles it using a decryption key, thereby restoring the multiplexed program stream. Next, the broadcast receiving terminal device 20 separates the program stream into each stream and decodes each separated stream, thereby generating each program material data.

At this time, the control unit 270 of the broadcast receiving terminal device 20 determines the audio broadcasting format of the selected program (S1002). That is, the control unit 270 determines whether the audio broadcasting format of the selected program is monaural broadcast/stereo broadcast or dual audio broadcast based on the program control information.

When the broadcast receiving terminal device 20 determines that the audio broadcasting method is monaural/stereo broadcasting, it then determines whether the current setting state is display or non-display (S1003). When the broadcast receiving terminal device 20 determines that the current setting state is display, it turns on the output of character and graphic data (S1004). On the other hand, when the broadcast receiving terminal device 20 determines that the current setting state is non-display, it turns off the output of character and graphic data (S1005). In other words, in this embodiment, the broadcast receiving terminal device 20 does not switch the audio data to be output.

Next, the broadcast receiving terminal device 20 superimposes the subtitle data and/or character and graphic data onto the video data to generate display data (S1006). If there is no output of subtitle data and/or character and graphic data, null data is superimposed onto the video data.

The broadcast receiving terminal device 20 controls the display device so that a screen based on the output display data is displayed, and also controls the speaker so that audible sound is output based on the output audio data (S1007).

If the broadcast receiving terminal device 20 determines in S1002 that the audio broadcasting method is not monaural/stereo broadcasting, it superimposes subtitle data and/or text and graphic data onto the video data in accordance with the current setting state, generates display data (S1006), and controls the display and audio output (S1007). As described in other embodiments, the broadcast receiving terminal device 20 may also perform processing in accordance with monaural/stereo broadcasting.

As described above, according to this embodiment, when a program is broadcast in either mono or stereo, the viewer can use the audio mode switching button provided on the broadcast receiving terminal device 20 and/or the remote controller to superimpose text and graphic images onto the main video image of the program on the screen, or hide the text and graphic images.

[Fifth embodiment]
This embodiment is a variation of the above embodiment, and is characterized in that in a digital broadcasting service using a broadcast signal based on a multiplexed transmission stream, a stream based on character and graphic images corresponding to each of a plurality of audio modes (e.g., main audio mode and secondary audio mode) is included in the transmission stream and broadcast, and the character and graphic images corresponding to the selected audio mode are displayed superimposed on the main moving image on the screen of the broadcast receiving terminal device.

Figure 11 is a block diagram showing an example of a schematic configuration of a broadcasting company system in a digital broadcasting service system according to the fifth embodiment of the present invention.

As shown in the figure, the broadcasting company system 10' of this embodiment differs from that shown in the above embodiment in that the character and graphic insertion unit 130' outputs first character and graphic data corresponding to the main audio and secondary audio, respectively, to the character and graphic encoding unit 125', which in turn outputs a first character and graphic stream and a second character and graphic stream, respectively. The multiplexing unit 150 multiplexes the first character and graphic stream and the second character and graphic stream together with other streams. The other components are the same as those in the above embodiment, so a description will be omitted.

Figure 12 is a block diagram showing an example of the general configuration of a broadcast receiving terminal device in a digital broadcast service system according to the fifth embodiment of the present invention.

As shown in the figure, the broadcast receiving terminal device 20' of this embodiment differs from the above embodiment in that a character/graphics decoding unit 245 decodes each of the first character/graphics stream and the second character/graphics stream separated from the program stream by the separation unit 230, thereby outputting the first character/graphics data and the second character/graphics data, and in that a character/graphics selection unit 255 is provided instead of the switch 253b.

That is, under the control of the control unit 270, the character/graphic selection unit 255 outputs either the first character/graphic data or the second character/graphic data to the synthesis unit 254 in conjunction with the selection operation of the audio selection unit 252. That is, when the viewer selects the main audio mode, under the control of the control unit 270, the audio selection unit 252 selects the first audio data, and the character/graphic selection unit 255 selects the corresponding first character/graphic data. On the other hand, when the viewer selects the secondary audio mode, under the control of the control unit 270, the audio selection unit 252 selects the second audio data, and the character/graphic selection unit 255 selects the corresponding second character/graphic data.

As described above, according to this embodiment, the broadcaster transmits text and graphic data corresponding to each audio mode in the transmission stream, so that the text and graphic images corresponding to the audio mode selected by the viewer are appropriately superimposed on the main video image of the program on the screen.

In other words, the character and graphic image will be displayed on the screen of the broadcast receiving terminal device 20 only in the audio mode specified by the broadcasting company. More specifically, when the current audio mode is the main audio mode, the broadcast receiving terminal device 20 selects the first audio data and also selects the corresponding first character and graphic data, outputs it to the synthesis unit 254, and synthesizes it by superimposing it on the video data. As a result, audio based on the first audio data is output from the speaker SPK, while the display device DSP displays a screen in which the first character and graphic image corresponding to, for example, the audio mode is superimposed on the video captured by the television camera.

When the current audio mode is the secondary audio mode, the broadcast receiving terminal device 20 selects the second audio data and also selects the second character and graphic data, outputs it to the synthesis unit 254, and synthesizes it by superimposing it on the video data. As a result, audio based on the second audio data is output from the speaker SPK, while the display device DSP displays a screen in which the second character and graphic image, for example handwritten and input with a pen, is superimposed on the image captured by the television camera.

(Modification)
As a variation of this embodiment, the broadcast receiving terminal device 20 may be controlled so that, in response to operation by the viewer, the audio mode can be switched and text and graphic images superimposed on the main moving image of the program on the screen can be selectively displayed or hidden.

In other words, the broadcast receiving terminal device 20 of this embodiment is configured to switch between audio modes and to switch between displaying and hiding either the first character/graphic image or the second character/graphic image corresponding to the audio mode, for example, each time the viewer operates the audio switch button on the remote controller. In this case, as shown in the modified example of the third embodiment (Figure 9), the broadcast receiving terminal device 20 performs control to cyclically switch between states ST31 to ST34 each time the audio switch button is operated.

For example, assume that the current setting state is state ST31. Therefore, the main sound is output from the speaker SPK, and a moving image with the first character/graphic image superimposed is displayed on the display device DSP. When the viewer presses the sound switch button once in state ST31, the control unit 270 changes the setting state to state ST32. Therefore, the control unit 270 controls the character/graphic selection unit 255 to turn off. As a result, the output from the speaker SPK remains the main sound, and a moving image without the first character/graphic image superimposed is displayed on the display device DSP.

Next, when the viewer presses the audio switch button once in state ST32, the control unit 270 changes the setting state to state ST33. Therefore, in order to switch the audio mode from the main audio mode to the secondary audio mode, the control unit 270 controls the audio selection unit 252 to switch to output secondary audio, and controls the character/graphic selection unit 255 to select the second character/graphic image. As a result, the secondary audio is output from the speaker SPK, and the display device DSP displays a moving image with the second character/graphic image superimposed thereon.

Next, when the viewer presses the audio switch button once in state ST33, the control unit 270 changes the setting state to state ST34. Therefore, the control unit 270 controls the character/graphic selection unit 255 to be turned off. As a result, the output from the speaker SPK remains the secondary audio, and the display device DSP displays a moving image without the second character/graphic image superimposed.

Next, in state ST34, when the viewer presses the audio switch button once, the control unit 270 returns the setting state to state ST31. Therefore, in order to switch the audio mode from the main audio mode to the secondary audio mode, the control unit 270 controls the audio selection unit 252 to switch to secondary audio output, and controls the character/graphic selection unit 255 to select the first character/graphic image. As a result, the main audio is output from the speaker SPK, and a moving image with the first character/graphic image superimposed is displayed on the display device DSP.

Next, the processing in the broadcast receiving terminal device of this embodiment will be described. The processing in the broadcast receiving terminal device is basically the same as the processing shown in FIG. 7, except for the character and graphic data output control processing of S703. FIG. 13 is a flowchart showing an example of processing in a broadcast receiving terminal device of a digital broadcast service system according to the fifth embodiment of the present invention, and specifically shows the details of the character and graphic data output control processing corresponding to S703 shown in FIG. 7.

That is, in the output control process of the character/graphic data of this embodiment, as shown in FIG. 13, the broadcast receiving terminal device 20 judges which is the current audio mode (S1301). If the broadcast receiving terminal device 20 judges that the current audio mode is the main audio mode, it then judges whether the setting state of the audio switch button specifies that the first character/graphic image is displayed (S1302). If the broadcast receiving terminal device 20 judges that the setting state specifies that the first character/graphic image is displayed, it selects and outputs the first audio data and turns on the output of the first character/graphic data (S1303). On the other hand, if the broadcast receiving terminal device 20 judges that the setting state specifies that the first character/graphic image is not displayed, it selects and outputs the first audio data and turns off the output of the first character/graphic data and the second character/graphic data (S1304).

On the other hand, in S1301, if the broadcast receiving terminal device 20 determines that the current audio mode is not the main audio mode (it is the secondary audio mode), it then determines whether the setting state of the audio switch button specifies that the second character/graphic image is displayed (S1305). If the broadcast receiving terminal device 20 determines that the setting state specifies that the second character/graphic image is displayed, it selects and outputs the second audio data and also selects and outputs the second character/graphic data (S1306). On the other hand, if the broadcast receiving terminal device 20 determines that the setting state specifies that the second character/graphic image is not displayed (No in S1305), it selects and outputs the second audio data and turns off the output of the first character/graphic data and the second character/graphic data (S1307).

As described above, the broadcast receiving terminal device 20 operates to switch between displaying and hiding the text and graphic images linked to the audio mode each time the audio switch button is operated. Therefore, the viewer can display or hide the text and graphic images even if they are associated with the audio mode.

The above embodiments are merely examples for explaining the present invention, and are not intended to limit the present invention to these embodiments. The present invention can be implemented in various forms without departing from the gist of the invention.

For example, in the methods disclosed herein, steps, operations, or functions may be performed in parallel or in a different order, provided that the results are not inconsistent. The steps, operations, and functions described are provided merely as examples, and some of the steps, operations, and functions may be omitted or combined into one, or other steps, operations, or functions may be added, without departing from the spirit of the invention.

In addition, although various embodiments are disclosed in this specification, specific features (technical matters) in one embodiment can be added to or replaced with specific features in another embodiment, with appropriate modifications, and such forms are also included in the gist of the present invention.

1... Digital broadcasting service system 10... Broadcasting company system 110... Program operation device 120... Encoding section 121... Video encoding section 122... Still image encoding section 123... Audio encoding section 124... Subtitle encoding section 130... Character and graphic insertion section 140... Program information generation section 150... Multiplexing section 160... Scrambling section 170... Modulation section 180... Transmission section 20... Broadcast receiving terminal device 210... Tuner/modulation section 220... Descrambling section 230... Separation section 240... Decoding section 250... Output control section 251... Video/still image switching section 252... Audio selection section 253a, 253b... Switch 254... Synthesis section 255... Character and graphic selection section 260... Data broadcasting engine 270... Control section 280... Operation reception section DSP... Display device SPK... Speaker

Claims (5)

A broadcast receiving terminal device adapted for digital broadcasting services,
a front-end processing unit that generates at least video data, first audio data, second audio data, subtitle data, and text and graphic data based on a broadcast signal of a received program;
a display control unit that controls the display of a program image on a screen of a display device based on the display data;
an audio control unit that performs control so that audio based on either the first audio data or the second audio data is output from a speaker when the audio broadcasting method of the program is dual audio broadcasting ,
the display control unit generates the display data by synthesizing the character and graphic data on the video data in a superimposed manner in conjunction with output of either the first audio data or the second audio data selected in accordance with an operation of a viewer.
Broadcast receiving terminal equipment. A broadcast receiving terminal device adapted for digital broadcasting services,
a front-end processing unit that generates at least video data, audio data, and text and graphic data based on a broadcast signal of a program to be received;
a display control unit that controls so that a program image in either the first form or the second form is displayed on a screen of a display device;
a sound control unit that controls so that sound based on the audio data is output from a speaker,
the program image of the first form is an image including at least a moving image based on the video data and a character and graphic image based on the character and graphic data,
the second type of program image is an image including the moving image but not including the text and graphic image,
When the program is configured in either a monaural broadcasting format or a stereo broadcasting format, the display control section controls the program image in either the first format or the second format to be selectively displayed on the screen in response to an operation of an audio switching button of a remote controller associated with the broadcast receiving terminal device.
Broadcast receiving terminal equipment. The display control unit is
a switching control unit for controlling whether or not the character and graphic data is to be output;
a synthesis unit for synthesizing the video data and the character and graphic data,
The switching control unit is
Each time the voice switching button is operated, control is performed so as to switch whether or not the character and graphic data is to be output to the synthesis unit.
3. A broadcast receiving terminal device according to claim 2. A broadcast receiving terminal device adapted for digital broadcasting services,
a front-end processing unit that generates at least video data, first audio data, second audio data, character and graphic data, and program control information based on a broadcast signal of a program received;
a display control unit that controls so that a program image in either the first form or the second form is displayed on a screen of a display device;
an audio processing unit that controls so that audio based on either the first audio data or the second audio data is output from a speaker according to an audio mode;
the program image of the first form is an image including at least a moving image based on the video data and a character and graphic image based on the character and graphic data,
the second type of program image is an image including the moving image but not including the text and graphic image,
When the program control information includes information for designating display of a character/graphic image in a first sound mode, the display control unit controls so that the program image in the first form is displayed on the screen, and the sound processing unit controls so that the sound based on the first audio data is output from the speaker;
When the program control information includes information for specifying display of a character/graphic image in a second audio mode, the display control unit controls so that the program image in the first form is displayed on the screen, and the audio processing unit controls so that the audio based on the second audio data is output from the speaker.
Broadcast receiving terminal equipment. The display control unit is
a switching control unit for controlling whether or not the character and graphic data is to be output;
a synthesis unit for synthesizing the video data and the character and graphic data,
The switching control unit is
When the program control information includes information for specifying display of a character/graphic image in the first audio mode, the character/graphic data is output to the synthesis unit;
When the program control information includes information for specifying display of a character/graphic image in the second audio mode, the control unit controls so as not to output the character/graphic data to the synthesis unit.
5. A broadcast receiving terminal device according to claim 4.

Images