JP3927713B2 - Broadcast receiving apparatus and method thereof - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a broadcast receiving apparatus and method thereof, and more particularly to a broadcast receiving apparatus and method capable of receiving a digital television broadcast and reproducing images and sound.
[0002]
[Prior art]
In recent years, digital television broadcasting using satellite broadcasting and cable broadcasting has been started. By realizing digital broadcasting, images and voice Many effects are expected, such as improving the quality of sound including sound, increasing the types and volume of programs using compression technology, providing new services such as interactive services, and evolving reception formats.
[0003]
FIG. 1 is a block diagram showing a configuration example of a digital broadcast receiver 10 using satellite broadcasting.
[0004]
First, television (TV) information transmitted by a broadcasting satellite is received by the antenna 1. The received TV information is selected by the tuner 2 and demodulated. Thereafter, although not shown, error correction processing, billing processing, descrambling processing, and the like are performed if necessary. Next, various data multiplexed as TV information is separated by the multiplexed signal separation circuit 3. TV information is separated into image information, sound information and other additional data. Each separated data is decoded by the decoding circuit 4. Of the data thus decoded, the image information and the sound information are converted to analog by the D / A conversion circuit 5 and reproduced by the television receiver (TV) 6. On the other hand, the additional data has a role as program sub-data and is involved in various functions.
[0005]
Further, the VTR 7 is used for recording and reproducing the received TV information. The receiver 10 and the VTR 7 are connected by a digital interface such as IEEE1394. The VTR 7 has a digital recording format, and records TV information in a bitstream by, for example, the D-VHS format. Not only D-VHS bitstream recording but also other consumer digital recording formats such as DV format and digital recording devices using various disk media can record TV information of digital TV broadcasts. It is. However, format conversion may be necessary.
[0006]
[Problems to be solved by the invention]
When a television program in terrestrial broadcasting or digital television broadcasting is reproduced on a home television, it is common to display the video sent from the broadcasting station as it is. In other words, an act of effectively changing the display form (layout), for example, not displaying an object in the video as necessary, or changing the size of the object is not performed. Such a function to change the display format effectively is necessary from the viewpoint of adding new functions of effective display methods in the process of multi-channel and multi-programming with the development of digital television broadcasting. It is considered one of the things.
[0007]
The following are examples of situations where you want to set the layout. Baseball broadcasts have different display layouts depending on the broadcasting station, even if they are programs of the same category. For this reason, for example, in order to view a score display object or the like in a common layout regardless of the broadcasting station, it is desirable that the user can set a layout that suits his / her preference. However, layout setting is difficult at present.
[0008]
The present invention relates to an image in digital television broadcasting. (Video) playback format The purpose is to provide new functions.
[0009]
[Means for Solving the Problems]
The present invention has the following configuration as one means for achieving the above object.
[0010]
A broadcast receiving apparatus according to the present invention includes a receiving unit that receives a digital data sequence of a television broadcast, a decoding unit that decodes an image, sound, and system data from the received digital data sequence, and the decoded system data. And control means for controlling the playback mode of the decoded image and / or sound based on the category information indicating the broadcast content included in the system data. And an image object constituting the decoded image Playback format corresponding to Setting data Is read from the storage means and applied to the broadcast program being received.
[0012]
The broadcast receiving method according to the present invention receives a digital data sequence of a television broadcast, decodes an image, sound and system data from the received digital data sequence, and based on the decoded system data, decodes the decoded image and / or Alternatively, each step of controlling the sound reproduction form includes category information indicating broadcast contents included in the system data. And image objects constituting the decoded image Playback format corresponding to Setting data Is read from the storage means and applied to the broadcast program being received.
[0014]
DETAILED DESCRIPTION OF THE INVENTION
[Overview]
This embodiment uses the concept of an object, which is a feature of MPEG4 coding, to enable the display position to be changed in units of objects, and is specific to the user and unified according to the program content regardless of the broadcasting station. Realize layout image (video) display. An object is a background image, a speaker and its voice, and MPEG4 encoding encodes / decodes each object and combines each object to express one scene.
[0015]
In the broadcast system using MPEG4, the specific layout setting function of the present embodiment is capable of manipulating the displayed image in units of objects for the display of real-time image information, and is unified according to the category information of the program A function for setting an image (video) display of the arranged layout. The layout corresponding to this category includes a predetermined position and a position arbitrarily set by the user.
[0016]
Also, in the present embodiment, object layout and layout control are performed when a layout corresponding to a category of a program is performed with reference to object information indicating the contents of the object.
[0017]
As described above, according to the present embodiment, it is possible to eliminate the drawbacks that are displayed in different layouts depending on the broadcasting station even for programs in the same category, and a layout that is unified for each category regardless of the broadcasting station. Can display common objects. In addition to having a default layout, it is also possible to arbitrarily set a layout preferred by the user. Therefore, it is possible to improve the visual effect and user interface quality of the user who views the digital television broadcast, and to add a new function to the video display of the television broadcast.
[0018]
In the following, a configuration example of a receiving apparatus that receives a digital television broadcast using the MPEG4 encoding method will be described as a receiving apparatus according to an embodiment of the present invention. First, a technology related to MPEG4 will be described in detail for each field. To do.
[0019]
[Outline of MPEG4]
[General structure of the standard]
The MPEG4 standard consists of four major items. Three of these items are similar to MPEG2, and are a visual part, an audio part, and a system part.
[0020]
● Visual part
An object encoding method that handles natural images, composite images, moving images, still images, and the like is defined as a standard. Also included are an encoding method suitable for correcting and repairing transmission path errors, a synchronous reproduction function, and hierarchical encoding. In terms of expression, “video” means a natural image, and “visual” includes a composite image.
[0021]
● Audio part
An object encoding method for natural sounds, synthesized sounds, and sound effects is defined as a standard. The video part and the audio part are devised to increase the encoding efficiency by defining a plurality of encoding methods and appropriately selecting a compression method suitable for the characteristics of each object.
[0022]
● System part
It stipulates the multiplexing processing of encoded video objects and sound objects and vice versa. In addition, buffer memory, time axis control and readjustment functions are included in this part. The video objects and sound objects encoded in the above visual part and audio part are integrated into the multiplexed stream of the system part together with scene configuration information describing the position, appearance time and disappearance time of the object in the scene. . As a decoding process, each object is separated / decoded from the received bit stream, and a scene is reconstructed based on the scene configuration information.
[0023]
[Object encoding]
In MPEG2, encoding is performed in units of frames or fields, but in order to realize reuse and editing of content, MPEG4 handles video data and audio data as objects (objects). There are the following types of objects:
sound
Natural image (background video: 2D fixed video)
Natural image (main subject video: no background)
Composite image
Character image
[0024]
FIG. 2 shows an example of a system configuration in which these are simultaneously input and encoded. The sound object encoder 5001, natural image object encoder 5002, composite image object encoder 5003, and character object encoder 5004 respectively encode the objects. At substantially the same time as this encoding, the scene description information encoder 5005 encodes the scene configuration information indicating the relationship between the objects in the scene. The encoded object information and scene description information are encoded into an MPEG4 bit stream by the data multiplexer 5006.
[0025]
In this way, on the encoding side, a combination of a plurality of visual objects and audio objects is defined to express one scene (screen). As for visual objects, a scene combining a natural image and a composite image such as computer graphics can be configured. Further, by adopting the above-described configuration, for example, a synchronized reproduction of the subject video and the sound can be performed using a text-to-speech synthesis function. Note that transmission / reception or recording / reproduction is performed in the bit stream state.
[0026]
The decoding process is a reverse process of the previous encoding process. The data separator 5007 separates and distributes the MPEG4 bit stream into each object. Objects such as separated sounds, natural images, composite images, and characters are decoded into object data by corresponding decoders 5008 to 5011. The scene description information is also decoded by the decoder 5012 almost simultaneously. Using these decoded information, the scene synthesizer 5013 synthesizes the original scene.
[0027]
On the decoding side, partial changes such as the position of visual objects included in the scene and the order of audio objects can be made. The object position can be changed by dragging, and the language can be changed by the user changing the audio object.
[0028]
In order to synthesize a scene by freely combining a plurality of objects, the following four items are defined.
[0029]
● Object coding
A visual object, an audio object, and an AV (audio visual) object obtained by combining them are set as encoding targets.
[0030]
● Scene synthesis
In order to define scene composition information and a composition method for composing a visual object, an audio object, and an AV object into a desired scene, a language in which Virtual Reality Modeling Language (VRML) is modified is used.
[0031]
● Multiplexing and synchronization
The format of a stream (elementary stream) in which each object is multiple-synchronized is determined. Quality of service (QOS) can also be set when this stream is sent to the network or stored in the recording device. QOS parameters include transmission path conditions such as maximum transmission rate, error rate, and transmission method, and decoding capability.
[0032]
● User operation (interaction)
A method for synthesizing visual objects and audio objects on the user terminal side is defined. An MPEG4 user terminal separates data sent from a network or a recording device into elementary streams and decodes each object. A scene is reconstructed from a plurality of encoded data based on the scene configuration information sent simultaneously.
[0033]
Figure 3 shows an example of a system configuration that takes user operations (editing) into consideration. FIG. 4 is a block diagram on the encoder side of the VOP processing circuit for the video object, and FIG. 5 is a block diagram on the decoder side.
[0034]
[VOP (Video Object Plane)]
Video encoding in MPEG4 is performed by dividing a target video object into a shape (Shape) and a picture (Texture). The unit of this video data is called VOP. FIG. 6 is a block diagram showing the overall configuration of VOP encoding and decoding.
[0035]
For example, when an image is composed of two objects, a person and a background, each frame is divided into two VOPs and encoded. Information constituting each VOP is object shape information, motion information, and texture information, as shown in FIG. 7A. On the other hand, the decoder separates the bitstreams for each VOP and decodes them individually, and then combines them to form an image.
[0036]
Thus, by introducing the VOP structure, when the image to be processed is composed of a plurality of video objects, it can be divided into a plurality of VOPs and individually encoded / decoded. When the number of VOPs is 1 and the object shape is a rectangle, encoding is performed in units of frames as in the past, as shown in FIG. 7B.
[0037]
There are three types of VOPs: intra-frame coding (I-VOP), forward prediction (P-VOP), and bidirectional prediction (B-VOP). The prediction unit is a 16 × 16 pixel macroblock (MB).
[0038]
Bidirectional prediction B-VOP is a method for predicting a VOP from both the past VOP and the future VOP in the same way as MPEG1 and MPEG2 B pictures. Then, four types of modes of direct coding / forward coding / backward coding / bidirectional coding can be selected for each macroblock. This mode can be switched in MB or block units. Bi-directional prediction using P-VOP motion vector scaling.
[0039]
[Shape coding]
In order to handle an image in object (object) units, the shape of the object must be known at the time of encoding and decoding. Further, in order to express an object such as glass through which an object behind can be seen, information indicating the transparency of the object is required. The information on the shape of the object and the transparency of the object are collectively referred to as shape information. The encoding of shape information is called shape encoding.
[0040]
[Size conversion processing]
Binary shape coding is a method of coding a boundary pixel by determining whether the pixel is outside or inside the object. Therefore, the smaller the number of pixels to be encoded, the smaller the amount of generated code. However, if the macroblock size to be encoded is reduced, the original shape code is degraded and transmitted to the receiving side. Therefore, how much the original information is degraded by the size conversion is measured, and a macro block size as small as possible is selected as long as a size conversion error equal to or less than a predetermined threshold value is obtained. Specific size conversion ratios include three types: full size, vertical / horizontal 1/2 times, and vertical / horizontal 1/4 times.
[0041]
The shape information of each VOP is given as an 8-bit α value and is defined as follows.
α = 0: Outside the corresponding VOP
α = 1 to 254: Displayed in a translucent state with other VOPs
α = 255: Display area of the corresponding VOP only
[0042]
Binary shape coding is a case where the α value takes 0 or 255, and the shape is expressed only inside and outside the corresponding VOP. Multi-level shape coding is a case where α values can take all values from 0 to 255, and can represent a state in which a plurality of VOPs are semitransparently superimposed.
[0043]
Similar to texture coding, motion compensated prediction with one pixel accuracy is performed for each block of 16 × 16 pixels. When the entire object is encoded in-plane, the shape information is not predicted. As the motion vector, a difference between motion vectors predicted from adjacent blocks is used. The obtained difference value of the motion vector is encoded and then multiplexed into a bit stream. In MPEG4, the shape information of the block unit predicted by motion compensation is binary shape encoded.
[0044]
● Feathering
In addition, even in the case of a binary shape, feathering (border shape smoothing) is used to smoothly change the boundary from opaque to transparent. The feathering includes a linear feathering mode that linearly interpolates boundary values and a feathering filter mode that uses a filter. Multi-valued shapes with constant opacity have a constant alpha mode that can be combined with feathering.
[0045]
[Texture coding]
It encodes the luminance component and color difference component of an object, and processes in the order of DCT (Discrete Cosine Tranfer), quantization, predictive coding, and variable length coding in field / frame units.
[0046]
DCT uses a block of 8 × 8 pixels as a processing unit, but if the object boundary is within the block, the pixel outside the object is compensated by the average value of the object. After that, a 4-tap two-dimensional filter process is performed to prevent a phenomenon in which a large pseudo peak occurs in the DCT coefficient.
[0047]
Quantization uses either the ITU-T Recommendation H.263 quantizer or the MPEG2 quantizer. If MPEG2 quantizer is used, nonlinear quantization of DC component and frequency weighting of AC component are possible.
[0048]
The in-plane coding coefficient after quantization is predictively coded between blocks before variable length coding, and redundant components are deleted. In particular, MPEG4 predictively encodes both DC and AC components.
[0049]
As shown in Fig. 8, texture coding AC / DC predictive coding examines the difference (gradient) of the corresponding quantization coefficient between blocks adjacent to the target block, and uses the smaller quantization coefficient for prediction. . For example, when the DC coefficient x of the block of interest is predicted, if the DC coefficients of the corresponding adjacent blocks are a, b, and c, the following occurs.
| a-b | If <| b-c |, use DC coefficient c for prediction
If | a-b | ≥ | b-c |, use DC coefficient a for prediction
[0050]
When predicting the AC coefficient X of the block of interest, the coefficient used for prediction is selected in the same manner as described above, and then normalized with the quantization scale value QP of each block.
[0051]
Predictive coding of DC components is performed by examining the difference in DC components between adjacent blocks (vertical gradient) between adjacent blocks and the difference in DC components (horizontal gradient) between blocks adjacent to the left and right. The difference from the DC component of the block is encoded as a prediction error.
[0052]
The prediction encoding of the AC component uses the coefficient corresponding to the adjacent block in accordance with the prediction encoding of the DC component. However, since there is a possibility that the value of the quantization parameter is different between blocks, the difference is obtained after normalization (quantization step scaling). Presence / absence of prediction can be selected for each macroblock.
[0053]
Thereafter, the AC component is zigzag scanned and three-dimensional (Last, Run and Level) variable length encoded. Here, Last is a 1-bit value indicating the end of a non-zero coefficient, Run is a zero duration, and Level is a non-zero coefficient value.
[0054]
For variable length coding of the DC component subjected to in-plane coding, either a variable length coding table for DC component or a variable length table for AC component is used.
[0055]
[Motion compensation]
In MPEG4, a video object plane (VOP) having an arbitrary shape can be encoded. As described above, VOP includes in-plane coding (I-VOP), forward prediction coding (P-VOP), and bidirectional prediction coding (B-VOP) depending on the type of prediction. Use macroblocks of 16 lines x 16 pixels or 8 lines x 8 pixels. Therefore, there are macroblocks that straddle the VOP boundary. In order to improve the prediction efficiency of the VOP boundary, padding (complementation) and polygon matching (matching of only the object portion) are performed on the macroblock on the boundary.
[0056]
[Wavelet coding]
Wavelet transformation is a transformation method that uses a plurality of functions obtained by enlarging / reducing / translating one solitary wave function as a transformation base. This still image coding mode (Wavelet Transform) using wavelet transform is various, especially when dealing with a computer graphics (CG) image combined with a natural image. It is suitable as a high-quality encoding method with spatial resolution. In wavelet coding, an image can be coded all at once without being divided into blocks. Therefore, block distortion does not occur even at a low bit rate, and mosquito noise can be reduced. In this way, MPEG4 still image coding mode has a wide range of scalability from low-resolution and low-quality images to high-resolution and high-quality images, processing complexity, and coding efficiency trade-offs depending on the application. Can be adjusted.
[0057]
[Hierarchical coding (scalability)]
In order to realize scalability, a hierarchical structure of syntax as shown in FIGS. 9A and 9B is configured. Hierarchical coding is realized, for example, by coding “difference information” that improves the image quality of the base layer in the reinforcement layer, with the base layer as the lower layer and the reinforcement layer as the upper layer. In the case of spatial scalability, the base layer represents a low-resolution moving image, and the “base layer + reinforcement layer” represents a high-resolution moving image.
[0058]
In addition to improving the image quality of the entire image hierarchically, there is a function of improving the image quality of only the object region in the image. For example, in the case of temporal scalability, the base layer is obtained by coding the entire image at a low frame rate, and the reinforcement layer is obtained by coding data for improving the frame rate of a specific object in the image.
[0059]
● Time scalability
The temporal scalability shown in FIG. 9A can layer the frame speed and increase the frame speed of the object in the reinforcement layer. Whether to hierarchize can be set for each object. There are two types of reinforcement layers, and type 1 consists of a part of base layer objects. Type 2 consists of the same objects as the base layer.
[0060]
● Spatial scalability
Spatial scalability shown in FIG. 9B layers the spatial resolution. The base layer can be down-sampled of any size and is used for prediction of the reinforcement layer.
[0061]
[Sprite coding]
A sprite is a planar object that can express movement, rotation, deformation, and the like as a whole, such as a background image in a three-dimensional space image. This method of encoding a planar object is called sprite encoding.
[0062]
There are four types of sprite coding: static / dynamic and online / offline. More specifically, a static sprite that is configured to send object data to a decoder in advance and transmit only global motion coefficients in real time, and is obtained by direct conversion of a template object. A dynamic sprite obtained by predictive coding from the previous sprite in time. An off-line sprite that is encoded in advance by in-plane encoding (I-VOP) and transmitted to the decoder side. There are online sprites that are created simultaneously at the encoder and decoder during encoding.
[0063]
Techniques studied for sprite coding include static sprite coding, dynamic sprite coding, and global motion compensation.
[0064]
● Static sprite coding
Static sprite coding is a method of expressing an image by previously coding a background (sprite) of the entire video clip and geometrically converting a part of the background. Some of the cut out images can express various deformations such as translation, enlargement / reduction, and rotation. In this regard, as shown in FIG. 10A, expressing viewpoint movement in a three-dimensional space by moving, rotating, enlarging / reducing, or deforming an image is called warp.
[0065]
Warp types include perspective transformation, affine transformation, isotropic expansion (a) / rotation (θ) / movement (c, f), and translation, and are represented by the equations in FIG. 10B. Movement, rotation, enlargement / reduction, deformation, etc. are represented by the coefficients of the equation shown in FIG. 10B. Sprite generation is performed off-line before the start of encoding.
[0066]
In this way, static sprite coding is realized by cutting out a partial region of the background image and warping the region. A partial area included in the sprite (background) image shown in FIG. 11 is warped. For example, the background image is an image of a spectator seat in a tennis game, and the warped area is an image including a moving object such as a tennis player. In static sprite encoding, only the geometric transformation parameters are encoded, and the prediction error is not encoded.
[0067]
● Dynamic sprite coding
In static sprite encoding, sprites are generated before encoding. On the other hand, in dynamic sprite encoding, sprites can be updated online while encoding. Also, it differs from static sprite coding in that the prediction error is coded.
[0068]
● Global motion compensation (GMC)
Global motion compensation is a technique that compensates motion by representing motion of an entire object as a single motion vector without dividing the motion into blocks, and is suitable for motion compensation of a rigid body. It is the same as the static sprite coding in that the reference image becomes the immediately preceding decoded image instead of the sprite and the prediction error is encoded. However, the fact that a memory for storing sprites is not required and that shape information is not required is different from static sprite encoding and dynamic sprite encoding. This is effective for the movement of the entire screen and images including zoom.
[0069]
[Scene structure description information]
Objects are synthesized based on the scene configuration information. In MPEG4, configuration information for synthesizing each object into a scene is transmitted. When the individual encoded objects are received, the scene configuration information can be used to synthesize the scene as intended by the transmission side.
[0070]
The scene configuration information includes the display time and display position of the object, and these are described as tree-like node information. Each node has relative time information and relative space coordinate position information on the time axis with respect to the parent node. Languages for describing scene configuration information include BIFS (Binary Format for Scenes) with modified VRML and AAVS (Adaptive Audio-Visual Session Format) using Java (TM). BIFS is a format that describes MPEG4 scene configuration information in binary. AAVS is based on Java (TM) and has a high degree of freedom and is positioned to supplement BIFS. FIG. 12 is a diagram showing a configuration example of scene description information.
[0071]
[Scene description]
Scene description is done by BIFS. Here, we will focus on scene graphs and nodes, which are common concepts between VRML and BIFS.
[0072]
The node specifies attributes such as light source, shape, material, color, and coordinates, and grouping of lower nodes accompanied by coordinate transformation. Taking the object-oriented idea, the arrangement and appearance of each object in the three-dimensional space is determined by tracing a tree called a scene graph from the node at the vertex and inheriting the attribute of the upper node. If a media object, for example, a bit stream of MPEG4 video is allocated to a node corresponding to a leaf in synchronization, a moving image can be combined with other graphics and displayed in a three-dimensional space.
[0073]
Differences from VRML are as follows. The MPEG4 system supports the following with BIFS:
(1) MPEG4 video VOP coding two-dimensional overlap description and MPEG4 audio composition description
(2) Continuous media stream synchronization processing
(3) Dynamic behavior representation of objects (eg sprites)
(4) Standardized transmission format (binary)
(5) Dynamically change the scene description during the session
[0074]
Except for VRML nodes such as Extrusion, Script, Proto, and ExtemProto, almost all VRML nodes are supported by BIFS. The new MPEG4 special nodes added in BIFS are as follows.
(1) Node for 2D / 3D composition
(2) Node for 2D graphics and text
(3) Animation node
(4) Audio node
[0075]
It should be noted that VRML did not support 2D compositing except for special nodes such as the background, but BIFS has extended the description to handle text, graphic overlay, and MPEG4 video VOP encoding on a pixel-by-pixel basis. Yes.
[0076]
In the animation node, a special node for MPEG4 CG image such as a face composed of 3D mesh is defined. There is a message (BIFS Update) that can dynamically replace, delete, add and change attributes in the scene graph, and it is possible to display a new moving image on the screen or add a button during the session become. BIFS can be realized by replacing VRML reserved words, node identifiers, and attribute values with binary data almost on a one-to-one basis.
[0077]
[MPEG4 audio]
FIG. 13 shows the types of MPEG4 audio encoding methods. Audio and sound coding includes parametric coding, CELP (Code Excited Linear Prediction) coding, and time / frequency transform coding. Furthermore, it also incorporates SNHC (Synthetic Natural Hybrid Coding) audio functions and includes SA (Structured Audio) coding and TTS (Text to Speech) coding. SA is a structural description language for synthesized musical sounds including MIDI (Music Instrument Degital Interface), and TTS is a protocol for sending intonation and phonological information to an external text-to-speech synthesizer.
[0078]
FIG. 14 shows the configuration of the audio encoding method. In FIG. 14, the input sound signal is preprocessed (201), and the three encodings of parametric encoding (204), CELP encoding (205) and time / frequency encoding (206) are used according to the band. Then, the signal is divided (202) and input to an encoder suitable for each. Also, the signal analysis control (203) analyzes the input audio signal and generates control information for assigning the input audio signal to each encoder.
[0079]
Subsequently, the parametric coding core (204), the CELP coding core (205), and the time / frequency transform coding core (206), which are different encoders, perform coding processing based on each coding method. Execute. These three types of encoding will be described later. Audio data subjected to parametric coding and CELP coding is subjected to small step enhancement (207), and audio data subjected to time / frequency transform coding and small step enhancement is subjected to large step enhancement (208). The small step enhancement (207) and the large step enhancement (208) are tools for reducing distortion generated in each encoding process. Thus, the audio data enhanced by a large step becomes an encoded sound bitstream.
[0080]
The above is the description of the configuration of the sound encoding system in FIG. 75. Next, each encoding system will be described with reference to FIG.
[0081]
● Parametric coding
A sound signal including an audio signal and a musical sound signal is expressed by parameters such as frequency, amplitude, and pitch, and encoded. It includes harmonic vector drive coding (HVXC) for audio signals and individual spectrum (IL) coding for musical signals.
[0082]
HVXC coding is mainly intended for 2k to 4kbps speech coding. Voice signals are classified into voiced and unvoiced sounds, and the voiced sound is a harmonic of the residual signal of the linear prediction coefficient (LPC). ) Vector quantize the structure. For unvoiced sounds, the prediction residual is subjected to vector excitation coding as it is.
[0083]
IL coding is aimed at coding a musical sound of 6 k to 16 kbps, and a signal is modeled with a line spectrum and coded.
[0084]
CELP encoding
In this method, an input sound signal is separated into spectral envelope information and sound source information (prediction error) and encoded. The spectral envelope information is represented by a linear prediction coefficient calculated from the input sound signal by linear prediction analysis. MPEG4 CELP encoding includes narrowband CELP with a bandwidth of 4 kHz and wideband CELP with a bandwidth of 8 kHz. A bit rate can be selected between k and 24 kbps.
[0085]
● Time / frequency transform coding
This is an encoding method aiming at high sound quality. This includes an AAC (Advanced Audio Coding) compliant scheme and TwinVQ (Transform-domain Weighted Interleave Vector Quantization). This time / frequency transform coding incorporates an auditory psychological model, and has a mechanism for adaptive quantization using the auditory masking effect.
[0086]
The AAC compliant system is a mechanism for frequency-converting audio signals using DCT and performing adaptive quantization while using the auditory masking effect. The adaptive bit rate is 24k to 64kbps.
[0087]
The TwinVQ method flattens the MDCT coefficient of an audio signal using a spectral envelope obtained by linear prediction analysis of the audio signal. After interleaving, vector quantization is performed using two code lengths. The adaptive bit rate is 6k-40kbps.
[0088]
[System structure]
The MPEG4 system part defines multiplexing, demultiplexing and composition. Hereinafter, the system structure will be described with reference to FIG.
[0089]
In multiplexing, each object stream that is output from the video encoder and audio encoder, and elementary streams such as scene configuration information that describes the spatio-temporal arrangement of each object are packetized at the access unit layer. The In the access unit layer, a time stamp and a reference clock for synchronization in units of access units are added as headers. The packetized stream is then multiplexed in the FlexMux layer in units of display and error resilience and sent to the TransMux layer.
[0090]
In the TransMux layer, an error correction code is added in the protection sublayer according to the necessity of error resilience. Finally, it is sent out to the transmission path as a single TransMux stream in the multiple sublayer (Mux Sub Layer). The TransMux layer is not defined in MPEG4, but the Internet protocol UDP / IP (User Datagram Protocol / Internet Protocol), MPEG2 transport stream (TS), ATM (Asynchronous Transfer Mode) AAL2 (ATM Adaptation layer2), Existing network protocols such as a videophone multiplexing system using telephone lines (ITU-T recommendation H.223) and digital audio broadcasting can be used.
[0091]
It is also possible to bypass the access unit layer and the FlexMux layer so that the overhead of the system layer is reduced and can be easily embedded in a conventional transport stream.
[0092]
On the decoding side, in order to synchronize each object, a buffer (DB: Decoding Buffer) is provided after the demultiplexing (separation) to absorb the arrival time of each object and the deviation in decoding time. A buffer (CB: Composition Buffer) is also provided before composition to adjust the display timing.
[0093]
[Basic structure of video stream]
FIG. 16 shows the layer structure. Each hierarchy is called a class, and each class has a header. The header is various code information such as start code, end code, ID, shape and size.
[0094]
● Video stream
A video stream is composed of a plurality of sessions. A session is a series of complete sequences.
VS: Session consists of multiple objects
VO: Video object
VOL: Object is a sequence of objects including multiple layers
GOV: Object consists of multiple layers
VOP: Object layer consists of multiple planes
However, the plane is an object for each frame.
[0095]
[Error-resistant bitstream structure]
In MPEG4, the encoding method itself has resistance against transmission errors in order to cope with mobile communication (wireless communication) and the like. Error correction in the existing standard system is mainly performed on the system side, but the error rate is very high in networks such as PHS (Personal Handy phone System), and errors that cannot be corrected on the system side are in the video coding part. It is expected to leak. Considering this, MPEG4 assumes various error patterns that could not be corrected on the system side, and is an error-resistant coding method that suppresses error propagation as much as possible even under such an environment. Has been. Here, a specific technique of error resilience relating to image coding and a bit stream structure for that purpose will be described.
[0096]
● Reversible VLC (RVLC) and bidirectional decoding
As shown in FIG. 17, when it is confirmed that an error has occurred during decoding, the decoding process is temporarily stopped and the next synchronization signal is detected. At the stage where the next synchronization signal has been detected, the bit stream decoding process is performed in the reverse direction. The starting point of decoding increases without new additional information, and the amount of information that can be decoded when an error occurs can be increased as compared with the conventional case. “Bidirectional decoding” is realized by such a variable-length code that can be decoded from the forward direction as well as from the reverse direction.
[0097]
● Transmit important information multiple times
As shown in FIG. 18, a configuration capable of transmitting important information multiple times is introduced to enhance error tolerance. For example, in order to display each VOP at the correct timing, a time stamp is required, and this information is included in the first video packet. Even if this video packet is lost due to an error, decoding can be resumed from the next video packet by the bidirectional decoding structure. However, since this video packet has no time stamp, the display timing is not known after all. Become. For this reason, a structure has been introduced in which each video packet is flagged as HEC (Header Extension Code), and after this, important information such as a time stamp can be added. After the HEC flag, a time stamp and a VOP encoding mode type can be added.
[0098]
In the event of a loss of synchronization, decoding begins at the next synchronization recovery marker (RM), but each video packet contains the information required for it, the number of the first MB contained in the packet, and the quantum for that MB. The step size is placed immediately after RM. After that, the HEC flag is inserted. When HEC = '1', TR and VCT are added immediately after that. With these HEC information, even if the first video packet cannot be decoded and is discarded, decoding and display after the video packet set with HEC = '1' will be performed correctly. Whether or not HEC is set to “1” can be freely set on the encoding side.
[0099]
● Data partitioning
Since the encoding side repeats the encoding process in units of MB to form a bit stream, if an error is mixed in the middle, the subsequent MB data cannot be decoded. On the other hand, if multiple MB information is grouped into several groups, each is placed in the bitstream, and marker information is included at the boundary of each group, the bitstream will contain errors and the subsequent data Even if the data cannot be decoded, it is possible to re-synchronize with the marker at the end of the group and correctly decode the data of the next group.
[0100]
Based on the above idea, a data partitioning method (Data Partitioning) for grouping motion vectors and texture information (DCT coefficients, etc.) in units of video packets is employed. In addition, motion markers (MM:
Motion Marker) is placed.
[0101]
Even if an error is mixed in the motion vector information, the DCT coefficient that comes after the MM can be correctly decoded, so that MB data corresponding to the motion vector before the error mixing can be accurately reproduced together with the DCT coefficient. Even if an error is mixed in the texture portion, if the motion vector is correctly decoded, an image that is accurate to some extent can be interpolated and reproduced (concealed) using the motion vector information and the decoded previous frame information.
[0102]
● Variable-length interval synchronization method
Here, a synchronization recovery technique composed of variable-length packets will be described. An MB group including a synchronization signal at the head is called a “video packet”, and the number of MBs included in the MB group can be freely set on the encoding side. When an error is mixed in a bit stream using a variable length code (VLC), the subsequent codes cannot be synchronized and cannot be decoded. Even in such a case, subsequent information can be correctly decoded by detecting the next synchronization recovery marker.
[0103]
[Byte alignment]
In order to maintain consistency with the system, information is multiplexed in units of integer multiples of bytes. The bit stream has a byte alignment structure. To perform byte alignment, a stuff bit is inserted at the end of each video packet. Further, the stuff bit is also used as an error check code in the video packet.
[0104]
The stuff bit is composed of a code such as “01111” such that the first bit is “0” and all other bits are “1”. In other words, if the last MB in the video packet is correctly decoded, the next code is always' 0 ', and after that there should be a sequence of'1's that is one bit shorter than the stuff bit length. is there. If a pattern that violates this rule is detected, it means that the previous decoding has not been performed correctly, and it can be detected that an error has been mixed in the bitstream.
[0105]
As described above, "Overview of the international standard MPEG4 is determined" (Nikkei Electronics, September 22, 1997, p.147-168), "The whole picture of MPEG4 that has been visible" (Text of the Institute of Image Information and Television Engineers, 1997.10.2), The latest standardization trend and image compression technology (Nippon Industrial Technology Center seminar data 1997.2.3) etc. was referred and the MPEG4 technology was explained.
[0106]
[First Embodiment]
[Constitution]
Hereinafter, a television broadcast receiver according to an embodiment of the present invention will be described in detail with reference to the drawings. FIG. 19 is a block diagram showing a configuration example of the television broadcast receiving apparatus according to the embodiment of the present invention.
[0107]
The digital television broadcast signal is selected and received by the satellite antenna 21 and the tuner 23 for satellite broadcasting and by the tuner 24 via the cable 22 for cable broadcasting, depending on the broadcasting form. Thus, the television information received from the satellite broadcast or cable broadcast is selected by the data selector 43 as one data string, demodulated by the demodulation circuit 25, and error-corrected by the error correction circuit 26.
[0108]
Subsequently, the television information is separated into each data multiplexed by the multiplexed data separation circuit 27, that is, image data, sound data, and other system data (additional data). Of these, the sound data is decoded by the sound decoding circuit 28 to become stereo audio data A (L), A (R), and the sound control unit 30 adjusts the volume and sound field localization and the sound multiplexing such as main / sub sound. After the correspondence to is processed, a sound to be output is selected, converted into an analog signal by a digital-analog converter (D / A) 29, and reproduced by a speaker 31.
[0109]
On the other hand, the image data is decoded by an image decoding circuit 32 including a plurality of decoders that perform decoding processing corresponding to each object in the image data. This decoding method is object-based decoding based on the MPEG4 image decoding method already described. The decoded image data becomes images v (1) to v (i) corresponding to the number of objects, and various processes based on the display are performed by the display control unit 34.
[0110]
The display control performed by the display control unit 34 combines the objects into one output image, whether to display each object, enlargement / reduction of each object, where to display the object, and the like. Further, the display control also performs various display processes such as synthesis of the object and the character image (time display, index title, etc.) generated by the character generation circuit 40. These display controls are performed according to the control of the system controller 38 based on the arrangement information of each object, that is, the scene description information from the scene description data conversion circuit 39.
[0111]
The formed display image is converted into an analog form by the D / A 33 and displayed on the CRT 35, or sent to a liquid crystal display (LCD) 44 or the like as a digital signal and displayed.
[0112]
On the other hand, system data (including scene description data and additional data) is decoded by the system data decoding circuit 36. The category information added to the program is detected by the category information detection unit 37 from the decrypted system data. The detected category information is input to the system controller 38 and becomes a reference for command generation in layout setting. In addition, data related to the scene description is input to the scene description data conversion unit 39 from the decoded system data. Other system data (object information representing the contents of the object in commands) is input to the system controller 38 as various commands. The additional data may include a document such as a program title index.
[0113]
Object information is assigned to each object as a title by a command set commonly used by each television station. At the time of reception, by analyzing the object information, the contents of the object can be determined and classified. In the present embodiment, a layout setting function is realized that uses this object information to place an object having designated object information at a set position.
[0114]
Using the scene description data configured by the scene description data conversion unit 39, the arrangement and composition of each object in the display control unit 34, and the volume and sound field localization in the sound control unit 30 are set. Further, by adjusting the scene description data conversion unit 39 and controlling the display control unit 34 based on the control of the system controller 38, the objects are laid out in an arrangement different from the basic layout, that is, when the layout is set. Placement control can be performed. This layout setting method will be described later.
[0115]
When generating a display image that is not handled as an object, such as a time display screen or a title index, the character generation circuit 40 is used. Under the control of the system controller 38, a time display character is generated from the time data included in the additional data or the time information generated inside the receiver using a memory 42 such as a ROM storing character data. . The same applies to the title index. The image generated here is combined or the like in the display control unit 34.
[0116]
Further, the user can input various commands via the instruction input unit 45. The position adjustment in the layout setting process can be performed based on the user's instruction input. That is, correction of the layout position and input of a new set value are performed from the instruction input unit 45. In accordance with the instruction input value, the system controller 38 appropriately controls the operation of each unit so that a desired output (display, reproduction) form can be obtained.
[0117]
[Layout settings]
Layout setting for discriminating category information and placing objects in a predetermined manner for each category can be executed by two methods. The first method is a method of setting a layout using pre-programmed default settings using layout setting data stored in the memory 41 from the factory, and the second method is a layout arbitrarily set by the user. The layout setting data is stored in the memory 41 for each category and used.
[0118]
Here, a specific layout setting method will be described. FIG. 20 is a diagram for explaining a method for setting position data at the time of layout setting, and FIG. 21 is a diagram for explaining an image and an instruction input method for layout setting.
[0119]
There are two methods for positioning the object. The first method is a method of correcting (shifting) the position of the basic layout defined by the scene description data, and the second method is a method in which the user newly sets the position of the object at an arbitrary location. Both can be selected by a selector 302 shown in FIG. 20 in accordance with a user operation.
[0120]
First, the shift method which is the first method will be described. Image data is input as an object, and the basic position of the object is represented by position data (X0, Y0) specified by scene description data. When the user wants to shift the object, the correction amount (ΔX, ΔY) is added to the position data (X0, Y0) by the adder 301, and the new position data (X ′, Y ′) is added to the object layout setting data. Become. A new setting method, which is the second method, will be described. Regardless of the basic position data, a completely new position (X, Y) of the object is set, and this is used as position data (X ′, Y ′) instead of the basic position data. In this way, the layout setting position data set by the user is replaced with the basic layout object position data defined by the scene description data and displayed.
[0121]
This completes the description of the method for setting the layout of the designated image object. Object information for determining the target object is also necessary data as part of the layout setting data. The display process is controlled by the system controller 38, and control data at this time, object information for determining the target object, and layout setting data are stored in the memory 41 as user layout setting data corresponding to the category. .
[0122]
Next, FIG. 12 will be described. FIG. 12 illustrates the position setting method described so far. In the display device 303 such as a CRT, when the object 306 (position is (X0, Y0)) at the basic position to be operated is shifted to the shift position 307, the correction amount at that time is added to the basic position data, The final position data (layout setting data) is (X ′, Y ′) = (X0 + ΔX, Y0 + ΔY). If the user arbitrarily places an object at the newly set position 308, the position data (layout setting data) becomes (X ′, Y ′) = (X, Y). The setting method described in FIG. 11 is illustrated in this way.
[0123]
FIG. 21 shows a mouse 304 and a remote controller 305 as examples of pointing devices included in the instruction input unit 45. By using the mouse 304 or using the direction input keys of the remote controller 305 (which may be a cross key, joystick, joypad, etc.), it is possible to easily manipulate the movement of a free object. Note that the position where the object is moved and the position to be newly set can be selected from several preset positions such as the four corners and the center of the screen.
[0124]
Category information is included in the television broadcast data. By using this category information, the set layout can be converted into data in association with the category information for each program and stored as layout setting data. As the storage location, a nonvolatile memory 41 such as an EEPROM is used. When the category information stored in the memory 41 is detected from the television broadcast data, the system controller 38 controls the scene description data conversion unit 39 and the display control unit 34 based on the layout setting data corresponding to the category information. Then, display images and play sound with the set layout.
[0125]
Next, layout setting data will be described. The layout setting data includes default setting data programmed and held in advance and data set by the user. Basically, the user setting data is based on the object arrangement information obtained from the scene description data, and in addition to the object arrangement information, the position of the object when the user sets the layout is converted into data, and the control data of each part and What is necessary is just to memorize | store as layout setting data with the object information used as object. The scene description data has already been described with reference to FIG. 12, but the information for designating the time at which each object should be displayed and the position at which each object should be displayed is arranged in a tree shape. It is.
[0126]
As other layout setting data configurations, as shown in FIG. 22, on / off data indicating whether or not to display the object, display position data when the display position is expressed two-dimensionally on the X and Y axes In addition, by holding data indicating the size, it can be used as layout setting data of a target object.
[0127]
FIG. 23 is a diagram showing a configuration of a general MPEG4 bit stream. In the database of objects 1 to 5 in FIG. 23, program contents, natural image objects, sound objects, and objects such as CG (although the types of objects differ depending on the program) are incorporated. For example, in baseball broadcast broadcasting, a background object (sprite), a person, other natural image objects, a composite image object with a score display, a sound object, and the like are applicable. In addition, scene description information and additional data are multiplexed as system data in the bitstream. The additional data includes category information and object information.
[0128]
FIGS. 24 and 25 are diagrams showing examples of screen settings in a baseball relay broadcast, and FIGS. 26 and 27 are diagrams showing display examples of a baseball relay broadcast.
[0129]
In baseball broadcast broadcasting, there are a score display object 310 and a count display object 311 shown in FIGS. 24 and 26 as objects for setting a layout, and a batting rate display object 312 shown in FIGS. 25 and 27. The above three objects are indispensable in baseball broadcast broadcasting, but their display positions vary depending on the broadcasting station, so they are suitable objects for setting the layout according to this embodiment. These objects are composite image objects using CG or the like, but the type of the object is not limited in this embodiment.
[0130]
After shifting to the layout setting mode, the user may place an object at an arbitrary position on the television screen, that is, a preferred position or an easy-to-see position by the above-described method while viewing the screen.
[0131]
In this way, by using the layout setting function of the present embodiment, the timing of displaying the object at the default setting or the user-set position as in one scene of the baseball relay broadcast shown in FIGS. A score display object 310, a count display object 311 and a batting percentage display object 312 can be displayed for each (scene). This layout display is the same regardless of the broadcasting station.
[0132]
After the layout setting data once set is held, the layout setting function is activated by detecting the same category information, and it is determined whether or not the object is the target object from the object information. In the case of the target object, the layout is automatically displayed at the position based on the stored layout setting data at the display timing (scene). In addition, when the data structure of object information differs for every broadcasting station, you may make it reset object information.
[0133]
[Operation procedure]
28 and 29 are flowcharts for explaining an example of the operation procedure of the television broadcast receiving apparatus according to the present embodiment. FIG. 28 is a flow when the user sets a layout, and FIG.
[0134]
In the layout setting mode shown in FIG. 28, an object for setting a layout is selected from the objects constituting the image data in the television information (step S1). Then, the user places the selected (designated) object at an arbitrary position (step S2). When the arrangement of the selected object is completed, it is determined whether or not to end the layout setting (step S3), and when setting the layout for other objects, the process returns to step S1 to repeat the object selection and arrangement. When the layout setting is completed, the position of each object for which the layout is set is converted into data. Then, the category information, object information, position data, and control data of each part of the object are integrated and stored in the memory 41 as layout setting data (step S4).
[0135]
In the display mode shown in FIG. 29, television information is received (step S11), and program category information is detected from the system data added to the television information (step S12). The category information is information that adds information corresponding to the category (genre) of the program to the system data using a command set unified by each broadcasting station, and is used to divide the contents of the program roughly. It is. If the program category information is different for each broadcasting station, resetting means for ensuring consistency between broadcasting stations may be provided.
[0136]
Subsequently, it is determined whether layout setting data corresponding to the detected category information has already been saved (step S13). In the case of a category in which layout setting data is not stored, a television broadcast video is displayed with the basic layout sent from the broadcasting station (step S14).
[0137]
In the case of the category in which the layout setting data is stored, the layout setting data corresponding to the category information detected from the memory 41 is read (step S15), and the object information recorded in the layout setting data appears when the object information appears. It stands by so that control which changes the layout of an object can be performed. Therefore, in step S16, objects that are not targeted for layout setting are displayed in the basic layout, and the layout setting target object is displayed in the set layout at the display timing (scene) of the target object.
[0138]
The display state of step S14 or step S16 is maintained until the program ends or the reception of a new program is started after moving to another channel. When reception of a new program is started, the current layout setting is reset, and the flow is repeated from the initial state of television broadcast reception in step S11.
[0139]
In the present embodiment, “baseball broadcast” is taken up as a program category. However, the present invention is not limited to this, and the same can be applied to “soccer broadcast” and categories of programs other than sports.
[0140]
As described above, according to the present embodiment, a user who views a digital television broadcast can arbitrarily set an object layout in accordance with program category information. Therefore, video display according to the program category and the user's preference is possible, and the quality improvement of the visual and auditory user interface can be expected, and the TV program display with more flexibility for the user. Is possible.
[0141]
When layout setting is performed for each category information of a program with reference to object information indicating the contents of the object, layout setting for only the specified object can be performed by performing object classification and arrangement control.
[0142]
Moreover, even in the same program category, it is possible to eliminate the drawbacks that are displayed in different layouts depending on the broadcasting station, and it is possible to display a common object with a unified layout for each program category regardless of the broadcasting station. it can.
[0143]
Second Embodiment
Hereinafter, a television broadcast receiver according to a second embodiment of the present invention will be described. Note that in the present embodiment, the same reference numerals are given to substantially the same configurations as those in the first embodiment, and detailed description thereof will be omitted.
[0144]
In the second embodiment, an object layout setting in a television broadcast that uses and substitutes an image encoded by an encoding method other than MPEG4 as one MPEG4 object will be described.
[0145]
Here, an example in which MPEG2 is used as a natural image encoding method will be described. That is, an explanation will be given regarding a television broadcast receiving apparatus that transmits an image encoded with MPEG2 (hereinafter, may be referred to as “MPEG2 image”) multiplexed in an MPEG4 bitstream, received, and displayed. The layout setting method in the second embodiment is the same as that described in the first embodiment, and the basic configuration and operation of the television broadcast receiving apparatus are the same as those described with reference to FIG. However, since the details of the sound decoding circuit 28, the image decoding circuit 32, and the system data decoding circuit 36 shown in FIG. 19 are different in relation to the decoding method of the television broadcast in the second embodiment, these are shown in FIG. 30 and FIG. It explains using.
[0146]
FIG. 30 shows an encoding unit installed in a system for transmitting a television broadcast based on MPEG4 in a broadcasting station on the transmission side. The data multiplexer 5006 multiplexes the output from the encoders 5001 to 5005 of the sound, natural image, synthesized image, character, and scene description information objects described in FIG. The MPEG2 bit stream 61 extracted by the commercial broadcast equipment, relay system, or DVD (Digital Video Disc) playback is multiplexed into the MPEG4 bit stream.
[0147]
FIG. 31 is a configuration example of a decoding unit mounted on the side that decodes an MPEG4 bit stream, that is, a television broadcast receiving apparatus. 31 includes a sound decoding circuit 28, an image decoding circuit 32, a system data decoding circuit 36, and a scene description data conversion which are decoding systems and related circuits constituting the television broadcast receiving apparatus of the second embodiment. Part 39 is included.
[0148]
The received MPEG4 bit stream is separated into respective data by a data separator 5007 before decoding. Of the separated data, the MPEG4 object, which is a sound, a natural image, a composite image, characters, and scene description information, is decoded by decoding units 5008 to 5012 corresponding to the objects. The MPEG2 data multiplexed together with the MPEG4 object is decoded by a dedicated MPEG2 decoder 62 provided separately from the MPEG4 object decoder. The MPEG2 decoder 62 may be configured to use a part of the MPEG4 image decoding circuit 32.
[0149]
Information for displaying the video of the television program is composed of the sound, image and system description data that are decoded in this way, each object and MPEG2 data is synthesized into a form to be output to the television by the scene synthesis unit 5013, Output as scene information.
[0150]
Next, description will be made with reference to FIG. 26 when an MPEG4 television broadcast image including an MPEG2 image is displayed using the layout setting method described in the first embodiment. In the second embodiment, it is assumed that the relay image 401 including the background and players in the entire baseball relay image shown in FIG. 26 is an MPEG2 image. The other objects, the score display object 310, the count display object 311, and the other objects corresponding to the development are composed of MPEG4 data. In other words, this is a video display example of an MPEG4 television broadcast including an MPEG2 image. An example of an MPEG4 bit stream at this time is shown in FIG.
[0151]
In the MPEG4 bit stream shown in FIG. 32, baseball broadcast image 401 data, which is an MPEG2 data stream, is multiplexed as object 2. An MPEG2 data stream is generally composed of three types of data: audio, video, and system data (MPEG2 additional information). The object 2 is multiplexed with an MPEG2 data stream divided into segments of a predetermined amount in accordance with a predetermined timing adjustment related to transmission. In MPEG2 and MPEG4, there is an encoding / decoding circuit that can be shared at a lower level. Therefore, if necessary, the encoding / decoding circuit is shared and processing that avoids waste related to encoding / decoding is performed.
[0152]
As described above, the layout setting as described in the first embodiment is possible even for an MPEG4 system TV broadcast including an image and / or sound data encoded by the MPEG2 system.
[0153]
For the display image layout setting data, as in the first embodiment, based on the object arrangement information obtained from the scene description information, the position data of the object whose layout has been changed by the user is calculated, the program category information, The target object information and the control data of each part are stored as layout setting data in association with each other. The operation related to the display is the same as in the first embodiment.
[0154]
The second embodiment is an MPEG4 television broadcast in which MPEG2 images are multiplexed. Therefore, when combining with MPEG2 content, for example, an image relay system used for on-site relaying, the output of the MPEG2 device is complicated data. It can be diverted to the MPEG4 broadcasting system without conversion, and it is easy to handle because of the affinity between MPEG2 and MPEG4. Of course, the present invention can be used not only for relay images but also for multiple image output examples such as a material video display using a DVD, which is a typical MPEG2 video device, or other MPEG2 devices.
[0155]
In addition, since there are many encoding / decoding circuits that can be shared between MPEG2 and MPEG4, in addition to improving the efficiency of the system, the circuit configuration is effective without requiring a complicated configuration. Of course, even in the case of a software decoder, the efficiency of the system can be improved.
[0156]
According to the present embodiment, in addition to the effects of the first embodiment, the television information encoded with MPEG2 can be used for the MPEG4 television system, so that conventional contents can be used as they are, and MPEG2 is bothered. Since there is no need to convert the data to MPEG4, it is easy to handle and very effective.
[0157]
[Third Embodiment]
Hereinafter, a television broadcast receiver according to a third embodiment of the present invention will be described. Note that in the present embodiment, the same reference numerals are given to substantially the same configurations as those in the first embodiment, and detailed description thereof will be omitted.
[0158]
The layout setting target object of the present embodiment is not limited to the object included in the broadcast data, and may be an image generated in the receiving apparatus. As an example of an image generated in such a receiving apparatus, time information and titles composed of character information and the like, and CG and other graphics can be considered. In the present embodiment, the display positions of time information, titles, graphics, and the like are automatically changed in correspondence with program categories.
[0159]
For character images, for example, time information, the character generation unit uses time data that is included in the additional data of the MPEG4 bitstream and is sent as a basis for time display, or a clock signal in the television broadcast receiver. Generated at 40. Further, the character generating unit 40, when there is a command for performing time display by the clock in the receiving device in the additional data, or when a time display command is independently issued by the system controller 38, A time display image corresponding to each command is output using a clock. It is to be noted that the time generation image 103 is actually generated, that is, the character generation operation is played by the character generation unit 40 and the memory 42 in which the character data is stored, and the display control unit 34 performs image synthesis. The system controller 38 generates and displays the time display image 103 by controlling them.
[0160]
The same operation can be realized by using a time stamp included as one information in the subcode of the MPEG2 data stream. The image generation related to the title and graphics is the same, and a predetermined image is generated, combined, and displayed at an appropriate timing in accordance with a command included in the additional data. In the baseball broadcast broadcast taken up in the first embodiment, for example, instead of the score display object 310, additional information indicating the team name and score is extracted from the additional data, and an image representing the team name and score is generated. . Thus, if an image generated in the receiving apparatus is handled like an object included in broadcast data, a system that is easier to handle can be constructed.
[0161]
Regarding the layout setting of the character and image generated in the receiving apparatus of this embodiment, the layout position (object position) of the generated character and image is converted into data, the position data, program category information, the character and image ( Object) identification information and control data of each part including control of character and image generation operations are stored as layout setting data. The setting of the position data is the same as the method described with reference to FIGS. In addition, the character generation data read from the memory 42 and the character generation operation generated by the character generation unit 40 are controlled in correspondence with the command to generate an appropriate character.
[0162]
The generated character and image are combined with the received object and displayed as in the first embodiment. Also in this case, the composition and display method are controlled by the program category information.
[0163]
According to the present embodiment, in addition to the effects of the first embodiment, the layout setting function can be realized for characters and images generated in the receiving apparatus, so the load on communication (transmission) is reduced. It has the effect of reducing.
[0164]
In this way, digital TV broadcasting can be easily integrated with a personal computer (PC), and layout settings, etc. currently performed on the PC desktop can be customized for TV images. The compatibility between broadcasting and PC will be improved, and the effect of market expansion can be expected in the field of digital composite products.
[0165]
[Other Embodiments]
Note that the present invention can be applied to a system including a plurality of devices (for example, a host computer, an interface device, a reader, and a printer), and a device (for example, a copying machine and a facsimile device) including a single device. You may apply to.
[0166]
Another object of the present invention is to supply a storage medium storing software program codes for realizing the functions of the above-described embodiments to a system or apparatus, and the computer (or CPU or MPU) of the system or apparatus stores the storage medium. Needless to say, this can also be achieved by reading and executing the program code stored in the. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) running on the computer based on the instruction of the program code. It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.
[0167]
Further, after the program code read from the storage medium is written into a memory provided in a function expansion card inserted in the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. It goes without saying that the CPU or the like provided in the card or the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.
[0168]
【The invention's effect】
As described above, according to the present invention, an image (video) in digital television broadcasting is recorded. Reproduction type New functions can be provided.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration example of a digital broadcast receiver using satellite broadcasting;
FIG. 2 is a block diagram illustrating a configuration example in which a plurality of types of objects are simultaneously input and encoded;
FIG. 3 is a diagram showing a configuration example of a system taking user operation (editing) into consideration;
FIG. 4 is a block diagram on the encoder side of a VOP processing circuit related to a video object;
FIG. 5 is a block diagram on the decoder side of a VOP processing circuit related to a video object;
FIG. 6 is a block diagram showing the overall configuration of VOP encoding and decoding;
FIG. 7A is a diagram showing information constituting a VOP;
FIG. 7B is a diagram showing information constituting a VOP;
FIG. 8 is a diagram for explaining AC / DC predictive coding of texture coding;
FIG. 9A is a diagram for explaining a hierarchical structure of syntax for realizing scalability;
FIG. 9B is a diagram for explaining a hierarchical structure of syntax for realizing scalability;
FIG. 10A is a diagram for explaining a warp;
FIG. 10B is a diagram for explaining types of warp;
FIG. 11 is a diagram for explaining a warp;
FIG. 12 is a diagram showing a configuration example of scene description information;
FIG. 13 is a diagram showing the types of MPEG4 audio encoding methods;
FIG. 14 is a diagram showing the configuration of an audio encoding system;
FIG. 15 is a diagram for explaining the MPEG4 system structure;
FIG. 16 is a diagram for explaining the MPEG4 layer structure;
FIG. 17 is a diagram for explaining bidirectional decoding;
FIG. 18 is a diagram for explaining multiple transmissions of important information;
FIG. 19 is a block diagram showing a configuration example of a television broadcast receiver according to an embodiment of the present invention;
FIG. 20 is a diagram for explaining a method of setting position data when setting a layout;
FIG. 21 is a diagram for explaining an image and instruction input method for layout setting;
FIG. 22 is a diagram for explaining the configuration of layout setting data;
FIG. 23 is a diagram showing the structure of a general MPEG4 bit stream;
FIG. 24 is a diagram showing an example of a video display mode according to the present embodiment;
FIG. 25 is a diagram showing an example of a video display mode according to the present embodiment;
FIG. 26 is a diagram showing an example of a video display mode according to the present embodiment;
FIG. 27 is a diagram showing an example of a video display mode according to the present embodiment;
FIG. 28 is a flowchart for explaining an operation procedure example of the television broadcast receiving apparatus of the present embodiment;
FIG. 29 is a flowchart for explaining an operation procedure example of the television broadcast receiving apparatus of the embodiment;
FIG. 30 is a block diagram showing a configuration example of an encoding unit installed in a system for transmitting a television broadcast by MPEG4;
FIG. 31 is a block diagram illustrating a configuration example of a decoding unit mounted on a television broadcast receiving device;
FIG. 32 is a diagram illustrating an example of an MPEG4 bit stream including an MPEG2 image.

Claims (11)

Receiving means for receiving a digital data string of a television broadcast;
Decoding means for decoding image, sound and system data from the received digital data sequence;
Control means for controlling a reproduction mode of the decoded image and / or sound based on the decoded system data;
The control means reads out category information indicating the broadcast contents included in the system data , and setting data of a reproduction form corresponding to the image object constituting the decoded image from the storage means, and converts them into a broadcast program being received. A broadcast receiver characterized by being applied. When the category information included in the decoded system data corresponds to the reproduction form stored in the storage means, the control means applies the reproduction form setting data regardless of the broadcasting station. 2. The broadcast receiving apparatus according to claim 1, wherein The reproduction form of the configuration data broadcast according to claim 1 or claim 2, characterized in that stored in the storage means in association with the object information indicating the category information and the contents of the image object Receiver device. Further comprising a setting means for manually setting the layout of the image object,
The layout set by the setting unit is stored in the storage unit as setting data of the reproduction form together with the category information and object information indicating the content of the image object. The broadcast receiving apparatus described in 2. The control unit reads the category information and the reproduction mode setting data corresponding to the image object constituting the decoded image from the storage unit, and corresponds to the object information associated with the reproduction mode setting data. 5. The broadcast receiving apparatus according to claim 3, wherein the layout of an image object to be controlled is controlled. Receive digital data stream of TV broadcast,
Decoding image, sound and system data from the received digital data sequence;
Based on the decoded system data, each step of controlling the playback mode of the decoded image and / or sound,
The control of the playback mode is performed by reading out category information indicating the broadcast content included in the system data and playback mode setting data corresponding to the image object constituting the decoded image from the storage means and receiving the broadcast program The broadcast receiving method characterized by being applied to. When the category information included in the decoded system data corresponds to the playback mode stored in the storage unit, the playback mode control applies the playback mode setting data regardless of the broadcasting station. The broadcast receiving method according to claim 6, wherein: The reproduction form of the configuration data broadcast according to claim 6 or claim 7, characterized in that stored in the storage means in association with the object information indicating the category information and the contents of the image object Reception method. And a step of manually setting the layout of the image object,
8. The set layout is stored in setting data of the storage unit as information indicating the reproduction mode together with the category information and object information indicating the contents of the image object. Broadcast receiving method described in 1. The playback mode control, the category information and reads the setting data of the reproduction mode corresponding to the image objects constituting the decoded image, wherein the object information associated with the setting data of the reproduction mode from the storage means 10. The broadcast receiving method according to claim 8, wherein a layout of an image object corresponding to the image object is controlled. And controls the broadcast receiving apparatus, or computer-readable remembers medium characterized by computer program for executing a broadcast received described is recorded in one of claims 10 claim 6.

Images