JP2009077137A - Digital broadcast receiver and reception method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide images of high quality as much as possible when an error occurs in stream data in a weak hierarchy due to degradation of a reception state and the stream data is replaced with data in a middle hierarchy and a strong hierarchy. <P>SOLUTION: After a transmission stream in a plurality of hierarchies is separated into fundamental streams of images and sounds and is decoded, data is subdivided to perform error detection per macro block or slice in the case of images and per frame in the case of sounds, and data is successively replaced with image data and sound data in the middle hierarchy and the strong hierarchy when the error occurs in streams in the weak hierarchy, and images are reproduced by synthesized frames. When data is replaced with image data in the middle hierarchy and the strong hierarchy, super-resolution processing of generating image data of high resolution from a plurality of pieces of image data of low resolution is performed, and data of which the resolution is enhanced is used for synthesis of images. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a digital broadcast receiving apparatus and a receiving method for suitably receiving a digital broadcast transmitted by a hierarchical transmission method.

  In recent years, digitization of television broadcasting has been progressing. In Japan, digital broadcasting by a communication satellite was started in 1996, digital broadcasting by a broadcasting satellite in 2000, and terrestrial digital broadcasting in 2003. These are mainly realized by using the MPEG system and digital modulation technology, which are moving image compression coding technologies, and by digitalizing and transmitting information, high quality broadcasting and multi-channel are realized. . On the other hand, in digital broadcasting, when the radio wave reception state is below a predetermined level, the quality is extremely deteriorated compared to analog broadcasting. In particular, in terrestrial digital broadcasting, it is required to enhance resistance to multipath interference of radio waves and enable mobile reception, and this problem is important. Therefore, in digital terrestrial broadcasting, BST-OFDM (Band Segment Transmission Orthogonal Frequency Division Multiplexing) system is adopted as a transmission path encoding system, and by using different transmission parameters for each group of OFDM segments obtained by subdividing the transmission band, Multi-layer transmission with different degradation tolerance is performed.

  When the same broadcast content is transmitted in multiple hierarchies, a receiver that can receive multiple hierarchies can select and play hierarchies according to the reception state. When the reception state is good in all layers, a layer (referred to as layer A) from which high-quality video reproduction can be expected is selected. When the reception state deteriorates and normal reproduction cannot be performed in the A layer, the B layer that can expect high-quality video reproduction after the A layer is selected. Further, when normal reproduction cannot be performed in the B layer, the C layer of the next quality is selected. By adding such a function to the receiver, error tolerance can be enhanced.

  In Patent Document 1, when a transmission stream is divided into units equal to or less than a frame, for example, MB (macroblock), error detection is performed, and errors are detected in the A layer, the portions are sequentially converted into the B layer and the C layer. There has been disclosed a technique for generating a composite output frame that is supplemented with hierarchical video data and audio data, and displaying and outputting it.

  On the other hand, super-resolution processing is known as a technique for generating a high-resolution digital image from a plurality of low-resolution digital images. For example, in Patent Document 2, as super-resolution processing, sampling is performed using n sets of data obtained by sampling n times at the same sampling interval while changing the sampling position for the same signal. A technique for canceling the generated aliasing component and restoring the high-frequency component of the original signal at or above the Nyquist frequency is disclosed.

JP 2005-319701 A JP-A-8-336046

  In Patent Document 1, for example, a portion corresponding to an MB having an error in a weak hierarchy is supplemented with MBs in a middle hierarchy and a strong hierarchy to generate a composite output frame.

  In that case, there is a problem that the image of the replaced middle layer and the strong layer MB, particularly the strong layer having the lowest resolution, is deteriorated in image quality and difficult to view.

  In view of such a problem, the present invention, when reception conditions deteriorate, an error occurs in stream data in a weak layer, and when the stream data is replaced with a middle layer or a strong layer, a video with the highest possible quality is displayed. An object of the present invention is to provide a digital broadcast receiving apparatus and a receiving method that can be reproduced.

  In the digital broadcast receiving apparatus of the present invention, in the digital broadcast receiving apparatus that receives a digital broadcast of a multi-layer transmission system, a receiving unit that demodulates the received transmission stream, and a video stream and an audio stream for each layer from the demodulated transmission stream A separation unit for separating the video stream, a video decoding unit for decoding the video stream, an audio decoding unit for decoding the audio stream, and dividing the decoded video data into a size equal to or smaller than a frame, A video processing unit that selects the divided video data according to the priority of the video and generates a video signal, and divides the decoded audio data into a size equal to or smaller than the frame, and performs the division according to the priority of the designated hierarchy A high-resolution video data from a plurality of low-resolution video data A super-resolution processing unit that performs super-resolution processing on video data that could not be normally decoded by the video decoding unit, and the video processing unit performs the super-resolution processing. The video data subjected to the image processing is selected, and the audio processing unit selects the audio data corresponding to the next highest priority for the audio data that could not be normally decoded by the audio decoding unit.

  Here, the received multi-layer transmission stream includes a stream of a weak layer, a middle layer, and a strong layer in descending order of quality.

  Further, the division of the video data with a size equal to or smaller than the frame is, for example, division in units of macroblocks and division in units of status.

  More specifically, the digital broadcast receiving apparatus further includes a frame rate conversion unit that performs frame rate conversion, and the frame rate conversion unit performs frame processing on the video data subjected to the super-resolution processing position. Try to change the rate.

  More specifically, in the digital broadcast receiving apparatus, the frame rate conversion unit changes the frame rate using the position estimation information of the super-resolution processing.

  More specifically, the digital broadcast receiving apparatus includes a motion detection unit that performs motion detection on the decoded video data, and the super-resolution processing unit performs the video decoding according to a detection result of the motion detection unit. The video data that could not be normally decoded in the part is switched between whether or not super-resolution processing is performed.

  According to the present invention, when the reception status deteriorates, an error occurs in the stream data in the weak hierarchy, and when the stream data is replaced with the middle hierarchy or the strong hierarchy, a video with the highest possible quality can be reproduced. A digital broadcast receiving apparatus and a receiving method can be provided.

  Embodiments according to the present invention will be described below with reference to FIGS.

  Note that the digital broadcast receiving apparatus in the following description is intended for digital terrestrial television broadcasting as an example, but the present invention is not limited to this, and can be applied to a receiving apparatus that receives signals of a multi-layer transmission scheme.

[Outline of digital terrestrial television broadcasting]
Before describing the operation of the digital broadcast receiving apparatus according to the present embodiment, first, the format and outline of a transmission signal in the target terrestrial digital television broadcast will be briefly described with reference to FIGS.
FIG. 2 is a diagram illustrating a transmission spectrum of the BST-OFDM scheme in terrestrial digital broadcasting.
FIG. 3 is a diagram illustrating the characteristics of each layer.
FIG. 4 is a diagram illustrating an example of the configuration of an MPEG transport stream.

  As shown in FIG. 2, in the terrestrial digital television broadcast employed in this embodiment, the TV channel bandwidth of 6 MHz is divided into 13 OFDM segments, and frequency interleaving is performed with segment 0 as the center. These OFDM segments are divided into a maximum of three groups, and hierarchical transmission is performed. Each layer is composed of transmission parameters and one or more OFDM segments that are different for each layer, and is called a strong layer, a middle layer, and a weak layer in order from the lowest required CN ratio. When the number of hierarchies is 2, the names of strong hierarchies and weak hierarchies are used. There are three types of transmission parameters for fixed reception, mobile reception, and portable reception, and they are used in combination with each layer according to the layer transmission pattern.

  FIG. 3 shows a comparison of the relative characteristics between the layers for each parameter in the figure. In the strong hierarchy, the required CN ratio is the lowest (highest degradation tolerance), but since the transmission information rate is limited, both the resolution and the frame rate are set to the lowest. On the contrary, the weak hierarchy has the highest required CN ratio (low degradation resistance), but can set the highest resolution and frame rate. The middle layer has characteristics intermediate between the strong layer and the weak layer. With these hierarchical transmissions, it is possible to enhance degradation resistance and to receive at mobile terminals. In particular, for mobile terminals, only the central OFDM segment (segment 0) is used in the strong hierarchy. A partial reception service for transmitting a simple video of H.264 standard is prepared, and stable reception with a low required CN ratio is realized (so-called “one-segment broadcasting”).

  An MPEG transport stream shown in FIG. 4 is used as the transport stream of the terrestrial digital television broadcast of this embodiment. This MPEG transport stream is a transport stream defined by the MPEG2 system standard (ISO / IEC13818-1). The MPEG transmission stream is a system stream obtained by multiplexing a basic video stream and a basic audio stream, and is configured in a packet format so as to be suitable for transmission. As shown in FIG. 4, the packets 0, 1, 2, 4 and 5 of the MPEG transport stream include a basic stream constituting a video access unit. The packets 3 and 6 of the MPEG transport stream include a basic stream that constitutes an audio access unit. Here, the access unit is basic stream data necessary for decoding a video or audio frame. In the head packet of the access unit, display time information of a frame decoded from the access unit is also included.

Embodiment 1
Hereinafter, a first embodiment according to the present invention will be described with reference to FIGS. 1 and 5 to 9.
First, the configuration of the digital broadcast receiving apparatus according to the first embodiment of the present invention will be described with reference to FIG.
FIG. 1 is a block diagram showing a configuration of a digital broadcast receiving apparatus according to the first embodiment of the present invention.

  As shown in FIG. 1, the signal input unit of the digital broadcast receiving apparatus according to the present embodiment includes a TS (transport stream) receiving unit 101 and a TS separating unit 102. The video signal system includes three systems of decoders (MPEG-2 decoders 104 and 105, H.264 decoder 106), buffer memories 107 to 109, scalers 110 to 112, super-resolution processors 151 and 152, and an error detector 131. To 133, a video composition unit 113, a smoothing processing unit 114, and a video display unit 115. The audio signal system includes three systems of AAC decoders 116 to 118, buffer memories 119 to 121, error detection units 141 to 143, an audio switching unit 125, a smoothing processing unit 126, and an audio display unit 127. The video display unit 115 and the audio display unit 127 may be built in the receiving apparatus or connected to the outside.

Next, the operation of the digital broadcast receiving apparatus according to the first embodiment of the present invention will be described with reference to FIG. 1 and FIGS.
FIG. 5 is a diagram illustrating how the TS separation unit 102 separates the video and audio basic streams of each layer from the MPEG transport stream 80.
FIG. 6 is a diagram for explaining the super-resolution processing by the super-resolution processing units 151 and 152.
FIG. 7 is a diagram schematically illustrating selective synthesis performed by the video synthesis unit 113 and scaling performed by the scalers 110 to 112.
FIG. 8 is a flowchart showing video processing of the digital broadcast receiving apparatus according to the first embodiment of the present invention.
FIG. 9 is a flowchart showing audio processing of the digital broadcast receiving apparatus according to the first embodiment of the present invention.

  The digital broadcast receiving apparatus of this embodiment receives a digital television broadcast radio wave by the TS receiving unit 101 shown in FIG. 1, performs OFDM demodulation and error correction processing, and converts the MPEG transmission stream of each layer into a TS separating unit. To 102.

  The TS separation unit 102 separates the video and audio basic streams from the MPEG transmission stream of each layer obtained by the TS reception unit 101. Here, the basic stream of video includes MPEG-2 standard (ITU-T H.262 | ISO / IEC13818-2) in the weak and middle layers, and H.264 in the strong layer. The H.264 standard (ITU-T H.264 | ISO / IEC14496-10) shall be used. In addition, the MPEG-2 AAC standard (ISO / IEC13818-7) is used for each layer in the audio basic stream.

  FIG. 5 shows how a video stream and an audio stream of each layer are extracted from the transmission stream.

  For example, in the weak hierarchical video basic stream 81, packets 0, 5, 10, and 13 are extracted and used as access units. Also, in the weak hierarchical audio basic stream 82, packets 1, 7, and 11 are extracted and used as access units.

  The basic video streams of each layer are respectively MPEG-2 decoders 104 and 105 and H.264. H.264 decoder 106 outputs. The audio basic stream of each layer is output to the AAC decoders 116, 117, and 118, respectively. Note that the format of the video and audio basic streams is not limited to the above standard, and other standards may be used. In that case, a decoder corresponding to the standard may be used as the decoder of each layer.

  First, video signal processing will be described. MPEG-2 decoders 104, 105 and H.264. The H.264 decoder 106 decodes the weak, middle, and strong hierarchical video basic streams, and stores the decoded frames in the buffer memories 107 to 109. Here, it is assumed that the size and frequency of the decoded frame in each layer are different. Further, the error detection units 131 to 133 detect errors in the video basic streams of the weak, middle, and strong layers, respectively. If there is a part that cannot be decoded in the frame, the display time information of the frame and the position information of the part that cannot be decoded in the frame are output to the video composition unit 113.

  The middle layer and weak layer decoded frames stored in the buffer memories 108 and 109 are input to the super-resolution processing units 151 and 152, respectively, and are subjected to super-resolution processing.

  The super-resolution processing is a technique for generating high-resolution image data from a plurality of low-resolution image data as described in the background art.

  Hereinafter, the super-resolution processing will be briefly described with reference to FIG.

  First, as shown in FIG. 6A, a plurality of adjacent (here, three) original images 600 to 602 are prepared and aligned with decimal pixel accuracy. FIG. 6B shows a state where the other images 601 and 602 are aligned with the image 600 as the reference position. Positioning also takes into account movements such as rotation and enlargement / reduction. Here, the deviation of the sampling position of each image is estimated from a plurality of given original images themselves (position estimation). In this way, one composite image with the sampling position shifted from the integer pixel position is generated. Thereafter, as shown in FIG. 6C, the high-resolution image 603 is generated by re-sampling at the target resolution. When re-sampling, the effect of surrounding pixels is convolved by interpolating and increasing the number of pixels (sampling points) using a low-pass filter (high-band interpolation) or by applying an inverse function of the point spread function. Determine the pixel value. As a result, the high-frequency component of the original image can be restored and a high-resolution image with less blur can be generated.

  The weak layer frames are input as they are from the buffer memory 107 to the scaler 110, and the middle and strong layer frames are subjected to super-resolution processing by the super-resolution processing units 151 and 152, respectively, and then the scalers 111 and 112, respectively. And scaled to match the size and format of the output frame at each scaler. At this time, for example, a known scaling technique such as linear interpolation can be used.

  Based on the error information received from the error detection units 131 to 133, the video synthesis unit 113 selects and synthesizes the scaled frame data of each layer in units of frames, slices, or macroblocks. That is, the portion that could not be decoded is replaced with data of another layer. At that time, the priority order of the hierarchy to be selected is specified in the order of weak, medium and strong hierarchy.

  In the present embodiment, the received stream is separated into video and audio basic streams, which are decoded, and then the quality (error) is evaluated based on the result of the decoding process. Therefore, it is possible to subdivide the data size as a quality evaluation unit. As a result, as will be described later, for example, processing can be performed in units of macroblocks for video and in units of frames for audio.

  FIG. 7 shows how the scalers 110 to 112 each perform scaling, and the video composition unit 113 selectively combines them as a composite output frame. FIG. 9A shows a weak layer frame, and FIG. 9B shows a middle layer. Frame (c) shows a strong hierarchical frame, and (d) is a synthesized frame obtained by synthesizing these data.

  In the weak hierarchy output frame (a) having the highest priority, it is assumed that the macroblocks (MB) indicated by A1 and A2 cannot be decoded (error MB), and the other MB indicated by A0 is normal. This error MB (A1, A2) is replaced with the corresponding portions B1, B2 of the image (b) subjected to the super-resolution processing in the middle layer output frame. At that time, B1 is normal, but B2 cannot be used because it overlaps the MB in which the error occurred. Therefore, the corresponding portion C2 of the image (c) subjected to the super-resolution processing in the stronger hierarchical frame is replaced. At this time, since the frame size differs depending on the hierarchy, scaling processing (B1 → B1 ′, C2 → C2 ′) is performed to match the size and format of the composite frame (d).

  Further, when the data of each layer is combined in the frame, the smoothing processing unit 114 performs a smoothing process on the boundary portion. In the smoothing process, for example, a nonlinear filter having a low-pass characteristic such as a linear low-pass filter or an order statistical filter can be used. The video display unit 115 outputs the frames generated by the video synthesis unit 113 and the smoothing processing unit 114 according to the display time information.

  Next, audio signal processing will be described. The AAC decoders 116 to 118 decode the weak, middle, and strong hierarchical audio basic streams, respectively, and store the decoded frames in the buffer memories 119 to 121. In addition, when the error detection units 141 to 143 detect an error in the audio basic stream of each layer and there is a frame that cannot be decoded, the error detection units 141 to 143 output the display time information of the frame to the audio switching unit 125.

  The voice switching unit 125 selects and switches frame data of each layer on a frame basis based on the error information received from the error detection units 141 to 143. The smoothing processing unit 126 performs a smoothing process on the boundary portion when switching data of different layers between frames. The voice display unit 127 outputs the frames generated by the voice switching unit 125 and the smoothing processing unit 126 according to the display time information.

  Next, the video processing of the digital broadcast receiving apparatus according to the first embodiment of the present invention will be described step by step with reference to the flowchart of FIG.

  Here, a case where error detection is performed in units of macro blocks and video data is synthesized in units of macro blocks by the video synthesis unit 113 will be described.

  The video composition unit 113 includes MPEG-2 decoders 104 and 105 and H.264. Based on the display time information from the H.264 decoder 106, a frame to be displayed next is determined (S501). The frame to be displayed is a frame having the latest display time from the current time. Then, the following processing is sequentially performed for each macro block (MB) constituting the frame (S502).

  First, it is determined from the display time information from the MPEG-2 decoder 104 and the error position information from the error detection unit 131 whether or not weak layer data is valid in the current macroblock (S503). If the data is valid, scaling processing is performed by the scaler 110 (S504), and the data is selected as data of the macroblock (S505).

  If the weak layer data is not valid in S503, it is determined from the display time information from the MPEG-2 decoder 105 and the error position information from the error detection unit 132 whether the middle layer data in the macroblock is valid. (S506). If the data is valid, the super-resolution processing of the middle layer is performed on the data (S521), the data subjected to the super-resolution processing is scaled (S507), and the data of the macroblock Is selected (S508).

  If the data in the middle layer is not valid in S506, It is determined from the display time information from the H.264 decoder 106 and the error position information from the error detection unit 133 whether or not the data of the strong layer in the macroblock is valid (S509). If the data is valid, the data is subjected to super-resolution processing of a strong hierarchy (S522), the data subjected to the super-resolution processing is scaled (S510), and the data of the macroblock Is selected (S511).

  If the data of the strong hierarchy is not valid in S509, error concealment processing is performed (S512). In the error concealment process, for example, macroblock data at the same position in the previous frame is repeatedly used.

  When the data of the macroblock is selected, a smoothing process is performed on the boundary between adjacent macroblocks as necessary (S513). In the smoothing process, for example, a smoothing filter process is performed when the hierarchy of adjacent macroblocks is different from that of the macroblock.

It is determined whether or not the processing of the frame has been completed (S514). If the processing has not been completed, the process returns to S502 to process the next macroblock. In this way, the processing from S503 to S513 is repeated for each macroblock until all the data of the frame is selected. After all the data of the frame is determined, the frame is displayed according to the display time information (S515). When the processing of the frame is completed, the process returns to S501 and the next frame is processed.

  With the above operation, with respect to the video frame, it is possible to select high-quality video data by selecting error-free data in macroblock units in accordance with the priority specified in the order of the weak, middle, and strong hierarchies. Further, by performing the smoothing process on the boundary between macroblocks having data of different hierarchies, it is possible to prevent occurrence of block distortion in the frame.

  Next, the audio processing of the digital broadcast receiving apparatus according to the first embodiment of the present invention will be described step by step with reference to the flowchart of FIG.

  The audio switching unit 125 determines the next frame to be displayed based on the display time information from the AAC decoders 116, 117, and 118 (S601). The next frame to be displayed is a frame having the latest display time from the current time.

  First, it is determined from the display time information from the AAC decoder 116 and the error frame information from the error detection unit 141 whether or not weak layer data is valid in the current frame (S602). If the data is valid, the data is selected as the frame (S603).

  If the weak layer data is not valid in S602, it is determined from the display time information from the AAC decoder 117 and the error frame information from the error detection unit 142 whether the middle layer data in the frame is valid (S604). ). If the data is valid, the data is selected as the frame (S605).

  If the data in the middle layer is not valid in S604, it is determined from the display time information from the AAC decoder 118 and the error frame information from the error detection unit 143 whether or not the data in the strong layer in the frame is valid (S606). ). If the data is valid, the data is selected as the frame (S607).

  If the strong hierarchy data is not valid in S606, error concealment processing is performed (S608). For example, the error concealment process generates frame data that is smoothly connected and attenuated from the previous frame.

  After the data of the frame is selected, the frame boundary is smoothed as necessary (S609). In the smoothing process, for example, a smoothing filter process is performed when the hierarchy of the previous frame is different from the hierarchy of the frame.

  After all the data of the frame is determined, the frame is displayed according to the display time information (S610). When the processing of the frame is completed, the process returns to S601 and the next frame is processed.

  With the above operation, regarding the audio frame, it is possible to select high-quality audio data by selecting data with no error according to the priority specified in the order of the weak, middle and strong hierarchies. Further, by performing the smoothing process on the boundary between frames having data of different hierarchies, it is possible to prevent noise caused by data discontinuity.

  As described above, in the digital broadcast receiving apparatus and receiving method according to the present embodiment, video data and audio data are processed independently, and the highest unit that is not affected by transmission errors in each subdivided data unit. Since a quality layer is selected and playback is performed with super-resolution processing when necessary, quality degradation can be minimized.

  In the above embodiment, the quality of video data is determined in units of macroblocks, and the quality of audio data is determined in units of frames. However, the present invention is not limited to this. For example, for video data, the effect of the present invention can be expected even at a slice level and a frame level that are larger than a macroblock.

  In the above embodiment, the number of macroblocks determined to be invalid in the video signal processing is replaced with data of another layer. If the number of replaced macroblocks is large in one frame, discontinuity at the boundary portion is conspicuous, and the image quality as a whole deteriorates. In such a case, it is better to switch the data unit to be selected to a larger frame unit or slice unit. Therefore, it is effective to add a control for counting the number of invalid macroblocks in the frame and switching the unit to be selected from the macroblock unit to the frame or slice unit when the frequency exceeds the threshold.

  Furthermore, in the above-described embodiment, the three decoders, the buffer memory, and the scaler are provided corresponding to the weak, middle, and strong hierarchies. However, these have the same functions, and the common hardware is provided. You may make it process in a time division.

  In the above embodiment, the decoding of all three systems of the weak, middle and strong layers is executed in parallel regardless of whether the reception quality of each layer is good or not. When it is determined that the data in the weak layer is invalid, it may be switched to execute the decoding of the signal in the middle layer and further in the strong layer which are given priority next. Thereby, there exists an effect which can reduce power consumption.

[Embodiment 2]
Hereinafter, a second embodiment according to the present invention will be described with reference to FIGS. 10 and 11.
FIG. 10 is a block diagram showing a configuration of a digital broadcast receiving apparatus according to the second embodiment of the present invention.
FIG. 11 is a flowchart showing video processing of the digital broadcast receiving apparatus according to the second embodiment of the present invention.

  In the first embodiment, the super resolution processing is performed on the data of the MB in the middle hierarchy or the weak hierarchy for the MB having the error, and the synthesized frame is output.

  In this embodiment, in addition to the processing of the first embodiment, frame rate conversion is performed to smooth the output of video. In the description of the present embodiment, differences from the first embodiment will be mainly described.

  First, the configuration of a digital broadcast receiving apparatus according to the second embodiment of the present invention will be described using FIG.

  In the present embodiment, in addition to the configuration of the digital broadcast receiving apparatus of the first embodiment, FRC (Frame Rate Conversion) units 161 and 162 are added.

  The FRC unit 161 generates an interpolated frame for the frame that has been subjected to the super-resolution processing of the middle layer, performs frame rate conversion so as to match the frame rate of the weak layer, and the FRC unit 162 An interpolated frame is generated for a frame that has been subjected to hierarchical super-resolution processing, and frame rate conversion is performed so as to match the frame rate of the weak hierarchy.

  Next, the operation added to the first embodiment in the video processing of the digital broadcast receiving apparatus according to the second embodiment of the present invention will be described with reference to the flowchart of FIG.

  In the present embodiment, a middle-tier FRC process (S531) is added between the middle-tier super-resolution processing (S521) in FIG. 8 showing the processing of the first embodiment and the middle-tier scale (S507). Then, an interpolated frame is generated for the frame that has been subjected to the super-resolution processing of the middle layer, and frame rate conversion is performed so as to match the frame rate of the weak layer. In addition, a strong hierarchy FRC process (S532) is added between the strong hierarchy super-resolution process (S522) and the middle hierarchy scale (S510), and the frame subjected to the strong hierarchy super-resolution process is added. Thus, an interpolated frame is generated and frame rate conversion is performed so as to match the frame rate of the weak layer.

  Frame rate conversion matches the frame rate of weak layers with high frame rates, so it is necessary to generate interpolated frames for moving images, but it uses the information of the position estimation results in super-resolution processing. By generating the interpolation frame, there is an effect that the calculation for generating the interpolation frame in the FRC process (S531, S532) can be omitted, the calculation amount can be reduced, and the power consumption and the processing time can be suppressed. .

[Embodiment 3]
Hereinafter, a third embodiment according to the present invention will be described with reference to FIGS. 12 and 13.
FIG. 12 is a block diagram showing a configuration of a digital broadcast receiving apparatus according to the third embodiment of the present invention.
FIG. 13 is a flowchart showing video processing of the digital broadcast receiving apparatus according to the third embodiment of the present invention.

  In the first embodiment, the super resolution processing is performed on the data of the MB in the middle hierarchy or the weak hierarchy for the MB having the error, and the synthesized frame is output.

  In this embodiment, in addition to the processing of the first embodiment, motion detection of video frames is performed, and super-resolution processing is not performed for MBs that do not move. Also in the description of the present embodiment, parts different from those in the first embodiment will be described mainly.

  First, the configuration of a digital broadcast receiving apparatus according to the third embodiment of the present invention will be described with reference to FIG.

FIG.
In this embodiment, motion detection units 171 and 172 are added to the configuration of the digital broadcast receiving apparatus of the first embodiment.

  The motion detection unit 171 performs motion detection on the MB of the frame decoded by the MPEG-2 decoder 105 that processes the frame in the middle layer, and the super-resolution processing unit 151 applies to the MB of the frame having no motion. Is instructed not to perform super-resolution processing. In addition, the motion detection unit 172 processes the H.264 frame for processing a strong layer frame. The H.264 decoder 106 performs motion detection on the MB of the decoded frame, and instructs the super-resolution processing unit 152 not to perform the super-resolution processing on the MB of the frame having no motion.

  Next, the operation added to the first embodiment in the video processing of the digital broadcast receiving apparatus according to the third embodiment of the present invention will be described with reference to the flowchart of FIG.

  In the present embodiment, it is determined whether or not middle-layer data is valid (S506). If the data is valid, motion detection is performed on the MB of the frame (S542), and motion is detected. If so, middle-layer super-resolution processing is performed (S521). If no motion is detected, middle-layer super-resolution processing is skipped.

  Further, it is determined whether or not the data of the strong hierarchy is valid (S506). If the data is valid, motion detection is performed on the MB of the frame (S542), and when motion is detected. Then, the strong hierarchy super-resolution process is performed (S522), and when no motion is detected, the strong hierarchy super-resolution process is skipped.

  Video motion detection can be performed by examining the value of the motion vector of the frame and the value of the DCT transform coefficient of the reference frame for motion compensated prediction.

  This is because motion detection is performed and super-resolution processing is not performed even on video data with no motion, so unnecessary super-resolution processing is omitted, and calculation is thereby performed. This is because the amount is reduced and the power consumption and the processing time are not increased.

[Embodiment 4]
The fourth embodiment according to the present invention will be described below with reference to FIG.

  The digital broadcast receiving apparatus according to the first embodiment realizes the function with hardware that is blocked for each function.

  In this embodiment, the function of the digital broadcast receiving apparatus of the present invention is realized by software.

  FIG. 14 is a block diagram showing a configuration of a digital broadcast receiving apparatus according to the fourth embodiment of the present invention.

  As shown in FIG. 14, the digital broadcast receiving device 7 of the present embodiment is connected to a system bus 70 by a central processing unit 71, a main storage unit 72, and an input / output unit 73. The input / output unit 73 is connected to an auxiliary storage unit 74, a TS receiving unit 75, and a display unit 76. The display unit 76 may be connected to the outside of the digital broadcast receiving device 7. The auxiliary storage unit 74 stores the program and data for performing the processing of the central processing unit 71, the transmission stream of each layer received by the TS receiving unit 75, the video / audio basic stream, and the data of the decoded frame and the output frame. To do. The main storage unit 72 sequentially stores a program read from the auxiliary storage unit 74 and data being processed. The central processing unit 71 performs processing according to a program in the main storage unit 72.

  The digital television transmission stream received by the TS receiving unit 75 is stored in the main storage unit 72 by the processing of the central processing unit 71. The central processing unit 71 sequentially separates the basic video and audio streams of each layer from the transmission stream and stores them in the main storage unit 72. The central processing unit 71 sequentially decodes these video and audio basic streams, and stores the decoded frame, display time information, and error position information in the main storage unit 72. Further, the central processing unit 71 performs selection / combination / switching / smoothing processing of video and audio decoded frames, super-resolution processing, and the like based on the display time information and error position information. To display. The processing content is the same as the flowcharts of FIGS. 8 and 9 of the first embodiment. The central processing unit 71 sequentially performs these processes in a time-sharing manner.

  The central processing unit 71 may perform the FRC process performed in the second embodiment and the motion detection process performed in the third embodiment.

  With the above configuration, the same function as that of the first embodiment can be obtained also in the digital broadcast receiving apparatus of this embodiment.

[Other Embodiments and Features]
In each of the above embodiments, terrestrial digital television broadcasting has been described as an example of a transmission system of three layers of a strong layer, a middle layer, and a weak layer, but the present invention can be applied to a transmission method of an arbitrary plurality of layers. Also, the priority order of the hierarchy to be selected is the order of weak, medium, and strong hierarchy, but this can be arbitrarily changed and designated according to the purpose of viewing.

  According to the receiving device of each of the above embodiments, when receiving a digital broadcast that performs multi-layer transmission such as digital terrestrial television broadcasting, if the transmission stream includes an error due to a deterioration in the reception status, Even when the data is replaced with data of a strong hierarchy, it is possible to minimize degradation of the quality of the video and audio to be viewed. As a result, it is possible to improve the resistance to reception degradation, and it can be effectively applied to a mobile receiver or the like in which the reception environment is likely to change.

  In addition, the receiving device of each of the above embodiments can be applied not only to a television incorporating the same but also to a device (HDD recorder) having a recording unit that records a received signal on a recording medium such as a hard disk.

It is a block diagram which shows the structure of the digital broadcast receiver which concerns on 1st embodiment of this invention. It is a figure showing the transmission spectrum of the BST-OFDM system in terrestrial digital broadcasting. It is a figure which shows the characteristic of each hierarchy. It is a figure which shows an example of a structure of an MPEG transmission stream. 3 is a diagram illustrating a state in which a TS and a basic video stream are separated from an MPEG transport stream 80 by a TS separation unit 102. FIG. It is a figure explaining the super-resolution process by the super-resolution process parts 151 and 152. FIG. It is the figure which showed typically the selection composition which the image | video synthetic | combination part 113 performs, and the scaling which the scalers 110-112 perform. It is a flowchart which shows the video processing of the digital broadcast receiver which concerns on 1st embodiment of this invention. It is a flowchart which shows the audio | voice process of the digital broadcast receiver which concerns on 1st embodiment of this invention. It is a block diagram which shows the structure of the digital broadcast receiver which concerns on 2nd embodiment of this invention. It is a flowchart which shows the video processing of the digital broadcast receiver which concerns on 2nd embodiment of this invention. It is a block diagram which shows the structure of the digital broadcast receiver which concerns on 3rd embodiment of this invention. It is a flowchart which shows the video processing of the digital broadcast receiver which concerns on 3rd embodiment of this invention. It is a block diagram which shows the structure of the digital broadcast receiver which concerns on 4th embodiment of this invention.

Explanation of symbols

  101 ... TS receiver, 102 ... TS separator, 104,105 ... MPEG-2 decoder, 106 ... H. H.264 decoder, 107-109, 119-121 ... buffer memory, 110-112 ... scaler, 113 ... video composition unit, 114, 126 ... smoothing processing unit, 115 ... video display unit, 116-118 ... AAC decoder, 125 ... audio Switching unit, 127 ... Audio display unit, 131 to 133, 141 to 143 ... Error detection unit, 151 and 152 ... Super-resolution processing unit, 161 and 162 ... FRC processing unit, 171 and 172 ... Motion detection unit, 7 ... Digital Broadcast receiving device, 70 ... system bus, 71 ... central processing unit, 72 ... main storage, 73 ... input / output unit, 74 ... auxiliary storage, 75 ... TS receiver, 77 ... display.

Claims (14)

In a digital broadcast receiver that receives a digital broadcast of a multi-layer transmission system,
A receiver for demodulating the received transmission stream;
A separation unit for separating a video stream and an audio stream for each layer from the demodulated transmission stream;
A video decoding unit for decoding the video stream;
An audio decoder for decoding the audio stream;
A video processing unit that divides the decoded video data into a size equal to or smaller than a frame, selects the divided video data according to the priority of the specified hierarchy, and generates a video signal;
An audio processing unit that divides the decoded audio data into a size equal to or smaller than a frame, selects the divided audio data according to the priority of the designated hierarchy, and generates an audio signal;
A super-resolution processing unit that generates high-resolution video data from multiple layers of low-resolution video data,
The super-resolution processing unit performs super-resolution processing on low-resolution video data corresponding to video data that could not be normally decoded in the video decoding unit,
The video processing unit, for video data that could not be decoded normally in the video decoding unit, select the video data subjected to the super-resolution processing corresponding to the next priority layer,
The digital broadcast receiving apparatus, wherein the audio processing unit selects audio data corresponding to a layer having the next highest priority for audio data that could not be normally decoded by the audio decoding unit. The digital broadcast receiver according to claim 1,
The received transmission stream includes a stream of a weak layer, a middle layer, and a strong layer in descending order of quality,
The digital broadcast receiving apparatus, wherein the video processing unit and the audio processing unit preferentially select the video data and the audio data in the order of the quality. The digital broadcast receiver according to claim 1,
The digital broadcast receiving apparatus according to claim 1, wherein the video processing unit divides the video data into units of macroblocks and selects from them. The digital broadcast receiver according to claim 1,
The digital broadcast receiving apparatus according to claim 1, wherein the video processing unit divides the video data into slice units and selects from the slice units. The digital broadcast receiver according to claim 1,
Furthermore, a frame rate conversion unit that performs frame rate conversion is provided,
The digital broadcast receiving apparatus, wherein the frame rate conversion unit performs frame rate conversion on the video data subjected to the super-resolution processing. The digital broadcast receiver according to claim 5, wherein
The digital broadcast receiving apparatus, wherein the frame rate conversion unit performs frame rate conversion using position estimation information of the super-resolution processing. The digital broadcast receiver according to claim 1,
A motion detection unit that performs motion detection on the decoded video data,
According to the detection result of the motion detection unit, whether or not the super-resolution processing unit performs super-resolution processing on low-resolution data corresponding to video data that cannot be normally decoded by the video decoding unit. A digital broadcast receiver characterized by switching. In a digital broadcast receiving method for receiving a digital broadcast of a multi-layer transmission system,
Demodulating the received transmission stream;
Separating a video stream and an audio stream for each layer from the demodulated transmission stream;
Decoding the video stream;
Decoding the audio stream;
A step of performing super-resolution processing for generating high-resolution video data from low-resolution video data of a plurality of hierarchies for video data that could not be normally decoded in the step of decoding the video stream;
The decoded video data is divided into a size equal to or smaller than a frame, and the video data that has not been successfully decoded in the step of decoding the video stream according to the designated hierarchical priority, Selecting video data that has been subjected to the corresponding super-resolution processing of generating a video signal;
The decoded audio data is divided into a size equal to or smaller than the frame, and the audio data that cannot be normally decoded in the step of decoding the audio stream in accordance with the priority of the specified hierarchy, the next priority hierarchy And selecting the corresponding audio data. A digital broadcast receiving method comprising: The digital broadcast receiving method according to claim 8,
The received transmission stream includes a stream of a weak layer, a middle layer, and a strong layer in descending order of quality,
A digital broadcast receiving method, wherein the video data and the audio data are preferentially selected in the order of quality. The digital broadcast receiving method according to claim 8,
A digital broadcast receiving method, characterized in that the video data is divided into macroblock units and selected from these. The digital broadcast receiving method according to claim 8,
A digital broadcast receiving method, wherein the video data is divided into slices and selected from the slices. The digital broadcast receiving method according to claim 8,
The digital broadcast receiving method further comprising a step of performing frame rate conversion on the video data subjected to the super-resolution processing. A digital broadcast receiving method according to claim 12,
The step of performing the frame rate conversion comprises performing frame rate conversion using position estimation information of the super-resolution processing. The digital broadcast receiving method according to claim 8,
Furthermore, it has a step of performing motion detection on the decoded video data,
According to the detection result of the step of performing motion detection, switching of whether or not to perform super-resolution processing on low-resolution data corresponding to video data that could not be normally decoded by the video decoding unit Digital broadcast receiving method.

Images