JP2005123990A - Portable communication terminal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a portable information terminal which decodes transmitted video data streams coded according to the MPEG-2 with a low power at a low cost utilizing ground wave digital television broadcasts, without using the complete decoding in the MPEG-2. <P>SOLUTION: The terminal previously separates codes concerning with I-pictures, P-pictures and B-pictures from a video stream separated from an MPEG-2 transport stream according to picture coding type codes. A video data stream discarder (6) discards the P-picture and B-picture information or the B-picture information only, then discards discrete cosine transform coefficients at the high frequency side from the I-picture and the P-picture information according to the display resolution of a display (32). For the P-pictures, the terminal executes an interpolating process of moving vectors in the video stream according to the display resolution, and decodes only information required for the display on a mobile apparatus. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a technique for decoding video data included in a broadcast signal for terrestrial digital television broadcasting.

  MPEG (Moving Picture Experts Group) -2 has been standardized as a television image encoding method. MPEG-2 uses a P picture (forward interframe predictive coding information) that is interframe-coded using forward prediction and a bi-directional prediction based on an intra-coded I picture (intraframe encoded information) as a reference. It is standardized as ISO-13818 that a television image is compressed and transmitted using an inter-coded B picture (bidirectional interframe predictive coding information).

  Since a television image is encoded and transmitted at 30 frames / second or 60 fields / second per second, the decoder installed in the receiver must create and display an image at that speed, It is necessary to have a high processing capacity. In this regard, as a technique that can cope with a relatively low processing capability, Patent Document 1 and Patent Document 2 describe a processing load of a thinning decoding process on a B picture or a P picture and a B picture from a transmitted encoded frame. Techniques for reducing are described. Patent Document 3 describes a technique for performing parallel display of a sub-screen of another channel, which has been thinned by decoding the decoded data, to reduce the display size, on the main screen of another channel. Further, Japanese Patent Application Laid-Open No. 2004-228620 describes that a scalar quantity of a motion vector is converted with respect to image data whose horizontal and vertical directions are compressed.

JP 2000-299867 A

JP 2002-218472 A JP2000-32475A JP-A-11-262011

  In recent years, service forms in mobile devices typified by mobile phones have made remarkable progress. In addition, terrestrial digital television broadcasting has begun to be tested and has become more realistic. The present inventor has studied the reception of terrestrial digital television broadcasts on mobile devices. According to this, the display screen is much smaller than the display screen of the television set due to the nature of the mobile device, and the power consumption is reduced due to the nature of the battery drive. Since the prior art is not focused on receiving terrestrial digital television broadcasts on mobile devices, such restrictions are not considered at all. The present inventor has found that it is necessary to consider not only the temporal resolution but also the spatial resolution (display resolution) in consideration of the display resolution on the mobile device.

  An object of the present invention is to provide a portable communication terminal apparatus capable of displaying an image by reducing both time resolution and spatial resolution for a broadcast signal by terrestrial digital television broadcasting.

  Another object of the present invention is to decode at low cost and low power without completely decoding all MPEG-2 encoded video data streams transmitted using terrestrial digital television broadcasting. An object of the present invention is to provide a portable communication terminal device that can be used.

  The above and other objects and novel features of the present invention will be apparent from the description of this specification and the accompanying drawings.

  The following is a brief description of an outline of typical inventions disclosed in the present application.

  That is, the mobile communication terminal device according to the present invention can receive a broadcast signal by terrestrial digital television broadcasting, and has a data processing unit and a display unit. The data processing unit extracts an MPEG-2 first video data stream from the received broadcast signal, and encodes intra-frame coding information and forward inter-frame prediction coding information included in the extracted first video data stream. , And discarding at least the bidirectional inter-frame prediction coding information among the bidirectional inter-frame prediction coding information, leaving the intra-frame coding information, and generating a second video data stream related to the remaining coding information, Of the encoded information of the second video data stream, only a part of information corresponding to the display resolution of the display unit is decoded. From the above, it is possible to reduce temporal resolution by discarding at least bidirectional inter-frame predictive coding information for the first video data stream. The spatial resolution can be reduced by decoding only part of the information corresponding to the display resolution of the display unit with respect to the encoded information of the second video data stream. The encoded information is, for example, an orthogonal transform coefficient such as a discrete cosine transform coefficient that constitutes intra-frame encoded information called an I picture.

  In the case of adding a reception function of a broadcast signal by terrestrial digital television broadcasting to a mobile communication terminal device that originally has a decoding function for an MPEG-4 video data stream, the data processing unit uses the received broadcast signal from the received broadcast signal. The first video data stream of MPEG-2 is extracted, and the intra-frame encoding information, the forward inter-frame prediction encoding information, and the bidirectional inter-frame prediction encoding information included in the extracted first video data stream are extracted. , Discarding at least bidirectional inter-frame predictive coding information, leaving intra-frame coding information, generating a second video data stream relating to the remaining coding information, and among the coding information of the second video data stream The third video data that conforms to MPEG-4 with some information corresponding to the display resolution of the display unit Into a stream, it may be configured to decode the third video data stream which has been converted.

  As a specific mode of the present invention, the data processing unit may include only a block-unit DC (direct current) component or a block-unit DC for intra-frame coding information included in the second video data stream. A component and a part of AC (alternating current) component are used as the part of information. The DC component means an average value of encoding information such as a discrete cosine transform coefficient of the block, or a difference between the average value of the block and the average value of the previous block. The AC component means a frequency component in units of pixels subjected to orthogonal transformation. Some of the AC components to be left are low frequency components with relatively low spatial frequencies in the horizontal and vertical directions. This is because orthogonal transformation has a property that relatively large values concentrate on low frequency components.

  As a specific form of the present invention, the data processing unit corresponds to a size of a motion vector corresponding to the display resolution for the forward interframe predictive coding information included in the second video data stream. Let me change it.

  In consideration of interlace scanning, the encoded information included in the second video data stream may be information corresponding to either an odd frame or an even frame in the interlace scan.

  The received broadcast signal constitutes an MPEG-2 transport stream, for example. It does not prevent you from being a program stream. In the case of the MPEG-2 transport stream, the data processing unit first divides the stream into a first video data stream and an audio data stream. When recording a television image, the data processing unit multiplexes the second video data stream and the audio data stream, and stores the multiplexed information in a memory as a transport stream.

  The mobile communication terminal apparatus according to the present invention may have a high frequency unit and a baseband unit for performing mobile communication. In short, a mobile phone is added with a broadcast signal receiving function by terrestrial digital television broadcasting.

  In another aspect of the present invention, the data processing unit extracts a first video data stream from the received broadcast signal, and encodes intra-frame coding information included in the extracted first video data stream, Among the forward inter-frame predictive coding information and the bi-directional inter-frame predictive coding information, at least the bi-directional inter-frame predictive coding information is discarded, the intra-frame coded information is left, and the second coding related to the remaining coded information is stored. A video data stream is generated, and a part of information corresponding to the display resolution of the display unit among the encoded information of the second video data stream is set as an image display target.

  The effects obtained by the representative ones of the inventions disclosed in the present application will be briefly described as follows.

  That is, according to the present invention, the video data stream according to MPEG-2 is decoded with the resolution in the time direction and the resolution in the spatial direction (display resolution) being reduced, so that the performance of the decoder can be lowered. As a result, MPEG-2 decoding is possible even in fields where power consumption is severe, such as mobile terminals. Therefore, it is possible to decode the video data stream encoded by MPEG-2 transmitted using terrestrial digital television broadcasting at a low cost and with low power without completely decoding.

  Further, by using an MPEG-4 decoder installed in a mobile terminal, a decoding process for MPEG-2 video data without installing an expensive MPEG-2 decoder (MPEG-2 decoder). Can be performed.

  FIG. 10 shows an example of a portable communication terminal device according to the present invention. The mobile communication terminal device PDA shown in the figure includes a mobile communication function such as a mobile phone together with a reception / playback function for terrestrial digital television broadcasting. For example, a reception / playback function for terrestrial digital television broadcasting is added to a mobile phone.

  The mobile communication function includes a high-frequency unit 34 that transmits high-frequency signals via the antenna 1 and receives high-frequency signals from the antenna 1, a baseband unit 35 that performs baseband processing of transmission / reception signals, and data to be transmitted. This is realized by an application processor unit 36 that controls editing, display of received data, key input, call voice processing, a microphone 37, a speaker 23, and a display unit 32.

  The reception / reproduction function of the terrestrial digital television broadcast includes a tuner 2 that captures a broadcast signal of the terrestrial digital television broadcast from the antenna 1 and performs entropy decoding such as Huffman decoding processing on the captured broadcast signal, for example, MPREG- 2 demodulator 3 for transport stream, demultiplexer 4 for separating MPEG-2 transport stream into video data stream and audio data stream for one channel, and image for image reproduction by inputting video data stream An image data processing unit 30 that performs processing such as decoding, an audio data processing unit 31 that performs processing such as audio decoding for audio reproduction by inputting an audio data stream, a display unit 32 that displays images, and outputs audio This is realized by the speaker 23.

  FIG. 1 shows an example of a functional module that realizes a reception / playback function for terrestrial digital television broadcasting. The signal captured from the antenna 1 is converted into an MPEG-2 transport stream by the tuner 2 and the demodulator 3 and input to the demultiplexer 4. The demultiplexing unit 4 separates the transport stream into a video data stream and an audio data stream for one channel. The video data stream is supplied to the preceding video stream analysis unit 5, and the audio data stream is supplied to the decoding processing unit 15. The preceding video stream analysis unit 5 operates to detect a start code of the video data stream and to find a picture coding type code. The picture coding types are B picture, I picture and P picture. The video data stream is input to the video stream discarding unit 6 as it is, and operates so as to discard, for example, information related to the B picture based on the information from the preceding video stream analyzing unit 5. In the video stream output from the video stream discarding unit 6, the information code regarding the B picture is discarded and input to the decoding processing unit 7. The decoding processing unit 7 decodes information related to the I picture and P picture, and supplies the decoded information to the DCT coefficient reduction unit 8 and the vector interpolation unit 13. The DCT coefficient reduction unit 8 determines a DCT coefficient to be reduced according to the resolution of the display device 24 and outputs it to the inverse quantization unit 9. The inverse quantization unit 9 inversely quantizes the reduced DCT coefficients according to the quantization parameter of the decoded information and outputs the result to the inverse DCT unit 10. The motion vector information output from the decoding processing unit 7 is vector-interpolated by the vector interpolation unit 13 according to the resolution of the display device 24 and output to the motion compensation unit 14. In the case of a P picture, the motion compensation unit 14 reads a reference image from the image memory 12 based on the interpolated vector information, and the read reference image is added by the adder 11 with the output from the inverse DCT unit 10. Are output to the image memory 12. In the case of an I picture, the output from the motion detection unit 14 is 0, and the output from the inverse DCT unit 10 is stored in the image memory 12 through the adder 11 as it is. The image data stored in the image memory 12 is displayed on the display device 24 at a display timing by the display control unit 20. The display device 24 is, for example, a liquid crystal display.

  On the other hand, the audio data stream separated by the demultiplexing unit 4 is decoded by the decoding processing unit 15, and the inverse quantization unit 16, the spectrum processing unit 17, the IDCT (Inverse Discrete Cosine Transform) unit 18, and the subband synthesis unit 19. Through the speaker 23. The circuit portion of the audio processing is realized using a technique such as MPEG-2 AAC (Advanced Audio Coding).

  The video data stream output from the video stream discarding unit 6 is again multiplexed with the audio data stream by the multiplexing / demultiplexing unit 21 and stored in the storage medium 22 such as a memory as an MPEG-2 transport stream. You can also The transport stream stored in the storage medium 22 is demultiplexed by the multiplexing / demultiplexing unit 21, the video data stream is input to the decoding processing unit 7, and the audio data stream is input to the decoding processing unit 15. Can be subjected to the decoding process described above.

  FIG. 2 shows an example of discard processing by the video stream discard unit 6. The image frames of the video data stream are I picture, P picture, and B picture. The video stream discarding unit 6 outputs information relating to the I picture and P picture as a video data stream, and discards the encoded data relating to the B picture. Of course, if the video stream discarding unit 6 also discards information related to the P picture, it goes without saying that only the encoded data related to the I picture is output. The P picture is screen information encoded only from the information, and is generated without using inter-frame prediction. P picture is screen information generated by performing prediction from I picture or P picture. A B picture is inserted between an I picture and a P picture, and is based on forward inter-frame prediction predicted from a backward (past) reproduced image, reverse inter-frame prediction predicted from the front (future), or internal prediction. Generated by inter-frame prediction. Since the B picture is not used for the next prediction, it is discarded. Even if it is discarded as compared with the I and P pictures, the degradation of image quality is slight. The I and P pictures are images used for the next prediction. However, since the picture quality degradation due to discard is expected to be lower in the P picture than in the I picture, the I picture is not discarded. The discarding process by the video stream discarding unit 6 reduces the time resolution.

  FIG. 3 illustrates the data structure of an encoded I picture. As shown in FIG. 3, the I-picture code string includes picture layer information and slice layer information. The slice layer information includes macro block information. The macro block information includes block information. One block information has 8 × 8 pixel information. In FIG. 3, the data structure is partially simplified, but details thereof can be referred to in a known MPEG-2 standard or the like.

  FIG. 4 shows a specific example of processing for reducing the spatial resolution for the I picture output from the video stream discarding unit 6. The block information in FIG. 3 is decoded into 8 × 8 DCT coefficient information by the decoding processing unit 7. In the 8 × 8 DCT coefficient information, the horizontal axis indicates the horizontal spatial frequency, the vertical axis indicates the vertical spatial frequency, # 00 is a DC (direct current) component as an average value of DCT coefficients, and # 01 to # 77 are AC (alternating current) component. The DCT coefficient of the AC component is the order of # 01, # 10, # 20, # 11, # 02, # 03, # 12, # 21, # 30, # 40, # 31, # 22,. It becomes smaller in order in the oblique scan direction. In the DCT coefficient information, the DCT coefficient reduction unit 8 reduces a DCT coefficient having a small value, that is, a component on the high frequency side, according to the resolution of the display device 24. This is because orthogonal transforms such as discrete cosine transform have the property that relatively large values concentrate on low frequency components.

  The example in the figure is an example when a display device 24 of horizontal 1/2 and vertical 1/2 of a video stream is assumed. When decoding with the same resolution without reducing the DCT coefficients, the image 100 having the size surrounded by the dotted line in the image memory 12 in the figure is decoded. When the DCT coefficient is reduced to ½ in both horizontal and vertical directions, and the inverse DCT operation is performed by the inverse DCT unit 10 based on the reduced DCT coefficient, 4 × 4 pixel data is output as pixel data. . Since the 4 × 4 pixel data is included in the I picture here, the output from the motion compensation unit 14 is 0, and the pixel data is supplied to the image memory 12 through the adder 11. The image size generated in the image memory 12 is 101. The size indicated by 100 represents the image size of the I picture when the spatial resolution is not lowered.

  FIG. 5 illustrates the data structure of the encoded P picture. In the code sequence of the P picture, there are picture layer information and slice layer information as shown in the figure. The slice layer information includes macro block information. The macro block information includes block information. Here, the macroblock information includes a motion vector used for inter-frame prediction. Although the data structure is partially simplified in FIG. 5, details thereof can be referred to in the known MPEG-2 standard.

  FIG. 6 shows a specific example of processing for reducing the spatial resolution for the P picture output from the video stream discarding unit 6. In the case of a P picture, a motion vector is output for each macroblock. The motion vector information is decoded by the decoding processing unit 7 to be vector information MV (X, Y). The vector interpolation unit 13 performs vector interpolation based on the resolution of the display device 24. In the example in the figure, it is shown that the horizontal and vertical 1/2 display devices are taken into consideration, and the vector is also set to an interpolation vector by halving the X component and the Y component, respectively. Based on the information of the interpolation vector, the motion detection unit 14 reads the image data from 101 in the image memory 24, adds the read image data and the pixel data described in FIG. 4 with the adder 11, and stores it in the image memory 24. Output as an image 201. Images 100 and 200 indicated by dotted lines in the image memory 24 indicate the image size when the video stream is decoded as it is.

  FIG. 7 shows an example of operation when DCT coefficient reduction is performed on the discarded bit stream in time series. From the reduced video data stream, DCT coefficients are reduced according to the display image resolution. Here, in conformity with the description of FIG. 4, the DCT coefficient is reduced to ¼. The display image frame is ¼ the original size.

  FIG. 8 shows an example of operation related to time resolution and spatial resolution reduction processing in consideration of interlaced scanning in time series. In this case, in order to reduce the temporal resolution, the B picture or the P picture is left with the I picture remaining for the information corresponding to either the odd frame (top field) or the even frame (bottom field) in the interlace scanning. Discard, discard all other odd or even frames. For example, in FIG. 8, information related to a top field of an I picture and a top field of a P picture is passed from the bitstream discard unit 6 and other information is discarded. In this case, if the display device is 1/4 of the decoded image size, the vertical direction has already been halved by dropping the field, so only the horizontal direction is halved. Thus, the reduction of the DCT coefficient may be executed. The display timing is as shown by the image display frame item in the figure.

  FIG. 9 shows an example of a communication portable terminal device when MPEG-4 is used like a mobile terminal such as a cellular phone. That is, the main part of the configuration in the case where means for decoding an MPEG-4 video data stream is already provided will be described. As described above, the video data stream separated by the multiplexing / separating unit 4 includes the preceding video stream analyzing unit 5, the video stream discarding unit 6, the decoding processing unit 7, the DCT coefficient reducing unit 8, and the vector interpolating unit 13. In this way, reduced DCT coefficients and interpolated motion vector information are obtained. The MPEG-4 bit stream creation unit 40 converts information necessary for MPEG-4 among the information decoded by the decoding processing unit 7, for example, information on the image size, the reduced DCT coefficient and the interpolation motion vector into the MPEG- 4 is encoded according to the code 4 to generate a video data stream compliant with MPEG-4. The generated MPEG-4 compliant video data stream is decoded by the MPEG-4 decoder 41 already provided to realize the functions necessary for the mobile terminal, and is transmitted via the display control unit 20 and the display device 24. Displayed.

  According to the mobile communication terminal PDA described above, the MPEG-2 video data stream is decoded with the temporal resolution and the spatial resolution (display resolution) reduced, so that the burden of decoding processing can be reduced. Become. As a result, MPEG-2 decoding is possible even in fields where power consumption is severe, such as mobile terminals. Therefore, an MPEG-2 encoded video data stream transmitted using terrestrial digital television broadcasting can be decoded inexpensively and with low power without performing MPEG-2 complete decoding.

  Further, by using an MPEG-4 decoder installed in a mobile terminal, a decoding process for MPEG-2 video data without installing an expensive MPEG-2 decoder (MPEG-2 decoder). Can be performed.

  Furthermore, since the transport stream in which the resolution in the time direction and the resolution in the spatial direction are reduced can be recorded in a memory or the like, the processing load when the same image is reproduced next time can be reduced.

  Although the invention made by the present inventor has been specifically described based on the embodiments, it is needless to say that the present invention is not limited thereto and can be variously modified without departing from the gist thereof. For example, when the DCT coefficient is reduced, all AC components may be reduced in relation to the display size, and only the DC component may remain. The present invention is not limited to a mobile phone and can be applied to various mobile terminals.

It is a block diagram which illustrates the structure for implement | achieving the reception reproduction | regeneration function of terrestrial digital television broadcasting. It is a timing chart which shows an example of the discard process by a video stream discard part. It is explanatory drawing which illustrates the data structure of I picture. It is a flowchart which shows the specific example of the process which reduces spatial resolution with respect to the I picture output from a video stream discard part. It is explanatory drawing which illustrates the data structure of P picture. It is a flowchart which shows the specific example of the process which reduces a spatial resolution with respect to P picture output from a video stream discard part. It is a timing chart which shows the operation example when performing DCT coefficient reduction with respect to the bit stream after discard. It is a timing chart which shows the operation example regarding the reduction process of the time resolution when considering interlace scanning, and spatial resolution. FIG. 2 is a block diagram illustrating a configuration of a main part for realizing a reception / playback function of digital terrestrial television broadcasting in a communication portable terminal device when MPEG-4 is used like a mobile terminal such as a cellular phone. is there. 1 is a block diagram generally showing a mobile communication terminal device according to the present invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Antenna 2 Tuner 3 Demodulation part 4 Demultiplexing part 5 Predecessor video stream analysis part 6 Video stream discard part 7 Decoding processing part 8 DCT coefficient reduction part 9 Inverse quantization part 10 Inverse DCT part 11 Adder 12 Image memory 13 Vector complement Unit 14 motion compensation unit 20 display control unit 21 multiplexing unit / demultiplexing unit 22 storage medium 23 speaker 24 display device 30 image data processing unit 31 audio data processing unit 32 display unit 34 high frequency unit 35 baseband unit 36 application processor unit 40 MPEG-4 video stream creation unit 41 MPEG-4 decoder

Claims (10)

A portable communication terminal device capable of receiving a broadcast signal by terrestrial digital television broadcasting and having a data processing unit and a display unit,
The data processing unit extracts an MPEG-2 first video data stream from the received broadcast signal,
Discard at least bi-directional inter-frame prediction coding information among intra-frame coding information, forward inter-frame prediction coding information, and bi-directional inter-frame prediction coding information included in the extracted first video data stream. Leaving intra-frame encoded information and generating a second video data stream for the remaining encoded information;
A mobile communication terminal device that decodes only a part of the encoded information of the second video data stream according to the display resolution of the display unit. A portable communication terminal device capable of receiving a broadcast signal by terrestrial digital television broadcasting and having a data processing unit and a display unit,
The data processing unit extracts an MPEG-2 first video data stream from the received broadcast signal,
Discard at least bi-directional inter-frame prediction coding information among intra-frame coding information, forward inter-frame prediction coding information, and bi-directional inter-frame prediction coding information included in the extracted first video data stream. Leaving intra-frame encoded information and generating a second video data stream for the remaining encoded information;
Of the encoded information of the second video data stream, a part of information corresponding to the display resolution of the display unit is converted into a third video data stream compliant with MPEG-4,
A mobile communication terminal device that decodes the converted third video data stream. The data processing unit may include only a block-unit DC component or a block-unit DC component and a part of an AC component for the intra-frame coding information included in the second video data stream. The mobile communication terminal device according to claim 1, wherein the mobile communication terminal device is the information. The said data processing part changes the magnitude | size of the motion vector of the forward inter-frame prediction encoding information contained in the said 2nd video data stream corresponding to the said display resolution. Mobile communication terminal device. The encoded information included in the second video data stream is information corresponding to one of an odd frame and an even frame in interlace scanning. Mobile communication terminal device. 3. The mobile communication terminal device according to claim 1, wherein the received broadcast signal constitutes an MPEG-2 transport stream. The mobile communication terminal device according to claim 6, wherein the data processing unit divides the transport stream into a first video data stream and an audio data stream. 8. The decoding apparatus according to claim 7, wherein the data processing unit includes a memory capable of multiplexing the second video data stream and the audio data stream and storing the multiplexed information as a transport stream. 9. The mobile communication terminal apparatus according to claim 1, further comprising a high frequency unit and a baseband unit for performing mobile communication. A portable communication terminal device capable of receiving a broadcast signal by terrestrial digital television broadcasting and having a data processing unit and a display unit,
The data processing unit extracts a first video data stream from the received broadcast signal,
Discard at least bi-directional inter-frame prediction coding information among intra-frame coding information, forward inter-frame prediction coding information, and bi-directional inter-frame prediction coding information included in the extracted first video data stream. Leaving intra-frame encoded information and generating a second video data stream for the remaining encoded information;
The mobile communication terminal device, wherein a part of information according to a display resolution of the display unit is displayed as an image display target among the encoded information of the second video data stream.

Images