JP4735558B2 - Information processing apparatus and information processing method - Google Patents

Description

  The present invention relates to a moving image compression technique, and more particularly to an information processing apparatus and an information processing method for decoding encoded moving image data.

  In recent years, for example, a system capable of mutually viewing moving images taken by a plurality of terminal devices such as a TV conference system has been proposed. In such a system, each terminal device transmits a moving image captured by the own device to other terminal devices, and displays a plurality of moving images sent from other terminal devices on the same screen. Accordingly, the user of each terminal device can perform a conversation or the like while watching the moving image of the user of another terminal device.

  In such a system, in order to reduce the communication load, encoding is performed on a moving image to be transmitted and received. For example, H.M. When a moving image compression technique called H.264 or MPEG-4 (for example, see Non-Patent Document 1) is used, a moving image is encoded with reference to a spatially encoded I picture and other pictures. Are encoded into image data (hereinafter referred to as encoded data) including a plurality of consecutive pictures including a P picture or a B picture. The encoded data can be reproduced as a moving image by being decoded by the terminal device on the receiving side.

  In the system described above, encoding of moving images and decoding of encoded data are performed by software installed in each terminal device. For this reason, there is no problem in the operation of the terminal device as long as it is transmission / reception of encoded data by a small number of terminal devices, for example, about 3 or 4 units. If this is the case, each terminal device must simultaneously decode a large number of encoded data, which increases the load on the CPU, and as a result, the moving image may not be reproduced normally.

  In order to prevent such a situation, conventionally, for example, a method for reducing the load on the decoding process in the terminal device such as decoding only the I picture or changing the frame rate has been proposed (for example, Patent Documents). 1).

Japanese Patent Laid-Open No. 9-74548 International Telecommunication Union, “ITU-T Recommendation H.264 Advanced video coding for generic audiovisual services”, [online], [searched January 23, 2007], Internet <http://www.itu.int/rec/ T-REC-H.264 / en>

  However, the method of decoding only I pictures makes it difficult to provide high-quality moving images because the frame rate during moving image reproduction becomes extremely low. If the number of I pictures is increased in order to solve this problem, the image quality at the time of decoding decreases in a low bit rate environment. In order to improve the image quality, it is necessary to increase the bit rate. However, since the I picture has a larger amount of data than the P picture and B picture, it occupies much of the network bandwidth and affects other communications. May come out.

  The method for changing the frame rate is good when the frame rate is high, but when the frame rate is low, it is difficult to reproduce a high-quality moving image.

As described above, conventionally, it has been difficult to reduce the computational load on the terminal device and to reproduce high-quality moving images.
Accordingly, an object of the present invention is to provide an information processing apparatus and an information processing method capable of reducing a calculation load and improving a moving image.

In order to solve the above-described problems, an information processing apparatus according to the present invention receives encoded data including a plurality of consecutive encoded pictures, and determines whether or not the input picture is an I picture. When the picture is an I picture, the picture is output. On the other hand, when the picture is not an I picture, whether or not a predetermined other picture input before the picture has been decoded. If the other picture has been decoded, a selection unit that does not output the picture and a decoding unit that decodes the picture output from the selection unit are provided . not a picture picture is not the immediately preceding, to have a structure that reference picture is not an I-picture that is more temporally distant I picture or decoding And butterflies.

Further, another information processing apparatus according to the present invention receives encoded data including a plurality of consecutive encoded pictures, determines whether or not the input picture is an I picture, and the picture is If the picture is an I picture, the picture is output. On the other hand, if the picture is not an I picture, it is determined whether a predetermined other picture input prior to the picture has been decoded. If the picture is decoded, the picture is not output. If the other picture is not decoded, it is determined whether or not the reference picture to be referenced by the picture is decoded. If the picture has been decoded, the picture is output, and if the reference picture has not been decoded, the selection means that does not output the picture, and output from the selection means E Bei was a decoding means for decoding the picture, the encoded data is a picture not I pictures, not immediately before the picture that is not an I-picture that is more temporally distant I picture or decoding It has a structure to be referred to .

Also, the information processing method according to the present invention receives encoded data including a plurality of consecutive encoded pictures, determines whether or not the input picture is an I picture, and the picture is an I picture. If the picture is not an I picture, it is determined whether or not a predetermined other picture input before the picture has been decoded, and the other picture is output. a selection step does not output the picture if but decoded, possess a decoding step of decoding the outputted picture by the selection step, the encoded data, a picture not I-picture, the immediately preceding It is characterized by having a structure that refers to a picture that is not a far-in-time I picture or a picture that is not a decoded I picture .

Further, another information processing method according to the present invention receives encoded data including a plurality of consecutive encoded pictures, determines whether or not the input picture is an I picture, and the picture is If the picture is an I picture, the picture is output. On the other hand, if the picture is not an I picture, it is determined whether a predetermined other picture input prior to the picture has been decoded. If the picture is decoded, the picture is not output. If the other picture is not decoded, it is determined whether or not the reference picture to be referenced by the picture is decoded. A selection step of outputting the picture if the picture has been decoded, and not outputting the picture if the reference picture has not been decoded, and the selection step Possess a decoding step for decoding a more output picture, encoded data, the picture is not an I-picture is not a last minute, not the I picture that is more temporally distant I picture or decoding It has a structure that refers to a picture .

  According to the present invention, when an input picture is not an I picture, if a predetermined other picture input before the picture is decoded, the picture is not decoded. The load can be reduced. In addition, since it is possible to decode pictures other than I pictures, it is possible to improve the quality of moving images.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In this embodiment, a case where the present invention is applied to a terminal device in a TV conference system will be described as an example.

[Configuration of TV conference system]
As shown in FIG. 1, the TV conference system according to the present embodiment includes a server 1 and a plurality of terminal devices 2a to 2n, each of which is a communication line 3 including a LAN (Local Area Network), the Internet, or the like. Connected by. Since the terminal devices 2a to 2n have the same configuration, they are hereinafter referred to as the terminal device 2 for convenience.

(Server configuration)
The server 1 performs call control of the terminal devices 2a to 2n constituting the TV conference system, and establishes communication between the terminal devices 2.

(Configuration of terminal device)
The terminal device 2 is composed of a known computer used by a user who uses the TV conference system. As shown in FIG. 2, the terminal device 2 includes an external I / F unit 4, an operation input unit 5, an image processing unit 6, an audio processing unit 7, a storage unit 8, and a control unit 9. Is done.

  The external I / F unit 4 includes a communication circuit, performs data communication with the server 1 and other terminal devices 2 through the communication line 3 and the like, and exchanges various data with each connected device.

  The operation input unit 5 includes a keyboard and a mouse, and detects a user operation and outputs it to the control unit 9.

  The image processing unit 6 includes an image processing circuit. The moving image captured by the camera 6a is input to the control unit 9, and the moving image input from the control unit 9 is output from the monitor 6b. Control and so on.

  The audio processing unit 7 includes a signal processing circuit, encodes audio input from the microphone 7a into audio data and outputs the audio data to the control unit 9, or decodes audio data input from the control unit 9 and outputs from the speaker 7b. Output, and has functions such as volume control and echo canceller.

  The storage unit 8 includes a storage device such as a memory or a hard disk, and stores control data 8a and a program 8b. Here, the control data 8 a is information used for controlling the terminal device 2. The program 8b is an operation program for the terminal device 2 and is stored in the storage unit 8 in advance. The program 8b may be stored in the storage unit 8 via a recording medium such as a DVD (Digital Versatile Disk) or a CD (Compact disk) or a network.

  The control unit 9 includes a microprocessor such as a CPU and its peripheral circuits, and reads and executes the program 8b from the storage unit 8 to cause the hardware and the program 8b to cooperate with each other, thereby encoding means 11 and decoding means. 12 and the TV conference means 13 are realized.

  The encoding means 11 is an H.264 standard. Based on the H.264 standard, a plurality of continuous frames (images) generated by the camera 6a and the image processing unit 6 are encoded to generate encoded data composed of a plurality of continuous pictures (encoded images). . As shown in FIG. 3, the encoding unit 11 includes a difference calculation unit 111, a DCT unit 112, a quantization unit 113, an inverse quantization unit 114, an IDCT unit 115, an addition calculation unit 116, A blocking filter 117, a motion compensation prediction unit 118, a frame memory 119, a motion detection unit 120, an intra prediction unit 121, a switch 122, and an encoded data output unit 123 are provided. Details of the image encoding processing operation by the encoding means 11 having such a configuration will be described later.

  The decoding unit 12 decodes encoded data received from another terminal device 2 via the external I / F unit 4 to generate image data. As shown in FIG. 4, the decoding unit 12 includes a selection unit 150, a decoding unit 131, an inverse quantization unit 132, an IDCT unit 133, an addition operation unit 134, a deblocking filter 135, a frame memory 136, a motion compensation prediction. Unit 137, intra prediction unit 138, and switch 139. Such a decoding means 12 is provided for each terminal device 2 of the other party when exchanging encoded data with a plurality of terminal devices 2 at the same time as in the TV conference system. Details of the decryption processing operation by the decryption means 12 will be described later.

  The TV conference means 13 implements a TV conference system by exchanging encoded data and audio data with another terminal device 2 via the communication line 3 and the external I / F unit 4. Specifically, first, as a result of the call control of the server 1, the quantity of the decoding means 12 is set according to the quantity of the terminal devices 2 used for the TV conference. Further, the encoded data generated by the encoding unit 11 and the audio data generated by the audio processing unit 7 are transmitted to each of the other terminal devices 2. In addition, the audio data received from the other terminal device 2 is output from the speaker 7 b by the audio processing unit 7. Also, the encoded data received from each of the other terminal devices 2 is decoded by the corresponding decoding means 12, and each of the generated frames is simultaneously performed at a predetermined position and size on the monitor 6b by the image processing unit 6. Display. For example, as shown in FIG. 5, the moving image from the terminal device 2 of the speaking user is displayed largely at the center of the screen as indicated by the symbol x, and the moving image from the terminal device 2 of the other user is As indicated by the symbol y, it may be displayed in a small size. By doing in this way, it is possible to have a conversation with a user of another terminal device 2 while watching each moving image.

[Encoding processing operation]
Next, the encoding processing operation by the encoding means 11 will be described with reference to FIG. There are two types of encoding: intra prediction encoding and motion compensation prediction encoding. These are switched by the switch 122. Here, first, intra prediction encoding will be described.

(Intra prediction coding)
Intra-predictive encoding is basically compression encoding within one frame. Specifically, the frame input from the image processing unit 6 is input to the DCT unit 112 and subjected to frequency conversion, then quantized by the quantization unit 113, and variable-length encoded by the encoded data output unit 123. Applied. As a result, an I picture is output.

  The residual data output from the quantization unit 113 is generated when the inverse quantization unit 114 and the IDCT unit 115 sequentially perform inverse quantization and inverse frequency conversion, and the deblocking filter 117 performs encoding. After processing for reducing block distortion is performed, it is stored in the frame memory 119 as reconstructed image data.

(Motion compensated prediction coding)
The motion compensation predictive encoding basically encodes a change from another frame (reference frame). Specifically, when a frame is input from the image processing unit 6, the motion detection unit 120 detects a motion vector based on the input frame and the reconstructed image data stored in the frame memory 119. The motion compensation prediction unit 118 generates predicted image data based on the motion vector and the reconstructed image data stored in the frame memory 119. The difference calculation unit 111 calculates the difference between the input frame and the predicted image data, and generates difference image data. The difference image data is input to the DCT unit 112 and subjected to frequency conversion, and then the encoded data output unit 123 performs variable length encoding. Thereby, P picture and B picture are output.

  Note that the residual data output from the quantization unit 113 is sequentially subjected to inverse quantization and inverse frequency conversion by the inverse quantization unit 114 and the IDCT unit 115, and the addition calculation unit 116 adds the predicted image data, After processing for reducing block distortion caused when encoding is performed by the blocking filter 117, it is stored in the frame memory 119 as reconstructed image data.

  H. as described above. The encoded data generated based on the H.264 standard generally has a configuration called a NAL unit. As shown in FIG. 6, the NAL unit includes a NAL header and an RBSP (Raw Byte Sequence Payload). The NAL header includes a fixed bit, nal_ref_ido including a flag indicating whether the picture is a reference picture, and nal_unit_type including an identifier indicating the type of the NAL unit. The RBAP is composed of slice data composed of compressed data and a slice header composed of header information related to the slice data. This slice header can identify which picture the reference picture is.

[Decryption processing operation]
Next, the operation of decoding encoded data by the decoding unit 12 will be described with reference to FIG.

  First, in the input encoded data, each picture is discriminated based on a predetermined condition by the discriminating means 152 of the selection unit 150, and only the picture decoded by the output means 151 is selectively outputted based on the discrimination result. The Details of the picture selection operation by the selection unit 150 will be described later.

  There are two types of decoding for the selected picture: intra prediction decoding and motion compensated prediction decoding. These are switched by a switch 139. Here, first, intra prediction decoding will be described.

(Intra prediction decoding)
Intra-prediction decoding restores the original frame from only the I picture. The picture selected by the selection unit 150 is subjected to variable length transformation by the decoding unit 131, subjected to inverse quantization and inverse frequency transformation in order by the inverse quantization unit 132 and the IDCT unit 133, and then decoded by the deblocking filter 135. The block distortion generated during the conversion is alleviated and output as a frame. This frame is also stored in the frame memory 136. Thus, an I picture is decoded without referring to other pictures.

(Motion compensated prediction decoding)
Next, motion compensation predictive decoding will be described. This motion compensation predictive decoding restores the original frame from the reference picture and the motion vector. When a picture is input from the selection unit 150, the picture is subjected to variable length transformation by the decoding unit 131, and inverse quantization and inverse frequency transformation are sequentially performed by the inverse quantization unit 132 and the IDCT unit 133, and residual decoding is performed. Output as digitized data. At this time, the motion compensation prediction unit 137 generates predicted image data based on the frame stored in the frame memory 136. The residual encoded data and the predicted image data generated as described above are added by the addition operation unit 134, and the block distortion generated when decoding is performed by the deblocking filter 135 is alleviated. Is output as This frame is also stored in the frame memory 136.

[Picture selection operation]
Next, the selection operation of the picture to be decoded by the selection unit 150 will be described with reference to FIGS.

  First, when the encoded data is input (step S1), the selection unit 150 sequentially determines whether or not each picture is to be decoded by the determination unit 152 according to the following procedure.

  The discriminating means 152 discriminates whether or not the input picture (hereinafter referred to as “input picture”) is an I picture by the I picture discriminating means 152a (step S2). This determination is performed by referring to the nal_unit_type of the NAL header in the NAL unit described with reference to FIG.

  When the input picture is an I picture (step S2: YES), the I picture determination unit 152a selects the input picture as a picture to be decoded and outputs it from the output unit 151 (step S6). As a result, in the present embodiment, all I pictures are decoded.

  As an example, a description will be given of a selection operation performed by the selection unit 150 with respect to encoded data composed of consecutive pictures of codes a to l illustrated in FIG. In FIG. 8, the characters shown in the rectangle indicate the type of each picture, and the arrows indicate the reference pictures to which each picture refers. For example, a picture with code b is a P picture, which means that an I picture with code a is referenced.

  In the case of the encoded data in FIG. 8, the picture with the code a and the picture with the code g are I pictures. Accordingly, the I picture determination unit 152a selects the pictures of the code a and the code g as the pictures to be encoded and outputs them from the output unit 151. Thereby, as shown in FIG. 9, the frame based on the picture of the code | symbol a and the code | symbol g is produced | generated. In FIG. 9, the upper rectangle represents each picture of the encoded data corresponding to FIG. 8, and the lower rectangle connected to these pictures by an arrow represents a frame output from the decoding means 13. When a person's picture frame is decoded, it means a frame corresponding to the selected picture, and a frame marked with “x” means a frame corresponding to the selected picture when discarded.

  On the other hand, when the input picture is not an I picture (step S2: NO), the determination unit 152 determines whether or not a picture existing at a predetermined position before the input picture is decoded by the predetermined picture determination unit 152b. (Step S3). The predetermined picture discriminating means 152b refers to the frame memory 136 and discriminates whether or not the picture inputted before the input picture (the picture existing immediately before in the present embodiment) has been decoded.

  When the immediately preceding picture has been decoded (step S3: YES), the predetermined picture discriminating means 152b does not select the input picture as a picture to be decoded, that is, selects it as a picture to be discarded and does not output it from the output means 151 ( Step S5). If the immediately preceding picture is decoded in this way, even if the immediately following picture is discarded, it is considered that the difference is hardly perceived by human eyes. Therefore, in the present embodiment, when the previous picture is decoded, it is possible to reduce the number of pictures to be decoded while suppressing the deterioration of the image quality by discarding the input picture. 2 calculation load can be reduced.

  For example, the pictures b and h in FIG. 8 are obtained by decoding the previous picture, that is, the picture of the code a or g. Therefore, the predetermined picture discriminating unit 152b selects the pictures b and h as the pictures to be discarded and does not output them from the output unit 151. Thereby, as shown in FIG. 9, the frame based on the picture of the code | symbol b and the code h is not produced | generated.

  When the immediately preceding picture has not been decoded (step S3: NO), the determination unit 152 decodes a picture (hereinafter referred to as “reference picture”) referred to by the input picture by the reference picture determination unit 152c. It is determined whether or not there is (step S4). As described above, the RBSP slice header of the NAL unit shown in FIG. 6 includes information indicating which picture is the reference picture of the input picture. Therefore, the reference picture determination unit 152c specifies the reference picture of the input picture from the slice header of the RBSP and searches the frame memory 136 for the reference picture to determine whether or not the reference picture of the input picture has been encoded. To do.

  When the reference picture is decoded (step S4: YES), the reference picture determination unit 152c selects the input picture as a picture to be decoded and outputs it from the output unit 151 (step S6). On the other hand, when the reference picture is not decoded (step S4: NO), the reference picture determination unit 152c does not select the input picture as a picture to be decoded, that is, selects it as a picture to be discarded and does not output it from the output unit 151. (Step S5). If the picture to be referenced is discarded, the input picture cannot be decoded with sufficient quality, and the computation load increases. For this reason, in this Embodiment, when the picture to refer is discarded, the calculation load of the terminal device 2 can be reduced by discarding an input picture.

  For example, in the picture with the code d shown in FIG. 8, the previous picture is not decoded, but the picture with the code a to be referenced is decoded. Therefore, the reference picture determination unit 152c selects the picture with the code d as a picture to be decoded and outputs it from the output unit 151. On the other hand, as for the picture of code c, the picture immediately before is not decoded and the picture of code b which is a picture to be referenced is not decoded. Therefore, the reference picture determination unit 152 c selects the picture with the code c as a picture to be discarded and does not output it from the output unit 151. As a result, as shown in FIG. 9, a frame based on the picture with the code d is generated, whereas a frame based on the picture with the code c is not generated.

  The picture output from the selection unit 150 by such a selection operation is input to the decoding unit 131 and is decoded by each component of the decoding unit 12 to be finally output as a frame. Here, the selection results by the selection unit 150 for the decoded data shown in FIG. 8 are summarized as shown in FIG. 9 and the following.

Since the picture with the code a is determined to be an I picture in step S2, it is output from the output means 151 and decoded.
Since it is determined in step S3 that the previous picture has been decoded, the picture with the code b is not output from the output means 151, that is, discarded.
The picture with the code c is not output from the output means 151 because it is determined that the picture referred to in step S4 has not been decoded, that is, discarded.
Since it is determined that the picture referred to in step S4 has been decoded, the picture with the code d is output from the output unit 151 and decoded.
Since it is determined in step S3 that the previous picture has been decoded, the picture with the code e is not output from the output means 151, that is, discarded.
Since it is determined in step S4 that the picture referred to by the code f is decoded, it is output from the output means 151 and decoded.
Since the picture with the code g is determined to be an I picture in step S2, it is decoded.
Since it is determined in step S3 that the previous picture has been decoded, the picture with the code h is not output from the output means 151, that is, discarded.
Since it is determined that the picture referred to in step S4 has been decoded, the picture with the code i is output from the output means 151 and decoded.
Since it is determined in step S3 that the immediately preceding picture has been decoded, the picture with the code j is not output from the output means 151, that is, discarded.
Since it is determined that the picture referred to in step S4 has been decoded, the picture with the code k is output from the output unit 151 and decoded.
Since it is determined in step S3 that the previous picture has been decoded, the picture with the code l is not output from the output means 151, that is, discarded.

  As described above, when the selection unit 150 performs the selection operation on the encoded data including the pictures indicated by the codes a to l, only the pictures of the codes a, d, f, g, i, and k are decoded. Is done. As a result, the number of pictures to be decoded is reduced, so that the calculation load on the terminal device 2 can be reduced. Further, since not only the I picture but also the P pictures of the codes d, f, i, and k are decoded, the image quality can be improved as compared with the case of decoding only the I picture. Further, since the code c having no picture to be referred to is not decoded, it is possible to prevent the image quality from being deteriorated and to reduce the calculation load.

  When the selection result by the selection unit 150 as shown in FIG. 9 is output and the selected picture is decoded by the decoding means 12, the frame memory 136 of the decoding means 12 stores the result shown in FIG. ) To (f), the frame is stored. In FIG. 10, it is assumed that a picture corresponding to a frame of code g is an IDR (Instantaneous Decoding Refresh) picture.

  That is, as shown in FIGS. 10A to 10C, when the picture of codes a, d, and f is selected as a picture to be decoded by the selection unit 150, each of the other components of the decoding means 12 Pictures are sequentially decoded and corresponding frames are stored in the frame memory 136.

  However, when the picture with the code g is selected as a picture to be decoded by the selection unit 150, the picture with the code g is an IDR (Instantaneous Decoding Refresh) picture, and therefore, the frames with the codes a, d, and f from the frame memory 136. Is cleared, and only the frame of code g is stored.

  Next, when the picture of codes g and i is selected as a picture to be decoded by the selection unit 150, it is sequentially decoded by the other components of the decoding means 12, and the corresponding frame is framed together with the frame of the code g. Stored in

  In this way, since only the picture selected by the selection unit 150 is decoded, the number of pictures stored in the frame memory 136 can be reduced. Thereby, it becomes possible to reduce the operation of the decoding means 12, particularly the deblocking filter 135 and the motion compensation prediction unit 137, and as a result, the calculation load of the terminal device 2 can be reduced. Further, by referring to such a frame memory 136, the operations of the predetermined picture determination unit 152b and the reference picture determination unit 152c of the determination unit 152 can be realized.

  As described above, according to the present embodiment, it is determined whether or not an input picture is an I picture. If the picture is not an I picture, a predetermined other input before the picture is determined. It can be determined whether or not the current picture is decoded, and if the other picture is decoded, the picture is not output, thereby reducing the computation load associated with the decoding. In addition, since it is possible to decode pictures other than the I picture, the image quality can be improved as a result.

  When the encoded data is exchanged with a plurality of terminal devices 2 at the same time as in the TV conference system, the decoding means 12 is provided for each terminal device 2 that is a communication partner. Since the decryption means 12 is realized by the CPU reading and executing the program 8b, the amount of the decryption means 12 increases as the number of the counterpart terminal device 2 increases. The load will also increase. Therefore, in the present embodiment, it is possible to reduce the processing of each decoding unit 12 by selectively decoding only a predetermined picture from the received encoded data by the selection unit 150. As a result, the calculation load on the CPU of the terminal device 2 can be reduced.

  In the present embodiment, for example, as illustrated in FIGS. 8 and 9, the encoded data is described as being composed of an I picture and a P picture, but the encoded data is an I picture, a P picture, and Needless to say, it may be composed of B pictures. In this case, for example, by determining whether or not to decode a B picture based on whether or not a predetermined picture and a reference picture previously input are decoded, the calculation load of the terminal device 2 is reduced. In addition to the reduction, the image quality of the moving image can be improved.

  In the present embodiment, the predetermined picture discriminating unit 152b identifies whether or not the picture immediately before the input picture has been decoded. However, the picture to be identified is not limited to this. It is possible to set freely as appropriate, such as a few previous pictures and a predetermined number of pictures before and immediately before.

  In FIG. 5, all the pictures may be decoded for the code data from the terminal device 2 that reproduces a moving image having a large display area indicated by the symbol x. When a picture that decodes encoded data that displays a decoded frame with a large display area and encoded data that displays a decoded frame with a small display area under the same conditions is selected, the larger display area You may feel that the image quality is lower than the smaller display area. For this reason, for encoded data having a large display area, all pictures are decoded. Thereby, user experience quality can be improved. At this time, for encoded data having a small display area, a picture to be decoded is selected by the method described above. Thereby, the calculation load of the terminal device 2 can be reduced.

  Further, the moving image based on the frame output from the decoding unit 12 may be reproduced at the same frame rate as the moving image based on the image data obtained by decoding all the pictures of the encoded data. In this case, a frame that has been normally reproduced before this frame may be continuously displayed in the frame corresponding to the discarded picture. Thereby, a moving image can be smoothly reproduced.

  The present invention can be applied to various apparatuses that decode encoded data.

It is a figure showing typically the composition of the TV conference system concerning the present invention. It is a block diagram which shows the structure of a terminal device. It is a block diagram which shows the structure of an encoding means. It is a block diagram which shows the structure of a decoding means. It is a figure which shows the example of a display of the monitor of the terminal device in a video conference system. It is a figure which shows the structure of a NAL unit typically. It is a flowchart which shows selection operation | movement of the picture to decode. It is a figure which shows an example of decoding data typically. It is a figure which shows typically the result of the selection operation | movement of the picture to decode with respect to the decoding data of FIG. (A)-(f) is a figure which shows typically the state of the frame memory which stored the picture decoded by the selection result of FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Server, 2, 2a, 2b, 2n ... Terminal device, 3 ... Communication line, 4 ... External I / F part, 5 ... Operation input part, 6 ... Image processing part, 6a ... Camera, 6b ... Monitor, 7 ... Audio processing unit, 7a ... microphone, 7b ... speaker, 8 ... storage unit, 8a ... control data, 8b ... program, 9 ... control unit, 11 ... encoding means, 12 ... decoding means, 13 ... TV conference means, 111 ... difference calculation unit, 112 ... DCT unit, 113 ... quantization unit, 114 ... dequantization unit, 115 ... IDCT unit, 116 ... addition calculation unit, 117 ... deblocking filter, 118 ... motion compensation prediction unit, 119 ... frame Memory 120, motion detector 121, intra prediction unit 122, switch 123, encoded data output unit 131, decoding unit 132, inverse quantization unit 133, IDCT unit 134, addition operation unit 13 Deblocking filter, 136 Frame memory, 137 Motion compensation prediction unit, 138 Intra prediction unit, 139 Switch, 150 Selection unit, 151 Output unit, 152 Discrimination unit, 152a I picture discrimination unit, 152b ... predetermined picture discrimination means, 152c ... reference picture discrimination means.

Claims (4)

Encoded data including a plurality of consecutive encoded pictures is input, it is determined whether or not the input picture is an I picture, and if the picture is an I picture, the picture is output , the picture may not be the I picture, it is determined whether or not certain other picture input earlier than the picture is being decoded, it outputs the picture if the other picture is decoded No selection means,
E Bei and decoding means for decoding the outputted picture from the selection means,
2. The information processing apparatus according to claim 1, wherein the encoded data has a structure in which a picture that is not an I picture refers to a picture that is not immediately before and that is farther in time or a picture that is not a decoded I picture . Encoded data including a plurality of consecutive encoded pictures is input, it is determined whether or not the input picture is an I picture, and if the picture is an I picture, the picture is output If the picture is not an I picture, it is determined whether or not a predetermined other picture input before the picture has been decoded. If the other picture has been decoded, the picture is output. If the other picture is not decoded, it is determined whether or not a reference picture to be referenced by the picture is decoded. If the reference picture is decoded, the picture is output. Selection means for not outputting the picture unless the reference picture is decoded;
Decoding means for decoding the picture output from the selection means ,
2. The information processing apparatus according to claim 1, wherein the encoded data has a structure in which a picture that is not an I picture refers to a picture that is not immediately before and that is farther in time or a picture that is not a decoded I picture . Encoded data including a plurality of consecutive encoded pictures is input, it is determined whether or not the input picture is an I picture, and if the picture is an I picture, the picture is output , the picture may not be the I picture, it is determined whether or not certain other picture input earlier than the picture is being decoded, it outputs the picture if the other picture is decoded Not a selection step and
Possess a decoding step of decoding the outputted picture by the selection step,
The information processing method characterized in that the encoded data has a structure in which a picture that is not an I picture refers to an I picture that is not immediately before, is farther in time, or is not a decoded I picture . Encoded data including a plurality of consecutive encoded pictures is input, it is determined whether or not the input picture is an I picture, and if the picture is an I picture, the picture is output If the picture is not an I picture, it is determined whether or not a predetermined other picture input before the picture has been decoded. If the other picture has been decoded, the picture is output. If the other picture is not decoded, it is determined whether or not a reference picture to be referenced by the picture is decoded. If the reference picture is decoded, the picture is output. A selection step of not outputting the picture unless the reference picture is decoded;
Possess a decoding step of decoding the outputted picture by the selection step,
The information processing method characterized in that the encoded data has a structure in which a picture that is not an I picture refers to an I picture that is not immediately before, is farther in time, or is not a decoded I picture .

Images