JP2008211294A - Encoding device and encoding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an encoding device and an encoding method that can reduce an calculation load. <P>SOLUTION: A main encoder 11-1 selects motion compensation prediction encoding processing or intra-prediction performed for one frame and carries out encoding through the selected processing, and sub-encoders 11-2a to 11-2n perform the processing selected by the main encoder 11-1 on the one frame, and then, the sub-encoder 11-2a to 11-2n can omit selecting operation for the encoding processing performed for the one frame unlike the main encoder 11-1, thereby reducing the operation load. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a moving image compression technique, and more particularly to an encoding apparatus and an encoding method for encoding a moving image.

  In recent years, for example, a system capable of mutually viewing moving images taken by a plurality of terminal devices such as a TV conference system has been proposed. In such a system, each terminal device transmits a moving image captured by the own device to other terminal devices, and displays a plurality of moving images sent from other terminal devices on the same screen. Accordingly, the user of each terminal device can perform a conversation or the like while watching the moving image of the user of another terminal device.

  In such a system, in order to reduce the communication load, encoding is performed on a moving image to be transmitted and received. For example, H.M. When a moving image compression technique called H.264 or MPEG-4 (for example, see Non-Patent Document 1) is used, a moving image is encoded with reference to a spatially encoded I picture and other pictures. Are encoded into image data (hereinafter referred to as encoded data) including a plurality of consecutive pictures including a P picture or a B picture. The encoded data can be reproduced as a moving image by being decoded by the terminal device on the receiving side.

  However, depending on the communication environment, there may be a difference in the bandwidth of the transmission path. Even if the encoding is performed as described above, a reception delay or the like of the encoded data occurs. Playback may not be smooth. For this reason, in the terminal device on the transmission side, the compression rate of encoded data and the like are determined according to the bandwidth of the transmission path of the terminal device on the reception side (see, for example, Patent Document 1). For example, as shown in FIG. 6, encoders 101 and 102 having different compression rates are provided in a terminal device on the transmission side, and a plurality of pieces of encoded data having different compression rates are generated from the same moving image. Thereby, since the encoded data with the compression rate corresponding to the communication environment is sent to the terminal device on the receiving side, no reception delay or the like occurs, and as a result, the moving image can be reproduced smoothly.

JP 2002-344972 A International Telecommunication Union, “ITU-T Recommendation H.264 Advanced video coding for generic audiovisual services”, [online], [searched January 23, 2007], Internet <http://www.itu.int/rec/ T-REC-H.264 / en>

However, in the terminal device on the transmission side, if a plurality of encoders are provided according to the difference in compression rate, all the processing at the time of encoding is performed in each encoder, so that the load on the CPU increases. was there.
Accordingly, the present invention has been made to solve the above-described problems, and an object thereof is to provide an encoding device and an encoding method capable of reducing the calculation load.

  In order to solve the above-described problems, an encoding apparatus according to the present invention can perform any of motion compensation prediction encoding processing and intra prediction encoding processing for one frame from one image data including a plurality of frames. A first encoder that encodes the first frame by the selected process, and a second encoder that encodes the first frame by the process selected by the first encoder It is characterized by having. In the present invention, the other encoder performs the encoding by the process selected by the first encoder.

  In the above encoding apparatus, the other encoder encodes one frame using the parameters used when the first encoder encodes, that is, the data used to create the predicted image data. May be. Here, in the first encoder, when encoding one frame, the prediction mode is referred to in the case of intra prediction encoding processing, and the optimum block size is referred to in the case of motion compensation prediction encoding processing. Parameters such as frames and motion vectors are generated. Such parameters are transmitted to other encoders, and are used when encoding is performed in the other encoders.

  In the encoding method according to the present invention, which of the motion compensation predictive encoding process and the intra predictive encoding process is performed on one frame from one image data composed of a plurality of frames in the first encoder? A first step for encoding the first frame by the selected process, and a second step for causing the second encoder to encode the first frame by the process selected in the first step. And 2 steps.

  According to the present invention, the first encoder selects whether to perform motion compensation prediction encoding processing or intra prediction encoding processing for one frame, performs encoding by the selected processing, In the second encoder, by performing the process selected by the first encoder on the one frame, the encoding process performed on the first frame in the second encoder like the first encoder is performed. Since the selection operation can be omitted, the calculation load can be reduced.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In this embodiment, a case where the present invention is applied to a terminal device in a TV conference system will be described as an example.

[Configuration of TV conference system]
As shown in FIG. 1, the TV conference system according to the present embodiment includes a server 1 and a plurality of terminal devices 2a to 2n, each of which is formed by a network 3 including a LAN (Local Area Network) or the Internet. It is connected. Since the terminal devices 2a to 2n have the same configuration, they are hereinafter referred to as the terminal device 2 for convenience.

(Server configuration)
The server 1 performs call control of the terminal devices 2a to 2n constituting the TV conference system, and establishes communication between the terminal devices 2.

(Configuration of terminal device)
The terminal device 2 is composed of a known computer used by a user who uses the TV conference system. As shown in FIG. 2, the terminal device 2 includes an external I / F unit 4, an operation input unit 5, an image processing unit 6, an audio processing unit 7, a storage unit 8, and a control unit 9. Is done.

  The external I / F unit 4 includes a communication circuit, performs data communication with the server 1 and other terminal devices 2 via the network 3 and the like, and exchanges various data with each connected device.

  The operation input unit 5 includes a keyboard and a mouse, and detects a user operation and outputs it to the control unit 9.

  The image processing unit 6 includes an image processing circuit. The moving image captured by the camera 6a is input to the control unit 9, and the moving image input from the control unit 9 is output from the monitor 6b. Control and so on.

  The audio processing unit 7 includes a signal processing circuit, encodes audio input from the microphone 7a into audio data and outputs the audio data to the control unit 9, or decodes audio data input from the control unit 9 and outputs from the speaker 7b. Output, and has functions such as volume control and echo canceller.

  The storage unit 8 includes a storage device such as a memory or a hard disk, and stores control data 8a and a program 8b. Here, the control data 8 a is information used for controlling the terminal device 2. The program 8b is an operation program for the terminal device 2 and is stored in the storage unit 8 in advance. The program 8b may be stored in the storage unit 8 via a recording medium such as a DVD (Digital Versatile Disk) or a CD (Compact disk) or a network.

  The control unit 9 includes a microprocessor such as a CPU and its peripheral circuits, and reads and executes the program 8b from the storage unit 8 to cause the hardware and the program 8b to cooperate with each other, thereby encoding means 11 and decoding means. 12 and the TV conference means 13 are realized.

  The encoding means 11 is an H.264 standard. Based on the H.264 standard, a plurality of continuous frames (images) generated by the camera 6a and the image processing unit 6 are encoded to generate encoded data composed of a plurality of continuous pictures (encoded images). . As shown in FIG. 2B, the encoding unit 11 includes a main encoder 11-1 and sub-encoders 11-2a to 11-2n that generate encoded data having a compression rate different from that of the main encoder. Composed. In the sub-encoders 11-2a to 11-2n, the same components and the same components as those of the main encoder 11-1 are denoted by the same names and description thereof is omitted as appropriate.

  The main encoder 11-1 selectively performs one of an intra prediction process and a motion compensation prediction process on the input moving image to generate encoded data, and information on the selected encoding process and the selected code The information regarding the parameters used in the conversion process is transmitted to the sub-encoder 11-2. As shown in FIG. 3, the main encoder 11-1 includes a difference calculation unit 111, a DCT unit 112, a quantization unit 113, an inverse quantization unit 114, an IDCT unit 115, an addition calculation unit 116, and a motion compensation prediction unit. 118, a frame memory 119, an intra prediction unit 121, a switch 122, and an encoded data output unit 123. Details of the encoding processing operation by the main encoder 111 will be described later.

  The sub-encoders 11-2a to 11-2n, based on the selection result of the encoding process received from the main encoder 11-1 and the parameters used for the encoding process, output the main encoder 11- 1 is performed at a different compression rate. As shown in FIG. 3, the sub encoders 11-2a to 11-2n include a difference calculation unit 111, a DCT unit 112, a quantization unit 113, an inverse quantization unit 114, an IDCT unit 115, an addition calculation unit 116, A prediction mode detection unit 117, a motion compensation prediction unit 118, a frame memory 119, a motion detection unit 120, an intra prediction unit 121, a switch 122, and an encoded data output unit 123 are provided. Such sub-encoders 11-2a to 11-2n are generated according to the number of pieces of encoded data having different compression rates. Details of the encoding processing operation by the sub-encoders 11-2a to 11-2n will be described later. Further, since the sub-encoders 11-2a to 11-2n have the same configuration, they are hereinafter referred to as the sub-encoder 11-2 for convenience.

  The decoding unit 12 decodes encoded data received from another terminal device 2 via the external I / F unit 4 to generate image data.

  The TV conference means 13 implements a TV conference system by exchanging encoded data and audio data with other terminal devices 2 via the network 3 and the external I / F unit 4. Specifically, first, as a result of the call control of the server 1, the quantity of encoded data to be generated according to the communication environment, protocol, apparatus performance, etc. of the terminal apparatus 2 used for the TV conference, and each encoded data A protocol, a compression rate, etc. are determined, and the quantity of the secondary encoder 11-2 is set according to the quantity. In addition, the encoded data generated by the main encoder 11-1 and the sub-encoder 11-2 of the encoding unit 11 and the audio data generated by the audio processing unit 7 are transmitted to each of the other terminal devices 2. In addition, the audio data received from the other terminal device 2 is output from the speaker 7 b by the audio processing unit 7. Also, the encoded data received from each of the other terminal devices 2 is decoded by the decoding means 12, and the generated frames are simultaneously displayed at a predetermined position and size on the monitor 6b by the image processing unit 6. By doing in this way, it is possible to have a conversation with a user of another terminal device 2 while watching each moving image.

[Encoding processing operation by the main encoder]
Next, the encoding processing operation by the main encoder 11-1 of the encoding means 11 will be described with reference to FIGS.

  First, when image data including a plurality of frames is input from the image processing unit 6 (step S1), the main encoder 11-1 divides each input frame into units of macroblocks, and then performs intra prediction encoding processing. And prediction image data is produced | generated by a motion compensation prediction encoding process (step S2). Note that division in units of macroblocks may be performed by the image processing unit 6 or by providing a functional unit that performs division processing in the main encoder 11-1.

  In the case of intra prediction encoding processing, the main encoder 11-1 detects the optimal prediction mode related to the luminance and color difference signal of the input frame by the prediction mode detection unit 117, and then detects by the intra prediction unit 121. Based on the predicted mode and the frame, predicted image data is generated. Note that the prediction mode detection unit 117 also outputs the prediction mode used when decoding the frame to the intra prediction unit 121 of the sub-encoder 11-2. Here, when there are a plurality of sub-encoders 11-2, the prediction mode detection unit 117 outputs the prediction mode detected by the main encoder 11-1 to each of the sub-encoders 11-2.

  In the case of the motion compensation predictive encoding process, the main encoder 11-1 uses the motion detection unit 120 based on the input frame and the reconstructed image data stored in the frame memory 119, and the optimum block size, reference frame, and After detecting a motion vector or the like, the motion compensation prediction unit 118 generates predicted image data based on the detected parameters and the reconstructed image data stored in the frame memory 119. The motion detection unit 120 also outputs the optimal block size, reference frame, motion vector, and the like used when decoding the frame to the motion compensation prediction unit 118 of the sub encoder 11-2. Here, when there are a plurality of sub-encoders 11-2, the motion detection unit 120 outputs the block size detected by the main encoder 11-1, a frame to be referenced, a motion vector, and the like to each of the sub-encoders 11-2. To do.

  When the prediction image data is generated by the intra prediction encoding process and the motion compensation prediction encoding process, the main encoder 11-1 uses the switch 122 to calculate the difference between each prediction image data and the input frame. By generating image data and comparing them, the more efficient one of the intra prediction encoding process and the motion compensation prediction encoding process is selected (step S3). Here, the switch 122 selects the encoding process with the smaller amount of the generated difference image data, that is, the encoding process with the smaller difference between the predicted image data and the input frame as being efficient.

  When the encoding process to be performed is selected, the main encoder 11-1 causes the switch 122 to output information related to the selected encoding process (hereinafter referred to as selection information) to the switch 122 of the sub-encoder 11-2. (Step S4). Also, the selection information is transmitted to each of the sub encoders 11-2 when there are a plurality of sub encoders 11-2.

  While outputting the selection information, the main encoder 11-1 outputs the predicted image data generated by the encoding process selected by the switch 122 to the difference calculation unit 111, and is input by the difference calculation unit 111. Difference image data obtained by taking the difference between the obtained frame and the predicted image data is generated (step S5).

  When the difference image data is generated, the main encoder 11-1 causes the DCT unit 112 and the quantization unit 113 to perform frequency conversion and quantization on the difference image data (step S6). At this time, the main encoder 11-1 performs predetermined compression of the encoded data by causing the quantization unit 113 to perform quantization according to the communication environment or performance of the destination terminal device of the generated encoded data. Adjust the rate.

  When quantization is performed, the main encoder 11-1 causes the encoded data output unit 123 to perform variable-length encoding (step S7). Thereby, the input frame is encoded by the encoding process selected by the switch 122, and is output as encoded data (step S8). As the encoded data, an I picture is output when the intra prediction encoding process is selected, and a B picture or a P picture is output when the motion compensation prediction encoding process is selected.

  Note that the residual data output from the quantization unit 113 is sequentially subjected to inverse quantization and inverse frequency conversion by the inverse quantization unit 114 and the IDCT unit 115 (step S9), and the addition calculation unit 116 obtains predicted image data. After the addition (step S10), it is stored in the frame memory 119 as reconstructed image data (step S11). The reconstructed image data stored in the frame memory 119 is used when the above-described motion compensation prediction encoding process is performed.

  Thus, according to the present embodiment, selection information related to the encoding process performed on one frame by the main encoder 11-1 is output to the sub-encoder 11-2. Further, the parameters used when encoding the one frame are also output to the sub-encoder 11-2.

[Encoding processing operation by sub encoder]
Next, with reference to FIGS. 3 and 5, the encoding processing operation by the sub-encoder 11-2 of the encoding means 11 will be described.

  First, when image data composed of a plurality of frames is input from the image processing unit 6, the sub-encoder 11-2 divides each input frame into macroblock units (step S 21). Note that division in units of macroblocks may be performed by the image processing unit 6 or by providing a functional unit that performs division processing in the sub-encoder 11-2.

  When the frame is input, the secondary encoder 11-2 receives the selection information from the main encoder 11-1 (step S22), and performs an encoding process corresponding to the selection information (step S23).

  For example, when selection information indicating that intra prediction encoding processing has been performed on an arbitrary frame is received, the intra prediction unit 121 of the sub-encoder 11-2 generates predicted image data of the arbitrary frame. At this time, the intra prediction unit 121 uses the parameters used when the main encoder 11-1 generates predicted image data of the arbitrary frame, that is, the prediction mode detected by the prediction mode 117, for the arbitrary frame. Predictive image data is generated.

  On the other hand, when the selection information indicating that the motion compensation prediction encoding process has been performed on an arbitrary frame is received, the motion compensation prediction unit 118 of the sub-encoder 11-2 generates predicted image data of the arbitrary frame. . At this time, the motion compensation prediction unit 118 uses the parameters used when the main encoder 11-1 generates predicted image data of the arbitrary frame, that is, the optimum block size, reference frame, and motion detected by the motion detection unit 120. Predictive image data of the arbitrary frame is generated using a vector or the like.

  When the predicted image data is generated by the selected encoding process, the sub-encoder 11-2 causes the switch 122 to output the predicted image data generated based on the selection information to the difference calculation unit 111, and this difference calculation. The unit 111 generates difference image data obtained by taking a difference between the input frame and the predicted image data (step S24).

  When the difference image data is generated, the sub encoder 11-2 causes the DCT unit 112 and the quantization unit 113 to perform frequency conversion and quantization on the difference image data (step S25). At this time, the sub-encoder 11-2 causes the quantization unit 113 to perform quantization according to the communication environment or performance of the destination terminal device of the generated encoded data, thereby reducing the compression rate of the encoded data. adjust.

  When quantization according to the communication environment or performance of the destination terminal device is performed, the sub-encoder 11-2 causes the encoded data output unit 123 to convert the variable length code to the residual data output from the quantization unit 113. (Step S26). Thereby, the input frame is encoded by the same encoding process as that of the main encoder 11-1, and is output as encoded data (step S27). Also in this case, as the encoded data, an I picture is output when the intra prediction encoding process is selected, and a B picture or a P picture is output when the motion compensated prediction encoding process is selected.

  Note that the residual data output from the quantization unit 113 is sequentially subjected to inverse quantization and inverse frequency transform by the inverse quantization unit 114 and the IDCT unit 115 (step S28), and the addition calculation unit 116 obtains predicted image data. After the addition (step S29), it is stored in the frame memory 119 as reconstructed image data (step S30). The reconstructed image data stored in the frame memory 119 is used when the above-described motion compensation prediction encoding process is performed.

  As described above, the sub-encoder 11-2 encodes one frame by the same encoding process as that of the main encoder 11-1, based on the selection information acquired from the main encoder 11-1. For this reason, in the sub-encoder 11-2, the operation is omitted to select whether to perform intra prediction encoding or motion compensation prediction encoding, and thus it is possible to reduce the calculation load. Specifically, since the switch 122 does not have to perform an operation for selecting the predicted image data to be output, the calculation load for this operation can be reduced. Also, the sub encoder 11-2 does not cause both the intra prediction unit 121 and the motion compensation prediction unit 118 to generate predicted image data as in the main encoder 11-1, but rather the intra prediction unit 121 and the motion compensation prediction unit 118. Since it is only necessary to generate predicted image data in either one of them, the calculation load can be reduced.

  Further, when performing the intra prediction encoding, the sub encoder 11-2 generates prediction image data using the prediction mode acquired from the main encoder 11-1, and thus generates a prediction mode like the main encoder 11-1. Since there is no need to do this, the calculation load can be reduced. Further, the sub-encoder 11-2 generates predicted image data using the optimum block size, reference frame, motion vector, and the like acquired from the main encoder 11-1 when performing motion compensation predictive coding. Since it is not necessary to generate these parameters as in the encoder 11-1, the calculation load can be reduced.

  As described above, according to the present embodiment, the main encoder 11-1 selects which of the motion compensation prediction encoding process and the intra prediction encoding process is performed on one frame, and this selection. The sub-encoder 11-2 performs the process selected by the main encoder 11-1 on the one frame, so that the sub-encoder 11-2 performs the same process as the main encoder 11-1 in the sub-encoder 11-2. In addition, since the selection operation of the encoding process performed on the one frame can be omitted, the calculation load can be reduced.

  The present invention can be applied to various apparatuses that generate a plurality of encoded data from the same image data.

It is a figure which shows typically the structure of the video conference system which concerns on this invention. (A) is a block diagram showing the configuration of the terminal device, (b) is a diagram showing the configuration of the encoding means. It is a block diagram which shows the structure of an encoding means in detail. It is a flowchart which shows the encoding process operation by the main encoder. It is a flowchart which shows the encoding process operation | movement by a subencoder. It is a block diagram which shows the structure of the conventional encoder.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Server, 2, 2a, 2b, 2n ... Terminal device, 3 ... Network, 4 ... External I / F part, 5 ... Operation input part, 6 ... Image processing part, 6a ... Camera, 6b ... Monitor, 7 ... Audio | voice Processing unit, 7a ... microphone, 7b ... speaker, 8 ... storage unit, 8a ... control data, 8b ... program, 9 ... control unit, 11 ... encoding means, 11-1 ... main encoder, 11-2, 11-2a 11-2n ... sub encoder, 12 ... decoding means, 13 ... TV conference means, 111 ... difference calculation section, 112 ... DCT section, 113 ... quantization section, 114 ... dequantization section, 115 ... IDCT section, 116 ... addition operation unit, 117 ... deblocking filter, 118 ... motion compensation prediction unit, 119 ... frame memory, 120 ... motion detection unit, 121 ... intra prediction unit, 122 ... switch, 123 ... encoded data output unit, 124 Switch.

Claims (3)

A selection is made between motion compensation predictive coding processing and intra prediction coding processing for one frame from one image data consisting of a plurality of frames, and the first frame is coded by the selected processing. 1 encoder,
An encoding apparatus comprising: a second encoder that encodes the first frame by a process selected by the first encoder. The encoding apparatus according to claim 1, wherein the second encoder encodes the first frame using a parameter used when the first encoder encodes. The first encoder is made to select which one of motion compensation prediction encoding processing and intra prediction encoding processing is to be performed on one frame from one image data composed of a plurality of frames. A first step of encoding one frame;
And a second step of causing the second encoder to encode the first frame by the process selected in the first step.

Images