JP2018113525A - Moving picture encoding device and moving picture encoding method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress distortion due to encoding by forming the encoding more efficiently in a fade period.SOLUTION: A moving picture encoding device which performs encoding by selecting, frame by frame, a plurality of encoding methods including a first encoding method for performing inter-frame predictive encoding based upon a temporally past image having been already encoded as a reference image and a second encoding method for performing the inter-frame predictive encoding based upon temporally past and future images having been already encoded as reference images has encoding method selecting means for selecting an encoding method for a frame to be encoded from the plurality of encoding methods including the first encoding method and the second encoding method according to fade information indicating whether the frame to be encoded is an image in a fade period. The encoding method selecting means increases a rate at which the second encoding method is selected more when the fade information indicates that the frame is within the fade period than when the frame is not in the fade period.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a moving image encoding apparatus and a moving image encoding method capable of encoding edited moving image data.

  In recent years, digitalization of information related to so-called multimedia such as audio signals and image signals has been rapidly progressing, and in response to this, compression encoding / decoding technology of image data has attracted attention. Since compression encoding and decoding techniques can reduce the storage capacity necessary for storing image data and the bandwidth necessary for transmission, this technique is extremely important for the multimedia industry.

  The compression encoding technique compresses the information amount / data amount by utilizing the high autocorrelation (that is, redundancy) of many images. The redundancy of image data includes temporal redundancy and two-dimensional spatial redundancy, and temporal redundancy can be reduced by using block-based motion detection and motion compensation. On the other hand, spatial redundancy can be reduced using discrete cosine transform (DCT). An image encoding method using these techniques is described in H.264. H.264 / MPEG-4 PART10 (AVC) (hereinafter referred to as H.264).

  H. In H.264, an I picture that is encoded using only that frame, a P picture that encodes a difference from a reference image using a past frame that has already been encoded as a reference image, and a temporal that has already been encoded In addition, there is a B picture that encodes a difference using a past frame, a future frame, or an average value of a past frame and a future frame as a reference image. A moving image is encoded by combining these plurality of encoding methods.

  In general, in consideration of reproduction from the middle of a stream at the time of reproduction, encoding is performed with an I picture that does not use correlation between frames at intervals of about 0.5 seconds. In general, encoding is performed by arranging two B pictures between an I picture and a P picture, and between a P picture and a P picture.

Japanese Patent Laying-Open No. 2005-303362

  By the way, there is a fade in a technique often used when editing a moving image. Fade is a moving image effect in which moving images gradually appear (fade in) and gradually disappear (fade out), and is often used at the joint between moving images and moving images. In addition, there is a type in which a moving image is gradually switched (cross fade) by superimposing a fade-out of a moving image before joining and a fade-in of a moving image after joining. Such a moving image of the fade portion has a low correlation between frames, and the data amount is compressed using temporal redundancy of image data. In the case of encoding with H.264, there is a problem that the compression efficiency is lowered.

  Patent Document 1 proposes a technique for reducing distortion due to encoding of a fade period by increasing the ratio of the code amount of a P picture and a B picture in a fade period as compared with the case of a non-fade period. However, in Patent Document 1, the compression efficiency of encoding using the correlation between frames in the fade period remains low, and the amount of code partially increases.

  Therefore, an object of the present invention is to perform encoding more efficiently in a fade period and to suppress distortion due to encoding.

  In order to achieve the above object, a video encoding apparatus according to the present invention includes a first encoding method that performs interframe predictive encoding using a previously encoded temporally past image as a reference image, A moving image code that selects and encodes a plurality of encoding methods for each frame, including a second encoding method that performs inter-frame predictive encoding using encoded past and future images as reference images A plurality of encoding methods including a first encoding method and a second encoding method according to fade information indicating whether a frame to be encoded is an image within a fade period. A coding method selection unit that selects a coding method of a frame to be coded from the coding method, and the coding method selection unit indicates that the fade information is within a fade period; Within the fade period Than not is characterized by increasing the proportion of selecting the second encoding method.

  In order to achieve the above object, a moving image encoding method according to the present invention includes a first encoding method that performs interframe predictive encoding using a temporally past image that has already been encoded as a reference image, A moving image code that selects and encodes a plurality of encoding methods for each frame, including a second encoding method that performs inter-frame predictive encoding using encoded past and future images as reference images A plurality of encoding methods including a first encoding method and a second encoding method according to fade information indicating whether or not a frame to be encoded is an image within a fade period. A coding method selection step for selecting a coding method of a frame to be coded from the coding method, and the coding method selection step indicates that the fade information is within a fade period; , Than not within Edo period, characterized by increasing the proportion of selecting the second encoding method.

  ADVANTAGE OF THE INVENTION According to this invention, it can encode more efficiently in a fade period and can suppress the distortion by encoding.

2 is a block diagram for explaining an example of a configuration of a moving image encoding apparatus 100 according to Embodiment 1. FIG. It is a figure for demonstrating an example of a structure of the image editing system to which the moving image encoder 100 is applied. It is the figure which showed typically selection of the image and encoding method of a cross-fade period. 6 is a flowchart for explaining the operation of the encoding method selection circuit according to the first embodiment. 10 is a flowchart for explaining an operation of an encoding scheme selection circuit in the second embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiments.
[Embodiment 1]
FIG. 2 is a diagram for explaining an example of the configuration of an image editing system to which the moving image encoding apparatus 100 is applied.

  The image editing apparatus 201 includes a decoding unit that decodes a moving image or a still image, reads the encoded image data recorded on the recording medium 203 through the recording / reproducing unit 202, and decodes the image data by the decoding unit. The decoded image data is edited by the image editing apparatus 201 and then transmitted to the encoding apparatus 100. The encoding device 100 encodes the edited image transmitted from the image editing device 201 as a moving image, and records the generated encoded image data on the recording medium 203 via the recording / reproducing device 202. Note that the image used for image editing is output as a moving image regardless of whether it is a still image or a moving image.

  The image editing apparatus 201 can perform at least a fade process, and fade information for identifying whether or not each frame of an image transmitted to the encoding apparatus 100 is a frame in a fade-processed period, The data is transmitted to the encoding device 100. Note that the fade process has an effect that the image gradually appears with time (fade in), an effect that gradually disappears (fade out), and further includes an effect that the image gradually changes with time (cross fade).

  FIG. 1 is a block diagram for explaining an example of a configuration of a moving image encoding apparatus 100 that predictively encodes moving image data. In FIG. 1, the moving picture encoding apparatus 100 includes an encoding method selection circuit 120, a frame memory 102, an inter prediction circuit 103 that searches for a motion vector, and an intra prediction circuit 104 that selects an intra prediction method. Yes. Furthermore, an intra / inter selection circuit 105 that selects a prediction method of inter prediction or intra prediction, a predicted image generation circuit 106, a subtractor 107, an integer conversion circuit 108, and a quantization circuit 109 are provided. ing.

  Furthermore, an inverse quantization circuit 110, an inverse integer conversion circuit 111, an adder 112, an in-loop filter 113, an entropy encoding circuit 115, a quantization control circuit 116, and a code amount control circuit 117 are provided. Yes. The frame memory 102 also includes a reference image memory that stores a reference image used for inter prediction. The moving image data transmitted from the image editing apparatus 201 is sequentially stored in the frame memory 102 in the order of the first frame, the second frame, the third frame,. Image data is extracted from the frame memory 102 in the order of encoding, for example, the third frame, the first frame, the second frame,...

  Here, the encoding methods include intra encoding that encodes only image data within a frame, and inter encoding that encodes including prediction between frames. A picture to be inter-encoded is a P picture that predicts one reference frame for a motion compensation unit (MC block) and B that predicts up to two reference frames for an MC block. There is a picture. In the case of referring to two reference frames in a B picture, encoding is performed using an image obtained by averaging the pixel values of the two reference frames for each pixel as a reference frame. On the other hand, a picture to be subjected to intra coding is an I picture.

  Note that the order of frames to be encoded is different from the order of input frames in order to enable prediction (rearward prediction) with not only past frames but also temporally future frames. The selection of the encoding method for each frame is performed by the encoding method selection circuit 120, and the selected encoding method is notified to each processing circuit in the moving image encoding apparatus 100.

  In the case of performing intra coding, image data of a coding target block serving as a coding unit is read from the frame memory 102 and input to the intra prediction circuit 104. In Embodiment 1, the unit of one encoding target block is 16 horizontal pixels × 16 vertical pixels. Further, the read pixel data around the encoding target block is also read from the frame memory 102 and input to the intra prediction circuit 104.

  The intra prediction circuit 104 selects an intra prediction method based on the correlation between the encoding target block and a plurality of intra prediction image data generated from pixel data around the encoding target block, and an intra / inter selection circuit. To 105. When performing intra coding, the intra / inter selection circuit 105 always selects intra prediction and notifies the prediction image generation circuit 106 of the selection result. The predicted image generation circuit 106 generates intra predicted image data from reconstructed image data output from an adder 112 described later according to the prediction method selected by the intra / inter selection circuit 105.

  The subtractor 107 receives the above-described intra prediction image data and the image data of the encoding target block read from the frame memory 102, and calculates the difference image data of the pixel values of the image of the encoding target block and the intra prediction image. Output to the integer conversion circuit 108. The integer conversion circuit 108 performs integer conversion on the input difference image data of the pixel value, and the quantization circuit 109 performs quantization processing on the signal converted by the integer conversion circuit 108.

  The entropy coding circuit 115 entropy codes the transform coefficient quantized by the quantization circuit 109 and outputs a stream. Here, the quantization coefficient in the quantization circuit 109 is calculated by the quantization control circuit 116 from the code amount generated in the entropy encoding circuit 115, the target code amount set from the code amount control circuit 117, and the like. Further, the transform coefficient quantized by the quantization circuit 109 is also input to the inverse quantization circuit 110. The inverse quantization circuit 110 inversely quantizes the input transform coefficient, and the inverse integer transform circuit 111 performs an inverse integer transform process on the inversely quantized signal.

  The adder 112 receives and adds the inverse integer-converted data and the intra predicted image data generated by the predicted image generation circuit 106. The data after the addition becomes decoded reconstructed image data, which is input to the above-described predicted image generation circuit 106 and used to generate intra predicted image data. In addition, the reconstructed image data is subjected to encoding distortion reduction processing by the in-loop filter 113 and is stored in the reference image memory of the frame memory 102 as reference image data used in inter-encoding described later.

  On the other hand, when inter coding is performed, image data of an encoding target block serving as a coding unit is read from the frame memory 102 and input to the inter prediction circuit 103. The inter prediction circuit 103 reads reference image data from the reference image memory, detects a motion vector from the image data to be encoded and the reference image data, and notifies the intra / inter selection circuit 105 of the motion vector.

  Depending on the frame, it is possible to select inter prediction or intra prediction for each encoding target block. The intra prediction circuit 104 operates as described above, and notifies the intra / inter selection circuit 105 of the result of intra prediction. The intra / inter selection circuit 105 receives the result of the inter prediction circuit 103 and the result of the intra prediction circuit 104, selects, for example, the prediction method with the smaller difference value, and notifies the prediction image generation circuit 106 of the prediction method. The subtractor 107 calculates the difference between the image to be encoded and the predicted image, and generates difference image data. The difference image data is output to the integer conversion circuit 108, and the subsequent processing is the same as in the case of the intra coding described above.

  The operation of the encoding method selection circuit 120 will be described in detail. Here, it is assumed that the edited image data output from the editing apparatus 201 includes a switching portion by cross fading from the image A to the image B, and the operation during the switching period will be described. FIG. 3A shows temporal changes of images before and after image switching in edited image data. Further, FIG. 3B shows a change in fade information corresponding thereto. A process in which the encoding method selection circuit 120 determines the picture type of the encoding target frame will be described with reference to the flowchart of FIG.

  The encoding method selection circuit 120 determines whether or not to initialize the B counter, which is a counter that counts how many B pictures are continuous, in S401. When the B counter is not initialized (No in S401), the process proceeds to S403. On the other hand, when the B counter is initialized at the start of recording (Yes in S401), the process proceeds to S402. In S402, the value of the B counter is initialized to 0, and the process proceeds to S403. In step S403, whether or not the encoding target frame is included in the fade period is determined based on the fade information notified from the image editing apparatus or the like. When it is a fade period (Yes in S403), the process proceeds to S404, and in S404, the number of B, which is a set value of consecutive B pictures, is set to 3, and the process proceeds to S406.

  On the other hand, if it is not the fade period (No in S403), the process proceeds to S405, the number of B is set to 2 in S405, and the process proceeds to S406. In S406, it is determined whether or not the value of the B counter is the same as the value of the B number. If the value of the B counter is not the same as the value of the B number (No in S406), the process proceeds to S410. In S410, the encoding method is determined to be B picture, and 1 is added to the value of the B counter in S412. The process ends. On the other hand, if the value of the B counter is the same as the value of the B number in S406 (Yes in S406), the process proceeds to S407. In S407, it is determined whether 0.5 seconds or more have elapsed since the previous selection of the I picture.

  If 0.5 seconds or more have elapsed (Yes in S407), it is determined in S408 that the encoding method is I picture, the value of the B counter is set to 0 in S411, and the process ends. On the other hand, if 0.5 seconds or more have not elapsed (No in S407), it is determined in S409 that the encoding method is P picture, the value of the B counter is set to 0 in S411, and the process ends.

  With the above operation, the encoding method in the vicinity of the fade period is selected as shown in FIG. An arrow in FIG. 3D indicates that the original image of the arrow is used as a reference image at the time of encoding at the tip of the arrow. For comparison, FIG. 3C shows selection of an encoding method when the methods in the first and second embodiments are not applied. Similarly, the arrow in FIG. 3C indicates a reference image. In FIG. 3C, the same encoding method is selected in the fade period as in the non-fade period.

  When the method according to the first embodiment is applied, the ratio at which the B picture is selected in the fade period is larger than that in the period other than the fade period, and the coding efficiency is improved. Since the fade period is an image obtained by combining the past image and the future image in time, the B picture that can use the image obtained by combining the past image and the future image in time as the reference image This is because the encoding efficiency is higher than that of a P picture that can only refer to an image.

[Embodiment 2]
In the second embodiment, the configuration of the video encoding apparatus in the second embodiment is the same as that described in the first embodiment, and here, differences from the first embodiment will be mainly described. In the second embodiment, the operation of the encoding method selection circuit 120 is different. The process in which the encoding method selection circuit 120 determines the picture type of the encoding target frame will be described with reference to the flowchart of FIG.

  The encoding method selection circuit 120 determines whether or not to initialize the B counter, which is a counter that counts how many B pictures are continuous, in S401. When the B counter is not initialized (No in S401), the process proceeds to S403. On the other hand, when the B counter is initialized at the start of recording (Yes in S401), the process proceeds to S402. In S402, the value of the B counter is initialized to 0, and the process proceeds to S403. In step S403, whether or not the encoding target frame is included in the fade period is determined based on the fade information notified from the image editing apparatus or the like.

  When it is a fade period (Yes in S403), the process proceeds to S404, and in S404, the number of B, which is a set value of consecutive B pictures, is set to 3, and the process proceeds to S406. On the other hand, if it is not the fade period (No in S403), the process proceeds to S405, the number of B is set to 2 in S405, and the process proceeds to S406. In S406, it is determined whether or not the value of the B counter is the same as the value of the B number. If the value of the B counter is not the same as the value of the B number (No in S406), the process proceeds to S501. In S501, it is determined whether it is a fade period and the value of the B counter is 1.

  If it is a fade period and the value of the B counter is 1 (Yes in S501), the encoding method is determined to be the reference B picture in S502, and the process proceeds to S412. The reference B picture is an encoding method that may be used as a reference image of another frame in the B picture. On the other hand, if it is a fade period and the value of the B counter is not 1 (No in S501), it is determined in S410 that the encoding method is B picture, and the process proceeds to S412. In S412, 1 is added to the value of the B counter, and the process ends.

  On the other hand, if the value of the B counter is the same as the value of the B number in S406 (Yes in S406), the process proceeds to S407. In S407, it is determined whether 0.5 seconds or more have elapsed since the previous selection of the I picture. If 0.5 seconds or more have elapsed (Yes in S407), it is determined in S408 that the encoding method is I picture, the value of the B counter is set to 0 in S411, and the process ends. On the other hand, if 0.5 seconds or more have not elapsed (No in S407), it is determined in S409 that the encoding method is P picture, the value of the B counter is set to 0 in S411, and the process ends.

  With the above operation, the encoding method in the vicinity of the fade period is selected as shown in FIG. In addition, the arrow in FIG.3 (e) has shown using the original image of an arrow as a reference image at the time of the encoding of the tip of an arrow. For comparison, FIG. 3C shows selection of an encoding method when the methods in the first and second embodiments are not applied. Similarly, the arrow in FIG. 3C indicates a reference image. In FIG. 3C, the same encoding method is selected in the fade period as in the non-fade period.

  When the method according to the second embodiment is applied, the encoding efficiency is improved because the temporal distance between the two reference images is equal for all the B pictures in the fade period. The fade period is an image in which the past image and the future image are synthesized in time. In general, the composition ratio changes at a constant speed with respect to time, so the intervals become equal in time. This is because the coding efficiency of a B picture that uses an image obtained by combining an average of past reference images and future reference images as a reference image increases.

[Embodiment 3]
The various functions, processes, and methods described in the first and second embodiments can also be realized by using a program by a personal computer, a microcomputer, a CPU (Central Processing Unit), or the like. Hereinafter, in the third embodiment, a personal computer, a microcomputer, a CPU, and the like are referred to as “computer X”. In the third embodiment, a program for controlling the computer X and realizing the various functions, processes, and methods described in the first and second embodiments is referred to as “program Y”.

  The various functions, processes, and methods described in the first and second embodiments are realized by the computer X executing the program Y. In this case, the program Y is supplied to the computer X via a computer-readable storage medium. The computer-readable storage medium according to the third embodiment includes at least one of a hard disk device, a magnetic storage device, an optical storage device, a magneto-optical storage device, a memory card, a volatile memory, and a nonvolatile memory. The computer-readable storage medium in the third embodiment is a non-transitory storage medium.

DESCRIPTION OF SYMBOLS 100 Moving image encoder 120 Coding method selection circuit 102 Frame memory 103 Inter prediction circuit 104 Intra prediction circuit

Claims (6)

A first encoding method for performing inter-frame predictive encoding using an already encoded temporally past image as a reference image, and inter-frame prediction using an already encoded temporally past and future image as a reference image A video encoding device that selects and encodes a plurality of encoding methods including a second encoding method that performs encoding for each frame,
From the plurality of encoding methods including the first encoding method and the second encoding method according to fade information indicating whether or not the frame to be encoded is an image within a fade period, Encoding method selection means for selecting an encoding method of a frame to be encoded,
The encoding method selection means increases the ratio of selecting the second encoding method when the fade information indicates that it is within a fade period, compared to when it is not within a fade period. A video encoding device. The second encoding method is:
A referenceable second encoding method that can be used as a reference image when decoding a frame of a decoded image when encoding another frame;
The decoded image frame can be selected from any one of the second non-referenceable encoding methods that cannot be used as a reference image when encoding other frames. The moving image encoding apparatus according to 1. The encoding method selection means, when the fade information indicates that it is within a fade period, selects the second encoding method that cannot be referred to every other frame,
For a frame in which the second non-referenceable encoding method is not selected, either the first encoding method or the referenceable second encoding method is alternately selected. The moving image encoding apparatus according to claim 2. A first encoding method for performing inter-frame predictive encoding using an already encoded temporally past image as a reference image, and inter-frame prediction using an already encoded temporally past and future image as a reference image A video encoding method that selects and encodes for each frame a plurality of encoding methods including a second encoding method that performs encoding,
From the plurality of encoding methods including the first encoding method and the second encoding method according to fade information indicating whether or not the frame to be encoded is an image within a fade period, An encoding method selection step of selecting an encoding method of a frame to be encoded;
In the encoding method selection step, when the fade information indicates that it is within a fade period, the ratio of selecting the second encoding method is increased as compared with a case where the fade information is not within a fade period. A video encoding method. The second encoding method is:
A referenceable second encoding method that can be used as a reference image when decoding a frame of a decoded image when encoding another frame;
The decoded image frame can be selected from any one of the second non-referenceable encoding methods that cannot be used as a reference image when encoding other frames. 5. The moving image encoding method according to 4. In the encoding method selection step, when the fade information indicates that it is within a fade period, the second non-referenceable encoding method is selected every other frame,
For a frame in which the second non-referenceable encoding method is not selected, either the first encoding method or the referenceable second encoding method is alternately selected. The moving image encoding method according to claim 5.