JP5486670B2 - Video encoding apparatus and control method thereof - Google Patents

Description

  The present invention relates to a moving picture coding apparatus and a control method thereof, and more particularly, to a moving picture coding apparatus and a control method thereof for performing efficient coding.

  Recent advances in digital signal processing technology enable high-efficiency encoding of large amounts of digital information, such as moving images, still images, and audio, so that they can be recorded on a small recording medium or transmitted via a communication medium. It has become. By applying such technology, a moving image encoding apparatus capable of converting a moving image taken by a television broadcast or a video camera into a stream has been developed.

  As a moving image encoding method, MPEG2 method or H.264 is used. The H.264 (aka AVC) system is well known. In particular, H.C. The H.264 system is improved so that moving picture compression coding can be performed more efficiently than the MPEG2 system by adopting inter-picture coding that enables reference to any picture.

  However, H.C. In the H.264 system, an arbitrary picture can be referred to, but some pictures are clearly unsuitable for reference. If such a picture can be excluded from the reference target, it can be said that the encoding efficiency is further improved. Thus, for example, Patent Document 1 discloses a technique for avoiding reference to a picture taken with a flash.

JP 2006-115336 A

  On the other hand, H. In the H.264 system, encoding truncation processing may be performed. The encoding truncation process will be specifically described below.

  FIG. 4 is a diagram illustrating a state of the decoder buffer in which the encoded stream is stored. When the encoded stream is decoded, the encoded stream is input to the decoder buffer at a predetermined rate as time advances. On the other hand, data corresponding to the code amount of the picture to be decoded is removed from the decoder buffer at every decoding time. Therefore, in order to prevent the decoder buffer from overflowing and to prevent underflow, the amount of data to be removed must be appropriately controlled. For this purpose, it is necessary to appropriately control the code amount of the picture. For example, in FIG. 4, it is necessary that data for the code amount of a picture to be decoded at time T does not exceed the code amount S.

  As a means for appropriately controlling the code amount of a picture in this way, in the production of video software that is commercially available for a DVD player or the like, a multi-pass encoding that repeats the encoding of a picture several times is used. . However, many apparatuses that require encoding in real time, such as recording with a video camera, can only perform one-time encoding, and one-pass encoding is generally used. Therefore, when the generated code amount of a picture is larger than expected, it is necessary to increase the compression rate in the middle of the picture and suppress the code amount within a predetermined amount. Specifically, the high-frequency component of the video is extremely reduced, and the amount of generated codes is extremely reduced. This process is referred to as an encoding abort process in this specification.

  FIG. 3 is a diagram illustrating an example of a screen when an abort process occurs in the middle of a picture. FIG. 3 shows an example of a decoding result when encoding is executed in order from the upper left of the picture and the truncation process occurs in the middle of the screen. In the upper screen portion 301, the video is appropriately decoded, whereas in the lower screen portion 302, the video is decoded into a block-like video.

  When inter-picture predictive coding is performed using a picture that has been subjected to the truncation process as a reference picture, the block-like video portion has a very low correlation with the corresponding portion of the picture to be coded. There has been a problem that the amount of generated codes increases greatly.

  In view of the above-described problems, the present invention provides a moving picture coding apparatus and a control method thereof that improve coding efficiency when a moving picture is coded using a coding scheme that can cause truncation processing. One purpose.

Further, the above object is for each picture constituting the moving image, and setting means for setting a picture type in encoding, in accordance with the picture type set by the setting means, Picture in each picture constituting a moving picture When the integrated value of the generated code amount of the first picture encoded by the encoding means and the encoding means that compresses and encodes by either the encoding method or the inter-picture predictive encoding method exceeds a predetermined value In addition, an instruction means for instructing the encoding means to increase the encoding compression rate from the middle of the first picture and to reduce the subsequent generated code amount of the first picture, and according to the instruction of the instruction means A second picture that is encoded after the first picture and is encoded with reference to the first picture using an inter-picture predictive encoding scheme, As encoded using puncturing the coding scheme also achieved by the video encoding apparatus; and a changing means for changing the picture type set by the setting means.

  With such a configuration, according to the present invention, it is possible to improve the encoding efficiency when a moving image is encoded using an encoding method that can cause a truncation process.

It is a block diagram which shows the function structural example of the moving image encoder of embodiment of this invention. , , , It is a figure which shows an example of the reference and referenced reference of a picture. It is a figure which shows degradation of the picture by the truncation process. It is a figure which shows transition of the accumulation | storage amount in a decoder buffer. , In the embodiment of the present invention, it is a flowchart showing an example of a processing procedure from performing the abort process to changing the reference picture. It is a block diagram which shows the function structural example of the moving image encoding device used as background art.

(First embodiment)
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 6 is a block diagram showing an example of a functional configuration of a moving picture coding apparatus as a background art of the present invention.
In FIG. 6, 900 is an input unit, and 901 is an intra-picture encoding unit. Reference numeral 902 denotes an inter-picture prediction encoding unit, and reference numeral 903 denotes a picture type setting unit. Further, 904 is a selector, and 905 is a generated code amount calculation unit. Reference numeral 906 denotes a stream output unit, and reference numeral 907 denotes an abort control unit. Reference numeral 908 denotes a local decoding unit.

  The video signal (moving image data) input to the input unit 900 is supplied to the intra-picture encoding unit 901 or the inter-picture prediction encoding unit 902, respectively. The intra-picture coding unit 901 performs coding independently one by one without referring to other pictures using the spatial correlation of the pictures constituting the video. Standards based on ISO / IEC13818-2, ITU-T H.264, etc. In the standard based on H.264, an image encoded by such an encoding method is defined as an I picture (also referred to as an intra-picture encoded picture or an I slice).

  On the other hand, the inter-picture prediction encoding unit 902 performs encoding using similarity between pictures having different display times, that is, temporal correlation. In the above-described standard, an image encoded by such an encoding method is a P picture (also referred to as inter-picture forward predictive encoded picture or P slice) or B picture (inter-picture bi-directional predictive encoded picture or B It is also called a slice). In addition, when performing inter-picture predictive coding, a picture to be referenced with respect to a picture to be coded is referred to as a reference picture. The reference picture is not an input image itself but a decoded image obtained by locally decoding a coded picture that has already been encoded by the local decoding unit 908. By using this reference picture, the propagation of the degradation error is prevented.

  The picture type setting unit 903 instructs the selector 904 about the type of the picture indicated by I, P, or B described above. The selector 904 selects the intra-picture coding unit 901 when the instructed picture type is the type (I) of the picture performed by the intra-picture coding method, and the code encoded by the intra-picture coding unit 901 is selected. The generated data is supplied to the generated code amount calculation unit 905. On the other hand, when the designated picture type is a picture type (P or B) performed by the inter-picture predictive coding method, the inter-picture predictive coding unit 902 is selected, and the inter-picture predictive coding unit 902 performs coding. The encoded data is supplied to the generated code amount calculation unit 905.

  The generated code amount calculation unit 905 outputs the encoded data supplied from the selector 904 to the stream output unit 906 and calculates an integrated value of the amount of encoded data (generated code amount). If the integrated value exceeds a predetermined value, the truncation control unit 907 receives the integrated value calculated by the generated code amount calculation unit 905, and sends it to the intra-picture encoding unit 901 or the inter-picture prediction encoding unit 902. Send an abort instruction signal.

FIG. 1 is a block diagram illustrating an example of a functional configuration of a video encoding device according to an embodiment of the present invention.
The moving image encoding apparatus 10 includes an input unit 100, an intra-picture encoding unit 101, an inter-picture prediction encoding unit 102, a picture type setting unit 103, and a selector 104. The intra-picture coding unit 101, the inter-picture prediction coding unit 102, and the selector 104 are collectively referred to as a coding unit 20. In addition, the moving image coding apparatus 10 includes a generated code amount calculation unit 105, a stream output unit 106, an abort control unit 107, and a local decoding unit 108. In the present embodiment, the moving image encoding apparatus 10 further includes a picture control unit 110, a picture type resetting unit 111, a buffer unit 112, and a reference picture setting unit 113. Each component of the moving image encoding device 10 is controlled by a microcomputer (not shown).

  The video signal (moving image data) input to the input unit 100 is supplied to the intra-picture encoding unit 101 and the inter-picture prediction encoding unit 102, respectively. The intra-picture coding unit 101 performs coding independently one by one using the spatial correlation of pictures constituting a moving image. Standards based on ISO / IEC13818-2, ITU-T H.264, etc. In the standard based on H.264, an image encoded by such an encoding method is defined as an I picture (also referred to as an intra-picture encoded picture or an I slice).

  On the other hand, the inter-picture predictive encoding unit 102 performs predictive encoding using similarity between pictures having different display times, that is, temporal correlation. In the above-described standard, an image encoded by such an encoding method is a P picture (also referred to as inter-picture forward predictive encoded picture or P slice) or B picture (inter-picture bi-directional predictive encoded picture or B It is also called a slice). In addition, when performing inter-picture predictive coding, a picture to be referenced with respect to a picture to be coded is referred to as a reference picture. The reference picture is not an input image itself but a decoded image obtained by locally decoding a coded picture that has already been encoded by the local decoding unit 108. Note that a plurality of pieces of reference picture data are held in a frame memory included in the inter-picture predictive encoding unit 102. By using this reference picture in inter-picture predictive coding, propagation of the degradation error is prevented.

  The picture type setting unit 103 instructs the selector 104 on the type of the picture indicated by I, P, or B described above. The selector 104 selects the intra-picture encoding unit 101 when the instructed picture type is a picture type (I) performed by the intra-picture encoding method, and the code encoded by the intra-picture encoding unit 101 is selected. The generated data is supplied to the generated code amount calculation unit 105. On the other hand, when the designated picture type is a picture type (P or B) performed by the inter-picture predictive coding method, the inter-picture predictive coding unit 102 is selected and the inter-picture predictive coding unit 102 performs coding. The encoded data is supplied to the generated code amount calculation unit 105.

  The generated code amount calculation unit 105 outputs the encoded data supplied from the selector 104 to the buffer unit 112 and calculates an integrated value of the amount of encoded data (generated code amount). When the integrated value exceeds a predetermined value, the truncation control unit 107 receives the integrated value calculated by the generated code amount calculation unit 105, and sends it to the intra-picture encoding unit 101 or the inter-picture prediction encoding unit 102. Send an abort instruction signal. Further, the abort control unit 107 also transmits an abort instruction signal to the picture control unit 110. The abort instruction signal is information indicating that the encoding has been aborted, and in what number picture the encoding has been aborted.

  FIG. 5A and FIG. 5B are flowcharts illustrating an example of a procedure related to an encoding abort process and an encoding control process associated therewith in the present embodiment. Specifically, when the encoding of the I picture or the P picture is interrupted in the middle, another picture encoded after the picture for which the encoding has been interrupted is the I or P for which the encoding has been terminated. The encoding of the other picture is controlled so as not to refer to the picture.

  First, in step S501, the generated code amount calculation unit 105 outputs the encoded data supplied from the selector 104 to the stream output unit 106 via the buffer unit 112, and also includes the amount of encoded data (generated code amount). ) Is calculated. Next, in step S502, the generated code amount calculation unit 105 compares the calculated integrated value with a predetermined value to determine whether or not to perform the encoding truncation process in the middle of the I picture or P picture. decide. Specifically, when the integrated amount exceeds a predetermined value, it is determined that the abort process is performed. If it is determined that the truncation process is not performed as a result of this determination, the abort control unit 107 transmits an abort instruction signal to the intra-picture encoding unit 101 or the inter-picture prediction encoding unit 102 and the picture control unit 110. do not do.

  On the other hand, if it is determined in step S502 that the abort process is to be performed, the process proceeds to step S503. In step S503, the abort control unit 107 receives the integrated value from the generated code amount calculation unit 105, and transmits an abort instruction signal to the intra-picture coding unit 101 or the inter-picture prediction coding unit 102. Further, an abort instruction signal is transmitted to the picture control unit 110. The picture control unit 110 that has received the abort instruction signal sends a control signal to the picture type setting unit 103 to forcibly change the picture type of the subsequent picture so as not to refer to the picture whose encoding has been aborted. Alternatively, a control signal is sent to the reference picture setting unit 113 and the picture type resetting unit 111 to forcibly change a previously encoded picture stored in the buffer unit 112 to be a reference picture. Such picture control accompanying the termination of encoding in step S503 will be further described below with reference to the flowchart of FIG. 5B and FIG. 2 showing an example of a picture reference / reference relationship.

  2A to 2D are diagrams illustrating an example of a reference / reference relationship between pictures. 2A shows a reference / referenced relationship between pictures when no truncation processing occurs, and FIGS. 2B to 2D show a reference / referenced relationship between pictures when truncation processing occurs.

  In the arrangement of pictures as shown in FIG. 2A, when the truncation process does not occur, the second picture 202 and the third picture 203 refer to the first picture 201. However, if truncation processing occurs in the first picture 201 and the second picture 202 and the third picture 203 refer to the first picture 201 that has deteriorated greatly, the temporal correlation becomes low, and the coding is not performed. Efficiency will decrease. In this way, it is determined in step S511 whether there is a picture that refers to a picture that has undergone the truncation process. If such a picture exists, the picture control process is performed in accordance with the conditions of steps S512, S514, and S516.

  In step S511, the picture control unit 110 determines whether or not the picture to be coded refers to a picture that has been coded. This determination is based on the picture type to be set to the picture to be encoded and the picture that has been encoded and stored based on the integrated value received when the generated code amount calculation unit 105 determines the execution of the abort process. Based on the information.

Then, when the encoding target picture refers to a picture whose encoding has been terminated,
(1) The reference picture is changed (2) The picture type of the encoding target picture is changed, so that the encoding target picture is controlled so as not to refer to the picture whose encoding has been terminated.

In the case of the example illustrated in FIG. 2A, in step S511, the picture control unit 110 detects that the second picture 202 refers to the first picture 201 in which the truncation processing has occurred.
In step S512, the picture control unit 110 determines whether there is another picture that can be referred to by the second picture 202.

In the example of FIG. 2B, the picture control unit 110 can refer to the fourth picture 204 temporally before the first picture with the same picture type (P) as the first picture, instead of the first picture. It is shown that it is determined that.
In this case, in step S513, the picture control unit 110 sends a control signal to the reference picture setting unit 113 so that the second picture 202 refers to the fourth picture 204. Accordingly, the reference picture setting unit 113 changes the reference picture for the second picture 202 from the first picture 201 to the fourth picture 204. This change can be made by changing the reference picture ID. The inter-picture prediction encoding unit 102 acquires the decoding result of the fourth picture 204 from the local decoding unit 108 and encodes the second picture 202.

  If it is determined in step S512 that there is no other picture that can be referred to, in S514, the picture control unit 110 determines whether or not the picture type of the current picture to be encoded can be changed to an I picture for intra-picture encoding.

  If it can be changed, the picture control unit 110 sends a control signal to the picture type setting unit 103 in S515 to forcibly change the picture type to an I picture. For example, in the example of FIG. 2A, the third picture 203 (P picture) is determined to have no other pictures that can be referred to in S512. Further, the third picture 203 can be changed to an I picture. Therefore, as shown in FIG. 2C, the picture type of the third picture 203 is forcibly changed to a picture (I picture) based on the intra-picture coding method. That is, the third picture 203 is encoded by the intra-picture encoding unit 101.

  If there is no other picture that can be referred to and the picture type cannot be changed, the picture control unit 110 searches for a coded picture stored in the buffer unit 112 that has not been terminated in step S516. To do. In this case, it goes without saying that a picture that is temporally close to the encoding target picture is searched. Then, it is determined whether there is a picture that is not referenced (non-reference picture) and can be changed to a reference picture among pictures obtained by the search.

  Here, for convenience, in the example of FIG. 2A, it is assumed that the second picture 202 is a picture that has no other picture that can be referred to and whose picture type cannot be changed. Then, it is assumed that the fifth picture 205, which is the immediately preceding B picture, can be changed to a non-reference picture and a reference picture.

  In this case, in step S517, the picture control unit 110 sends a control signal to the picture type resetting unit 111 to forcibly change the picture type of the preceding picture. In the example illustrated in FIG. 2D, the fifth picture 205 is not used as a reference picture but is encoded as a B picture. However, since the truncation process has been performed on the first picture 201, the code indicating the picture type in the encoded stream stored in the buffer unit 112 is changed and changed as a reference picture. As a result, the second picture 202 can refer to the fifth picture 205 instead of the first picture 201. In this way, during a period in which the B picture that should have originally referred to the first picture 201 is encoded, a picture other than the aborted picture is referred to.

As a result, the picture whose image quality has been degraded due to the truncation is limited to only the first picture 201 that has been censored, and the picture type and reference picture are changed in the encoding of subsequent pictures such as the second picture 202 and the third picture 203. . Therefore, pictures with temporal correlation can be referred to, and coding efficiency can be improved.
Note that the determination method in steps S512, S514, and S516 can be determined as appropriate. Further, the order of priority of determination and processing may not be in the order shown in FIG. 5B.

  As shown in the flowchart of FIG. 5B, in the present embodiment, in encoding that refers to a picture that has been subjected to encoding truncation, first, truncation occurs in the frame memory in the inter-picture predictive encoding unit 102. It is determined whether there is another reference picture that has not been received. If the other reference picture is suitable for prediction, the process of S513 is performed. On the other hand, if there is no reference picture suitable for prediction in the frame memory, it is determined whether or not the picture to be encoded can be changed to an I picture. If the I picture can be changed, the process of S515 is performed. If the I picture cannot be changed, it is determined whether or not there remains an encoded B picture that can be reset to the reference B picture in the buffer unit 112. judge. If there is a picture that can be changed to the reference B picture, the process of S517 is performed. Note that if none of the determination conditions is satisfied, encoding may be performed while referring to a picture in which encoding has been aborted.

  Note that the priority order of the above-described determination conditions is an example of the embodiment, and other priority orders may be adopted or the priority order may be adaptively set due to group of picture (GOP) configuration restrictions or code amount restrictions. You may add the process which changes.

  In the present embodiment, an example of a moving image encoding apparatus that inputs a video signal from the input unit 100 has been described. However, the present invention may be applied to an imaging apparatus that further includes an imaging unit that captures a subject and converts it into a video signal. .

  As described above, in the present embodiment, since a picture greatly deteriorated by the truncation process is not treated as a reference picture, the temporal correlation can be maintained high, and encoding can be performed efficiently.

(Other embodiments according to the present invention)
Each means constituting the moving picture coding apparatus and each step of the moving picture coding method in the embodiment of the present invention described above can be realized by operating a program stored in a RAM or ROM of a computer. This program and a computer-readable recording medium recording the program are included in the present invention.

  Further, the present invention can be implemented as, for example, a system, apparatus, method, program, or recording medium. Specifically, the present invention may be applied to a system including a plurality of devices. The present invention may be applied to an apparatus composed of a single device.

  Note that the present invention may supply a software program (in the embodiment, a program corresponding to the flowcharts shown in FIGS. 5A and 5B) for realizing the functions of the above-described embodiments directly or remotely to the system or apparatus. Including. This includes the case where the system or the computer of the apparatus is also achieved by reading and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, and the like.

  Examples of the recording medium for supplying the program include a flexible disk, a hard disk, an optical disk, and a magneto-optical disk. Further, there are MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD-R) and the like.

  As another program supply method, there is a method of connecting to a homepage on the Internet using a browser of a client computer. The computer program itself of the present invention or a compressed file including an automatic installation function can be downloaded from the homepage by downloading it to a recording medium such as a hard disk.

  It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  As another method, the program of the present invention is encrypted, stored in a recording medium such as a CD-ROM, distributed to users, and encrypted from a homepage via the Internet to users who have cleared predetermined conditions. Download the key information to be solved. It is also possible to execute the encrypted program by using the key information and install the program on a computer.

  Further, the functions of the above-described embodiments are realized by the computer executing the read program. Furthermore, based on the instructions of the program, an OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments can be realized by the processing.

  As another method, the program read from the recording medium is first written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Then, based on the instructions of the program, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are also realized by the processing.

DESCRIPTION OF SYMBOLS 10 Moving image encoding apparatus 100 Input part 101 Intra picture encoding part 102 Inter picture prediction encoding part 103 Picture type setting part 104 Selector 105 Generated code amount calculation part 106 Stream output part 107 Abort control part 108 Local decoding part 110 Picture Control unit 111 Picture type resetting unit 112 Buffer unit

Claims (4)

Setting means for setting a picture type at the time of encoding for each picture constituting a moving image;
Encoding means for compressing and encoding each picture constituting the moving picture by either an intra-picture encoding scheme or an inter-picture predictive encoding scheme according to the picture type set by the setting means;
When the integrated value of the generated code amount of the first picture encoded by the encoding means exceeds a predetermined value, the encoding compression rate is increased from the middle of the first picture, and the first picture Instruction means for instructing the encoding means to reduce the subsequent generated code amount of the picture;
In accordance with an instruction from the instruction unit, a second encoded image is encoded after the first picture and is encoded with reference to the first picture using the inter-picture predictive encoding method. A moving image coding apparatus comprising: a changing unit that changes a picture type set by the setting unit so that a picture is encoded using the intra-picture encoding method. Setting means for setting a picture type at the time of encoding for each picture constituting a moving image;
Moving picture coding comprising: coding means for compressing and coding each picture constituting the moving picture by either an intra-picture coding scheme or an inter-picture predictive coding scheme according to a picture type set by the setting means An apparatus control method comprising:
The instruction means increases the coding compression rate from the middle of the first picture when the integrated amount of the generated code quantity of the first picture encoded by the encoding means exceeds a predetermined value. An instruction step for instructing the encoding means to reduce the subsequent generated code amount of the first picture;
The changing means has a relationship of being encoded after the first picture in accordance with the instruction and being encoded with reference to the first picture using the inter-picture predictive encoding scheme. And a changing step of changing a picture type set by the setting means so that the picture is encoded using the intra-picture encoding method. A program for causing a computer to execute each step of the control method according to claim 2 . A computer-readable recording medium storing a program for causing a computer to execute the steps of the control method according to claim 2 .

Images