JP2006311135A - Moving image encoding device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To attain efficient reencoding by suppressing deterioration in picture quality due to reencoding. <P>SOLUTION: A moving image encoding device stores reencoding control information required for reencoding in an invalid packet of a transport stream to perform two-pass variable rate through a recording medium 3, and when performing the reencoding, redistributes encoding quantity to the whole one-pass recording signal to suppress deterioration in picture quality due to reencoding and attain efficient reencoding. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a moving image encoding apparatus that performs video signal re-encoding processing using an encoding method such as MPEG.

  2. Description of the Related Art Conventionally, video signal recording devices and playback devices that employ an encoding method such as MPEG have been studied. As for MPEG, the video standards are ISO-IEC1172-2, ITU-TH. 262 / 1SO-IEC13818-2, the system standard is ITU-TH. 222 / ISO-IEC13818-1 is described in detail. In order to convert the rate of the video stream encoded by this encoding method, a method of decoding the stream and decoding the output video signal again can be considered. The rate conversion is usually aimed at conversion to a low rate, and the rate is set according to the capacity of the recording medium and the recording time.

  In MPEG2, VBV is defined not only for fixed transfer rate (CBR) data but also for variable transfer rate (VBR) data, so that it can be defined in two ways to support various applications. Yes. This regulation is distinguished by the value of vbv_delay, which is when vbv_delay is not all the maximum value oxffff or when vbv_delay of each picture is oxffff. When vbv_delay is oxffff, the data input rate for the VBV is determined by the VBV buffer occupancy, and the peak rate Rmax when the VBV buffer is empty, and zero when there is no empty. In the initial state, the VBV buffer occupancy is zero, and decoding is started when the buffer is full.

  In particular, at a low rate, in order to suppress degradation of image quality as much as possible, encoding at a variable rate that varies the encoding rate according to the difficulty level of the video signal is effective. As an encoding method using a variable rate, one-pass variable rate encoding is conceivable for an apparatus such as an optical disk recorder that requires real-time performance, and is proposed in Japanese Patent Laid-Open No. 9-23423 (Patent Document 1). In this encoding, the difficulty level of an input image is obtained every predetermined time, and an assigned code amount is determined based on this, and encoding is performed for a certain time according to the capacity of the target recording medium. The relationship between the difficulty level of the input image and the assigned code amount is standardized in advance by statistical processing or the like using a general input signal.

  In an authoring system or the like that does not require real-time performance, a two-pass encoding method disclosed in Japanese Patent Application Laid-Open No. 6-141298 (Patent Document 2), that is, a video signal recorded in advance on a recording medium. A method is employed in which provisional encoding is performed on the video signal, an assigned code amount corresponding to the difficulty level of the video signal is calculated, and encoding is performed again using this result.

  As the MPEG stream, a programming stream (PS) used in DVD or the like and a transport stream (TS) of the MPEG2 multiplexing standard are used in BS digital broadcasting or the like. Transport streams have also been adopted for disc media such as Blu-ray. As for the video recording apparatus, for example, a hybrid component device including both an optical disk drive and a hard disk drive has become common.

  By the way, when using a method of decoding an encoded video bitstream for rate conversion, MPEG-coding the video signal again using a video signal encoding device, and outputting the bitstream, There is concern about deterioration. As a countermeasure, it is conceivable to apply 1-pass variable rate coding to the decoded video signal. However, in the one-pass variable rate encoding, it is difficult to accurately allocate the code amount to the input video when the difficulty level of the input video cannot be predicted. For example, when a video having a high difficulty level is continuously input at the end of a certain time, the image quality may be deteriorated as compared with the fixed transfer rate encoding when the video is encoded.

As a method of rate conversion, for example, in a hybrid component device, a method of recording at a high rate once on a hard disk and converting to a rate according to the remaining capacity of the optical disk medium to be recorded and the recording time at another time can be considered. When such a configuration is adopted, high-quality rate conversion is possible by performing two-pass variable encoding. In particular, efficient variable rate control is possible by calculating the local target code amount, etc., from the rate converted from the encoded information for all recorded video signals, and re-encoding based on this. Become. However, for this purpose, it is necessary to secure the calculation time after the first pass. In addition, it is necessary to manage encoding information for the first pass, storage of control information for the second pass, synchronization with the video signal, and the like.
Japanese Patent Laid-Open No. 9-23423 JP-A-6-141298

  The present invention has been made in view of the above-described problems of the prior art. When a video signal once encoded is re-encoded to a predetermined rate, the entire recorded signal is recorded. It is an object of the present invention to provide a moving picture coding apparatus that can perform efficient rate conversion by redistributing code amounts.

  The moving image encoding apparatus according to the first aspect of the present invention encodes an input video signal at a predetermined first encoding rate during pass 1 recording, and feeds back a decoded video signal fed back during pass 2 recording. A video encoder for generating a video ES stream by encoding at a second encoding rate indicated by the re-encoding control information and generating the encoding information, and a video ES stream from the video encoder multiplexed with an audio ES stream And generating a transport stream, and at the time of pass 1 recording, a multiplexing processing unit for writing the encoded information in the invalid packet for each picture, and a recording medium for recording the transport stream output from the multiplexing processing unit And the transport at the time of pass 1 recording recorded on the recording medium at the time of pass 2 recording. Re-encoding control information for each picture is calculated from all the encoding information for each picture written in the invalid packet of the stream, and the re-encoding control is performed on the contents of the invalid packet in which the original encoding information has been written. A re-encoded information generation unit that replaces the information with the information and re-records the information on the recording medium, reads the transport stream recorded on the recording medium at the time of pass 1 recording, and records the video ES stream and audio ES The stream is separated and output, and at the same time, the separation processing unit that extracts the re-encoding control information for each picture from the invalid packet, and the video ES stream is decoded to generate the decoded video signal at the time of pass 2 recording. In addition, the re-encoding control information for each picture extracted by the separation processing unit is output and converted from the sequence at the time of encoding, and the decoded image is converted. In synchronism with the signal is obtained both a video decoder that outputs to the video encoder.

  According to a second aspect of the present invention, in the moving picture coding apparatus according to the first aspect, the invalid packet for each picture is set to a position before the head packet of a packet of a video ES stream in which the invalid packet is multiplexed. It is what.

  According to the moving image encoding apparatus of the present invention, when pass 2 recording is performed by re-encoding and recording a video signal encoded at a predetermined rate at pass 1 recording, the video signal is encoded and recorded at pass 1 recording. Efficient rate conversion can be performed by redistributing the code amount over the entire recording signal.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  (First Embodiment) FIG. 1 is a block diagram showing the configuration of a moving picture coding apparatus according to one embodiment of the present invention. The moving image encoding device has a two-pass encoding function, and includes a video encoder 1, a multiplexing processing unit 2, a recording medium 3, a re-encoding information generation unit 4, a separation processing unit 5, And a video decoder 6.

  In the first (pass 1) encoding, the video encoder 1 inputs an external video signal and recording mode information for the a port. The recording mode information includes identification information of one of normal recording / pass 1 recording / re-encoding (pass 2) recording that does not presuppose rate conversion for the video encoder 1 and encoded average rate information. The video encoder 1 outputs the encoded video ES stream and the encoded information to the multiplexing processing unit 2. In pass 1, re-encoding control information is not input, and code amount control is a process inside the video encoder 1.

  Here, “pass 1 recording” is a process from initial encoding, recording of encoded information, and re-encoded information generation / recording on the premise of re-encoding. “Re-encoding (pass 2) recording” is a process from decoding of a stream generated in pass 1 and re-encoding using re-encoding control information to recording of the generated stream. “Normal recording” is an encoding recording process that does not assume re-encoding, that is, does not generate re-encoded information.

  The multiplexing processing unit 2 receives the video ES stream, the audio ES stream, and the encoded information, generates a transport stream (TS), and records it on the recording medium 3. When the recording mode information is “pass 1 recording”, the input encoded information is multiplexed into the transport stream by writing into invalid packets. Here, the recording medium 3 is an optical disk, a hard disk, or the like, but may have a plurality of these configurations.

  When the recording mode information is “pass 1 recording”, the re-encoded information generation unit 4 uses all the encoded information for each picture written in the invalid packet of the transport stream of pass 1 recorded on the recording medium 3. Re-encoding control information for each picture is calculated, and the contents of the invalid packet in which the original encoding information has been written are replaced with the re-encoding control information and recorded on the recording medium 3.

  On the other hand, when the recording mode information is “pass 2 recording”, the separation processing unit 5 reads the transport stream (TS) recorded on the recording medium 3 during “pass 1 recording”, and the video ES stream and audio The ES stream is separated and output. When the recording mode information is “pass 2 recording”, re-encoding control information selected from the encoding average rate set in the recording mode information is extracted from the invalid packet and output to the video decoder 6.

  The video decoder 6 receives the video ES stream, performs a decoding process, and outputs a decoded video signal. Further, when the recording mode information is “pass 2 recording”, the video decoder 6 inputs the re-encoding control information separated by the separation processing unit 5, and re-encodes the control information for each picture from the sequence at the time of encoding. Video output and conversion, and output in synchronization with the decoded video signal.

  In “pass 2 recording”, a decoded video signal is input from the video decoder 6 to the video encoder 1 through the b port, and re-encoding control information is also input. When the recording mode information is “pass 2 recording”, the video encoder 1 performs code amount control by this re-encoding control information, re-encodes the decoded video signal, and outputs a video ES stream. During this “pass 2 recording”, the video encoder 1 does not output encoded information. When the recording mode information is “pass 2 recording”, the multiplexing processing unit 2 inputs the re-encoded video ES stream and the audio ES stream, multiplexes them, and generates a transport stream (TS) again. And recorded on the recording medium 3.

  FIG. 2 is a block diagram showing a detailed internal configuration of the video encoder 1 in the above embodiment. The video encoder 1 includes a first image memory 11, a subtractor 12, a DCT unit 13, a quantization unit 14, a VLC unit 15, an activity detection unit 16, a code amount control unit 17, and encoding information. A generation unit 18, a motion compensation prediction unit 19, an inverse quantization unit 20, an inverse DCT unit 21, an adder 22, and a second image memory 23 are provided.

  The input video signal of the a port is encoded when “pass 1 recording” is encoded, and the video signal input to the b port from the video decoder 6 when the re-encoding process is executed with “pass 2 recording”. It is inputted and stored in the image memory 11. The first image memory 11 stores a plurality of frames of image data. The stored image data is processed in blocks of luminance data (Y) in units of 16 × 16 pixels and color data (Cb / Cr) in units of 16 × 8 pixels. Process as a block of × 8 pixels.

  Similarly, the blocked image data is reproduced by locally decoding the image data read from the first image memory 11 and the frame (field) read from the second image memory 23 by the motion compensation prediction unit 19. A motion vector for prediction between frames (inter-field) is detected from the image data. The motion compensation prediction unit 19 further performs motion compensation from the motion vector and the reproduced image data read from the second image memory 23, and outputs motion compensation image data, a motion vector, and a prediction mode.

  The subtracter 12 receives the image data read from the first image memory 11 and the motion compensated video data output from the motion compensation prediction unit 19 and performs subtraction between them to output difference image data. The DCT unit 13 receives difference image data, performs DCT (Discrete Cosine Transform), and outputs DCT coefficients. The quantization unit 14 receives DCT coefficients, performs quantization, outputs quantized image data, and outputs quantization information.

  The VLC unit 15 receives the quantized data, the motion vector, and the prediction mode, outputs the video ES stream by variable-length encoding them, and outputs the generated code amount and picture type for each picture. Here, as the generated code amount in units of pictures, for example, the total sum of the code amounts in units of a predetermined block (16 × 16 pixels) obtained by variable length coding is used. In addition, since I and P pictures require a reproduced image to be used later as reference data for motion compensation prediction, inverse quantization and local decoding of inverse DCT are performed.

  The activity detection unit 16 calls image data from the first image memory 11 and calculates and outputs the sum of activities for each picture. An activity indicates a change in a block unit in the screen. FIG. 3 is a diagram illustrating an example of activity calculation, in which a change is obtained in units of 8 × 8 pixel blocks. That is, an absolute value difference (Absolute Difference) between pixels (Pixels) adjacent to each other in the horizontal direction and the vertical direction is calculated, and a total for each block unit is calculated. The activity (Act) increases when the change in the block unit in the screen is large, and the activity decreases when the change in the block unit in the screen is small. Then, the activity detection unit 16 performs the above calculation for the entire area of the picture, and calculates and outputs the total sum for the entire picture. The unit block of activity (Act) is a block (8 × 8 pixels) which is the minimum unit of MPEG encoding.

  The code amount control unit 17 that is characteristic as a component in the video encoder 1 in the moving picture encoding apparatus of the present embodiment inputs when the recording mode information is “normal recording” or “pass 1 recording”. Code amount control is performed from the generated code amount / picture type and activity, and when the recording mode information is “pass 2 recording”, control is performed based on the target code amount for each input picture and the recoding control information of the picture type. Is added to control the code amount, and the quantization control information is output to the quantization unit 14.

  The generated code amount and the quantization scale are almost inversely proportional, and the quantization scale is decreased to increase the generated code amount, and the quantization scale is increased to decrease the code amount. Also, because the P picture is predicted from the previous I picture or P picture, and the B picture is predicted from the I picture and P picture on both sides in time, other pictures are linked with the deterioration of the I picture. to degrade. Therefore, the picture quality of the I picture is the most important and the code must be well allocated. Therefore, for the purpose of suppressing image quality deterioration of the reference I picture and P picture, re-encoding is performed with the same picture type as the picture type at the time of “pass 1 recording” at the time of “pass 2 recording”.

  The encoding information generation unit 18 having the same characteristics calculates an average quantization scale by inputting the quantization information for each picture when the recording mode information is “pass 1 recording”, and inputs the input activity and generated code amount. / Constructs and outputs encoded information together with picture type. As the quantization information, for example, an in-plane average quantization scale is used. When the recording mode information is “normal recording” or “pass 2 recording”, no output is performed.

  The inverse quantization unit 20 performs inverse quantization with the quantized data as an input, and outputs DCT coefficients. The inverse DCT unit 21 performs inverse DCT with a DCT coefficient as an input, and outputs a differential decoded image. The adder 22 adds the difference decoded image and the motion compensated image data output from the motion compensation prediction unit 19 as inputs, and outputs decoded image data. The second image memory 23 stores decoded image data.

  FIG. 4 is a block diagram showing an internal configuration of the multiplexing processing unit 2 in the above embodiment. The multiplexing processing unit 2 includes an encoded information TS packet generation unit 31, a TS packet generation unit 32, and a TS_MUX unit 33.

  When the recording mode information is “pass 1 recording”, the encoded information TS packet generation unit 31 receives the encoded information as input and configures and outputs the encoded information in an invalid packet. The TS packet generator 32 receives the input video ES stream, audio ES stream, etc. as input, and generates and outputs TS packets. The TS_MUX unit 33 multiplexes these TS packets, outputs a transport stream (TS), and records it on the recording medium 3. FIG. 5 is a diagram showing an example of a packet insertion position. V0, V1, etc. indicate video data packets, A0, A1, etc. indicate audio data packets, and N indicates an invalid packet. As shown in FIG. 5, by placing an invalid packet N before the beginning V0 of a video packet of the same picture as the encoded information, management of the video stream and the encoded information can be facilitated. Note that when the recording mode information is “normal recording” or “pass 2 recording”, a packet composed of encoded information is not generated and is not multiplexed in the transport stream.

  FIG. 6 is a diagram showing the structure of a TS packet. The MPEG2 TS multiplexing is described in detail in the standard, but the invalid packet generated by the encoded information TS packet generator 31 is NullPacketID (0x1fff) in the packet ID (PID) and the unit start display is '01. ',' 01 '(only payload) is set in the adaptation field control, and encoded information is configured in the payload as shown in FIG. The data identification is for identifying whether the payload of the invalid packet is composed of encoded information or re-encoding control information, or is a payload of a normal invalid packet. When the payload of the invalid packet is encoding information or re-encoding control information, a picture type, generated code amount, quantization information, and activity are set as the picture unit encoding information.

  FIG. 8 is a block diagram showing an internal configuration of the re-encoding information generation unit 4 in the above embodiment. The re-encoded information generation unit 4 includes an encoded information separation processing unit 41, a buffer memory 42, a re-encoding information calculation unit 43, and a re-encoding control information multiplexing processing unit 44.

  The encoded information separation processing unit 41 calls a transport stream (TS) recorded on the recording medium 3, detects invalid packets, extracts encoded information, and stores it in the buffer memory 42. The re-encoding information calculation unit 43 generates re-encoding control information for each rate set for each picture based on the encoding information of all pictures and stores the re-encoding control information in the buffer memory 42. The re-encoding control information multiplexing processing unit 44 calls the transport stream from the recording medium 3, detects an invalid packet in which the encoding information is configured, and replaces the payload of the same picture with the configuration shown in FIG. The above process is performed only when the recording mode information is “pass 1 recording” to prepare for the re-encoding process of “pass 2 recording”.

  Here, the set rate identification is for identifying a re-encoding control information area for each set rate. There are various methods for calculating re-encoded information. Basically, the amount of generated code of each picture, the average quantization scale, and the activity are weighted for each picture type, and the average over the previous time is calculated. Is the calculation standard. On the other hand, the above parameters of each picture are compared to calculate the code amount distribution ratio. Then, the target code amount is calculated by multiplying the set rate.

  FIG. 10 is a diagram showing a target code amount in units of frames at each encoding average rate when M = 3. Usually, a larger amount of code is allocated to the reference I and P pictures, but when the set average rate is low, weighting is performed so as to reduce the difference in the target code amount from the I and P pictures in order to suppress deterioration of the B picture. I do.

  FIG. 11 is a block diagram showing the internal configuration of the separation processing unit 5 of the above embodiment. The separation processing unit 5 includes a TS_DEMUX unit 51, a re-encoding control information extraction unit 52, and an ES stream extraction unit 53.

  The TS_DEMUX unit 51 reads the transport stream recorded on the recording medium 3, detects and separates video data packets, audio data packets, and invalid packets and outputs them. The re-encoding control information extraction unit 52 performs re-encoding based on the average rate of re-encoding set in the input recording mode and the set rate identification of the invalid packet in which the input re-encoding control information is configured. Extract and output control information. The ES stream extraction unit 53 extracts the video ES stream and the audio ES stream from the input video data packet and audio data packet, respectively, and outputs them. In TS_DEMUX 51, when the re-encoding average rate is the same as when “pass 1 recording” is performed, that is, when rate conversion is not necessary, it is possible to remove invalid packets and record them on the recording medium 3.

  FIG. 12 is a block diagram showing the internal configuration of the video decoder 6 in the above embodiment. The video decoder 6 includes a VLD unit (variable length code decoder) 61, an inverse quantization unit 62, an inverse DCT unit 63, an adder 64, a motion compensation prediction unit 65, a buffer memory 66, and re-encoding. And a control information rearranging unit 67.

  The video ES stream is input to the VLD unit 61. The VLD unit 61 decodes a variable-length code for this video ES stream for each block, and sends motion vector information and quantized image data to the inverse quantization unit 62. Output. The inverse quantization unit 62 inversely quantizes the quantized image data and outputs the DCT image data (DCT coefficient) to the inverse DCT unit 63. The inverse DCT unit 63 performs an inverse DCT process with the DCT coefficient as an input, and outputs difference image data. This difference image data is input to the adder 64. In the adder 64, the difference image data and the image data from the motion compensation prediction unit 65 are added and output as a decoded video signal and stored in the buffer memory 66.

  The motion compensation prediction unit 65 performs motion compensation from the decoded image data stored in the buffer memory 66 and the motion vector information from the VLD unit 61, and outputs a decoded video signal. The re-encoding control information rearrangement unit 67 receives the re-encoding control information and the recording mode information, and records the re-encoding control information in the buffer memory 66 when the recording mode information is “pass 2 recording”. Then, as shown in FIG. 13, the re-encoding control information synchronized with the image of the decoded video signal to be output is read from the buffer memory 66 and output in the same picture sequence as the decoded video data. In FIG. 13, I0 is an I picture, P3, P6, etc. are P pictures, B1, B2, etc. are B pictures.

  According to the moving picture coding apparatus of the present embodiment, “pass 2 recording” in which a once-encoded video bitstream is decoded, the video signal is MPEG-encoded again using a video decoder, and a bitstream is output. When performing re-encoding from all the encoded information for each picture with re-encoding with an appropriate code amount distribution according to the content of the moving image at the time of re-encoding, degradation of image quality at the time of rate conversion can be suppressed, By configuring the encoded information and the re-encoding control information as invalid packets of the MPEG2 transport stream, each information can be easily managed and accessed by being positioned before the head of the video stream packet of each picture. Also, even if a stream that has been re-encoded by the moving picture encoding apparatus of the present embodiment is reproduced by another normal function video decoder, the invalid packet is skipped and normal reproduction is possible. When encoding control information is generated, re-encoding starts immediately even if the rate is set at the time of re-encoding by calculating and storing in advance the set multiple re-encoding average rates. In addition, when the re-encoding is not performed, the code amount of the transport stream can be reduced by removing the multiplexed invalid packet.

The block diagram which shows the structure of the moving image encoder of one embodiment of this invention. The block diagram which shows the structure of the video encoder in the said embodiment. The figure which shows an example of an activity calculation method. The block diagram which shows the structure of the multiplexing process part in the said embodiment. The figure which shows an example of the insertion position of TS packet. The figure which shows an example of TS packet structure. The figure which shows an example which comprises encoding information in a payload. The block diagram which shows the structure of the re-encoding information generation part in the said embodiment. The figure which shows an example of a structure of the replacement of a payload. The figure which showed the target code amount of the frame unit by each encoding average rate when M = 3. The block diagram which shows the structure of the separation process part in the said embodiment. The block diagram which shows the structure of the video decoder in the said embodiment. The figure which shows a picture sequence.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Video encoder 2 Multiplexing process part 3 Recording medium 4 Re-encoding information production | generation part 5 Separation part 6 Video decoder 11, 23 Image memory 12 Subtractor 13 DCT part 14 Quantization part 15 VLC part 16 Activity detection part 17 Code amount Control unit 18 Encoding information generation unit 19, 65 Motion compensation prediction unit 20, 62 Inverse quantization unit 21, 63 Inverse DCT unit 22, 64 Adder 31 TS packet generation unit 32 Encoding information TS packet generation unit 33 TS_MUX unit 41 Encoding information separation processing unit 42, 66 Buffer memory 43 Re-encoding information calculation unit 44 Re-encoding control information multiplexing processing unit 51 Re-encoding control information extracting unit 52 TS_DEMUX unit 53 ES stream extracting unit 61 VLD unit 67 Re Reorder encoding control information

Claims (2)

The second encoding that the encoded video signal is encoded at a predetermined first encoding rate at the time of pass 1 recording and the re-encoding control information that is fed back the decoded video signal that is fed back at the time of pass 2 recording. A video encoder for encoding at a rate to generate a video ES stream and for generating encoded information;
A multiplexing process unit that multiplexes a video ES stream from the video encoder with an audio ES stream to generate a transport stream, and writes the encoded information for each picture in an invalid packet during pass 1 recording;
A recording medium for recording a transport stream output from the multiplexing processing unit;
At the time of pass 1 recording, re-encoding control information for each picture is calculated from all the encoding information for each picture written in the invalid packet of the transport stream recorded at the time of pass 1 recording on the recording medium, A re-encoding information generating unit that replaces the content of the invalid packet in which the encoding information is written with the re-encoding control information and re-records it on the recording medium,
At the time of pass 2 recording, the transport stream recorded on the recording medium at the time of pass 1 recording is read out, the video ES stream and the audio ES stream are separated and output, and at the same time, re-encoding control information for each picture is recorded. A separation processing unit for extracting from the invalid packet;
During pass 2 recording, the video ES stream is decoded to generate a decoded video signal, and the re-encoding control information for each picture extracted by the separation processing unit is output and converted from the sequence at the time of encoding, A video encoding device comprising: a video decoder that outputs to the video encoder together with a decoded video signal. 2. The moving picture encoding apparatus according to claim 1, wherein the invalid packet for each picture is set to a position before a head packet of a packet of a video ES stream in which the invalid packet is multiplexed.

Images