JP5853757B2 - Moving picture coding apparatus and moving picture coding method - Google Patents

Description

  The present invention relates to a moving picture coding apparatus and a moving picture coding method that perform low delay coding and high image quality coding and generate respective coded data.

  In recent years, an environment in which a viewer can freely select and view a favorite program from a large number of real-time broadcasts by digital terrestrial television broadcasting or Internet live distribution is being prepared.

  In the real-time broadcasting described above, the time lag between the distribution source and the distribution destination is one problem. In digital broadcasting, video shot on the distribution source side is encoded, and the encoded data is transferred to the viewer through radio waves.

  On the distribution destination side (viewer side), the received encoded data is decoded and output to a monitor such as a TV. Since time required for encoding and decoding is generated in digital broadcasting, this processing time causes a time lag compared to conventional analog broadcasting.

  In addition, in Internet live delivery, in addition to the delay due to the processing time described above, network delay is also added to the time lag. Particularly in live broadcast programs, it is required to minimize the delay time required for encoding / transfer.

  FIG. 1 is a diagram for explaining the time lag. As shown in FIG. 1, encoding delay, communication delay, and decoding delay cause time lag.

  Thereby, in digital broadcasting, there is a problem that smooth exchange with the relay destination cannot be performed due to the influence of the time lag, which may cause discomfort to the viewer. In addition, in Internet distribution and videophone, there is a problem that communication is difficult to take due to the delay described above. Therefore, it is required to reduce the time lag as much as possible.

  On the other hand, in moving picture coding, there is a unit called GOP (Group Of Picture) as a unit in which a plurality of pictures are combined into one, and it is used as a starting point for random access or editing during reproduction.

  As a general GOP structure, an IBBP structure is often used, and since there is a B picture for backward reference, a so-called coding delay occurs. In order to reduce the coding delay, a GOP structure called an IPPP structure is mainly used.

  FIG. 2 is a diagram for explaining the IBBP structure and the IPPP structure. In the example shown in FIG. 2, in the IBBP structure, the B picture is backward-referenced and is encoded after the backward picture, so that an encoding delay for two pictures occurs.

  On the other hand, in the IPPP structure, encoding can be performed in the input order of images (pictures), so that no encoding delay occurs. However, in the IPPP structure, since backward reference is not performed, the compression rate is reduced as compared with the IBBP structure, and the image quality is affected.

  In the following, the GOP structure of I and P pictures is referred to as a low-delay version, and the GOP structure of I, P, and B pictures is referred to as a high-quality version. Also, encoding with the low delay version is called low delay encoding, and encoding with the high quality version is called high quality encoding.

  Here, some users place importance on image quality and others place importance on delay time, so the user can freely select a low-delay encoded stream and a high-quality encoded stream. Hope you can. The distribution side can provide a flexible program by transmitting both low-delay encoding and high-quality encoding data, but it is necessary to generate two types of encoded data simultaneously.

  As a technique for generating a plurality of encoded data, for example, a first encoding is performed as an I picture and a P picture, and a locally decoded image of an image encoded by the first encoding is used as a predicted image. There is a technique for performing the encoding of 2 and the third encoding.

JP 2001-177841 A

  However, according to the prior art, the bitstream encoded by the second encoding or the third encoding cannot be independently decoded, and the bitstream encoded by the first encoding It is necessary to decode in combination.

  In addition, when generating encoded data that can be decoded independently of the low-delay version and the high-quality version described above, since the low-delay version and the high-quality version are encoded, There has been a problem that the amount of processing required for the conversion increases.

  Accordingly, the disclosed technology has been made in view of the above problems, and a moving image that can reduce the amount of processing required for encoding when generating two types of encoded data of a low-delay version and a high-quality version. An object is to provide an image encoding device and a moving image encoding method.

A moving image encoding apparatus according to an aspect of the disclosure encodes a moving image with a low-delay version indicating a GOP structure of an I picture, a P picture, and a high quality version indicating a GOP structure of an I picture, a P picture, and a B picture. An intra-prediction unit that processes a common I picture in the low-delay version and the high-quality version, and the low-delay version and the high-quality for a reference P picture to be referred to An inter prediction unit that processes a common P picture in a version and processes the high-quality version of a B picture using auxiliary information including motion vector information of an unreferenced non-reference P picture processed in the low delay version; .

  According to the disclosed technology, when generating two types of encoded data of a low-delay version and a high-quality version, it is possible to reduce the amount of processing required for encoding.

The figure for demonstrating a time lag. The figure for demonstrating an IBBP structure and an IPPP structure. 1 is a block diagram illustrating an example of a functional configuration of a moving image encoding device according to Embodiment 1. FIG. FIG. 3 is a diagram for explaining an outline of encoding processing according to the first embodiment. FIG. 3 is a diagram illustrating an example of a reference area of a low-delay version and a high-quality version in the first embodiment. The figure which shows the comparison result of the operation amount of the conventional method and the method 1. FIG. 5 is a flowchart illustrating an example of a picture encoding process according to the first embodiment. 9 is a flowchart illustrating an example of an encoding process for a non-reference P picture with low delay in the first embodiment. 9 is a flowchart illustrating an example of MB processing for a non-reference P picture according to the first embodiment. 9 is a flowchart illustrating an example of inter prediction processing for a non-reference P picture according to the first embodiment. 6 is a flowchart illustrating an example of an encoding process for a high-quality B picture according to the first exemplary embodiment. 9 is a flowchart illustrating an example of MB processing for a B picture according to the first embodiment. 9 is a flowchart illustrating an example of inter prediction processing for a B picture according to the first embodiment. FIG. 9 is a block diagram illustrating an example of a functional configuration of a moving image encoding device according to a second embodiment. FIG. 10 is a diagram illustrating an example of a reference area of a low-delay version and a high-quality version in the second embodiment. 10 is a flowchart illustrating an example of a picture encoding process according to the second embodiment. 9 is a flowchart illustrating an example of an encoding process for a low-delay non-reference P picture and a high-quality B picture according to the second embodiment. 9 is a flowchart illustrating an example of MB processing by an inter prediction unit according to the second embodiment. 10 is a flowchart illustrating an example of inter prediction processing according to the second embodiment.

  Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. In the moving picture encoding apparatus described below, there are cases where encoding is performed in units of pictures and processing is performed in units of macroblocks (MB). First, a video encoding apparatus that performs encoding in units of pictures will be described.

  Hereinafter, a GOP structure including an I picture and a P picture is referred to as a low-delay version, and a GOP structure including an I picture, a P picture, and a B picture is referred to as a high-quality version. Also, encoding with the low delay version is called low delay encoding, and encoding with the high quality version is called high quality encoding.

[Example 1]
<Configuration>
FIG. 3 is a block diagram illustrating an example of a functional configuration of the video encoding device 10 according to the first embodiment. 3 includes a storage unit 101, an overall control unit 102, an inter prediction unit 103, an intra prediction unit 104, a determination unit 105, a selector (SEL) 106, and a prediction error generation. Unit 107, transform / quantization unit 108, entropy coding unit 109, inverse quantization / inverse transform unit 110, decoded pixel generation unit 111, and deblocking filter unit 112.

  In the example illustrated in FIG. 3, the moving image encoding apparatus 10 is configured to perform high-quality encoding after performing low-delay encoding on one picture. And high-quality encoding may be performed in parallel.

  The storage unit 101 is a memory that stores data related to encoding. The storage unit 101 is, for example, an SDRAM as an external memory. The storage unit 101 includes an original image storage unit 121, an auxiliary information storage unit 122, a reference image storage unit 123, and a stream storage unit 124. Each unit of the storage unit 101 may be provided with a respective storage area.

  The original image storage unit 121 stores the input original image. The auxiliary information storage unit 122 stores auxiliary information including a motion vector of an unreferenced non-reference P picture encoded by the low-delay encoding by the inter prediction unit 103.

  The reference image storage unit 123 stores a reference image used in the inter prediction unit 103. The stream storage unit 124 stores a stream encoded by low-delay encoding and a stream encoded by high-quality encoding. Data stored in each storage unit will be described later.

  The overall control unit 102 appropriately controls low delay encoding and high image quality encoding. For example, the overall control unit 102 encodes an I picture that can be commonly used in the low-delay version and the high-quality version during low-delay encoding in the intra-prediction unit 104, and performs encoding at the time of high-quality encoding. The intra prediction unit 104 is notified that there is no need.

  The overall control unit 102 determines the picture type of the current image, and rearranges the pictures in the encoding order when the high-quality version is used.

  The inter prediction unit 103 performs a motion search using the original image read from the original image storage unit 121 and the reference image read from the reference image storage unit 123, and finds an optimal motion vector. The inter prediction unit 103 outputs a prediction pixel signal whose motion is compensated by the motion vector to the selector 106.

  The inter prediction unit 103 encodes a P picture that is common to the low-delay version and the high-quality version with respect to the reference P picture to be referred to. The inter prediction unit 103 encodes a high-quality B picture using auxiliary information including motion vector information of an unreferenced non-reference P picture encoded in the low-delay version.

  The reference P picture represents a P picture that serves as a reference picture, and the non-reference P picture represents a P picture that does not serve as a reference picture. The auxiliary information includes a motion vector, a SAD calculation result, and an encoding cost calculated from the motion vector and the SAD calculation result.

  Since the inter prediction unit 103 shares the reference image between the low-delay version and the high-quality version, the low-delay version encodes a picture that becomes a B picture in the high-quality version as a non-reference P picture.

  When the inter prediction unit 103 encodes a high-quality version of a B picture, for the forward reference, the inter prediction unit 103 determines the reference area using auxiliary information of the non-reference P picture encoded in the low-quality version. No need to search. For example, the reference area can be determined using a motion vector included in the auxiliary information. Thereby, the process concerning a search can be reduced.

  The intra prediction unit 104 encodes a common I picture in the low-delay version and the high-quality version. Therefore, since the intra prediction unit 104 uses the I-picture encoding result in common for the low-delay version and the high-quality version, it is not necessary to code the I-picture twice for the low-delay version and the high-quality version. .

  The intra prediction unit 104 selects an optimum prediction pattern from various intra-screen prediction patterns using, for example, an original image read from the original image storage unit 121 and a local decoded image as I picture encoding. The intra prediction unit 104 outputs the prediction pixel signal generated by the selected prediction pattern to the selector 106.

  The determination unit 105 selects the one with the smaller code amount in the inter prediction and the intra prediction, and controls the selector 106 to output the selected prediction pixel signal.

  The selector 106 outputs the prediction pixel signal of either the inter prediction unit 103 or the intra prediction unit 104 to the prediction error generation unit 107 under the control of the determination unit 105.

  The prediction error generation unit 107 acquires the original image and the prediction pixel signal output from the selector 106, and generates a prediction error signal by calculating a difference between the original image and the prediction pixel signal.

  The transform / quantization unit 108 performs orthogonal transform processing such as discrete cosine transform on the prediction error signal to generate orthogonal transform coefficients, and quantizes the generated orthogonal transform coefficients.

  The entropy coding unit 109 entropy codes the quantized transform coefficient generated by the transform / quantization unit 108. The entropy coding unit 109 also entropy codes the information on the motion vector used in the inter prediction unit 103.

  The entropy encoding unit 109 stores the entropy encoded encoded stream (encoded data) in the stream storage unit 124.

  The inverse quantization and inverse transform unit 110 performs an inverse quantization process on the quantized orthogonal transform coefficient, and performs an inverse orthogonal transform process on the inversely quantized transform coefficient. The inverse quantization and inverse transform unit 110 outputs a signal generated by inverse transform to the decoded pixel generation unit 111. Here, the generated signal is a signal corresponding to the prediction error signal.

  The decoded pixel generation unit 111 generates a decoded pixel by adding the signal generated by the inverse quantization and inverse conversion unit 110 and the predicted pixel signal output by the selector 106. The generated decoded pixel is output to the deblocking filter unit 112.

  The deblocking filter unit 112 applies, for example, an H.D. The deblocking filter process prescribed | regulated to H.264 is performed. The decoded pixel subjected to the deblocking filter is stored in the reference image storage unit 123 as a reference image (reference picture).

  Each unit of the prediction error generation unit 107 to the deblocking filter unit 112 performs processing with a low-delay version and then performs processing with a high-quality version. The stream storage unit 124 stores a low-delay version encoded stream and a high-quality version encoded stream. The entropy encoding unit 109 stores, in the stream storage unit 124, encoded data that can be used in common for the low-delay version and the high-quality version so that each encoded stream is included.

  Thereby, the moving image encoding device 10 can generate both the low-delay version and the high-quality version of the encoded stream while reducing the encoding process.

<Outline of encoding process>
Next, an outline of the encoding process between the low-delay version and the high-quality version when encoding in units of pictures will be described. FIG. 4 is a diagram for explaining the outline of the encoding process in the first embodiment. FIG. 4A shows an original image input at regular intervals. Time T represents the input time of the original image.

  FIG. 4B shows an example of the encoding result when the original image is encoded with the low delay version. In the example shown in FIG. 4B, the GOP structure is an IPPP structure. The encoding time represents the time when the original image is encoded with reference to the time T.

  As shown in FIG. 4B, in the low delay version of the IPPP structure, when an original image is input, encoding is possible without delay. A common I picture and a common P picture shown in FIG. 4B represent pictures common to the low-delay version and the high-quality version. The common P picture is a reference P picture.

  Here, the inter prediction unit 103 stores the motion search result as auxiliary information in the storage unit 101 when processing the low-delay version of the non-reference P picture (motion search, motion compensation).

  For example, at the encoding time 1, the inter prediction unit 103 reads the original image at time T = 1 and the image at time T = 0 as a reference image from the storage unit 101, and performs motion search and motion compensation. At this time, the inter prediction unit 103 writes the motion search result as auxiliary information 1 in the storage unit 101. The inter prediction unit 103 performs the same process as the process of the encoding time 1 for the non-reference P picture (the encoding times 2, 4, 5, 7, 8, 10, 11,...).

  FIG. 4C shows an example of an encoding result when the original image is encoded with a high quality version. In the example shown in FIG. 4C, the GOP structure is an IBBP structure. Since the common I picture and the common P picture are encoded in the low-delay version, this encoding process may not be performed in the high-quality version.

  In high-quality encoding, the original image is buffered and the order is changed before encoding. Also, the original image that was a non-reference P picture in the low-delay version becomes a B picture in the high-quality version. This is because a common reference image is used for the low-delay version and the high-quality version.

  Here, when the inter prediction unit 103 processes the high-quality version of the B picture, for the forward reference, the inter prediction unit 103 performs the forward search by using the auxiliary information obtained when processing the non-reference P picture in the low-delay version. You don't have to.

  For example, at the encoding time 4 shown in FIG. 4C, the inter prediction unit 103 stores the original image 1, only the reference region indicated by the auxiliary information 1, and the image at time T = 3 as the reference image. Read from 101, and perform backward search and motion compensation. At this time, the inter prediction unit 103 may not perform the forward search by using the auxiliary information 1.

  If the difference value between the reference area and the original image is included as auxiliary information, reading of the reference area becomes unnecessary. This is because the reference area can be generated from the original image and the difference value.

  The inter prediction unit 103 performs a process similar to the process of the encoding time 4 on the B picture (encoding times 5, 7, 8, 10, 11,...) And omits the process related to the forward search. Can do.

  FIG. 5 is a diagram illustrating an example of reference regions of the low delay version and the high quality version in the first embodiment. 5A shows a group of pictures that are subjected to low delay encoding, and FIG. 5B shows a group of pictures that are encoded with high image quality.

  In the example shown in FIG. 5A, the image of the input original image number (hereinafter also referred to simply as “number”) 4 is a non-reference P picture, and the image obtained by decoding the image of number 3 (hereinafter also referred to as “decoded image”). Refers to an area.

  In the example shown in FIG. 5B, the image of number 1 is a B picture, and the areas of the decoded image of number 0 and the decoded image of number 3 are referred to. At this time, the inter prediction unit 103 does not need the forward search as described above, and only needs to read the reference region using the auxiliary information. For the backward search, the inter prediction unit 103 reads a part of the decoded image of number 3 as a search area, and performs a motion search in this search area.

  As a result, for forward search, only the reference area needs to be read, so that the memory bandwidth to be used can be reduced. In addition, when including the difference value of a reference area and an original image in auxiliary information, it is not necessary to read the area to refer.

<Comparison>
Next, the amount of calculation between the conventional method and the method 1 according to the first embodiment is compared. FIG. 6 is a diagram illustrating a comparison result of calculation amounts between the conventional method and the method 1. The conventional method is a method of encoding each of the low-delay version and the high-quality version independently.

  It is assumed that the amount of calculation in each process is 1, and 1 GOP includes 15 pictures. For example, it is assumed that GOP of IBBPBBPBBPBBPBB is processed for high image quality, and IPPPPPPPPPPPPPPP GOP is processed for low-delay version.

  First, in the low-delay version of the conventional method, the one-sided reference of the inter prediction process is 14 because the number of P is 14, and the intra prediction process is 15 because 15 pictures are processed. The transformation, quantization processing, inverse transformation, inverse quantization processing, entropy coding processing, and deblocking filter processing are 15 because the processing is performed for the number of pictures.

  In the high-quality version of the conventional method, the single-sided reference in the inter prediction process is 4 because the number of P is 4, and the double-sided reference is 10 because the number of B is 10. When this is converted into one side, the processing amount is 4 + 10 × 2 = 24. The processing after intra prediction is the same as the low-latency version.

  The low delay version of Method 1 has the same processing amount as the low delay version of the conventional method. However, the processing amount of the high-quality version of Method 1 can be greatly reduced compared to the high-quality version of the conventional method. That is, in the high-quality version of Method 1, the single-sided reference of the inter prediction process is performed for the P picture in common with the low-delay version, and the process is not necessary. 10.

  The double-sided reference in the inter prediction process is 0 because only a backward reference is required even for a B picture. When these are converted into one side, the processing amount is 10, which indicates that the amount of processing is greatly reduced as compared with 24 of the conventional method.

  The amount of processing after the intra prediction process is 10 because the common I picture and the common P picture (reference P picture) are shared with the low-delay version and only the B picture can be processed.

  According to FIG. 6, it can be said that the technique 1 in the first embodiment can significantly reduce the processing amount as compared with the conventional technique.

<Operation>
Next, the operation of the moving image processing apparatus 10 in the first embodiment will be described. FIG. 7 is a flowchart illustrating an example of a picture encoding process according to the first embodiment. Here, N in the flowcharts in FIG. 7 and subsequent figures indicates the input original image number, and m indicates the reference P picture interval in the GOP structure.

  In step S101, the moving image encoding apparatus 10 inputs the original image N and writes it in the storage unit 101 (original image storage unit 121).

  In step S102, the overall control unit 102 determines whether or not the I picture is encoded. If it is an I picture encoding (step S102-YES), the process proceeds to step S103, and if it is not an I picture encoding (step S102-NO), the process proceeds to step S104.

  In step S <b> 103, each of the intra prediction unit 104, the prediction pixel generation unit 107 to the deblocking filter unit 112 encodes a common I picture with low delay / high image quality for the original image N.

  In step S104, the overall control unit 102 determines whether or not the reference P picture is encoded. If it is encoding of a reference P picture (step S104-YES), it will progress to step S105, and if it is not encoding of a reference P picture (step S104-NO), it will progress to step S106.

  In step S105, each unit such as the inter prediction unit 103, the intra prediction unit 104, the prediction pixel generation unit 107 to the deblocking filter unit 112 encodes a common P picture with low delay / high image quality with respect to the original image N. Do.

  In step S106, the inter prediction unit 103 encodes the low-delay version of the non-reference P picture for the original image N. Details of this processing will be described later with reference to FIGS.

  In step S107, the overall control unit 102 determines whether N−m is 0 or more. If Nm is 0 or more (step S107-YES), the process proceeds to step S108, and if Nm is less than 0 (step S107-NO), the process is terminated.

  In step S108, the inter prediction unit 103 performs high-quality B-picture encoding on the original image N-m. Details of this processing will be described later with reference to FIGS. When the processing for one picture is completed, processing similar to that shown in FIG. 7 is performed for the next picture. Note that the example shown in FIG. 7 shows processing in which the low-delay version and the high-quality version are performed in parallel.

(Encoding process of non-reference P picture)
FIG. 8 is a flowchart illustrating an example of an encoding process for a non-reference P picture with low delay in the first embodiment. In step S201 illustrated in FIG. 8, the inter prediction unit 103 performs processing on the macroblock (MB) in the picture. Details of this processing will be described later with reference to FIG.

  In step S202, the inter prediction unit 103 determines whether the processing has been completed for all MBs. If all the MBs have been processed (step S202—YES), the process is terminated. If all the MBs have not been processed (step S202—NO), the process returns to step S201, and the next MB is processed. .

  FIG. 9 is a flowchart illustrating an example of MB processing for a non-reference P picture according to the first embodiment. In step S301 illustrated in FIG. 9, the inter prediction unit 103 reads the original image N from the original image storage unit 121.

  In step S302, the inter prediction unit 103 performs an inter prediction process. Details of this processing will be described later with reference to FIG.

  In step S303, the intra prediction unit 104 performs an intra prediction process. In step S304, the determination unit 105 determines which one of the encoding cost of the prediction pixel processed by the inter prediction unit 103 and the encoding cost of the prediction pixel processed by the intra prediction unit 104 is smaller, and selects the selector 106. Control. The selector 106 outputs the predicted pixel instructed by the determination unit 105.

  In step S305, each unit of the prediction error generation unit 107 to the decoded pixel generation unit 111 encodes information on the MB or transform coefficient.

  FIG. 10 is a flowchart illustrating an example of inter prediction processing for a non-reference P picture according to the first embodiment. In step S <b> 401 illustrated in FIG. 10, the inter prediction unit 103 reads the forward reference image from the reference image storage unit 123.

  In step S402, the inter prediction unit 103 performs forward motion search on the read reference image search range.

  In step S403, the inter prediction unit 103 calculates the encoding cost at the optimum reference position.

  In step S404, the inter prediction unit 103 writes and stores the motion vector information at the optimum reference position, the SAD calculation result, and the coding cost in the auxiliary information storage unit 122 as auxiliary information.

(B picture encoding process)
FIG. 11 is a flowchart illustrating an example of an encoding process for a high-quality B picture according to the first embodiment. In step S501 illustrated in FIG. 11, the inter prediction unit 103 performs processing on a macroblock (MB) in a picture. Details of this processing will be described later with reference to FIG.

  In step S502, the inter prediction unit 103 determines whether the processing has been completed for all MBs. If all MBs have been processed (step S502—YES), the process is terminated. If all MBs have not been processed (step S502—NO), the process returns to step S501 and the next MB is processed. .

  FIG. 12 is a flowchart illustrating an example of MB processing for a B picture according to the first embodiment. In step S601 illustrated in FIG. 12, the inter prediction unit 103 reads the original image Nm from the original image storage unit 121.

  In step S602, the inter prediction unit 103 performs an inter prediction process. Details of this processing will be described later with reference to FIG.

  In step S603, the intra prediction unit 104 performs an intra prediction process. In step S604, the determination unit 105 determines which one of the encoding cost of the prediction pixel processed by the inter prediction unit 103 and the encoding cost of the prediction pixel processed by the intra prediction unit 104 is smaller. The determination unit 105 controls the selector 106 so that the prediction pixel with the smaller encoding cost is output. The selector 106 outputs the predicted pixel instructed by the determination unit 105.

  In step S605, each unit of the prediction error generation unit 107 to the decoded pixel generation unit 111 encodes information about the MB or transform coefficient.

  FIG. 13 is a flowchart illustrating an example of inter prediction processing for a B picture according to the first embodiment. In step S701 illustrated in FIG. 13, the inter prediction unit 103 reads the backward reference image from the reference image storage unit 123.

  In step S702, the inter prediction unit 103 performs a backward motion search for the search range of the read reference image.

  In step S703, the inter prediction unit 103 reads auxiliary information corresponding to the MB of the processed image from the auxiliary information storage unit 122.

  In step S704, the inter prediction unit 103 calculates a coding cost using the reference region obtained from the read auxiliary information and the reference region obtained in the backward motion search.

  As described above, according to the first embodiment, when generating two types of encoded data of the low-delay version and the high-quality version, it is possible to reduce the processing amount for encoding.

[Example 2]
Next, the moving picture coding apparatus 20 according to the second embodiment will be described. In the second embodiment, the low-delay version and the high-quality version are encoded in parallel in MB units.

<Configuration>
FIG. 14 is a block diagram illustrating an example of a functional configuration of the video encoding device 20 according to the second embodiment. In the example illustrated in FIG. 14, the storage unit 201, the overall control unit 202, the inter prediction unit 203, the first intra prediction unit 204-1, the second intra prediction unit 204-2, the first determination unit 205-1, and the second determination A section 205-2, a first selector 206-1, and a second selector 206-2.

  In addition, the moving image encoding device 20 includes a first prediction pixel generation unit 207-1, a second prediction pixel generation unit 207-2, a first transformation, a quantization unit 208-1, a second transformation, and a quantization unit 208-. 2, the first entropy encoding unit 209-1, the second entropy encoding unit 209-2, the first inverse quantization unit, the inverse transform unit 210-1, the second inverse quantization, the inverse transform unit 210-2, the first The first decoded pixel generation unit 211-1, the second decoded pixel generation unit 211-2, and the deblocking filter unit 212 are included.

  The first units shown in FIG. 14 perform low-delay encoding, and the second units perform high-quality encoding. The configuration other than the overall control unit 202 and the inter prediction unit 203 illustrated in FIG. 14 is different from the first embodiment in that the low-delay version and the high-quality version are encoded, but the content of the processing is the same as that of the first embodiment. Therefore, the description thereof is omitted.

  The overall control unit 202 notifies the inter prediction unit 203 of the two images to be processed and the respective reference images. Further, the overall control unit 202 notifies the inter prediction unit 203 whether the processing target image is a low-delay version or a high-quality version.

  The inter prediction unit 203 performs a low-delay version inter prediction process or a high-quality version inter prediction process on each of the two images notified by the overall control unit 202. In the inter prediction unit 203, the processing contents of the low-delay version and the high-quality version are the same as in the first embodiment.

  At this time, the inter prediction unit 203 makes the reference picture referenced by the low-delay version P picture the same as the reference picture referenced backward by the high-quality version B picture.

  As a result, the same picture is referred to in the low-delay encoding and the high-quality encoding that are processed in parallel, so that it is possible to reduce the memory band for reading the reference image.

<Outline of encoding process>
Next, an outline of encoding processing between the low-delay version and the high-quality version when parallel encoding is performed in units of MB will be described with reference to FIG.

  For example, the overall control unit 202 reads the original image at time T = 1, 4 and the decoded image at time T = 0, 3 that is the reference image at the encoding time 4 shown in FIG. To control.

  At this time, the overall control unit 202 instructs the inter prediction unit 203 to perform high-quality encoding of the original image at time T = 1 and low-delay encoding of the original image at time T = 4.

  The inter prediction unit 203 performs high-quality encoded B picture processing on the original image MB at time T = 1, and does not perform low-delay encoding on the original image MB at time T = 4. The reference P picture is processed.

  At this time, the inter prediction unit 203 writes and stores the low-delay version motion search result as auxiliary information 4 in the auxiliary information storage unit 222.

  Also, the inter prediction unit 203 reads the auxiliary information 1 from the auxiliary information storage unit 222 in the high-quality-encoded B picture process, omits the forward search process, and performs the backward search process. The inter prediction unit 203 commonly uses a decoded image of time T = 3 as a reference image in the low-delay version and the high-quality version.

  FIG. 15 is a diagram illustrating an example of reference regions of the low-delay version and the high-quality version in the second embodiment. In the example shown in FIG. 15, high-quality encoding of the original image of number 1 and low-delay encoding of the original image of number 4 are encoded in MB units in parallel.

  As shown in FIG. 15, the picture referred to by the number 4 original picture is the reference picture number 3, and the picture referenced backward by the number 4 original picture is the number 3 reference picture and has the same search range. become. The reason why the search ranges are the same is that, in parallel processing in MB units, MBs at the same position are processed in the low-delay version and the high-quality version.

  Therefore, as shown in FIG. 15, the reference picture of number 3 can be used simultaneously in the low-delay version and the high-quality version, so that the reference picture is compared with the case where it is processed separately for each picture shown in FIG. It is possible to reduce the memory bandwidth for reading pictures.

<Operation>
Next, the operation of the video encoding device 20 in the second embodiment will be described. FIG. 16 is a flowchart illustrating an example of a picture encoding process according to the second embodiment. Note that the processing illustrated in FIG. 16 illustrates a case where low-delay encoding and high-quality encoding are performed in parallel in units of MB.

  In step S801, the moving image encoding apparatus 20 inputs the original image N and writes it in the storage unit 201 (original image storage unit 221).

  In step S802, the overall control unit 202 determines whether or not the I picture is encoded. If it is an I picture encoding (step S802-YES), the process proceeds to step S803, and if it is not an I picture encoding (step S802-NO), the process proceeds to step S804.

  In step S803, each unit such as the intra prediction unit 204, the prediction pixel generation unit 207, and the deblocking filter unit 212 performs encoding of a common I picture with low delay / high image quality on the original image N. For example, the first processing unit encodes a common I picture.

  In step S804, the overall control unit 202 determines whether or not the reference P picture is encoded. If it is the encoding of the reference P picture (step S804-YES), the process proceeds to step S805, and if it is not the encoding of the reference P picture (step S804-NO), the process proceeds to step S806.

  In step S805, each unit such as the inter prediction unit 203, the intra prediction unit 204, the prediction pixel generation unit 207 to the deblocking filter unit 212 encodes a common P picture with low delay / high image quality for the original image N. Do. For example, the first processing unit encodes a common P picture.

  In step S806, the inter prediction unit 203 determines whether Nm is 0 or more. If Nm is greater than or equal to 0 (step S806: YES), the process proceeds to step S807, and if Nm is less than 0 (step S806: NO), the process proceeds to step S808.

  In step S807, the inter prediction unit 203 and the first processing unit encode a low-delay non-reference P picture with respect to the original image N, and the inter prediction unit 203 and the second processing unit perform the original image N Encode a high-quality B picture for -m. Details of this processing will be described later with reference to FIGS.

  In step S808, the inter prediction unit 203 and the first processing unit encode a low-delay non-reference P picture with respect to the original image N. This process is the same as the process of FIGS.

(Parallel encoding processing of non-reference P picture and B picture)
FIG. 17 is a flowchart illustrating an example of an encoding process for a low-delay non-reference P picture and a high-quality B picture according to the second embodiment. In step S901 illustrated in FIG. 17, the inter prediction unit 203 performs processing on the macroblock (MB) in the picture. Details of this processing will be described later with reference to FIG.

  In step S902, the inter prediction unit 203 determines whether the processing has been completed for all MBs. If all MBs have been processed (step S902-YES), the process is terminated. If all MBs have not been processed (step S902-NO), the process returns to step S901, and the process is performed on the next MB. .

  FIG. 18 is a flowchart illustrating an example of MB processing by the inter prediction unit 203 according to the second embodiment. In step S1001 illustrated in FIG. 18, the inter prediction unit 203 reads the original image N from the original image storage unit 221.

  In step S1002, the inter prediction unit 203 reads the original image N-m from the original image storage unit 221.

  In step S1003, the inter prediction unit 203 performs an inter prediction process. Details of this processing will be described later with reference to FIG.

  In step S1004, the first intra prediction unit 204-1 performs an intra prediction process on a low-delay non-reference P picture.

  In step S1005, the second intra prediction unit 204-2 performs an intra prediction process on a high-quality B picture.

  In step S1006, the first determination unit 205-1 encodes the prediction pixel coding cost for the non-reference P picture processed by the inter prediction unit 203 and the prediction pixel code processed by the first intra prediction unit 204-1. It is determined which of the conversion costs is lower, and the first selector 206-1 is controlled. The first selector 206-1 outputs the predicted pixel instructed by the first determination unit 205-1.

  In step S1007, the second determination unit 205-2 performs the encoding cost of the prediction pixel for the B picture processed by the inter prediction unit 203 and the encoding cost of the prediction pixel processed by the second intra prediction unit 204-2. Which is smaller is controlled, and the second selector 206-2 is controlled. The second selector 206-2 outputs the predicted pixel instructed by the second determination unit 205-2.

  In step S1008, each processing unit of the first prediction error generation unit 207-1 to the first decoded pixel generation unit 211-1 performs encoding of information on the MB of the non-reference P picture or a transform coefficient.

  In step S1009, each processing unit of the second prediction error generation unit 207-2 to the second decoded pixel generation unit 211-2 performs encoding of information regarding the MB of the B picture or a transform coefficient.

  FIG. 19 is a flowchart illustrating an example of inter prediction processing according to the second embodiment. In step S1101 illustrated in FIG. 19, the inter prediction unit 203 reads from the reference image storage unit 223 a reference image that is commonly used by the low-delay version and the high-quality version (hereinafter also referred to as a common reference image).

  In step S1102, the inter prediction unit 203 performs forward motion search on the search range of the read common reference image for the low-delay version non-reference P picture.

  In step S1103, the inter prediction unit 203 performs a backward motion search for the read range of the common reference image for the high-quality B picture.

  In step S1104, the inter prediction unit 203 reads the auxiliary information corresponding to the high-quality B picture from the auxiliary information storage unit 222.

  In step S1105, the inter prediction unit 203 calculates the coding cost at the optimum reference position for the low-delay version non-reference P picture.

  In step S1106, the inter prediction unit 203 calculates the coding cost at the optimum reference position for the high-quality B picture.

  In step S1107, the inter prediction unit 203 writes the motion vector information at the optimum reference position in the non-reference P picture, the SAD calculation result, and the coding cost as auxiliary information in the auxiliary information storage unit 222 and stores them.

  As described above, according to the second embodiment, when generating two types of encoded data of the low-delay version and the high-quality version, it is possible to reduce the memory bandwidth while reducing the processing amount for encoding.

  Various units other than the storage unit of each embodiment described above can employ various integrated circuits. In addition, a part of each unit described in each embodiment can be a separate integrated circuit. For example, examples of the integrated circuit include ASIC (Application Specific Integrated Circuit) and FPGA (Field Programmable Gate Array). The overall control unit may be implemented by a CPU (Central Processing Unit) or the like.

  Although each embodiment has been described in detail above, it is not limited to a specific embodiment, and various modifications and changes other than the above embodiment are possible within the scope described in the claims. .

In addition, the following additional remarks are disclosed regarding the above Example.
(Appendix 1)
A moving picture coding apparatus for coding a moving picture with a low delay version showing a GOP structure including an I picture, a P picture, and a high quality version showing a GOP structure including an I picture, a P picture, and a B picture. ,
An intra prediction unit for processing a common I picture in the low-delay version and the high-quality version;
Auxiliary information including motion vector information of a non-referenced P picture that is processed in the low-delay version and that is processed in the low-delay version is processed for the reference P picture to be referenced. An inter prediction unit for processing the high-quality version of the B picture;
A video encoding device comprising:
(Appendix 2)
A storage unit for storing the auxiliary information;
The inter prediction unit
Storing the auxiliary information when the non-reference P picture is processed in the low-delay version in the storage unit;
The moving picture coding apparatus according to claim 1, wherein when processing the high-quality B picture, a reference area for forward reference is determined using auxiliary information read from the storage unit.
(Appendix 3)
When encoding the low-delay version and the high-quality version in macroblock units in parallel,
The inter prediction unit
The moving picture coding apparatus according to Supplementary Note 1 or 2, wherein a reference picture referenced by the low-delay version P picture and a reference picture referenced backward by the high-quality version B picture are the same.
(Appendix 4)
The moving image encoding apparatus according to any one of appendices 1 to 3, wherein the auxiliary information includes a motion vector, an SAD operation result, and an encoding cost calculated from the motion vector and the SAD operation result.
(Appendix 5)
Executed by a moving picture coding apparatus that performs coding of a moving picture with a low-delay version indicating a GOP structure including an I picture and a P picture and a high quality version indicating a GOP structure including an I picture, a P picture, and a B picture. A video encoding method comprising:
Intra-coding the I picture common to the low-delay version and the high-quality version,
For the reference P picture to be referred to, inter-coding of the P picture common to the low delay version and the high quality version,
A moving picture coding method including a process of inter-coding the high-quality B picture using auxiliary information including motion vector information of a non-referenced non-reference P picture coded in the low-delay version.
(Appendix 6)
When encoding the low-delay version and the high-quality version in macroblock units in parallel,
The inter encoding process includes:
6. The moving picture coding method according to appendix 5, wherein a reference picture referenced by the low-delay version P picture and a reference picture referenced backward by the high-quality version B picture are the same.

10, 20 Moving picture encoding device 101, 201 Storage unit 102, 201 Overall control unit 103, 203 Inter prediction unit 104 Intra prediction unit 105 Determination unit 204-1 First intra prediction unit 204-2 Second intra prediction unit 205-1 1 determination unit 205-2 second determination unit

Claims (5)

A moving image encoding apparatus that encodes a moving image with a low-delay version indicating a GOP structure of an I picture, a P picture, and a high quality version indicating a GOP structure of an I picture, a P picture, and a B picture,
An intra prediction unit for processing a common I picture in the low-delay version and the high-quality version;
Auxiliary information including motion vector information of a non-referenced P picture that is processed in the low-delay version and that is processed in the low-delay version is processed for the reference P picture to be referenced. An inter prediction unit for processing the high-quality version of the B picture;
A video encoding device comprising: A storage unit for storing the auxiliary information;
The inter prediction unit
Storing the auxiliary information when the non-reference P picture is processed in the low-delay version in the storage unit;
2. The moving picture encoding apparatus according to claim 1, wherein when processing the high-quality B picture, a reference area for forward reference is determined using auxiliary information read from the storage unit. When encoding the low-delay version and the high-quality version in macroblock units in parallel,
The inter prediction unit
3. The moving picture coding apparatus according to claim 1, wherein a reference picture referenced by the low-delay version P picture and a reference picture referenced backward by the high-quality version B picture are the same.   4. The moving picture encoding apparatus according to claim 1, wherein the auxiliary information includes a motion vector, a SAD operation result, and an encoding cost calculated from the motion vector and the SAD operation result. 5. A moving image executed by a moving image encoding apparatus that encodes a moving image using a low-delay version indicating a GOP structure of I pictures and P pictures and a high-quality version indicating a GOP structure of I pictures, P pictures, and B pictures An encoding method comprising:
Intra-coding the I picture common to the low-delay version and the high-quality version,
For the reference P picture to be referred to, inter-coding of the P picture common to the low delay version and the high quality version,
A moving picture coding method including a process of inter-coding the high-quality B picture using auxiliary information including motion vector information of a non-referenced non-reference P picture coded in the low-delay version.

Images