WO2013073422A1

WO2013073422A1 - Video encoding device

Info

Publication number: WO2013073422A1
Application number: PCT/JP2012/078784
Authority: WO
Inventors: 信博知原; 昌史高橋; 山口　宗明; 雅俊近藤
Original assignee: 株式会社日立国際電気
Priority date: 2011-11-17
Filing date: 2012-11-07
Publication date: 2013-05-23
Also published as: JP2013110466A

Abstract

Provided is a video encoding device for carrying out high-quality, low-latency encoding processing by suppressing variations in the amount of code while also enhancing subjective image quality. A video encoding device for encoding video comprising a plurality of images lined up side by side in time series by intra-frame prediction or inter-frame prediction, wherein the plurality of images are segmented into a reference region, which is a region for referencing pixel values in the intra-frame prediction or inter-frame prediction, and a non-reference region, which is a region where the pixel values are not referenced, and the reference region and non-reference region in the plurality of images are determined so that the reference region and non-reference region in the plurality of images alternate at a predetermined interval that is established in advance on the time series. During encoding of the plurality of images, the non-reference region is quantized by coarser steps than the reference region.

Description

Video encoding device

The present invention relates to a technique for encoding a moving image.

In recent years, video coding technology has become an indispensable technology due to the increase in video distribution content due to the development of broadband networks and the use of large-capacity storage media such as DVDs and large-screen video display devices. In addition to the high resolution of the imaging device and the display device, a technology for encoding at a high resolution in the moving image encoding technology is indispensable.

The moving image encoding process is a process of converting an original image (input image) input to the moving image encoding apparatus into a stream having a smaller data amount. For example, as one of moving image encoding technologies capable of encoding with high resolution and high image quality, the international standard H.264 is available. The H.264 / AVC standard (Non-Patent Document 1) exists.

H. In the encoding processing by H.264 / AVC, processing is generally performed in units called macroblocks composed of 16 × 16 pixels with respect to the original image. H. In H.264 / AVC, image prediction is performed using characteristics that image information generally has, that is, characteristics that have high correlation between adjacent pixels and between frames, and furthermore, high-frequency components that are difficult for human vision to perceive changes. Reduce redundant information. Thereby, an input image in a wide transmission band (for example, about 1.5 Gbps in HD-SDI (High Definition Serial Digital Interface)) is applied to a low transmission band (for example, about 15 Mbps in terrestrial digital broadcasting).

By the way, H. There are mainly two prediction methods used in H.264 / AVC encoding: intra prediction and inter prediction. In the in-screen prediction, a plurality of prediction methods are prepared according to combinations of block sizes and prediction directions serving as prediction units. Also in the inter-screen prediction, a plurality of prediction methods are prepared according to the size of a block which is a prediction unit. H. In H.264 / AVC, an encoding method with high image quality and high compression is realized by dynamically selecting these prediction methods according to the target image quality and code amount.

Here, referring to FIG. An outline of H.264 / AVC encoding will be described. FIG. 1 is a diagram illustrating a configuration of an image encoding device that performs H.264 / AVC encoding processing.

In order to perform an encoding process using intra prediction, first, the mode selection unit 930 selects the intra prediction unit 910. Then, the stream 91 is obtained from the original image 90 through the intra prediction unit 910, the orthogonal transform unit 940, the quantization unit 950, and the variable length coding unit 980. In the encoding process using inter-screen prediction, the mode selection unit 930 selects the inter-screen prediction unit 920. Then, the stream 91 is obtained from the original image 90 through the inter-screen prediction unit 920, the orthogonal transform unit 940, the quantization unit 950, and the variable length encoding unit 980.

Next, the quantization executed by the quantization unit 950 will be described.

H. In quantization performed by H.264 / AVC encoding, the quantization granularity is adjusted using the quantization coefficient D84 determined by the encoding control unit 990. If the quantization granularity is small, the code amount is large, but the image is likely to be closer to the original image, and if the quantization granularity is large, the code amount is small but the image is likely to be separated from the original image.

Therefore, H. In H.264 / AVC encoding, encoding with high image quality and high compression efficiency is realized by dynamically selecting the quantization coefficient D84 according to the target image quality and code amount.

The following is H. Each process of H.264 / AVC encoding will be described.

The in-screen prediction unit 910 receives an original image 90 and a reconstructed image 92 that is a peripheral image. The reconstructed image 92 is an image configured by adding the restored difference image 97 output from the inverse orthogonal transform unit 970 and the predicted image 95 output by the mode selection unit 930, and is the same as the original image (that is, It is an image group of a plurality of past frames including a frame (currently being encoded). The reconstructed image 92 referred to in intra-screen prediction is the same frame as the original image 90.

Then, the intra-screen prediction unit 910 selects an appropriate intra-screen prediction mode (intra-screen prediction method) from the original image 90 and the reconstructed image 92 by the intra-screen prediction process, and represents the mode information of the intra-screen prediction mode. An intra-screen prediction information D81, an intra-screen prediction image 93 which is a prediction result, and an intra-screen prediction error representing a difference between the original image 90 and the intra-screen prediction image 93 are generated.

The inter-screen prediction unit 920 receives the input of the original image 90 and the reconstructed image 92 generated from the original image before and after (past or future), and receives the inter-screen prediction information D82, the inter-screen prediction image 94, and the original image An inter-screen prediction error representing a difference between 90 and the inter-screen prediction image 94 is generated.

The mode selection unit 930 uses the intra-screen prediction error input from the intra-screen prediction unit 910 and the inter-screen prediction error input from the inter-screen prediction unit 920 according to the encoding mode selection algorithm, and performs intra-screen prediction and inter-screen prediction. Determine any coding mode of prediction. When the intra prediction is selected, the intra prediction image 93 is output as the prediction image 95, and when the inter prediction is selected, the inter prediction image 94 is output as the prediction image 95.

Note that the encoding mode selection algorithm greatly affects the code amount and image quality of the stream 91, and therefore there are various methods depending on the content of the original image 90 to be encoded and the purpose of video encoding.

The orthogonal transform unit 940 generates a frequency component D83 from the difference image 96 that is a difference between the original image 90 and the predicted image 95 by orthogonal transform processing. Note that the unit of prediction (block) and the unit of orthogonal transformation are not necessarily the same, and the difference image 96 may be further divided and orthogonally transformed.

The quantization unit 950 performs a quantization process from the quantization coefficient D84 input from the encoding control unit 990 and the frequency component D83 input from the orthogonal transform unit 940, and generates a quantized value D85 with a reduced amount of information. Output.

The inverse quantization unit 960 performs an inverse quantization process on the quantized value D85 to generate a restored frequency component D86.

The inverse orthogonal transform unit 970 performs an inverse orthogonal transform process on the reconstructed frequency component D86 to generate a reconstructed difference image 97. Then, the generated restored differential image 97 and the predicted image 95 output by the mode selection unit 930 are added together in the same block, and stored as a reconstructed image 92 in a storage device such as a memory.

The variable length encoding unit 980 encodes the quantized value D85 and the intra-screen prediction information D81 or the inter-screen prediction information D82 into a data string having a smaller data amount, and outputs it as a stream 91.

The stream buffer 1000 once buffers the stream 91, and then outputs it to the transmission path or a subsequent decoder. The code amount of the buffered stream 91 is output to the encoding control unit 990 as code amount information D87.

The encoding control unit 990 determines the quantization coefficient D84 according to the rate control algorithm using the code amount information D87 input from the stream buffer 1000, and outputs this to the quantization unit 950.

Here, since the rate control algorithm based on the control of the quantization coefficient D84 has a great influence on the moving picture coding efficiency, various systems exist depending on the use of the moving picture coding. As a technique for improving moving picture coding efficiency by a rate control algorithm based on the control of the quantization coefficient D84, for example, an image coding technique described in Patent Document 1 can be cited.

Patent Document 1 discloses a method for improving objective image quality by referring to a predicted image closer to an original image, in which the quantization coefficient is reduced in inter-screen prediction by managing the reference relationship in inter-screen prediction in units of pictures. Is disclosed. Further, in Patent Document 1, when a sharp image and a dull image are repeatedly displayed every other frame, a visual characteristic called “repetitive illusion of a dull image” that looks like a sharp image as a whole by human vision is utilized. A method for improving subjective image quality is also disclosed.

FIG. 22 is a diagram for explaining a method for improving the encoding efficiency using the “repetitive illusion of a sharp image” in the image encoding technique described in Patent Document 1. As shown in FIG. 22, when alternately displaying a high-quality image with a reduced quantization coefficient and a low-quality image with a reduced quantization amount by increasing the quantization coefficient every other frame, “sharpness” is displayed. Due to the “repetitive illusion of images”, the entire moving image appears to be equivalent to a high-quality moving image with a small quantization coefficient in human vision.

Therefore, in the image coding technique of Patent Document 1, it is possible to reduce the code amount by increasing the quantization coefficient while maintaining the subjective image quality.

JP 2008-31824 A JP 2008-153802 A JP-A-7-162863

By the way, it is generally known that in moving picture encoding, a delay occurs during encoding and decoding.

After an image is input to the moving image encoding device, the code output from the moving image encoding device is input to the moving image decoding device, and a decoding process is executed to display the same image. The time difference from display to display is the delay time, and in a general moving picture encoding apparatus and moving picture decoding apparatus, the delay is about several hundred milliseconds to several seconds.

In recent years, for the purpose of simultaneous use with uncompressed image data and the like, there is a need for low delay in moving picture encoding and decoding such as a delay time of 16 milliseconds or less.

In order to realize low-delay video encoding and decoding, it is necessary to suppress fluctuations in the amount of data generated. In general, image data is transmitted at a fixed bit rate determined from the data amount of the entire image data, and the transmitted image data is stored and processed in a buffer. When a large amount of code data having a bit rate or higher is generated in this area, data in the buffer is insufficient for processing in this area, and waiting for transmission of code data to the buffer occurs. If all processing is not completed for this area, this area cannot be displayed, resulting in an increase in delay time in displaying the entire image.

That is, the delay time is determined by the time for processing the assumed maximum code generation amount, and this delay time is greatly influenced by the difference between the bit rate and the maximum code generation amount, that is, the variation amount of the generated code amount.

FIG. 23 is a diagram for explaining a code amount variation by quantization control in units of pictures in the image coding technique described in Patent Document 1.

In the image coding technique of Patent Document 1, a non-reference picture having a large quantization coefficient and a reference picture having a small quantization coefficient are determined, and the quantization coefficient is controlled in units of pictures.

For this reason, the code amount of non-reference pictures is significantly smaller than that of reference pictures, and the code amount fluctuation between pictures is large. Therefore, buffering for several pictures is required for non-reference pictures. There was a problem that a delay occurred.

Accordingly, in view of the above problems, the present invention provides a moving image coding capable of performing high-quality and low-delay coding processing by suppressing fluctuations in the code amount while improving the image quality in the moving image coding processing. An object is to provide an apparatus, a moving image encoding method, and a moving image encoding program.

In order to solve the above-described problem, according to one aspect of the present invention, there is provided a moving image encoding apparatus that encodes a moving image including a plurality of images arranged in time series by intra prediction or inter prediction. A plurality of images are divided into a reference region that is a region that refers to a pixel value in the intra prediction or the inter-screen prediction and a non-reference region that is a region that does not refer to a pixel value, and A reference control unit that determines the reference region and the non-reference region in the plurality of images so that the reference region and the non-reference region in the plurality of images are switched at a predetermined interval in FIG. And a quantization unit that performs quantization with a width (gradation) coarser than the reference region for the non-reference region when the image is encoded. There is provided.

According to this configuration, it is possible to encode a moving image with high image quality while suppressing delay by suppressing variation in code amount.

According to another aspect of the present invention, the reference control unit may be configured such that a ratio of the reference area to the non-reference area in a certain unit area in any of the certain unit areas included in the image. The reference area and the non-reference area may be determined so as to be the same.

According to this configuration, it is possible to suppress fluctuations in the code amount within a certain unit area.

According to another aspect of the present invention, the video encoding device further includes an intra-screen prediction unit that performs the encoding by the intra-screen prediction on the plurality of images, and the intra-screen prediction unit includes: In the image, the encoding is performed with reference to only the pixels in the reference area according to the positional relationship between the current macroblock that is the macroblock to be encoded and the reference area or the non-reference area. It may be.

According to this configuration, since only the pixels in the reference area with high image quality are referred to in the intra prediction, it is possible to avoid a decrease in encoding efficiency.

According to another aspect of the present invention, the video encoding device further includes an inter-screen prediction unit that performs the encoding by the inter-screen prediction on the plurality of images, and the inter-screen prediction unit includes: When performing the inter-screen prediction, the motion vector search range may refer to at least one of the plurality of images serving as the reference region.

According to this configuration, since only the pixels in the reference area having high image quality are referred to in inter-screen prediction, it is possible to avoid a decrease in encoding efficiency.

According to the present invention, it is possible to encode a moving image with high image quality while suppressing delay by suppressing fluctuations in code amount.

It is a figure which shows an example of a structure of the moving image encoder which concerns on one Embodiment of this invention. The figure which shows the 1st specific example of the management method of the reference area and non-reference area | region for every picture which the reference control part in the moving image encoder which concerns on one Embodiment performs. The figure which shows the code amount per macroblock line in an encoding object picture. The figure which shows the fluctuation | variation of the code amount for every 1 macroblock line in an encoding object picture. The schematic diagram which shows a response | compatibility with the search range of current MB, and inter-screen prediction restriction | limiting information. The schematic diagram which shows a response | compatibility with the search range of current MB, and inter-screen prediction restriction | limiting information. The schematic diagram which shows a response | compatibility with the search range of current MB, and inter-screen prediction restriction | limiting information. H. The figure which shows the kind of prediction direction in the prediction in a screen with respect to 4x4 block in a H.264 / AVC specification. The figure which shows an example of the restriction | limiting of the prediction mode in a screen in each positional relationship of a reference area and the current macroblock. The figure which shows the 2nd specific example of the management method of the reference area and non-reference area which concern on one Embodiment of this invention. The figure which shows the code amount per picture in an encoding object picture. The figure which shows the fluctuation | variation of the code amount for every picture in an encoding object picture. The schematic diagram which shows a response | compatibility with the search range of current MB, and inter-screen prediction restriction | limiting information. The schematic diagram which shows a response | compatibility with the search range of current MB, and inter-screen prediction restriction | limiting information. The schematic diagram which shows a response | compatibility with the search range of current MB, and inter-screen prediction restriction | limiting information. The figure which shows an example of the restriction | limiting of the prediction mode in a screen in each positional relationship of a reference area and the current macroblock. The figure which shows the 3rd specific example of the management method of the reference area and non-reference area which concern on one Embodiment of this invention. The figure which shows the code amount per macroblock line in an encoding object picture. The schematic diagram which shows a response | compatibility with the search range of current MB, and inter-screen prediction restriction | limiting information. The schematic diagram which shows a response | compatibility with the search range of current MB, and inter-screen prediction restriction | limiting information. Conventional H.264. The block diagram of the image coding apparatus which performs a H.264 / AVC encoding process. The figure explaining the improvement method of the encoding efficiency using the conventional "repetitive illusion of a sharp image". The figure explaining the code amount fluctuation | variation by the quantization control of the conventional picture unit. Configuration diagram of transcoder according to embodiment 1

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings referred to in the following description, the same parts as those in the other drawings are denoted by the same reference numerals.

(Configuration of video encoding device)
FIG. 1 is a diagram illustrating an example of a configuration of a video encoding apparatus according to the present embodiment. The video encoding apparatus 1 includes an intra-screen prediction unit 110, an inter-screen prediction unit 120, a mode selection unit 130, an orthogonal transform unit 140, a quantization unit 150, an inverse quantization unit 160, and an inverse orthogonal transform. Unit 170, variable-length encoding unit 180, encoding control unit 190, stream buffer 200, and reference control unit 210.

Here, since the configuration other than the encoding control unit 190 and the reference control unit 210 is the same as the configuration of the conventional moving image encoding apparatus shown in FIG. 21, it will be briefly described below.

1 can be realized by, for example, LSI (Large Scale Integration), FPGA (Field-Programmable Gate Array), and the like, and various image processing for performing moving image encoding processing is possible. It can be applied to equipment. Further, for example, it can be realized by a configuration similar to that of a general computer such as a memory (not shown) such as a CPU (Central Processing Unit) and a RAM (Random Access Memory), a storage device such as a hard disk, and a network interface. That is, the function of each component of the moving image encoding device 1 can be realized by, for example, the CPU reading and executing a program stored in a hard disk or the like.

The intra-screen prediction unit 110 acquires the original image 10, the reconstructed image 12 that is an image located around the original image 10, and intra-screen prediction restriction information D11 (described later) output from the reference control unit 210. Then, the intra-screen prediction information D1, the intra-screen prediction image 13 that is the prediction result of the intra-screen prediction, and the intra-screen prediction error representing the difference between the original image 10 and the intra-screen prediction image 13 are generated.

The intra-screen prediction information D1 includes an intra-screen prediction block type that is information indicating the block size for which intra-screen prediction has been performed, and intra-screen prediction mode information indicating the direction of intra-screen prediction.

The inter-screen prediction unit 120 includes the original image 10 and a past original image (on the time axis, an original image located in front of the original image 10. The same applies hereinafter) or a future original image (on the time axis, The reconstructed image 12 generated from the original image located after the original image 10 (the same applies hereinafter) and the inter-screen prediction restriction information D12 (described later) output from the reference control unit 210 are acquired, and the inter-screen is acquired. The prediction information D2, the inter-screen prediction image 14 which is the prediction result of the inter-screen prediction, and the inter-screen prediction error representing the difference between the original image 10 and the inter-screen prediction image 14 are generated.

The inter-screen prediction information D2 includes an inter-screen prediction block type indicating a block size when inter-screen prediction is performed, and motion vector information as a result of motion compensation.

The mode selection unit 130 selects a prediction mode (intra-screen prediction) according to the mode selection algorithm from the intra-screen prediction error output from the intra-screen prediction unit 110 and the inter-screen prediction error output from the inter-screen prediction unit 120. Or, prediction between screens) is determined. When “intra-screen prediction” is selected, the intra-screen prediction image 13 is output as the predicted image 15. When “inter-screen prediction” is selected, the inter-screen prediction image 14 is output as the predicted image 15.

The orthogonal transform unit 140 generates a frequency component D3 from the difference image 16 that is a difference between the original image 10 and the predicted image 15 by orthogonal transform processing.

The quantization unit 150 performs a quantization process from the quantization coefficient D4 output from the encoding control unit 190 and the frequency component D3 output from the orthogonal transform unit 140, and obtains a quantization value D5 with a reduced amount of information. Output.

The inverse quantization unit 160 performs an inverse quantization process on the quantization value D5 output from the quantization unit 150 to generate a restored frequency component D6.

The inverse orthogonal transform unit 170 performs an inverse orthogonal transform process on the restoration frequency component D6 output from the inverse quantization unit 160 to generate the restoration difference image 17.

Then, the restored difference image 17 and the predicted image 15 selected by the mode selection unit 130 are added together and stored as a reconstructed image 12 in a storage device such as a memory. In addition, since it is not necessary to obtain the reconstructed image 12 for a “non-reference region” described later, the inverse quantization unit 160 and the inverse orthogonal transform unit 170 may stop processing.

The variable length encoding unit 180 encodes the quantized value D5 and the intra-screen prediction information D1 or the inter-screen prediction information D2 into a data string having a smaller data amount, and outputs the data stream 11 as a stream 11.

The stream buffer 200 acquires the stream 11 output from the variable-length encoding unit 180, buffers the stream 11, and then puts the stream 11 in a first-in first-out manner in a transmission path connected to the moving image encoding apparatus 1, or a subsequent decoder. Output in the method. The code amount (accumulated amount) of the buffered stream 11 is output to the encoding control unit 190 as code amount information D7.

The encoding control unit 190 acquires the code amount information D7 from the stream buffer 200, and calculates the quantization coefficient D4 according to the rate control algorithm. Further, if the region determination result D13 (described later) of the current MB (macroblock, the same applies hereinafter) output from the reference control unit 210 indicates “reference region”, the region is left with the quantization coefficient unchanged. If the determination result D13 indicates “non-reference region”, the offset value is added to the quantization coefficient (or quantization parameter), and the result is output to the quantization unit 150.

Here, the “reference area” is an area that is permitted to be referred to by others, that is, when performing a macroblock prediction process in inter-screen prediction or intra-screen prediction, inter-screen prediction is searched for inter-screen reference. An area that is a range means an area that is the target of pixel reference within the screen in intra prediction, and `` non-reference area '' is an area that is prohibited from being referred to by others. An area that is not used as a search range refers to an area that is not subject to pixel reference within the screen in the intra prediction.

The reference control unit 210 determines the positions of the reference area and the non-reference area for each picture, holds the position management information, and includes the current MB, which is the macro block targeted for prediction processing, in either the non-reference area or the reference area Judge whether or not Then, according to the determination result, an area determination result D13 indicating whether the current MB is included in the reference area or the non-reference area is output to the encoding control unit 190. Further, based on the managed reference area information, the intra-screen prediction restriction information D11 is provided to the intra-screen prediction unit 110, and the inter-screen prediction restriction information D12 is provided to the inter-screen prediction unit 120. This will be described in detail later.

Hereinafter, the reference control unit 210, which is a feature of the video encoding apparatus according to the present embodiment, will be described in detail below with reference to the drawings.

(Reference control unit 210)
[Specific example 1: Right-and-left division of encoding target picture]
FIG. 2 is a diagram for explaining an example of a reference area and non-reference area management method for each picture executed by the reference control unit 210 of the video encoding device 1 according to the present embodiment.

In this specific example, a case will be described in which the reference control unit 210 divides each picture into left and right, determines one as a reference area and the other as a non-reference area, and performs control for switching each picture.

The reference control unit 210 sets the left half of the encoding target picture 403 as a reference area and the right half as a non-reference area in the encoding target picture 403 that is the first picture in time series (display order; the same applies hereinafter). In the subsequent pictures (pictures 402 and 401), the area that was the non-reference area in the previous picture in time series is set as the reference area, and the area that was the reference area is set as the non-reference area. Also, the reference control unit 210 holds position management information indicating the position (left half or right half) of the reference area or non-reference area for each picture for the current and past several pictures. Then, based on this position management information, it is determined whether the current MB being processed is included in the reference region or the non-reference region, and the determination result is output to the encoding control unit 190 as a region determination result D13.

(Effect on image quality)
As a result, the non-reference area and the reference area are alternately displayed on both the left and right sides of the encoding target picture, so that the subjective image quality is maintained using the “repetitive illusion of sharp images”. Is possible. Furthermore, the amount of codes can be reduced.

Note that this specific example is characterized in that the left and right reference areas and the non-reference areas are switched for each picture, and the reference area and the non-reference area in the first picture 403 may be reversed (that is, The right half of the picture 403 may be a reference area and the left half may be a non-reference area).

(Effect on code amount fluctuation)
Next, as described with reference to FIG. 2, a description will be given of code amount fluctuations when the reference control unit 210 divides the encoding target picture into a non-reference region and a reference region on the left and right.

FIG. 3 is a diagram for explaining the code amount per macro block line (hereinafter referred to as “MBL”) in the encoding target picture. FIG. 4 is a diagram showing a change in code amount for each MBL.

As shown in FIG. 3, the moving image encoding process is performed on the encoding target picture 401 in MB units according to the raster scan order indicated by the arrows.

Here, as in this specific example, when the left half of the encoding target picture 401 is a reference area and the right half is a non-reference area, a non-reference area with a small amount of code and a reference area with a large amount of code within 1 MBL The area included in is always constant in any MBL (see FIG. 3). Therefore, in the case of this specific example, as shown in FIG. 4, the code amount is almost the same in 1 MBL units, and the fluctuation between pictures is small, so that the code amount fluctuation can be suppressed to within 1 MBL unit.

Therefore, according to the encoding method of this specific example, it is possible to reduce the capacity of the stream buffer 200 that absorbs code amount fluctuations, and to realize a low delay.

In this specific example, the encoding target picture is divided into left and right, but the present invention is not limited to this. If the ratio of the reference area and the non-reference area included in every fixed number of MBs is always the same, and the non-reference area and the reference area are switched at regular intervals, the “repetitive illusion of sharp images” can be used. It is possible to realize a delay. Therefore, for example, a method of dividing a picture diagonally, a method of dividing a picture into three or more regions, and a method of dividing by a stripe pattern may be used.

(About intra-screen prediction restriction information and inter-screen prediction restriction information)
Here, the intra-screen prediction restriction information D11 and the inter-screen prediction restriction information D12 output by the reference control unit 210 will be described.

As shown in the example of FIG. 2, when the picture area to be encoded is divided into a high-quality reference area and a low-quality non-reference area on the left and right sides, the delay of the code amount is suppressed and the delay is reduced while maintaining the image quality. Is possible. However, on the other hand, when the picture area is divided into a high-quality reference area and a low-quality non-reference area on the left and right sides of the picture, coding efficiency is obtained when pixel values in the non-reference area are referred to in intra prediction and inter prediction. Gets worse.

Therefore, the reference control unit 210 provides intra-screen prediction restriction information D11 to the intra-screen prediction unit 110 for intra-screen prediction of the current MB and inter-screen prediction for inter-screen prediction so that image quality degradation does not propagate from the non-reference region. The inter-screen prediction restriction information D12 is output to the unit 120.

(Restriction of prediction between screens)
First, the inter-screen prediction restriction information D12 that the reference control unit 210 outputs to the inter-screen prediction unit 120 will be described.

The inter picture prediction restriction information D12 includes a reference picture number. The “reference picture number” is a number indicating a reference image on which a motion vector search is performed in the inter-screen prediction process. The inter-screen prediction process is a process of searching for a pixel block similar to the current MB within a search range centered on the position of the current MB or a position predicted therefrom.

5 to 7 are diagrams conceptually illustrating an example of the inter-screen prediction restriction information D12 that the reference control unit 210 outputs to the inter-screen prediction unit 120 in this specific example.

As shown in FIG. 5, the reference control unit 210 included the search range 41 of the encoding target picture 401 in the past in the coding order (that is, the left half region was the reference region). ) Among the pictures, the picture number of the picture 403 which is the image temporally closest to the encoding target picture 401 is included and designated in the inter-screen prediction restriction information D12.

In addition, as shown in FIG. 6, if the search range 41 exists in the right-side non-reference area, the coding is performed on the image in the coding order in the right half area in the past. The picture number of the picture 402 that is the image temporally closest to the target picture 401 is specified by being included in the inter-screen prediction restriction information D12.

Further, as shown in FIG. 7, when the current MB is located near the center of the encoding target picture 401 and the search range 41 extends over both the left and right areas, the left half area is referred to in the past. The reference area of the picture 403 that is the most recent image that was the area (more specifically, the same area as the search area 41 in the reference area; the same applies to FIG. 15) and the nearest area in which the right half was the reference area in the past The picture numbers of both images are designated so as to refer to both the reference area of the picture 402 that is the image of the image (more specifically, the same area portion as the search area 41 in the reference area, which is the same in FIG. 15). .

The inter-screen prediction unit 120 that has received such inter-screen prediction restriction information D12 basically performs inter-screen prediction with reference to only the reference area of the designated picture number.
Therefore, according to this specific example, the inter-picture prediction always uses the reference picture encoded with a small quantization coefficient. As a result, the inter-picture prediction accuracy is improved and the moving picture coding efficiency is also improved. To do.

(In-screen prediction limit)
Next, the intra-screen prediction restriction information D11 that the reference control unit 210 outputs to the intra-screen prediction unit 110 will be described.

The in-screen prediction restriction information D11 includes mode restriction information. The reference control unit 210 sets the mode restriction information so that the intra-screen prediction unit 110 refers to the pixel value only from the reference region without referring to the pixel value from the non-reference region. The “mode restriction information” will be described below.

FIG. It is a figure which shows the kind of prediction mode (prediction direction) in the prediction in a screen with respect to 4x4 block prescribed | regulated to H.264 / AVC standard. FIG. 9 is a diagram for explaining an example of the restriction of the intra prediction mode in each positional relationship between the reference area and the current MB.

As illustrated in FIG. 9A, the reference control unit 210 has a non-reference area on the left side of the current picture 401a, a current MB in the reference area, and is adjacent to the right side of the non-reference area. Designates mode restriction information indicating that the use of the intra prediction mode referring to the pixel value on the left side is prohibited as the intra prediction restriction information D11 (FIG. 9C, “position A” column).

Also, as shown in FIG. 9B, when the right side of the encoding target picture 401b is a non-reference area and the current MB is adjacent to the left side of the non-reference area, a screen for referring to the pixel value on the right side The mode restriction information indicating that the use of the intra prediction mode is prohibited is specified as the intra prediction restriction information D11 (FIG. 9 (c) “position B” column).

When the positional relationship between the reference area and the current MB is other than the above, that is, the left side of the encoding target picture is a non-reference area and the current MB is not adjacent to the right side of the non-reference area, and the right side of the encoding target picture Is a non-reference area and the current MB is not adjacent to the left side of the non-reference area, it is not necessary to limit the intra prediction mode. Therefore, the reference control unit 210 does not specify mode restriction information. At this time, for example, the intra-screen prediction restriction information D11 may include information indicating that the prediction mode is not restricted.

[Specific example 2: Upper and lower division of encoding target picture]
FIG. 10 is a diagram for explaining another example of a method for managing a reference area and a non-reference area for each picture performed by the reference control unit 210. In this specific example, the reference control unit 210 divides each picture into upper and lower parts, and performs control for switching each picture as a reference area and the other as a non-reference area.

As illustrated in FIG. 10, for example, the reference control unit 210 sets the upper half of the encoding target picture 503, which is the first picture in time series, as a reference area, the lower half as a non-reference area, and the subsequent pictures ( In the pictures 502 and 501), on the time axis, an area that was a non-reference area in the previous picture is set as a reference area, and an area that was a reference area is set as a non-reference area. Further, the reference control unit 210 holds position management information indicating the position of this reference area or non-reference area for the current and past several pictures. Then, based on this position management information, it is determined whether the current MB being processed is included in the reference region or the non-reference region, and the determination result is output to the encoding control unit 190 as a region determination result D13.

(Effect on code amount fluctuation)
Hereinafter, as described with reference to FIG. 10, a description will be given of code amount fluctuations when the reference control unit 210 divides an encoding target picture into a non-reference area and a reference area in the upper and lower directions.

FIG. 11 is a diagram for explaining the code amount per picture in the picture to be encoded. Further, FIG. 12 is a diagram illustrating a change in code amount for each picture.

As shown in FIG. 11, the moving image encoding process is performed on the encoding target picture 501 in MB units according to the raster scan order indicated by the arrows.

Here, as in this specific example, when the upper half of the encoding target picture 501 is a reference area and the lower half is a non-reference area, one picture of a non-reference area with a small code amount and a reference area with a large code amount The area contained within is always constant for any picture. Therefore, in this specific example, as shown in FIG. 12, the code amount becomes almost the same code amount in one picture unit, and the code amount fluctuation can be suppressed to within one picture unit.

As for the image quality, similarly to the first specific example, the non-reference area and the reference area are alternately displayed on the upper and lower sides of the encoding target picture. Can be used to maintain subjective image quality. Also, the amount of codes can be reduced.

This specific example is characterized in that the upper and lower reference areas and the non-reference areas are switched for each picture, and the reference area and the non-reference area in the first picture 503 may be reversed (that is, The lower half of the picture 503 may be a reference area, and the upper half may be a non-reference area).

(Restriction of prediction between screens)
As in the first specific example, the inter-screen prediction restriction information D12 output from the reference control unit 210 to the inter-screen prediction unit 120 includes a reference picture number, and the “reference picture number” It is a number indicating a reference image for performing motion vector search.

13 to 15 are diagrams conceptually illustrating an example of the inter-screen prediction restriction information D12 that the reference control unit 210 outputs to the inter-screen prediction unit 120 in this specific example.

As shown in FIG. 13, if the search range 51 of the encoding target picture 501 exists in the upper half reference area, the reference control unit 210 refers to the upper half area in the past on the time axis. The picture number of the picture 503 that is the image temporally closest to the encoding target picture 501 among the images that were the area is included and designated in the inter-screen prediction restriction information D12.

Also, as shown in FIG. 14, if the search range 51 exists in the lower half non-reference area, the encoding target of the images whose lower half area was the reference area in the past on the time axis. The picture number of the picture 502 that is the closest image in time to the picture 501 is specified by being included in the inter-screen prediction restriction information D12.

As shown in FIG. 15, when the current MB is located near the center of the encoding target picture 501 and the search range 51 extends over both upper and lower areas, the upper half area is referred to in the past. The picture numbers of both images are designated so as to refer to both the reference area of the picture 503 which is the most recent image that was the area and the reference area of the picture 502 which is the most recent image whose lower half was the reference area in the past. .

Therefore, according to this specific example, the inter-picture prediction always uses the reference picture encoded with a small quantization coefficient. As a result, the inter-picture prediction accuracy is improved and the moving picture coding efficiency is also improved. To do.

Also, moving images generally have a large lateral movement. In this specific example, the search range can be ensured horizontally longer than in the case of the specific example 1, and thus there is an advantage that the inter-screen prediction accuracy in the horizontal direction can be improved.

(In-screen prediction limit)
FIG. 16 is a diagram for explaining an example of the restriction of the intra prediction mode in each positional relationship between the reference area and the current MB.

As in the first specific example, the intra-screen prediction restriction information D11 output from the reference control unit 210 to the intra-screen prediction unit 110 includes mode restriction information. As illustrated in FIG. 16A, the reference control unit 210 sets the upper pixel when the upper side of the encoding target picture 501a is a non-reference area and the current MB is adjacent to the lower side of the non-reference area. The mode restriction information indicating that the in-screen prediction mode that refers to the value is prohibited is designated as the in-screen prediction restriction information D11 (FIG. 16B, “position C” column).

When the positional relationship between the reference area and the current MB is other than the above, that is, when the upper side of the encoding target picture is a non-reference area and the current MB is not adjacent to the lower side of the non-reference area, When the lower side is a non-reference area, there is no need to limit the intra prediction mode. Therefore, the reference control unit 210 does not specify mode restriction information. At this time, for example, the intra-screen prediction restriction information D11 may include information indicating that the prediction mode is not restricted.

In this specific example, since the prediction mode in the horizontal direction of the picture is not limited in the intra prediction mode, there is an advantage that the prediction accuracy is improved compared to the above specific example 1 for an image having a horizontal movement. .

[Specific example 3: Divide the encoding target picture into a plurality]
FIG. 17 is a diagram for explaining another example of a method for managing a reference area and a non-reference area for each picture performed by the reference control unit 210. In this specific example, the reference control unit 210 divides each picture into three in the horizontal direction, and performs control for switching each of the pictures as a reference area and the remaining one as a non-reference area for each picture. Do.

As illustrated in FIG. 17, for example, the reference control unit 210 sets the right area of the encoding target picture 604 that is the first picture in time series as a non-reference area, the center and the left as reference areas, and the next picture In the following (

pictures

603, 602, 601), if the area that was the non-reference area in the previous picture is right, the area is the center, left if it is the center, and if it is left, the right area is the non-reference area.

The reference control unit 210 refers to the current MB being processed based on the position management information indicating the position (left, center, right) of this reference area or non-reference area, as in the first specific example and the second specific example. The region determination result D13 indicating whether the region is included in the region or the non-reference region is output to the encoding control unit 190.

In this specific example, the picture is divided into three parts, one of which is a non-reference area and two of which are reference areas, and the non-reference areas are moved in order of right, center, and left in time series. However, the present invention is not limited to this. When the picture is divided into a plurality of parts, the number of divisions and the order of movement of the non-reference areas may be other methods as long as the setting pattern of the non-reference areas and the reference areas is repeated in a predetermined cycle in time series. . For example, each picture may be divided into more regions. This period does not need to be linked with the GOP period (IDR frame insertion interval).

(Effect on code amount fluctuation)
Hereinafter, as described with reference to FIG. 17, a description will be given of fluctuations in code amount when the reference control unit 210 divides the encoding target picture into three and manages the positions of the non-reference area and the reference area.

FIG. 18 is a diagram illustrating the code amount per 1 MBL in the encoding target picture.

As shown in FIG. 18, the moving image encoding process is performed on the encoding target picture 601 in MB units according to the raster scan order indicated by the arrows.

Here, as in this specific example, when the right of the encoding target picture 601 is a non-reference area and the center and the left are reference areas, 1 MBL of a non-reference area with a small code amount and a reference area with a large code amount The area contained therein is always constant in any MBL (see FIG. 18). Therefore, in the case of this specific example, the code amount is almost the same in 1 MBL unit as in the above specific example 1, and the code amount variation can be suppressed to within 1 MBL unit.

As for image quality, the proportion of non-reference areas with low image quality in a picture is narrower than in the first specific example, and when played back as a moving image, the interval at which non-reference areas appear at the same location in each picture is It becomes longer than the specific example 1. Therefore, even when the frame rate is as low as 30 fps, it is possible to receive the effect of improving the subjective image quality of the “repetitive illusion of sharp images”.

(Restriction of prediction between screens)
19 and 20 are diagrams conceptually illustrating an example of the inter-screen prediction restriction information D12 that the reference control unit 210 outputs to the inter-screen prediction unit 120 in this specific example.

If the search range 61 of the encoding target picture 601 exists in the left reference area, the reference control unit 210 encodes the encoding target picture among the images whose left area was the reference area in the past on the time axis. The picture number of the image temporally closest to 501 is specified by being included in the inter-screen prediction restriction information D12. Further, if the search range 61 exists in the center area of the encoding target picture 601, the time for the encoding target picture 601 in the image whose center area has been the reference area in the past on the time axis is temporal. The picture number of the closest image is included and specified in the inter-screen prediction restriction information D12. The same applies when the search range 61 exists in the right region of the encoding target picture 601. That is, the picture number of the image temporally closest to the encoding target picture 601 among the images whose right region has been the reference region in the past on the time axis is specified by being included in the inter-screen prediction restriction information D12.

Further, as shown in FIGS. 19 and 20, when the search range 61 extends over a plurality of areas such as the left and center of the encoding target picture 601 or the center and right, The picture number of the image is included in the inter-screen prediction restriction information D12 so as to refer to the reference area of the image in which the area at the same position as the position where the search range extends is the reference area.

In the examples of FIGS. 19 and 20, a case where only a single picture is referred to as a reference image is shown. However, as described with reference to FIG. You may come to do. For example, in the example of FIG. 19, the search range 61 extends over the left and center areas of the encoding target picture 601, but in this case, the picture that is the latest image in which the left area was the reference image in the past. Both the reference image 603 and the picture 602 that is the most recent image whose central area was the reference area in the past may be used. At this time, the reference control unit 210 specifies the picture numbers of the

pictures

602 and 603 by including them in the inter-screen prediction restriction information D12.

As described above, according to this specific example, inter-picture prediction always uses a reference image encoded with a small quantization coefficient. As a result, inter-picture prediction accuracy is improved and moving picture coding is performed. Efficiency is also improved.

In addition, a wider search range than the first specific example in which one picture is divided into two regions can be secured for a single reference image. As a result, inter-screen prediction accuracy is improved and moving picture coding efficiency is also improved. There is an advantage of improvement.

(In-screen prediction limit)
The restriction on intra prediction is the same as in the first specific example. That is, when the current MB is adjacent to the non-reference area (for example, as shown in FIG. 20), the reference control unit 210 sets the mode restriction information so as not to refer to the pixel value from the non-reference area, and the intra-screen prediction restriction. Designated as information D11.

A sharp image and a sharp image are mixed in a moving image that is encoded and decoded according to the specific example described above. The “repetitive illusion of a sharp image” is effective for human vision, but can be a disturbance to image processing such as image recognition. Periodic sharp changes can be detected on the decoding side by observing the code amount or SAD value per macroblock, and it can be estimated whether it is a reference area or a non-reference area, but position management on the encoding side Information cannot be completely restored, and an extra processing load is generated.
In addition, transcoding, which is once decoded and re-encoded with another method when it is desired to change the encoding method, is widely performed, but once a sharply repeated moving image material maintains its sharp pattern. If transcoding is not performed, the image quality is significantly degraded.

In this example, in the video encoding apparatus, the variable length encoding unit 180 encodes information (position management information) for distinguishing between a reference area and a non-reference area, and user information (for example, H.264 / AVC encoding). SEI (supplemental enhancement information) message) in the stream and output from the stream buffer. Then, the decoding device that receives the stream decodes the information and provides it to downstream image processing.

FIG. 24 is a configuration diagram of the transcoder according to the present embodiment.
The moving picture decoding apparatus 2 is the same as the H.264 standard. This is a general decoder that decodes an H.264 / AVC encoded stream and sequentially outputs image data. The motion information (motion vector) used for motion compensation and the position management information decoded from the stream are output together.
The moving picture decoding apparatus 1 ′ is almost the same as the moving picture decoding apparatus 1 described in the first to third specific examples. However, the reference control unit 210 of the video decoding device 1 arbitrarily determines the positions of the reference region and the non-reference region, whereas the reference control unit 210 ′ of the video encoding device 1 ′ of the present example. Is different in that the position management information received from the video decoding device 2 is followed to determine the position. Basically, what was once a non-reference area may be maintained as a non-reference area, and there is no problem as long as a former reference area becomes a non-reference area. Note that the motion information from the video decoding device 2 is provided to the inter-screen prediction unit 120 ′, which is useful for reducing the amount of motion search processing.

In the transcoder of the present embodiment, the moving picture decoding apparatus 1 ′ and the moving picture decoding apparatus 2 have been described as being integrated so that data can be transferred through a memory or the like. When connected by a baseband signal transmission medium such as SDI (bit serial digital interface), the position management information can be included in the ancillary data (auxiliary data).

(Appendix)
It should be noted that the scope of the present invention is not limited to the illustrated and described exemplary embodiments, but includes all embodiments that provide the same effects as those intended by the present invention. Further, the scope of the invention can be defined by any desired combination of particular features among all the disclosed features.

Some or all of the above-described embodiments can be described as in the following supplementary notes. However, the following supplementary notes are merely examples of the present invention, and the present invention is not limited to such cases.

A moving image encoding program for causing a computer to execute a moving image encoding method for encoding a moving image including a plurality of images arranged in time series by intra prediction or inter prediction.
The plurality of images are divided into a reference region that is a region that refers to a pixel value in the intra prediction or inter-screen prediction, and a non-reference region that is a region that does not refer to a pixel value, and the time series A first processing step for determining the reference region and the non-reference region in the plurality of images so that the reference region and the non-reference region in the plurality of images are switched at a predetermined interval set forth above; ,
A moving image encoding program for executing a process including: a second processing step of performing quantization on the non-reference area with a coarser width than the reference area when encoding the plurality of images .

The present invention relates to MPEG (trademark), VC-1 (SMPTE 421M), H.264, etc. The present invention can be applied to a moving picture coding system based on predictive coding such as H.265 / HEVC, and includes a video recording apparatus including a digital video camera using such a moving picture coding system, a video transmission system, a television studio device, a video phone / conference It can be used in a device or software for a personal computer that realizes these functions.

DESCRIPTION OF SYMBOLS 1 Moving image encoder, 10 Original image, 11 Stream, 12 Reconstructed image, 13 Intra-screen prediction image, 14 Inter-screen prediction image, 15 Prediction image,
16 differential images, 17 restored differential images, 41 search ranges, 51 search ranges, 61 search ranges, 90 original images, 91 streams, 92 reconstructed images, 93 intra prediction images,
94 inter-screen prediction image, 95 prediction image, 96 difference image, 97 restored differential image 110 intra-screen prediction unit, 120 inter-screen prediction unit, 130 mode selection unit, 140 orthogonal transform unit, 150 quantization unit, 160 inverse quantization unit ,
170 Inverse orthogonal transform unit, 180 Variable length coding unit, 190 Coding control unit 200 Stream buffer, 210 Reference control unit 401 Picture to be coded, 401a, 401b Picture to be coded, 402 picture, 403 picture, 501 Code target Picture,
501a encoding target picture, 502 picture, 503 picture 601 encoding target picture, 602 picture, 603 picture, 910 intra prediction unit, 920 inter prediction unit, 930 mode selection unit,
940 orthogonal transform unit, 950 quantization unit, 960 inverse quantization unit, 970 inverse orthogonal transform unit, 980 variable length coding unit, 990 encoding control unit, 1000 stream buffer,
D1 intra-screen prediction information, D2 inter-screen prediction information, D3 frequency component, D4 quantization coefficient, D5 quantized value, D6 restored frequency component, D7 code amount information,
D11 intra-screen prediction restriction information, D12 inter-screen prediction restriction information, D13 region determination result, D81 intra-screen prediction information, D82 inter-screen prediction information, D83 frequency component,
D84 quantization coefficient, D85 quantization value, D86 restoration frequency component, D87 code amount information

Claims

A video encoding device that encodes a video consisting of a plurality of images arranged in time series by intra prediction or inter prediction,
The plurality of images are divided into a reference region that is a region that refers to a pixel value in the intra prediction or inter-screen prediction, and a non-reference region that is a region that does not refer to a pixel value, and the time series A reference control unit that determines the reference region and the non-reference region in the plurality of images so that the reference region and the non-reference region in the plurality of images are switched at a predetermined interval above;
A moving image coding apparatus comprising: a quantization unit configured to perform quantization with a coarser width than the reference region for the non-reference region when the plurality of images are encoded.
In the image, the reference control unit is configured so that a ratio of the reference area and the non-reference area in a certain unit area is the same in any certain unit area included in the image. The moving picture coding apparatus according to claim 1, wherein the non-reference area is determined.
An intra-screen prediction unit that performs the encoding by the intra-screen prediction on the plurality of images;
The intra-screen prediction unit refers to only the pixels in the reference area according to the positional relationship between the current macroblock that is the encoding target macroblock and the reference area or the non-reference area in the image. The moving image encoding apparatus according to claim 1, wherein the encoding is performed.
An inter-screen prediction unit that performs the encoding by the inter-screen prediction for the plurality of images;
The inter-screen prediction unit refers to at least one of the plurality of images whose motion vector search range is the reference region when performing the inter-screen prediction. The moving image encoding device according to claim 1.
This is a moving image encoding method executed by a moving image encoding apparatus that includes a reference control unit and a quantization unit and encodes a moving image composed of a plurality of images arranged in time series by intra prediction or inter prediction. And
The reference control unit divides the plurality of images into a reference region that refers to a pixel value in the intra prediction or inter prediction and a non-reference region that does not refer to a pixel value. In addition, the reference area and the non-reference area in the plurality of images are determined so that the reference area and the non-reference area in the plurality of images are switched at predetermined intervals on the time series. A first step;
The quantization unit includes a second step of performing quantization with a coarser granularity than the reference region for the non-reference region when the plurality of images are encoded. Image coding method.