JP2006211304A - Device, method, and program for video image coding and decoding - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve a problem that, if information showing a scanning order is stored in a video image stream, a code amount increases and coding efficiency worsens. <P>SOLUTION: The device creates a determination formula for determining scanning order from a reference image for coding images, and determines scanning order for each image block using the determination formula. Moreover, it is possible to obtain scanning order for each image block determined by a coding device also in a decoding device by storing the determination formula in the image stream. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a video encoding apparatus and method for encoding video and generating a video stream, and a video decoding apparatus and method for generating decoded video by decoding the video stream.

  Video is no longer inseparable from our daily lives, and it can be viewed on various display terminals such as personal computers, mobile terminals, TVs, and high-definition TVs through transmission methods such as the Internet, mobile phone networks, broadcast waves, and storage media. It became an important existence to let you enjoy the information. The video transmitted through the transmission means is compressed into a video stream having a smaller amount of data using a video encoding technique in order to efficiently transmit information.

  Also, it is required to transmit one video to a plurality of users connected to transmission means having different speeds, and it has a data structure consisting of several layers, and a video stream as needed after encoding. Research and development of a hierarchical coding method that can change the transmission amount of the video is progressing. In the hierarchical coding scheme, video is layered and coded into a base layer stream that can be decoded alone and an enhancement layer stream for improving the image quality of the base layer stream.

  However, in recent years, the amount of information held by video due to higher resolution of cameras and video displays has become enormous, and a technology for further increasing the efficiency of these video encoding methods is required.

  Here, the video refers to continuous images, that is, moving images.

  In the video encoding technique, an input image is divided into, for example, image blocks for every 8 × 8 pixels, and DCT conversion (Discrete Cosine Transformation) is performed to convert the input image into DCT coefficients representing a frequency.

  This uses the characteristic that the video statistically contains a lot of low frequencies and does not contain high frequencies, and the frequency that is not visually important to humans is set to 0 by quantization that divides each coefficient by an appropriate value. It is possible to reduce the amount.

  FIG. 12A shows DCT coefficients of an image block of 8 × 8 pixels before quantization, and FIG. 12B shows DCT coefficients after quantization. In FIG. 12 (b), quantization is performed by dividing by a larger value as the frequency becomes higher.

  Rearranging the quantized DCT coefficients is referred to as scanning, and the statistical values that are zeroed by quantization by rearranging the DCT coefficients from low frequency components to high frequency components in the zigzag scan order shown in FIG. It is possible to bias the high frequency coefficients having small absolute values to the second half of the order.

  Further, the DCT coefficients rearranged by scanning are encoded using an “End-of-Block signal” that indicates that “the subsequent coefficients are all 0”, that is, EoB, so that the latter half of 0 can be efficiently converted into one signal. It is possible to express. The order from the beginning of the scan order until the last DCT coefficient other than 0 appears is called the scan length. If the scan length is shortened, the code amount is reduced and the coding efficiency is improved.

  Here, when there are two video streams from which the same decoded image can be obtained, a video stream with a smaller code amount is said to have good coding efficiency. In addition, when there are two video streams with the same code amount, a video stream with a cleaner decoded image is said to have good encoding efficiency.

  Here, there are various definitions of a cleaner image, but an image that is more beautiful when viewed by humans or an image with less error from the original image.

  In Patent Document 1, the edge of an image block is examined, and the scan order is appropriately selected, so that more zeros are biased in the latter half of the scanned DCT coefficient to improve the coding efficiency. Here, there are various definitions of the edge. Here, the edge in the horizontal direction is represented by a difference between two pixels adjacent to the left and right, and the edge in the vertical direction is represented by a difference between two pixels adjacent in the vertical direction. Shall.

  FIG. 14 shows a configuration of an encoding apparatus to which Patent Document 1 is applied. The edge extraction unit 1401 calculates the amount of edges in the horizontal and vertical directions for each image block before DCT conversion. The block classification unit 1402 selects a scan order suitable for each image block from FIGS. 13A, 13B, and 13C, for example, according to the amount of the edge. For example, if the image block includes many horizontal edges, the horizontal frequency is included. Therefore, the horizontal scan shown in FIG. If many vertical edges are included, the vertical scan shown in FIG. 13C is selected. The adaptive DCT encoding unit 1403 performs encoding by DCT conversion or scanning according to the scanning order selected for each image block.

  Here, by storing in the video stream a flag indicating which scan order is selected for each image block, it is possible to specify a different scan order for each image block in the decoding apparatus.

  In Patent Document 2, the distribution of DCT coefficients of an image block temporally and spatially adjacent to an image block to be encoded is used, and the scanning order is appropriately selected, so that the latter half of the scanned DCT coefficient is increased. The encoding efficiency is improved by biasing 0.

  FIGS. 16A and 16B show image blocks obtained by dividing an image every 8 × 8 pixels. FIG. 16A shows an image block 1601 to be encoded and an image block 1602 around it. FIG. 16B shows an image block 1603 at the same position one frame before the image block to be encoded and an image block 1604 adjacent thereto. In the method of Patent Document 2, the distribution of DCT coefficients of these image blocks 1602, 1603, and 1604 is used.

  FIG. 15 shows a configuration of an encoding apparatus to which Patent Document 2 is applied. The scanning method selection unit 1504 selects a scan order in which the DCT coefficients of image blocks to be encoded and temporally and spatially adjacent image blocks can be rearranged to have the shortest scan length.

Here, if the selection method of the image block to be referenced is matched between the encoding device and the decoding device, the decoding device can store the image without storing a flag indicating which scan order is selected for each image block. It is possible to specify a different scan order for each block.
JP-A-10-126775 JP 2003-250157 A

  However, in the method of Patent Document 1, the video stream increases by the code amount of the flag indicating the scan order, and the encoding efficiency deteriorates.

  Similarly, when the method of Patent Document 1 is applied to an enhancement layer of hierarchical encoding, a flag indicating the scan order for each image block must be stored in the video stream, resulting in poor encoding efficiency.

  In addition, the method of Patent Document 2 does not necessarily mean that the distribution of DCT coefficients of the image block to be encoded, the surrounding image block, and the image block at the same position in another image are similar, and the scan length becomes long. Inefficiency.

  Further, when the encoded image block is predictive encoding such as motion prediction compensation encoding, a problem occurs. An image block 1605 in FIG. 16C represents a reference image block of the image block 1601 to be encoded in FIG. Generally, in the predictive coding, the image block 1605 is selected so that the difference between the image blocks 1601 and 1605 is small, so that the DCT coefficient distribution of the image block 1601 is similar to that of the image block 1605. However, in Patent Document 2, only the DCT coefficient of the image block located at the position where the image is divided every 8 × 8 pixels can be used, and the image block 1605 not located at the position divided every 8 × 8 pixels is used. Information is not available. The DCT coefficient distribution of the image blocks 1602, 1603, and 1604 is not necessarily similar to that of the image block 1601, and the encoding efficiency is not necessarily improved.

  Similarly, when the method of Patent Document 2 is applied to an enhancement layer of hierarchical coding, the image block around the image block to be coded in the enhancement layer, the same position of the frame before and after the time, the surrounding image block, and the DCT The coefficient distribution is not always similar, and the coding efficiency is not necessarily improved.

  The video encoding apparatus of the present invention refers to a scan order, which is an order in which an encoded image block, which is a set of several pixels in an image to be encoded, is encoded and stored in a video stream. A scan determination unit that determines a scan determination formula for a reference image block to be generated, a scan determination formula generation unit that generates the scan determination formula using a reference image that is referred to during encoding, and the scan determination formula into a video stream A configuration having a scan judgment expression storing means for storing is adopted.

  With this configuration, it is not necessary to describe the scan order for each image block, the scan order of all the encoded image blocks can be expressed only by the scan determination formula, and a short scan length can be used for each encoded image block. The scan order that can be encoded can be given, and the encoding efficiency can be improved.

  Further, the video decoding apparatus according to the present invention is characterized in that, in the above configuration, the scan determination formula storage means stores the information of the scan determination formula at a position of the video stream before the code of the encoded image block. To do.

  With this configuration, even when the video stream is interrupted, it is possible to decode up to an encoded image block in the middle.

  In the coding order determining apparatus according to the present invention, in the above configuration, the scan determination formula generation unit compares a horizontal edge and a vertical edge of the reference image, and compares the horizontal edge with the vertical edge. It has a feature of generating the scan determination formula that causes the scan determining means to determine the scan order in preference to the horizontal direction if more than the edges are included, and in priority to the vertical direction if more vertical edges are included than the horizontal edges. .

  With this configuration, it is possible to generate the scan determination formula using the edge strength of the reference image having a strong correlation with the encoded image block, and to give the scan order that can be encoded with a short scan length for each encoded image block. It is possible to improve the encoding efficiency.

  In the coding order determining apparatus according to the present invention, in the above configuration, the scan determination formula generation unit compares the horizontal high-frequency coefficient of the DCT coefficient of the reference image with the vertical high-frequency coefficient, and converts the horizontal high-frequency coefficient into the vertical high-frequency coefficient. If more are included, the horizontal direction is prioritized, and if more vertical high-frequency coefficients are included than the horizontal high-frequency coefficient, the scan determination formula is generated to cause the scan determination unit to determine the scan order with priority to the vertical direction.

  With this configuration, it is possible to generate the scan determination formula using DCT coefficients representing edges, and to provide the scan order that can be encoded with a short scan length for each encoded image block, thereby improving encoding efficiency. Is possible.

  In the coding order determining apparatus according to the present invention, in the above configuration, the scan determination formula generation unit predicts the characteristics of the encoded image block from the statistics of information for each reference image block, and calculates the scan determination formula. It has the feature to generate.

  With this configuration, it is possible to generate the scan determination formula using the information of the reference image block and the statistics of the entire reference image, and to give the scan order that can be encoded with a short scan length for each encoded image block, It is possible to improve the encoding efficiency.

  The coding order determining apparatus according to the present invention is characterized in that, in the above-described configuration, the scan determination expression generation unit generates the scan determination expression for each of the plurality of encoded image blocks.

  With this configuration, it is possible to generate the scan determination formula according to the characteristics of each frame of an image or each region in the image, and to give the scan order that can be encoded with a short scan length for each encoded image block. It is possible to improve the encoding efficiency.

  Further, the coding order determining apparatus of the present invention has the feature that, in the above configuration, the scan determination formula generation means changes the value of the parameter included in the scan determination formula using the past scan determination formula.

  With this configuration, the scan determination formula is generated by predicting the feature of the image being encoded from the features of the past image and the past reference image, and the scan that can be encoded with a short scan length for each encoded image block An order can be given and encoding efficiency can be improved.

  The coding order determining apparatus according to the present invention is characterized in that, in the above-described configuration, the scan determination expression generation unit generates the scan determination expression using a result of scanning the encoded image block.

  With this configuration, it is possible to check the encoding efficiency in the scan order according to the scan determination formula to generate the scan determination formula, and to give the scan order that can be encoded with a short scan length for each encoded image block. It is possible to improve the efficiency.

  Further, the coding order determining apparatus according to the present invention is characterized in that, in the above configuration, the scan determination formula generation means prepares and switches a plurality of the scan determination formulas in advance.

  Thereby, according to the feature of the image, it is possible to adaptively switch to the scan determination formula that shortens the scan length, and to give the scan order that can be encoded with a short scan length for each encoded image block. Efficiency can be improved.

  The coding order determining apparatus according to the present invention has the above-described configuration, in the hierarchical coding in which the base layer capable of decoding a single video and the decoded image of the base layer are hierarchized into an enhancement layer for improving the image quality. The scan determination formula generation unit generates the scan determination formula for determining the scan order of the encoded blocks of the enhancement layer corresponding to the reference image block of the base layer, using the base layer decoded image as the reference image. It is characterized by that.

  With this configuration, it is possible to generate the scan determination formula using the decoded image of the base layer having a strong correlation with the enhancement layer, and to give the scan order that can be encoded with a short scan length for each encoded image block. It is possible to improve the efficiency.

  The coding order determining apparatus according to the present invention has the above-described configuration, wherein the scan determination formula generation unit uses the image referred to in the predictive coding as the reference image in the predictive coding of the video, and the scan determination formula is It has the feature to generate.

  With this configuration, the scan determination formula is generated using the reference image of predictive encoding having a strong correlation with the encoded image block, and the scan order that can be encoded with a short scan length is provided for each encoded image block. It is possible to improve the encoding efficiency.

  In addition, the coding order determination apparatus according to the present invention is characterized in that, in the above configuration, the predictive coding is inter-screen predictive coding.

  With this configuration, the scan determination formula is generated using the reference image of inter-picture predictive coding having a strong correlation with the coded image block, and the scan order that can be coded with a short scan length is provided for each coded image block It is possible to improve the coding efficiency.

  In addition, the coding order determination apparatus of the present invention has the feature that, in the above configuration, the predictive coding is intra-screen predictive coding.

  With this configuration, the scan determination formula is generated using the reference image of intra prediction encoding having a strong correlation with the encoded image block, and the scan order that can be encoded with a short scan length is provided for each encoded image block It is possible to improve the coding efficiency.

  In the coding order determining apparatus according to the present invention, in the above configuration, the predictive coding is performed when the code amount of the coded image block is the smallest among the reference image block candidates of the predictive coding. The reference image block predicted by the determination formula is used for predictive coding.

  With this configuration, a reference image block for predictive coding can be selected so that the scan length given by the scan order of the coded image block determined by the scan determination formula is shortened and coding efficiency is improved. It is possible to improve the efficiency.

  In the coding order determining apparatus according to the present invention, in the configuration described above, the scan determination formula generation unit uses the information used for prediction by the prediction encoding to calculate the scan determination formula corresponding to the reference image block. It has the feature to generate.

  With this configuration, it is possible to generate the scan determination formula using more detailed information of the reference image and to give the scan order that can be encoded with a short scan length for each encoded image block, thereby improving encoding efficiency. It is possible.

  The video decoding apparatus according to the present invention refers to a scan order, which is an order in which an encoded image block, which is a set of several pixels in an encoded image, is encoded and stored in a video stream. A scan determination unit that determines a scan determination formula for a reference image block to be performed, and a scan determination formula acquisition unit that acquires the scan determination formula from a video stream.

  With this configuration, it is not necessary to describe the scan order for each image block in the video stream, and the scan order of all the encoded image blocks can be expressed only by the scan determination formula, and a video stream with high encoding efficiency is decoded. It is possible to

  The present invention relates to a video coding method for performing video coding, which encodes a coded image block, which is a set of several pixels in an image to be coded, executed by a video coding device. A scan determination processing step for determining a scan order, which is an order to be converted into a video stream and stored in a video stream, using a scan determination formula for other reference image blocks to be referred to at the time of encoding, and a reference image to be referred to at the time of encoding A scan determination expression generation processing step for generating the scan determination expression, and a scan determination expression storage processing step for storing the scan determination expression in a video stream.

  The present invention is a video decoding method for decoding a video stream, and an encoded image block which is a set of several pixels in an encoded image, which is executed by a video decoding device. A scan determination processing step for determining a scan order, which is an order of encoding and storing in a video stream, by a scan determination formula for a reference image block to be referred to at the time of decoding, and a scan determination for acquiring the scan determination formula from the video stream A formula acquisition processing step.

  The present invention relates to a scan order, which is an order in which an encoded image block, which is a set of several pixels in an image to be encoded, is encoded and stored in a video stream in order to perform video encoding. A scan determination processing step that determines a scan determination formula for other reference image blocks that are referred to in encoding, and a scan determination formula generation process that generates the scan determination formula using a reference image that is referred to in encoding And a scan determination formula storage processing step for storing the scan determination formula in a video stream.

  The present invention provides a computer for executing decoding of a video stream, a scan determination expression acquisition processing step for acquiring the scan determination expression from a video stream, and a reference image to be referred to when the scan determination expression and decoding are performed. And a scan determination processing step for determining the scan order.

  In the present invention, it is possible to improve encoding efficiency in video encoding.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(Embodiment 1)
In the first embodiment, in hierarchical coding in which video is hierarchized into a base layer and an enhancement layer and encoded, a scan determination formula for generating the scan order of the DCT coefficients of the image blocks of the enhancement layer is used for the base layer. It is generated from the decoded image, and the DCT coefficients of the enhancement layer image block are encoded by the scan order generated by the scan determination formula.

  Although this embodiment will be described with respect to encoding of video, that is, continuous images, it may be used for encoding still images.

  FIG. 1 shows a configuration of a video encoding apparatus 100 to which a video encoding method according to Embodiment 1 of the present invention is applied.

  In FIG. 1, the video encoding device 100 includes a video signal input unit 101, a base layer encoding unit 102, a base layer output unit 103, and an enhancement layer encoding unit 110. The enhancement layer coding unit 110 includes a difference unit 111, a DCT unit 112, a scanning unit 113, a variable length coding unit 114, an enhancement layer output unit 115, a scan determination formula generation unit 116, and a scan determination unit 117.

  The video signal input unit 101 inputs a video as an original image frame by frame from the outside of the video encoding device 110 and outputs it to the base layer encoding unit 102 and the enhancement layer encoding unit 110. Also, the presence / absence of a video input from the outside of the video encoding device 110 is determined. If there is no video input, the processing is terminated.

  The base layer encoding unit 102 encodes the original image input from the video signal input unit 101 to generate a base layer stream, and outputs the base layer stream to the base layer output unit 116. Also, the base layer stream is decoded to generate a base layer decoded image, which is output to the enhancement layer encoding unit 110.

  Base layer output section 103 outputs the base layer stream input from base layer encoding section 102 to the outside of video encoding apparatus 100.

  The difference unit 111 generates a difference image by taking the difference between the original image input from the video signal input unit 101 and the base layer decoded image input from the base layer encoding unit 102 and outputs the difference image to the DCT unit 112.

  When the resolution of the base layer decoded image and the original image are different, the base layer decoded image is changed to the same resolution as the original image.

  The DCT unit 112 divides the difference image input from the difference unit 111 into image blocks of 8 × 8 pixels, performs DCT conversion for each image block, generates a DCT coefficient, and outputs the DCT coefficient to the scan unit 113.

  In this embodiment, the image block is 8 × 8 pixels for simplicity of explanation, but the size of the image block is not particularly limited.

  The scan unit 113 rearranges the DCT coefficient input from the DCT unit 112 using the scan order input from the scan determination unit 117 to generate a scanned DCT coefficient, and outputs the generated DCT coefficient to the variable length encoding unit 114.

  The variable length coding unit 114 performs variable length coding on the scanned DCT coefficient input from the scanning unit 113 to generate an enhancement layer stream, and outputs the enhancement layer stream to the enhancement layer output unit 115.

  Here, the variable length encoding unit 114 may be encoding using quantization such as the MPEG-4 AVC method described in ISO / IEC 14496-10, or MPEG-4 described in ISO / IEC 14496-2 Amendment 2. Encoding using a bit plane such as FGS may be used. The present invention can be applied to any coding scheme in which DCT coefficients are reordered by scanning and coded using EoB with 0 being biased to the second half of the scanning order.

  In the case of encoding that performs quantization, the variable length encoding unit 114 performs quantization processing before performing variable length encoding.

  The variable length coding method may be any method such as Huffman coding or arithmetic coding.

  The enhancement layer output unit 115 stores the scan determination formula input from the scan determination formula generation unit 116 in the enhancement layer stream input from the variable length encoding unit 114, and outputs the stored scan determination formula to the outside of the video encoding device 100.

  The storage of the scan determination formula will be described later.

  The scan determination formula generation unit 116 generates a scan determination formula using the base layer decoded image input from the base layer encoding unit 102 and outputs the scan determination formula to the enhancement layer output unit 115 and the scan determination unit 117.

  In the present embodiment, it is assumed that a scan determination formula is stored for each enhancement layer stream of one frame. By generating the scan order based on this, a short scan length applied to the image characteristics for each frame can be obtained. However, the frequency of storing the scan determination formula is not particularly limited.

  A method for generating the scan determination formula will be described later.

  The scan determination unit 117 generates a scan order for each image block using the base layer decoded image input from the base layer decoding unit 102 and the scan determination formula input from the scan determination formula generation unit 116, and sends it to the scan unit 113. Output.

  The scan determination formula generation unit 116 corresponds to the scan determination formula generation unit of the present invention, the scan determination unit 117 corresponds to the scan determination unit, and the enhancement layer output unit 115 is integrated into the scan determination formula storage unit.

  The scan determination formula generation process performed by the scan determination formula generation unit 116 will be described in detail below.

  The enhancement layer encoding unit 110 encodes detailed information of the original image that cannot be expressed in the base layer decoded image. In the difference image, pixels having a large absolute value are concentrated around the contour of the object, and the difference image indicates that the contour of the base layer decoded image is refined. In addition, it can be seen that strong edges of the base layer decoded image are concentrated around the contour of the object, and the difference image and the edges of the base layer decoded image have a strong correlation. In this embodiment, a method for generating the scan order from the edge characteristics is generated by utilizing this fact.

  The image block at the same position of the base layer decoded image corresponding to the encoded image block of the enhancement layer is set as a reference image block.

  The DCT coefficient represents the intensity of the frequency component. When an image block including a strong edge is subjected to DCT conversion, a DCT coefficient having a large absolute value is generated at a high frequency. Also, the distribution of DCT coefficients varies depending on the type of edge, and a coded image block including many horizontal edges has a large absolute value in the horizontal high-frequency DCT coefficient, and a coded image including many vertical edges. The block has a large absolute value biased to the vertical high frequency DCT coefficient.

  Therefore, for example, for the encoded image block corresponding to the reference image block including many horizontal edges, the horizontal scan in FIG. 13B giving priority to the horizontal direction corresponds to the reference image block including many vertical edges. By selecting the vertical scan of FIG. 13C giving priority to the vertical direction for the encoded image block to be performed, the scan length becomes shorter than when the zigzag scan is applied, and the encoding efficiency is improved. If it cannot be said that there are more edges in either the horizontal direction or the vertical direction, the zigzag scan in FIG. 13A is selected.

  For the sake of simplicity, it is assumed that one of FIGS. 13A, 13B, and 13C is selected as the scan order, but the present invention is not limited to this.

  The scan determination formula generation unit 116 generates a scan determination formula that gives a scan order in which the encoded image block can be efficiently encoded from the edge of the reference image block. Depending on the determination of the scan determination formula, a scan order for encoding the encoded image block inefficiently is given. For example, the scan determination formulas of (Formula 1) to (Formula 4) are generated from the reference image block.

  Prior to the method for generating the scan determination formula, generation of the scan order using the scan determination formula will be described.

  The scan determination unit 118 calculates the strength of the edge of the reference image block using (Equation 1) and (Equation 2), and scans the encoded image block using the determination equations (Equation 3) and (Equation 4) for the reference image block. Select the order. If the determination formula (Equation 3) is true, the horizontal scan is selected, if the determination equation (Equation 4) is true, the vertical scan is selected, and if both are false, the zigzag scan is selected.

  (Formula 1) is a formula for calculating the horizontal edge of the reference image block. Edge represents a value representing the strength of the edge, the subscript H at the upper left of Edge represents the edge in the horizontal direction, and the subscript i at the lower right of Edge represents the serial number of the image block. Further, p represents a value of the pixel, and x and y represent the coordinates of the pixel in the image block. λ is a value greater than zero. Therefore, (Formula 1) represents a calculation formula of a value obtained by summing up the λ power of the absolute difference value between the adjacent pixels in the 8 × 8 image block.

  (Formula 2) is a formula for calculating the vertical edge of the reference image block. The subscript V at the upper left of Edge represents a vertical edge. Therefore, (Formula 2) represents a formula for calculating the sum of the λ power of the absolute value of the difference from the vertically adjacent pixels in the 8 × 8 image block, as in (Formula 1).

  (Formula 3) is a determination formula in the case of selecting the horizontal scan for the encoded image block. α_1 and α_2 take appropriate values of 1 or more. Edge_H with an overline represents an average value of edges in the horizontal direction of the base layer decoded image. Therefore, (Equation 3) represents that in the reference image block, the strength of the edge in the horizontal direction exceeds the strength of the edge in the vertical direction to some extent and exceeds the strength of the edge in the horizontal direction in average.

  (Formula 4) is a determination formula in the case of selecting the vertical scan for the encoded image block. β_1 and β_2 take appropriate values of 1 or more. Edge_V with an overline represents the average value of all image blocks at the edge in the vertical direction of the base layer decoded image. Therefore, (Equation 3) represents that, in the reference image block, the strength of the vertical edge exceeds the strength of the vertical edge to some extent and exceeds the average vertical edge strength.

  Next, a method for generating a scan determination formula will be described. For simplicity of explanation, it is assumed that λ in (Expression 1) and (Expression 2) is a fixed value, for example, 2.

  The scan determination formula generation unit 116 calculates the edge strength calculated by (Formula 1) and (Formula 2) for all reference image blocks. Then, statistical values such as average and variance are obtained, and the values of α_1, α_2, β_1, and β_2 in (Equation 3) and (Equation 4) are determined. As a method for determining the values of α_1, α_2, β_1, and β_2 from the statistical values of the edge strength, a correspondence table or the like obtained in advance through experiments is used.

  For example, α_1 is determined to be twice the value obtained by dividing the average value of the horizontal edge intensity by the average value of the vertical edge intensity. If the average value of the horizontal edge intensity is 1000 or less, α_2 is determined to be 2, and if not, α_2 is determined to be 3. β_1 and β_2 are similarly determined.

  The information α_1, α_2, β_1, and β_2 of the scan determination formula determined as described above is output to the scan determination unit 117, and the enhancement layer output unit 115 stores the information in the enhancement layer stream.

  FIG. 2 shows an example of the structure of the enhancement layer stream. A head portion 201 of the enhancement layer stream is a header including a flag indicating that this video stream represents enhancement layer information and information such as image resolution. A portion 203 is a code of the image block generated by the variable length encoding unit 114. The information of the scan determination formula is stored in the portion 202. For example, α_1, α_2, β_1, and β_2 only take values from 1 to 16, and these values are expressed by 4 bits. 0000 represents 1 and 1111 represents 16.

  The position in the video stream in which the scan determination formula is stored is assumed to be before the code of the image block. If there is no scan judgment formula in the decoding process, the scan order of the image blocks cannot be specified. If there is a scan judgment formula after the image block, all the image blocks are decoded when the video stream is interrupted in the middle. I can't. If the scan determination formula is stored before the code of the image block, even when the video stream is interrupted in the middle, the image block in the middle can be decoded.

  A feature of the present invention is that calculation formulas and parameters necessary for scan generation are stored in a video stream, and the scan order of image blocks is matched in the encoding and decoding processes. It is not important and may be other than the above method.

  Note that it is possible to prepare multiple conditions for edge strength calculation formulas and scan judgment formulas, in which case the parameters indicating which edge strength formulas and scan judgment formulas are used are extended layers. Store in stream.

  Accordingly, it is possible to adaptively switch to a scan determination formula that shortens the scan length according to the feature of the image. A switching method according to the feature of the image may be obtained in advance by experiments. For example, switching according to the code amount of the base layer stream.

  The value of λ may not be a fixed value. For example, in a certain frame, the value of λ is determined to be 2 and encoding is performed. If the number of horizontal scans or vertical scans selected in the frame is 10% or less of the total number of image blocks, λ is set to 3 in the next frame. If not, leave it at 2 otherwise.

  Thus, by selecting an edge strength calculation formula suitable for the image to be encoded, a scan determination formula that gives a scan order that can be encoded efficiently is generated.

  The formula for calculating the edge strength is not limited to (Formula 1) and (Formula 2).

  For example, the intensity of the horizontal edge of the reference image block may be a value obtained by totaling 8 columns in the vertical direction for the dispersion of 8 pixels in the horizontal direction. The vertical edge strength is also the same.

  For example, DCT conversion is performed on the reference image block to generate DCT coefficients as shown in FIG. 3, and the horizontal edge strength is the absolute sum of horizontal high-frequency DCT coefficients 301 surrounded by a broken line, and the vertical edge strength is a broken line. The absolute sum of the vertical high-frequency DCT coefficients 302 surrounded by

  Thereby, when the DCT coefficient of the reference image block has already been calculated, the processing load for calculating the edge strength is reduced.

  The determination formula for scan generation is not limited to (Equation 3) and (Equation 4).

  For example, (Equation 3) may be added on condition that the vertical edge strength is equal to or lower than the average value, and (Equation 4) is added on the condition that the horizontal edge strength is equal to or lower than the average value. Further, a determination condition with a threshold value may be added to (Expression 3) or (Expression 4).

  The gist of the present invention is to store a scan judgment formula indicating calculation formulas and parameters necessary for scan generation in a video stream, and to match the scan order of image blocks in encoding and decoding processing. The expression itself may be other than the above method.

  The scan determination formula generation unit 116 may input the DCT coefficient of the encoded image block from the DCT unit 112.

  Thereby, for example, by actually scanning the DCT coefficients of all the coded image blocks in the scan order obtained by a combination of several α_1, α_2, β_1, β_2, λ, and examining the shortest scan order, It is possible to determine α_1, α_2, β_1, β_2, and λ with good coding efficiency.

  The scan determination formula generation unit 116 may output the intermediate information generated when generating the scan determination formula to the scan determination unit 117.

  As a result, it is possible to omit duplicate processing between the generation of the scan determination formula and the selection of the scan order.

  Next, the operation of the video encoding apparatus 100 configured as described above will be described. FIG. 4 is a flowchart showing an example of the operation of the video encoding apparatus 100 according to the first embodiment shown in FIG. The flowchart shown in FIG. 4 is executed by software by executing a control program stored in a storage device (not shown) such as a ROM or a flash memory by a CPU (not shown). It is also possible to do so.

  First, in step S <b> 401, the video signal input unit 101 inputs a video as an original image frame by frame from the outside of the video encoding device 110, and outputs it to the base layer encoding unit 102 and the enhancement layer encoding unit 110.

  Next, in step S <b> 402, the base layer encoding unit 102 encodes the original image input from the video signal input unit 101 to generate a base layer stream, and outputs the base layer stream to the base layer output unit 116. Also, the base layer stream is decoded to generate a base layer decoded image, which is output to the enhancement layer encoding unit 110.

  Next, in step S403, the difference unit 111 generates a difference image by taking the difference between the original image input from the video signal input unit 101 and the base layer decoded image input from the base layer encoding unit 102, and a DCT unit. To 112.

  Next, in step S404, the DCT unit 112 divides the difference image input from the difference unit 111 into image blocks of 8 × 8 pixels, performs DCT conversion for each image block, generates a DCT coefficient, and scan unit It outputs to 113.

  Next, in step S <b> 405, the scan determination formula generation unit 116 generates a scan determination formula using the base layer decoded image input from the base layer encoding unit 102, and sends it to the enhancement layer output unit 115 and the scan determination unit 117. Output.

  Next, in step S <b> 406, the scan determination unit 117 uses the base layer decoded image input from the base layer decoding unit 102 and the scan determination formula input from the scan determination formula generation unit 116 to set the scan order for each image block. Generate and output to the scan unit 113.

  In step S407, the scan unit 113 rearranges the DCT coefficients input from the DCT unit 112 using the scan order input from the scan determination unit 117 to generate scanned DCT coefficients, and the variable length encoding unit 114 Output to.

  Next, in step S <b> 408, the variable length coding unit 114 performs variable length coding on the scanned DCT coefficient input from the scanning unit 113 to generate an enhancement layer stream, and outputs the enhancement layer stream to the enhancement layer output unit 115.

  Next, in step S409, the enhancement layer output unit 115 stores the scan judgment formula input from the scan judgment formula generation unit 116 in the enhancement layer stream input from the variable length coding unit 114, and the video coding apparatus 100 externally. Output to. Also, the base layer output unit 103 outputs the base layer stream input from the base layer encoding unit 102 to the outside of the video encoding device 100.

  Finally, in step S410, the video signal input unit 101 determines the presence / absence of a video input from the outside of the video encoding device 110. If there is no video input, the processing ends. Otherwise, the process returns to step S401.

  Note that step S405 corresponds to the scan determination formula generation processing step of the present invention, step S406 corresponds to the scan determination processing step, and step S409 corresponds to the scan determination formula storage processing step.

  As described above, according to the first embodiment, the video encoding apparatus 100 uses the base layer decoded image whose scan determination formula generation unit 116 has a high correlation with the difference image to generate the base layer decoded image block. A scan determination formula for specifying the scan order of the encoded image block is generated from the edge, and the scan determination unit 117 uses the scan determination formula for the base layer decoded image, so that the information on the scan order is displayed for each image block. It is possible to specify a scan order having a short scan length for an enhancement layer image block without storing it in a stream, and to improve the coding efficiency of hierarchical coding.

  Further, according to the first embodiment, the video encoding device 100 stores the scan order information for each image block in the video stream by the enhancement layer encoding unit 115 storing the scan determination formula in the video stream. Without this, the scan order of all the encoded image blocks can be expressed only by the scan determination formula, and the encoding efficiency can be improved.

(Embodiment 2)
In the second embodiment, in decoding of the video stream output from the video encoding apparatus 100 to which the first embodiment is applied, a scan for generating the reverse scan order of the DCT coefficients of the image block of the enhancement layer The determination formula is acquired from the video stream, and the DCT coefficients of the enhancement layer image block are decoded by the scan order generated by the scan determination formula.

  FIG. 5 shows a configuration of a video decoding apparatus 500 to which the video decoding method according to Embodiment 2 of the present invention is applied.

  In FIG. 5, the video decoding apparatus 500 includes a video signal output unit 501, a base layer decoding unit 502, a base layer input unit 503, and an enhancement layer decoding unit 510. The enhancement layer decoding unit 510 includes a difference unit 511, an IDCT unit 512, an inverse scan unit 513, a variable length decoding unit 514, an enhancement layer input unit 515, a scan determination formula acquisition unit 516, and a scan determination unit 517.

  Base layer input section 503 outputs a base layer stream input from the outside of video decoding apparatus 500 to base layer decoding section 502. Also, the presence / absence of a video stream input from the outside is determined. If there is no video stream input, the process ends.

  Base layer decoding section 502 decodes the base layer stream input from base layer input section 503, generates a base layer decoded image, and outputs the base layer decoded image to enhancement layer decoding section 510.

  Here, the base layer decoded image matches the base layer decoded image generated by the video encoding device 100.

  The enhancement layer input unit 515 outputs the enhancement layer stream input from the outside of the video decoding device 500 to the variable length decoding unit 514 and the scan determination formula acquisition unit 516.

  The variable length decoding unit 514 performs variable length decoding on the enhancement layer stream input from the enhancement layer input unit 515 to generate a scanned DCT coefficient, and outputs the generated DCT coefficient to the inverse scanning unit 513.

  Note that in the case of encoding in which the video encoding device 100 performs quantization, the variable length decoding unit 514 performs inverse quantization processing after performing variable length decoding. Inverse quantization is a process of multiplying by a numerical value obtained by dividing the DCT coefficient.

  The inverse scan unit 513 generates the DCT coefficient by scanning the scanned DCT coefficient input from the variable length decoding unit 514 in the reverse order of the scan order input from the scan determination unit 517, and outputs the DCT coefficient to the IDCT unit 512. .

  The IDCT unit 512 performs IDCT conversion on the DCT coefficient input from the inverse scan unit 512 to generate a decoded image block. Decoded image blocks that have been divided into a plurality of regions are combined to generate a decoded difference image, which is output to the adder 511.

  Here, the IDCT (Inverse Discrete Cosine Transform) conversion is an inverse process for returning the DCT converted coefficient to the original.

  The scan determination formula acquisition unit 516 acquires a scan determination formula from the enhancement layer stream input from the enhancement layer input unit 515 and outputs the scan determination formula to the scan determination unit 517.

  The scan determination unit 517 generates a scan order for each image block with respect to the base layer decoded image input from the base layer decoding unit 502 using the scan determination formula input from the scan determination formula acquisition unit 516, and performs reverse processing. The data is output to the scan unit 513.

  The addition unit 511 generates a decoded image by adding the base layer decoded image input from the base layer decoding unit 502 and the decoded differential image input from the IDCT unit 512, and outputs the decoded image to the video signal output unit 501. .

  The video signal output unit 501 outputs the decoded image input from the adding unit 511 to the outside of the video decoding device 500.

  The scan determination formula acquisition unit 516 corresponds to the scan determination formula acquisition unit of the present invention, and the scan determination unit 517 corresponds to the scan determination unit.

  Next, the operation of the video decoding apparatus 500 configured as described above will be described. FIG. 6 is a flowchart showing an example of the operation of the video decoding apparatus 500 according to the second embodiment shown in FIG. Note that the flowchart shown in FIG. 6 is executed by software by executing a control program stored in a storage device (not shown) such as a ROM or a flash memory by a CPU (not shown). It is also possible to do so.

  First, in step S <b> 601, the base layer input unit 503 outputs the base layer stream input from the outside of the video decoding apparatus 500 to the base layer decoding unit 502. Also, the enhancement layer input unit 515 outputs the enhancement layer stream input from the outside of the video decoding apparatus 500 to the variable length decoding unit 514 and the scan determination formula acquisition unit 516.

  Next, in step S602, base layer decoding section 502 decodes the base layer stream input from base layer input section 503, generates a base layer decoded image, and outputs the base layer decoded image to enhancement layer decoding section 510.

  Next, in step S603, the variable length decoding unit 514 performs variable length decoding on the enhancement layer stream input from the enhancement layer input unit 515 to generate a scanned DCT coefficient, and outputs the generated DCT coefficient to the inverse scan unit 513.

  Next, in step S <b> 604, the scan determination formula acquisition unit 516 acquires a scan determination formula from the enhancement layer stream input from the enhancement layer input unit 515 and outputs the scan determination formula to the scan determination unit 517.

  In step S605, the scan determination unit 517 uses the scan determination formula input from the scan determination formula acquisition unit 516 for the base layer decoded image input from the base layer decoding unit 502 for each image block. A scan order is generated and output to the reverse scan unit 513.

  Next, in step S606, the IDCT unit 512 generates a decoded image block by performing IDCT conversion on the DCT coefficient input from the inverse scan unit 512, and generates a decoded differential image by combining the divided image blocks. And output to the adding unit 511.

  Next, in step S <b> 607, the inverse scan unit 513 scans the scanned DCT coefficients input from the variable length decoding unit 514 in the reverse order of the scan order input from the scan determination unit 517 to generate DCT coefficients. , Output to the IDCT unit 512.

  Next, in step S608, the addition unit 511 generates a decoded image by adding the base layer decoded image input from the base layer decoding unit 502 and the decoded differential image input from the IDCT unit 512 to generate a video image. The signal is output to the signal output unit 501.

  Next, in step S <b> 609, the video signal output unit 501 outputs the decoded image input from the addition unit 511 to the outside of the video decoding device 500.

  Finally, in step S610, the base layer decoding unit 502 determines whether or not there is a video stream input from the outside. If there is no video stream input, the process ends. Otherwise, the process returns to step S601.

  Note that step S604 corresponds to the scan determination formula acquisition processing step of the present invention, and step S605 corresponds to the scan determination processing step.

  As described above, according to the second embodiment, in video decoding apparatus 500, enhancement layer decoding section 515 acquires a scan determination formula from a video stream, and scan determination section 517 applies to a base layer decoded image. By using the scan determination formula, it is possible to specify the scan order without including the scan order of all the image blocks of the enhancement layer in the video stream, and the encoding efficiency is improved.

(Embodiment 3)
In the third embodiment, in video coding using prediction, a scan determination formula for generating a scan order of DCT coefficients of a coded image block is used as a reference image block for predictive coding of the coded image block. And the DCT coefficients are encoded according to the scan order generated by the scan determination formula.

  Unlike the first and second embodiments, this embodiment targets a video stream that is not hierarchically encoded.

  Further, although the present embodiment will be described with respect to encoding of video, that is, continuous images, it may be used for encoding of still images.

  FIG. 7 shows a configuration of a video encoding apparatus 700 to which the video encoding method according to Embodiment 3 of the present invention is applied.

  In FIG. 7, a video encoding apparatus 700 includes a video signal input unit 701, a predictive encoding unit 702, a DCT unit 703, a scanning unit 704, a variable length encoding unit 705, a stream output unit 706, a decoding unit 707, a scan determination. An expression generation unit 708 and a scan determination unit 709 are included.

  The video signal input unit 701 inputs video from the outside of the video encoding device 710 frame by frame as an original image and outputs it to the prediction encoding unit 702. Also, the presence / absence of a video input from the outside of the video encoding device 710 is determined. If there is no video input, the process is terminated.

  The prediction encoding unit 702 generates a difference image by prediction encoding using the original image input from the video signal input unit 701 and the decoded image block input from the decoding unit 707, and outputs the difference image to the DCT unit 703. In addition, the reference image used for predictive encoding is output to the scan determination formula generation unit 708. Also, prediction information is generated by predictive encoding and output to the stream output unit 706.

  Prediction coding will be described later.

  The DCT unit 703 divides the difference image input from the predictive encoding unit 702 into 4 × 4 pixel image blocks, performs DCT transformation and quantization for each image block, generates quantized DCT coefficients, and scan unit 704 and the decryption unit 707.

  Here, the image block is 8 × 8 pixels in the first embodiment, but in this embodiment, the image block is 4 × 4 pixels for the sake of simplicity of explanation. However, the size of the image block is not particularly limited.

  The scan unit 704 rearranges the DCT coefficients input from the DCT unit 703 using the scan order input from the scan determination unit 709 to generate a scanned DCT coefficient, and outputs the generated DCT coefficient to the variable length encoding unit 705.

  The variable length coding unit 705 generates a video stream by variable length coding the scanned DCT coefficient input from the scanning unit 704 and outputs the video stream to the stream output unit 706.

  The stream output unit 706 stores the prediction information input from the prediction encoding unit 702 and the scan determination expression input from the scan determination expression generation unit 708 in the video stream input from the variable length encoding unit 705, and the video encoding device Output outside 700.

  The decoding unit 707 performs IDCT conversion on the quantized DCT coefficient input from the DCT unit 703 to generate a decoded image block, and outputs the decoded image block to the prediction encoding unit 702.

  The scan determination formula generation unit 708 generates a scan determination formula using the reference image input from the prediction encoding unit 702 and outputs the scan determination formula to the stream output unit 706 and the scan determination unit 709.

  The scan determination unit 709 generates a scan order for each encoded image block using the reference image input from the base layer decoding unit 702 and the scan determination formula input from the scan determination formula generation unit 708, and outputs the scan order to the scan unit 704. To do.

  The scan determination formula generation unit 708 corresponds to the scan determination formula generation unit of the present invention, the scan determination unit 709 corresponds to the scan determination unit, and the stream output unit 706 corresponds to the scan determination formula storage unit.

  The predictive encoding performed by the predictive encoding unit 702 according to the present embodiment and the scan determination formula generation method performed by the scan determination formula generation unit 708 will be described below.

  The predictive coding unit 702 performs inter-screen predictive coding and intra-screen predictive coding used in MPEG-4 AVC described in ISO / IEC 14496-10, which is a known technique.

  The inter-picture prediction encoding will be described with reference to FIG.

  An area similar to the encoded image block 1601 in FIG. 16A is selected from past and future decoded images already stored in the video stream as an image block 1605 in FIG. 1605.

  In Patent Document 2, it is possible to select an image block to be referred to only in units of image blocks to be subjected to DCT conversion represented by the image block 1602 in FIG. 16A and the image blocks 1603 and 1604 in FIG. 16B. In the invention, it is possible to select an image block including a pixel at an arbitrary position, such as an image block 1605 in FIG.

  In the video stream, the prediction information indicating the image having the reference image block 1605, the position of the reference image block 1605, and the like, and the DCT coefficient of the difference between the reference image block 1605 and the encoded image block 1602 are encoded and stored. In general, since the reference image block 1605 is selected so that the difference from the encoded image block 1601 is small, inter-picture predictive encoding is performed by using the encoded image block 1601 as a video stream with a smaller code amount than when not using it. Can be stored.

  The predictive encoding unit 702 has a storage area for storing past and future decoded images for use in predictive encoding.

  In inter-picture predictive encoding, the reference image block is selected so that the difference from the encoded image block is small. For example, in the reference image, a reference image block is selected that minimizes the absolute sum of squares of differences for each pixel from the encoded image block. Therefore, the edge characteristics of the two image blocks are similar, and the determination formula generation method described in the first embodiment can be used in the same manner.

  In addition, the scan determination formula may include a reference image block selection method for predictive coding.

  Thereby, for example, if the intensity of the horizontal edge of the reference image block is equal to or greater than a certain threshold value, the rule is to select a reference image block in which the horizontal scan of the difference DCT coefficient from the encoded image block is the shortest scan. This makes it possible to specify the scan order from the reference image block.

  The intra prediction encoding will be described with reference to FIG.

  Reference numeral 801 in FIG. 8A denotes an image block of 4 × 4 pixels to be encoded. The surrounding cells represent the decoded pixels that have already been encoded and stored in the video stream. In intra prediction encoding, a reference image block is obtained by copying a decoded pixel around an encoded image block to the position of an image block 801 according to one of the prediction directions in FIG. 8B.

  The prediction information indicating the prediction direction and the DCT coefficient of the difference between the image block 801 and the reference image block are encoded and stored in the video stream. In general, since the reference image block is generated so that the difference from the image block 801 is small, the intra-frame predictive coding stores the coded image block 1602 in the video stream with a smaller code amount than when not using it. Is possible.

  In the intra prediction encoding, the reference image block is selected so that the difference from the encoded image block is small. For example, a prediction direction in which the absolute sum of squares of differences for each pixel from the encoded image block is selected from all prediction directions is selected. Therefore, the edge characteristics of the two image blocks are similar, and the determination formula generation method described in the first embodiment can be used in the same manner.

  Further, the prediction direction of the intra prediction shown in FIG. 8B may be included in the conditions of the scan determination formula.

  Thereby, for example, if the prediction direction is just beside, there is no edge in the horizontal direction, and the processing load is reduced by omitting the calculation of the strength of the edge in the horizontal direction.

  Next, the operation of the video encoding apparatus 700 configured as described above will be described. FIG. 9 is a flowchart showing an example of the operation of the video encoding apparatus 700 according to the third embodiment shown in FIG. The flowchart shown in FIG. 9 is executed by software by executing a control program stored in a storage device (not shown) such as a ROM or a flash memory by a CPU (not shown). It is also possible to do so.

  First, in step S <b> 901, the video signal input unit 701 inputs a video as an original image frame by frame from the outside of the video encoding device 710 and outputs it to the predictive encoding unit 702.

  Next, in step S902, the prediction encoding unit 702 generates a difference image by predictive encoding using the original image input from the video signal input unit 701 and the decoded image block input from the decoding unit 707, and The data is output to the DCT unit 703. Also, prediction information is generated by predictive encoding and output to the stream output unit 706.

  Next, in step S903, the DCT unit 703 divides the difference image input from the predictive encoding unit 702 into 4 × 4 pixel image blocks, performs DCT transformation and quantization for each image block, and performs quantization DCT. A coefficient is generated and output to the scanning unit 704 and the decoding unit 707.

  Next, in step S904, the decoding unit 707 generates a decoded image block by performing IDCT conversion on the quantized DCT coefficient input from the DCT unit 703, and outputs the decoded image block to the prediction encoding unit 702.

  In step S 905, the scan determination formula generation unit 708 generates a scan determination formula using the reference image input from the prediction encoding unit 702 and outputs the scan determination formula to the stream output unit 706 and the scan determination unit 709.

  In step S906, the scan determination unit 709 generates a scan order for each encoded image block using the reference image input from the base layer decoding unit 702 and the scan determination formula input from the scan determination formula generation unit 708. And output to the scan unit 704.

  Next, in step S907, the scan unit 704 generates a scanned DCT coefficient by rearranging the DCT coefficients input from the DCT unit 703 using the scan order input from the scan determination unit 709, and the variable length encoding unit 705. Output to.

  In step S <b> 908, the variable length coding unit 705 generates a video stream by variable length coding the scanned DCT coefficient input from the scanning unit 704, and outputs the video stream to the stream output unit 706.

  Next, in step S909, the stream output unit 706 adds the prediction information input from the prediction encoding unit 702 and the scan determination expression input from the scan determination expression generation unit 708 to the video stream input from the variable length encoding unit 705. Store and output to the outside of the video encoding device 700.

  Finally, in step S910, the video signal input unit 701 determines whether there is a video input from the outside of the video encoding device 710. If there is no video input, the process ends. Otherwise, the process returns to step S901.

  Note that step S905 corresponds to the scan determination formula generation processing step of the present invention, step S906 corresponds to the scan determination processing step, and step S909 corresponds to the scan determination formula storage processing step.

  As described above, according to the third embodiment, in video encoding apparatus 700, scan determination expression generation section 708 generates a scan determination expression using a reference image for predictive encoding, and scan determination section 709 predicts. By using the scan determination formula for the reference image for encoding, it is possible to generate a scan order with a short scan length for all image blocks, and to improve the encoding efficiency of predictive encoding. is there.

  Further, according to the third embodiment, in video encoding apparatus 700, scan determination formula generation section 708 generates a scan determination formula using prediction information, and predictive encoding section 702 uses scan determination formula information. By using the prediction, it is possible to generate a scan order having a shorter scan length, and to improve the encoding efficiency.

(Embodiment 4)
In the fourth embodiment, in decoding a video stream output from the video encoding apparatus 700 to which the third embodiment is applied, a scan for generating a reverse scan order of DCT coefficients of an image block to be decoded The determination formula is acquired from the video stream, and the DCT coefficient of the image block is decoded by the scan order generated by the scan determination formula.

  FIG. 10 shows the configuration of a video decoding apparatus 1000 to which the video decoding method according to Embodiment 4 of the present invention is applied.

  In FIG. 10, the video decoding apparatus 1000 includes a video signal output unit 1001, a predictive decoding unit 1002, an IDCT unit 1003, an inverse scan unit 1004, a variable length decoding unit 1005, a stream input unit 1006, and a scan determination formula acquisition unit 1007. And a scan determination unit 1008.

  The stream input unit 1006 inputs a video stream from the outside of the video decoding apparatus 1000 and outputs the video stream to the variable length decoding unit 1005, the prediction decoding unit 1002, and the scan determination formula acquisition unit 1007. Also, the presence / absence of a video stream input from the outside is determined. If there is no video stream input, the process ends.

  The variable length decoding unit 1005 performs variable length decoding on the video stream input from the stream input unit 1006 to generate a scanned DCT coefficient, and outputs the generated DCT coefficient to the inverse scanning unit 1004.

  The inverse scan unit 1004 scans the scanned DCT coefficients input from the variable length decoding unit 1005 in the reverse order of the scan order input from the scan determination unit 1008 to generate quantized DCT coefficients, and sends them to the IDCT unit 5J03. Output.

  The IDCT unit 1003 performs inverse quantization and IDCT conversion on the quantized DCT coefficient input from the inverse scan unit 1004 to generate a decoded image block, and outputs the decoded image block to the predictive decoding unit 1002.

  The scan determination formula acquisition unit 1007 acquires the scan determination formula from the video stream input from the stream input unit 1006 and outputs the scan determination formula to the scan determination unit 1008.

  The scan determination unit 1008 generates a scan order for each image block using the scan determination formula input from the scan determination formula acquisition unit 1007 for the reference image input from the predictive decoding unit 1002, and sends the scan order to the inverse scan unit 1004. Output.

  The predictive decoding unit 1002 acquires prediction information from the video stream input from the stream input unit 1006, generates a decoded image by predictive decoding using the decoded image block input from the IDCT unit 1003, and outputs a video signal. Output to the unit 1001. In addition, the reference image used for predictive decoding is output to the scan determination unit 708.

  The video signal output unit 1001 outputs the decoded image input from the predictive decoding unit 1002 to the outside of the video decoding apparatus 1000.

  The scan determination formula acquisition unit 1007 corresponds to the scan determination formula acquisition unit of the present invention, and the scan determination unit 1008 corresponds to the scan determination unit.

  Next, the operation of the video decoding apparatus 1000 configured as described above will be described. FIG. 11 is a flowchart showing an example of the operation of the video decoding apparatus 1000 according to the fourth embodiment shown in FIG. The flowchart shown in FIG. 11 is executed by software by executing a control program stored in a storage device (not shown) such as a ROM or a flash memory by a CPU (not shown). It is also possible to do so.

  First, in step S1101, the stream input unit 1006 inputs a video stream from the outside of the video decoding apparatus 1000, and outputs the video stream to the variable length decoding unit 1005, the predictive decoding unit 1002, and the scan determination formula acquisition unit 1007.

  Next, in step S1102, the variable length decoding unit 1005 performs variable length decoding on the video stream input from the stream input unit 1006, generates a scanned DCT coefficient, and outputs the generated DCT coefficient to the inverse scanning unit 1004.

  Next, in step S <b> 1103, the scan determination formula acquisition unit 1007 acquires a scan determination formula from the video stream input from the stream input unit 1006 and outputs the scan determination formula to the scan determination unit 1008.

  In step S1104, the scan determination unit 1008 generates a scan order for each image block using the scan determination formula input from the scan determination formula acquisition unit 1007 for the reference image input from the predictive decoding unit 1002. And output to the reverse scan unit 1004.

  Next, in step S1105, the inverse scan unit 1004 scans the scanned DCT coefficients input from the variable length decoding unit 1005 in the reverse order of the scan order input from the scan determination unit 1008 to obtain the quantized DCT coefficients. Generate and output to the IDCT unit 5J03.

  Next, in step S 1106, the IDCT unit 1003 performs inverse quantization and IDCT transformation on the quantized DCT coefficient input from the inverse scan unit 1004, generates a decoded image block, and outputs the decoded image block to the predictive decoding unit 1002.

  Next, in step S1107, the predictive decoding unit 1002 acquires prediction information from the video stream input from the stream input unit 1006, and uses the decoded image block input from the IDCT unit 1003 to decode a decoded image by predictive decoding. Is output to the video signal output unit 1001. In addition, the reference image used for predictive decoding is output to the scan determination unit 708.

  Next, in step S1108, the video signal output unit 1001 outputs the decoded image input from the predictive decoding unit 1002 to the outside of the video decoding apparatus 1000.

  Finally, in step S1109, the stream input unit 1006 determines whether there is a video stream input from the outside. If there is no video stream input, the process ends. Otherwise, the process returns to step S1101.

  Note that step S1103 corresponds to the scan determination formula acquisition processing step of the present invention, and step S1104 corresponds to the scan determination processing step.

  As described above, according to the fourth embodiment, in the video decoding apparatus 1000, the scan determination formula acquisition unit 1007 acquires the scan determination formula from the video stream, and the scan determination unit 517 converts it into a reference image for prediction encoding. On the other hand, by using the scan determination formula, it is possible to specify the scan order without including the scan order of all image blocks in the video stream, and the encoding efficiency is improved.

  INDUSTRIAL APPLICABILITY The present invention is useful because it is possible to improve encoding efficiency by generating a scan order suitable for each image block.

  In addition, the present invention is particularly applicable to the hierarchical coding scheme, and is useful because it can improve the coding efficiency of the enhancement layer.

  Therefore, after encoding a video stream, it is suitable for the purpose of decoding by adjusting the code amount.

  In addition, the present invention is particularly applicable to a predictive coding scheme, and is useful because it can improve the coding efficiency of predictive coding.

  Therefore, it is suitable for encoding video, that is, continuous images. That is, it is suitable for a video encoding system of a system that transmits and receives video while dynamically changing the amount of the video stream via a network whose communication speed varies. Further, the present invention is suitable for a video encoding system of a system that distributes one video content to a plurality of users having different communication speeds and terminal processing capabilities at different image quality and communication speeds. Further, the present invention is suitable for a video encoding method of a video storage apparatus that changes the storage capacity without decoding and re-encoding after storing the video stream.

The figure which shows the structure of the video coding apparatus by Embodiment 1 of this invention. Diagram showing an example of the structure of the enhancement layer stream The figure which shows the example of the horizontal high frequency of a DCT coefficient, and a vertical high frequency The flowchart which shows operation | movement of the video coding apparatus by Embodiment 1 of this invention. The figure which shows the structure of the video decoding apparatus by Embodiment 2 of this invention. The flowchart which shows operation | movement of the video decoding apparatus by Embodiment 2 of this invention. The figure which shows the structure of the video coding apparatus by Embodiment 3 of this invention. Diagram showing an example of in-screen prediction The flowchart which shows operation | movement of the video coding apparatus by Embodiment 3 of this invention. The figure which shows the structure of the video decoding apparatus by Embodiment 4 of this invention. The flowchart which shows operation | movement of the video decoding apparatus by Embodiment 4 of this invention. Diagram showing an example of quantization Diagram showing an example of scan order The figure which shows the structure of the video coding apparatus by patent document 1 The figure which shows the structure of the video coding apparatus by patent document 2 Conceptual diagram of Patent Document 2

Explanation of symbols

101, 701 Video signal input unit 102 Base layer encoding unit 103 Base layer output unit 110 Enhancement layer encoding unit 111 Difference unit 112, 703 DCT unit 113, 704 Scan unit 114, 705 Variable length encoding unit 115 Enhancement layer output unit 116, 516, 708, 907 Scan determination formula generation unit 117, 517, 709, 908 Scan determination unit 501, 901 Video signal output unit 502 Base layer decoding unit 503 Base layer input unit 510 Enhancement layer decoding unit 511 Addition unit 512 903 IDCT unit 513 904 Reverse scan unit 514 905 Variable length decoding unit 515 Enhancement layer input unit 702 Predictive coding unit 706 Stream output unit 707 Decoding unit 902 Predictive decoding unit 906 Stream input unit 201 Enhancement layer stream head 202 code scan judgment formula 203 image blocks

Claims (20)

The scan order, which is the order in which an encoded image block, which is a set of several pixels in the image to be encoded, is encoded and stored in the video stream is determined by the scan judgment formula for the reference image block to be referred to at the time of encoding. A scan determination means for determining; a scan determination expression generation means for generating the scan determination expression using a reference image referred to in encoding; a scan determination expression storage means for storing the scan determination expression in a video stream; A video encoding device. The video encoding apparatus according to claim 1, wherein the scan determination formula storage unit stores the information of the scan determination formula at a position of the video stream before the code of the encoded image block. The scan determination formula generation means compares the horizontal edge and the vertical edge of the reference image, and if the horizontal edge is included more than the vertical edge, the horizontal direction is given priority, and the vertical edge is determined. 2. The video encoding apparatus according to claim 1, wherein if there are more edges than in the horizontal direction, the scan determination formula is generated to cause the scan determination unit to determine the scan order in preference to the vertical direction. The scan determination formula generation unit compares the horizontal high-frequency coefficient and the vertical high-frequency coefficient of the DCT coefficient of the reference image, and if the horizontal high-frequency coefficient is included more than the vertical high-frequency coefficient, the horizontal direction is given priority, and the vertical high-frequency coefficient is set as the horizontal high-frequency coefficient. 4. The video encoding apparatus according to claim 3, wherein when the number of coefficients is greater than the number of coefficients, the scan determination formula is generated to cause the scan determination unit to determine a scan order in preference to the vertical direction. The video encoding apparatus according to claim 1, wherein the scan determination formula generation unit generates the scan determination formula by predicting a feature of the encoded image block from statistics of information for each reference image block. The video encoding apparatus according to claim 1, wherein the scan determination formula generation unit generates the scan determination formula for each of the plurality of encoded image blocks. The video encoding apparatus according to claim 1, wherein the scan determination formula generation unit changes a parameter value included in the scan determination formula using the past scan determination formula. The video encoding apparatus according to claim 1, wherein the scan determination expression generation unit generates the scan determination expression using a result of scanning the encoded image block. The video encoding apparatus according to claim 1, wherein the scan determination formula generation unit prepares and switches a plurality of scan determination formulas in advance. In the hierarchical coding for hierarchizing and encoding a base layer capable of decoding video alone and a decoded image of the basic layer into an enhancement layer for improving the image quality, the scan determination formula generating means converts the base layer decoded image into the base layer decoded image The video encoding apparatus according to claim 1, wherein the scan determination formula for determining the scan order of the encoded block of the enhancement layer corresponding to the reference image block of the base layer is generated as a reference image. The video encoding apparatus according to claim 1, wherein the scan determination expression generation unit generates the scan determination expression using an image referred to in the prediction encoding as the reference image in predictive encoding of a video. The video encoding apparatus according to claim 11, wherein the predictive encoding is inter-picture predictive encoding. The video encoding device according to claim 11, wherein the predictive encoding is intra-screen predictive encoding. The predictive coding uses the reference image block predicted by the scan determination formula for predictive coding when the code amount of the coded image block is the smallest among the reference image block candidates of the predictive coding. The video encoding device according to claim 11. 12. The video encoding apparatus according to claim 11, wherein the scan determination formula generation unit generates the scan determination formula corresponding to the reference image block using information used by the prediction encoding for prediction. The scan order, which is the order in which the encoded image block, which is a set of several pixels in the encoded image, is encoded and stored in the video stream is determined by the scan determination formula for the reference image block that is referred to at the time of decoding. A video decoding apparatus configured to include: a scan determination unit that determines; and a scan determination formula acquisition unit that acquires the scan determination formula from a video stream. A video encoding method for encoding video, wherein the video encoding device executes
Scan determination formula for other reference image blocks that are referred to when encoding the scan order, which is the order in which an encoded image block that is a set of several pixels in the image to be encoded is encoded and stored in the video stream Scanning determination processing step determined by:
A scan determination expression generation processing step for generating the scan determination expression using a reference image referred to in encoding;
And a scan determination expression storing processing step of storing the scan determination expression in a video stream. A video decoding method for decoding a video stream, which is executed by a video decoding device,
The scan order, which is the order in which an encoded image block, which is a set of several pixels in the encoded image, is encoded and stored in the video stream is determined by the scan determination formula for the reference image block that is referred to at the time of decoding. A video decoding method comprising: a scan determination processing step for determining; and a scan determination formula acquisition processing step for acquiring the scan determination formula from a video stream. Computer to perform video encoding,
Scan determination formula for other reference image blocks that are referred to when encoding the scan order, which is the order in which an encoded image block that is a set of several pixels in the image to be encoded is encoded and stored in the video stream Scanning determination processing step determined by:
A scan determination expression generation processing step for generating the scan determination expression using a reference image referred to in encoding;
A video encoding program that operates as a scan determination formula storage processing step for storing the scan determination formula in a video stream. A computer to perform the decoding of the video stream,
The scan order, which is the order in which the encoded image block, which is a set of several pixels in the encoded image, is encoded and stored in the video stream is determined by the scan determination formula for the reference image block that is referred to at the time of decoding. A video decoding program that operates as a scan determination processing step to determine and a scan determination formula acquisition processing step to acquire the scan determination formula from a video stream.

Images