JP5038367B2 - Scalable video encoding method, scalable video encoding device, and scalable video encoding program - Google Patents

Description

  The present invention relates to a scalable video encoding method and apparatus for encoding a video in a scalable manner, and a scalable video encoding program used for realizing the scalable video encoding method. The present invention relates to a scalable moving picture coding method and apparatus for realizing the reduction, and a scalable moving picture coding program used for realizing the scalable moving picture coding method.

  In response to the background of various display terminals and network environments in recent years, JVT (Joint Video Team) has a space / time / SNR for AVC (Advanced Video Coding Standard). An encoding scheme SVC (Scalable Video Coding) to which the above scalability has been added has been studied (for example, see Non-Patent Document 1).

  SVC adopts three prediction methods, namely, Inter prediction, Intra prediction, and inter-layer prediction, and performs redundancy removal inherent in time, space, and layers. The prediction modes that can be used in SVC are listed below.

[Inter prediction]
・ Skip mode (Skip)
・ Direct mode (Direct)
16 × 16 block size motion prediction mode (P16 × 16)
16 × 8 block size motion prediction mode (P16 × 8)
8 × 16 block size motion prediction mode (P8 × 16)
8 × 8 block size motion prediction mode (P8 × 8)
[Intra prediction]
16 × 16 block size intra prediction mode (I16 × 16)
・ 8 × 8 block size intra prediction mode (I8 × 8)
・ 4 × 4 block size intra prediction mode (I4 × 4)
[Inter-layer prediction]
・ BLskip mode (BLskip)
・ IntraBL mode (IntraBL)
Each 8 × 8 block when performing P8 × 8 can be further divided into block sizes of 8 × 4, 4 × 4, and 4 × 4. In SVC, one of these prediction mode search candidates is selected as the optimum prediction mode for each macroblock.

  An example of a method for determining the optimum prediction mode is given below.

  In JSVM provided by JVT as an SVC reference encoder (see, for example, Non-Patent Document 2), the encoding cost including the code amount and the encoding distortion is calculated in each prediction mode, and all the predictions listed above are calculated. The prediction mode with the lowest coding cost among the modes is determined as the optimum.

  Further, in Patent Document 1 shown below, a vector obtained by extrapolating / interpolating a motion vector of a reference frame to an encoding target frame is generated, and coordinates of each pixel of the moved macroblock are obtained thereby to match the pixels. The number of times is counted for each pixel. Then, prediction mode search candidates are narrowed down according to the score value calculated from the count number of each pixel in the encoding target macroblock. This narrowing down method is H.264. H.264 / AVC is proposed for speeding up the prediction mode search. The present invention is also applicable to SVC, which is the same prediction mode search mechanism as H.264 / AVC.

  Further, in Patent Document 2 shown below, for example, nine screens of blocks that perform intra-frame coding using pixel values of adjacent coding blocks in order to perform intra-frame coding at high speed. An inner prediction error is obtained, and a prediction mode of the block is determined based on the inner prediction error. Next, when the prediction mode of the block is determined using the intra prediction mode of the adjacent coded block and the two prediction modes match, the prediction mode is selected as it is. The prediction mode with the lower coding cost is selected.

  However, in the determination method of the optimal prediction mode in JSVM of Non-Patent Document 2, since prediction mode search candidates are not narrowed down, high coding performance can be realized, but a huge amount of time is required for the prediction mode search. A prediction mode (for example, an intra prediction mode in a still region) that is clearly not likely to be selected in consideration of the characteristics of an image in a macroblock is also searched for, which is wasteful.

  Further, narrowing down prediction mode search candidates in Patent Document 1 is a method for determining whether or not to perform intra prediction, and therefore, the effect of reducing the Inter prediction mode search that requires a longer calculation time than the search in intra prediction mode. There is no. That is, the room for improvement is left as it is for the search in the Inter prediction mode.

  In addition, since narrowing down prediction mode search candidates in Patent Document 2 is narrowing down only to intra prediction, similarly to narrowing down prediction mode search candidates in Patent Document 1, a long calculation time is required as compared to the search in Intra prediction mode. There is no reduction effect of searching for the Inter prediction mode. That is, the room for improvement is left as it is for the search in the Inter prediction mode.

  Against this backdrop, the present inventor based on the information on the optimal prediction mode selected in scalable video coding performed without restriction on the use of the prediction mode in Non-Patent Document 3 below. A correspondence table describing the correspondence between the combination of the optimum prediction mode and the occurrence rate by obtaining the occurrence rate of the combination of the optimum prediction mode of the upper layer and the lower layer selected in the spatially corresponding block. When the upper layer block is encoded, the information of the optimal prediction mode selected by the encoding of the spatially corresponding block of the lower layer is acquired, and the acquired optimal prediction mode is obtained. Disclosed an invention that narrows down search candidates for a prediction mode to be searched by encoding a block of a higher layer based on the information and its correspondence table

JP 2006-033451 A JP-A-2005-184241

T. Wiegand, G. Sullivan, J. Reichel, H. Schwarz and M. Wien: "Joint Draft ITU-T Rec. H.264 ｜ ISO / IEC 14496-10 / Amd.3 Scalable video coding," ISO / IEC JTC1 / SC29 / WG11 and ITU-T SG16 Q.6, JVT-X201,2007.http: //ftp3.itu.ch/av-arch/jvt-site/2007_06_Geneva/JVTX201.zip J. Reichel, H. Schwarz and M. Wien: "Joint Scalable Video Model JSVM-11," ISO / IEC JTC1 / SC29 / WG11 and ITU-T SG16 Q.6, JVT-X202, 2007. http: // ftp3 .itu.ch / av-arch / jvt-site / 2007_06_Geneva / JVTX202.zip Kazuya Hayase, Yukihiro Bando, Masayuki Takamura, Kazuo Uekura, Yoshiyuki Yashima: "Acceleration of mode selection using inter-layer prediction mode correlation in SVC", Image Coding Symposium PCSJ2008.

  The invention disclosed by the present inventor in Non-Patent Document 3 uses the correlation of prediction modes between layers in scalable video coding to narrow down search candidates for prediction modes of macroblocks in an enhancement layer. Therefore, it is possible to realize high speed scalable video encoding.

  However, the invention disclosed in Non-Patent Document 3 assumes a layer structure in which upper layer blocks and lower layer blocks correspond one-to-one spatially, and does not have such a layer structure. Remains a problem that cannot be applied.

  That is, the invention disclosed in Non-Patent Document 3 can be applied when a power-of-two resolution scalability is established between the upper layer and the lower layer, but when a power-of-two resolution scalability is not established, This is not applicable because there are a plurality of optimum prediction modes in the image area immediately below the macroblock of the enhancement layer.

  By the way, even when a lower layer video is generated by cutting out a part of the video from the upper layer video, the upper layer block and the lower layer block have a layer structure that does not spatially correspond one-to-one. That is, this problem occurs in such a case as well.

  The present invention has been made in view of such circumstances, and in scalable video coding that achieves scalability by layer structure, narrowing down prediction mode search candidates for higher layers using the correlation of optimal prediction modes between layers The purpose of this is to realize a new scalable moving picture coding technique that increases the speed, and in this realization, the layer structure has a spatially one-to-one correspondence between the upper layer block and the lower layer block. It is an object to make it possible to increase the speed when a non-corresponding structure is adopted.

[1] Basic concept of the present invention In order to achieve this object, the present invention provides a scalable video coding that realizes scalability by a layer structure.
(I) Prediction mode correspondence rate table (describes the correlation of optimum prediction modes between layers
(Ii) Speeding up the prediction mode search is realized by two processes of narrowing down prediction mode search candidates using the prediction mode correspondence rate table.

  At this time, considering that the layer structure has a structure in which the block of the upper layer and the block of the lower layer do not spatially correspond to each other, processing is performed so as to generate a prediction mode correspondence rate table, In consideration of this, processing is performed so as to narrow down prediction mode search candidates using the prediction mode correspondence rate table.

  Hereinafter, the description proceeds according to an example as shown in FIG. That is, it is assumed that both layer L and layer L-1 are encoded with a hierarchical B structure of IBBBP. Arrows in the figure indicate prediction reference destinations. The encoding target layer is L, the encoding target frame is B2b, and the generation target frame of the prediction mode correspondence rate table is B2a. Also, the frame of layer L-1 at the same time B2b is B'2b, and the frame of layer L-1 at the same time B2a is B'2a. It is assumed that encoding is performed in the order from the lowest time level, and in the same time level, the coding is performed in the order of frames with the earlier time. Further, it is assumed that layers are encoded in ascending order of level.

  Next, a large flow of processing according to the present invention will be described with reference to the flowchart shown in FIG.

  In the present invention, in the case of scalable coding of a moving image, as shown in the flowchart of FIG. 2, first, in step S101, 1 is set to the variable n, and in the subsequent step S102, all frames are coded. If it is determined whether or not all frames have been encoded, the process ends.

  On the other hand, when it is determined that all the frames are not encoded according to the determination process in step S102, the process proceeds to step S103, and one unprocessed frame is selected according to the order from the first frame, and in subsequent step S104. The selected frame is encoded by performing the prediction without limiting the use of the prediction mode defined as usable.

  Subsequently, in step S105, the value of the variable n is incremented by one, and in the subsequent step S106, it is determined whether or not the value of the variable n has become larger than a predetermined threshold value N1 (N1 is an integer of 1 or more). Thus, when it is determined that it is not larger than the threshold value N1, the process returns to the process of step S102 to encode the frame without limiting the use of the prediction mode defined as usable. continue.

  On the other hand, when it is determined in step S106 that the value of the variable n has become larger than the threshold value N1, the process proceeds to step S107 to correlate the optimum prediction mode between layers having a data structure as described later. A prediction mode correspondence rate table describing the sex is generated.

  Subsequently, in step S108, the variable n is set to 1, and in the subsequent step S109, it is determined whether or not all the frames have been encoded, and when it is determined that all the frames have been encoded, The process ends.

  On the other hand, when it is determined that all the frames are not encoded according to the determination process in step S109, the process proceeds to step S110, and one unprocessed frame is selected according to the order from the first frame, and in the subsequent step S111. The selected frame is encoded by performing prediction while narrowing down prediction mode search candidates using the prediction mode correspondence rate table.

  Subsequently, in step S112, the value of the variable n is incremented by one, and in the subsequent step S113, it is determined whether or not the value of the variable n has become larger than a predetermined threshold value N2 (N2 is an integer of 1 or more). Thus, when it is determined that it is not larger than the threshold value N2, by returning to the process of step S109, it is possible to encode the frame while narrowing down prediction mode search candidates using the prediction mode correspondence rate table. continue.

  On the other hand, when it is determined in step S113 that the value of the variable n is larger than the threshold value N2, it is determined that the prediction mode correspondence rate table needs to be updated, and the process returns to step S101. As a result, the processing of step S101 to step S113 is continued while updating the prediction mode correspondence rate table.

  As described above, in the present invention, when a moving image is scalable encoded, a prediction mode that describes the correlation of the optimum prediction mode between layers based on the encoding result after encoding N1 frames. A correspondence rate table is generated, and then N2 frames are encoded. The N2 frames are searched while narrowing down prediction mode search candidates using the generated prediction mode correspondence rate table. It processes so that encoding may be repeated.

(I) Prediction mode correspondence rate table generation processing Next, the prediction mode correspondence rate table generation processing executed in step S107 will be described. The following description is an example in the case of performing spatial scalable encoding at an arbitrary magnification generated by layer L-1 video L clipping processing.

  Here, in this cut-out process, for example, layer L-1 video is generated by removing the upper, lower, left, and right end portions of the layer L video, whereby the upper layer block and the lower layer block are generated. And take a layer structure that does not spatially correspond to one to one.

  According to the processing in step S104, the generation target frame B2a and the B′2a immediately below the generation target frame B2a of the prediction mode correspondence rate table shown in FIG. 1 have already been encoded, and the optimal prediction mode has already been selected.

  When the frames B2a and B'2a are encoded, information on the selected optimum prediction mode is stored in the buffer. Then, the optimal prediction mode of the macroblock (hereinafter abbreviated as MB) of the frame B2a stored in the buffer, and the optimal prediction mode of the MB to which the block included in the spatially corresponding image area of the frame B′2a belongs Check the correspondence with.

  FIG. 3 shows these spatial positional relationships. When the layer L-1 video is generated by cutting out the layer L video, there may be a maximum of four blocks of the frame B′2a spatially corresponding to the MB of the frame B2a. Here, as an example, it is assumed that there may be four, and these blocks are defined as BLK1 to BLK4, respectively.

  The generation of the prediction mode correspondence rate table is performed by the following two procedures.

  First, when the MB of the frame B2a is a certain prediction mode, which prediction mode is selected in the BLK1 to BLK1 of the frame B'2a immediately below it is analyzed, and the number of pixels of the corresponding combination is calculated. Then, the number of pixels is input to each cell of the table as shown in FIG. 4, and the numerical values are cumulatively added. For example, P8 × 16 is selected in 25 pixels in the image area immediately below the MB in which P16 × 16 is selected in the frame B2a.

  Next, a prediction mode correspondence rate table as shown in FIG. 5 is generated. The numerical values in FIG. 5 are obtained by dividing the number of pixels in each cell by the cumulative number of pixels in each prediction mode on the horizontal axis in FIG. This indicates the ratio of each prediction mode selected by the MB of the frame B2a immediately above when a certain prediction mode is selected in a certain image region of the frame B'2a.

  The selection method of the optimal prediction mode in the frames B2a and B′2a may be the JSVM method described in Non-Patent Document 2 or the method of narrowing down prediction mode search candidates as described in Patent Document 1. In this example, the generation target frame of the prediction mode correspondence rate table is one encoded frame in the same time layer as the encoding target frame, but is not limited thereto. An encoded frame (for example, frame B1) of a different time layer may be a generation target. Further, the prediction mode correspondence rate table may be calculated by accumulating the plurality of frames with a plurality of frames as generation targets (for example, frames B1 and B2a). That is, any frame that has been encoded in the encoding target layer and the layer immediately below the encoding target layer can be a generation target frame of the prediction mode correspondence rate table.

(Ii) Prediction mode search candidate narrowing process using prediction mode correspondence rate table Next, the prediction mode search candidate narrowing process using the prediction mode correspondence rate table executed in step S111 will be described.

  In accordance with the value of the prediction mode correspondence rate in the prediction mode correspondence rate table generated in step S107, the prediction mode search candidates are narrowed down in each MB of the encoding target frame B2b. The numerical value in this prediction mode correspondence rate table is regarded as the occurrence probability of each prediction mode in the encoding target frame B2b. That is, when the optimum prediction mode j (prediction mode on the vertical axis of the prediction mode correspondence rate table shown in FIG. 5) is selected in an image region with the frame B′2b, the spatial correspondence of the encoding target frame B2b The probability that the prediction mode i (the prediction mode on the horizontal axis of the prediction mode correspondence rate table shown in FIG. 5) is optimal in the MB to which the image region belongs is described in each cell.

The encoding target macroblock of the encoding target frame B2b is denoted as MB _L, and the block of the frame B′2b of the layer L-1 located spatially at the same position as MB _L is represented by BLK1 _L−1 , BLK2 _L−1 , Indicated as BLK3 _L-1 and BLK4 _L-1 .

  First, according to any one of the following three methods (a) to (c), the reference prediction mode to be input to the vertical axis j of the prediction mode correspondence rate table is determined.

  Here, with respect to the reference prediction mode, it is possible to determine one prediction mode as the reference prediction mode and it is also possible to determine a plurality of prediction modes as the reference prediction mode, but in the example shown below, It is assumed that one reference prediction mode is determined.

(A) Prediction mode with maximum accumulated area Among BLK1 _{L-1 to} BLK4 _L-1 , a prediction mode of a block with the maximum cumulative area is identified, and is used as a reference prediction mode of the prediction mode correspondence rate table And At this time, if there is a prediction mode in which the accumulated area is the same, it is determined according to a predetermined method. For example, the following methods (b) and (c) can be considered. The blocks targeted for the cumulative area need not remain in BLK1 _{L-1 to} BLK4 _L-1 . You may include the block around the said area | region in the predetermined range. Further, referring to distance information or the like, the area may be weighted to obtain an accumulation.

(B) Prediction mode with highest priority Prioritization is performed between the initial prediction modes in advance, and the prediction with the highest priority among the prediction modes that BLK1 _{L-1 to} BLK4 _L-1 can take. The mode is identified and used as the reference prediction mode of the prediction mode correspondence rate table. As an example of how to set the priority, a method of increasing the priority in the order in which narrowing down is performed with emphasis on high speed, and conversely, the priority is increased in the order of narrowing down with emphasis on coding performance. Possible methods.

(C) Prediction mode to be referred to in inter-layer prediction A prediction mode to be referred to between layers is investigated and set as a reference prediction mode in the prediction mode correspondence rate table.

When the reference prediction mode to be input to the vertical axis j of the prediction mode correspondence rate table is determined in this way, next, the prediction mode correspondence rate table and the reference prediction mode are collated, and each macroblock MB _L is encoded. Investigate the probability that the prediction mode can be optimal. Here, the reference prediction mode is a prediction mode input on the vertical axis of FIG. Then, the prediction mode search candidates are narrowed down based on the probability that the optimum prediction mode can be obtained. Two examples of narrowing down are shown below.

(A) Refinement method 1
The narrowing down method 1 is a method of narrowing down prediction mode search candidates using a prediction mode search candidate narrowing threshold.

  In this narrowing down method 1, a prediction mode search candidate narrowing down threshold t% is provided, and a prediction mode less than this threshold t% is excluded from search candidates. The value of the threshold value t is given from the outside. As an example of a method for determining a value, a method for determining a value that suppresses deterioration in coding performance within an allowable range by performing a plurality of encoding processes may be considered.

Here, the time of obtaining the reference prediction mode information, reads out the probability that can be the optimum prediction mode for each prediction mode in the MB _L from the prediction mode correspondence rate table (correspondence rate), compared with the prediction mode search candidate narrowing down threshold value If this method is used, the comparison process becomes complicated.

  Therefore, in advance, the correspondence rate of the prediction mode correspondence rate table is binarized by thresholding the correspondence rate of the prediction mode correspondence rate table with the prediction mode search candidate narrowing threshold.

  FIG. 6 illustrates a result of narrowing down prediction mode search candidates when the prediction mode search candidate narrowing threshold is set to 5% in the prediction mode correspondence rate table shown in FIG. In the figure, ○ indicates a search candidate, and × indicates a prediction mode excluded from the search candidate.

(B) Refinement method 2
The narrowing-down method 2 is a method of setting only the prediction mode with the maximum prediction mode correspondence rate as a search candidate.

  A prediction mode that maximizes the prediction mode correspondence rate is set as a search candidate. Normally, this is narrowed down to one prediction mode, but when there are a plurality of prediction mode search candidates giving the maximum value, they are all set as search candidates.

Here, at the time when the information of the reference prediction mode is acquired, the probability (correspondence rate) that can be the optimum prediction mode of each prediction mode in MB _L is read from the prediction mode correspondence rate table, and the correspondence rate of the maximum value is determined from among them. If the method of specifying is used, the specifying process becomes complicated.

  Therefore, the correspondence rate of the maximum value included in the correspondence rate of the prediction mode correspondence rate table is specified in advance, and the correspondence rate of the prediction mode correspondence rate table is binarized.

  FIG. 7 illustrates a result of narrowing down when the maximum prediction mode is set as a prediction mode search candidate in the prediction mode correspondence rate table shown in FIG. In the figure, ○ indicates a search candidate, and × indicates a prediction mode excluded from the search candidate.

[2] Configuration of the Present Invention Next, the configuration of the present invention will be described.

  The scalable video encoding device of the present invention speeds up prediction mode search in scalable video encoding in which a layer structure in which an upper layer block and a lower layer block do not spatially correspond one-to-one is processed. (1) Based on the optimal prediction mode selected by scalable coding performed without restriction on the use of the prediction mode, the upper layer block and the image region of the lower layer immediately below it are selected. Generating means for obtaining an occurrence rate of the combination of the optimum prediction modes and generating a correspondence table describing a correspondence relationship between the combination of the optimum prediction modes and the occurrence rate; and (2) encoding a block of an upper layer. (3) acquisition means for acquiring information on the optimum prediction mode selected in the image region of the lower layer immediately below the image area; Based on the selection means for selecting the optimum prediction mode to be processed from the acquired optimum prediction modes, and (4) the optimum prediction mode selected by the selection means and the occurrence rate described in the correspondence table generated by the generation means , A prediction mode in which an effective combination is extracted from the combinations of optimal prediction modes described in the correspondence table, and the optimal prediction mode of the upper layer possessed by the extracted combination is searched by encoding the block of the upper layer Determination means for determining as a search candidate, (5) scalable coding for limiting the use of a prediction mode executed using a correspondence table, and use of a prediction mode executed without using a correspondence table And a control means for controlling to repeat non-scalable coding alternately.

  When adopting this configuration, the selecting means selects the optimum prediction mode to be processed based on the size of the overlap between the upper layer block and the lower layer image area, or prioritizes a preset prediction mode. When there is a prediction mode to be selected based on the ranking or to be referred to in inter-layer prediction, the optimal prediction mode to be processed may be selected in a manner that gives priority to the prediction mode. .

  Here, the selection unit may select all of the optimum prediction modes acquired by the acquisition unit as the optimum prediction modes to be processed.

  Further, the generation means considers that the block of the upper layer and the block of the lower layer do not spatially correspond one-to-one, and for each of the optimum prediction modes selected in the block of the upper layer, In some cases, the number of pixels in the image area of the immediately lower layer is aggregated for each optimum prediction mode selected in the image area, and a correspondence table is generated based on the aggregation result.

  In the scalable video encoding apparatus of the present invention configured as described above, the selection unit is configured so that the upper layer block and the lower layer block do not correspond one-to-one spatially, and thus the optimum obtained by the obtaining unit is obtained. Considering that there may be a plurality of prediction modes, the optimum prediction mode to be processed is selected from among them.

  For example, when selecting one optimal prediction mode to be processed from among the optimal prediction modes acquired by the acquisition unit, if there is one optimal prediction mode acquired by the acquisition unit, select the optimal prediction mode On the other hand, if there are a plurality of optimum prediction modes acquired by the acquisition means, the optimum prediction mode selected in the image region of the immediately lower layer with the largest overlap with the block of the upper layer is selected or set in advance. In accordance with the priority order of the predicted prediction modes, the optimal prediction mode with the highest priority among the plurality of optimal prediction modes is selected, and there is a prediction mode referred to in inter-layer prediction among the plurality of optimal prediction modes. In that case, the prediction mode is selected.

  Here, the optimum prediction mode selected by the selection means corresponds to the reference prediction mode described above.

  Upon receiving the selection process by the selection unit, the determination unit specifies the occurrence rate associated with the optimum prediction mode by referring to the correspondence table using the information of the optimum prediction mode selected by the selection unit as a key. , Extract the optimal prediction mode combination that has an occurrence rate that is larger than a predetermined threshold included in the specified occurrence rate, or the optimum that has the occurrence rate that shows the largest value included in the specified occurrence rate A combination of prediction modes is extracted, or a combination of optimum prediction modes having a predetermined number of occurrence rates selected in descending order of the values included in the specified occurrence rate, and the extracted optimum prediction mode Is determined as a prediction mode search candidate to be searched for by encoding the block of the upper layer.

  Further, in order to realize that the determination process by the determination unit becomes an efficient process, the scalable video encoding device of the present invention is based on the occurrence rate value described in the correspondence table in advance. The effective optimum prediction mode combinations are extracted by narrowing down the optimum prediction mode combinations described in (1), and prediction mode correspondence information describing the extracted effective optimum prediction mode combinations is generated.

  In this case, the scalable video coding apparatus of the present invention (1) based on the optimal prediction mode selected by the scalable coding performed without limiting the use of the prediction mode, Correspondence table generating means for determining the occurrence rate of the combination of the optimum prediction modes selected in the image region of the immediately lower layer and generating a correspondence table describing the correspondence between the combination of the optimum prediction modes and the occurrence rate And (2) extracting effective optimum prediction mode combinations by narrowing down the optimum prediction mode combinations described in the correspondence table based on the occurrence rate values described in the correspondence table generated by the correspondence table generating means. A prediction mode correspondence information generating means for generating prediction mode correspondence information describing the combination of the extracted effective optimum prediction modes; A) acquiring means for acquiring the information of the optimal prediction mode selected in the image area of the immediately lower layer when encoding a block of the upper layer; and (4) from among the optimal prediction modes acquired by the acquiring means. A selection mode for selecting an optimum prediction mode to be processed, and (5) a prediction mode for searching by encoding a block in a higher layer by referring to prediction mode correspondence information having a combination of the optimum prediction mode selected by the selection unit (6) Scalable coding for limiting the use of a prediction mode executed using a correspondence table, and use of a prediction mode executed without using a correspondence table And a control means for controlling to repeat non-scalable coding alternately.

  When taking this configuration, the prediction mode correspondence information generating means predicts by extracting, as shown in FIG. 6, an effective combination of optimum prediction modes having an occurrence rate that indicates a value larger than a predetermined threshold. The mode correspondence information is generated or, as shown in FIG. 7, the optimum prediction mode combination having the highest occurrence rate indicating the largest value is selected from among the optimum prediction mode combinations having the same optimum prediction mode for the lower layer. The prediction mode correspondence information is generated by extracting as a valid one, or the combination of a predetermined number of optimum prediction modes selected in the order of the occurrence rate indicating the large value is extracted as a valid one. May be generated.

  The scalable video encoding method of the present invention realized by the operation of each of the above processing means can also be realized by a computer program, and this computer program is recorded on an appropriate computer-readable recording medium and provided. Or provided via a network, installed when implementing the present invention, and operated on a control means such as a CPU, thereby realizing the present invention.

  In the present invention, in scalable video coding in which a layer structure in which an upper layer block and a lower layer block do not spatially correspond to each other is processed, the correlation of the optimum prediction mode between layers is used. Thus, the encoding time can be reduced because the prediction mode search candidates for the higher layer are narrowed down.

  And, in the present invention, when the encoding time is reduced by narrowing down the prediction mode search candidates, the narrowing is performed based on the correspondence relationship between the optimal prediction modes between layers in the encoded frame. By avoiding the risk that the optimum prediction mode is omitted by narrowing down, it is possible to suppress a decrease in encoding performance that may occur by narrowing down prediction mode search candidates.

It is explanatory drawing which shows an example of the flame | frame used as the production | generation object of a prediction mode correspondence rate table, and an encoding object frame. It is a flowchart which shows the big flow of the process of this invention. It is explanatory drawing of the spatial positional relationship of an upper layer and a lower layer. It is explanatory drawing of the table | surface created for the production | generation of a prediction mode corresponding | compatible ratio table | surface. It is explanatory drawing of a prediction mode corresponding | compatible ratio table. It is explanatory drawing of the narrowing-down result of a prediction mode search candidate. It is explanatory drawing of the narrowing-down result of a prediction mode search candidate. It is a flowchart of the scalable moving image encoding process performed by this invention. It is a flowchart of the scalable moving image encoding process performed by this invention. It is a flowchart of the scalable moving image encoding process performed by this invention. It is an apparatus block diagram of the scalable moving image encoder which comprises this invention. It is an apparatus block diagram of the scalable moving image encoder which comprises this invention. It is an apparatus block diagram of the scalable moving image encoder which comprises this invention. It is a figure for demonstrating the effectiveness of this invention.

  Hereinafter, the present invention will be described in detail according to embodiments.

  8 to 10 show flowcharts of scalable video encoding processing executed according to the present invention.

  FIG. 8 is an overall flowchart of the scalable video encoding process executed according to the present invention. Each of FIGS. 9 and 10 is an example of a detailed flowchart of the process executed in step S201 of the flowchart of FIG. It is.

  Next, according to these flowcharts, the scalable video encoding process executed by the present invention will be described in detail.

  Here, the encoding process of the present invention is a process for the enhancement layer, and a non-scalable single layer encoding process is applied to the base layer. An example of the single layer encoding process is the encoding process of the base layer portion of the SVC reference encoder JSVM mentioned in Non-Patent Document 2.

  First, the processing of steps S201 to S206 executed in the flowchart of FIG. 8 will be described.

  Step S201: The initial value of the prediction mode search candidate to be searched for in the encoding target macroblock (MB) is read, and finally the prediction mode search candidate to be searched for in the encoding target MB is determined and stored in the register. Details of this processing will be described later with reference to FIGS.

  Step S202: Read prediction mode search candidate information stored by the process of step S201 from the register, execute a search for each prediction mode search candidate, determine one optimal prediction mode to be used for encoding, and then the information Is stored in a register. As an example of a method for determining the optimum prediction mode, there is a method for optimizing a prediction mode that minimizes a coding cost expressed by a linear sum of a code amount and coding distortion performed in JSVM.

  Step S203: Read information on the optimal prediction mode in the encoding target MB from the register, perform motion compensation in the optimal prediction mode, generate a prediction residual signal, and store it in the buffer.

  Step S204: The prediction residual signal is read from the buffer, the prediction residual signal is encoded, and the encoded data is stored in the buffer. As an example of this process, there is a series of processes of DCT, quantization, and variable length coding in the SVC reference encoder JSVM described in Non-Patent Document 2.

  Step S205: A process for determining whether or not encoding of all MBs is completed. When encoding of all MBs is completed, the encoding process is terminated, and encoded data of each MB and necessary data are stored from the buffer. Other header information is read and output as final encoded data. On the other hand, if not all MBs have been encoded, the process proceeds to step S206.

  Step S206: Move to the next encoding target MB and perform the process of step S201.

  Next, an example of specific contents of the process executed in step S201 will be described by explaining the process of steps S301 to S307 executed in the flowchart of FIG.

  Step S301: Information specifying whether or not the encoding target MB is a prediction mode search candidate narrowing target MB to which the present invention is applied is read, and if it is a prediction mode search candidate narrowing target MB, the process of step S302 is performed. On the other hand, if it is not the prediction mode search candidate narrowing-down target MB, the initial value of the prediction mode search candidate is output as the final prediction mode search candidate.

  Step S302: The designation information of the encoded frame to be calculated in the prediction mode correspondence rate table is read from the outside, and the prediction mode information of the designated frame is stored in the register.

  Step S303: Read prediction mode information (information on the optimal prediction mode used in encoding) in the calculation target frame of the prediction mode correspondence ratio table, and the correspondence ratio (occurrence between the encoding target layer and the layer immediately below it) Rate) is calculated and stored in a register as a prediction mode correspondence rate table. That is, according to the procedure described above, a table for generating a prediction mode correspondence rate table as shown in FIG. 4 is created, and based on the table, a prediction mode correspondence rate table as shown in FIG. Store it.

  Step S304: One or more optimum prediction mode groups in the image region immediately below the encoding target MB are read, and one reference prediction mode to be used for the vertical axis of the prediction mode correspondence rate table is determined from the group according to a certain rule, Output it to the register. Here, as an example of the norm of determining the reference prediction mode, (a) the prediction mode with the maximum accumulated area, (b) the prediction mode with the highest priority, and (c) the prediction to be referred to in the inter-layer prediction. Mode, etc. are applicable.

  Step S305: Read the information of the prediction mode correspondence rate table portion associated with the reference prediction mode, and store it in the buffer.

  Step S306: Read a prediction mode search candidate narrowing threshold value and store it in a register.

  Step S307: Read the information of the prediction mode correspondence rate table part associated with the reference prediction mode from the buffer, read the prediction mode search candidate narrowing threshold value from the register, and the correspondence rate (occurrence rate) is equal to or higher than the prediction mode search candidate narrowing threshold value. Only the prediction mode is set as a prediction mode search candidate, and the information is stored in the register. Here, in this setting / storage, only the prediction mode search candidate associated with the optimum prediction mode obtained by encoding the base layer is selected and set / stored.

  In this way, in the flowchart of FIG. 8, when the process of step S201 is executed according to the process of the flowchart of FIG. 9, based on the prediction mode correspondence rate table having the data structure as shown in FIG. It processes so that prediction mode search candidates may be narrowed down in the form as shown in FIG.

  Next, another example of the specific content of the process executed in step S201 will be described by explaining the process of steps S401 to S406 executed in the flowchart of FIG.

  Step S401: Information specifying whether or not the encoding target MB is a prediction mode search candidate narrowing target MB to which the present invention is applied is read, and if it is a prediction mode search candidate narrowing target MB, the process of step S402 is performed. On the other hand, if it is not the prediction mode search candidate narrowing-down target MB, the initial value of the prediction mode search candidate is output as the final prediction mode search candidate.

  Step S402: The designation information of the encoded frame to be calculated in the prediction mode correspondence rate table is read from the outside, and the prediction mode information of the designated frame is stored in the register.

  Step S403: Read prediction mode information (information on the optimal prediction mode used in encoding) in the calculation target frame of the prediction mode correspondence rate table, and correspond to the optimal prediction mode (occurrence of the encoding target layer and the layer immediately below it). Rate) is calculated and stored in a register as a prediction mode correspondence rate table. That is, according to the procedure described above, a table for generating a prediction mode correspondence rate table as shown in FIG. 4 is created, and based on the table, a prediction mode correspondence rate table as shown in FIG. Store it.

  Step S404: Read one or more optimum prediction mode groups in the image region immediately below the encoding target MB, and determine one reference prediction mode to be used for the vertical axis of the prediction mode correspondence rate table from among them, according to a certain rule, Output it to the register. Here, as an example of the norm of determining the reference prediction mode, (a) the prediction mode with the maximum accumulated area, (b) the prediction mode with the highest priority, and (c) the prediction to be referred to in the inter-layer prediction. Mode, etc. are applicable.

  Step S405: Read the information of the prediction mode correspondence rate table portion associated with the reference prediction mode, and store it in the buffer.

  Step S406: Read information of the prediction mode correspondence rate table portion associated with the reference prediction mode from the buffer, set only the prediction mode with the highest correspondence rate (occurrence rate) as a prediction mode search candidate, and store the information in the register To do. Here, in this setting / storage, only the prediction mode search candidate associated with the optimum prediction mode obtained by encoding the base layer is selected and set / stored.

  Thus, in the flowchart of FIG. 8, when the process of step S201 is executed according to the process of the flowchart of FIG. 10, based on the prediction mode correspondence rate table having the data structure shown in FIG. It processes so that prediction mode search candidates may be narrowed down in the form as shown in FIG.

  FIG. 11 to FIG. 13 show a device configuration of a scalable video encoding device provided with the present invention.

  FIG. 11 shows the overall apparatus configuration of the scalable video encoding apparatus comprising the present invention. FIGS. 12 and 13 each show the detailed apparatus configuration of the prediction mode search candidate determination unit 102 shown in FIG. It is an example.

  Next, the scalable video encoding apparatus including the present invention will be described in detail according to these apparatus configuration diagrams.

  Here, the scalable video encoding apparatus provided with the present invention is a processing apparatus for an enhancement layer, and applies a non-scalable single layer encoding process to the base layer. An example of the single layer encoding process is the encoding process of the base layer portion of the SVC reference encoder JSVM mentioned in Non-Patent Document 2.

  First, the overall configuration of the scalable video encoding apparatus including the present invention will be described with reference to FIG.

  Prediction mode search candidate initial value storage unit 101: Reads the initial value of the prediction mode search candidate and outputs it to the register.

  Prediction mode search candidate determination unit 102: reads an initial value of a prediction mode search candidate, determines a prediction mode search candidate to be finally searched, outputs information of the finally determined prediction mode search candidate to a register, The process moves to the optimum prediction mode determination unit 103. The detailed configuration of this processing unit will be described later with reference to FIGS.

  Optimal prediction mode determination unit 103: Reads prediction mode search candidates from a register, executes a search for each prediction mode search candidate, determines one optimal prediction mode to be used for encoding, and stores the information as an optimal prediction mode storage unit To 104. As an example of a method for determining the optimum prediction mode, there is a method for optimizing a prediction mode that minimizes a coding cost expressed by a linear sum of a code amount and coding distortion performed in JSVM.

  Prediction residual signal generation unit 105: Reads the optimal prediction mode in the encoding target MB from the optimal prediction mode storage unit 104, performs motion compensation in the optimal prediction mode, generates a prediction residual signal, and outputs it to the buffer .

  Prediction residual signal encoding unit 106: Reads the prediction residual signal in the encoding target MB from the buffer, encodes the prediction residual signal, and outputs the encoded data to the buffer. An example of this processing is H.264. Application of a series of processes such as DCT, quantization, and variable length coding of the H.264 / AVC reference encoder JM and the SVC reference encoder JSVM described in Non-Patent Document 2 is conceivable.

  All MB completion determination unit 107: Performs a determination process of whether or not encoding of all MBs is completed. When encoding of all MBs is completed, the encoding process ends and final encoding is performed. Data is output, and when the encoding of all MBs is not completed, the process proceeds to the encoding target MB update unit 108.

  Encoding target MB update unit 108: Moves to the next encoding target MB, and performs the process of the prediction mode search candidate determination unit 102.

  Next, an example of a detailed configuration of the prediction mode search candidate determination unit 102 will be described with reference to FIG.

  Prediction mode search candidate narrowing-down target MB designation information storage unit 201: Reads information for designating whether or not the MB is a target for narrowing down prediction mode search candidates, and outputs it to a register.

  Prediction mode search candidate narrowing-down target MB determining unit 202: Predictive mode search candidate narrowing-down target MB designation information storage unit 201 reads MB designation information for narrowing down prediction mode search candidates, and the encoding target MB is a MB for narrowing down. If the MB is to be narrowed down, the process proceeds to the prediction mode correspondence rate table generation unit 206. If the MB is not to be narrowed down, the initial value of the prediction mode search candidate is finalized. Is determined as a candidate prediction mode search and output.

  Prediction mode correspondence rate calculation target frame designation information storage unit 203: Reads the designation information of the encoded frame that is the calculation target of the prediction mode correspondence rate, and outputs it to the register.

  Target frame enhancement layer optimal prediction mode storage unit 204: Optimal prediction mode in the encoding target layer for a frame for which the prediction mode correspondence rate indicated by the designation information read by the prediction mode correspondence rate calculation target frame designation information storage unit 203 is calculated. Read information and output to register.

  Target frame direct layer optimum prediction mode storage unit 205: For a frame for which the prediction mode correspondence rate indicated by the designation information read in the prediction mode correspondence rate calculation target frame designation information storage unit 203 is calculated, in the layer immediately below the encoding target layer Reads the optimal prediction mode information and outputs it to the register.

  Prediction mode correspondence rate table generation unit 206: Reads the optimal prediction mode information in the encoding target layer of the calculation target frame of the prediction mode correspondence rate from the target frame extension layer optimal prediction mode storage unit 204 and stores the optimal prediction mode layer immediately below the target frame. The optimal prediction mode information in the layer immediately below the encoding target layer of the encoding target layer of the target frame for calculating the prediction mode correspondence rate is read from the unit 205, and the optimal prediction mode between the macro block of the encoding target layer and the image region immediately below the target mode is read. The correspondence rate (occurrence rate) is calculated and output to the prediction mode correspondence rate table storage unit 207 as a prediction mode correspondence rate table. That is, according to the procedure described above, a table for generating a prediction mode correspondence rate table as shown in FIG. 4 is created, and based on the table, a prediction mode correspondence rate table as shown in FIG. The data is output to the correspondence rate table storage unit 207.

  Reference prediction mode determination unit 208: Reads one or more optimum prediction mode groups in the immediately lower layer from the target frame immediately lower layer optimal prediction mode storage unit 205, and uses them as a vertical axis of the prediction mode correspondence rate table from among them according to a certain rule One reference prediction mode is determined and output to the reference prediction mode storage unit 209. Here, as an example of the norm of determining the reference prediction mode, (a) the prediction mode with the maximum accumulated area, (b) the prediction mode with the highest priority, and (c) the prediction to be referred to in the inter-layer prediction. Mode, etc. are applicable.

  Prediction mode correspondence rate narrowing threshold storage unit 210: Reads a prediction mode search candidate narrowing threshold and outputs it to a register.

  Prediction mode correspondence rate threshold comparison unit 211: Reads the prediction mode correspondence rate table from the prediction mode correspondence rate table storage unit 207, reads the reference prediction mode from the reference prediction mode storage unit 209, and further reduces the prediction mode correspondence rate threshold storage unit The prediction mode correspondence rate narrowing threshold is read from 210, the occurrence probability of the optimal prediction mode of the encoding target MB associated with the reference prediction mode is investigated, and only the prediction mode whose occurrence probability is equal to or higher than the prediction mode correspondence rate narrowing threshold is final Is set as a typical prediction mode search candidate and output.

  In this way, the apparatus configuration shown in FIG. 12 performs processing to narrow down prediction mode search candidates in the form shown in FIG. 6 based on the prediction mode correspondence rate table having the data structure shown in FIG. It is.

  Next, another example of the detailed configuration of the prediction mode search candidate determination unit 102 will be described with reference to FIG.

  Prediction mode search candidate narrowing-down target MB designation information storage unit 301: Reads information for designating whether or not the prediction mode search candidate narrows down the prediction mode search candidate, and outputs it to a register.

  Prediction mode search candidate narrowing target MB determination unit 302: Predictive mode search candidate narrowing target MB designation information storage unit 301 reads MB designation information for narrowing down prediction mode search candidates, and the encoding target MB is a MB for narrowing down. If the MB is to be narrowed down, the process proceeds to the prediction mode correspondence rate table generation unit 306. If the MB is not to be narrowed down, the initial value of the prediction mode search candidate is finalized. Is determined as a candidate prediction mode search and output.

  Prediction mode correspondence rate calculation target frame designation information storage unit 303: Reads designation information of an encoded frame that is a calculation target of the prediction mode correspondence rate, and outputs it to a register.

  Target frame enhancement layer optimal prediction mode storage unit 304: prediction mode correspondence rate calculation target frame designation information storage unit 303 for the prediction target for the prediction mode correspondence rate pointed to by the designation information read in the coding information layer optimum prediction mode in the coding target layer Read information and output to register.

  Target frame immediately below layer optimum prediction mode storage unit 305: Prediction mode correspondence rate calculation target frame designation information storage unit 303 for a frame that is a calculation target of the prediction mode correspondence rate indicated by the designation information read in the encoding target layer immediately below the encoding target layer Reads the optimal prediction mode information and outputs it to the register.

  Prediction mode correspondence rate table generation unit 306: Reads the optimal prediction mode information in the encoding target layer of the target frame extension layer optimal prediction mode storage unit 304 and calculates the prediction mode correspondence rate, and stores the optimal prediction mode layer immediately below the target frame. The optimal prediction mode information in the layer immediately below the encoding target layer of the target frame for calculating the prediction mode correspondence rate is read from the unit 305, and the optimal prediction mode between the macroblock of the encoding target layer and the image region immediately below the target mode is read. The correspondence rate (occurrence rate) is calculated and output to the prediction mode correspondence rate table storage unit 307 as a prediction mode correspondence rate table. That is, according to the procedure described above, a table for generating a prediction mode correspondence rate table as shown in FIG. 4 is created, and based on the table, a prediction mode correspondence rate table as shown in FIG. This is output to the correspondence rate table storage unit 307.

  Reference prediction mode determination unit 308: One or more optimum prediction mode groups in the immediately lower layer are read from the target frame immediately below layer optimum prediction mode storage unit 305, and are used as the vertical axis of the prediction mode correspondence rate table from among them according to a certain rule. One reference prediction mode is determined and output to the reference prediction mode storage unit 309. Here, as an example of the norm of determining the reference prediction mode, (a) the prediction mode with the maximum accumulated area, (b) the prediction mode with the highest priority, and (c) the prediction to be referred to in the inter-layer prediction. Mode, etc. are applicable.

  Occurrence rate maximum prediction mode identification unit 310: Reads a prediction mode correspondence rate table from the prediction mode correspondence rate table storage unit 307, reads a reference prediction mode from the reference prediction mode storage unit 309, and stores the encoding target MB for the reference prediction mode. The occurrence probability of the optimum prediction mode is investigated, and the prediction mode with the maximum occurrence probability is set as the final prediction mode search candidate and output.

  In this way, the apparatus configuration shown in FIG. 13 performs processing to narrow down prediction mode search candidates in the form shown in FIG. 7 based on the prediction mode correspondence rate table having the data structure shown in FIG. It is.

  Next, the effectiveness of the present invention will be described.

  As described above, in the Non-Patent Document 3, the present inventor uses the correlation of prediction modes between layers in scalable video coding to narrow down search candidates for prediction modes of macroblocks in the enhancement layer. The invention to do was disclosed.

  According to the present invention, search candidates for prediction modes of macroblocks in the enhancement layer can be narrowed down, so that it is possible to realize a high speed scalable video encoding process.

  However, the present invention can be applied only when the optimal prediction mode of the image area directly corresponding to the block to be encoded has one optimal prediction mode, and is applicable when there are a plurality of optimal prediction modes. Can not.

  When the power scalability resolution of 2 is not established between the upper layer and the lower layer, or when the lower layer video is generated by cutting out a part of the video from the upper layer video, Since there are a plurality of optimal prediction modes in the image region directly corresponding to the space, the invention disclosed in Non-Patent Document 3 cannot be applied.

  On the other hand, the present invention can be applied even when the image area directly corresponding to the encoding target block has a plurality of optimum prediction modes. The invention disclosed in Non-Patent Document 3 Also, the prediction mode selection of the encoding target block can be executed at high speed.

  For example, as shown in FIG. 14, a video with a 1920 × 1080 pixel size is used as an upper layer, and a video with a size of 1440 × 1080 pixels generated by multiplying it by 3/4 in the horizontal direction is used as a lower layer. Assume that scalable video coding has been performed.

  In this case, in the invention disclosed in Non-Patent Document 3, the upper layer is applied according to the constraint that it can be applied when only one optimal prediction mode has an image area immediately below that spatially corresponds to the encoding target block. When encoding is performed, the fast prediction mode search can be performed only with the number of macroblocks that is ½ of the total number of macroblocks. On the other hand, since the present invention can be performed on all macroblocks, the number of macroblocks on which prediction mode search processing is narrowed is doubled compared to the invention disclosed in Non-Patent Document 3. To do.

  That is, as shown in FIG. 14, among the four blocks a, b, c, and d in the lower layer, the blocks a and d have only one optimal prediction mode, while the block b has The optimal prediction mode derived from the left macroblock A and the optimal prediction mode derived from the right macroblock B have two optimal prediction modes. With the optimal prediction mode derived from the macroblock B and the optimal prediction mode derived from the right macroblock C, there are two optimal prediction modes.

  From the above, in the invention disclosed in Non-Patent Document 3, when encoding the upper layer, only the two macroblocks positioned above the blocks a and d out of the four blocks a, b, c and d are high-speed. In contrast to the prediction mode search, in the present invention, since the high-speed prediction mode search can be performed on all four macroblocks positioned above the four blocks a, b, c, and d, the prediction mode search process is performed. As compared with the invention disclosed in Non-Patent Document 3, the number of macroblocks to be narrowed is increased twice.

Similarly, according to the present invention, compared to the invention disclosed in Non-Patent Document 3, the number of macroblocks to be narrowed down in the prediction mode search process is
(1) 9/4 times for scalable video coding with spatial scalability of 1920 × 1080 pixel size and 1280 × 720 pixel size (2) Spatial scalability of 1920 × 1080 pixel size and 720 × 480 pixel size (3) In scalable video coding with spatial scalability of 1920 × 1080 pixel size and 640 × 480 pixel size, it will increase to 3/2 times.

  Therefore, according to the present invention, compared with the invention disclosed in Non-Patent Document 3, the calculation time required for encoding can be greatly reduced.

  Although the present invention has been described according to the illustrated embodiment, the present invention is not limited to this. For example, in the embodiment, the present invention has been described according to the application to the hierarchical layer configuration of the base layer and the extension layer, but the application of the present invention is not limited to such a hierarchical layer configuration.

  In the embodiment, the configuration is such that one prediction mode is selected as the reference prediction mode, but a configuration in which a plurality of prediction modes are selected as the reference prediction mode is also possible.

  The present invention can be applied to scalable video coding that achieves scalability by a layer structure, and the coding time can be reduced by applying the present invention.

101 prediction mode search candidate initial value storage unit 102 prediction mode search candidate determination unit 103 optimal prediction mode determination unit 104 optimal prediction mode storage unit 105 prediction residual signal generation unit 106 prediction residual signal encoding unit 107 all MB completion determination unit 108 Encoding target MB update unit 201 Prediction mode search candidate narrowing target MB designation information storage unit 202 Prediction mode search candidate narrowing target MB determination unit 203 Prediction mode correspondence rate calculation target frame designation information storage unit 204 Target frame enhancement layer optimum prediction mode storage unit 205 Predictive layer optimum prediction mode storage unit immediately below target frame 206 Prediction mode correspondence rate table generation unit 207 Prediction mode correspondence rate table storage unit 208 Reference prediction mode determination unit 209 Reference prediction mode storage unit 210 Prediction mode correspondence rate narrowing threshold storage unit 211 Prediction mode Response rate threshold Comparison unit 301 Prediction mode search candidate narrowing target MB designation information storage unit 302 Prediction mode search candidate narrowing target MB determination unit 303 Prediction mode correspondence rate calculation target frame designation information storage unit 304 Target frame enhancement layer optimal prediction mode storage unit 305 Directly under target frame Layer optimal prediction mode storage unit 306 Prediction mode correspondence rate table generation unit 307 Prediction mode correspondence rate table storage unit 308 Reference prediction mode determination unit 309 Reference prediction mode storage unit 310 Maximum occurrence rate prediction mode identification unit

Claims (12)

A scalable video encoding method for encoding a video in a scalable manner,
Occurrence of a combination of the optimal prediction mode selected in the upper layer block and the image region of the lower layer immediately below it based on the optimal prediction mode selected in scalable coding performed without limiting the use of the prediction mode Generating a correspondence table that describes the correspondence between the combination of the optimal prediction modes and the occurrence rate,
When encoding a block of the upper layer, a process of obtaining information on the optimal prediction mode selected in the image region of the lower layer immediately below the block,
Selecting an optimum prediction mode to be processed from the obtained optimum prediction modes;
Based on the selected optimum prediction mode and the occurrence rate described in the correspondence table, an effective combination is extracted from the combinations of optimum prediction modes described in the correspondence table, and the extracted combination has Determining the optimal prediction mode of the upper layer as a prediction mode search candidate for searching by encoding the block of the upper layer,
A scalable video encoding method characterized by the above. A scalable video encoding method for encoding a video in a scalable manner,
Occurrence of a combination of the optimal prediction mode selected in the upper layer block and the image region of the lower layer immediately below it based on the optimal prediction mode selected in scalable coding performed without limiting the use of the prediction mode Generating a correspondence table that describes the correspondence between the combination of the optimal prediction modes and the occurrence rate,
Based on the value of the occurrence rate, a combination of effective optimum prediction modes is extracted by narrowing down the combination of optimum prediction modes described in the correspondence table, and the combination of the extracted effective optimum prediction modes is described. A process of generating prediction mode correspondence information;
When encoding a block of the upper layer, a process of obtaining information on the optimal prediction mode selected in the image region of the lower layer immediately below the block,
Selecting an optimum prediction mode to be processed from the obtained optimum prediction modes;
A step of determining a prediction mode search candidate to be searched by encoding a block of a higher layer by referring to the prediction mode correspondence information having the selected optimum prediction mode in combination,
A scalable video encoding method characterized by the above. The scalable video encoding method according to claim 2,
In the process of generating the prediction mode correspondence information, extracting a combination of optimum prediction modes having an occurrence rate indicating a value larger than a predetermined threshold as effective,
A scalable video encoding method characterized by the above. The scalable video encoding method according to claim 2,
In the process of generating the prediction mode correspondence information, a combination of optimum prediction modes having the highest occurrence rate indicating the largest value is extracted as an effective one from combinations of optimum prediction modes having the same optimum prediction mode for lower layers. Or extracting a combination of a predetermined number of optimum prediction modes that are selected in the order of occurrence rates showing large values as effective,
A scalable video encoding method characterized by the above. The scalable video encoding method according to any one of claims 1 to 4,
In the process of selecting the optimum prediction mode, selecting the optimum prediction mode to be processed based on the size of the overlap between the upper layer block and the lower layer image region,
A scalable video encoding method characterized by the above. The scalable video encoding method according to any one of claims 1 to 4,
In the process of selecting the optimum prediction mode, selecting the optimum prediction mode to be processed based on the preset priority order of the prediction modes,
A scalable video encoding method characterized by the above. The scalable video coding method according to any one of claims 1 to 6,
In the process of selecting the optimum prediction mode, if there is a prediction mode to be referred to in inter-layer prediction in the obtained optimum prediction mode, selecting the prediction mode as the optimum prediction mode to be processed;
A scalable video encoding method characterized by the above. The scalable video coding method according to any one of claims 1 to 7,
In the process of generating the correspondence table, for each of the optimal prediction modes selected in the block of the upper layer, the number of pixels in the image region of the lower layer immediately below the block is determined for each optimal prediction mode selected in the image region. Counting and generating the correspondence table based on the counting result,
A scalable video encoding method characterized by the above. The scalable video coding method according to any one of claims 1 to 8,
The scalable coding that restricts the use of the prediction mode executed using the correspondence table and the scalable coding that does not restrict the use of the prediction mode executed without using the correspondence table are alternately repeated. To have a process of controlling
A scalable video encoding method characterized by the above. A scalable video encoding device for encoding a video in a scalable manner,
Occurrence of a combination of the optimal prediction mode selected in the upper layer block and the image region of the lower layer immediately below it based on the optimal prediction mode selected in scalable coding performed without limiting the use of the prediction mode Means for generating a correspondence table describing the correspondence between the combination of the optimum prediction mode and the occurrence rate,
Means for acquiring information of the optimal prediction mode selected in the image region of the lower layer immediately below when encoding a block of the upper layer;
Means for selecting an optimum prediction mode to be processed from the obtained optimum prediction modes;
Based on the selected optimum prediction mode and the occurrence rate described in the correspondence table, an effective combination is extracted from the combinations of optimum prediction modes described in the correspondence table, and the extracted combination has Means for determining the optimal prediction mode of the upper layer as a prediction mode search candidate for searching by encoding the block of the upper layer,
A featured scalable video encoding apparatus. A scalable video encoding device for encoding a video in a scalable manner,
Occurrence of a combination of the optimal prediction mode selected in the upper layer block and the image region of the lower layer immediately below it based on the optimal prediction mode selected in scalable coding performed without limiting the use of the prediction mode Means for generating a correspondence table describing the correspondence between the combination of the optimum prediction mode and the occurrence rate,
Based on the value of the occurrence rate, a combination of effective optimum prediction modes is extracted by narrowing down the combination of optimum prediction modes described in the correspondence table, and the combination of the extracted effective optimum prediction modes is described. Means for generating prediction mode correspondence information;
Means for acquiring information of the optimal prediction mode selected in the image region of the lower layer immediately below when encoding a block of the upper layer;
Means for selecting an optimum prediction mode to be processed from the obtained optimum prediction modes;
Means for determining a prediction mode search candidate to be searched by encoding a block of a higher layer by referring to the prediction mode correspondence information having the selected optimal prediction mode in combination,
A featured scalable video encoding apparatus.   A scalable video encoding program for causing a computer to execute the scalable video encoding method according to any one of claims 1 to 9.

Images