JP6767299B2 - Video coding device - Google Patents

Description

The present invention relates to a video coding device.

As a standard for video coding, H.I. 264 / AVC (Advanced Video Coding) and H.A. A video coding method called 265 / HEVC (High Efficiency Video Coding) has been established. In HEVC, a plurality of picture types are defined: an I (Intra) picture which is an intra picture, a P (Predictive) picture which is an inter picture, and a B (Bi-directional predictive) picture. Each frame is assigned one of the picture types.

The I picture is usually a picture type defined every several frames, and is a picture type to which only the intra prediction mode of predicting from the encoded pixels in the coded target frame is applied. The P picture and the B picture are picture types to which the intra prediction mode or the inter prediction mode of predicting from the encoded frame is selectively applied. The P picture uses the past frame of the time series as the reference frame, while the B picture uses the past and future frames as the reference frame.

The frame group from one picture to the next I picture is called GOP (Group Of Picture), and the coding structure such as the interval of I picture and the reference structure can be freely set for each GOP. Typical coding structures include a low-delay coding coding structure shown in FIG. 17 and a random access coding coding structure shown in FIG. 18, and in general, random access coding has higher coding efficiency. Is said to be expensive.

When defining the coding structure, the coding efficiency can be improved by adaptively defining the I-picture or P-picture interval according to the image to be encoded. For example, for a sequence of video frames with a scene change in the middle of the video as shown in FIG. 19, the I picture or P picture is adaptively adapted to the scene change rather than the fixed I picture or P picture interval. The coding efficiency is improved by selecting the following frames.

In addition, the coding efficiency can be improved by adaptively changing the reference structure, such as setting the reference structure as shown in FIGS. 20 and 21 hierarchically. For example, when the adaptive structure 1 shown in FIG. 20 and the adaptive structure 2 shown in FIG. 21 are compared, the adaptive structure 2 shown in FIG. 21 has two maximum layers, so that the maximum layers are two layers. The distance is small, and improvement in coding efficiency is expected.

A method for selecting a coded structure having a higher coding efficiency has also been proposed (see, for example, Non-Patent Documents 1 and 2). The two typical techniques proposed will be described below. In the technique shown in Non-Patent Document 1, the coded structure is defined according to the following procedure. The original image is reduced to 1/2 size in each of the vertical and horizontal directions (step Sx1). The first frame is encoded as an I picture and used as a reference frame (step Sx2). For each frame up to N sheets ahead, the cost when the P picture is used is calculated, and the frame with the smallest calculated cost is defined as the P picture. Here, the cost is the sum of all the blocks of the smaller coding cost when each coded block of the frame is coded in the intra prediction mode and the inter prediction mode (step Sx3).

An intermediate frame that references them is defined between the reference frame and the frame selected as the P picture in step Sx3. For the frames located between the reference frame and the intermediate frame, the cost is calculated when these frames are B pictures that refer to the reference frame and the intermediate frame. Further, for the frame located between the intermediate frame and the P picture, the cost is calculated when these frames are B pictures that refer to the intermediate frame and the P picture. The combination that minimizes the calculated cost is selected as the final coded structure (step Sx4). The frame designated as the P picture in step Sx3 is used as the reference frame, and steps Sx3 and Sx4 are repeatedly executed to adaptively determine the coding structure.

In the technique described in Non-Patent Document 2, the coded structure is defined according to the following procedure. The first frame is encoded as an I picture (step Sy1). A motion search is performed between the I picture and the next frame, and the motion vector of each coded block is set to S ₁ (k) (where 1 ≦ k ≦ N _blk : N _blk is the number of blocks in the frame). (Step Sy2).

A motion search is performed for each frame of the nth frame (n ≧ 3) with the first frame, and the motion vector of each coded block divided by the inter-frame distance is defined as _Sn (k). (Step Sy3). The average of the motion vector difference _{(E = Σ (S 1 -S} n) / N blk) is calculated (step Sy4). A predetermined threshold value is set to th, and if E <th, the nth frame is set to B picture, and if not, it is set to P picture (step Sy5).

“VideoLAN, a project and a non-profit organization”, [online], VideoLAN Organization, [Search on March 17, 2017], Internet (URL: https://www.videolan.org/developers/x265.html ) Adriana Dumitras and Barry G. Haskell, “I / P / B FRAME TYPE DECISION BY COLLINEARITY OF DISPLACEMENTS”, 2004 International Conference on Image Processing (ICIP), p2769-p2772

However, as shown in the above procedure, in the technique described in Non-Patent Document 1, although a reduced image is used, processing close to actual coding is performed on a plurality of combinations. In addition, there are some combinations in which the calculation results are wasted, and as a result, there is a problem that the method has a large amount of calculation. Further, the technique described in Non-Patent Document 1 is premised on allocating two layers as the layer of the B picture, and when trying to apply a deeper hierarchical structure with good coding efficiency to this technique, the number of combinations There is a problem that the number increases.

Even in the technique described in Non-Patent Document 2, there is a problem that the amount of calculation is large because there is a combination in which the calculation result at the time of determination is wasted. Further, the technique described in Non-Patent Document 2 has a problem that only one layer can be applied as the layer of the B picture. Further, in the technique described in Non-Patent Document 2, since only the motion vector is used for the determination, there is a problem that the determination cannot be made accurately for a complicated image and the coding efficiency is poor.

In view of the above circumstances, an object of the present invention is to provide a technique capable of obtaining a coded structure having high coding efficiency with a low calculation amount.

One aspect of the present invention is a video coding apparatus that performs inter-frame predictive coding, and from a frame group acquired from input video information, a reference group including a predetermined number of three or more consecutive frames is provided. Based on the reference group generation unit to be generated, the first and last frames of the reference group generated by the reference group generation unit, and any one of the determination target frames between the first and last frames. An evaluation value calculation unit that calculates an evaluation value related to an error between frames, an evaluation value between the determination target frame and the first frame among the evaluation values calculated by the evaluation value calculation unit, and the determination target. evaluation value between the frame last frame, or the said determination target frame and the first frame and the end of the previous SL determination target frame based on the relative value calculated from the smaller among the evaluation values between frames first A picture that selects the picture type of the frame included in the frame group based on the judgment result of the evaluation value determination unit that determines whether or not the reference relationship between the image and the last frame is combined. The evaluation value determination unit includes a type selection unit, and is a video coding device that determines to be combined when the relative value is less than the first relative evaluation threshold .

One aspect of the present invention is the above-mentioned video coding apparatus, and the evaluation value calculation unit is a first one-way evaluation value regarding an error between the determination target frame and the first frame. And the second one-way evaluation value regarding the error between the frames between the determination target frame and the last frame, and between the determination target frame and the first and last frames. The bidirectional evaluation value related to the error is calculated, and the evaluation value determination unit selects the smaller evaluation value of the second one-way evaluation value and the bidirectional evaluation value, and the selected evaluation value is said to be the same. The relative value is calculated as the magnitude of the relative value with respect to the first one-way evaluation value, and the determination target frame and the first one are based on the calculated relative value and a predetermined relative evaluation threshold value. Determines whether to combine the reference relationship with the last frame.

One aspect of the present invention is the above-mentioned video coding apparatus, wherein the predetermined relative evaluation threshold value further includes a second relative evaluation threshold value having a value larger than that of the first relative evaluation threshold value. and de, the evaluation value determination unit, before SL relative value, when the a first value below the second relative evaluation threshold value relative evaluation threshold or more, and the bidirectional evaluation value, predetermined Based on the absolute evaluation threshold, it is determined whether or not to combine the reference relationship between the determination target frame and the first and last frames.

One aspect of the present invention is the video coding apparatus, wherein the reference group generation unit is the first frame or the last frame of the reference group determined to be combined by the evaluation value determination unit. A new reference group including the reference group and a number of consecutive frames larger than the predetermined three or more frames is generated so as to be the determination target frame, and the evaluation value calculation unit is described. Based on the first and last frames of the new reference group and the determination target frame, the evaluation value regarding the error between the frames is calculated, and the evaluation value determination unit is calculated by the evaluation value calculation unit. Among the evaluation values, the evaluation value between the determination target frame and the first frame of the new reference group, the evaluation value between the determination target frame of the new reference group and the last frame, or the evaluation value. new reference group determination target frame and the first frame and the determination target frame and the first and last of the reference group before SL new based on the relative value calculated from the smaller among the evaluation values between the last frame determines whether to combine the reference relationship with the frame, if the relative value is less than the first relative evaluation threshold, it determines that binds.

One aspect of the present invention is the above-mentioned video coding apparatus, and when the reference group generation unit generates another reference group adjacent to the reference group, the last frame of the reference group is the said. The reference group including the predetermined three or more consecutive frames is generated by overlapping so as to be the first frame of the other adjacent reference groups, and the two adjacent reference groups When the evaluation value determination unit determines that the reference relationship between the determination target frame and the first and last frames is combined in both cases, the two adjacent reference groups are combined to generate the new reference group. , The overlapping frames are set as the determination target frames.

One aspect of the present invention is the above-mentioned video coding apparatus, and the reference group generation unit is
According to the hierarchy number of values of the coding structure to build Ru been predetermined et al, repeated to generate the new reference group.

One aspect of the present invention is the above-mentioned video coding apparatus, in which the picture type selection unit acquires the frame group from the input video information and selects the first frame of the acquired frame group as an I picture. When the evaluation value determination unit determines that the reference relationship between the determination target frame and the first and last frames is combined, the picture type selection unit determines the determination target frame. The first and last frames are selected as B pictures with reference frames as reference frames, the I picture or the frame not selected as the B picture is selected as the P picture, and the selected frames are selected in the display order of the frame group. The frame of the past P-picture closest to is used as the reference frame of the selected frame, and when there is no frame of the past P-picture closest to the selected frame, the frame of the I-picture is used. Let it be the reference frame of the selected frame.

One aspect of the present invention is the above-mentioned video coding apparatus, and the number of the predetermined three or more sheets is three.

INDUSTRIAL APPLICABILITY According to the present invention, it is possible to obtain a coding structure having high coding efficiency with a low amount of calculation.

It is a block diagram which shows the structure of the image coding apparatus in 1st Embodiment of this invention. It is a block diagram which shows the structure of the coded structure construction part of the same embodiment. It is a flowchart which shows the flow of the coded structure construction process of the same embodiment. It is a flowchart which shows the flow of the combination determination processing of the same embodiment. It is a figure (the 1) which shows the specific example of the coded structure construction process of the same embodiment. It is a figure (the 2) which shows the specific example of the coded structure construction process of the same embodiment. It is a figure (the 3) which shows the specific example of the coded structure construction process of the same embodiment. It is a figure (the 4) which shows the specific example of the coded structure construction process of the same embodiment. It is a figure which shows the specific example of the combination determination processing of the same embodiment. It is a figure which shows the evaluation value calculated in the specific example of the combination determination processing of the same embodiment. It is a block diagram which shows the structure of the coded structure construction part of the 2nd Embodiment of this invention. It is a flowchart (the 1) which shows the flow of the coded structure construction process of the same embodiment. It is a flowchart (No. 2) which shows the flow of the coded structure construction process of the same embodiment. It is a figure (the 1) which shows the specific example of the coded structure construction process of the same embodiment. It is a figure (the 2) which shows the specific example of the coded structure construction process of the same embodiment. It is a figure (the 3) which shows the specific example of the coded structure construction process of the same embodiment. It is a figure which shows the coding structure of low delay coding in inter prediction coding. It is a figure which shows the coding structure of random access coding in inter prediction coding. It is a figure which shows the difference between a fixed structure and an adaptive structure in inter-predictive coding. It is a figure (the 1) for demonstrating the adaptive structure in inter-predictive coding. It is a figure (the 2) for demonstrating the adaptive structure in inter-predictive coding.

(First Embodiment)
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a video coding device 1 according to the first embodiment of the present invention. The video coding device 1 includes a coding structure construction unit 10, a subtractor 11, an orthogonal conversion / quantization unit 12, a variable length coding unit 13, an inverse quantization / inverse orthogonal conversion unit 14, an adder 15, and an intra prediction unit. A loop filter unit 17, a decoding picture storage unit 18, an inter-prediction unit 19, and an intra-inter changeover switch unit 20 are provided.

The coded structure construction unit 10 takes in input video information from the outside, selects a number of frames according to a predetermined GOP size from the captured input video information, and generates a frame group. Further, the coded structure construction unit 10 constructs a coded structure by selecting a picture type, a reference structure, and the like of each frame of the generated frame group. Further, the coded structure construction unit 10 outputs a coded target frame in units of CU (Coding Unit) blocks according to the coding order indicated by the constructed coded structure.

The subtractor 11 includes image information of the coded target frame output by the coding structure construction unit 10 and prediction image information output by the intra prediction unit 16 or the inter prediction unit 19 via the intra / inter changeover switch unit 20. Calculate the difference between. Further, the subtractor 11 outputs the difference image information based on the calculated difference to the orthogonal conversion / quantization unit 12.

The orthogonal conversion / quantization unit 12 performs orthogonal conversion and quantization on the difference image information output by the subtractor 11, and outputs the difference image information to the variable length coding unit 13 and the inverse quantization / inverse orthogonal conversion unit 14. The variable-length coding unit 13 generates coded data by variable-length coding the quantization coefficient output by the orthogonal conversion / quantization unit 12, and outputs the generated coded data to the outside.

The inverse quantization / inverse orthogonal conversion unit 14 performs inverse quantization and inverse orthogonal conversion on the quantization coefficient output by the orthogonal conversion / quantization unit 12, decodes the difference image information, and outputs the difference image information to the adder 15. To do. The adder 15 includes the decoding difference image information output by the inverse quantization / inverse orthogonal conversion unit 14 and the prediction image information output by the intra prediction unit 16 or the inter prediction unit 19 via the intra / inter changeover switch unit 20. Calculate the sum of. Further, the adder 15 outputs the decoded image information based on the calculated sum to the intra prediction unit 16 and the loop filter unit 17.

The intra prediction unit 16 uses the decoded image information output by the adder 15 as a reference frame to generate the intra prediction image information of the coded target block included in the coded target frame.

The loop filter unit 17 applies a loop filter to the decoded image information output by the adder 15, and writes and stores the decoded image information to which the loop filter is applied in the decoded picture storage unit 18. The inter-prediction unit 19 uses the decoded image information stored in the decoded picture storage unit 18 as a reference frame, and uses the inter-prediction image information of the coding target block included in the coding target frame output by the coding structure construction unit 10 as a reference frame. Generate.

The intra-inter changeover switch unit 20 switches between the intra-prediction unit 16 and the inter-prediction unit 19 according to the prediction mode of the coded block, and the intra-prediction image information or the inter-prediction unit output by the intra-prediction unit 16. The inter-prediction image information output by 19 is output to the subtractor 11 and the adder 15.

With the above configuration, the video coding apparatus 1 generates a frame group from the input video information captured from the outside, and raster scans each coded frame of the generated frame group to generate a CU block. The video coding apparatus 1 repeatedly encodes each generated CU block as a code target block in the order of raster scan, and outputs coded data for each code target block. The video coding apparatus 1 encodes the input video information to be encoded by repeatedly performing this process for the frames included in all the frame groups generated from the input video information.

In the process of coding, in order for the inter-prediction unit 19 to encode the frame of the P picture, the reference frame for the frame of the P picture needs to be stored in the decoded picture storage unit 18 in advance. Further, in order to encode the frame of the B picture, it is necessary that two reference frames for the frame of the B picture are stored in the decoded picture storage unit 18 in advance. The coded structure construction unit 10 constructs a coded structure that satisfies the requirements of these coding orders.

FIG. 2 is a block diagram showing the configuration of the coded structure construction unit 10. The coded structure construction unit 10 includes a coded frame selection unit 101, an I picture selection unit 102, a reference group generation unit 103, a combination determination unit 104, a picture type selection unit 105, a picture type storage unit 106, and a block output unit 107. To be equipped.

The coded target frame selection unit 101 selects a number of frames according to a predetermined GOP size from the input video information, generates a frame group, and outputs the frame group to the I picture selection unit 102 and the block output unit 107. The I-picture selection unit 102 selects the first frame of the frame group as the I-picture, and outputs the frame group to the reference group generation unit 103 with the first frame as the starting frame.

The reference group generation unit 103 selects a predetermined number of consecutive frames (for example, three) starting from the first frame of the frame group, and sets a combination of the selected predetermined number of consecutive frames as the first reference group. Generate. Further, the reference group generation unit 103 selects a predetermined number of consecutive frames from the frames that serve as the starting point, and generates a combination of the selected predetermined number of consecutive frames as a reference group. Further, the reference group generation unit 103 generates a new reference group by combining the two adjacent reference groups when it is determined by the join determination unit 104 that the two adjacent reference groups are both combined.

Further, the reference group generation unit 103 determines the combination of the generated reference group information, that is, the image information of the frame included in the reference group, and the frame number for specifying the frame of the frame included in the reference group in the frame group. Output to unit 104 and picture type selection unit 105.

The combination determination unit 104 determines whether or not to combine the frames included in the reference group generated and output by the reference group generation unit 103. Here, combining the frames included in the reference group means that the determination target frame, which is a frame between the first and last frames, is a B picture in which the first and last frames are reference frames. The combination determination unit 104 includes an evaluation value calculation unit 110, an evaluation value determination unit 111, and a determination result storage unit 112.

The evaluation value calculation unit 110 calculates the evaluation value of the reference group generated by the reference group generation unit 103. Here, the evaluation value is the value of the three intercosts calculated as follows. The intercost is, for example, an evaluation value related to an error when motion search is performed between frames. For example, the minimum value of the absolute error sum is applied, and this evaluation value is the RD (Rate-Distortion) cost. It is a value different from the coding cost that cannot be obtained without actually performing coding such as. Here, the minimum value of the absolute error sum represents the smallest value among the values obtained by detecting a plurality of motion vectors and calculating the absolute error sum for each motion vector.

First, the first frame, the determination target frame between the first and last frames, and the last frame are S frame, M frame, and E frame, respectively. When the reference group includes a predetermined number of consecutive frames (for example, 3 frames), the determination target frame becomes an intermediate frame, and when two adjacent reference groups are combined as described later, the determination target frame Is the last frame of the previous reference group in the display order.

The total of all the blocks of the one-way intercost (hereinafter referred to as L0 cost) when the S frame is used as the reference frame for the M frame in a predetermined block size unit is calculated. Next, the total of all blocks of the one-way intercost (hereinafter referred to as L1 cost) when the E frame is used as the reference frame in a predetermined block size unit predetermined for the M frame is calculated. Further, the total of all blocks of the bidirectional intercost (hereinafter referred to as Bi cost) when the S frame and the E frame are used as reference frames in a predetermined block size unit predetermined for the M frame is calculated.

The evaluation value determination unit 111 refers to whether or not the frames of the reference group should be combined based on the L0 cost, L1 cost, and Bi cost calculated by the evaluation value calculation unit 110, that is, the M frame, the S frame, and the E frame. Performs a combination determination process for determining whether or not to use a B picture as a frame. The evaluation value determination unit 111 writes and stores the determination result in the determination result storage unit 112 in association with the information indicating the reference group, and outputs the determination result to the picture type selection unit 105. The determination result storage unit 112 stores the determination result determined by the evaluation value determination unit 111 for each reference group.

The picture type selection unit 105 selects the picture type of the frame included in the reference group generated by the reference group generation unit 103 according to the determination result output by the combination determination unit 104. Further, when the reference group generation unit 103 determines that three consecutive frames cannot be selected, the picture type selection unit 105 selects the remaining frames as P pictures. Further, the picture type selection unit 105 selects the picture type for all the frames, and then selects the frame selected as the past P picture closest to the frame in the display order with respect to the frame selected as the P picture. Select as the reference frame for the frame.

Here, the past P picture closest in the display order is the closest P picture with a number younger than the frame to which the reference frame is assigned when consecutive numbers are assigned in the display order from the first frame that becomes the I picture. That is. If there is no past P picture closest to the display order, the picture type selection unit 105 selects the I picture as a reference frame.

The picture type storage unit 106 stores information indicating the picture type of the I picture in association with the information indicating the frame selected as the I picture by the I picture selection unit 102. Further, the picture type storage unit 106 stores information indicating the picture type of the B picture or P picture selected by the picture type selection unit 105 in association with the information indicating each frame together with the information indicating the reference frame.

The block output unit 107 receives the frame group output by the coded target frame selection unit 101, temporarily writes it in the internal storage area, and stores it. When the picture type selection is completed and the coded structure is determined, the block output unit 107 reads frames from the internal storage area in the order based on the defined coded structure, and raster scans each of the read frames. It is decomposed into CU blocks in the order of, and output as a coded block.

(Coded structure construction process of the first embodiment)
Next, the processing by the coded structure construction unit 10 of the first embodiment will be described. FIG. 3 is a flowchart showing the flow of the coded structure construction process by the coded structure construction unit 10. Further, FIG. 4 is a flowchart showing the flow of the subroutine of the join determination process by the join determination unit 104 called in the coded structure construction process.

The coded target frame selection unit 101 selects a number of frames according to a predetermined GOP size from the input video information to generate a frame group, and transmits the generated frame group to the I picture selection unit 102 and the block output unit 107. Output. The I-picture selection unit 102 selects the picture type of the first frame of the frame group as the I-picture, and writes and stores information indicating that the frame is selected as the I-picture in the picture-type storage unit 106. Further, the I-picture selection unit 102 sets the first frame as the starting frame and outputs the frame group to the reference group generation unit 103 (step S101).

The reference group generation unit 103 determines whether or not three consecutive frames can be selected from the frame group starting from the starting frame (step S102). When the reference group generation unit 103 determines that three consecutive frames can be selected (step S102-YES), the reference group generation unit 103 displays the three consecutive frames A, B, in the display order. Generate a reference group to be included as a C frame. The determination target frame in the reference group is the B frame.

The reference group generation unit 103 outputs the generated reference group to the combination determination unit 104 and the picture type selection unit 105 (step S105). The join determination unit 104 performs the join determination process using the A, B, and C frames included in the reference group output by the reference group generation unit 103 as S, M, and E frames (step S106).

(Join judgment processing by the join judgment unit)
In the subroutine of the combination determination process shown in FIG. 4, the evaluation value calculation unit 110 calculates three evaluation values of L0 cost, L1 cost, and Bi cost based on the S frame, the M frame, and the E frame (step S201). ..

The evaluation value determination unit 111 selects the smaller value of the Bi cost and the L1 cost (step S202), divides the selected value by L0, and calculates a relative value (step S203). If the values of Bi cost and L1 cost are the same, either one may be selected, so one of them is selected. When the processing of steps S202 and S203 is expressed by a mathematical formula, it becomes the following equation (1).

Relative value = min (Bi cost, L1 cost) / L0 cost ... (1)

The evaluation value determination unit 111 determines the magnitude of the calculated relative value based on the predetermined relative evaluation thresholds Th_min and Th_max (step S204). When the evaluation value determination unit 111 determines that the calculated relative value is less than the value of Th_min (step S204: <Th_min), the evaluation value determination unit 111 outputs "True" indicating that the frames included in the reference group are combined (Step S204: <Thh_min). Step S205). On the other hand, when the evaluation value determination unit 111 determines that the calculated relative value exceeds the value of Th_max (step S204:> Th_max), "False" indicating that the frames included in the reference group are not combined. Is output (step S206).

On the other hand, when the evaluation value determination unit 111 determines that the calculated relative value is equal to or greater than the value of Th_min and less than or equal to the value of Thmax (step S204: ≧ Th_min and ≦ Th_max), the magnitude of the Bi cost value. Is determined based on a predetermined absolute evaluation threshold Th_abs (step S207). When the evaluation value determination unit 111 determines that the Bi cost value is less than the value of Th_abs (step S207: <Th_abs), the evaluation value determination unit 111 outputs "True" indicating that the frames included in the reference group are combined (True). Step S208). On the other hand, when the evaluation value determination unit 111 determines that the Bi cost value is equal to or greater than the Th_abs value (step S207: ≧ Th_abs), the evaluation value determination unit 111 outputs “False” indicating that the frames included in the reference group are not combined (False). Step S206).

In the processing of steps S205, S206, and S208, the evaluation value determination unit 111 writes and stores the information of "True" or "False" indicating the determination result in association with the information indicating the reference group in the determination result storage unit 112. Then, the subroutine of the join judgment process is terminated.

Returning to FIG. 3, the picture type selection unit 105 determines the determination result output by the combination determination unit 104 (step S107). When the determination result is "False" (step S107-False), the picture type selection unit 105 selects information indicating that the B frame picture type of the reference group is selected as the P picture and the B frame is selected as the P picture. It is written and stored in the picture type storage unit 106. The picture type selection unit 105 outputs information starting from the B frame to the reference group generation unit 103 (step S108). When the reference group generation unit 103 receives the information output by the picture type selection unit 105 starting from the B frame, the reference group generation unit 103 proceeds to the process of step S102 with the B frame as the starting frame.

On the other hand, when the determination result is "True" (step S107-True), the picture type selection unit 105 selects the picture type of the B frame as the B picture with the A frame and the C frame as the reference frame, and the B frame is B. Information indicating that the picture has been selected and information indicating that the A frame and the C frame are reference frames are written and stored in the picture type storage unit 106. Further, the picture type selection unit 105 selects the picture type of the C frame as the P picture, and writes and stores the information indicating that the C frame is selected as the P picture in the picture type storage unit 106. The picture type selection unit 105 outputs information indicating a reference group whose determination result is “True” to the reference group generation unit 103 (step S109).

When the determination result output by the picture type selection unit 105 receives the information indicating the reference group of "True", the reference group generation unit 103 refers to the determination result storage unit 112 of the combination determination unit 104. Based on the information stored in the determination result storage unit 112, the reference group generation unit 103 determines whether or not the determination result of the reference group immediately before the reference group being processed is "True" in the display order. (Step S110).

When the reference group generation unit 103 determines in the display order that the determination result of the reference group immediately before the reference group being processed is not "True" (step S110-NO), the C frame is used as the starting frame (step S110-NO). Step S116), the process proceeds to step S102.

When the reference group generation unit 103 determines in the display order that the determination result of the reference group immediately before the reference group being processed is "True" (step S110-YES), the reference group generation unit 103 generates a new reference group. To do. That is, the reference group generation unit 103 sets the first frame and the last frame of the previous reference group as α frames and β frames, respectively, and sets the last frame of the reference group being processed as a γ frame, and sets α and β. , Generates a new reference group containing γ frames. The β frame is also the first A frame of the reference group being processed, and is a determination target frame in the reference group. The reference group generation unit 103 outputs the generated new reference group information to the combination determination unit 104 and the picture type selection unit 105 (step S111).

The coupling determination unit 104 performs the coupling determination processing on the α, β, and γ frames included in the reference group output by the reference group generation unit 103. The evaluation value calculation unit 110 performs the coupling determination process shown in FIG. 4 with each of the α, β, and γ frames as S, M, and E frames (step S112).

The picture type selection unit 105 determines the determination result output by the combination determination unit 104 (step S113). When the determination result is "False" (step S113-False), the picture type selection unit 105 starts from the γ frame, which is the last frame of the new reference group, that is, the C frame of the reference group being processed before joining. The information indicating that the above is to be output to the reference group generation unit 103 (step S115). The reference group generation unit 103 receives the information indicating that the C frame output by the picture type selection unit 105 is the starting frame, and proceeds to the process of step S102 with the C frame as the starting frame (step S116).

On the other hand, when the determination result is "True" (step S113-True), the picture type selection unit 105 selects the picture type of the β frame as the B picture with the α frame and the γ frame as the reference frame, and the β frame is B. Information indicating that the picture has been selected and information indicating that the α frame and the γ frame are reference frames are written and stored in the picture type storage unit 106. Further, the picture type selection unit 105 selects the picture type of the γ frame as the P picture, and writes and stores the information indicating that the γ frame is selected as the P picture in the picture type storage unit 106 (step S114).

The picture type selection unit 105 outputs to the reference group generation unit 103 information indicating that the γ frame, which is the last frame of the new reference group, that is, the C frame of the reference group being processed before joining is used as the starting frame. (Step S115). The reference group generation unit 103 receives the information indicating that the C frame output by the picture type selection unit 105 is the starting frame, and proceeds to the process of step S102 with the C frame as the starting frame (step S116).

On the other hand, in step S102, when the reference group generation unit 103 determines that three consecutive frames cannot be selected (step S102-NO), the instruction information for selecting the remaining frames of the frame group as P pictures. Is output to the picture type selection unit 105.

The picture type selection unit 105 receives the instruction information from the reference group generation unit 103, selects the picture type of the remaining frame as the P picture, and stores information indicating that the remaining frame is selected as the P picture. It is written in the part 106 and stored (step S103). The picture type selection unit 105 selects as a reference frame the frame of the past P picture closest to the frame in the display order for each of the frames selected as the P picture. If there is no past P picture closest to the display order, the I picture is selected as the reference frame of the frame. The picture type selection unit 105 writes and stores the information of the reference frame selected for each P picture in the picture type storage unit 106 (step S104).

As a result, the picture type storage unit 106 stores the information of the finally constructed coded structure. The block output unit 107 temporarily stores the frame group received from the coded target frame selection unit 101 in the internal storage area, and internally in the order based on the coded structure stored in the picture type storage unit 106. Read a frame from the storage area of. The block output unit 107 decomposes each of the read frames into CU blocks in the order of raster scan and outputs them as a coded target block, and the coded structure construction unit 10 ends the processing for one frame group.

(Specific example of coded structure processing of the first embodiment)
Next, an example in which the coded structure construction process of the first embodiment shown in FIG. 3 is applied to the frame group shown in FIG. 5A will be described with reference to FIGS. 5 to 8. The coded target frame selection unit 101 generates the frame group shown in FIG. 5A from the input video information. The frame group includes scene changes in the frames of frame number 8 and frame number 9.

As shown in FIG. 5B, the I-picture selection unit 102 selects the picture type of the frame with frame number 0 as the I-picture, and the picture type storage unit 106 stores information indicating that the frame is selected as the I-picture. Write in and memorize. Further, the I-picture selection unit 102 sets the frame of the frame number 0 as the starting frame (step S101).

The reference group generation unit 103 selects three consecutive frames of frame numbers 0, 1 and 2 with the frame of frame number 0 as the starting frame (step S102-YES). The reference group generation unit 103 generates a reference group 301 including frames 0, 1 and 2 as frames A, B, and C, and uses the generated reference group 301 information as a combination determination unit 104 and a picture type selection unit 105. Is output to (step S105). In the reference group 301, the determination target frame is the frame of frame number 1.

It is assumed that the combination determination unit 104 performs the combination determination process shown in FIG. 4 on the reference group 301, and since there is no scene change here, the evaluation value determination unit 111 outputs “True” (step S106). .. Since the determination result output by the combination determination unit 104 is "True" (step S107-True), the picture type selection unit 105 sets the picture type of the frame of frame number 1 to the frame of frame number 0 and frame number 2. Select as the B picture to be the reference frame. The picture type selection unit 105 writes in the picture type storage unit 106 information indicating that the frame of frame number 2 is selected as the B picture and information indicating that the frames of frame number 0 and frame number 2 are reference frames. Remember with.

Further, the picture type selection unit 105 selects the picture type of the frame of the frame number 2 as the P picture, and writes and stores the information indicating that the frame of the frame number 2 is selected as the P picture in the picture type storage unit 106. .. The picture type selection unit 105 outputs information indicating the reference group 301 whose determination result is “True” to the reference group generation unit 103 (step S109).

As shown in FIG. 5C, the reference group generation unit 103 refers to the determination result storage unit 112, and since the reference group immediately before the reference group 301 does not exist (step S110-NO), the frame. With the frame number 2 as the starting frame (step S116), three consecutive frames with frame numbers 2, 3 and 4 are selected (step S102-YES). The reference group generation unit 103 generates a reference group 302 including frames of frame numbers 2, 3 and 4 as frames A, B, and C, and uses the generated reference group 302 information as a combination determination unit 104 and a picture type selection unit 105. Is output to (step S105). In the reference group 302, the determination target frame is the frame of frame number 3.

It is assumed that the combination determination unit 104 performs the combination determination process shown in FIG. 4 on the reference group 302, and since there is no scene change here as well, the evaluation value determination unit 111 outputs “True” (step S106). .. Since the determination result output by the combination determination unit 104 is "True" (step S107-True), the picture type selection unit 105 sets the picture type of the frame of frame number 3 to the frame of frame number 2 and frame number 4. Select as the B picture to be the reference frame. The picture type selection unit 105 writes in the picture type storage unit 106 information indicating that the frame of frame number 3 is selected as the B picture and information indicating that the frames of frame numbers 2 and 4 are reference frames. Remember with.

Further, the picture type selection unit 105 selects the picture type of the frame number 4 as the P picture, and writes and stores the information indicating that the frame of the frame number 4 is selected as the P picture in the picture type storage unit 106. The picture type selection unit 105 outputs information indicating the reference group 302 whose determination result is “True” to the reference group generation unit 103 (step S109).

As shown in FIG. 6D, the reference group generation unit 103 refers to the determination result storage unit 112, and determines that the determination result of the reference group 301 immediately before the reference group 302 is “True”. (Step S110-YES). The reference group generation unit 103 generates a reference group 303 including frames of frame numbers 0, 2, and 4 as frames α, β, and γ. In the reference group 303, the determination target frame is the frame of frame number 2. The reference group generation unit 103 outputs the generated information of the reference group 303 to the combination determination unit 104 and the picture type selection unit 105 (step S111).

It is assumed that the combination determination unit 104 performs the combination determination process shown in FIG. 4 on the reference group 303, and since there is no scene change here as well, the evaluation value determination unit 111 outputs “True” (step S112). .. Since the determination result is "True" (step S113-True), the picture type selection unit 105 selects the picture type of the frame of frame number 2 as the B picture with the frames of frame number 0 and frame number 4 as reference frames. To do.

Information indicating that the frame of the frame number 2 is a P picture has already been written in the picture type storage unit 106. Therefore, the picture type selection unit 105 stores information indicating that the frame of frame number 2 is selected as the B picture and information indicating that the frames of frame number 0 and frame number 4 are reference frames. Is overwritten and stored (step S114). The picture type selection unit 105 outputs information indicating the frame number 4, which is the next starting frame, to the reference group generation unit 103 (step S115).

As shown in FIG. 6E, the reference group generation unit 103 selects three consecutive frames of frame numbers 4, 5 and 6 with the frame of frame number 4 as the starting frame (step S116) (step). S102-YES). The reference group generation unit 103 generates a reference group 304 including frames of frame numbers 4, 5 and 6 as frames A, B, and C, and uses the generated reference group 304 information as a combination determination unit 104 and a picture type selection unit 105. Is output to (step S105). In the reference group 304, the determination target frame is the frame of frame number 5.

It is assumed that the combination determination unit 104 performs the combination determination process shown in FIG. 4 on the reference group 304, and since there is no scene change here as well, the evaluation value determination unit 111 outputs “True” (step S106). .. Since the determination result output by the combination determination unit 104 is "True" (step S107-True), the picture type selection unit 105 sets the picture type of the frame of frame number 5 to the frames of frame number 4 and frame number 6. Select as the B picture to be the reference frame. The picture type selection unit 105 writes in the picture type storage unit 106 information indicating that the frame of frame number 5 is selected as the B picture and information indicating that the frames of frame numbers 4 and 6 are reference frames. Remember with.

Further, the picture type selection unit 105 selects the picture type of the frame number 6 as the P picture, and writes and stores the information indicating that the frame of the frame number 6 is selected as the P picture in the picture type storage unit 106. The picture type selection unit 105 outputs information indicating the reference group 304 whose determination result is “True” to the reference group generation unit 103 (step S109).

As shown in FIG. 6F, the reference group generation unit 103 refers to the determination result storage unit 112, and determines that the determination result of the reference group 303 immediately before the reference group 304 is “True”. (Step S110-YES). The reference group generation unit 103 generates a reference group 305 including frames of frame numbers 0, 4, and 6 as frames α, β, and γ. In the reference group 305, the determination target frame is the frame of frame number 4. The reference group generation unit 103 outputs the generated information of the reference group 305 to the combination determination unit 104 and the picture type selection unit 105 (step S111).

It is assumed that the combination determination unit 104 performs the combination determination process shown in FIG. 4 on the reference group 305, and since there is no scene change here as well, the evaluation value determination unit 111 outputs “True” (step S112). .. Since the determination result is "True" (step S113-True), the picture type selection unit 105 selects the picture type of the frame of frame number 4 as the B picture using the frames of frame number 0 and frame number 6 as reference frames. To do.

Information indicating that the frame has the frame number 4 is a P picture has already been written in the picture type storage unit 106. Therefore, the picture type selection unit 105 stores information indicating that the frame of frame number 4 is selected as the B picture and information indicating that the frames of frame number 0 and frame number 6 are reference frames. Is overwritten and stored (step S114). The picture type selection unit 105 outputs information indicating a frame number 6 to be the next starting frame to the reference group generation unit 103 (step S115).

As shown in FIG. 7 (g), the reference group generation unit 103 selects three consecutive frames of frame numbers 6, 7, and 8 with the frame of frame number 6 as the starting frame (step S116) (step). S102-YES). The reference group generation unit 103 generates a reference group 306 including frames of frame numbers 6, 7, and 8 as frames A, B, and C, and uses the generated reference group 306 information as a combination determination unit 104 and a picture type selection unit 105. Is output to (step S105). In the reference group 306, the determination target frame is the frame of frame number 7.

It is assumed that the combination determination unit 104 performs the combination determination process shown in FIG. 4 on the reference group 306, and since there is no scene change here as well, the evaluation value determination unit 111 outputs “True” (step S106). .. Since the determination result output by the combination determination unit 104 is "True" (step S107-True), the picture type selection unit 105 sets the picture type of the frame of frame number 7 to the frame of frame number 6 and frame number 8. Select as the B picture to be the reference frame. The picture type selection unit 105 writes in the picture type storage unit 106 information indicating that the frame of frame number 7 is selected as the B picture and information indicating that the frames of frame numbers 6 and 8 are reference frames. Remember with.

Further, the picture type selection unit 105 selects the picture type of the frame number 8 as the P picture, and writes and stores the information indicating that the frame of the frame number 8 is selected as the P picture in the picture type storage unit 106. The picture type selection unit 105 outputs information indicating the reference group 306 whose determination result is “True” to the reference group generation unit 103 (step S109).

As shown in FIG. 7H, the reference group generation unit 103 refers to the determination result storage unit 112, and determines that the determination result of the reference group 305 immediately before the reference group 306 is “True”. (Step S110-YES). The reference group generation unit 103 generates a reference group 307 including frames of frame numbers 0, 6, and 8 as frames α, β, and γ. In the reference group 307, the determination target frame is the frame of frame number 6. The reference group generation unit 103 outputs the generated information of the reference group 307 to the combination determination unit 104 and the picture type selection unit 105 (step S111).

It is assumed that the combination determination unit 104 performs the combination determination process shown in FIG. 4 on the reference group 307, and since there is no scene change here as well, the evaluation value determination unit 111 outputs “True” (step S112). .. Since the determination result is "True" (step S113-True), the picture type selection unit 105 selects the picture type of the frame of frame number 6 as the B picture using the frames of frame number 0 and frame number 8 as reference frames. To do.

Information indicating that the frame of frame number 6 is a P picture has already been written in the picture type storage unit 106. Therefore, the picture type selection unit 105 stores information indicating that the frame of frame number 6 is selected as the B picture and information indicating that the frames of frame number 0 and frame number 8 are reference frames. Is overwritten and stored (step S114). The picture type selection unit 105 outputs information indicating a frame number 8 to be the next starting frame to the reference group generation unit 103 (step S115).

As shown in FIG. 7 (i), the reference group generation unit 103 selects three consecutive frames of frame numbers 8, 9, and 10 with the frame of frame number 8 as the starting frame (step S116) (step). S102-YES). The reference group generation unit 103 generates a reference group 308 including frames of frame numbers 8, 9, and 10 as frames A, B, and C, and uses the generated reference group 308 information as a combination determination unit 104 and a picture type selection unit 105. Is output to (step S105). In the reference group 308, the determination target frame is the frame of frame number 9.

The coupling determination unit 104 performs the coupling determination processing shown in FIG. 4 on the reference group 308. As shown in FIG. 5A, since there is a scene change between the frame number 8 and the frame number 9, it is assumed that the evaluation value determination unit 111 outputs "False" here (step S106). Since the determination result output by the combination determination unit 104 is "False" (step S107-False), the picture type selection unit 105 selects the picture type of the frame of frame number 9 as the P picture. The picture type selection unit 105 writes and stores information indicating that the frame of frame number 9 is selected as the P picture in the picture type storage unit 106. The picture type selection unit 105 uses the frame of frame number 9 as the starting frame (step S108).

As shown in FIG. 8 (j), the reference group generation unit 103 cannot select three consecutive frames starting from the frame of frame number 9 (step S102-NO). The reference group generation unit 103 outputs instruction information for selecting the picture type of the frame of the frame number 10 which is the remaining frame as a P picture to the picture type selection unit 105. In response to the instruction information, the picture type selection unit 105 selects the frame of frame number 10 as the P picture, and writes information indicating that the frame of frame number 10 is selected as the P picture in the picture type storage unit 106. It is stored (step S103).

The picture type selection unit 105 refers to the information stored in the picture type storage unit 106, and refers to the past P picture closest to the display order for the frames of frame numbers 8, 9 and 10 selected as the P picture. Select the frame of as the reference frame. If there is no past P picture closest to the display order, the frame of the I picture is selected as the reference frame. The picture type selection unit 105 writes and stores information indicating the selected reference frame in the picture type storage unit 106 (step S104).

As for the frame of frame number 8, since there is no past P picture closest to the display order, the frame of the I picture of frame number 0 becomes the reference frame. For the frame of frame number 9, the past P picture closest in the display order is the frame of frame number 8. For the frame of frame number 10, the past P picture closest in the display order is the frame of frame number 9.

When the coded structure constructed based on the information stored in the picture type storage unit 106 is layered, the coded structure has a layered structure of four layers of B pictures as shown in FIG. 8 (k). ..

(Specific example of combination judgment processing)
Next, an example of the coupling determination processing by the coupling determination unit 104 will be described with reference to FIGS. 9 and 10. FIG. 9 shows an example of S, M, and E frames to which the combination determination process is applied, and FIG. 9 (i) shows an example in which there is no scene change in the three frames of S, M, and E frames (ii). ) Is an example in which there is a scene change between the M frame and the E frame, and (iii) is an example in which there is a scene change between the S frame and the M frame. 9 (iv) and 9 (v) are examples in which there is a partial scene change from the S frame to the E frame.

In this example, it is assumed that "0.8" and "1.2" are set as the relative evaluation thresholds Th_min and Th_max, respectively, and "350" is set as the absolute evaluation thresholds Th_abs.

In the case of FIG. 9 (i), the evaluation value calculation unit 110 of the combination determination unit 104 sets the evaluation values of the L0, L1, and Bi costs as “150”, “150”, and “100”, respectively, as shown in FIG. Is calculated (step S201). When the equation (1) is applied by the evaluation value determination unit 111, the relative value = min (100, 150) / 150 = 100/150 = 2/3 (steps S202, S203). Since 2/3 <0.8 (step S204, <Th_min), the evaluation value determination unit 111 outputs "True" (step S205).

In the case of FIG. 9 (ii), the evaluation value calculation unit 110 of the combination determination unit 104 has "150", "500", and "400" as the evaluation values of the L0, L1, and Bi costs, respectively, as shown in FIG. Is calculated (step S201). When the equation (1) is applied by the evaluation value determination unit 111, the relative value = min (400,500) / 150 = 400/150 = 8/3 (steps S202 and S203). Since 8/3> 1.2 (step S204,> Th_max), the evaluation value determination unit 111 outputs “False” (step S206).

In the case of FIG. 9 (iii), the evaluation value calculation unit 110 of the combination determination unit 104 sets the evaluation values of the L0, L1, and Bi costs as “500”, “150”, and “400”, respectively, as shown in FIG. Is calculated (step S201). When the equation (1) is applied by the evaluation value determination unit 111, the relative value = min (400, 150) / 500 = 150/500 = 0.3 (steps S202, S203). Since 0.3 <0.8 (step S204, <Th_min), the evaluation value determination unit 111 outputs "True" (step S205).

In the case of FIG. 9 (iv), the evaluation value calculation unit 110 of the combination determination unit 104 sets the evaluation values of the L0, L1, and Bi costs as “300”, “400”, and “300”, respectively, as shown in FIG. Is calculated (step S201). When the equation (1) is applied by the evaluation value determination unit 111, the relative value = min (300, 400) / 300 = 300/300 = 1 (steps S202 and S203). Since 0.8 ≦ 1 ≦ 1.2 (step S204, ≧ Th_min and ≦ Th_max), the evaluation value determination unit 111 determines the magnitude of the Bi cost value based on the absolute evaluation threshold value (step S207). ). Since the Bi cost value "300" <Th_abs "350" (step S207, <Thh_abs), the evaluation value determination unit 111 outputs "True" (step S208).

In the case of FIG. 9 (v), the evaluation value calculation unit 110 of the combination determination unit 104 uses “400”, “500”, and “400” as the evaluation values of the L0, L1, and Bi costs, respectively, as shown in FIG. Is calculated (step S201). When the equation (1) is applied by the evaluation value determination unit 111, the relative value = min (400,500) / 400 = 400/400 = 1 (steps S202 and S203). Since 0.8 ≦ 1 ≦ 1.2 (step S204, ≧ Th_min and ≦ Th_max), the evaluation value determination unit 111 determines the magnitude of the Bi cost value based on the absolute evaluation threshold value (step S207). ). Since the Bi cost value “400” ≧ Th_abs “350” (step S207, ≧ Th_abs), the evaluation value determination unit 111 outputs “False” (step S206).

According to the configuration of the first embodiment described above, in the coding structure construction unit 10, the coding target frame selection unit 101 selects a number of frames according to a predetermined GOP size from the input video information to form a frame group. Generate. The I-picture selection unit 102 selects the first frame of the frame group as the I-picture and uses it as the starting frame. The reference group generation unit 103 selects three consecutive frames starting from the starting frame, and generates a reference group using the selected three frames as A, B, and C frames. The evaluation value calculation unit 110 of the combination determination unit 104 calculates three evaluation values, L0 cost, L1 cost, and Bi cost, based on the A, B, and C frames included in the reference group. The evaluation value determination unit 111 selects the smaller value of the Bi cost and the L1 cost, and determines based on the relative value based on the value of the L0 cost and the predetermined relative evaluation thresholds Th_min and Th_max. It is determined whether or not the reference relationship between the B frame, which is the target frame, and the A and C frames is combined. Further, when the relative value is not more than the value of Th_min and is included in the range not more than the value of Th_max, the evaluation value determination unit 111 further bases on the value of Bi cost and the absolute evaluation threshold Th_abs. , It is determined whether or not the reference relationship between the B frame, which is the determination target frame, and the A and C frames is combined. When the evaluation value determination unit 111 determines that the reference relationship between the B frame and the A and C frames is to be combined, the picture type selection unit 105 selects the B frame as the B picture with the A and C frames as the reference frames. .. That is, the coded structure construction unit 10 does not perform an operation that actually performs coding, calculates three evaluation values based on three frames, and based on the calculated three evaluation values, 3 It is determined whether or not to combine the reference relationships of two frames. Therefore, in the coded structure construction unit 10 of the first embodiment, it is possible to obtain a coded structure having high coding efficiency with a low calculation amount.

In addition, the reference group generation unit 103 combines two adjacent reference groups determined by the evaluation value determination unit 111 to combine the reference relationship between the determination target frame and the first and last frames, and determines the overlapping frame. Repeat the process of creating a new reference group. The coupling determination unit 104 performs the coupling determination processing based on the α, β, and γ frames of the newly generated reference group by the reference group generation unit 103. As a result, the coded structure construction unit 10 of the first embodiment can construct a coded structure having two or more layers deep, and can improve the coding efficiency. There is. For example, as described above, the example of FIG. 8K shows that it is possible to construct a coded structure having a hierarchical structure of four layers of B pictures.

(Second Embodiment)
FIG. 11 is a block diagram showing the configuration of the coded structure construction unit 10a according to the second embodiment of the present invention. The coded structure construction unit 10a is a functional unit that is applied in place of the coded structure construction unit 10 of the video coding apparatus 1 of the first embodiment. When the coded structure construction unit 10a of the second embodiment is applied, the code of the video coding device is replaced with "1" and represented by "1a". In the second embodiment, the same configurations as those in the first embodiment are designated by the same reference numerals, and the configurations different from those in the coded structure construction unit 10 of the first embodiment will be described below.

The coded structure construction unit 10a includes a coded frame selection unit 101, an I picture selection unit 102a, a reference group generation unit 103a, a combination determination unit 104, a picture type selection unit 105a, a picture type storage unit 106, and a block output unit 107. And a reference group storage unit 108.

In the coded structure construction unit 10a, the I picture selection unit 102a selects the first frame of the frame group as the I picture. The reference group generation unit 103a sets the initial values to i = 0 and j = 0, and sets a reference group including three frames whose frame numbers are "2j", "2j + 2 ⁱ ", and "2j + 2 ^{(i + 1)} " from the frame group. Repeat while j is less than N / 2, adding 1 to j until it can no longer be generated. Here, the value of N is the maximum value of the frame number given to the frame included in the frame group, and is a value obtained by subtracting "1" from the number of frames in the frame group.

The reference group generation unit 103a, and "2j" is selected frame number, "2j + ^{2 i",} in each of the combinations of the frame of the "2j ^{+ 2 (i + 1)",} the three frames A, B, and C frame, A Generate a reference group Ref [i] [j] with indexes of i and j including frames B and C (however, when generating a reference group including frames A, B and C, set i = 0. It is fixed and only j is a variable). Further, the reference group generation unit 103a writes and stores the generated reference groups Ref [0] [j] in the reference group storage unit 108.

Further, the reference group generation unit 103a sets the initial value to i = 1, j = 0, and sets the combination determination unit 104 in both the reference groups Ref [i-1] [j] and Ref [i-1] [j + 1]. The detection of the combination in which the judgment result determined by is "True" is repeated while adding 2 to j and while j is less than N / 2, and further adding 1 to i, i is depth_max. Repeat for less than. Here, depth_max is a value indicating the number of layers of the coded structure to be constructed, which is predetermined, and in the case of three layers, depth_max is "3".

In addition, the reference group generation unit 103a is the first of Ref [i-1] [j] in each of the detected combinations of the reference groups Ref [i-1] [j] and Ref [i-1] [j + 1]. The last frame and the last frame of Ref [i-1] [j + 1] are selected as α, β, and γ frames. Further, the reference group generation unit 103a writes and stores a new reference group including the selected α, β, and γ frames in the reference group storage unit 108 as the reference group Ref [i] “j / 2”.

The picture type selection unit 105a selects the picture type of the frame included in the reference groups Ref [i] [j] according to the determination result output by the combination determination unit 104. If the picture type selection unit 105a determines that the reference group generation unit 103a cannot select a frame having a frame number of "2j", "2j + 2 ⁱ ", or "2j + 2 ^{(i + 1)} ", the picture type selection unit 105a selects the remaining frame as a P picture. Select as.

Further, the picture type selection unit 105a selects the picture type for all the frames, and then selects the frame selected as the past P picture closest to the frame in the display order with respect to the frame selected as the P picture. Select as the reference frame for the frame. If there is no past P picture closest to the display order, the frame of the I picture is selected as the reference frame of the frame. The reference group storage unit 108 stores the reference groups Ref [i] [j] generated by the reference group generation unit 103a.

(Coded structure construction process of the second embodiment)
Next, the processing by the coded structure construction unit 10a of the second embodiment will be described. 12 and 13 are flowcharts showing the flow of the coded structure construction process by the coded structure construction unit 10. The processes shown in FIGS. 12 and 13 are continuous processes in which the processes are connected at the locations indicated by reference numerals A. The process up to the point indicated by the reference numeral A shown in FIG. 12 is the process of constructing the coded structure of the first layer indicated by i = 0, and the process shown in FIG. 13 is the process indicated by i = 1 and 2. This is a process for constructing the coded structure of the second and third layers shown.

The coded target frame selection unit 101 selects a number of frames according to a predetermined GOP size from the input video information to generate a frame group. The coded target frame selection unit 101 assigns a frame number from “0” to the generated frame group in the display order and outputs the frame number to the I picture selection unit 102a and the block output unit 107. The I-picture selection unit 102a selects the picture type of the first frame of the frame group, that is, the frame of frame number 0, as the I-picture. The I-picture selection unit 102a writes and stores information indicating that the frame having the frame number 0 is selected as the I-picture in the picture type storage unit 106 (step S301).

The reference group generation unit 103a sets i = 0 and j = 0 (step S302), and whether or not three frames having frame numbers "2j", "2j + 2 ⁱ ", and "2j + 2 ^{(i + 1)} " can be selected from the frame group. (Step S303).

When the reference group generation unit 103a determines that three frames can be selected (step S303-YES), the reference group generation unit 103a includes the three frames as A, B, and C frames in the display order. Generate reference groups Ref [0] [j]. The reference group generation unit 103a writes and stores the generated reference groups Ref [0] [j] in the reference group storage unit 108 (step S305). The determination target frame in the reference group is the B frame.

When the information of the reference group Ref [0] [j] is written to the reference group storage unit 108 by the reference group generation unit 103a, the combination determination unit 104 includes A included in the written reference group Ref [0] [j]. , B, C frames are set as S, M, E frames, and the combination determination process shown in FIG. 4 is performed (step S306).

The picture type selection unit 105a determines the determination result output by the combination determination unit 104 (step S307). When the determination result is "False" (step S307-False), the picture type selection unit 105a selects the B frame and C frame picture types of the reference groups Ref [0] [j] as P pictures. The picture type selection unit 105a writes and stores information indicating that the B frame and the C frame of the reference groups Ref [0] [j] are selected as P pictures in the picture type storage unit 106 (step S308).

On the other hand, when the determination result is "True" (step S307-True), the picture type selection unit 105a sets the picture type of the B frame of the reference groups Ref [0] [j] as the reference frame A frame and the C frame. Select as B picture. The picture type selection unit 105a stores information indicating that the B frame of the reference group Ref [0] [j] is selected as the B picture and information indicating that the A frame and the C frame are reference frames. Write to 106 and memorize it. Further, the picture type selection unit 105a selects the picture type of the C frame of the reference group Ref [0] [j] as the P picture. The picture type selection unit 105a writes and stores information indicating that the C frame of the reference group Ref [0] [j] is selected as the P picture in the picture type storage unit 106 (step S309).

After the processing of step S308 and step S309, the picture type selection unit 105a outputs instruction information for continuing the processing to the reference group generation unit 103a. The reference group generation unit 103a adds "1" to the value of j (step S310), and determines whether or not the value of j is less than N / 2 (step S311). When the reference group generation unit 103a determines that the value of j is less than N / 2 (step S311-YES), the reference group generation unit 103a repeats the process from step S303. On the other hand, when the reference group generation unit 103a determines that the value of j is not less than N / 2 (step S311-NO), the reference group generation unit 103a starts the process shown in FIG. 13 via the portion indicated by the reference numeral A.

On the other hand, when the reference group generation unit 103a determines that the three frames cannot be selected (step S303-NO), the picture type selection unit 105a provides instruction information for selecting the remaining frames of the frame group as P pictures. Output to. The picture type selection unit 105a receives the instruction information from the reference group generation unit 103a, selects the picture type of the remaining frame as the P picture, and stores information indicating that the remaining frame is selected as the P picture. It is written and stored in the unit 106 (step S304). The reference group generation unit 103a outputs the instruction information to the picture type selection unit 105a, and then starts the process shown in FIG. 13 via the portion indicated by reference numeral A.

As shown in FIG. 13, the reference group generation unit 103a sets i = 1, j = 0 (step S312), refers to the determination result storage unit 112 of the combination determination unit 104, and refers to the reference group Ref [i-1] [ It is determined whether or not the determination results of [j] and Ref [i-1] [j + 1] are both “True” (step S313). When the reference group generation unit 103a determines that both the determination results of the reference groups Ref [i-1] [j] and Ref [i-1] [j + 1] are not "True" (step S313-NO), the process To step S318.

On the other hand, when the reference group generation unit 103a determines that the determination results of the reference groups Ref [i-1] [j] and Ref [i-1] [j + 1] are both "True" (step S313-YES). ), The first and last frames of the reference group Ref [i-1] [j] and the last frame of the reference group Ref [i-1] [j + 1] are selected as α, β, and γ frames. .. The reference group generation unit 103a generates a reference group Ref [i] [j / 2] including the selected α, β, γ frames, and uses the generated reference group Ref [i] [j / 2] as a reference group storage unit. It is written in 108 and stored (step S314).

When the information of the reference group Ref [i] [j / 2] is written in the reference group storage unit 108 by the reference group generation unit 103a, the combination determination unit 104 writes the reference group Ref [i] [j / 2]. The combination determination process shown in FIG. 4 is performed using the α, β, and γ frames included in the above as S, M, and E frames (step S315).

The picture type selection unit 105a determines the determination result output by the combination determination unit 104 (step S316). When the determination result is "False" (step S316-False), the picture type selection unit 105a outputs instruction information for advancing the process to step S318 to the reference group generation unit 103a.

On the other hand, when the determination result is "True" (step S316-True), the picture type selection unit 105a refers to the α frame and the γ frame as the picture types of the β frames of the reference groups Ref [i] [j / 2]. Select as the B picture. The picture type selection unit 105a provides information indicating that the β frame of the reference group Ref [i] [j / 2] is selected as the B picture and information indicating that the α frame and the γ frame are reference frames. It is written in the storage unit 106 and stored. Further, the picture type selection unit 105a selects the picture type of the γ frame of the reference group Ref [i] [j / 2] as the P picture. The picture type selection unit 105a writes and stores information indicating that the γ frame of the reference group Ref [i] [j / 2] is selected as the P picture in the picture type storage unit 106. The picture type selection unit 105a outputs instruction information for advancing the process to step S318 to the reference group generation unit 103a (step S317).

The reference group generation unit 103a adds "2" to the value of j (step S318), and determines whether or not the value of j is less than N / 2 (step S319). When the reference group generation unit 103a determines that the value of j is less than N / 2 (step S319-YES), the reference group generation unit 103a repeats the processing from step S313.

On the other hand, when the reference group generation unit 103a determines that the value of j is not less than N / 2 (step S319-NO), "1" is added to the value of i to set j = 0 (step S320).
The reference group generation unit 103a determines whether or not the value of i is less than depth_max (here, depth_max = 3) (step S321). When the reference group generation unit 103a determines that the value of i is less than depth_max (step S321-YES), the reference group generation unit 103a repeats the process from step S313. On the other hand, when the reference group generation unit 103a determines that the value of i is not less than depth_max (step S321-NO), the reference group generation unit 103a outputs instruction information for causing the picture type selection unit 105a to select a reference frame for the frame of the P picture. ..

Upon receiving the instruction information, the picture type selection unit 105a selects the frame of the past P picture closest to the frame in the display order as the reference frame of the frame for each of the frames selected as the P picture. If there is no past P picture closest to the display order, the frame of the I picture is selected as the reference frame of the frame. The picture type selection unit 105a writes and stores the information of the reference frame selected for each P picture in the picture type storage unit 106 (step S322).

As a result, the picture type storage unit 106 stores the information of the finally constructed coded structure. The block output unit 107 temporarily stores the frame group received from the coded target frame selection unit 101 in the internal storage area, and internally in the order based on the coded structure stored in the picture type storage unit 106. Read a frame from the storage area of. The block output unit 107 decomposes each of the read frames into CU blocks in the order of raster scan and outputs them as a coded target block, and the coded structure construction unit 10a ends the processing for one frame group.

(Specific example of the coded structure construction process of the second embodiment)
Next, with reference to FIGS. 14 to 16, an example in which the coded structure construction process of the second embodiment shown in FIGS. 12 and 13 is applied to the frame group shown in FIG. 14 (a). explain. The coded target frame selection unit 101 generates the frame group shown in FIG. 14A from the input video information. The frame group includes scene changes in the frames of frame number 4 and frame number 5.

As shown in FIG. 14B, after the frame of frame number 0 is selected as the I picture by the I picture selection unit 102a, the frame of the frame number (0, 1, 2) is included by the reference group generation unit 103a. The reference group Ref [0] [0] is written to the reference group storage unit 108 (steps S303 and S305). The combination determination unit 104 performs the combination determination process shown in FIG. 4 on the reference groups Ref [0] [0] written in the reference group storage unit 108. Here, since there is no scene change, the combination determination unit 104 It is assumed that the evaluation value determination unit 111 of the above outputs "True" (step S315).

Since the determination result output by the combination determination unit 104 is "True", the picture type selection unit 105a is a B picture in which the picture type of the frame of frame number 1 is the frame of frame number 0 and the frame of frame number 2 is the reference frame. Select as. The picture type selection unit 105 writes in the picture type storage unit 106 information indicating that the frame of frame number 2 is selected as the B picture and information indicating that the frames of frame number 0 and frame number 2 are reference frames. Remember with.

Further, the picture type selection unit 105a selects the picture type of the frame of the frame number 2 as the P picture, and writes and stores the information indicating that the frame of the frame number 2 is selected as the P picture in the picture type storage unit 106. (Step S309).

The reference group generation unit 103a adds 1 to j while the value of j is less than N / 2, and refers to the reference group Ref [0] [1] including the frame of the frame number (2, 3, 4), and the frame number. Reference group Ref [0] [2] including frames of (4,5,6), reference group Ref [0] [3] including frames of frame number (6,7,8), frame number (8,9) , 10) The reference group Ref [0] [4] including the frame is written in the reference group storage unit 108 and stored. The combination determination unit 104 performs the combination determination process shown in FIG. 4 every time a new reference group is written in the reference group storage unit 108. As a result, as shown in FIG. 14B, for the reference group Ref [0] [2] including the scene change, "False" is output as the determination result, and for the reference groups other than the reference groups Ref [0] [2]. Outputs "True".

As a result, the picture type selection unit 105a selects the picture type as shown in FIG. 14 (b) for the frames of frame numbers 1 to 10. That is, each of the reference groups Ref [0] [0], Ref [0] [1], Ref [0] [3], and Ref [0] [4] determined to be "True" by the combination determination unit 104. The B frame of the determination target frame of is selected as a B picture that refers to each of the A and C frames, and the C frame is selected as a P picture. On the other hand, as for the frames of the reference group Ref [0] [2] determined to be "False" by the combination determination unit 104, the B frame and the C frame are selected as P pictures.

As shown in FIG. 13, the reference group generation unit 103a sets i = 1, j = 0 (step S312), and first, the reference group Ref [0] stored in the determination result storage unit 112 of the combination determination unit 104. ] [0] and the determination result of the reference group Ref [0] [1] are referred to. Since the determination results of the reference group Ref [0] [0] and the reference group Ref [0] [1] are both “True” (step S313-YES), the reference group generation unit 103a can use the reference group Ref [0]. 0] The first and last frames of [0] and the last frame of the reference group Ref [0] [1], that is, the frames of frame numbers 0, 2, and 4 are included as α, β, and γ frames of the reference group Ref [ 1] Generate [0].

As shown in FIG. 15C, the reference group generation unit 103a writes the generated reference groups Ref [1] and [0] to the reference group storage unit 108 and stores them (step S314). The combination determination unit 104 performs the combination determination process shown in FIG. 4 on the reference groups Ref [1] [0] written in the reference group storage unit 108, and since there is no scene change here, the evaluation value determination unit It is assumed that 111 outputs "True" (step S315). Since the determination result is "True" (step S316-True), the picture type selection unit 105a sets the picture type of the frame number 2 which is the determination target frame of the reference groups Ref [1] [0] to the frame number 0. And the frame of frame number 4 is selected as the B picture with the frame as the reference frame.

Information indicating that the frame of the frame number 2 is a P picture has already been written in the picture type storage unit 106. Therefore, the picture type selection unit 105a stores information indicating that the frame of frame number 2 is selected as the B picture and information indicating that the frames of frame number 0 and frame number 4 are reference frames. Overwrite and memorize.

Further, the picture type selection unit 105a selects the picture type of the frame number 4 which is the γ frame as the P picture. Information indicating that the frame of frame number 4 is a P picture has already been written in the picture type storage unit 106, but the picture type selection unit 105a has selected the frame of frame number 4 as a P picture. The information indicating that is overwritten in the picture type storage unit 106 and stored (step S317).

The reference group generation unit 103a adds 2 to the value of j to make j = 2 (step S318), and since the value of j is less than N / 2 (N = 10) (step S319-YES), the combination determination is made. The determination results of the reference group Ref [0] [2] and the reference group Ref [0] [3] stored in the determination result storage unit 112 of the unit 104 are referred to. Since the determination result of the reference group Ref [0] [2] is "False", the reference group generation unit 103a determines the determination results of the reference group Ref [0] [2] and the reference group Ref [0] [3]. Is not both "True" (step S313-NO). Therefore, the reference groups Ref [1] [1] shown in FIG. 15 (c) for convenience of explanation are not generated by the reference group generation unit 103a.

The reference group generation unit 103a adds 2 to the value of j to make j = 4 (step S318), and since the value of j is less than N / 2 (N = 10) (step S319-YES), the reference group Attempts are made to refer to the determination results of Ref [0] [4] and the reference group Ref [0] [5]. However, since the reference group Ref [0] [5] does not exist, the reference group generation unit 103a determines that both the reference group Ref [0] [4] and the reference group Ref [0] [5] are ". It is determined that it is not "True" (step S313-NO). The reference group generation unit 103a adds 2 to the value of j to make j = 6 (step S318), and the value of j is not less than N / 2 (N = 10) (step S319-NO). Add "1" to the value to make i = 1 and j = 0 (step S320).

Since the value of i is less than depth_max (depth_max = 3) (step S321-YES), the reference group generation unit 103a has the reference group Ref [1] stored in the determination result storage unit 112 of the combination determination unit 104. Attempts are made to refer to the determination results of [0] and the reference groups Ref [1] [1]. However, as described above, since the reference group Ref [1] [1] has not been generated, the reference group generation unit 103a has the reference group Ref [0] [4] and the reference group Ref [0] [5]. It is determined that both of the determination results are not "True" (step S313-NO). Therefore, the reference groups Ref [2] and [0] shown in FIG. 15 (d) for convenience of explanation are not generated by the reference group generation unit 103a.

After that, the processes of steps S318 and S319 are repeated until j is not less than N / 2, and "1" is added to the value of i in step S320 to make i = 2, j = 0. Since the value of i is less than depth_max (step S321-YES), the processing is continued, but as described above, the reference groups Ref [2] and [0] shown in FIG. 15 (d) are the reference group generation units. Since it is not generated by 103a, the determination in step S313 is "NO". Again, the processing of steps S318 and S319 is repeated until j is no less than N / 2, and "1" is added to the value of i in step S320 to make i = 3, j = 0. Since the value of i is not less than depth_max (step S321-NO), the reference group generation unit 103a outputs instruction information for causing the picture type selection unit 105a to select a reference frame for the frame of the P picture.

Upon receiving the instruction information, the picture type selection unit 105a selects the frame of the past P picture closest to the frame in the display order as the reference frame of the frame for each of the frames selected as the P picture. If there is no past P picture closest to the display order, the frame of the I picture is selected as the reference frame of the frame. The picture type selection unit 105a writes and stores the information of the reference frame selected for each P picture in the picture type storage unit 106 (step S322). As a result, the picture type storage unit 106 stores the coded structure having the hierarchical structure of the two layers of B pictures shown in FIG. 16 (e).

According to the configuration of the second embodiment described above, in the coding structure construction unit 10a, the coding target frame selection unit 101 selects a number of frames according to a predetermined GOP size from the input video information to form a frame group. Generate. The I-picture selection unit 102a selects the first frame of the frame group as the I-picture. The reference group generation unit 103a sets i = 0 and j = 0, and repeats the frames whose frame numbers are "2j", "2j + 2 ⁱ ", and "2j + 2 ^{(i + 1)} " from the frame group while adding "1" to j. A reference group Ref [0] [j] is generated which is selected and includes the three selected frames as A, B, and C frames. The evaluation value calculation unit 110 of the combination determination unit 104 calculates three evaluation values, L0 cost, L1 cost, and Bi cost, based on the A, B, and C frames included in the reference group. The evaluation value determination unit 111 selects the smaller value of the Bi cost and the L1 cost, and determines based on the relative value based on the value of the L0 cost and the predetermined relative evaluation thresholds Th_min and Th_max. It is determined whether or not the reference relationship between the B frame, which is the target frame, and the A and C frames is combined. Further, when the relative value is not more than the value of Th_min and is included in the range not more than the value of Th_max, the evaluation value determination unit 111 further bases on the value of Bi cost and the absolute evaluation threshold Th_abs. , It is determined whether or not the reference relationship between the B frame, which is the determination target frame, and the A and C frames is combined. When the evaluation value determination unit 111 determines that the reference relationship between the B frame and the A and C frames is combined, the picture type selection unit 105a selects the B frame as the B picture with the A and C frames as the reference frames. .. That is, the coded structure construction unit 10a does not perform an operation that actually performs coding, calculates three evaluation values based on three frames, and based on the calculated three evaluation values, 3 It is determined whether or not to combine the reference relationships of two frames. Therefore, in the coded structure construction unit 10a of the second embodiment, it is possible to obtain a coded structure having high coding efficiency with a low calculation amount.

Further, the reference group generation unit 103a combines two adjacent reference groups determined by the evaluation value determination unit 111 to combine the reference relationship between the determination target frame and the first and last frames, and determines the overlapping frame. Repeat the process of creating a new reference group. The coupling determination unit 104 performs the coupling determination processing based on the α, β, and γ frames of the newly generated reference group by the reference group generation unit 103a. As a result, the coded structure construction unit 10a of the second embodiment can construct a coded structure having two or more layers deep, and can improve the coding efficiency. There is.

Further, in the coded structure construction units 10 and 10a of the first and second embodiments, the minimum value of the absolute error sum when the motion search is performed between frames is calculated as the evaluation value, and only the motion vector is calculated. Therefore, it is possible to accurately perform the combination judgment even for a complicated input video. Therefore, the coded structure construction units 10 and 10a of the first and second embodiments can obtain a coded structure having high coding efficiency with a low calculation amount.

In the coded structure construction units 10 and 10a of the first and second embodiments, as described with reference to FIG. 4, before performing the absolute evaluation of the Bi cost using the absolute evaluation threshold value in step S207, In step S204, a relative evaluation is performed based on the L0 cost. Therefore, a robust determination can be realized. On the other hand, in the technique described in Non-Patent Document 1, absolute evaluation is performed using only the minimum value of the coding cost as the evaluation value. Therefore, when the technique of the technique described in Non-Patent Document 1 is applied to the frames shown in FIGS. 9 (ii) and 9 (iii), assuming that the intercost is sufficiently large, the same value of "150" is obtained in both cases. It will be calculated. Therefore, in the technique described in Non-Patent Document 1, unlike the evaluation value determination unit 111 of the first and second embodiments, FIG. 9 (ii) is not combined, and FIG. 9 (iii) is used. It is not possible to make different determinations to combine.

When combining in both FIGS. 9 (ii) and 9 (iii), it is considered effective to combine in FIG. 9 (iii) because there is a merit of referring to the E frame which is the same scene in the M frame. On the other hand, in FIG. 9 (ii), since the scenes of the M frame and the E frame are different, there is a high possibility that the determination will be erroneous. If they are not combined, on the contrary, there is a high possibility that FIG. 9 (iii) is an erroneous determination. As described above, the evaluation value determination unit 111 in the first and second embodiments may make a different determination that the one shown in FIG. 9 (ii) is not combined and the one shown in FIG. 9 (iii) is combined. Therefore, the coding efficiency can be improved in both cases. This is an evaluation that reflects the fact that when focusing on the M frame, the merit of combining is obtained when the correlation with the E frame is high, that is, when the L1 cost or the Bi cost is sufficiently smaller than the L0 cost. Because it is.

Further, in the first and second embodiments described above, the combination determination unit 104 units the determination target frame of the reference group generated by the reference group generation units 103 and 103a and the three frames of the first and last frames. However, the configuration of the present invention is not limited to the embodiment, although it is premised that the combination determination process is performed. For example, it may be applied to the technique described in Non-Patent Document 1. As described above, in the technique described in Non-Patent Document 1, an intermediate frame that refers to the reference frame and the selected P frame is defined between the reference frame and the selected P frame. For the frames located between the reference frame and the intermediate frame, the cost is calculated when these frames are B pictures that refer to the reference frame and the intermediate frame. Further, for the frame located between the intermediate frame and the P picture, the cost is calculated when these frames are B pictures that refer to the reference frame and the intermediate frame. Then, the combination that minimizes the calculated cost is selected as the final coding structure. Instead of these costs, the coupling determination unit 104 may be applied to the technique in Non-Patent Document 1 to construct a coded structure using the three evaluation values calculated by the coupling determination unit 104.

In the first and second embodiments described above, the B frame is selected as the P picture, the C frame is selected as the P picture, and the γ frame is selected as the P picture according to the result of the combination determination. Further, when three frames cannot be selected, the remaining frames are selected as P pictures, but the configuration of the present invention is not limited to the embodiment. For example, frames to be P-pictures may be selected in advance at predetermined intervals from I-pictures, or after all frames to be B-pictures are selected as a result of the combination determination, the I-pictures and B-pictures may be selected. A frame other than the frame may be a P picture.

Further, in the first and second embodiments described above, three consecutive frames are generated as a reference group, but the number may be three or more.

Further, in the first and second embodiments described above, adjacent reference groups are generated so that there is one overlapping frame in the adjacent reference group, but one or more frames are duplicated. Adjacent reference groups may be generated as such. When a plurality of frames overlap, the judgment target frame is selected from any of the overlapping frames.

Further, in the first and second embodiments described above, the reference group generation units 103 and 103a generate a new reference group based on the determination result of the adjacent reference group to construct a coded structure having two or more layers. However, the process of constructing only one layer may be performed.

Further, in the first and second embodiments described above, the reference group generation units 103 and 103a combine past adjacent reference groups in the display order to generate a new reference group. The embodiment of the above is not limited to the embodiment. For example, adjacent reference groups in the future may be combined in display order, or adjacent reference groups in both the past and future may be combined.

Further, in the first and second embodiments described above, the evaluation value determination unit 111 makes a determination based on the relative evaluation thresholds Th_min and Th_max and the absolute evaluation threshold Th_abs. In this determination, the evaluation value determination unit 111 makes a determination based on whether or not it is less than Th_min, whether or not it exceeds Th_max, and whether or not it is less than Th_abs, but whether or not it is less than. The determination as to whether or not the amount is exceeded is an example, and the determination may be made based on whether or not it is Th_min or less, whether or not it is Th_max or more, and whether or not it is Th_abs or less.

The video coding devices 1 and 1a according to the first and second embodiments described above may be realized by a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by a computer system and executed. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, or a storage device such as a hard disk built in a computer system. Further, a "computer-readable recording medium" is a communication line for transmitting a program via a network such as the Internet or a communication line such as a telephone line, and dynamically holds the program for a short period of time. It may also include a program that holds a program for a certain period of time, such as a volatile memory inside a computer system that serves as a server or a client in that case. Further, the above program may be for realizing a part of the above-mentioned functions, and may be further realized for realizing the above-mentioned functions in combination with a program already recorded in the computer system. It may be realized by using a programmable logic device such as FPGA (Field Programmable Gate Array).

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and the design and the like within a range not deviating from the gist of the present invention are also included.

10 ... Coded structure construction unit, 101 ... Coded frame selection unit, 102 ... I picture selection unit, 103 ... Reference group generation unit, 104 ... Combination determination unit, 105 ... Picture type selection unit, 106 ... Picture type storage unit , 107 ... Block output unit, 110 ... Evaluation value calculation unit, 111 ... Evaluation value judgment unit, 112 ... Judgment result storage unit

Claims (8)

A video coding device that performs inter-frame predictive coding.
A reference group generation unit that generates a reference group including a predetermined number of three or more consecutive frames from a frame group acquired from input video information.
Based on the first and last frames of the reference group generated by the reference group generation unit and any one of the determination target frames between the first and last frames, the evaluation value regarding the error between the frames is determined. Evaluation value calculation unit to be calculated and
Of the evaluation values calculated by the evaluation value calculation unit, the evaluation value between the determination target frame and the first frame, the evaluation value between the determination target frame and the last frame, or the determination target frame. determining whether to combine the reference relationship between the first frame and the before and SL determination target frame based on the relative value calculated from the smaller among the evaluation values between the last frame first and last frame and Evaluation value judgment unit and
A picture type selection unit that selects a picture type of the frame included in the frame group based on the determination result of the evaluation value determination unit, and
Equipped with a,
The evaluation value determination unit is a video coding device that determines that the relative values are combined when the relative values are less than the first relative evaluation threshold value . The evaluation value calculation unit
A first one-way evaluation value regarding an error between the determination target frame and the first frame, and a second regarding an error between the determination target frame and the last frame. A one-way evaluation value and a two-way evaluation value regarding an error between the frames to be determined and both the first and last frames are calculated.
The evaluation value determination unit
The smaller evaluation value of the second one-way evaluation value and the two-way evaluation value is selected, and the relative value of the selected evaluation value relative to the first one-way evaluation value is set. A claim for calculating a value and determining whether or not to combine a reference relationship between the determination target frame and the first and last frames based on the calculated relative value and a predetermined relative evaluation threshold value. Item 2. The video coding apparatus according to item 1. The predetermined relative evaluation threshold value further includes a second relative evaluation threshold value having a value larger than that of the first relative evaluation threshold value.
The evaluation value determination section,
Before SL relative value, when the a first value below the second relative evaluation threshold value relative evaluation threshold or more, and the bidirectional evaluation value, based on an absolute evaluation threshold defined in advance, The video coding apparatus according to claim 2, wherein it is determined whether or not the reference relationship between the determination target frame and the first and last frames is combined. The reference group generation unit
From the predetermined number of three or more sheets including the reference group so that the first frame or the last frame of the reference group determined to be combined by the evaluation value determination unit becomes the determination target frame. Generates a new reference group that also contains a large number of consecutive frames,
The evaluation value calculation unit
Based on the first and last frames of the new reference group and the determination target frame, the evaluation value regarding the error between the frames is calculated.
The evaluation value determination unit
Among the evaluation values calculated by the evaluation value calculation unit, the evaluation value between the determination target frame and the first frame of the new reference group, the determination target frame of the new reference group, and the last frame. evaluation value between frames, or the determination of the reference group before SL new based on the relative value calculated from the smaller among the evaluation values between the new reference group determination target frame and the first frame and the last frame determines whether to combine the reference relationship between the target frame beginning and ending frames, if the relative value is less than the first relative evaluation threshold, determines that binds, claims 1 to 3 The video coding apparatus according to any one of the above. The reference group generation unit
When generating another reference group adjacent to the reference group, the three predetermined frames are overlapped so that the last frame of the reference group becomes the first frame of the other adjacent reference group. The reference group including the above number of consecutive frames is generated.
When the evaluation value determination unit determines that the reference relationship between the determination target frame and the first and last frames is combined in both of the two adjacent reference groups, the two adjacent reference groups are combined. The video coding apparatus according to claim 4, wherein the new reference group is generated, and the overlapping frames are set as the determination target frames. The reference group generation unit
According to the hierarchy number of values of the coding structure to build Ru been predetermined et al, repeated to generate the new reference group, the image encoding apparatus according to claim 5. The picture type selection unit
It is provided with an I-picture selection unit that acquires the frame group from the input video information and selects the first frame of the acquired frame group as an I-picture.
The picture type selection unit
When the evaluation value determination unit determines that the reference relationship between the determination target frame and the first and last frames is combined, the determination target frame is selected as a B picture having the first and last frames as reference frames. ,
The I picture or the frame not selected as the B picture is selected as the P picture, and the frame of the past P picture closest to the selected frame in the display order of the frame group is selected. Claims 1 to 6 are the reference frame of the frame, and when there is no frame of the past P picture closest to the selected frame, the frame of the I picture is the reference frame of the selected frame. The video coding apparatus according to any one of the above. The video coding apparatus according to any one of claims 1 to 7, wherein the predetermined number of 3 or more sheets is 3.

Images