JP6159292B2 - Motion vector detection apparatus, motion vector detection method, and motion vector detection program - Google Patents

Description

  The present invention relates to a motion vector detection apparatus, a motion vector detection method, and a motion vector detection program that detect a motion vector when video coding is performed.

  In inter-picture prediction of moving picture coding, a motion vector representing how much an object has moved is detected using a temporally past or future coded frame of the encoding target frame as a reference frame, and the motion vector indicates Compression is performed by taking the difference from the area. H. H.264 / AVC and H.264 In H.265 / HEVC, P picture and B picture are defined as picture types for inter-screen prediction. The P picture uses only one reference frame, and the B picture uses two reference frames.

  The motion vector is detected in units of blocks, and the block most similar to the target block is searched from the reference frame. At this time, if a full search is performed, the amount of calculation becomes enormous, and therefore a search range is generally set. In order to further reduce the amount of calculation, the following method has been proposed.

  H. In the H.264 / AVC standard, there is a motion vector prediction method called a temporal direct mode, which is a technique for scaling a motion vector at the same spatial position to an already encoded frame to obtain a predicted motion vector of a prediction target frame. . By using this predicted motion vector as a starting point for motion search, a low-computation search can be realized without expanding the search range even when the time distance between the target frame and the reference frame is long (for example, non-patent literature) 1).

MPEG-4 AVC (ISO / IEC 14496-10)

  However, if the predicted motion vector is calculated by scaling the motion vectors at the same spatial position as in this method, the continuity of the object is not accurately taken into consideration. FIG. 13 is an explanatory diagram showing problems when the same spatial position is used. As shown in FIG. 13, when different objects are adjacent to each other, the predicted motion vector may refer to another object, and the search accuracy may be reduced due to an incorrect predicted motion vector.

  FIG. 14 is an explanatory diagram showing a problem when occlusion occurs. As shown in FIG. 14, when an occlusion area is generated in the encoded frame, the calculated motion vector is highly likely to have no correlation in the object at the same spatial position in the target frame. Therefore, if it is used as it is for the predicted motion vector of the target frame, the motion vector search accuracy may be reduced. In order to solve these problems, it may be considered not to use the error when the motion search error (such as the absolute error sum) exceeds a certain threshold. However, the threshold varies greatly depending on the type of image and the scene, so there is a problem that unless an appropriate threshold is set, an occlusion area cannot be detected or an area where no occlusion has occurred is excluded. .

  The present invention has been made in view of such circumstances, and a motion vector detection device, a motion vector detection method, and a motion vector detection program capable of improving motion vector detection accuracy while suppressing an increase in the amount of calculation. The purpose is to provide.

  The present invention is a motion vector detection device that detects a motion vector when video coding is performed, and determines whether or not there is a motion vector projected from a previous frame in a prediction block to be processed. When it is determined that there is a motion vector projected by the determination unit and the determination unit, a predicted motion vector is calculated from all the projected motion vectors and set as a motion search start point. If it is determined that there is not, a start point setting unit that uses a zero vector as a start point of motion search, a motion search unit that performs a motion search based on the start point, and based on the motion search result, The evaluation value calculating means for calculating the evaluation value of the inter-screen prediction mode and the evaluation value are compared, and the motion vector of the prediction block that becomes the minimum evaluation value is the next motion search target. Characterized in that it comprises a motion vector projection means for projecting the frame.

  In the present invention, the evaluation value calculation means further calculates an evaluation value of an intra prediction mode of the prediction block, and the motion vector projection means evaluates the evaluation value of the inter prediction mode when the evaluation value of the inter prediction mode is The motion vector is projected onto the next motion search target frame only when the value is less than or equal to the value.

  According to the present invention, the motion search means determines whether or not to refer to a frame whose order is later than the current target frame among a plurality of motion search target frames. Is referred to, a motion search using the original image is performed, and the obtained motion vectors are connected and projected.

  In the present invention, the motion search means uses a motion vector calculated using the original image as a starting point after encoding a reference frame in which a motion search using the original image is encoded. A feature is that the motion search is performed again in a search range smaller than the used search range.

  In the present invention, it is determined that the motion vector is not projected from the previous frame in the prediction target block by the determination means, and the motion vector having the shortest distance to the prediction target block is projected around the motion vector. Further, an interpolation means for performing interpolation is provided.

  The present invention is a motion vector detection method for detecting a motion vector when video coding is performed, and determines whether or not there is a motion vector projected from a previous frame in a prediction block to be processed. When it is determined that there is a motion vector projected by the determination step and the determination step, a predicted motion vector is calculated from all the projected motion vectors and set as a motion search start point. If it is determined that there is not, a start point setting step using a zero vector as a start point for motion search, a motion search step for performing motion search based on the start point, and based on the motion search result, An evaluation value calculation step for calculating an evaluation value of the inter-screen prediction mode, and the motion vector of the prediction block that is the smallest evaluation value by comparing the evaluation value The characterized by having a motion vector projection step of projecting to the next motion search target frame.

  The present invention is a motion vector detection program for causing a computer to function as the motion vector detection device.

  According to the present invention, it is possible to improve the motion vector detection accuracy while suppressing an increase in the calculation amount.

It is a block diagram which shows the structure of the video coding apparatus in 1st Embodiment of this invention. It is a block diagram which shows the structure of the prediction motion vector production | generation part 12 shown in FIG. It is a flowchart which shows operation | movement of the prediction motion vector production | generation part 12 shown in FIG. It is a figure which shows the processing order of a motion search. It is a figure which shows the production | generation method of the prediction motion vector for utilizing with the next motion search object frame in step S10. It is a block diagram which shows the structure of the prediction motion vector production | generation part 12 by 2nd Embodiment. It is a flowchart which shows operation | movement of the prediction motion vector production | generation part 12 shown in FIG. It is a block diagram which shows the structure of the motion vector predictor production | generation part 12 by 3rd Embodiment. It is a flowchart which shows operation | movement of the motion vector predictor production | generation part 12 shown in FIG. It is a figure showing the mode of the production | generation of a prediction motion vector. It is a block diagram which shows the structure of the prediction motion vector production | generation part 12 by 5th Embodiment. It is a flowchart which shows operation | movement of the prediction motion vector production | generation part 12 shown in FIG. It is explanatory drawing which shows the problem in the case of utilizing the same space position. It is explanatory drawing which shows a problem when an occlusion generate | occur | produces.

<First Embodiment>
Hereinafter, a motion vector detection device according to a first embodiment of the present invention will be described with reference to the drawings. In the following description, the target frame indicates a frame that is the next target for motion search (a frame on which a motion vector is projected), and the motion search target frame indicates a frame that is currently performing a motion search. In the following description, HEVC will be described as an example, but the present invention is not limited to HEVC. Although the case where the highest layer interval (GOP) of the reference structure is 4 will be described, the present invention is not limited to this.

  FIG. 1 is a block diagram showing a configuration of a video encoding apparatus in the embodiment. The motion vector detection apparatus according to the present invention is used in a video encoding apparatus as shown in FIG. The video encoding device 100 includes a subtraction unit 1, an orthogonal transform / quantization unit 2, a variable length encoding unit 3, a mode determination unit 4, an inverse quantization / inverse orthogonal transform unit 5, and an addition unit 6. , An intra / inter switch 7, an intra prediction unit 8, an inter prediction unit 9, a loop filter unit 10, a decoded picture memory 11, a predicted motion vector generation unit 12, and a motion vector memory 13.

  The subtraction unit 1 calculates the difference between the prediction signal output from the intra prediction unit 8 or the inter prediction unit 9 for each prediction size of the encoding target block of the input picture that is a picture constituting the input video, Output to the orthogonal transform / quantization unit 2. The orthogonal transformation / quantization unit 2 performs orthogonal transformation and quantization on the difference and outputs the result. The variable length encoding unit 3 converts the output of the orthogonal transform / quantization unit 2 into encoded data.

  The intra / inter switch 7 performs switching according to the prediction mode of each encoding target block. The inverse quantization / inverse orthogonal transform unit 5 returns the block signal output from the orthogonal transform / quantization unit 2 to a differential signal including an error. The adding unit 6 converts the prediction signal output from the intra prediction unit 8 or the inter prediction unit 9 and the difference signal into a restored signal. The intra prediction unit 8 inputs this restored signal and uses it for intra prediction of the subsequent block.

  The output of the adding unit 6 is output to the loop filter unit 10 after one picture is completed. The loop filter unit 10 applies a loop filter. This output is stored in the decoded picture memory 11. The reference frame stored in the decoded picture memory 11 is input to the inter prediction unit 9 and used for inter prediction of the subsequent picture.

  The inter prediction unit 9 inputs an input video and a reference frame from the decoded picture memory 11 for each prediction block size, and performs motion search and motion compensation, or a motion vector when a motion vector has already been calculated. A motion vector is called from the memory 13 to perform motion compensation, and a prediction signal is output to the intra / inter switch 7. The motion vector memory 13 stores the motion vector of each prediction block size from the inter prediction unit 9 and outputs the motion vector to the prediction motion vector generation unit 12.

  The predicted motion vector generation unit 12 generates a predicted motion vector and outputs it to the inter prediction unit 9. The mode determination unit 4 receives the input video, the restoration signal, and the code amount from the variable length coding unit 3 as input, calculates an evaluation value, and outputs the predicted block size having the smallest evaluation value to the motion vector predictor generation unit 12. .

  Next, the configuration of the motion vector predictor generator 12 shown in FIG. 1 will be described with reference to FIG. FIG. 2 is a block diagram showing a configuration of the motion vector predictor generator 12 shown in FIG. The predicted motion vector generation unit 12 receives the block size information and the motion vector of the previous frame, projects a motion vector, and predicts a motion vector from the output of the motion vector projection unit 121. And a vector calculation unit 122.

  Next, the operation of the motion vector predictor generator 12 shown in FIG. 2 will be described with reference to FIG. FIG. 3 is a flowchart showing the operation of the motion vector predictor generator 12 shown in FIG. First, motion search is performed in order from the closest to the reference frame for a plurality of frames that refer to the same reference frame as follows in units of GOP (step S1). FIG. 4 is a diagram illustrating a processing order of motion search. In this example, the forward motion search is sequentially performed from F2 to F5 referring to the same reference frame F1. If there is a picture that does not refer to F1, that picture is removed from the motion search order.

  Next, the coding block of the motion search target frame is set as a processing target in the raster scan order (step S2), and a prediction block in the coding block is set as a processing target (step S3). The predicted motion vector calculation unit 122 determines whether or not there is a motion vector projected from the previous frame in the prediction block to be processed (step S4). If there is a projected motion vector, the predicted motion vector is projected. A predicted motion vector is calculated from all the motion vectors that have been set, and is set as a motion search start point (predicted motion vector) (step S5). For the calculation of the predicted motion vector, an average, a weighted average based on an absolute error sum during motion search, a median, or the like is used.

On the other hand, when no motion vector is projected, the motion vector predictor calculation unit 122 sets the zero vector as a start point (step S6). Then, the predicted motion vector calculation unit 122 performs a motion search based on the start point, and stores the calculated motion vector in the motion vector memory 13 (step S7). Next, the motion vector predictor calculation unit 122 calculates the evaluation value of the inter prediction mode of the target prediction block size (step S8). For the evaluation value, for example, the RD cost obtained by the following formula is used.
RD cost = D + λR
Here, D is an error between the target prediction block and the reference block, R is a code amount, and λ is a Lagrangian constant. When the evaluation values in all prediction blocks are calculated, the evaluation values are compared (step S9). Then, the motion vector projecting unit 121 projects the motion vector of the prediction block having the minimum evaluation value onto the next motion search target frame (step S10).

FIG. 5 is a diagram showing a predicted motion vector generation method for use in the next motion search target frame in step S10. In this example, after a motion search for F2 is performed, a predicted motion vector to be used in the motion search for F3 is generated. The predicted motion vector is calculated by scaling according to the time distance. For example, the predicted motion vector mvp3 = (mvp3x, mvp3y) of F3 is calculated by the following equation using the motion vector mv2 = (mv2x, mv2y) calculated in F2 and the picture display order number (T).
mvp3 = (T3-T1) / (T2-T1) × mv2

The predicted motion vector is projected so as to be extended to the next motion search target frame. For example, when the motion vector obtained in F2 is projected onto F3, assuming that the upper left coordinate position of the motion vector calculated in F2 is (x2, y2), the predicted block size is based on the projected F3 position (x3, y3). Project in pixels on the minute area.
(X3, y3) = (x2 + mv3x0−mv2x, y2 + mv3y0−mv2y)

  These are carried out until all the coding blocks of the motion search target frame are completed. These procedures are performed until the motion search of F5. When the search up to F5 is completed, encoding of F5 is performed. Thereafter, the backward motion search is performed. As shown in FIG. 4, the motion search is performed in the same order as in the forward direction in the order of F4 to F1, with F5 as a reference frame. When encoding, normal motion search is performed only on reference frames that have not been obtained, and encoding is performed on reference frames that have already been obtained using motion vectors stored in the motion vector memory. .

  Also, the calculated evaluation value of the inter-screen prediction mode is stored and reused at the time of encoding. Furthermore, the motion search for the reference frame that has not been obtained may be performed only with the optimal prediction block size determined previously, or the prediction block size other than the optimal prediction block size may be excluded to reduce the calculation amount.

  In this way, motion search of a plurality of frames that refer to the same reference frame is performed in order of time closest to the reference frame, and the calculated motion vector is projected onto the next motion search target frame so as to be extended. A motion search is performed as the start point of the motion search in the projected area. At this time, the evaluation value of the inter-screen prediction mode is calculated, and only the prediction block size that minimizes the evaluation value is set as a projection target.

  According to this configuration, by performing a motion search of a plurality of frames that refer to the same reference frame in the order close to the reference frame, projecting the motion vector obtained by the motion search onto a frame that is a target of the next motion search, Since the predicted motion vector is generated in consideration of the motion of the object, the search accuracy is improved. At this time, by calculating the evaluation values of the plurality of inter-screen prediction modes and comparing the evaluation values, it is possible to determine the optimal prediction block size that best reflects the motion of the object in the inter-screen prediction. The prediction motion vector accuracy is improved by using only the motion vector having the optimal prediction block size as the projection target.

Second Embodiment
Next, a motion vector detection device according to a second embodiment of the present invention will be described. FIG. 6 is a block diagram showing the configuration of the motion vector predictor generator 12 according to the second embodiment. In this figure, the same parts as those shown in FIG. 2 are denoted by the same reference numerals, and the description thereof is omitted. The difference between the predicted motion vector generation unit 12 shown in this figure and the predicted motion vector generation unit 12 shown in FIG. 2 is that an intra / inter determination unit 123 is newly provided.

  Next, the operation of the motion vector predictor generator 12 shown in FIG. 6 will be described with reference to FIG. FIG. 7 is a flowchart showing the operation of the motion vector predictor generator 12 shown in FIG. In FIG. 7, the same parts as those in the processing operation shown in FIG. The processing operation shown in this figure is different from the processing operation shown in FIG. 3 in that steps S11 and S12 are newly added.

  First, motion search is performed in order from the closest to the reference frame for a plurality of frames that refer to the same reference frame as follows in units of GOP (step S1). Subsequently, the coding block of the motion search target frame is set as a processing target in the raster scan order (step S2), and a prediction block in the coding block is set as a processing target (step S3). The predicted motion vector calculation unit 122 determines whether or not there is a motion vector projected from the previous frame in the prediction block to be processed (step S4). If there is a projected motion vector, the predicted motion vector is projected. A predicted motion vector is calculated from all the motion vectors that have been set, and is set as a motion search start point (predicted motion vector) (step S5). For the calculation of the predicted motion vector, an average, a weighted average based on an absolute error sum during motion search, a median, or the like is used.

On the other hand, when no motion vector is projected, the motion vector predictor calculation unit 122 sets the zero vector as a start point (step S6). Then, the predicted motion vector calculation unit 122 performs a motion search based on the start point, and stores the calculated motion vector in the motion vector memory 13 (step S7). Next, the motion vector predictor calculation unit 122 calculates the evaluation value of the inter prediction mode of the target prediction block size (step S8). For the evaluation value, for example, the RD cost obtained by the following formula is used.
RD cost = D + λR
Here, D is an error between the target prediction block and the reference block, R is a code amount, and λ is a Lagrangian constant.

  Next, the motion vector predictor calculation unit 122 calculates not only the evaluation value of the inter prediction mode of the target prediction block size but also the evaluation value of the intra prediction mode (step S11). The evaluation value is calculated using an original image or a restored image predicted by intra prediction.

  Next, the predicted motion vector calculation unit 122 compares the evaluation values when the evaluation values in all the prediction blocks are calculated (step S9). The intra / inter determination unit 123 compares the evaluation values of the inter-screen prediction mode and the intra-screen prediction mode (step S12), and only when the evaluation value of the inter-screen prediction mode is equal to or less than the evaluation value of the intra-screen prediction mode. The motion vector is projected onto the next motion search target frame (step S10). The evaluation value of the intra prediction mode calculated here is also saved and reused when encoding later. Further, the calculation amount may be reduced by not calculating the bi-directional prediction mode including Skip / Merge for the block whose evaluation value in the inter-screen prediction mode is smaller than the evaluation value in the intra-screen prediction mode.

  As described above, when the evaluation value of the intra prediction mode is calculated at the time of motion search and compared with the inter prediction mode, if the evaluation value of the intra prediction mode is small, the motion vector of the area is excluded from the projection target. To do.

  According to this configuration, by calculating the evaluation value of the intra prediction and comparing it with the inter prediction, it is possible to relatively determine the block area where the intra prediction is selected, thereby eliminating the threshold setting problem. In addition, the motion vector of the in-screen region that causes the estimation error such as the occlusion region is excluded, and the prediction motion vector accuracy is further improved.

<Third Embodiment>
Next, a motion vector detection device according to a third embodiment of the present invention will be described. FIG. 8 is a block diagram showing a configuration of the motion vector predictor generator 12 according to the third embodiment. In this figure, the same parts as those shown in FIG. 6 are denoted by the same reference numerals, and the description thereof is omitted. The difference between the predicted motion vector generating unit 12 shown in this figure and the predicted motion vector generating unit 12 shown in FIG. 6 is that a motion vector connecting unit 124 is newly provided. 8 is a diagram illustrating an example in which a motion vector coupling unit 124 is added to FIG. 6, but a configuration in which a motion vector coupling unit 124 is added to FIG. 2 may be used.

  Next, the operation of the motion vector predictor generator 12 shown in FIG. 8 will be described with reference to FIG. FIG. 9 is a flowchart showing the operation of the motion vector predictor generator 12 shown in FIG. In FIG. 9, the same parts as those in the processing operation shown in FIG. The processing operation shown in this figure is different from the processing operation shown in FIG. 7 in that steps S13 and S14 are newly added. 9 is a diagram illustrating an example in which steps S13 and S14 are added to FIG. 7, but a processing operation in which steps S13 and S14 are added to FIG. 3 may be used.

  First, motion search is performed in order from the closest to the reference frame for a plurality of frames that refer to the same reference frame as follows in units of GOP (step S1). Subsequently, the coding block of the motion search target frame is set as a processing target in the raster scan order (step S2), and a prediction block in the coding block is set as a processing target (step S3). The predicted motion vector calculation unit 122 determines whether or not there is a motion vector projected from the previous frame in the prediction block to be processed (step S4). If there is a projected motion vector, the predicted motion vector is projected. A predicted motion vector is calculated from all the motion vectors that have been set, and is set as a motion search start point (predicted motion vector) (step S5). For the calculation of the predicted motion vector, an average, a weighted average based on an absolute error sum during motion search, a median, or the like is used.

  On the other hand, when no motion vector is projected, the motion vector predictor calculation unit 122 sets the zero vector as a start point (step S6). Then, the predicted motion vector calculation unit 122 performs a motion search based on the start point, and stores the calculated motion vector in the motion vector memory 13 (step S7).

Next, the motion vector connecting unit 124 determines whether to refer to a frame whose order is later than the current target frame among the plurality of motion search target frames (step S13). As a result of this determination, when the frame of the target frame refers to a later frame, the motion vector concatenation unit 124 performs a motion search using the original image and stores it in the motion vector memory 13 (step S14). For example, in the example shown in FIG. 4, since F2 refers to F3, a motion search is performed using the original image for F3 during the motion search of F2. At this time, the predicted motion vector mvp3 to be projected is obtained by the following expression when the motion vector for F1 of F2 is mv2_F1 = (mv2x_F1 mv2y_F1) and the motion vector for F3 is mv2_F3 = (mv2x_F3, mv2y_F3).
mvp3 = mv2_F1-mv2_F3

Also, the position to project is
(X3, y3) = (x2-mv2x_F3, y2-mv2y_F3)
It becomes. FIG. 10 is a diagram illustrating how a predicted motion vector is generated. At this time, when calculating the evaluation value of the inter-screen prediction mode, it is calculated in one of the forward direction, the backward direction, and the bidirectional direction. Further, the backward motion search does not follow the motion search order (backward direction) in FIG. 3, but is performed in the same order as the forward motion search order, and is encoded by calling from the motion vector memory during encoding.

Next, the motion vector predictor calculation unit 122 calculates an evaluation value of each prediction block (step S8). For the evaluation value, for example, the RD cost obtained by the following formula is used.
RD cost = D + λR
Here, D is an error between the target prediction block and the reference block, R is a code amount, and λ is a Lagrangian constant.

  Next, the motion vector predictor calculation unit 122 calculates not only the evaluation value of the inter prediction mode but also the evaluation value of the intra prediction mode for each prediction block size (step S11). The evaluation value is calculated using an original image or a restored image predicted by intra prediction.

  Next, the predicted motion vector calculation unit 122 compares the evaluation values when the evaluation values in all the prediction blocks are calculated (step S9). Then, the intra / inter determination unit 123 compares the evaluation values of the inter-screen prediction mode and the intra-screen prediction mode (step S12), and calculates a motion vector only when the evaluation value of the inter-screen prediction mode is equal to or lower than the intra-screen prediction mode. Projecting to the next motion search target frame (step S10). The evaluation value of the intra prediction mode calculated here is also saved and reused when encoding later. Further, the calculation amount may be reduced by not calculating the bi-directional prediction mode including Skip / Merge for the block whose evaluation value in the inter-screen prediction mode is smaller than the evaluation value in the intra-screen prediction mode.

  As described above, when there is a reference frame of a motion search target frame in a plurality of frames (the order is later than the current motion search target frame) referring to the same reference frame, a motion search is performed using the original image. Then, a combination of the calculated first motion vector and the motion vector in the second direction calculated using the original image is projected.

  According to this configuration, when a frame whose motion search order is later in a plurality of frames that refer to the same reference frame is used as a reference frame, a motion search is performed using an original image (not a restored image) Concatenate and project vectors. By doing so, it is possible to generate a highly accurate predicted motion vector even when the motion of the object is not continuous.

<Fourth embodiment>
Next, a motion vector detection device according to a fourth embodiment of the present invention will be described. The motion vector detection device according to the fourth embodiment differs from the third embodiment in the following points. In the third embodiment, the motion vector in the backward direction is retrieved from the motion vector memory and used for the encoding when the search is performed using the original image, but the reference frame is encoded in the fourth embodiment. Then, the motion vector searched using the original image after the restoration is used as a starting point, and the search range smaller than the search range when the original image is used again is redone.

  As described above, after the reference frame that has been subjected to motion search using the original image is encoded, the search range that is smaller than the search range that was used for the original image with the second motion vector calculated using the original image as the starting point. Repeat the search again.

  According to this configuration, after the reference frame on which the motion search is performed using the original image is encoded, the motion search is performed again using the restored image. At this time, the search is performed in a search range smaller than the search range used in the original image, using the motion vector when the original image is used as a search start point. In the third embodiment, since the search is performed using the original image, if the motion vector is used for encoding as it is, the encoding efficiency is reduced as compared with the case of searching using the original restored image. In order to avoid this, it is necessary to redo the motion search again, but if it is simply redone, the amount of computation will increase accordingly. Therefore, by re-starting the motion search in the search range smaller than the search range used in the original image using the vector calculated in the original image as a starting point, the calculation result when using the original image is not wasted and the amount of calculation is small. The motion vector can be obtained again.

<Fifth Embodiment>
Next, a motion vector detection device according to a fifth embodiment of the present invention will be described. FIG. 11 is a block diagram illustrating the configuration of the motion vector predictor generator 12 according to the fifth embodiment. In this figure, the same parts as those shown in FIG. 8 are denoted by the same reference numerals, and the description thereof is omitted. The difference between the predicted motion vector generation unit 12 shown in this figure and the predicted motion vector generation unit 12 shown in FIG. 8 is that a motion vector interpolation unit 125 is newly provided. 11 is a diagram illustrating an example in which a motion vector interpolation unit 125 is added to FIG. 8, but a configuration in which a motion vector interpolation unit 125 is added to FIGS. 2 and 6 may be used.

  Next, the operation of the motion vector predictor generator 12 shown in FIG. 11 will be described with reference to FIG. FIG. 12 is a flowchart showing the operation of the motion vector predictor generator 12 shown in FIG. In FIG. 12, the same reference numerals are given to the same portions as the processing operation shown in FIG. The processing operation shown in this figure is different from the processing operation shown in FIG. 9 in that steps S15 and S16 are newly added. FIG. 12 is a diagram illustrating an example in which steps S15 and S16 are added to FIG. 11, but a processing operation in which steps S15 and S16 are added to FIGS.

  First, motion search is performed in order from the closest to the reference frame for a plurality of frames that refer to the same reference frame as follows in units of GOP (step S1). Subsequently, the coding block of the motion search target frame is set as a processing target in the raster scan order (step S2), and a prediction block in the coding block is set as a processing target (step S3). The predicted motion vector calculation unit 122 determines whether there is a motion vector projected from the previous frame in the predicted block to be processed (step S4).

  When a motion vector is not projected from the previous frame in the prediction target block at the time of motion search and a motion vector is projected around (step S15), the motion vector interpolation unit 125 applies the most to the prediction target block. Interpolation is performed with a motion vector having a small distance (step S16). When there are a plurality of motion vectors having the shortest distance, interpolation is performed using an average of them, a weighted average, a median, or the like. A zero vector is set when none is projected around.

  On the other hand, if there is a projected motion vector, a predicted motion vector is calculated from all the projected motion vectors, and is set as a motion search start point (predicted motion vector) (step S5). For the calculation of the predicted motion vector, an average, a weighted average based on an absolute error sum during motion search, a median, or the like is used.

  Next, when no motion vector is projected, the predicted motion vector calculation unit 122 sets the zero vector as a start point (step S6). Then, the predicted motion vector calculation unit 122 performs a motion search based on the start point, and stores the calculated motion vector in the motion vector memory 13 (step S7).

Next, the motion vector concatenation unit 124 determines whether to refer to a frame whose order is later than the current target frame among the plurality of motion search target frames (step S13). When referring to a later frame, the motion vector connecting unit 124 performs a motion search using the original image and stores the motion search in the motion vector memory 13 (step S14). For example, in the example shown in FIG. 4, since F2 refers to F3, a motion search is performed using the original image for F3 during the motion search of F2. At this time, the predicted motion vector mvp3 to be projected is obtained by the following expression when the motion vector for F1 of F2 is mv2_F1 = (mv2x_F1 mv2y_F1) and the motion vector for F3 is mv2_F3 = (mv2x_F3, mv2y_F3).
mvp3 = mv2_F1-mv2_F3

Also, the position to project is
(X3, y3) = (x2-mv2x_F3, y2-mv2y_F3)
It becomes. FIG. 10 is a diagram illustrating how a predicted motion vector is generated. At this time, when calculating the evaluation value of the inter-screen prediction mode, it is calculated in one of the forward direction, the backward direction, and the bidirectional direction. Further, the backward motion search does not follow the motion search order (backward direction) in FIG. 3, but is performed in the same order as the forward motion search order, and is encoded by calling from the motion vector memory during encoding.

Next, the motion vector predictor calculation unit 122 calculates an evaluation value of each prediction block (step S8). For the evaluation value, for example, the RD cost obtained by the following formula is used.
RD cost = D + λR
Here, D is an error between the target prediction block and the reference block, R is a code amount, and λ is a Lagrangian constant.

  Next, the motion vector predictor calculation unit 122 calculates not only the evaluation value of the inter prediction mode but also the evaluation value of the intra prediction mode for each prediction block size (step S11). The evaluation value is calculated using an original image or a restored image predicted by intra prediction.

  Next, the predicted motion vector calculation unit 122 compares the evaluation values when the evaluation values in all the prediction blocks are calculated (step S9). Then, the intra / inter determination unit 123 compares the evaluation values of the inter-screen prediction mode and the intra-screen prediction mode (step S12), and calculates a motion vector only when the evaluation value of the inter-screen prediction mode is equal to or lower than the intra-screen prediction mode. Projecting to the next motion search target frame (step S10). The evaluation value of the intra prediction mode calculated here is also saved and reused when encoding later. Further, the calculation amount may be reduced by not calculating the bi-directional prediction mode including Skip / Merge for the block whose evaluation value in the inter-screen prediction mode is smaller than the evaluation value in the intra-screen prediction mode.

  In this manner, the region where the motion vector is not projected is interpolated from the surrounding region where the motion vector is projected and which is the shortest distance.

  According to this configuration, by interpolating the unprojected area from the nearest surrounding area being projected, it is possible to perform a search using the predicted motion vector even in the unprojected area, with high accuracy. The motion vector can be calculated with a small amount of calculation.

  You may make it implement | achieve the prediction motion vector production | generation part 12 in embodiment mentioned above with a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside a computer system serving as a server or a client in that case may be included and a program held for a certain period of time. Further, the program may be for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in the computer system. It may be realized using hardware such as PLD (Programmable Logic Device) or FPGA (Field Programmable Gate Array).

  As mentioned above, although embodiment of this invention has been described with reference to drawings, the said embodiment is only the illustration of this invention, and it is clear that this invention is not limited to the said embodiment. is there. Therefore, additions, omissions, substitutions, and other modifications of the components may be made without departing from the technical idea and scope of the present invention.

  It can be applied to applications in which it is essential to improve the detection accuracy of motion vectors.

  DESCRIPTION OF SYMBOLS 1 ... Subtraction part, 2 ... Orthogonal transformation / quantization part, 3 ... Variable length encoding part, 4 ... Mode determination part, 5 ... Dequantization / inverse orthogonal transformation part, 6 ... Adder, 7 ... Intra / inter switch, 8 ... Intra prediction unit, 9 ... Inter prediction unit, 10 ... Loop filter unit, 11 ... Decoded picture memory, 12. ..Predicted motion vector generation unit, 13 ... motion vector memory

Claims (7)

A motion vector detection device that detects a motion vector when performing video encoding,
Determining means for determining whether or not there is a motion vector projected from the previous frame in the prediction block to be processed;
When it is determined by the determination means that there is a projected motion vector, a predicted motion vector is calculated from all the projected motion vectors and set as a motion search start point, and it is determined that there is no motion vector. A starting point setting means having a zero vector as a starting point for motion search,
Motion search means for performing a motion search based on the start point;
An evaluation value calculating means for calculating an evaluation value of an inter-screen prediction mode of the prediction block based on the motion search result;
A motion vector detecting device comprising: a motion vector projecting unit that compares the evaluation values and projects the motion vector of the prediction block having the minimum evaluation value onto a next motion search target frame. The evaluation value calculation means further calculates an evaluation value of the intra prediction mode of the prediction block,
The motion vector projecting unit projects the motion vector onto a next motion search target frame only when the evaluation value of the inter-screen prediction mode is equal to or lower than the evaluation value of the intra-screen prediction mode. The motion vector detection device described in 1.   The motion search means determines whether or not to refer to a frame whose order is later than the current target frame among a plurality of motion search target frames, and when the order of the motion search target frames refers to a later frame The motion vector detection apparatus according to claim 1, wherein a motion search using the original image is performed, and the obtained motion vectors are connected and projected.   The motion search means uses a search range used for the original image with a motion vector calculated using the original image as a starting point after a reference frame subjected to motion search using the original image is encoded. The motion vector detection apparatus according to claim 3, wherein the motion search is performed again within a smaller search range.   Interpolation that interpolates with the motion vector having the shortest distance to the prediction target block when it is determined by the determination means that the motion vector is not projected from the previous frame in the prediction target block and the surrounding motion vector is projected The motion vector detection device according to claim 1, further comprising means. A motion vector detection method for detecting a motion vector when performing video encoding,
A determination step of determining whether or not there is a motion vector projected from the previous frame in the prediction block to be processed;
If it is determined that there is a motion vector projected by the determination step, a predicted motion vector is calculated from all the projected motion vectors and set as a motion search start point, and it is determined that there is no motion vector. A starting point setting step using a zero vector as a starting point for motion search,
A motion search step for performing a motion search based on the starting point;
An evaluation value calculating step for calculating an evaluation value of an inter-screen prediction mode of the prediction block based on the motion search result;
A motion vector detection method comprising: comparing the evaluation values and projecting the motion vector of the prediction block having the minimum evaluation value onto a next motion search target frame.   A motion vector detection program for causing a computer to function as the motion vector detection device according to any one of claims 1 to 5.

Images