CN102413270B - Method and device for correcting boundary region motion vector - Google Patents

Abstract

The present invention belongs to the frame rate conversion technology in the field of digital video processing. It provides a method for correcting the motion vector of the boundary area, which includes the following steps: S1, determining the boundary offset block and the boundary block of the video frame; S2, converting the boundary The motion vector of the offset block is assigned to the boundary block such that the boundary block points in the same direction as the motion vector of the boundary offset block. The present invention assigns the credible motion vector of the boundary offset block to the boundary block, so that the boundary block points to the same direction as the motion vector of the boundary offset block, that is, if the boundary block and the boundary offset block belong to the same object, then After such processing, their motion vectors can be kept consistent, thereby improving the image quality of the boundary area of the video frame.

Description

Method and device for correcting boundary region motion vector

technical field

The invention relates to frame rate conversion technology in the field of digital video processing, in particular to a method and device for correcting motion vectors in boundary regions based on global motion vectors.

Background technique

For hardware-implemented frame rate conversion, motion estimation and motion compensation techniques are usually used. Motion estimation is mainly to find the best matching block in the current frame and the previous or next frame. The best matching simple metric is the one with the smallest numerical difference between them. After the best matching block is found, the displacement between it and the current block is recorded as a motion vector. This motion vector is later used in the motion compensation process to obtain the best matching block.

The goal of frame rate conversion is to generate intermediate frames between previous and previous frames. Therefore, the motion vectors are adjusted according to the position of the previous frame and the intermediate frame. The block pointed by the adjusted motion vector is taken as the current block of the intermediate frame. However, motion estimation does not perform well if there is motion at the boundaries of video frames, since objects appearing in the current frame may not have entered the previous frame. Similarly, an object that was present in the previous frame may not be present in the next frame. Therefore, motion estimation may find another matching block in the previous frame, as long as it has the smallest numerical difference, but this block does not actually belong to this object.

Existing implementations do not do special handling of video frame boundaries.

Contents of the invention

In order to solve the above-mentioned technical problems, the object of the present invention is to provide a method for correcting the motion vector of the boundary area based on the global motion vector, by assigning the credible motion vector of the boundary offset block to the boundary block, so that the boundary block points to and borders Offsets the motion vectors of the blocks in the same direction, thus improving image quality in border regions of the video frame.

Corresponding to the method, the object of the present invention is to provide a device for applying the above method.

In order to achieve one of the above-mentioned purposes of the invention, a method for correcting a motion vector in a boundary area of the present invention includes the following steps:

S1. Determine the boundary offset block and the boundary block of the video frame;

S2. Assign the motion vector of the boundary offset block to the boundary block, so that the boundary block points to the same direction as the motion vector of the boundary offset block.

As a further improvement of the present invention, the step S1 specifically includes the following steps:

S11. Developing a global motion vector for each frame of the video frame;

S12. Split the global motion vector to obtain a first motion vector and a second motion vector;

S13. Convert the motion vector of the video frame into a block index according to the size of the block, wherein the first motion vector corresponds to the first block index, and the second motion vector corresponds to the second block index;

S14. Determine the distribution of the boundary offset block and the boundary block according to the block index.

As a further improvement of the present invention, the step S14 specifically includes:

Obtain the block whose block index is equal to the first block index on the left side of the video frame and the block whose block index is equal to the second block index on the right side of the video frame to form a ring-shaped boundary offset block area;

A block whose block index is outside the boundary offset block area is defined as a boundary block.

As a further improvement of the present invention, the method of "assigning the motion vector of the boundary offset block to the boundary block" in the step S2 includes direct copying of the motion vector, or offset, or stretching, or the above-mentioned methods any combination of .

As a further improvement of the present invention, the step of "assigning the motion vector of the boundary offset block to the boundary block" further includes:

The motion vector is transmitted by the formula MV2=A*MV1+B, where MV1 is the motion vector of the boundary offset block, MV2 is the motion vector of the boundary block, and A and B are configurable parameters.

In order to achieve the purpose of the above another invention, the present invention also provides a device for correcting the motion vector of the boundary area, which includes the following units:

A boundary block acquisition unit, used to determine a boundary offset block and a boundary block of a video frame;

A motion vector transmitting unit is configured to assign the motion vector of the boundary offset block to the boundary block, so that the boundary block points to the same direction as the motion vector of the boundary offset block.

As a further improvement of the present invention, the boundary block acquisition unit specifically includes the following units:

A global motion vector acquisition unit, configured to develop a global motion vector for each frame of the video frame;

a global motion vector splitting unit, configured to split the global motion vector to obtain a first motion vector and a second motion vector;

The conversion unit is used to convert the motion vector of the video frame into a block index according to the size of the block, wherein the first motion vector corresponds to the first block index, and the second motion vector corresponds to the second block index;

The boundary obtaining unit is configured to determine the distribution of the boundary offset block and the boundary block according to the block index.

As a further improvement of the present invention, the boundary learning unit is specifically used for:

Obtain the block whose block index is equal to the first block index on the left side of the video frame and the block whose block index is equal to the second block index on the right side of the video frame to form a ring-shaped boundary offset block area;

A block whose block index is outside the boundary offset block area is defined as a boundary block.

As a further improvement of the present invention, the way of "assigning the motion vector of the boundary offset block to the boundary block" in the motion vector transfer unit includes direct copying of the motion vector, or offset, or stretching, or Any combination of the above methods.

As a further improvement of the present invention, the motion vector transfer unit is also used for:

The motion vector is transmitted by the formula MV2=A*MV1+B, where MV1 is the motion vector of the boundary offset block, MV2 is the motion vector of the boundary block, and A and B are configurable parameters.

Compared with the prior art, the beneficial effects of the present invention are: the present invention assigns the credible motion vector of the boundary offset block to the boundary block, so that the boundary block points to the same direction as the motion vector of the boundary offset block, that is If the boundary block and the boundary offset block belong to the same object, after such processing, their motion vectors can be kept consistent, thereby improving the image quality of the boundary area of the video frame.

Description of drawings

FIG. 1 is a flow chart of a method for correcting motion vectors in boundary regions in an embodiment of the present invention;

Fig. 2 is a schematic diagram of the splitting principle of the global motion vector in an embodiment of the present invention;

Fig. 3 is a schematic diagram of the principle of transferring the motion vector of the boundary offset block to the boundary block in an embodiment of the present invention;

FIG. 4 is a block diagram of a device for correcting motion vectors in boundary regions in an embodiment of the present invention.

Detailed ways

The present invention will be described in detail below in conjunction with specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, method, or functional changes made by those skilled in the art according to these embodiments are included in the protection scope of the present invention.

Please refer to FIG. 1, in a specific embodiment of the present invention, the method for correcting the motion vector of the boundary area includes the following steps:

S1. Determine the boundary offset block and the boundary block of the video frame;

The present invention aims to make the motion vector at the boundary of the video frame consistent with the motion of the object or scene by performing special processing on the boundary of the video frame. So determining the boundary of the video frame is the key link.

In order to achieve the ultimate goal of the above steps, step S1 can be specifically divided into the following steps to achieve:

S11. Developing a global motion vector for each frame of the video frame;

A global motion vector is first developed for each frame. This global motion vector is generated by accumulating the number of occurrences of the motion vector for each block (eg, a block of size 8*8) in each frame. Preferably, in the present invention, the motion vector corresponding to the highest count is designated as the global motion vector.

S12. Split the global motion vector to obtain a first motion vector and a second motion vector;

Referring to Figure 2, the global motion vectors are used to determine whether the motion vectors at video frame boundaries are plausible. According to the distance between the previous frame and the intermediate frame, the global motion vector is first split into two parts (the first motion vector and the second motion vector). For example, if the horizontal component (Global MV) of the global motion vector is 16 and points to the left, it means that the global motion is from left to right, the middle frame is 3/4 from the previous frame, and the middle frame is 1 from the next frame /4, so the global motion vector is split into two parts, one part is equal to 12, and the other part is equal to 4. The part equal to 12 is called the uncovered motion vector (MV (uncovered)), that is, the first motion vector; the part equal to 4 is called the covered motion vector (MV (covered)), that is, the second motion vector.

S13. Convert the motion vector of the video frame into a block index according to the size of the block, wherein the first motion vector corresponds to the first block index, and the second motion vector corresponds to the second block index;

In order to facilitate the determination of the boundary offset block and boundary block corresponding to the video frame, the motion vector of the video frame must be converted into a corresponding block index. The process of converting the motion vector of the video frame into a block index is realized by the size of the block. For example, for a block of 4x4 size, the block index of the first motion vector is 12/4=3, and the block index of the second motion vector Then 4/4=1. Among them, after the conversion, the block index with a block index of 3 is called the exposed block offset (the first block index), and the block index with the block index of 1 is called the covered block offset (the second block index).

S14. Determine the distribution of the boundary offset block and the boundary block according to the block index.

Referring to Figure 3, the step S14 specifically includes:

Get the block with the block index equal to the first block index on the left side of the video frame and the block with the block index equal to the second block index on the right side of the video frame to form a ring-shaped boundary offset block area (the gray part in the figure), these Blocks are collectively defined as boundary offset blocks 101 of video frames;

It is worth mentioning that each boundary offset block 101 includes a motion vector with a certain direction (horizontally left, right, or vertically upward, downward).

After that, the block whose block index is outside the boundary offset block area can be defined as the boundary block 102 . Wherein, being located outside the boundary offset block means being located at the periphery of the circular portion in the figure.

S2. Assign the motion vector of the boundary offset block to the boundary block, so that the boundary block points to the same direction as the motion vector of the boundary offset block.

In this embodiment, in order to reduce the error caused by motion estimation in the boundary area of the video frame, the motion vector of the boundary offset block is transferred to its adjacent boundary blocks. Therefore, if the boundary block and the boundary offset block belong to the same object, after such processing, their motion vectors can be kept consistent.

Wherein, in the present invention, the method of "assigning the motion vector of the boundary offset block to the boundary block" in step S2 includes direct copying of the motion vector, or offset, or stretching, or any of the above methods combination.

Direct copy is generally applicable to the middle part of the boundary, and the boundary block adjacent to the boundary offset block can be directly copied to a vector whose direction and size are equal to the corresponding motion vector of the boundary offset block;

And in some parts, the boundary block adjacent to the boundary offset block can be given to a vector (stretch) with the same direction and a different size than the corresponding motion vector of the boundary offset block;

At the corner of the boundary (see Figure 3), the boundary block adjacent to the boundary offset block can be assigned a vector (offset) with a different direction and the same size (or different) from the corresponding motion vector of the boundary offset block;

Certainly, in other implementation manners of the present invention, the way of "assigning the motion vector of the boundary offset block to the boundary block" may also be a combination of the above-mentioned ways (copying, stretching, offsetting).

If the above-mentioned step of "assigning the motion vector of the boundary offset block to the boundary block" uses a more optimized formula to realize the transmission of the motion vector, that is: MV2=A*MV1+B, Among them, MV1 is the motion vector of the boundary offset block, MV2 is the motion vector of the boundary block, and A and B are configurable parameters.

Next, the device for correcting the motion vector of the boundary area of the present invention will be described with reference to FIG. 4 . In a specific embodiment of the present invention, the device includes:

Boundary block acquisition unit 10, for determining the boundary offset block and the boundary block of the video frame;

The present invention aims to make the motion vector at the boundary of the video frame consistent with the motion of the object or scene by performing special processing on the boundary of the video frame. So determining the boundary of the video frame is the key link.

In order to achieve the ultimate purpose of this unit, the boundary block acquisition unit 10 can be specifically divided into the following units for realization:

The global motion vector acquisition unit 11 is used to develop a global motion vector for each frame of the video frame;

A global motion vector is first developed for each frame. This global motion vector is generated by accumulating the number of occurrences of the motion vector for each block (eg, a block of size 8*8) in each frame. Preferably, in the present invention, the motion vector corresponding to the highest count is designated as the global motion vector.

A global motion vector splitting unit 12, configured to split the global motion vector to obtain a first motion vector and a second motion vector;

Referring to Figure 2, the global motion vectors are used to determine whether the motion vectors at video frame boundaries are plausible. According to the distance between the previous frame and the intermediate frame, the global motion vector is first split into two parts (the first motion vector and the second motion vector). For example, if the horizontal component (Global MV) of the global motion vector is 16 and points to the left, it means that the global motion is from left to right, the middle frame is 3/4 from the previous frame, and the middle frame is 1 from the next frame /4, so the global motion vector is split into two parts, one part is equal to 12, and the other part is equal to 4. The part equal to 12 is called the uncovered motion vector (MV (uncovered)), that is, the first motion vector; the part equal to 4 is called the covered motion vector (MV (covered)), that is, the second motion vector.

The conversion unit 13 is configured to convert the motion vector of the video frame into a block index according to the size of the block, wherein the first motion vector corresponds to the first block index, and the second motion vector corresponds to the second block index;

In order to facilitate the determination of the boundary offset block and boundary block corresponding to the video frame, the motion vector of the video frame must be converted into a corresponding block index. The process of converting the motion vector of the video frame into a block index is realized by the size of the block. For example, for a block of 4x4 size, the block index of the first motion vector is 12/4=3, and the block index of the second motion vector Then 4/4=1. Among them, after the conversion, the block index with a block index of 3 is called the exposed block offset (the first block index), and the block index with the block index of 1 is called the covered block offset (the second block index).

The boundary obtaining unit 14 is configured to determine the distribution of the boundary offset blocks and boundary blocks according to the block index.

Referring to Fig. 3, the boundary acquisition unit 14 is specifically used for:

Get the block with the block index equal to the first block index on the left side of the video frame and the block with the block index equal to the second block index on the right side of the video frame to form a ring-shaped boundary offset block area (the gray part in the figure), these Blocks are collectively defined as boundary offset blocks 101 of video frames;

It is worth mentioning that each boundary offset block 101 includes a motion vector with a certain direction (horizontally left, right, vertically upward, downward, or a combination thereof).

After that, the boundary obtaining unit 14 can define the block whose block index is outside the boundary offset block area as the boundary block 102 . Wherein, being located outside the boundary offset block means being located at the periphery of the circular portion in the figure.

The motion vector transmitting unit 20 is configured to assign the motion vector of the boundary offset block to the boundary block, so that the boundary block points to the same direction as the motion vector of the boundary offset block.

In this embodiment, in order to reduce the error caused by motion estimation in the boundary area of the video frame, the motion vector of the boundary offset block is transferred to its adjacent boundary blocks. Therefore, if the boundary block and the boundary offset block belong to the same object, after such processing, their motion vectors can be kept consistent.

Among them, in the present invention, the way of "assigning the motion vector of the boundary offset block to the boundary block" in the motion vector transfer unit 20 includes direct copying of the motion vector, or offset, or stretching, or the above-mentioned any combination of methods.

Direct copy is generally applicable to the middle part of the boundary, and the boundary block adjacent to the boundary offset block can be directly copied to a vector whose direction and size are equal to the corresponding motion vector of the boundary offset block;

And in some parts, the boundary block adjacent to the boundary offset block can be given to a vector (stretch) with the same direction and a different size than the corresponding motion vector of the boundary offset block;

At the corner of the boundary (see Figure 3), the boundary block adjacent to the boundary offset block can be assigned a vector (offset) with a different direction and the same size (or different) from the corresponding motion vector of the boundary offset block;

Certainly, in other implementation manners of the present invention, the way of "assigning the motion vector of the boundary offset block to the boundary block" may also be a combination of the above-mentioned ways (copying, stretching, offsetting).

If the above-mentioned step of "assigning the motion vector of the boundary offset block to the boundary block" uses a more optimized formula to realize the transmission of the motion vector, that is: MV2=A*MV1+B, Among them, MV1 is the motion vector of the boundary offset block, MV2 is the motion vector of the boundary block, and A and B are configurable parameters.

Compared with the prior art, the method and corresponding device in the technical solution of the present invention assign the credible motion vector of the boundary offset block to the boundary block, so that the boundary block points to the same direction as the motion vector of the boundary offset block , that is, if the boundary block and the boundary offset block belong to the same object, after such processing, their motion vectors can be kept consistent, thereby improving the image quality of the boundary area of the video frame.

The device implementations described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network elements. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. It can be understood and implemented by those skilled in the art without creative effort.

For the convenience of description, when describing the above devices, functions are divided into various units and described separately. Of course, when implementing the present application, the functions of each unit can be implemented in one or more pieces of software and/or hardware.

The device implementations described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network elements. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. It can be understood and implemented by those skilled in the art without creative effort.

This application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including storage devices.

It should be understood that although this description is described according to implementation modes, not each implementation mode only contains an independent technical solution, and this description in the description is only for clarity, and those skilled in the art should take the description as a whole, and each The technical solutions in the embodiments can also be properly combined to form other embodiments that can be understood by those skilled in the art.

The series of detailed descriptions listed above are only specific descriptions for feasible implementations of the present invention, and they are not intended to limit the protection scope of the present invention. Any equivalent implementation or implementation that does not depart from the technical spirit of the present invention All changes should be included within the protection scope of the present invention.

Claims (6)

1. A method for correcting boundary region motion vectors, characterized in that it comprises the steps: S1. Determine the boundary offset block and the boundary block of the video frame; S2. Assign the motion vector of the boundary offset block to the boundary block, so that the boundary block points to the same direction as the motion vector of the boundary offset block; The step S1 specifically includes the following steps: S11. Developing a global motion vector for each frame of the video frame; S12. Split the global motion vector according to the distance between the previous frame and the intermediate frame to obtain a first motion vector and a second motion vector; S13. Convert the motion vector of the video frame into a block index according to the size of the block, wherein the first motion vector corresponds to the first block index, and the second motion vector corresponds to the second block index; S14. Determine the distribution of the boundary offset block and the boundary block according to the block index; Described step S14 specifically comprises: Obtain the block whose block index is equal to the first block index on the left side of the video frame and the block whose block index is equal to the second block index on the right side of the video frame to form a ring-shaped boundary offset block area; A block whose block index is outside the boundary offset block area is defined as a boundary block. 2. The method according to claim 1, wherein the method of "assigning the motion vector of the boundary offset block to the boundary block" in the step S2 includes direct copying of the motion vector, or offsetting , or stretching, or any combination of the above methods. 3. The method according to claim 1 or 2, wherein the step of "assigning the motion vector of the boundary offset block to the boundary block" further comprises: The motion vector is transmitted by the formula MV2=A*MV1+B, where MV1 is the motion vector of the boundary offset block, MV2 is the motion vector of the boundary block, and A and B are configurable parameters. 4. A device for correcting motion vectors in boundary regions, characterized in that it comprises the following units: A boundary block acquisition unit, used to determine a boundary offset block and a boundary block of a video frame; a motion vector transfer unit, configured to assign the motion vector of the boundary offset block to the boundary block, so that the boundary block points to the same direction as the motion vector of the boundary offset block; The boundary block acquisition unit specifically includes the following units: A global motion vector acquisition unit, configured to develop a global motion vector for each frame of the video frame; a global motion vector splitting unit, configured to split the global motion vector according to the distance between the previous frame and the intermediate frame to obtain a first motion vector and a second motion vector; The conversion unit is used to convert the motion vector of the video frame into a block index according to the size of the block, wherein the first motion vector corresponds to the first block index, and the second motion vector corresponds to the second block index; a boundary obtaining unit, configured to determine the distribution of the boundary offset block and the boundary block according to the block index; The boundary learning unit is specifically used for: Obtain the block whose block index is equal to the first block index on the left side of the video frame and the block whose block index is equal to the second block index on the right side of the video frame to form a ring-shaped boundary offset block area; A block whose block index is outside the boundary offset block area is defined as a boundary block. 5. The device according to claim 4, wherein the manner of "assigning the motion vector of the boundary offset block to the boundary block" in the motion vector transfer unit includes direct copying of the motion vector, or Offset, or stretch, or any combination of the above. 6. The device according to claim 4 or 5, wherein the motion vector transfer unit is further used for: The motion vector is transmitted by the formula MV2=A*MV1+B, where MV1 is the motion vector of the boundary offset block, MV2 is the motion vector of the boundary block, and A and B are configurable parameters.

Images