JP5958133B2 - Video processing apparatus and video processing method - Google Patents

Description

  The present invention relates to a video processing apparatus including a motion determination unit that determines whether the entire screen is moving uniformly.

  In recent years, a moving picture shooting function is standardly installed in a mobile phone, and it is in an environment where a moving picture can be easily taken. However, when a person unfamiliar with moving image shooting takes a picture, the image appears blurred due to the influence of camera shake. In order to remove camera shake from a photographed camera shake image, camera shake correction is performed by video signal processing.

  To perform camera shake correction by video signal processing, first, an input image is divided into blocks, and a motion vector indicating motion from a past frame is calculated in each block. Next, a global vector indicating the motion of the entire screen is calculated from the motion vector obtained for each block. A camera shake vector is calculated from this global vector, and camera shake correction is performed by shifting the display position of the image.

  If the global vector is calculated from all the motion vectors, if the subject is not a stationary object, there is a possibility that a global vector affected by the motion of the subject is obtained. Therefore, the conventional motion detection circuit performs reliability determination based on the correlation calculation value and the divergence degree with respect to the motion vector at the periphery of the screen and the motion vector at the center of the screen to determine whether the subject is a stationary object. . If the subject is a stationary object and the entire screen moves uniformly, then the motion vector of the entire screen is a global vector, and if the subject is not a stationary object, the motion vector of the screen periphery is the global vector. When the subject is a stationary object, the global vector is calculated from more motion vectors, so the accuracy is increased. If the subject is not a stationary object, the global vector is calculated while avoiding the influence of the motion of the subject in the center. (See Patent Document 1 below)

Japanese Patent No. 2595841

  However, the conventional motion detection circuit needs to calculate the correlation calculation value and the divergence. For example, when implemented as an additional function in a device such as a mobile phone, the circuit scale for executing such a calculation is large. There is a possibility that it cannot be secured.

  The present invention has been made in view of the above, and an object of the present invention is to obtain a video processing apparatus capable of realizing whether or not the entire screen is moving uniformly with a small circuit scale.

Depending on which of the regions obtained by dividing the specific region set on the coordinate plane, the coordinate of the motion vector component of each block obtained from the input video with the X coordinate value as the X coordinate value and the Y component as the Y coordinate value A motion vector distribution statistic unit that associates a motion vector with a region and calculates motion vector distribution information indicating the number of motion vectors associated with each region of the specific region, and a plurality of adjacent motion vector distribution information When the number of motion vectors associated with a region is larger than a threshold value stored in advance , a motion determination unit is provided that determines that the entire screen is moving uniformly.

  According to the present invention, it is possible to determine whether the entire screen is moving uniformly with a small circuit scale, without calculating a correlation calculation value or a divergence degree.

1 is a block diagram showing a video processing apparatus 10 according to a first embodiment. FIG. 4 is a diagram showing an example of a motion vector according to the first embodiment. FIG. 4 is a diagram showing an example of motion vector distribution according to the first embodiment. FIG. 6 is a diagram showing an example of the shape and division of a specific area 31 according to the first embodiment. FIG. 6 shows an example of motion vector distribution information VDIS1 according to the first embodiment. 5 is a flowchart showing a process flow of a motion determination unit 13 according to the first embodiment. FIG. 4 is a diagram illustrating an example of an outermost periphery of a specific region 31 according to the first embodiment. FIG. 4 shows an example of an adjacent region according to the first embodiment. FIG. 4 shows an example of four adjacent areas according to the first embodiment. The figure which shows the example which computed the number of the motion vectors applicable to the four adjacent area | region which concerns on Embodiment 1 by using the component of a motion vector as a coordinate value. 9 is a flowchart showing a process flow of a motion determination unit 13 according to the second embodiment. FIG. 10 is a diagram illustrating an example of a motion vector serving as a representative value of a region according to the second embodiment.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing a video processing apparatus 10 according to the first embodiment.
The input processing unit 11 divides the input video into blocks, calculates a motion vector indicating motion from a past frame in each block, and outputs the motion vector to the motion vector distribution statistical unit 12 and the global vector calculation unit 14. The motion vector distribution statistical unit 12 calculates the motion vector distribution information VDIS1 from the motion vector and outputs the motion vector distribution information VDIS1 to the motion determination unit 13. The motion determination unit 13 generates motion determination information VDET2 from the motion vector distribution information VDIS1 and outputs it to the global vector calculation unit 14. The global vector calculation unit 14 calculates a global vector GV3 using the motion determination information VDET2 and the motion vector, and outputs the global vector GV3 to the camera shake correction unit 15. The camera shake correction unit 15 outputs the real global vector RGV4 using the global vector GV3 and returns it to the motion vector distribution statistics unit 12 and outputs an image subjected to camera shake correction. Although the display video is output in FIG. 1, it may be output to the display device, or the video data may be stored in the storage device.

FIG. 2 is a diagram illustrating an example of a motion vector according to the first embodiment. The motion vector output from the input processing unit 11 is calculated as a vector indicating motion from a past frame in each block by dividing the input video into blocks.
FIG. 3 is a diagram illustrating an example of motion vector distribution according to the first embodiment. The motion vector component (represented by (X, Y)) obtained from the input video is drawn as a coordinate value on the coordinate plane, and the actual global vector component input from the camera shake correction unit 15 is used as the coordinate value. This is a distribution in which a specific area 31 having a value as the center point C is set and the specific area 31 is divided into areas A00 to A55. FIG. 3A shows a motion vector expressed by a vector, and FIG. 3B shows a vector component expressed by a vector component. The reason for determining the center point C of the specific area 31 based on the real global vector will be described later.
FIG. 4 is a diagram illustrating an example of the shape and division of the specific region 31 according to the first embodiment.
FIG. 5 is a diagram showing an example of the motion vector distribution information VDIS1 according to the first embodiment. N00 to N55 have motion vector components as coordinate values, and the coordinate values indicate the number of motion vectors corresponding to the areas A00 to A55. For example, N00 is the number of motion vectors corresponding to the region A00, where the motion vector component is a coordinate value. Note that the number of motion vectors is called the number of motion vectors.

Next, the operation of the motion vector distribution statistical unit 12 will be described.
When the motion vector for each block obtained by dividing the input video is input, the motion vector distribution statistics unit 12 draws the motion vector component as a coordinate value on the coordinate plane, and sets the specific region 31 with the center point as C. The motion vector distribution statistical unit 12 further divides the specific region 31 into regions A00 to A55, uses the motion vector components as coordinate values, counts the number of motion vectors corresponding to each region, and The motion vector distribution information VDIS1 shown in FIG. 5 is generated by associating with the region, and is output to the motion determination unit 13. The motion determination unit 13 determines from the motion vector distribution information VDIS1 whether the entire screen is moving uniformly, and outputs the determined result as motion determination information VDET2.

  In the present embodiment, the shape of the specific region 31 is a rectangle, but other shapes such as a rhombus and a circle may be used. The specific area is divided by the horizontal component and the vertical component, but may be divided by other methods and the number of divisions is not limited. The shape and division method of the specific region 31 may be, for example, the method shown in FIG. Further, when the motion vector distribution information VDIS1 is calculated by counting the motion vectors belonging to each region, in this embodiment, the unit is one frame, but may be a plurality of frames. Furthermore, it may be counted in field units that are images formed by collecting only odd lines or only even lines from one frame.

Next, the operation of the motion determination unit 13 will be described.
FIG. 6 is a flowchart illustrating a process flow of the motion determination unit 13 according to the first embodiment.
FIG. 7 is a diagram illustrating an example of the outermost periphery of the specific region 31 according to the first embodiment.
FIG. 8 is a diagram illustrating an example of adjacent regions according to the first embodiment.
It is assumed that the motion determination unit 13 holds threshold values for Th0, Th1, and Th2 in advance. When the motion vector distribution information VDIS1 is input from the motion vector distribution statistics unit 12, the process starts from step S61 and proceeds to step S62. In step S62, the motion determination unit 13 refers to the motion vector distribution information VDIS1, and determines whether the value of the region where the number of motion vectors is the maximum is greater than the threshold Th0. For example, in the present embodiment, a region where the number of motion vectors is the maximum is a region A13. If the number of motion vectors N13 in the area A13 is greater than the threshold value Th0, the process proceeds to step S66, and if not, the process proceeds to step S63.

  In step S <b> 63, the motion determination unit 13 determines whether the region A <b> 13 where the number of motion vectors is the maximum is the outermost region of the specific region 31. The outermost area of the specific area 31 is an area in contact with the outside of the specific area, and is an area indicated by hatching in FIG. In addition, the motion determination unit 13 determines whether the total value of the number of motion vectors in the region A13 where the number of motion vectors is the maximum and the adjacent region is greater than the threshold Th1. The adjacent area of the area A13 is an area indicated by hatching in FIG. If the area A13 in which the number of motion vectors is maximum is not the outermost peripheral area of the specific area 31, and the total value N13 + N02 + N03 + N04 + N12 + N14 + N22 + N23 + N24 of the number of motion vectors in the area A13 and its adjacent areas is greater than the threshold Th1, the process proceeds to step S66. Otherwise, the process proceeds to step S64.

  In step S63, it is determined whether the region where the number of motion vectors is the maximum is not the outermost periphery of the specific region. The number of adjacent regions is different depending on whether the region is the outermost periphery or the outermost periphery. This is because the determination cannot be made under the same conditions when determining whether the total value of the number of motion vectors of the maximum area and the adjacent area is larger than the threshold Th1.

FIG. 9 is a diagram illustrating an example of four adjacent regions according to the first embodiment.
FIG. 10 is a diagram illustrating an example in which the number of motion vectors corresponding to the four adjacent regions according to the first embodiment is calculated using the motion vector components as coordinate values.
In step S64, the motion determination unit 13 calculates the number of motion vectors included in four adjacent regions as shown in FIG. FIG. 10 shows the result of calculating the total number of motion vectors included in the four adjacent areas for the areas A00 to A55. For example, M40 indicates a total value N40 + N41 + N50 + N51 of the number of motion vectors included in the areas A40, A41, A50, and A51. When calculating the total number of four adjacent motion vectors, the areas are shifted one by one in the x direction or the y direction. For example, the value of M41 is the total value N41 + N42 + N51 + N52 of the number of motion vectors included in the areas A41, A42, A51, and A52. In the present embodiment, there are four adjacent regions, but other values may be used. By counting the number of motion vectors contained in multiple adjacent regions, the motion vector distribution is constant even when the motion vector distribution is concentrated near the boundary of the region and the motion vectors are distributed and counted in the adjacent region. It is possible to determine whether or not the area is concentrated. When the number of motion vectors included in the four adjacent regions is larger than the threshold Th2, the process proceeds to step S66, otherwise the process proceeds to step S65.

  In step S65, the motion determination unit 13 sets a bit indicating that the entire screen is not moving uniformly in the motion determination information VDET2, proceeds to step S67, and ends the process. In step S66, the motion determination unit 13 sets a bit indicating that the entire screen is moving uniformly in the motion determination information VDET2, proceeds to step S67, and ends the process.

Next, the global vector calculation unit 14 and the camera shake correction unit 15 in FIG. 1 will be described.
The motion determination information VDET2 output from the motion determination unit 13 and the motion vector output from the input processing unit 11 are input to the global vector calculation unit 14, and the global vector calculation unit 14 calculates the global vector GV3. The global vector GV3 is calculated by an existing method. For example, the global vector calculation unit 14 refers to the motion determination information VDET2 and calculates a global vector from the motion vector of the entire screen when the entire screen is moving uniformly. In addition, when the entire screen does not move uniformly, the global vector calculation unit 14 calculates a global vector from the motion vector at the periphery of the screen while avoiding the movement of the subject at the center of the screen.

  The global vector calculation unit 14 outputs the global vector GV3 to the camera shake correction unit 15, and the camera shake correction unit 15 calculates the real global vector RGV4 and outputs it to the motion vector distribution statistics unit 12, and also uses the global vector GV3 to correct camera shake. To output the corrected display image. The real global vector RGV4 is a vector indicating the motion of the entire screen corrected for camera shake, and is calculated by an existing method. For example, the camera shake correction unit 15 calculates the actual global vector RGV4 by obtaining the average of the global vectors GV3 of the past several frames. When obtaining the average of the global vectors GV3 of the past several frames, the camera shake correction unit 15 may weight the global vectors GV3 of the frames close to the current frame and take the average. Further, the camera shake correction unit 15 may obtain the real global vector RGV4 by applying low-pass filter processing in which the frequency characteristics are adaptively changed according to the video to the global vector GV3. The camera shake correction unit 15 outputs the real global vector RGV4 thus obtained to the motion vector distribution statistics unit 12. In addition, the camera shake correction unit 15 subtracts the real global vector RGV4 from the global vector GV3, thereby calculating a camera shake vector, correcting the camera shake of the video, and outputting the corrected display video.

  The motion vector distribution statistical unit 12 sets the real global vector RGV4 (Vx, Vy) as the center point of the specific region 31 when determining the center point of the specific region 31. Since the distribution of motion vectors is concentrated in the vicinity of (Vx, Vy), the coordinates of the motion vector components as coordinate values are obtained by setting the real global vector RGV4 (Vx, Vy) as the center point of the specific region 31. The value is easily included in the specific area 31.

  Therefore, in the present embodiment, the motion vector and the region are associated with each other depending on whether the coordinate value of the motion vector component of each block obtained from the input video corresponds to the region obtained by dividing the specific region 31. The entire screen is uniform based on the motion vector distribution statistical unit 12 that calculates the motion vector distribution information VDIS1 indicating the number of motion vectors associated with each of the 31 regions, the motion vector distribution information VDIS1, and a threshold value stored in advance. A motion determination unit 13 that determines whether the motion is moving, an input processing unit that calculates a motion vector from the input video, and a motion that indicates whether the entire screen input from the motion determination unit 13 is performing a uniform motion Based on the determination information and the motion vector, a global vector calculation unit 14 that calculates a global vector, Accordingly, the image stabilization unit 15 includes an image stabilization unit 15 that calculates an actual global vector indicating the motion of the entire screen subjected to the image stabilization, and performs image stabilization on the input video based on the actual global vector. When the maximum value of the number of motion vectors associated with a region is larger than a threshold value stored in advance, or the region where the number of motion vectors associated with the region is the maximum is not the outermost periphery of the specific region 31 and If the number of motion vectors associated with the region where the number of attached motion vectors is the maximum and its adjacent region is greater than the threshold stored in advance, or the motion vectors associated with a plurality of adjacent regions If the maximum value of the number is larger than the threshold value stored in advance, the video processing apparatus 10 determines that the entire screen is moving uniformly. It is not necessary to calculate the correlation operation value or emittance can vector. In addition, instead of holding the motion vector itself, the specific area 31 is set, and only the number of corresponding motion vectors is stored in each area obtained by dividing the specific area 31 with the motion vector component as a coordinate value. The determination of whether or not the whole is moving uniformly can be realized with a small circuit scale.

  In addition, since the motion vector distribution statistical unit determines the center of the specific area based on the real global vector, the video processing apparatus 10 moves the specific area 31 according to the motion of the video to move the motion vector distribution information VDIS1. The coordinate value having the motion vector component as the coordinate value is easily included in the specific region 31, and the accuracy of determination of whether the entire screen is moving uniformly can be improved.

Embodiment 2. FIG.
In the first embodiment described above, the motion vector distribution statistical unit 12 divides the specific region 31 into the regions A00 to A55, counts the number of motion vectors corresponding to each region using the motion vector components as coordinate values, and motion vectors. The distribution information VDIS1 is output, and the motion determination unit 13 determines whether the motion of the entire screen is uniform based on the motion vector distribution information VDIS1 and a threshold stored in advance. Shows an embodiment in which it is determined whether the motion of the entire screen is uniform by calculating the variance of motion vectors included in the specific region 31.
In addition, since parts other than the motion determination part 13 in Embodiment 2 are the same as Embodiment 1, description is omitted.

Next, the operation of the motion determination unit 13 will be described.
FIG. 11 is a flowchart illustrating a process flow of the motion determination unit 13 according to the second embodiment.
FIG. 12 is a diagram illustrating an example of a motion vector serving as a representative value of a region according to the second embodiment. The center value of the area Aij is set as the representative value (Xi, Yj) of the motion vector of the area. For example, in the case of the region A33, the motion vector (X3, Y3) is a representative value.
It is assumed that the motion determination unit 13 holds threshold values for Th3 and Th4 in advance. When the motion vector distribution information VDIS1 is input from the motion vector distribution statistic unit 12, the process starts from step S111. In step S112, the motion determination unit 13 calculates the approximate approximate value (Xm, Ym) is calculated, and the process proceeds to step S113. Nij is motion vector distribution information VDIS1, Xj, and Yi are the X component and Y component of the representative value of the motion vector of each region.

ステップＳ１１３において、動き判定部１３は、動きベクトル分布情報ＶＤＩＳ１と特定領域３１の領域に対応付けられた動きベクトルの平均の近似値（Ｘｍ，Ｙｍ）とより、特定領域３１の領域に対応付けられた動きベクトルの分散の近似値（Ｘｖ，Ｙｖ）を数式３〜４で算出する。

In step S <b> 113, the motion determination unit 13 is associated with the region of the specific region 31 based on the motion vector distribution information VDIS <b> 1 and the average approximate value (Xm, Ym) of the motion vector associated with the region of the specific region 31. The approximate value (Xv, Yv) of the variance of the motion vector is calculated by Equations 3-4.

処理は、ステップＳ１１４へ進む。ステップＳ１１４において、動き判定部１３はＸｖが閾値Ｔｈ３より小さく、かつ、Ｙｖが閾値Ｔｈ４より小さいかを判定する。Ｘｖが閾値Ｔｈ３より小さく、かつ、Ｙｖが閾値Ｔｈ４より小さい場合、処理はステップＳ１１６へ進み、それ以外の場合、処理はステップＳ１１５へ進む。ステップＳ１１５において、動き判定部１３は動き判定情報ＶＤＥＴ２に画面全体が一様な動きをしていないことを示すビットを設定し、ステップＳ１１７へ進み、処理を終了する。ステップＳ１１６において、動き判定部１３は動き判定情報ＶＤＥＴ２に画面全体が一様な動きをしていることを示すビットを設定し、ステップＳ１１７へ進み、処理を終了する。

The process proceeds to step S114. In step S114, the motion determination unit 13 determines whether Xv is smaller than the threshold Th3 and Yv is smaller than the threshold Th4. If Xv is smaller than threshold Th3 and Yv is smaller than threshold Th4, the process proceeds to step S116. Otherwise, the process proceeds to step S115. In step S115, the motion determination unit 13 sets a bit indicating that the entire screen is not moving uniformly in the motion determination information VDET2, proceeds to step S117, and ends the process. In step S116, the motion determination unit 13 sets a bit indicating that the entire screen is moving uniformly in the motion determination information VDET2, proceeds to step S117, and ends the process.

  As described above, in the present embodiment, the motion determination unit 13 calculates an approximate value of the variance of the motion vector associated with the specific region 31 based on the motion vector distribution information VDIS1 and the representative value of the motion vector of the region. When the calculated approximate value of variance is smaller than the threshold value stored in advance, since it is determined that the entire screen is moving uniformly, it is necessary for the video processing apparatus 10 to calculate the correlation calculation value of the motion vector. Absent. In addition, instead of holding the motion vector itself, the specific area 31 is set, and only the number of corresponding motion vectors is stored in each area obtained by dividing the specific area 31 with the motion vector component as a coordinate value. The determination of whether or not the whole is moving uniformly can be realized with a small circuit scale.

DESCRIPTION OF SYMBOLS 10 Image processing apparatus 11 Input processing apparatus 12 Motion vector distribution statistics part 13 Motion determination part 14 Global vector calculation part 15 Camera shake correction part 31 Specific area

Claims (6)

Depending on which of the regions obtained by dividing the specific region set on the coordinate plane, the coordinate of the motion vector component of each block obtained from the input video with the X coordinate value as the X coordinate value and the Y component as the Y coordinate value A motion vector distribution statistic unit that associates the motion vector with the region and calculates motion vector distribution information indicating the number of motion vectors associated with each region of the specific region;
For the motion vector distribution information , when the number of motion vectors associated with a plurality of adjacent regions is greater than a threshold value stored in advance , a motion determination unit that determines that the entire screen is moving uniformly ,
A video processing apparatus comprising: An input processing unit that calculates the motion vector from the input video;
A global vector calculation unit that calculates a global vector based on the motion determination information indicating whether the entire screen input from the motion determination unit is moving uniformly and the motion vector;
On the basis of the global vector, the image stabilization unit calculates the actual global vector representing the motion of the entire screen without the camera shake, before based on Kigu global vector, and outputs the image stabilization of the input image,
The video processing apparatus according to claim 1, further comprising: 3. The motion vector distribution statistical unit according to claim 1, further comprising: determining a center of the specific region based on a real global vector indicating a motion of the entire screen that does not include camera shake . 4. Video processing device. When the maximum value of the number of motion vectors associated with the region is larger than a threshold value stored in advance,
Alternatively, the region where the number of associated motion vectors is the maximum is not the outermost periphery of the specific region, and the region corresponding to the maximum number of associated motion vectors corresponds to the adjacent region. If the sum of the number of motion vectors attached is greater than a pre-stored threshold ,
The video processing apparatus according to any one of claims 1 to 3 whole screen is characterized in that it comprises the motion determination unit determines to be a uniform motion. Based on the motion vector distribution information and a representative value of the motion vector representing the motion vector associated with the region, an approximate value of the variance of the motion vector associated with the specific region is calculated,
4. The movement determination unit according to claim 1, further comprising: determining that the entire screen is moving uniformly when the approximate value of the variance is smaller than a threshold value held in advance. 5. The video processing apparatus described. Depending on which of the regions obtained by dividing the specific region set on the coordinate plane, the coordinate of the motion vector component of each block obtained from the input video with the X coordinate value as the X coordinate value and the Y component as the Y coordinate value A motion vector distribution statistic step that associates the motion vector with the region and calculates motion vector distribution information indicating the number of the motion vectors associated with each region of the specific region;
For the motion vector distribution information , when the number of motion vectors associated with a plurality of adjacent regions is greater than a threshold value that is stored in advance , a motion determination step that determines that the entire screen is performing a uniform motion; ,
A video processing method.

Images