JPH10134193A - Method and device for calculating moving vector - Google Patents

Abstract

PROBLEM TO BE SOLVED: To evaluate the accuracy of a calculated moving vector by calculating matching degrees at respective positions while making a block on one image scan in the other image, and evaluating and deciding the best value. SOLUTION: The image signal obtained by a video camera 10 is converted by an A/D converter 11 into a sequential digital image signal, which is stored in frame buffers 13a and 13b alternately through a changeover switch 12. A moving vector calculation part 14 calculates a moving vector from the different- time image signals stored in the two frame buffers 13a and 13b. At this time, a block area is set for the stored image, a block matching is performed according to the image of the area, and matching degrees at respective positions within the matching range and an evaluated value for the result are calculated. Then the found evaluated value is compared with a previously set threshold value and when it is decided that the result of the matching is accurate, the moving vector is recorded.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for calculating a motion vector for calculating a motion vector in an image.

[0002]

2. Description of the Related Art In various image processing apparatuses, some of the processing steps include a process of calculating a movement vector. For example, in a moving object detection device that detects and extracts a moving object based on image information, a moving vector is obtained from images captured at different times, and the moving vector is analyzed to extract the moving object. Further, for example, in an image compression apparatus, a movement vector is obtained from images captured at different times, and the compression efficiency is enhanced by using the movement vector.

There are various methods for obtaining a motion vector, such as a block matching method and a gradient method. Many image processing apparatuses use the block matching method.

Literature 1, "Takeru Azain, Tomoharu Nagao," Image Processing and Recognition ", Shokodo, 1992, pp. 164-167] For example, in the above Literature 1, corresponding points are determined between consecutive frames. A method for determining a movement vector by using the method is disclosed. In this method, a block centered on a point for which a corresponding point is to be obtained in one image is considered, and the target block is scanned on the other image to search for the best matching position, and the center of that position is obtained as a corresponding point. Things.
Such a method is generally called a block matching method.

[0005] Various examples of the quantity (matching degree) indicating the goodness of matching between the blocks are described in the above-mentioned reference 1, and for example, the sum of absolute values RPQ of the luminance difference shown in equation (1) can be used. It should be noted that the sum 1 in the equation (1) is such that x is -L
To L, and y is from -L to L.

RPQ = ΣΣ | Fj (Xq + x, Yq + y) −Fi (Xp + x, Yp + y) | (1) As shown in FIG.
The upper point P (Xp, Yp) and the point Q on the other image Fj
It indicates the degree of coincidence of blocks around each of (Xq, Yq), and the closer the value of RPQ is to 0, the better the coincidence is. Here, the size of the block is a square of 2L + 1 pixel on a side.

[0007]

Naturally, when a motion vector is obtained by the block matching method, a position that best matches a block on one image must be uniquely determined on the other image.

However, for example, the degree of coincidence is substantially the same at each position within a portion such as a road surface portion in a road image or a wall or floor portion in an indoor image where the change in brightness is small. Points cannot be uniquely determined. Also, for a portion having a repetitive pattern such as a balustrade of a bridge, a pedestrian crossing, or a tiled floor, since there are a plurality of image regions having the same pattern, the position giving the best match is also unique. It is difficult to decide.

For such a portion, it is possible to prevent erroneous association to some extent by, for example, restricting the range of block matching to the vicinity of the original position. However, in a case where there is no pattern at all, or in a portion where it is difficult to obtain a correct movement vector due to a fine repetition pattern, it is necessary to devise so that an uncertain movement vector is not calculated.

Therefore, there is a need for a method and apparatus for calculating a motion vector that can evaluate the certainty of the calculated motion vector when calculating the motion vector by the block matching method.

[0011]

According to a first aspect of the present invention, there is provided a moving vector calculating method for calculating a moving vector from two images photographed at different times. A process of calculating a matching score indicating goodness of matching at each position while scanning the block region set in the image on the other image, and (2) based on a plurality of higher-order values having a higher matching score, Forming an evaluation value of the likelihood of the movement vector.

According to a second aspect of the present invention, there is provided a moving vector calculating method for calculating a moving vector from two images photographed at different times, wherein (1) a multi-resolution image for each of the two inputted images; (2)
A process of calculating a matching score indicating goodness of matching at each position while scanning the block region set in one image on the other image, and (3) a process of calculating a matching score based on a plurality of upper values having a higher matching score. The process of forming an evaluation value of the likelihood of the movement vector, and (4) when the calculated evaluation value of the likelihood of the movement vector indicates that it is likely, the best value of the degree of coincidence is given. A motion vector is obtained according to the position, and when the evaluation value of the likelihood does not indicate that it is likely, the low-resolution image of the multi-resolution image is the same as the captured image for the block region including the block region. A motion vector is obtained by processing, or a motion vector for a low-resolution image obtained in advance by the same processing as that for a captured image is Characterized in that it comprises a process for the motion vector of the block area.

Further, the third invention provides a motion vector calculating apparatus for calculating a motion vector from two images photographed at different times, wherein (1) a block area set in one image is set on the other image. And (2) an evaluation value of the likelihood of the motion vector based on a plurality of values on the upper side having a higher degree of coincidence while scanning And moving vector evaluation value calculation means for forming

According to a fourth aspect of the present invention, there is provided a moving vector calculating apparatus for calculating a moving vector from two images taken at different times, wherein a multi-resolution image is created for each of the two inputted images. A multi-resolution image generating means, (1) a matching degree calculating means for calculating a matching degree indicating goodness of matching at each position while scanning a block area set in one image on the other image, and (2) ) A moving vector evaluation value calculation means for forming an evaluation value of the likelihood of the movement vector based on the plurality of values on the upper side having a high degree of coincidence, and (3) the evaluation value of the likelihood of the calculated movement vector is If it is determined that the likelihood is high, the motion vector is obtained from the position where the best value of the degree of coincidence is given, and the evaluation value of the certainty is not certain. In this case, for the low-resolution image of the multi-resolution image, for a block area including the block area, a motion vector is obtained by the same processing as that for the captured image, or the motion vector is obtained in advance by the same processing as for the captured image. A moving vector determining unit that sets a moving vector of the low-resolution image as a moving vector of the block area.

According to the first and third aspects of the present invention, it is possible to obtain a motion vector whose reliability is evaluated based on a plurality of coincidences in block matching.

Further, according to the second and fourth aspects of the present invention, it is possible to obtain a motion vector whose reliability is evaluated based on a plurality of coincidences in block matching, and obtain a motion vector for an original image. Since the movement vector obtained for the low-resolution image is output to the block area that cannot be obtained, the number of block areas in which the movement vector takes an effective value can be increased.

[0017]

BEST MODE FOR CARRYING OUT THE INVENTION

(A) First Embodiment Hereinafter, a first embodiment of a movement vector calculation method and apparatus according to the present invention will be described in detail with reference to the drawings.

In the first embodiment, the degree of coincidence is calculated at each position while scanning a block on one image on the other image, and the best value is sufficiently better than the values of the other degrees of coincidence. It is characterized in that only a certain movement vector is calculated by judging whether or not the motion vector is a motion vector.

FIG. 3 is a block diagram showing the configuration of a system to which the method and apparatus for calculating a motion vector according to the first embodiment are applied.

In FIG. 3, an image signal (analog signal) obtained by a video camera 10 is sequentially converted into a digital image signal by an analog / digital converter (A / D converter) 11, The data is alternately supplied and stored in the buffer 13a or 13b.
The frame buffers 13a and 13b store image signals photographed at different times (hereinafter, simply referred to as images in some cases).
At appropriate time intervals. The movement vector calculation unit 14 includes two frame buffers 13
The movement vector is calculated from the image signals having different times stored in a and 13b.

FIG. 1 is a flowchart showing the processing contents of the movement vector calculation unit 14 in FIG.

First, a block area for performing block matching is set for the image stored in the frame buffer 13a (step SP21).

Next, block matching is performed based on the image of the block area, and the degree of coincidence at each position in the matching range and the evaluation value for the result are calculated (step SP22). The center of the set block area,
The vector connecting the center of the block with the highest degree of coincidence is the movement vector to be obtained. The evaluation value indicates the likelihood of the obtained movement vector.

Then, the obtained evaluation value is compared with a preset threshold value T, and it is determined whether or not the obtained movement vector is reliable (step SP2).
3). That is, for example, when the other image stored in the frame buffer 13b has little change in luminance,
If the result of block matching is uncertain in a case where the same pattern is repeated, step SP2
The evaluation value obtained in Step 2 is a small evaluation. Therefore, by comparing this evaluation value with an appropriate threshold value, it is determined whether or not the obtained movement vector is reliable.

If the result of the comparison between the evaluation value and the threshold value indicates that the result of the block matching is reliable, the obtained movement vector is recorded (step S).
P24). On the other hand, if the result of the comparison between the evaluation value and the threshold value determines that the result of the block matching is uncertain, the obtained movement vector is not recorded.

The processing loop consisting of steps SP21 to SP24 as described above is repeatedly performed for each point for obtaining a movement vector (step SP25).

Hereinafter, the processing in step SP22 will be described in detail. The processing of step SP22 is roughly
It is divided into a block matching process and an evaluation value calculation process, and will be described in detail in this order.

First, referring to the block matching explanatory diagram of FIG. 2 described above, the processing contents of the block matching in step SP22 (step SP4 in FIG. 4 described later)
1) will be described.

As shown in FIG. 2, the block area Bp set in step SP21 is a square of 2L + 1 pixel on a side at the center of the image Fi at the point P (Xp, Yp). Also, on the other image Fj, the point P
The range for searching for a point corresponding to is called a search window W,
The search window W is set to a range where the movement amount of the center position of the block is less than or equal to ± w pixels in each axis direction from P.
In the block matching, a block B having the best matching degree with the processing target block area Bp within the search window W
is to find q.

Assuming that the degree of coincidence between a block centered at a point (dx, dy) away from the point P and the block area Bp to be processed is R (dx, dy), the above equation (1) is obtained. Is used to calculate the degree of coincidence R (dx, d
y) is expressed by equation (2). The sum 総 in the equation (2) is for x from -L to L and y for -L to L. Further, the degree of coincidence R (dx, dy) is-
Each position within the range of w ≦ dx ≦ w and −w ≦ dy ≦ w is sequentially obtained (step SP in FIG. 4 described later).
48).

R (dx, dy) = ΣΣ | Fj (Xp + dx + x, Yp + dy + y) −Fi (Xp + x, Yp + y) | (2) Next, the process of calculating the evaluation value in step SP22 will be described. FIG. 4 is a flowchart showing the process of step SP22 showing the details of the process of calculating the evaluation value.

Here, the evaluation value is the coincidence R (dx, d
This indicates how much the best value in the search window W of y) is better than the others. In the following description, it is assumed that the degree of coincidence (a smaller value indicates a better coincidence) R (dx, dy) is obtained by equation (2), and the difference between the minimum value of the degree of coincidence and the second minimum value is used as an evaluation value. calculate.

First, in performing the processing, a variable R1 representing the minimum value of the degree of coincidence and a variable R2 representing the second minimum value are initialized. The initial values of these variables R1 and R2 are set to the worst values that can be taken as the values of the degree of coincidence. In the following procedure, a variable D for storing the movement amount and a deviation (dxm, dym) up to the coordinates of the point giving the minimum coincidence are used.

Next, a certain point (X
The degree of coincidence R (dx, dy) between the block centered at (p + dx, Yp + dy) and the processing target block area Bp is obtained (step SP41). In FIG. 4, it is indicated as R.

Thereafter, the obtained degree of coincidence R is compared with the second minimum value variable R2 (step SP42). If the obtained degree of coincidence R is equal to or greater than the second minimum value variable R2 at that time, it will not be a candidate for a motion vector and cannot be used for calculating an evaluation value. The process proceeds to the calculation of the degree of coincidence for the center block (steps SP48 and SP41). On the other hand, when the obtained degree of coincidence R is smaller than the second minimum value variable R2 at that time, the obtained degree of coincidence R is changed to the minimum value variable R2.
1 (step SP43).

If the obtained degree of coincidence R is smaller than the minimum value variable R1, the currently obtained degree of coincidence R is the smallest among the degrees of coincidence up to now, and thus the minimum value variable R1
Is assigned to a second minimum value variable R2 (a new second minimum value), the matching degree R obtained this time is assigned to a minimum value variable R1 (a new minimum matching degree value), and from the point P Point (Xp +
dx, Yp + dy) 移動 (dx ² + dy ² )
Is substituted for the movement variable D (it is not always necessary to take the square root). Further, the coordinate deviation (dx, dy) between the center P of the processing target block area Bp and the center of the block for which the degree of coincidence is obtained is represented by a movement vector variable (dxm, dym).
(Step SP44).

On the other hand, if the comparison result in step SP43 shows that the obtained degree of coincidence R is equal to or larger than the minimum value variable R1, the condition R1 in step SP43>
If R does not hold, it is checked whether the obtained degree of coincidence R matches the minimum value variable R1 (step SP4).
5).

If they do not match, the matching degree R found this time is the second smallest among the matching degrees so far. Therefore, the matching degree R found this time is substituted into the second minimum value variable R2 ( (Step SP47), the process proceeds to the calculation of the degree of coincidence for a block centered on another point in the search window W (steps SP48 and SP41).

On the other hand, the obtained degree of coincidence R is the minimum value variable R
If it is equal to 1, the moving amount √ (dx ² + dy ² ) is compared with the moving amount variable D (step SP4).
6). If the moving amount is smaller than the moving amount variable D, the process proceeds to step SP44 assuming that the matching degree R obtained this time is the smallest of the matching degrees so far, and the moving amount is set to the moving amount variable D. In the above case, it is determined that the coincidence R obtained this time is the second smallest among the coincidences up to now, and the routine goes to step SP47.

The above processing is repeated for all points in the search window W (step SP
48), the minimum value R of the matching degree in the search window W
The first and second minimum values R2 can be obtained. Then, at the end of the repetition processing, the movement vector variables (dxm, dy)
m) as a result of block matching at point P,
In addition, the difference R2 between the minimum coincidence value R1 and the second minimum value R2.
R1 is obtained as an evaluation value Cp for the result of block matching (step SP49).

For example, when the image in the search window W has a small change in luminance, or when the same pattern is repeated, and the result of block matching is uncertain, the matching degree in the search window W Since the difference between the minimum value and the second minimum value becomes small, the obtained evaluation value Cp becomes a small value. Therefore, by comparing this evaluation value Cp with an appropriate threshold value, it can be determined whether or not the obtained movement vector is reliable.
When the minimum matching degree is obtained at a plurality of positions, the evaluation value Cp obtained according to the above procedure is zero.

As described above, in step SP23, the obtained evaluation value Cp is compared with the preset threshold value T, and it is determined whether or not the obtained movement vector is reliable. If the vector is reliable, the motion vector variable (dxm, dym) is recorded as the motion vector at the point P in step SP24.

As described above, according to the first embodiment, the evaluation value Cp is calculated as the difference between the minimum value of the degree of coincidence obtained as a result of block matching and the second minimum value, and this is used as the threshold value. Since the comparison is made, it is possible to evaluate the certainty of the calculated movement vector, and to prevent the output of an uncertain movement vector.

As a result, the accuracy of the processing result of the processing using the movement vector can be improved. For example, a moving object detection device that detects and extracts a moving object from image information can improve the extraction accuracy of a moving object.

(B) Second Embodiment Next, a second embodiment of the movement vector calculation method and apparatus according to the present invention will be described in detail with reference to the drawings.

The second embodiment is an improvement on the first embodiment. In the first embodiment, the minimum value and the second minimum value of the coincidence are obtained while sequentially comparing the coincidences obtained by the block matching, and the minimum value and the second minimum value of the finally obtained coincidence are obtained. In practice, the matching value near the position where the best match is given often indicates a value close to the best value. In such a case, even if the obtained motion vector is good, the evaluation value may be low. For this reason, it is practical to find a position that gives the best match once, and then devise the vicinity of the position so as not to use it for calculating the evaluation value.

Therefore, in the second embodiment, the matching score calculated in the search window W is temporarily stored in the matching score storage device, and the best matching score is obtained in a portion excluding the vicinity of the position giving the best match. The difference from the overall best match was calculated as an evaluation value. In other words, the area for obtaining the second minimum value in the first embodiment is limited.

FIG. 5 is a block diagram showing the configuration of the system according to the second embodiment, in which the same or corresponding parts as those in FIG. 3 are denoted by the same reference numerals. As is clear from the comparison between FIG. 5 and FIG. 3, a coincidence storage unit 55 is further added to the system configuration of the first embodiment.

As shown in FIG. 7, the coincidence degree storage unit 55 stores the coincidence degree value corresponding to the coordinate deviation value (dx, dy) from the point P with the coordinate deviation value (dx, dy) from the point P. dy) is stored as a two-dimensional array.

FIG. 6 is a flowchart showing the contents of the processing in the movement vector calculation section 14 of the second embodiment.

As is clear from the comparison between FIG. 1 and FIG.
In the second embodiment, a block matching process (step SP60) and an evaluation value calculation process (step SP61) are clearly divided, and the processing contents are different from those of the first embodiment. Other processes are the same as in the first embodiment. Therefore, the content of the block matching process and the evaluation value calculation process of the second embodiment will be described below.

In the block matching process (step SP60) of the second embodiment, block matching is performed on the processing target block area determined in step SP21, and the degree of coincidence obtained for each point in the search window W is calculated as follows. As shown in FIG. 7, as a two-dimensional array,
A process for storing in the coincidence storage unit 55 is performed.

In the evaluation value calculation processing (step SP61) of the second embodiment, although the flowchart is not shown, first, the matching degree stored in the matching degree storage unit 55 is calculated.
The position that shows the best match (the minimum value of the degree of matching) is searched, and then the position that shows the best match is searched for in a portion excluding its vicinity. Then, the difference between the two degrees of coincidence is calculated as an evaluation value.

Hereinafter, this procedure will be described with reference to FIG.
In the following description, it is assumed that the degree of coincidence of Expression (2) is used, as in the description of the first embodiment.

In FIG. 8, (dxm, dym) is a position (correctly, a coordinate deviation value from the point P) at which the smallest one of the stored coincidences is given. The search for this position is performed by sequentially scanning the coincidence in the coincidence storage 55 in an appropriate procedure. Next, the minimum value of the degree of coincidence R (dx, dy) in a range (a portion shown by oblique lines in FIG. 8) apart from the position (dxm, dym) by a certain value or more (to the second minimum value of the first embodiment) Applicable). And this value R
(Dx, dy) and the overall minimum value R (dxm, dym)
Is defined as an evaluation value Cp. In this case, the evaluation value Cp is
It is obtained by equation (3). Note that the minimum value search range W '(the shaded portion in FIG. 8) in the expression (3) is a range given by the expression (4), and dx or dy is smaller than dxm or dym from the fixed value n pixels, respectively. It is an area far away.

Cp = min {R (dx, dy) -R (dxm, dym)} (3) W ′ = {(dx, dy) || dx-dxm |> n and | dx-dxm |> n 4 (4) Also according to the second embodiment, the evaluation value Cp is calculated as the difference between the minimum value of the degree of coincidence obtained as a result of block matching and the second minimum value, and is compared with the threshold value. Therefore, the reliability of the calculated movement vector can be evaluated, and the output of an uncertain movement vector can be prevented.

Here, according to the second embodiment, the second minimum value of the coincidence is searched for from the area excluding the vicinity of the position of the overall minimum value. The certainty can be more appropriately evaluated. For example, almost the same degree of coincidence is obtained in the vicinity of the position where the minimum value of the degree of coincidence is given, and despite the fact that a reliable motion vector has been obtained, the evaluation value is equal to or less than the threshold value. It is possible to avoid a situation in which the information is not recorded.

(C) Third Embodiment Next, a third embodiment of the motion vector calculation method and apparatus according to the present invention will be described in detail with reference to the drawings.

The third embodiment is also an improvement on the first embodiment. As described above, in block matching, the position that best matches a block on one image must be uniquely determined on the other image. However, for a portion where the change in brightness is small, the degree of coincidence is substantially the same at each position inside the portion, so that a corresponding point cannot be uniquely determined. Therefore, in the first embodiment, an uncertain movement vector is eliminated by calculating an evaluation value for the result of block matching and comparing the evaluation value with a threshold value. However, since this evaluation value is obtained by using the result of actual block matching, the same processing is required for an unnecessary movement vector. In the third embodiment, a function of determining whether or not to perform block matching in advance is provided to reduce the processing amount as a whole.

The configuration of the system according to the third embodiment is the same as that of FIG. 3 described above, and the processing of the movement vector calculation unit 14 is different from that of the first embodiment.

Therefore, the processing contents of the movement vector calculation unit 14 according to the third embodiment will be described below with reference to the flowchart shown in FIG.

As is clear from the comparison between FIG. 9 and FIG. 1, the processing for determining the processing target block area in step SP90 is different from that in the first embodiment.
A comparison process between the feature value of P91 and the threshold value F is newly added, and other processes (steps SP22 to SP25)
Is the same as in the first embodiment. So, below,
The process of determining the processing target block area in step SP90 and the process of comparing the feature amount with the threshold F in step SP91 will be described.

In the process of determining the block area to be processed (step SP90) in the third embodiment, not only the block area Bp to be processed is determined, but also the feature quantity representing the feature of the image content of the block area Bp is calculated. I do.

Here, the feature quantity can determine whether or not the image content has sufficient complexity, which is suitable for calculation of a motion vector (block matching). In other words, it is possible to determine whether the luminance change in the block area Bp is poor and is not suitable for the calculation of the motion vector, or whether the luminance change is more than a certain level and is suitable for the calculation of the motion vector. The feature amount is, for example, a total sum F of the luminance change of the pattern in the block represented by Expression (5).
(Bp) can be applied. Note that the sum ΣΣ in the equation (5) is such that x is from −L to L−1, and y is −L.
To L-1.

F (Bp) = {| Fi (Xp + x + 1, Yp + y) -Fi (Xp + x, Yp + y) | + | Fi (Xp + x, Yp + y + 1) -Fi (Xp + x, Yp + y) |} (5) Next, in step SP91, the obtained feature value F (Bp) representing the complexity is compared with a threshold value F set in advance to determine whether or not to perform block matching. When the feature value F (Bp) is equal to or greater than the threshold value F, the process after the block matching process is performed on the block area Bp to obtain a moving vector. On the other hand, if the feature value F (Bp) is smaller than the threshold value F, the process proceeds to the next process block area setting process without performing block matching.

According to the third embodiment as well, the evaluation value Cp is calculated as the difference between the minimum value of the coincidence obtained as a result of block matching and the second minimum value, and this is compared with the threshold value. , The reliability of the calculated movement vector can be evaluated, and the output of an uncertain movement vector can be prevented.

In addition to this, according to the third embodiment,
Before performing block matching, a feature area representing the complexity of image content included in the processing target block area is calculated, and this is compared with a threshold to determine whether to perform block matching. Therefore, for example, for a region such as a road surface portion of a road image or a wall surface portion of an indoor image, in which a reliable movement vector is unlikely to be obtained by block matching due to a small change in image content (pattern), it is determined in advance. Block matching can not be performed, and the amount of calculation as a whole can be reduced.

(D) Fourth Embodiment Next, a fourth embodiment of the movement vector calculation method and apparatus according to the present invention will be described in detail with reference to the drawings.

In the fourth embodiment, a multi-resolution image is generated, and a motion vector is calculated with reference to the multi-resolution image.

FIG. 10 is a block diagram showing the configuration of a system according to the fourth embodiment, in which the same or corresponding parts as those in FIG. 3 are denoted by the same reference numerals. FIG.
As is clear from the comparison between FIG. 0 and FIG. 3, the multi-resolution image generation unit 16a is added to the system configuration of the first embodiment.
And 16b.

Each multi-resolution image generating section 16a, 16b
Generates multi-resolution images for the images stored in the corresponding frame buffers 13a and 13b. Therefore, the movement vector calculation unit 14 of the fourth embodiment calculates the movement vector with reference to the multi-resolution image, as described later.

Here, the multi-resolution image refers to a series of images obtained by reducing the size of the original image at various scales as shown in FIG. For example, by setting the scale to 、 and setting the average luminance value of four pixels adjacent in the vertical and horizontal directions of the original image to the luminance of one pixel of the reduced image, an image smaller by one step is generated. Is performed as a source image, and a multi-resolution image is generated by repeating such processing. In addition, the creation of a multi-resolution image is described in Reference 2 “PJBurt,“ Smart Sensing within a Pyra
mid Vision Machine '', Proceeding IEEE, Vol.76, N
o.8, pp.1006-1015, 1988].

Here, when blocks of the same size in the vertical and horizontal directions are considered for the original image and the reduced image, the blocks of the reduced image correspond to the reciprocal of the square of the reduced scale in the original image. Usually, the change (complexity) of the image content is large.

FIG. 12 is a flowchart showing a processing procedure of the movement vector calculation unit 14 according to the fourth embodiment.

The moving vector calculation section 14 repeats the processing shown in FIG. 12 in order from the low resolution image to the higher resolution image (step SP121). The processing of the first low-resolution image is slightly different from that shown in FIG. 12, and is the same as the above-described processing shown in FIG. That is, when the evaluation value Cp is equal to or smaller than the threshold value T, the movement vector recording operation is not performed.

In the processing of each resolution image, the evaluation value Cp
The process up to the comparison of the threshold value T with the threshold value T is the same as in the first embodiment. Here, when the evaluation value Cp is larger than the threshold value T, the movement vector obtained in the process of the current step SP22 is recorded.
In the following case, among the motion vectors obtained in advance by block matching with a lower resolution image, a motion vector relating to a block including the processing block is called (step SP120), and this is set as a motion vector in the processing block. Record.

Also in the fourth embodiment, the evaluation value Cp is calculated as the difference between the minimum value of the degree of coincidence obtained as a result of block matching and the second minimum value, and is compared with the threshold value. , The reliability of the calculated movement vector can be evaluated, and the output of an uncertain movement vector can be prevented.

In addition to this, according to the fourth embodiment,
Since the multi-resolution image is created and the motion vector at each resolution is sequentially obtained, the number of pixels from which the motion vector can be obtained can be increased. That is, in the fourth embodiment, the moving vector in the low-resolution image is also calculated, and one block in the low-resolution image includes a wider range than the block in the original image. If another pattern is included, a more reliable movement vector can be calculated by including a pattern change or a pattern having no periodicity in the block. Therefore, the motion vector obtained in the image of the current resolution can be supplemented by the motion vector obtained in the low resolution image, and more motion vectors can be obtained.

(E) Other Embodiments As described above, the use of the method and apparatus for calculating a motion vector according to the present invention makes it possible to obtain more motion vectors with higher accuracy than conventional motion vectors obtained by block matching. Such a method and apparatus can be applied to various apparatuses using a movement vector such as observation of a traffic flow and an air flow, tracking of a moving object, and the like, and their performance can be improved.

In the description of each embodiment, the size of the block area to be processed is assumed to be a square having 2L + 1 pixels on a side, but the shape may be arbitrary. At this time, a representative point may be set at an appropriate place in the block, and a movement vector may be output as a movement amount of the representative point before and after matching.

The third embodiment presupposes the processing of the first embodiment, and incorporates the calculation of the characteristic amount and the determination as to whether or not to execute the block matching based on the characteristic amount in the processing. However, the processing of the second embodiment may be presupposed, and the processing may include calculation of a feature amount and determination of whether or not to execute block matching based on the feature amount.

Further, in the fourth embodiment, the motion vector is calculated with reference to the multi-resolution image on the premise of the processing of the first embodiment. On the premise of the processing of the third embodiment, the calculation of the movement vector with reference to the multi-resolution image may be performed.

Further, in each of the above embodiments,
Although the method of calculating the movement vector from the image with the oldest imaging time to the new image has been described, the present invention can be applied to the calculation of the movement vector from the image with the new imaging time to the old image.

In the fourth embodiment, the motion vector is obtained by processing in order from the low-resolution image to the high-resolution image. Conversely, the motion vector of the captured image is obtained from the beginning. The calculation may be performed, and the motion vector for the low-resolution image may be calculated for the block area that cannot be calculated.

Further, in each of the above embodiments, the evaluation value Cp of the block matching is determined by comparing the minimum value of the coincidence with the second value.
Although the difference from the minimum value is shown, another evaluation value may be used. For example, it may be a ratio between the minimum value of the degree of coincidence and the second minimum value, and an average of the second minimum value and the third minimum value;
The difference from the minimum value may be used.

Further, in each of the above embodiments,
Although the moving vector with a low evaluation value Cp is not recorded (output), the moving vector with a low evaluation value Cp is also recorded together with the evaluation value Cp, and the use or non-use of the moving vector with a low evaluation is left to a subsequent device. You may do it.

[0087]

As described above, according to the movement vector calculation method according to the first invention and the movement vector calculation apparatus according to the third invention, the block area set in one image is scanned on the other image. While calculating the degree of coincidence indicating the goodness of the coincidence at each position, and forming the evaluation value of the degree of certainty of the movement vector based on the plurality of values on the upper side with the higher degree of coincidence. Can be obtained.

Further, according to the movement vector calculation method according to the second invention and the movement vector calculation apparatus according to the fourth invention, a multi-resolution image is created for each of the two input images, and one of the images is created. While scanning the block area set in the above on the other image, the degree of coincidence representing the goodness of coincidence at each position is calculated, and the likelihood of the movement vector is determined based on a plurality of values on the upper side where the degree of coincidence is high. If the evaluation value is formed and the calculated evaluation value of the likelihood of the movement vector indicates that it is likely, the movement vector is obtained by the position where the best value of the degree of coincidence is given, and the evaluation value of the certainty is If it is not probable that the low-resolution image of the multi-resolution image has a block area including the block area, Then, a movement vector is obtained by the same processing, or a movement vector for a low-resolution image obtained in advance by the same processing as for a captured image is set as a movement vector of the block area. An evaluated motion vector can be obtained, and a motion vector obtained for a low-resolution image is output for a block region where a motion vector cannot be obtained for an original image. It is possible to increase the number of block areas that take values.

[Brief description of the drawings]

FIG. 1 is a processing flowchart of a moving block calculation unit according to the first embodiment.

FIG. 2 is an explanatory diagram of block matching.

FIG. 3 is a block diagram illustrating a system configuration according to the first embodiment.

FIG. 4 is a flowchart illustrating an evaluation value calculation process according to the first embodiment.

FIG. 5 is a block diagram illustrating a system configuration according to a second embodiment.

FIG. 6 is a processing flowchart of a moving block calculation unit according to the second embodiment.

FIG. 7 is an explanatory diagram illustrating storage contents of a coincidence storage unit according to the second embodiment;

FIG. 8 is an explanatory diagram of an evaluation value calculation method according to the second embodiment.

FIG. 9 is a processing flowchart of a moving block calculation unit according to the third embodiment.

FIG. 10 is a block diagram illustrating a system configuration according to a fourth embodiment.

FIG. 11 is an explanatory diagram of a multi-resolution image.

FIG. 12 is a processing flowchart of a moving block calculation unit according to the fourth embodiment.

[Explanation of symbols]

10 Video camera, 11 A / D converter, 12 Changeover switch, 13a, 13b Frame buffer, 14
Movement vector calculation unit, 16a, 16b... Multi-resolution image generation unit, 55..

Claims (12)

[Claims] 1. A moving vector calculating method for calculating a moving vector from two images captured at different times, wherein a block area set in one image is scanned on the other image while matching a position in each position. A first process of calculating a degree of coincidence representing goodness; and a second process of forming an evaluation value of the likelihood of a movement vector based on a plurality of values on the higher side having a higher degree of coincidence. A moving vector calculation method. 2. The method according to claim 1, further comprising: a third process of calculating a feature amount representing the complexity of the image within a block area set in one image and comparing the calculated feature amount with a threshold value. The moving vector calculation method according to claim 1, wherein when the complexity of the image in the set block area is small, the calculation of the movement vector for the block area is not performed. 3. A moving vector calculating method for calculating a moving vector from two images captured at different times, a fourth process of creating a multi-resolution image for each of the two input images, A first process of calculating the degree of coincidence representing the degree of coincidence at each position while scanning the block area set in the image of the other image on the other image, and based on a plurality of upper values having a higher degree of coincidence A second process for forming an evaluation value of the likelihood of the movement vector, and, if the calculated evaluation value of the likelihood of the movement vector indicates that it is likely, the position at which the best value of the coincidence is given And obtain a movement vector, and if the above-mentioned certainty evaluation value does not indicate that it is likely,
For the low-resolution image of the multi-resolution image, for a block area including the block area, a movement vector is obtained by the same processing as that for the captured image, or a low-resolution image previously obtained by the same processing as for the captured image. A fifth process of setting a motion vector of an image as a motion vector of the block area. 4. A method for calculating a feature amount representing the complexity of an image in a block area set in one image and comparing the calculated feature amount with a threshold value, wherein the comparison result obtained by the third processing is In the case where the image indicates that the complexity of the image in the block area is small, the first processing is not performed on the block area, and the low-resolution image of the multi-resolution image includes the block area. About the block area,
A motion vector is obtained by the same processing as for the captured image, or a motion vector for a low-resolution image obtained in advance by the same processing as for the captured image is set as the motion vector of the block area. Item 3. The movement vector calculation method according to item 3. 5. The method according to claim 1, wherein the second processing is performed for the top two with a high degree of coincidence.
The moving vector calculation method according to claim 1, wherein a difference between the two values is formed as an evaluation value of the likelihood of the moving vector. 6. The movement according to claim 5, wherein the value having the second highest degree of coincidence in the second processing is detected from outside a predetermined area around a position where the degree of coincidence is first highest. Vector calculation method. 7. A movement vector calculation apparatus for calculating a movement vector from two images captured at different times, wherein a block area set in one image is scanned on the other image while matching the position at each position. Matching degree calculating means for calculating a matching degree representing goodness, and moving vector evaluation value calculating means for forming an evaluation value of the likelihood of a moving vector based on a plurality of higher-order values having a high matching degree A motion vector calculation device characterized by the above-mentioned. 8. An image feature amount calculating means for calculating a feature amount representing the complexity of the image in a block area set in one image and comparing the calculated feature amount with a threshold value, wherein the comparison by the image feature amount calculating means is performed. 8. The motion vector calculating apparatus according to claim 7, wherein when the result indicates that the complexity of the image in the set block area is small, the calculation of the motion vector for the block area is not executed. 9. A moving vector calculating device for calculating a moving vector from two images captured at different times, a multi-resolution image generating means for generating a multi-resolution image for each of the two input images, A coincidence calculating means for calculating a degree of coincidence at each position while scanning the block area set in one image on the other image; and a plurality of higher-order values based on the higher coincidence. A moving vector evaluation value calculating means for forming an evaluation value of the likelihood of the moving vector; and, if the calculated evaluation value of the likelihood of the moving vector indicates that the likelihood is certain, the best value of the degree of coincidence is determined. In the case where the movement vector is obtained according to the given position and the evaluation value of the certainty is not shown to be likely,
For the low-resolution image of the multi-resolution image, for a block area including the block area, a movement vector is obtained by the same processing as that for the captured image, or a low-resolution image previously obtained by the same processing as for the captured image. A moving vector determining unit that determines a moving vector of the image as a moving vector of the block area. 10. A feature amount representing the complexity of an image in a block area set in one image is calculated,
The image processing apparatus further includes an image feature amount calculating unit that compares the moving vector with the threshold value. If the comparison result by the image feature amount calculating unit indicates that the complexity of the image in the set block area is small, the moving vector determining unit However, without calculating the motion vector for this block area, for the low-resolution image of the multi-resolution image, for the block area including the block area, by obtaining the motion vector by the same process as for the captured image, or ,
10. The movement vector calculation device according to claim 9, wherein a movement vector for a low-resolution image obtained in advance by the same processing as that for a captured image is used as a movement vector of the block area. 11. The moving vector evaluation value calculating means,
8. The method according to claim 7, wherein a difference between the top two values having a high degree of coincidence is formed as an evaluation value of the likelihood of the movement vector.
The movement vector calculation device according to any one of claims 10 to 10. 12. The moving vector evaluation value calculating means,
The movement vector calculation device according to claim 11, wherein a value having the second highest degree of coincidence is detected from outside a predetermined area around a position where the degree of coincidence is first highest.