JP2002315007A - Image processing equipment and method, recording medium, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To calculate a moving vector between a standard image and a reference image, without producing a large number of interpolated image, i.e., without increasing the calculation cost. SOLUTION: A moving vector detecting part 15 obtains the standard image and the reference image from an image memory 14, calculates the moving vectors of the standard image and the reference image, and outputs the vectors to an image-correcting part 16 via a bus 18. The part 16 reads the reference image from the image memory 14, moves the read reference image in parallel by the amount corresponding to the moving vector, and convents the image into a corrected image, which is supplied to an output part 17 via the bus 18. The output part 17 carries out buffering of the corrected image supplied from the image correcting part 16 via the bus 18 and output the image to a display unit 3 of the last stage at a prescribed rate.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to an image processing apparatus and method, a recording medium, and a program.
The present invention relates to an image processing apparatus and method, a recording medium, and a program suitable for use in detecting a motion vector between a plurality of continuously shot images.

[0002]

2. Description of the Related Art Conventionally, when a video camera is photographed by holding it with a hand without being fixed by a tripod or the like, a moving image photographed due to camera shake vibrates up, down, left and right, and is difficult to see when reproduced. There was a problem that it became a screen. As a method for solving such a problem, a process of converting a moving image in which the screen vibrates to a moving image in which the screen does not vibrate, that is, a so-called stabilizer process is known.

[0003] In a technique for compression-encoding a moving image, a compression process using motion compensation is known.

[0004] As a technique common to the stabilizer processing and the compression processing by motion compensation, there is a technique called a block matching method. The block matching method is a technique of detecting a corresponding pixel between two images (a reference image and a reference image) shot continuously, and calculating a motion amount (a motion vector between the images).

[0005] In the stabilizer processing, a plurality of still images constituting a moving image are obtained by a block matching method.
For example, the first still image is set as the reference image, the still images captured thereafter are set as the reference images, the motion vectors of the reference image and the reference image are calculated, and the entire reference image is translated by the motion vector. You. Thereby, the moving image whose screen vibrates is corrected to a moving image whose screen does not vibrate.

In the compression processing based on motion compensation, a motion vector between a reference image and a reference image is calculated by a block matching method, and the data amount is reduced by reproducing the reference image using the reference image and the motion vector. Let me.

A conventional block matching method will be described with reference to a reference image A and a reference image B. Hereinafter, a pixel located at the coordinates (i, j) of the reference image A is referred to as a pixel A (i, j).
j). Similarly, the coordinates (i, j) of the reference image B
Is described as a pixel B (i, j). The coordinates (i, j) are integers indicating the horizontal position or the vertical position of the image, respectively.

First, an evaluation function for evaluating the degree of similarity (hereinafter referred to as appropriateness) between a reference image A and an image obtained by translating a reference image B by a vector (x, y) includes:
For example, it is calculated using the following equation (1).

(Equation 1) ... (1)

Since equation (1) is an error function,
The smaller the value, the higher the appropriateness.

Here, all values assumed to be motion vectors between the reference image A and the reference image B are substituted for the vector (x, y). Among them, the vector (x, y) having the highest appropriateness value is the most appropriate vector (x
_fit , y _fit ). That is, an image obtained by translating the reference image A by the most appropriate vector (x _fit , y _fit ) becomes an image substantially equal to the reference image B.

By the way, the most appropriate vector (x _fit ,
Since y _fit ) is not always an integer, the value of the vector (x, y) must take decimals into account. However, the reference image B has a pixel B represented by an integer.
Since only (i, j) exists, in order to apply the non-integer vector (x, y) to the operation of equation (1),
It is necessary to interpolate non-existent pixels having non-integer coordinates using a reference image B including pixels of coordinates represented by real integers to generate an interpolated image B 'including interpolated pixels.

[0012] For example, as shown in FIG. 1, pixel P nonexistent in the reference image B 'is the following equation pixels P ₁ to P ₄ actually exists on the vertical and horizontal coordinates (hereinafter, referred to as interpolation formula )
And interpolate. Pixel P '= B (x + i, y + j) = B (m + p + i, n + q + j) = (1-p) (1-q) B (m + i, n + j) + (1-p) qB (m + i, n + 1 + j) + p (1 −q) B (m + 1 + i, n + j) + pqB (m + 1 + i, n + 1 + j)

Here, m and n are integers, and p and q are 0
It is a decimal number less than 1 or more. Hereinafter, p and q are described as phases.

In order to set the accuracy of the motion vector calculated by the block matching method to, for example, 1/128, the values of x and y of the vector (x, y) are respectively reduced to 1/128.
It is necessary to change the value in units of 128 and substitute it into equation (1). For that purpose, 16384 (= 128 × 12
8) It is necessary to generate interpolated images for different phases.

FIG. 2 shows the conventional block matcher described above.
6 is a flowchart showing a processing procedure of a logging method. Sand
In step S1, the reference image B is used to generate an interpolation image.
Image B ' ₁Or B '_k(In this case, k = 16384)
To achieve. In step S2, the reference image A and each interpolated image
The appropriateness with respect to the image B 'is calculated using Expression (1). Stay
In step S3, the reference image A calculated in step S2
And the maximum value of the appropriateness of each interpolated image B 'is detected,
The corresponding vector (x, y) is obtained by using the reference image A and the reference image
Let B be the motion vector.

[0016]

As described above,
In the conventional block matching method, since an operation for generating a large number of interpolated images B ′ _{1 to} B ′ _k is required in step S1, the operation cost increases, and the time required to complete a series of processes increases. There were challenges.

The present invention has been made in view of such a situation.
And many interpolated images B ' ₁Or B '_kGenerate
Without, that is, without increasing the computational cost,
A motion vector between the reference image and the reference image can be calculated
The purpose is to be.

[0018]

An image processing apparatus according to the present invention comprises a setting means for setting a search range of a motion vector, a dividing means for dividing a search range into a small area, and an extreme value of an evaluation value in a small area. First calculating means for calculating; second calculating means for calculating evaluation values respectively corresponding to the four end points of the minute area; extreme values of the evaluation values calculated by the first calculating means; Determining means for determining an optimum evaluation value corresponding to a micro area from the optimum evaluation value among the four evaluation values respectively corresponding to the four endpoints calculated by the means; and two determining means respectively corresponding to two different micro areas. By comparing the optimal evaluation values, a holding unit that holds one of the more optimal evaluation values, and a motion vector corresponding to a position in a search range corresponding to the optimal evaluation value finally held by the holding unit, Calculation to calculate Characterized in that it comprises a stage.

The first and second calculating means calculate the auto-correlation value of the reference image, the auto-correlation value of the reference image, and the cross-correlation value of the reference image and the reference image by using existing pixels. Then, each calculation result is multiplied by a coefficient consisting of a decimal number, and each multiplication result is added or subtracted to calculate an evaluation value.

According to the image processing method of the present invention, a setting step of setting a search range of a motion vector, a dividing step of dividing the search range into a small area, and a first calculation for calculating an extreme value of an evaluation value in the small area Steps and 4 of micro area
A second calculation step for calculating an evaluation value corresponding to each of the end points; an extreme value of the evaluation value calculated in the processing in the first calculation step;
A determination step of determining an optimum evaluation value corresponding to a minute area from an optimum evaluation value among four evaluation values respectively corresponding to end points, and comparing two optimum evaluation values respectively corresponding to two different minute areas. And a calculation step of calculating a motion vector corresponding to a position in a search range corresponding to the optimum evaluation value finally held in the processing of the holding step. And a step.

According to the recording medium program of the present invention, a setting step of setting a search range of a motion vector, a dividing step of dividing the search range into minute areas, and a first step of calculating an extreme value of an evaluation value in the minute areas. A calculation step, and a second step of calculating evaluation values respectively corresponding to the four end points of the minute area.
Calculation step, the extreme value of the evaluation value calculated in the processing of the first calculation step, and the optimum evaluation of the four evaluation values respectively corresponding to the four end points calculated in the processing of the second calculation step A determination step of determining an optimum evaluation value corresponding to a micro area from the values, and a holding step of comparing two optimum evaluation values respectively corresponding to two different micro areas to hold one of the more optimum evaluation values And a calculating step of calculating a motion vector corresponding to a position in the search range corresponding to the optimum evaluation value finally held in the processing of the holding step.

The program according to the present invention includes a setting step of setting a search range of a motion vector, a dividing step of dividing the search range into a small area, and a first calculation step of calculating an extreme value of an evaluation value in the small area. A second calculation step for calculating evaluation values respectively corresponding to the four end points of the micro area, an extreme value of the evaluation value calculated in the processing in the first calculation step, and a calculation in the processing in the second calculation step. A determination step of determining an optimum evaluation value corresponding to a micro area from the optimum evaluation values among the four evaluation values respectively corresponding to the four end points; and two optimum evaluation values respectively corresponding to two different micro areas. In comparison, a holding step of holding one of the more optimum evaluation values, and a position in the search range corresponding to the optimum evaluation value finally held in the processing of the holding step, Characterized in that to execute can a calculation step of calculating a vector computer.

In the image processing apparatus, method, and program according to the present invention, a search range of a motion vector is set, and the search range is divided into minute areas. The extreme value of the evaluation value in the micro area is calculated, and the evaluation value corresponding to each of the four end points of the micro area is calculated. Also, the extreme values of the calculated evaluation values and the four end points corresponding to the four calculated end points, respectively.
The optimum evaluation value corresponding to the micro area is determined from the optimum evaluation value of the two evaluation values. Further, two optimal evaluation values respectively corresponding to two different micro areas are compared, one of the more optimal optimal evaluation values is held, and a search range corresponding to the finally retained optimal evaluation value is held. A motion vector is calculated according to the position.

[0024]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A configuration example of an image processing apparatus according to an embodiment of the present invention will be described with reference to FIG.
The image processing device 2 converts a moving image (still images continuously shot) supplied from the photographing device 1 into a moving image whose screen does not vibrate, and outputs the moving image to the display device 3.

In the image processing apparatus 2, the control unit 11
It controls each unit of the image processing apparatus 2 via the bus 18 in accordance with information of a user operation input from the operation input unit 12. The operation input unit 12 receives a user operation and outputs information on the operation to the control unit 11 via the bus 18. The input unit 13 outputs a moving image input from the imaging device 1 to the image memory 14 via the bus 18.
The image memory 14 stores a moving image (still images continuously shot) supplied from the input unit 13 via the bus 18.

The motion vector detecting section 15 includes the image memory 1
4 sequentially reads out the still images constituting the moving image stored therein. For example, the first image is set as the reference image A, and the images to be read thereafter are sequentially set as the reference images B, and the motion vectors of the reference image A and the reference image B are sequentially set. The calculated motion vector is output to the image correction unit 16 via the bus 18.

The image correction section 16 reads out the reference image A from the image memory 14 and supplies the reference image A as it is to the output section 17 via the bus 18 as a corrected image A '. Further, the image correction unit 16 reads the reference image B from the image memory 14 and
The read reference image B is translated in parallel by an amount corresponding to the corresponding motion vector, converted into a corrected image B′n (n = 1, 2,...), And supplied to the output unit 17 via the bus 18. The output unit 17 buffers the corrected images A ′ and B′n supplied from the image correction unit 16 via the bus 18 and outputs the buffered images to the subsequent display device 3 at a predetermined rate.

Next, the motion vector detecting unit 15 changes from m in the horizontal direction to (m + 1) and from n in the vertical direction to (n
The concept of the process for finding the optimal phase (p, q) in the range up to +1) will be described.

In the equation (1) for calculating the appropriateness between the reference image A and the image obtained by translating the reference image B by the vector (x, y), B (x + i, y + j) is used as the interpolation formula. Equation (1) can be transformed into the following equation (2).

(Equation 2) ... (2)

The right side of the equation (2) can be transformed into the following equation (3) using the phase (p, q) as a variable. The phase (p, q) which is a variable is a decimal number greater than or equal to 0 and less than 1. Let α be the value shown by equation (3).

(Equation 3) ... (3)

Now, the object is to detect the vector (x, y) having the highest value of the appropriateness, that is, the vector (x, y) having the minimum value calculated by the equation (1). , The extreme value of the equation (3) in which the phase (p, q) is a variable may be obtained.

That is, the equation (3) is replaced by the phase (p,
q), and the phase (p, q) at which these values become 0 may be obtained as shown in the following equations (4) and (5). Hereinafter, the appropriateness corresponding to the phase (p, q) at which the expression (3) becomes an extreme value is described as appropriateness V5.

[0033]

(Equation 4) ... (4)

[0034]

(Equation 5) ... (5)

However, since it is conceivable that the equation (3) is a monotonically increasing function (or a monotonically decreasing function) for the phase (p, q) as a variable, the variable p =
The value obtained by substituting 0 and q = 0 is referred to as appropriateness V1, the value obtained by substituting variables p = 0 and q = 1 in equation (3) is referred to as appropriateness V2, and the variables p = 1 and q = The value obtained by substituting 0 into the appropriateness degree V3
And a value obtained by substituting the variables p = 1 and q = 1 into the equation (4) is prepared as the appropriateness V4.

Then, the most appropriate value, that is, the minimum value is selected from the appropriateness levels V1 to V5, and the corresponding variables p and q are determined.

Thus, from m in the horizontal direction to (m +
Up to 1), the optimal phase (p, q) in the range from n to (n + 1) in the vertical direction can be obtained.

As is clear from Equations (3) to (5), Equation (3) is obtained by calculating the value of the autocorrelation of the reference image A,
A value obtained by multiplying the value of the autocorrelation of the reference image B by a coefficient comprising the phase (p, q) is multiplied by a coefficient comprising the phase (p, q) by the value of the cross-correlation between the reference image A and the reference image B. It can be seen that the value is composed of Equations (4) and (5)
Is a value obtained by multiplying the value of the autocorrelation of the reference image B by a coefficient composed of the phase (p, q), and the value of the cross-correlation between the reference image A and the reference image B is represented by a coefficient composed of the phase (p, q). It can be seen that the value is composed of the multiplied value.

The order of the operation of the equation (3) is as follows.
After the value of the autocorrelation of the reference image A, the value of the autocorrelation of the reference image B, and the value of the cross-correlation between the reference image A and the reference image B are calculated, the coefficients composed of their phases (p, q) are multiplied. Then, the multiplied values are added (subtracted).

By performing the above-described process of determining the optimum phase (p, q) for all combinations of (m, n) in the search range, the reference image A and the reference image B The motion vector (x, y) between them can be obtained.
The motion vector detection processing by the motion vector detection unit 15 will be described with reference to the flowchart in FIG.

In step S11, the control unit 11
The still image constituting the moving image stored in the image memory 14 is input to the motion vector detecting unit 15. The motion vector detection unit 15 sets the input first image as the reference image A,
The next image is referred to as a reference image B.

In step S12, the control unit 11
Based on a user operation on the operation input unit 12, a search range of a motion vector is designated for the motion vector detection unit 15. That is, the minimum value m _min and the maximum value m _max of m, and the minimum value n _min and the maximum value n _{max of} the row n are specified. At this time, the user sets the reference image A and the reference image B
Since the deviation amount (the magnitude of the motion vector) from the above can be predicted to some extent, the search range is specified so as to encompass the value. Note that a preset initial value may be used as the search range.

In step S13, the motion vector detecting section 15 sets the variables m and n to their minimum values m _min ,
Initialize to n _min . Also, the motion vector detection unit 15
Substitutes a very large value for the appropriateness V _fit .

In step S14, the motion vector detecting section 15 sets (p = 0, q = 0), (p = 0, q =
1), (p = 1, q = 0), and (p = 1, q = 1)
Are respectively substituted into Expression (3) to calculate four types of α,
These are substituted for appropriate values V1 to V4.

In step S15, the motion vector detector 15 determines whether or not (p, q) that satisfies the equations (4) and (5) exists. If it is determined that (p, q) that satisfies the equations (4) and (5) exists, the process proceeds to step S16. In step S16, the motion vector detection unit 15 satisfies Expressions (4) and (5) (p,
The value of q) is substituted into equation (3) to calculate α, and α is substituted into an appropriate value V5.

If it is determined in step S15 that (p, q) that satisfies the equations (4) and (5) does not exist, the processing in step S16 is skipped.

In step S17, the motion vector detecting section 15 determines the most appropriate value (the minimum value in this case) among the appropriate values V1 to V5, and sets the determined appropriate value to the appropriate value V. And (p, q) corresponding to the appropriate value V
Holds the value of

For example, when the appropriateness V1 among the appropriate values V1 to V5 is the minimum, the appropriateness V is set to the appropriateness V1, and p = 0 and q = 0 are held. Further, for example, when the appropriateness V2 among the appropriate values V1 to V5 is the minimum, the appropriateness V is set to the appropriateness V2, and p = 0 and q = 1.
Is held. Further, for example, when the appropriateness V5 among the appropriate values V1 to V5 is the minimum, the appropriateness V is set to the appropriateness V5, and the expressions (4) and (5) are satisfied (p, p).
The value of q) is retained.

In step S18, the motion vector detection unit 15 determines whether or not the appropriateness V set in step S17 is more appropriate than the appropriateness _Vfit , that is, in this case, the appropriateness V is equal to the appropriateness _Vfit. It is determined whether it is smaller than. If it is determined that the appropriateness V is more appropriate than the appropriateness _Vfit , the process proceeds to step S19.

[0050] In step S19, the motion vector detecting section 15 substitutes a variable m to a variable m _fit, by substituting the variable n to the variable n _fit substitutes variable p to the variable p _fit, the appropriateness V _{fi t} The appropriateness V is substituted.

[0051] However, in step S18, if the appropriateness V is determined not to be appropriate than appropriateness V _fit, the process of step S19 is skipped.

In step S20, the motion vector detecting section 15 increments the variable m by one. In step S21, the motion vector detector 15, the variable m is equal to or a maximum value m or _max or less. If it is determined that the variable m is less than or equal to its maximum value _mmax ,
The process returns to step S14, and the subsequent processes are repeated. Thereafter, when it is determined in step S21 that the variable m is not _smaller than the maximum value _mmax , the process proceeds to step S22.

In step S22, the motion vector detection
The output unit 15 sets the variable m to its minimum value m _minReinitialize to
You. Further, the motion vector detecting unit 15 sets the variable n to 1
Increment. In step S23, the motion
The vector detecting unit 15 determines that the variable n is the maximum value n_maxBelow
It is determined whether or not there is. Variable n is its maximum value n_maxLess than
If it is determined that the process is
Therefore, the subsequent processing is repeated. Then, step S2
3, the variable n has its maximum value n_maxNot less
If it is determined, the process proceeds to step S24.

In step S24, the motion vector detector 15 calculates x = m _fit + p _fit , y = n _fit as the motion vector (x, y) between the reference image A and the reference image B.
+ To calculate the q _{f it.} The calculated motion vector (x,
y) is supplied to the image correction unit 16.

As described above, in the processing for detecting the motion vector (x, y) of the present invention, there is no processing for generating the interpolated image B ′ using the reference image B.
It is possible to reduce the calculation cost until the motion vector is obtained.

First, an optimal motion vector (m, n) = (m _tmp , n _tmp ) is obtained with integer precision, and then the search range is set to a value close to the motion vector (m _tmp , n _tmp ) (for example,
By limiting the range to (± 2 range), a vector that moves faster can be detected. In this case, in step S12, the minimum value m _min of the variable m is set to m _tmp -2,
The maximum value m _max of the variable m is set to m _tmp +2, and the minimum value n of the variable n
_It is sufficient to set _min to n _tmp -2 and the maximum value n _max of the variable n to n _tmp +2.

Incidentally, in the above description, the reference image A
And the image obtained by translating the reference image B by the vector (x, y) is used to calculate the evaluation function using Expression (1). ').

(Equation 6) ・ ・ (1 ')

Since the equation (1 ′) is a normalized cross-correlation function, it is evaluated that the larger the value is, the higher the appropriateness is.

When equation (1 ′) is used as the evaluation function, the above-described motion vector detection processing may be changed as follows.

That is, in step S 11, the control unit 11 causes the motion vector detection unit 15 to input a still image constituting a moving image stored in the image memory 14. The motion vector detecting unit 15 sets the input first image as the reference image A, and sets the next image as the reference image B.

In step S12, the control unit 11
Based on a user operation on the operation input unit 12, a search range of a motion vector is designated for the motion vector detection unit 15. That is, the minimum value m _min and the maximum value m _max of m, and the minimum value n _min and the maximum value n _{max of} the row n are specified.

In step S13, the motion vector detecting section 15 sets the variables m and n to their minimum values m _min ,
Initialize to n _min . Also, the motion vector detection unit 15
Substitutes a very small value for the appropriateness V _fit .

In step S14, the motion vector detecting section 15 sets (p = 0, q = 0), (p = 0, q =
1), (p = 1, q = 0), and (p = 1, q = 1)
Are respectively substituted into the following equation (3 ′) to calculate four types of β, and each is substituted into appropriate values V1 to V4. However, the equation (3 ′) is a further modified version of the following equation (2 ′) obtained by substituting the interpolation equation into the equation (1 ′).

(Equation 7) ... (2 ')

[0064]

(Equation 8) ... (3 ') However,

(Equation 9)

In step S15, the motion vector detection unit 15 calculates the following equation (4 ') in which the result of partial differentiation of equation (3') with p is 0, and the result of partial differentiation of equation (3 ') with q. Is determined whether or not (p, q) that satisfies the following expression (5 ′) in which

(Equation 10) ... (4 ')

[0066]

[Equation 11] ... (5 ')

Equations (4 ') and (5') are satisfied (p,
If it is determined that q) exists, the process proceeds to step S1
Proceed to 6. In step S16, the motion vector detection unit 15 satisfies the equations (4 ′) and (5 ′) (p, q).
Is substituted into the equation (3 ′) to calculate β, and β is set to an appropriate value V
Substitute for 5.

If it is determined in step S15 that (p, q) that satisfies the equations (4 ') and (5') does not exist, the processing in step S16 is skipped.

In step S17, the motion vector detecting section 15 determines the most appropriate value (the largest value in this case) among the appropriate values V1 to V5, and sets the determined appropriate value to the appropriate value V. And (p, q) corresponding to the appropriate value V
Holds the value of

In step S18, the motion vector detecting section 15 determines whether or not the appropriateness V set in step S17 is more appropriate than the appropriateness V _fit , that is, in this case, the appropriateness V is equal to the appropriateness V _fit It is determined whether it is greater than or equal to. If it is determined that the appropriateness V is more appropriate than the appropriateness _Vfit , the process proceeds to step S19.

[0071] In step S19, the motion vector detecting section 15 substitutes a variable m to a variable m _fit, by substituting the variable n to the variable n _fit substitutes variable p to the variable p _fit, the appropriateness V _{fi t} The appropriateness V is substituted.

[0072] However, in step S18, if the appropriateness V is determined not to be appropriate than appropriateness V _fit, the process of step S19 is skipped.

In step S20, the motion vector detecting section 15 increments the variable m by one. In step S21, the motion vector detector 15, the variable m is equal to or a maximum value m or _max or less. If it is determined that the variable m is less than or equal to its maximum value _mmax ,
The process returns to step S14, and the subsequent processes are repeated. Thereafter, when it is determined in step S21 that the variable m is not _smaller than the maximum value _mmax , the process proceeds to step S22.

In step S22, the motion vector detection
The output unit 15 sets the variable m to its minimum value m _minReinitialize to
You. Further, the motion vector detecting unit 15 sets the variable n to 1
Increment. In step S23, the motion
The vector detecting unit 15 determines that the variable n is the maximum value n_maxBelow
It is determined whether or not there is. Variable n is its maximum value n_maxLess than
If it is determined that the process is
Therefore, the subsequent processing is repeated. Then, step S2
3, the variable n has its maximum value n_maxNot less
If it is determined, the process proceeds to step S24.

In step S24, the motion vector detector 15 calculates x = m _fit + p _fit , y = n _fit as the motion vector (x, y) between the reference image A and the reference image B.
+ To calculate the q _{f it.}

As described above, the motion vector (x, y) can be calculated by using the normalized cross-correlation function as the evaluation function.

Note that the above description corresponds to the case where four pixels existing at the top, bottom, left and right of the interpolation point are used for interpolation of the reference image. However, the present invention uses another interpolation method (for example, a method in which a real pixel exists near the interpolation point). (When nine pixels are used).

Incidentally, the above-described series of processing can be executed by hardware such as the image processing apparatus 2, but can also be executed by software. When a series of processing is executed by software, a program constituting the software can execute various functions by installing a computer built into dedicated hardware or installing various programs. It is installed from a recording medium into a possible general-purpose personal computer or the like.

As shown in FIG. 15, the recording medium is a magnetic disk 41 (including a floppy disk) on which the program is recorded and an optical disk 42, which are distributed separately from the computer to provide the program to the user.
(CD-ROM (Compact Disc-Read Only Memory), DVD (Digita
l Versatile Disc), magneto-optical disc 43 (M
D (including Mini Disc), or a package medium including a semiconductor memory 44, etc., and also provided to a user in a state where the program is incorporated in a computer in advance. And a hard disk included in the hard disk drive 38.

In this specification, the steps for describing the program recorded on the recording medium are not limited to the processing performed in chronological order according to the described order, but are not necessarily performed in chronological order. Alternatively, it also includes individually executed processing.

In this specification, the system is
It represents the entire device composed of a plurality of devices.

[0082]

As described above, according to the image processing apparatus, method, and program of the present invention, the optimum value of the extreme value of the evaluation value in the minute area and the optimum value of the four evaluation values respectively corresponding to the four end points are obtained. From the evaluation values, the optimum evaluation value corresponding to the micro area is determined, the two optimum evaluation values corresponding to the two different micro areas are compared, one of the more optimum evaluation values is held, and finally, Since the motion vector is calculated corresponding to the position in the search range corresponding to the held optimal evaluation value, the reference image and the reference image are generated without generating many interpolation images, that is, without increasing the calculation cost. It is possible to calculate a motion vector between the image and the image.

[Brief description of the drawings]

FIG. 1 is a diagram for explaining a process of generating an interpolated image B ′ by interpolating a reference image B;

FIG. 2 is a flowchart illustrating an example of a conventional motion vector detection process.

FIG. 3 is a block diagram illustrating a configuration example of an image processing apparatus 2.

FIG. 4 is a flowchart illustrating a motion vector detection process by a motion vector detection unit 15;

FIG. 5 is a block diagram illustrating a configuration example of a general-purpose personal computer.

[Explanation of symbols]

2 image processing device, 11 control unit, 12 operation input unit, 13 input unit, 14 image memory, 15 motion vector detection unit, 16 image correction unit, 17 output unit,
31 CPU, 41 magnetic disk, 42 optical disk, 43 magneto-optical disk, 44 semiconductor memory

Continued on the front page F term (reference) 5C059 KK19 LB18 NN03 NN08 NN32 PP04 SS14 SS20 UA34 UA39 5J064 AA02 BB01 BB03 BB04 BC01 BD03 5L096 FA45 HA04

Claims (5)

[Claims] An image processing apparatus for detecting a motion vector between the reference image and the reference image based on an evaluation value for evaluating a degree of similarity between the reference image and the reference image; Setting means for setting a range; dividing means for dividing the search range into minute regions; first calculating an extreme value of the evaluation value in the minute regions
Calculating means for calculating the evaluation values respectively corresponding to the four end points of the minute area; the extreme value of the evaluation value calculated by the first calculating means; Determining means for determining an optimum evaluation value corresponding to the small area from an optimum evaluation value among the four evaluation values respectively corresponding to the four end points calculated by the calculating means; and two different small areas. Holding means for comparing the two optimum evaluation values respectively corresponding to the above, and holding one of the more optimum evaluation values; and the search range corresponding to the optimum evaluation value finally held by the holding means. Calculating means for calculating the motion vector corresponding to a position in the image processing apparatus. 2. The method according to claim 1, wherein the first and second calculating means calculate an auto-correlation value of the reference image, an auto-correlation value of the reference image, and a cross-correlation value of the reference image and the reference image. The method according to claim 1, wherein the evaluation value is calculated by using each existing pixel, multiplying each calculation result by a coefficient having a decimal number, and adding or subtracting each multiplication result. Image processing device. 3. An image processing method of an image processing device for detecting a motion vector between the reference image and the reference image based on an evaluation value for evaluating a degree of similarity between the reference image and the reference image, A setting step of setting a search range of a motion vector; a dividing step of dividing the search range into minute regions; a first step of calculating an extreme value of the evaluation value in the minute regions
A second calculating step of calculating the evaluation values respectively corresponding to the four end points of the micro area; an extreme value of the evaluation value calculated in the processing of the first calculating step; A determining step of determining an optimum evaluation value corresponding to the micro area from an optimum evaluation value among the four evaluation values respectively corresponding to the four endpoints calculated in the processing of the second calculation step; A holding step of comparing the two optimal evaluation values respectively corresponding to the two small areas, and holding one of the more optimal evaluation values; and the optimal evaluation finally held in the processing of the holding step Calculating a motion vector corresponding to a position in the search range corresponding to a value. 4. A program for image processing for detecting a motion vector between the reference image and the reference image based on an evaluation value for evaluating a degree of similarity between the reference image and the reference image, A setting step of setting a search range of a motion vector; a dividing step of dividing the search range into minute regions; and a first calculating an extreme value of the evaluation value in the minute regions.
A second calculating step of calculating the evaluation values respectively corresponding to the four end points of the micro area; an extreme value of the evaluation value calculated in the processing of the first calculating step; A determining step of determining an optimum evaluation value corresponding to the micro area from an optimum evaluation value among the four evaluation values respectively corresponding to the four end points calculated in the processing of the second calculation step; A holding step of comparing the two optimal evaluation values respectively corresponding to the two small areas, and holding one of the more optimal evaluation values; and the optimal evaluation finally held in the processing of the holding step Calculating a motion vector corresponding to a position in the search range corresponding to a value. Recording media. 5. A computer for detecting a motion vector between the reference image and the reference image based on an evaluation value for evaluating a degree of similarity between the reference image and the reference image, A setting step of setting; a dividing step of dividing the search range into minute regions; and a first calculating an extreme value of the evaluation value in the minute regions.
A second calculating step of calculating the evaluation values respectively corresponding to the four end points of the micro area; an extreme value of the evaluation value calculated in the processing of the first calculating step; A determining step of determining an optimum evaluation value corresponding to the micro area from an optimum evaluation value among the four evaluation values respectively corresponding to the four end points calculated in the processing of the second calculation step; A holding step of comparing the two optimal evaluation values respectively corresponding to the two small areas, and holding one of the more optimal evaluation values; and the optimal evaluation finally held in the processing of the holding step A calculating step of calculating the motion vector corresponding to a position in the search range corresponding to a value.