JP3145831B2 - Image processing method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method for deriving an optical flow of an image, for example, when deriving a distance from an object using an imaging means.

[0002]

2. Description of the Related Art Conventionally, when a distance to an object is determined by image processing, image data of the object imaged by an image pickup means 1 such as a single CCD camera is converted from analog to digital as shown in FIG. A / D conversion) unit 2
A / D-converted, and the A / D-converted image data is stored in the frame memory 3, and the image processing means 4 performs a matching process between the image at a certain time and the image after a predetermined time such as the time corresponding to one frame. After that, the optical flow is calculated to derive the distance to the object.

When an optical flow is derived in order to obtain a distance to an object as described above, matching processing between an image at a certain time and an image after a predetermined time is performed as described above. This is done by searching for where the same pattern as the density pattern of the image data in the micro window for several pixels in the image of the image exists in the image after a predetermined time.

[0004] That is, assuming that the imaging surface of the imaging means 1 is an XY coordinate plane as shown in FIG. 10, an image at a certain time on this coordinate plane becomes as shown by a solid line in FIG. When the image after a predetermined time has moved as shown by the broken line in FIG. 10, the above-mentioned minute window is shifted by one pixel to calculate the correlation value C given by Expression 1, and as shown in FIG. By finding the position where the correlation value C is minimized, matching is achieved.

[0005]

(Equation 1)

In Equation 1 and FIG. 11, I1 represents the density value of the image at a certain time, I2 represents the density value of the image after a predetermined time, and m and n represent the positions in the X and Y directions, respectively.

[0007] Such matching processing is performed not only when deriving an optical flow which is a movement vector of an image, but also in still image processing such as image recognition.

[0008]

However, since the conventional matching process is performed by shifting the window by one pixel and obtaining a correlation value, it takes a long time to perform the calculation.
For example, it is not suitable for a case where high-speed processing is required, such as calculating an inter-vehicle distance with a vehicle ahead mounted on a vehicle.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and has as its object to greatly reduce the time required for computation.

[0010]

An image processing method according to the present invention performs analog / digital conversion processing on image data obtained by imaging an object by an imaging means, and converts the image data at a certain time by the imaging means into a predetermined time. In an image processing method for deriving an optical flow by searching for a point corresponding to an image after a time, a line drawing image is formed by performing line drawing conversion on the image at the certain time, and a line pattern of the obtained line drawing image is obtained. Assuming that a density gradient exists in the normal direction, a virtual density gradient is set, and a gradient value of the set virtual density gradient, a tangent angle of a first point on the line pattern, and the first point And the first on a line pattern obtained from the image after the predetermined time
A constraint formula for optical flow is derived based on the distance between the second point closest to the point and the angle between the line connecting the two points and the tangent at the second point, and based on the constraint formula It is characterized in that matching processing is performed within a predetermined range.

The above-mentioned virtual density gradient may be set by filtering along a line pattern using an averaging filter to derive a constraint equation of an optical flow by a normal gradient method.

[0012]

According to the present invention, a virtual density gradient is set based on a tangent of a line pattern obtained by line drawing conversion,
When an optical flow is derived by combining optical flow constraint expressions obtained from tangent angles, matching processing only needs to be performed within a predetermined range based on the constraint expression, and processing time can be reduced by limiting the matching processing range.

The same applies to the case where an optical flow constraint equation is obtained from a virtual density gradient by filtering.

[0014]

【Example】

(First Embodiment) FIG. 1 is a flowchart for explaining the operation of a first embodiment of the image processing method according to the present invention, and FIGS.
Is an operation explanatory diagram.

Here, the apparatus applied to this embodiment is shown in FIG.
Therefore, the function of the image processing means 4 is different from that of the related art, although FIG.

That is, a frame image at a certain time t is line-drawing-converted by a predetermined differential operator or the like to form a line-drawing image, and the density gradient is obtained in the normal direction of the line pattern of the obtained line-drawing image.
A virtual density gradient, that is, a so-called virtual density gradient, is set assuming that the distribution exists, the gradient value of the set virtual density gradient, the tangent angle θ of the first point on the line pattern, the A distance d between the first point and a second point closest to the first point on the line pattern obtained from the image after a predetermined time Δt has elapsed from the time t, and a line connecting these points is defined as a second point. The optical flow is calculated based on the angle φ between the point and the tangent.

More specifically, the above-described line drawing image is a binary image represented by a line having a thickness of "1" representing the contour of an object. Thus, a line pattern indicated by L1 in FIG. 2 is obtained, and a line image indicated by L2 in FIG. 2 is obtained by performing a line drawing conversion of the frame image Δt time after the time t. The steepest virtual density gradient Gt is set in the normal direction perpendicular to the tangents T1 and T2. At this time, each tangent T1, T2
If it is provided at an arbitrary position on each of the line patterns L1 and L2,
Good. In general, the gradient of the density distribution is such that the X direction and the Y direction
Defined by the derivative of the direction, the gradient at which angle to the coordinates
Changes from 0 degrees to the steepest value depending on
In this embodiment, the normal direction perpendicular to the tangents T1 and T2
Is assumed to have the steepest virtual concentration gradient.
The virtual density gradient Gt is set.

Therefore, as shown in FIG.
1, a tangent angle θ at an arbitrary first point P1 (x, y, t), a gradient value G of a virtual density gradient at the point P1, a point P1
And the second point P on the line pattern L2 closest to this point P1
2 and the line connecting both points P1 and P2 and the second
Angle between the tangent T2 and the normal perpendicular to the point P2 at
Based on the equation, the virtual concentration gradient
X direction, Y direction, T direction (time direction)
Density gradients Vx, Vy, Vt (hereinafter the same) as values are obtained.

[0019]

(Equation 2)

[0020]

(Equation 3)

[0021]

(Equation 4)

At this time, Vx, Vy and Vt are shown in FIG.
(A) and (b), as shown in the moving image processing field.
Optical flow using the gradient method generally known in
When applied to the constraint equation in the case of deriving ー, the optical flow constraint equation represented by Equation 5 is established from Vx, Vy, and Vt given by Equations 2 to 4. This
In Equation 5, u and v are X,
This is the Y-direction component.

[0023]

(Equation 5)

The above tangent angle θ is obtained by the calculation of Expression 6 using a chain code. In Expression 6, Np is the number of chain codes, and Fi is the value of the chain code.

[0025]

(Equation 6)

In the method using the chain code, for example, as shown in FIG. 5, within a range of 3 × 3 pixels, “0” to “7” is determined depending on where the pixel adjacent to the center pixel is located at positions “0” to “7”. Since the direction to an adjacent pixel can be known from this chain code, the tangent angle θ can be obtained by using the chain code. The tangent angle θ at the center point in the pixel range shown in FIG.

[0027]

(Equation 7)

Then, the optical flow (u, v) falls within the range satisfying the relational expression of Expression 5, that is, the matching is performed only on the thick line shown in FIG. Can be derived,
The calculation time can be greatly reduced as compared with the conventional case.

Therefore, as shown in the flowchart of FIG. 1, after the calculations of Vx, Vy, and Vt are performed according to Equations 2 to 4 (Step S1), the constraint equation of the optical flow obtained along each line pattern is calculated. The optical flows are derived by solving the equations 8 and 9 simultaneously (step S2).

[0030]

(Equation 8)

[0031]

(Equation 9)

As described above, according to the first embodiment, the virtual density gradient is set on the basis of the tangent of the line pattern obtained by the line drawing conversion, and the optical flow constraint equation obtained from the tangent angle is simultaneously established to form the optical flow. The flow is derived. At this time, since the matching process only needs to be performed within a predetermined range based on the constraint formula, the time required for the calculation can be significantly reduced as compared with the related art.

(Second Embodiment) FIG. 8 is a flow chart for explaining the operation of a second embodiment of the present invention.

As described in the first embodiment, when the line drawing image obtained by the line drawing conversion is complicated, filtering using a simple averaging filter is performed along the line drawing as shown in FIG. To set a virtual density gradient (step T1), and calculate Vx, V
After obtaining y and Vt (step T2), an optical flow constraint equation represented by the above-described equation 5 is obtained, and the optical flow constraint equations are simultaneously established along a line pattern before and after a predetermined time has elapsed. (Step T3), an optical flow is derived.

[0035]

(Equation 10)

[0036]

[Equation 11]

[0037]

(Equation 12)

When the optical flow is derived in this manner, the matching process may be performed only within a predetermined range based on the constraint formula of the optical flow as in the first embodiment.

At this time, the density gradient obtained by the filtering is not necessarily linear, but becomes non-linear as the curvature of the line pattern increases. However, a density gradient substantially linear is obtained, and there is no practical problem.

Therefore, according to the second embodiment, the same effect as that of the first embodiment can be obtained.

[0041]

As described above, according to the image processing method of the present invention, a virtual density gradient is set for a line image obtained by line image conversion, and a constraint equation for an optical flow is derived from the set virtual density gradient. Since the matching process is performed within a predetermined range based on the constraint formula, the processing speed of the matching process can be limited, and the calculation time can be significantly reduced as compared with the conventional case. This is effective when high-speed processing is required, such as calculating the inter-vehicle distance to a vehicle ahead.

[Brief description of the drawings]

FIG. 1 is a flowchart for explaining the operation of a first embodiment of the image processing method of the present invention.

FIG. 2 is an operation explanatory diagram of the first embodiment.

FIG. 3 is an operation explanatory diagram of the first embodiment.

FIG. 4 is an operation explanatory diagram of the first embodiment.

FIG. 5 is an operation explanatory diagram of the first embodiment.

FIG. 6 is an operation explanatory diagram of the first embodiment.

FIG. 7 is an operation explanatory diagram of the first embodiment.

FIG. 8 is a flowchart for explaining the operation of the second embodiment of the present invention.

FIG. 9 is a block diagram of an image processing apparatus on which the present invention is based.

FIG. 10 is an operation explanatory diagram of a conventional example.

FIG. 11 is an operation explanatory diagram of a conventional example.

[Explanation of symbols]

 1 imaging means 4 image processing means

──────────────────────────────────────────────────続 き Continued on the front page (58) Fields investigated (Int. Cl. ⁷ , DB name) G01B 11/00-11/30 G06T 1/00 315 G06T 1/00 400 G06T 7/60 180

Claims (2)

(57) [Claims] An image data obtained by imaging an object by an imaging means is subjected to analog / digital conversion processing, and a point corresponding to the image at a certain time and the image after a predetermined time is searched by the imaging means. an image processing method for deriving an optical flow, a line drawing image to form an image at the certain time by the line drawing conversion, concentrated in the normal direction of the line pattern of the obtained said line drawing image
Assuming that a degree gradient exists, a virtual density gradient is set, and a gradient value of the set virtual density gradient, a tangent angle of a first point on the line pattern, the first point and the predetermined time The optical distance is determined based on the distance between the second point closest to the first point on the line pattern obtained from the subsequent image and the angle between the line connecting the two points and the tangent at the second point. An image processing method comprising: deriving a constraint equation of a flow and performing a matching process within a predetermined range based on the constraint equation. 2. The image processing method according to claim 1, wherein
A virtual density gradient is set by performing filtering by an averaging filter along a line pattern of the line image obtained by the line image conversion, and a constraint formula of an optical flow is derived by a gradient method based on the set virtual density gradient. Characteristic image processing method.