JP3022330B2 - Moving image recognition device and moving image recognition method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image recognition apparatus, and more particularly to a moving image recognition apparatus that is robust against noise in an image.

[0002]

2. Description of the Related Art Conventionally, this kind of moving image recognition apparatus has been proposed to realize robust recognition against noise contained in an image, for example, as described in Japanese Patent Application Laid-Open No. 03-292574. The influence of noise in the image data has been suppressed by adding the images taken as a series for each pixel. Also, as a method of using a time series of images, for example, the IEICE Transactions, 1994
Year, Vol. J77-D-II, No. 10, pp. 2
A technique called the DDT method as shown in JP-A No. 019-2026 is used.

The operation of the DTT method will be described with reference to FIG. 19, FIG. 20, FIG. 21 and FIG.

Referring to FIG. 19, a moving image recognition device described in this document is an image input device 191 such as a CCD camera.
Processing area setting means 19 for setting a processing area in an image
2, a feature data extracting unit 193 for extracting feature data of an object from a processing area in an image, and a feature data projecting unit 1 for projecting feature data on a direction axis of the object to obtain one-dimensional data.
94, a DTT image generating unit 195 for generating a DTT image by arranging one-dimensional data in a time-series direction, a DTT image binarizing unit 196 for binarizing the DTT image, and an object from the binarized DTT image. Means for detecting an object from a DTT image for detecting an image.

The single frame image provided from the image input device 191 is supplied to a processing area setting means 192. The processing area setting means 192 sets a processing area 201 covering the moving range of the object in the image as shown in FIG. The image in which the processing area 201 is set is supplied to the feature data extracting means 193, and the feature data extracting means 193 extracts the object 203 to be detected from the set processing area 201.
Extract feature data for The feature data is supplied to the feature data projection unit 194, and the feature data projection unit 19
Reference numeral 4 denotes the moving direction of the object as shown in FIG.
The projection is performed on a direction axis 204 substantially parallel to 02 to generate one-dimensional data. The one-dimensional data at each time is DTT
The DTT image generating means 1
95 arranges the one-dimensional data in time series,
A two-dimensional image (DTT image) including a direction axis and a time axis as shown in FIG. 21 is generated. The DTT image is supplied to the DTT image binarization unit 196, and the DTT image binarization unit 196 binarizes the supplied DTT image. The binarized DTT image is an object detection unit 1 from the DTT image.
97, and an object detection unit 197 from the DTT image.
Detects a uniform area from the supplied binarized DTT image and outputs it as an image representing one object.

In the DTT method, as shown in FIG. 22, in a spatio-temporal image in which past and present frames 32 and 33 are arranged in a time-series direction, a certain cutting plane 211 is considered and its cross section is analyzed to obtain an object 203. Are aware of how they are moving.

[0007]

In a conventional moving image processing apparatus for adding images corresponding to images, a feature point representing the same feature of the object to be recognized is generated between frames because of minute movement between frames. As a result, the feature points do not overlap well because of the shift to different positions, and it is difficult to distinguish between the noise included in the image and the feature points. Therefore, in this case, it is necessary to associate feature points representing the same feature between frames, but the association between frames is essentially a difficult problem, and the accuracy, reliability, and recognition speed of the recognition device deteriorate. There is a problem that it causes a decrease.

On the other hand, a moving image recognition apparatus using a spatiotemporal image such as the DDT method recognizes a recognition target in a moving image without associating the frames with each other. The image recognition device is originally for obtaining information on global movement of an object, and cannot recognize the shape of a recognition target.

SUMMARY OF THE INVENTION An object of the present invention is to solve the above-mentioned problems and to use a time-series information without associating frames with each other using a spatio-temporal image to be robust against noise in the image. Another object of the present invention is to provide a moving image recognition device capable of recognizing a shape of a recognition target.

[0010]

A spatio-temporal image generating means for generating and outputting a spatio-temporal image by inputting a spatio-temporal image and arranging them in the time axis direction, and A feature point extracting means for extracting and outputting a feature point; and a recognition target to which the feature point is inputted and up to a current frame.
The position and shape of the object and the spatio-temporal model of the object modeling the minute movement of the recognition object between each frame are matched, and the distance between the feature point and the spatio-temporal model is minimized.
A spatial model matching means when outputting correct the space-time model parameter values so that the space when said corrected
The spatio-temporal model defined by the model parameter values and the current
The position of the object in the current frame as the intersection area with the frame
And having a target object estimating means for recognizing the location and shape.

According to the means of the present invention, the spatio-temporal model of the object to be recognized in the spatio-temporal model matching means is obtained by modeling a minute movement of the object to be recognized between frames. By matching the model directly to the feature points of the recognition target and absorbing the slight movement of the recognition target between frames into the model, time-series information can be obtained without associating feature points between frames. The used moving image recognition can be realized.

According to the means of the present invention, the shape of the recognition target can be recognized by incorporating the shape of the recognition target in each frame into the spatiotemporal model of the recognition target.

[0013]

Next, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of the first embodiment of the present invention. In the first embodiment, a road image is fetched from a camera mounted on an automobile, and the position and shape of a white line representing a driving lane in the image are estimated from the image. Here, a case is considered where the white line forms a straight line.

Referring to FIG. 1, according to a first embodiment of the present invention, a road image input device 11 such as a CCD camera and a road image data for storing road image data input within a certain period in one frame unit are provided. A storage device 12, a spatio-temporal road image generating means 13 for arranging road images input within a certain period in a time series to generate a spatio-temporal road image, and a white line position estimation result at a previous time in the spatio-temporal road image , A white line candidate point extraction area setting means 14 for setting an area for extracting white line candidate points, a white line candidate point extraction means 15 for extracting white line candidate points from a spatiotemporal road image, and a white line candidate point and a white line Spatio-temporal model matching means 16 for matching the spatio-temporal model, and white line estimating means 1 for estimating the position and shape of the white line in the current frame as an intersection of the white line spatio-temporal model and the current frame. Including the door.

Referring to FIG. 2, the spatio-temporal model matching means 16 calculates a sum of distances between each white line candidate point and the spatio-temporal model.
Parameter update end determining means 22 for determining whether to end the model parameter update based on the distance between the white line candidate point models, and calculates the first derivative and the second derivative of the model parameter of the distance between the white line candidate point models And a model parameter updating means 24 for updating model parameters by an iterative method such as the Newton method based on the distance differential value.

Next, the operation of the first embodiment will be described in detail with reference to FIG.

First, the world coordinates and camera coordinates are determined as shown in FIG.

The road image data provided from the road image input device 11 is supplied to a road image storage device 12. The road image storage device 12 stores the road image data together with the time information for a certain period (steps 401 and 40).
2). The road image storage device 12 supplies N-frame road images (N is a predetermined number) for a certain period including the current frame to the spatio-temporal road image generation means 13. The spatio-temporal road image generating means 13 arranges the road image N frames in time series to generate a three-dimensional spatio-temporal road image as shown in FIG. 3 (step 403). Spatio-temporal road image generation means 13
Means for extracting the spatiotemporal road image from the white line candidate point extraction area setting means 1
4 The white line candidate point extraction area setting means 14 sets a processing area centered on the white line position estimated at the previous time in the spatiotemporal road image (step 404).

However, in the road image, the higher the position is in the upper part of the image, that is, in the −z direction in FIG.
Since the dimensional real space region becomes narrow, it is natural that the processing region also becomes narrower toward the upper part of the image. That is, the size of each frame is M × N, and the spatiotemporal white line model is z = f
(X, t), the processing area width at the bottom of the image is W _proc ^low ,
Assuming that the vanishing point z coordinate at each time t of the white line is Z _van (t), the processing area width W _proc (z, t) at each z coordinate is as shown in FIG.

[0021]

(Equation 1)

It is assumed that

The white line candidate point extraction area setting means 14 supplies the spatiotemporal road image in which the processing area has been set to the white line candidate point extraction means 15. The white line candidate point extracting means 15 extracts white line candidate points that are considered to be points on the white line from the processing area of the spatiotemporal road image (step 405). At this time, if the white line candidate point does not exist in the processing area, the operation is terminated because it is impossible to estimate the white line. If a white line candidate point exists, its coordinate value is supplied to the spatiotemporal model matching means 16 (step 406).

Here, the spatiotemporal white line model will be described. The white line on the image has the same position and the same shape in each frame when the own vehicle is traveling only in a direction completely parallel to the white line as shown in FIG. Let's make a plane. However, in actuality, the own vehicle undergoes a lateral shift or a change in the running direction as shown in FIG. 9 during running, and the gradient of the road also changes as shown in FIG. Therefore, in the spatiotemporal road image, the white line is not a plane but a curved surface. Therefore, in the present embodiment, it is assumed that the x-coordinate and z-coordinate of the vanishing point of the white line on the two-dimensional image and the inclination of the white line linearly change with respect to time t.
Define b, c, d, e, and f as model parameters as a curved surface represented by x- (e + ft) + (a + bt) z- (a + bt) (c + dt) = 0 (2). If the above model is a spatio-temporal model of a white line on the right side of the host vehicle, the vanishing point of the white line on the left side is common to the white line on the right side as shown in FIG. Therefore, the spatiotemporal white line model on the left side is defined as x− (e + ft) + (g + bt) z− (g + bt) (c + dt) = 0 (3) using g as a new model parameter. Therefore, the left and right spatiotemporal white line models are expressed as x- (e + ft) + (a + bt) z- (a + bt) (c + dt) = 0 x- (e + ft) + (g + bt) z- (g + bt) (c + dt) = 0 (4) Is defined as

The spatio-temporal model matching means 16 first sets the model parameters estimated at the previous time as initial parameters in the spatio-temporal white line model (step 407). Next, in step 408, the distance between the spatiotemporal white line model (Equation (4)) in which the parameters are set and each white line candidate point (x _i , z _i , t _i ) is calculated as the following equation (5).

[0026]

(Equation 2)

In step 409, the sum of the distances calculated for each white line candidate point is calculated as in the following equation (6).

[0028]

(Equation 3)

In the following, a model parameter that minimizes the sum of distances in the equation (6) is calculated as an optimal model parameter, an intersection line between the spatiotemporal white line model and the current frame is calculated, and the intersection line is defined as the current frame. Then, the position and shape of the white line are output as the white line, and the operation is completed. Specifically, for example, the following method can be used.

In step 409, the distance 1
Next, the weighted sum of the secondary differential values is expressed by the following equations (7) and (8).
Calculate as follows.

[0031]

(Equation 4)

The weights w _left and w _right are weighted in inverse proportion to the number of white line candidate points N _left and N _right for each of the left and right when calculating the sum of the left and right white lines. Is taken. That is, the weights w _left and w _right are represented by the following equation (9), where v is a predetermined constant.

[0033]

(Equation 5)

By assigning such weights, when the number of white line candidate points is extremely reduced in one of the left and right processing regions due to noise, the model is drawn too much toward the other white line. Can be prevented.

The sum of the distances is compared with a predetermined threshold value in step 410. If the sum of the distances is equal to or less than the threshold value, it is determined that the spatiotemporal white line model is sufficiently close to the white line image in the spatiotemporal road image. , Calculate the intersection between the spatiotemporal white line model and the current frame, and use that intersection as the white line in the current frame,
Output the position and shape of the white line and end the operation (Step 4)
12 and 413).

If the sum of the distances is larger than the threshold value, the process proceeds to step 411, and in step 411, the process proceeds to step 409.
Newto from the sum of the first and second derivatives of the distance calculated in
The spatio-temporal road model parameters are updated by the n method, and the process returns to step 408. Hereinafter, the loop is repeated until the sum of the distances becomes equal to or less than the threshold value or the number of loops exceeds a predetermined number.

If the number of loops exceeds a predetermined number, the spatio-temporal white line model and the spatio-temporal white line model are used as the optimal model parameters in the loops in which the sum of the distances is the minimum in each loop up to that time. The intersection line with the current frame is calculated, the intersection line is set as the white line in the current frame, and the position and shape of the white line are output, and the operation is completed.

Next, a second embodiment of the present invention will be described in detail with reference to the drawings. In this embodiment,
The case where the white line forms a curve is considered.

Referring to FIG. 11, a second embodiment of the present invention includes a road image input device 11 such as a CCD camera and the like.
A road image storage device 12 that stores road image data input within a certain period in units of one frame, and a spatio-temporal road that generates spatio-temporal road images by arranging road images input within a certain period in chronological order An image generation unit 13; a white line candidate point extraction area setting unit 14 for setting an area for extracting a white line candidate point based on a white line position estimation result at a previous time in a spatio-temporal road image; A white line candidate point extracting unit 15 for extracting candidate points; a spatiotemporal tangent model matching unit 111 for matching the white line candidate point with a spatiotemporal model of a tangent line below the white line curve image; a white line candidate point and a spatiotemporal curve model And a white line estimating means 1 for estimating the position and shape of the white line in the current frame of the white line
7 is included.

Spatiotemporal curve model matching means 112
Referring to FIG. 12, the white line candidate point model distance calculating means 21 for calculating the sum of the distances between each white line candidate point and the spatiotemporal curve model, and updating the curve model parameters based on the white line candidate point model distance Parameter update end determining means 22 for determining whether or not to perform the calculation, distance differential value calculating means 23 for calculating the first derivative and second derivative of the curve model parameter of the distance between the white line candidate point models, and differentiation of the distance. A curve model parameter updating unit 121 for updating the curve model parameters based on the values by an iterative method such as the Newton method.

Next, the operation of the second embodiment will be described in detail with reference to FIG.

When the white line forms a curve, the number of parameters of the spatiotemporal white line model increases.
That is, the position of the white line is obtained using the spatiotemporal tangent model of the tangent 133 of the white line curve near the own vehicle,
Next, using the spatiotemporal curve model, the curvature radius 132 of the curve
(FIG. 13). Since here spacetime indirect line model is a space model when the straight portion of the white line, as in the first embodiment x = (e + ft) + (a + bt) z- (a + bt) (c + dt) ≡f r (z, t ) x = (e + ft) + (g + bt) z- (g + bt) (c + dt) ≡f l (z, t) is defined as (10). At this time, the spatio-temporal model of the white line curve is defined as a new parameter related to the radius of curvature, m.

[0043]

(Equation 6)

Is defined as In this way, by dividing the spatiotemporal white line model into two stages, it is possible to reduce the dimension of the parameter space at each matching stage, and to simplify, stabilize, and speed up the matching.

In the second embodiment, white line candidate points are extracted as in the first embodiment, and if there are white line candidate points, the spatiotemporal white line model parameters estimated at the previous time are used as initial values. Set to the spatiotemporal white line model (Step 4
07), the coordinates of the white line candidate point are supplied to the spatiotemporal tangent model matching means 111. The spatiotemporal tangent model matching unit 111 estimates a model parameter other than the m by matching the spatiotemporal tangent model with the white line candidate point at the lower part of the image, and supplies the model parameter to the spatiotemporal curve model matching unit 112.

Spatiotemporal curve model matching means 112
In step 142, a parameter other than the estimated m, and m space curve model when the initial value is set (Equation (11)) in step 141, the white line candidate point _{(x i, z i, t} i ) Is calculated as Expression (12).

[0047]

(Equation 7)

In step 409, the sum of the distances calculated for each white line candidate point is calculated as in the following equation (13).

[0049]

(Equation 8)

Hereinafter, a model parameter that minimizes the sum of the distances in the equation (13) is calculated as an optimal model parameter, and an intersection between the spatiotemporal white line model and the current frame is calculated. The position of the white line as the white line in the
Output the shape and finish the operation. Specifically, for example, the following method can be used.

In step 409, the distance 1
Next, the weighted sum of the secondary differential values is expressed by the following equations (14) and (1).
Calculate as in 5).

[0052]

(Equation 9)

The weights w _left and w _right used in the summation in equations (14) and (15) are the same as those in the first embodiment. The sum of the distances is compared with a predetermined threshold in step 410, and if the sum of the distances is equal to or less than the threshold, the spatio-temporal curve model determines that the spatio-temporal road image is sufficiently close to the white line image in the spatio-temporal road image. Model matching means 112
Supplies the curve model parameters to the white line estimating means 17. The white line estimating means 17 calculates the line of intersection between the spatiotemporal curve model and the current frame, outputs the position and shape of the white line as the white line in the current frame (step 413).
Finish the operation.

If the sum of the distances is larger than the threshold value, the process proceeds to step 143, and in step 143, the process proceeds to step 409.
Newto from the sum of the first and second derivatives of the distance calculated in
m is updated by the n method, and the process returns to step 142. Hereinafter, the loop is repeated until the sum of the distances becomes equal to or less than the threshold value or the number of loops exceeds a predetermined number.

When the number of loops exceeds a predetermined number, the spatio-temporal curve model matching means 112 sets the m in the loop in which the total sum of the distances is the minimum in each of the loops up to that time as an optimal parameter. It is supplied to the estimating means 17. The white line estimating means 17 calculates the intersection line between the spatiotemporal white line model and the current frame, outputs the position and shape of the white line as the white line in the current frame (step 413), and ends the operation.

Next, a third embodiment of the present invention will be described in detail with reference to the drawings. In this embodiment,
It is assumed that the user does not know whether the white line forms a curve or a straight line.

Referring to FIG. 15, a third embodiment of the present invention includes a road image input device 11 such as a CCD camera and the like.
A road image storage device 12 that stores road image data input within a certain period in units of one frame, a spatio-temporal road image generation unit 13 that generates a spatio-temporal road image from a road image input within a certain period, A white line candidate point extraction area setting unit 14 for setting an area for extracting a white line candidate point in the spatiotemporal road image, a white line candidate point extracting unit 15 for extracting a white line candidate point from the spatiotemporal road image, and a white line candidate point. A spatio-temporal straight line model matching unit 16 for matching a spatio-temporal straight line model, a spatio-temporal tangent model matching unit 111 for matching a white line candidate point and a spatio-temporal tangent model of a curve, and a white line candidate point and a spatio-temporal curve model A spatiotemporal curve model matching means 112 for matching;
A matching degree comparing unit 151 that determines the shape of the road by comparing the degrees of matching in the two spatiotemporal model matching units; and a white line estimating unit 17 that estimates the position and shape of the white line in the two-dimensional white line image. including.

Next, the operation of the third embodiment will be described in detail with reference to FIG.

Steps up to step 405 of extracting a white line candidate point are the same as in the first and second embodiments. Step 4
At 07, the model parameters estimated at the previous time are set as initial values in the spatiotemporal linear model and the spatiotemporal curve model. Next, the spatiotemporal linear model matching means 16
In step 161, the white line candidate point is matched with the spatiotemporal straight line model (step 161), and the sum of the straight line model parameters and the distance between the spatiotemporal straight line model and the white line candidate point is supplied to the matching degree comparing means 151. Next, the spatiotemporal tangent model matching unit 111 matches the white line candidate point at the lower part of the image with the spatiotemporal tangent model of the curve (step 141), and supplies the tangent model parameters to the spatiotemporal curve model matching unit 112. The spatiotemporal curve model matching unit 112 matches the white line candidate point with the spatiotemporal curve model (step 162), and supplies the curve model parameters and the sum of the distances between the spatiotemporal curve model and the white line candidate point to the matching degree comparing unit 151. I do. The matching degree comparing means 151 compares the sum of the distances between the spatiotemporal model supplied from both the spatiotemporal linear model matching means 16 and the spatiotemporal curve model matching means 112 and the white line candidate point as data representing the degree of matching ( Step 163). Here, if the sum of the distances of the straight-line models is equal to or less than the sum of the distances of the curve models, the matching degree comparison unit 151 supplies the straight-line model parameters to the white line estimation unit 17. The white line estimating means 17 calculates an intersection line between the spatiotemporal straight line model and the current frame in step 412, sets this intersection line as a white line in the current frame, outputs the position and shape of the white line in step 413, and ends the operation. .

If the sum of the distances for the curve model is smaller in step 163, the matching degree comparing means 151 supplies the curve model parameters to the white line estimating means 17. The white line estimating means 17 calculates an intersection line between the spatiotemporal curve model and the current frame in step 144, sets this intersection line as a white line in the current frame, outputs the position and shape of the white line in step 413, and ends the operation. .

Next, a fourth embodiment of the present invention will be described.
This will be described in detail with reference to the drawings. In the present embodiment, a case is considered in which an initial value of a model parameter is not given.

Referring to FIG. 17, a fourth embodiment of the present invention includes a road image input device 11 such as a CCD camera and the like.
A road image storage device 12 that stores road image data input within a certain period in units of one frame, a spatio-temporal road image generation unit 13 that generates a spatio-temporal road image from a road image input within a certain period, A first white line candidate point extracting means 171 for extracting a white line candidate point without defining a processing area in a lower part of the spatiotemporal road image, and a white line candidate point and a spatiotemporal straight line model are predetermined in the lower part of the spatiotemporal road image. First spatiotemporal linear model matching means 172 for matching based on the obtained initial model parameters, white line candidate point extraction area setting means 14 for setting an area for extracting a white line candidate point in the spatiotemporal road image, A white line candidate point extracting means 15 for extracting a white line candidate point from a white line candidate point extraction region of a road image;
The second spatio-temporal straight-line model matching means 16 for matching the model parameters calculated by the spatio-temporal straight-line model matching means 172 as the initial model parameters, and the spatio-temporal tangent for matching the white line candidate point and the spatio-temporal tangent model of the curve Model matching means 111
Initial curvature radius setting means 173 for outputting a plurality of curvature radii appropriately determined in advance as initial curvature radii; and model parameters calculated by the spatiotemporal connection model matching means 111 for white line candidate points and the spatiotemporal curve model. And a spatio-temporal curve model matching unit 112 that matches the initial curvature radius supplied by the initial curvature radius setting unit 173 as an initial model parameter, and a degree of matching between the two spatio-temporal model matching units to determine the shape of the road. It includes a matching degree comparison unit 151 to be determined, and a white line estimation unit 17 for estimating the position and shape of the white line in the two-dimensional image of the white line.

Hereinafter, the configuration is the same as that of the third embodiment of the present invention.

Next, the operation of the fourth embodiment will be described in detail with reference to FIG. As in the first, second, and third embodiments, up to step 403, a spatio-temporal road image is generated. Next, white line candidate points are extracted from the entire lower region of the spatiotemporal image appropriately determined (step 18).
1) In step 182, the white line candidate point is matched with the spatiotemporal linear model. However, unlike the first, second and third embodiments, the initial parameters in this matching use predetermined values. The spatiotemporal model parameters calculated in this step are used as initial model parameters in the following steps.

Since the initial model parameters of the spatiotemporal linear model have been obtained as described above, matching with the white line candidate point is performed in the same manner as in the third embodiment.
The calculated sum of the straight line model parameters and the distance between the spatiotemporal straight line model and the white line candidate point are supplied to the matching degree comparing means 151.

For the spatiotemporal curve model, a plurality of initial curvature radii are prepared in advance in the initial curvature radius setting means 173, and a processing area is set for each initial curvature radius based on the above initial model parameters, and a white line candidate is set. The points are extracted and the white line candidate points are matched with the spatiotemporal curve model, and the curve model parameters for each initial radius of curvature and the sum of the distances between the spatiotemporal curve model and the white line candidate points are supplied to the matching degree comparing means 151 ( Steps 184 and 185, 162).

In the matching degree comparing means 151,
The sum of the distances between the spatiotemporal model and the white line candidate points supplied from both the spatiotemporal straight line model matching means 16 and the spatiotemporal curve model matching means 112 is compared as data representing the degree of matching (step 163). Here, if the sum of the distances for the straight line model is equal to or less than all of the sums of the distances for the curve model with respect to the respective initial radii of curvature, the matching degree comparison unit 151 supplies the straight line model parameters to the white line estimation unit 17. The white line estimating means 17 calculates an intersection line between the spatiotemporal straight line model and the current frame in step 412, sets this intersection line as a white line in the current frame, outputs the position and shape of the white line in step 413, and ends the operation. .

In step 163, if the curve model for a certain initial radius of curvature gives the minimum sum of distances, the matching degree comparing means 151 supplies the curve model parameters for the initial radius of curvature to the white line estimating means 17. The white line estimating means 17 performs step 14
In step 4, the intersection line between the spatiotemporal curve model and the current frame is calculated, and this intersection line is set as a white line in the current frame. In step 413, the position and shape of the white line are output, and the operation ends.

[0069]

The first effect is that a moving image recognition robust to noise in an image using time-series information is realized without associating frames.

The reason is that the recognition of the recognition object using the time-series information is realized by matching the feature points of the recognition object with the spatio-temporal object model in the spatio-temporal image. By incorporating small movements of the recognition target between frames into the target model, matching between the feature points and the spatio-temporal target model can be performed while absorbing the shift of the feature points between frames into the spatio-temporal target model. What you can do.

A second effect is to provide a moving image recognition device capable of recognizing the shape of an object.

The reason is that the spatiotemporal object model can incorporate the shape of each object frame.

[Brief description of the drawings]

FIG. 1 is a configuration diagram showing a first embodiment according to the present invention.

FIG. 2 is a configuration diagram showing a spatiotemporal model matching unit.

FIG. 3 is a diagram illustrating a spatio-temporal image and a spatio-temporal model.

FIG. 4 is a flowchart showing the operation of the first embodiment according to the present invention.

FIG. 5 is a diagram showing world coordinates and camera coordinates.

FIG. 6 is a diagram showing a white line candidate point extraction area setting unit.

FIG. 7 is a diagram showing the relationship between left and right white lines and vanishing points.

FIG. 8 is a diagram showing a positional relationship between a white line and a host vehicle.

FIG. 9 is a diagram showing a positional relationship between a white line and a host vehicle.

FIG. 10 is a diagram showing a gradient of the ground.

FIG. 11 is a configuration diagram showing a second embodiment according to the present invention.

FIG. 12 is a configuration diagram showing a spatiotemporal curve model matching unit.

FIG. 13 is a diagram showing a positional relationship between a white line forming a curve and the host vehicle.

FIG. 14 is a flowchart showing the operation of the second embodiment according to the present invention.

FIG. 15 is a configuration diagram showing a third embodiment according to the present invention.

FIG. 16 is a flowchart showing the operation of the third embodiment according to the present invention.

FIG. 17 is a configuration diagram showing a fourth embodiment according to the present invention.

FIG. 18 is a flowchart showing the operation of the fourth embodiment according to the present invention.

FIG. 19 is a configuration diagram showing a configuration of a conventional invention.

FIG. 20 is a diagram illustrating a process according to a conventional invention.

FIG. 21 is a diagram showing an object recognition method according to a conventional invention.

FIG. 22 is a diagram showing features of the conventional invention.

[Description of Signs] 11 Road image input device 12 Road image storage device 13 Spatio-temporal road image generation means 14 White line candidate point extraction area setting means 15 White line candidate point extraction means 16 Spatio-temporal model matching means 17 White line estimation means 21 White line candidate points Model distance calculation means 22 Parameter update end determination means 23 Distance differential value calculation means 24 Model parameter update means 31 Spatiotemporal model of white line 32 Current frame 33 Past frame 51 Camera focus 52 Ground 53 Camera focal length 71 Vanishing point 72 Horizon 73 White line 81 Own vehicle 101 Horizontal line 111 Spatio-temporal tangent model matching means 112 Spatio-temporal curve model matching means 121 Curve model parameter updating means 131 Running direction 132 Curvature radius 133 Tangent line of white line 151 Matching degree comparing means 171 First White line candidate point extracting means 172 first spatio-temporal linear model matching means 173 initial curvature radius setting means 191 image input means 192 processing area setting means 193 feature data extracting means 194 feature data projecting means 195 DTT image generating means 196 Binarization means 197 Object detection means from DTT image 201 Processing area 202 Object movement direction 203 Object 204 Direction axis 211 Cutting plane 212 Object image

Continuation of the front page (51) Int.Cl. ⁷ identification symbol FI H04N 7/18 H04N 7/18 G (58) Investigation field (Int.Cl. ⁷ , DB name) G06T 7/20 H04N 7/18 B60R 21 / 00 JICST file (JOIS)

Claims (10)

(57) [Claims] 1. A spatio-temporal image is generated by arranging a spatio-temporal image in a time axis direction, a feature point extracted from the spatio-temporal image up to the present time , a position of a recognition target object up to a current frame,
Shape and matching a spatial model when slight motion modeled objects recognition object between frames, before
The distance between the feature point and the spatiotemporal model is minimized.
And output the modified spatio-temporal model parameter values.
Spatio-temporal model defined by specified spatio-temporal model parameter values
By finding the intersection area between the model and the current frame,
A moving image recognition method comprising recognizing a position and a shape of an object in a current frame . 2. A spatio-temporal image generating means for generating and outputting a spatio-temporal image by inputting a spatio-temporal image and arranging them in a time axis direction, and extracting a feature point from the spatio-temporal image by inputting the spatio-temporal image. A feature point extracting means for outputting , and the position, shape and position of the recognition target object up to the current frame after inputting the feature point
Matching the spatio-temporal model of the object modeling the minute movement of the recognition object between each frame and the feature points
A spatio-temporal model that minimizes the distance from the spatio-temporal model
A spatial model matching means when modifying outputting del parameter values, the spatial model parameter values when said corrected
Intersection area of the current frame and the spatiotemporal model defined by
Video recognition apparatus characterized by having a target object estimating means for recognizing the position and shape of the current frame of the object as a. 3. The spatial image is a spatial image of a road taken from inside a vehicle, the recognition target is a white line representing a driving lane on the road, and the spatiotemporal model is a current frame.
Position and shape of the white line in the
A spatio-temporal white line model in a spatio-temporal image modeling small movements
The moving image recognition method according to claim 1, wherein the moving image is a Dell . 4. The spatial image is a spatial image of a road taken from inside a vehicle, the object to be recognized is a white line representing a driving lane in the road, and the spatiotemporal model is a current frame.
Position and shape of the white line in the
Space white mode when the small movements in the modeled space-time image
The feature point extracting means inputs a spatio-temporal road image, extracts and outputs white line candidate points considered to be points on a white line, and the spatio-temporal model matching means inputs a white line candidate point and outputs the spatio-temporal image. Spatio-temporal white line model matching with white line model
The moving image recognition device according to claim 2, wherein the moving image recognition device outputs a Dell parameter . 5. A spatio-temporal model of a white line, wherein the spatio-temporal model matching means inputs a white line candidate point, and models a minute movement of a vehicle and a change of a road gradient between frames by a linear function with respect to time. 5. The moving image recognition device according to claim 4, wherein the moving image recognition device outputs the model parameter by matching with the following. 6. The spatio-temporal model matching means includes:
Enter the white line candidate points in the image, each frame odor
Modeled in a spatiotemporal image assuming that the white line forms a straight line
The moving image recognition device according to claim 4 or 5, wherein the moving image recognition device outputs a spatiotemporal linear model parameter by matching with the spatiotemporal white line model. 7. The spatio-temporal model matching means includes:
Enter the white line candidate points in the image, each frame odor
The moving image recognition apparatus according to claim 4 or 5, wherein the moving image recognition apparatus outputs a spatiotemporal curve model parameter by matching with a spatiotemporal white line model modeled in the spatiotemporal image assuming that the white line forms a curve. 8. A spatiotemporal tangent model matching means for inputting the white line candidate point, matching the spatiotemporal model of the tangent of the white line at the lower part of the image, and outputting a tangent model parameter, wherein the spatiotemporal model matching means comprises 6. The moving image recognition apparatus according to claim 4, wherein the moving image recognition device outputs the curve model parameters by matching the white line candidate points with the spatiotemporal model of the white line that is modeled as forming a curve. 9. A spatio-temporal model matching means for inputting the white line candidate points, matching the spatio-temporal model of the white line modeled as a white line forming a straight line, and outputting a straight line model parameter, and the curve model parameter and the straight line model 8. A matching degree comparing means for inputting a parameter, comparing a matching degree between a white line candidate point and a spatiotemporal model in the two model parameters, and outputting a model parameter having a larger matching degree. 9. The moving image recognition device according to 8. 10. A lower feature point extracting means for inputting the spatiotemporal road image and outputting a lower white line candidate point at the lower part of the image among the white line candidate points, and inputting the lower white line candidate point to form a white line at a lower part of the image. An initial spatio-temporal model matching means for matching with a tangent spatio-temporal model and outputting initial model parameters, wherein the spatio-temporal model matching means inputs the white line candidate points and the initial model parameters and sets the initial model parameters to initial values. 10. The moving image recognition apparatus according to claim 4, wherein the white line candidate point is matched with a spatiotemporal model of the white line to output model parameters.