WO2015170706A1

WO2015170706A1 - Traveling body detection method and program

Info

Publication number: WO2015170706A1
Application number: PCT/JP2015/063213
Authority: WO
Inventors: 貴裕東
Original assignee: 日本電産エレシス株式会社
Priority date: 2014-05-07
Filing date: 2015-05-07
Publication date: 2015-11-12

Abstract

A device (10) having: an imaging unit (110) that captures, across a plurality of frames, an image including periphery information for a vehicle; a feature point extraction unit (120) that extracts, from the captured plurality of image frames, a plurality of feature points included in first and second target images for processing that have been captured at two different times; a hypothesis generation unit (140) that selects two first feature points and two second feature points, each being two feature points, from the plurality of feature points for both the extracted first target image and second target image, generates a first line component that connects the two feature points selected in the first target image and a second line component that connects the two feature points in the second target image, and, if the first line component and the second line component are parallel, generates selection conditions being conditions to be fulfilled by a movement vector for one target object including two feature points, on the basis of the movement vectors for each of the two feature points between the first target image and the second target image; and a detection unit (160) that detects a traveling body on the basis of the selected feature points.

Description

Moving object detection method and program

The present invention relates to a moving object detection method and a program.

In recent years, a technique for recognizing an obstacle present around a moving body using a camera mounted in a vehicle interior of the moving body such as a vehicle has become widespread.
As a conventional technique for recognizing an obstacle existing around a moving body, there is a method of recognizing a surrounding state by clustering feature amounts extracted from surrounding information as in Patent Document 1, for example.

JP2011-14037A

However, such a conventional method requires a large amount of calculation.

An object of the present invention has been made in view of the above-described problems, and is to provide a technique capable of detecting a moving object with a small calculation amount.

According to one embodiment of the present invention,
A method for detecting a moving body different from the host vehicle based on a target image including surrounding information of the host vehicle, capturing the target image over a plurality of frames, storing the image information, and storing the image A preparatory step of extracting two frame images having different imaging times from information as a first target image and a second target image;
A plurality of first feature points included in the first target image are extracted, two first target feature points are selected from the first feature points, and a plurality of second feature points included in the second target image are selected. A feature point extracting step of extracting and associating the two first feature points with the two second feature points of the second target image indicating the same part on the target image;
A detection step of comparing the first line segment connecting the two first feature points with the second line segment connecting the two second feature points, and detecting their parallelism and magnification rate;
An estimation step of estimating a range of a moving body different from the host vehicle based on the parallelism or the enlargement ratio and the two first feature points or the two second feature points;
There is provided a method for detecting a moving object including:

Also, according to one embodiment of the present invention,
Since the parallelism and / or the enlargement ratio of the first line segment and the second line segment coincide with each other, the image portion including the first feature point or the second feature point is different from the own vehicle. There is provided a moving object detection method including recognizing that it is a part of one moving body.

Also, according to one embodiment of the present invention,
Whether or not the parallelism matches is provided by a method of detecting a moving body that is estimated to match even if the angle formed between the first line segment and the second line segment is within 5 °.

Also, according to one embodiment of the present invention,
In the feature point extraction step, a target region is set in the first target image, the two first feature points in different sets included in the target region, and the target associated with the two first feature points Two second feature points indicating the same part on the image are sequentially extracted, and in the detection step, a set in which the parallelism and the enlargement ratio in each set of the feature points match each other is stored. Then, a method for detecting a moving body is provided, which estimates a range of a moving body different from the host vehicle based on information including the number of sets (number of supports) that match these feature points.
Also, according to one embodiment of the present invention,
A moving object that is a different set of feature points, and that when the enlargement ratios are the same, the respective image portions including these different feature points are both included in the same moving object. A method of detecting is provided.
Also, according to one embodiment of the present invention,
Provided is a method for detecting a moving body, in which, when the enlargement ratios in the respective feature point sets are within 5% of the other, they are assumed to match each other.
Also, according to one embodiment of the present invention,
A program that is recorded in a non-volatile storage medium that detects a moving body different from the host vehicle based on a target image including surrounding information of the host vehicle and that is executed by a computer,
A preparation step of capturing the target image over a plurality of frames, storing these image information, and extracting two frame images having different imaging times as the first target image and the second target image from the image information;
A plurality of first feature points included in the first target image are extracted, two first target feature points are selected from the first feature points, and a plurality of second feature points included in the second target image are selected. A feature point extracting step of extracting and associating the two first feature points with the two second feature points of the second target image indicating the same part on the target image;
A detection step of comparing the first line segment connecting the two first feature points with the second line segment connecting the two second feature points, and detecting their parallelism and magnification rate;
An estimation step of estimating a range of a moving body different from the host vehicle based on the parallelism or the enlargement ratio and the two first feature points or the two second feature points,
Since the parallelism and / or the enlargement ratio of the first line segment and the second line segment coincide with each other, the image portion including the first feature point or the second feature point is different from the own vehicle. Recognized as part of one mobile object,
A program recorded on a computer-readable storage medium is provided.

According to the present invention, it is possible to detect a moving object located around the host vehicle with a small amount of calculation.

It is a block diagram which shows notionally the processing structure of the apparatus containing the moving object detection method in 1st Embodiment. It is a flowchart which shows the flow of a process of the apparatus containing the moving object detection method in 1st Embodiment. It is a flowchart which shows the flow of a process of the apparatus containing the moving object detection method in 1st Embodiment. It is a figure which shows an example of a 1st object image. It is a figure which shows an example of a 2nd target image. It is a figure which shows the movement vector of each feature point calculated | required based on a 1st target image and a 2nd target image. It is a figure for demonstrating the flow in which a hypothesis production | generation part sets the object area | region ROI. It is a figure which shows the state which plotted the point p of FIG. 4, the point q, and the point p 'of FIG. 5, and the point q' on the same coordinate axis.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same reference numerals are given to the same components, and the description will be omitted as appropriate.

[Processing configuration]
FIG. 1 is a block diagram conceptually showing the processing configuration of an apparatus including a moving object detection method in the first embodiment. The moving object detection apparatus 10 includes an imaging unit 110, a feature point extraction unit 120, a stationary object identification unit 130, a hypothesis generation unit 140, a hypothesis selection unit 150, and a detection unit 160.

The imaging unit 110 captures an image including a scene around the host vehicle over a plurality of frames. Here, “capturing an image over a plurality of frames” may mean capturing a moving image composed of a plurality of still images, or simply a plurality of still images at arbitrary intervals. May be taken.

The feature point extraction unit 120 is included in each of two target images (first target image and second target image) captured at two different times to be processed from a plurality of image frames captured by the imaging unit 110. Extract a plurality of feature points. The feature points extracted here may include feature points related to stationary objects and feature points related to moving objects. Then, the plurality of feature points included in the first target image are related to the plurality of feature points on the second target image indicating the same part on the target image, for example, the end of the tire. This association is performed by detecting a movement vector or the like. As a result, feature points on images taken at different times indicate the same part in an actual moving object. As described above, the above-described association is performed with respect to the plurality of feature points of the first target image and the second target image extracted here. These associations may be performed for each feature point extraction process described later, or all feature points may be associated in advance in a lump.

The stationary object specifying unit 130 specifies a feature point related to a stationary object among the feature points included in each image. The determination of whether or not the object is a stationary object will be described later. Thereby, the feature point regarding a moving object can be discriminate | determined among each feature point extracted by the feature point extraction part 120. FIG.

The hypothesis generation unit 140 includes two feature points “two first feature points” and a plurality of feature points of the first target image and the second target image extracted by the feature point extraction unit 120, respectively. Select “Two second feature points”. Then, a first line segment connecting the two feature points of the extracted first target image and a second line segment connecting the two feature points of the first target image are generated. Next, when the first line segment and the second line segment are parallel, the two feature points are based on the respective movement vectors of the two feature points between the first target image and the second target image. A selection condition, which is a condition to be satisfied by the movement vector of the object including, is generated. This will be described later.

When the hypothesis generation unit 140 generates a plurality of selection conditions, the hypothesis selection unit 150 selects a selection condition with the highest accuracy among the plurality of selection conditions.

The detection unit 160 selects a feature point that satisfies a selection condition by a movement vector between two images (first target image and second target image) captured by the imaging unit 110, and based on the selected feature point. Detect moving objects.

In addition, each component of the moving object detection apparatus 10 shown in FIG. 1 is not a hardware unit configuration but a functional unit block. Each component of the moving object detection apparatus 10 includes an arbitrary computer CPU, memory, a program for realizing the components shown in the figure loaded in the memory, a storage medium such as a hard disk for storing the program, and a network connection interface. It is realized by any combination of hardware and software. There are various modifications of the implementation method and apparatus.

[Operation example]
Hereinafter, an operation example of the moving object detection device 10 according to the present embodiment will be described with reference to FIGS. 2 and 3. 2 and 3 are flowcharts showing the flow of processing of the moving object detection apparatus in the first embodiment.

The feature point extraction unit 120 acquires a first image (first target image) and a second image (second target image) from the images captured by the imaging unit 110 (S101). The first target image and the second target image are captured at different timings. The feature point extraction unit 120 processes each image using an arbitrary algorithm, and extracts each feature point included in the image (S102). The stationary object specifying unit 130 calculates a movement vector of each feature point based on the correspondence between each feature point extracted for each image (S103). Furthermore, the stationary object specifying unit 130 detects the vanishing point of the image related to the first target image using an algorithm for detecting the vanishing point of the image (S104). Then, the stationary object specifying unit 130 selects one of the feature points extracted in S102 (S105), and compares it with a condition (stationary object condition) for determining whether or not the feature point relates to a stationary object. (S106). Specifically, the stationary object specifying unit 130 determines whether or not each feature point has a movement vector that coincides with a direction extending radially from the vanishing point of the image when the movement component due to camera rotation is canceled. Thus, the feature point related to the stationary object can be determined. When the feature point satisfies the stationary object condition (S106: YES), the stationary object specifying unit 130 excludes the feature point from the subsequent processing targets (S107). On the other hand, when the feature point does not satisfy the stationary object condition (S106: NO), the stationary object specifying unit 130 does not execute the process of S107. Then, the stationary object specifying unit 130 determines whether or not the stationary object condition has been confirmed for all feature points (S108). When unidentified feature points remain (S108: NO), the stationary object specifying unit 130 repeats the processing of S105 to S107. On the other hand, when all the feature points have been confirmed (S108: YES), the process proceeds to S109.

Here, the operation of the stationary object specifying unit 130 will be described with reference to FIGS. 4 to 6 are diagrams for explaining the operation of the stationary object specifying unit 130. FIG. 4 is a diagram illustrating an example of the first target image. FIG. 5 is a diagram illustrating an example of the second target image. FIG. 6 is a diagram illustrating a movement vector of each feature point obtained based on the first target image and the second target image.

4, the x mark is the vanishing point v detected in S104, and the points a, b, p, and q are feature points included in the first target image. In addition, the point a, the point b, the point p, and the point q exist as the point a ′, the point b ′, the point p ′, and the point q ′, respectively, in the second target image shown in FIG. Although the first target image and the second target image include feature points other than these feature points, they are not shown for convenience of explanation. The stationary object specifying unit 130 uses a point a, a point b, a point p, a point q, a point a ′, a point b ′, and a point p by using an arbitrary algorithm that specifies the correspondence of feature points between a plurality of images. The correspondence relationship with “and point q” is specified. Then, the stationary object specifying unit 130 calculates a movement vector for each feature point, as shown in FIG. 6, based on the correspondence between the feature points and the coordinate position in each image. 4 and 5 exemplify a case where the vehicle on which the camera is mounted goes straight. Here, as shown in FIG. 6, the movement vector aa ′ at the point a and the movement vector bb ′ at the point b coincide with the direction extending radially from the vanishing point v. On the other hand, the direction of the movement vector pp ′ at the point p and the movement vector qq ′ at the point q does not coincide with the direction extending radially from the vanishing point v. That is, point a and point b satisfy the stationary object condition, and point p and point q do not satisfy the stationary object condition. Therefore, the stationary object specifying unit 130 determines that the point a and the point b are feature points related to the stationary object, and excludes them from the subsequent processing targets.

The hypothesis generation unit 140 sets a target region ROI (Region of Interest) based on the remaining feature points excluding the feature points related to the stationary object (S109). A flow in which the hypothesis generation unit 140 sets the target region ROI will be described with reference to FIG. FIG. 7 is a diagram for explaining a flow in which the hypothesis generation unit 140 sets the target region ROI. As shown in FIG. 7, the hypothesis generation unit 140 recognizes an area below the horizon h that can be estimated from the vanishing point v as a ground area, and calculates a ground projection point of each feature point. Then, the hypothesis generation unit 140 calculates the distances from the reference position (for example, the installation position of the camera, etc.) to each feature point based on the calculated ground projection point. Then, the hypothesis generation unit 140 sets a region within a predetermined range as the target region ROI with reference to the feature point having the closest distance.

However, this is only an example, and various methods are conceivable for the hypothesis generation unit to set the target region ROI as follows.
(1) A rectangle surrounding a vehicle pattern found by image pattern recognition is defined as ROI.
(2) A rectangle about 2 m square in the direction and distance of the radar detection target is set as the ROI.
(3) A vehicle existence hypothesis is established for every 1 m in depth and 0.5 m in the width direction up to a distance of 50 m with respect to one's own lane and the adjacent lane, and a 2 m square is defined as ROI for each hypothesis.
(4) An ROI is set for the entire field of view, and a moving object having the maximum support described later is detected. The feature points belonging to the support are removed, and the phase object having the next largest support is detected. Repeat these. Thereby, different moving bodies can be detected sequentially.
The method for setting the target region ROI is not limited to these methods.

As described above, by setting the ROI in conjunction with the image recognition function and the radar detection processing, efficient detection processing can be performed while reducing the calculation time.

The hypothesis generation unit 140 arbitrarily selects a feature point pair from the feature points included in the target region ROI (S110). Then, the hypothesis generation unit 140 determines whether or not the feature point pair selected in S110 satisfies the trapezoidal condition (S111).

Here, the trapezoidal conditions are explained. As a premise, if the unevenness on the back of the object is sufficiently small with respect to the distance to the object with respect to the moving object common between the two images, the moving object in the other image from each feature point coordinate belonging to the moving object in one image Conversion to the feature point coordinates belonging to can be expressed by translation and enlargement / reduction. Hereinafter, deriving a feature point of another image by translation and enlargement / reduction of a feature point of an image is referred to as similar translation. Here, when a pair of feature points belonging to the same moving object is selected in each image, the four selected feature points form a trapezoid. Specifically, as shown in FIG. 8, the points p and q in FIG. 4 and the points p ′ and q ′ in FIG. 5 form a trapezoid TRP. FIG. 8 is a diagram showing a state in which the points p and q in FIG. 4 and the points p ′ and q ′ in FIG. 5 are plotted on the same coordinate axis. Thus, “a pair of feature points corresponding to two images forms a trapezoid”, in other words, a case where the angle formed by the straight line pq and the straight line p′q ′ can be recognized as parallel is a trapezoidal condition. Call. Here, the straight line pq and the straight line p′q ′ do not need to be completely parallel, and include a case where the straight line pq and the straight line p′q ′ are inclined within a certain range (for example, a range of ± 5%). The trapezoidal condition is proved as follows.

Assuming that a feature point belonging to a certain moving object has changed between two images with a translation vector t = (tx, ty) and an enlargement ratio α, the following Expressions 1 and 2 are established. Here, the coordinate origin can be arbitrarily determined. For example, if the center of the target area ROI is the coordinate origin, the meaning of enlargement and parallel movement can be understood more intuitively.

According to Equation 1 and Equation 2 above, p′−q ′ = α * (pq), so that two sides (side pq and side p′q ′) are parallel in the quadrangle (pp′q′q). I understand that That is, it can be said that the quadrangle (pp′q′q) forms a trapezoid. Further, based on the four points forming the trapezoid, the enlargement ratio α and the translation vector t can be expressed by the following equations 3 and 4.

Here, the establishment of the trapezoidal condition can be confirmed by whether or not the following simple equation is satisfied within a predetermined tolerance. The coordinate transformation calculation of Equation 1 and Equation 2 requires 4 multiplications and 8 additions / subtractions, whereas when the following Equation 5 is used, 2 multiplications and 5 additions / subtractions are required.

3, when the feature point pair selected in S110 does not satisfy the trapezoidal condition (S111: NO), the hypothesis generation unit 140 reselects the feature point pair (S110). On the other hand, when the feature point pair selected in S110 satisfies the trapezoidal condition (S111: YES), the hypothesis generation unit 140 generates a similar translation hypothesis from the selected feature point pair (S112). The similar parallel movement hypothesis can also be called a selection condition for selecting feature points belonging to a moving object. Here, the similar translation hypothesis is that each feature point constituting the same object has the same enlargement ratio and translation vector. Specifically, the hypothesis generation unit 140 calculates the enlargement ratio α and the parallel movement vector t based on the movement vectors related to the two feature points. Then, the hypothesis generation unit 140 verifies the movement vector of each feature point included in the target region ROI, and calculates the number of feature points (supports) that satisfy the similar parallel movement hypothesis generated in S112 (S113). Here, the feature point (support) satisfying the similar translation hypothesis is specifically a feature point having an enlargement factor and a translation vector that coincide with the enlargement factor α and the translation vector t generated in S112. . The part corresponding to the feature point having the same selection condition is a condition that is recognized as a part of one moving body. Note that “match” here is not so-called perfect match, but to some extent (for example, in the case of parallelism, the mutual angle is within 5 °, and in the case of magnification, one is within ± 5% of the other. Etc.). Then, the hypothesis generation unit 140 determines whether a predetermined number of similar parallel movement hypotheses has been generated (S114). When a predetermined number of similar translational hypotheses have not been generated (S114: NO), the hypothesis generation unit 140 selects a new feature point pair (S110), and repeats the processing of S111 to S113 a predetermined number of times. On the other hand, when a predetermined number of similar translation hypotheses are generated (S114: YES), the hypothesis selection unit 150 selects selection conditions (that is, enlargement ratio α and translation) extracted based on the plurality of generated similar translation hypotheses. A feature point group of selection conditions having the maximum number of support is selected from the conditions defined by the vector t) (S115). The detection unit 160 recognizes a region related to the moving object based on the feature point group satisfying the similar parallel movement hypothesis selected in S115 (S116). Then, the hypothesis generation unit 140 excludes the feature points included in the moving object area recognized in S116 from the processing target (S117), and repeats the above-described processing. This process is repeated until there are no more feature points to be processed. Through this iterative process, it is possible to sequentially extract and estimate a plurality of moving bodies different from the host vehicle included in one target image pair (the first target image and the second target image). . Since the moving object can be recognized based on one similar parallel movement hypothesis generated in S112, the processes in S114 and S115 are not necessary. The presence of the processes of S114 and S115 can improve the accuracy of the similar translation hypothesis.

As described above, according to the present embodiment, each feature point belonging to the moving object can be easily calculated based on the similar parallel movement hypothesis calculated using at least two feature points among the plurality of feature points belonging to the moving object. It is possible to discriminate and specify an area related to the moving object. Thereby, according to this embodiment, a moving object can be detected with a small calculation amount. Since more hypotheses can be generated and tested in a limited calculation time, the probability of selecting an optimal similar translation transformation is increased.

Further, in the present embodiment, it is possible to determine a feature point group that moves in the same manner between a plurality of images based on the similar parallel movement hypothesis. Here, since each feature point belonging to the same object follows the movement of the object, it is considered that the feature point moves in substantially the same manner between different images. Therefore, according to this embodiment, the area | region regarding each moving object can be specified only by confirming the motion of each feature point between images based on the similar parallel movement hypothesis. As a result, for example, an exceptional object that is likely to be missed by a method such as pattern recognition, for example, a positive sample distribution when generating a pattern discriminator has an appearance probability higher than a similar negative sample in the vicinity. It is possible to detect by supplementing an object that is difficult to recognize because it is small) or an unknown object (for example, an object for which a similar positive sample is not prepared in advance).

In this embodiment, it is sufficient that at least two feature points are selected by random sampling, and the probability of generating a correct hypothesis by random sampling can be increased. For example, when eight feature points are selected from samples in which 50% is noise, the probability that all selected feature points do not contain noise is about 1/256. On the other hand, according to the present embodiment, since at least two points may be selected, the probability that all feature points do not include noise can be reduced to about ¼ at maximum. As a result, the number of samplings required to obtain a correct hypothesis can be reduced.

In the above embodiment, an example using a pair of feature points is shown. However, a third feature point that satisfies the trapezoidal condition by the pair of feature points is extracted, and each feature point pair and each third feature point are extracted. Based on this, one similar translation hypothesis may be generated. Specifically, three trapezoids can be generated from each feature point pair and the third feature point in each of the two images. Then, for each of the three trapezoids, the enlargement ratio α and the translation vector t are calculated, and one similar translation hypothesis is calculated using the average value, intermediate value, and the like. By doing so, the accuracy of the similar translation hypothesis can be improved. As a result, it is expected that correct parallel translation can be detected with fewer hypothesis generation times.

DESCRIPTION OF SYMBOLS 10 Moving object detection apparatus 110 Image pick-up part 120 Feature point extraction part 130 Stationary object specification part 140 Hypothesis generation part 150 Hypothesis selection part 160 Detection part

Claims

A method of detecting a moving body different from the own vehicle based on a target image including surrounding information of the own vehicle,
Capture the target image over a plurality of frames, store these image information,
A preparation step of extracting two frame images having different imaging times from the image information as a first target image and a second target image;
A plurality of first feature points included in the first target image are extracted, two first target feature points are selected from the first feature points, and a plurality of second feature points included in the second target image are selected. A feature point extracting step of extracting and associating the two first feature points with the two second feature points of the second target image indicating the same part on the target image;
A detection step of comparing the first line segment connecting the two first feature points with the second line segment connecting the two second feature points, and detecting their parallelism and magnification rate;
An estimation step of estimating a range of a moving body different from the host vehicle based on the parallelism or the enlargement ratio and the two first feature points or the two second feature points;
For detecting moving objects including
Since the parallelism and / or the enlargement ratio of the first line segment and the second line segment coincide with each other, the image portion including the first feature point or the second feature point is different from the own vehicle. The method for detecting a moving body according to claim 1, wherein the moving body is recognized as a part of one moving body.
3. The method for detecting a moving body according to claim 1, wherein whether or not the parallelism matches is estimated to match even if an angle between the first line segment and the second line segment is within 5 °.
In the feature point extraction step, a target region is set in the first target image, the two first feature points in different sets included in the target region, and the target associated with the two first feature points Sequentially extract two second feature points indicating the same part on the image,
In the detection step, a set in which the parallelism and the enlargement ratio in the respective feature point sets match each other is stored, and information including the number of sets (support number) in which these feature points match is stored. 4. The method for detecting a moving body according to claim 1, wherein a range of the moving body different from the own vehicle is estimated.
5. It is presumed that the image portions including the different feature points are included in the same moving object when the enlargement factors are different from each other in the different sets of the feature points. To detect moving objects.
The method for detecting a moving object according to claim 4 and 5, wherein the enlargement ratio in each of the set of feature points is estimated to match each other when one is within 5% of the other.
Based on the target image including the surrounding information of the host vehicle, a moving body different from the host vehicle is detected.
A program recorded in a non-volatile storage medium and executed by a computer,
A preparation step of capturing the target image over a plurality of frames, storing these image information, and extracting two frame images having different imaging times as the first target image and the second target image from the image information;
A plurality of first feature points included in the first target image are extracted, two first target feature points are selected from the first feature points, and a plurality of second feature points included in the second target image are selected. Extract and
A feature point extracting step for associating the two first feature points with the two second feature points of the second target image indicating the same part on the target image;
A detection step of comparing the first line segment connecting the two first feature points with the second line segment connecting the two second feature points, and detecting their parallelism and magnification rate;
An estimation step of estimating a range of a moving body different from the host vehicle based on the parallelism or the enlargement ratio and the two first feature points or the two second feature points,
Since the parallelism and / or the enlargement ratio of the first line segment and the second line segment coincide with each other, the image portion including the first feature point or the second feature point is different from the own vehicle. Recognized as part of one mobile object,
A program recorded on a computer-readable storage medium.