JPH0290381A - Running path discriminating method - Google Patents

Abstract

PURPOSE:To execute discrimination at high speed by dividing an image obtained by bringing a running path to image pickup into areas of this side and a remote side, extracting a first straight lien corresponding to one running path end in the this side area and extracting a second straight line of the other which passes through the vicinity of an intersection to its area boundary. CONSTITUTION:Image data of a running path which is inputted by a camera 1 is stored temporarily in an image memory 3. Subsequently, a straight line group corresponding to the feature of an image is obtained by a DDA arithmetic part 5 through a pre-processing part 4, and sent to a straight line extracting/ selecting part 9 after filtering, sorting and clustering are performed. The image is divided into a this side area and remote side area, and in the this side area, the first straight line corresponding to one running path end is extracted. Next, the second straight line corresponding to the other running path end is extracted from the straight line group which passes through the vicinity of the intersection of both boundaries and the first straight line. A discriminating part 10 discriminates the shape of the running path, based on the first straight line and the second straight line, and the shape of the running path can be recognized at high speed.

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial Field of Application] The present invention relates to a roadway discrimination method that can easily determine the direction in which the edges of the roadway continue when the shape of the roadway is recognized by image processing.

[Conventional technology]

In order to recognize the shape of a travel road, it is necessary to extract its edges (traveling road edges), and it is important to easily determine the direction in which they continue. For this purpose, it is necessary to process the image data obtained by the camera, and there is a method called Hough transformation, for example (Japanese Patent Laid-Open No. 62-24310, No. 62
-70916 etc.). According to this, for each edge point of the differential image of the original image obtained as the image feature amount, Ho
By performing the gh transformation, a group of straight lines corresponding to the characteristics of the pixel distribution can be obtained.

[Problem to be solved by the invention]

However, while the number of straight lines obtained in this way is extremely large, if the traveling route is curved, many straight lines with different attributes will be included, so the direction in which the traveling route is continuous on the original image It is not easy to determine. By the way, if we examine an image obtained by capturing a driving route from the viewpoint of the linear arrangement of its feature quantities, we will find that there are areas in the image where features with a linear arrangement of feature quantities often appear and areas where they do not appear often. be. In boat fishing, it is easy to extract a straight line that approximates the arrangement of features in areas where the above-mentioned features often appear, but it is difficult to extract the above-mentioned straight lines in areas where features do not appear much.

However, since the conventional method is based on performing image processing on these areas all at once, the shape of the traveling road cannot be determined easily and quickly. The present invention has been made to solve such problems.

[Means to solve the problem]

A first aspect of the driving path determination method according to the present invention is a driving path determination method in which the shape of the driving path is determined from the image feature amounts included in the image obtained by imaging the driving path. A first step of dividing into at least two regions including a first region in which the most characteristic appears and a second region adjacent to the first region in a direction in which the arrangement is continuous, and characteristics in each region. a second step of finding a straight line group consisting of a plurality of straight lines corresponding to the amount, and a third step of extracting the first straight line that approximates the arrangement of the feature amounts that appear corresponding to one running road edge in the first region. Then, the arrangement of feature values that appear corresponding to one running road edge in the second region and the approximated second straight line are extracted from a group of straight lines passing near the intersection of the first straight line and the boundary of the first and second regions. The fourth step is to
and a fifth step of determining the shape of the travel path based on the second straight line.

Further, in the second aspect of the present invention, in addition to the first and second steps, in the third step, the arrangement of the feature amounts appearing corresponding to one and the other traveling road edges in the first region is approximated. The first pair of straight lines are extracted, and in the fourth step, the second pair of straight lines approximated by the features that appear corresponding to one and the other road edge in the second area are extracted. The straight lines are extracted from a group of straight lines that pass near the intersection of the region boundary and the first pair of straight lines, and in the fifth step, the shape of the travel path is determined based on the first and second pairs of straight lines. .

[Effect]

According to the present invention, a first straight line (straight line pair) is first extracted in a region (first region) where it is easy to extract an arrangement of image features and an approximate straight line, and based on this, an arrangement of image features is approximated. A second straight line (straight line pair) is extracted from a region (second region) in which it is difficult to extract a straight line. The first straight line and the second straight line intersect near the boundary between the first and second regions.

〔Example〕

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 shows the basic concept of the invention. First, assume that a scene captured by a camera mounted on a traveling vehicle is, for example, as shown in FIG. 1(a). That is,
There is a guardrail 52 at the left end of the running road with the running road edge 51 as a boundary, a furrow 53 on the right side, and the running road edge 51 extends to the horizon 54. Such a scene is captured as an original image, and the edge data obtained by differentiation is used as a dot 56.
For example, if expressed as shown in FIG. 1(b). Therefore, by performing the Hough transformation described later on this dot 56 to obtain a Hough curve, and counting the intersection points, a straight line L as shown in Fig. 1(C) is obtained.
A group of straight lines including -L5 etc. is obtained.

When the results of this Hough transformation are subjected to filtering processing, sorting processing, and clustering processing, which will be described later, it is possible to obtain characteristic lines and approximate straight lines of the edge data (dots 56) as representative values. Therefore, if we focus on this representative value and perform processing,
Although the straight line corresponding to the running road edge 51 can be found,
As is clear from Figure 1(c), the lower part of the original image (
On the near side), the feature of the straight line corresponding to the road edge 51 appears relatively clearly, whereas on the upper side (far side) of the original image, the straight line corresponding to the road edge 51 corresponds to another scene. It is mixed with straight lines, making it difficult to distinguish the characteristic parts.

Therefore, a first area A1 is set as shown in FIG. 1(d) for the near side area where the arrangement of dots 56 characteristically appears in the original image, and the area corresponding to the road edge 51 is set in this area. Find a straight line. To find this straight line, it is sufficient to check the above-mentioned representative values, and in most cases, the straight line with the maximum representative value corresponds to the road edge 51. Now, assuming that the straight lines p and ρ'1 corresponding to the road edge 51 correspond to the maximum representative value and the next representative value, this straight line j71°g'1 is expressed as shown in Fig. 1(d). be painted.

Next, a second area A2 adjacent to this first area A1 is
The second area A2 is set as shown in Figure (e), and the second area A2 is set in the direction in which Ω and ρ'1 in the first area A1 are continuous. Then, straight lines p and ρ'2 corresponding to the road edge 51 in the second area A2 are extracted. In this case, the straight lines Ω, ρ′
The intersection points P and Q of and #l)' should be located near the boundary line between the first area A1 and the second area A2, as shown in FIG.
, ρ'l is found, the second ■ Ω in area A2, j! ′ are points P, Q
It can be found from among the straight lines passing through the vicinity of 2 2 l 1 . Moreover,
Since the intersection of the straight lines p and Ω'2 can be predicted to be near the horizon H8, this extraction is easy.

Furthermore, a third area A3 is set as shown in FIG. 1(f) following the second area A2, and a straight line 1. Find 1'3. This straight line ρ, 11'
Also, the straight lines ρ, ρ′2 in the second region A2
can be easily found as well. In this way, the present invention first performs processing to extract an approximate straight line for a region (first region AI) in which the arrangement of image data (dots) is characteristic, and then extracts an approximate straight line based on this. A
2 or less).

Next, the present invention will be sequentially explained according to examples.

FIG. 2 is a conceptual diagram illustrating a device to which an embodiment of the present invention is applied as a combination of function realizing means.

As shown in the figure, the image data of the driving route captured by the camera 1 is converted into digital data by the A/D converter 2 and temporarily stored in the image memory 3.

The image data read out from the image memory 3 is sent to the preprocessing section 4.
Here, edge detection by differentiation, threshold processing by a look-up table (LUT), area setting, coordinate transformation, etc. are performed. The DDA calculation section 5 performs so-called digital differential analysis (Digital Difference Analysis).
ial analysis).
The u gh transformation is done in a pipelined manner.

Then, by this Hough transformation, a group of straight lines corresponding to the features of the image is obtained.

The data related to the straight line group obtained by Hough transformation is given to the neighborhood filtering section 6, where it is subjected to, for example, 8 neighborhood filtering processing, and then given to the peak sorting section 7, and furthermore, it is given to the peak sorting section 7 by the clustering processing in the clustering section 8 to A value is selected and sent to the straight line extraction/selection section 9. The data regarding the extracted and selected straight lines is sent to the determining section 10, where the shape of the traveling path is determined. The determination output is sent to an external device (not shown) and also to the monitor control unit 11,
It is displayed on the monitor 12 together with the image of the running route. Note that the operation of the above function realizing means is controlled by the controller 13. Further, the initial setting section 14 is used to initialize data such as travel road width.

Next, the effects of the embodiment applied to the above device will be sequentially explained.

FIG. 3 is a flowchart 1 showing the overall process flow. First, when starting the process, initial settings are performed (step 201). This initial setting includes the driving road width WR.
In addition to the setting, the depression angle θ of camera 1. , camera] lens focal length f1 of the camera 1 and the mounting height H of the camera 1. This will be explained with reference to FIGS. 4 and 5.

As shown in FIG. 4(a), a traveling vehicle 3 is placed on a traveling path 31.
2 exists, and it is assumed that the camera 1 is fixed to the traveling vehicle 32. Then, the depression angle θ of camera 1. The mounting height H8 can be set as a known value based on the structure. Further, the lens focal length f can also be set as a known value based on the structure of the camera 1. In addition, Fig. 4 (b
) is an enlarged view of the positional relationship of the optical system in figure (a). On the other hand, the traveling road width WR is set as shown in FIG. 5. Once the above data is set, the fifth
The running road width wr at the lower end of the image as shown in the figure can be determined. That is, the running road width wr on the image is wr = (WR' ・f) / ((L + H - 1anθ) COSθ)low
co.

...(1) becomes. Here, L is low in FIG. 4(b) and I L =H/jan(H+jan (y
/f))ow c o
ow... (2) WR' is determined from the inclination angle ψ in FIG. 5(b) as WR' = WR/sin ψ - (3).

Next, in step 202 in FIG. 3, the horizon on the image is calculated. This horizon position H is shown in Figure 4(b)
, it is calculated as H=f-tan θo-(4). Then, as step 204 in FIG. 3, a first area A1, a second area A2, and a third area A3 are set. That is, as shown in FIGS. 1(d), (e), and (f), the near side of the running path 31 is defined as a first area A1, the front side is defined as a second area A2, and the far side is defined as a third area 8. 3
shall be.

The ratio of the first area A1 to the second area A2 and the ratio of the second area A2 to the third area A3 are each about 3:2,
In addition, the upper end of the third area A3 is made to substantially coincide with the horizon position H. In addition, in the embodiment, in order to simplify the explanation, the first . Second area AI, A2
There are two areas.

After the above-mentioned pre-processing is completed, the image data processing is repeated. That is, first, in step 205 in FIG. 3, image data is input as digital data from the image memory 3 in FIG. 2, and preprocessing section 4 performs predetermined preprocessing.

Here, for example, edge detection such as 5 obel is performed, and the DDA calculation unit 5 in FIG.
Perform Hough transformation. Is it possible to perform this Hough conversion in a pipeline system using multiple DDA calculation circuits (not shown) connected in series? For an overview of the process, see, for example, U.S. Pat. It is shown in Sho 63-112243. That is, an edge image is obtained by differentiating the original image, and a straight line is drawn from a predetermined origin to each edge point therein. Then, if the distance between this origin and each edge point is ρ, and the angle between this straight line and the horizontal axis is θ, and the value of θ is changed, H
A sine curve called an ough curve can be drawn on the ρ, θ plane.

This Hough curve is different for each edge, and these curves have intersection points on the ρ and θ planes, so by examining the histogram of these intersection points, we can calculate the alignment of the edge points and the approximate straight line g with the values of ρ and θ. It can be found by

This will be briefly explained in the sixth drawing.

Now, the edge point on the x-y plane (original image plane) is the sixth
EP, EP as shown in figure (a).

Assume that there are 3 points for EP2. Then, each edge point EP
One Hough curve (sine curve) for each o-EP2 is drawn on the ρ-θ plane, and the two Hough curves due to the edge points EPo and EP are (ρ
, θ ), and the two Hough curves formed by edge points EP 1 and EP2 have an intersection at (ρ , C2).

Therefore, as shown in Figure 6(b), 15 of EPo-EP14
If these edge points are lined up, one Hough curve will be drawn for each edge on the ρ-θ plane. And the 9 R's of edge point EP-EP8
The Hough curve intersects at (ρ, θ), and the three Hough curves from edge point EP8 to CCE Pto intersect at (ρ, θ).
ρb.

For the five Hough curves of edge points EP1o-EP14, they intersect at (ρ, θ).
a. To bite.

As is clear from the above description, by counting the frequency of intersections of Hough curves, it is possible to obtain a straight line that approximates the arrangement of edge points. Then, by weighting this with the brightness of the edge points, it becomes possible to recognize the characteristic arrangement of data in the image by knowing the histogram of the intersection points.

Next, the filtering process corresponding to step 210 in FIG. 3 will be explained with reference to FIG.

FIG. 7 shows count values (a histogram of intersection points weighted by brightness) at the ρ and θ coordinates. Here, the 8-neighborhood filtering process applies 1 j to the count value C0 at a certain (ρ6.θ,)
1. This is done by comparing the count values with eight nearby count values. That is, the count value of (ρ, θ, ) ~ 1-I J-1 (ρ, , θ, ) is C1-1,,+-1
i ten l J+1 C9, when closed, C9, either color 1+1.3
+1 When it is greater than 1+Conte value, this is extracted as a peak. Therefore, for example, in the range indicated by the symbol F1 in FIG. 7, (ρ,
θ )-7 is extracted as a peak, but (ρ , θ )-6, indicated by symbol F2, is not extracted as a peak.

After the filtering process, a sorting process is performed. This is to rearrange the data in descending order of the count value C described above, and may be realized by software or by dedicated hardware.

After the above sorting process is completed, representative values are selected by clustering process (step 213 in FIG. 3). This will be explained with reference to FIGS. 8 to 10.

FIG. 8 shows the sorted data. Here, the count values C, C, C2, . . . C are set as C>C>C> . . .>C by the sorting process. For the OJ, 2n-1 rusk ring process, set =o and i=1 as shown in the flowchart of FIG. 9 in step 501.
), the processing of steps 502 to 515 is repeated. Therefore, a general explanation of this process using the above i and k will be as follows.

First, in step 502, it is checked whether the i-th straight line that is ink is already a member of another straight line, and if it is not a member, ρ −Δρ≦ρ・≦ρ +Δρ is 1 1 is checked for θ −Δθ≦θ, ≦θ and +Δθ (step 504). Then, only when the above relationship is established, the i-th straight line is determined to be a member of the i-th straight line (step 505). Next, 1 is added to i (step 507), and only when data is present (step 508), it is checked whether the i-th count value C3 is too small compared to the maximum value (step 510). This is because if the count value is too small, there is little need for processing.

If it is not too small, the process returns to step 502, but if it is too small, 1 is added to k (step 511).
), another clustering is started at step 513. That is, it is checked whether the kth straight line is already an interpolation of another straight line (step 51B), and if it is a member, 1 is added to k in step 511, and step 513 is performed again. executed. If they are not friends, step 502 is executed again for the i (=+l)th one after the kth one (step 515).

A specific example of clustering performed by the above process is shown in FIG. In the same figure, the first representative value is p =
200 (dots), θ=45Cdegl, and the second representative value is p=2401:dots:l,
It can be seen that θ=123Cdeg).

After the above processing, straight line extraction/selection of the first area A1 shown in step 214 in FIG. 3 is performed.

FIGS. 11 to 13 are flowcharts thereof.

To extract straight lines in the first area A1, as shown in the flowchart of FIG. It is checked whether it is on the right side or on the left side (step 552). When it is on the right side, the other first line ρ2 should be on the left side, so its theoretical value is calculated (step 554), and when it is on the left side, the other first line Ω2 should be on the right side, so its theoretical value is found. required (step 5
55). Here, the straight line N = (ρ, θ)1 1 on the right side of the straight line ρ1 = (ρ, θ)
The theoretical value of 2r 2r 2r is ρ = HΦsin θ2°2r.

=1 θ2. = tan ([(ρ1/cos θ1+Wr
]/H8 Left straight line Ill = (ρ, θ) Theory 2
ρ 2. & 2. The Q value is = H-5in 02Ω ρ2Ω O e =jan -' ([p/cos e
'-wr: 12 Ω 11/
It is found as H8.

Then, until there is a representative value other than the representative value corresponding to the first straight line Ω1 that is close to the theoretical value (step 557)
, the above operations are repeated (step 558).

Here, if one of the first straight lines p1 is as shown in FIG. 14, the theoretical value can be found from the relationship between the travel road image width wr and the horizon position H, and the representative value corresponding to this theoretical value Assume that the straight line is L2 in FIG. 14 as the processing result of step 560. At this time, a straight line determination operation is performed using this as a candidate straight line for the other first straight line Ω2 (step 561).

The details of the process in step 56] are as shown in FIG. First, a group of candidate straight lines for the other first straight line ρ2 is created (step 571).

There are ways to create this group, such as grouping together representative values within a predetermined range, or interpolating the number of divisions during Hough transformation, but in any case, in addition to the straight line L2 close to the theoretical value, the final A group is created that includes the other first straight line g2 selected as follows (see FIG. 14).

Next, the number of edge points EP in FIG. 14 below each candidate straight line is counted (step 572), and the one with the largest count is set as the final other first straight line Ω2. In this way, the original first straight line g2 can be selected without being confused by straight lines, L2', etc.

When the processing of the first area A1 as described above is completed, the processing of the 13th area A1 is completed.
Processing of the second area A2 as shown in the figure is performed.

First, in step 581, the first pair of straight lines g in the first area A1 is
, ρ2 and the upper end of the first area A1] P and Q are determined as shown in FIG. Next, a straight line passing near point P is selected as a candidate for one of the second straight lines g3 (step 582). Then, when performing the straight line determination operation as shown in Fig. 12 on this straight line (584), the nearby straight line (
For example, one of the original second straight lines 93 can be extracted without being confused by L3) in FIG. Thereafter, the other second straight line Ω4 is extracted (step 585), and this extraction is performed in the same manner as steps 551 to 560 in FIG. Finally, by performing the straight line determination operation as shown in FIG. 12, the other second straight line Ω4 can be extracted without being confused by the arrow 4 in FIG.

When the above processing is completed, the shape of the travel path is finally determined (step 217 in FIG. 3).

〔Effect of the invention〕

As described in detail above, according to the present invention, for a sequence of image features, a first straight line (straight line pair) is first extracted in a region (first region) in which it is easy to extract a straight line that approximates it. , based on this, the arrangement of the image features and the second straight line (straight line pair) in the region (second region) where it is difficult to extract an approximate straight line
is extracted. Here, by utilizing the fact that the first straight line and the second straight line intersect near the boundary of the first and second regions, the second straight line (straight line pair) can be easily extracted. . Therefore, it becomes possible to quickly and easily determine the travel route.

[Brief explanation of drawings]

FIG. 1 is an explanatory diagram of the basic concept of an embodiment of the present invention, FIG. 2 is a conceptual diagram of a function realizing means in a device to which the embodiment is applied, and FIG. 3 is a flowchart showing the overall processing. Figure 4 is an explanatory diagram of the mounting position of the camera, Figure 5 is an explanatory diagram of the relationship between the vehicle condition on the road and the image, and Figure 6 is:
FIG. 7 is an explanatory diagram of the Hough transformation. FIG. 7 is an explanatory diagram of the filtering process. FIG. 8 is an explanatory diagram of the results of the Hough transformation and sorting process. FIG. 9 is an explanatory diagram of the clustering process. An explanatory diagram of a specific example of the results of the clustering process, FIG. 11 is a flowchart for extracting and selecting straight lines in the first area A1, FIG. 12 is a flowchart for straight line confirmation operation, and FIG. 13 is a flowchart for straight line determination in the first area A1 A2
FIG. 14 is a diagram illustrating the extraction and selection of straight lines Ω1 to g4 on an image. DESCRIPTION OF SYMBOLS 1... Camera, 2... A/D conversion part, 3... Image memory, 5... DDA calculation part, 6... Neighborhood filtering part, 7... Peak sorting part, 8...・Clustering section, 9...Line extraction/selection section, 10...
- Discrimination unit, 11... Monitor control unit, 12... Monitor, 13... Controller, 14... Initial setting unit, Ω
1... One first straight line, ρ2... Other first straight line, g3... One second straight line, Ω4... Other second straight line, A1... First Area, A2...second area,
A3...Third area. Patent Applicant: Honda Motor Co., Ltd. Representative Patent Attorney Yoshiki Hase

Claims (1)

[Scope of Claims] 1. In a driving route discrimination method for determining the shape of an image from image feature quantities included in an image obtained by capturing an image of a running road, the image is characterized in that the arrangement of the image feature quantities is the most characteristic. a first step of dividing into at least two regions including a first region that appears and a second region adjacent to the first region in a direction in which the arrangement continues, and corresponding to the image feature amount on the image; a second step of finding a straight line group consisting of a plurality of straight lines, and a third step of extracting a first straight line that approximates the arrangement of the image features appearing in the first region corresponding to one of the road edges. and a second straight line approximated by the arrangement of the image features appearing in the second region corresponding to one road edge, and a second straight line approximated by the boundary between the first and second regions and the first straight line. A driving path determination method comprising: a fourth step of extracting from a group of passing straight lines; and a fifth step of determining the shape of the driving path based on the first and second straight lines. 2. The driving path determination method according to claim 1, wherein the second step is executed prior to the first step. 3. In a driving road discrimination method that determines the shape of an image from image features included in an image obtained by capturing an image of a driving road, the image is defined as a first region in which the arrangement of the image features most characteristically appears; a first step of dividing into at least two regions including the first region and an adjacent second region in a direction in which the arrangement is continuous; and a plurality of straight lines on the image corresponding to the image feature amount. a second step of obtaining a group of straight lines; a third step of extracting a first pair of straight lines that approximates the arrangement of the image features that appear corresponding to one and the other road edge in the first region; A second pair of straight lines approximated by the arrangement of the image features that appear corresponding to one and the other road edge in the second region,
a fourth step of extracting from a group of straight lines passing near the intersection of the boundary of the first and second regions and the first pair of straight lines; and determining the shape of the traveling path based on the first and second pair of straight lines. A driving route determination method, comprising: a fifth step. 4. The driving route determination method according to claim 3, wherein the second step is executed prior to the first step. 5. The third step is to extract one of the first straight lines that approximates the arrangement of the image features that appear in the first region corresponding to one of the road edges, and to The fourth step includes the step of selecting the other first straight line that appears corresponding to the other running road edge based on preset running road width information, and the fourth step includes selecting one of the running road edges in the second area. extracting one of the second straight lines approximated to the arrangement of the image features that appears corresponding to from a group of straight lines passing near the intersection of the boundary of the first and second regions and the one of the first straight lines, and 4. The method according to claim 3, further comprising a step of selecting the other second straight line that appears corresponding to the edge of the other running road based on the one second straight line and preset running road width information. Driving route determination method.