JP2001022918A - Method and device for recognizing traveling course of vehicle - Google Patents

Abstract

PROBLEM TO BE SOLVED: To recognize an image of a traveling course by discriminating traveling lines, such as median line and shoulder line forming the traveling course with high accuracy in a short time by determining objects on both bands in position relation of continuation as the same traveling line. SOLUTION: A CPU detects respective left and right detected band-shaped objects, namely, traveling lines A1 and A2 forming the real traveling course and a pair of band-shaped objects with high possibility to be both the left and right side traveling lines A1 and A2 forming actual traveling course in a linear part of a road sign B. As traveling line determining processing to be executed next to the left/right traveling line detecting processing, the CPU decides whether the first band-shaped object detected from first road surface images AL and AR provided at a previous time point in two road surface images LA and LR provided successively with the forward movement of a vehicle and whether the second band-shaped object provided at a following time point exist in the continuous position relation between both the road surface images AL and AR at least and both the band-shaped objects in the continuous position relation are determined as the same traveling lines A1 and A2.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and apparatus for recognizing a traveling road by recognizing a traveling road by determining a traveling line from a road surface image of a traveling road on which a vehicle travels.

[0002]

2. Description of the Related Art Conventionally, when a vehicle travels on a traveling road divided by a traveling line (a so-called white line) such as a center separation line and a road shoulder line, the vehicle itself recognizes the traveling road, and a driver can recognize the traveling road from the traveling road. There are attempts to inform the driver of deviations, actively assist driving, and even perform automatic driving. On this occasion,
A method of recognizing a traveling road by recognizing an image of a traveling line forming the traveling road has been studied.

[0003] As a method of recognizing a traveling line by image recognition, a method based on template matching for extracting an area corresponding to a standard pattern of a traveling line from an image of a traveling path picked up by a CCD camera, and a differential processing of a shading gradient of the image by a differentiation process. A method of recognizing a contour of a traveling line by extracting the line is known.

[0004] Among them, in the traveling line recognition method using template matching, the state of the traveling line with respect to the vehicle is recognized by extracting an area where the standard pattern of the traveling line matches from the road surface image of the traveling road. For this reason, a number of standard patterns are prepared in accordance with the state of the traveling line obtained as an image, that is, bending to the left and right, the degree of bending, and the like.
It is determined whether or not the partial area of the image including the running line has a correlation with any of the standard patterns. Therefore, in the method using template matching, it is possible to recognize a traveling line with high accuracy even if an unnecessary object such as a road sign on the traveling road is included in the image. However, there is a problem that the number of calculations for measuring the correlation between the partial area including the travel line and each standard pattern becomes enormous, and it takes time to recognize the travel line.

On the other hand, in a method of recognizing a contour of a travel line from an image by differential processing, an obtained image is subjected to differential processing to extract a contour of an object, and the obtained contour is a characteristic of the travel line. The traveling line is recognized by determining whether or not the vehicle is equipped. Therefore, the number of calculations required for recognition of the traveling line is smaller than that of the template matching method, and the time required for recognition is reduced.

[0006]

However, in the method of extracting the contour line of the center separation line or the shoulder line, the method of detecting the contour of a paving stone on a traveling road surface, a shadow formed on the traveling road surface, a road sign, and the like is also required. Become. Then, there is a problem that it is not easy to discriminate an object other than the traveling line from the traveling line, and it is not easy to recognize the traveling line with high accuracy.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and has as its object to reduce the traveling time of a central separation line, a road shoulder line or the like which forms a traveling path in a shorter time and with high accuracy. It is an object of the present invention to provide a method and apparatus for recognizing a traveling path in a vehicle capable of recognizing a traveling path by distinguishing a line.

[0008]

According to a first aspect of the present invention, there is provided an image pickup apparatus, which is imaged in a predetermined positional relationship with respect to a vehicle, and is provided with a traveling axis set in a traveling direction of the vehicle. Expressed by coordinates with the horizontal axis set in a direction perpendicular to the traveling direction, from a road surface image sequentially captured including at least one of the left and right traveling lines forming a traveling path on which the vehicle travels, Of each image unit forming a line area extending in the horizontal axis direction from each position of the traveling axis,
An edge detection process of generating edge data in which the density change value in the horizontal axis direction is made to correspond to each position of the horizontal axis; and a plurality of divided areas divided by a predetermined division width in the traveling axis direction. An image dividing process for dividing, and, for each of the divided regions, a density change value at the same position on the horizontal axis of each of the peripheral data from the edge data generated for each of the line regions of the divided region. And the cumulative change value reflecting at least one of the frequencies of the density change values that are equal to or more than a predetermined value at the same position on the horizontal axis of each edge data is made to correspond to each position on the horizontal axis. Edge region detection processing for generating edge region data, and for each of the divided regions, in the edge region data generated for the divided region, the sign of the cumulative change value is the same and The position range which is equal to or larger than a predetermined size; A boundary region detection process of detecting a boundary region in which edges passing through both end edges of the divided region that face each other in the traveling axis direction, and the traveling axis of the divided region in which each of the boundary regions is detected. Position with respect to
From the position of the boundary area with respect to the horizontal axis detected in each of the divided areas, the sign of the value of the cumulative change value in the boundary area, and the division width of the divided area, the margin of each of the boundary areas is determined. Is constituted by the recognition object where
A band-shaped object detection process for detecting a band-shaped display portion that is present in at least a continuous positional relationship with a partial region of the road surface image formed by each divided region in which each of the boundary regions is detected;
In the two road surface images sequentially obtained with the traveling of the vehicle, a first band-like object detected from a first road surface image obtained at a previous time and a second band-like object obtained at a later time It is determined whether or not the second belt-shaped object detected from the road surface image is present at least in a continuous positional relationship between the two road surface images, and the two belt-shaped objects having the continuous positional relationship are traveled in the same traveling state. This is a method for recognizing a traveling route in a vehicle that performs a traveling line determination process for determining a traveling line.

According to a second aspect of the present invention, in the first aspect of the present invention, the road surface image is a road surface image including left and right traveling lines forming the traveling road, and is different from the road surface image. Based on the positions of the plurality of band-shaped objects detected by the respective processes up to the band-shaped object detection process with respect to the horizontal axis, based on the distance between the left and right traveling lines forming the actual traveling path, The left and right traveling line detection processing for detecting a pair of band-shaped objects in which the distance between both of the band-shaped objects is equal to the distance between the traveling lines on both the left and right sides that form the actual traveling path is performed. The line recognition processing is characterized in that the traveling line is determined for each of the detected left and right band-shaped objects.

According to a third aspect of the present invention, in the first or second aspect of the present invention, travel line data indicating an imaging state on the road surface image of the travel line determined in the travel line determination process is stored. And running line data generation processing generated from the first or second road surface image.

According to a fourth aspect of the present invention, there is provided a travel system in which a vehicle travels expressed by coordinates of a traveling axis set in a traveling direction of the vehicle and a horizontal axis set in a direction orthogonal to the traveling direction. An image pickup means for sequentially taking a road surface image of the traveling road including the traveling line of the road in a predetermined posture with respect to the vehicle, and a traveling road recognition method according to any one of claims 1 to 3, An apparatus for recognizing a traveling road in a vehicle, comprising: an image processing unit that performs image processing on each of the road surface images.

(Operation) According to the first aspect of the present invention,
In the edge detection processing, a travel line is calculated from the density change value in the edge data, which is a density change value between image units forming a line region extending in the horizontal axis direction at each position of the traveling axis of the road surface image,
That is, the existence of the center separation line, the edge of the road shoulder line, the edge of the road sign, and the edge of the recognition obstacle such as the paving stone imaged at the same density as the traveling line are detected together with the position with respect to the horizontal axis. You.
In the image division processing, the edge area detection processing, and the boundary area detection processing, for each of the division areas divided by the predetermined division width in the traveling direction, the margin generated from the margin data of each line area constituting the division area From the position range where the cumulative change value that is equal to or greater than the predetermined size exists in the area data, the edge of the traveling line passing through the opposite edges in the traveling direction of the divided area or the edge of the road sign exists Is detected along with its position range. At this time, for an obstacle such as a paving stone smaller than the division width of the division area, the magnitude of the accumulated change value is smaller than the predetermined magnitude even if the obstacle is imaged at the same density as the traveling line. Not detected from The recognition target having an edge in the boundary area detected in each divided area is a straight line of a running line or a road sign extending linearly in a band shape, and a non-straight line of a road sign not extending linearly in a band shape. If it is not a part, the recognition target object having an edge in each boundary region is a band-shaped object existing in a positional relationship at least continuous with the divided region in which each boundary region is detected. Based on this precondition, in the belt-shaped object detection process, the position of the divided region where the boundary region is detected with respect to the traveling axis, the position of the boundary region detected in the divided region with respect to the horizontal axis, and the accumulation in the boundary region From the positive / negative of the value of the change value and the division width of the divided region, a traveling line that is a strip-shaped object existing at least in a state where the boundary region where the edge exists is continuously connected to each of the detected divided regions, that is, overtaking A line, an overtaking prohibition line, a part of a road shoulder line, or a straight part of a road sign is detected, and a non-linear part of a road sign that is not a belt-shaped object is not detected. Also, if the belt-shaped objects detected in the two road surface images photographed at sufficiently short time intervals compared to the traveling speed of the vehicle are the same traveling line and are not linear portions of the road sign, then 2 There is at least a continuous positional relationship between the two road surface images. Based on these preconditions, in the traveling line determination processing, the same continuous overtaking prohibition line, the overtaking line composed of broken lines, the road shoulder line, or a road surface marker having a straight line portion, which is a belt-shaped object detected in each road surface image. Out of them, an overtaking prohibited line, an overtaking line, and a road shoulder line are determined. Therefore, there is no need for a large amount of arithmetic processing for measuring the correlation between a plurality of standard plates and the road surface image as in the method of recognizing a traveling line by template matching. Paving stones whose length along the traveling path is less than the predetermined value are not detected, and shadows, road markings, and the like having the same length or more than the predetermined value are determined to be non-traveling lines.

According to the second aspect of the present invention, the first aspect is provided.
In addition to the operation of the invention described in the above, if the detected left and right band-shaped objects are traveling lines on both the left and right sides forming an actual traveling path, the traveling lines are continuous at the same interval. Based on this precondition, in the left and right traveling line detection processing, a plurality of band-shaped objects detected, that is, traveling lines that form an actual traveling path, traveling lines that do not form a traveling path and exist in a discontinuous state, A pair of band-like objects having a high possibility of being both left and right traveling lines forming an actual traveling path are detected from the straight portions of the road sign. Therefore, even if the actual traveling path is not formed and the discontinuous traveling line is on the traveling road surface, the left and right traveling lines forming the actual traveling path are detected.

According to the invention described in claim 3, according to claim 1 of the present invention,
Alternatively, in addition to the operation of the invention described in claim 2, in the traveling line data generation processing, traveling line data indicating an imaging state of the determined traveling line in the road surface image is generated.

According to the fourth aspect of the present invention, the image processing means converts the road surface image of the travel road including the travel line of the travel road sequentially imaged in a predetermined posture with respect to the vehicle by the image processing means. A travel path is recognized by the travel path recognition method according to any one of claims 1 to 3.

[0016]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below with reference to FIGS. FIG. 3 is a schematic configuration diagram of the vehicle traveling road recognition device 10. The vehicle travel path recognition device 10 includes a CCD camera 11 as an imaging unit.
a, 11b, and an image processing unit 12 as image processing means. The CCD cameras 11a and 11b
The left and right door mirrors ML and MR of the vehicle C are provided respectively.

FIG. 4 shows a photographing area of a traveling road photographed by each of the CCD cameras 11a and 11b. Also, FIG.
CCD camera 11a provided on left side door mirror ML
6 shows a road surface image of the traveling road surface taken by the user, and FIG. 6 shows a road surface image taken by the CCD camera 11b provided on the right side door mirror MR. Each CCD camera 11a, 11b
Is fixed in a predetermined posture with respect to the vehicle, and is represented by coordinates of a traveling axis Y set in the traveling direction of the vehicle C and a horizontal axis X set in a direction orthogonal to the traveling direction. Captures a road surface image of the travel road including a travel line forming a travel road on which the vehicle travels. Further, the CCD camera 11a is provided so as to capture a left traveling line forming a traveling road on which the vehicle travels, for example, a left road surface image AL including only the road shoulder line A1, and the CCD camera 11b is configured to photograph the traveling road. It is provided so as to capture the right road surface image AR including only the right traveling line to be formed, for example, the center separation line (overtaking line, overtaking prohibited line) A2. Also, each CCD camera 11a,
Reference numeral 11b is provided to capture road surface images AL and AR including the road surface sign B displayed on the traveling road. Each of the CCD cameras 11a and 11b captures a road surface image formed of pixels arranged two-dimensionally in m rows × n columns. The CCD cameras 11a and 11b output, for each pixel, a pixel signal having an intensity corresponding to the density of the traveling road surface imaged in pixel units.

The image processing unit 12 includes an A / D converter 1
3, CPU 14, input / output port 15, ROM 16, RA
M17, an image memory 18, a timer 19, and the like. RO
In M16, a control program for a traveling road recognition process executed by the CPU 14 is stored.

In accordance with the control program, the CPU 14 repeatedly executes a travel path recognition process for recognizing a travel path formed by the left and right traveling lines A1 and A2, for example, every 100 milliseconds.

As the traveling road recognition processing, the CPU 14 executes traveling road surface photographing processing, image division processing, edge detection processing, edge area detection processing, boundary area detection processing, band-shaped object detection processing,
The left and right traveling line detection processing, the traveling line determination processing, and the traveling line data generation processing are sequentially executed.

As the road surface photographing process, the CPU 14
The right and left CCD cameras 11a and 11b are controlled to scan each time a predetermined time elapses based on the time counted by the timer 19, and the one-frame pixel signals output by the CCD cameras 11a and 11b are output to the A / D converter 13a. , 13b through m rows × n columns of pixel data (hereinafter simply referred to as pixels)
Road surface image data (hereinafter simply referred to as road surface image)
And stored in the image memory 18 respectively. That is,
As the traveling road surface photographing process, the CPU 14 sequentially photographs left and right road surface images AL and AR of the traveling road that change as needed with the same positional relationship as the vehicle travels.

As the traveling path recognition processing, the CPU 14 executes each processing from the image division processing to the traveling line detection processing on each of the road surface images AL and AR on the left and right sides of the traveling road. Hereinafter, the same processing will be described as a road surface image AR on the right side of the traveling road
However, the same applies to the road surface image AL on the left side of the traveling road.

As the edge detection processing, the CPU 14 calculates a density change value in the horizontal axis direction of each pixel row forming a line region extending in the horizontal axis direction from each position of the traveling axis Y with respect to the right road surface image AR. Edge data in which the (differential value) corresponds to each position on the horizontal axis X is generated. That is, the CPU 14 generates m pieces of edge data for one road surface image AR.
FIG. 7B shows a pixel row forming a line area LA extending in the horizontal axis direction from one position Q1 on the traveling axis Y of the road surface image AR shown in FIGS. 6 and 7A. The margin data obtained by taking the density change value is represented by a graph showing the density change value with respect to the horizontal axis X. The edge data is generated by differentiating pixel data of each pixel of a pixel row forming one line area LA, and indicates a point where the image density greatly changes in the line area LA. That is, in the edge data, the presence of the density change value that is equal to or greater than the predetermined magnitude | α | is determined based on the edges of the traveling lines A1 and A2, the edges of the road marking B, and the edges of the traveling lines A1 and A2. The presence of an edge of an obstacle such as a paving stone C imaged at the same density as that of the obstacle is shown together with its position.

As an image dividing process, the CPU 14 converts the right road surface image AR stored in the image memory 18 into a plurality of divided areas divided by a predetermined dividing width wd in the traveling axis direction as shown in FIG. In FIG. 6, Ad1 to Ad
Divide into nine. The division width wd at which each of the divided areas Ad1 to Ad9 is divided in the traveling axis direction is a travel line, that is, the road shoulder line A based on the density of the image captured by each pixel of the road surface image AR.
1 and the center separation line A2, an object on the traveling road surface which may be erroneously recognized as the traveling lines A1 and A2, for example, outside of an obstacle such as a paving stone C laid on the road surface. The size is set as small as possible within a range larger than the diameter. Note that the number of divided areas Ad of the road surface image AR differs depending on the size of the road surface image AR in the traveling axis direction, the set division width wd, and the like.
It is divided into two divided areas Ad.

As the edge area detection processing to be executed next to the edge detection processing and the image division processing, the CPU 14 determines, for each of the divided areas Ad1 to Ad9, the divided areas Ad1 to Ad9 for the right road surface image AR. From the edge data generated for each line area LA, a cumulative change value which is an addition value of a density change value equal to or more than a predetermined magnitude | α | Edge area data corresponding to each position of X is generated. FIG. 7A shows the divided area Ad3, and FIG. 7C is a graph showing the edge area data generated from the edge data generated in each line area LA of the divided area Ad3. In the marginal area data, the position range in which the cumulative change value is equal to or greater than the predetermined magnitude | β | is a traveling line passing through the opposite edges in the traveling direction of the divided area Ad, that is, the side of the road shoulder line A1. The area where the edge, the edge of the center separation line A2, or the edge of the road sign B is present.

As the boundary area detection processing to be executed next to the edge area detection processing, the CPU 14 determines, for each of the divided areas Ad on the right road surface image AR, the edge generated for the divided area Ad. In the area data, the positive / negative of the cumulative change value is the same as the horizontal axis X and a predetermined magnitude |
The position range that is equal to or greater than β | is detected as a boundary region where edges that pass through the divided region Ad in the traveling direction, that is, edges that pass through the upper edge LU and the lower edge LL, are present. For example, regarding the divided region Ad3, three boundary regions ε are obtained from the peripheral region data shown in the graph of FIG.
1, ε2 and ε3 are detected. That is, as the boundary area detection processing, the CPU 14 performs one or both sides of the road shoulder line A1, the center separation line A2, or the road surface marker B passing through both end edges LU, LL opposed to each other in the traveling axis direction of each divided area Ad. Detect edges. On the other hand, the CPU 14 determines that the obstacle such as the paving stone C smaller than the division width wd of the division area Ad is captured at the same density as the traveling lines A1 and A2 as the boundary area detection processing. No detection is performed because the magnitude of the cumulative change value corresponding to the edge is less than the predetermined magnitude | β |. It is unknown whether the edge detected by the CPU 14 in the boundary area detection processing is inclined rightward or leftward with respect to the traveling axis Y.

As a band-shaped object detection process to be executed next to the boundary region detection process, the CPU 14 determines the position of the divided region Ad where each boundary region is detected with respect to the traveling axis Y and the position detected in each of the divided regions Ad. It is constituted by a recognition target object having an edge in each boundary region from the position of the boundary region with respect to the horizontal axis X, the sign of the value of the cumulative change value in the boundary region, and the division width wd of the division region Ad. And detecting a strip-shaped object existing at least in a continuous relationship with a partial area of the road surface image AR formed by each divided area Ad in which each boundary area is detected. That is, the recognition target existing in each of the divided areas Ad in which the boundary area is detected is a part of the running lines A1 and A2 extending in a belt shape or a straight line portion of the road marking B, and the road marking does not extend in the belt shape. If it is not the non-linear portion of B, the recognition target object having an edge in each boundary area is a strip-shaped object existing in a positional relationship at least continuous with the divided area in which each boundary area is detected. Based on these preconditions, the CPU 14 determines that the road-side line A1 and the center separation line (overtaking line, overtaking prohibition line) A, which are belt-shaped objects.
2, or the linear part of the road sign B is detected, and the non-linear part of the road sign B which is not a belt-shaped object is not detected. More specifically, for example, one of the lower edges of the road surface image AR
The strip-shaped object existing in a state continuous from the point to one point on the right edge is detected as being possibly a part of the traveling lines A1 and A2 or a part of the straight part of the road sign B. . Further, a belt-shaped object existing in a continuous positional relationship from one point on the lower edge to one point on the upper edge of the road surface image AR and having no intermediate point is detected as a possibility of being an overtaking line. You.

The processing for detecting a strip-shaped object will be described in detail. The position range of one boundary region detected in the boundary region detection processing with respect to the horizontal axis X is such that the edge existing in the boundary region is the traveling line A.
1 and A2, the divided area A
This is the same value as the interval of the passing position at each edge of the edge when passing through both edges in the traveling direction with respect to the horizontal axis X.
Therefore, based on the division width wd of the division region Ad and the position range of the boundary region, as shown in FIG.
The inclination angle θ with respect to the traveling direction of the traveling line passing through d can be determined. However, it cannot be determined whether the direction of the inclination is right or left with respect to the traveling direction. In the belt-shaped object detection processing, the CPU 14 sets two edges that are inclined at a predetermined inclination angle θ to the right or left with respect to the traveling direction with respect to the boundary area of each divided area Ad detected in the boundary area detection processing. Suppose. In addition, the CPU 14 determines that a pair of boundary areas whose interval with respect to the horizontal axis X is within a predetermined range and whose cumulative change value is positive or negative has edges on both left and right sides of one traveling line A1, A2. Assume a boundary region. And
A recognition target existing in a partial area of the road surface image AL or AR formed by the divided area Ad where each boundary area is detected and present in the divided area Ad in a state facing any one of directions assumed in each boundary area It is determined whether there is a band-shaped object having a continuous positional relationship at least in the partial region.

In the left and right traveling line detection processing to be executed next to the band-shaped object detection processing, the CPU 14 applies the left and right road surface images AL and AR detected by the respective processing up to the band-shaped object detection processing. Road surface image AL, A
Based on the position of each band-shaped object R with respect to the horizontal axis X and the distance between the left and right traveling lines A1 and A2 forming the actual traveling path, the distance between both of the band-shaped objects is actually determined. A pair of band-shaped objects having the same distance as the travel lines A1 and A2 on both left and right sides forming the travel path are detected. That is,
If the left and right band-shaped objects detected respectively in the left and right road surface images AL and AR are the traveling lines A1 and A2 on both the left and right sides that form the actual traveling road, they continue at the same interval. Based on the preconditions, the CPU 14 detects the detected left and right band-shaped objects, that is, the traveling lines A1, A forming the actual traveling path.
2. A pair of strip-like objects having a high possibility of being the left and right traveling lines A1 and A2 forming the actual traveling path are detected from the straight line portion of the road sign B.

As a traveling line determination process to be executed next to the left / right traveling line detection process, the CPU 14 uses the first road surface images AL and AR sequentially obtained as the vehicle advances to obtain the first road image AL obtained at the previous time. The first band-shaped object detected from the road surface images AL and AR of the second road surface image and the second band-shaped object obtained at a later point in time exist at least in a continuous positional relationship between the two road surface images AL and AR. It is determined whether or not the two belt-shaped objects having a continuous positional relationship are the same traveling lines A1 and A2. That is, the belt-shaped objects detected in the two road surface images AR photographed at sufficiently short time intervals compared with the traveling speed of the vehicle are the same traveling line A.
1, if it is A2 and it is not the straight part of the road sign B, 2
There is at least a continuous positional relationship between the two road surface images.
Based on these preconditions, the CPU 14
From the road shoulder line A1, the center separation line (continuous overtaking prohibition line, the overtaking line consisting of a broken line) A2, or the road sign B having a straight portion, The separation line (overtaking prohibited line and overtaking line) A2 is determined as the traveling line.

As the traveling line data generation processing to be executed next to the traveling line determination processing, the CPU 14 determines the road image A for the left and right traveling lines A1 and A2 determined in the traveling line determination processing.
Travel line data indicating an imaging position on L and AR is generated from the first or second road surface image.

Next, the operation of the above-structured traveling road recognition apparatus will be described. When the traveling path recognition device is activated when the vehicle is traveling, the CP of the image processing unit is activated.
U14 repeatedly executes the travel path recognition process every predetermined time. 1 and 2 show a flowchart of the traveling road recognition processing.

In the traveling path recognition processing, the CPU 14
First, in step (hereinafter simply referred to as S) 10, a traveling road surface photographing process is executed. As a result, the left and right road surface images AL and AR of the traveling road that change at any time with the same positional relationship as the vehicle travels are sequentially captured.

Next, in S11, the CPU 14 executes the edge detection processing for each line area of the left and right road surface images AL and AR as the edge detection processing. Next, the CPU 14
In S12, as the edge detection processing and the image division processing,
The road surface images AL and AR are divided at a predetermined division width with respect to the traveling axis Y, and for one divided region Ad, the density change value is plotted against the edge data of the divided region Ad with respect to the horizontal axis X. Is stored and the density change value thereof is stored. At S13, it is determined that the processing at S12 has been performed for all the divided areas Ad. As a result, for each of the divided areas Ad of the road surface images AL and AR, the road shoulder line A1, the center separation line A2, the road surface sign B, and the density equivalent to each of the road surface displays are present in each of the line regions forming the divided region Ad. The presence of the edge of the paving stone C or the like to be imaged is detected together with the position with respect to the horizontal axis X.

Next, in S14, the CPU 14 determines the divided area Ad in which the edge detection processing has been completed in S14.
Is generated by adding the edge data generated in each line area. Further, in step S15, the CPU 14 detects a position range in which the positive and negative cumulative change values are the same and equal to or greater than the predetermined size | β | from the peripheral region data generated for the divided region Ad in the boundary region detection process. And
In S16, each detected position range and its sign are stored. As a result, the presence of the road shoulder line A1, the center separation line A2, or the edge of the road sign B passing through the divided area Ad in the traveling direction is detected together with the position range. On the other hand, even if the density is the same as that of the traveling lines A1 and A2, the obstacle such as the paving stone C smaller than the division width wd of the division area Ad has a value of the cumulative change value smaller than the predetermined magnitude | β | Not detected from becoming

When the CPU 14 determines in S17 that the edge area detection processing and the boundary area detection processing have been completed for all the divided areas Ad with respect to the left and right road surface images AL and AR, the processing proceeds to S18. Execute the object detection processing. As a result, each divided area A in which the boundary area is detected
The road shoulder line A1, a part of the center separation line A2, or a straight line portion of the road surface marker B, which is a belt-shaped object existing in at least a partial region of the road surface images AL and AR formed by d, is detected and is not a band-shaped object. The non-linear portion of the road sign B is not detected.

Next, in S19, the CPU 14 executes a left / right traveling line detection process. As a result, a plurality of band-shaped objects detected from the left and right road surface images AL and AR, that is, the traveling lines A1 and A2 forming the actual traveling road, exist in a discontinuous state without forming the actual traveling road. A pair of belt-like objects including the left and right traveling lines A1 and A2 forming the actual traveling path are detected from the traveling line and the straight line portion of the road sign B.

Next, in S20, the CPU 14 executes a traveling line determination process. As a result, two road surface images AL,
The same continuous road shoulder line A1, center separation line (overtaking line, overtaking prohibition line) A2, or road surface sign B having a straight line portion, which is a band-shaped object detected by AR.
Of these, the road shoulder line A1 and the center separation line A2 are determined.

At the end of the traveling road recognition processing, the CPU 14 executes S
At 21, a traveling line data generation process is executed. As a result, travel line data indicating the imaging position of each of the left and right travel lines A1 and A2 determined as the travel lines forming the actual travel path in the road surface images AL and AR is generated.

According to the present embodiment described in detail above, the following effects can be obtained. (1) There is no need for a large amount of arithmetic processing to measure the correlation between a plurality of standard plates and a road surface image in a travel line recognition method using template matching. or,
A recognition obstacle such as a paving stone C whose length along the traveling path is less than a predetermined value is not detected, and a shadow whose length is equal to or more than a predetermined value,
It is determined that the road signs B and the like are not the traveling lines A1 and A2.
As a result, in a short time, and with high accuracy,
1. The traveling road such as the center separation line A2 or the like can be determined to recognize the traveling road as an image.

(2) A belt-shaped object which is not continuous at the same interval as a pair of left and right and does not form an actual traveling path is not recognized as actual traveling lines A1 and A2, and forms an actual traveling path. Only the traveling lanes A1 and A2 are detected. As a result, for example, when a temporary traveling path is used as an actual traveling path at the time of road construction and the like, and the traveling line of the previous traveling path remains in a discontinuous state, it may not be continued. In addition, it is possible to recognize a continuous traveling line and recognize a correct traveling image without recognizing an incorrect traveling road from traveling lines that do not form an actual traveling road.

(3) Since the traveling line data indicating the imaging position of the determined traveling line in the road surface image is generated, the positional relationship of the vehicle with respect to the traveling lines A1 and A2 can be grasped from the traveling line data. For this reason, based on the travel line data, when the vehicle is not traveling, it is necessary to detect a deviation or departure from the center of the travel path in a state where the travel lane is not to be changed, and warn the driver, or Steering wheels can be automatically controlled so as not to deviate from the road.

Hereinafter, embodiments other than the above-described embodiment embodying the present invention will be listed as other examples. The imaging means for capturing the road surface image is not limited to the CCD camera, but may be, for example, a CMOS camera.

The parts of the vehicle provided with the left and right cameras for photographing the road surface images including the traveling lines on both the left and right sides of the traveling road are not limited to the left and right door mirrors, but may be doors, tire roofs, bumpers, etc. Good.

One road surface image photographed so as to include both left and right traveling lines, instead of performing left and right traveling line detection processing based on a pair of left and right road surface images AL and AR separately photographed by cameras provided on the left and right. The left and right traveling line detection processing may be performed based on

The edge area data is a cumulative value in which at least one of the density change value at the same position on the horizontal axis X and the frequency of the density change value at or above the predetermined value at the same position is reflected in each edge data. It is only necessary to detect a boundary area where edges passing through both end edges of the divided area that are opposed to each other in the traveling axis direction exist based on the change value. Therefore, the edge area data is not limited to data in which the added value of the density change value equal to or more than the predetermined value in the edge data of all the line areas of the divided area Ad corresponds to each position on the horizontal axis X. In addition, for example, the edge area data may be generated by directly adding the density change value at the same position with respect to the horizontal axis X in each edge data. Alternatively, the edge data may be edge area data in which the frequency of a density change value that is equal to or greater than a predetermined value at the same position on the horizontal axis X corresponds to each position on the horizontal axis X. Alternatively, the edge area data may be a frequency in which the frequency reflecting the density change value at the same position with respect to the horizontal axis X in each edge data corresponds to each position on the horizontal axis X.

The line area for generating the edge data is not limited to the pixel row, and may be a line area formed by a set of pixels arranged in the traveling direction. In this case, the edge data may be, for example, data indicating a density change value of road surface image data generated from a value obtained by averaging pixel signals of a plurality of pixels at each position on the horizontal axis X.

The traveling path recognition device detects, based on the traveling path data, deviation of the vehicle from the traveling path on either the left or right side or deviation from the traveling path when the vehicle is not in a state of changing lanes. Alternatively, a driving assist device may be configured together with the driving condition monitoring device that warns the driver. In this configuration, the operation of the vehicle can be indirectly assisted by detecting the travel line that forms the travel path, and there is no need to embed a vehicle guidance line or a vehicle detection sensor in the travel path.

Further, based on the traveling road data, the traveling road recognition device detects deviation of the vehicle on either side of the traveling road or deviation from the traveling road when the vehicle is not in the state of changing the traveling lane. The traveling assist device may be configured together with the traveling state monitoring device that detects and warns the driver. In this configuration, the driving of the vehicle can be directly assisted by detecting the traveling line that forms the traveling path, and there is no need to embed a vehicle guidance line or a vehicle detection sensor in the traveling path.

Hereinafter, in addition to the inventions described in the claims, the technical ideas grasped from the above-described embodiments and the respective examples will be described together with their effects. (1) Based on the travel path recognition device according to claim 4 and the travel path data, when the vehicle is not in a state of changing the travel lane, a deviation of the vehicle on the left or right side with respect to the travel path, or A traveling assist device including a traveling state monitoring device that detects a deviation from a traveling road and warns a driver. According to such a configuration, the operation of the vehicle can be indirectly assisted by detecting the travel line forming the travel path, and there is no need to embed a vehicle guidance line or a vehicle detection sensor in the travel path.

(2) On the basis of the travel path recognition device according to claim 4 and the travel path data, when the vehicle is not in a state of changing the travel lane, the deviation of the vehicle on the left or right side with respect to the travel path. Or a driving assistance device that detects deviation from the traveling road, controls the steering of the steered wheels, and returns the steering wheel to the center of the traveling road of the vehicle. According to such a configuration, the driving of the vehicle can be directly assisted by detecting the travel line that forms the travel path, and there is no need to embed a vehicle guidance line or a vehicle detection sensor in the travel path.

[0052]

As described in detail above, according to the first to fourth aspects of the present invention, a center separation line, a road shoulder line, etc., which form a traveling path in a shorter time and with higher accuracy. And the travel road can be recognized by image recognition.

According to the invention described in any one of claims 2 to 4, even if a traveling line that is not continuous and does not form an actual traveling path is on the traveling road surface, the continuous traveling line is determined. It is possible to recognize an image of a correct traveling road.

According to the third or fourth aspect of the present invention, traveling data indicating the positional relationship of the vehicle with respect to the traveling road is output.

[Brief description of the drawings]

FIG. 1 is a flowchart of a traveling road recognition process.

FIG. 2 is a flowchart of a traveling road recognition process.

FIG. 3 is a schematic block diagram of a travel path recognition device.

FIG. 4 is a schematic diagram showing a traveling path and a photographing area of each CCD camera.

FIG. 5 is a schematic diagram showing a road surface image obtained by a left CCD camera.

FIG. 6 is a schematic diagram showing a road surface image obtained by a right CCD camera.

7A is a schematic diagram of a divided area, FIG. 7B is a graph of one edge data of a line area, and FIG. 7C is a graph of one edge area data of a divided area.

[Explanation of symbols]

11a, 11b... CCD camera as imaging means, 12
... Image processing unit as image processing means, AL, AR
... Road surface images, Ad1 to Ad9 ... Divided areas, C: Vehicles, L
A: line region, X: horizontal axis, Y: traveling axis, ε1, ε2, ε3
... Boundary area.

Continued on front page F term (reference) 5B057 AA16 BA02 CA02 CA08 CA12 CA16 CC02 DA07 DB02 DB05 DB09 DC16 DC22 5H180 AA01 CC04 FF27 LL06 5L096 AA03 AA06 BA04 CA05 FA06 FA14 FA69 GA07

Claims (4)

[Claims] 1. An image is captured in a predetermined positional relationship with respect to a vehicle, and is represented by coordinates of a traveling axis set in a traveling direction of the vehicle and a horizontal axis set in a direction orthogonal to the traveling direction. And images forming line regions extending in the horizontal axis direction from respective positions of the traveling axis from road surface images sequentially captured including at least one of the left and right traveling lines forming a traveling path on which the vehicle travels. An edge detection process of generating edge data in which a density change value in the unit of the horizontal axis corresponds to each position of the horizontal axis; and a plurality of sections divided by a predetermined division width in the traveling axis direction. Image division processing for dividing into divided areas, For each of the divided areas, from the margin data generated for each of the line areas of the divided area, at the same position on the horizontal axis of each of the margin data Addition value of density change value, and
The marginal area data in which at least one of the frequencies of the density change values that are equal to or more than the predetermined value is reflected at the same position on the horizontal axis of each of the marginal data, corresponding to each position on the horizontal axis. Edge region detection processing to be generated, and for each of the divided regions, in the edge region data generated for the divided region, the sign of the cumulative change value is the same and is equal to or greater than a predetermined size. Boundary region detection processing for detecting a position range with respect to the horizontal axis as a boundary region in which edges passing through both end edges of the divided region opposing each other in the traveling axis direction, and each of the boundary regions is detected. The position of the divided region with respect to the traveling axis, the position of the boundary region detected in each of the divided regions with respect to the horizontal axis, the sign of the cumulative change value in the boundary region, and the division of the divided region width Therefore, each boundary area is constituted by a recognition target having an edge, and each boundary area exists in a positional relationship at least continuous with a partial area of the road surface image formed by each detected divided area. A band-shaped object detection process for detecting a band-shaped display unit, and a first band-shaped object detected from a first road-surface image obtained at a previous time point in the two road-surface images sequentially obtained as the vehicle travels. It is determined whether or not the target object and the second belt-shaped target object detected from the second road surface image obtained at a later point of time are present in at least a continuous positional relationship between the two road surface images, and are determined to be continuous. A method for recognizing a traveling path in a vehicle, comprising: performing a traveling line determination process for determining the two belt-shaped objects in a positional relationship as being the same traveling line. 2. The road surface image is a road surface image including left and right traveling lines forming the traveling road, and is detected from the road surface image by each processing up to the band-shaped object detection processing. Based on the positions of the plurality of band-shaped objects with respect to the horizontal axis and the distance between the left and right traveling lines forming the actual traveling path, the distance between the two of the band-shaped objects is determined as the actual travel. Along with performing left and right traveling line detection processing for detecting a pair of band-shaped objects that are equivalent to the interval between the left and right traveling lines forming the road, the traveling line recognition processing is performed on the detected left and right band-shaped objects. The method according to claim 1, wherein the travel line is determined. 3. A travel line data generation process for generating travel line data indicating an imaging state on the road surface image of the travel line determined in the travel line determination process from the first or second road surface image. 3. The method for recognizing a traveling path in a vehicle according to claim 1 or claim 2. 4. A traveling axis set in a traveling direction of the vehicle;
A road surface image of the traveling road, including a traveling line of the traveling road on which the vehicle travels, represented by coordinates with a horizontal axis set in a direction orthogonal to the traveling direction, is sequentially captured in a predetermined posture with respect to the vehicle. An apparatus for recognizing a traveling road in a vehicle, comprising: an imaging unit that performs image processing; and an image processing unit that performs image processing on each of the road surface images by the traveling road recognition method according to any one of claims 1 to 3.