JP2014238693A - Stop line detection device, moving body control device, and program for stop line detection - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable the accurate detection of a sign of a stop line on the road.SOLUTION: There is provided a stop line detection device that detects a stop line present on the road surface on which a moving body travels on the basis of photographed images obtained by photographing the direction of travel of the moving body. The stop line detection device detects, from the photographed images, a structure installed at a branch location where the traveling road surface on which the moving body travels branches to another traveling road surface and dividing a traveling road surface area and a non-traveling road surface area, and a long belt-like sign on the road extending in the width direction of the road, and detects a sign on the road present at a position distant from the position of the detected structure by a predetermined distance as a sign of a stop line on the road.

Description

  The present invention relates to a stop line detection device that detects a road marking of a stop line formed on a road surface, a mobile body control device that mounts the stop line detection device and controls the travel of a mobile body, and a stop line detection device. It is about the program.

  In recent years, with the diversification of driving support systems such as ACC (Adaptive Cruise Control) mounted on vehicles in order to improve the safety of vehicles that are moving objects, image recognition devices have come to be used a lot. . In ITS (Intelligent Transport System), many systems perform image recognition processing while driving. Patent Document 1 discloses a technique for detecting a road marking of a stop line formed on a road surface in front of an intersection or a pedestrian crossing.

  In the stop line detection apparatus of Patent Document 1, the traveling direction of the host vehicle is imaged from before entering an intersection on a roadway having a plurality of vehicle lanes. A road marking part for a stop line is detected based on a road marking part for each traveling direction detected from the captured image. When traveling toward the intersection, the road marking section according to the direction of travel is first seen on the road surface of the traveling road. The road marking section classified by the traveling direction is a road marking section with an arrow indicating the traveling direction after passing the intersection for each vehicle lane. Passing through the road marking sections classified by the direction of travel, and traveling further toward the intersection, the road marking section for the stop line and the road marking section for the bicycle-only zone can be seen in order. The road marking part of the stop line and the road marking part of the bicycle traffic band are long white strips extending in the road width direction and have no characteristic shape. Moreover, the difference of each width | variety in the driving | running | working direction which is a direction orthogonal to a road surface width direction is small, and it is difficult to distinguish each other from a captured image. For these reasons, it is difficult to detect the road marking part of the stop line from the picked-up image, distinguishing it from the road marking part of the bicycle traffic band. On the other hand, the arrow display of the road marking section classified by the traveling direction has a characteristic shape, and the road marking section classified by the traveling direction can be detected relatively easily. It is known that a road marking part for a stop line is formed at a position away from the position of the road marking part for each traveling direction by a predetermined distance toward the intersection. The road marking section for the direction of traveling direction is detected from the picked-up image, and the road marking section for a dedicated bicycle passage is determined by whether or not it is at a predetermined distance from the position of the road marking section for the traveling direction section detected. It is possible to detect the road marking part of the stop line.

  However, in the stop line detection device of Patent Document 1, it is difficult to accurately detect the road marking part of the stop line as follows. Generally, the road surface marking part formed on the traveling road surface is formed by painting. Of these road marking sections, the road marking section for each direction of travel is formed throughout the vehicle traffic zone so that the vehicle driver can easily see the travel direction after passing the intersection of the traveling vehicle traffic zone. Has been. If the direction of the arrow in the traveling direction section of the vehicle traffic zone that has been traveling is different from the travel direction that the user wants to travel after passing the intersection, the travel is changed to another vehicle traffic zone in the direction that the vehicle travels. In this case, the vehicle travels across the vehicle lane, and the frequency of traveling on the road marking portion classified by the traveling direction increases, and the tire of the vehicle is painted on the road marking portion classified by the traveling direction. Easy to pass over. As a result, the painted portion of the road marking portion classified by the traveling direction is easily scraped off, and the display of the road marking portion classified by the traveling direction is gradually faded. In such a captured image obtained by capturing an image on the traveling road surface, the road surface marking section classified by the traveling direction is blurred. It is difficult to detect with high accuracy the road marking section classified by the traveling direction from the captured image. In the captured image, the edge image of the vehicle traveling on the road surface is shown as a frame indicating the outline of the vehicle, and the edge image of the road marking portion of the stop line is also shown as a frame indicating the outline of the stop line. ing. For this reason, it cannot be distinguished whether the frame line extracted from the captured image is the outline of the vehicle or the outline of the stop line. It is difficult to detect the road marking part of the stop line from the captured image with high accuracy.

  Patent Document 1 also describes a navigation system using the technology of the stop line detection device. In this navigation system, the stop line detection technique described above is used. The road marking part of the stop line is detected from the road marking part classified by the traveling direction provided at the intersection as a feature, and the current position of the host vehicle when passing the stop line on the road surface is detected. A map based on the current position of the host vehicle is displayed and the current position of the host vehicle is displayed on the display map. If the current position of the host vehicle is erroneously detected as described above, the map or the position of the host vehicle on the map is erroneously displayed. The stop line detection device of Patent Document 1 is applicable not only to the correction of the position display of the own vehicle in the navigation system but also to other systems, but in other systems applicable if the current position of the own vehicle is mistaken. Will also cause problems.

  The present invention has been made in view of the above problems, and an object thereof is to provide a stop line detection device, a moving body control device, and a stop line detection program capable of accurately detecting a road surface marking portion of a stop line. It is to be.

  In order to achieve the above object, a first aspect of the present invention provides a stop line detecting apparatus for detecting a stop line existing on a traveling road surface of a moving body based on a captured image obtained by imaging the traveling direction of the moving body. A road surface that is provided at a branching point that branches from the traveling road surface area where the vehicle is traveling to another traveling road surface area, and that divides the traveling road surface area from the outside of the traveling road surface area, and a long belt-like road surface that extends in the width direction of the road surface A sign indicating unit is detected from the captured image, and the road marking unit existing at a position away from the detected position of the structure by a predetermined distance is detected as a road marking unit of a stop line. is there.

  ADVANTAGE OF THE INVENTION According to this invention, the stop line detection apparatus which can detect the road marking part of a stop line correctly, a mobile body control apparatus, and the program for stop line detection can be provided.

It is a schematic diagram which shows schematic structure of a moving body control system. It is a block diagram which shows the structure containing the stop line detection apparatus of this embodiment. It is a functional block diagram containing the stop line detection apparatus of this embodiment. It is a figure which shows the example of a left-right luminance image and a parallax image. (A) is explanatory drawing which shows an example of the parallax value distribution of a parallax image, (b) is explanatory drawing which shows V-Map of the parallax value frequency distribution for every horizontal direction area | region of the parallax image of the figure (a). (A) is explanatory drawing which shows an example of the parallax value distribution of a parallax image, (b) is explanatory drawing which shows U-Map of the parallax value frequency distribution for every vertical direction area | region of the parallax image of the figure (a), (c) ) Is an explanatory diagram showing a U-Height-Map whose value is the highest row of the parallax value frequency distribution, and (d) is an explanatory diagram showing a U-Low-Map whose value is the lowest row of the parallax value frequency distribution. FIG. (A) is explanatory drawing which shows an example of a luminance image, (b) is explanatory drawing which shows U-Map, (c) is explanatory drawing which shows U-Low-Map, (d) shows U-Height-Map. It is explanatory drawing. It is a flowchart which shows the stop line detection process flow of this embodiment. (A) is explanatory drawing which shows an example of a luminance image, (b) is explanatory drawing which shows V-Map, (c) is explanatory drawing which shows U-Map, (d) shows road surface recognition on V-Map. (E) is an explanatory diagram showing a modified U-Map, (f) is an explanatory diagram showing a U-Low-Map, (g) is an explanatory diagram showing a U-Height-Map, and (h) is a modified diagram. FIG. 6 is an explanatory diagram showing a completed luminance image, and (i) is an explanatory diagram showing a luminance image including a stop line recognition display.

Hereinafter, an embodiment in which a stop line detection device according to the present invention is used in a mobile control system will be described.
FIG. 1 is a schematic diagram showing a schematic configuration of a moving body control system. The moving body control system shown in the figure includes an image pickup unit 101 having an image pickup means for picking up an image of an area in front of the traveling direction of the host vehicle 100 as a moving body. For example, the imaging unit 101 is installed near a room mirror (not shown) of the windshield 103 of the host vehicle 100. Various data such as captured image data obtained by imaging by the imaging unit 101 is input to an image analysis unit 102 as image processing means. The image analysis unit 102 analyzes the data transmitted from the imaging unit 101 and detects a road surface marking portion of a stop line on the road surface existing in the imaging area. The detection result of the image analysis unit 102 is sent to the vehicle travel control unit 104. The vehicle travel control unit 104 performs, for example, the following control based on the detection result of the road marking part of the stop line detected by the image analysis unit 102. When the driver is not going to stop by operating the brake pedal when the traffic light ahead is red, the driver of the host vehicle 100 is notified of a warning to stop, or the brake is automatically operated and controlled. Carry out driving support control.

  FIG. 2 is a block diagram showing a configuration including the stop line detection device of the present embodiment. The stop line detection apparatus according to the present embodiment corresponds to an MPU (Micro Processing Unit) 14 that performs parallax map generation and stop line detection processing. Stereo images picked up by two cameras 11 and 12 mounted in parallel on the left and right are supplied to an FPGA (Field Programmable Gate Array) 13 and parallax data is calculated at high speed from the stereo images acquired by the two cameras 11 and 12. Is done. An MPU (Micro Processing Unit) 14 performs parallax map generation and stop line detection processing. The frame buffer 15 stores captured images, parallax data, and detection result images. The memory 16 stores data specifying the road surface height difference feature amount, the stop line width feature amount, and the stop line position feature amount from the cameras 11 and 12 to be used for the detection process. The image display unit 17 displays a stop line detection result and a camera image. Each of the two cameras 11 and 12 images the front of the vehicle and supplies a left luminance image and a left luminance image to the FPGA 13. The FPGA 13 generates a parallax image having a parallax value based on the stereo image of the left and right luminance images and supplies the parallax image to the frame buffer 15. The frame buffer 15 stores the left and right luminance images captured by the cameras 11 and 12, the parallax image generated by the FPGA 13, and the stop line detection result. The MPU 14 displays the road marking of the stop line based on the left-right luminance image and the parallax image supplied from the frame buffer 15, the road surface height difference feature quantity, the stop line width feature quantity, and the stop line position feature quantity stored in the memory 16. To detect. The detection result image is stored in the frame buffer 15 and displayed on the image display unit 17.

  FIG. 3 is a functional block diagram including the stop line detection device of the present embodiment. The stop line detection device 10 of the present embodiment is realized by the MPU 14 or the like. In the figure, a stereo image acquisition unit 21 acquires stereo images from two cameras 11 and 12 as shown in FIG. As illustrated in FIG. 4, the parallax image generation unit 22 performs a stereo matching process on the partial images between the acquired stereo images, and generates a parallax image including the parallax between the partial images. When the matching processing in the parallax image generation unit 22 is realized by hardware processing, for example, SSD (Sum of Squared Difference), ZSSD (Zero-mean Sum of Squared Difference), SAD (Sum of Absolute Difference), ZSAD ( A method such as Zero-mean Sum of Absolute Difference can be used. In the matching process, only the parallax value can be calculated for each pixel. For this reason, it is necessary to use an estimated value when a sub-pixel level parallax value of less than one pixel is required. As the estimation method, for example, an equiangular straight line method, a quadratic curve method, or the like can be used. However, since an error occurs in the estimated parallax value at the sub-pixel level, EEC (estimated error correction) or the like for reducing the estimated error may be used. Examples of left and right luminance images and parallax images are shown in FIG.

  The feature amount storage unit 23 stores a road surface height difference feature amount, a stop line width feature amount, and a stop line position feature amount. The road surface height difference feature amount is used as height information for discriminating a structure such as a curb, an object other than a structure such as a curb, and a road surface marking part of road information formed on the road surface. The structure is a curb that is a boundary portion that divides the traveling road surface area and the outside of the traveling road surface area. The object is, for example, a building such as a building if it is outside the travel road surface area, and a vehicle or a pedestrian if it is within the travel road surface area. The stop line width feature quantity is used as width information in the road surface width direction of the road for determining whether or not the road marking part is a stop line. Stop line position feature is the distance from the curb or guardrail at the intersection of the intersection to the road marking part of the stop line, and whether the road marking part formed on the road surface is the road marking part of the stop line It is used as distance information for determining

  The parallax map generation unit 24 generates four parallax maps described later as histograms of parallax data on the parallax image. The parallax map here refers to a map indicating a parallax value distribution on a parallax image. The detection processing calculation unit 25 performs a stop line detection process from the parallax map and the luminance image. The stop line detection apparatus 10 according to the present embodiment includes a parallax map generation unit 24 and a detection processing calculation unit 25. The detection result display unit 26 displays an image of the detection result. In addition, as a boundary part other than protective fences, such as a curb and a guardrail, what is necessary is just a structure which divides a roadway (traveling road surface area | region) and a sidewalk (outside the traveling road surface area | region).

  Here, the parallax maps generated in the present embodiment are V-Map (horizontal parallax histogram information), U-Map (vertical parallax histogram information), U-Low-Map, and U-Height-Map. When the parallax image generation process is performed by the parallax image generation unit 22 of FIG. 3, the parallax map generation unit 24 executes a V-Map generation process for generating a V-Map. Each parallax pixel data included in the parallax image data is a set (x, y, d) of the x-direction position of the left-right direction position on the parallax image, the y-direction position of the vertical position on the parallax image, and the parallax value d. ). This is converted into three-dimensional coordinate information (d, y, f) in which d is set on the X-axis, y is set on the Y-axis, and frequency f is set on the Z-axis, or this three-dimensional coordinate information (d, y, f) 2D coordinate information (d, y) limited to information exceeding a predetermined frequency threshold is generated as horizontal direction parallax histogram information.

  Lateral direction parallax histogram information in which three-dimensional histogram information composed of three-dimensional coordinate information (d, y, f) is distributed in an XY two-dimensional coordinate system is referred to as V-Map. More specifically, the parallax map generation unit 24 calculates a parallax value frequency distribution for each lateral region of the parallax image obtained by dividing the captured image into a plurality of segments. Information indicating this disparity value frequency distribution is V-Map. As an example, the width of each horizontal region is one pixel. When parallax image data having a parallax value distribution as shown in FIG. 5A is input, the parallax map generation unit 24 sets the number of data exceeding a predetermined frequency threshold of each parallax value for each horizontal region. A disparity value frequency distribution, which is a distribution, is calculated and output as V-Map. That is, a two-dimensional orthogonal coordinate system in which parallax value frequency distribution information of each lateral region is taken as a y-axis position on the parallax image (a vertical position in the captured image) on the Y-axis and a parallax value d on the X-axis. By expressing the above, a V-Map as shown in FIG. 5B can be obtained. This V-Map can also be expressed as an image in which pixels having pixel values corresponding to the frequency f are distributed on the two-dimensional orthogonal coordinate system.

  Moreover, in the parallax map production | generation part 24, U-Map production | generation processing which produces | generates U-Map, U-Height-Map production | generation processing which produces | generates U-Height-Map, U-Low-Map which produces | generates U-Low-Map Execute the generation process. Vertical parallax obtained by converting each parallax pixel data (x, y, d) into three-dimensional coordinate information (x, d, f) in which x is set on the X axis, d is set on the Y axis, and frequency f is set on the Z axis. Generated as histogram information. Alternatively, two-dimensional coordinate information (x, d) limited to information exceeding a predetermined frequency threshold is generated from the three-dimensional coordinate information (x, d, f) as vertical parallax histogram information.

  Also, as shown in FIG. 6B, vertical disparity histogram information obtained by distributing three-dimensional histogram information composed of three-dimensional coordinate information (x, d, f) in an XY two-dimensional coordinate system is represented by U -Called Map. Specifically, the parallax map generation unit 24 calculates a parallax value frequency distribution for each vertical region of the parallax image obtained by dividing the captured image into a plurality of segments. Information indicating this disparity value frequency distribution is U-Map. As an example, the width of each vertical region is one pixel.

  When parallax image data having a parallax value distribution as shown in FIG. 6A is input, the parallax map generation unit 24 sets the number of data exceeding a predetermined frequency threshold of each parallax value for each vertical direction area. A disparity value frequency distribution, which is a distribution, is calculated and output as a U-Map. That is, the information of the disparity value frequency distribution of each vertical direction area is a two-dimensional orthogonal coordinate system in which the x-axis position (horizontal direction position in the captured image) on the parallax image is taken on the x-axis and the parallax value d is taken on the y-axis. By expressing the above, a U-Map as shown in FIG. 6B can be obtained. This U-Map can also be expressed as an image in which pixels having pixel values corresponding to the frequency f are distributed on the two-dimensional orthogonal coordinate system.

  Furthermore, when the parallax image data having the parallax value distribution shown in FIG. 6A is input, the parallax map generation unit 24 uses U-Height-Map as shown in FIG. 6C and FIG. 6D. A U-Low-Map such as U-Height-Map is a maximum height map of U-Map parallax data, and the number of rows with the largest y-coordinate corresponding to each parallax value becomes the value of U-Height-Map. U-Low-Map is a minimum height map of U-Map parallax data, and the number of rows with the smallest y coordinate corresponding to each parallax value becomes the value of U-Low-Map. FIG. 7 is a diagram illustrating each relationship between a luminance image and U-Map, U-Low-Map, and U-Height-Map. The building or vehicle displayed in the luminance image shown in FIG. 7A is displayed in a straight line portion that includes height information and is linearly approximated on the U-Map shown in FIG. 7B. In the U-Low-Map shown in FIG. 7C, parallax data of a boundary portion (a structure such as a curb or a guardrail) of the road surface region is generated. In the U-Height-Map shown in FIG. 7D, parallax data of buildings and vehicles on the luminance image is generated.

Next, stop line detection processing in the stop line detection apparatus of the present embodiment will be described according to the processing flow of FIG. Fig.9 (a)-FIG.9 (i) are the figures which show the image of the processing content in each process.
8, the detection processing calculation unit 25 of FIG. 3 generates a V-Map as shown in FIG. 9B and a U-Map parallax map as shown in FIG. 9C based on the parallax image. . A parallax map of U-Low-Map as shown in FIG. 9F and U-Height-Map as shown in FIG. 9G is generated (steps S101 and S102). In the V-Map of FIG. 9B, linear approximation processing such as least square method or Hough transform processing is performed to approximate a line and remove noise (step S103). In U-Map, noise removal and line approximation are performed (step S104). As shown in FIG. 9D, a road surface area is extracted on the V-Map (step S105).

  Next, detection processing is performed on the four generated parallax maps (FIGS. 9B, 9C, 9F, and 9G) and the luminance image (FIG. 9A) to obtain a stop line. A road marking part is detected. A line extracted from the V-Map is temporarily detected as a road surface. Based on the information of the line extracted from the U-Map, as shown in FIG. It is removed (step S106). Then, an actual height deviation is obtained for the line remaining on the U-Map based on the simple height difference and the parallax data on the U-Low-Map and the U-Height-Map. Compared with a predetermined value of the road surface height difference feature amount, the straight line portion on the U-Map is classified into a structure such as a curbstone, an object, and a road surface marking. More specifically, distance information is obtained based on parallax data. There is a triangular relationship between the distance and the actual height deviation, and there is a direct proportional relationship between the actual height deviation and the height difference between U-Height-Map and U-Low-Map. As a result, the actual height deviation can be calculated by calculating the distance based on the parallax data and using the height difference based on the distance information. Moreover, in order to classify the straight line portion on the U-Map into a structure such as a curbstone, an object, and a road surface marking portion, the actual height deviation is different from the straight line portion extracted on the V-Map. When the amount is smaller than the feature amount, the straight line portion is recognized as a structure such as a curbstone and detected. When it is larger than the road surface height difference feature amount, the straight line portion is recognized as an object such as a building and detected. When the actual height deviation is smaller than the road height difference feature amount with respect to the straight line portion in the traveling road surface area extracted on the V-Map, it is recognized as a road marking portion and detected. When it is larger than the road surface height difference feature amount, it is recognized as an object such as a vehicle or a pedestrian (step S107). Next, the distance from the end point of the branch point in the road marking on the U-Map to the end point of the road marking portion is calculated, and the distance is compared with the stop line width feature amount of a predetermined value, except for the stop line. Are classified into a road marking section and a road marking section of a stop line. Specifically, when it is smaller than the stop line width feature amount, a road marking part other than the stop line is detected. If it is larger than the stop line width feature amount, it is detected as a road marking part of the stop line (step S108). Then, the image outside the traveling road surface area detected on the U-Map, the image of the object, and the image of the road marking part other than the stop line are respectively removed from the luminance image (step S109).

  Next, in step S109, a detection process is performed on the luminance image from which each image has been removed, information on a straight line portion is extracted as shown in FIG. 9H, and the extracted straight line portion is detected as a road marking portion. Further, for all road marking parts, the end point information (Y coordinate) is compared with the road surface information detected on the V-Map, and the distance is calculated for each road marking part. For example, on the luminance image, the position coordinates of each road marking part and the position coordinates of a structure such as a curb at the branching point of an intersection are known. On the parallax image, it is difficult to display the road marking portion because it has almost no height. However, since the structure such as a curb at the branching point of the intersection has a height, the structure is easily displayed. The distance from the structure such as a curb to the camera can be calculated from the parallax data. Thus, the distance from the structure such as the curb to the camera is calculated from the parallax data of the structure such as the curb that is the same as the Y coordinate of the position coordinate in the road marking unit. A difference (L1) between a distance from a structure such as a curbstone to the camera at a branch point of the intersection and a distance from each road marking unit to the camera is calculated. And the distance (L2) of the driving | running | working direction between structures, such as a curb stone in the branch location of an intersection, and each road marking part is calculated | required, respectively. By the Hough transform, a straight line portion can be detected as a line segment having an end point, and end point information of the straight line portion is obtained. The straight line portion is approximated on the luminance image and detected as all road marking portions. The disparity values in the disparity information on the V-Map at each location of the structure such as the curbstone that is the same as the end point coordinates (Y coordinate) of each road marking portion are read, and the distance information of each road marking portion is calculated (step S110). ). Compare each distance between a structure such as a curb at each branch point of an intersection and each road marking section with a predetermined value of the stop line position feature amount. Is detected as a stop line. On the other hand, when it is larger than the stop line position feature amount, the road marking part is detected as a road marking part other than the stop line (step S111).

  Finally, the stop line status is displayed based on the positional relationship on the luminance image. Specifically, it provides information that the stop line is ahead of the traveling direction of the host vehicle. For example, as shown in FIG. 9 (i), the detected stop line is displayed surrounded by a frame, for example. The detection result display unit 26 in FIG. 3 has a function of outputting various images from the camera and detection result images (images in which stop lines are superimposed and displayed with lines and frames) as an image for the user to confirm.

  In the stop line detection processing flow shown in FIG. 8, the curbstone, the object, and the road marking unit are classified based on the height information, and then the road marking unit based on the stop line width feature amount of the classified road marking unit. To extract. Next, the road marking part of the stop line is detected based on the distance between the road marking part and a structure such as a curb stone provided at the intersection of the intersection and the stop line position feature amount. It is not necessary to limit to the order of processing. Further, after classifying a structure such as a curbstone, an object, and a road marking unit based on height information, a stop line may be detected based on the stop line width feature amount of the classified road marking unit. In other words, the road marking part can be extracted by distinguishing structures such as curbs and objects based on the height information, and the road surface of the stop line can be extracted by comparing the width in the road width direction of the road marking part with the stop line width feature quantity. Detect with marking part. In addition, the road marking portion is extracted by comparing the width in the road width direction of the road marking portion in the traveling road surface area with the stop line width feature amount, and a structure such as a curb stone provided at a branch point of the intersection with the road marking portion and The stop line can be detected by comparing the distance between the stop line and the stop line position feature amount.

  In the stop line detection process in the stop line detection apparatus of the above-described embodiment, the road marking part of the stop line is detected based on the road surface height difference feature quantity, the stop line width feature quantity, and the stop line position feature quantity. On the road surface, a road marking part and a paint part for displaying various information are formed. If there is no road marking part or paint part with almost the same width as the road marking part of the stop line in them, the thing which has a height component based on the road height difference feature quantity without using the above stop line position feature quantity Detect the image. And the road marking part of a stop line can be specified based on a stop line width feature-value with respect to road marking parts other than the image. That is, in the stop line detection process flow of FIG. 8, each process of steps S101 to S109 is performed, and if there is no road marking part or paint part having the same width as the road marking part of the stop line, the process of step S109 is performed. Furthermore, the road marking part of the stop line can be detected. As a result, the road marking part of the stop line can be detected with high accuracy even in the traveling road surface area other than the vicinity of the intersection.

What has been described above is merely an example, and the present invention has a specific effect for each of the following modes.
(Aspect 1)
In a stop line detection device that detects a stop line existing on a traveling road surface of a moving body based on a captured image obtained by imaging the traveling direction of the moving body, from the traveling road surface area where the moving body is traveling to another traveling road surface area A structure that is provided at a branching point that diverges and divides the traveling road surface region and the outside of the traveling road surface region, and a long belt-like road marking portion that extends in the road surface width direction is detected from the captured image, and the detected The road marking part existing at a position away from the position of the structure by a predetermined distance is detected as a road marking part of a stop line.
According to this, as described in the above embodiment, there may be an intersection such as a crossroad or a T-junction that is a branching point divided into other roads on the road. In many cases, there are traffic lights at intersections of the crossroads, but there are intersections where there are traffic lights and intersections where there are no traffic lights. In the case of a crossroad intersection and a T-junction intersection with a traffic light, on the road surface immediately before the intersection, a road marking part of a stop line is formed on the road surface as a mark for stopping the vehicle when the traffic light is red Has been. In the case of a T-shaped intersection without a traffic light, a road marking portion for a stop line for temporary stop is formed on the road surface of a roadway (traveling road surface area) that is not a priority road before the intersection with the priority road. In addition, when there is a traveling road surface area and an area outside the traveling road surface area, generally, a boundary between the traveling road surface area and the outside of the traveling road surface area and along the traveling road surface area is provided at the boundary portion along the running road surface area. Etc. are provided. Unlike the road marking portion formed by painting, the structure is extremely rarely passed by the vehicle and is hardly scraped by the vehicle tire, so the shape of the structure does not change. The captured image always shows the structure clearly. As a result, the structure can be accurately detected from the captured image. As a result, the road marking part that exists at a predetermined distance from the position of the structure detected from the captured image can be accurately detected as the road marking part of the stop line. When this stop line detection invention is used in, for example, a driving support system, the vehicle can be automatically stopped at the stop line by accurately detecting the road marking part of the stop line, and the driving burden on the driver is reduced. Can be reduced.
(Aspect 2)
In (Aspect 1), based on parallax image information generated by the parallax image generation unit 22 or the like from a plurality of captured images obtained by imaging the traveling direction of the moving body by a plurality of imaging units such as the cameras 11 and 12, The vertical parallax histogram information indicating the frequency distribution of the parallax values in each row region obtained by dividing the captured image into a plurality of left and right directions is generated by the parallax map generation unit 24 and the like, and based on the generated vertical parallax histogram information Detect structures.
According to this, as described in the above embodiment, it is possible to distinguish and detect an object such as a building, a vehicle, and a curb stone provided at a branching point of an intersection on a captured image, and stop line detection processing. Can be executed in a short processing time.
(Aspect 3)
In (Aspect 1), the traveling road surface area of the moving body is detected from the captured image by the parallax map generation unit 24 and the like, and the detection processing calculation unit 25 and the like are located at a position away from the position of the structure by a predetermined distance in the traveling road surface area. An existing road marking part is detected as a road marking part of a stop line.
According to this, as described in the above embodiment, the road marking part of the stop line is formed on the traveling road surface, and by detecting the traveling road surface area, an object other than the traveling road surface area, such as an advertisement signboard or The destination display board can be deleted from the detection target. As a result, the number of detection objects is reduced, and the stop line detection process can be executed in a short processing time.
(Aspect 4)
In (Aspect 3), imaging is performed based on parallax image information generated by the parallax image generation unit 22 or the like from a plurality of captured images obtained by imaging the traveling direction of the moving body by a plurality of imaging units such as the cameras 11 and 12. The parallax map generating unit 24 generates horizontal parallax histogram information indicating the frequency distribution of the parallax values in each row region obtained by dividing the image into a plurality of parts in the vertical direction, and travels based on the generated horizontal parallax histogram information. A road surface area is detected.
According to this, as described in the above embodiment, the portion of the horizontal direction parallax histogram information corresponding to the traveling road surface area is limited to a relatively narrow range. By appropriately setting this range, the part of the lateral parallax histogram information corresponding to the traveling road surface area is extracted by distinguishing from the part of the lateral parallax histogram information not corresponding to the traveling road surface area. By detecting the traveling road surface area with high accuracy, an object outside the traveling road surface area can be reliably deleted from the detection target. Thereby, the stop line detection process can be executed in a short processing time.
(Aspect 5)
In (Aspect 1), the predetermined distance is calculated based on parallax image information generated based on a plurality of captured images obtained by imaging the traveling direction of the moving body by a plurality of imaging units.
According to this, the parallax image is generated by the parallax image generation unit 22 as described in the above embodiment. A predetermined distance can be calculated based on the parallax values of mutually corresponding portions on the parallax image, and a road marking unit existing at the predetermined distance can be accurately detected. Thereby, the road marking part of a stop line can be detected with high accuracy.
(Aspect 6)
In (Aspect 1), the road surface marking portion of the stop line has at least a road surface width in the road surface width direction or a travel direction width in the moving body travel direction orthogonal to the road surface width direction.
According to this, as described in the above embodiment, the road marking portion of the stop line is a long strip-shaped road marking portion extending in the road width direction, and the road width in the road width direction or the moving body traveling direction. The driving direction width is regulated by ministerial ordinances such as road structure ordinance. It is possible to distinguish a road marking portion that does not have a specified width from among road marking portions that exist at a predetermined distance from the position of the structure detected from the captured image. Thereby, the road marking part of a stop line can be detected with high accuracy.
(Aspect 7)
In a stop line detection device that detects a stop line existing on a traveling road surface of a moving body based on a captured image obtained by capturing the traveling direction of the moving body, the stop line detecting device has a height component in a traveling road surface area where the moving body is traveling. An image of an object and an image of a long belt-shaped road marking portion extending in the road width direction are detected from the captured image, and an image of the road marking portion other than an image of an object having a height component in the traveling road surface area Among them, a road marking portion having a predetermined road width in the road width direction is detected as a road marking portion of a stop line.
According to this, as explained about the above-mentioned embodiment, there is a road marking part and a paint part for displaying various information on the traveling road surface. If there is no road marking part or paint part with the same width as the road marking part of the stop line in them, position information that there is a stop line at a predetermined distance from the position of the structure around the intersection Without consideration, the stop line can be detected from the captured image based on the image of the object having the height component in the traveling road surface area and the image of the road surface marking unit. Thereby, the road marking part of the stop line which exists in the traveling road surface area | regions other than the intersection periphery can be detected with high precision.
(Aspect 8)
Stop line detection means such as a stop line detection device 10 that picks up an image of the periphery of the moving body as an imaging region and detects a road marking portion of a stop line existing on the traveling road surface from a plurality of captured images obtained by the plurality of imaging means. And a moving body control device such as the vehicle travel control unit 104 that controls a device mounted on the moving body based on the detection result of the stop line detecting means. (Aspect 1) to (Aspect 7) are used.
According to this, as described in the above embodiment, based on the detection result of the stop line, for example, when the driver does not operate the brake pedal when the traffic light ahead is red, It is possible to execute the driving support control such as notifying the driver of the vehicle of the warning to stop the vehicle or performing the driving support control such as automatically controlling the brake of the host vehicle with high accuracy.
(Aspect 9)
In a stop line detection program for causing a computer to execute a step of detecting a stop line existing on a traveling road surface of a moving body based on a captured image obtained by imaging the traveling direction of the moving body, a traveling road surface area in which the moving body is traveling A structure that is provided at a branching point that branches from the road surface to another road surface area, and that divides the road surface area and the road surface area outside, and a long belt-like road marking portion that extends in the road surface width direction, And detecting the road marking portion present at a position away from the detected position of the structure by a predetermined distance as a road marking portion of a stop line by a computer.
According to this, as described in the above embodiment, the road marking part of the stop line can be accurately detected. This program can be distributed or obtained in a state of being recorded on a recording medium such as a CD-ROM. It is also possible to distribute and obtain signals by placing this program and distributing or receiving signals transmitted by a predetermined transmission device via transmission media such as public telephone lines, dedicated lines, and other communication networks. Is possible. At the time of distribution, it is sufficient that at least a part of the computer program is transmitted in the transmission medium. That is, it is not necessary for all data constituting the computer program to exist on the transmission medium at one time. The signal carrying the program is a computer data signal embodied on a predetermined carrier wave including the computer program. Further, the transmission method for transmitting a computer program from a predetermined transmission device includes a case where data constituting the program is transmitted continuously and a case where it is transmitted intermittently.
(Aspect 10)
In a stop line detection program for causing a computer to execute a step of detecting a stop line existing on a traveling road surface of a moving body based on a captured image obtained by imaging the traveling direction of the moving body.
An image of an object having a height component in the traveling road surface area where the moving body is traveling and an image of a long belt-shaped road surface marking portion extending in the road surface width direction are detected from the captured image, and the traveling road surface area is detected. The computer executes a step of detecting a road marking portion having a predetermined road width in the road width direction as a road marking portion of the stop line in the road marking portion image other than the image of the object having the height component inside Let
According to this, as described in the above embodiment, the road marking part of the stop line can be accurately detected. This program can be distributed and obtained in the same manner as in the ninth aspect.

10 Stop Line Detection Device 11 Camera 12 Camera 13 FPGA
14 MPU
15 frame buffer 16 memory 17 image display unit 21 stereo image acquisition unit 22 parallax image generation unit 23 feature amount storage unit 24 parallax map generation unit 25 detection processing calculation unit 26 detection result display unit 100 own vehicle 101 imaging unit 102 image analysis unit 103 Windshield 104 Vehicle travel control unit

JP 2013-30007 A

Claims (10)

In a stop line detection device that detects a stop line existing on the traveling road surface of the moving body based on a captured image obtained by imaging the traveling direction of the moving body,
A structure that divides the traveling road surface area and the outside of the traveling road surface area, and is formed in a long strip extending in the road surface width direction, provided at a branching point that branches from the traveling road surface area where the mobile body is traveling to another traveling road surface area. The road marking portion is detected from the captured image, and the road marking portion existing at a position away from the detected position of the structure by a predetermined distance is detected as a road marking portion of a stop line. Stop line detection device. The stop line detection device according to claim 1,
Parallax within each row region obtained by dividing the captured image into a plurality of left and right directions based on parallax image information generated from a plurality of captured images obtained by capturing the traveling direction of the moving body with a plurality of imaging means A stop line detection device that generates vertical disparity histogram information indicating a frequency distribution of values and detects the structure based on the generated vertical disparity histogram information. The stop line detection device according to claim 1,
Detecting a traveling road surface area of the moving body from the captured image, and detecting the road surface marking portion existing at a predetermined distance from the position of the structure in the traveling road surface area as a road surface marking portion of a stop line. A characteristic stop line detection device. In the stop line detection device according to claim 3,
Based on parallax image information generated from a plurality of captured images obtained by capturing the traveling direction of the moving body with a plurality of imaging means, a parallax value in each row region obtained by dividing the captured image into a plurality of vertical directions A stop line detecting device, wherein horizontal disparity histogram information indicating a frequency distribution of the vehicle is generated, and the traveling road surface area is detected based on the generated horizontal disparity histogram information. The stop line detection device according to claim 1,
The stop line detection apparatus characterized in that the predetermined distance is calculated based on parallax image information generated based on a plurality of captured images obtained by imaging the traveling direction of the moving body by a plurality of imaging means. The stop line detection device according to claim 1,
The stop line detection device according to claim 1, wherein the road marking portion of the stop line has at least a predetermined road surface width in the road surface width direction or a predetermined travel direction width in a moving body travel direction orthogonal to the road surface width direction. In a stop line detection device that detects a stop line existing on the traveling road surface of the moving body based on a captured image obtained by imaging the traveling direction of the moving body,
An image of an object having a height component in the traveling road surface area where the moving body is traveling and an image of a long belt-shaped road surface marking portion extending in the road surface width direction are detected from the captured image, and the traveling road surface area is detected. A road marking part having a predetermined road surface width in the road width direction is detected as a road marking part of a stop line in an image of a road marking part other than an image having an inner height component. Stop line detection device. Stop line detection means for picking up an image of the periphery of the moving body as an imaging region and detecting a road surface marking portion of a stop line existing on the traveling road surface from a plurality of captured images obtained by the plurality of imaging means, and the stop line detection means In the moving body control device comprising the moving body control means for controlling the device mounted on the moving body based on the detection result of
A moving body control device using the object detection device according to claim 1 as the stop line detection means. In a stop line detection program for causing a computer to execute a step of detecting a stop line existing on a traveling road surface of a moving body based on a captured image obtained by imaging the traveling direction of the moving body.
A structure that divides the traveling road surface area and the outside of the traveling road surface area, and is formed in a long strip extending in the road surface width direction, provided at a branching point that branches from the traveling road surface area where the mobile body is traveling to another traveling road surface area. And detecting the road surface marking portion present at a position away from the detected position of the structure by a predetermined distance as a road surface marking portion of a stop line. A program for detecting a stop line. In a stop line detection program for causing a computer to execute a step of detecting a stop line existing on a traveling road surface of a moving body based on a captured image obtained by imaging the traveling direction of the moving body.
An image of an object having a height component in the traveling road surface area where the moving body is traveling and an image of a long belt-shaped road surface marking portion extending in the road surface width direction are detected from the captured image, and the traveling road surface area is detected. The computer executes a step of detecting a road marking portion having a predetermined road width in the road width direction as a road marking portion of the stop line in the road marking portion image other than the image of the object having the height component inside A program for detecting a stop line.