JP4692831B2 - Peripheral situation recognition device and method - Google Patents

Description

  The present invention relates to a surrounding situation recognition device and method for recognizing the presence of an obstacle around a host vehicle based on image information from an imaging device and feature information acquired from a database, and the surrounding situation recognition device. The present invention relates to a navigation device and a vehicle control device used.

  A technique for recognizing an obstacle existing around a host vehicle using a camera or a radar mounted on the vehicle is already known. Regarding such a technique, for example, Patent Document 1 below discloses the following obstacle detection device. This device detects at least the presence / absence of a crossing road or the number of lanes as a road configuration on the traveling direction ahead of the traveling path of the host vehicle or a predicted traveling locus based on information acquired from a navigation device or the like. Then, depending on the current position of the host vehicle and the direction of travel (straight, left / right, lane change, etc.), and further depending on the driver's gaze direction detected by the gaze detection device, the radar detection direction and the camera Set the imaging direction. Then, the control device calculates the possibility that the moving object or obstacle detected by the radar or camera will interfere with the traveling of the vehicle, and according to this possibility, occupant protection such as a braking device, various transmission devices, and an air bag Activate the device.

  Here, the radar transmits a transmission signal such as a laser beam or a millimeter wave in an appropriate detection direction, and receives a reflection signal generated by reflecting the transmission signal by an obstacle or the like. Then, based on these reflection signal and transmission signal, a relative distance, a relative speed, and the like to an obstacle in a region within a predetermined detection area are calculated. The camera is, for example, a CCD camera that can capture an image in the visible light region or the infrared region, and images the outside world in an appropriate detection direction. Then, the image processing unit performs image processing on the image output from the camera, and detects other vehicles around the host vehicle, obstacles such as pedestrians, signs, and the like. Further, along with these detection results, information on the relative distances from the host vehicle to each obstacle recognized is generated and output.

Japanese Patent Laying-Open No. 2005-231450 (page 6-16, FIG. 1-12)

  However, as in the case of the above-described device, in a configuration that directly recognizes other vehicles, pedestrians, and the like around the host vehicle by image recognition using an imaging device such as a radar or a camera, erroneous recognition is actually performed. Since it occurs frequently, there is a problem that it is difficult to make a practical device. In other words, in addition to objects to be recognized such as vehicles and pedestrians, various objects such as trees, fences, and sidewalks exist around the actual road, and these become noise components, so that the object to be recognized is accurately recognized. Is disturbed. In particular, when performing image recognition, it is difficult to configure an apparatus that can accurately recognize all of the shapes, colors, and the like of recognition targets such as vehicles. In order to overcome these problems and enable highly accurate recognition, a high-performance and expensive system is required.

  The present invention has been made in view of the above problems, and its object is to recognize the shape and color of an obstacle as a recognition target when recognizing an obstacle around the host vehicle using image recognition. The surrounding situation recognition device and the surrounding situation recognition method that are not easily affected by various differences such as the above, or that can be noise components that exist in the vicinity, and that can stably recognize obstacles, have a relatively simple configuration. It is to be realized.

In order to achieve the above object, the feature configuration of the surrounding situation recognition device according to the present invention is an image information acquisition means for acquiring image information from an imaging device capable of imaging the periphery of the host vehicle, and features on the surroundings of the host vehicle. Feature information acquisition means for acquiring feature information from the database, image recognition means for performing image recognition of the feature included in the image information based on the feature information, and based on the image recognition result Obstacle recognition means for recognizing the presence of obstacles in the vicinity of the host vehicle, wherein the feature information acquisition means obtains feature information about continuous features arranged along the road. The obstacle recognizing means acquires the feature that should be included in the image information based on the feature information, when all or part of the image of the feature cannot be recognized by the image recognizing means. , Own vehicle and this Recognizes that an obstacle exists between the feature further comprises a arrangement relative to the vehicle of the feature with the continuous based on the feature information, the based on the image recognition results by the image recognition unit The position and the size of the obstacle are recognized based on the arrangement of the unrecognizable portion of the continuous feature .

  According to this characteristic configuration, the obstacle existing around the host vehicle is recognized based on the image recognition result for the feature indicated in the feature information acquired from the database. Therefore, there is an advantage that it is difficult to be affected by the shape and color of the obstacle as compared with the case where the obstacle is directly recognized. In addition, since image recognition is performed on a feature whose position, shape, color, and the like are specified in advance based on the feature information, there is an advantage that it is difficult to be influenced by noise components that may be present in the vicinity. Therefore, it can be set as the apparatus which can perform the stable recognition of the obstacle around the own vehicle by a comparatively simple structure.

Moreover, if comprised as mentioned above , according to the arrangement | positioning of the said continuous feature, it will become possible to recognize the arrangement | positioning of an obstacle continuously along a road. Therefore, it is possible to recognize the arrangement of obstacles around the host vehicle in more detail.

Further, if configured as described above, it is assumed that there is an obstacle in a portion where image recognition is not performed despite the position where the continuous feature should be image-recognized based on the feature information. The position and size of the obstacle can be calculated and recognized based on the position. Therefore, the presence state of the obstacle can be recognized in more detail.

Here, the vehicle further includes a vehicle lane recognizing unit that recognizes the vehicle lane that is the lane in which the vehicle is traveling, and the feature information acquisition unit includes a feature for a feature located in the back of the lane adjacent to the vehicle lane. Object information is acquired from a database, and the obstacle recognizing means recognizes, for the feature that should be included in the image information based on the feature information, all or a part of the image of the feature. When it cannot be recognized by the means, it is preferable to recognize that there is an obstacle on the lane adjacent to the own lane .

According to configuration of this, the recognition of the obstacle existing on a lane adjacent to the own lane, will be performed based on the image recognition results for features shown in the feature information obtained from the database. Therefore, there is an advantage that it is difficult to be affected by the shape and color of the obstacle as compared with the case where the obstacle is directly recognized. In addition, since image recognition is performed on a feature whose position, shape, color, and the like are specified in advance based on the feature information, there is an advantage that it is difficult to be influenced by noise components that may be present in the vicinity. Therefore, it can be set as the apparatus which can perform the stable recognition of the obstruction on the lane adjacent to the own lane with a comparatively simple structure.

  Here, it is preferable to further include lane change determination means for determining whether or not a lane change to a lane adjacent to the own lane is possible based on a recognition result by the obstacle recognition means.

  According to this configuration, it is possible to determine whether or not the lane of the host vehicle can be changed based on the recognition result of the presence condition of the obstacle on the lane adjacent to the host lane by the obstacle recognition unit. Therefore, it can be set as the apparatus which can perform the stable determination of the possibility of a lane change with a comparatively simple structure.

Here, it is particularly preferable that the continuous feature includes at least one of a lane marking provided on the road surface and a curb arranged along the road.
As for these features, various things other than the road surface are rarely imaged around and behind them, and the color difference from the road surface is large. Therefore, image recognition is relatively easy, and it becomes possible to recognize obstacles more stably.

  Further, the obstacle recognizing means is a ratio of a portion of the feature that can be recognized by the image recognizing means with respect to the continuous features that should be included in the image information based on the feature information. It is preferable to adopt a configuration for recognizing the presence of the obstacle based on the above.

  If constituted in this way, based on the ratio of the part that was able to recognize the image about the continuous feature that should be included in the image information, the higher the ratio, the less the ratio that the obstacle exists, It becomes possible to easily recognize the presence of an obstacle.

  A characteristic configuration of the navigation device according to the present invention includes a surrounding situation recognition device having the above-described configuration, and a route guidance unit that performs route guidance of the host vehicle. The route guide unit is configured to change the lane of the host vehicle. When it is necessary and the lane change determination means determines that the lane change is possible, the lane change guidance is provided.

  According to this feature configuration, even when the lane change of the host vehicle is necessary, the lane change guidance is provided only when the lane change to the lane adjacent to the host lane is possible. It becomes possible to improve the safety of the machine. Moreover, it is possible to prevent the driver from feeling troublesome by not performing guidance for lane change in a situation where lane change is not possible.

  A characteristic configuration of a vehicle control device according to the present invention includes a surrounding situation recognition device having the above-described configuration, and a control unit that performs operation control of each unit of the host vehicle, wherein the control unit is the obstacle recognition unit. The operation control of each part of the host vehicle is performed based on the recognition result of the vehicle.

  According to this characteristic configuration, for example, the control of each part of the vehicle for preventing the host vehicle from approaching the obstacle or decelerating or the like according to the presence situation of the obstacle around the host vehicle or such Appropriate vehicle control, such as assisting the driver's operation for control, can be performed.

The characteristic configuration of the surrounding situation recognition method according to the present invention includes an image information acquisition step for acquiring image information from an imaging device capable of imaging the periphery of the host vehicle, and acquires feature information about the features around the host vehicle from the database. Included in the image information based on the feature information, and an image recognition step for performing image recognition of the feature included in the image information based on the feature information. An obstacle recognition step for recognizing that there is an obstacle between the vehicle and the feature when all or a part of the image of the feature is not recognized by the image recognition step; comprising a, in the feature information acquiring step, acquires feature information about the feature that a continuous arranged along the road, in the obstacle recognition step, further, based on the feature information The position of the obstacle based on the arrangement of the continuous feature based on the own vehicle and the arrangement of the unrecognizable portion of the continuous feature based on the image recognition result of the image recognition step. And the point is to recognize the size .

  According to this characteristic configuration, the obstacle existing around the host vehicle is recognized based on the image recognition result for the feature indicated in the feature information acquired from the database. Therefore, there is an advantage that it is difficult to be affected by the shape and color of the obstacle as compared with the case where the obstacle is directly recognized. In addition, since image recognition is performed on a feature whose position, shape, color, and the like are specified in advance based on the feature information, there is an advantage that it is difficult to be influenced by noise components that may be present in the vicinity. Therefore, it is possible to stably recognize obstacles around the host vehicle using an apparatus having a relatively simple configuration.

Moreover, if comprised as mentioned above , according to the arrangement | positioning of the said continuous feature, it will become possible to recognize the arrangement | positioning of an obstacle continuously along a road. Therefore, it is possible to recognize the arrangement of obstacles around the host vehicle in more detail.

Further, if configured as described above, it is assumed that there is an obstacle in a portion where image recognition is not performed despite the position where the continuous feature should be image-recognized based on the feature information. The position and size of the obstacle can be calculated and recognized based on the position. Therefore, the presence state of the obstacle can be recognized in more detail.

Here, further comprising a self-lane recognition step recognizes the own lane on which the vehicle is in the lane during driving, the feature information acquiring step, before Symbol for feature located behind from the lane adjacent to the own lane The feature information is acquired from the database, and the obstacle recognition step is configured to recognize the image of all or a part of the feature for the feature to be included in the image information based on the feature information. It is preferable to include an obstacle recognition step of recognizing that an obstacle exists on the lane adjacent to the own lane when the step cannot be recognized .

According to configuration of this, the recognition of the obstacle existing on a lane adjacent to the own lane, will be performed based on the image recognition results for features shown in the feature information obtained from the database. Therefore, there is an advantage that it is difficult to be affected by the shape and color of the obstacle as compared with the case where the obstacle is directly recognized. In addition, since image recognition is performed on a feature whose position, shape, color, and the like are specified in advance based on the feature information, there is an advantage that it is difficult to be influenced by noise components that may be present in the vicinity. Therefore, it is possible to stably recognize an obstacle on the lane adjacent to the own lane using an apparatus having a relatively simple configuration.

1. First Embodiment First, a first embodiment of the present invention will be described with reference to the drawings. In the present embodiment, a case where the surrounding situation recognition device 1 according to the present invention is applied to a navigation device 2 will be described. FIG. 1 is a block diagram schematically showing the overall configuration of the surrounding situation recognition apparatus 1 according to the present embodiment. The surrounding situation recognition device 1 according to the present embodiment is located behind the adjacent lane Lb of the own lane La included in the image information G captured by the imaging device 4 based on the feature information F acquired from the map database DB. Image recognition of the lane marking fa1 located at is performed (see FIGS. 7 and 8). And when there exists a part which cannot recognize an image about the lane line fa1 which should be included in the image information G, a process of recognizing that an obstacle B such as another vehicle Cb exists on the adjacent lane Lb is performed. Further, based on the recognition result of the obstacle B, it is determined whether or not the lane change of the own vehicle Ca to the adjacent lane Lb is possible, and the driver only when the lane change is necessary and the lane change is possible. Guidance for urging to change lanes.

  As shown in FIG. 1, each functional unit of the surrounding situation recognition device 1, specifically, an image information acquisition unit 5, a vehicle position calculation unit 6, a map information acquisition unit 7, a feature information extraction unit 8, an image The recognizing unit 9, the obstacle recognizing unit 10, the navigation calculating unit 11, the own lane recognizing unit 12, and the lane change determining unit 13 are various types of input data with an arithmetic processing unit such as a CPU as a core member. A functional unit for performing processing is implemented by hardware or software (program) or both. The map database DB is a device having a recording medium capable of storing information and its driving means, such as a hard disk drive, a DVD drive equipped with a DVD-ROM, and a CD drive equipped with a CD-ROM. It is provided as a hardware configuration. Hereinafter, the configuration of each unit will be described in detail.

1-1. Imaging device 4
The imaging device 4 includes an imaging device such as a CCD sensor or a CMOS sensor, and a lens that constitutes an optical system for guiding light to the imaging device. The imaging device 4 is provided so as to be capable of imaging at least a road surface around the host vehicle Ca. In this example, as shown in FIG. 2, the host vehicle Ca is provided with a front camera capable of imaging the periphery of the traveling direction as the imaging device 4. An alternate long and short dash line in FIG. 2 indicates an imaging region of the imaging device 4.

1-2. Image information acquisition unit 5
The image information acquisition unit 5 functions as an image information acquisition unit that acquires image information G captured by the imaging device 4. Here, the image information acquisition unit 5 takes in analog imaging information imaged by the imaging device 4 at a predetermined time interval, converts it into image information G of a digital signal, and stores it in the image memory 21. The time interval for capturing the image information G at this time can be set to, for example, about 10 to 50 ms. Thereby, the image information acquisition part 5 can acquire the image information G of the several frames which imaged the periphery of the own vehicle Ca substantially continuously. The image information G once stored in the image memory 21 is output to the image recognition unit 9 and the own lane recognition unit 12.

1-3. Own vehicle position calculation unit 6
The own vehicle position calculation unit 6 functions as own vehicle position information acquisition means for acquiring own vehicle position information S indicating the current position of the own vehicle Ca. Here, the vehicle position calculation unit 6 is connected to the GPS receiver 22, the direction sensor 23, and the vehicle speed sensor 24. Here, the GPS receiver 22 is a device that receives GPS signals from GPS (Global Positioning System) satellites. This GPS signal is normally received every second and output to the vehicle position calculation unit 6. The own vehicle position calculation unit 6 can analyze the signal from the GPS satellite received by the GPS receiver 22 and acquire information such as the position (latitude and longitude), the traveling direction, and the moving speed of the own vehicle. The direction sensor 23 is a sensor that detects the traveling direction of the host vehicle Ca or a change in the traveling direction. The azimuth sensor 23 includes, for example, a gyro sensor, a geomagnetic sensor, an optical rotation sensor attached to the rotating part of the handle, a rotary resistance volume, an angle sensor attached to the wheel part, and the like. Then, the direction sensor 23 outputs the detection result to the vehicle position calculation unit 6. The vehicle speed sensor 24 is a sensor that detects the vehicle speed and movement distance of the host vehicle Ca. The vehicle speed sensor 24 is, for example, a vehicle speed pulse sensor that outputs a pulse signal every time a drive shaft or wheel of the vehicle rotates by a certain amount, a yaw / G sensor that detects the acceleration of the host vehicle Ca, and an integration of the detected acceleration. Configured by a circuit or the like. Then, the vehicle speed sensor 24 outputs vehicle speed information as the detection result to the vehicle position calculation unit 6.

  And the own vehicle position calculating part 6 performs the calculation which pinpoints the present position of the own vehicle Ca by a well-known method based on the output from these GPS receiver 22, the direction sensor 23, and the vehicle speed sensor 24. FIG. In addition, the vehicle position calculation unit 6 acquires the map information E around the vehicle position from the map database DB, and performs the known map matching based on the map information E to indicate the current position of the vehicle vehicle Ca in the map information E. Correction on the road is also performed. And the own vehicle position calculation unit 6 uses the current position information of the own vehicle Ca as the calculation result as the own vehicle position information S, the map information acquisition unit 7, the feature information extraction unit 8, the own lane recognition unit 12, and It outputs to the calculation part 11 for navigation. At this time, the vehicle position information S is represented as position information represented by latitude and longitude, for example. The own vehicle position information S indicates the position of the own vehicle Ca in the direction along the road, but when the road has a plurality of lanes, the own vehicle La that the own vehicle Ca is traveling is specified. It's not as detailed as it can.

1-4. Map database DB
The map database DB is a database that stores map information of various locations. FIG. 3 is an explanatory diagram showing the contents of the map information E stored in the map database DB. As shown in this figure, the map database DB stores a road network layer X1, a road shape layer X2, and a feature layer X3, and these constitute map information E.

  The road network layer X1 is a layer indicating connection information between roads. Specifically, it has information on a large number of nodes N having position information on a map expressed by latitude and longitude, and information on a large number of links L constituting the road by connecting the two nodes N. It is configured. Each link L includes information such as the type of road (type of highway, toll road, national road, prefectural road, etc.) and link length as link information.

  The road shape layer X2 is stored in association with the road network layer X1 and indicates the shape of the road. Specifically, information on a number of road shape complementary points S that are located between two nodes N (on the link L) and have position information on the map expressed in latitude and longitude, and each road shape complementary point Information of the road width W in S.

  The feature layer X3 is stored in association with the road network layer X1 and the road shape layer X2, and is a layer in which information of various features f provided on the road and in the vicinity of the road, that is, the feature information F is recorded. It is. The feature f in which the feature information F is stored in the feature layer X3 is roughly classified into a paint-type feature f and a three-dimensional feature f. Examples of the paint features f include, for example, lane lines that divide the lane along the road (including various lane lines such as a solid line, a broken line, and a double line), and traffic according to the traveling direction that specifies the traveling direction of each lane. Various road markings provided on the road surface such as section markings, pedestrian crossings, stop lines, speed indications, and zebra zones are included. The three-dimensional feature f includes, for example, various solid objects provided on or around the road such as curbs, guardrails, various signs, and traffic lights provided along both sides of the road.

  The feature information F includes the arrangement information, type information, and form information of each feature f as its contents. Here, the arrangement information includes information on the position (latitude and longitude) and direction of the feature f on the map. The type information is information indicating the type of each feature f, and specifically includes “division lines (including line types such as solid lines, broken lines, and double lines)”, “traveling direction markings according to traveling direction”, Includes information such as the type of road marking such as “pedestrian crossing”, the type of feature f itself such as “curbstone”, “guardrail”, and “traffic light”, and the type of sign such as “temporary stop” and “parking prohibition sign”. . In addition to information on the characteristic shape and color of each feature f, the form information includes height information from the road surface in the case of a three-dimensional feature f.

1-5. Map information acquisition unit 7
The map information acquisition unit 7 acquires map information E around the host vehicle Ca from the map database DB based on the host vehicle position information S output from the host vehicle position calculation unit 6. Then, the map information acquisition unit 7 outputs the map information E to the own vehicle position calculation unit 6, the feature information extraction unit 8, the own lane recognition unit 12, and the navigation calculation unit 11. At this time, the range in which the map information acquisition unit 7 acquires the map information E differs depending on the output destination of the map information E. That is, the map information acquisition unit 7 acquires and outputs the map information E in a range corresponding to the range of the map information E required at the output destination. For example, map information E in a relatively narrow range around the host vehicle Ca is used for map matching in the host vehicle position calculation unit 6 and host lane recognition in the host lane recognition unit 12. On the other hand, for the map display on the display unit 27 by the navigation calculation unit 11, a relatively wide range of map information E is used according to the display scale. The map information acquisition unit 7 acquires and outputs necessary map information E according to these uses.

1-6. Own lane recognition unit 12
The own lane recognition unit 12 functions as own lane recognition means for recognizing the own lane La that is the lane in which the own vehicle Ca is traveling. Here, the own lane recognition unit 12 performs image recognition of the image information G imaged by the imaging device 4, and obtains the image recognition result and the map information E around the own vehicle position indicated by the own vehicle position information S. By comparing, the own lane La is recognized. For example, when the image information G as shown in FIG. 4 is acquired, and the map information E around the host vehicle Ca as shown in FIG. 5 is acquired, the host lane La is one of the three lanes. It can be recognized as a center lane. In other words, in the image shown in the image information G shown in FIG. 4, the broken lane markings fa2 and fa3 are located on both sides with respect to the center in the width direction of the image, which is the position of the host vehicle Ca, and further on both sides thereof. Solid line dividing lines fa1 and fa4 are located respectively. On the other hand, according to the map information E shown in FIG. 5, the road on which the host vehicle Ca is traveling is three lanes, and the feature information Fa1 and Fa4 of the solid line on the both sides in the width direction of the road, the width of the road It can be seen that the feature information Fa2 and Fa3 of the broken lane markings that divide each lane are located on the center side in the direction. Therefore, the own lane recognition unit 12 can recognize that the own lane La is the central lane among the three lanes by comparing these pieces of information. In addition, the own lane recognition unit 12 continuously recognizes the own lane La according to the traveling of the own vehicle Ca and accumulates the information, thereby performing a process of increasing the recognition accuracy of the own lane La. The own lane recognition unit 12 outputs information of the recognition result of the own lane La to the feature information extraction unit 8 and the navigation calculation unit 11.

1-7. Feature information extraction unit 8
The feature information extraction unit 8 functions as a feature information extraction unit that extracts, from the map information E, the feature information F about the feature f around the host vehicle Ca that is image-recognized by the image recognition unit 9. Therefore, in the present embodiment, the map information acquisition unit 7 and the feature information extraction unit 8 constitute a feature information acquisition unit 17. Here, the feature information extraction unit 8 is based on the own vehicle position information S, the information on the recognition result of the own lane La by the own lane recognition unit 12, and the lane change request information output from the navigation calculation unit 11 described later. Then, the feature information F about the feature f located behind the adjacent lane Lb adjacent to the lane change direction indicated by the lane change request information with respect to the own lane La is extracted from the map information E and acquired. Here, the feature f located behind the adjacent lane Lb is a feature f located further on the lane change direction side (right side or left side) than the adjacent lane Lb. At this time, the feature information extraction unit 8 extracts the feature information F about the continuous feature f arranged along the road. Here, as such a continuous feature f, a solid line or a broken demarcation line fa provided on the road surface, a curb fb arranged along the road, and the like are preferable. The feature information F about the lane marking fa is extracted. Note that the symbol “fa” is used as a generic symbol for various lane markings such as “fa1” to “fa4”.

  Here, as an example, processing for extracting the feature information F by the feature information extraction unit 8 from the map information E shown in FIG. 6 will be described. In FIG. 6, the symbol “Ca ′” indicates the position of the host vehicle Ca indicated by the host vehicle position information S and the information on the recognition result of the host lane La by the host lane recognition unit 12. As shown in this figure, in this example, the own lane La is a central lane among the three lanes. Further, it is assumed that the lane change request information from the navigation calculation unit 11 indicates a change request to the right lane. In this case, the feature information extraction unit 8 extracts the feature information Fa1 of the solid lane line (roadway center line) fa1 located behind (right side) the adjacent lane (right lane) Lb on the right side of the own lane La. . In this example, the map information E output from the map information acquisition unit 7 to the feature information extraction unit 8 is information in a range that substantially matches the imaging region of the imaging device 4. In addition, the imaging area by the imaging device 4 is determined in advance based on the mounting position of the imaging device 4 in the host vehicle Ca, the imaging direction, the imaging angle of view, and the like. The feature information F extracted by the feature information extraction unit 8 is output to the image recognition unit 9.

  In addition, the feature information extraction unit 8 determines, from the map information E, the continuation feature f around the host vehicle Ca, in this example, the lane lines fa1 to fa4 for identifying the host lane La and the adjacent lane Lb. The feature information Fa1 to Fa4 is extracted and output to the arrangement recognition unit 26 of the obstacle recognition unit 10. The arrangement recognition unit 26 recognizes the positional relationship between the own lane La and the adjacent lane Lb based on the feature information F.

1-8. Image recognition unit 9
The image recognition unit 9 functions as an image recognition unit that performs image recognition of the feature f included in the image information G based on the feature information F output from the feature information extraction unit 8. Specifically, the image recognition unit 9 first extracts an image of a road marking (paint) such as a lane marking fa using a contour extraction by binarization processing, edge detection processing, or the like on the image information G. Thereafter, the image recognizing unit 9 determines the image of the feature f in the image information G based on information such as the position and shape of the feature f indicated in the feature information F output from the feature information extracting unit 8. Is identified, and image recognition of the feature f is performed. In this example, the feature f shown in the feature information F output from the feature information extraction unit 8 is a solid line section located at the back (right side) of the adjacent lane (right lane) Lb on the right side of the own lane La. It is a line (roadway center line) fa1. 7 and 8 show examples of the image information G acquired in each situation and the recognition result of the image information G by the image recognition unit 9. FIG. 7 shows (b) image information G and (c) the recognition result taken in a situation where there is no obstacle B such as another vehicle Cb around as shown in (a). FIG. 8 shows (b) image information G and (c) the recognition result taken in a situation where another vehicle Cb exists in the adjacent lane Lb on the right side of the own lane La as shown in (a). .

  As shown in FIG. 7A, when there is no obstacle B in the adjacent lane Lb on the right side of the own lane La, the image information G includes the image information G as shown in FIG. That all of the solid lane line fa1 located behind the right adjacent lane Lb that should be included in the image (all of the portions that should be included in the image information G of the lane line fa1) are captured. Become. Therefore, as shown in FIG. 7C, the image recognition unit 9 can recognize all the images of the lane marking fa1. On the other hand, as shown in FIG. 8A, when there is an obstacle B such as another vehicle Cb in the adjacent lane Lb on the right side of the own lane La, the image information is displayed as shown in FIG. In G, a part of the lane marking fa1 that should be included in the image information G is hidden behind the obstacle B and cannot be imaged. Therefore, as shown in FIG. 8C, the image recognizing unit 9 cannot recognize a part of the image of the lane marking fa1, and intermittently recognizes the image of the lane marking fa1. . Then, information of the image recognition result by the image recognition unit 9 as shown in FIG. 7C or FIG. 8C is output to the obstacle recognition unit 10.

1-9. Obstacle recognition unit 10
The obstacle recognizing unit 10 functions as an obstacle recognizing unit that recognizes the presence state of the obstacle B around the host vehicle Ca based on the image recognition result by the image recognizing unit 9. In the present embodiment, the obstacle recognition unit 10 recognizes the presence state of the obstacle B on the adjacent lane Lb adjacent to the own lane La. In other words, here, the obstacle recognizing unit 10 selects all of the features f or the features f to be included in the image information G based on the feature information F output from the feature information extracting unit 8. When some images cannot be recognized by the image recognition unit 9, it is recognized that the obstacle B exists on the lane Lb adjacent to the own lane La. Here, the feature f to be included in the image information G based on the feature information F is the feature f from which the feature information F has been extracted by the feature information extraction unit 8, and is determined by the imaging device 4. It is the feature f (whole or part) included in the imaging region of the image information G. Such a feature f is a target of image recognition by the image recognition unit 9. In this example, the feature f is a solid lane line fa1 located behind the adjacent lane Lb on the right side of the own lane La as described above.

  Then, when the obstacle B is present on the adjacent lane Lb on the right side of the own lane La, the obstacle recognizing unit 10 hides all or part of the lane marking fa1 of the solid line behind the obstacle B. The obstacle B is indirectly recognized using the fact that the image cannot be recognized. Specifically, as shown in FIG. 7C, the image recognition unit 9 recognizes all images of the solid lane marking fa <b> 1 located behind the right adjacent lane Lb that should be included in the image information G. If it is possible, the obstacle recognition unit 10 recognizes that the obstacle B does not exist on the adjacent lane Lb on the right side of the own lane La. On the other hand, as shown in FIG. 8C, the image recognition unit 9 recognizes all or a part of the image of the solid lane line fa1 located behind the right adjacent lane Lb that should be included in the image information G. When the obstacle cannot be recognized, the obstacle recognition unit 10 recognizes that the obstacle B exists on the adjacent lane Lb on the right side of the own lane La. That is, the obstacle recognition unit 10 does not recognize the obstacle B by directly recognizing the obstacle B such as another vehicle Cb, but hides behind the obstacle B when the obstacle B exists. Obstacle recognition is indirectly performed using the image recognition result of the feature f that is easy to recognize, such as the lane marking fa1. Here, the position, shape, color, and the like of the lane marking fa are specified in advance, and it is unlikely that various things other than the road surface are imaged around and behind the lane line fa. The difference is also great. Therefore, image recognition is easier than in the case of recognizing an obstacle B such as a vehicle having various shapes and colors, and is less susceptible to noise components that may be present in the vicinity. Therefore, the obstacle recognition stability can be improved by indirectly performing the obstacle recognition using the image recognition result of the lane marking fa.

  In the present embodiment, the obstacle recognition unit 10 includes a recognition rate determination unit 25 that determines a recognition rate for a continuous feature f. And recognition of whether the above obstacles B exist is performed based on the recognition rate determined by the recognition rate determination part 25. FIG. This is because even if a part of the feature f that should be included in the image information G such as the lane marking fa1 cannot be recognized, it may not be caused by the presence of the obstacle B. This is to increase the recognition accuracy by recognizing the obstacle B in consideration. That is, in reality, even if the obstacle B does not exist, the feature is affected by various factors such as dirt and faintness of the feature f to be image-recognized or an error in image recognition. There is a case where all of f cannot be completely recognized. Accordingly, in consideration of these factors, the obstacle recognition unit 10 performs a process of recognizing that the obstacle B exists when the recognition rate determined by the recognition rate determination unit 25 is less than a predetermined threshold.

  Here, the recognition rate is 100 [%] when all of the portions that should be included in the image information G of the continuous feature f that is the recognition target are completely recognized. The area ratio of the recognized part is expressed as a percentage. For example, when the range in which the recognition rate can be lowered due to various factors such as dirt or fading of the feature f or an error in image recognition is set to 10%, it is recognized that the obstacle B exists. The threshold for the recognition rate to be 90%. Therefore, for example, the recognition rate of the lane marking fa1 based on the image recognition result shown in FIG. 7C is 95 [%], and the recognition rate of the lane marking fa1 based on the image recognition result shown in FIG. In the case of [%], the obstacle recognition unit 10 recognizes that the obstacle B does not exist in the case of FIG. 7C, and recognizes that the obstacle B exists in the case of FIG. 11).

  Further, the obstacle recognition unit 10 includes an arrangement recognition unit 26. The arrangement recognizing unit 26 arranges the continuous feature f based on the feature information F with reference to the own vehicle Ca, and the arrangement of the unrecognizable portion of the continuous feature f based on the image recognition result. Based on the above, a process for recognizing the position and size of the obstacle B is performed. FIG. 9 is an explanatory diagram of processing for recognizing the position and size of the obstacle B performed by the arrangement recognition unit 26 in this example. In this figure, as in the situation shown in FIG. 8A, the feature f located behind the adjacent lane Lb on the right side of the own lane La is a solid lane line fa1, and is on the adjacent lane Lb on the right side. The situation is that there is another vehicle Cb. In this case, the arrangement recognizing unit 26 recognizes, as the information of the image recognition result by the image recognizing unit 9, a solid line lane line fa1 located behind the adjacent lane Lb on the right side of the own lane La as shown in FIG. Result information is entered. Further, the feature information Fa1 to Fa4 of the lane markings fa1 to fa4 is input to the arrangement recognition unit 26 as the feature information F from the feature information extraction unit 8.

And the arrangement | positioning recognition part 26 derives | leads-out the obstruction area | region Bc which shows the position and magnitude | size estimated that the obstruction B exists based on these feature information F (Fa1-Fa4) and an image recognition result. Therefore, the arrangement recognizing unit 26 first determines the angle formed by the start end P1 and the end P2 of the unrecognizable part of the lane marking fa1 with the center of the host vehicle Ca as the reference position P0 based on the image recognition result by the image recognizing unit 9. An angle θ2 formed by θ1 and the position P3 in the traveling direction of the host vehicle Ca on the lane line fa1 and the start point P1 is derived. Here, the traveling direction position P3 of the host vehicle Ca on the lane marking fa1 is an intersection of a line passing through the reference position P0 and orthogonal to the traveling direction of the host vehicle Ca and the lane marking fa1. Here, the start end P1, the end point P2, and the traveling direction position P3 are points on the center line of the demarcation line fa1. The angle θ1 is the distance W between the reference position P0 and the lane line fa1, the length L1 from the start end P1 to the end point P2 of the unrecognizable part of the lane line fa1, and the traveling direction position of the host vehicle Ca on the lane line fa1. Based on the length L2 from P3 to the starting point P1, it is calculated and derived by the following equation (1).
θ1 = tan−1 [(W × L1) / {W × W + L2 × (L1 + L2)}] (1)
Further, the angle θ2 is calculated and derived by the following equation (2).
θ2 = tan−1 (L2 / W) (2)

  Then, as shown in FIG. 9, the arrangement recognizing unit 26 is based on the angles θ1 and θ2, and the arrangement of the own lane La and the adjacent lane Lb specified by the feature information Fa1 to Fa4 of the lane markings fa1 to fa4. The obstacle region Bc is derived. The derivation of the obstacle region Bc is performed based on an obstacle table in which the position and size of the obstacle region Bc are defined in advance according to the values of the angle θ1 and the angle θ2, for example. For example, if the obstacle B is assumed to be a vehicle located approximately in the vicinity of the center on the adjacent lane Lb, the obstacle table can be defined using statistical data on the width and length of various vehicles. That is, the values of the angle θ1 and the angle θ2 and the positional relationship between the own lane La and the adjacent lane Lb are defined as an area within the range of the angle θ1 and in the adjacent lane Lb where the vehicle as the obstacle B exists. The Therefore, a rectangular area having a size corresponding to a vehicle having an appropriate width and length that falls within this area can be defined as the obstacle area Bc using statistical data of the width and length of various vehicles. This obstacle table is prepared in advance for various angles θ1 and θ2, and is formed into a table in advance.

1-10. Lane change determination unit 13
The lane change determination unit 13 functions as a lane change determination unit that determines whether or not the lane change to the adjacent lane Lb adjacent to the own lane La is based on the recognition result by the obstacle recognition unit 10. In the present embodiment, the lane change determination unit 13 basically determines that the lane change is possible when the obstacle recognition unit 10 recognizes that no obstacle B exists on the adjacent lane Lb. When the obstacle recognition unit 10 recognizes that the obstacle B exists on the adjacent lane Lb, it is determined that the lane change is impossible. In the present embodiment, the obstacle recognition unit 10 includes the arrangement recognition unit 26. Therefore, even when it is recognized that the obstacle B exists on the adjacent lane Lb, the vehicle Ca changes the lane based on the recognition result of the position and size of the obstacle B by the arrangement recognition unit 26. If there is sufficient space on the adjacent lane Lb, the lane change determination unit 13 determines that the lane change is possible at the side position of the space.

1-11. Calculation unit 11 for navigation
The navigation calculation unit 11 performs calculation processing for displaying the vehicle position using the display device 27 and for route guidance using the display device 27 and the audio output device 28. Specifically, the calculation unit 11 for navigation acquires the map information E around the host vehicle Ca from the map database DB by the map information acquisition unit 7 and displays a map image on the display device 27, and also displays the map image. On top of this, a process for displaying the vehicle mark superimposed on the vehicle position information S output from the vehicle position calculation unit 6 is performed. Further, the navigation calculation unit 11 selects one of the display device 27 and the audio output device 28 based on the guide route and the vehicle position information S set in advance so as to connect the departure point and the destination point by a known method. Alternatively, route guidance is performed using both. Therefore, the navigation calculation unit 11, the display device 27, and the audio output device 28 function as a route guidance unit 31 that provides route guidance for the host vehicle Ca. Although not shown, the navigation calculation unit 11 is connected to various other known configurations necessary for a navigation device such as a user interface such as a remote controller and a touch panel.

Further, in the present embodiment, the navigation calculation unit 11 includes a lane change request generation unit 29 that generates lane change request information when a lane change of the host vehicle Ca is necessary to proceed according to the guide route. . The lane change request generation unit 29 is traveling on a road with a plurality of lanes, and needs to change course at an intersection or the like in order to proceed according to the set guide route. When the own lane La recognized by 12 is not an appropriate lane for changing the course, it is determined that the lane change of the own vehicle Ca is necessary. When it is determined that the lane change is necessary, the lane change request generation unit 29 generates lane change request information for requesting the lane change to the appropriate lane side for the course change. The lane change request information generated here is output to the feature information extraction unit 8.
When the lane change request information is generated by the lane change request generation unit 29 and the lane change determination unit 13 determines that the lane change is possible, the navigation calculation unit 11 performs the lane change request for the driver. Give guidance. In this case, the lane change guidance is performed by using one or both of the display device 27 and the voice output device 28 by displaying a graphic or a message for prompting the lane change to the appropriate lane side for changing the course, voice guidance, or the like. Do.

1-12. Next, a specific example of the surrounding situation recognition method executed in the navigation device 2 including the surrounding situation recognition device 1 according to the present embodiment and the lane change guidance method using the same will be described with reference to FIGS. It demonstrates according to the flowchart shown. Here, first, the image information acquisition unit 5 acquires the image information G captured by the imaging device 14 (step # 01). Next, the vehicle position calculation unit 6 acquires the vehicle position information S (step # 02). Then, the map information acquisition unit 7 acquires map information E around the host vehicle Ca from the map database DB based on the host vehicle position information S acquired in step # 01 (step # 03). Thereafter, the own lane recognition unit 12 recognizes the own lane La, which is the lane in which the own vehicle Ca is traveling (step # 04).

  The lane change request generation unit is a case where the lane change of the host vehicle Ca is not necessary and is based on the guide route set by the navigation calculation unit 11 and the host lane La recognized by the host lane recognition unit 12. If the lane change request has not been generated at 29 (step # 05: NO), the process ends as it is. On the other hand, when the lane change request is generated in the lane change request generation unit 29 (step # 05: YES), the feature information extraction unit 8 obtains the feature information F about the feature f around the host vehicle Ca. Extracted from the map information E (step # 06). In this example, the feature information F about the feature f located behind the adjacent lane Lb adjacent to the lane change direction side indicated by the lane change request information with respect to the own lane La is extracted from the map information E. Next, the image recognition unit 9 performs image recognition of the feature f included in the image information G based on the feature information F extracted in step # 06 (step # 07). Then, the obstacle recognition unit 10 recognizes the presence state of the obstacle B around the host vehicle Ca based on the image recognition result in step # 07 (step # 08). This obstacle recognition process will be described later in detail according to the flowchart of FIG.

  Thereafter, the lane change determination unit 13 determines whether or not the lane change to the adjacent lane Lb adjacent to the own lane La is possible based on the recognition result by the obstacle recognition unit 10 (step # 09). If it is determined that the lane change is impossible (step # 10: NO), the process ends as it is. On the other hand, when it is determined that the lane change is possible (step # 10: YES), the navigation calculation unit 11 guides the lane change using one or both of the display device 27 and the audio output device 28. (Step # 11). The process ends here.

  Next, the obstacle recognition process in step # 08 will be described. In this example, as shown in FIG. 7 and FIG. 8, image recognition is performed on a solid lane line fa1 (an example of a continuous feature f) located behind the adjacent lane Lb on the right side of the own lane La, Based on the recognition result, a process of recognizing the presence state of the obstacle B on the adjacent lane Lb adjacent to the own lane La is performed. Therefore, as shown in FIG. 11, in the obstacle recognition unit 10, first, in the recognition rate determination unit 25, based on the image recognition result in step # 07, a solid lane line fa <b> 1 as the continuous feature f is obtained. The recognition rate is determined for (step # 21). Here, as described above, the threshold of the recognition rate for recognizing that the obstacle B exists is 90 [%]. Therefore, it is determined whether or not the recognition rate is less than 90 [%]. If the recognition rate is less than 90 [%] (step # 22: YES), it is recognized that the obstacle B exists (step # 23). ) The position recognition unit 26 recognizes the position and size of the obstacle B (step # 24). On the other hand, when the recognition rate is 90% or more (step # 22: NO), it is recognized that the obstacle B does not exist (step # 25). Thus, the obstacle recognition process ends.

2. Second Embodiment Next, a second embodiment of the present invention will be described. This embodiment demonstrates the case where the surrounding condition recognition apparatus 1 which concerns on this invention is applied to the navigation apparatus 2 which also served as the vehicle control apparatus 3. FIG. FIG. 12 is a block diagram schematically showing the overall configuration of the surrounding situation recognition apparatus 1 according to the present embodiment. The surrounding situation recognition device 1 according to the present embodiment exists in the traveling direction of the host vehicle Ca included in the image information G captured by the imaging device 4 based on the feature information F acquired from the map database DB. Image recognition of the lane marking fa arranged on the road R1 intersecting the intersection K is performed (see FIGS. 14 and 16). And when there exists a part which cannot recognize an image about the division line fa which should be included in the image information G, the process which recognizes that the obstruction B which affects the passage of the intersection K exists is performed. Further, based on the recognition result of the obstacle B, guidance for prompting the driver of the host vehicle Ca to decelerate is performed, or deceleration control of the host vehicle Ca is performed.

  As shown in FIG. 12, each functional unit of the surrounding situation recognition device 1, specifically, an image information acquisition unit 5, a vehicle position calculation unit 6, a map information acquisition unit 7, a feature information extraction unit 8, an image The recognizing unit 9, the obstacle recognizing unit 10, the navigation calculating unit 11, the intersection recognizing unit 14, the traveling state determining unit 15, and the vehicle control unit 16 are configured to input data using a calculation processing device such as a CPU as a core member. On the other hand, a functional unit for performing various processes is implemented by hardware or software (program) or both. In addition, the map database DB is a record capable of storing information, such as a hard disk drive, a DVD drive equipped with a DVD-ROM, a CD drive equipped with a CD-ROM, as in the first embodiment. A device having a medium and its driving means is provided as a hardware configuration. Hereinafter, the configuration of each part will be described focusing on differences from the first embodiment.

  In the present embodiment, the image information G acquired by the image information acquisition unit 5 is output to the image recognition unit 9. Vehicle speed information as a detection result of the vehicle speed sensor 24 is output to the vehicle position calculation unit 6 and the traveling state determination unit 15. The own vehicle position calculation unit 6 outputs the own vehicle position information S as a calculation result to the map information acquisition unit 7, the feature information extraction unit 8, the intersection recognition unit 14, and the navigation calculation unit 11. The map information acquisition unit 7 outputs the map information E to the own vehicle position calculation unit 6, the feature information extraction unit 8, the intersection recognition unit 14, and the navigation calculation unit 11.

2-1. Intersection recognition unit 14
The intersection recognition unit 14 functions as an intersection recognition unit that recognizes an intersection K existing in the traveling direction of the host vehicle Ca. Here, the intersection recognition unit 14 determines the traveling direction of the host vehicle Ca based on the map information E acquired by the map information acquisition unit 7 and the host vehicle position information S output from the host vehicle position calculation unit 6. It is recognized whether or not the intersection K exists within the predetermined distance Lk. Here, it is preferable that the predetermined distance Lk from the host vehicle Ca that acquires the information of the intersection K is determined within a range in which the feature f in the intersection K can be recognized by the imaging device 4, for example, 100 [m]. Etc. For example, when map information E around the host vehicle Ca as shown in FIG. 13 is acquired, and the distance from the position of the host vehicle Ca to the center of the intersection K is equal to or less than a predetermined distance Lk, the intersection The recognition unit 14 recognizes that the intersection K exists in the traveling direction of the host vehicle Ca. When the intersection recognition unit 14 recognizes that the intersection K exists within the predetermined distance Lk in the traveling direction of the host vehicle Ca, the distance indicating the distance from the position of the host vehicle Ca to the intersection K to the traveling state determination unit 15. Output information. Therefore, this intersection recognition unit 14 also functions as a distance information acquisition unit that acquires distance information to the intersection K. When the intersection recognition unit 14 recognizes that the intersection K exists within the predetermined distance Lk in the traveling direction of the host vehicle Ca, the feature information extraction unit 8 uses the map information E acquired by the map information acquisition unit 7. Output to.

2-2. Traveling state determination unit 15
The traveling state determination unit 15 functions as a traveling state determination unit that determines whether the vehicle speed is high with respect to the distance to the intersection K. Here, the traveling state determination unit 15 is based on the vehicle speed information indicating the traveling speed of the host vehicle Ca acquired by the vehicle speed sensor 24 and the distance information to the intersection K output from the intersection recognition unit 14. The running state is determined. Specifically, the traveling state determination unit 15 determines that the braking distance at the current vehicle speed of the host vehicle Ca is from the host vehicle Ca to the intersection K when the host vehicle Ca is based on stopping before the intersection K. If the distance to the intersection is sufficiently short, it is determined that the vehicle speed is low with respect to the distance to the intersection K. On the other hand, if the braking distance at the current vehicle speed of the host vehicle Ca is close to or exceeds the distance from the host vehicle Ca to the intersection K, it is determined that the vehicle speed is higher than the distance to the intersection K. To do. Note that the braking distance at each speed of the host vehicle Ca varies depending on the weather, road surface conditions, and the like. Therefore, it is preferable to use a braking distance under bad conditions as a criterion for determination. Moreover, it is suitable also as a structure which acquires information, such as a weather and a road surface state, and changes the braking distance used as a reference | standard based on such information. The determination in the traveling state determination unit 15 can be performed using a table that defines a threshold vehicle speed that determines that the vehicle speed of the host vehicle Ca is high in relation to the distance from the position of the host vehicle Ca to the intersection K. Then, the traveling state determination unit 15 outputs traveling state information indicating whether the vehicle speed is high or low with respect to the distance to the intersection K to the navigation calculation unit 11.

2-3. Feature information extraction unit 8
The feature information extraction unit 8 is a feature information extraction unit that extracts, from the map information E, the feature information F about the feature f arranged on the road R1 that intersects the intersection K existing in the traveling direction of the host vehicle Ca. Function. Therefore, in the present embodiment, the map information acquisition unit 7 and the feature information extraction unit 8 constitute a feature information acquisition unit 17. Here, the feature information extraction unit 8 has continuity arranged along the road R1 intersecting the intersection K based on the own vehicle position information S and the deceleration request information output from the navigation calculation unit 11. The feature information F about the feature f is extracted from the map information E and acquired. Here, as such a continuous feature f, a solid line or a broken lane line fa provided on the road surface, a curb arranged along the road, and the like are preferable. The feature information F about the line fa is extracted.

  Here, as an example, a process for extracting the feature information F by the feature information extraction unit 8 from the map information E shown in FIG. 13 will be described. In FIG. 13, the symbol “Ca ′” indicates the position of the host vehicle Ca indicated in the host vehicle position information S. As shown in this figure, in this example, there is a T-junction intersection K in the traveling direction of the host vehicle Ca. The road R0 on which the host vehicle Ca is traveling is a one-lane one-way road, and the road R1 that intersects the road R0 at the intersection K is a one-lane two-lane road. Further, it is assumed that deceleration request information is output from the navigation calculation unit 11. In this case, the feature information extraction unit 8 includes the feature information Fa5 of the solid line (the roadway outer line) fa5 arranged on the near side of the road R1 and the solid line of the solid line arranged in the center in the width direction of the road R1. The feature information Fa6 of the line (roadway center line) fa6 and the feature information Fa7 of the solid line (roadway outer line) fa7 arranged on the back side of the road R1 are extracted (see FIG. 14 for fa5 to fa7). . In this example, the map information E output from the map information acquisition unit 7 to the feature information extraction unit 8 is information in a range that substantially matches the imaging region (see FIG. 14) by the imaging device 4. In addition, the imaging area by the imaging device 4 is determined in advance based on the mounting position of the imaging device 4 in the host vehicle Ca, the imaging direction, the imaging angle of view, and the like. The feature information F extracted by the feature information extraction unit 8 is output to the image recognition unit 9.

2-4. Image recognition unit 9
The image recognition unit 9 functions as an image recognition unit that performs image recognition of the feature f included in the image information G based on the feature information F output from the feature information extraction unit 8. Specifically, the image recognition unit 9 first extracts an image of a road marking (paint) such as a lane marking fa using a contour extraction by binarization processing, edge detection processing, or the like on the image information G. Thereafter, the image recognizing unit 9 determines the image of the feature f in the image information G based on information such as the position and shape of the feature f indicated in the feature information F output from the feature information extracting unit 8. Is identified, and image recognition of the feature f is performed. In this example, the feature f shown in the feature information F output from the feature information extraction unit 8 is three solid lines arranged along the road R1 that intersects at the intersection K in the traveling direction of the host vehicle Ca. The lane markings are fa5 to fa7. 14 and 16 show examples of the situation around the host vehicle Ca and the intersection K, respectively. 15 and 17 show examples of the image information G acquired in each situation and the recognition result of the image information G by the image recognition unit 9. Here, FIG. 14 shows a situation where the obstacle B does not exist around the intersection K, and FIG. 16 shows a situation where the obstacle B that affects the traffic at the intersection K exists. FIG. 15 shows (a) image information G captured in the situation shown in FIG. 14 and (b) the image recognition result. FIG. 17 shows (a) image information G and (b) the image recognition result captured in the situation shown in FIG. 14 and 16 show a situation where a T-junction intersection K exists in the traveling direction of the host vehicle Ca, similarly to the contents of the map information E shown in FIG. The road R0 on which the host vehicle Ca is traveling is a one-lane one-way road, and the road R1 that intersects the road R0 at the intersection K is a one-lane two-lane road. A stop line fc1 exists in the traveling direction of the host vehicle Ca.

  As shown in FIG. 14, when there is no obstacle B around the intersection K, as shown in FIG. 15A, the image information G includes a road R1 that should be included in the image information G. All of the division lines fa5 to fa7 arranged along the line (all of the parts that should be included in the image information G of the division lines fa5 to fa7) are captured. Therefore, as shown in FIG. 15B, the image recognition unit 9 can recognize all the images of the lane markings fa5 to fa7. On the other hand, in the situation shown in FIG. 16, the building fd exists at the left corner before the intersection K as an obstacle B that affects the traffic at the intersection K, and other vehicles are in the lane on the front side of the road R1. Cb is present. For this reason, as shown in FIG. 17A, the image information G includes part of the lane lines fa5 to fa7 arranged along the road R1 that should be included in the image information G. It is impossible to capture the image behind the object B. Therefore, as shown in FIG. 17B, the image recognizing unit 9 cannot recognize a part of the lane markings fa5 to fa7. At this time, since the building fd is located on the near side of the road R1, the three division lines fa5, fa6, and fa7 are hidden behind the building fd and are not imaged. Therefore, in this area, the image recognition unit 9 cannot recognize the images of the three lane markings fa5, fa6, and fa7. On the other hand, since the other vehicle Cb is located on the lane on the near side of the road R1, the two lane lines fa6 and a7 are hidden behind the other vehicle Cb and are not imaged. Therefore, in this area, the image recognition unit 9 cannot recognize the images of the two lane markings fa6 and a7. 14 and 16, the hatched lane markings fa5 to fa7 indicate portions of the lane markings fa5 to fa7 that can be recognized by the image recognition unit 9. 14 and 16 indicate an imaging region of the imaging device 4. The information of the image recognition result by the image recognition unit 9 as shown in FIG. 15B or FIG. 17B is output to the obstacle recognition unit 10.

2-5. Obstacle recognition unit 10
The obstacle recognizing unit 10 functions as an obstacle recognizing unit for recognizing the presence state of the obstacle B that affects the passage of the intersection K in the traveling direction of the host vehicle Ca based on the image recognition result by the image recognizing unit 9. . In the present embodiment, the obstacle recognizing unit 10 determines all the features f for the feature f to be included in the image information G based on the feature information F output from the feature information extracting unit 8. Alternatively, when some images cannot be recognized by the image recognition unit 9, it is recognized that there is an obstacle B that affects the traffic at the intersection K. Here, as the obstacle B that affects the traffic at the intersection K, for example, the obstacle B such as a building fd, a tree, a parked vehicle, or the like that obstructs the prospect of the intersection K, or the road R1 that intersects the intersection K moves. An obstacle B such as a vehicle Cb or a pedestrian is included. Further, the feature f to be included in the image information G based on the feature information F is the feature f from which the feature information F has been extracted by the feature information extraction unit 8, and is an image by the imaging device 4. It is the feature f (whole or part) included in the imaging region of the information G. Such a feature f is a target of image recognition by the image recognition unit 9. In this example, the feature f is three solid dividing lines fa5 to fa7 arranged along the road R1 intersecting at the intersection K in the traveling direction of the host vehicle Ca as described above.

  The obstacle recognition unit 10 is arranged along the road R1 when there is an obstacle B before or on the road R1 intersecting at the intersection K in the traveling direction of the host vehicle Ca. Obstacle B is indirectly recognized by making use of the fact that all or a part of one or more of the lane markings fa5 to fa7 is hidden behind the obstacle B and cannot be recognized. Specifically, as shown in FIG. 15B, when all the images of the three lane markings fa5 to fa7 that should be included in the image information G can be recognized by the image recognition unit 9, the failure The object recognition unit 10 recognizes that there is no obstacle B that affects the passage of the intersection K such as the building fd that obstructs the prospect of the intersection K or the vehicle Cb that moves on the road R1. On the other hand, as shown in FIG. 17B, the image recognition unit 9 can recognize all or a part of one or more of the three lane markings fa5 to fa7 that should be included in the image information G. If not, the obstacle recognition unit 10 recognizes that there is an obstacle B that affects the traffic at the intersection K. That is, the obstacle recognizing unit 10 does not recognize the obstacle B by directly recognizing the obstacle B such as the building fd or another vehicle Cb as in the first embodiment. The obstacle recognition is indirectly performed using the image recognition result of the feature f that is easy to recognize the image such as the lane markings fa5 to fa7 that are hidden behind the obstacle B.

  The obstacle recognizing unit 10 can recognize the lane markings that could not be recognized when all or a part of one or more of the three lane markings fa5 to fa7 could not be recognized by the image recognition unit 9. It is recognized that the obstacle B exists on the near side of fa. Thereby, the obstacle recognition unit 10 can roughly recognize the position of the obstacle B in the direction parallel to the traveling direction of the host vehicle Ca. That is, for example, as shown in FIG. 16, the building fd is located on the left side corner before the intersection K and is closer to the host vehicle Ca than the three lane markings fa5 to fa7. All three of fa7 are hidden behind the building fd and cannot be recognized. Therefore, as shown in the left part of FIG. 17B, the obstacle recognizing unit 10 cannot recognize the three lane lines fa5 to fa7, and the frontmost lane line fa5. It is recognized that there is an obstacle B on the near side. On the other hand, the other vehicles Cb on the lane on the near side of the road R1 are located on the own vehicle Ca side of the two lane markings fa6 and fa7 out of the three lane markings fa5 to fa7. The lane markings fa6 and fa7 are hidden behind the vehicle Cb and cannot be recognized. Therefore, when the obstacle recognizing unit 10 cannot recognize the images of the two lane markings fa6 and fa7 among the three lane markings fa5 to fa7, as shown in the right part of FIG. , It is recognized that the obstacle B is present on the near side of the lane marking fa6 on the near side.

  In the present embodiment, the obstacle recognition unit 10 recognizes the presence state of the obstacle B by dividing the area A1 on the right side and the area A2 on the left side from the intersection K toward the traveling direction of the host vehicle Ca. Do. Here, as shown in FIGS. 14 and 16, the vehicle is divided into a right region A <b> 1 and a left region A <b> 2 on the boundary line a extending in the traveling direction from the host vehicle Ca. Then, the obstacle recognizing unit 10 recognizes whether or not the obstacle B exists for each of the right area A1 and the left area A2. By recognizing the obstacle B in this way, it becomes possible to appropriately use the recognition result of the presence state of the obstacle B according to the traveling direction of the host vehicle Ca at the intersection K. That is, for example, when the host vehicle Ca makes a left turn at the intersection K, information on the presence state of the obstacle B for the right region A1 is important, and when the host vehicle Ca makes a straight or right turn at the intersection K, the right side Information on the presence status of the obstacle B in both the area A1 and the left area A2 is important.

The obstacle recognition unit 10 includes a recognition rate determination unit 25 that determines the recognition rate for the continuous feature f, as in the first embodiment. Also, the threshold of the recognition rate for recognizing that the obstacle B is present is set to 90 [%] as in the first embodiment. Therefore, for example, when the recognition rate of the three lane markings fa5 to fa7 based on the image recognition result shown in FIG. 15B is 95% for both the right region A1 and the left region A2, the obstacle The recognition unit 10 recognizes that the obstacle B does not exist in both the right area A1 and the left area A2. On the other hand, if the recognition rates of the three lane markings fa5 to fa7 based on the image recognition result shown in FIG. 17B are 60% for the right region A1 and 30% for the left region A2, The object recognition unit 10 recognizes that the obstacle B exists in both the right area A1 and the left area A2.
In the present embodiment, the obstacle recognition unit 10 does not include the arrangement recognition unit 26 for recognizing the position and size of the obstacle B. However, as in the first embodiment, a configuration including the arrangement recognition unit 26 is also a preferred embodiment.

2-6. Calculation unit 11 for navigation
As described above, the navigation calculation unit 11 is connected to the display device 27 and the audio output device 28 that are used for displaying the position of the host vehicle Ca and for route guidance. In this embodiment, the navigation calculation unit 11 includes a deceleration necessity determination unit 30 that determines whether or not the host vehicle Ca needs to be decelerated at the intersection K. Based on the map information E, the deceleration necessity determination unit 30 is not clear that the road R0 on which the host vehicle Ca is traveling intersects at the intersection K is a priority road, and the traveling state determination unit 15 When the traveling state information output from the vehicle information indicates that the vehicle speed is high with respect to the distance to the intersection K, the obstacle recognition unit 10 further includes an obstacle B that affects the traffic at the intersection K. When it recognizes, it determines with the deceleration of the own vehicle Ca being required. At this time, the deceleration necessity determination unit 30 determines the necessity of deceleration of the host vehicle Ca based on the presence state of the obstacle B in each of the area A1 on the right side and the area A2 on the left side from the intersection K by the obstacle recognition unit 10. judge. That is, when it is determined that the host vehicle Ca makes a left turn at the intersection K based on the guidance route set in advance or the operation state of the turn signal in the navigation calculation unit 11, the deceleration necessity determination unit 30 The necessity of deceleration of the own vehicle Ca is determined based on the presence state of the obstacle B in the area A1. On the other hand, when it is determined that the host vehicle Ca makes a right turn or goes straight at the intersection K, the deceleration necessity determination unit 30 automatically determines based on the presence state of the obstacle B in both the right area A1 and the left area A2. The necessity for deceleration of the vehicle Ca is determined.

  And the arithmetic operation part 11 for navigation performs the alert | report for the warning which prompts a driver | operator to decelerate, when it determines with the deceleration necessity determination part 30 having the deceleration necessity of the own vehicle Ca. This notification is performed by using one or both of the display device 27 and the voice output device 28, by displaying a graphic or message for prompting deceleration, voice guidance, or the like. Therefore, the navigation calculation unit 11, the display device 27, and the voice output device 28 also function as a notification unit 32 for performing notification to the driver. The navigation calculation unit 11 is connected to a vehicle control unit 16 described later. When the navigation calculation unit 11 determines that the deceleration necessity determination unit 30 determines that the host vehicle Ca needs to be decelerated, the navigation calculation unit 11 outputs a deceleration control command to the vehicle control unit 16, and Deceleration control is performed.

  Further, the deceleration necessity determination unit 30 determines whether or not there is a deceleration request at the intersection K existing in the traveling direction of the host vehicle Ca at the stage before the obstacle recognition unit 10 recognizes the presence state of the obstacle B. Make a decision. In other words, the deceleration necessity determination unit 30 recognizes that the intersection K exists within the predetermined distance Lk in the traveling direction of the host vehicle Ca by the intersection recognition unit 14 and the traveling state determination unit 15 travels the host vehicle Ca. When the state is determined, a deceleration request is determined. The deceleration necessity determination unit 30 does not clearly determine that the road R0 on which the vehicle Ca is traveling is a priority road with respect to the road R1 intersecting at the intersection K based on the map information E, and further determines the traveling state. When the traveling state information output from the unit 15 indicates that the vehicle speed is high with respect to the distance to the intersection K, it is determined that there is a deceleration request. When determining that there is a deceleration request, the deceleration necessity determination unit 30 determines that there is a possibility that deceleration is required at the intersection K, and generates deceleration request information. The deceleration request information generated here is output to the feature information extraction unit 8.

2-7. Vehicle control unit 16
The vehicle control unit 16 functions as a control unit that controls the operation of each unit of the host vehicle Ca. Here, the vehicle control part 16 performs operation control of each part for decelerating the own vehicle Ca, when the deceleration control command is output from the calculating part 11 for navigation. As such deceleration control, specifically, braking by wheel brakes provided on each wheel of the host vehicle Ca, braking by applying engine braking by turning off the throttle and lowering the gear ratio of the transmission, etc. Is done.

2-8. Next, the surrounding situation recognition method executed in the navigation device 2 that also serves as the vehicle control device 3 including the surrounding situation recognition device 1 according to this embodiment, and the deceleration guide method and the deceleration control method using the same. A specific example will be described with reference to the flowchart shown in FIG. Here, first, the image information acquisition unit 5 acquires the image information G captured by the imaging device 14 (step # 31). Next, the vehicle position calculation unit 6 acquires the vehicle position information S (step # 32). Then, the map information acquisition unit 7 acquires map information E around the host vehicle Ca from the map database DB based on the host vehicle position information S acquired in step # 01 (step # 33). Thereafter, the intersection recognition unit 14 recognizes whether or not the intersection K exists within the predetermined distance Lk in the traveling direction of the host vehicle Ca (step # 34).

  If it is determined in step # 34 that the intersection K does not exist within the predetermined distance Lk in the traveling direction of the host vehicle Ca (step # 35: NO), the process ends. On the other hand, when it is recognized that the intersection K exists within the predetermined distance Lk in the traveling direction of the host vehicle Ca (step # 35: YES), the travel state determination unit 15 then determines the distance to the intersection K. It is determined whether or not the vehicle speed of the host vehicle Ca is high (step # 36). Next, the deceleration necessity determination unit 30 of the navigation calculation unit 11 determines whether or not there is a deceleration request at the intersection K recognized in step # 34 (step # 37). If it is determined that there is no deceleration request (step # 37: NO), the process ends. On the other hand, when it is determined that there is a deceleration request (step # 37: YES), the feature information extraction unit 8 then places the road R1 that intersects the intersection K existing in the traveling direction of the host vehicle Ca. The feature information F for the feature f thus extracted is extracted from the map information E (step # 38). In this example, the feature information F about the continuous feature f arranged along the road R1 intersecting the intersection K is extracted from the map information E. Next, the image recognition unit 9 performs image recognition of the feature f included in the image information G based on the feature information F extracted in step # 38 (step # 39).

  Thereafter, the obstacle recognition unit 10 recognizes the presence state of the obstacle B that affects the passage of the intersection K in the traveling direction of the host vehicle Ca based on the image recognition result of step # 39 (step # 40). This obstacle recognition process is almost the same as the obstacle recognition process in the first embodiment shown in the flowchart of FIG. 11 except that the obstacle recognition of the obstacle B in step # 24 is not performed. . When it is recognized in step # 40 that no obstacle B exists (step # 41: NO), the process ends. On the other hand, when it is recognized in step # 40 that the obstacle B exists (step # 41: YES), the deceleration necessity determination unit 30 of the navigation calculation unit 11 determines the necessity for deceleration of the host vehicle Ca ( Step # 42). If it is determined in step # 42 that the host vehicle Ca does not need to be decelerated (step # 43: NO), the process ends. On the other hand, if it is determined in step # 42 that the host vehicle Ca needs to be decelerated (step # 43: YES), the navigation calculation unit 11 and the vehicle control unit 16 warn the driver to decelerate. One or both of the notification for the control and the operation control of each part for decelerating the host vehicle Ca is executed (step # 44). The process ends here.

3. Other Embodiments (1) In the first embodiment described above, the navigation device 2 has been described as a configuration that does not include the vehicle control unit 16. However, the navigation device 2 including the surrounding situation recognition device 1 according to the first embodiment has a configuration including the vehicle control unit 16 as in the second embodiment, and also serves as the vehicle control device 3. 2 is also one of the preferred embodiments.

(2) In each of the above-described embodiments, a case has been described in which the imaging device 4 includes a front camera capable of imaging the periphery of the traveling direction. However, the configuration of the imaging device 4 is not limited to this. Therefore, for example, the imaging device 4 includes a back camera that captures the rear of the host vehicle Ca in addition to the front camera, a side camera that captures the left and right sides of the host vehicle Ca, and the like. It is also a preferred embodiment that the obstacle B is recognized using the image information G.

(3) In each of the above embodiments, the case has been described in which the feature information extraction unit 8 extracts the feature information F of the lane marking fa and performs image recognition. However, the feature f to be subjected to image recognition is not limited to the lane marking fa. That is, in another preferred embodiment, the image recognition target is a feature f such as a road marking other than a curb or road marking fa. It is also possible to use a three-dimensional feature f such as a guardrail as an object of image recognition. Even if the feature f other than the lane marking fa is the target of image recognition, if the target feature is a continuous feature f arranged along a road such as a curb or a guardrail, it is an obstacle. It is more preferable because the arrangement of B can be continuously recognized along the road.

(4) In each of the above-described embodiments, examples in the case where the surrounding situation recognition device 1 according to the present invention is applied to the navigation device 2 and the vehicle control device 3 have been described. However, the peripheral situation recognition apparatus 1 according to the present invention is also applicable to other apparatuses, for example, a seat belt tensioner that adjusts the tension of the seat belt according to the peripheral situation of the host vehicle Ca, and other vehicle safety devices. Is possible.

  According to the present invention, it is possible to recognize the presence of an obstacle around the host vehicle based on the image information from the imaging device and the feature information acquired from the database. Therefore, the present invention can be suitably used for guidance by a navigation device, vehicle control by a vehicle control device, and the like.

The block diagram which shows typically the whole structure of the periphery condition recognition apparatus which concerns on 1st embodiment of this invention. Explanatory drawing which shows arrangement | positioning of an imaging device Explanatory diagram showing the contents of the map information stored in the map database The figure which shows an example of the image information imaged with the imaging device The figure which shows an example of the map information around the own vehicle Explanatory drawing of processing to extract feature information from map information (A) The figure which shows the example of the image recognition result of (b) image information and (c) the image information acquired in the condition where there is no obstacle around (A) The figure which shows the example of (b) image information acquired in the condition where an obstacle exists in the adjacent lane of the own lane, and (c) the image recognition result Explanatory diagram of processing to recognize the position and size of the obstacle The flowchart which shows the surrounding condition recognition method which concerns on 1st embodiment of this invention, and a lane change guidance method using the same The flowchart which shows the obstruction recognition processing of step # 08 of FIG. The block diagram which shows typically the whole structure of the periphery condition recognition apparatus which concerns on 2nd embodiment of this invention. The figure which shows an example of the map information around the intersection which exists in the own vehicle and its advancing direction The top view which shows the situation where there is no obstacle around the intersection which exists in the advancing direction of the own vehicle The figure which shows the example of (a) image information and (b) the image recognition result imaged in the condition shown by FIG. The top view which shows the situation where the obstacle which affects the traffic of the intersection which exists in the traveling direction of the own vehicle exists The figure which shows the example of (a) image information and (b) the image recognition result imaged in the condition shown by FIG. Flowchart showing a surrounding situation recognition method and a deceleration guide method and a deceleration control method using the same according to the second embodiment of the present invention.

1: peripheral situation recognition device 2: navigation device 3: vehicle control device 4: imaging device 5: image information acquisition unit (image information acquisition means)
6: own vehicle position calculation unit 7: map information acquisition unit 8: feature information extraction unit 9: image recognition unit (image recognition means)
10: Obstacle recognition unit (obstacle recognition means)
11: Calculation unit for navigation 12: Own lane recognition unit (own lane recognition means)
13: Lane change determination unit 14: Intersection recognition unit (distance information acquisition means)
15: Traveling state determination unit (traveling state determination means)
16: Vehicle control unit (control means)
17: Feature information acquisition means 24: Vehicle speed sensor (vehicle speed information acquisition means)
25: Recognition rate determination unit 26: Arrangement recognition unit 27: Display device 28: Audio output device 29: Lane change request generation unit 30: Deceleration necessity determination unit 31: Route guidance unit 32: Notification unit G: Image information S: Self Vehicle position information E: Map information F: Feature information DB: Map database La: Own lane Lb: Adjacent lane B: Obstacle Ca: Own vehicle f: Feature K: Intersection

Claims (9)

Image information acquisition means for acquiring image information from an imaging device capable of imaging the periphery of the host vehicle;
Feature information acquisition means for acquiring feature information about features around the vehicle from a database;
Image recognition means for performing image recognition of a feature included in the image information based on the feature information;
Obstacle recognition means for recognizing the presence of obstacles around the vehicle based on the image recognition result,
The feature information acquisition means acquires feature information about a continuous feature arranged along a road,
The obstacle recognizing unit is configured to detect the vehicle when the image recognizing unit cannot recognize all or a part of an image of the feature that should be included in the image information based on the feature information. And the feature is recognized as an obstacle, and the arrangement of the continuous feature based on the feature information based on the own vehicle and the image recognition result by the image recognition means A peripheral situation recognition device for recognizing the position and size of the obstacle based on the arrangement of the unrecognizable portion of the continuous feature based thereon . It further comprises own lane recognition means for recognizing the own lane that is the lane in which the own vehicle is traveling ,
The feature information acquisition means acquires feature information about a feature located behind a lane adjacent to the own lane from a database ,
The obstacle recognizing means, for the features that should be included in the image information based on the feature information, when all or part of the images of the features cannot be recognized by the image recognizing means, The surrounding state recognition apparatus according to claim 1, wherein an obstacle is recognized on the lane adjacent to the own lane.   The surrounding state recognition device according to claim 2, further comprising a lane change determination unit that determines whether or not a lane change to a lane adjacent to the own lane is possible based on a recognition result by the obstacle recognition unit. The surrounding situation recognition device according to any one of claims 1 to 3, wherein the continuous feature includes at least one of a lane marking provided on a road surface and a curb arranged along the road. . The obstacle recognizing unit is based on a ratio of a portion of the feature that can be recognized by the image recognizing unit with respect to the continuous feature that should be included in the image information based on the feature information. The surrounding state recognition device according to any one of claims 1 to 4 , wherein the presence state of the obstacle is recognized. A surrounding situation recognition device according to claim 3, and a route guidance means for performing route guidance of the host vehicle,
The navigation device for guiding the lane change when the lane change of the host vehicle is necessary and the lane change determination unit determines that the lane change is possible. A surrounding situation recognition device according to any one of claims 1 to 5 , and a control means for controlling the operation of each part of the host vehicle,
The said control means is a vehicle control apparatus which controls operation | movement of each part of the own vehicle based on the recognition result by the said obstacle recognition means. An image information acquisition step of acquiring image information from an imaging device capable of imaging the periphery of the host vehicle;
A feature information acquisition step of acquiring feature information about features around the vehicle from a database;
An image recognition step for recognizing an image of a feature included in the image information based on the feature information;
For a feature that should be included in the image information based on the feature information, when all or a part of the image of the feature cannot be recognized by the image recognition step, between the vehicle and the feature An obstacle recognition step for recognizing that an obstacle exists in the
Equipped with a,
In the feature information acquisition step, the feature information about the continuous features arranged along the road is acquired,
In the obstacle recognizing step, further, the arrangement of the continuous features based on the own vehicle of the continuous features based on the feature information and the continuous features based on the image recognition result in the image recognition step. A surrounding situation recognition method for recognizing the position and size of the obstacle based on the arrangement of unrecognizable portions . A vehicle lane recognition step for recognizing the vehicle lane that is the lane in which the vehicle is traveling ;
The feature information obtaining step obtains pre Symbol feature information about feature located behind from the lane adjacent to the own lane from the database,
In the obstacle recognition step, when the feature which should be included in the image information on the basis of the feature information cannot be recognized by the image recognition step, all or a part of the feature can be recognized in the own lane. The surrounding situation recognition method according to claim 8, wherein an obstacle is recognized in an adjacent lane.

Images