JP2018128906A - Vehicle control device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a vehicle control device capable of recognizing a road environment even when performing a plurality of types of recognition processing.SOLUTION: A vehicle control device sets a recognition result of individual recognition parts 80 and 82, that is, weighting coefficients C11 and C12 for lane information, on the basis of state information indicating a state of inside and outside of a vehicle, and estimates the lane information as an integrated road environment using the plurality of recognition result and the weighting coefficient C11 and C12 corresponding to each recognition result.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a vehicle control apparatus that recognizes a road environment and controls automatic traveling of a vehicle.

  A vehicle capable of traveling by automatic driving performs control such as acceleration / deceleration, steering, braking, etc. while recognizing the surrounding road environment. For example, road environments such as lanes and stop lines are recognized based on image information captured by a camera and map information stored by a navigation device.

  In Patent Document 1, either a road shape estimated from road data (map information) or a road shape estimated from an image (image information) taken by a photographing means (camera) is used as the reliability of each road shape. An apparatus for selecting based on is disclosed. Patent Document 2 discloses an apparatus that generates a virtual travel lane based on map data (map information) when the accuracy of recognition of the travel lane is low based on a camera shooting result (image information). .

JP 2001-291197 A (Claim 1) Japanese Patent Laying-Open No. 2015-005132 (paragraphs [0041] and [0042])

  The devices of Patent Documents 1 and 2 recognize the road environment using either image information or map information. As described above, when performing a plurality of different types of recognition processes, it is difficult to switch between a recognition process based on image information and a recognition process based on map information at an appropriate timing. If the switching timing is incorrect, the recognition accuracy of the road environment may be lowered.

  The present invention has been made in consideration of such problems, and an object of the present invention is to provide a vehicle control device that can recognize a road environment even when a plurality of different types of recognition processes are performed.

  The present invention is a vehicle control device that recognizes a road environment and controls automatic driving without relying on a driver for at least a part of vehicle driving, and recognizes the road environment based on input information. A weighting coefficient setting unit that sets a weighting coefficient for a recognition result of each of the recognition units based on state information indicating a state inside and outside the vehicle, and a plurality of the recognition results and the weighting coefficient for the recognition results. And a road environment estimation unit for estimating the integrated road environment.

  According to the present invention, a weighting coefficient for a recognition result of each recognition unit is set based on state information indicating a state inside and outside the vehicle, and an integrated road is obtained using a plurality of recognition results and weighting coefficients corresponding to each recognition result. Estimate the environment. In this way, by changing the weighting coefficient and integrating the recognition results using the changed weighting coefficient as the final recognition result, from one recognition process to another recognition process. Instantaneous switching is not required. For this reason, when the recognition process is switched, an event that the recognition accuracy of the road environment becomes low does not occur, and the road environment can be always recognized.

  The vehicle control device includes a reliability determination unit that individually determines the reliability of the plurality of recognition units based on the state information, the road environment is lane information, and one of the recognition units is a camera An image information recognition unit for recognizing the lane information based on image information captured by the image information recognition unit, and the weighting coefficient setting unit is configured so that the reliability of the image information recognition unit decreases as the reliability of the image information recognition unit decreases. You may make it change the said weighting coefficient with respect to a recognition result. According to the above configuration, the lane information (integrated road environment) can be accurately recognized because the weighting coefficient is appropriately changed.

  The state in which the reliability of the image information recognition unit is low may be a situation in which the vehicle travels an intersection. According to the above configuration, since the weighting coefficient of the recognition result based on the image information of the camera is changed at an intersection where no lane mark exists, the estimation accuracy of the lane information as the integration result can be increased.

  The image information recognition unit may generate a virtual lane using the past image information in the state where the reliability is low. According to the above configuration, since a virtual lane is generated even in a situation where no lane mark exists, a recognition result based on image information can be output. Therefore, the estimation accuracy of the lane information as the integration result can be increased.

  The image information recognition unit may generate a virtual lane using the lane information that can be recognized based on the image information in the state where the reliability is low. According to the above configuration, since a virtual lane is generated even in a situation where no lane mark exists, a recognition result based on image information can be output. Therefore, the estimation accuracy of the lane information as the integration result can be increased.

  One of the recognition units is a map information recognition unit that recognizes the lane information based on map information, and the road environment estimation unit includes the weighting coefficient for the recognition result of the image information recognition unit, and the map information. Based on the weighting coefficient for the recognition result of the recognition unit, the lane information recognized by the image information recognition unit based on the image information, and the lane information recognized by the map information recognition unit based on the map information. Thus, the lane information as the integrated road environment may be estimated. According to the above configuration, the lane information recognized based on the image information captured by the camera, the lane information recognized based on the map information, and the lane information corresponding to the reliability of each recognition unit are estimated. Therefore, appropriate lane information can be acquired.

  The vehicle control device includes a reliability determination unit that individually determines the reliability of the plurality of recognition units based on the state, the road environment is stop line information, and one of the recognition units is a camera An image information recognizing unit that recognizes the stop line information based on image information captured by the image information recognizing unit as the reliability of the image information recognizing unit decreases. The weighting coefficient for the recognition result may be changed. According to the above configuration, the stop line information (integrated road environment) can be accurately recognized because the weighting coefficient is appropriately changed.

  The state information includes information related to the recognition timing of the road environment by the recognition unit, and the reliability determination unit determines that the latest recognition timing of any of the recognition units is the latest of other recognition units. When the reliability is closer to the determination timing of the reliability than the recognition timing, the reliability of any of the recognition units may be higher than the reliability of the other recognition units. According to the above configuration, since the reliability of the recognition unit having a relatively new recognition result is increased, the accuracy of the stop line information (integrated road environment) as the integration result can be increased.

  According to the present invention, when the recognition process is switched, an event that the recognition accuracy of the road environment becomes low does not occur, and the road environment can be recognized.

FIG. 1 is a configuration diagram of an information providing system. FIG. 2 is a configuration diagram of a vehicle including the vehicle control device according to the first embodiment. FIG. 3 is a functional block diagram of the lane recognition unit used in the first embodiment. FIG. 4 is an explanatory diagram of the situation in the first embodiment and the reliability around the intersection. FIG. 5 is an explanatory diagram of various timings and reliability. FIG. 6 is a flowchart of processing performed in the first embodiment. FIG. 7 is a configuration diagram of a vehicle including the vehicle control device according to the second embodiment. FIG. 8 is a functional block diagram of the stop line recognition unit used in the second embodiment. FIG. 9 is an explanatory diagram of the situation of the second embodiment. FIG. 10 is a flowchart of processing performed in the second embodiment.

  DESCRIPTION OF EMBODIMENTS Hereinafter, a vehicle control apparatus according to the present invention will be described in detail with reference to the accompanying drawings by giving preferred embodiments.

[1 Information providing system 10]
The configuration of the information providing system 10 used in each embodiment will be described with reference to FIG. The information providing system 10 includes an information providing server 12, a plurality of autonomous driving vehicles 16 (hereinafter also referred to as “vehicles 16”), a roadside device 18, and a broadcasting station 20.

  The information providing server 12 sends traffic information, road regulation information, weather information (including weather information and weather forecast information), traffic signals to the vehicle 16 via at least one of the roadside device 18 and the broadcasting station 20. Provide information. Although only one information providing server 12 is shown in FIG. 1, a plurality of information providing servers 12 may be provided according to information to be provided. The roadside machine 18 is provided along the road. The broadcasting station 20 is provided in each region.

[2 First Embodiment]
In the first embodiment, a lane 92 (see FIG. 4) in which the vehicle 16 travels is used as a road environment. The lane 92 is recognized by a plurality of modules, and the recognized lane information is integrated to obtain integrated lane information. Is estimated.

[2-1 Configuration of Vehicle 16]
The configuration of the vehicle 16 and the vehicle control device 26 according to the first embodiment will be described with reference to FIG. In this specification, the term “automatic driving” refers not only to “full automatic driving” in which the driving control of the vehicle 16 is automatically performed, but also “partial automatic driving” in which driving control of the vehicle 16 is partially performed automatically This concept includes “support”. In the present specification, it is assumed that the vehicle 16 is capable of switching between driving control, braking control, and steering control between automatic control and manual control. The vehicle 16 is configured to execute automatic driving as an outside world information acquisition device 22, a vehicle sensor 24, a vehicle control device 26, a driving force device 28, a steering device 30, a braking device 32, and a notification device. 34.

  The outside world information acquisition device 22 includes one or more cameras 38, a plurality of radars 40, a plurality of LIDARs 42, a navigation device 44, and a communication unit 46. The camera 38 images the front of the vehicle 16. The radar 40 irradiates an electromagnetic wave around the vehicle 16 and detects a reflected wave with respect to the electromagnetic wave irradiation. The LIDAR 42 irradiates a laser around the vehicle 16 and detects scattered light in response to the laser irradiation. It is also possible to use a fusion sensor that fuses image information captured by the camera 38 and detection information acquired by the radar 40.

  The navigation device 44 measures the current position of the vehicle 16 using detection information from the satellite positioning device, the vehicle sensor 24, and the like while referring to the map information stored in the storage unit 44a, and the user designates from the current position. A travel route to the destination is generated. The navigation device 44 includes an operation switch (including a touch panel), a display, and a speaker as a user interface, displays the generated travel route, and provides voice guidance for the travel route.

  The communication unit 46 includes a plurality of communication devices. The first communication device 46a can receive wide-area traffic jam information, weather information, and the like transmitted from the broadcast station 20. The second communication device 46b can receive traffic information, regulation information, traffic signal information, and the like transmitted from the roadside device 18. The third communication device 46 c can transmit and receive various types of information to and from the third communication device 46 c provided in the other vehicle 16.

  The vehicle sensor 24 includes a plurality of sensors that detect various behaviors of the vehicle 16. For example, a speed (vehicle speed) sensor that detects the speed (vehicle speed) of the vehicle 16, an acceleration sensor that detects acceleration / deceleration, a lateral G sensor that detects lateral G, a yaw rate sensor that detects an angular velocity around the vertical axis of the vehicle 16, a vehicle An azimuth sensor that detects the direction of the vehicle 16, a gradient sensor that detects the gradient of the vehicle 16, and the like. Further, the vehicle sensor 24 includes an operation detection sensor that detects the presence / absence, operation amount, and operation position of each operation device (accelerator pedal, steering wheel, brake pedal, shift lever, direction instruction lever, etc.). For example, an accelerator pedal sensor that detects an accelerator depression (opening) amount, a steering angle sensor that detects an operation amount (steering angle) of a steering wheel, a torque sensor that detects steering torque, a brake pedal sensor that detects a brake depression amount, A shift sensor for detecting a shift position is included.

  The vehicle control device 26 includes one or more ECUs, and includes a CPU 50, a storage device 52, a clock 54, and the like. In the present embodiment, the CPU 50 executes the program stored in the storage device 52, thereby realizing each function realizing unit 60, 62, 64, 66, 68, 70 described later. The function implementing units 60, 62, 64, 66, 68, and 70 can be realized by hardware including an integrated circuit. The function implementation units 60, 62, 64, 66, 68, and 70 will be described in [2-2] below.

  The driving force device 28 includes a driving force ECU and a driving source of the vehicle 16 such as an engine and / or a driving motor. The driving force device 28 generates a traveling driving force (torque) for the vehicle 16 to travel in accordance with a control command output from the vehicle control unit 66, and transmits the traveling driving force (torque) to the wheels via the transmission or directly.

  The steering device 30 includes an EPS (electric power steering system) ECU and an EPS device. The steering device 30 changes the direction of the wheel (steering wheel) in accordance with a control command output from the vehicle control unit 66.

  The braking device 32 is, for example, an electric servo brake that uses a hydraulic brake together, and includes a brake ECU and a brake actuator. The braking device 32 brakes the wheel in accordance with a control command output from the vehicle control unit 66.

  The notification device 34 includes a notification ECU, a display device, and / or an acoustic device. The notification device 34 notifies a manual operation request, warning, guidance, and the like in accordance with a notification command output from the notification control unit 68.

[2-2 Function realization units 60, 62, 64, 66, 68, 70]
Each function realization part 60, 62, 64, 66, 68, 70 of the vehicle control apparatus 26 is demonstrated using FIG.2 and FIG.3. The outside world recognition unit 60 is based on various information acquired by the outside world information acquisition device 22, and the road environment around the vehicle 16 {road structure, lane 92 (lane mark 90), stop line 200 (see FIG. 9), traffic light , Signs, etc.} and traffic participants (other vehicles, pedestrians, obstacles, etc.). The external recognition unit 60 according to the first embodiment is divided into a lane recognition unit 72 and other recognition unit 74.

  As illustrated in FIG. 3, the lane recognition unit 72 includes an image information recognition unit 80, a map information recognition unit 82, a reliability determination unit 84, a weighting coefficient setting unit 86, and a lane information estimation unit 88. The image information recognition unit 80 recognizes lane information based on image information captured by the camera 38. For example, edge extraction such as Hough transform is performed, the presence or absence of the lane marks 90 and their positions are recognized, and the lane 92 on which the vehicle 16 travels is recognized. Hereinafter, the image information recognition unit 80 is also referred to as a “recognition unit 80”. The map information recognition unit 82 recognizes lane information based on the map information stored in the storage unit 44 a of the navigation device 44. The navigation device 44 measures the traveling position of the vehicle 16. The map information recognition unit 82 acquires the node of the lane 92 ahead of the travel position from the map information, and recognizes the lane 92 in which the vehicle 16 travels. Hereinafter, the map information recognition unit 82 is also referred to as a “recognition unit 82”. In this specification, information that can specify the position of the lane 92, such as the position information of the lane mark 90 and the position information of the node, is collectively referred to as lane information.

  The reliability determination unit 84 determines the reliability R11 of the image information recognition unit 80 and the reliability R12 of the map information recognition unit 82 based on the inside and outside states of the vehicle 16. The reliability R11, R12 of the recognition units 80, 82 changes as the state inside and outside the vehicle 16 changes. In the present specification, information indicating a state that correlates with the reliability R11 or R12 of the recognition units 80 and 82 among the internal and external states of the vehicle 16 is referred to as state information. A specific example of the state information will be described in [2-3] below. The reliability determination unit 84 obtains a state parameter P1 by digitizing one or more types of state information based on a predetermined rule. Then, the reliability R11, R12 is determined using the map (table) M1 that associates the state parameter P1 with the reliability R11, R12. The map M1 and the predetermined rules for digitization are stored in the storage device 52.

  The weighting coefficient setting unit 86 sets the weighting coefficient C11 for the recognition result of the image information recognition unit 80 and the weighting coefficient C12 for the recognition result of the map information recognition unit 82 based on the reliability R11, R12. A specific method for setting the weighting coefficients C11 and C12 will be described in [2-4] below.

  The lane information estimation unit 88 estimates integrated lane information (integrated road environment) using the recognition results of the image information recognition unit 80 and the map information recognition unit 82 and the weighting coefficients C11 and C12. The integrated lane information means that the lane information recognized by the image information recognition unit 80 based on the image information and the lane information recognized by the map information recognition unit 82 based on the map information are integrated according to the weighting coefficients C11 and C12. It is a thing. A specific method for estimating the integrated lane information will be described in [2-4] below.

  Returning to FIG. 2, the description of the external recognition unit 60 will be continued. The other recognizing unit 74 is configured to store external information other than the lane 92 based on at least one of the image information of the camera 38, the detection information of the radar 40, the detection information of the LIDAR 42, the map information of the navigation device 44, and the communication information of the communication unit 46. Recognize position and behavior.

  The own vehicle position recognition unit 62 recognizes the current position and posture of the vehicle 16 based on the position information of the vehicle 16 measured by the navigation device 44. Apart from this, it is also possible to measure the current position of the vehicle 16 using detection information from the satellite positioning device, the vehicle sensor 24, etc. without using the navigation device 44, and to recognize the current position and posture of the vehicle 16. is there.

  The track generation unit 64 causes the vehicle 16 to travel along the travel route generated by the navigation device 44, based on the recognition result of the external environment recognition unit 60 and the recognition result of the vehicle position recognition unit 62. Generate travel trajectory and target speed.

  The vehicle control unit 66 outputs a control command to the driving force device 28, the steering device 30, and the braking device 32. The vehicle control unit 66 outputs a control command so that the vehicle 16 travels at the target speed along the target travel trajectory generated by the trajectory generation unit 64 during automatic driving, and the vehicle sensor during manual driving. A control command is output based on the detection result of 24 (operation detection sensor).

  The notification control unit 68 outputs a notification command to the notification device 34 when notification to the driver is necessary, for example, when automatic driving is stopped.

  The automatic operation control unit 70 performs overall control of automatic operation. The automatic driving control unit 70 starts or stops the automatic driving according to the operation of the automatic driving switch performed by the driver. Moreover, the automatic driving | operation control part 70 stops automatic driving | operation, when the manual operation of one of operation devices is detected by the vehicle sensor 24 during automatic driving | operation.

[2-3 Relationship between state information and reliability]
As described above, the reliability determination unit 84 determines the reliability R11 and R12 of the recognition units 80 and 82 based on one or a plurality of state information indicating the internal and external states of the vehicle 16. Specific examples of the state information include, for example, information on the intersection 94 (see FIG. 4), information indicating the curvature of the travel route, information indicating the weather in the travel region, information indicating the current time, lane information (showing the road environment) Information) indicating the freshness of the information.

  The information regarding the intersection 94 is information indicating whether or not the intersection 94 is within a predetermined distance from the vehicle 16, the distance from the vehicle 16 to the intersection 94, the size of the intersection 94, and the like. Information regarding the intersection 94 can be acquired from the map information of the navigation device 44. As shown in FIG. 4, the lane mark 90 does not exist at the intersection 94. In this case, the image information recognition unit 80 generates a virtual lane 92 ′. This means that the reliability R11 is low. The reliability R11 is low when there is an intersection 94, that is, when a virtual lane 92 'is generated. In addition, the reliability R11 decreases as the distance between the vehicle 16 and the intersection 94 decreases, and increases as the distance increases. Further, the reliability R11 decreases as the intersection 94 increases, and increases as the intersection 94 decreases. On the other hand, the reliability R12 is contrary to the reliability R11. The reliability R12 is high when there is an intersection 94, that is, when a virtual lane 92 'is generated. That is, the reliability R12 increases as the distance between the vehicle 16 and the intersection 94 decreases, and decreases as the distance increases. In addition, the reliability R12 increases as the intersection 94 increases, and decreases as the intersection 94 decreases.

  Information indicating the curvature of the travel path can be acquired from the map information of the navigation device 44, and can also be acquired from the image information when the travel path can be photographed by the camera 38. The reliability R11 decreases as the curvature decreases, and increases as the curvature increases. On the other hand, the reliability R12 increases as the curvature decreases, and decreases as the curvature increases.

  Information indicating the weather in the travel area can be acquired from weather information transmitted from the information providing server 12 via the broadcast station 20 or the roadside device 18. The reliability R11 is lower as the weather is closer to rain (snow), and is higher as the weather is closer to sunny. On the other hand, the reliability R12 increases as the weather is closer to rain (snow), and decreases as the weather is closer to sunny.

  Information indicating the current time can be acquired from the clock 54. The reliability R11 is lower as the time is closer to night, and is higher as the time is closer to daytime. On the other hand, the reliability R12 increases as the time is closer to night, and decreases as the time is closer to daytime.

  The freshness of the lane information (information indicating the road environment) will be described with reference to FIG. The freshness of the lane information is information indicating which of the lane information recognized by the image information recognition unit 80 and the lane information recognized by the map information recognition unit 82 is newer. The freshness of the lane information is determined from the lane information recognition timing 110 by the image information recognition unit 80 and the lane information recognition timing 112 by the map information recognition unit 82. The lane information recognition cycle T1 by the image information recognition unit 80 and the lane information recognition cycle T2 by the map information recognition unit 82 are different. For this reason, there is a difference between the recognition timing 110 and the recognition timing 112. The reliability R11 and R12 of the recognition units 80 and 82 that have performed recognition processing at the closest timing retroactively from the determination timing 114 of the reliability R11 and R12 by the reliability determination unit 84 are relatively high. The reliability R11 is low when the time difference between the recognition timing 110 and the determination timing 114 is larger than the time difference between the recognition timing 112 and the determination timing 114, and is high when the time difference is small. On the other hand, the reliability R12 is high when the time difference between the recognition timing 110 and the determination timing 114 is larger than the time difference between the recognition timing 112 and the determination timing 114, and is low when the time difference is small.

[2-4 Weighting coefficient setting method and integrated lane information estimation method]
The weighting coefficient setting unit 86 and the lane information estimation unit 88 set the weighting coefficient by either method 1 or method 2 below, and estimate integrated lane information. As an example of the “recognition result of the recognition units 80 and 82” in the following description, an angle in the extending direction of the lane 92 (lane mark 90) with respect to the reference direction (for example, the traveling direction of the vehicle 16) can be given.

[2-4-1 Method 1]
The weighting coefficient setting unit 86 sets the weighting coefficient (C11 or C12) for the recognition results of the recognition units 80 and 82 having relatively high reliability among the recognition results of the recognition units 80 and 82 as the reference value 1. Further, the weighting coefficient (C11 or C12) for the recognition results of the recognition units 80 and 82 with relatively low reliability is obtained from a predetermined conversion formula or map using the recognition results of the recognition units 80 and 82 and the reliability R11 and R12. calculate. For example, when the reliability R11 of the recognition unit 80 is higher than the reliability R12 of the recognition unit 82, the weighting coefficient setting unit 86 has (weighting coefficient C12 × recognition result of the recognition unit 82) (weighting coefficient C11 × recognition unit 80). The weighting coefficient C12 is calculated so as to approach and not exceed the recognition result. The weighting coefficient C12 is set so that (weighting coefficient C12 × recognition result of the recognition unit 82) approaches (weighting coefficient C11 × recognition result of the recognition unit 80) as the reliability R12 is lower. Similarly, when the reliability R12 of the recognition unit 82 is higher than the reliability R11 of the recognition unit 80, the weighting coefficient setting unit 86 has (weighting coefficient C11 × recognition result of the recognition unit 80) (weighting coefficient C12 × recognition unit). The weighting coefficient C11 is calculated so as to approach and not exceed the recognition result of 82). The weighting coefficient C11 is set so that (weighting coefficient C11 × recognition result of the recognition unit 80) approaches (weighting coefficient C12 × recognition result of the recognition unit 82) as the reliability R11 is lower.

  The lane information estimation unit 88 adds (weighting coefficient C11 × recognition result of recognition unit 80) and (weighting coefficient C12 × recognition result of recognition unit 82), and adds the total value to the number of recognition units 80 and 82 (here, Divide by 2) and use the result as the integrated lane information.

[2-4-2 Method 2]
The weighting coefficient setting unit 86 calculates the weighting coefficients C11 and C12 so that the total value of the weighting coefficients C11 and C12 is 1. At this time, the ratio of the weighting coefficients C11 and C12 is made equal to the ratio of the reliability R11 and R12. That is, the weighting coefficients C11 and C12 are calculated from both equations of C11 + C12 = 1 and R11: R12 = C11: C12.

  The lane information estimation unit 88 adds (weighting coefficient C11 × recognition result of the recognition unit 80) and (weighting coefficient C12 × recognition result of the recognition unit 82), and uses the calculation result as integrated lane information. Unlike the calculation of method 1, the calculation of method 2 does not divide by the number of recognition units 80 and 82.

[2-4-3 Other methods]
The weighting coefficient setting unit 86 can also calculate the weighting coefficients C11 and C12 by a method other than the above methods 1 and 2. In short, any process is used as long as the weighting coefficients C11 and C12 for the recognition results of the recognition units 80 and 82 having relatively high reliability R11 and R12 are heavier than the other weighting coefficients C11 and C12. May be.

[2-5 Processes Executed in First Embodiment]
The process of 1st Embodiment is demonstrated using FIG. A series of processing described below is performed repeatedly at predetermined intervals with the outside recognition unit 60 as a main body. A series of processes described below is started at a predetermined timing. For example, the other recognition unit 74 is started when the intersection 94 is recognized in front of the vehicle 16. Moreover, you may start at another timing.

  In step S1, it is determined whether automatic driving is in progress. When traveling control by automatic driving is performed (step S1: YES), the process proceeds to step S2. On the other hand, when the traveling control by the automatic driving is not performed (step S1: NO), the processing is temporarily ended without executing the processing after step S2.

  In step S <b> 2, the image information recognition unit 80 recognizes lane information based on the latest image information taken by the camera 38. Further, the map information recognition unit 82 recognizes lane information ahead of the current traveling position based on the map information stored in the storage unit 44 a of the navigation device 44.

  As shown in FIG. 4, the image information recognition unit 80 generates a virtual lane 92 ′ when the recognition accuracy of the lane mark 90 is low. The virtual lane 92 ′ can be generated by performing linear interpolation or curve interpolation using a part of the recognizable lane marks 90. For example, the lane mark 90 does not exist at the intersection 94. The image information recognition unit 80 specifies the exit path 98 from the intersection 94 based on the planned travel route. Then, linear interpolation or curve interpolation is performed using the lane mark 90 that can be recognized near the exit 96o of the approach path 96 to the intersection 94 and the lane mark 90 that can be recognized near the entrance 98i of the exit path 98, and the virtual lane mark 90 '. And a virtual lane 92 'is generated.

  The image information recognition unit 80 does not use the lane mark 90 at the exit 96o of the approach path 96 to the intersection 94 and the entrance 98i of the exit path 98 from the intersection 94, but simply recognizes the lane mark 90 that can be recognized at that time. The virtual lane mark 90 ′ may be generated by extending in front of the vehicle 16 according to the shape.

  The image information recognition unit 80 may generate the virtual lane mark 90 ′ using image information acquired in the past. The storage device 52 sequentially stores the position information of the lane mark 90 a certain time before the current time or a certain distance before the current position. The image information recognition unit 80 may generate the virtual lane mark 90 ′ by using the position information and extending the past lane mark 90 in front of the vehicle 16 according to the shape. The storage device 52 may store information on the direction of the lane mark 90 instead of the position information of the past lane mark 90. In that case, the image information recognition unit 80 extends the past lane mark 90 in the direction to generate a virtual lane mark 90 ′.

  In addition, the image information recognition unit 80 acquires the latest image information or image information of a predetermined time before the current time from the other vehicle 16 via the third communication device 46c, and uses the acquired information to obtain the virtual lane mark 90. 'May be generated. Moreover, the image information of the driving | running | working position accumulate | stored in an external server via the communication apparatus which is not shown in figure may be acquired, and virtual lane mark 90 'may be produced | generated using the acquired information. In this case, each vehicle 16 receives image information from an external server via the communication device, and transmits image information captured by the camera 38 to the external server via the communication device.

  In step S3, the reliability determination unit 84 obtains the state parameter P1. As described above, the reliability determination unit 84 acquires one or a plurality of state information and obtains the state parameter P1 based on a predetermined rule.

  In step S4, the reliability determination unit 84 determines the reliability R11 of the image information recognition unit 80 and the reliability R12 of the map information recognition unit 82 based on the state parameter P1 and the map M1. As an example, it is assumed that the determination processing of the reliability R11, R12 is performed based on the state parameter P1 corresponding to the information related to the intersection 94. As shown in FIG. 4, since no lane mark 90 exists at the intersection 94, a virtual lane 92 ′ is set. This means that the reliability R11 is low. On the other hand, the reliability R12 increases as the distance from the intersection 94 increases, and decreases as the distance from the intersection 94 increases.

  As another example of step S4, it is assumed that the reliability R11, R12 is determined based on the state parameter P1 corresponding to the freshness of the lane information. As shown in FIG. 5, when the recognition result of the image information recognition unit 80 is newer than the recognition result of the map information recognition unit 82, the reliability R11 is high and the reliability R12 is low. On the other hand, when the recognition result of the map information recognition unit 82 is newer than the recognition result of the image information recognition unit 80, the reliability R12 is high and the reliability R11 is low.

  In step S5, the weighting coefficient setting unit 86 sets the weighting coefficients C11 and C12 according to the reliability R11 and R12 by the method 1 or the method 2 described in [2-4] above. In step S6, the lane information estimation unit 88 estimates the integrated lane information by the method 1 or the method 2 described in [2-4] above.

[2-6 Summary of First Embodiment]
The vehicle control device 26 according to the first embodiment relates to a device that recognizes lane information and controls automatic driving that does not rely on a driver for at least a part of vehicle driving. The vehicle control device 26 recognizes different recognition units 80 and 82 that recognize lane information based on input information, and recognition results of the individual recognition units 80 and 82 based on state information indicating the internal and external states of the vehicle 16. A lane information estimation unit 88 (road) that estimates lane information as an integrated road environment using a plurality of recognition results and weighting factors C11 and C12 for recognition results and weighting factors C11 and C12. Environmental estimation unit).

  According to the first embodiment, the weighting coefficients C11 and C12 for the recognition results of the individual recognition units 80 and 82 are set based on the state information, and the plurality of recognition results and the weighting coefficients C11 and C12 corresponding to the respective recognition results are set. Is used to estimate lane information as an integrated road environment. In this way, by changing the weighting coefficients C11 and C12 and using the changed weighting coefficients C11 and C12 to integrate the respective recognition results as final recognition results, There is no need to instantly switch to another recognition process. For this reason, when the recognition process is switched, the phenomenon that the recognition accuracy of the lane information is lowered does not occur, and the lane information can be always recognized.

  The vehicle control device 26 includes a reliability determination unit 84 that individually determines the reliability R11 and R12 of the plurality of recognition units 80 and 82 based on the state information. One of the recognition units 80 and 82 is an image information recognition unit 80 that recognizes lane information based on image information captured by the camera 38. The weighting coefficient setting unit 86 changes the weighting coefficient C11 for the recognition result of the image information recognition unit 80 as the reliability R11 of the image information recognition unit 80 decreases. According to the above configuration, the lane information (integrated road environment) can be accurately recognized because the weighting coefficient C12 is appropriately changed.

  The state in which the reliability R11 of the image information recognition unit 80 is low is a situation in which the vehicle 16 travels through the intersection 94. According to the above configuration, since the weighting coefficient C11 of the recognition result based on the image information of the camera 38 is changed at the intersection 94 where the lane mark 90 does not exist, the estimation accuracy of the lane information as the integration result can be increased.

  The image information recognition unit 80 generates a virtual lane 92 ′ using past image information in a state where the reliability R11 is low. According to the above configuration, since the virtual lane 92 ′ is generated even in the situation where the lane mark 90 does not exist, the recognition result based on the image information can be output. Therefore, the estimation accuracy of the lane information as the integration result can be increased.

  The image information recognition unit 80 generates the virtual lane 92 ′ using lane information that can be recognized based on the image information in a state where the reliability R11 is low. According to the above configuration, since the virtual lane 92 ′ is generated even in the situation where the lane mark 90 does not exist, the recognition result based on the image information can be output. Therefore, the estimation accuracy of the lane information as the integration result can be increased.

  One of the recognition units 80 and 82 is a map information recognition unit 82 that recognizes lane information based on map information. The lane information estimation unit 88 includes a weighting coefficient C11 for the recognition result of the image information recognition unit 80, a weighting coefficient C12 for the recognition result of the map information recognition unit 82, and the lane information that the image information recognition unit 80 recognizes based on the image information. Based on the lane information recognized by the map information recognition unit 82 based on the map information, the lane information as the integrated road environment is estimated.

  According to the above configuration, the lane information recognized based on the image information photographed by the camera 38, the lane information recognized based on the map information, and the reliability R11, R12 of each recognition unit 80, 82 Since the corresponding lane information as the integrated road environment is estimated, appropriate lane information can be acquired.

  The state information includes information regarding the recognition timing of the road environment by the recognition units 80 and 82. The reliability determination unit 84 performs arbitrary recognition when the latest recognition timing of the arbitrary recognition units 80 and 82 is closer to the determination timing of the reliability R11 and R12 than the latest recognition timing of the other recognition units 80 and 82. The reliability R11, R12 of the units 80, 82 is set higher than the reliability R11, R12 of the other recognition units 80, 82. According to the above configuration, since the reliability R11 and R12 of the recognition units 80 and 82 having relatively new recognition results are increased, the accuracy of the lane information as the integrated road environment can be increased.

[3 Second Embodiment]
The second embodiment uses a stop line as a road environment, recognizes the stop line 200 by a plurality of modules, and integrates the recognized stop line information to estimate integrated stop line information.

[3-1 Configuration of Vehicle 16A]
The configuration of the vehicle 16A will be described with reference to FIG. The configuration of the vehicle 16A is common in many respects to the configuration of the vehicle 16 shown in FIG. For this reason, the same code | symbol is attached | subjected to a common structure and the description is abbreviate | omitted. The vehicle 16A has a vehicle control device 26A, and the vehicle control device 26A has a CPU 50A. The external recognition unit 160 is realized by the CPU 50A executing the program stored in the storage device 52A.

[3-2 External recognition unit 160]
The external recognition unit 160 will be described with reference to FIGS. The external environment recognition unit 160 is based on various information acquired by the external environment information acquisition device 22, and the road environment around the vehicle 16A {road structure, lane 92 (lane mark 90), stop line 200, traffic light, sign, etc.} and Recognize traffic participants (other vehicles, pedestrians, obstacles, etc.). The external environment recognition unit 160 according to the second embodiment is divided into a stop line recognition unit 172 and another recognition unit 174.

  As shown in FIG. 8, the stop line recognition unit 172 includes a first image information recognition unit 180, a second image information recognition unit 182, a map information recognition unit 183, a reliability determination unit 184, and a weighting coefficient setting unit. 186 and a stop line information estimation unit 188. The first image information recognition unit 180 performs texture extraction based on image information captured by the camera 38, and recognizes the presence / absence of the stop line 200 and its position. Hereinafter, the first image information recognition unit 180 is also referred to as a “recognition unit 180”. The second image information recognition unit 182 performs edge extraction such as Hough transform based on image information captured by the camera 38, and recognizes the presence / absence of the stop line 200 and its position. Hereinafter, the second image information recognition unit 182 is also referred to as a “recognition unit 182”. The map information recognition unit 183 recognizes the presence / absence of the stop line 200 and the position thereof based on the map information stored in the storage unit 44a of the navigation device 44. The navigation device 44 measures the traveling position of the vehicle 16A. The map information recognition unit 183 acquires information on the stop line 200 existing within a predetermined distance ahead of the travel position from the map information, and recognizes the presence and position of the stop line 200. Hereinafter, the map information recognition unit 183 is also referred to as a “recognition unit 183”.

  The reliability determination unit 184 determines the reliability R21 of the first image information recognition unit 180, the reliability R22 of the second image information recognition unit 182 and the reliability R23 of the map information recognition unit 183 based on the internal and external states of the vehicle 16A. judge. The reliability R21, R22, R23 of the recognition units 180, 182, and 183 changes as the inside / outside state of the vehicle 16A changes. Examples of the state information indicating the state correlated with the reliability R21, R22, R23 of the recognition units 180, 182, and 183 include information regarding the distance from the vehicle 16A to the stop line 200, information indicating the curvature of the road, Information indicating the weather, information indicating the current time, information indicating the freshness of stop line information (information indicating the road environment), and the like. The reliability determination unit 84 obtains a state parameter P2 by digitizing one or more types of state information based on a predetermined rule. Then, the reliability R21, R22, R23 is determined using a map (table) M2 that associates the state parameter P2 with the reliability R21, R22, R23. The map M2 and the predetermined rules for digitization are stored in the storage device 52A.

  The weighting coefficient setting unit 186 includes a weighting coefficient C21 for the recognition result of the first image information recognition unit 180, a weighting coefficient C22 for the recognition result of the second image information recognition unit 182 and a weighting coefficient for the recognition result of the map information recognition unit 183. C23 is set based on the reliability R21, R22, R23. As a specific method for setting the weighting coefficients C21, C22, and C23, the methods 1 and 2 described in the above [2-4] or other methods can be used.

  The stop line information estimation unit 188 uses the recognition results of the first image information recognition unit 180, the second image information recognition unit 182 and the map information recognition unit 183 and the weighting coefficients C21, C22, and C23 to integrate integrated stop line information (integrated road). Environment). The integrated stop line information refers to stop line information that the first image information recognition unit 180 recognizes through texture extraction of image information, stop line information that the second image information recognition unit 182 recognizes through edge extraction of image information, The stop line information recognized by the map information recognition unit 183 based on the map information is integrated according to the weighting factors C21, C22, and C23.

  Returning to FIG. 7, the description of the external recognition unit 160 will be continued. The other recognition unit 174 is external information other than the stop line 200 based on at least one of the image information of the camera 38, the detection information of the radar 40, the detection information of the LIDAR 42, the map information of the navigation device 44, and the communication information of the communication unit 46. Recognize the position and behavior.

[3-3 Processes Executed in Second Embodiment]
The process of 2nd Embodiment is demonstrated using FIG. A series of processes described below is mainly executed by the outside recognition unit 60A and is repeatedly executed at predetermined time intervals.

  In step S11, it is determined whether or not automatic driving is in progress. If traveling control by automatic driving is being performed (step S11: YES), the process proceeds to step S12. On the other hand, when the traveling control by the automatic driving is not performed (step S11: NO), the processing is temporarily ended without executing the processing after step S12.

  In step S12, the first image information recognition unit 180 performs texture extraction based on the latest image information captured by the camera 38, and recognizes stop line information. The second image information recognition unit 182 performs edge extraction based on the latest image information taken by the camera 38 and recognizes stop line information. Further, the map information recognition unit 183 recognizes stop line information ahead of the current traveling position based on the map information stored in the storage unit 44 a of the navigation device 44.

  In step S13, the presence or absence of the stop line 200 is determined. As a result of the recognition performed in step S12, when the stop line 200 is within the predetermined range in front of the vehicle 16A (step S13: YES), the process proceeds to step S14. On the other hand, when there is no stop line 200 in the predetermined range ahead of the vehicle 16A (step S13: NO), the process is temporarily terminated without executing the processes after step S14.

  In step S14, it is determined whether or not it is necessary to stop at the stop line 200. The other recognizing unit 174 recognizes the attribute of the stop line 200 (whether it is a temporary stop point) based on the map information. Further, the presence / absence of a traffic signal on the stop line 200 and the traffic signal instruction are recognized based on the information acquired from the information providing server 12 via the communication unit 46. Then, it is determined whether or not it is necessary to stop when the vehicle 16A reaches the stop line 200. If it is necessary to stop (step S14: YES), the process proceeds to step S15. On the other hand, when it is not necessary to stop (step S14: NO), the processing is temporarily terminated without executing the processing after step S15.

  In step S15, the reliability determination unit 184 obtains the state parameter P2. As described above, the reliability determination unit 184 acquires one or a plurality of state information and obtains the state parameter P2 based on a predetermined rule.

  In step S16, the reliability determination unit 184 determines the reliability R21 of the first image information recognition unit 180 based on the state parameter P2 and the map M2. Further, the reliability R22 of the second image information recognition unit 182 is determined based on the state parameter P2 and the map M2. Further, the reliability R23 of the map information recognition unit 183 is determined based on the state parameter P3 and the map M2. As an example, it is assumed that the determination process of the reliability R21, R22, R23 is performed based on the state parameter P2 corresponding to the information related to the distance from the vehicle 16A to the stop line 200. As shown in FIG. 9, the reliability R21 of the first image information recognition unit 180 is relatively high in the range of a short distance (less than L2) from the stop line 200, and the distance from the stop line 200 becomes longer than L2. Lower. In addition, the reliability R22 of the second image information recognition unit 182 is relatively high in the range of the intermediate distance (L2 or more and less than L1) from the stop line 200, and as the distance from the stop line 200 becomes shorter than L2, or As the distance from the stop line 200 becomes longer than L1, the distance decreases. In addition, the reliability R23 of the map information recognition unit 183 is relatively high in the range of a long distance (L1 or more) from the stop line 200, and decreases as the distance from the stop line 200 becomes shorter than L1.

  In step S17, the weighting coefficient setting unit 186 sets the weighting coefficients C21, C22, and C23 according to the reliability levels R21, R22, and R23 by the methods 1 and 2 described in [2-4] above or other methods. In step S18, the stop line information estimation unit 188 estimates the integrated stop line information by the methods 1 and 2 described in [2-4] above or other methods.

[3-4 Summary of Second Embodiment]
The vehicle control device 26A according to the second embodiment relates to a device that recognizes stop line information and controls automatic traveling that does not rely on a driver for at least part of vehicle traveling. The vehicle control device 26A includes different types of recognition units 180, 182, and 183 that recognize stop line information based on input information, and individual recognition units 180 and 182 based on state information indicating the internal and external states of the vehicle 16A. , 183, a weighting coefficient setting unit 186 that sets weighting coefficients C21, C22, and C23 for the recognition result, and a plurality of recognition results and weighting coefficients C21, C22, and C23 for the recognition result, and a stop line 200 as an integrated road environment Is provided with a stop line information estimation unit 188 (road environment estimation unit).

  According to the second embodiment, the recognition results of the individual recognition units 180, 182, and 183, that is, the weighting coefficients C21, C22, and C23 for the stop line information are set based on the state information, and a plurality of recognition results and each recognition result are set. The stop line information as the integrated road environment is estimated using the weighting coefficients C21, C22, and C23 corresponding to. In this way, by changing the weighting coefficients C21, C22, and C23 and using the changed weighting coefficients C21, C22, and C23 to integrate the respective recognition results, the stop line information obtained as a final recognition result is used. This eliminates the need to clearly switch between a plurality of recognition processes. For this reason, when the recognition process is switched, the phenomenon that the recognition accuracy of the stop line information is reduced does not occur, and the stop line information can be recognized.

  The vehicle control device 26 includes a reliability determination unit 184 that individually determines the reliability R21, R22, and R23 of the plurality of recognition units 180, 182, and 183 based on the state. One of the recognition units 180, 182, and 183 is a first image information recognition unit 180 or a second image information recognition unit 182 that recognizes stop line information based on image information captured by the camera 38. The weighting coefficient setting unit 186 changes the weighting coefficient C21 for the recognition result of the first image information recognition unit 180 as the reliability R21 of the first image information recognition unit 180 decreases. Further, as the reliability R22 of the second image information recognition unit 182 becomes lower, the weighting coefficient C22 for the recognition result of the second image information recognition unit 182 is changed. According to the above configuration, the stop line information (integrated road environment) can be accurately recognized because the weighting coefficient is appropriately changed.

  In addition, the vehicle control apparatus according to the present invention is not limited to the above-described embodiment, and it is needless to say that various configurations can be adopted without departing from the gist of the present invention.

16, 16A ... Vehicle 26, 26A ... Vehicle control device 38 ... Camera 80 ... Image information recognition unit (recognition unit)
82, 183 ... Map information recognition unit (recognition unit)
84: Reliability determination unit 86, 186 ... Weighting coefficient setting unit 88 ... Lane information estimation unit (road environment estimation unit)
90 ... Lane mark 90 '... Virtual lane mark 92' ... Virtual lane 94 ... Intersection
180 ... 1st image information recognition part (recognition part)
182 ... Second image information recognition unit (recognition unit)
188 ... Stop line information estimation unit (road environment estimation unit)
200 ... Stop line

Claims (8)

A vehicle control device for recognizing a road environment and controlling automatic driving without relying on a driver for at least a part of vehicle driving,
A plurality of different recognition units for recognizing the road environment based on input information;
A weighting coefficient setting unit that sets a weighting coefficient for a recognition result of each of the recognition units based on state information indicating a state inside and outside the vehicle;
A vehicle control device comprising: a road environment estimation unit that estimates an integrated road environment using a plurality of the recognition results and the weighting coefficient for the recognition results. The vehicle control device according to claim 1,
A reliability determination unit that individually determines the reliability of the plurality of recognition units based on the state information,
The road environment is lane information,
One of the recognition units is an image information recognition unit that recognizes the lane information based on image information captured by a camera.
The weighting coefficient setting unit changes the weighting coefficient for the recognition result of the image information recognition unit as the reliability of the image information recognition unit decreases. The vehicle control device according to claim 2,
The vehicle control apparatus according to claim 1, wherein the state in which the reliability of the image information recognition unit is low is a situation in which the vehicle travels an intersection. In the vehicle control device according to claim 2 or 3,
The said image information recognition part produces | generates a virtual lane using the said previous image information in the said state where the said reliability becomes low. The vehicle control apparatus characterized by the above-mentioned. In the vehicle control device according to claim 2 or 3,
The vehicle information control apparatus, wherein the image information recognition unit generates a virtual lane using the lane information that can be recognized based on the image information in the state where the reliability is low. The vehicle control device according to any one of claims 2 to 5,
One of the recognition units is a map information recognition unit that recognizes the lane information based on map information,
The road environment estimation unit is configured to recognize the weighting coefficient for the recognition result of the image information recognition unit, the weighting coefficient for the recognition result of the map information recognition unit, and the image information recognition unit to recognize based on the image information. The vehicle control device, wherein the lane information as the integrated road environment is estimated based on lane information and the lane information recognized by the map information recognition unit based on the map information. The vehicle control device according to claim 1,
A reliability determination unit that individually determines the reliability of the plurality of recognition units based on the state,
The road environment is stop line information,
One of the recognition units is an image information recognition unit that recognizes the stop line information based on image information captured by a camera,
The weighting coefficient setting unit changes the weighting coefficient for the recognition result of the image information recognition unit as the reliability of the image information recognition unit decreases. The vehicle control device according to any one of claims 2 to 7,
The state information includes information related to the recognition timing of the road environment by the recognition unit,
The reliability determination unit, when the latest recognition timing of any of the recognition units is closer to the determination timing of the reliability than the latest recognition timing of the other recognition units, A vehicle control device characterized in that the reliability is higher than the reliability of the other recognition units.

Images