JP2023142006A - Vehicle control device, vehicle control method and program - Google Patents

Abstract

To improve a specification rate of a travel lane of a vehicle.SOLUTION: A vehicle control device according to an embodiment comprises: a recognition unit which recognizes a peripheral situation of a vehicle; a driving control unit which controls one or both of steering and speed of the vehicle on the basis of the peripheral situation recognized by the recognition unit; an acquisition unit which acquires map information including lane information of the periphery of the vehicle and reference information for specifying the position of the vehicle; and a specification unit which specifies a travel lane of the vehicle on the basis of information on a road on which the vehicle travels acquired from the map information on the basis of the reference information and the type of a section line that divides the one or more lanes in the periphery of the vehicle recognized by the recognition unit. The specification unit specifies the travel lane of the vehicle without using at least information on the type for the specific road section line in the section lines recognized by the recognition unit.SELECTED DRAWING: Figure 2

Description

The present invention relates to a vehicle control device, a vehicle control method, and a program.

In recent years, research has been progressing on autonomous driving, which recognizes the surrounding conditions of a vehicle and automatically controls the driving of the vehicle. In this regard, the state of road display lines is recognized to detect information about the driving environment, and the marking lines that separate the current driving lane are estimated and controlled based on the line types stored in advance in the storage unit. There are techniques for determining the content and determining the position of the vehicle using camera recognition results and map information (for example, see Patent Documents 1 to 4).

Japanese Patent Application Publication No. 2000-105898 Japanese Patent Application Publication No. 2010-221859 Japanese Patent Application Publication No. 10-300494 JP2013-032953A

However, in the conventional technology, when there are predetermined road division lines that are difficult to recognize from images originally captured by a camera, the lane in which the vehicle is traveling may not be identified or may be identified incorrectly.

Aspects of the present invention have been made in consideration of such circumstances, and an object thereof is to provide a vehicle control device, a vehicle control method, and a program that can improve the identification rate of the driving lane of a vehicle. Make it one.

A vehicle control device, a vehicle control method, and a program according to the present invention employ the following configuration.
(1): A vehicle control device according to one aspect of the present invention includes a recognition unit that recognizes a surrounding situation of a vehicle, and one or both of steering and speed of the vehicle based on the surrounding situation recognized by the recognition unit. an acquisition unit that acquires map information including lane information around the vehicle and reference information for specifying the position of the vehicle; Identifying the driving lane of the vehicle based on the acquired information about the road on which the vehicle travels and the type of marking line that separates each of one or more lanes around the vehicle recognized by the recognition unit. a identifying unit, the identifying unit identifies the driving lane of the vehicle without using at least the type information for a specific road marking line among the marking lines recognized by the recognizing unit. It is a control device.

(2): In the aspect of (1) above, the type of the specific road marking line is a deceleration broken line at least on either the left or right side of the vehicle.

(3): In the aspect of (1) or (2) above, the identification unit at least The driving lane of the vehicle is specified based on the type of the other lane marking.

(4): In the aspect of (1) or (2) above, when there is a possibility that the type of one of the left and right marking lines of the vehicle is a deceleration broken line, The lane in which the vehicle is traveling is specified based on the road shoulder markings outside the road in which the vehicle is traveling.

(5): In any one of the aspects (1) to (4) above, the identification unit is configured such that the type of one or both of the left and right road marking lines of the vehicle is a deceleration broken line. If there is a possibility, the lane in which the vehicle is traveling is specified based on solid lines other than road marking lines on the left and right sides of the vehicle.

(6): In any one of the aspects (1) to (5) above, the driving control unit is configured to control at least a first driving mode and a task that is more imposed on the occupant of the vehicle than in the first driving mode. The vehicle is caused to run in one of a plurality of driving modes including a second driving mode in which the driving mode is severe, and when the identifying section identifies a driving lane of the vehicle, the first driving mode is executed. , when the driving lane of the vehicle is not specified by the specifying section, the second driving mode is executed.

(7): In any one of the aspects (1) to (5) above, the driving control unit selects one of a plurality of driving modes in which tasks imposed on the occupant of the vehicle differ. When the driving mode is executed and the vehicle is driven, and the driving lane of the vehicle is not specified by the identifying section, the driving mode that is being executed is continued and the vehicle is driven.

(8): The vehicle control method according to one aspect of the present invention provides a driving operation in which a computer recognizes a surrounding situation of a vehicle and controls one or both of the steering and speed of the vehicle based on the recognized surrounding situation. The control is executed to acquire map information including lane information around the vehicle and reference information for specifying the position of the vehicle, and the vehicle acquired from the map information is driven based on the reference information. The driving lane of the vehicle is specified based on the information on the road to be recognized and the type of lane markings that separate each of one or more lanes around the recognized vehicle, and the lane in which the vehicle is traveling is identified. This vehicle control method specifies the driving lane of the vehicle without using at least the above-mentioned types of information regarding the marking lines.

(9): The program according to one aspect of the present invention causes a computer to recognize a surrounding situation of a vehicle, and controls driving control for controlling one or both of the steering and speed of the vehicle based on the recognized surrounding situation. is executed to obtain map information including lane information around the vehicle and reference information for specifying the position of the vehicle, and the vehicle obtained from the map information runs based on the reference information. The driving lane of the vehicle is identified based on the road information and the type of lane markings that separate each of one or more lanes around the recognized vehicle, and the lane in which the vehicle is traveling is identified. This program specifies the driving lane of the vehicle without using at least the above types of information regarding road marking lines.

According to the aspects (1) to (9) above, it is possible to improve the identification rate of the vehicle driving lane.

1 is a configuration diagram of a vehicle system 1 using a vehicle control device according to an embodiment. FIG. 2 is a functional configuration diagram of a first control section 120 and a second control section 160 according to the embodiment. FIG. 3 is a diagram illustrating an example of the relationship between a driving mode, a control state of a vehicle M, and a task. 18 is a diagram showing an example of the contents of a determination table 182. FIG. It is a figure which shows an example of a 1st line type pattern. It is a figure which shows an example of a 2nd line type pattern. It is a figure which shows an example of a 3rd line type pattern. It is a figure which shows an example of a 4th line type pattern. It is a figure which shows an example of a 5th line type pattern. It is a figure which shows an example of a 6th line type pattern. It is a flowchart which shows an example of the flow of operation control processing performed by automatic operation control device 100.

DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments of a vehicle control device, a vehicle control method, and a program of the present invention will be described with reference to the drawings. In the following, a case will be explained in which the left-hand driving regulations are applied, but if the right-hand driving regulations are applied, left and right can be read in reverse.

[overall structure]
FIG. 1 is a configuration diagram of a vehicle system 1 using a vehicle control device according to an embodiment. The vehicle on which the vehicle system 1 is installed (hereinafter referred to as vehicle M) is, for example, a two-wheeled, three-wheeled, four-wheeled vehicle, etc., and its driving source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or any of these. It's a combination. The electric motor operates using electric power generated by a generator connected to an internal combustion engine, or electric power discharged from a secondary battery or a fuel cell. Moreover, below, as an example, an embodiment in which a vehicle control device is applied to an automatic driving vehicle will be described. Automatic driving means, for example, automatically controlling one or both of the steering and speed of the vehicle M to perform driving control. The driving control of the vehicle M may include various driving supports such as ACC (Adaptive Cruise Control), ALC (Auto Lane Changing), and LKAS (Lane Keeping Assistance System), for example. The operation of an autonomous vehicle may be partially or entirely controlled by manual operation by a passenger (driver).

The vehicle system 1 includes, for example, a camera (an example of an imaging unit) 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14, an object recognition device 16, a communication device 20, and an HMI (Human Machine Interface) 30. , a vehicle sensor 40 , a navigation device 50 , an MPU (Map Positioning Unit) 60 , a driver monitor camera 70 , a driving operator 80 , an automatic driving control device 100 , a driving force output device 200 , and a brake device 210 and a steering device 220. These devices and devices are connected to each other via multiplex communication lines such as CAN (Controller Area Network) communication lines, serial communication lines, wireless communication networks, and the like. Note that the configuration shown in FIG. 1 is just an example, and a part of the configuration may be omitted, or another configuration may be added. A combination of the camera 10, the radar device 12, and the LIDAR 14 is an example of the "external sensor ES." The external sensor ES may include another detection unit (for example, sonar) that recognizes the surrounding situation of the vehicle, and may also include the object recognition device 16. The HMI 30 is an example of an "output device." The automatic driving control device 100 is an example of a "vehicle control device."

The camera 10 is, for example, a digital camera that uses a solid-state imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). Camera 10 is attached to any location on vehicle M. For example, when capturing an image of the front of the vehicle M, the camera 10 is attached to the top of the front windshield, the back surface of the room mirror, or the like. Further, when capturing an image of the rear of the vehicle M, the camera 10 is attached to the upper part of the rear windshield, the back door, or the like. Further, when capturing images of the side and rear sides of the vehicle M, the camera 10 is attached to a door mirror or the like. For example, the camera 10 periodically and repeatedly images the surroundings of the vehicle M. Camera 10 may be a stereo camera. Further, the camera 10 may be provided with a plurality of cameras (for example, a first camera and a second camera), and the plurality of cameras may take images in the same direction, and normally the first camera takes images, If a predetermined condition is satisfied, the second camera or both the first camera and the second camera may be used to capture an image. The predetermined condition is, for example, a road marking line (hereinafter referred to as a marking line) that demarcates a lane included in the road on which the vehicle M travels based on an image captured by a camera (hereinafter referred to as a camera image). This is a case of recognizing. The lane markings may include, for example, line segment information other than lane markings that separate lanes.

The radar device 12 emits radio waves such as millimeter waves around the vehicle M, and detects radio waves (reflected waves) reflected by an object to detect at least the position (distance and direction) of the object. The radar device 12 is attached to an arbitrary location on the vehicle M. The radar device 12 may detect the position and velocity of an object using an FM-CW (Frequency Modulated Continuous Wave) method.

The LIDAR 14 irradiates light (or electromagnetic waves with a wavelength close to light) around the vehicle M and measures scattered light. The LIDAR 14 detects the distance to the target based on the time from light emission to light reception. The irradiated light is, for example, pulsed laser light. LIDAR 14 is attached to any location on vehicle M.

The object recognition device 16 performs sensor fusion processing on detection results from some or all of the camera 10, radar device 12, and LIDAR 14 included in the external sensor ES, and recognizes the position, type, speed, etc. of the object. do. The object recognition device 16 outputs the recognition result to the automatic driving control device 100. The object recognition device 16 may output the detection results of the camera 10, radar device 12, and LIDAR 14 as they are to the automatic driving control device 100. The object recognition device 16 may be omitted from the vehicle system 1.

The communication device 20 communicates with other vehicles existing around the vehicle M using, for example, a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), or a wireless base. Communicate with various server devices via the station.

The HMI 30 presents various information to the occupant of the vehicle M under the control of the HMI control unit 170, and also receives input operations from the occupant. The HMI 30 includes, for example, various display devices, speakers, switches, microphones, buzzers, touch panels, keys, and the like. Various display devices include, for example, LCD (Liquid Crystal Display) and organic EL (Electro Luminescence) display devices. The display device is provided, for example, near the front of the driver's seat (the seat closest to the steering wheel) on the instrument panel, and is installed at a position where the passenger can see it through a gap between the steering wheels or through the steering wheel. Further, the display device may be installed at the center of the instrument panel. Further, the display device may be a HUD (Head Up Display). The HUD projects an image onto a portion of the front windshield in front of the driver's seat, thereby making the virtual image visible to the eyes of a passenger seated in the driver's seat. The display device displays images generated by an HMI control unit 170, which will be described later. Further, the HMI 30 may include an operation changeover switch and the like that mutually switches between automatic operation and manual operation by an occupant. The switch includes, for example, a blinker switch (direction indicator) 32. The turn signal switch 32 is provided, for example, on a steering column or a steering wheel. The blinker switch 32 is an example of an operation unit that receives an instruction from a passenger to change lanes of the vehicle M, for example.

The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the vehicle M, an acceleration sensor that detects the acceleration, a yaw rate sensor that detects the angular velocity around the vertical axis, a direction sensor that detects the direction of the vehicle M, and the like. Further, the vehicle sensor 40 may include a steering angle sensor that detects the steering angle of the vehicle M (the angle of a steering wheel or the operating angle of a steering wheel). Further, the vehicle sensor 40 may include a position sensor that acquires the position of the vehicle M. The position sensor is, for example, a sensor that acquires position information (longitude/latitude information) from a GPS (Global Positioning System) device. Further, the position sensor may be a sensor that acquires position information using a GNSS (Global Navigation Satellite System) receiver 51 of the navigation device 50.

The navigation device 50 includes, for example, a GNSS receiver 51, a navigation HMI 52, and a route determining unit 53. The navigation device 50 holds first map information 54 in a storage device such as an HDD (Hard Disk Drive) or a flash memory. GNSS receiver 51 identifies the position of vehicle M based on signals received from GNSS satellites. The position of the vehicle M may be specified or complemented by an INS (Inertial Navigation System) using the output of the vehicle sensor 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, keys, and the like. The navigation HMI 52 may be partially or completely shared with the aforementioned HMI 30. For example, the route determining unit 53 determines a route (hereinafter referred to as a map) from the position of the vehicle M specified by the GNSS receiver 51 (or any input position) to the destination input by the occupant using the navigation HMI 52. upper route) is determined with reference to the first map information 54.

The first map information 54 includes, for example, information in which lane information is added to each road (hereinafter referred to as lane information). The lane information includes, for example, nodes indicating the start and end of a road section, and links expressing the road shape between the nodes. In addition, lane information includes the number of lanes (number of parallel lanes), the number of increases/decrements, and the direction of lane increase/decrease (information indicating whether the lanes increase or decrease on the left or right with respect to the direction of travel of the road) in a predetermined section such as a road unit. may be included. Further, the lane information may include, for example, information regarding the type of marking line (for example, solid line, broken line, deceleration broken line, road shoulder line). The deceleration broken line is a road surface marking that makes the width of the road appear narrower to the driver of the vehicle, for example, in a road section where there are many collisions between vehicles. The deceleration broken line is provided, for example, at a downhill section of a road, a curved section, an intersection with poor visibility, etc., along the shape of a lane boundary line, outside line, or center line. By making the vehicle width appear narrower to the driver by using the deceleration broken line, an effect of decelerating the vehicle M during manual driving can be expected. Further, the first map information 54 may include distances and curvatures of road sections, road types (for example, expressways, general roads), POI (Point Of Interest) information, and the like. The route on the map is output to the MPU 60.

The navigation device 50 may provide route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may be realized, for example, by the functions of a terminal device such as a smartphone or a tablet terminal owned by a passenger. The navigation device 50 may transmit the current position and destination to the navigation server via the communication device 20, and obtain a route equivalent to the route on the map from the navigation server. Note that the first map information 54 may be stored in the storage unit 180 instead of the navigation device 50.

The MPU 60 includes, for example, a recommended lane determining section 61. The recommended lane determining unit 61 divides the route on the map provided from the navigation device 50 into a plurality of blocks (for example, divided into blocks of 100 [m] with respect to the vehicle traveling direction), and divides the route on the map provided by the navigation device 50 into blocks (for example, divided into blocks of 100 [m] in the direction of vehicle travel), and calculates the first map information 54 for each block. Determine recommended lanes from lane information. Further, the recommended lane determining unit 61 may determine recommended lanes for each road stored in the first map information 54. For example, the recommended lane determining unit 61 determines which lane from the left (or right) the vehicle should drive. When there is a branch point on the route on the map, the recommended lane determining unit 61 determines a recommended lane so that the vehicle M can travel on a reasonable route to the branch destination.

The driver monitor camera 70 is, for example, a digital camera using a solid-state imaging device such as a CCD or CMOS. The driver monitor camera 70 can be mounted at any location in the vehicle M, for example, at a position and orientation where it can image the head of an occupant (hereinafter referred to as the driver) seated in the driver's seat of the vehicle M from the front (in a direction that images the face). Can be installed at any location. For example, the driver monitor camera 70 is attached to the top of a display device provided in the center of the instrument panel of the vehicle M.

The driving controls 80 include, for example, the steering wheel 82, an accelerator pedal, a brake pedal, a shift lever, and other controls. A sensor is attached to the driving operator 80 to detect the amount of operation or the presence or absence of the operation, and the detection result is sent to the automatic driving control device 100, the driving force output device 200, the brake device 210, and the steering device. 220 is output to some or all of them. The steering wheel 82 is an example of an "operator that accepts a steering operation by the driver." The operator does not necessarily have to be annular, and may be in the form of a deformed steering wheel, a joystick, a button, or the like. A steering wheel grip sensor 84 is attached to the steering wheel 82 . The steering wheel grip sensor 84 is realized by a capacitance sensor or the like, and automatically sends a signal that can detect whether or not the driver is gripping the steering wheel 82 (meaning that the driver is in contact with the steering wheel 82 in a state where force can be applied to it). Output to the operation control device 100.

The automatic driving control device 100 includes, for example, a first control section 120, a second control section 160, an HMI control section 170, and a storage section 180. The first control unit 120, the second control unit 160, and the HMI control unit 170 are each realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). In addition, some or all of these components are hardware (circuits) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), and GPU (Graphics Processing Unit). (including circuitry), or may be realized by collaboration between software and hardware. The program may be stored in advance in a storage device such as the HDD or flash memory (storage device equipped with a non-transitory storage medium) of the automatic driving control device 100, or may be stored in a removable device such as a DVD or CD-ROM. The information may be stored in a storage medium, and may be installed in the HDD or flash memory of the automatic driving control device 100 by attaching the storage medium (non-transitory storage medium) to a drive device. A combination of the action plan generation section 140 and the second control section 160 is an example of the "operation control section." The HMI control section 170 is an example of an "output control section."

The storage unit 180 may be realized by the various storage devices described above, or an SSD (Solid State Drive), an EEPROM (Electrically Erasable Programmable Read Only Memory), a ROM (Read Only Memory), or a RAM (Random Access Memory). . The storage unit 180 stores, for example, a determination table 182, programs, and other various information. The determination table 182 is a table that is referred to, for example, to specify the driving lane of the vehicle M. The determination table 182 will be described later. Further, the first map information 54 may be stored in the storage unit 180.

FIG. 2 is a functional configuration diagram of the first control section 120 and the second control section 160 according to the embodiment. The first control unit 120 includes, for example, a recognition unit 130, an action plan generation unit 140, and a mode determination unit 150. The first control unit 120 realizes, for example, a function based on AI (Artificial Intelligence) and a function based on a model given in advance in parallel. For example, in the "recognize intersections" function, intersection recognition using deep learning, etc., and recognition based on pre-given conditions (pattern-matchable signals, road markings, etc.) are executed in parallel. This may be achieved by scoring and comprehensively evaluating. This ensures the reliability of automated driving.

The recognition unit 130 recognizes the position, speed, acceleration, and other states of objects around the vehicle M based on information input from the external sensor ES. The position of the object is recognized, for example, as a position on absolute coordinates with a representative point (center of gravity, drive shaft center, etc.) of the vehicle M as the origin, and is used for control. The position of an object may be expressed by a representative point such as the center of gravity or a corner of the object, or may be expressed by an area. The "state" of an object may include the acceleration or jerk of the object, or the "behavioral state" (eg, whether it is changing lanes or is about to change lanes).

Further, the recognition unit 130 recognizes, for example, the lane in which the vehicle M is traveling (driving lane). For example, the recognition unit 130 recognizes the left and right marking lines of the vehicle M from the camera image captured by the camera 10, and recognizes the driving lane based on the positions of the recognized marking lines. Note that the recognition unit 130 recognizes not only lane markings but also road shoulders, curbs, median strips, guardrails, fences, walls, and other targets (road boundaries, road boundaries) that can identify lane positions. It may also recognize the driving lane. In this recognition, the position of the vehicle M acquired from the navigation device 50 and the processing results by the INS may be taken into consideration. Additionally, the recognition unit 130 may recognize stop lines, obstacles, red lights, toll booths, and other road events.

When recognizing the driving lane, the recognition unit 130 recognizes the position and attitude of the vehicle M with respect to the driving lane. For example, the recognition unit 130 determines the deviation of the reference point of the vehicle M from the center of the lane and the angle made with respect to a line connecting the center of the lane in the traveling direction of the vehicle M as the relative position and attitude of the vehicle M with respect to the traveling lane. May be recognized. Alternatively, the recognition unit 130 may recognize the position of the reference point of the vehicle M with respect to any side edge (demarcation line or road boundary) of the driving lane as the relative position of the vehicle M with respect to the driving lane. good. Note that the recognition of the driving lane by the recognition unit 130 and the recognition of the position and attitude of the vehicle M with respect to the driving lane may be executed by the identification unit 153, which will be described later.

In principle, the action plan generation unit 140 drives the vehicle M in the recommended lane determined by the recommended lane determination unit 61, and furthermore, the action plan generation unit 140 automatically (driver's operation) Generates a target trajectory to be traveled in the future (independent of the current situation). The target trajectory includes, for example, a velocity element. For example, the target trajectory is expressed as a sequence of points (trajectory points) that the vehicle M should reach. A trajectory point is a point that the vehicle M should reach every predetermined traveling distance along the road (for example, about several [m]), and apart from that, a trajectory point is a point that the vehicle M should reach every predetermined distance traveled along the road (for example, about several [m]), and apart from that, it is a point that the vehicle M should reach at a predetermined sampling time (for example, about 0 tenths [sec]). A target velocity and acceleration for each are generated as part of the target trajectory. Further, the trajectory point may be a position to be reached by the vehicle M at each predetermined sampling time. In this case, information on target speed and target acceleration is expressed by intervals between trajectory points.

The action plan generation unit 140 may set automatic driving events (functions) when generating the target trajectory. Automated driving events include constant speed driving events, low speed following driving events, lane change events, branching events, merging events, takeover events, etc. The action plan generation unit 140 generates a target trajectory according to the activated event.

The mode determining unit 150 determines the driving mode of the vehicle M to be one of a plurality of driving modes in which the tasks imposed on the driver differ (in other words, a plurality of modes in which the degree of automation differs). The mode determining unit 150 includes, for example, a driver state determining unit 151, a first obtaining unit 152, a specifying unit 153, and a mode change processing unit 154. These individual functions will be described later. The first acquisition unit 152 is an example of an “acquisition unit”.

FIG. 3 is a diagram showing an example of the relationship between the driving mode, the control state of the vehicle M, and the task. In the example of FIG. 3, there are five driving modes of the vehicle M, for example, mode A to mode E. In FIG. 3, modes A and B are examples of "first operation modes," and modes C, D, and E are examples of "second operation modes." Note that the operating modes may include modes other than modes A to E, and may include operating modes other than the first operating mode and the second operating mode. From mode A to mode E, the control state, that is, the degree of automation (control degree) of driving control of vehicle M, is highest in mode A, then decreasing in order of mode B, mode C, and mode D, and lowest in mode E. . Conversely, the tasks imposed on the driver (occupant) are the lightest in mode A, followed by mode B, mode C, and mode D, which become more severe in that order, and mode E, in which manual driving is performed, is the most severe. Note that in modes B to E, the control state is not automatic driving, so the automatic driving control device 100 is responsible for ending control related to automatic driving and transitioning to driving support or manual driving. The contents of each mode will be illustrated below.

In mode A, the vehicle is in an automatic driving state, and the driver is not required to monitor the surroundings of the vehicle M or grip the steering wheel 82 (hereinafter referred to as "steering grip"). Surrounding monitoring includes at least monitoring of the direction in which the vehicle M is traveling (for example, forward). The front means a space in the direction of travel of the vehicle M that is visually recognized through the front windshield. However, even in mode A, the driver is required to be able to quickly shift to manual driving in response to a request from the system centered on the automatic driving control device 100. Note that the term "automatic driving" as used herein means that both the steering and the speed of the vehicle M are controlled without depending on the driver's operations. In mode A, for example, conditions such as vehicle M is traveling at a predetermined speed (for example, about 50 [km/h]) or less on a motorway such as an expressway, and there is a vehicle in front to be followed are present. This is a driving mode that can be executed when the conditions are met, and is sometimes referred to as TJP (Traffic Jam Pilot) mode. If this condition is no longer met, mode determining unit 150 changes the driving mode of vehicle M to mode B.

Furthermore, while mode A is being executed, the occupant can execute a second task. The second task is, for example, an act other than driving that is permitted by the occupant while the vehicle M is automatically driving. Second tasks include, for example, watching TV, using terminal devices (e.g., smartphones and tablets) owned by passengers (e.g., making calls, sending and receiving emails, using SNS (Social Networking Service), browsing the web, etc.), eating, etc. is included.

In mode B, the driver is in a driving support state, and the driver is tasked with monitoring the surroundings of the vehicle M (hereinafter referred to as surrounding monitoring), but is not tasked with gripping the steering wheel 82. For example, in mode B, the lane change of the vehicle M is performed based on the route setting to the destination by the navigation device 50, etc., based on the judgment of the vehicle system 1, without accepting a lane change instruction from the occupant. A lane change is to move the vehicle M from the own lane in which the vehicle M is traveling to an adjacent lane adjacent to the own lane, and may include a lane change based on branching or merging. The main driver in modes A and B is the vehicle system 1.

In mode C, the vehicle is in a driving support state, and the driver is tasked with monitoring the surroundings and gripping the steering wheel 82. For example, in mode C, when the vehicle system 1 determines that it is necessary for the vehicle M to change lanes, an inquiry is made to the occupant via the HMI 30, and approval for the lane change by the occupant is received from the HMI 30, etc. In this case, driving assistance is provided to perform a lane change. Lane change control in mode B and mode C is a system-based lane change.

Mode D is a driving mode that requires a certain amount of driving operation by the driver regarding at least one of the steering and acceleration/deceleration of the vehicle M. For example, in mode D, driving assistance such as ACC (Adaptive Cruise Control) and LKAS (Lane Keeping Assist System) is performed. Furthermore, in mode D, when an instruction to cause the vehicle M to change lanes by operating the blinker switch 32 by the driver is received, driving assistance is performed to execute the lane change in the instructed direction. The lane change in mode D is a lane change based on the driver's intention. The driver's operation of the blinker switch 32 is an example of a driving operation. Furthermore, the driving operations in mode D may include driving operations for controlling steering or acceleration/deceleration.

In mode E, the vehicle M enters a manual driving state in which both the steering and acceleration/deceleration of the vehicle M require driving operations by the driver. In both modes D and E, the driver is naturally required to monitor the surroundings of the vehicle M. The driving subject in modes C to E is the driver.

Mode determining unit 150 changes the driving mode of vehicle M to a driving mode in which the task is more severe when the driver does not perform a task related to the determined driving mode.

For example, if the driver is unable to shift to manual driving in response to a request from the system while running Mode A (for example, if the driver continues to look outside the permissible area, or if a sign that driving becomes difficult is detected. mode determination unit 150 causes the HMI control unit 170 to execute control using the HMI 30 to prompt the driver to shift to manual driving in mode E. In addition, the mode determining unit 150 causes the HMI control unit 170 to perform control to prompt the transition to manual driving, and if the driver does not respond even after a predetermined period of time has elapsed, or if the driver is not in a state to perform manual driving. If it is estimated, control is performed such as gradually decelerating the vehicle M while approaching the target position (for example, the shoulder of the road) and stopping automatic driving. Further, after stopping the automatic driving, the vehicle M enters the mode D or E, and it becomes possible to start the vehicle M by manual operation by the driver. The same applies to "stop automatic operation" below.

In mode B, if the driver is not monitoring the front, the mode determining unit 150 uses the HMI 30 to urge the driver to monitor the front, and if the driver does not comply, the mode determining unit 150 moves the vehicle M toward the target position and gradually moves the vehicle M toward the target position. control such as stopping the vehicle and stopping automatic operation. In mode C, if the driver is not monitoring the front or not gripping the steering wheel 82, the mode determining unit 150 uses the HMI 30 to have the driver monitor the front and/or grip the steering wheel 82. If the driver does not respond, the vehicle M is brought to a target position and gradually stopped, and automatic driving is stopped.

The driver condition determination unit 151 determines whether the occupant (driver) is in a condition suitable for driving. For example, the driver state determination unit 151 monitors the driver's state for the above mode change and determines whether the driver's state is appropriate for the task. For example, the driver condition determination unit 151 analyzes the image captured by the driver monitor camera 70 and performs posture estimation processing, and determines whether the driver is in a position that cannot shift to manual driving in response to a request from the system. Determine. In addition, the driver state determination unit 151 analyzes the image captured by the driver monitor camera 70 and performs line-of-sight estimation processing to determine whether the driver is monitoring the surroundings (more specifically, the front) of the vehicle M. Determine whether If it is determined that the driver is not in a state suitable for the task for a predetermined period of time or more, the driver state determining unit 151 determines that the driver is not in a state suitable for driving the task. Further, when it is determined that the driver is in a state suitable for the task, the driver state determining unit 151 determines that the driver is in a state suitable for driving the task. Further, the driver state determination unit 151 may determine whether or not the occupant is in a state where a driving change is possible.

The first acquisition unit 152 acquires the first map information 54. The first acquisition unit 152 also acquires reference information for identifying the position of the vehicle M. The reference information is, for example, position information of the vehicle M detected by the vehicle sensor 40 or a camera image captured by the camera 10. Further, the reference information may include part or all of the recognition result by the recognition unit 130.

The specifying unit 153 refers to the first map information 54 based on the position information of the vehicle M included in the reference information, and specifies the lane in which the vehicle M is traveling among the one or more lanes included in the road on which the vehicle M is traveling. do. Further, the identification unit 153 identifies the driving lane based on, for example, the road information acquired from the first map information 54 and the type of marking line recognized by the recognition unit 130. In addition, when at least one of the road information and the type of marking line includes a specific road marking line, the identifying unit 153 selects a specific road marking from among the recognition results around the vehicle M recognized by the recognizing unit 130. As for lines, at least the driving lane is specified without using type information.

The specific road marking line is, for example, line segment information that is drawn or formed on the road along the marking line that partitions the driving lane of the vehicle M, rather than the marking line that partitions the driving lane existing on the road. . Further, the specific road marking line is line segment information in which the type of marking line cannot be detected (recognized), but the existence of the line segment can be recognized. The specific road marking line is, for example, a deceleration broken line, but may be other line segment information. Depending on the recognition result of the recognition unit 130, the deceleration broken line may be mistakenly detected as a normal broken line or a solid line. Therefore, due to false detection, the number of road lanes obtained from the map information may differ from the number of lanes obtained from the camera image, or there is a possibility that the driving lane of the vehicle may be incorrectly specified. Therefore, when determining that there is a possibility that a specific road marking line exists based on the recognition pattern of a plurality of marking lines existing around the vehicle M, the identifying unit 153 provides information on the type of the specific road marking line. The lane in which vehicle M is traveling is identified as a lane in which there is a possibility that a marking line exists. Further, the identifying unit 153 may identify the driving lane of the vehicle M based on the line types of other lane markings excluding the specific road lane marking. In the following example, it is assumed that the specific road marking line is a deceleration broken line, and the lane in which the vehicle M is traveling is specified using marking lines other than the specific road marking line.

Further, the specifying unit 153 specifies the driving lane of the vehicle M at a predetermined timing. The predetermined timing may be, for example, a predetermined cycle, the start of automatic driving, the start of travel on a specific road such as an expressway, or the timing at which road sections are switched in the first map information 54. Further, the predetermined timing may be, for example, the timing when the number of lanes in the map information and the number of lanes obtained from the camera image become unmatched, the current driving lane is reset, and it becomes necessary to specify the driving lane again. The timing may be the timing when the driving lane of M has not been identified and there is no change in the road within a predetermined distance, or the timing may be the timing when the driver performs an operation to start automatic driving. Further, the predetermined timing may be, for example, a timing at which the number of lanes on the road on which the vehicle M travels increases or decreases or increases or decreases in the near future. The details of the function of the specifying unit 153 will be described later.

The mode change processing unit 154 determines the driving mode of the vehicle M based on the determination result of the driver condition determination unit 151, the determination result of the identification unit 153, and the like. Furthermore, the mode change processing unit 154 may decide to continue the current operating mode or to switch to another mode. Further, the mode change processing unit 154 performs various processes for changing to the driving mode determined by the mode determining unit 150. For example, the mode change processing unit 154 instructs a driving support device (not shown) to operate, causes the HMI control unit 170 to output information for prompting the driver to take action, or causes the HMI 30 to take action according to the driving mode. It instructs the plan generation unit 140 to generate a target trajectory based on the plan generation unit 140.

The second control unit 160 controls the driving force output device 200, the brake device 210, and the steering device 220 so that the vehicle M passes the target trajectory generated by the action plan generation unit 140 at the scheduled time. do.

The second control unit 160 includes, for example, a second acquisition unit 162, a speed control unit 164, and a steering control unit 166. The second acquisition unit 162 acquires information on the target trajectory (trajectory point) generated by the action plan generation unit 140, and stores it in a memory (not shown). The speed control unit 164 controls the travel driving force output device 200 or the brake device 210 based on the speed element associated with the target trajectory stored in the memory. The steering control unit 166 controls the steering device 220 according to the degree of curvature of the target trajectory stored in the memory. The processing of the speed control section 164 and the steering control section 166 is realized, for example, by a combination of feedforward control and feedback control. As an example, the steering control unit 166 performs a combination of feedforward control according to the curvature of the road in front of the vehicle M and feedback control based on the deviation from the target trajectory.

The HMI control unit 170 notifies the occupant of predetermined information via the HMI 30. The predetermined information includes, for example, information related to the running of the vehicle M, such as information regarding the state of the vehicle M and information regarding driving control. The information regarding the state of the vehicle M includes, for example, the speed of the vehicle M, the engine speed, the shift position, and the like. In addition, information related to driving control includes, for example, inquiries regarding whether to change lanes, information regarding whether or not to execute a driving mode, information regarding changing the driving mode, and information imposed on the occupant to switch the driving mode. (task request information for the occupant), information regarding the driving control status (for example, the content of the event being executed), and the like. Further, the predetermined information may include information that is not related to driving control of the vehicle M, such as a television program and content (for example, a movie) stored in a storage medium such as a DVD. Further, the predetermined information includes, for example, the current position and destination of the vehicle M, information regarding the remaining amount of fuel, information indicating whether the driving lane of the vehicle M has been identified, and the remaining distance until the driving mode is switched. , the direction of lane increase/decrease, the number of lanes to increase/decrease, the number of lanes running parallel to the driving lane (number of parallel lanes), etc.

For example, the HMI control unit 170 may generate an image including the above-mentioned predetermined information and display the generated image on the display device of the HMI 30, generate sound indicating the predetermined information, and display the generated sound on the HMI 30. It may also be output from a speaker. Further, the HMI control unit 170 may output information received by the HMI 30 to the communication device 20, the navigation device 50, the first control unit 120, etc.

The driving force output device 200 outputs driving force (torque) for driving the vehicle to the driving wheels. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, a transmission, and the like, and an ECU (Electronic Control Unit) that controls these. The ECU controls the above configuration according to information input from the second control unit 160 or information input from the driving operator 80.

The brake device 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the second control unit 160 or information input from the driving operator 80 so that brake torque corresponding to the braking operation is output to each wheel. The brake device 210 may include, as a backup mechanism, a mechanism that transmits hydraulic pressure generated by operating a brake pedal included in the driving operator 80 to a cylinder via a master cylinder. Note that the brake device 210 is not limited to the configuration described above, and may be an electronically controlled hydraulic brake device that controls an actuator according to information input from the second control unit 160 and transmits the hydraulic pressure of the master cylinder to the cylinder. Good too.

Steering device 220 includes, for example, a steering ECU and an electric motor. For example, the electric motor applies force to a rack and pinion mechanism to change the direction of the steered wheels. The steering ECU drives the electric motor in accordance with information input from the second control unit 160 or information input from the driving operator 80 to change the direction of the steered wheels.

[Functions of specific parts and details of operation control]
Hereinafter, the details of the function of the specifying section 153 and the details of the operation control based on the processing contents of the specifying section 153 will be explained. In the following description, it is assumed that the driver status determining unit 151 has determined that the driver is appropriately performing the task assigned according to the driving mode, and that the driving status is determined based on the processing content by the identifying unit 153. An example in which the mode is determined will be explained. Note that when the driver state determining unit 151 determines that the driver is not performing the task imposed according to the mode, the mode determining unit 150 Decide to change to a corresponding mode or decide to perform control to stop automatic operation.

First, the identifying unit 153 acquires, for example, information regarding the marking lines around the vehicle M recognized by the recognizing unit 130. To explain specifically, first, the recognition unit 130, for example, generates an image (camera The system analyzes the image), extracts edge points in the image that have a large difference in brightness from adjacent pixels, and connects the edge points to recognize targets that can identify marking lines and lane positions on the image plane. The recognition unit 130 also extracts image information from the image through feature extraction, image enhancement processing, etc., and performs matching by referring to the extracted image information and a predefined model for pattern matching. The lane markings and the above-mentioned targets may be recognized through processing. Further, the recognition unit 130 may recognize type information such as type (solid line, broken line) and color for each marking line based on the image analysis result, or may recognize the type of target object. The recognition unit 130 may also recognize the positional relationship between the recognized marking lines and targets, and the positional relationship (relative position) between the marking lines and targets and the vehicle M. Further, the recognition unit 130 may recognize characters displayed on a road sign or characters drawn on a road.

Note that the marking lines around the vehicle M recognized by the recognition unit 130 are, for example, the left and right marking lines closest to the vehicle M, and the left and right marking lines closest to the vehicle M after the left and right marking lines. Hereinafter, the closest marking line on the left side of the vehicle M will be referred to as the "first left marking line", and the next closest marking line (the marking line that is further back from the vehicle M than the first marking line on the left) will be referred to as the "first marking line on the left". It shall be referred to as the "second left partition line." In addition, the closest marking line on the right side of the vehicle M is referred to as the "first marking line on the right side," and the next closest marking line (the marking line that is further back from the first marking line on the right side when viewed from the vehicle M) is referred to as the "first marking line on the right side." It shall be referred to as the "second right partition line."

The identification unit 153 refers to the first map information 54 based on the position information included in the reference information acquired by the first acquisition unit 152, and acquires information on the road on which the vehicle M runs. The road information is, for example, information regarding the number of lanes. Further, the road information may include information regarding the number of lanes on one side, information indicating whether there is two-way traffic, and information indicating whether the road on which the vehicle M is traveling is a deceleration section. may be included.

Next, the identifying unit 153 determines whether the vehicle M Identify the driving lane. In addition, for example, when a specific road marking line that is difficult to recognize in a camera image is included, the identifying unit 153 sets a special rule (predetermined relaxed condition) without immediately determining that the driving lane of the vehicle M cannot be identified. It is used to identify the driving lane. For example, when the type of marking line recognized by the recognition unit 130 includes a marking line that may be a deceleration broken line, the specifying unit 153 determines whether the vehicle M Identify the driving lane.

More specifically, when it is determined that there is a possibility that the type of at least one of the left and right marking lines of the vehicle M is a deceleration broken line, for example, the identification unit 153 identifies the marking line determined to be a deceleration broken line. The lane in which the vehicle M is traveling is specified using the lane markings excluding the marking lines. Note that the deceleration broken line portion may be used to specify the position of the vehicle M with respect to the driving lane, assuming that some line segment exists.

For example, if there is a possibility that the type of one of the left and right lane markings of the vehicle M is a deceleration broken line, the identification unit 153 determines the driving lane of the vehicle M based on the type of the other lane marking. Identify. In addition, when there is a possibility that the type of one of the left and right marking lines of the vehicle M is a deceleration broken line, the identification unit 153 determines whether the vehicle M The driving lane of M may be specified. In addition, when there is a possibility that the type of one or both of the left and right lane marking lines of the vehicle M is a deceleration broken line, the identification unit 153 determines the type of lane markings based on solid lines other than the left and right road lane markings of the vehicle M. The lane in which the vehicle M is traveling may be specified.

In addition, when performing the above-mentioned identification, the identification unit 153 may refer to the determination table 182 stored in the storage unit 180 in advance, and identify the driving lane of the vehicle M when a corresponding condition is satisfied.

FIG. 4 is a diagram showing an example of the contents of the determination table 182. The determination table 182 is information in which determination conditions and lane marking conditions are associated with determination results. In the example of the determination table 182 shown in FIG. 4, a condition ID is also included as identification information for identifying each determination condition. The determination start condition is a condition for the specifying unit 153 to specify the driving lane. "Always" includes the meaning of a predetermined period. In addition, deceleration display detection means that the recognition unit recognizes that a deceleration display that encourages deceleration of the vehicle M is drawn on the road on which the vehicle M travels, or a road sign or signboard that prompts deceleration.

The marking line conditions include conditions for the line types of two marking lines on each of the left and right sides of the vehicle M. The "-" part included in the marking line condition in FIG. 4 indicates that the marking line can be of any line type (it does not matter, even if it is not detected). The determination result specifies the lane in which the vehicle M is traveling when the conditions shown in the determination start condition and the marking line condition are satisfied.

Below, the line types of the lane markings around the vehicle M and the identification of driving lanes based on the line types will be explained in several patterns.

<First line type pattern>
FIG. 5 is a diagram showing an example of the first line type pattern. In the example of FIG. 5, it is assumed that the vehicle M is traveling at a speed VM toward the extending direction of the road RD1 (X-axis direction in the figure). Regarding the example in FIG. 5, the lot line RL11 corresponds to the second left lot line, the lot line RL12 corresponds to the first left lot line, the lot line RL13 corresponds to the first right lot line, and the lot line RL14 corresponds to the right side lot line. It shall correspond to the second lot line. The same shall apply to subsequent drawings.

In the example of FIG. 5, it is assumed that the recognition unit 130 has not detected (recognized) the lane markings RL11 and RL14 (undetected), but has recognized that the lane markings RL12 and RL13 are solid lines. In this case, the specifying unit 153 specifies that the driving lane of the vehicle M is an end lane of a single lane (single lane) because the condition ID "C001" shown in FIG. 4 is satisfied. Further, in the first line type pattern, when the recognition unit 130 recognizes that at least one of the marking lines RL11 and RL is a road shoulder instead of not being detected, the identifying unit 153 similarly determines whether the vehicle M is traveling Identify the lane as the end lane of a single lane.

<Second line type pattern>
FIG. 6 is a diagram showing an example of the second line type pattern. In the example of FIG. 6, it is assumed that the recognition unit 130 recognizes that the marking line RL11 is a road shoulder (or roadside strip), the marking line RL12 is a solid line, and the marking line RL13 is a broken line. Further, in the second line type pattern, the marking line RL14 may be undetected, or some marking line may be detected. In this case, the specifying unit 153 specifies that the lane in which the vehicle M is traveling is the end lane (the leftmost lane) of the plurality of lanes, since the condition ID "C002" shown in FIG. 4 is satisfied.

<Third line type pattern>
FIG. 7 is a diagram showing an example of the third line type pattern. In the example of FIG. 7, the recognition unit 130 recognizes the drawing of the characters "Slow Down" on the road, and further recognizes that the marking line RL11 is the road shoulder, the marking line RL12 is a special line, and the marking line RL13 is a broken line. It is assumed that Further, in the third line type pattern, the marking line RL14 may be undetected, or some marking line may be detected. The special line is, for example, a line that cannot be recognized as a solid line or a broken line, and includes, for example, a deceleration broken line, a deceleration mark, a zebra zone, a line partially erased due to scratches, dirt, etc. In other words, the special line is a line that may be a deceleration broken line. In this case, in order to satisfy the condition ID "C003" shown in FIG. Identify the end lane (leftmost lane). Further, in the third line type pattern, when the marking line RL12 is a solid line or a broken line and the marking line RL13 is a special line, the identifying unit 153 similarly determines that the driving lane of the vehicle M is in the deceleration broken line section. It may be specified that it is an edge lane. Further, instead of recognizing the drawing of the characters "deceleration" on the road, it may be recognized from the first map information 54 that the road is in a deceleration section. As described above, in the case of the third line type pattern, when there is a possibility that one of the left and right marking lines RL12 and RL13 of the vehicle M is a deceleration broken line, The driving lane of vehicle M is specified based on the type of the other lane marking. In addition, in the case of the third line type pattern, as shown in FIG. Identify.

<Fourth line type pattern>
FIG. 8 is a diagram showing an example of the fourth line type pattern. In the example of FIG. 8, it is assumed that the recognition unit 130 recognizes that the marking line RL11 is a solid line, and that the marking lines RL12 and RL13 are broken lines. Further, in the fourth line type pattern, the marking line RL14 may be undetected, or some marking line may be detected. In this case, the specifying unit 153 specifies that the driving lane of the vehicle M is the second lane (second lane from the left end) of a road with three or more lanes on each side, in order to satisfy the condition ID “C004” shown in FIG. do. Note that in the fourth line type pattern, the specifying unit 153 refers to the first map information 54 based on the position information of the vehicle M, and sets the condition of "C004" when the road RD1 is a road with three or more lanes on each side. It may be determined that the following is satisfied.

<Fifth line type pattern>
FIG. 9 is a diagram showing an example of the fifth line type pattern. In the example of FIG. 9, it is assumed that the recognition unit 130 recognizes that the lane markings RL11 and RL14 are solid lines, and that the lane markings RL12 and RL13 are broken lines. In this case, the specifying unit 153 specifies that the driving lane of vehicle M is the second lane (second lane from the left end) of a road with two lanes on each side, in order to satisfy the condition ID "C005" shown in FIG. . In addition to the above conditions, the specifying unit 153 refers to the first map information 54 based on the position information of the vehicle M, and determines that the condition of "C005" is satisfied when the road RD1 is a road with two lanes on each side. You may.

<Sixth line type pattern>
FIG. 10 is a diagram showing an example of the sixth line type pattern. In the example of FIG. 10, the recognition unit 130 recognizes the drawing of the characters "Slow down" on the road, and further recognizes that the marking line RL11 is a solid line, and that the marking lines RL12 and RL13 are special lines. do. Further, in the fourth line type pattern, the marking line RL14 may be undetected, or some marking line may be detected. In this case, the specifying unit 153 specifies that the driving lane of the vehicle M is the second lane (second lane from the left end) of the deceleration broken line section, since the condition ID "C006" shown in FIG. 4 is satisfied. The specifying unit 153 refers to the first map information 54 based on the position information of the vehicle M in addition to the above-mentioned conditions, and determines that the condition of "C006" is satisfied when the road RD1 is a road in a deceleration section. Good too. Further, in the sixth line type pattern, when at least one of the lane markings RL12 and RL13 is recognized as a broken line, the identification unit 153 similarly determines that the driving lane of the vehicle M is in the deceleration broken line section. It may also be specified that it is the second lane (second lane from the left end). Further, instead of recognizing the drawing of the characters "deceleration" on the road, it may be recognized from the first map information 54 that the road is in a deceleration section. As described above, in the case of the sixth line type pattern, when there is a possibility that one or both of the left and right road marking lines of the vehicle M is of the type deceleration broken line, The driving lane of the vehicle M is specified based on the position of a solid line other than the road marking lines on the left and right of the vehicle M (in the example of FIG. 10, RL11).

In this way, by identifying the driving lane of vehicle M using the first to sixth line type patterns described above, for example, even if there is a marking line that is difficult to recognize from a camera image, it is possible to It is possible to accurately identify the driving lane. Therefore, even if there is a marking line that is difficult to recognize from the camera image, the first driving mode can be continued without immediately switching from the first driving mode to the second driving mode.

Note that if the above-mentioned first to sixth line type patterns (condition IDs "C001" to "C006") do not apply, the identification unit 153 determines whether the driving lane of the vehicle M is one or more lanes included in the road RD1. It is assumed that it is not possible to specify which lane the vehicle is traveling in.

The mode change processing unit 154 executes the first driving mode when the driving lane of the vehicle M is specified by the identifying unit 153, and causes the vehicle M to run without deviating from the specified driving lane, or Driving controls such as following the vehicle and changing lanes to drive towards the destination are executed. Further, the mode change processing unit 154 causes the second driving mode to be executed when the driving lane of the vehicle M is not specified by the specifying unit 153. Thereby, when it is not possible to specify which lane on the road the vehicle M is traveling on, automatic driving can be restricted and the vehicle M can be driven more safely.

Furthermore, if the identifying unit 153 does not specify the driving lane of the vehicle M, the mode change processing unit 154 may cause the vehicle M to continue traveling in the current driving mode. In this way, more stable driving can be achieved by maintaining the current driving mode without switching the driving mode in a situation where the driving lane of the vehicle M cannot be specified.

Note that when the driving lane of vehicle M cannot be specified, whether to switch to the second driving mode or continue the current driving mode (first driving mode or second driving mode) is determined by vehicle M, for example. The decision may be made based on the surrounding situation and road information obtained from the first map information 54. For example, if there are a predetermined number or more of other vehicles around vehicle M (within a predetermined distance), recognition unit 130 may not be able to recognize the marking line due to the influence of the other vehicles. Therefore, if there are more than a predetermined number of other vehicles around the vehicle M, the mode change processing unit 154 continues the current driving mode even if the driving lane of the vehicle M cannot be identified; In some cases, it switches to the second operation mode. Furthermore, the mode change processing unit 154 refers to the first map information 54, acquires information on roads around the vehicle M based on the position information of the vehicle M, and obtains information about the number of road lanes on the road within a predetermined distance from the vehicle M. If there is no increase or decrease or the curvature of the road is small (below the threshold), the current driving mode is continued, and if there is an increase or decrease in the number of lanes or the curvature of the road is large (greater than the threshold), the second driving mode is activated. You can also switch to the mode. Thereby, a more appropriate driving mode can be executed depending on the surrounding situation of the vehicle M.

Furthermore, when the identification unit 153 is unable to identify the driving lane of the vehicle M, the HMI control unit 170 may cause the HMI 30 to output that the driving lane of the vehicle M cannot be identified, and notify the occupant. Thereby, the situation of the vehicle M can be notified to the occupant, and it is possible to perform driving control according to the situation, such as having the occupant monitor the surroundings of the vehicle or switching to manual operation as necessary.

[Processing flow]
Next, the flow of processing executed by the automatic driving control device 100 of the embodiment will be explained. In addition, below, among the processes executed by the automatic driving control device 100, the process of specifying the driving lane of the vehicle M and the process of switching the driving mode based on the result of the specifying process will be mainly explained. Further, the process of this flowchart may be repeatedly executed at a predetermined timing, for example.

FIG. 11 is a flowchart showing an example of the flow of the driving control process executed by the automatic driving control device 100. In the example of FIG. 11, the recognition unit 130 recognizes the surrounding situation of the vehicle M (step S100). Next, the mode determining unit 150 causes the vehicle to run in one of a plurality of preset driving modes based on surrounding conditions and the like (step S102).

Next, the mode determining unit 150 acquires the first map information, the recognition result by the recognition unit 130, and reference information for specifying the position of the vehicle M (step S104), and based on the acquired information, Processing for identifying the driving lane is performed (step S106). For example, the specifying unit 153 of the mode determining unit 150 determines whether or not a specific road marking line is included in at least one of the road information and the marking line type among the acquired information (step S108). When it is determined that a specific road marking line is included, the identifying unit 153 adopts a special rule (predetermined relaxation condition) without immediately determining that the lane in which vehicle M is traveling cannot be specified, and at least includes the marking line. Identify the driving lane without using line type information. More specifically, the specifying unit 153 specifies the driving lane using the lane markings excluding the specific road marking line (step S110). Further, in the process of step S108, if it is determined that the specific road marking line is not included, the identifying unit 153 specifies the driving lane using the recognized marking line without adopting the special rule (step S112). ).

Next, the mode change processing unit 154 determines whether the driving mode of the vehicle M is the first driving mode (step S114). If it is determined that the first driving mode is being executed, the mode change processing section 154 determines whether the driving lane of the vehicle M has been specified by the specifying section 153 (step S116). If it is determined that the driving lane is specified, the mode change processing unit 154 continues the first driving mode based on the specified driving lane (step S118), and determines that the driving lane is not specified. If so, control is performed to switch from the first operation mode to the second operation mode (step S120). Thereby, the processing of this flowchart ends. Further, in the process of step S114, if it is determined that the first operation mode is not being executed, the process of this flowchart ends.

[Modified example]
Some modifications of this embodiment will be described below. For example, in the embodiment described above, when the identification unit 153 cannot specify the driving lane of the vehicle M, the driving control unit changes the lane so as to position the vehicle M in the leftmost or rightmost lane of the road on which the vehicle M is traveling. You may do so. Thereby, the driving lane can be specified more accurately. Note that the driving control unit may cause the specifying unit 153 to perform a process of specifying the driving lane of the vehicle M after the vehicle M is positioned in the left-most or right-most lane of the road. Thereby, when the driving lane cannot be specified, the driving lane can be specified more quickly.

Further, in the embodiment described above, in addition to the first map information 54, map information (second map information) more accurate than the first map information 54 is held, and the second map information When a situation arises in which information cannot be acquired, control may be performed to specify the driving lane using the first map information 54 or the image captured by the camera 10.

Here, the second map information is, for example, map information in which road information is defined for each lane in a shorter section than the first map information 54. Further, the second map information may include, for example, information on the center of the lane or information on the boundary between the lanes. Further, the second map information may include road information, traffic regulation information, address information (address/zip code), facility information, telephone number information, and the like. The second map information may be updated at any time by the communication device 20 communicating with other devices. The second map information may be held in a storage device such as the HDD or flash memory of the MPU 60, or may be stored in the storage unit 180, for example.

For example, if the MPU 60 or the storage unit 180 has second map information, the recommended lane determining unit 61 divides the route on the map provided from the navigation device 50 into a plurality of blocks, and blocks the blocks by referring to the second map information. Determine the recommended lane for each vehicle. In this case, the recommended lane determining unit 61 may determine which lane from the left the vehicle M should drive using the lane information included in the second map information.

Here, for example, if the vehicle M is executing the first driving mode and the second map information is unavailable due to an abnormality in the data of the second map information or an abnormality during updating of the second map information, The mode determining unit 150 specifies the driving lane of the vehicle M as described above based on the information acquired from the first map information 54 and continues the first driving mode. Thereby, even in a situation where the second map information is not available, a driving mode with a high automation rate can be executed.

Further, in the embodiment, when switching from the first driving mode to the second driving mode, the mode determining unit 150 selects one of the plurality of modes included in the second driving mode according to the driving state and driving environment of the vehicle M. , may decide which mode to switch to. The driving state is, for example, the state of the driver determined by the driver state determining unit 151. The driving environment includes, for example, the shape of the road around the vehicle M, the number of lanes, branching, presence or absence of merging, the number and relative positions of other vehicles existing in the vicinity, and the like. For example, when the conditions for switching from the first driving mode to the second driving mode are met, if the number of lanes in which the vehicle M is traveling cannot be identified and the number of lanes is three, the mode determining unit 150 switches to mode C; In the case of a lane, it is decided to switch to mode D, and in the case of five or more lanes, it is decided to switch to mode E. Thereby, the vehicle M can be driven in a more appropriate mode depending on the driving state and driving environment. In addition, in the above-described embodiment, the description has been made assuming that the specific road division line is mainly a deceleration dashed line, but instead of (or in addition to) the deceleration dashed line, another line segment corresponding to the above-mentioned specific road division line may be used. The deceleration broken line may be read as another line segment as appropriate, depending on the traffic regulations (road traffic law) of the foreign country to which this embodiment is applied.

According to the embodiment described above, the vehicle control device includes a recognition unit 130 that recognizes the surrounding situation of the vehicle M, and one of the steering or speed of the vehicle M based on the surrounding situation recognized by the recognition unit 130. Or acquire the driving control unit (action plan generation unit 140, second control unit 160) that controls both, map information including lane information around the vehicle M, and reference information for specifying the position of the vehicle M. information on the road on which the vehicle M travels, which is acquired from map information based on the reference information, and one or more lanes around the vehicle M recognized by the recognition unit 130. and a specifying unit 153 that specifies the driving lane of the vehicle M based on the type of the marking line that partitions the road, and the specifying unit 153 identifies the specific road marking line among the marking lines recognized by the recognition unit 130. By specifying the driving lane of the vehicle M without using at least the above type of information, it is possible to improve the identification rate of the driving lane of the vehicle. Therefore, it is possible to continue the first driving mode using the information on the identified driving lane, or to change the degree of driving control in a more appropriate situation.

Specifically, according to the embodiment, even if a specific road marking line that is difficult to recognize in a camera image is included, the special rule (predetermined By specifying the driving lane by adopting the relaxation condition (relaxation condition), it is possible to improve the identification rate of the driving lane and suppress erroneous identification. For example, according to the embodiment, when the positions and types (line types) of multiple lane markings obtained from camera images meet predetermined judgment conditions, the combination of types determines which position among multiple lanes (parallel lanes) It is possible to identify whether the vehicle is running. Further, according to the embodiment, for example, it can be determined whether the vehicle M is traveling on the edge lane of the road based on the position of the solid line, or when the vehicle M has dashed lines on both sides, By specifying the driving lane using road shoulder markings or the like, it is possible to specify the driving lane more accurately.

Further, according to the embodiment, based on the recognition results of the two left and right lane marking lines of the vehicle M (the first lane marking line on the left side, the second lane marking line on the left side, the first lane marking line on the right side, and the second lane marking line on the right side), Since the driving lane is specified, the identification rate of the driving lane can be improved without increasing the load of recognition processing. Further, according to the embodiment, even if high-precision map information is not installed in the vehicle M, the position of the vehicle M (driving lane ) can be specified more accurately, and execution of the first operation mode can be continued. This eliminates the need for constant updating of map information and management by a map server, which is required for high-precision maps, so operational costs can be reduced.

The embodiment described above can be expressed as follows.
a storage device that stores the program;
comprising a hardware processor;
By the hardware processor executing a program stored in the storage device,
Recognizes the surrounding situation of the vehicle,
Executing driving control to control one or both of the steering and speed of the vehicle based on the recognized surrounding situation,
Obtaining map information including lane information around the vehicle and reference information for identifying the location of the vehicle;
The information on the road on which the vehicle travels is acquired from the map information based on the reference information, and the recognized type of marking line that separates each of one or more lanes around the vehicle. Identify the driving lane,
identifying the driving lane of the vehicle without using at least the type of information for a specific road marking line among the recognized marking lines;
Vehicle control device.

Although the mode for implementing the present invention has been described above using embodiments, the present invention is not limited to these embodiments in any way, and various modifications and substitutions can be made without departing from the gist of the present invention. can be added.

DESCRIPTION OF SYMBOLS 1... Vehicle system, 10... Camera, 12... Radar device, 14... LIDAR, 16... Object recognition device, 20... Communication device, 30... HMI, 32... Turn signal switch, 40... Vehicle sensor, 50... Navigation device, 60... MPU, 70... Driver monitor camera, 80... Driving operator, 82... Steering wheel, 84... Steering grip sensor, 100... Automatic driving control device, 120... First control section, 130... Recognition section, 140... Action plan generation section , 150...Mode determination section, 151...Driver state determination section, 152...First acquisition section, 153...Identification section, 154...Mode change processing section, 160...Second control section, 162...Second acquisition section, 164... Speed control unit, 166...Steering control unit, 170...HMI control unit, 180...Storage unit, 200...Travel driving force output device, 210...Brake device, 220...Steering device, M...Vehicle

Claims (9)

A recognition unit that recognizes the surrounding situation of the vehicle;
a driving control unit that controls one or both of the steering and speed of the vehicle based on the surrounding situation recognized by the recognition unit;
an acquisition unit that acquires map information including lane information around the vehicle and reference information for identifying the position of the vehicle;
Based on information about the road on which the vehicle travels, which is obtained from the map information based on the reference information, and the type of marking line that separates each of one or more lanes around the vehicle, which is recognized by the recognition unit. and a specifying unit that specifies the driving lane of the vehicle,
The identification unit identifies the driving lane of the vehicle without using at least the type information for a specific road marking line among the marking lines recognized by the recognition unit.
Vehicle control device. The specific road marking line is a type of road marking line on either the left or right side of the vehicle that is a deceleration broken line.
The vehicle control device according to claim 1. When there is a possibility that the type of one of the left and right lane markings of the vehicle is a deceleration broken line, the identification unit identifies the driving lane of the vehicle based on at least the type of the other lane marking. do,
The vehicle control device according to claim 1 or 2. The identifying unit is configured to identify the vehicle based on a road shoulder marking line outside the vehicle's travel road when there is a possibility that the type of one of the left and right marking lines of the vehicle is a deceleration broken line. identify the driving lane,
The vehicle control device according to claim 1 or 2. When there is a possibility that one or both of the left and right road marking lines of the vehicle are of a type of deceleration broken line, the identification unit determines the type of road marking line other than the left and right road marking lines of the vehicle. , identifying the driving lane of the vehicle;
The vehicle control device according to any one of claims 1 to 4. The operation control section includes:
Running the vehicle by executing any one of a plurality of driving modes including at least a first driving mode and a second driving mode in which tasks imposed on the occupant of the vehicle are more severe than in the first driving mode;
executing the first driving mode when the identifying unit identifies the driving lane of the vehicle;
executing the second driving mode when the driving lane of the vehicle is not identified by the identifying unit;
The vehicle control device according to any one of claims 1 to 5. When the driving control unit causes the vehicle to run by executing one of a plurality of driving modes in which tasks are imposed on occupants of the vehicle and the driving lane of the vehicle is not specified by the identifying unit. the vehicle continues to run in the current driving mode;
The vehicle control device according to any one of claims 1 to 5. The computer is
Recognizes the surrounding situation of the vehicle,
Executing driving control to control one or both of the steering and speed of the vehicle based on the recognized surrounding situation,
Obtaining map information including lane information around the vehicle and reference information for identifying the location of the vehicle;
The information on the road on which the vehicle travels is acquired from the map information based on the reference information, and the recognized type of marking line that separates each of one or more lanes around the vehicle. Identify the driving lane,
identifying the driving lane of the vehicle without using at least the type of information for a specific road marking line among the recognized marking lines;
Vehicle control method. to the computer,
Makes the vehicle aware of its surroundings,
Execute driving control to control one or both of the steering and speed of the vehicle based on the recognized surrounding situation;
acquiring map information including lane information around the vehicle and reference information for identifying the location of the vehicle;
Based on the information on the road on which the vehicle travels, which is acquired from the map information based on the reference information, and the type of lane markings that separate each of one or more lanes around the recognized vehicle, identify the driving lane of
identifying the driving lane of the vehicle without using at least the type of information for a specific road marking line among the recognized marking lines;
program.

Images