US10031519B2 - Driving support device, driving support system, and driving support method - Google Patents

Driving support device, driving support system, and driving support method Download PDF

Info

Publication number
US10031519B2
US10031519B2 US15/501,176 US201615501176A US10031519B2 US 10031519 B2 US10031519 B2 US 10031519B2 US 201615501176 A US201615501176 A US 201615501176A US 10031519 B2 US10031519 B2 US 10031519B2
Authority
US
United States
Prior art keywords
vehicle
automatic driving
icon
image
sign
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
US15/501,176
Other languages
English (en)
Other versions
US20170364066A1 (en
Inventor
Kazushige Yamada
Osamu YAMAJI
Azusa Gojyo
Satoshi Kajita
Toshiaki Mori
Makoto Okada
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panasonic Automotive Systems Co Ltd
Original Assignee
Panasonic Intellectual Property Management Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Panasonic Intellectual Property Management Co Ltd filed Critical Panasonic Intellectual Property Management Co Ltd
Assigned to PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LTD. reassignment PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: GOJYO, Azusa, MORI, TOSHIAKI, KAJITA, SATOSHI, OKADA, MAKOTO, YAMADA, KAZUSHIGE, YAMAJI, OSAMU
Publication of US20170364066A1 publication Critical patent/US20170364066A1/en
Application granted granted Critical
Publication of US10031519B2 publication Critical patent/US10031519B2/en
Assigned to PANASONIC AUTOMOTIVE SYSTEMS CO., LTD. reassignment PANASONIC AUTOMOTIVE SYSTEMS CO., LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LTD.
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/0011—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement
    • G05D1/0016—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement characterised by the operator's input device
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B60K35/10—Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B60K35/20—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
    • B60K35/28—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K37/00—Dashboards
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W10/00—Conjoint control of vehicle sub-units of different type or different function
    • B60W10/20—Conjoint control of vehicle sub-units of different type or different function including control of steering systems
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
    • B60W30/18—Propelling the vehicle
    • B60W30/18009—Propelling the vehicle related to particular drive situations
    • B60W30/18163—Lane change; Overtaking manoeuvres
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/143—Touch sensitive instrument input devices
    • B60K2360/1438—Touch screens
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/16—Type of output information
    • B60K2360/171—Vehicle or relevant part thereof displayed
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/16—Type of output information
    • B60K2360/175—Autonomous driving
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/20—Optical features of instruments
    • B60K2360/31—Virtual images
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B60W2050/146—Display means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/215—Selection or confirmation of options
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2710/00—Output or target parameters relating to a particular sub-units
    • B60W2710/20—Steering systems

Definitions

  • the present invention relates to a technique that supports a driving operation instruction of a driver to a vehicle during automatic driving.
  • Level 1 is a driving support system that automatically performs one of acceleration, deceleration, and steering
  • level 2 is a driving support system that automatically performs two or more of acceleration, deceleration, and steering in coordination. In all cases, a driver is required to be involved in a driving operation.
  • Autonomy level 4 is a fully-automatic driving system that automatically performs all of acceleration, deceleration, and steering, and thus a driver is not involved in a driving operation.
  • Autonomy level 3 is a semi-fully-automatic driving system that automatically performs all of acceleration, deceleration, and steering, but if necessary, a driving operation is performed by a driver.
  • a form of automatic driving a form in which a driver does not operate an existing driving operation unit such as a steering, an accelerator pedal, or the like but instructs a specific driving operation such as traffic lane change, passing, following traveling, or the like to a vehicle by issuing a command to a vehicle is considered.
  • a user interface in which there are fewer erroneous operations is required.
  • a technique that extracts a guide sign from the original image and selectively displays the guide sign based on a traveling direction or the like is known (for example, refer to PTL 1).
  • a driving support device includes an image output unit that outputs image information including a vehicle object representing a vehicle and a sign object representing sign information, to a display unit.
  • the driving support device further includes an operation signal input unit that receives an operation signal of a user for moving, in the image displayed on the display unit, the vehicle object to the position of the sign object or for moving the sign object to the position of the vehicle object.
  • the driving support device further includes a command output unit that outputs an automatic driving control command corresponding to the display content in the sign object when the operation signal is received, to an automatic driving control unit that controls automatic driving.
  • an aspect in which an expression of the present invention is converted into a device, a system, a method, a program, a recording medium in which a program is recorded, and an automatic drive vehicle equipped therewith by arbitrarily combining the above components is also effective.
  • a driver can intuitively and conveniently instruct a specific driving operation to a vehicle.
  • FIG. 1 is a block diagram illustrating a configuration of a vehicle according to an embodiment of the present invention.
  • FIG. 2 is a diagram illustrating an example of a basic sequence of detection unit, automatic driving controller, HMI controller, display unit, and input unit in FIG. 1 .
  • FIG. 3 is a diagram illustrating an example of a basic flowchart for explaining processing of HMI controller in FIG. 1 .
  • FIG. 4 is a flowchart for explaining update processing of a reception mode.
  • FIG. 5 is a flowchart illustrating an example of determination processing in a case where a gesture operation for instructing a specific operation using a road guide sign is input in step S 9 of FIG. 3 .
  • FIG. 6 is a flowchart illustrating a processing example of issuing a command corresponding to the content of the road guide sign by a gesture operation.
  • FIG. 7A and FIG. 7B are diagrams illustrating an example of issuing a course change instruction command or a route change instruction command using a destination guide sign at an intersection.
  • FIG. 8A and FIG. 8B are diagrams illustrating an example of issuing a course change instruction command or a route change instruction command using a traffic lane guide sign.
  • FIG. 9A and FIG. 9B are diagrams illustrating an example of issuing a setting instruction command for setting a passing spot using an exit guide sign of a toll road.
  • FIG. 10A and FIG. 10B are diagrams illustrating another example of issuing a setting instruction command for setting a passing spot using an exit guide sign of a toll road.
  • FIG. 11A and FIG. 11B are diagrams illustrating an example of issuing a destination setting instruction command for setting a destination using a parking lot guide sign.
  • FIG. 12A to FIG. 12C are diagrams illustrating another example of issuing a course change instruction command using a destination guide sign at an intersection.
  • FIG. 13A and FIG. 13B are diagrams illustrating an example of issuing a traffic lane change instruction command and a course change instruction command using a guide sign on a traffic lane.
  • the present disclosure has been made in consideration of such a situation, and an object thereof is to provide a technique that allows a driver to intuitively and conveniently instruct a specific driving operation to a vehicle.
  • FIG. 1 is a block diagram illustrating a configuration of vehicle 1 according to an embodiment of the present invention, and illustrates a configuration related to automatic driving.
  • Vehicle 1 an own vehicle with an automatic driving mode includes driving support device (human-machine interface (HMI) controller) 10 , automatic driving control device (automatic driving controller) 20 , display device 30 , detection unit 40 , and driving operation unit 50 .
  • driving support device human-machine interface (HMI) controller
  • HMI human-machine interface
  • automatic driving control device automated driving controller
  • display device 30 display device 30
  • detection unit 40 detection unit
  • driving operation unit 50 driving operation unit 50 .
  • Display device 30 includes display unit 31 and input unit 32 .
  • Display device 30 may be a head unit such as a car navigation system, a display audio, or the like, a portable terminal device such as a smart phone, a tablet, or the like, or a dedicated console terminal device.
  • Display unit 31 is a liquid crystal display, an organic electro-luminescence (EL) display, or a heads-up display (HUD).
  • Input unit 32 is a user interface that receives an input of a user.
  • Display unit 31 and input unit 32 may be an integrated touch panel display.
  • Such as a proximity touch panel that can detect proximity of a hand on a touch panel or a touch pad or a position of a finger due to a hover operation, one that receives a gesture input at a position apart from a predetermined distance from the display unit may be used.
  • Input unit 32 may have an input device such as a mouse, a stylus pen, a trackball, or the like for assisting the gesture input.
  • a pen that emits visible light or infrared light may also be used.
  • Display unit 31 and input unit 32 may be physically separated from each other, instead of being an integrated touch panel display.
  • input unit 32 includes a sensor such as a camera or the like, and may be a non-contact type input device that allows a gesture operation input in the air.
  • a gesture operation input in the air For example, an operation method that starts to drag with a gesture by pointing a target with a finger and making a thumb finger and an index finger close and shut together, and ends to drag with a gesture by making a thumb finger and an index finger separated from each other, is considered.
  • Driving support device 10 and display device 30 may be connected to each other by wire communication such as a dedicated line, a controller area network (CAN), or the like, or may be connected to each other by wire communication or wireless communication such as USB, Ethernet (registered trademark), Wi-Fi (registered trademark), Bluetooth (registered trademark), or the like.
  • wire communication such as a dedicated line, a controller area network (CAN), or the like
  • wire communication or wireless communication such as USB, Ethernet (registered trademark), Wi-Fi (registered trademark), Bluetooth (registered trademark), or the like.
  • Detection unit 40 includes position information acquisition unit 41 , sensor 42 , speed information acquisition unit 43 , and map information acquisition unit 44 .
  • Position information acquisition unit 41 acquires a current position of vehicle 1 from a global positioning system (GPS) receiver.
  • Sensor 42 is a general term for various sensors for detecting a situation outside the vehicle and a state of vehicle 1 .
  • a sensor for detecting a situation outside the vehicle for example, a camera, a millimeter wave radar, a light detection and ranging (laser imaging detection and ranging, LIDAR), a temperature sensor, a pressure sensor, a humidity sensor, an illumination sensor, or the like is mounted.
  • a road situation in which the own vehicle travels or an environmental situation including weather, and an own vehicle peripheral situation such as a traveling position or a traveling state of an another vehicle traveling at the periphery of the own vehicle are considered. Any information outside the vehicle that can be detected by the sensor may be considered.
  • a sensor for detecting a state of vehicle 1 for example, an acceleration sensor, a gyro sensor, a geomagnetic sensor, an inclination sensor, or the like is mounted.
  • Speed information acquisition unit 43 acquires the current speed of vehicle 1 from a vehicle speed sensor.
  • Map information acquiring unit 44 acquires map information around the current position of vehicle 1 from a map database. Map database may be recorded on a recording medium in vehicle 1 , or may be downloaded from a map server via a network in use.
  • Detection unit 40 and automatic driving control device 20 are connected to each other by wire communication such as a dedicated line, a USB, an Ethernet (registered trademark), a controller area network (CAN), or the like.
  • wire communication such as a dedicated line, a USB, an Ethernet (registered trademark), a controller area network (CAN), or the like.
  • a configuration in which data acquired and detected by detection unit 40 is directly output from detection unit 40 to driving support device 10 may be employed.
  • Driving operation unit 50 includes steering 51 , brake pedal 52 , accelerator pedal 53 , and blinker switch 54 .
  • acceleration, deceleration, steering, and blinker blink are a target of automatic control by automatic driving control device 20 .
  • FIG. 1 an operation unit when these controls are performed manually is drawn.
  • Information indicating that driving operation unit 50 is slightly moved by a driver in a manual way may output to driving support device 10 .
  • Steering 51 is an operation unit for steering the vehicle.
  • the steering actuator can be electronically controlled by a steering electronic control unit (ECU).
  • ECU steering electronic control unit
  • Brake pedal 52 is an operation unit for decelerating vehicle 1 .
  • brake pedal 52 When brake pedal 52 is depressed by a driver, the vehicle is decelerated via a brake actuator.
  • the brake actuator can be electronically controlled by a brake ECU.
  • Accelerator pedal 53 is an operation unit for accelerating vehicle 1 .
  • an engine rotation speed and/or a motor rotation speed is controlled via an accelerator actuator.
  • the engine rotation speed is controlled.
  • the motor rotation speed is controlled.
  • both of the engine rotation speed and the motor rotation speed are controlled.
  • the accelerator actuator can be electronically controlled by at least one of an engine ECU and a motor ECU.
  • Blinker switch 54 is an operation unit for blinking a blinker so as to notify a course of the vehicle to the outside.
  • the blinker controller includes a drive circuit such as a relay controlling power supply to a blinker lamp, or the like.
  • Each of the steering ECU, the brake ECU, the engine ECU, the motor ECU, and the blinker controller and automatic driving control device 20 are connected to each other by wired communication such as a CAN, a dedicated line, or the like.
  • Each of the steering ECU, the brake ECU, the engine ECU, the motor ECU, and the blinker controller respectively transmits a state signal indicating the state of each of a steering, a brake, an engine, a motor, and a blinker lamp to automatic driving control device 20 .
  • each of the steering ECU, the brake ECU, the engine ECU, and the motor ECU drives the corresponding actuator according to a control signal supplied from automatic driving control device 20 .
  • a manual driving mode a configuration in which an instruction is directly transferred from each of steering 51 , brake pedal 52 , and accelerator pedal 53 to the corresponding actuator in a mechanical way may be employed, or a configuration in which electronic control intervenes via the corresponding ECU may be employed.
  • the blinker controller turns on/off the blinker lamp according to the control signal supplied from automatic driving control device 20 or an instruction signal from blinker switch 54 .
  • Automatic driving control device 20 is an automatic driving controller that has an automatic driving control function, and includes control unit 21 , memory unit 22 , and input-output unit 23 .
  • the configuration of control unit 21 may be realized by cooperation between hardware resources and software resources or only hardware resources.
  • As the hardware resources a processor, a read only memory (ROM), a random access memory (RAM), and other large scale integrated (LSI) circuits can be used, and as software resources, an operating system, an application, and a program such as a firmware or the like can be used.
  • Memory unit 22 includes a non-volatile recording medium such as a flash memory or the like.
  • Input-output unit 23 performs various communication control according to various communication formats.
  • Control unit 21 calculates a control value for controlling an automatic control target such as a traveling direction of vehicle 1 by applying various parameter values collected from detection unit 40 and various ECUs to an automatic driving algorithm. Control unit 21 transfers the calculated control value to the ECU or controller of each control target. In the present embodiment, the control value is transferred to the steering ECU, the brake ECU, the engine ECU, and the blinker controller. In a case of an electric vehicle or a hybrid car, the control value is transferred to the motor ECU instead of the engine ECU or in addition to the engine ECU.
  • Driving support device 10 is a human machine interface (HMI) controller for performing an interface function between vehicle 1 and a driver, and includes determination unit 11 , image generation unit 12 , instruction unit 13 , and input-output unit 14 .
  • Determination unit 11 , image generation unit 12 , and instruction unit 13 can be realized by cooperation between hardware resources and software resources, or only hardware resources.
  • As the hardware resources a processor, a ROM, a RAM, and other LSI circuits can be used, and as software resources, an operating system, an application, and a program such as a firmware or the like can be used.
  • Input-output unit 14 performs various communication control according to various communication formats.
  • Input-output unit 14 includes image output unit 14 a , operation signal input unit 14 b , command output unit 14 c , and vehicle information input unit 14 d .
  • Image output unit 14 a outputs an image generated by image generation unit 12 to display unit 31 .
  • Operation signal input unit 14 b receives an operation signal that is input from input unit 32 by an operation of a driver, a passenger, or a user outside the vehicle, and outputs the operation signal to determination unit 11 .
  • Command output unit 14 c outputs the command instructed by instruction unit 13 to automatic driving controller 20 .
  • Vehicle information input unit 14 d receives detection data acquired by detection unit 40 or vehicle information generated by automatic driving controller 20 , and outputs the detection data or the vehicle information to image generation unit 12 .
  • Automatic driving controller 20 and HMI controller 10 are directly connected to each other by a signal line.
  • a configuration in which automatic driving controller 20 and HMI controller 10 are connected to each other via a CAN may be employed.
  • a configuration in which automatic driving controller 20 and HMI controller 10 are integrated into one controller may be also employed.
  • FIG. 2 is a diagram illustrating an example of a basic sequence of detection unit 40 , automatic driving controller 20 , HMI controller 10 , display unit 31 , and input unit 32 in FIG. 1 .
  • Detection unit 40 detects own vehicle position information, own vehicle traveling road information, and own vehicle peripheral information, and outputs the detected information to automatic driving controller 20 (P 1 ).
  • Automatic driving controller 20 outputs the own vehicle position information acquired from detection unit 40 , the own vehicle traveling road information, and the own vehicle peripheral information to HMI controller 10 (P 2 ).
  • HMI controller 10 generates a schematic diagram including the own vehicle and the own vehicle peripheral situation based on the information acquired from automatic driving controller 20 (P 3 ).
  • HMI controller 10 outputs the generated schematic diagram to display device 30 , and the schematic diagram is displayed on display unit 31 (P 4 ).
  • a user who views the schematic diagram displayed on display unit 31 comes into contact with input unit 32 (P 5 ).
  • Display unit 31 outputs coordinate data at a position at which the contact is detected to HMI controller 10 (P 6 ).
  • HMI controller 10 determines a type of the command based on the coordinate data acquired from display device 30 (P 7 ).
  • HMI controller 10 receives an additional input until a certain period of time has elapsed (P 8 to P 12 ).
  • HMI controller 10 regenerates a schematic diagram indicating that the command is being instructed (P 8 ).
  • HMI controller 10 outputs the regenerated schematic diagram to display device 30 , and the regenerated schematic diagram is displayed on display unit 31 (P 9 ).
  • HMI controller 10 In a case where the command corresponding to the gesture operation by the touch of a user does not exist, HMI controller 10 generates a schematic diagram including an error message, and outputs the schematic diagram to display device 30 , and the schematic diagram is displayed on display unit 31 .
  • HMI controller 10 When a user who views the schematic diagram indicating that the command is being instructed comes into contact with input unit 32 (P 10 ), display unit 31 outputs the coordinate data at a position at which the contact is detected to HMI controller 10 (P 11 ). HMI controller 10 performs additional command processing based on the coordinate data acquired from display device 30 (P 12 ). In a case where there is no input of a new command in the additional command processing (P 12 ), HMI controller 10 issues a command determined in P 7 to automatic driving controller 20 (P 13 and P 14 ). In a case where a new command is input in the additional command processing (P 12 ), HMI controller 10 issues a new command to automatic driving controller 20 .
  • HMI controller 10 cancels the issue of a command. Processing of overwriting and canceling of the original command due to a new command may be performed by automatic driving controller 20 . In this case, after the command determination processing in P 7 and P 12 , HMI controller 10 transmits the command to automatic driving controller 20 , and performs processing of overwriting and canceling according to an internal state of automatic driving controller 20 .
  • Detection unit 40 periodically detects the own vehicle position information, the own vehicle traveling road information, and the own vehicle peripheral information, and outputs the information to automatic driving controller 20 (P 15 ).
  • Automatic driving controller 20 determines whether or not control instructed by the command issued from HMI controller 10 is executable (P 16 ), based on the information. In a case where it is determined that the control is executable, automatic driving controller 20 outputs a control start notification to HMI controller 10 (P 17 ).
  • HMI controller 10 regenerates a schematic diagram including a message indicating that the control is being performed (P 18 ).
  • HMI controller 10 outputs the regenerated schematic diagram to display device 30 , and the regenerated schematic diagram is displayed on the display unit 31 (P 19 ).
  • automatic driving controller 20 calculates a control value for controlling driving operation unit 50 that performs the issued command by applying various parameter values collected from detection unit 40 or various ECUs to an automatic driving algorithm, and transfers the control value to the ECU or controller of each control target.
  • the control value is a specific value for automatically controlling control target such as a traveling direction of vehicle 1 or the like.
  • Driving operation unit 50 operates based on the specific control value. When a predetermined control value or the detection data acquired by detection unit 40 is a predetermined value (in a predetermined range), and when automatic driving controller 20 determines that a condition of the issued command is satisfied, driving operation unit 50 determines that performing of the command is completed.
  • HMI controller 10 When a control completion notification is received from automatic driving controller 20 , HMI controller 10 generates a schematic diagram including a message indicating that control is completed, and outputs the generated schematic diagram to display device 30 . During a period for which an operation from a user is not received, HMI controller 10 generates a schematic diagram including a message indicating that an operation is not received, and outputs the generated schematic diagram to display device 30 .
  • FIG. 3 is a diagram illustrating an example of a basic flowchart for explaining processing of HMI controller 10 in FIG. 1 .
  • Determination unit 11 of HMI controller 10 checks whether or not a driving mode is an automatic driving mode or a manual driving mode (S 1 ). In the manual driving mode (N in S 2 ), the process ends. In a case of the automatic driving mode (Y in S 2 ), processing is performed as follows.
  • Image generation unit 12 of HMI controller 10 generates a schematic diagram including the own vehicle and the own vehicle peripheral situation based on the own vehicle position information, the own vehicle traveling road information, and the own vehicle peripheral information that are input from automatic driving controller 20 , and draws the generated schematic diagram on display unit 31 (S 4 ).
  • Determination unit 11 checks whether or not a reception mode is a reception enable mode in which an operation from a user can be received or a reception disable mode in which an operation from a user cannot be received (S 5 ). In a case where the reception mode is the reception disable mode (N in S 6 ), the process ends.
  • determination unit 11 determines whether or not there is a contact to input unit 32 by a user (S 7 ). In a case where there is no contact to input unit 32 by a user (N in S 8 ), predetermined-time elapse determination processing (S 12 ) to be described later is performed. In a case where there is a contact to input unit 32 by a user (Y in S 8 ), determination unit 11 determines a control command according to a gesture operation input by a user (S 9 ). The details of the control command determination processing will be described later.
  • step S 9 In a case where the control command determined in step S 9 is not a cancel command (N in S 10 ), image generation unit 12 displays that the command is being instructed on display unit 31 (S 11 ).
  • operation reception disable is displayed on display unit 31 (S 14 )
  • determination unit 11 updates the reception mode from the reception enable mode to the reception disable mode (S 15 )
  • instruction unit 13 outputs the determined control command to automatic driving controller 20 (S 16 ).
  • a predetermined time has elapsed (N in S 12 )
  • the process transitions to step S 3 .
  • step S 10 in a case where the determined control command is a cancel command (Y in S 10 ), cancel is displayed (S 110 ), and the process ends.
  • step S 13 in a case where there is no control command determined in step S 9 (N in S 13 ), an input error is displayed (S 111 ), and the process ends.
  • Automatic driving controller 20 periodically detects the own vehicle position information, the own vehicle traveling road information, and the own vehicle peripheral information from detection unit 40 . Since the own vehicle peripheral situation constantly changes, after the control command is output to automatic driving controller 20 , there is a case where it is determined that the control command is not executable. For example, after a following instruction, there is a case where other cars interrupt between the own vehicle and an another vehicle.
  • image generation unit 12 displays that the control is being performed on display unit 31 (S 18 ), and starts counting by activating a timer (S 19 ). In a case where it is determined that the control is not executable by automatic driving controller 20 (N in S 17 ), image generation unit 12 displays a control disable error on display unit 31 (S 112 ).
  • FIG. 4 is a flowchart for explaining update processing of a reception mode.
  • determination unit 11 of HMI controller 10 updates the reception mode from the reception disable mode to the reception enable mode (S 114 ).
  • the count value of the timer may be changed according to the own vehicle peripheral situation.
  • determination unit 11 may update the reception mode from the reception disable mode to the reception enable mode.
  • control command an example in which a control command for instructing a specific operation using a road guide sign is issued will be described.
  • a user inputs a gesture operation for instructing a specific operation using a road guide sign to input unit 32 .
  • a specific example of the gesture operation will be described later.
  • FIG. 5 is a flowchart illustrating an example of determination processing in a case where a gesture operation for instructing a specific operation using a road guide sign is input in step S 9 of FIG. 3 .
  • Determination unit 11 of HMI controller 10 determines whether or not an own vehicle mark is present at a contact start position (S 9 a ). In a case where the own vehicle mark is not present at the contact start position (N in S 9 a ), determination unit 11 determines that the gesture operation is a control command of another operation in which a road guide sign is not used (S 9 b ).
  • generation unit 12 draws a drop enable area in the schematic diagram, and displays the area on display unit 31 (S 9 c ). At this time, generation unit 12 generates a drop enable area of a road guide sign in the schematic diagram.
  • Information on the drop enable area of the road guide sign is acquired from a local map information database that is prepared in advance or a map information database on a network.
  • a coordinate of an object such as a place name, an arrow, or the like in the sign may be acquired by using image recognition technology based on a road guide sign image captured in a real time by a vehicle-mounted camera.
  • Determination unit 11 receives a touch event generated in input unit 32 (S 9 d ), and determines the type of the touch event (S 9 e ). In a case where the type of the touch event is a movement (movement in S 9 e ), generation unit 12 draws a prediction trajectory/route candidate of vehicle 1 in the schematic diagram, and the prediction trajectory/route candidate of vehicle 1 is displayed on display unit 31 (S 9 f ).
  • determination unit 11 determines whether or not a contact end position is present in the area of the road guide sign (S 9 g ). In a case where the contact end position is not present in the area of the road guide sign (N in S 9 g ), determination unit 11 determines that the gesture operation is a control command of another operation in which the road guide sign is not used (S 9 h ). In a case where the contact end position is present in the area of the road guide sign (Y in S 9 g ), determination unit 11 determines whether or not the area of the road guide sign is an area of a lane guide sign or an uphill lane sign (S 9 i ).
  • determination unit 11 determines whether or not the lane of a lane guide sign or an uphill lane sign is a right-turn-only lane or a left-turn-only lane (S 9 j ). In a case where the lane is not a right-turn-only lane or a left-turn-only lane (N in S 9 j ), determination unit 11 determines that the gesture operation is a traffic lane change instruction command for instructing the traffic lane change to the lane (S 9 k ).
  • determination unit 11 determines that the gesture operation is a traffic lane change instruction command for instructing the traffic lane change to the lane, and a right turn instruction or a left turn instruction (S 9 l ).
  • step S 9 i in a case where the dropped area is not an area of a lane guide sign or an uphill lane sign (N in S 9 i ), determination unit 11 determines whether or not the dropped area is an area in which an arrow indicating a direction or a way is displayed (S 9 m ). In a case where the dropped area is an area in which an arrow indicating a direction or a way is displayed (Y in S 9 m ), determination unit 11 determines that the gesture operation is a traveling direction change instruction command for instructing the traveling direction change to the direction (S 9 n ).
  • determination unit 11 determines whether or not the dropped area is an area in which a place name indicating a city or a region is displayed (S 9 o ). In a case where the dropped area is an area in which a place name indicating a city or a region is displayed (Y in S 9 o ), determination unit 11 acquires a traveling direction corresponding to the place name from the map information database (S 9 p ). Determination unit 11 determines that the gesture operation is a traveling direction change instruction command for instructing the traveling direction change to the acquired traveling direction (S 9 q ).
  • step S 9 o in a case where the dropped area is not an area in which a place name indicating a city or a region is displayed (N in S 9 o ), determination unit 11 determines whether or not the dropped area is an area in which a road name is displayed (S 9 r ). In a case where the dropped area is an area in which a road name is displayed (Y in S 9 r ), generation unit 12 generates an image (for example, a menu or a dialog) or a voice that requests a selection of an inbound and an outbound, and notifies a user of the image or voice (S 9 s ).
  • an image for example, a menu or a dialog
  • voice that requests a selection of an inbound and an outbound
  • determination unit 11 determines that the gesture operation is a traveling direction change instruction command for instructing the traveling direction change to the inbound (S 9 u ). In a case where an outbound is selected by a user (N in S 9 t ), determination unit 11 determines that the gesture operation is a traveling direction change instruction command for instructing the traveling direction change to the outbound (S 9 v ).
  • step S 9 r in a case where the dropped area is not an area in which a road name is displayed (N in S 9 r ), determination unit 11 determines whether or not the dropped area is an area in which a sign of an entrance, an exit, a parking area, or a service area of a toll road is displayed (S 9 w ). In a case where the dropped area is an area in which a sign of an entrance, an exit, a parking area, or a service area of a toll road is displayed (Y in S 9 w ), determination unit 11 determines that the gesture operation is a setting instruction command for setting the entrance, exit, parking area, or service area of the toll road to a passing spot (S 9 x ).
  • determination unit 11 determines whether or not the dropped area is an area in which a sign of a parking lot, a shelter, or an emergency phone is displayed (S 9 y ). In a case where the dropped area is an area in which a sign of a parking lot, a shelter, or an emergency phone is displayed (Y in S 9 y ), determination unit 11 determines that the gesture operation is a setting instruction command for setting the parking lot, shelter, or road shoulder at which an emergency phone is located to a destination (S 9 z ). In step S 9 y , in a case where the dropped area is not a sign of a parking lot, a shelter, or an emergency phone (N in S 9 y ), generation unit 12 displays an error message on display unit 31 (S 9 zz ).
  • FIG. 6 is a flowchart illustrating a processing example of issuing a command according to the content of the road guide sign by a gesture operation.
  • Determination unit 11 of HMI controller 10 receives a touch event (DOWN) from the touch panel (S 50 ).
  • the touch event (DOWN) is an event representing a contact state change on the touch panel from a non-contact state to a contact state by a finger or a pen.
  • Determination unit 11 determines whether or not a coordinate detected by the touch event (DOWN) is present in a display area of the own vehicle icon (S 51 ). In a case where the coordinate detected by the touch event (DOWN) is present outside the display area of the own vehicle icon (N in S 51 ), it is determined that the gesture operation is not an operation instruction corresponding to content of a road guide sign, and the process ends.
  • determination unit 11 receives a touch event (MOVE) from the touch panel (S 52 ).
  • the touch event (MOVE) is an event representing a change from a contact state on a certain point of the touch panel to a contact state on another point of the touch panel by a finger or a pen.
  • determination unit 11 receives a touch event (UP) from the touch panel (S 53 ).
  • the touch event (UP) is an event representing a contact state change on the touch panel from a contact state to a non-contact state by a finger or a pen.
  • Determination unit 11 determines whether or not a coordinate detected by the touch event (UP) is present in a display area of a road guide sign (S 54 ). In a case where the coordinate detected by the touch event (UP) is present in the display area of the road guide sign (Y in S 54 ), instruction unit 13 issues a command according to the content of the road guide sign to automatic driving controller 20 (S 55 ). In a case where the coordinate detected by the touch event (UP) is present outside the display area of the road guide sign (N in S 54 ), it is determined that the gesture operation is not an operation instruction corresponding to content of the road guide sign, and the process ends.
  • FIG. 7A and FIG. 7B are diagrams illustrating an example of issuing a course change instruction command or a route change instruction command using a destination guide sign at an intersection.
  • an own vehicle icon V 1 As display forms of the own vehicle, the road, and the guide sign, various display forms are considered. A real photographed image may be used, and a fine CG image or an animation image may be used. Display of the own vehicle is not limited to an icon, and the own vehicle may be displayed as a more simple mark or character, or may be displayed as a real photographed image. In other words, there is no problem as long as the own vehicle is displayed on a screen as an object in any display form.
  • a driver wants to change the course of the own vehicle to any one course displayed on a destination guide sign S 1 at an intersection, as illustrated in FIG. 7A , a driver drags the own vehicle icon V 1 , and as illustrated in FIG. 7B , drops the own vehicle icon V 1 onto a place name “Machida” N 1 of a desired destination in the destination guide sign S 1 . Accordingly, a course change instruction command for changing the course of the own vehicle to a course of the desired destination is issued. Alternatively, a route change instruction command for changing the route of the own vehicle to a route in which the desired destination is set as a passing spot or a destination is issued.
  • FIG. 8A and FIG. 8B are diagrams illustrating an example of issuing a course change instruction command or a route change instruction command using a traffic lane guide sign.
  • a driver wants to change the course of the own vehicle to any one course displayed on traffic lane guide signs S 2 a and S 2 b , as illustrated in FIG. 8A , a driver drags the own vehicle icon V 1 , and as illustrated in FIG. 8B , drops the own vehicle icon V 1 onto a desired traffic lane guide sign “Sagamihara” S 2 b .
  • a course change instruction command for changing the course of the own vehicle to a course of the desired destination is issued.
  • a route change instruction command for changing the route of the own vehicle to a route in which the desired destination is set as a passing spot or a destination is issued.
  • FIG. 9A and FIG. 9B are diagrams illustrating an example of issuing a setting instruction command for setting a passing spot using an exit guide sign of a toll road.
  • a driver wants to exit a toll road through the next exit, as illustrated in FIG. 9A , a driver drags the own vehicle icon V 1 , and as illustrated in FIG. 9B , drops the own vehicle icon V 1 onto an exit guide sign “Yokohama Machida” S 3 . Accordingly, a passing spot setting instruction command for setting the exit to a passing spot is issued.
  • a route change instruction command is issued.
  • FIG. 10A and FIG. 10B are diagrams illustrating another example of issuing a setting instruction command for setting a passing spot using an exit guide sign of a toll road.
  • a driver wants to exit a toll road through a desired exit
  • a driver drags the own vehicle icon V 1
  • a passing spot setting instruction command for setting the exit to a passing spot is issued.
  • a route change instruction command is issued.
  • FIG. 11A and FIG. 11B are diagrams illustrating an example of issuing a destination setting instruction command for setting a destination using a parking lot guide sign.
  • a driver wants to park the own vehicle in a parking lot, as illustrated in FIG. 11A , a driver drags the own vehicle icon V 1 , and as illustrated in FIG. 11B , drops the own vehicle icon V 1 onto a parking lot guide sign S 5 . Accordingly, a destination setting instruction command for setting the parking lot to a destination is issued.
  • automatic driving controller 20 reroutes the original destination of the own vehicle to a new destination.
  • FIG. 12A to FIG. 12C are diagrams illustrating another example of issuing a course change instruction command using a destination guide sign at an intersection.
  • a driver wants to change the course of the own vehicle to any one course displayed on a destination guide sign S 6 at an intersection, as illustrated in FIG. 12A , a driver drags the own vehicle icon V 1 , and as illustrated in FIG. 12B , drops the own vehicle icon V 1 onto a road name “Gyoko road” R 1 on the destination guide sign S 6 .
  • a destination display window R 2 of the inbound (Machida direction) and the outbound (Zama direction) of the road is displayed.
  • the inbound or outbound of the destination display window R 2 is touched by a driver. Accordingly, a course change instruction command for changing the course of the own vehicle to the course in the touched direction is issued.
  • FIG. 13A and FIG. 13B are diagrams illustrating an example of issuing a traffic lane change instruction command and a course change instruction command using a guide sign on a traffic lane.
  • a driver wants the own vehicle to turn right, as illustrated in FIG. 13A , a driver drags the own vehicle icon V 1 , and as illustrated in FIG. 13B , drops the own vehicle icon V 1 onto a right-turn-only lane sign M 1 on a traffic lane.
  • a traffic lane change instruction command for instructing the traffic lane change to the right-turn-only lane and a right turn instruction command are issued.
  • generation unit 12 may hold the original image (own vehicle icon before movement) in the schematic diagram, and delete the original image when the own vehicle icon is dropped.
  • generation unit 12 may draw a trajectory of the drag of the own vehicle icon using a dotted line in the schematic diagram.
  • generation unit 12 may invert the color of the road, and return the color of the road to the original color thereof when the own vehicle icon is dropped.
  • generation unit 12 may change (invert, thin, or the like) the color of the own vehicle icon.
  • the own vehicle icon may return to the original position thereof, and an error message such as “operation disabled” may be displayed. Examples of a case where an operation is disabled includes approach of following vehicle, a traffic lane change prohibited area, exceeding of a limiting speed, or the like.
  • generation unit 12 may change (invert, thin, or the like) a color of a background such as a road or the like. When an operation is enabled, the color of the background is returned to the original color thereof.
  • generation unit 12 changes a color of the drop disable area.
  • generation unit 12 performs notification using an error sound or a vibration.
  • generation unit 12 may change the color of the own vehicle icon.
  • the drag operation (movement of the own vehicle icon) may be disabled.
  • An error message such as “operation disabled” may be displayed.
  • generation unit 12 may change the color of the background such as a road or the like.
  • the color of the background is returned to the original color thereof.
  • generation unit 12 performs notification using an error sound or a vibration in a state where the drag operation (movement of the own vehicle icon) is disabled.
  • generation unit 12 may display a state of the own vehicle as a ghost, and display a trajectory of the own vehicle. During the period for which the own vehicle icon is dropped and the own vehicle completes processing according to the control command, generation unit 12 may change a display state of the own vehicle icon (flashing, a color change, a size, a position, or the like). During the period for which the own vehicle icon is dropped and processing according to the control command is completed, determination unit 11 may queue a next instruction as an additional operation (reserve a control to be performed after the current control is completed).
  • the gesture operation of the icon is a simple operation, and thus a driver is released from a driving operation in the related art such as turning of steering 51 , depressing of accelerator pedal 53 , or the like.
  • a driving operation in the related art such as turning of steering 51 , depressing of accelerator pedal 53 , or the like.
  • it is possible to easily instruct the operation according to the content of the road guide sign by displaying a schematic diagram including a road guide sign and moving the own vehicle icon onto the road guide sign.
  • a driver can perform a confirmation of a peripheral situation and an operation instruction on the touch panel at the same time, and thus the sight line of the driver does not move.
  • an operation for linking the own vehicle icon and the sign other than the operation may be used.
  • a gesture operation corresponding to a control command is described as a drag-and-drop operation or the like, a touch-and touch operation may be used.
  • a predetermined gesture operation is preferably used, but an operation that is customizable by a driver may be used.
  • a comment, an icon, or an arrow may be displayed by display unit 31 , or a guide display or a voice guidance may be provided by display unit 31 .
  • HMI controller 10 may be realized using a CPU in a portable device such as a smart phone or a tablet that is used as display device 30 .
  • a portable device that is used as display device 30 and automatic driving controller 20 are directly connected to each other.
  • Functions of HMI controller 10 may be realized by a CPU in a head unit such as a car navigation device, a display audio, or the like.
  • a dedicated LSI on which HMI controller 10 is mounted may be included in a head unit.
  • a command according to the display content in a sign is issued by dragging and dropping the own vehicle icon onto a sign object indicating a road guide sign is described.
  • a command according to the display content in a sign can be also issued by dragging and dropping the sign object onto the own vehicle icon.
  • the embodiments may be specified by the following items.
  • a driving support device ( 10 ) including;
  • an image output unit ( 14 a ) that outputs an image including an own vehicle object representing an own vehicle and a sign object representing a sign, to a display unit ( 31 );
  • an operation signal input unit ( 14 b ) that receives an operation signal of a user for moving, in the image displayed on the display unit ( 31 ), the own vehicle object to the position of the sign object or for moving the sign object to the position of the own vehicle object;
  • a command output unit ( 14 c ) that outputs a command according to the display content in the sign object when the operation signal is received, to an automatic driving control unit ( 20 ) that controls automatic driving.
  • sign is a sign on road traffic, and may be a guide sign or a traffic sign.
  • the driving support device ( 10 ) according to Item 1, in which the command output unit ( 14 c ) outputs a command for setting, as a destination or a passing spot, a place specified by the display content in the sign object to which the own vehicle object is moved, to the automatic driving control unit ( 20 ).
  • a user can intuitively and conveniently perform an operation for setting a destination or a passing spot.
  • the driving support device ( 10 ) according to Item 1 or 2, in which the command output unit ( 14 c ) outputs a command for instructing the own vehicle to enter a traffic lane in the sign object to which the own vehicle object is moved, to the automatic driving control unit ( 20 ).
  • “enter” includes right turn, left turn, U-turn, and traffic lane change.
  • a user can intuitively and conveniently perform an operation for instructing right turn, left turn, U-turn, and traffic lane change.
  • the driving support device ( 10 ) in which when the operation signal input unit ( 14 b ) receives an operation signal of the user for moving the own vehicle object in the image displayed on the display unit ( 31 ) to a traffic lane or a road name in the sign object, the image output unit ( 14 a ) outputs an image including a selection object for selecting an inbound or an outbound of the traffic lane, to the display unit ( 31 ).
  • a user can intuitively and conveniently instruct the own vehicle to travel in an inbound or an outbound when changing the course of the own vehicle to another road.
  • a display device ( 30 ) that displays an image
  • a driving support device ( 10 ) that outputs an image to the display device ( 30 ),
  • the driving support device ( 10 ) includes:
  • a driving support method including;
  • An automatic drive vehicle ( 1 ) including:
  • an image output unit ( 14 a ) that outputs an image including an own vehicle object representing an own vehicle and a sign object representing a sign, to a display unit ( 31 );
  • an operation signal input unit ( 14 b ) that receives an operation signal of a user for moving, in the image displayed on the display unit ( 31 ), the own vehicle object to the position of the sign object or for moving the sign object to the position of the own vehicle object;
  • a command output unit ( 14 c ) that outputs a command according to the display content in the sign object when the operation is received;
  • an automatic driving control unit ( 20 ) that executes the output command.
  • the present invention can be used in a vehicle equipped with automatic driving mode.

Landscapes

  • Engineering & Computer Science (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Automation & Control Theory (AREA)
  • Chemical & Material Sciences (AREA)
  • Combustion & Propulsion (AREA)
  • Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • General Physics & Mathematics (AREA)
  • Remote Sensing (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Human Computer Interaction (AREA)
  • Theoretical Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Traffic Control Systems (AREA)
  • Navigation (AREA)
  • Control Of Driving Devices And Active Controlling Of Vehicle (AREA)
  • Instrument Panels (AREA)
  • Instructional Devices (AREA)
  • User Interface Of Digital Computer (AREA)
US15/501,176 2015-07-31 2016-07-19 Driving support device, driving support system, and driving support method Active US10031519B2 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2015-152851 2015-07-31
JP2015152851A JP5957744B1 (ja) 2015-07-31 2015-07-31 運転支援装置、運転支援システム、運転支援方法、運転支援プログラム及び自動運転車両
PCT/JP2016/003383 WO2017022199A1 (ja) 2015-07-31 2016-07-19 運転支援装置、運転支援システム、運転支援方法及び自動運転車両

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2016/003383 A-371-Of-International WO2017022199A1 (ja) 2015-07-31 2016-07-19 運転支援装置、運転支援システム、運転支援方法及び自動運転車両

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/039,149 Continuation US10656639B2 (en) 2015-07-31 2018-07-18 Driving support device, driving support system, and driving support method

Publications (2)

Publication Number Publication Date
US20170364066A1 US20170364066A1 (en) 2017-12-21
US10031519B2 true US10031519B2 (en) 2018-07-24

Family

ID=56513772

Family Applications (2)

Application Number Title Priority Date Filing Date
US15/501,176 Active US10031519B2 (en) 2015-07-31 2016-07-19 Driving support device, driving support system, and driving support method
US16/039,149 Active 2036-09-06 US10656639B2 (en) 2015-07-31 2018-07-18 Driving support device, driving support system, and driving support method

Family Applications After (1)

Application Number Title Priority Date Filing Date
US16/039,149 Active 2036-09-06 US10656639B2 (en) 2015-07-31 2018-07-18 Driving support device, driving support system, and driving support method

Country Status (5)

Country Link
US (2) US10031519B2 (ja)
JP (1) JP5957744B1 (ja)
CN (1) CN107851394B (ja)
DE (1) DE112016003473T5 (ja)
WO (1) WO2017022199A1 (ja)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180321673A1 (en) * 2015-07-31 2018-11-08 Panasonic Intellectual Property Management Co., Ltd. Driving support device, driving support system, and driving support method
US20190077414A1 (en) * 2017-09-12 2019-03-14 Harman International Industries, Incorporated System and method for natural-language vehicle control
US10527450B2 (en) * 2016-03-29 2020-01-07 Toyota Motor Engineering & Manufacturing North America, Inc. Apparatus and method transitioning between driving states during navigation for highly automated vechicle

Families Citing this family (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5910904B1 (ja) * 2015-07-31 2016-04-27 パナソニックＩｐマネジメント株式会社 運転支援装置、運転支援システム、運転支援方法、運転支援プログラム及び自動運転車両
EP3548843B1 (en) 2016-11-30 2021-01-20 Nissan North America, Inc. Interface for mapping remote support to autonomous vehicles
BR112019010723A2 (pt) 2016-11-30 2019-10-01 Nissan North America Inc teleoperação de carros autônomos para negociar situações problemáticas
JP2018117320A (ja) * 2017-01-20 2018-07-26 株式会社東芝 電子ミラーの映像合成装置及び映像合成方法
JP6916647B2 (ja) * 2017-03-31 2021-08-11 本田技研工業株式会社 車両制御システム、車両制御方法、および車両制御プログラム
DE102017208931A1 (de) * 2017-05-29 2018-11-29 Volkswagen Aktiengesellschaft Verfahren zum Unterstützen eines Benutzers eines Kraftfahrzeugs bei einer Fahrt mit dem Kraftfahrzeug, Navigationseinrichtung und Kraftfahrzeug
JP6690612B2 (ja) * 2017-08-03 2020-04-28 トヨタ自動車株式会社 車外報知装置
JP7035447B2 (ja) 2017-10-26 2022-03-15 トヨタ自動車株式会社 車両制御装置
JP6745294B2 (ja) * 2018-04-04 2020-08-26 本田技研工業株式会社 車両制御装置、車両制御方法、及びプログラム
CN108334258A (zh) * 2018-04-11 2018-07-27 刘连波 自动驾驶辅助装置、自动驾驶辅助方法以及自动驾驶辅助系统
US10838417B2 (en) 2018-11-05 2020-11-17 Waymo Llc Systems for implementing fallback behaviors for autonomous vehicles
JP7206886B2 (ja) 2018-12-19 2023-01-18 トヨタ自動車株式会社 運転支援装置
JP7205236B2 (ja) * 2019-01-08 2023-01-17 トヨタ自動車株式会社 リモート走行システム及びリモート走行アプリケーションソフト
US11016485B2 (en) 2019-03-28 2021-05-25 Nissan North America, Inc. Teleoperation for exception handling
JP7268481B2 (ja) * 2019-05-22 2023-05-08 トヨタ自動車株式会社 情報処理装置、自律走行車両、情報処理方法及びプログラム
DE102020106380A1 (de) 2020-03-09 2021-09-09 Bayerische Motoren Werke Aktiengesellschaft Verfahren zum Betreiben eines Fahrerassistenzsystems eines Fahrzeugs durch Anpassung von angezeigten Objekten, Fahrerassistenzsystem sowie Fahrzeug
KR20220085860A (ko) * 2020-12-15 2022-06-23 현대자동차주식회사 차량 및 그 제어방법
WO2022205159A1 (zh) * 2021-03-31 2022-10-06 华为技术有限公司 一种车辆控制方法及装置

Citations (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005041433A (ja) 2003-07-25 2005-02-17 Denso Corp 車両誘導装置、および経路判定プログラム
US20060224316A1 (en) 2005-03-31 2006-10-05 Xanavi Informatics Corporation Navigation system
JP2008032629A (ja) 2006-07-31 2008-02-14 Xanavi Informatics Corp ナビゲーション装置
WO2008099483A1 (ja) 2007-02-15 2008-08-21 Pioneer Corporation 表示制御装置、表示制御方法、表示制御プログラムおよび記録媒体
US20100023257A1 (en) 2006-12-11 2010-01-28 Mitsubishi Electric Corporation Navigation apparatus
JP2010198578A (ja) 2009-02-27 2010-09-09 Toyota Motor Corp 移動軌跡生成装置
US20120173069A1 (en) * 2010-12-29 2012-07-05 GM Global Technology Operations LLC Vehicle operation and control system for autonomous vehicles on full windshield display
US20120310465A1 (en) * 2011-06-02 2012-12-06 Harman International Industries, Incorporated Vehicle nagivation system
US20130289858A1 (en) * 2012-04-25 2013-10-31 Alain Anthony Mangiat Method for controlling and communicating with a swarm of autonomous vehicles using one-touch or one-click gestures from a mobile platform
US20140088855A1 (en) * 2012-09-27 2014-03-27 Google Inc. Determining changes in a driving environment based on vehicle behavior
JP2014074627A (ja) 2012-10-03 2014-04-24 Denso Corp 車両用ナビゲーションシステム
US20140278029A1 (en) * 2013-03-15 2014-09-18 Carnegie Mellon University Methods And Software For Managing Vehicle Priority In A Self-Organizing Traffic Control System
US20140365228A1 (en) * 2013-03-15 2014-12-11 Honda Motor Co., Ltd. Interpretation of ambiguous vehicle instructions
JP2015011458A (ja) 2013-06-27 2015-01-19 株式会社デンソー 車両用情報提供装置
US20150244826A1 (en) * 2014-02-25 2015-08-27 Here Global B.V. Method and apparatus for providing selection and prioritization of sensor data
WO2015141308A1 (ja) 2014-03-18 2015-09-24 日産自動車株式会社 車両操作装置
US9581460B1 (en) * 2016-03-29 2017-02-28 Toyota Motor Engineering & Manufacturing North America, Inc. Apparatus and method transitioning between driving states during navigation for highly automated vechicle
US20170197637A1 (en) * 2015-07-31 2017-07-13 Panasonic Intellectual Property Management Co., Ltd. Driving support device, driving support system, driving support method, and automatic drive vehicle
US20170203763A1 (en) * 2015-07-31 2017-07-20 Panasonic Intellectual Property Management Co., Ltd. Driving support device, driving support system and driving support method
US20170225702A1 (en) * 2015-07-31 2017-08-10 Panasonic Intellectual Property Management Co., Ltd. Driving support device, driving support system, and driving support method

Family Cites Families (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005307491A (ja) * 2004-04-20 2005-11-04 Yanmar Co Ltd 作業車用誤操作防止装置
JP2008072412A (ja) * 2006-09-14 2008-03-27 Fujifilm Corp 撮影装置および方法並びにプログラム
JP2008099483A (ja) * 2006-10-13 2008-04-24 Matsushita Electric Ind Co Ltd モータ駆動装置およびこれを具備する冷蔵庫
JP5182260B2 (ja) * 2009-09-02 2013-04-17 ソニー株式会社 操作制御装置、操作制御方法およびコンピュータプログラム
JP2011095238A (ja) * 2009-09-30 2011-05-12 Aisin Aw Co Ltd ナビゲーション装置及びプログラム
FR2957266B1 (fr) * 2010-03-11 2012-04-20 Parrot Procede et appareil de telecommande d'un drone, notamment d'un drone a voilure tournante.
WO2013021685A1 (ja) * 2011-08-11 2013-02-14 株式会社Ｊｖｃケンウッド 車両における操作対象装置の制御装置及び制御方法、ステアリングホイール
DE102012002304A1 (de) * 2012-02-06 2013-08-08 Audi Ag Vorrichtung zum automatisierten Führen eines Kraftwagens und Verfahren zum Betreiben eines Kraftwagens
US9429943B2 (en) * 2012-03-05 2016-08-30 Florida A&M University Artificial intelligence valet systems and methods
CN104487308B9 (zh) * 2012-07-24 2017-07-07 丰田自动车株式会社 驾驶辅助装置
US9126694B2 (en) * 2013-07-15 2015-09-08 Honeywell International Inc. Display systems and methods for providing displays having an integrated autopilot functionality
US9505383B2 (en) * 2013-10-29 2016-11-29 Medallion Instrumentation Systems, Llc Removable vehicle operation instrument with remote control capability and related method
EP2902864B1 (en) * 2014-01-30 2017-05-31 Volvo Car Corporation Control arrangement for autonomously driven vehicle
CN103971128B (zh) * 2014-05-23 2017-04-05 北京理工大学 一种面向无人驾驶车的交通标志识别方法
EP3624081A1 (en) * 2014-10-30 2020-03-18 Mitsubishi Electric Corporation In-vehicle device and autonomous vehicle including such an in-vehicle device
EP3240714B1 (en) * 2014-12-29 2023-08-30 Robert Bosch GmbH Systems and methods for operating autonomous vehicles using personalized driving profiles
JP5957744B1 (ja) * 2015-07-31 2016-07-27 パナソニックＩｐマネジメント株式会社 運転支援装置、運転支援システム、運転支援方法、運転支援プログラム及び自動運転車両
US10824985B2 (en) * 2016-03-09 2020-11-03 Amazon Technologies, Inc. Interactive mobile pick-up unit notification
KR101747375B1 (ko) * 2016-09-12 2017-06-15 아이피랩 주식회사 차량 원격제어 장치 및 방법, 원격 운전 시스템
US10257582B2 (en) * 2017-03-17 2019-04-09 Sony Corporation Display control system and method to generate a virtual environment in a vehicle
US10496362B2 (en) * 2017-05-20 2019-12-03 Chian Chiu Li Autonomous driving under user instructions
CA3072776A1 (en) * 2017-08-25 2019-02-28 Cubic Corporation Remote operation of non-driving functionality autonomous vehicles

Patent Citations (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005041433A (ja) 2003-07-25 2005-02-17 Denso Corp 車両誘導装置、および経路判定プログラム
US20060224316A1 (en) 2005-03-31 2006-10-05 Xanavi Informatics Corporation Navigation system
JP2006284218A (ja) 2005-03-31 2006-10-19 Xanavi Informatics Corp ナビゲーション装置
JP2008032629A (ja) 2006-07-31 2008-02-14 Xanavi Informatics Corp ナビゲーション装置
US20100023257A1 (en) 2006-12-11 2010-01-28 Mitsubishi Electric Corporation Navigation apparatus
JPWO2008072412A1 (ja) 2006-12-11 2010-03-25 三菱電機株式会社 ナビゲーション装置
WO2008099483A1 (ja) 2007-02-15 2008-08-21 Pioneer Corporation 表示制御装置、表示制御方法、表示制御プログラムおよび記録媒体
US20100030462A1 (en) 2007-02-15 2010-02-04 Pioneer Corporation Display control apparatus, display control method, display control program, and recording medium
JP2010198578A (ja) 2009-02-27 2010-09-09 Toyota Motor Corp 移動軌跡生成装置
US20120022716A1 (en) 2009-02-27 2012-01-26 Toyota Jidosha Kabushiki Kaisha Movement trajectory generator
US20120173069A1 (en) * 2010-12-29 2012-07-05 GM Global Technology Operations LLC Vehicle operation and control system for autonomous vehicles on full windshield display
US20120310465A1 (en) * 2011-06-02 2012-12-06 Harman International Industries, Incorporated Vehicle nagivation system
US20130289858A1 (en) * 2012-04-25 2013-10-31 Alain Anthony Mangiat Method for controlling and communicating with a swarm of autonomous vehicles using one-touch or one-click gestures from a mobile platform
US20140088855A1 (en) * 2012-09-27 2014-03-27 Google Inc. Determining changes in a driving environment based on vehicle behavior
JP2014074627A (ja) 2012-10-03 2014-04-24 Denso Corp 車両用ナビゲーションシステム
US20140278029A1 (en) * 2013-03-15 2014-09-18 Carnegie Mellon University Methods And Software For Managing Vehicle Priority In A Self-Organizing Traffic Control System
US20140365228A1 (en) * 2013-03-15 2014-12-11 Honda Motor Co., Ltd. Interpretation of ambiguous vehicle instructions
JP2015011458A (ja) 2013-06-27 2015-01-19 株式会社デンソー 車両用情報提供装置
US20150244826A1 (en) * 2014-02-25 2015-08-27 Here Global B.V. Method and apparatus for providing selection and prioritization of sensor data
WO2015141308A1 (ja) 2014-03-18 2015-09-24 日産自動車株式会社 車両操作装置
US20170197637A1 (en) * 2015-07-31 2017-07-13 Panasonic Intellectual Property Management Co., Ltd. Driving support device, driving support system, driving support method, and automatic drive vehicle
US20170203763A1 (en) * 2015-07-31 2017-07-20 Panasonic Intellectual Property Management Co., Ltd. Driving support device, driving support system and driving support method
US20170225702A1 (en) * 2015-07-31 2017-08-10 Panasonic Intellectual Property Management Co., Ltd. Driving support device, driving support system, and driving support method
US9581460B1 (en) * 2016-03-29 2017-02-28 Toyota Motor Engineering & Manufacturing North America, Inc. Apparatus and method transitioning between driving states during navigation for highly automated vechicle

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
International Search Report of PCT application No. PCT/JP2016/003383 dated Oct. 25, 2016.

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180321673A1 (en) * 2015-07-31 2018-11-08 Panasonic Intellectual Property Management Co., Ltd. Driving support device, driving support system, and driving support method
US10656639B2 (en) * 2015-07-31 2020-05-19 Panasonic Intellectual Property Management Co., Ltd. Driving support device, driving support system, and driving support method
US10527450B2 (en) * 2016-03-29 2020-01-07 Toyota Motor Engineering & Manufacturing North America, Inc. Apparatus and method transitioning between driving states during navigation for highly automated vechicle
US20190077414A1 (en) * 2017-09-12 2019-03-14 Harman International Industries, Incorporated System and method for natural-language vehicle control
US10647332B2 (en) * 2017-09-12 2020-05-12 Harman International Industries, Incorporated System and method for natural-language vehicle control

Also Published As

Publication number Publication date
US20180321673A1 (en) 2018-11-08
JP2017032412A (ja) 2017-02-09
CN107851394B (zh) 2021-12-28
US10656639B2 (en) 2020-05-19
DE112016003473T5 (de) 2018-04-12
WO2017022199A1 (ja) 2017-02-09
CN107851394A (zh) 2018-03-27
JP5957744B1 (ja) 2016-07-27
US20170364066A1 (en) 2017-12-21

Similar Documents

Publication Publication Date Title
US10656639B2 (en) Driving support device, driving support system, and driving support method
US10604158B2 (en) Driving support device, driving support system and driving support method
US10532763B2 (en) Driving support device, driving support system, and driving support method
US10435033B2 (en) Driving support device, driving support system, driving support method, and automatic drive vehicle
US10160451B2 (en) Driving support device, driving support system and driving support method
US10017189B2 (en) Driving support device, driving support system, and driving support method
JP6621032B2 (ja) 運転支援装置、運転支援システム、運転支援方法、運転支援プログラム及び自動運転車両
JP6575915B2 (ja) 運転支援装置、運転支援システム、運転支援方法、運転支援プログラム及び自動運転車両
JP6090727B2 (ja) 運転支援装置、運転支援システム、運転支援方法、運転支援プログラム及び自動運転車両
JP6681604B2 (ja) 運転支援装置、運転支援システム、運転支援方法、運転支援プログラム及び自動運転車両
JP6671019B2 (ja) 運転支援装置、運転支援システム、運転支援方法、運転支援プログラム、自動運転車両及び自動運転制御システム
JP6558738B2 (ja) 運転支援装置、運転支援システム、運転支援方法、運転支援プログラム及び自動運転車両

Legal Events

Date Code Title Description
AS Assignment

Owner name: PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LT

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:YAMADA, KAZUSHIGE;YAMAJI, OSAMU;GOJYO, AZUSA;AND OTHERS;SIGNING DATES FROM 20161111 TO 20161118;REEL/FRAME:042005/0630

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4

AS Assignment

Owner name: PANASONIC AUTOMOTIVE SYSTEMS CO., LTD., JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LTD.;REEL/FRAME:066703/0113

Effective date: 20240207