JP6575915B2 - Driving support device, driving support system, driving support method, driving support program, and autonomous driving vehicle - Google Patents

Description

  The present invention relates to a technique for assisting a driver with a driving operation instruction to a vehicle during automatic driving.

  In recent years, development related to automatic driving of automobiles has been promoted. As for automatic operation, the automation levels defined in 2013 by NHTSA (National Highway Traffic Safety Administration) are no automation (level 0), automation of specific functions (level 1), automation of complex functions (level 2), semi-automatic operation ( Level 3) and fully automatic operation (level 4). Level 1 is a driving support system that automatically performs one of acceleration, deceleration, and steering. Level 2 is a driving support system that automatically performs two or more of acceleration, deceleration, and steering in harmony. is there. In either case, the driver remains involved in the driving operation. The automation level 4 is a fully automatic traveling system that automatically performs all of acceleration, deceleration, and steering, and the driver is not involved in the driving operation. The automation level 3 is a quasi-fully automatic traveling system in which acceleration, deceleration, and steering are all automatically performed, and a driver performs a driving operation as necessary.

  As a form of automatic driving, the driver does not operate the existing driving operation unit such as steering and accelerator pedal, but by issuing commands to the vehicle side, specific driving operation such as lane change, overtaking, following driving, etc. Can be considered to the vehicle side. In this form, a user interface with few erroneous operations is required. There is known one that cuts out a guide sign from an original image and selectively displays the guide sign based on the traveling direction or the like (for example, see Patent Document 1).

JP 2014-074627 A

  However, when it is applied to automatic driving, it takes time to confirm the sign and to indicate the course using another means (steering, accelerator, etc.) in another location. In addition, in a design in which a switch is provided for each specific driving operation such as lane change, overtaking, and follow-up driving, there is a possibility of erroneous operation when the switch is operated without looking at the hand while checking the situation around the vehicle. Also, the correspondence between switch operation and automatic driving control is not intuitive, and selecting instructions from many switches or selecting functions that change according to the situation by reducing the number of switches is a complicated task. However, there is a problem that a wide range of users including those who have not driven until now and those who want to continue driving even if their driving ability has declined cannot use an autonomous driving vehicle without training.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide a technique by which a driver can instruct a vehicle to perform a specific driving operation intuitively and simply.

  In order to solve the above-described problem, a driving support device according to an aspect of the present invention includes an image output unit that outputs image information including a vehicle object representing the vehicle and a sign object representing sign information on the display unit, An operation signal input unit for receiving an operation signal of a user for moving the vehicle object in the image displayed on the display unit to the position of the sign object or the sign object to the position of the vehicle object; A command output unit that outputs an automatic driving control command corresponding to the display content in the sign object to an automatic driving control unit that controls automatic driving.

  Note that any combination of the above-described components, the expression of the present invention converted between an apparatus, a system, a method, a program, a recording medium on which the program is recorded, and an autonomous driving vehicle equipped with the same are also included in the present invention. It is effective as an embodiment of

  According to the present invention, the driver can instruct the vehicle to perform a specific driving operation intuitively and simply.

It is a block diagram which shows the structure of the vehicle which concerns on embodiment of this invention. It is a figure which shows an example of the basic sequence of the detection part of FIG. 1, an automatic driving controller, an HMI controller, a display part, and an input part. It is a figure which shows an example of the basic flowchart for demonstrating the process of the HMI controller of FIG. It is a flowchart for demonstrating the update process of reception mode. FIG. 4 is a flowchart illustrating an example of a determination process when a gesture operation for instructing a specific operation is input using a road guide sign in step S9 of FIG. 3. It is a flowchart which shows the process example which issues the command according to the content of the road guidance sign by gesture operation. FIGS. 7A and 7B are diagrams illustrating an example in which a route change instruction command or a route change instruction command is issued using an intersection destination guide sign. FIGS. 8A to 8B are diagrams illustrating an example in which a route change instruction command or a route change instruction command is issued using a lane guide sign. FIGS. 9A to 9B are diagrams illustrating an example in which a setting instruction command for setting a waypoint is issued by using an exit guide sign on a toll road. FIGS. 10A and 10B are diagrams showing another example in which a setting instruction command for setting a waypoint is issued using a toll road exit guide sign. FIGS. 11A and 11B are diagrams illustrating an example of issuing a destination setting instruction command for setting a destination using a parking lot guide sign. FIGS. 12A to 12C are diagrams illustrating another example of issuing a route change instruction command using a destination guide sign at an intersection. FIGS. 13A to 13B are diagrams illustrating an example in which a lane change instruction command and a course change instruction command are issued using a guide sign on a lane.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a vehicle 1 according to an embodiment of the present invention, which is a configuration related to automatic driving. A vehicle 1 (own vehicle) equipped with an automatic driving mode includes a driving support device 10, an automatic driving control device 20, a display device 30, a detection unit 40, and a driving operation unit 50.

  The display device 30 includes a display unit 31 and an input unit 32. The display device 30 may be a head unit such as a car navigation system or display audio, may be a mobile terminal device such as a smartphone or a tablet, or may be a dedicated console terminal device.

  The display unit 31 is a liquid crystal display, an organic EL display, or a head-up display (HUD). The input unit 32 is a user interface that receives user input. The display unit 31 and the input unit 32 may be an integrated touch panel display. In addition, it can accept gesture input at a predetermined distance from the display unit, such as a proximity touch panel that can detect the proximity of the hand on the touch panel or touch pad and the position of the finger by a hover operation. Also good. Furthermore, the input unit 32 may include an input device that assists gesture input, such as a mouse, a stylus pen, or a trackball. A pen that emits visible light or infrared light may be used.

  Note that the display unit 31 and the input unit 32 may be physically separated instead of an integrated touch panel display. For example, the input unit 32 may be a non-contact type input device that includes a sensor such as a camera and is capable of inputting an air gesture operation. For example, an operation method such as pointing with a pointing object, starting dragging with the gesture of closing the thumb and index finger close, and ending dragging with the gesture of releasing the thumb and index finger can be considered.

  The driving support device 10 and the display device 30 may be connected by a wired communication such as a dedicated line or a CAN (Controller Area Network), or may be USB, Ethernet (registered trademark), Wi-Fi (registered trademark). , Bluetooth (registered trademark) or the like may be connected by wired communication or wireless communication.

  The detection unit 40 includes a position information acquisition unit 41, a sensor 42, a speed information acquisition unit 43, and a map information acquisition unit 44. The position information acquisition unit 41 acquires the current position of the vehicle 1 from the GPS receiver. The sensor 42 is a general term for various sensors for detecting the situation outside the vehicle and the state of the vehicle 1. For example, a camera, millimeter wave radar, LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging), a temperature sensor, an atmospheric pressure sensor, a humidity sensor, an illuminance sensor, and the like are mounted as sensors for detecting a situation outside the vehicle. The situation outside the vehicle may be the road situation where the host vehicle including the sign information, the environment including the weather, and the surrounding situation of the host vehicle are considered. Any information outside the vehicle that can be detected by the sensor may be used. The sign information in the present embodiment includes road guidance signs, intersection destination guidance signs, lane guidance signs, toll road exit guidance signs, and the like, but is not limited thereto. For example, an acceleration sensor, a gyro sensor, a geomagnetic sensor, a tilt sensor, and the like are mounted as sensors for detecting the state of the vehicle 1. The speed information acquisition unit 43 acquires the current speed of the vehicle 1 from the vehicle speed sensor. The map information acquisition unit 44 acquires map information around the current position of the vehicle 1 from the map database. The map database may be recorded on a recording medium in the vehicle 1, or may be downloaded from a map server via a network when used.

  The detection unit 40 and the automatic operation control device 20 are connected by wired communication such as a dedicated line, USB, Ethernet (registered trademark), CAN (Controller Area Network) or the like. The data acquired and detected by the detection unit 40 may be directly output from the detection unit 40 to the driving support device 10.

  The driving operation unit 50 includes a steering 51, a brake pedal 52, an accelerator pedal 53, and a winker switch 54. In the automatic driving mode according to the present embodiment, acceleration, deceleration, steering, and blinker blinking are objects of automatic control by the automatic driving control device 20, and FIG. 1 shows an operation unit for performing these controls manually. I'm drawing. In addition, you may output the information which the driver | operator moved the driving operation part 50 a little manually to the driving assistance apparatus 10. FIG.

  The steering 51 is an operation unit for steering the vehicle. When the steering wheel 51 is rotated by the driver, the traveling direction of the vehicle is controlled via the steering actuator. The steering actuator can be electronically controlled by a steering ECU (Electronic Control Unit).

  The brake pedal 52 is an operation unit for decelerating the vehicle 1. When the brake pedal 52 is depressed by the driver, the vehicle is decelerated through the brake actuator. The brake actuator can be electronically controlled by the brake ECU.

  The accelerator pedal 53 is an operation unit for accelerating the vehicle 1. When the accelerator pedal 53 is depressed by the driver, the engine speed and / or the motor speed are controlled via the accelerator actuator. The engine speed is controlled in a pure engine car, the motor speed is controlled in a pure electric car, and both are controlled in a hybrid car. The accelerator actuator can be electronically controlled by the engine ECU and / or the motor ECU.

  The turn signal switch 54 is an operation unit for blinking a turn signal for notifying the outside of the vehicle path. When the turn signal switch 54 is turned on / off by the driver, the turn signal is turned on / off via the turn signal controller. The winker controller includes a drive circuit such as a relay for controlling power feeding to the winker lamp.

  The steering ECU, the brake ECU, the engine ECU, the motor ECU, and the winker controller are connected to the automatic driving control device 20 by wired communication such as a CAN or a dedicated line. The steering ECU, the brake ECU, the engine ECU, the motor ECU, and the winker controller respectively transmit state signals indicating the states of the steering, the brake, the engine, the motor, and the winker lamp to the automatic driving control device 20.

  In the automatic operation mode, the steering ECU, the brake ECU, the engine ECU, and the motor ECU drive each actuator in accordance with a control signal supplied from the automatic operation control device 20. In the manual operation mode, the steering 51, the brake pedal 52, and the accelerator pedal 53 may be mechanically transmitted directly to each actuator, or may be configured to be electronically controlled via each ECU. It may be. The winker controller turns on / off the winker lamp in response to a control signal supplied from the automatic operation control device 20 or an instruction signal from the winker switch 54.

  The automatic operation control device 20 is an automatic operation controller that implements an automatic operation control function, and includes a control unit 21, a storage unit 22, and an input / output unit 23. The configuration of the control unit 21 can be realized by cooperation of hardware resources and software resources, or only by hardware resources. Processors, ROM, RAM, and other LSIs can be used as hardware resources, and programs such as an operating system, application, and firmware can be used as software resources. The storage unit 22 includes a nonvolatile recording medium such as a flash memory. The input / output unit 23 executes various communication controls corresponding to various communication formats.

  The control unit 21 applies various parameter values collected from the detection unit 40 and various ECUs to the automatic driving algorithm, and calculates a control value for controlling an automatic control target such as the traveling direction of the vehicle 1. The control unit 21 transmits the calculated control value to each control target ECU or controller. In this embodiment, it is transmitted to a steering ECU, a brake ECU, an engine ECU, and a winker controller. In the case of an electric vehicle or a hybrid car, the control value is transmitted to the motor ECU instead of or in addition to the engine ECU.

  The driving support apparatus 10 is an HMI (Human Machine Interface) controller for executing an interface function between the vehicle 1 and the driver, and includes a determination unit 11, a generation unit 12, an instruction unit 13, and an input / output unit 14. . The determination unit 11, the generation unit 12, and the instruction unit 13 can be realized by cooperation of hardware resources and software resources, or only by hardware resources. Processors, ROM, RAM, and other LSIs can be used as hardware resources, and programs such as an operating system, application, and firmware can be used as software resources. The input / output unit 14 executes various communication controls according to various communication formats. The input / output unit 14 includes an image output unit 14a, an operation signal input unit 14b, a command output unit 14c, and a vehicle information input unit 14d. The image output unit 14 a outputs the image generated by the generation unit 12 to the display unit 31. The operation signal input unit 14 b receives an operation signal generated by the driver, an occupant, or a user outside the vehicle made to the input unit 32 and outputs the operation signal to the determination unit 11. The command output unit 14 c outputs the command instructed by the instruction unit 13 to the automatic operation controller 20. The vehicle information input unit 14 d receives detection data acquired by the detection unit 40 and vehicle-side information generated by the automatic driving controller 20 and outputs the information to the generation unit 12.

  The automatic operation controller 20 and the HMI controller 10 are directly connected by a signal line. In addition, the structure connected via CAN may be sufficient. Further, a configuration is possible in which the automatic operation controller 20 and the HMI controller 10 are integrated into one controller.

  FIG. 2 is a diagram illustrating an example of a basic sequence of the detection unit 40, the automatic operation controller 20, the HMI controller 10, the display unit 31, and the input unit 32 of FIG. The detection unit 40 detects the vehicle position information, the vehicle traveling road information including the sign information, and the vehicle periphery information and outputs the detected information to the automatic driving controller 20 (P1). The automatic driving controller 20 outputs the vehicle position information, the vehicle traveling road information, and the vehicle surrounding information acquired from the detection unit 40 to the HMI controller 10 (P2). The HMI controller 10 generates a schematic diagram including the host vehicle and the surrounding situation of the host vehicle based on the information acquired from the automatic driving controller 20 (P3). The HMI controller 10 outputs the generated schematic diagram to the display device 30 and displays it on the display unit 31 (P4).

  The user who sees the schematic diagram displayed on the display unit 31 contacts the input unit 32 (P5). The display unit 31 outputs the coordinate data in which contact is detected to the HMI controller 10 (P6). The HMI controller 10 determines the type of command based on the coordinate data acquired from the display device 30 (P7). The HMI controller 10 accepts additional inputs until a predetermined time has elapsed (P8 to P12). After determining the command, the HMI controller 10 regenerates a schematic diagram indicating that the command is being instructed (P8). The HMI controller 10 outputs the regenerated schematic diagram to the display device 30 and displays it on the display unit 31 (P9). When there is no command corresponding to the gesture operation by the user's contact, a schematic diagram including an error message is generated and output to the display device 30 and displayed on the display unit 31.

  When the user who sees the schematic diagram showing that the command is being instructed touches the input unit 32 (P10), the display unit 31 outputs the coordinate data in which the contact is detected to the HMI controller 10 (P11). The HMI controller 10 executes additional command processing based on the coordinate data acquired from the display device 30 (P12). If there is no new command input in the additional command processing (P12), the HMI controller 10 issues the command determined in P7 to the automatic operation controller 20 (P13, P14). When a new command is input in the additional command process (P12), the new command is issued to the automatic operation controller 20. If the newly input command is a cancel command, cancel the command issue. The overwriting / cancellation processing of the original command by a new command may be performed by the automatic operation controller 20, and in that case, the HMI controller 10 transmits the command to the automatic operation controller 20 after the command determination processing in P7 and P12. The overwriting / cancellation process is performed according to the state of the automatic operation controller 20.

  The detection unit 40 periodically detects the vehicle position information, the vehicle traveling road information, and the vehicle periphery information and outputs the detected information to the automatic driving controller 20 (P15). Based on the information, the automatic operation controller 20 determines whether or not the control instructed by the command issued from the HMI controller 10 can be executed (P16). If it can be executed, the automatic operation controller 20 outputs a control start notification to the HMI controller 10 (P17). Upon receiving the control start notification, the HMI controller 10 regenerates a schematic diagram including a message indicating that control is being performed (P18). The HMI controller 10 outputs the regenerated schematic diagram to the display device 30 and displays it on the display unit 31 (P19). Although not shown, the automatic operation controller 20 applies various parameter values collected from the detection unit 40 and various ECUs to the automatic operation algorithm, and performs automatic control such as the traveling direction of the vehicle 1 for executing the issued command. A specific control value for controlling the driving operation unit 50 is calculated and transmitted to each ECU or controller to be controlled. The driving operation unit 50 operates based on specific control values. The driving operation unit 50 determines that the command execution is complete when the predetermined control value or the detection data of the detection unit 40 reaches a predetermined value (range) and the automatic operation controller 20 determines that the condition of the issued command is satisfied. .

  When receiving the control completion notification from the automatic operation controller 20, the HMI controller 10 generates a schematic diagram including a message indicating that the control is completed and outputs the schematic diagram to the display device 30. During a period in which no operation is accepted from the user, a schematic diagram including a message indicating that no operation is accepted is generated and output to the display device 30.

  FIG. 3 is a diagram showing an example of a basic flowchart for explaining the processing of the HMI controller 10 of FIG. The determination unit 11 of the HMI controller 10 checks whether the operation mode is the automatic operation mode or the manual operation mode (S1). In the case of the manual operation mode (N in S2), the process is terminated. In the case of the automatic operation mode (Y in S2), the following process is performed.

  The sensor information input from the detection unit 40 to the automatic operation controller 20 is updated as needed (S3). The generation unit 12 of the HMI controller 10 includes the own vehicle position information, the own vehicle traveling road information including the sign information, and the surrounding conditions of the own vehicle and the own vehicle based on the surrounding information of the own vehicle. A schematic diagram is generated, and the generated schematic diagram is drawn on the display unit 31 (S4). The determination unit 11 checks whether the reception mode is an acceptable mode in which an operation from the user can be accepted or an unacceptable mode in which an operation from the user cannot be accepted (S5). If it is in the unacceptable mode (N in S6), the process ends. When it is in the acceptable mode (Y in S6), the determination unit 11 determines whether or not there is a user's contact with the input unit 32 (S7). When there is no contact (N of S8), the fixed time passage determination process (S12) described later is executed. When there is contact (Y in S8), the determination unit 11 determines a control command in accordance with a gesture operation input by the user (S9). Details of this determination process will be described later.

  When the control command determined in step S9 is not a cancel command (N in S10), the generation unit 12 displays a command instruction on the display unit 31 (S11). When a certain time has elapsed after the control command is determined (Y in S12), if there is a control command determined in step S9 (Y in S13), the display unit 31 displays that the operation cannot be accepted (S14), and the determination unit 11 updates the acceptance mode from the acceptable mode to the unacceptable mode (S15), and the instruction unit 13 outputs the determined control command to the automatic operation controller 20 (S16). Until a predetermined time elapses (N in S12), the process proceeds to step S3.

  In step S10, when the determined control command is a cancel command (Y in S10), cancel is displayed (S110), and the process ends. In step S13, if there is no control command determined in step S9, an input error is displayed (S111), and the process ends. The automatic operation controller 20 periodically detects the vehicle position information, the vehicle traveling road information, and the vehicle surrounding information from the detection unit 40. Since the surrounding situation changes from moment to moment, it may be determined that the control command cannot be executed after the control command is output to the automatic operation controller 20. For example, there is a case where another vehicle has interrupted between the own vehicle and another vehicle after the follow-up instruction. When the automatic operation controller 20 determines that the control command can be executed (Y in S17), the generation unit 12 displays on the display unit 31 that the control is being performed (S18), activates a timer, and starts counting. (S19). When the automatic operation controller 20 determines that control is impossible (N in S17), the generation unit 12 displays an uncontrollable error on the display unit 31 (S112).

  FIG. 4 is a flowchart for explaining the reception mode update process. When the timer count value reaches a set value (for example, 10 seconds) (Y in S113), the determination unit 11 of the HMI controller 10 updates the reception mode from the unacceptable mode to the acceptable mode (S114). The count value of the timer may be changed according to the surrounding situation of the own vehicle. When the determination unit 11 receives a notification indicating the completion of control from the automatic driving controller 20 or determines from the behavior of the vehicle 1 that the control according to the control command is completed, the reception mode can be received from the unacceptable mode. You may update to mode.

  Hereinafter, in the present embodiment, an example in which a control command for instructing a specific operation using a road guide sign is issued as a control command will be described. The user inputs a gesture operation for instructing a specific operation to the input unit 32 using a road guide sign. A specific example of the gesture operation will be described later.

  FIG. 5 is a flowchart illustrating an example of determination processing when a gesture operation that instructs a specific operation using a road guide sign is input in step S9 of FIG. The determination unit 11 of the HMI controller 10 determines whether or not the vehicle mark is present at the contact start position (S9a). When the own vehicle mark does not exist (N of S9a), the determination unit 11 determines that the control command does not use the road guide sign (S9b). When the vehicle mark is present (Y in S9a), the generation unit 12 draws a droppable area in the schematic diagram and displays it on the display unit 31 (S9c). In that case, the production | generation part 12 produces | generates the dropable area of a road guide sign in a schematic diagram. Information regarding the dropable area of the road sign is acquired from a local map information database prepared in advance or a map information database on the network. Moreover, you may acquire object coordinates, such as a place name and an arrow in a sign, from the road guidance sign image image | photographed live by the vehicle-mounted camera using an image recognition technique.

  The determination unit 11 receives the touch event generated at the input unit 32 (S9d), and determines the type of the touch event (S9e). When the touch event type is movement (movement of S9e), the generation unit 12 draws the predicted trajectory / route candidate of the vehicle 1 in the schematic diagram and causes the display unit 31 to display the candidate (S9f).

  When the touch event type is contact end (contact end of S9e), the determination unit 11 determines whether or not the contact end position is an area of a road guide sign (S9g). When the area is not a road information sign area (N in S9g), the determination unit 11 determines that the control command does not use the road information sign (S9h). When the contact end position is an area of a road guide sign (Y in S9g), the determination unit 11 determines whether the road guide sign area is an area of a lane guide or an uphill lane sign (S9i). If the area is a sign for a lane guide or an uphill lane (Y in S9i), it is determined whether the corresponding lane is a right turn lane or a left turn lane (S9j). If the lane is not a right turn lane or a left turn lane (N in S9j), the determination unit 11 determines a lane change instruction command to the lane (S9k). When it is a right turn lane or a left turn lane (Y in S9j), the determination unit 11 determines a lane change instruction command to the lane and a right turn instruction or a left turn instruction command (S91).

  In step S9i, when the dropped area is not a lane guidance or an uphill lane sign area (N in S9i), the determination unit 11 determines whether the dropped area is an area in which an arrow indicating a direction / direction is displayed. (S9m). When it is an area where an arrow indicating a direction / direction is displayed (Y in S9m), the determination unit 11 determines that the command is an instruction to change the traveling direction in the direction (S9n). When the dropped area is not an area where an arrow indicating the direction / direction is displayed (N in S9m), the determination unit 11 determines whether the dropped area is an area where a place name indicating a city / region is displayed. (S9o). When the place name indicating the city / region is displayed (Y in S9o), the determination unit 11 acquires the traveling direction corresponding to the place name from the map information database (S9p). The determination part 11 determines with the acquired traveling direction change instruction command to the traveling direction (S9q).

  In step S9o, if the dropped area is not an area where a place name indicating a city / region is displayed (N in S9o), the determination unit 11 determines whether the dropped area is an area where a road name is displayed. (S9r). When the road name is displayed in the area (Y in S9r), the generation unit 12 generates an image (for example, a menu or a dialog) or a sound requesting selection of uplink and downlink and notifies the user (S9s). When uplink is selected by the user (Y in S9t), the determination unit 11 determines that the command is an upward direction change direction command (S9u). When downlink is selected by the user (N of S9t), the determination unit 11 determines that the command is a direction change instruction command in the downward direction (S9v).

  In step S9r, when the dropped area is not the area where the road name is displayed (N in S9r), the determination unit 11 displays a sign indicating that the dropped area is an entrance, exit, parking area or service area of the toll road. It is determined whether or not the area is the same (S9w). When it is an area where a sign indicating a toll road entrance, exit, parking area or service area is displayed (Y in S9w), the determination unit 11 passes through the corresponding toll road entrance, exit, parking area or service area. Is determined to be a setting instruction command for setting to (S9x).

  When the dropped area is not an area where the sign of the entrance, exit, parking area or service area of the toll road is displayed (N in S9w), the determination unit 11 determines whether the dropped area is a parking lot, a rest area, or an emergency call. It is determined whether the area is where the sign is displayed (S9y). When the parking lot, the rest area, or the area where the emergency telephone sign is displayed (Y of S9y), the determination unit 11 sets the road shoulder where the parking area, the rest area, or the emergency telephone is located as the destination. The setting command is determined (S9z). In step S9y, when the dropped area is not a parking lot, a shelter, or an emergency telephone sign (N in S9y), the generation unit 12 displays an error message on the display unit 31 (S9zz).

  Hereinafter, a specific example of the gesture operation using the road information sign will be described. In the following example, it is assumed that a touch panel display in which the display unit 31 and the input unit 32 are integrated is used.

  FIG. 6 is a flowchart illustrating an example of a process for issuing a command corresponding to the content of a road guide sign by a gesture operation. The determination unit 11 of the HMI controller 10 receives a touch event (DOWN) from the touch panel (S50). The touch event (DOWN) is an event indicating a change from a non-contact state to a contact state with a finger or a pen on the touch panel. The determination unit 11 determines whether or not the coordinate at which the touch event (DOWN) is detected is within the display area of the vehicle icon (S51). When it is outside the display area (N in S51), it is determined that the operation instruction is not in accordance with the content of the road guide sign, and the process is terminated.

  If it is inside the display area (Y in S51), the determination unit 11 receives a touch event (MOVE) from the touch panel (S52). A touch event (MOVE) is an event that represents a change from a contact state of a point on the touch panel with a finger or a pen to a contact state of another point. Thereafter, the determination unit 11 receives a touch event (UP) from the touch panel (S53). The touch event (UP) is an event representing a change from a contact state with a finger or a pen to the touch panel to a non-contact state.

  The determination unit 11 determines whether or not the coordinates at which the touch event (UP) is detected are within the display area of the road information sign (S54). If it is inside the display area (Y in S54), the instruction unit 13 issues a command corresponding to the content of the road guide sign to the automatic operation controller 20 (S55). When it is outside the display area (N in S54), it is determined that the operation instruction is not in accordance with the content of the road guide sign, and the process is terminated.

  FIGS. 7A and 7B are diagrams illustrating an example in which a route change instruction command or a route change instruction command is issued using an intersection destination guide sign. In the schematic diagram shown in FIG. 7A, the host vehicle icon V1, the road, and the destination guide sign S1 are displayed. Various display forms are conceivable for the display form of the vehicle, the road, and the guide sign. A live-action image may be used, or a precise CG image or animation image may be used. The display of the own vehicle is not limited to icons, and may be displayed with simpler marks and characters, or may be displayed with a live-action image. That is, the object may be displayed on the screen in some display form.

  When the driver wants to change the course to any one of the directions displayed in the intersection destination guide sign S1, the vehicle icon V1 is dragged as shown in FIG. 7 (a), and as shown in FIG. 7 (b). The own vehicle icon V1 is dropped on the place name “Machida” N1 of the desired destination in the destination guide sign S1. As a result, a route change instruction command to the route of the destination is issued. Alternatively, a route change instruction command for a route having the destination as a route or destination is issued.

  FIGS. 8A to 8B are diagrams illustrating an example in which a route change instruction command or a route change instruction command is issued using a lane guide sign. When the driver wants to change the course to one of the courses displayed on the lane guidance signs S2a and S2b, drag the own vehicle icon V1 as shown in FIG. The car icon V1 is dropped on a desired lane guide sign “Sagamihara” S2b. As a result, a route change instruction command to the route of the destination is issued. Alternatively, a route change instruction command for a route having the destination as a route or destination is issued.

  FIGS. 9A to 9B are diagrams illustrating an example in which a setting instruction command for setting a waypoint is issued by using an exit guide sign on a toll road. When the driver wants to get off from the next exit of the toll road, the user drags the vehicle icon V1 as shown in FIG. 9A, and the vehicle icon V1 as shown in FIG. Drop it on “Yokohama Machida” S3. Thereby, a waypoint setting instruction command for setting the exit to the waypoint is issued. When the route is changed by setting the waypoint, a route change instruction command is also issued.

  FIGS. 10A and 10B are diagrams showing another example in which a setting instruction command for setting a waypoint is issued using a toll road exit guide sign. When the driver wants to get off from a desired exit on the toll road, the user drags the vehicle icon V1 as shown in FIG. 10 (a), and displays the vehicle icon V1 as a plurality of exit guides as shown in FIG. 10 (b). Of the signs S4a, S4b, and S4c, drop on the desired exit guide sign “Yokohama Machida” S4c. Thereby, a waypoint setting instruction command for setting the exit to the waypoint is issued. When the route is changed by setting the waypoint, a route change instruction command is also issued.

  FIGS. 11A and 11B are diagrams illustrating an example of issuing a destination setting instruction command for setting a destination using a parking lot guide sign. When the driver wants to park his / her own vehicle in the parking lot, the driver drags the own vehicle icon V1 as shown in FIG. 11 (a) and displays the own vehicle icon V1 as shown in FIG. 11 (b). Drop on top. Thereby, a destination setting instruction command for setting the parking lot as a destination is issued. When the destination is set, the automatic operation controller 20 reroutes to a new destination.

  FIGS. 12A to 12C are diagrams illustrating another example of issuing a route change instruction command using a destination guide sign at an intersection. When the driver wants to change the course to any one of the directions displayed in the destination guidance sign S6 at the intersection, the driver drags the vehicle icon V1 as shown in FIG. 12 (a), and as shown in FIG. 12 (b). The own vehicle icon V1 is dropped on the road name “Yuki road” R1 in the destination guide sign S6. When the own vehicle icon V1 is dropped on the road name, as shown in FIG. 12C, the destination display window R2 for the upward direction (Machida direction) and the downward direction (Zama direction) of the road is displayed. The driver touches the destination display window R2 in the upward direction or the downward direction. Thereby, a course change instruction command to the course in the touched direction is issued.

  FIGS. 13A to 13B are diagrams illustrating an example in which a lane change instruction command and a course change instruction command are issued using a guide sign on a lane. When the driver wants to make a right turn, drag the own vehicle icon V1 as shown in FIG. 13 (a), and drop the own vehicle icon V1 onto the right turn dedicated lane sign M1 as shown in FIG. 13 (b). . Thereby, the lane change instruction command and the right turn instruction command to the right turn exclusive lane are issued.

  In the above example, the generation unit 12 leaves the original image (the vehicle icon before the movement) in the schematic diagram until the vehicle icon is dropped after being dragged, and the original image is displayed when the icon is dropped. You may turn it off. In addition, the drag unit 12 may draw a drag trajectory in the schematic diagram with a dotted line from when the vehicle icon is dragged to when it is dropped. Moreover, the generation unit 12 may invert the color of the road until the vehicle icon is dropped after being dragged, and return it to the original color when the icon is dropped.

  In addition, when the vehicle icon is dragged and dropped, the generation unit 12 changes the color of the vehicle icon when there is a non-dropable area (such as an opposite lane) or when an operation (lane change, etc.) is not possible ( Inversion, thinning, etc.). At the time of dropping, the vehicle icon may be returned to the original vehicle position, and an error message such as “operation not possible” may be displayed. Examples of cases where operation is not possible include approach of a following vehicle, lane change prohibition area, overspeed limit, and the like.

  Also, during the period from dragging the vehicle icon to dropping it, the generation unit 12 may change the color of the background such as a road (inverted, lightened, etc.) when the operation is impossible. When the operation becomes possible, the original color is restored. Further, the generation unit 12 changes the color of the non-dropable area from when the own vehicle icon is dragged to when it is dropped. In addition, when the vehicle icon is dragged and dropped, the generation unit 12 notifies an error sound or vibration when there is a drop-impossible area or when an operation is not possible.

  If the operation cannot be started when dragging the own vehicle icon is started, the generation unit 12 may change the color of the own vehicle icon. Alternatively, the drag operation (movement of the own vehicle icon) may be disabled. Also, an error message such as “operation not possible” may be displayed. If the operation cannot be started when dragging of the vehicle icon is started, the generation unit 12 may change the background color such as a road. When the operation becomes possible, the original color is restored. If the operation cannot be started when the drag of the own vehicle icon is started, the generation unit 12 notifies an error sound or vibration in a state where the drag operation (moving of the own vehicle icon) is not possible.

  From the time when the own vehicle icon is dropped until the processing corresponding to the control command is completed, the generation unit 12 may display the state of the moving vehicle as a ghost and display the track of the own vehicle. In addition, the generation unit 12 changes the display state (flashing, color change, size, position, etc.) of the vehicle icon from when the vehicle icon is dropped until the vehicle completes the process according to the control command. Also good. In addition, from the time when the vehicle icon is dropped until the processing corresponding to the control command is completed, the determination unit 11 queues the next instruction by an additional operation (reservation of control performed after the current control is completed). May be.

  As described above, according to the present embodiment, various operation details can be transmitted to the automatic operation controller 20 by moving the icon displayed on the touch panel with a gesture. The icon gesture operation is a simple operation, and is released from conventional driving operations such as the driver turning the steering wheel 51 and depressing the accelerator pedal 53. For example, by displaying a schematic diagram including a road guide sign and moving the vehicle icon to the road guide sign, an operation corresponding to the content of the road guide sign can be easily instructed. The driver can check the surrounding situation and perform an operation instruction on the touch panel at the same time, so that the line of sight does not move. Therefore, the possibility of erroneous operation is reduced, and safer driving can be realized. In addition, in order to issue the control command, an operation for associating a vehicle icon and a sign other than the operation described above may be used. In addition, although the gesture operation corresponding to the control command has been described by drag and drop, touch and touch may be used. A predetermined gesture or operation may be used, but the driver may be able to customize it. In addition, an explanation, an icon, or an arrow may be displayed on the display unit 31, a guide may be displayed, or voice guidance may be given so that the correspondence between the gesture operation and the control command can be understood.

  The present invention has been described based on the embodiments. It is understood by those skilled in the art that these embodiments are exemplifications, and that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are also within the scope of the present invention. By the way.

  For example, the HMI controller 10 is assumed to be mounted by a dedicated LSI, but the function of the HMI controller 10 may be realized using a CPU in a mobile device such as a smartphone or a tablet used as the display device 30. In this case, the portable device used as the display device 30 and the automatic operation controller 20 are directly connected. Further, the function of the HMI controller 10 may be realized by a CPU in a head unit such as a car navigation device or display audio. In addition, a configuration in which a dedicated LSI on which the HMI controller 10 is mounted may be included in the head unit.

  In the above-described embodiment, the example in which the command corresponding to the display content of the sign is issued by dragging and dropping the own vehicle icon onto the sign object indicating the road guide sign has been described. Similarly, a command corresponding to the display content of the sign can be issued by dragging and dropping the sign object and the vehicle icon.

  The embodiment may be specified by the following items.

[Item 1]
An image output unit (14a) for outputting an image including a vehicle object representing the vehicle and a sign object representing a sign to the display unit (31);
An operation signal input unit (14b) for accepting a user operation for moving the vehicle object in the image displayed on the display unit (31) to the position of the sign object or moving the sign object to the position of the vehicle object. )When,
When receiving the operation, a command output unit (14c) that outputs a command corresponding to the display content in the sign object to an automatic driving control unit (20) that controls automatic driving;
A driving support device (10) comprising:
The “sign” is a sign relating to road traffic, and may be a guide sign or a traffic sign.
Thereby, a user can perform operation according to the content of a sign intuitively and simply.
[Item 2]
The command output unit (14c) outputs, to the automatic driving control unit (20), a command for setting a place specified by a display content in the sign object to which the host vehicle object is moved as a destination or a waypoint. The driving support device (10) according to item 1, wherein the driving support device (10) is provided.
Thereby, the user can intuitively and easily perform the setting operation of the destination or waypoint.
[Item 3]
The command output unit (14c) outputs to the automatic driving control unit (20) a command for instructing the vehicle to enter the lane in the sign object to which the vehicle object has been moved. The driving assistance device (10) according to item 1 or 2.
“Enter” includes a right turn, a left turn, a U-turn, and a lane change.
Thus, the user can intuitively and easily perform operations for instructing a right turn, a left turn, a U-turn, and a lane change.
[Item 4]
When the operation signal input unit (14b) receives a user operation for moving the vehicle object in the image displayed on the display unit (31) to a lane or road name in the sign object, the image output unit (14a) The driving assistance device (10) according to item 3, wherein an image including a selection object for selecting an up direction and a down direction of the lane is output to the display unit (31).
Thereby, when the user changes the course to another road, it can be intuitively and easily instructed whether to go in the upward direction or the downward direction.
[Item 5]
A display device (30) for displaying an image;
A driving support device (10) for outputting an image to the display device (30),
The driving assistance device (10)
An image output unit (14a) for outputting an image including an own vehicle object representing the own vehicle and a sign object representing a sign to the display device (30);
An operation signal input unit (14b) for accepting a user operation for moving the vehicle object in the image displayed on the display device (30) to the position of the sign object or moving the sign object to the position of the vehicle object. )When,
When receiving the operation, a command output unit (14c) that outputs a command corresponding to the display content in the sign object to an automatic driving control unit (20) that controls automatic driving;
A driving support system (10, 30) comprising:
Thereby, a user can perform operation according to the content of a sign intuitively and simply.
[Item 6]
Outputting to the display unit (31) an image including an own vehicle object representing the own vehicle and a sign object representing a road sign;
Receiving a user operation to move the vehicle object in the image displayed on the display unit (31) to the position of the sign object, or to move the sign object to the position of the vehicle object;
When receiving the operation, outputting a command corresponding to the display content in the sign object to an automatic driving control unit (20) that controls automatic driving;
A driving support method comprising:
Thereby, a user can perform operation according to the content of a sign intuitively and simply.
[Item 7]
A process of outputting an image including a vehicle object representing the vehicle and a sign object representing a sign to the display unit (31);
A process of accepting a user operation to move the vehicle object in the image displayed on the display unit (31) to the position of the sign object or to move the sign object to the position of the vehicle object;
When receiving the operation, a process of outputting a command corresponding to the display content in the sign object to an automatic driving control unit (20) that controls automatic driving;
A driving support program for causing a computer to execute.
Thereby, a user can perform operation according to the content of a sign intuitively and simply.
[Item 8]
An image output unit (14a) for outputting an image including a vehicle object representing the vehicle and a sign object representing a sign to the display unit (31);
An operation signal input unit (14b) for accepting a user operation for moving the vehicle object in the image displayed on the display unit (31) to the position of the sign object or moving the sign object to the position of the vehicle object. )When,
When receiving the operation, a command output unit (14c) that outputs a command corresponding to the display content in the marker object;
An automatic operation control unit (20) for executing the output command;
An autonomous driving vehicle (1) comprising:
Thereby, a user can perform operation according to the content of a sign intuitively and simply.

  DESCRIPTION OF SYMBOLS 1 vehicle, 10 HMI controller, 11 determination part, 12 production | generation part, 13 instruction | indication part, 14 input / output part, 20 automatic operation controller, 21 control part, 22 memory | storage part, 23 input / output part, 30 display apparatus, 31 display part, 32 input unit, 40 detection unit, 41 position information acquisition unit, 42 sensor, 43 speed information acquisition unit, 44 map information acquisition unit, 50 driving operation unit, 51 steering, 52 brake pedal, 53 accelerator pedal, 54 winker switch.

  The present invention is applicable to a vehicle equipped with an automatic driving mode.

Claims (7)

An image output unit that outputs image information including a host vehicle object representing the host vehicle and parking lot guide sign information for guiding the user to a predetermined parking lot, on the display unit;
An operation signal input unit for receiving an operation signal of a user for moving the vehicle object in the image displayed on the display unit to the position of the parking lot guide sign information ;
Upon receiving the operation signal, a command output unit that outputs a control command for setting the destination of the host vehicle to the predetermined parking lot, to an automatic driving control unit that controls automatic driving;
A driving support apparatus comprising: The image further includes own vehicle traveling road information representing a road on which the own vehicle is traveling ,
The own vehicle object is positioned in the own vehicle traveling road information .
The driving support device according to claim 1. The user moves the vehicle object to a position of the parking lot guide sign information using a touch panel.
The driving support apparatus according to claim 1 or 2. A display device for displaying an image;
A driving support device that outputs image information to the display device,
The driving support device includes:
An image output unit that outputs image information including an own vehicle object representing the own vehicle and parking lot guide sign information for guiding a user to a predetermined parking lot, on the display device;
An operation signal input unit that receives an operation signal of a user that moves the vehicle object in the image displayed on the display device to the position of the parking lot guide sign information ;
Upon receiving the operation signal, a command output unit that outputs a control command for setting the destination of the host vehicle to the predetermined parking lot, to an automatic driving control unit that controls automatic driving;
Driving support system including. Outputting to the display unit image information including a host vehicle object representing the host vehicle and parking lot guide sign information for guiding the user to a predetermined parking lot ;
Receiving a user operation signal for moving the vehicle object in the image displayed on the display unit to the position of the parking lot guide sign information ;
When receiving the operation signal, outputting a control command for setting the destination of the host vehicle to the predetermined parking lot to an automatic driving control unit that controls automatic driving;
A driving support method comprising: A process of outputting image information including a vehicle object representing the vehicle and parking lot guide sign information for guiding the user to a predetermined parking lot on the display unit;
A process of receiving a user operation signal for moving the vehicle object in the image displayed on the display unit to the position of the parking lot guide sign information ;
When receiving the operation signal, a process of outputting a control command for setting the destination of the host vehicle to the predetermined parking lot to an automatic driving control unit that controls automatic driving;
A driving support program that causes a computer to execute. An image output unit that outputs image information including a host vehicle object representing the host vehicle and parking lot guide sign information for guiding the user to a predetermined parking lot, on the display unit;
An operation signal input unit for receiving an operation signal of a user for moving the vehicle object in the image displayed on the display unit to the position of the parking lot guide sign information ;
When receiving the operation signal, a command output unit that outputs a control command for setting the destination of the host vehicle to the predetermined parking lot;
An automatic operation control unit for executing the output command;
A self-driving vehicle comprising:

Images