JP6913316B2 - Information presentation system - Google Patents

Description

The present invention relates to an information presentation system for presenting information related to driving to a driver who drives a moving body.

It is well known that the information presentation system mounted on a moving body such as a vehicle includes a so-called car navigation system, which provides driving-related information to the driver. This type of technology provides the driver with travel-related information such as the course and road conditions of the traveling body when the driver drives the moving body. However, the provision of driving-related information was unidirectional from the information provision system to the driver, and was not interactive. Therefore, the driver only unilaterally follows the information provided by the information providing system, and does not establish a relationship like a passenger.

Therefore, the concept of an in-vehicle robot has emerged, and an information presentation system having an interactive relationship has been developed (see Non-Patent Documents 1 and 2). This type of technology enables mutual information transmission in addition to driving-related information, and applies technologies such as image processing or voice recognition using video, audio, light, or movement of the drive unit as a medium. By doing so, it attempts to introduce a conversational element.

However, according to the above technology, it is premised that the in-vehicle robot and the driver have a one-to-one interaction. For example, when the in-vehicle robot starts a conversation while driving, the driver must respond. There was a problem that it caused an element that could not be concentrated on driving. On the other hand, if the vehicle does not respond to the conversation of the in-vehicle robot, no effective interaction is formed, and the navigation system is merely installed, which may not be different from the conventional one.

Therefore, the inventors of the present application have developed an in-vehicle information presentation system for the purpose of reducing the cognitive load using multi-person conversation (see Non-Patent Document 3). This technology realizes multi-person conversation by an in-vehicle robot in order to solve the problem of one-to-one interaction, and is based on the concept that the driver is not required to participate in the conversation. That is, when the in-vehicle robot executes a multi-person conversation, the driver has a degree of freedom in participating in the conversation, and this degree of freedom realizes a loose communality in the multi-person conversation. As a result, when concentrating on driving, it was allowed not to respond to the conversation, which reduced the cognitive load on the driver.

Japanese Unexamined Patent Publication No. 2017-19436

Kohei Ito et al., "Prototype of In-Vehicle Robot" PIONEER R & D, Vol. 17, No. 1/207 Naohiko Ichihara et al., "In-vehicle camera application technology-image recognition car navigation and in-vehicle robot-" PIONEER R & D, Vol. 17, No. 1/207 Muneshi Yoshikawa et al., "Interaction Design in Multi-person Interaction Driving Agent (NAMIDA)" P-29, HAI Symposium 2016

The above-mentioned in-vehicle robot is exclusively for transmitting navigation information to the driver, and in addition to the navigation information, provides daily conversation or related information as a topic. In addition, a multi-person conversation type in-vehicle robot is excellent in that it reduces the cognitive load of the driver, but the information provided is also limited to navigation information and additional information, and is related to driving operations. Information, that is, information directly related to the running of the moving body, was not included. This was due to the difficulty of transmitting information related to driving by voice or the like.

By the way, in recent moving bodies, as represented by the automatic braking system, various sensors and computer control can protect the traveling body from the driver's operation mistakes, and further. , The technology related to the automatic driving system is rapidly advancing, and it is being realized that the moving body is automatically driven. In this way, by mounting various sensors on the moving body, the information that the driver must pay attention to is also detected by the sensor, and the blind spot area is also detected by the sensor, and based on this information. It has a configuration that can be controlled (see Patent Document 1).

The widely accepted definition of the above-mentioned autonomous driving system is that it is divided into levels 0 to 4 in stages according to the degree of automation. According to this definition, level 4 is a state of fully automatic operation, but level 3 is a state in which an automatic operation state and an operation state by manual operation are mixed. Therefore, the states of Levels 3 and 4 cause an event in which the moving body travels in a situation where the driver does not perform the driving operation.

However, in the above-mentioned automatic driving system, especially in the automatic driving state at level 3, whether the moving body is in a state where it can be automatically driven or in a state where it should be operated by manual operation is controlled by the moving body. Since it is controlled by the system, it was not easy to objectively grasp it outside the control system (driver). Then, these changes in the situation were only made known to the driver by the notification device, and the driver could not know what kind of situation judgment was made during the automatic driving. Therefore, the driver cannot be relieved even in the automatic driving state, and has no choice but to confirm the safety by his own observation of the running state of the moving body. That is, the technology related to the interface (HMI: Human Machine Interface) between the automatic driving system and the driver has not been developed.

The present invention has been made in view of the above points, and an object of the present invention is to visually present to the driver information directly related to traveling provided by sensors mounted on a moving body. In addition to providing a system for this purpose, it is also to provide an information presentation system that can make the driver aware of the situation in which the moving body is inputting information related to traveling.

Therefore, the present invention is an information presenting system provided for a manned mobile body and presenting information related to the running of the moving body to a driver, and the running information indicating the running state of the moving body, the same. An information acquisition means for acquiring one or more of the external environment information necessary for traveling outside the moving body and the driver information related to the driver operating the moving body from the sensors, and the information acquisition. The information includes a control means for generating presentation information to be presented to the driver from the information acquired by the means, and an information sensing means for making the driver aware of the presentation information generated by the control means. The recognizing means includes a plurality of display bodies, each of which causes the driver to be aware of the same morphological change or individually by a single morphological change.

According to the above configuration, one or more of driving information, external environment information and driver information are acquired, information to be presented to the driver is generated based on the information, and the information is presented to the driver by a conscious means. It is a thing. Here, as a means of awareness, a plurality of display bodies are provided, and the display bodies are used to arouse the driver's consciousness and present driving-related information by changing the form of the plurality of display bodies. The form of is not only by sound (or sound), but also by the shape, color, pattern or combination of these, and in addition to changing the form as an image, moving or rotating the display body, etc. The morphology may be changed by driving. Although it is possible to display the form by characters, in order to be able to visually recognize the moment, the form recognition by the change in appearance can be mainly focused, and in the case of voice, it can be a secondary content.

The moving body of the present invention includes an automatic driving control device that controls a part or all of the moving stroke, and a switching means that enables switching between the moving driving mode by the automatic driving control device and the manual driving mode by the driver. In the case of such a moving body, the control means may generate presentation information indicating that switching to the automatic driving mode is possible, or suggest switching to the manual driving mode. It is possible to generate the presentation information to be presented. Then, when the driver switches to the automatic driving mode, the presentation information is generated so as to indicate that the information that the moving body should need for driving is acquired among the external environment information. It has a structure.

Further, the information system of the present invention can be provided with a means (eye tracker) for acquiring information on the driver's vision at the same time, and the driver's line of sight is used as input information to form a mode of awareness in the awareness means. It is possible to reflect it. In such an embodiment, the driver's line of sight is detected in both automatic driving and manual driving, and the state of awareness by the sensing means is synchronized with the driver's line of sight in the same direction. It is also possible to express the state of watching. If the display body that constitutes the recognition means imitates the state of a living thing, it is possible to realize a recognition method in which the driver and the display body have a cooperative relationship. In addition, when the driver's line of sight frequently points in a specific direction during driving (especially during automatic driving), it is possible to input the matter that the driver is paying particular attention to as special information, and this information is processed. As a method, it is also possible to form a mode in which the driver gives instructions on points to be noted when traveling the moving body.

According to the present invention, since the driving-related information is displayed by changing the form by a plurality of display bodies, when the driver is performing the driving operation, the driving-related information can be visually obtained while concentrating on the driving. Can be done. In addition, in a situation where the moving body is traveling by automatic driving, what kind of information is obtained during automatic driving is presented through the display body, and the driver is in a situation other than the display body. It enables comparison, that is, comparison with driving-related information based on one's own observation. Therefore, even when the moving body travels by the automatic driving operation, the driver can visually recognize that the normal situation judgment is made, and the driver can be relieved.

It is explanatory drawing which shows the outline of the Embodiment of this invention. It is explanatory drawing which shows the structure of the HMI controller and HMI. It is explanatory drawing which shows one aspect of the specific form of HMI. It is explanatory drawing which shows the example of mounting HMI on a moving body. It is explanatory drawing which shows the processing flow in an HMI controller. It is explanatory drawing which shows the processing flow in an HMI controller. It is explanatory drawing which shows the control method of the display body. It is explanatory drawing which shows the control method of the display body. It is explanatory drawing which shows the modification about the specific form of HMI.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 shows an outline of the system of the present embodiment. The information presentation system according to the present embodiment is mounted on a moving body such as a vehicle, and it is premised that the moving body is equipped with the automatic driving system 1. Further, the automatic driving system 1 enables switching between automatic driving by the automatic driving control device and manual driving operated by the driver DR by the operation of the driver DR while the moving body is traveling.

As shown in FIG. 1, the automatic driving system 1 includes information acquisition means for obtaining various kinds of information. For example, there are a GPS receiving unit 11 for obtaining position information, a map database 12, an external sensor 13, an internal sensor 14, and the like. This information is input to the ECU (Electronic Control Unit) 16 and can be controlled by the ECU to determine whether or not the operation can be switched and to control the vehicle during automatic operation.

The external sensor 13 is a group of sensors for acquiring an external environment necessary for traveling outside the moving body, and includes a camera, a radar, a lidar (LIDAR: Laser Imaging Detection and Ranking), and the like. The camera is for photographing a predetermined external environment, and is used for acquiring information necessary for detecting an object such as a preceding vehicle or an obstacle. In addition, image processing distinguishes traffic lights, pedestrians, structures, etc., and detects the display state of traffic lights and the display contents of road signs. Radar is used to detect obstacles outside the moving body using radio waves. The rider is used to detect obstacles outside the moving body using light. In order to obtain information on the external environment, input devices other than these may be added, and the obtained information may be duplicated. Further, if the acquisition of duplicate information confuses control, some sensors may be omitted.

The internal sensor 14 is sensors for acquiring the traveling state of the moving body, and includes a vehicle speed sensor, an acceleration sensor, a yaw rate sensor, and the like. The vehicle speed sensor detects the rotational speed on the wheels or axles. The acceleration sensor detects the acceleration in the left-right direction as well as the acceleration in the front-rear direction. The yaw rate sensor detects the rotational angular velocity around the vertical axis of the center of gravity of the moving body. With these sensors, at least the basic state of the state in which the moving body is running (or stopped) can be acquired. By adding this information to GPS information, map data, or the like and processing it, it is possible to detect the driving situation based on the traveling plan.

The ECU 16 includes an external situation recognition unit and a position recognition unit, and recognizes the external situation and position information as external environment information based on the information input from the GPS receiving unit 11, the map database 12, and the external sensor 13. Items that should be recognized as external conditions include the width and shape of the road, the presence or absence of obstacles outside the moving body, and the display contents of traffic lights and road signs.

Further, the ECU 16 includes an operation control unit, and controls the accelerator operation, the brake operation, and the steering operation while referring to the external environment information. These operations are performed by a throttle actuator, a brake actuator, and a steering actuator, and each operation is controlled by each actuator. In a system in which a travel plan is generated in advance, travel based on the travel plan is realized.

Further, the ECU 16 includes an operation input amount detecting unit, a traveling state recognition unit, a driver state recognition unit, an operation switching unit, and the like. The operation input amount detection unit, the driver state recognition unit, and the operation switching unit recognize the information detected by the driver information sensor 15 described later, and the running state recognition unit recognizes the running state by the internal sensor 14. Is what you do.

The information recognized or controlled by the ECU 16 is output to the HMI controller 20, and the presented information is output to the HMI 30. The details of the HMI controller 20 and the HMI 30 will be described later, but the outline of the HMI controller 20 includes an information input unit, a generation unit for presenting information to the driver DR, an output unit, and the like. The signal output from the output unit is input to the HMI 30, and the information for notifying the driver DR is displayed. The HMI is provided with a plurality of display bodies, and for example, the form is deformed by driving an actuator. In addition to the case where the plurality of display bodies are aware of the presented information by simultaneous and similar morphological changes, there are cases where the individual display bodies are individually changed in morphology to comprehensively recognize the presented information. Differences in these awareness methods may be distinguished between stable driving and emergencies, and may be distinguished between automatic driving and manual driving. In either case, the driver DR can confirm the presented information displayed by the display body. The HMI is an interface for inputting / outputting information between the mobile body and the driver DR, and can be provided with a microphone or the like for voice recognition as an input, or can be provided with an operation button. good.

Further, the automatic driving system 1 is provided with a driver information sensor 15. The driver information sensor 15 detects the state of the driver DR operating the moving body, and detects whether or not the operation is appropriately performed based on the information displayed by the display body. In addition, it is equipped with an eye tracker for obtaining information on the line of sight. The tactile sensor is provided on the steering wheel and detects whether the steering wheel can be operated by the driver DR, and the operation input amount sensor detects the operation amount by the operation of the driver DR. The changeover operation switch detects that the driver DR has performed a changeover operation between automatic operation and manual operation.

The outline of the information presentation system according to the present embodiment is as described above, but the automatic operation system 1 may be provided with other functions. Further, the control unit in the HMI controller 20 may be operated by the control device of the ECU 16, but in the present embodiment, in order to explain the function of the HMI, a controller configured as an individual controller is illustrated.

FIG. 2 shows the configuration of the HMI controller 20 and the HMI 30. As shown in this figure, the HMI controller 20 includes a control unit (control means) 21 and an input / output unit 22. The input / output unit 22 inputs information on various sensors recognized as external environment information (external status, position information, etc.) and running information (running state, etc.) in the above-mentioned ECU 16, and also outputs information from the HMI 30. Enter. Further, the control signal generated by the control unit 21 is output to the HMI 30. The sensors and the like that acquire the information input to the input / output unit 22 are specific information acquisition means, but it is considered that the sensors and the like that acquire the information input to the input / output unit 22 function as the information acquisition means by the configuration including the input information module of the control unit 21. Can be done.

The control unit 21 includes an input information recognition module, a presentation information control module, a presentation information generation module, and a driver state recognition module. The information input from the input / output unit 22 is recognized as external environment information, traveling information, and driver information by the input information recognition module or the driver state recognition module. This information is organized as information to be presented by the presentation information control module. An output signal is generated by the presentation information generation module, and the organized information is output to the HMI 30 via the input / output unit 22.

The HMI 30 includes a display body 31, a speaker 33, a video display unit 34, and the like, and functions as an information sensing means. The information sensing means may include all of these, but may be configured by at least one of them. Further, the speaker 33 and the image display unit 34 may be built in the display body 31.

The display body 31 is for visually presenting the presented information, and is composed of an aggregate of a plurality of display bodies (first to nth) 31a, 31b, ... 31n. The reason why the plurality of display bodies 31a to 31n are configured is to intuitively obtain information visually by an information presentation method that changes the form of each display body 31a to 31b without displaying characters or figures. be. In order to change the form of the display bodies 31a to 31n, the HMI 30 is provided with an actuator 32. For example, when the display bodies 31a to 31n are composed of spherical rotating bodies, the driving force for rotating the rotating bodies is changed. In addition, various actuators enable a wide variety of changes in form.

The speaker 33 presents information as voice. For example, in an emergency, when the information presented by the display body 31 is insufficient, the information is presented as voice. It is not necessary for the speaker 33 to output only the presentation information by voice, and when there is no information to be output by voice as in normal times, the speaker 33 displays by synchronizing with the morphological changes of the plurality of display bodies 31a to 31n. A plurality of people may have a conversation with the body 31.

The image display unit 34 displays an image on the display body 31 or a display unit (display panel or the like) that replaces the display body 31. When the car navigation system is installed, it can be shown that the vehicle is in a traveling state according to the traveling plan. Furthermore, it is also possible to present the image acquired from the camera among the acquired external environment information. In the case of displaying an image on the display body 31, an image that causes a morphological change of the individual display bodies 31a to 31n may be presented. The display bodies 31a to 31n in this case are manufactured as a structure having a function of displaying an image.

The HMI 30 may be configured to include an input operation unit 35. The input operation unit 35 includes, for example, a touch panel and a changeover switch. In the case of a touch panel, it can also be used as an image display unit 34, and when linked with a navigation system, it can function as an input device when changing a traveling route, or is presented as image information. In that case, it may be used as an image display switching. The changeover switch includes a changeover switch between automatic operation and manual operation, and can be installed in place of the changeover operation switch of the driver information sensor 15 of the automatic operation system 1 (FIG. 1). The input operation unit 35 may also use a voice recognition system or the like, and may acquire the voice emitted by the driver DR and use it as an input.

The driver DR visually recognizes the morphological change of the display body 31 having the above configuration, and recognizes the presented information from the contents displayed therein. The change in form is intuitively recognizable and is displayed within the range of presented information to that extent. In addition, information that cannot be recognized from the display body 31 can be obtained from the voice information output from the speaker.

Further, an eye tracking system (eye tracker) 15a is provided as an external device (a part of the driver information sensor) so that the line-of-sight information of the driver DR who should operate the moving body can be obtained. This is to confirm whether the line of sight of the driver DR is moving to an object (traffic light, obstacle, etc.) to be noted during manual driving, and the display body 31 is displayed during automatic driving. The visual state of information and external objects is confirmed by the line of sight.

As for the use of the line-of-sight information, for example, in the case of manual driving, if the driver DR does not shift the line of sight to an object to be noted by inputting the line-of-sight information, the display body 31 is displayed by changing the form. , If the line of sight does not move after that, it may be presented as voice information. Further, at the time of automatic operation, it is determined whether or not the driver DR confirms the content displayed by the display body 31, and for example, the display body 31 indicates that the state should be switched to manual operation. If so, it is used to check if the driver DR has seen it. When the display body 31 is not viewed even though the operation should be switched to the manual operation, it can be used as a judgment criterion such as awakening by voice information.

Further, the line-of-sight information can be used as information for following the line-of-sight of the driver DR so that the driver DR can sympathize with the display body 31. The driver frequently moves his / her line of sight in order to obtain information on the outside of the moving body in both the automatic driving and the manual driving, and the content displayed by the display body 31 relates to the direction of the line of sight. It is supposed to be. That is, if the display content displayed by the display body 31 is visually and intuitively recognizable by the driver DR, the display content can be set to the direction to be watched exclusively, and the direction to be watched is During stable running, the direction of the driver's DR line of sight can be set. In this case, if the display body 31 imitates the mode of an organism as described later, it is possible to show an aspect as if a cooperative relationship (joint attention posture) between the two is formed. Furthermore, if the display content is to alternately gaze at the driver and his / her line-of-sight direction, it is possible to express an action similar to a kind of pet animal's gesture (so-called gaze alternation).

Further, since the line-of-sight information is information indicating an object to be watched by the driver DR, it is important information when the moving body is driven. Therefore, among the line-of-sight information, an object that exists in a direction in which the line of sight is repeatedly directed, or an object in a direction in which the line of sight is directed for a relatively long time can be recognized as an object to be particularly watched. The recognition of the object is stored in the ECU 16 to promote the improvement of safety during automatic operation.

The information acquired by the eye tracking system 15a may be directly input from the input / output unit 22 of the HMI controller 20 to the control unit 21, but it is uniquely output to the ECU 16 and the HMI from the ECU 16 together with other input information. It may be configured to output to the controller 20.

Here, a specific example of the HMI 30 using a plurality of display bodies 31a to 31n will be described. FIG. 3 shows a specific form of the HMI 30. As shown in FIG. 3, the specific form has a configuration having three display bodies 31a, 31b, and 31c. The three display bodies 31a, 31b, 31c are installed in the three through holes 37a, 37b, 37c provided in the base 36, and their heads are exposed from the surface. Each display body 31a, 31b, 31c has at least a spherical head shape, and can rotate while being loosely fitted in the through holes 37a, 37b, 37c. Further, inside the base 36, three actuators for driving the individual display bodies 31a, 31b, and 31c are provided so that the rotation can be driven and controlled. Therefore, the three display bodies 31a, 31b, and 31c have a structure in which the actuators can be individually operated by the output from the HMI controller 20 to rotate the spherical head.

Further, the spherical heads of the respective display bodies 31a, 31b, 31c are partially distinguished from the surroundings by a translucent member, and the facial morphological portions 38a, 38b, 38c are formed by the distinguished portions. By installing two LEDs each inside the facial morphological portions 38a, 38b, 38c, the eye morphological portions 39a1, 39a2, 39b1, 39b2 are formed. Although not shown in the figure, a similar eye shape portion is formed on the third display body 31c. Further, although the figure does not show the speaker or the image display unit, it can be installed on the base 36 or may be provided separately. Further, the HMI controller 20 may be provided inside the base 36, and may also be provided as a separate body. Needless to say, if the ECU has a function as an HMI controller, it does not need to be installed specially.

As described above, by forming the facial morphological portions 38a to 38c and the eye morphological portions 39a1 to 39b2, the display bodies 31a, 31b, and 31c are shaped like three display organisms. Therefore, the through holes 37a, 37b, and 37c of the base 36 serve as the neck of the living thing, and the head rotates to imitate a state in which the direction of the face (line of sight) is changed. Therefore, by changing the face (line of sight), the change in morphology is realized. The change in the line of sight at this time can be visually and intuitively displayed to the driver as indicating the viewing direction of an organism such as an animal. By changing the line of sight, it is possible to express a situation in which the driver is paying attention in collaboration with the driver, and in an emergency, it is possible to express a special aspect (a state in which the creature is in a hurry).

For example, for all three display bodies 31a, 31b, and 31c, by directing the face (line of sight) in the same direction, the existence (presentation information) of important cautions is presented in that direction, and for example, one of these 1 By directing the face (line of sight) toward the driver, the two display bodies 31c can express a state in which the driver's state is being watched. Further, all of the three display bodies 31a, 31b, and 31c rotate left and right in small steps to express a form in a hurry, and it is also possible to show presentation information in an emergency. Furthermore, by setting the faces (line of sight) of the three display bodies 31a, 31b, and 31c to face each other, it is possible to express that the state is a conversation among a plurality of speakers. In particular, during automatic driving, driving is controlled by the automatic driving system, but the face (line of sight) should be directed toward the location of various information (information sensed by the sensor) acquired by the automatic driving system. By operating the display bodies 31a, 31b, and 31c, it is possible to make the driver sensuously present the state in which the automatic driving system is traveling.

The eye portion morphology portions 39a1 to 39b2 are made to appear by emitting light from an LED or the like, but the morphology may be changed by changing the emission color. In this case, there is a method in which the vehicle is in a relatively stable state during automatic operation and is lit in green to indicate the condition, and is lit in red to call attention during manual operation. Then, when indicating the switchable state, there is a method of making the user aware by blinking or the like. The change in the emission color and the rotation of the head may be displayed in a superimposed manner.

Further, the face morphology portions 38a to 38c and the eye morphology portions 39a1 to 39b2 as described above may have a configuration in which an image is projected from the inside with the entire head as an image display screen. In this case, it is not necessary to drive the head itself to rotate, and by editing the image, the orientations of the facial morphological portions 38a to 38c and the eye morphological portions 39a1 to 39b2 are changed by the images projected on the head. It may be a method of visually recognizing it as if it were.

The HMI 30 having the above configuration is installed in a place that can be easily confirmed by the driver. For example, when the moving body to be installed is a vehicle, as shown in FIG. 4, if it is installed on the dashboard surface at an easily visible position in front of the steering wheel or diagonally forward, the display body 31a can be installed even during manual operation. , 31b, 31c can be confirmed. In this way, by being able to obtain the presented information visually and intuitively instead of displaying it in characters, it is possible to instantly grasp the points of caution during manual driving, and the operation of the automatic driving system during automatic driving. The state can be grasped visually.

Next, a control method by the HMI controller 20 will be described. As described above, the display of the presented information by the HMI 30 may be by voice or image, but here, the control method for the display body will be exclusively described. 5 and 6 show a control flow. FIG. 5 is a control flow during automatic operation (this is referred to as an automatic operation mode), and FIG. 6 is a control flow during manual operation (this is referred to as a manual operation mode).

As shown in FIG. 5, in the automatic driving mode, the sensor information (acquired information) is read (S101), and further, the driving information is read (S102). Based on the read information, first, it is determined whether or not it is possible to continue the automatic driving (driving in the automatic driving mode) (S103). This judgment is made by comparing the sensor information (detected value) acquired by the sensors with the threshold value related to these numerical values. Here, the determination is made by the HMI controller, but it is usually processed by the ECU. When automatic driving is possible, display processing is performed as an automatic driving mode state. That is, information (display information) to be presented to the driver during automatic driving is selected, the information is generated as display information (S104), and is generated as drive control information for each of a plurality of display bodies (S105). Output to the drive unit (actuator) (S106). Since the acquired information changes from moment to moment, the control is repeated because the sensor information is read again (S101) unless the HMI controller is terminated (S107).

In the determination of whether or not automatic driving is possible (S103), if automatic driving is not possible, information for prompting travel switching (switching to manual driving) is generated (S111). As the switching information to manual driving, stepwise display information can be set. For example, there are cases where there is an urgency, manual operation is required in the near future, manual operation or automatic operation is possible, but manual operation is preferable. Here, a case where manual operation is preferable will be described as an example. In such a case, the drive control information of the drive unit (actuator) is generated (S112) along with the display information for encouraging manual operation, and is output to the drive unit (S113). After that, information on the changeover switch and the like is acquired, and it is determined whether or not the operation mode has been changed (S114). If the operation mode has not been switched, the same drive control information is output to the drive unit until the switching is completed. When the switching is completed, the control is then changed to the manual operation mode.

When the drive information is based on the display bodies 31a, 31b, 31c exemplified above, the display information at the time of automatic driving is an object (pedestrian, preceding vehicle, etc.) that should be generally watched among the external environment information. The face (line of sight) may be directed toward a traffic light, etc., or the face (line of sight) of some of the display bodies may be directed toward the driver. On the other hand, in the case of display information prompting switching to manual operation, the faces (line of sight) of all the display bodies may be directed toward the driver, or all of them may wiggle (rotate back and forth to the left and right). Can be mentioned.

In the manual operation mode, as shown in FIG. 6, sensor information (acquired information) is read (S201) and driving information is read (S102), and based on these information, there is a possibility of automatic driving. Is determined (S203). This judgment is the same as the judgment of the continuity of automatic driving, and the sensor information (detection value) is compared with the threshold value. When the automatic operation is impossible, the manual operation is continued, so that the displayed information is generated according to the manual operation (S204). Based on the display information, drive control information is generated (S205), drive information is output (S206), and unless the HMI controller is terminated, the information is read (S201) and repeated.

When it is determined in the possibility determination (S203) of automatic driving that automatic driving is possible, it is displayed that switching to automatic driving is possible (S211 to S213). In presenting information that autonomous driving is possible, it does not encourage automatic driving, but merely indicates that it is possible. This is because even if automatic driving is possible, the automatic driving mode should not be forced. Therefore, after the information that the automatic operation is possible is presented, the operation mode switching is determined (S214), but even if the switching is not performed, the same control as the manual operation mode is performed. (S204 to S207). When the switching to the automatic operation is completed, the control shifts to the automatic operation mode.

When the drive information is based on the display bodies 31a, 31b, 31c exemplified above, the information presented during manual driving is an object (pedestrian, preceding vehicle, etc.) that should be generally watched among the external environmental information. It is possible to change the face (line of sight) of each display body simultaneously or individually according to the priority (importance) of the traffic light, etc.). Further, the face (line of sight) of a part of the display body may be directed toward the driver to display the state in which the driver's line of sight information is acquired. Display information indicating that automatic driving is possible includes directing the faces (line of sight) of all display bodies in the direction of travel.

The above-mentioned eye morphology is displayed by light from an LCD or the like, but since there may be a morphological change that changes the color of this light, in the automatic operation mode, the eye morphology is green. The current mode may be visually displayed as red in the manual operation mode. Further, the form may be changed so as to blink the light as a display for prompting the mode switching or the case where the mode can be switched.

Here, a typical display body control method will be described. FIG. 7 is a control flow of the display body during automatic driving, and FIG. 8 is a control flow of the display body during manual driving. Since the parts already explained are duplicated, they are omitted here.

As shown in FIG. 7, during automatic operation, the information to be displayed (display information) is read (the display information created above is used) (S301), so that manual operation can be performed. It is determined whether the switching information is included (S302). When the switching information is not included, the automatic operation state is continued, so that the information to the display body is controlled according to the content. At this time, when the number of information to be displayed is large, the number is compared with the number of display bodies (S303), and when the number is larger than the number of display bodies, specifically, one of the display bodies is used. Except, when there is more display information than the remaining number, the priority is determined for the information to be displayed (S304). The priority of the display information is determined according to the setting order in which the order is set in advance for each piece of information. For example, pedestrian information can be ranked first among external environmental information, traffic light information can be ranked second, and speed information among driving information can be ranked third. Ranking is set for the elements that should be important in. Further, for example, when there is no object to be paid special attention to, the priority may be set to follow the driver's line of sight so as to follow the driver's line of sight.

According to the above priority, the necessary display is performed on a plurality of display bodies (total number-1). In the case of the above-mentioned example, the necessary information here is such that the face (line of sight) of the display body is directed in the direction to be watched. In particular, in the case of automatic driving, steering is automatically operated, and acceleration, deceleration, or stop is performed, but it is possible to display the external environment that causes these operations to the driver. It becomes important. Therefore, in the case of steering, the priority may be changed according to the case of acceleration / deceleration, and in this case, the traveling information is acquired at the same time. It should be noted that a large number of display information may be sequentially moved down from the first order, the display contents may be changed stepwise by each display body, and control may be performed so that all the display information is displayed in a short time.

Arbitrary information is controlled to be displayed for the remaining one display body. The arbitrary display information is a display for the driver, and for example, in the above example, the face (line of sight) is directed toward the driver to display a state in which the driver is being observed. Is. In this way, by giving a degree of freedom to one display body and not displaying the display information, it is an important display means for displaying information such as an emergency. That is, when displaying urgent display information, it is possible to indicate an urgent state by changing the form of the residual display body. Arbitrary information is like turning the face (line of sight) alternately in two directions, the driver's eyes and the direction that follows the line of sight, so as to reproduce the gaze alternation, which is a type of animal gesture. May be.

When the read information includes the information for switching to manual operation, the presence or absence of urgency is first determined (S311). The case where there is no urgency is when manual operation is desirable, but it is not a situation where it is necessary to switch to manual operation. In such a case, the switching information will be displayed. All display bodies will be used for this switching display. After this display, it is determined whether or not the operation has been switched to manual operation (S313), and if it is switched, the control shifts to the display body control during manual operation (S314), but if it cannot be switched, during normal automatic operation. It is controlled as a display in.

Further, when it is determined that there is an urgency in the switching information to the manual operation, the display control is performed to notify the entire display body that the manual operation should be urgently switched (S321). Then, the completion of the switching is confirmed (S322), and if the switching is performed, the control of the display body during manual operation is started (S323). If the switching is not made, it is determined whether the running in the automatic driving can be continued (S324), and if the running is possible, the above display control is continued, but if the running is not possible, the ECU results as a result. Will result in an emergency stop, and display control will end (S325).

As shown in FIG. 8, the control of the display body during manual operation is performed when the presence or absence of automatic operation switching information is determined based on the read information (S401) (S402) and the switching information is not included. Displays display information according to the priority order using the remaining display bodies except for one display body (S403 to 405). The display method according to the priority and the display of arbitrary information by one display body (S406) are the same as in the case of display body control during automatic operation, and the display control ends. The above process is repeated until (407).

When the read information includes the switching information, it is displayed that the switching is possible by using all the display bodies (S411). The display method is as described above. Then, the switching state is determined (S412), and if it is switched, it shifts to the display body control during automatic operation (S413), but if it cannot be switched, it is processed as the display body control during normal manual operation. Will be done (S403 to S407).

As described above, in the control of the display body, not all of them are controlled in the same manner, but various states are made known to the driver by the display body by giving a degree of freedom to at least one display body. Is possible. Then, as described above, when the display body has the appearance of a living thing, the direction of the face (line of sight) is controlled, and what the sensors detect, in particular, according to the direction. At the time of automatic driving, whether these sensors are functioning normally, and also, in such a state, the moving body controlled by the automatic driving system continues to run while paying attention to what. It is possible for the driver to easily confirm whether or not the vehicle is operating.

The embodiment of the present invention is as described above, but the above embodiment shows an example of the present invention, and the present invention is not limited to the above embodiment. Therefore, the configuration shown in the above embodiment may be changed as appropriate. For example, FIGS. 3 and 4 have been shown and described as examples of specific display bodies. In this embodiment, three display bodies are illustrated, but the number can be increased or decreased as appropriate. Further, the shape of the base 26 shown in the drawing is an arch shape extending in three directions, but this shape can also be changed as appropriate. That is, when the number of display bodies is four, one display body can be arranged on each side with the center of the base 26 as the center, and the arch-shaped base 26 is circular or rectangular. It may be formed in a box body.

Further, although the individual display bodies 31a, 31b, and 32c are installed on the base 26, they may be configured by individually self-supporting display bodies without providing the base 26. Specifically, the egg-shaped display bodies 131a, 131b, 131c as shown in FIG. 9 can be installed at any place. This egg-shaped display body can stand on its own by constructing it with a low center of gravity, and has a head and eyes that imitate the appearance of living things, and the whole rotates or an image is projected on the head. It is possible to change the form by making it. Further, the morphology may be changed by deforming the outer shape exhibiting an egg shape.

A display panel may be provided in place of or additionally of the display body of the above form. A display object that imitates the appearance of a living thing may be displayed as an image in the panel. By using the display panel also as a touch panel, the input operation unit can be integrally configured with the display body.

1 Automatic driving system 11 GPS receiver 12 Map database 13 External sensor 14 Internal sensor 15 Driver information sensor 15a Eye tracking system 16 ECU
20 HMI controller 21 Control unit 22 Input / output unit 30 HMI
31 Display 31a, 31b, 31c, 31n, 131a, 131b, 131c Individual display 32 Actuator 33 Speaker 34 Video display 35 Input operation 36 HMI base 37a, 37b, 37c Through holes 38a, 38b, 38c Face morphology Part 39a1, 39a2, 39b1, 39b2 Eye part morphology part

Claims (11)

An information presentation system provided for a manned mobile body and presenting information related to the running of the moving body to the driver.
A sensor detects one or more of the running information indicating the running state of the moving body, the external environmental information necessary for running outside the moving body, and the driver information related to the driver who operates the moving body. Make the driver aware of the information acquisition means acquired from the class, the control means for generating the presentation information to be presented to the driver from the information acquired by the information acquisition means, and the presentation information generated by the control means. Equipped with information awareness means for
The information-aware means includes a plurality of display bodies, and the display bodies change their morphology by moving, rotating, or deforming a part or all of the display bodies, each of which is changed by the same morphological change or by the same morphological change. An information presentation system characterized in that the driver is made aware of each individual by a single form change. An information presentation system provided for a manned mobile body and presenting information related to the running of the moving body to the driver.
A sensor detects one or more of the running information indicating the running state of the moving body, the external environmental information necessary for running outside the moving body, and the driver information related to the driver who operates the moving body. Make the driver aware of the information acquisition means acquired from the class, the control means for generating the presentation information to be presented to the driver from the information acquired by the information acquisition means, and the presentation information generated by the control means. Equipped with information awareness means for
The information SatoshiTomo means comprises a plurality of display bodies, the display body state, and are not each be SatoshiTomo the driver by the same morphological changes, or individually by a single morphological changes,
The display body displays an image, and the control means has a function of generating the image, and in a partial area or an entire area of the display body, the facial morphology portion and the eye morphology portion An information presentation system characterized in that an image is displayed in such a manner that it moves, rotates, or is deformed, and the displayed form is changed by the movement, rotation, or deformation of the image. The moving body includes an automatic driving control device that controls a part or all of the moving stroke, and a switching means that enables switching between the automatic driving mode by the automatic driving control device and the manual driving mode by the driver. ,
The control means is characterized in that it generates presentation information that suggests that switching to the automatic operation mode is possible, or generates presentation information that suggests switching to the manual operation mode. The information presentation system according to claim 1 or 2. When the control means is switched to the automatic driving mode by the driver, the control means generates presentation information so as to show that the information necessary for the moving body to travel is acquired among the external environment information. The information presentation system according to claim 3 , characterized in that the information is to be used. The presented information includes external environment information that causes the moving body to change its traveling direction, decelerate, or stop, and at least one of the plurality of display bodies is the external environment. The information presentation system according to claim 4 , wherein the presentation information based on the information is displayed. The information acquisition means is adapted to obtain information about the driver's line of sight as at least the driver information, the display body according to claim 4 at least one is for displaying that it is getting an driver information Or the information presentation system according to 5. Claims 4 to 6 in which at least one display body displays the presentation information to the driver when the control means generates presentation information suggesting switching to the manual operation mode. Information presentation system described in any of. When the control means is switched to the manual operation mode by the driver, the control means generates presentation information relating to an element of the external environment information to be watched when the driver performs an operation of traveling the moving body. The information presentation system according to claim 3 , characterized in that it is a thing. The information presentation system according to any one of claims 1 to 8 , wherein the information awareness means includes a sound output means. The information presentation according to any one of claims 1 to 9 , wherein the display body is composed of a plurality of independent display units or a plurality of display units visually recognized inside a single display body. system. The information presentation system according to any one of claims 1 to 10 , wherein the form of the display body or the mode of deformation thereof imitates the mode of an organism.

Images