JP7007055B2 - Autonomous driving system - Google Patents

Description

The present invention relates to a technique for switching between automatic driving and manual driving in an automatic driving system.

As a technique for switching between automatic operation and manual operation in an automatic operation system, a technique for automatically switching between automatic operation and manual operation according to a situation is known (Patent Document 1).
In addition, as a technology for switching between automatic driving and manual driving in an automatic driving system, when switching from automatic driving to manual driving, the user is encouraged to see obstacles and the front of the vehicle, and the user's line-of-sight direction is detected. There is known a technique for determining that a user is in a state where manual driving is possible when the detected line-of-sight direction is toward the object for which visual recognition is promoted (for example, Patent Document 2).

Further, in a head-up display that displays a virtual image in front of an automobile by projecting an image onto a front window from a projector and reflecting the image, a technique is also known in which the distance to the virtual image in the front-rear direction is variable (for example). Patent Documents 3 and 4).

Japanese Unexamined Patent Publication No. 2001-1992 Japanese Unexamined Patent Publication No. 2015-185088 Japanese Unexamined Patent Publication No. 2016-118423 Japanese Unexamined Patent Publication No. 2016-161773

According to the above-mentioned technology for determining that the user is in a state where manual driving is possible when the user's line of sight is facing the direction of the object for which visual recognition is promoted, the user is looking at the direction of the object indiscriminately. However, there is a problem that the user is determined to be in a state where manual operation is possible even when the user does not pay attention to the object.

Therefore, the present invention more effectively leads the user's state to a state suitable for manual driving when switching from automatic driving to manual driving in the automatic driving system, and the user's state is suitable for manual driving. The task is to determine more accurately whether or not it is present.

In order to achieve the above-mentioned problems, the present invention provides an automatic driving system, which is mounted on a car and selectively performs automatic driving and manual driving of the car, to the front of the car for a user in the driver's seat of the car. A head-up display that displays a virtual image in front of the window and has a variable distance in the front-rear direction of the vehicle from the front window to the virtual image, and a head-up display in front of the vehicle when switching from automatic driving to manual driving. The head-up display is provided with a control means for setting a display position and displaying a virtual image of a predetermined figure at the set display position. In the control means, the higher the vehicle speed of the vehicle, the more the front window to the display position. The display position is set so that the distance in the front-rear direction of the automobile from the vehicle becomes large.

Here, in the control means, such an automatic driving system is used as the display position at the center position of the vehicle in the left-right direction of the vehicle, or at a position between the center position and the front of the driver's seat. Therefore, it may be configured to set a position within a predetermined range from the road surface above the road surface.

Here, the driver's gaze point suitable for manual driving is that the higher the speed of the vehicle, the farther forward the position. Therefore, in the present automatic driving system, the display position is set so that the distance from the front window to the display position in the front-rear direction of the vehicle increases as the vehicle speed of the vehicle increases, and the display position is predetermined. By displaying the virtual image of the figure, the gazing point of the user in the driver's seat can be guided to a position suitable for manual driving.

Further, in order to achieve the above-mentioned problems, the present invention provides an automatic driving system mounted on an automobile for selectively performing automatic driving and manual driving of the automobile, for a user in the driver's seat of the automobile. A head-up display that displays a virtual image in front of the front window of the vehicle and a variable distance in the front-rear direction of the vehicle from the front window to the virtual image, and an obstacle detecting means for detecting an obstacle in front of the vehicle. When switching from the automatic driving to the manual driving, a control means for setting a display position in front of the automobile and displaying a virtual image of a predetermined figure at the set display position on the head-up display is provided. In the control means, when the obstacle detecting means detects the obstacle, the display position is set so that the display position is the position of the obstacle in the up, down, left, right, front and back directions of the automobile. It is a thing.

Here, when there is an obstacle in front of the automobile, the driver's gaze point suitable for manual driving is the position of the obstacle. Therefore, in this automatic driving system, by displaying a virtual image of a predetermined figure at the position of an obstacle, the gaze point of the user in the driver's seat can be guided to a position suitable for manual driving.

Here, in the above automatic driving system, the head-up display may be configured to blink a virtual image of the predetermined figure.
Further, each of the above automatic driving systems is provided with a line-of-sight direction detecting means for detecting the line-of-sight direction of the user in the driver's seat, and the control means displays a virtual image of the predetermined figure on the head-up display. The switching from the automatic operation to the manual operation is suppressed at least until the line-of-sight direction detected by the line-of-sight direction detecting means is the direction in which the virtual image displayed at the display position is viewed. It is also preferable to do so.

By doing so, it can be confirmed that the gaze point of the user in the driver's seat has reached the display position which is a position suitable for manual driving, so that the user's state is prior to switching from automatic driving to manual driving. Can be more accurately determined whether or not is in a state suitable for manual operation.

Further, each of the above automatic driving systems is provided with a line-of-sight direction detecting means for detecting the line-of-sight direction of the user in the driver's seat, and when the control means switches from the automatic driving to the manual driving, the head-up display is displayed. Prior to displaying the virtual image of the predetermined figure at the set display position, the head of the predetermined figure is placed at the initial display position which is the position in the line-of-sight direction detected by the line-of-sight direction detecting means at that time. It may be configured to be displayed on the up display and then to move the position of the virtual image of the predetermined figure from the initial display position to the set display position on the head up display.

Further, in this case, in the control means, the head-up display is made to blink and display the virtual image of the predetermined figure at the initial display position, and then from the initial display position of the position of the virtual image of the predetermined figure. With the movement to the set display position, the head-up display may be configured to gradually reduce the blinking speed in the blinking display of the virtual image of the predetermined figure.

Further, the distance in the front-rear direction of the vehicle from the front window to the initial display position may be smaller than the distance in the front-rear direction of the vehicle from the front window to the display position. ..

By configuring the automatic driving system as described above, the user's gaze point in the driver's seat can be smoothly guided from the position in the line-of-sight direction at that time to the display position which is a position suitable for manual driving. become.

Here, also in the automatic driving system that displays the virtual image of the predetermined figure at the initial display position as described above, the virtual image of the predetermined figure is displayed at the display position set on the head-up display by the control means. After this, switching from the automatic operation to the manual operation can be suppressed at least until the line-of-sight direction detected by the line-of-sight direction detecting means is the direction in which the virtual image displayed at the display position is viewed. preferable.

As described above, according to the present invention, in the automatic driving system, when switching from the automatic driving to the manual driving, the user's state can be more effectively guided to the state suitable for the manual driving. Further, according to the present invention, it is possible to more accurately determine whether or not the user's state is suitable for manual operation.

It is a block diagram which shows the structure of the in-vehicle system which concerns on embodiment of this invention. It is a figure which shows the arrangement of the head-up display and the line-of-sight detection camera which concerns on embodiment of this invention. It is a flowchart which shows the manual operation switching process which concerns on embodiment of this invention. It is a figure which shows the display example of the head-up display which concerns on embodiment of this invention. It is a figure which shows the display example of the head-up display which concerns on embodiment of this invention. It is a figure which shows the display example of the head-up display which concerns on embodiment of this invention.

Hereinafter, embodiments of the present invention will be described.
FIG. 1 shows the configuration of the in-vehicle system according to the present embodiment.
The in-vehicle system is a system installed in an automobile, and as shown in the figure, the in-vehicle system includes a line-of-sight detection camera 1, a head-up display 2, a GPS receiver, a broadcast receiver, a storage device, a disk drive, an input device, and the like. It is provided with various peripheral devices 3, a voice output device 4, a manual driving posture detection system 5, a peripheral monitoring system 6, and a control device 7 for controlling each of the above parts.

The control device 7 is connected to an automatic driving system 100 that selectively performs automatic driving and manual driving of the automobile.
Here, the automatic operation system 100 includes an automatic operation mode and a manual operation mode as operation modes, and the automatic operation mode is an operation mode in which all various operations such as steering, braking, and acceleration are automatically controlled. The manual operation mode is an operation mode in which all automatic control of various operations such as steering, braking, and acceleration, or some operations of the various operations are not automatically controlled.

Then, the automatic driving system 100 automatically controls various operations such as steering, braking, and acceleration of the automobile in a form according to the driving mode while switching the driving mode according to the driving situation and the surrounding situation.

When the manual driving mode is set as the driving mode, the user must monitor the surrounding traffic conditions by himself / herself, and the user himself / herself needs to operate the operation of the automobile which is not automatically controlled by the automatic driving system 100. There is. On the other hand, when the automatic driving mode is set as the driving mode, the user does not need to operate the vehicle and is freed from the need to monitor the surrounding traffic conditions.

Next, in the manual driving posture detection system 5, the user's posture is manually driven based on whether the user's hand is holding the steering wheel of the car, whether the user's foot is placed on the pedal of the car, and the like. Detects whether you are in a position to do.

Further, the peripheral monitoring system 6 detects obstacles such as other vehicles existing in the vicinity of the automobile by using a radar, a camera, or the like.
Further, the line-of-sight detection camera 1 is arranged on a dashboard or the like, for example, as shown in FIG. 2a, and captures an image of the user's face including the user's eyeball.
The head-up display 2 is a display that projects an image from below onto the front window in front of the driver's seat, and the virtual image 200 based on the image projected on the front window is superimposed on the front view of the driver's seat. It is visible to the user in front of the car.

Here, in the following, the distance from the front window to the virtual image 200 will be referred to as a "display distance", and the position of the virtual image 200 will be referred to as a "display position".
Here, as shown in FIGS. 2a and 2b, the head-up display 2 according to the present embodiment is a head-up display 2 having a variable display distance, and the head-up display 2 displays according to the control from the control device 7. Set the distance.

Hereinafter, the manual operation switching process performed by the control device 7 in such a configuration will be described.
FIG. 3 shows the procedure of this manual operation switching process.
As shown in the figure, in the manual operation switching process, the control device 7 monitors the reception of the manual operation switching permission request from the automatic operation system 100 (step 302).
Here, when the operation mode is switched from the automatic operation mode to the manual operation mode, the automatic operation system 100 issues a manual operation switching permission request to the control device 7 prior to switching to the manual operation mode. Then, the automatic operation system 100 switches the operation mode from the automatic operation mode to the manual operation mode when the control device 7 responds with the manual operation switching permission.

When the control device 7 receives the manual driving switching permission request from the automatic driving system 100 (step 302), the control device 7 checks whether the peripheral monitoring system 6 has detected an obstacle within a predetermined distance in front of the vehicle (step 302). Step 304), if an obstacle is detected, the display position is set to the position of the obstacle detected by the peripheral monitoring system 6 (step 306), and the process proceeds to step 312.

On the other hand, if an obstacle is not detected within a predetermined distance in front of the vehicle (step 304), the current vehicle speed of the vehicle is acquired from the automatic driving system 100 (step 308), and the display position is set to the vehicle in the left-right direction. The center position in the left-right direction, the position at a predetermined height from the road surface in the up-down direction (for example, half the height of the standard vehicle height of a normal car), the distance according to the vehicle speed in the front-rear direction, and the distance to the front. The position is set (step 310), and the process proceeds to step 312. Here, in step 310, the display position in the front-rear direction is set to a position farther forward as the vehicle speed of the automobile increases. Further, in step 310, the display position in the left-right direction may be set to a predetermined position between the center position in the left-right direction of the automobile and the front of the driver's seat.

Then, when the process proceeds from step 306 or step 310 to step 312, the head-up display 2 displays a predetermined marker (graphic) so that the virtual image 200 of the marker can be visually recognized at the display position by the user in the driver's seat. .. That is, the display distance of the head-up display 2 is set to the distance to the display position in the front-back direction, and the position of the marker in the image projected by the head-up display 2 is operated by the virtual image 200 of the marker in the up-down, left-right directions. After setting the position to be visually recognized by the user in the seat, the marker is displayed on the head-up display 2.

Then, while acquiring the line-of-sight direction of the driver's seat user detected by the line-of-sight detection camera (step 314), it waits for the line-of-sight direction of the driver's seat user to be in the direction of viewing the virtual image 200 of the marker (step 316). ).

Then, if the line-of-sight direction of the user in the driver's seat is the direction of viewing the virtual image 200 of the marker (step 316), the voice output device 4 outputs a voice message to notify the user that the operation will be switched to manual operation soon. (Step 318).

Then, if the manual driving posture detection system 5 waits for the user's posture to detect that the user's posture is in the manual driving posture (step 320), and the user's posture becomes the manual driving posture, then A manual operation switching permission is issued to the automatic operation system 100 (step 322), and the process returns from step 302.

Here, as described above, the automatic operation system 100 switches the operation mode from the automatic operation mode to the manual operation mode when the control device 7 responds with the manual operation switching permission.
The manual operation switching process performed by the control device 7 has been described above.
According to such a manual operation switching process, when the automatic operation system 100 switches from the automatic operation mode to the manual operation mode, the following markers are displayed and the markers are visually recognized prior to the switching to the manual operation mode. Is confirmed.

That is, when an obstacle exists within a predetermined distance in front of the automobile, as shown in FIG. 4a1, the head-up display 2 visually confirms that the obstacle exists at the position of the obstacle (preceding vehicle 500 in the figure). The marker 400 is displayed. Here, the display distance d of the head-up display 2 in this case is set so as to match the distance from the front window of the automobile to the obstacle, as shown in FIG. 4a2.

Here, when an obstacle exists within a predetermined distance in front of the automobile, the driver's gaze point suitable for manual driving is the position of the obstacle.
Therefore, according to the present embodiment, the marker 400 can guide the gaze point of the user in the driver's seat to the position of the obstacle also in the front-rear direction of the automobile.
Further, when the line-of-sight direction of the driver's seat user is in the direction of the marker 400, it is considered that the user is gazing at the marker 400, that is, the gazing point of the user is the position of the obstacle. By detecting that the line-of-sight direction of the user in the driver's seat is in the direction of the marker 400 as in the present embodiment, it is possible to appropriately determine that the user's gaze position is in a position suitable for manual operation. Will be able to.

Next, if there is no obstacle within a predetermined distance in front of the vehicle, as shown in FIGS. 4b1 and 4c1, the center position (or the center position and the front of the driver's seat) in the left-right direction of the vehicle in the left-right direction. A marker 400 that is visually recognized as having a position at a predetermined height from the road surface in the vertical direction is displayed.

Here, FIG. 4b1 shows the display of the marker 400 when the vehicle speed of the vehicle is relatively slow, and FIG. 4c1 shows the display of the marker 400 when the vehicle speed of the vehicle is relatively high.
As shown in FIG. 4b2 and the display distance d when the vehicle speed of the vehicle is relatively slow, and the display distance d when the vehicle speed of the vehicle is relatively fast in FIG. 4c2, the faster the vehicle speed of the vehicle, the larger the display distance d. The marker 400 is displayed so that it can be visually recognized at a distant position in front of the vehicle as the vehicle speed increases.

Here, when there is no obstacle, the driver's gaze point suitable for manual driving is a position farther forward as the speed of the vehicle increases.
Therefore, according to the present embodiment, the marker 400 can guide the distance to the gazing point of the user in the driver's seat to a distance suitable for manual driving according to the speed of the automobile.
Further, when the line-of-sight direction of the driver's seat user is in the direction of the marker 400, the user is gazing at the marker 400, that is, the distance to the gazing point of the user is relative to the speed of the automobile at that time. Therefore, it is considered that the distance is suitable for manual driving. Therefore, as in the present embodiment, by detecting that the line-of-sight direction of the user in the driver's seat is in the direction of the marker 400, the gaze position of the user can be set. It becomes possible to appropriately determine whether or not the vehicle is in a position suitable for manual operation.

The embodiment of the present invention has been described above.
By the way, in the manual operation switching process in the above embodiment, the marker 400 may be blinked and displayed.
Alternatively, in the manual operation switching process in the above embodiment, the marker 400 may be displayed as follows.
That is, in step 312 of the manual operation switching process shown in FIG. 3, first, as shown in FIGS. 5a1 and 5a2, the direction seen from the driver's seat user of the marker 400 is the line-of-sight direction of the driver's seat user at that time. The marker 400 is blinked and displayed at a relatively fast blinking speed at a position where the distance (display distance d) in the front-rear direction to the marker 400 is relatively small.

When no obstacle is detected in step 304, the display of the marker 400 is moved from the position of FIGS. 5a1 and 5a2 to the display position calculated in step 310 while gradually reducing the blinking speed. .. That is, the display position of the marker 400 and the blinking speed are changed from FIGS. 5a1 and a2 to FIGS. 5b1, b2 and 5b1 and b2 to 5c1 and bc2.

Then, when the marker 400 is moved to the display position calculated in step 310, the process proceeds to step 314.
On the other hand, when an obstacle is detected in step 304, the display of the marker 400 is moved from the position of FIGS. 5a1 and 5a2 to the display position calculated in step 306 while gradually reducing the blinking speed. .. Then, when the marker 400 is moved to the display position calculated in step 306, the process proceeds to step 314.

By doing so, the marker 400 can smoothly guide the gaze point of the user in the driver's seat from the position in the line-of-sight direction at that time to the position suitable for manual driving.

Further, in the manual operation switching process of the above embodiment, prior to switching to the manual operation mode, the display of the marker 400 is moved up, down, front, back, left and right as shown in FIG. May be done.

That is, for example, if it can be detected in step 316 that the line-of-sight direction of the user is the direction in which the virtual image 200 of the marker 400 is viewed, the display of the marker 400 is moved up, down, front, back, left, and right for a certain period of time. Then, the display position of the marker 400 is returned to the display position where the marker 400 is displayed in step 312, and then, if the line-of-sight direction of the user in the driver's seat is the direction in which the virtual image 200 of the marker 400 is viewed, the subsequent steps 318 are performed. You may proceed to the process.

1 ... Camera for line-of-sight detection, 2 ... Head-up display, 3 ... Seed peripheral device, 4 ... Voice output device, 5 ... Manual driving posture detection system, 6 ... Peripheral monitoring system, 7 ... Control device, 100 ... Automatic driving system, 200 ... virtual image, 400 ... marker.

Claims (4)

It is an automatic driving system installed in a car that selectively performs automatic driving and manual driving of the car.
A head-up display that displays a virtual image in front of the front window of the vehicle for the user in the driver's seat of the vehicle and has a variable distance in the front-rear direction of the vehicle from the front window to the virtual image.
When switching from the automatic driving to the manual driving, a control means for setting a display position in front of the automobile and displaying a virtual image of a predetermined figure on the head-up display at the set display position .
It has a line-of-sight direction detecting means for detecting the line-of-sight direction of the user in the driver's seat.
The control means sets the display position so that the distance in the front-rear direction of the vehicle from the front window to the display position increases as the vehicle speed of the vehicle increases, and the virtual image of the predetermined figure is displayed. After displaying the head-up display at the set display position, the automatic operation is performed until at least the line-of-sight direction detected by the line-of-sight direction detecting means is the direction for viewing the imaginary image displayed at the display position. An automatic operation system characterized by suppressing switching from the above to the manual operation. It is an automatic driving system installed in a car that selectively performs automatic driving and manual driving of the car.
A head-up display that displays a virtual image in front of the front window of the vehicle for the user in the driver's seat of the vehicle and has a variable distance in the front-rear direction of the vehicle from the front window to the virtual image.
When switching from the automatic driving to the manual driving, a control means for setting a display position in front of the automobile and displaying a virtual image of a predetermined figure on the head-up display at the set display position.
It has a line-of-sight direction detecting means for detecting the line-of-sight direction of the user in the driver's seat.
The control means is
When switching from the automatic operation to the manual operation, the head-up display is in the line-of-sight direction detected by the line-of-sight direction detecting means at that time prior to displaying the virtual image of the predetermined figure at the set display position. A virtual image of the predetermined figure is blinked on the head-up display at the initial display position, which is a position, and then the position of the virtual image of the predetermined figure is set on the head-up display from the initial display position. While moving to the position, as the position of the virtual image of the predetermined figure moves from the initial display position to the set display position, the head-up display shows the blinking of the virtual image of the predetermined figure. An automatic driving system characterized by tapering the speed. The automatic driving system according to claim 2.
An automatic driving system characterized in that the front-rear distance of the vehicle from the front window to the initial display position is smaller than the front-rear distance of the vehicle from the front window to the display position. The automatic driving system according to claim 2 or 3.
After displaying the virtual image of the predetermined figure on the head-up display at the set display position, the control means displays at least the line-of-sight direction detected by the line-of-sight direction detecting means at the display position. An automatic operation system characterized in that switching from the automatic operation to the manual operation is suppressed until the direction of viewing the virtual image is reached.

Images