JP2007280203A - Information presenting device, automobile and information presenting method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To quickly present information required by a driver in detail. <P>SOLUTION: When the driver turns on an information request switch 7, the environment in front of a vehicle photographed by a front camera 3 is recognized (step S1), and a potential P representing the degree of driver's desire to an enlarged image is calculated by each front object (step S2). Among the front objects, the front object whose potential P is equal to or more than a predetermined value is selected (step S3), and further among them, only remote objects which can be seen by the driver and disabled from being recognized are selected (step S4). An enlarged image of the thus selected remote object is displayed in a position having a lower potential P in the vicinity of the remote object by a HUD 6 (step S6). <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an information presentation apparatus, an automobile equipped with the information presentation apparatus, and an information presentation method.

When an obstacle is detected, this obstacle is highlighted on the head-up display to alert the driver, and it is determined that the driver has recognized the obstacle based on the driver's line of sight. Then, in order to reduce annoyance, there existed what switched to the display which weakened emphasis of the obstruction (refer patent document 1).
JP 2005-135037 A

However, in the conventional example described in Patent Document 1 described above, it is merely avoiding unnecessary presentation of information already recognized by the driver. There is room for improvement from the perspective of strengthening the vehicle and expanding driving margins.
An object of the present invention is to quickly present details required by a driver in detail.

  In order to solve the above problems, an information presentation apparatus according to the present invention images an environment in front of a vehicle, selects a distant object that cannot be recognized by the driver from the captured image, and displays an enlarged image of the selected distant object. It is characterized by displaying.

  According to the information presenting apparatus according to the present invention, it is possible to quickly present details required by the driver by displaying an enlarged image of a distant object that cannot be recognized by the driver.

Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings.
<< One Embodiment >>
"Constitution"
FIG. 1 is a schematic configuration of the present invention. The automobile 1 includes, for example, a controller 2 constituted by a microcomputer, a front camera 3 provided on an inside mirror, for example, for imaging an environment ahead of the vehicle, and a driver camera 4 provided, for example, on a steering column, for imaging a driver's eyeball. And a head-up display (hereinafter referred to as HUD) 6 for projecting a display beam onto a predetermined range of the front window glass 5 to display an image.

When the information request switch 7 provided on the steering wheel 7 is turned on by the driver, the controller 2 executes the information presentation process of FIG. 2 at a predetermined sampling period.
First, in step S1, the environment ahead of the vehicle is recognized based on the image captured by the front camera 3.
In the subsequent step S2, the potential calculation process of FIG. 3 described later is executed, and for each forward object recognized in step S1, the degree of desire that the driver desires to display the enlarged image is determined. Is calculated as the potential P.
In the subsequent step S3, a front object having a potential P of a predetermined value or more is selected as an enlarged image display target.

In the subsequent step S4, only the distant object whose presence can be recognized by the driver and whose details cannot be recognized is selected from the front objects selected as the display target. Here, a distant object that is recognizable and unrecognizable is an object that is so far away that the area S on the image data is a predetermined value S ₁ or more and a predetermined value S ₂ or less, as shown in FIG. (S ₁ ≦ S ≦ S ₂ ). That is, if S <S ₁ , it is determined that the presence cannot be recognized, and if S> S ₂ , the details are determined to be recognizable. Of course, it is desirable that the predetermined values S ₁ and S ₂ are not uniform, but are changed according to the type of the front object.

In the subsequent step S5, the display position of the enlarged image is determined. Here, the position is determined in the vicinity of a distant object to be displayed as viewed from the driver and at a position where the potential P does not overlap with a front object having a potential P or higher, that is, a position having a low potential P.
In the subsequent step S6, the enlarged image is displayed by the HUD 6, and then the process returns to the predetermined main program.

Next, the potential calculation process of FIG. 3 will be described.
First, in step S21, the weight W is calculated for each front object according to the type and the type of the traveling road. Here, the front objects are classified into five types: “traffic signs”, “route guidance signs”, “cars”, “people”, and “signboards”, and the driving roads are “highways” and “intercity roads”. , “City road” and “residential road”. Then, as shown in FIG. 5, for example, by increasing the weight W of the “route guide sign” on the “highway” and increasing the weight W of “people” on the “residential road”, it is optimal for the driving scene. A simple weight W is calculated. FIG. 6 is an example of “intercity road”.

The information request switch 7 may be provided with an adjustment function that allows the driver to select a specific front object and change its weight W. For example, when the driver increases the weight W of “person”, this adjustment is prioritized, and the weight W of “person” is set to the maximum value as shown in FIG. Should be set.
In subsequent step S22, when calculating the potential P, it is determined whether to invalidate or validate the driver's line-of-sight direction. This may be determined by the driver's input through the information request switch 7, or may be determined based on whether or not a condition for detecting the line-of-sight direction is satisfied. Here, when invalidating the line-of-sight direction, the process proceeds to step S26 described later. On the other hand, when validating the line-of-sight direction, the process proceeds to step S23.

In step S23, the driver's line of sight is detected. For example, the eyeball center and the pupil center are calculated based on the image data captured by the driver camera 4, and the driver's line of sight is detected by calculating a vector from the eyeball center to the pupil center.
In subsequent step S24, the angle potential A is calculated for each front object according to the driver's line of sight and field of view. FIG. 8 shows a general field of view of the Japanese, and the gaze, static, and dynamic fields are determined centering on the viewpoint ahead of the line of sight, so refer to the control map of FIG. The angle potential A is calculated according to the viewing angle of the object. As shown in FIG. 9, the control map is set so that the angle potential A decreases in the range of 1 to 0 as the viewing angle increases (gazing angle <static viewing angle <dynamic viewing angle).

Since the driver's field of view also changes depending on the vehicle speed, referring to the control map of FIG. 10, the gaze angle, the static vision angle, and the dynamic viewing angle are set to decrease as the vehicle speed increases.
In the subsequent step S25, as shown in the following equation (1), the potential P is calculated for each forward object by multiplication of the weight W and the angle potential A, and then the potential calculation process in FIG. Here, x and y represent coordinates on the image data.
P (x, y) = W (x, y) × A (x, y) (1)
On the other hand, in step S26, the angle potential A of all front objects is set to “1” regardless of the driver's line-of-sight direction, and then the process proceeds to step S25.

<Action>
Now, it is assumed that the driver desires to present information on a distant object by turning on the information request switch 7.
At this time, the environment in front of the vehicle imaged by the front camera 3 is recognized (step S1), and for each front object, a potential P representing the degree of desire for the enlarged image of the driver is calculated (step S2). Then, a front object having a potential P equal to or higher than a predetermined value is selected (step S3), and only a far object that is recognizable and unrecognizable by the driver is selected (step S4). By displaying the enlarged image of the far object selected in this way at a position near the far object and at a low potential P by the HUD 6 (steps S5 and S6), information required by the driver can be quickly detailed. To present.

The potential P is calculated using two elements, the weight W and the angle potential A (step S25).
For example, when driving on an expressway, interest in route guidance signs increases, and when driving on a residential road, interest in people increases. The degree of interest is different. Therefore, the weight W as the first element is determined according to the type of the front object and the type of the traveling road (step S21).

Also, if the interest is high, the driver's line of sight is naturally directed there, so the angle potential A as the second element is determined according to the driver's line of sight and field of view (steps S23 and S24). In particular, the potential P of an object that is out of the driver's gaze is reduced. At this time, an accurate visual field is calculated by changing a gaze angle, a static visual angle, and a dynamic visual angle that specify each of the driver's gaze field, static field, and dynamic field of view according to the vehicle speed.
However, when the driver desires invalidity of the line-of-sight direction or when the line-of-sight detection accuracy is low (“No” in step S22), the angle potential A is set to “1” (step S26). The driver's line-of-sight direction is invalidated for the calculation of the potential P.

  FIG. 11 is a display example of the calculation result of the potential P when the line-of-sight direction is invalidated and an enlarged image according to the calculation result. The magnitude of the potential P is represented by the brightness of the color. This means that the potential P is high. Here, a state where the potential P of the “signboard” on the front left side, the “person” on the front right side, and the “route guidance sign” is greater than or equal to a predetermined value is shown (white area), and each enlarged image is displayed on the HUD 6. it's shown. In the case of “person”, at least the head above the shoulder is enlarged and displayed as an image. This makes it possible to recognize the contents of “signboards” and “route guidance signs” and the face of “people” even if they are so far away that it is difficult for the driver to recognize them. In addition, since the enlarged image is displayed in the vicinity of the distant object and in the region having a low potential P (black region), the display target and the object having a high potential P existing in the vicinity thereof are not hidden.

  FIG. 12 is a display example of a potential P calculation result when the line-of-sight direction is validated and an enlarged image according to the calculation result. Here, the driver's line of sight is directed toward the “right” of the front right side, and the potential P of the “signboard” on the left front side and the “route guidance sign” on the front right side is less than a predetermined value. Only the enlarged image of “person” whose P is equal to or greater than a predetermined value is displayed on the HUD 6. This avoids unnecessary presentation of an enlarged image and prevents the driver from feeling uncomfortable or increasing power consumption.

As the vehicle moves forward, the image data captured by the front camera 3 becomes clearer as the vehicle gradually approaches a distant object, and the enlarged image displayed on the HUD 6 becomes gradually clearer. When the distant object displaying the enlarged image approaches to a distance that can be recognized by the driver, the object is not recognized as a distant object that requires the display of the enlarged image, and the display of the enlarged image is terminated.
In addition, by providing the front camera 3 that captures the environment ahead of the vehicle on an inside mirror or the like that is higher than the viewpoint of the driver, a more stereoscopic image (bird view image) can be presented. .

《Application example》
In the above embodiment, the driver's eye is captured by the driver camera 4 and the driver's line of sight is detected based on the image data. However, the driver's line of sight is determined based on the driver's line-of-sight movement characteristics. May be estimated. That is, as shown in FIG. 13, if the steering operation is performed, the line of sight moves left and right, and if the vehicle speed increases, the line of sight moves upward. Therefore, the line of sight in the left and right direction is calculated (estimated) from the steering angle. Then, the line of sight in the vertical direction may be calculated (estimated) from the vehicle speed. According to this, it is possible to reduce the calculation burden compared to detecting the line-of-sight direction by image processing, and to easily calculate the driver's line-of-sight direction. Of course, either one of the detection result based on the image processing and the estimation result based on the steering angle and the vehicle speed may be selectively used, or both may be used.

"effect"
From the above, the front camera 3 corresponds to “imaging means”, the processing in steps S1 to S4 corresponds to “selection means”, the processing in HUD6 and steps S5 and S6 corresponds to “display means”, and step S23. , S24 corresponds to “calculation means”.
(1) An imaging unit that captures an environment in front of the vehicle, a selection unit that selects a distant object that cannot be recognized by the driver from an image captured by the imaging unit, and an enlarged image of the distant object selected by the selection unit are displayed. Display means.
In this way, by displaying an enlarged image of a distant object that cannot be recognized by the driver, the information required by the driver can be presented quickly and in detail, thereby enhancing the prediction ability and expanding the driving margin. To contribute.

(2) For each distant object that cannot be recognized by the driver, the selection means determines the degree of hope that the driver desires to display an enlarged image, and selects only distant objects whose determined degree of hope is equal to or greater than a predetermined value. To do.
Thereby, since only the enlarged image of the distant object determined to have a high degree of hope is displayed, only the information required by the driver can be presented. Therefore, it is possible to avoid unnecessary presentation of the enlarged image and to prevent the driver from feeling uncomfortable or increasing power consumption.

(3) The selection means determines the desired degree according to the type of the distant object and the type of the traveling road.
Thereby, it is possible to determine an appropriate degree of desire according to the traveling environment.
(4) A calculation unit that calculates the driver's line of sight and visual field is provided, and the selection unit determines a desired degree according to the driver's line of sight and visual field calculated by the calculation unit.
As a result, only the information required by the driver can be presented. Therefore, it is possible to avoid unnecessary presentation of the enlarged image and to prevent the driver from feeling uncomfortable or increasing power consumption.

(5) The calculating means calculates the visual field of the driver according to the vehicle speed.
Thereby, an accurate visual field can be calculated.
(6) The calculation means calculates the line of sight in the left-right direction according to the steering angle, and calculates the line of sight in the vertical direction according to the vehicle speed.
According to this, it is possible to reduce the calculation burden and more easily estimate the driver's line of sight than to detect the line of sight by image processing.

(7) The display means includes a head-up display capable of displaying an image superimposed on the environment in front of the vehicle as viewed from the driver, and the driver displays an enlarged image of the distant object selected by the selection means with the head-up display. When viewed from the distance, the object is displayed in the vicinity of the distant object and at a position that does not overlap with the distant object having the desired degree of the predetermined value or more.
As a result, it is possible to prevent the display object of the enlarged image and the object having a high degree of hope existing in the vicinity thereof from being hidden by the enlarged image.

(8) The display means displays an enlarged image of the distant object selected by the selection means when detecting the request operation of the driver who requests display of the enlarged image, that is, when the information sudden operation switch 7 is turned on.
As a result, the enlarged image can be displayed only when the driver desires to present information on a distant object, so that information can be presented in accordance with the driver's desire.
(9) The imaging unit images the environment ahead of the vehicle from a position higher than the driver's viewpoint.
Thereby, it is possible to present a more stereoscopic and easy-to-understand image (bird view image).

It is a schematic block diagram of this invention. It is a flowchart of an information presentation process. It is a flowchart of a potential calculation process. This is the relationship between the object area and visibility. A weight W corresponding to the object type and the road type. This is the weight W on the toll road. It is the weight W on the toll road after the driver's designation operation. This is a general field of view. An angle potential A corresponding to the viewing angle. The viewing angle depends on the vehicle speed. It is the example of a display of the enlarged image according to the calculation result of the potential P when invalidating a gaze direction, and the calculation result. It is the example of a display of the enlarged image according to the calculation result of the potential P when the gaze direction is validated, and the calculation result. It is the line-of-sight direction according to the steering angle and the vehicle speed.

Explanation of symbols

1 Car 2 Controller 3 Front Camera 4 Driver Camera 6 Head Up Display (HUD)
7 Information request switch

Claims (12)

  An image pickup means for picking up an environment in front of the vehicle, a selection means for selecting a distant object that cannot be recognized by the driver from an image picked up by the image pickup means, and a display for displaying an enlarged image of the distant object selected by the selection means And an information presentation device.   The selection means determines the degree of desire that the driver desires to display an enlarged image for each distant object that cannot be recognized by the driver, and selects only distant objects whose determined degree of hope is equal to or greater than a predetermined value. The information presentation apparatus according to claim 1.   The information presenting apparatus according to claim 2, wherein the selection unit determines the desired degree according to a type of the distant object and a type of a traveling road. A calculation means for calculating the driver's line of sight and visual field;
The information presentation apparatus according to claim 2, wherein the selection unit determines the desired degree in accordance with a driver's line of sight and visual field calculated by the calculation unit.   The information presentation apparatus according to claim 4, wherein the calculation unit calculates a driver's field of view according to a vehicle speed.   The information presentation apparatus according to claim 4, wherein the calculating unit calculates a gaze in the left-right direction according to a steering angle, and calculates a gaze in the vertical direction according to a vehicle speed.   The display means includes a head-up display capable of displaying an image superimposed on an environment ahead of the vehicle as viewed from the driver, and the head-up display displays an enlarged image of a distant object selected by the selection means. The information presentation according to any one of claims 2 to 6, wherein the information is displayed in the vicinity of the distant object and at a position that does not overlap with the distant object having the desired degree equal to or greater than a predetermined value. apparatus.   The said display means displays the enlarged image of the distant object which the said selection means selected, if the driver | operator's request | requirement operation which requests | requires the display of an enlarged image is detected. The information presentation apparatus described in 1.   The information presenting apparatus according to claim 1, wherein the imaging unit images an environment in front of the vehicle from a position higher than the viewpoint of the driver.   An information presentation apparatus that images an environment in front of a vehicle, selects a distant object that cannot be recognized by a driver from the captured image, and displays an enlarged image of the selected distant object. In a car equipped with an information presentation device,
The information presentation device includes:
An image pickup means for picking up an environment in front of the vehicle, a selection means for selecting a distant object that cannot be recognized by the driver from an image picked up by the image pickup means, and a display for displaying an enlarged image of the distant object selected by the selection means And a vehicle.   An information presentation method comprising imaging an environment in front of a vehicle, selecting a distant object that cannot be recognized by a driver from the imaged image, and displaying an enlarged image of the selected distant object.

Images