JP7057893B2 - Visual condition judgment device - Google Patents

Description

The present invention relates to a visual state determination device.

Patent Document 1 discloses that whether or not a viewer is in a tense state is estimated based on the line-of-sight direction, line-of-sight distribution, change in pupil diameter, and the like.

Japanese Patent Application Laid-Open No. 2002-367100

By the way, it is desired to determine the degree of immersion of the visual object with respect to the visual object (which can be said to be the degree of interest in the visual object). For example, in the case of a vehicle driver as a visual observer, a high degree of immersion in a visible object in front of the vehicle is preferable in that he / she pays sufficient attention to the situation in front of the vehicle, but on the other hand, a rear-view mirror, a side mirror, and a meter. It also shows an unfavorable tendency that it is easy to overlook something that should be visually recognized, such as a panel.

The present invention has been made in consideration of the above circumstances, and an object of the present invention is to provide a visual recognition state determining device capable of determining how much a viewer is immersed in a visual object. ..

In the present invention, basically, in the case of active visual recognition in which the viewer positively visually recognizes the visual object by his / her own will, the actual distance to the visual object and the visual object are the visual object. It was made based on the finding that the difference between the actual distance and the viewing distance is large when the person is passively visually recognizing, while the difference between the viewing distance and the viewing distance is small. Then, the viewing distance is determined based on the convergence angle formed by the lines of sight of the left and right eyes of the viewer.

Specifically, the following solution method is adopted in the present invention. That is, as described in claim 1,
Convergence angle acquisition means for acquiring the convergence angle formed by the line of sight of the left and right eyes of the viewer when the viewer is visually observing the visual object.
An actual distance detecting means for detecting the actual distance from the viewer to the visual object, and
Based on the first evaluation index set corresponding to the congestion angle acquired by the congestion angle acquisition means and the second evaluation index set corresponding to the actual distance detected by the actual distance detecting means. An immersive degree determining means for determining the degree to which the viewer is immersed in the visual object,
Equipped with
The first evaluation index is set as the convergence angle acquired by the convergence angle acquisition means, and the first evaluation index is set.
The second evaluation index is set as a convergence angle corresponding to the actual distance detected by the actual distance detecting means.
It is done like this. According to the above solution method, it is possible to determine the degree of immersion in the visual object based on the actual distance and the convergence angle. In particular, the two evaluation indices can be compared by the value of the convergence angle.

A preferred embodiment premised on the above-mentioned solution method is as described in claim 2 and below .

The immersive degree determining means determines the immersive degree based on the difference between the first evaluation index and the second evaluation index (corresponding to claim 2 ). In this case, when the above difference is small, it can be determined that the user is in an active visual recognition state in which the visual object is sufficiently recognized when the actual distance and the visual distance are matched or almost matched. .. On the contrary, when the above difference is large, it can be determined that the visual object is in a passive visual state in which the visual object is not sufficiently visually recognized.

The immersive degree determining means has a high degree of immersiveness when the number of times the difference between the first evaluation index and the second evaluation index is equal to or less than a predetermined number of times within a predetermined time set in advance. (Corresponding to claim 3 ). In this case, the degree of immersion (tendency) can be accurately determined based on the data over a predetermined time.

A display means that is set to a position that can be visually recognized by the viewer and whose display brightness can be changed is provided.
When the immersive degree determination means determines that the immersive degree is low, the display brightness of the display means is increased.
(Corresponding to claim 4 ). In this case, the degree of immersion can be increased by increasing the display brightness.

The display means is installed in the vehicle, and the viewer is regarded as the driver of the vehicle.
The display brightness of the display means is increased provided that the vehicle is automatically driven or stopped.
(Corresponding to claim 5 ). In this case, since the display brightness is increased on condition that the vehicle is automatically driven or stopped, the situation where the display brightness is increased while the vehicle is running causes the user to immerse only in a part of the visible area where the display brightness is increased. It is preferable to prevent it and make the surrounding situation visible evenly.

The congestion angle acquisition means has an image pickup means for photographing the face portion of the viewer, and acquires the convergence angle based on the image obtained by the image pickup means (corresponding to claim 6 ). In this case, the image pickup means can be used to accurately detect the convergence angle, that is, estimate the viewing distance.

The viewer is regarded as the driver of the vehicle.
The image pickup means is provided in the vehicle interior.
(Corresponding to claim 7 ). In this case, the degree of immersion of the driver of the vehicle can be determined.

According to the present invention, it is possible to determine how much the viewer is immersed in the visual object.

The simplified plan view which shows an example of the vehicle to which this invention is applied. A simplified plan view showing the relationship between the convergence angle and the actual distance. A characteristic diagram showing the correspondence between the convergence angle and the viewing distance. The figure which shows the distance correction according to the difference of a seat position. The figure which divides the visibility area of a driver into a plurality of small areas, and shows the main visual object in each small area. The figure which shows the situation which gazes at a display. The block diagram which shows the example of the control system of this invention. The flowchart which shows the control example of this invention. The flowchart which shows the control example of this invention. The figure which shows the setting example of the brightness increase amount of a display screen according to the degree of immersion.

Hereinafter, an embodiment of the present invention will be described with respect to a case where a visual person who is a target person for determining the degree of immersion is a driver who drives a car as a vehicle.

FIG. 1 is a simplified view of the driver's seat 1 of the vehicle V as viewed from above. The driver P is seated in the driver's seat 1. A display 3 is arranged as a display means on the upper surface of the instrument panel 2 extending in the vehicle width direction in front of the driver's seat 1 at a position easily visible to the driver P. The display 3 is used, for example, for a navigation device, and in addition to displaying map information, various warning information and the like are appropriately switched and displayed (display in a balloon format or the like may be used). The display brightness of the display 3 can be automatically changed.

The driver P is equipped with a glasses-type convergence angle detecting device 10. The convergence angle detecting device 10 has, for example, one or two eye cameras, and detects a convergence angle θ0, which is an angle formed by the line-of-sight directions of the left and right eyes, as shown in FIG. In FIG. 2, the visual object is, for example, a display 3. Further, the convergence angle detecting device 10 detects the distance between the eyes of the driver P. It should be noted that the detection of the convergence angle θ0 and the distance between the eyes is not performed by the glasses type, but is equipped on, for example, the leading edge of the roof panel, the rear-view mirror, etc., and the face of the driver P (particularly both eyes). It can be done by an appropriate method, such as by using an in-vehicle camera that is oriented toward.

When the distance between both eyes is the same, the smaller the convergence angle θ0, the larger the distance to the visual object (display 3 in FIG. 2), that is, the visual distance (conversely, the larger the convergence angle θ0, the larger the visual object. Distance, that is, the viewing distance becomes smaller). The relationship between the convergence angle θ0 and the corresponding viewing distance is shown by, for example, a characteristic line as shown in FIG. 3 when the distance between the eyes is set to a certain constant value. Specifically, assuming that the viewing distance is L and the distance between both eyes is D, the relationship between the convergence angle θ0 and the viewing distance L is as shown in the following equation (1).

L = (D / 2) × (1 / tan (θ0 / 2)) (1)
In FIG. 5, the front viewing area by the driver P is divided into a plurality of vertically and horizontally divided small areas (in the embodiment, a total of nine small areas of vertical 3 × horizontal 3). The main visual objects in each small area are listed in FIG.

FIG. 6 shows a situation in which the driver P is gazing at a small area where the display 3 exists. The part indicated by the broken line in the figure is the range where the center point per unit time exists, and one circle with a hollow indicates the center of the gazing point distribution.

In FIG. 6, since the position of the display 3 is fixed, the actual distance between the specific position of the driver's seat 1 and the display 3 can be known in advance. That is, the distance between the eye point position of the driver P seated in the driver's seat 1 and the display 3 can be known in advance. However, when the seat position of the driver's seat 1 changes in the front-rear direction, the distance between the eye point position of the driver P and the display 3 changes. The eye point position of the driver P is corrected as shown in FIG. 4 according to the position in the front-rear direction of the driver's seat 1. That is, as the front-rear position of the driver's seat 1, the position when the driver P has a standard physique is set as the standard position, and when the driver's seat 1 moves in the front-rear direction from the standard position, the distance of the movement is It will be corrected.

Specifically, the correction of the eye point position may be performed as follows. First, the standard position of the driver's seat 1 is X (coordinate position in the front-rear direction). In this case, when the driver's seat 1 is moved backward by "Xh1" from the standard position X, the eye point position of the driver P is set as "X + Xh1". On the contrary, when the driver's seat 1 is moved forward by "Xh2" from the standard position X, the eye point position of the driver P is set as "X-Xh2". Such correction of the eye point position according to the movement of the driver's seat 1 in the front-rear direction is performed not only for the display 3 but also for each viewing position such as a rear-view mirror and a meter.

FIG. 7 shows an example of the control system of the present invention. In the figure, U is a controller (control unit) configured by using a microcomputer mounted on a vehicle. In addition to the signal from the congestion angle detection device 10 described above, signals from various sensors S1 and S2 are input to the controller U. In the figure, S1 is a radar, which is provided in the vehicle interior near the front end of the vehicle or the upper end of the front window glass, and detects the actual distance to an object (visual object) located in front of the vehicle. It has become. Further, the controller U controls the display 3.

In FIG. 1, S2 is a seat position sensor, which detects the position in the front-rear direction of the seat cushion constituting the driver's seat 1. The eye point position of the driver P is specified according to the position in the front-rear direction of the driver's seat 1 detected by the seat position sensor S2. By specifying the eye point position of the driver P, the actual distance to various in-vehicle devices (for example, a navigation screen, a meter panel, a rear-view mirror, a side mirror, etc.) that are visible objects by the driver is specified. ..

Specifically, the controller U has a database DB that stores the actual distances to the various in-vehicle devices, assuming that the driver's seat 1 is in the standard position. Then, in this database DB, the eye point position is corrected as described above in correspondence with the position change of the driver's seat 1 in the front-rear direction from the standard position. The seat position sensor S2 and the database DB constitute a detection means for detecting the actual distance from the position of the driver's eyes to the visual object. When specifying (correcting) the eye point position, the height of the seat cushion of the driver's seat 1, the inclination angle of the seat back, and the like can also be taken into consideration.

The controller U determines the angle formed by the line of sight of the driver's left and right eyes, that is, the convergence angle θ0 from the image captured by the convergence angle detection device 10, and is a visual target by the driver based on this convergence angle θ0. It is designed to estimate the viewing distance to an object. As is known, it is determined that the smaller the convergence angle, the farther the object is visually recognized (conversely, the larger the convergence angle, the closer the object is visually recognized). Further, when the visual object is in front of the vehicle V, the actual distance to the visual object is detected by using the radar S1 (correction according to the front-rear position of the driver's seat 1). ).

When the viewing distance to the visual object estimated based on the convergence angle θ0 and the actual distance to the visual object are the same or almost the same, the driver P positively gazes at the visual object. It is in a state of being visually recognized, that is, in an active state. The longer the visual recognition in the active state continues, or the more repeatedly it occurs within a short period of time, the higher the degree of immersion (that is, the degree of interest) in the visual object is considered to be. On the other hand, when the above-mentioned viewing distance and the actual distance are significantly different from each other, the visual recognition is performed in the passive state (the state of being shown).

Next, a control example for determining the degree of immersion by the controller U will be described with reference to the flowcharts shown in FIGS. 8 and 9. 8 is a main flowchart, FIG. 9 is a sub flowchart, and the processing result in FIG. 9 is reflected in the control of FIG. Further, in the following description, Q indicates a step.

First, FIG. 9 is a control example for specifying a visual object, detecting the actual distance to the visual object, and converting the detected actual distance into the actual convergence angle θr. .. The control example of FIG. 9 will be described after the control example of FIG. 8 is described.

In FIG. 8 showing the main flowchart, first, in Q1, the convergence angle detection device 10 detects the convergence angle θ0 formed by the line of sight of the driver's left and right eyes. After that, in Q2, the absolute value of the deviation between the detected convergence angle θ0 and the actual convergence angle θr corresponding to the actual distance to the visual object described later is from a predetermined value A (1 degree in the embodiment). Is also determined whether it is small or not.

If NO in the determination of Q2 above, the product is returned as it is. If YES in the determination of Q2, the number of immersions Nb is counted up in Q3. Q The number of immersions Nb is cleared to 0 at the start of control. After 3, it is determined in Q4 whether or not a predetermined time (for example, 2 seconds to 3 seconds) has elapsed from the time when YES is first determined in Q2. If NO in the determination of Q4, it is returned.

If YES in the determination of Q4, it is determined in Q5 whether or not the number of immersions Nb is a predetermined value B (for example, 5 to 10 times) or more. If YES in the determination of Q5, it is determined in Q6 that the degree of immersion is large. In this Q5, the number of immersions Nb is cleared to 0 in preparation for the next determination (the measurement of the predetermined time used in Q4 is also cleared). It is also possible to display caution information so that the surrounding situation can be visually recognized evenly when the driver P continues to be highly immersive in the visual object (for example, display on the display 3).

If the determination in Q5 is NO, it is determined in Q7 that the degree of immersion is small. In this Q7, the number of immersions Nb is cleared to 0 in preparation for the next determination (the measurement of the predetermined time used in Q4 is also cleared).

After Q7, in Q8, it is determined whether or not the vehicle is currently in automatic driving or stopped. If the determination of Q8 is NO, the product is returned as it is. If YES in the determination of Q8, the screen average brightness of, for example, the display 3 as the display means is changed to be higher than the current brightness. As shown in FIG. 10, the higher the degree of immersion, the higher the degree of increasing the brightness.

Next, the flowchart of FIG. 9 will be described, and FIG. 9 is a process of determining the actual convergence angle θr corresponding to the actual distance to the visual recognition target. First, in Q21, the distance from the installation position of the radar S1 (for example, the front end of the vehicle) to the visual object outside the vehicle is detected. Then, the distance from the standard eye point position to the radar installation position is added to calculate the distance from the standard eye point position to the visual object outside the vehicle. In this way, for example, the actual distance to the pedestrian in front is defined as "X steps", the actual distance to the traffic light in front is defined as "X communication", and the others are detected in the same manner.

In Q22, the distance between the standard eye point position and various in-vehicle devices is acquired. For example, the distance to the room mirror is set to "X mirror", the distance to the display 3 is set to "X navigation", and other in-vehicle devices are detected in the same manner.

In Q23, as described above, the correction amount of the eye point position with respect to the standard position is determined according to the position in the front-rear direction of the driver's seat 1 (corresponding to the determination of "xh1" in FIG. 4). After that, in Q24, the center (average position) of the gazing point distribution in a unit time is determined (position determination of the hollow circle in FIG. 6). After that, it is determined that the region where the center position of the center point distribution determined in Q24 exists is the region (small region) where the visual object exists.

After Q25, in Q26, the actual distance Dr to the visual object in the region (small region) determined by Q25 is determined. When determining the actual distance Dr, the correction amount of the eye point position determined in Q23 is added.

In Q27, the actual distance Dr determined in Q26 is converted as the actual convergence angle θr (characteristics as shown in FIG. 3, which are also calculated based on the equation (1)). The actual convergence angle θr determined in this way is used for the processing in Q2 in FIG. The process of FIG. 9 may be performed in parallel with the process of FIG. 8 (also serves as an interrupt process to FIG. 8), and the process shifts to FIG. 8 after the process of FIG. 9 is performed. It can also be serialized.

Although the embodiments have been described above, the present invention is not limited to the embodiments, and appropriate modifications can be made within the scope of the claims. In the embodiment, the detected convergence angle θ0 is used as the first evaluation index, and the actual convergence angle θr corresponding to the detected actual distance Dr is used as the second evaluation index, but the evaluation index can also be used as the distance. That is, with reference to the viewing distance corresponding to the detected convergence angle θ0, this viewing distance can be used as the first evaluation index, while the detected actual distance Dr can be used as the second evaluation index. The display means is not limited to the display 3, and an appropriate display such as a head-up display that projects display information in front of the driver can be adopted.

The viewer is not limited to the driver P of the vehicle V, but may be a person who operates a vehicle such as an airplane, a ship, or a railroad train, and a moving object other than the vehicle (for example, a drone or a helicopter operated wirelessly). ) May be the operator, and may also be a person who monitors a predetermined range at a fixed position, such as a controller at an airfield. The degree of immersion is not limited to two stages, but can be determined in multiple stages of three or more stages. In addition, the degree of immersion can be determined in a short time (for example, 3 to 10 seconds). The present invention can be applied in various fields such as, for example, how much an advertising poster attracts viewers (in the case of an advertising poster, it is judged that the higher the degree of immersion is, the more preferable it is). can. Of course, the object of the present invention is not limited to what is specified, but also implicitly includes providing what is expressed as substantially preferable or advantageous.

The present invention can determine the degree of immersion in a visual object.

V: Vehicle P: Driver U: Controller S1: Radar S2: Seat position sensor 1: Driver's seat 3: Display (display means)
10: Convergence angle detector

Claims (7)

Convergence angle acquisition means for acquiring the convergence angle formed by the line of sight of the left and right eyes of the viewer when the viewer is visually observing the visual object.
An actual distance detecting means for detecting the actual distance from the viewer to the visual object, and
Based on the first evaluation index set corresponding to the congestion angle acquired by the congestion angle acquisition means and the second evaluation index set corresponding to the actual distance detected by the actual distance detecting means. An immersive degree determining means for determining the degree to which the viewer is immersed in the visual object,
Equipped with
The first evaluation index is set as the convergence angle acquired by the convergence angle acquisition means, and the first evaluation index is set.
The second evaluation index is set as a convergence angle corresponding to the actual distance detected by the actual distance detecting means.
A visual condition determination device characterized by this. In claim 1 ,
The immersive degree determining means is a visual recognition state determining device, characterized in that the immersive degree is determined based on the difference between the first evaluation index and the second evaluation index. In claim 2 ,
The immersive degree determining means has a high degree of immersiveness when the number of times the difference between the first evaluation index and the second evaluation index is equal to or less than a predetermined number of times within a predetermined time set in advance. A visual recognition state determination device characterized by determining that. In any one of claims 1 to 3 ,
A display means that is set to a position that can be visually recognized by the viewer and whose display brightness can be changed is provided.
When the immersive degree determination means determines that the immersive degree is low, the display brightness of the display means is increased.
A visual condition determination device characterized by this. In claim 4 ,
The display means is installed in the vehicle, and the viewer is regarded as the driver of the vehicle.
The display brightness of the display means is increased provided that the vehicle is automatically driven or stopped.
A visual condition determination device characterized by this. In any one of claims 1 to 5 ,
A visual recognition state determination device, wherein the convergence angle acquisition means has an image pickup means for photographing a face portion of a viewer and acquires a convergence angle based on an image obtained by the image pickup means. In claim 6 ,
The viewer is regarded as the driver of the vehicle.
The image pickup means is provided in the vehicle interior.
A visual condition determination device characterized by this.

Images