JP2016110269A - Manipulation input device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a manipulation input device that offers superior operability while securing vehicle driving safety.SOLUTION: A manipulation input device comprises; a touch pad 20 that constitutes an input unit configured to detect and acknowledge a state of manipulation made by a finger or the like; a screen display unit 30 that includes a screen corresponding to a manipulatable area of the touch pad 20 and displays the manipulation state; a camera 40 that constitutes an image capturing unit that captures an image of a user (driver) performing input manipulation on the touch pad 20 while looking at the screen display unit 30; and a control unit 51 that controls the touch pad 20, the screen display unit 30, and the camera 40. The control unit 51 includes a determination unit 52 that determines a state of visual recognition of the screen display unit 30 by the user (driver) photographed by the camera 40, and is configured to provide control to disable manipulation on the touch pad 20 or a display function of the screen display unit 30 in accordance with a result of determination by the determination unit 52.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an operation input device.

  2. Description of the Related Art Conventionally, there has been proposed an operation input device that can display an operation guidance screen according to the presence or absence of a finger or the like, for example, an operation input device that selects and inputs items from an input screen such as a car navigation device. (For example, Patent Document 1).

  The operation input device includes a display unit that displays an image, an operation input unit that performs operation input on the operation guide screen, a proximity detection unit that detects the approach of an object to the operation input unit, and proximity determination The trajectory measurement unit that measures the trajectory when it is determined that the object is close by the unit, the trajectory recording unit that records the trajectory measured by the trajectory measurement unit, and the operation input unit is determined to have been performed And a display control unit that controls the display unit. This operation input device is used for an operator (occupant) to input an operation input unit with a fingertip while visually recognizing an operation guide screen. For example, when the vehicle is equipped as a car navigation device, the operation input device An input operation can be performed while visually recognizing the operation guidance screen.

  Further, an operation input device having a display and an imaging device having a function as a touch panel has been proposed (for example, Patent Document 2).

  The operation input device is in a display gazing state in which a detection unit that detects a user's line of sight based on the user's face image acquired by the imaging device and a direction of the line of sight detected by the detection unit are directed to the display. When the determination means determines whether or not the display is in the display gazing state, the input operation on the touch panel is controlled so as to be accepted, and when the determination means determines that the display is not in the display gazing state, And a control means for controlling the input operation on the touch panel so as not to be accepted.

Japanese Patent Application Laid-Open No. 2012-118761 International Publication No. 2013/141161

  However, when the operation input device of Patent Literature 1 is mounted on a vehicle as a car navigation device and an operation input device that performs input operation control based on a user's face image acquired by the imaging device of Patent Literature 2 is applied, It is necessary to improve the convenience of the operator (occupant) while ensuring the safety of traveling. In addition, it is necessary to comply with the basic concept of handling the image display device while the vehicle is traveling.

  Accordingly, an object of the present invention is to provide an operation input device that is excellent in operability while ensuring the safety of vehicle travel.

[1] In order to achieve the above object, an input unit that detects and inputs an operation state, a screen display unit that includes a screen corresponding to the operation range of the input unit and displays the operation state, and the screen display unit An imaging unit that captures an operator who performs an input operation with the input unit while visually recognizing, and a control unit that controls the input unit, the screen display unit, and the imaging unit. A control unit that determines a viewing state of the operator imaged by the imaging unit with respect to the screen display unit, and controls to disable operation of the input unit or display of the screen display unit based on a determination result of the determination unit An operation input device mounted on a vehicle is provided.

[2] The operation input device according to [1], wherein the control unit performs the determination based on whether or not an integrated value of viewing time with respect to the screen display unit is equal to or greater than a predetermined time. May be.

[3] The operation according to [1] or [2], wherein the control unit controls the input unit or the screen display unit based on a vehicle speed of the vehicle and the determination result. It may be an input device.

  According to the operation input device of the present invention, it is possible to provide an operation input device excellent in operability while ensuring the safety of vehicle travel.

FIG. 1 is an overall perspective view showing the inside of a vehicle equipped with an operation input device according to an embodiment of the present invention. FIG. 2 is a block diagram showing the configuration of the operation input device according to the embodiment of the present invention. FIG. 3 is a flowchart showing an operation flow of the operation input device according to the embodiment of the present invention. 4A and 4B show the face and line of sight of the occupant seated in the driver's seat, where FIG. 4A is a diagram when the occupant is running and FIG. 4B is a diagram when the occupant is viewing the screen display unit. . FIG. 5 is a diagram for explaining the movement of the line of sight of the occupant.

(Embodiment of the present invention)
(Configuration of operation input device 10)
FIG. 1 is an overall perspective view showing the inside of a vehicle equipped with an operation input device according to an embodiment of the present invention. An operation input device 10 according to an embodiment of the present invention is an operation input device mounted on a vehicle, and is an operation input device that can ensure the safety of vehicle travel while keeping travel regulations.

  The operation input device 10 mounted on the vehicle 1 according to the embodiment of the present invention includes a touch pad 20 that is an input unit that detects and inputs an operation state with a finger or the like, and a screen corresponding to an operation range of the touch pad 20. A screen display unit 30 that displays an operation state, a camera 40 that is an imaging unit that images an operator (occupant) who performs an input operation with the touch pad 20 while viewing the screen display unit 30, the touch pad 20, and the screen A display unit 30 and a control unit 51 that controls the camera 40, and the control unit 51 includes a determination unit 52 that determines a visual recognition state of the operator (occupant) captured by the camera 40 with respect to the screen display unit 30. Based on the determination result of the determination unit 52, control is performed to disable the operation of the touch pad 20 or the display of the screen display unit 30.

  FIG. 2 is a block diagram showing the configuration of the operation input device according to the embodiment of the present invention. The touch pad 20, the screen display unit 30, and the camera 40 shown in FIG. 1 are each connected to an operation input ECU 50 for controlling the entire operation input device. The operation input ECU 50 is connected to an in-vehicle device, for example, a car navigation device 110, a vehicle speed sensor 120, and the like via the in-vehicle LAN 100. The operation input ECU 50 includes a control unit 51 for controlling the operation of the touchpad 20, the screen display unit 30, and the camera 40, and a determination unit 52 that determines a visual state described later.

(Touchpad 20)
The touch pad 20 as an input unit is a touch pad that is provided on the center console 4 of the vehicle 1 and detects, for example, a position (detection point) on the operation surface touched by an operation finger. For example, the occupant 2 can display a cursor display, movement, and the like corresponding to the touch point on the screen display unit 30 by performing a touch operation on the operation surface. In addition, it is possible to operate in-vehicle devices such as the car navigation device 110 connected via the in-vehicle LAN 100.

  As the touch pad 20, for example, a capacitive touch pad capable of detecting proximity of a finger, contact, and positions or movements of a plurality of detection fingers can be used.

  The input operation on the input unit is not limited to a touch operation on the operation surface, a tracing operation, or the like. For example, the input unit may be configured by a joystick or the like instead of the touch pad, or may be configured to perform an input operation by a button or a plurality of button arrays.

(Screen display unit 30)
The screen display unit 30 is, for example, a liquid crystal display panel provided on the instrument panel 3 of the vehicle 1. The screen display unit 30 displays a menu screen, a map screen, and the like of the car navigation device 110 that is a control target, and a cursor corresponding to an input operation of the touch pad 20. Here, the input operation is a touch operation, a swipe operation, a pinch-in operation, a pinch-out operation, a tap operation, a double tap operation, or the like.

  Note that the input unit and the screen display unit described above are not limited to being arranged separately. It may be a touch input device in which the input unit and the screen display unit are arranged so as to overlap with each other and the input surfaces of the screen display unit and the input unit are formed in the same region.

(Camera 40)
The camera 40 serving as an imaging unit uses, for example, a CCD (Charge Coupled Device) as an imaging element and has a configuration similar to a digital camera. An operation input ECU 50 (Electronic Control Unit) described later is used. Under this control, the occupant's face is imaged and the image data is sent to the operation input ECU 50.

  As shown in FIG. 1, the camera 40 is placed, for example, on the steering column post 5 and is disposed at a position suitable for imaging the face of the occupant 2 (operator).

(Operation input ECU 50)
The operation input ECU 50 is, for example, a one-chip LSI microprocessor mainly composed of a CPU, and is configured to operate according to a program stored in the memory 53. A touch pad 20, a screen display unit 30, and a camera 40 are connected to the operation input ECU 50. In-vehicle devices such as the car navigation device 110 and the vehicle speed sensor 120 are connected via the in-vehicle LAN 100.

  In the operation input ECU 50, a control unit 51 that controls the operation of the touch pad 20 or the display of the screen display unit 30, a determination unit 52 that determines a viewing state of the occupant 2 captured by the camera 40 with respect to the screen display unit 30, and , A memory 53 is provided. Moreover, the control part 51 calculates | requires the integration | accumulation time Ts of the visual recognition time which judges that the passenger | crew 2 is gazing at the object apparatus (screen display part 30) by the judgment part 52 mentioned later.

  The memory 53 is a storage medium including a semiconductor memory or the like that stores the program and temporarily stores image data from the camera 40. The memory 53 is a pattern diagram showing a predetermined speed (reference vehicle speed), a predetermined time (reference integrated time), an arrangement of the screen display unit 30 installed on the instrument panel 3 as viewed from the viewpoint of the occupant 2 of the driver seat, and Data (table) relating to the orientation and line of sight of the occupant 2 with respect to the screen display unit 30 is stored as reference data.

(Operation of the operation input device 10)
FIG. 3 is a flowchart showing an operation flow of the operation input device according to the embodiment of the present invention. 4A and 4B show the face and line of sight of the occupant seated in the driver's seat, where FIG. 4A is a diagram when the occupant is running and FIG. 4B is a diagram when the occupant is viewing the screen display unit. . FIG. 5 is a diagram for explaining the movement of the line of sight of the occupant. The process shown in FIG. 3 is executed by the operation input ECU 50 shown in FIG.

  When the operation of the operation input device 10 starts, the operation input ECU 50 first images the face of the occupant 2 with the camera 40 (S01). Image data from the camera 40 is taken into the operation input ECU 50 and stored in the memory 53.

  In most cases, the face of the occupant 2 faces the traveling direction as shown in FIG. However, when trying to operate the device (touch pad 20, screen display unit 30), as shown in FIG. 4B, the face is directed toward the operation target device, and the line of sight 22 is also directed toward the device. Come to face. In the present embodiment, the screen is directed toward the screen display unit 30 on which a moving cursor is displayed by the touch pad 20 to be operated.

  Therefore, the operation input ECU 50 reads the image data from the memory 53 and measures the face direction and the line of sight 22 of the occupant 2 (S02).

  Next, the final gaze point is estimated based on the reference data stored in the memory 53 with respect to the movement of the line of sight 2 of the occupant 2 (S03).

  This gaze point estimation will be described with reference to FIG. Here, the left eye is shown, but the same applies to the right eye. When the occupant 2 eyes 21 are looking at a road or the like, the line of sight 22 is in the line-of-sight direction e1. From this state, when the occupant 2 rotates his face in the A direction (left direction), the line-of-sight direction changes from e1 to e2, e3, and e4. Here, it is assumed that time t1 from e1 to e2, time t2 from e2 to e3, time t3 from e3 to e4, angle θ1 from e1 to e2, angle θ2 from e2 to e3, It is assumed that the angle θ3 is from e3 to e4.

  Further, once the line of sight 22 starts to move, it is considered that the line of sight 22 moves linearly in that direction at a constant speed, and the direction and speed are rarely changed in the middle. Therefore, when the time t1 is required for the line of sight 22 to move the angle θ1, the time required from the line of sight direction e2 to e4 is (t1 × 2) seconds after the line of sight direction e2. Accordingly, at time t1, (t1 × 2 = t2 + t3) seconds later is estimated as the time for the point of interest to reach the screen display unit 30.

  The operation input ECU 50 determines whether or not the touch-on is permitted (S04). If the touch-on is permitted, the process proceeds to step S05 (S04: Yes), and if the touch-on is not permitted, the process proceeds to step S10 (S04: No).

  In step S05, the table of reference data stored in the memory 53 with respect to the moving direction of the line of sight 22 is referred to at the time t1, and the target device (screen display unit 30 is displayed on the line of sight direction e4 after t1 × 2 seconds. ) Exists, the operation input ECU 50 can determine the presence of the screen display unit 30 in the line-of-sight direction within the angle range of (θ1 + θ2 + θ3) with reference to the table in the memory 53 (S05).

  Depending on the face of the occupant 2 and the movement speed of the line of sight, the estimation may not be completed at the angle θ1, but in such a case, the estimation is performed at an arbitrary time after the angle θ1. When it is determined that the screen display unit 30 does not exist in the direction from the line-of-sight direction e1 to e4 (S05: No), the process proceeds to step S13. If it is determined that the screen display unit 30 is present in the direction from the line-of-sight direction e1 to e4 (S05: Yes), the process proceeds to step S05.

  In specifying the target device, when the area of the target device is small, the target device is within the area if the line of sight 22 is facing the set area, even if the line of sight 22 around the target device is small. It can be processed as if it existed.

  Next, the operation input ECU 50 determines whether or not the vehicle speed is equal to or higher than a predetermined speed. A determination is made based on vehicle speed data input from the vehicle speed sensor 120 via the in-vehicle LAN 100 (S06). The predetermined speed can be arbitrarily set to, for example, 0 km / h, 5 km / h, 10 km / h, 15 km / h, and the like. The predetermined speed (reference vehicle speed) is set depending on whether safe traveling of the vehicle can be ensured even when the target device (screen display unit 30) is visually recognized.

  If the vehicle speed is not equal to or higher than the predetermined speed (S06: No), the process returns to step S01, and the subsequent processes are repeatedly executed. If the vehicle speed is equal to or higher than the predetermined speed (S06: Yes), the process proceeds to step S07.

  The control unit 51 of the operation input ECU 50 performs an accumulation process of the visual recognition time of the target device (screen display unit 30). The time during which the screen display unit 30 is watched is integrated to obtain an integrated time Ts (S07).

  The operation input ECU 50 determines whether or not the accumulated time Ts is equal to or longer than a predetermined time (S08). Here, the predetermined time is defined, for example, as “Regarding the handling of the image display device” of the Japan Automobile Manufacturers Association, “the total screen viewing time until the task is completed does not exceed 8 seconds” By complying, it can be set to 8 seconds. This predetermined time can be set based on the regulations of each country such as the NHTSA guidelines.

  If the accumulated time Ts is not equal to or longer than the predetermined time (8 seconds) (S08: No), the process returns to step S01, and the subsequent processes are repeatedly executed. If the accumulated time Ts is equal to or longer than the predetermined time (8 seconds) (S08: Yes), the process proceeds to step S09.

  The operation input ECU 50 does not permit touch-on by the touch pad 20 and performs control to shut down (gray out) the screen display of the screen display unit 30 (S09). That is, a control for disabling the operation of the touch pad 20 or the display of the screen display unit 30 is performed to obtain a display indicating that the operation is not possible. Alternatively, the display of the screen display unit 30 is turned off. The control for disabling the display of the screen display unit 30 satisfies, for example, the provision of “Handling of image display device” of the Japan Automobile Manufacturers Association.

  In step S10, the table of reference data stored in the memory 53 with respect to the movement direction of the line of sight 22 is referred to at the time t1, and the target device (screen display unit 30 is displayed on the line of sight e4 after t1 × 2 seconds. ) Exists, the operation input ECU 50 can determine the presence of the screen display unit 30 in the line-of-sight direction within the angle range of (θ1 + θ2 + θ3) with reference to the table in the memory 53 (S10).

  Depending on the face of the occupant 2 and the movement speed of the line of sight, the estimation may not be completed at the angle θ1, but in such a case, the estimation is performed at an arbitrary time after the angle θ1. When it is determined that the screen display unit 30 does not exist in the direction from the line-of-sight direction e1 to e4 (S10: No), the process returns to step S01, and the subsequent processes are repeatedly executed. If it is determined that the screen display unit 30 is present in the direction from the line-of-sight direction e1 to e4 (S10: Yes), the process proceeds to step S11.

  In step S10 described above, the same determination as in step S05 is performed. Similarly to step S05, when the area of the target device is small in specifying the target device, the line of sight 22 toward the periphery of the target device is determined. Even if it exists, if the line of sight 22 faces in the set area, it can be considered that the target device exists in the area.

  The control unit 51 of the operation input ECU 50 determines whether or not there is a touch on the touch pad 20 (S11). If it is determined that there is a touch (S11: Yes), the process proceeds to step S12. If it is not determined that there is a touch (S11: No), the process returns to step S01, and the subsequent processes are repeatedly executed.

  The operation input ECU 50 permits touch-on by the touch pad 20 (S12). The processing of this system returns to step S01, and the processing after step S05 is executed through steps S01 to S04.

  In step S <b> 13, the control unit 51 of the operation input ECU 50 performs an accumulation process of the non-viewing time of the target device (screen display unit 30). The times when the screen display unit 30 is not watched are integrated to obtain an integrated time Ts' (S13).

  The operation input ECU 50 determines whether or not the integration time Ts ′ of the non-visual time is equal to or longer than a predetermined time (S14). If the accumulated time Ts' is not equal to or longer than the predetermined time (S14: No), the process returns to step S01, and the subsequent processes are repeatedly executed. On the other hand, when the accumulated time Ts' is equal to or longer than the predetermined time (S14: Yes), the process proceeds to step S15.

  The operation input ECU 50 does not permit touch-on by the touch pad 20. However, since the screen display unit 30 is not watched, the screen display of the screen display unit 30 is continuously displayed (S15).

(Effect of the embodiment of the present invention)
The operation input device according to the embodiment of the present invention integrates the time when the screen display unit 30 is visually recognized during the touch-on-permitted state, and shifts to the touch-on disallowed state when the accumulated time Ts exceeds a certain time, and displays the screen. Is shut down (grayed out or displayed indicating that operation is not possible). Thereby, the operation input device excellent in operability can be provided while ensuring the safety of vehicle travel. In addition, for example, it is possible to satisfy the provisions of “The handling of image display devices” of the Japan Automobile Manufacturers Association and the regulations of each country such as the NHTSA guidelines.

  On the other hand, as shown in step S08, when the cumulative time Ts of the visual recognition time is equal to or shorter than the predetermined time, or as shown in step S05 (S05: No), it is determined that the screen display unit 30 is not watched. In the state, the screen display of the screen display unit 30 is continued. Thereby, since the screen display is not grayed out, convenience can be improved.

  As mentioned above, although embodiment of this invention was described, these embodiment is only an example and does not limit the invention which concerns on a claim. For example, in the embodiment of the present invention, the determination is made based on whether or not the visual integration time Ts is equal to or longer than a predetermined time. Further, the visual recognition time is integrated when it is determined that the touch-on is permitted. However, the visual recognition time may be integrated regardless of the determination result of whether the touch-on is permitted.

  In addition, these novel embodiments and modifications thereof can be implemented in various other forms, and various omissions, replacements, changes, and the like can be made without departing from the scope of the present invention. Can do. In addition, not all the combinations of features described in these embodiments are essential to the means for solving the problems of the invention. Furthermore, these embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 1 ... Vehicle 2 ... Passenger 3 ... Instrument panel 4 ... Center console 5 ... Steering column post 10 ... Operation input device,
DESCRIPTION OF SYMBOLS 20 ... Touchpad 21 ... Eye 22 ... Line of sight 30 ... Screen display part 40 ... Camera 50 ... Operation input ECU
51 ... Control unit 52 ... Judgment unit 53 ... Memory 100 ... In-vehicle LAN
110: Car navigation device 120 ... Vehicle speed sensor

Claims (3)

An input unit for detecting and inputting an operation state;
A screen display unit having a screen corresponding to the operation range of the input unit and displaying the operation state;
An imaging unit for imaging an operator who performs an input operation with the input unit while visually recognizing the screen display unit;
A control unit that controls the input unit, the screen display unit, and the imaging unit;
The control unit includes a determination unit that determines a viewing state of the operator captured by the imaging unit with respect to the screen display unit, and operates the input unit or the screen display unit based on a determination result of the determination unit. An operation input device mounted on a vehicle, characterized by performing control to disable display.   The operation input device according to claim 1, wherein the control unit performs the determination based on whether or not an integrated value of the viewing time with respect to the screen display unit is equal to or greater than a predetermined time.   The operation input device according to claim 1, wherein the control unit controls the input unit or the screen display unit based on a vehicle speed of the vehicle and the determination result.

Images