JPWO2017061183A1 - Human interface - Google Patents

Abstract

The human interface of the driving support system (1) displays a peripheral object information acquisition device (10) that acquires information on peripheral objects existing around the vehicle, and an icon indicating the peripheral objects superimposed on a real scene around the vehicle. According to the head-up display (20), the motion sensor (11) for detecting the finger of the driver's hand holding the steering wheel, and the position or movement of the detected finger, the driver A matching unit (31a) that matches a desired icon and a display control unit (31b) that superimposes and displays information on a peripheral object indicated by the matched icon on the head-up display (20).

Description

  The present invention relates to a human interface, and more particularly to a human interface used in a vehicle.

  In recent years, for example, a pre-crash safety system (automatic brake system) that detects a pedestrian or vehicle ahead and alerts the driver and automatically brakes the vehicle when there is a possibility of contact, etc. ) And advanced driver assistance systems (ADAS (Advanced Driver Assistance System)) such as an adaptive auto-cruise system that detects a preceding vehicle and keeps a distance between the preceding vehicles and travels at a predetermined speed. It is being advanced.

  On the other hand, Patent Document 1 discloses a system (human interface) that allows a driver to perform a query search operation while driving. In this system, the pointing of the driver's finger to the object (or item) is detected using the first input device, and the selection instruction of the object (or item) is received using the second input device. . More specifically, in this system, an image of a hand is picked up by a first input device (for example, a camera), and the display device (for example, head-up) corresponding to the position of the fingertip of the finger to be processed by processing the picked-up image. The position of an object or the like on the display (Head Up Display: HUD) is determined. Then, the second input device (for example, a microphone) is used to receive an instruction for selecting an object or the like by the driver, and an operation associated with the selected item or the like is executed.

JP 2014-179097 A

  By the way, in the above-described pre-crash safety system or the like, there is a situation where it is desired to present detailed information regarding detected objects such as pedestrians and vehicles. In such a situation, for example, when a plurality of objects are detected, if detailed information is presented for all the objects, the display information becomes complicated, which is not preferable from the viewpoint of safe driving. Therefore, in such a case, it is desirable to match (select) an object desired by the driver from a plurality of detected objects and present only the detailed information of the object.

  Here, as described above, according to the system described in Patent Document 1, the driver can perform a query search operation while driving. However, in this system (human interface), in order to select an object, etc., it is necessary to perform, for example, a pointing operation with a fingertip and an operation such as utterance, so the driver's attention selects an object, etc. rather than driving (Matching) may be devoted, and the time required for selection (matching) may be increased.

  The present invention has been made to solve the above-described problems, and an object existing around a vehicle desired by a driver without moving the line of sight to an operation switch or the like and without releasing both hands from the steering wheel. It is an object of the present invention to provide a human interface capable of matching and presenting the above information in a shorter time by a simpler operation.

  The human interface according to the present invention includes peripheral object information acquisition means for acquiring information on peripheral objects existing in the vicinity of the vehicle, and landmark information indicating the peripheral objects acquired by the peripheral object information acquisition means in an actual scene around the vehicle. By means of display means according to the position or movement of the finger detected by the display means for superimposing display, finger detection means for detecting the finger of the driver's hand holding the steering wheel of the vehicle, and finger detection means A matching unit that matches the landmark information desired by the driver from the superimposed marker information, and a display control unit that causes the display unit to superimpose and display information on peripheral objects indicated by the landmark information matched by the matching unit; It is characterized by providing.

  According to the human interface according to the present invention, information on surrounding objects existing around the vehicle is acquired, and landmark information indicating the acquired surrounding objects is displayed superimposed on the actual scene around the vehicle. Therefore, the driver can recognize a peripheral object that can provide information while keeping the line of sight forward, for example. At that time, only the mark information is superimposed and displayed, so that it is possible to prevent the displayed information from becoming complicated. On the other hand, the finger of the driver's hand holding the steering wheel of the vehicle is detected, and the driver is selected from the mark information superimposed on the detected finger's position (including presence / absence) or movement. The desired landmark information is matched. Then, information on the peripheral object indicated by the matched landmark information is displayed in a superimposed manner. Therefore, the driver matches the desired landmark information with a single motion (only the movement of the fingertip) while holding the steering wheel with both hands, and information on the surrounding objects indicated by the landmark information (detailed information) ) Can be displayed in a superimposed manner. As a result, without moving the line of sight to the operation switch or the like and without releasing both hands from the steering wheel, the information on the object around the vehicle desired by the driver can be obtained in a shorter time by a simpler operation, Matching can be presented.

  In the human interface according to the present invention, it is preferable that the finger detection means detects a finger protruding forward from the steering wheel. If it does in this way, it will become possible to detect the position or movement of a driver's finger reliably and accurately, and it will become possible to match mark information desired by the driver accurately using the detection result (detection information). .

  In the human interface according to the present invention, the finger detection means is disposed in an upper part of the vehicle interior of the vehicle and has a detection range directed downward from the disposed position. It is preferable that a finger protruding forward from the wheel enters and the steering wheel does not enter. In this way, when processing the detection information, it is not necessary to remove the steering wheel from the detection information, so that the processing load can be reduced and the processing time can be shortened.

  In the human interface according to the present invention, the matching means divides the detection range into a plurality of areas, and the driver is placed in any one of the plurality of areas according to the position or movement of the finger detected by the finger detection means. It is preferable to match the mark information corresponding to the region containing the finger when it is determined whether the finger is present. In this case, by determining whether or not a finger has entered the region set as the detection range, it is possible to determine the landmark information desired by the driver, so the processing load can be reduced and the processing time can be further shortened. It becomes.

  In the human interface according to the present invention, the matching unit acquires the direction in which the finger is pointing based on the position or movement of the finger detected by the finger detection unit, and the driver points with the finger from the acquired direction. It is preferable to match the mark information. In this case, based on the detected position or movement (trajectory) of the finger, the direction in which the finger is pointing is acquired, and the mark information that the driver is pointing with the finger is matched from the acquired direction. . Therefore, even when there are a large number of acquired peripheral objects and a large number of landmark information is superimposed and displayed, it is possible to accurately determine the landmark information desired by the driver from the large number of landmark information. It becomes possible.

  In the human interface according to the present invention, it is preferable that the finger detection means is constituted by a distance image sensor. In this way, three-dimensional position information (three-dimensional coordinates) indicating the position of the finger can be obtained with high accuracy by the distance image sensor. Furthermore, from the three-dimensional position information, for example, the direction or the like that the driver is pointing with a finger can be accurately acquired.

  In the human interface according to the present invention, the mark information is preferably an icon. In this way, it is possible to easily recognize the surrounding object that can provide information to the driver by the icon superimposed on the actual scene around the vehicle.

  According to the present invention, information on an object existing around a vehicle desired by a driver can be shortened by a simpler operation without moving the line of sight to an operation switch or the like and without releasing both hands from the steering wheel. Matching and presentation can be made in time.

It is a block diagram which shows the structure of the driving assistance system which concerns on 1st Embodiment. It is a figure which shows the detection range of the motion sensor which comprises a driving assistance system, (a) is a front view, (b) is a side view. In the driving assistance system concerning a 1st embodiment, it is a figure showing an example when the icon of a peripheral object is superimposed and displayed on the real scene ahead of vehicles. In the driving assistance system concerning a 1st embodiment, it is a figure showing an example when detailed information on a peripheral object is superimposed and displayed on a real scene ahead of a vehicle. It is a block diagram which shows the structure of the driving assistance system which concerns on 2nd Embodiment. In the driving assistance system concerning a 2nd embodiment, it is a figure showing an example when the icon of a peripheral object is superimposed and displayed on the real scene ahead of vehicles. In the driving assistance system concerning a 2nd embodiment, it is a figure showing an example when detailed information on a peripheral object is superimposed and displayed on a real scene ahead of a vehicle.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. In the drawings, the same reference numerals are used for the same or corresponding parts. Moreover, in each figure, the same code | symbol is attached | subjected to the same element and the overlapping description is abbreviate | omitted.

(First embodiment)
Here, the human interface (Human Machine Interface (HMI)) according to the present invention is used for AR (Augmented Reality) using a head-up display and around the vehicle (particularly the vehicle). A case will be described as an example in which the present invention is applied to a driving support system that presents information on an object (front object) existing in the front) to the driver. In the present embodiment, the forward direction of the vehicle is “front (front side)”, the backward direction of the vehicle is “rear (rear side)”, and the right side when facing the forward direction of the vehicle is “right (right side)” ) ”, And the left side when the vehicle is moving forward is“ left (left side) ”.

  First, the configuration of the driving support system 1 according to the first embodiment will be described with reference to FIGS. 1 and 2 together. FIG. 1 is a block diagram showing the configuration of the driving support system 1. FIG. 2 is a diagram showing a detection range of the motion sensor 11 constituting the driving support system 1, wherein (a) is a front view and (b) is a side view.

  The driving support system 1 superimposes an icon indicating a surrounding object that can provide information (corresponding to the landmark information described in the claims) on the actual scene around the vehicle, and the driver points out of the superimposed icons. Detailed information of the surrounding object corresponding to the (selected) icon is superimposed and displayed. At that time, the driving support system 1 detects a gesture by a finger (for example, an index finger) of a hand holding the steering wheel by the driver and matches an icon pointed by the driver with the finger. In particular, the driving support system 1 determines the icon that is pointed by dividing the finger detection range into a plurality of areas and determining whether or not the finger has entered an arbitrary area among the plurality of areas. In the present embodiment, an example will be described in which one icon is superimposed on the left side and the right side, and the detection range is divided into two areas, a left area and a right area.

  The driving support system 1 includes a peripheral object information acquisition device 10 (corresponding to the peripheral object information acquisition means described in the claims), a motion sensor 11 (corresponding to the finger detection means described in the claims), a head-up display 20 (corresponding to the display means described in the claims) and an electronic control unit (hereinafter referred to as “ECU (Electronic Control Unit)”) 31.

  The peripheral object information acquisition device 10 is a device that acquires information on peripheral objects existing around the vehicle (particularly in front of the vehicle). The peripheral object information acquisition device 10 may acquire information by detecting a peripheral object using a sensor mounted on the vehicle, or may acquire information on the peripheral object from the outside of the vehicle. The peripheral object information acquisition device 10 transmits the acquired peripheral object information (peripheral object information) to the ECU 31. The peripheral objects are, for example, automobiles, motorcycles, motorbikes, bicycles, pedestrians, animals, and the like. The peripheral object information is, for example, a position (for example, absolute coordinates such as relative coordinates and latitude / longitude with reference to the host vehicle), a moving speed, a moving direction, a type, a vehicle type, and an exhaust amount.

  As a sensor mounted on the vehicle, for example, a radar sensor such as a millimeter wave radar or a laser radar, an image sensor using a camera, or the like can be used. With these sensors, information such as the presence / absence of an object around the vehicle and the position, size, type, and the like of the object are acquired when the object exists. Furthermore, by acquiring time-series data of the position of the object, the moving speed, moving direction, etc. of the object can also be acquired. In the case of an image sensor, an object can be detected even when the surroundings are dark, such as at night, by using a near infrared camera or an infrared camera.

  Examples of the acquisition device from the outside of the vehicle include a vehicle-to-vehicle communication device and a road-to-vehicle communication device. When using an inter-vehicle communication device, for example, information such as position, moving speed, moving direction, type, vehicle type, and displacement can be acquired from other vehicles within the communicable range of the inter-vehicle communication device. . On the other hand, when a road-to-vehicle communication device is used, when the host vehicle enters the communicable range of the roadside device (traffic infrastructure), the position, moving speed, moving direction of the object acquired by the roadside device, And information such as type can be acquired. In addition, when acquiring information on a peripheral object by vehicle-to-vehicle communication or road-to-vehicle communication, information on a peripheral object that exists in a place where the vehicle (driver) becomes a blind spot can also be acquired.

  The motion sensor 11 is a position (including presence / absence) and movement (hand gesture / gesture) of a finger that comes forward (front glass side) from the steering wheel while the driver is holding the steering wheel with both hands (especially both hands). It is a sensor which detects. As the motion sensor 11, for example, it is preferable to use a distance image sensor of a TOF (Time Of Flight) method using near infrared rays.

  This distance image sensor receives, for example, an LED that emits near-infrared light and reflected light reflected by an object (for example, a finger of a hand), and outputs an electrical signal photoelectrically converted for each pixel (for example, CMOS image sensor or CCD image sensor), and a TOF processing unit that measures the time from near-infrared light emission to light reception and calculates the time and the distance from the light speed to the object for each pixel. .

  The motion sensor 11 uses the distance information for each pixel to position information (for example, three-dimensional relative to the motion sensor 11 on the basis of the motion sensor 11) of the object (the finger that is in front of the steering wheel) within the detection range. Get coordinates). For example, the motion sensor 11 determines whether or not an object exists by determining whether or not the distance is shorter than a predetermined value for each pixel, and if it is determined that the object exists, the object exists. The position information is acquired from the distance at each of the plurality of pixels in the region to be processed and the two-dimensional position of the pixel. The motion sensor 11 transmits detection information indicating the position information of the detected object to the ECU 31. Note that the motion sensor 11 may transmit distance information for each pixel to the ECU 31, and the ECU 31 may perform processing for determining whether or not an object exists from the distance information for each pixel and acquiring position information of the object.

  The motion sensor 11 is a predetermined location on the driver's seat side in the upper part of the passenger compartment of the vehicle and on the front side of the steering wheel (for example, a predetermined location on the front end of the driver's seat on the ceiling of the passenger compartment, or the driver's seat on the windshield At a predetermined location on the upper end of the side). The motion sensor 11 emits near-infrared light downward from the place where it is disposed, and has a detection range directed downward.

  As shown in FIG. 2, the detection range R of the motion sensor 11 is preferably set to a range in which the steering wheel W does not enter and the finger FF that protrudes forward from the steering wheel W enters. In particular, the detection range R is a finger pointing in any forward direction when the driver is holding the steering wheel W with both hands at the home position (for example, 10:10 when the vehicle is traveling straight). It is desirable to set the index finger to a range where only FF enters and no other fingers enter. Further, it is desirable that the detection range R is set to a range in which an instrument panel arranged in front of the steering wheel W does not enter.

  Here, the detection range R is defined by, for example, the left viewing angle LA, the right viewing angle RA, the front viewing angle FA, the rear viewing angle BA, and the detection distance L of the central axis A. The detection distance L is determined based on the vertical distance between the motion sensor 11 and the steering wheel W or the like. The left and right viewing angles LA and RA are determined based on the diameter (for example, 35 cm) of the steering wheel W and the detection distance L (for example, 30 °). The front and rear viewing angles FA and BA are determined based on the installation angle θ of the steering wheel W (for example, 15 °).

  The head-up display 20 is a display device that superimposes and displays various information on a real scene (landscape) in front of the vehicle. For example, the head-up display 20 projects an image (real image) toward the windshield, and reflects the image toward the driver by a half mirror provided in a part of the inside of the windshield. A virtual image is formed by forming an image in front of the windshield. Since the head-up display 20 is a well-known display device, description of the detailed configuration is omitted.

  When the head-up display 20 receives the display control information from the ECU 31, the head-up display 20 projects a predetermined image (for example, an icon indicating a peripheral object or detailed information of the peripheral object) on a predetermined display position of the windshield indicated by the display control information. To do.

  The ECU 31 is a control unit that comprehensively controls the driving support system 1. The ECU 31 includes, for example, a microprocessor that performs calculations, a ROM that stores a program for causing the microprocessor to execute each process, a RAM that stores various data such as calculation results, a backup RAM that stores the stored contents, and Input / output I / F and the like. In the ECU 31, a program stored in a ROM or the like is executed by a microprocessor, whereby a matching unit 31a (corresponding to the matching unit described in the claims) and a display control unit 31b (display described in the claims). Each function of the control means is realized.

  The matching unit 31a matches the icon that the driver is pointing with the finger FF from the left icon and the right icon superimposed on the head-up display 20. Specifically, the matching unit 31a divides the detection range R of the motion sensor 11 into a left side and a right side of the central axis A, and sets a left region and a right region in the detection range R. When the left icon and the right icon are superimposed, the matching unit 31a uses the position information of the object detected by the motion sensor 11 so that the driver's finger FF is in the left region or the right icon Determine if you are in the area. For example, the matching unit 31a determines that the driver's finger FF is in the area when the finger FF is continuously in any one of the areas for a predetermined time (for example, several seconds to several seconds). . When it is determined that the driver's finger FF is in the left area, the matching unit 31a determines that the driver is pointing to the left icon, and the driver's finger FF is in the right area. It is determined that the driver is pointing to the right icon.

  In the above-described matching unit 31a, the detection range R is divided into two regions, a left region and a right region, and an icon is matched corresponding to each region. However, the intermediate region, the left region, For example, the detection range R may be divided into three or more regions, and icons may be matched to correspond to the three or more regions, for example, when divided into three regions on the right side.

  The display control unit 31 b causes the head-up display 20 to superimpose and display an icon indicating a peripheral object (information-providing peripheral object) for which information is acquired by the peripheral object information acquisition device 10. In particular, the display control unit 31b divides the display range of the head-up display 20 into a left side and a right side, and displays icons one by one on the left side and the right side.

  Specifically, the display control unit 31b converts the positions of the peripheral objects in the coordinate system of the peripheral object information acquisition apparatus 10 into the positions in the coordinate system of the head-up display 20 for the left and right peripheral objects. Further, the display control unit 31b sets a position where the icon is superimposed and displayed near the position of the peripheral object in the coordinate system of the head-up display 20 for the left and right peripheral objects. Then, the display control unit 31b transmits to the head-up display 20 display information indicating the video information of the icons, the position of the superimposed display, and the like for each of the left and right peripheral objects. As for the icon pattern, color, etc., an icon that is easily visible to the driver is appropriately selected. The icons shown as an example in FIGS. 3, 4, 6 and 7 are pictures in which an exclamation mark (exclamation mark) is arranged in a triangle.

  In addition, when information on two or more peripheral objects is acquired on at least one of the left side and the right side, for example, priorities are given to the peripheral objects, and icons of the peripheral objects having the highest priority are superimposed and displayed. It is preferable to do so. Here, the priority order is, for example, whether the surrounding object is a moving object or a stationary object, whether it exists on the traveling road, whether it is moving toward the traveling road, It is preferable to determine whether or not it exists at a place where it becomes, depending on the moving speed or the like.

  Further, when any one of the icons is matched by the matching unit 31a, the display control unit 31b causes the head-up display 20 to superimpose and display the detailed information of the peripheral object indicated by the icon. Specifically, the display control unit 31b sets a position where detailed information is superimposed and displayed in the vicinity of the position of a peripheral object (may be an icon) in the coordinate system of the head-up display 20. Then, the display control unit 31b transmits to the head-up display 20 video information of detailed information (for example, video information including a character string of detailed information) and display control information indicating the position of superimposed display and the like. The detailed information to be superimposed is, for example, the type, the vehicle type, the displacement, the moving speed, the moving direction, and the like.

  When the detailed information is displayed in a superimposed manner, the superimposed display of all icons may be stopped, or only the superimposed display of the icons of peripheral objects that do not display the detailed information may be stopped. When the icon of the peripheral object that displays the detailed information is continuously displayed in a superimposed manner, the display control unit 31b transmits display control information for displaying the icon in a superimposed manner to the head-up display 20.

  Next, the operation of the driving support system 1 will be described. Here, the operation will be described using the example shown in FIGS. FIG. 3 is a diagram illustrating an example when the icons of the peripheral objects are superimposed and displayed on the actual scene in front of the vehicle in the driving support system 1. FIG. 4 is a diagram illustrating an example when the detailed information of the surrounding objects is superimposed and displayed on the actual scene ahead of the vehicle in the driving support system 1.

  In the example shown in FIGS. 3 and 4, the motorized bicycle O1 (FIG. 3) exists on the front side of the parked vehicle PV that is parked on the left side in front of the vehicle on which the driving support system 1 is mounted. In the example shown in FIG. 6, only the helmet of the motorbike O1 driver can be seen from the parked vehicle PV) and the animal (cat) O2 existing on the right side in front of the vehicle is in the vicinity of the vehicle in front. It is an object. The peripheral object information acquisition device 10 acquires each piece of information about the peripheral objects O1 and O2, and transmits a peripheral object information signal indicating these pieces of information to the ECU 31.

  For each peripheral object O1, O2, the ECU 31 converts the position of the peripheral object acquired by the peripheral object information acquisition device 10 into a position in the coordinate system of the head-up display 20, and displays an icon superimposed on the vicinity of the converted position. The display control information indicating the video information of the icon and the position of the superimposed display is transmitted to the head-up display 20. When the head-up display 20 receives the display control information, the head-up display 20 projects an icon image at a position indicated by the display control information. As a result, as shown in FIG. 3, the icon I1 is superimposed and displayed above the front side of the left parked vehicle PV in front of the vehicle, and the icon I2 is superimposed and displayed near the right cat O2.

  The driver looking forward can visually recognize these two icons I1 and I2 and easily recognize that there are surrounding objects capable of providing information by the icons I1 and I2. In the case of this example, the driver can visually recognize the peripheral object (cat O2) indicated by the icon I2, but cannot visually recognize the peripheral object indicated by the icon I1 in the blind spot of the parked vehicle PV. The detailed information of the surrounding object indicated by the icon I1 is to be obtained. In order to match the icon I1, the driver points the index finger FF of the left hand LH toward the icon I1 while holding the steering wheel W with both hands LH and RH and looking forward. At that time, the driver only has to put out the index finger FF so as to enter the region on the left side of the detection range R of the motion sensor 11. The motion sensor 11 detects the finger FF that has entered the detection range R, and transmits a detection signal indicating position information of the detected finger FF to the ECU 31.

  The ECU 31 determines that the finger FF is in the left region of the detection range R using the position information of the finger FF detected by the motion sensor 11, and the driver points to the left icon I1 by the operation with the finger FF. It is determined that Then, the ECU 31 sets a position where the detailed information of the peripheral object O1 is superimposed and displayed near the position of the icon I1, and heads up display control information indicating the video information of the detailed information of the peripheral object O1, the position of the superimposed display, and the like. It transmits to the display 20. When the head-up display 20 receives the display control information, the head-up display 20 projects an image of detailed information of the peripheral object O1 on the position indicated by the display control information.

  As a result, as shown in FIG. 4, detailed information DI of the peripheral object O1 (for example, a motorbike (50 cc), 5 km / h) is superimposed and displayed in the vicinity of the peripheral object O1 (icon I1). In this example, the icon I1 indicating the peripheral object O1 on which the detailed display DI is displayed in a superimposed manner continues to be displayed in a superimposed manner, and the superimposed display of the icon I2 indicating the peripheral object O2 is stopped.

  The driver looking forward sees this detailed information DI. From this detailed information D1, the driver knows that there is a motorized bicycle on the front side of the parked vehicle PV, and this motorized bicycle may come out on the road. Therefore, the driver can perform, for example, a driving operation for decelerating the vehicle, assuming that the motorized bicycle comes out when overtaking the parked vehicle PV.

  As described above in detail, according to the present embodiment, information on surrounding objects existing around the vehicle is acquired, and icons indicating the acquired surrounding objects are displayed superimposed on the actual scene around the vehicle. The Therefore, the driver can recognize a peripheral object that can provide information while keeping the line of sight forward, for example. On the other hand, the finger FF of the driver's hand holding the steering wheel W of the vehicle is detected, and depending on the detected position (detection region) of the finger FF, the driver can select from the superimposed icons. The desired icon is matched. Then, information on surrounding objects corresponding to the matched icon is displayed in a superimposed manner. Therefore, the driver matches a desired icon by a single operation (operation) while holding the steering wheel W with both hands, and superimposes and displays information on the surrounding object (detailed information) indicated by the icon. Can be made. As a result, the driver can display information on objects around the desired vehicle in a shorter time by moving the line of sight to the operation switch or the like and without releasing both hands from the steering wheel W by a simpler operation. Thus, matching can be presented.

  According to the present embodiment, by detecting the finger FF protruding forward from the steering wheel W, the position (detection region) of the driver's finger FF can be detected reliably and accurately, and the detection result It is possible to accurately match an icon desired by the driver using (detection information). Further, according to the present embodiment, the motion sensor 11 is disposed in the upper part of the vehicle interior, and the detection range directed downward from the disposed position is within the range where the steering wheel W does not enter and forward from the steering wheel W. Since it is set in the range where the protruding finger FF enters, it is not necessary to remove the steering wheel W from the detection information of the motion sensor 11, the processing load of the ECU 31 can be reduced, and the processing time can be shortened. .

  According to the present embodiment, since the motion sensor 11 is configured by a TOF type distance image sensor, the motion sensor 11 can obtain three-dimensional position information indicating the position of the finger FF with high accuracy. From the original position information, for example, it is possible to accurately acquire the direction of the driver pointing with the finger FF.

  According to the present embodiment, the detection range of the motion sensor 11 is divided into a plurality of (for example, two) areas, and the driver's finger enters an arbitrary area among the plurality of areas using the detection information of the motion sensor 11. Therefore, it is possible to reduce the processing load on the ECU 31 and to shorten the processing time. In addition, since a finger only needs to enter one of the regions, the driver can match icons with a rough pointing operation (operation).

(Second Embodiment)
Next, with reference to FIG. 5, the structure of the driving assistance system 2 which concerns on 2nd Embodiment is demonstrated. The driving support system 2 is different from the driving support system 1 according to the first embodiment in the matching method of the icon that the driver points with the finger. The driving support system 2 calculates the direction in which the finger is pointing and matches the icon desired by the driver from the pointing direction. In the first embodiment, the number of superimposed icons is limited according to the number of areas that divide the detection range. However, in the second embodiment, the number of icons superimposed is not limited.

  The driving support system 2 includes a peripheral object information acquisition device 10, a motion sensor 11, a head-up display 20, and an ECU 32. Since the peripheral object information acquisition device 10, the motion sensor 11, and the head-up display 20 are the same as those described in the first embodiment, description thereof will be omitted, and the ECU 32 will be described below.

  The ECU 32 is a control unit that comprehensively controls the driving support system 2. The ECU 32 has, for example, a microprocessor, a ROM, a RAM, a backup RAM, an input / output I / F, and the like, similar to the ECU 31 described above. In the ECU 32, a program stored in a ROM or the like is executed by a microprocessor, whereby a matching unit 32a (corresponding to the matching unit described in the claims) and a display control unit 32b (display described in the claims). Each function of the control means is realized.

  The matching unit 32a matches the icon that the driver points with the finger FF from the icons superimposed on the head-up display 20. Specifically, when the icon is superimposed and displayed, the matching unit 32a calculates the direction in which the driver's finger FF is pointing using the position information of the object (finger FF) detected by the motion sensor 11. . For example, the matching unit 32a obtains a virtual line extending from the steering wheel side to the windshield side using a plurality of three-dimensional coordinate data of the object, and obtains a direction in which the virtual line points from the steering wheel side to the windshield side.

  The matching unit 32a calculates the position indicated by the finger FF in the coordinate system of the head-up display 20 from the direction indicated by the finger FF. Then, the matching unit 32a compares the position of the finger FF with the position of the icon in the coordinate system of the head-up display 20 for each superimposed icon, and which icon the finger FF points to. Determine. For example, the matching unit 32a sets a predetermined determination area around the position of the icon, and the position pointed by the finger FF is continuously included in the determination area for a predetermined time (for example, several seconds to several commas). In this case, it is determined that the finger FF points to the icon.

  The display control unit 32 b causes the head-up display 20 to superimpose and display an icon indicating the peripheral object for which information is acquired by the peripheral object information acquisition device 10. Specifically, the display control unit 32b displays, for each peripheral object for which the peripheral object information acquisition device 10 has acquired information, an icon indicating the peripheral object by the same process as the display control unit 31b according to the first embodiment. The superimposition display position is obtained, and display control information indicating the video information of the icon and the superimposition display position is transmitted to the head-up display 20.

  In addition, the display control unit 32b causes the head-up display 20 to superimpose and display a cursor indicating the position pointed to by the driver's finger FF. Specifically, the display control unit 32b sets the position indicated by the finger FF obtained by the matching unit 32a as a position where the cursor is superimposed and displayed (for example, the center position of the carson, the tip position of the cursor), Display control information indicating the position of the video information and superimposed display is transmitted to the head-up display 20. As for the design, color, etc. of the cursor, those that are easy for the driver to visually recognize are appropriately selected. The cursor shown as an example in FIG. 6 is a picture of an arrow. The superimposed display of the cursor is stopped when the icons are matched. Note that the cursor may not be superimposed and displayed. Further, instead of displaying the cursor, for example, the icon pointed with the finger FF may be displayed in reverse (change the display color).

  Further, when one icon is matched by the matching unit 32a, the display control unit 32b superimposes and displays the detailed information of the peripheral object indicated by the icon on the head-up display 20. Specifically, the display control unit 32b obtains a position where the detailed information is superimposed and displayed by the same process as the display control unit 31b described above, and displays the video information of the detailed information and the display control information indicating the position of the superimposed display, etc. It transmits to the up display 20.

  Next, the operation of the driving support system 2 will be described. Here, the operation will be described with reference to the examples shown in FIGS. FIG. 6 is a diagram illustrating an example when the icons of the peripheral objects are superimposed and displayed on the actual scene in front of the vehicle in the driving support system 2. FIG. 7 is a diagram illustrating an example when the detailed information of the surrounding objects is superimposed and displayed on the actual scene ahead of the vehicle in the driving support system 2.

  In the example shown in FIGS. 6 and 7, the motorized bicycle O1 that exists on the front side of the parked vehicle PV that is parked on the left side in front of the vehicle on which the driving support system 2 is mounted, and the cat that exists on the right side in front of the vehicle. O2, pedestrians O3 and O4 trying to cross a pedestrian crossing in front of the vehicle, a preceding vehicle O5 existing in front of the vehicle, and an oncoming vehicle O6 are peripheral objects existing in front of the vehicle. The peripheral object information acquisition device 10 acquires each piece of information about these peripheral objects O1 to O6, and transmits a peripheral information signal indicating these pieces of information to the ECU 32.

  For each of the peripheral objects O1 to O6, the ECU 32 converts the position of the peripheral object acquired by the peripheral object information acquisition device 10 into a position in the coordinate system of the head-up display 20, and displays an icon superimposed on the vicinity of the converted position. The display control information indicating the video information of the icon and the position of the superimposed display is transmitted to the head-up display 20.

  When the head-up display 20 receives the display control information, the head-up display 20 projects an icon image at a position indicated by the display control information. As a result, as shown in FIG. 6, the icon I1 is superimposed on the front side of the left parked vehicle PV in front of the vehicle, the icon I2 is superimposed on the right side of the cat O2, and each pedestrian O3, Icons I3 and I4 are superimposed and displayed near O4, icon I5 is superimposed and displayed near the preceding vehicle O5, and icon I6 is superimposed and displayed near the oncoming vehicle O6.

  The driver looking forward looks at these six icons I1 to I6. In this example, the driver can visually recognize the surrounding objects (cat O2, each pedestrian O3, O4, each vehicle O5, O6) indicated by the icons I2 to I6, but enters the blind spot of the parked vehicle PV. Since the peripheral object indicated by the icon I1 is not visually recognized, detailed information on the peripheral object indicated by the icon I1 is obtained. In order to match the icon I1, the driver points the index finger FF of the left hand LH toward the icon I1 while looking forward while holding the steering wheel W with both hands LH and RH. The motion sensor 11 detects the finger FF that has entered the detection range R, and transmits position information of the detected finger FF to the ECU 32.

  The ECU 32 calculates the direction pointed by the finger FF using the position information of the finger FF detected by the motion sensor 11, and the position pointed by the finger FF in the coordinate system of the head-up display 20 from the pointing direction is calculated. calculate. Then, the ECU 32 transmits to the head-up display 20 display information indicating the video information of the cursor, the position of the superimposed display, and the like in order to superimpose and display the cursor at the position indicated by the finger FF. When the head-up display 20 receives the display control information, the head-up display 20 projects a cursor image at a position indicated by the display control information. Thereby, as shown in FIG. 6, the cursor C indicating the position where the driver is pointing with the finger FF is superimposed and displayed. Instead of displaying the cursor C, for example, the icons I1 to I6 pointed by the finger FF may be displayed in reverse (change the display color).

  The driver who is looking forward visually recognizes the cursor C and moves the finger FF while looking at the cursor C. When the cursor C is away from the matching target icon I1, the driver moves the finger FF so that the cursor C approaches the icon I1.

  When the cursor C comes close to the icon I1 or overlaps the icon 11, the ECU 32 indicates the direction pointed by the finger FF calculated from the position information of the finger FF detected by the motion sensor 11 (the position pointed by the finger FF). Thus, the icon I1 pointed by the driver with the finger FF is matched. Then, the ECU 32 transmits to the head-up display 20 video information of the detailed information of the peripheral object O1 indicated by the icon I1, display control information indicating the position of the superimposed display, and the like.

  The head-up display 20 projects an image of detailed information of the peripheral object O1 at a position indicated by the display control information. Thereby, as shown in FIG. 7, the detailed information DI is superimposed and displayed in the vicinity of the peripheral object O1. In the case of this example, the icon I1 indicating the peripheral object O1 on which the detailed display DI is superimposed is continuously displayed, and the superimposed display of the icons I2 to I6 indicating the peripheral objects O2 to O6 is stopped. The driver looking forward can visually recognize this detailed information DI as in the first embodiment described above, and there is a possibility that the motorized bicycle existing on the front side of the parked vehicle PV may come out on the road. Can know.

  According to the present embodiment, based on the three-dimensional position of the driver's finger FF detected by the motion sensor 11, the direction of the finger FF is acquired, and the driver moves the finger from the acquired direction. The icon pointed to by is matched. Therefore, even if there are many acquired peripheral objects and many icons are superimposed, the icon desired by the driver can be accurately determined from the many icons. . Further, according to the present embodiment, since the cursor is superimposed and displayed during the matching operation with the driver's finger FF, the position pointed to by the finger FF can be easily matched with a desired icon.

  Further, according to the present embodiment, the number of peripheral objects (icons) to be superimposed and displayed is not limited. Furthermore, since only the detailed information (desired detailed information) of the matched icon is displayed as the detailed information, it can be prevented from being complicated by the superimposed image.

  Although the embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various modifications can be made. For example, in the above embodiment, the head-up display 20 is used to superimpose and display the actual scene. However, instead of the head-up display 20, for example, a display device such as a head-mounted display may be used.

  In the above embodiment, the case of superimposing and displaying on the actual scene in front of the vehicle has been described as an example. However, the present invention can also be applied to the case of superimposing and displaying on the actual scene in other directions such as the rear of the vehicle. Further, in the above-described embodiment, the configuration is such that the icons of the peripheral objects present in the front are superimposed on the front real scene, but in addition to this, the icons of the peripheral objects existing in the rear or side may be superimposed and displayed. . For example, a vehicle that travels rearward or laterally is detected by a detection means that detects an object that exists behind or laterally, and an icon indicating the vehicle or the like is superimposed and displayed when the icon is pointed Detailed information about the vehicle and the side vehicle is superimposed and displayed. As a result, information corresponding to information obtained by a door mirror or the like can be obtained without the driver moving his / her line of sight from the front. In addition, an image acquired by a camera or the like is displayed as an alternative to a door mirror, and for example, the driver is notified by an icon only when a parallel running vehicle exists on the left and right, and the icon is pointed Sometimes, it is possible to display an image acquired by a camera (corresponding to door mirrorless).

  In the above embodiment, an icon indicates that there is a peripheral object that can provide information. However, other mark information such as a rectangular frame surrounding the peripheral object may be used to indicate that the peripheral object exists. Moreover, although the example which superimposes and displays the detailed information of a surrounding object with the character string was shown in the said embodiment, you may superimpose and display the detailed information of other forms, such as the image imaged with the camera. For example, when the image is dark at night or the like, the captured image of the infrared camera may be superimposed and displayed as detailed information, or the captured image of the camera that captures the rear or side may be superimposed and displayed as detailed information.

  In the above embodiment, the motion sensor constituted by the distance image sensor is shown as the finger detection means, but other finger detection means constituted by a three-dimensional camera, a radar sensor, or the like may be applied.

  In the above embodiment, the range where the steering wheel W does not enter is set as the detection range of the motion sensor 11, but a detection range where the steering wheel W enters may be set. In this case, for example, it is preferable to mask the information of the steering wheel W from the detection information of the motion sensor 11 and determine the icon (detailed information) desired by the driver using the detection information after masking.

  In the above-described embodiment, the motion sensor 11 (distance image sensor) is placed at the upper part of the vehicle interior on the driver's seat and in front of the steering wheel (for example, the front end of the driver's seat on the ceiling or the driver's seat on the windshield). However, instead of these positions, for example, it may be arranged between the windshield and the dashboard, or in the housing of the meter.

  Moreover, in the said embodiment, based on the three-dimensional position information of the finger | toe acquired by the motion sensor 11 (distance image sensor), it was set as the structure which acquires the direction which the driver is pointing and matches with an icon. However, instead of such a configuration, for example, the motion sensor 11 (distance image sensor) detects the three-dimensional coordinates of the fingertip, synchronizes the three-dimensional coordinates with the coordinates on the head-up display 20, and It is good also as a structure which performs coordinate matching. In this case as well, the number of icons displayed in a superimposed manner is not limited.

1, 2 Driving support system 10 Peripheral object information acquisition device (peripheral object information acquisition means)
11 Motion sensor (finger detection means)
20 Head-up display (display means)
31, 32 ECU
31a, 32a matching unit (matching means)
31b, 32b Display controller (display control means)

Claims (8)

Peripheral object information acquisition means for acquiring information on peripheral objects existing around the vehicle;
Display means for displaying mark information indicating the peripheral object acquired by the peripheral object information acquisition means superimposed on a real scene around the vehicle;
Finger detection means for detecting a finger of a driver's hand holding the steering wheel of the vehicle;
A matching means for matching the mark information desired by the driver from the mark information superimposed and displayed by the display means according to the position or movement of the finger detected by the finger detection means,
A human interface, comprising: a display control unit that causes the display unit to superimpose and display information on a peripheral object indicated by the landmark information matched by the matching unit.   The human interface according to claim 1, wherein the finger detection unit detects a finger coming out from the steering wheel. The finger detection means is disposed in the upper part of the vehicle interior of the vehicle, and has a detection range directed downward from the disposed location,
3. The human interface according to claim 2, wherein a finger protruding forward from the steering wheel enters the detection range, and the steering wheel does not enter. 3.   The matching unit divides the detection range into a plurality of regions, and a driver's finger enters one of the regions included in the plurality of regions according to the position or movement of the finger detected by the finger detection unit. The human interface according to claim 3, wherein when it is determined whether or not a finger is present, the landmark information corresponding to the region where the finger is present is matched.   The matching unit acquires a direction in which the finger is pointing based on the position or movement of the finger detected by the finger detection unit, and matches mark information pointed by the driver with the finger from the acquired direction. The human interface according to claim 3.   The human interface according to claim 1, wherein the finger detection unit is configured by a distance image sensor.   The human interface according to claim 6, wherein the position of the finger is a three-dimensional coordinate of the fingertip detected by the distance image sensor.   The human interface according to claim 1, wherein the mark information is an icon.

Images