JP2009227245A - Operation device for on-vehicle equipment - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent various information from becoming hard to be visually recognized when various information to be visually recognized by a driver and projected image elements virtually displayed on a display means overlap one another. <P>SOLUTION: The device captures a front scene of a vehicle by a front scene shooting camera 4 and recognizes information (first information) to be visually recognized by a driver such as a road sign 15 contained in a captured image and information (second information) not wanted to be visually recognized by a driver such as sun 21. Switches 10 of on-vehicle equipment is displayed on a virtual display 1. When the first information and the switches 10 are overlapped one another, the switches 10 are moved and transmissively displayed. When the second information and the switches 10 are overlapped one another, the switches 10 are highlighted. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an operating device for in-vehicle equipment in a vehicle.

  In recent vehicles (for example, automobiles), a virtual display is installed integrally with the front window or on the front side of the front window, and switches for in-vehicle devices (navigation device, audio device, air conditioner, etc.) are displayed on this display. What projects (projected image element) and displays it by superimposing on a front window is known (for example, refer patent document 1). This type of technology is called “head-up display (HUD)” or “windshield display (WSD)”. In this technique, since switches are displayed on the display, the driver has an advantage that the switches of the in-vehicle device can be operated while keeping his or her line of sight facing forward.

  In front of the traveling vehicle, there are various information (road signs, road depressions, sunbeam reflections, etc.) that the driver can visually recognize through the front window. For this reason, such information and the display of switches on the display may overlap. Then, it becomes difficult for the driver to visually recognize various kinds of information, and it becomes difficult to accurately operate the switches.

In the case of a conventional operating device, the display position of the switches displayed on the display is always constant. However, in the curved portion of the road (including the crank road), the driver often visually recognizes the area where the vehicle is about to turn rather than the area just before the vehicle. For this reason, the driver must move the line of sight greatly in order to operate the switches.
JP 2006-327526 A

  In view of the above-described problems, the present invention prevents various information from becoming difficult to visually recognize when various information to be visually recognized by a driver overlaps a projection image element virtually displayed on a display unit. Is an issue.

Means for Solving the Problems and Effects of the Invention

The present invention for solving the above problems is as follows.
Display means for virtually displaying buttons, switches and other projection image elements installed on or near the front window of the vehicle for operating equipment mounted on the vehicle in a state of being superimposed on the front window;
Operating finger detection means for detecting that the driver of the vehicle has operated the projection image element with a finger;
Forward landscape image acquisition means for capturing a forward landscape image of the vehicle and acquiring a forward landscape image;
Image processing means for extracting a preset avoidance target area from the acquired forward landscape image;
Control for performing control to move the display position or to change the display method when the display position of the projection image element on the display unit overlaps the avoidance target area extracted from the forward landscape image Means,
It is characterized by having.

  The operation device of the in-vehicle device according to the present invention is configured as described above, and when the projected image element displayed on the display unit overlaps information (avoidance target area) existing in the front landscape of the vehicle, The display position of the projection image element is moved or the display method is changed. When the display method is different, the projection image element is displayed in a transparent manner or highlighted. As a result, the driver can surely visually recognize the information existing in the front scenery of the vehicle even when the information existing in the front scenery of the vehicle overlaps the projection image element displayed on the display means.

  The avoidance target area includes road signs, obstacles on the road, depression of the road surface, sun position, reflection of solar rays on buildings, road surfaces, and the like. For this reason, the driver can surely visually recognize the front scenery even when the above-described information overlaps with the projection image element displayed on the display means.

  When the projected image element is displayed superimposed on the front window, an initial display position with respect to the front window is determined.

  By defining the display position of the projection image element as the initial display position (default position), the projection image element can be displayed at a position that is most easily visible to the driver. Then, when the projection image element and the information of the front landscape overlap, the projection image element is moved from the default position. This facilitates control of movement of the projection image element.

  Further, the display position of the projection image element may be changed following the change of the driver's visual recognition area while the vehicle is traveling, and may be arranged in the vicinity of the driver's visual recognition area.

  For example, when driving on a curved portion of a road, the driver often visually recognizes a region where the vehicle is about to turn. In such a case, the movement of the driver's line of sight can be reduced by disposing the projected image element in the vicinity of the previous area (viewing area) where the vehicle is about to turn instead of the default position.

  The front landscape image acquisition means may be a camera installed in the vehicle.

  Thereby, a camera can be installed in the substantially same position as a driver | operator's eyes | visual_axis. Of course, the camera may be installed outside the vehicle.

  Then, the control means captures a state in which the front landscape image acquisition means captures a front landscape of the vehicle, the image processing means extracts the avoidance target area from the acquired front landscape image, and the projection on the display means. It is possible to perform time-division display control that alternately repeats the state of displaying image elements.

  Examples of the present invention will be described. 1 is a schematic diagram of an operating device 100 according to an embodiment of the present invention, FIG. 2 is a block diagram of the operating device 100, FIG. 3 is a diagram showing a front landscape of the vehicle, and FIG. 4 is a diagram showing a front landscape of the vehicle on a curved road. It is.

  As shown in FIGS. 1 and 2, the operating device 100 according to the embodiment of the present invention includes a display 1 (display means) virtually provided in the vicinity of an instrument panel in a vehicle (in the case of the present embodiment, an automobile). ), A front landscape photographing camera 4 (front landscape image acquisition means) that captures a front landscape of the vehicle, and is installed almost directly above the display 1, And an operation finger photographing camera 7 (operation finger detection means) for photographing 6 and detecting its coordinate position. In FIG. 1, 8 is a handle.

  The display 1 includes buttons and switches (hereinafter collectively referred to as “switches 10” (projected image elements)) for operating in-vehicle devices such as a navigation device 9 and an air conditioner (not shown). It is displayed virtually superimposed on the front window 11 of the vehicle. The driver 5 operates the switches 10 displayed on the display 1 with his / her finger 6. The finger 6 at this time is photographed by the operation finger photographing camera 7, and its coordinate position is detected. Then, the ECU 13 (control means; see FIG. 2) compares the coordinate position of the finger 6 with the position of the switches 10 displayed on the display 1, and whether the driver 5 has operated the switches 10 on the display 1. Judge whether.

  As shown in FIG. 3, the forward scenery photographing camera 4 photographs the forward scenery seen through the front window 11. It is desirable to install the front landscape photographing camera 4 on the headrest 3 of the driver's seat 2 so as to be almost the same as the line of sight of the driver 5 in the driving state. In addition, a pair of front landscape photography cameras 4 are installed on both sides of the headrest 3 so that all the front scenery viewed through the front window 11 can be photographed, and two images photographed by the pair of front landscape photography cameras 4. May be synthesized. The photographed front landscape is subjected to predetermined image processing by the image processing unit 12 and taken into the ECU 13. The photographed front landscape has various information. In the case of the forward landscape shown in FIG. 3, the road information board 14, the road sign 15, the depression 16 on the road surface, the obstacle 17, the center line 18, the boundary 19 with the sidewalk, the building 20, the sun 21, and the sunlight are reflected. Information on the glass 22 of the building 20, the road surface 23, the snow block 24, and the headlight 25 of the oncoming vehicle.

  Here, when the vehicle travels on a straight road, the switches 10 are displayed in the lower right part of the front window 11 (this position is described as “default position” of the switches 10). However, when the vehicle travels on a curved portion of the road (including the crank road), the driver 5 visually recognizes a region where the vehicle is about to turn rather than a region immediately before the vehicle. For this reason, even when the vehicle tries to travel on the curved portion, if the switches 10 are displayed at the default position, the movement of the driver's line of sight increases. In order to avoid this, the operating device 100 according to the present embodiment detects whether the road is a straight road by detecting a ridgeline or a white line (a centerline 18 or a boundary line 19 with a sidewalk) of the road on which the vehicle travels. It is determined whether it is a road. Then, as shown in FIG. 4, when the road is a curved road, the switch 10 is moved to the previous area where the vehicle is about to turn, following the change in the visual recognition area of the driver 5 ( Later).

  The operation of the operating device 100 according to the present embodiment will be described. When the driver 5 turns on a key switch (not shown) of the vehicle, the display 1 and the operation finger photographing camera 7 are activated. Thereby, the switches 10 of the in-vehicle devices (for example, the navigation device 9 and the air conditioner) are displayed on the display 1 so as to be superimposed on the front window 11 (see FIG. 3). When the key switch is turned on, the switches 10 are displayed at the lower right (default position) of the front window 11. The driver 5 can visually recognize the switches 10 without greatly moving the line of sight for visually recognizing the front window 11. When the driver 5 operates the switches 10 of the display 1, the finger 6 of the driver 5 is photographed by the operation finger photographing camera 7, and the coordinate position is detected.

  As shown in FIG. 5, the front scenery of the vehicle photographed by the front scenery photographing camera 4 is taken into the ECU 13 as image data (step S1). In this image data, as information visually recognized by the driver 5, a road information board 14, a road sign 15, a depression hole 16 on the road surface, an obstacle 17, a center line 18, a boundary line 19 with a sidewalk, a building 20, and a sun 21 The glass 22 of the building 20 reflecting the sunlight, the road surface 23, the snow block 24, and the headlight 25 of the oncoming vehicle are displayed.

  These pieces of information are subjected to image processing by the image processing unit 12 (step S2). In other words, these pieces of information must be visually recognized by the driver 5 (road information board 14, road sign 15, road depression 16 and obstacle 17. These are hereinafter referred to as "first information". ) And information that the driver 5 does not want to visually recognize (the sun 21, the glass 22 of the building 20 that reflects the sunlight, the road surface 23, the snow block 24, and the headlight 25 of the oncoming vehicle). "Information".) And road information (center line 18, border 19 with sidewalk. These are hereinafter referred to as "third information"). The first information is recognized by image processing the image processing unit 12. Further, the second information is recognized by measuring the magnitude of the luminance by the image processing unit 12. Then, as shown in FIG. 6, first and second information display areas are extracted from the captured image that has been subjected to image processing. Each display area is described by adding a subscript “a” to a code indicating the corresponding information.

  Next, by recognizing the third information, it is determined whether the currently traveling road is a straight road or a curved road, and a road surface ahead is predicted (step S3). If it is a straight road (“Yes” in step S3), the switches 10 are displayed at the default position (step S4). And it is confirmed whether the display area of the 1st and 2nd information and the switches 10 have overlapped (step S5). If the first and second information display areas do not overlap with the switches 10 (“No” in step S5), the switches 10 are displayed as they are in the default position (step S6). If the display area of the first and second information overlaps the switches 10 (“Yes” in step S5), it is checked whether there is an area for moving the switches 10 (step S7). If there is an area for moving the switches 10 (“Yes” in step S7), as shown in FIG. 7, the switches 10 are moved from the default position to display the first and second information display areas (FIG. 7). In this case, the display is made so as not to overlap with the depression 16) on the road surface (step S8). If there is no area for moving the switches 10 (“No” in step S7) and the switches 10 overlap the display area of the first information, the switches 10 are displayed transparently (step S9). Thereby, the first information can be visually recognized through the switches 10. If the switches 10 overlap the display area of the second information, the switches 10 are highlighted (step S9). Thereby, it can avoid that the switches 10 become difficult to visually recognize by 2nd information.

  If the road is a curved portion in step S3 ("No" in step S3), the switches 10 are displayed at the destination where the vehicle is about to turn (step S10). The following steps S11 to S15 are exactly the same as steps S5 to S9 when the road is a straight road.

  In the operating device 100 according to the present embodiment, the image data of the front scenery photographed by the front scenery photographing camera 4 and the switches 10 perform time-division display control in which their display states are alternately repeated. As shown in FIG. 8, switches 10 are displayed on the display 1. This state is indicated by “A”. Then, when the forward scenery photographing camera 4 photographs the forward scenery, the display of the switches 10 on the display 1 is stopped. This state is indicated by “B”. Thereby, the switches 10 do not appear in the photographed front landscape. Then, the switches 10 are displayed on the display 1 again. While the switches 10 are displayed on the display 1, the front landscape photographed by the front landscape camera 4 is captured as image data, the avoidance target area is extracted, the overlapping state is confirmed, and the switch is switched as necessary. The operation of moving the display position of the class 10 is performed. Since the times “A” and “B” are performed in a minute time that is difficult for the driver 5 to recognize, it appears to the driver 5 that the switches 10 are moving continuously, There is no risk of disturbing operation.

  In this embodiment, the front landscape of the vehicle is acquired by performing image processing on an image captured by the front landscape photographing camera 4. However, a forward landscape image may be acquired from the map information of the navigation device 9.

  In the operating device 100 according to the present embodiment, it is determined whether or not each piece of information in the front landscape overlaps the switches 10. In addition to this, the overlap between the in-vehicle information and the switches 10 may be determined, and the switches 10 may be moved from the default position. As in-vehicle information, for example, reflection by a room lamp is assumed.

  In addition, by acquiring information on whether the vehicle is accelerating, decelerating, or turning, the display position of the switches 10 can be arranged at a position that is more easily visible to the driver 5. For example, the switches 10 are displayed further forward when the vehicle is accelerating, the switches 10 are displayed slightly forward when the vehicle is decelerating, and the switches 10 are displayed when the vehicle is turning. Or display in the turning direction.

It is the schematic of the operating device 100 of the Example of this invention. 3 is a block diagram of the operating device 100. FIG. It is a figure which shows the front scenery of a vehicle. It is a figure which shows the front scenery of a vehicle on a curved road. 3 is a flowchart showing the operation of the operating device 100. It is action explanatory drawing of the state which extracts the area | region where each information is displayed from the image | photographed front scenery image. It is action explanatory drawing which moves the display position of switches 10. It is action | operation explanatory drawing of the state which carries out time-sharing control of the display time of the switches 10 and the imaging | photography time of a front landscape.

Explanation of symbols

100 Operating device 1 Display (display means)
4 Front landscape camera (Front landscape image acquisition means)
5 Driver 6 Finger 7 Operating finger photographing camera (operating finger detection means)
10 Switches (projection image elements)
11 Front window 12 Image processing unit (image processing means)
13 ECU (control means)
14 Road information board (area to avoid)
15 road signs (areas to avoid)
16 Sinkhole (area to avoid)
17 Obstacle (Avoidance target area)
21 Sun (area to avoid)
22 Building glass reflecting sunlight (area to avoid)
23 Road surface that reflects sunlight (area to avoid)
24 Snow mass reflecting sun rays (area to avoid)
25 Headlight (Avoidance target area)

Claims (6)

Display means for virtually displaying buttons, switches and other projection image elements installed on or near the front window of the vehicle for operating equipment mounted on the vehicle in a state of being superimposed on the front window;
Operation finger detection means for detecting that the driver of the vehicle has operated the projection image element with a finger;
Forward landscape image acquisition means for capturing a forward landscape image of the vehicle and acquiring a forward landscape image;
Image processing means for extracting a preset avoidance target area from the acquired forward landscape image;
Control for performing control to move the display position or to change the display method when the display position of the projection image element on the display unit overlaps the avoidance target area extracted from the forward landscape image Means,
A device for operating an in-vehicle device.   2. The vehicle-mounted device according to claim 1, wherein the avoidance target region includes road signs, obstacles on a road, depression of a road surface, sun position, reflection of sunlight on a building, a road surface, and the like. Operating device.   3. The operating device for an in-vehicle device according to claim 1, wherein an initial display position with respect to the front window is determined when the projection image element is displayed superimposed on the front window.   The display position of the projection image element changes following the change of the driver's visual recognition area while the vehicle is traveling, and is arranged in the vicinity of the driver's visual recognition area. The operation apparatus for vehicle equipment according to any one of the above.   The operating device for an in-vehicle device according to any one of claims 1 to 4, wherein the front landscape image acquisition means is a camera installed in a vehicle.   The control means includes a state in which the front scenery image acquisition means captures a front scenery of a vehicle, the image processing means extracts the avoidance target area from the acquired front scenery image, and a projection image element on the display means. 6. The operating device for an in-vehicle device according to claim 1, wherein time-division display control that alternately repeats a state in which the information is displayed is performed.

Images