JP4026146B2 - VEHICLE IMAGE DISPLAY DEVICE, VEHICLE IMAGE DISPLAY METHOD, AND VEHICLE IMAGE DISPLAY PROGRAM - Google Patents

Description

  The present invention relates to an image display technique for a vehicle that transmits information to a driver by displaying an image so as to be superimposed on a real landscape.

  Conventionally, there is a system that provides information about various facilities to a driver of a vehicle.

  In particular, in recent years, as the traffic network has developed and the driver has moved freely over a wide area, it has become difficult for the driver to constantly grasp the complicated and changing road conditions, and the vehicle has moved to the destination by a vehicle such as an automobile. Navigation systems that provide guidance information for doing so are known and are in widespread use.

As an advanced form of such a navigation system, a system has been developed that displays an image in the vicinity of the front window so as to overlap the actual scenery. For example, Patent Document 1 discloses an apparatus that captures an actual scene with a camera and displays an image representing attribute information of the structure so as to be superimposed on the structure in the actual scene.
JP 11-0665431 A

  However, in the system as described above, a facility-related image is displayed unconditionally when the facility existing in the visual field becomes visible. Therefore, as a result, display of a large number of facility-related images is started at the same time, which may impair driving safety.

  For example, immediately after turning a sharp curve or immediately after exiting a fence, if a completely new landscape spreads out at once, a large number of images are displayed in the driver's field of view at once, and the driver is surprised. Or obstructed the driver ’s view.

  The present invention has been made to solve the above-described problems of the prior art, and the object of the present invention is to prevent the safety of driving from being hindered when an image relating to a facility is displayed superimposed on a real scene. , To control its display.

In order to achieve the above object, an apparatus according to the present invention provides:
In order to provide information on a facility existing in front of the vehicle, an image display device for a vehicle that displays a facility-related image related to the facility so as to be superimposed on an actual scenery that is visible from the driver of the vehicle through a window. Because
When the facility is visible, a predetermined facility-related image is displayed, and when the facility is invisible, the display control means performs display control in a display mode in which the predetermined facility-related image is not displayed.
The display control means limits the number of facility-related images whose display is started within a predetermined time.

  The display control means determines whether to start displaying the facility-related image within the predetermined time according to the type of the corresponding facility.

Vehicle guidance means for guiding the vehicle with the facility as an index;
The display control means excludes the facility that is used as the vehicle guidance index by the vehicle guidance means from the display restriction of the facility-related image.

The display control means starts the display within the predetermined time by delaying the display start timing of the facility-related image for a part of the facility-related image whose display is started within the predetermined time. It is characterized by limiting the number of facility-related images.

  The display control means further delays the display start of the facility-related image predicted to be displayed late among the plurality of facility-related images predicted to be displayed within the predetermined time. And

Wherein the display control means, when the facility related image displayed within the predetermined time is started is estimated that more than a predetermined number, to be displayed in order from the facility related image distance of closest facilities from the vehicle It is characterized by.

  The display control means regulates the display of the facility-related images whose time that is not visible among the plurality of facility-related images whose display is started within the predetermined time is a predetermined value or more. And

In order to achieve the above object, the method according to the present invention comprises:
A vehicle image display method for displaying a facility-related image related to the facility so as to be superimposed on an actual scenery that can be viewed through a window from a driver of the vehicle in order to provide information regarding the facility existing in front of the vehicle. Because
When the facility is visible, a predetermined facility-related image is displayed, and when the facility is not visible, the display control step performs display control in a display mode in which the predetermined facility-related image is not displayed.
The display control step limits the number of the facility-related images that are displayed within a predetermined time.

In order to achieve the above object, a program according to the present invention provides:
In order to provide information on a facility existing in front of the vehicle, an image display device for a vehicle that displays a facility-related image related to the facility so as to be superimposed on an actual scenery that is visible from the driver of the vehicle through a window. In addition,
A vehicle for executing a display control step of performing display control in a display mode in which a predetermined facility-related image is displayed when the facility is visible and a predetermined facility-related image is not displayed when the facility is not visible. An image display program for
The display control step limits the number of the facility-related images that are displayed within a predetermined time.

  According to the present invention, since an image related to a facility is displayed so as to be superimposed on the actual scenery, the driver can acquire information on the facility with a small amount of line of sight movement during driving, thereby improving safety. be able to. In addition, since the number of the facility-related images that are displayed within a predetermined time is limited, it is possible to ensure safety so as not to hinder driving and to prevent the driver from feeling uncomfortable. .

  Since the start timing of the display of facility-related images is delayed for some of the facilities that can be viewed within a predetermined time, it is necessary while limiting the number of facility-related images that are displayed within a predetermined time. Facility-related images can be displayed reliably.

  Of the multiple facility-related images that are displayed within a predetermined time, the start of the display of the facility-related images that are displayed late is delayed, so the facility-related images that are displayed almost simultaneously without changing the order of the display of the facility-related images The number of images can be limited.

  When it is estimated that the number of facilities that can be visually recognized within a predetermined time is greater than or equal to a predetermined number, since the facility-related images of the facilities with a short distance are displayed in order, the number of facility-related images displayed can be reduced efficiently.

  Of the plurality of facility-related images that are displayed within a predetermined time, the display is restricted for facility-related images that have been invisible for a predetermined time or more, so the number of facility-related images displayed can be reduced efficiently. be able to.

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. However, the relative arrangement of components, the display screen, and the like described in this embodiment are not intended to limit the scope of the present invention only to those unless otherwise specified.

  The vehicle image display apparatus according to the present invention provides information related to facilities existing in front of the vehicle, so that the facilities related to facilities are visible so as to be superimposed on the actual scenery visible from the driver of the vehicle through the window. An image is displayed. Hereinafter, as an embodiment of the vehicle image display apparatus according to the present invention, an in-vehicle system that provides information on various facilities to the driver by an image, voice, or the like in conjunction with a car navigation system will be described. In addition, in this specification, a facility is not limited to a building or a park, but includes roads and intersections, and further includes natural objects such as mountains.

<System configuration>
FIG. 1 is a diagram showing an overall configuration of a vehicle 1000 equipped with an in-vehicle system according to the present embodiment. The position of the eye (eye point) 10a of the driver 10 sitting in the driver's seat of the vehicle 1000 is a position corresponding to the body shape and posture. Then, as shown in FIG. 1, the driver 10 can see a real landscape spreading in front of the vehicle 1000 through the front window 1100. That is, the field of view 10b from the eye point includes a partial area of the front window 1100 and a real landscape spreading in front of the vehicle. Therefore, in the present embodiment, the display unit 110 is used to display an image (computer graphics) on the front window 1100 (or the vicinity thereof) and display the information on the facility in the actual scenery by overlapping the actual scenery. I will provide a. For this reason, the vehicle 1000 has an image according to the eye point camera 120 for detecting the position of the eye point 10a of the driver 10, the position of the eye point 10a of the driver 10, the vehicle position, 3D map data, and the like. Is generated and displayed on the display unit 110. The control unit 100 is provided.

  FIG. 2 is a diagram illustrating an example of an image displayed by the in-vehicle system according to the present embodiment. For convenience, the driver is not shown in FIG.

  As shown in FIG. 2, the driver can visually recognize mountains, roads, buildings, and the like through the front window 1100 as actual scenery. Then, the image 200 is displayed so as to be superimposed on the actual scenery. This image 200 is a facility-related image indicating information related to the facility 210 in the actual landscape. Here, at least an arrow 201 as a simple image indicating the position of the facility 210 and a detailed image for explaining the facility 210 The comment 203 is included. Here, since the facility 210 is a gas station, the comment 203 indicates “gas station”. As a comment, in addition to the type and name of the facility, the business content and advertisement of the facility may be displayed. In addition to arrows, a characteristic image such as a star may be used as a simple image. Furthermore, if a comment is displayed so that it can be seen superimposed on the facility, it is possible to omit a simple image if it is clear which facility the comment indicates. A circle 202 added to the rear end of the arrow is an image indicating that the facility 210 is set as a destination in the car navigation system. Of course, information indicating that the facility is a destination, waypoint, or registered location may be displayed as an image in another form. For example, characters such as “Destination” are displayed in the comment part. Also good.

<System hardware configuration>
Next, the internal configuration of the present system will be described with reference to FIGS. 3 and 4. FIG. 3A is a block diagram illustrating a hardware configuration of the in-vehicle system according to the present embodiment.

  As shown in FIG. 3A, the system includes a control unit 100 that controls the entire system, a display unit 110 that displays an image on a front window, an eye point camera 120 that captures a driver and detects an eye point position, A communication unit 130 that communicates with a server installed in an information center outside the vehicle via a network, a microphone 140 that receives instructions from the driver, a speaker 150 that provides information to the driver by voice, and instructions from the driver It includes an input operation unit 160 and various sensors 170 for detecting the behavior and position of the vehicle.

  If a head-up display using a hologram is used as the display unit 110, the focus of the driver's eyes can be made substantially the same as when viewing a real landscape. However, the display unit 110 here is not limited to this, and any unit can be used as long as it can display an image so as to be superimposed on the actual scenery when viewed from the driver. . For example, a configuration in which an image is projected by a projector on a translucent screen attached to the front window may be used, or a configuration in which a transflective liquid crystal display is attached to the front window and a liquid crystal display is driven to display a desired image may be used. The details of such a display method have been disclosed in various past documents (for example, Japanese Patent Laid-Open No. 1-219883 and Japanese Patent Laid-Open No. 4-125679).

  The eyepoint camera 120 is a camera that captures the driver's head, identifies the driver's eyes from the captured image, and calculates the position thereof.

  The communication unit 130 can access the communication network by downloading radio waves to the base station closest to the vehicle and download the latest map data from a specific server. The downloaded map data is stored in the control unit 100. Setting data relating to the display of facility-related images and vehicle guidance points (favorites, etc.) can be uploaded to the server from the driver's home or the like, and the setting data can also be downloaded by the communication unit 130. Furthermore, the setting data can be incorporated into the map data and downloaded to the communication unit 130 on the server side.

  The operation unit 160 includes, for example, input operation keys such as a numeric keypad and various touch switches and an interface circuit thereof, and receives operation input from a passenger such as a driver. The operation unit 160 may have a display for displaying various setting contents.

  The various sensors 170 include an antenna, a GPS receiving unit, a speed sensor, an acceleration sensor, a direction sensor, and the like. The antenna is a small high gain antenna such as a planar antenna or a helical antenna, for example, and receives radio waves from a plurality of GPS satellites and transmits them to the GPS receiving unit. The GPS receiving unit mainly performs spread spectrum communication using the DS (Direct Sequence) method, detects and demodulates the received radio waves from the GPS satellites, obtains positioning data, and based on this, obtains the current vehicle current by GPS navigation. Calculate the position and heading.

  The speed sensor, the acceleration sensor, and the azimuth sensor are composed of a sensor for detecting various self-contained navigation parameters required for the self-contained navigation and its interface circuit. The speed sensor is a sensor such as a rotary encoder, for example, and detects the speed of the vehicle by taking a vehicle speed pulse. The acceleration sensor detects the acceleration applied to the vehicle by taking pressure change information using inertia such as a diaphragm. The direction sensor detects the direction of the vehicle using a gyro mechanism such as a gimbal, for example. Note that the positioning of the current position and the determination of the heading are performed by mixing and correcting the data from GPS navigation and the data from autonomous navigation. 8-11170 and Japanese Patent Laid-Open No. 10-307036), details of which are disclosed, and a description thereof will be omitted.

  The control unit 100 fetches predetermined information from the eye point camera 120, the communication unit 130, the microphone 140, the operation unit 160, and the various sensors 170 every predetermined period in the course of executing various programs, and comprehensively stores such information. Thus, the image to be output to the display unit 110 and the sound to be output from the speaker 150 are derived.

  As shown in FIG. 3A, the control unit 100 includes a CPU (specifically a microprocessor) 101 for arithmetic control, a ROM (Read Only Memory) 102 for storing fixed values and various program modules, and various data. And a RAM (Random Access Memory) 103 for temporarily storing programs, an HDD (Hard Disk Drive) 104 for storing 3D map data 104 and various setting data 104b, and an input / output port (I / O) 106. Here, 3D map data and setting data are stored in the HDD 104, but this can be replaced by a drive of a recording medium such as a DVD or a CD-ROM, or a nonvolatile semiconductor memory such as a flash memory. is there.

  The CPU 101 controls the entire in-vehicle system by executing a main routine program stored in the ROM and various program modules.

  The ROM 102 extracts a visible facility from the 3D map data 104a, generates a facility-related image, the vehicle position, vehicle travel information, 3D map data 104a, destination setting information, and the like. The navigation module 102b that generates a route to guide the vehicle, the setting module 102c that generates or corrects the setting data 104b based on an operation on the operation unit 160, and the image captured by the eye point camera 140. An eye point detection module 102d for detecting an eye point position and a line-of-sight direction, a voice analysis module 102e for analyzing voice input from the microphone 140, and a voice synthesis module 102f for generating voice to be output from the speaker 150. I remember it.

  The RAM 103 temporarily stores a program execution area 103a for executing various programs, a map data storage area 103b for temporarily storing 3D map data 104a read from the HDD 104, and setting data 104b input by the setting module 102c. And a setting data storage area 103c stored in the vehicle and a vehicle data storage area 103d for temporarily storing data relating to the behavior and position of the vehicle input by the various sensors 160.

  The 3D map data 104a includes facility and shape data including buildings, parks, roads, intersections, and other natural objects such as mountains, and road information on road routes and road widths. It contains data such as road intersection location information.

  The setting contents stored in the HDD 104 include the following A and B, which can be set by the operation unit 150 or a personal computer connected to the server via a network. These items can also be set separately for each region, for each passenger, for each time zone, for each day of the week, and the like. An example of the setting screen is shown in FIGS. 3B and 3C.

A. Basic items A-1. Whether to display a facility-related image, whether to display a guidance information image, whether to display an advertisement, display priority when displaying these, facility-related images A-2. Facility-related image size (standard, 2x, 3x, automatically selected according to distance)
A-3. Number of facility-related images that can be displayed simultaneously (select from standard, minimum, less, more, maximum, etc.)
A-4. Whether or not the facility to be viewed and the facility-related image may be visible. If they overlap, do you want to prohibit display or shift the display position? A-5. Presence / absence of voice guidance and presence / absence of icons Detailed items B-1. What is displayed as the facility-related image, that is, whether the destination is displayed or the facility name is displayed. Whether to display an arrow for guiding the vehicle or a mark for guiding the vehicle B-2. Selection of facilities to be displayed for facility-related images (registered facilities, convenience stores (can be specified in more detail), roadside stations, gas stations (can be specified in more detail), amusement facilities (more Details can be specified), dining facilities (can be specified in more detail), event holding facilities (can be specified in more detail), sights (can be specified in more detail), etc. )
B-3. How much of the facility is present in the facility-related image (select from standard, narrow, wide, maximum, etc.)
B-4. Whether or not to limit the display time of facility-related images, i.e., continuously displayed time to a predetermined time (for example, 15 seconds), and whether to limit the total display time to a predetermined time (for example, 3 minutes) Or B-5. Destination setting function (whether to enable destination setting by voice, or to enable destination setting by pointing)
In addition, the display color and the display size can be set for each facility-related image or for each type of facility-related image. For the display color, a plurality of candidate colors (black, red, blue, green, white, etc.) can be selected. Then, one color is selected from the candidate colors selected here in consideration of the background color and the like. Candidate colors may be selectable collectively according to the theme. For example, if there is a theme for each season and the driver selects the theme of “autumn”, colors that shine against red and yellow may be selected as candidate colors.

<Functional configuration of system>
FIG. 4 is a block diagram showing a functional configuration of the in-vehicle system according to the present embodiment. When the CPU 101 reads out and executes the program module stored in the ROM 102 in FIG. 3, the display control unit 401, the navigation unit 402, the setting unit 403, the eye point detection unit 404, the voice analysis unit 405, and the voice shown in FIG. Each function of the combining unit 406 is realized.

  The navigation unit 402 derives a guidance route and the like based on the vehicle data obtained from the various sensors 106 and passes it to the display control unit 401. The eye point detection unit 404 extracts the eye point position using the captured image from the eye point camera 120 and passes it to the display control unit 401. The voice analysis unit 405 analyzes the voice input from the microphone 140, and as a result, extracts the driver's instruction and passes it to the display control unit 401.

  The display control unit 401 analyzes the map data read from the storage unit 410, identifies a visually recognizable facility, and displays an image so that the facility-related image and the vehicle guidance image are displayed at a position corresponding to the eye point position. Generate and send to the display unit 110. At this time, with reference to the setting data read from the storage unit 410, the voice analysis result in the voice analysis unit 405, guidance information in the navigation unit 402, or the like, what kind of display mode the facility-related image is to be displayed or not It is determined whether to display, and what kind of voice is to be output to the voice synthesis unit 406.

  The navigation unit 402 functions as a vehicle guidance unit that guides the vehicle using the facility as an index.

  The setting unit 403 stores the instruction from the driver input from the operation unit 150 in the storage unit 410 while reflecting the instruction on the facility-related image. Further, the storage unit 410 updates the stored map data with data downloaded from the server through the communication unit 130.

<Processing of this system>
A rough process performed in this system will be described with reference to the flowchart of FIG.

  First, when the engine of the vehicle is started, the map data 104a and the setting data 104b are read from the HDD 104 in step S501.

  In step S502, a facility capable of displaying a facility-related image is extracted from the map data. The facilities extracted here are all the facilities existing in front of the vehicle regardless of whether or not they can be visually recognized by the driver. At this time, extraction may be limited to those within a predetermined distance (for example, 1000 km). Also, in that case, if it is a small facility such as a restaurant or a gas station, only those that are relatively close to each other are extracted, and if it is a very large facility such as a mountain or a high-rise building, it is up to a relatively far distance, or You may perform the process which extracts irrespective of distance. Further, in step S502, the map data is used to classify the facility into a facility that can be viewed by the driver and a facility that cannot be viewed. Here, whether or not a facility is visible is determined by whether or not there is another facility on the line connecting the facility and the driver's eyepoint in the vehicle. Whether it is possible or not can be determined. For example, you may determine using the camera mounted in the vehicle. If the type of facility that is not the display target of the facility-related image is set in the setting data 104b, it may be narrowed down to the type of facility set as the display target at this time.

  When the facility is extracted in step S502, in step S503, display control of the facility related image is performed, and image data to be transferred to the display unit 110 is generated. Of course, when it is determined by the display control processing here that no facility-related image is displayed, image data is not generated.

  Finally, in step S504, the image data generated by the display control process in step S503 is transferred to the display unit 110.

  In step S503, display control is basically performed in such a manner that a predetermined facility-related image is displayed when the facility is visible, and a predetermined facility-related image is not displayed when the facility is not visible. However, it is determined whether or not a predetermined condition is satisfied. If the predetermined condition is satisfied, the display mode of the predetermined facility-related image is changed. Specifically, the display mode is controlled as shown in the following 1-4.

1. Display position The position of the eye point is detected, and calculation is performed so that the facility-related image is displayed around the position where the facility is actually visually recognized. At that time, if there is an obstacle to the visual recognition of the facility between the vehicle and the facility, a position avoiding the obstacle is calculated.

2. Display method The presence / absence of a landmark display indicating a destination, a transit point, or a registered place is determined. The display color of the facility related image is calculated in consideration of the background. Calculate the size of the facility-related image.

3. Display prohibition condition It is determined whether or not the number of facility-related images is equal to or greater than a predetermined number, and a facility-related image set to be preferentially displayed is selected as a display target. For facility-related images with low priority, display is restricted (for example, prohibited). It is determined whether or not a condition prohibiting the display of the facility-related image (when the vehicle is turning, when there is a lot of traffic, or when the sign disappears when the facility-related image is displayed) is determined.

4). Display termination conditions Conditions for terminating the display of facility-related images (if they are not visible, if they are displayed for a specified number of times, if they are displayed continuously for a specified time or intermittently, they are located outside the area to be displayed. To determine whether or not this is the case.

<Detailed display control processing>
Here, the process performed as a part of the display control process of step S503 will be described in detail with reference to the flowchart of FIG. The flowchart of FIG. 6 shows in detail how to change the display mode of a predetermined facility-related image when what predetermined condition is satisfied in step S503. The series of processes shown in this flowchart is intended to improve the convenience of the driver and prevent the driver from feeling uncomfortable, in particular by preventing blinking of facility-related images.

  First, in step S601, it is determined whether the facility is visible. If the facility is not visible, the process proceeds to step S602, and it is determined whether a facility-related image is displayed. If the facility-related image is not displayed, the process proceeds to step S603, and a viewable time during which the facility that cannot be viewed is temporarily visible is predicted. Then, the process proceeds to step S604, and it is determined whether the predicted visible time is equal to or shorter than a predetermined time. If the predicted visible time is equal to or shorter than the predetermined time, it is determined that the predetermined condition is satisfied, and the process proceeds to step S605. Thereafter, the facility-related image is controlled even if the facility is visible.

  If the facility is visible in step S601, the process advances to step S606 to determine whether the facility-related image is normally displayed (not regulated). If the facility-related image is normally displayed, the process proceeds to step S607, and an unrecognizable time when the visually recognizable facility is temporarily unrecognizable is predicted. Then, the process proceeds to step S608, and it is determined whether or not the predicted invisible time is equal to or shorter than a predetermined time. If the predicted invisible time is equal to or shorter than the predetermined time, it is determined that the predetermined condition is satisfied, and the process proceeds to step S609. Thereafter, even if the facility becomes invisible, the display of the facility-related image is continued. To control.

  Next, if a facility-related image is displayed in step S602, it is determined that the display continuation processing has been performed in step S609, and the process proceeds to step S610, where the facility that cannot be viewed is visible. Predict the invisible duration until. In step S611, it is determined whether or not the predicted unrecognizable duration time is equal to or longer than a predetermined time. If the predicted unrecognizable duration time is equal to or longer than the predetermined time, the process proceeds to step S612 and the facility related image is displayed. Regulate the display of

  If it is determined that the display is not normal in step S606, it is considered that the display of the facility-related image is being restricted. Therefore, the process proceeds to step S613, and the visible continuation time until the visually observable facility becomes invisible is displayed. In step S614, it is determined whether or not the predicted visible continuation time is equal to or longer than the predetermined time. If the predicted visible continuation time is longer than the predetermined time, the process proceeds to step S615 and the facility-related image is displayed. Release display restrictions.

  In steps S603, S607, S610, and S613, the visible time or the invisible time is predicted based on the traveling state of the vehicle and the position of the vehicle on the map. In addition, the predetermined time that is the threshold value in steps S604, S608, S611, and S614 may be different depending on the position of the facility viewed from the vehicle.

  In addition, as a method for regulating the facility-related image in step S605 and step S612, for example, the facility-related image is hidden, but the present invention is not limited to this, and the facility-related image is not limited to this. For example, in addition to non-display, it may be possible to display inconspicuously, display small, or limit the display time. It should be noted that it is desirable to be able to cancel the restriction on the display of the facility-related image based on the driver's operation.

  Next, another process performed as a part of step S503 will be described with reference to FIG. In step S503, when a predetermined condition is satisfied, the display of a predetermined facility-related image for a visually recognizable facility is restricted, but the flowchart in FIG. It shows in detail how to restrict the display of a predetermined facility-related image for a visually recognizable facility. The series of processes shown in this flowchart is that, especially when the visual recognition becomes possible, displaying a predetermined facility-related image unconditionally may cause an obstacle to driving or make the driver uncomfortable. It is taken into consideration.

  First, in step S701, the safety of the vehicle is determined. If it is determined that the safety exceeding the predetermined limit is not ensured, the process proceeds to step S702 to restrict the display of the facility-related image. Here, the safety of the vehicle can be determined based on, for example, whether the traveling speed of the vehicle is a predetermined value or less, whether the vehicle is traveling on a highway, whether the vehicle is traveling on a curve with poor visibility, or the like. is there. Further, the display may be regulated step by step according to the speed of the vehicle. For example, the number of facility-related images displayed may be reduced as the vehicle speed increases.

  Next, if it is determined in step S701 that there is no problem with safety, then in step S703, the number of visually recognizable facilities is counted, and it is determined whether or not the displayed facility-related images are equal to or less than a predetermined number. If the number of facility-related images to be displayed exceeds a predetermined number, the process proceeds to step S704. If there are no safety issues and the number of all facility-related images displayed is less than or equal to the specified number, there will be no obstacles to driving or unpleasant driving without additional special regulations. Therefore, the process ends without adding restrictions in this flow.

  In step S704, one of a plurality of visible facilities is extracted. Then, in step S705, the distance from the vehicle is determined for the extracted visible facility. If the determined distance to the facility is greater than or equal to the predetermined value, the process proceeds to step S706. Note that in step S502 described with reference to FIG. 5, if only facilities within a predetermined distance are extracted, step S705 can be omitted.

  Next, in step S707, the position history of the driver's line of sight detected by the eye point camera 140 is read. If the facility is once determined to be positioned ahead of the driver's line of sight, the process proceeds to step S706.

  Next, in step S708, it is determined whether or not other facilities are close and visible. If it is determined from the driver that other facilities are visually recognized within a predetermined range, the process proceeds to step S709, and it is determined whether or not the display importance is higher than other facilities.

  The display importance level is set from the operation unit 160 or a personal computer at the driver's home based on an instruction from the driver, and is stored in the HDD 104 as a part of the setting data 104b. Of course, the display importance may be set for each type of facility by default without waiting for an instruction from the driver.

  If the display importance is lower than that of other facilities that can be viewed in close proximity, the process proceeds to step S706.

  In step S706, the display of facility-related images is restricted. However, facilities that are used as indicators for guiding the vehicle by the navigation unit 402 are not subject to the display regulation of facility-related images.

  In step S710, it is determined whether the processes in steps S705 to S709 have been completed for all visible facilities. If the determination has not been completed, the process returns to step S704. When the processes in steps S705 to S708 have been completed for all facilities, the process according to this flow is terminated. Before finishing the process, count the number of facilities that can be viewed again, and if the number of facility-related images to be displayed exceeds the specified number, the type of facility that has a lower display importance level. May be excluded from the display target of the facility-related image.

  The facility-related image display control method in step S706 is the same as in step S605, and is not limited to non-display.

  Next, still another process performed as part of step S503 will be described with reference to FIG. The flowchart in FIG. 8 shows in detail how to restrict the display of a predetermined facility-related image for a visually recognizable facility under what predetermined conditions are established. The series of processes shown in this flowchart, in particular, prevents the driver from repetitively displaying an image that has already been recognized by the driver and does not provide any new information. The purpose is not to make it uncomfortable.

  The flowchart in FIG. 8 is triggered by the fact that each facility-related image is displayed. When the facility-related image is displayed, the display amount of the facility-related image for the facility is accumulated in step S801. Here, the number of times or time when the facility-related image is displayed within a predetermined period (for example, one month) is accumulated, and the past total display time is also accumulated. In addition, the display amount is accumulated for each travel route and travel direction of the vehicle. That is, even if the same facility-related images are displayed in a state where the vehicle is located in different travel routes and travel directions, they are not added up.

  Next, in step S802, it is determined whether or not the accumulated display amount is a predetermined amount or more. If the display amount is greater than or equal to the predetermined amount, the process proceeds to step S803, a confirmation operation by the driver is requested, and if there is a confirmation operation, the process proceeds to step S804, and the facility related image is displayed for the visible facility. The setting is restricted, and the process ends.

  If the accumulated display amount is equal to or smaller than the predetermined amount in step S802, the process proceeds to step S805. In step S805, it is determined whether the road around the facility has been traveled a predetermined number of times (or a predetermined distance) or more. Such a travel history is stored in the HDD 104 by the navigation unit 402. If it is determined in step S805 that the road around the facility has been traveled a predetermined number of times (or a predetermined distance), the process proceeds to step S803, and if there is a confirmation operation by the driver, it can be visually recognized in step S804. The setting is made to restrict the display of the facility-related image for the facility, and the process ends. That is, even if the actual display amount does not exceed the predetermined value, it is determined that the display amount is greater than or equal to the predetermined amount, and the display of the facility-related image is restricted.

  Next, still another process performed as part of step S503 will be described with reference to FIG. In FIG. 4, the display control unit 401 has already described that the display unit 110 displays a simple image indicating the location of the facility and a detailed image explaining the attribute of the facility as the facility-related image. Under what conditions, only a simple image or a detailed image will be displayed. Displaying detailed descriptions such as facility names for all facilities that can be visually recognized may complicate the overall display, which may cause discomfort to the driver and hinder driving. is there. However, if the display of such an image is limited to one copy and the others are not displayed, the information desired by the driver may not be obtained. Therefore, by performing the processing according to the flowchart shown in FIG. 9 (a), the entire facility-related image is displayed clearly, discomfort to the driver is prevented, driving safety is ensured, and driving is performed. Provide desired information to the person.

  First, in step S901, a simple image display (only a simple image such as an arrow is displayed) is displayed for some of a plurality of visible facilities, or a detailed image display (a simple arrow or the like is displayed for all facility related images). Combination of detailed images such as images and comments).

  Here, the number of detailed images displayed in the unit area (for example, 10 cm × 10 cm) and in the entire image area is limited to a predetermined number. That is, when a facility-related image exceeding a predetermined number (for example, 2) is displayed in the unit area, or when a facility-related image exceeding a predetermined number (for example, 20) is displayed in the entire image area, Display a simple image. Among a plurality of facilities that can be viewed from a state where they cannot be viewed within a predetermined time (for example, 5 seconds), detailed images are displayed for a predetermined number of facilities (for example, 5), and the remaining facilities are simply displayed. Display an image. This is to limit the number of detailed images that are displayed almost simultaneously, and control is performed so as not to surprise the driver by displaying too many images at once.

  If it is determined in step S901 that only a simple image is to be displayed, the process advances to step S902 to determine which facility is to display only the simple image. That is, a facility for displaying a simple image is selected. Here, a facility that displays only a simple image is selected according to the type of facility and the distance from the vehicle to the facility. In particular, when the facility is separated from the vehicle by a first distance (for example, 10 km) or more, the facility-related image is not displayed and is within the first distance and the second distance (for example, 5 km) or more. Is a display target of only a simple image, and is a display target of a detailed image when it is within the second distance. In addition, when limiting the number of detailed images to a predetermined number or less, the detailed images are to be displayed in order from the one with the highest display importance. Only the target to be displayed. In step S903, only the simple image is displayed for the facility selected in step S902, and the detailed image is displayed for the rest.

  If the condition for displaying only the simple image is not applied in step S901, that is, if the facility-related images displayed are sparse and few, the process proceeds to step S904, and all the facility-related images are detailed. Display by combining images and simple images.

  Next, processing for displaying a detailed image of a facility corresponding to the designated simple image when the simple image is designated based on the driver's operation will be described using the flowchart of FIG. 9B. First, in step S911, it is determined whether or not the driver has issued an instruction to switch from simple image display to detailed image display. If there is a switching instruction, the process proceeds to step S912, and the driver determines whether simple images are selected and specified one by one or all displayed simple images are specified in a lump. In step S913, the designated simple image is switched to display the corresponding detailed image. As a method for the driver to designate a simple image, there are a method in which the driver touches the display portion of the simple image with the display of the display unit 110 as a touch panel, and a method for designating with a pointing device provided on the remote control. . It is also effective to provide a button for switching between non-display, simple image display, and detailed image display on the operation unit 150 such as a remote control or a control panel.

  Next, still another process performed as part of step S503 will be described with reference to FIG. In step S503, the facility-related image is displayed under a predetermined condition even if the facility is invisible. FIG. 10 is a flowchart showing in detail how the facility-related image is displayed for a facility that cannot be viewed under what conditions. If a facility-related image is not displayed for a facility that cannot be visually recognized, the driver cannot obtain desired information at a desired timing, which is inconvenient. Therefore, when processing according to the flowchart shown in FIG. 10 (a) is performed and an image relating to the facility is displayed superimposed on the actual scenery, it is displayed in a form convenient for the driver, Provide information about facilities more effectively.

  In step S1001, out of the facilities extracted in step S502 in FIG.

  Next, in step S1002, it is determined in the navigation unit 402 whether the facility extracted in step S1001 is an index such as a destination, a relay point, or a registered place. That is, it is determined whether or not the facility is a navigation related facility.

  If it is determined in step S1002 that the facility that cannot be visually recognized is an index of the navigation unit 402, the process proceeds to step S1003, and the facility is a display target of the facility-related image. At this time, if the facility-related image of the facility that can be visually recognized and the facility-related image of the facility that cannot be visually recognized are displayed in a distinguishable manner, the driver can see the facility-related image from the facility that cannot be visually recognized, that is, the driver. It can be easily recognized that it shows information on facilities that are not. For example, a facility-related image of a facility that cannot be visually recognized may be displayed in a light color, an arrow may be displayed in a broken line, or a character may be displayed in a different font. It is also effective to use only facility-related images of facilities that cannot be viewed as simple images (such as arrows) that indicate the direction of the facility. Furthermore, for a facility that cannot be seen, if the facility-related image display mode (eg, size) is changed according to the distance to the facility, how far the facility is not visible to the driver. Can be easily recognized. In addition, if the facility-related image is displayed in a manner that indicates that the facility exists behind the obstacle at a position avoiding the obstacle that hides the facility when the facility is invisible, The location of the facility indicated by the facility-related image can be easily recognized.

  If it is determined in step S1002 that the index is not the index of the navigation unit 402, the process proceeds to step S1004, and the facility is not set as a facility-related image display target with reference to the setting data 104b by the driver. If YES in step S1003, the flow advances to step S1003. Here, for each type of facility (genre), it is possible to set the display of facility-related images for facilities that are not visible. For example, for types of facilities with high display importance, A facility-related image is displayed, but a facility with a low display importance level can be set to hide the facility-related image if the facility-related image is not visible.

  For facilities that are not navigation-related facilities and are not subject to special display settings, the processing of this flow is terminated, and facility-related images are not displayed as usual.

  In addition, after a certain facility becomes visible and the display of the facility-related image starts, when the facility is hidden behind some other facility, the processing of FIG. 10B is performed. Do.

  That is, first, in step S1011, a timer included in the CPU 101 or the like is operated simultaneously with the start of display of facility-related images. Next, in step S1012, it is detected that the facility has become invisible, and the process proceeds to step S1013 to determine whether or not the time indicated by the timer is equal to or longer than a predetermined time. That is, the time from when the facility-related image is displayed until the facility becomes invisible is counted and compared with a predetermined time.

  If the predetermined time or more has elapsed, the process of this flow is terminated, and the process of FIG. That is, as a result of the processing of FIG. 10A, if the determination in step 102 and step S1004 is YES, the display of the facility-related image is continued as a result.

  If it is determined in step S1013 that the predetermined time or more has not elapsed, the process proceeds to step S1014, and the facility-related image display is continued until the predetermined time or more elapses. In other words, the facility-related image once displayed is continuously displayed so that the continuous display time is equal to or longer than the predetermined time even if the corresponding facility becomes invisible. Thereby, the time for the driver to recognize the facility-related image is secured, and the driver can obtain information about the facility more reliably.

  Next, still another process performed as part of step S503 will be described with reference to FIG. In step S503, the facility-related image is displayed in various display modes according to the type of facility. FIG. 11 is a flowchart showing in detail what kind of facility and how the facility-related image is displayed.

  First, in step S1101, it is determined to which type (genre) the facility extracted in step S502 of FIG. 5 belongs. The types of facilities include, for example, stations, convenience stores, gas stations, restaurants, department stores, event venues, amusement parks, etc. For each type, the maximum display distance L, the number N of display-permitted images, whether to display invisible Whether or not can be set. Here, the facility-related images are set not to be displayed for the types of facilities having a large number of displayed facility-related images.

  In step S1102, the maximum display distance L of that type of facility is read from the setting data 104b in the HDD 104 and compared with the distance from the vehicle to the facility. For example, 10 km for a gas station, 5 km for a station, and 1 km for a convenience store.

  If it is farther than the maximum display distance L, the process proceeds to step S1103, and the display of the facility-related image of the facility is restricted. The display regulation method here is also the same as that in step S605, and is not limited to non-display. Note that the display of facility-related images is not restricted for facilities that are used as indicators for vehicle guidance.

  If the facility is closer than the maximum display distance, the process proceeds to step S1104, and among the facilities belonging to that type, whether or not the number of display-permitted images is equal to or less than the Nth display permitted image is counted from the facility closest to the vehicle. judge. That is, the number of facilities of the same type as the facility to be determined is counted at a position closer to the vehicle than the facility, and it is determined whether the number is less than the number of display-permitted images. If the number of display-permitted images is Nth or less, this flow is terminated and the facility-related images are displayed as usual. If it exceeds the number N of display permission images, it will progress to step S1103 and will control the display of a facility related image.

  Next, still another process performed as part of step S503 will be described with reference to FIG. In step S503, the number of facility-related images whose display is started within a predetermined time is limited. FIG. 12 is a flowchart showing details of such processing. The process of FIG. 12 prevents a decrease in driving safety due to the simultaneous display of a large number of facility-related images. For example, immediately after turning a sharp curve or immediately after passing through a fence, if a completely new landscape spreads out at once, a large number of images are displayed in the driver's field of view at once, and the driver is surprised. Or obstruct the driver's view. Therefore, the number of images displayed at a time is limited to ensure driving safety.

  First, in step S1201, the number of facility-related images whose display is started within a predetermined time (for example, 5 seconds) is calculated. This specifies in advance facilities that can be viewed almost simultaneously from the 3D map data, the vehicle position, and the traveling direction, and counts the facilities that are to be displayed for facility-related images.

  Next, in step S1202, it is determined whether or not the number of facility-related images counted in step S1201 is a predetermined number (for example, 10) or more. If the number is less than the predetermined number, the process proceeds to step S1203, and display of all facility-related images is started as usual.

  If it is determined in step S1202 that the number is greater than or equal to the predetermined number, the process advances to step S1204 to select a facility-related image that restricts display start. Here, the selection is made according to the type of the corresponding facility. In addition, facilities that are used as indicators for vehicle travel guidance (such as destinations, registered locations, and relay locations) are not subject to display restrictions on facility-related images. In addition, the facility-related image having a low display importance is preferentially set as a display start restriction target. Further, the display of the facility-related image for which the time during which the visual recognition is impossible is longer than a predetermined time is restricted. That is, the facility-related image that has been displayed until immediately before is not displayed and is displayed as usual.

  In step S1205, the display start timing is delayed for the selected facility-related image. That is, the display of the selected facility-related image is started after a predetermined time has elapsed. At this time, for example, only a simple image may be displayed first, and then a detailed image may be displayed after a predetermined time has elapsed.

  In particular, among the plurality of selected facility-related images, the facility-related image predicted to be displayed later (for example, after 5 seconds) than the facility-related image predicted to be displayed earlier (for example, after 1 second). Delay the start of image display. At this time, the images are displayed in order from the facility-related images of the facilities with a short distance. Here, the driver may be notified that the display start timing has been shifted.

(Effect of embodiment)
According to this embodiment, since the image regarding the facility is displayed so as to be superimposed on the actual scenery, the driver can acquire the facility information with a small amount of line-of-sight movement while driving, and the safety can be improved. Can be planned. In addition, since the number of facility-related images that are displayed within a predetermined time is limited, safety can be ensured so as not to hinder driving, and discomfort to the driver can be prevented.

<Other embodiments>
In the above-described embodiment, the above-described effect is achieved by the CPU 101 described above reading out and executing a program capable of realizing each processing described with reference to FIGS. 6 to 12 from some storage medium. As the medium, any storage device such as a readable / writable semiconductor memory and a hard disk as well as an optical disk, a magnetic disk, and a magneto-optical disk can be adopted, and such a storage medium and a program itself are also included in the scope of the present invention. It is. Therefore, data transmitted through a telecommunication line by using the Internet or the like can be included in the scope of the present invention. Moreover, the vehicle image display apparatus according to the present invention may be a part of the in-vehicle system as shown in the above embodiment, or may be a single dedicated unit.

1 is a diagram showing an overall configuration of a vehicle equipped with an in-vehicle system according to an embodiment of the present invention. It is a figure which shows an example of the image displayed by the vehicle-mounted system which concerns on embodiment of this invention. It is a block diagram which shows the hardware constitutions of the vehicle-mounted system which concerns on embodiment of this invention. It is a figure which shows an example of the setting screen displayed by the vehicle-mounted system which concerns on embodiment of this invention. It is a figure which shows an example of the setting screen displayed by the vehicle-mounted system which concerns on embodiment of this invention. It is a block diagram which shows the function structure of the vehicle-mounted system which concerns on embodiment of this invention. It is a flowchart for demonstrating briefly operation | movement of the vehicle-mounted system which concerns on embodiment of this invention. It is a flowchart for demonstrating in detail the operation | movement of the vehicle-mounted system which concerns on embodiment of this invention. It is a flowchart for demonstrating in detail the operation | movement of the vehicle-mounted system which concerns on embodiment of this invention. It is a flowchart for demonstrating in detail the operation | movement of the vehicle-mounted system which concerns on embodiment of this invention. It is a flowchart for demonstrating in detail the operation | movement of the vehicle-mounted system which concerns on embodiment of this invention. It is a flowchart for demonstrating in detail the operation | movement of the vehicle-mounted system which concerns on embodiment of this invention. It is a flowchart for demonstrating in detail the operation | movement of the vehicle-mounted system which concerns on embodiment of this invention. It is a flowchart for demonstrating in detail the operation | movement of the vehicle-mounted system which concerns on embodiment of this invention.

Claims (9)

In order to provide information on a facility existing in front of the vehicle, an image display device for a vehicle that displays a facility-related image related to the facility so as to be superimposed on an actual scenery that is visible from the driver of the vehicle through a window. Because
When the facility is visible, a predetermined facility-related image is displayed, and when the facility is invisible, the display control means performs display control in a display mode in which the predetermined facility-related image is not displayed.
The display control means limits the number of facility-related images whose display is started within a predetermined time.   2. The vehicle image display according to claim 1, wherein the display control unit determines whether to start displaying the facility-related image within the predetermined time according to a type of a corresponding facility. apparatus. Vehicle guidance means for guiding the vehicle with the facility as an index;
2. The vehicle according to claim 1, wherein the display control unit excludes the facility that is used as an index of guidance of the vehicle by the vehicle guide unit from being subject to display restriction of the facility-related image. Image display device. The display control means starts display within the predetermined time by delaying the display start timing of the facility-related image for a part of the plurality of facility-related images whose display is started within the predetermined time. The vehicle image display device according to claim 1, wherein the number of facility-related images is limited.   The display control means further delays the display start of the facility-related image predicted to be displayed late among the plurality of facility-related images predicted to be displayed within the predetermined time. The vehicle image display device according to claim 2. The display control means includes
Claims, characterized in that the facility related image displayed within the predetermined time is started when it is estimated that more than a predetermined number, and displays in order from distances near facility of the facilities related images from the vehicle The vehicle image display device according to 1.   The display control means regulates the display of the facility-related images whose time that is not visible among the plurality of facility-related images whose display is started within the predetermined time is a predetermined value or more. The vehicle image display device according to claim 1. A vehicle image display method for displaying a facility-related image related to the facility so as to be superimposed on an actual scenery that can be viewed through a window from a driver of the vehicle in order to provide information regarding the facility existing in front of the vehicle. Because
When the facility is visible, a predetermined facility-related image is displayed, and when the facility is not visible, the display control step performs display control in a display mode in which the predetermined facility-related image is not displayed.
The display control step limits the number of facility-related images whose display is started within a predetermined time. In order to provide information on a facility existing in front of the vehicle, an image display device for a vehicle that displays a facility-related image related to the facility so as to be superimposed on an actual scenery that is visible from the driver of the vehicle through a window. In addition,
A vehicle for executing a display control step of performing display control in a display mode in which a predetermined facility-related image is displayed when the facility is visible and a predetermined facility-related image is not displayed when the facility is not visible. An image display program for
The display control step limits the number of facility-related images whose display is started within a predetermined time.

Images