JP2010058631A - Vehicular information display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a vehicular information display device capable of visually providing various kinds of information through an HMD (Head Mount Display), and preventing fatigue of eyes. <P>SOLUTION: When an occupant holding a portable terminal M wears an HMD 30, ID is acquired from the portable terminal M, the occupant is indirectly specified by the ID, and the light quantity passing through a lens 32 (a liquid crystal panel 40) of the HMD 30 is adjusted based on the target illuminance prepared in advance in a memory of the portable terminal M so that the light quantity corresponding to the occupant related to the ID enters eyes E. The target illuminance is adjusted based on these pieces of information by incorporating the attributes such as the pupil color, the sex, and the age included in a memory 92 of the portable terminal M, and then, the light transmittance of the liquid crystal panel 40 of the HMD 30 is controlled based on the target illuminance after the adjustment. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a vehicle information display device.

  Driving support information provided by a navigation device or the like is greatly useful for improving convenience during driving. On the other hand, how to display such driving support information on which part of the vehicle is a problem from the viewpoint of safety in vehicle travel.

  For example, a technique for projecting and displaying driving support information on a front window has been proposed. However, the driver constantly moves the face direction and line of sight while driving, and the line of sight cannot be fixed on the information display part of the front window.

  On the other hand, Patent Document 1 displays driving support information on a head-mounted display device (MHD: head mounted display) so that the driving support information can be provided to the driver regardless of the head direction of the driver. Have been disclosed. Regarding HMD, Patent Document 1 discloses a compound eye type (goggles type) HMD that covers both eyes of a driver, and this HMD has left and right lenses formed of a color display liquid crystal panel (see FIG. 1). 2).

  Similar to Patent Document 1, Patent Document 2 also discloses a goggle-type HMD, and the environment around the occupant can be seen through the left and right lenses of the HMD, and the external situation of the vehicle is displayed as a stereoscopic image in front of the driver's eyes. Propose to do.

JP-A-7-294842 JP-A-10-206789

  The compound eye type HMD receives the light from the outside with the left and right eyes of the occupant wearing it, so if the illuminance of the light from the outside is large, it may feel dazzling and eventually induce eye fatigue. is there.

  An object of the present invention is to provide an information display device capable of not only visually providing various information through an HMD but also preventing eye fatigue.

According to the present invention, the above technical problem is
Vehicle information display apparatus including an HMD that includes left and right lenses whose light transmittance is controllable, and that can see the surrounding environment through the left and right lenses and can display the external situation of the vehicle as a stereoscopic image in front of the eyes Because
An illuminance sensor for detecting the illuminance of light incident on a passenger wearing the HMD;
Pupil opening degree detecting means for detecting an opening degree of a passenger's pupil wearing the HMD;
Light transmittance control means for controlling the light transmittance of the left and right lenses based on the opening degree of the occupant's pupil detected by the pupil opening degree detection means and the illuminance of the light detected by the illuminance sensor;
This is achieved by providing a vehicle information display device comprising communication network means for transmitting / receiving data to / from the illuminance sensor, the pupil opening degree detection means, and the light transmittance control means.

  That is, according to the present invention, the light transmittance of the left and right lenses is adjusted by the illuminance of the light incident on the occupant and the opening of the occupant's pupil. You can reduce fatigue.

In a preferred embodiment of the present invention,
Storage means constituting part of the communication network means;
In the storage means, the target illuminance is stored in advance for each occupant,
The light transmittance control means is controlled based on the target illuminance stored in the storage means. According to this embodiment, the illuminance of light entering the eyes of the occupant can be optimized at an early stage when the occupant wears the HMD.

In a preferred embodiment of the present invention,
The illuminance sensor is constituted by a sensor node that forms part of the communication network means and is attached to the HMD, and attribute data that can indirectly identify an occupant in the memory of the sensor node; The target illuminance data of the passenger is stored in association with the attribute data. Thus, by providing the sensor node attached to the HMD with the function of the illuminance sensor, it becomes easy not only to detect the illuminance of light incident on the occupant, but also to set the illuminance target illuminance to the sensor node of the HMD. By storing it, the association between the occupant and the target illuminance becomes clear, and the possibility that the target illuminance of another occupant is set can be reduced.

In a preferred embodiment of the present invention,
A portable terminal that can be incorporated into the communication network means;
In relation to the ID of the portable terminal, data on the target illuminance of the occupant who owns the portable terminal is stored in the memory of the portable terminal.

  The mobile terminal is typically a mobile phone. The mobile phone stores a unique ID, and the mobile phone can be specified by using this ID. Since it is normal that the person who possesses the mobile phone is specified, it is possible to specify the occupant who owns the mobile phone via the ID of the mobile phone. Further, by storing the target illuminance of the occupant in the memory of the mobile phone, and controlling the light transmittance of the lens of the HMD based on the target illuminance, the occupant carrying the mobile phone and the occupant's The target illuminance can be set without making a mistake with the target illuminance.

  In a preferred embodiment of the present invention, a plurality of sensor nodes constituting the communication network means have a detection function that can identify an occupant wearing an HMD. Specifically, the function of the pressure sensor is given to the sensor node installed on the seat, and the occupant can be identified from the detected weight by detecting the weight of the occupant seated on the seat.

In a preferred embodiment of the present invention,
The target illuminance is adjusted based on the occupant's pupil opening detected by the pupil opening detection means and the illuminance of light from the illuminance sensor. For example, a driver usually drives while changing his / her eyes and face direction. By adjusting the target illuminance in response to this change, the light transmittance of the HMD lens can be made to follow the situation change.

In a preferred embodiment of the present invention,
Attributes relating to the eyes of the occupant are stored in the memory of the portable terminal,
After taking in the attribute of the occupant and adjusting the target illuminance based on the attribute, control of the light transmittance of the lens is executed. The attributes relating to the eyes of the occupant can include the color, sex, and age of the pupil. By adjusting the target illuminance according to these factors, the target illuminance corresponding to individual differences can be set.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. In addition, this invention is not limited to the following embodiment, It shows only the specific example advantageous for implementation of this invention. In addition, not all combinations of features described in the following embodiments are indispensable as means for solving the problems of the present invention.

  FIG. 1 is a diagram showing a basic configuration of an information display apparatus according to this embodiment.

  In FIG. 1, reference numeral 10 denotes a control unit, which is typically configured by a computer system including a CPU, RAM, ROM, etc. (not shown), and the processing shown by each block in the illustrated control unit 10 is executed by the CPU. Is done.

  This information display device is, for example, a first front photographing camera 14 provided on the shoulder of the backrest 12 of the driver's seat toward the front of the vehicle, for example, a second front photographing camera provided on the upper portion of the instrument panel toward the front of the vehicle. 16, for example, a front blind spot photographing camera 18 which is provided at the lower part of the front door and obliquely forward on the outer side of the vehicle and photographs a blind spot ahead of the vehicle, for example, a rear blind spot photographing camera 20 which is provided at the upper part of the rear dashboard and directed toward the rear of the vehicle. These are connected to the camera image acquisition unit 22 in the control unit 10. These cameras 14 to 20 constitute image information acquisition means for acquiring image information around the own vehicle, and preferably function as an omnidirectional camera that obtains an image around the entire vehicle. Moreover, it is preferable that each camera is provided with a super-wide-angle lens in order to obtain a wider range of images. In addition, it is desirable that each of the second front photographing camera 16, the front blind spot photographing camera 18, and the rear blind spot photographing camera 20 is an infrared camera (a night vision camera) so that night photography can be performed. Furthermore, an illuminance sensor 24 that detects the illuminance of light incident on the face of the driver D is provided, and the output of the illuminance sensor 24 is also supplied to the camera image acquisition unit 22.

  Reference numeral 30 denotes an HMD (head-mounted display) as viewing angle assisting means worn by the driver D. The HMD 30 is an eyeglass-type HMD as shown in FIG. 2, and the left and right lenses 32 of the eyeglass-type HMD 30 are configured by a light transmissive liquid crystal color monitor. The occupant wearing the HMD 30 can see the environment around the occupant through the left and right lenses 32, and can see the external situation as a stereoscopic image in front of the eyes as will be described later. Of course, the HMD 30 may be a goggle type as disclosed in Patent Document 1.

  FIG. 3 is an exploded perspective view of the lens 32 of the HMD 30. The lens 32 includes a pair of protective panels 34 and 36, a color image display panel 38 and a liquid crystal panel 40 positioned between the protective panels 34 and 36. The color image display panel 38 is a red panel 38a. And a green panel 38b and a blue panel 38c.

  The color image display panel 38 constituting a part of the lens 32 is operated in cooperation with each other by the red, green, and blue panels 38a to 38c operating in accordance with the drive signal supplied from the display drive circuit 42. Display an image. Further, the light transmittance of the liquid crystal panel 40 decreases as the voltage of the drive signal supplied from the liquid crystal drive circuit 44 increases. In other words, when the voltage of the drive signal supplied from the liquid crystal drive circuit 44 becomes zero V, the light transmittance is maximized.

  A position sensor 50 is provided at a predetermined position in the passenger compartment (for example, a roof trim above the driver's seat). The position sensor 50 is composed of, for example, a magnetic sensor, and is configured to output a signal indicating the position and orientation of the HMD 30. The signal indicating the position and orientation includes parameters of position (x, y, z) and orientation (roll, pitch, yaw) with respect to the position sensor 50. A signal indicating the position and orientation of the HMD 30 output from the position sensor 50 is input to the line-of-sight direction estimation unit 52 in the control unit 10.

  The controller 10 holds the positional relationship of the first and second front photographing cameras 14, 16, the front blind spot photographing camera 18, and the rear blind spot photographing camera 20 with respect to the position of the position sensor 50 in advance by camera calibration. Therefore, the control unit 10 obtains the positional relationship between the first and second front photographing cameras 14 and 16, the front blind spot photographing camera 18, the rear blind spot photographing camera 20, and the HMD 30 based on the signal input from the position sensor 50. Is possible.

  In front of the driver D, a face image photographing camera 54 that photographs a face image including the pupil of the driver D wearing the HMD 30 is installed. The driver's face image and pupil opening degree data captured by the face image capturing camera 54 are supplied to the line-of-sight direction estimation unit 52 in the control unit 10. The installation position of the face image photographing camera 54 is arbitrary, but it may be attached to a dashboard of a vehicle, for example. The connection between the control unit 10 and the HMD 30 may be wired or wireless.

  The information display device according to the embodiment further includes various types of sensors such as a navigation unit 60 that receives GPS signals and acquires travel information of the vehicle such as route information and traffic information (navigation information) based on map data, and a vehicle speed sensor. Is included.

  The control unit 10 includes, in addition to the camera image acquisition unit 22 and the line-of-sight direction estimation unit 52, a lens distortion correction unit 64 that corrects lens distortion (aberration) of the acquired image, a geometric conversion / image cutting that performs image geometric transformation and image clipping. Navigation information based on data input from a clipping unit 66, a drawing position / size calculating unit 68 that calculates a drawing position and size of an image, a camera image processing unit 70 that processes a camera image, a navigation unit 60, and a vehicle state sensor 62 And the like, an information image creating unit 72 for creating an information image, and an image composition unit 74 are provided. The navigation unit 60 and the vehicle state sensor 62 are connected to an information image creation unit (2D image creation unit) 72 of the control unit 10.

  FIG. 4 is a flowchart showing image display control processing of the HMD 30 by the control unit 10 in the present embodiment.

  The camera image acquisition unit 11 receives image data from the first and second front photographing cameras 14 and 16, the front blind spot photographing camera 18, and the rear blind spot photographing camera 20, and sequentially receives output data from the illuminance sensor 24. (Step S1). In parallel with this, the line-of-sight direction estimation unit 52 receives a signal indicating the position and orientation of the HMD 30 from the position sensor 50 and face image data from the face image photographing camera 54 (step S2). The line-of-sight direction estimation unit 52 estimates the driver's face direction from the signal indicating the position and orientation of the HMD 30 from the position sensor 50, and based on the face image data from the face image capturing camera 54, The image area is separated from the pupil area, and the line-of-sight direction with respect to the face direction is estimated from the result. Then, from these results, the line-of-sight direction with respect to the reference position sensor 50 is estimated.

  In step S1, for example, when the estimated line-of-sight direction is determined to be in front of the vehicle, and the illuminance is determined to be brighter than a predetermined value based on the received data from the illuminance sensor 24, the first front photographing camera 14 and the front Image data of the blind spot photographing camera 18 is acquired. If the estimated line-of-sight direction is determined to be in front of the vehicle and the illuminance is determined to be dark below the predetermined value, the image data of the second front photographing camera (infrared camera) 16 is acquired. Further, when it is determined that the estimated line-of-sight direction is the rear of the vehicle, the image data of the rear blind spot photographing camera 20 is acquired.

  Next, the lens distortion correction unit 64 corrects distortion (aberration) that occurs depending on the lens characteristics of the camera that acquired the image data for the image data acquired in step S1 (step S3). This correction is performed based on camera internal parameters indicating the position of the camera that acquired the image data with respect to the position of the position sensor 50 obtained by camera calibration, for example.

  On the other hand, the line-of-sight direction estimation unit 52 obtains the position / orientation data of the HMD 30 based on the signal indicating the position / orientation of the HMD 30 from the position sensor 50 received in step S2 by a position calibration performed in advance. Correction is performed based on the HMD correction parameter (step S4).

  Next, the geometric conversion / cutout unit 66 performs cutout and geometric conversion of the image in accordance with the display area of the HMD 30 on the image data after the lens distortion correction obtained in step S3 (step S5). Specifically, for example, from the image data after lens distortion obtained in step S3, the HMD 30 has the same direction and size as the three-dimensional real space image S viewed from the driver's viewpoint. A necessary image region is cut out in accordance with the position and orientation with respect to the position sensor 50 and the line-of-sight direction estimated by the line-of-sight direction estimation unit 52, and affine transformation is performed.

  Next, as a countermeasure against glare, the camera image processing unit 70 recognizes a high-luminance object, performs spectral image processing, smoothes the luminance distribution, Spectral correction processing is performed to adjust the light so that there is no discomfort (step S6). Note that glare increases the degree of light scattering in the eyeball, which makes it difficult to see the object, and unpleasant glare that makes it easier to feel uncomfortable when there is a bright object in the field of view. Both generally increase with aging. The spectral correction process in step S6 deals with such glare.

  FIG. 5 is a flowchart showing an example of the spectral correction process in step S6. First, luminance distribution analysis is performed on the image data processed in step S5 (step S61). Then, it is determined whether or not a high-luminance object exists based on the obtained luminance distribution (step S62). If a high-luminance object exists, the luminance distribution is smoothed (step S63).

  Next, the camera image processing unit 70 performs follow-up delay correction processing (step S7). This is to cope with the slowing of eye movement when chasing a moving object with aging, and a specific processing example is shown in the flowchart of FIG.

  First, a moving object is extracted from the input moving image data (step S71), and it is determined whether there is a high-speed moving object (step S72). If a high-speed moving object is present, the moving direction, moving speed, and position of the moving object are detected (step S73), and the predicted position of the moving object is calculated based on the detection result (step S73). S74). Thereafter, the moving object is drawn at the predicted position (step S75).

  Next, the camera image processing unit 70 executes a dangerous object detection process (step S8). A specific example of this process is shown in the flowchart of FIG.

  First, objects such as pedestrians and other vehicles are extracted from the input image data (step S81), and an arrangement pattern of the extracted objects is extracted (step S82). Next, the gaze behavior model of the skilled driver D stored in the memory in advance is read (step S83). FIG. 8 shows a conceptual diagram of the gaze behavior model of the skilled driver D. Then, a neural network or reinforcement learning based on the gaze behavior model of the skilled driver D is configured, and using this, a high risk object is extracted from the object arrangement pattern extracted in step S82 (step S84). .

  When the high risk object is extracted, the image is displayed on the lens 32 of the HMD 30 as a camera image. At this time, an identification mark such as a surrounding frame line may be superimposed and displayed so that the driver can easily recognize the image as a high risk object.

  Next, the drawing position / size calculation unit 68 calculates the drawing position and the size of the 2D image displayed on the lens 32 of the HMD 30 (step S9). In addition, the information image creation unit 72 creates an information image (2D image) such as navigation information created based on data input from the navigation unit 60 and the vehicle state sensor 50. At this time, the information image creation unit 72 may include an arrow image or the like indicating the moving direction of the high-speed moving object detected in the follow-up delay correction process in step S7 in the information image.

Next, the image composition unit 56 determines whether to display the camera image on the lens 32 of the HMD 30 (step S10). For example, in at least one of the following cases, it is determined that the degree of danger in traveling the vehicle is relatively high, and the camera image is displayed on the lens 32 of the HMD 30.
(1) When the luminance distribution smoothing in step S63 is executed in the spectral correction process in step S6.
(2) When the processes in steps S73 to S75 are executed in the follow-up delay correction process in step S7.
(3) When there is a blind spot in the driver's line of sight.
(4) When the illuminance is dark.

  When it is determined that the camera image is displayed on the lens 32 of the HMD 30 in step S10, the image composition unit 56, based on the calculation result in step S9, the camera image obtained in the above-described steps S5 to S8, The information image creation unit 72 creates an image obtained by synthesizing the information image (2D image) such as navigation information created based on the data input from the navigation unit 60 and the vehicle state sensor 50 (step S11), and the synthesized image is displayed. The image is displayed on the lens 32 of the HMD 30 (step S12).

  On the other hand, if it is determined in step S10 that the camera image is not displayed on the lens 32 of the HMD 30, the image composition unit 56 creates the information image creation unit 72 based on data input from the navigation unit 60 and the vehicle state sensor 50. Only an information image (2D image) such as navigation information is drawn (step S13) and displayed on the lens 32 of the HMD 30 (step S12).

  An example of an image visually recognized by the driver by the image display control of the HMD 30 by the control unit 10 described above is shown in FIG.

  In the image shown in FIG. 9 (a), the sky in fine weather is greatly spread in front of the vehicle, and unpleasant glare is likely to occur. On the other hand, in the present embodiment, since the spectral correction process is performed in step S6 and the luminance distribution is smoothed, the glare is reduced. Further, in FIG. 9A, the bus 80 is moving so as to cross from the left front of the host vehicle. In contrast, in this embodiment, follow-up delay correction is performed in step S7, and the position of the bus 80, which is a moving object, is corrected. At this time, an arrow mark 81 indicating the moving direction may be superimposed and displayed as shown in the figure.

  In FIG. 9B, a character display “100 m ahead right turn” as navigation information and a right turn mark 82 with an arrow indicating that are superimposed. FIG. 9C shows that the preceding vehicle is determined to be a dangerous object in the dangerous object extraction process in step S8, and a surrounding frame line is superimposed on it in order to call the driver's attention. The message “Goods” and “Securing distance” are displayed. If the state shown in FIG. 9C is not resolved even after a predetermined time has passed, a voice warning message may be issued as shown in FIG. 9D.

  Here, it is preferable that the right turn mark 82 as the navigation information as shown in FIGS. 9B to 9D is displayed in a position and manner suitable for the line-of-sight direction of the driver. On the other hand, as shown in FIG. 9 (d), the traffic information in the navigation information such as “5KM forward traffic jam 1KM” is fixed at the lower end as shown in the figure regardless of the driver's line-of-sight direction. It may be displayed at the position.

  Although the driving support for the driver D has been described above, the glasses-type HMD 30 is not limited to the driver D, and is convenient for a passenger in the front passenger seat or the rear seat to wear it. When an occupant seated on a seat other than the driver's seat wears the HMD 30, for example, a movie or guidance on the surrounding environment is displayed via the HMD 30.

  10 and 11 are overall configuration diagrams of a network mounted on a vehicle, FIG. 12 is a block diagram showing sensor nodes, FIG. 13 is a block diagram showing gateway points, and FIG. FIG. 15 is a block diagram showing a mobile terminal, FIG. 15 is a diagram schematically showing devices controlled by a gateway point while visualizing network information between a sensor node and a gateway. FIG. It is the figure which represented typically the apparatus controlled by a gateway point while visualizing the network information between a node and a gateway.

  First, as shown in FIGS. 10 and 11, a plurality of sensor nodes SN and gateway points G are mounted on the vehicle. The HMD 30 described above also incorporates a pair of left and right sensor nodes SN. In this embodiment, the gateway point G includes a CPU, a memory, and a wireless communication interface built in the navigation device. As shown in FIG. 10, a device D (for example, an air conditioner having a plurality of air outlets) is connected to the gateway point G, and the temperature of air sent from each air outlet by a device control unit (to be described later) of the gateway point G. And the amount is controlled.

  The sensor node SN is a coin-sized size and relatively thin, and can be attached to various parts in the passenger compartment. In the present embodiment, as shown in FIG. 11, a plurality of seats are provided in the seats of each seat or in the roof portion corresponding to each seat. As shown in FIG. 11, the sensor nodes SN can be arranged three-dimensionally. The sensor nodes SN and the sensor nodes SN incorporated in the HMD 30 form a network (see FIG. 15) wirelessly.

  The gateway point G receives information by wireless communication from each sensor node SN, limits the operation of the navigation function and the TV function of the in-vehicle device, for example, the navigation device, and adjusts the cooling / heating according to each seat of the air conditioner (See FIG. 15). The gateway point G transmits and receives information (signals) to and from the sensor node SN, and also calculates, for example, the target temperature of each seat and the control amount of the air conditioner corresponding to each seat. This gateway point G also forms a network (see FIG. 15) wirelessly with the plurality of sensor nodes SN described above.

  As shown in FIG. 11, each sensor node SN has a power source such as a button battery, a processor as a CPU, and various sensors attached to the outside of the sensor node SN. For example, the illuminance sensor 24 described above is incorporated in the sensor node SN incorporated in the HMD 30. The sensor node SN mounted on the vehicle includes an illuminance sensor, a temperature sensor, a pressure sensor, and the like, and the pressure sensor functions as a seating sensor that detects the seating of an occupant. Further, the sensor node SN has an interface capable of wireless communication with other sensor nodes SN, a portable terminal M such as a cellular phone, and a gateway point G described later.

  In addition, a memory is mounted on the sensor node SN, and a program for processing information on the pressure sensor and inputting / outputting signals to / from other portable terminals M and other sensor nodes SN is stored in the memory. The software with built-in is stored. The software includes operating software and application software. The operating software includes an operating system, and the application software includes a program module. Further, application data is stored in the memory.

  The memory of the sensor node SN stores the detection result of the sensor of the sensor node SN. The processor (data processing unit) of the sensor node SN calculates a target control amount for controlling the device D based on the detection result by the sensor. This calculation may be performed using the detection result by the sensor as it is, or by using the detection result stored in the memory of the sensor node SN or the memory of the gateway point described later. Further, the processor of the sensor node SN can calculate a target control amount for controlling the device D based on the detection result stored in the predetermined database. The predetermined database is provided in the memory of the sensor node SN or the memory of the gateway point.

  The calculated target control amount is used by the processor and device control unit of the gateway point G, and the device control unit of the gateway point G controls the device D so that the target control amount is obtained. Further, the calculated target control amount is stored in the memory of the sensor node SN or the gateway point memory described later, and the processor and the device control unit of the gateway point G use the stored target control amount, The device D may be controlled so that the target control amount is obtained by the device control unit.

  As such a communication method of the sensor node SN, for example, there is one called “ZigBee”. The standard is “IEEE802.15.4”, the transmission speed (bps) is “250K”, the frequency band used is “2.4GHz (worldwide), 868MHz (Europe), 915MHz (US)”, and the transmission distance is “up to 10”. -75m "and power consumption (communication) is" <60mW ". In addition to this, there are other methods such as “weak wireless”, “specific low power wireless”, “Bluetooth”, and “UWB”.

  As illustrated in FIG. 13, the gateway point G includes a power source drawn from the battery of the vehicle, a processor as a CPU, an interface capable of wireless communication with a plurality of sensor nodes SN, a memory, and a device control unit. This memory stores software in which a program for inputting / outputting signals to / from the sensor node SN and the portable terminal M is built. The software includes operating software and application software. The operating software includes an operating system, and the application software includes a program module. Further, application data is stored in the memory.

  The gateway point G centrally manages information obtained from each sensor node SN and its network and controls external devices such as an air conditioner, a navigation device, an electric seat device, and a power window. It is also possible to store information in a server outside the vehicle through the gateway point G and the portable terminal M. The gateway point G stores information such as the ID of the mobile terminal M. Such information may be stored in each sensor node SN.

  The memory 88 of the gateway point G stores the detection result by the sensor of the sensor node SN. The processor (data processing unit) of the gateway point G calculates a target control amount for controlling the device D based on the detection result of the sensor of the sensor node SN. This calculation may be performed using the detection result of the sensor as it is, or is performed using the detection result stored in the memory 90 of the sensor node SN or the memory 88 of the gateway point. The processor of the gateway point G controls the device D based on the detection result stored in the predetermined database described above, that is, the database provided in the memory 90 of the sensor node SN or the memory 88 of the gateway point. It is also possible to calculate a target control amount for this purpose.

  The calculated target control amount is used by the processor and device control unit of the gateway point G, and the device control unit of the gateway point G controls the device D so that the target control amount is obtained. The target control amount stored in the memory of the sensor node SN or the gateway point memory 88 described later is used by the processor and device control unit of the gateway point G, and the target control amount is obtained by the device control unit. The device D may be controlled as described above.

  As shown in FIG. 14, the mobile terminal M, which is typically a mobile phone, includes mobile phone base middleware, a processor that is a CPU, an interface and a memory that can wirelessly communicate with a plurality of sensor nodes SN. This memory stores software in which a program for inputting / outputting signals to / from the sensor node SN is incorporated. The software includes operating software and application software. The operating software includes an operating system, and the application software includes a program module. Further, application data is stored in the memory. This mobile terminal M also forms a network (see FIG. 16) wirelessly with the plurality of sensor nodes SN described above.

  Next, the concept of the sensor network will be described. As described above, the sensor node SN includes a memory, an application, a wireless function, and the like in addition to a sensor that performs predetermined sensing. The sensor node SN can hold information obtained by the sensor and can transmit and receive wirelessly to / from another sensor node SN and the portable terminal M. With such a function, a certain sensor node SN can obtain information on other sensor nodes SN and mobile terminals M, that is, information obtained by sensors included in other sensor nodes SN and stored information. I can do it. In this way, a plurality of sensor nodes SN and mobile terminals M are connected by a network (see FIG. 15), and even if one sensor node SN is out of order or the mobile terminal M is not present, A sensor node SN can form a network. This makes it possible to form a network that does not require wiring, unlike in-vehicle LANs that are connected by wiring. In addition, the information of the sensor node SN at a certain position can be obtained in a multifaceted manner together with the information of other sensor nodes SN.

  Returning to FIG. 1, information from the illuminance sensor 24 configured by the sensor node SN installed in the HMD 30 and the face image photographing camera 54 including the pupil of the driver D is input to the liquid crystal drive circuit 44. A pupil opening degree detection processing circuit 86 is interposed between the facial image photographing camera 54 and the liquid crystal driving circuit 44, and the pupil opening degree detection processing circuit 86 detects the opening degree of the pupil of the driver D. The opening degree information is input to the liquid crystal drive circuit 44.

  FIG. 17 schematically shows a network incorporating a mobile terminal M, typically a mobile phone, possessed by an occupant wearing the HMD 30. In the gateway point G, information of a person who has boarded the vehicle in the past is stored. This information includes the weight of each person and the target illuminance suitable for the eyes of the individual. Further, when the person who uses the HMD 30 is limited to, for example, a driver or a family, the “weight-target illuminance” data of each person is stored in the memory 90 (FIG. 17) of the sensor node SN of the HMD 30. Also good. Further, the mobile terminal M has its memory 92 (FIG. 4) stored in its memory 92 ID and user ID, personal characteristics data of the user of the mobile terminal M (weight, pupil color, target illuminance, Age, average blood pressure, maximum blood pressure, minimum blood pressure, average pulse, etc.) are stored in advance.

  When the occupant carrying the portable terminal M wears the HMD 30, the ID is obtained from the portable terminal M, the occupant is indirectly identified by this ID, and the light quantity corresponding to the occupant associated with this ID is in the eye E. The amount of light passing through the lens 32 (liquid crystal panel 40) of the HMD 30 is adjusted based on the target illuminance prepared in advance. This target illuminance takes in attributes such as the color, sex, and age of the pupil included in the memory of the mobile terminal M, and after adjusting based on these information, the target illuminance of the liquid crystal panel 40 of the HMD 30 is adjusted based on the adjusted target illuminance. The light transmittance may be controlled.

  In addition to this basic control, the target value for the liquid crystal panel 40 is corrected based on the pupil opening obtained from the facial image photographing camera 54 and the actual illuminance so that the light amount preferable for the passenger wearing the HMD 30 enters the eye E. In addition, the light transmission degree of the liquid crystal panel 40 is feedback-controlled. With this feedback control, it is possible to generate a state in which a preferable amount of light enters the eye E in relation to actual illuminance for the passenger wearing the HMD 30. In particular, the driver is driving while changing the line of sight, the direction of the face, etc., and the illuminance entering the eye can be adjusted to a preferable illuminance in response to the change in the line of sight of the driver. Then, based on the target illuminance acquired by the feedback control, the stored values of the gateway point G, the sensor node SN of the HMD 30, and the memories 88, 90, 92 of the portable terminal M are updated.

  18 and 19 are flowcharts for explaining one specific example of the light transmission control of the HMD 30, and FIG. 19 shows a flow subsequent to FIG. First, referring to FIG. 18, the sensor node SN including the sensor node SN built in the HMD 30 forms a network with the gateway point D (S100), and each sensor node SN performs relay processing of other sensor nodes SN. By (S101), the process which detects the position of each sensor node SN is performed (S102).

  Further, data of each sensor node SN, for example, position data, occupant pupil opening degree data, and actual illuminance data are supplied to the gateway point G (S103), and the gateway point G receives this (S104).

  When a person carrying the portable terminal M such as a cellular phone gets on (S105), a network including the portable terminal M is formed (S106), and the position of the portable terminal M is detected (S107). Next, along with this position data, personal characteristic data such as ID, weight, blood pressure, pulse, and target illuminance data stored in advance in the portable terminal M are transmitted to the gateway point G via the sensor node SN (S108). . If this mobile ID is new, these pieces of information are stored in the memory 88 of the gateway point G (S109 in FIG. 19).

  Referring to FIG. 19, at gateway point G, based on the received location data of mobile terminal M and the location data of sensor node SN, sensor node SN closest to mobile terminal M is selected (S110). Of course, when a plurality of persons each have a portable terminal M, the nearest sensor node SN is selected for each portable terminal M. Then, based on the target illuminance of the ID of each mobile terminal M, the target illuminance data is transmitted to each sensor node SN (S111), and the corresponding sensor node SN receives this and executes the light amount control, and the HMD 30 The amount of light transmitted through the lens 32 is adjusted by the light transmittance of the liquid crystal panel 40 (S112). The liquid crystal panel 40 is feedback controlled as described above.

  For the occupant wearing the HMD 30, the amount of light entering the eye E is preferable by the light amount control in S112, but the pupil opening degree of the occupant is detected by the face image photographing camera 54 (S113). Is within the proper range, the process proceeds from step S114 to step S115, and the calculation of the target illuminance and the update of the stored data of the gateway point G are executed. Thereby, the optimal target illuminance of each person who compensated for individual differences in glare can be stored. The target illuminance data for update is supplied to the sensor node SN and the portable terminal M, and the data is updated (S116, S117).

  20 and 21 are flowcharts for explaining another specific example of the light transmission control of the HMD 30. FIG. 21 shows a flow subsequent to FIG. These examples in FIGS. 20 and 21 are examples when the occupant does not have the portable terminal M, and the weight of the occupant is detected by the pressure sensor (sensor node SN) of the seat on which the occupant is seated. The occupant is identified indirectly from the weight. Then, the target illuminance is extracted based on the detected weight from the data of “weight-target illuminance” stored in advance, and the light transmission degree of the liquid crystal panel 40 of the HMD 30 is controlled based on the target illuminance. The control of the liquid crystal panel 40 is feedback-controlled by the pupil opening degree of the occupant wearing the HMD 30, and the target illuminance is corrected so as to obtain an optimum illuminance. Then, the stored data of the gateway point G is updated with the corrected target illuminance.

  Assuming the above, referring to FIG. 20, the sensor node SN including the sensor node SN built in the HMD 30 forms a network with the gateway point D (S200). A process of detecting the position of each sensor node SN is performed by the relay process (S201) of the sensor node SN (S202). Further, data of each sensor node SN, for example, position data, occupant pupil opening degree data, and actual illuminance data are supplied to the gateway point G (S203), and the gateway point G receives this (S204).

  The sensor node SN installed in each seat forms a network with other sensor nodes SN and gateway points D (S205), and each sensor node SN is connected to each sensor by relay processing (S206) of the other sensor nodes SN. Processing for detecting the position of the node SN is performed (S207). Each sensor node SN of the seat on which the occupant wearing the HMD 30 is seated detects the pressure from the occupant seated on the seat, and this pressure data is transmitted to the gateway point G together with the position data for each sensor node SN. (S208).

  Referring to FIG. 21, gateway point G selects HND 30 and illuminance sensor 24 that are closest to pressure sensor node SN based on the received position data of sensor node SN (S209). Of course, when a plurality of people are in the vehicle, the HND 30 and the illuminance sensor 24 closest to the pressure sensor node SN of the corresponding seat are selected.

  The pressure sent from the pressure sensor node SN, that is, the occupant is indirectly specified based on the weight, and the weight is determined based on the weight-target illuminance data stored in advance based on the weight. The target illuminance is extracted, and the extracted target illuminance is transmitted to the corresponding sensor node SN (S210). The sensor node SN that has received this (S211) receives this and executes light amount control, and the amount of light transmitted through the lens 32 of the HMD 30 is adjusted by the light transmittance of the liquid crystal panel 40 (S212). The liquid crystal panel 40 is feedback controlled as described above. By performing this feedback control, when the occupant's line of sight, face orientation, and the like are changed, the illuminance entering the eyes can be adjusted to a preferable illuminance correspondingly.

  For the occupant wearing the HMD 30, the amount of light entering the eye E is preferable by the light amount control in S212, but the pupil opening degree of the occupant is detected by the face image photographing camera 54 and transmitted to the gateway point G. (S213) The gateway point G (S214) that has received this determines the range of the pupil opening is appropriate (S215), and if the pupil opening is within the appropriate range, the process proceeds from step S215 to step S216. Then, the calculation of the target illuminance and the update of the stored value of the memory 88 of the gateway point G are executed and transmitted to the sensor node SN. Thereby, the data of the memories 88 and 90 of the gateway point G and the sensor node SN are updated with the optimum target illuminance of each person compensated for the individual difference in glare (S217).

It is a figure which shows the basic composition of the information display apparatus for vehicles of an Example. It is a figure which shows the spectacles type HMD employ | adopted in the Example. It is a disassembled perspective view for demonstrating the structure of the lens on either side of the spectacles type HMD of FIG. It is a flowchart which shows the image display control process of HMD by the image process part in an Example. It is a flowchart which shows an example of the spectral correction process in an Example. It is a flowchart which shows an example of the follow-up delay correction process in an Example. It is a flowchart which shows an example of the dangerous object extraction process in an Example. It is a conceptual diagram of the gaze action model of an expert driver. It is a figure which shows the example of the image visually recognized by a driver by the image display control of HMD by an image process part. It is the figure which planarly viewed the communication network in a vehicle comprised with a sensor node and a gateway point. It is the figure which looked at the arrangement | positioning of a sensor node from the side. It is a block diagram of a sensor node. It is a block diagram of a gateway point. It is a block diagram of a portable terminal. It is the figure which represented typically the apparatus controlled by a gateway point while visualizing transfer of the network information between a sensor node and a gateway. It is the figure which represented typically the apparatus controlled by a gateway point while visualizing transfer of the network information between a portable terminal, a sensor node, and a gateway. It is the figure which represented typically the communication network between the sensor node with which the light transmission of the lens of HMD was mounted | worn, the sensor node with which HMD was mounted | worn, and a gateway point. It is a flowchart for demonstrating the 1st example of the specific procedure regarding control of the light transmittance of the lens of HMD. FIG. 19 is a flowchart for describing a first example of a specific procedure for controlling light transmittance of an HMD lens, following FIG. 18. It is a flowchart for demonstrating the 2nd example of the specific procedure regarding control of the light transmittance of the lens of HMD. FIG. 21 is a flowchart for describing a second example of a specific procedure for controlling the light transmittance of the HMD lens, following FIG. 20.

Explanation of symbols

14 First Front Shooting Camera 16 Second Front Shooting Camera 18 Front Blind Spot Shooting Camera 20 Back Blind Spot Shooting Camera 24 Illuminance Sensor 30 HMD
32 Lens (light transmissive liquid crystal)
34, 36 Protection panel 38 Color image display panel 40 Liquid crystal panel 42 Display drive circuit 44 Liquid crystal drive circuit 50 Position sensor 52 Gaze direction estimation unit 54 Face image camera 62 Vehicle state sensor 86 Pupil opening detection processing circuit 88 Gateway point memory 90 Memory of sensor node SN of HMD 92 Memory of mobile terminal M Mobile terminal (mobile phone)

Claims (7)

Vehicle information display apparatus including an HMD that includes left and right lenses whose light transmittance is controllable, and that can see the surrounding environment through the left and right lenses and can display the external situation of the vehicle as a stereoscopic image in front of the eyes Because
An illuminance sensor for detecting the illuminance of light incident on a passenger wearing the HMD;
Pupil opening degree detecting means for detecting an opening degree of a passenger's pupil wearing the HMD;
Light transmittance control means for controlling the light transmittance of the left and right lenses based on the opening degree of the occupant's pupil detected by the pupil opening degree detection means and the illuminance of the light detected by the illuminance sensor;
A vehicle information display device comprising: a communication network means for transmitting / receiving data to / from the illuminance sensor, the pupil opening degree detection means, and the light transmittance control means. Storage means constituting part of the communication network means;
In the storage means, the target illuminance is stored in advance for each occupant,
The vehicle information display device according to claim 1, wherein the light transmittance control unit is controlled based on a target illuminance stored in the storage unit.   The illuminance sensor is constituted by a sensor node that forms part of the communication network means and is mounted on the HMD, and attribute data that can indirectly identify an occupant in the memory of the sensor node; The vehicle information display device according to claim 1, wherein data of target illuminance of the occupant is stored in association with attribute data. A portable terminal that can be incorporated into the communication network means;
The vehicle information display device according to claim 3, wherein data of target illuminance of an occupant who owns the mobile terminal is stored in a memory of the mobile terminal in association with the ID of the mobile terminal.   The vehicle information display device according to claim 3, wherein a plurality of sensor nodes constituting the communication network means have a detection function capable of specifying an occupant wearing the HMD.   The vehicle information display device according to claim 3 or 4, wherein the target illuminance is adjusted based on an occupant's pupil opening detected by the pupil opening detection means and an illuminance of light from the illuminance sensor. Attributes relating to the eyes of the occupant are stored in the memory of the portable terminal,
The vehicle information display device according to claim 4, wherein after taking in an attribute of the occupant and adjusting the target illuminance based on the attribute, control of light transmittance of the lens is executed.

Images