JP6743254B2 - Video display device and control method - Google Patents

Description

The present invention relates to a video display device that can be mounted on a user's head and displays a video in front of the user's eyes, and a control method thereof.

The user-worn video display device is expected to be lighter and more compact, and the burden on the device user will be reduced. This type of image display device has an advantage that information can be obtained in a state where both hands can be freely used, and is expected to be applied to a wide range of purposes.

Patent No. 5228305

For example, as in the above-mentioned patent document, there has been proposed an apparatus that uses a spectacles-type or head-mounted type mounting unit to dispose a display section immediately in front of a user's eyes to perform display and display termination.

In a wearable image display device, it is difficult for the user of the device to remove his/her line of sight from the image, and it is desirable to reduce discomfort caused by disturbance of the image while maintaining the display. However, in order to terminate the display according to the judgment of the device as in Reference 1, it may be important to keep displaying the image depending on the use of the device, and in that case, it may be disadvantageous to the device user. Exists.

It is conceivable that this image distortion is caused by the display system of the image display device. For example, in the case of a video display device using liquid crystal, flicker of the screen when updating the video information of the screen may be considered, and in the case of a field sequential system (color time division system) video display device, color break (color breakup) may occur. To be In the case of a wearable image display device, these image distortions may be recognized by the device user as great discomfort when the device user moves. In addition, the display method for reducing the disturbance of the image increases the power consumption.

A video display device that can be mounted on a user's head for solving the above-mentioned problems, and a video display unit capable of switching between two or more display methods, and a control unit for instructing the video display unit of the display method. A first detector that detects the movement of the user's head, a second detector that detects the movement of the user's viewpoint, the output of the first detector, and the second detector. And an exercise determination unit that determines the exercise status of the device user based on the output of. The control unit instructs the video display unit to change the display method according to the determination result of the exercise determination unit.

According to the present invention, in the wearable image display device, it is possible to reduce the discomfort felt by the device user from the image with low power consumption.

Other objects, features and advantages of the present invention will be apparent from the following description of embodiments of the present invention with reference to the accompanying drawings.

1 is a block diagram of a video display device 10 according to a first embodiment. 3 is an example of an external view of the video display device 10. FIG. It is a figure when the video display apparatus 10 shown in FIG. 1B is mounted. 6 is another example of an external view of the video display device 10. FIG. It is a figure when the video display apparatus 10 shown in FIG. 1D is mounted. It is a figure when another example of the video display device 10 is attached. 3 is a block diagram showing an example of a video display unit 1001 in the video display device 10. FIG. FIG. 7 is a circuit diagram showing an example of a light source element 2003 in the image display unit 1001. 7 is a timing chart showing an example of a 1× speed display operation of the video display unit 1001 in the video display device 10. 7 is a timing chart showing an example of double-speed display operation of the video display unit 1001 in the video display device 10. 9 is a timing chart showing an example in which the video display unit 1001 in the video display device 10 performs double-speed display operation and frame complement. 7 is a timing chart showing an example of a 3× speed display operation of the video display unit 1001 in the video display device 10. 9 is a table showing a determination process by an exercise determination processing unit 1006. 9 is a flowchart showing a determination processing flow by a control unit 1003 and a motion determination processing unit 1006. It is an example of the outside world. 6 is a display example of a virtual image displayed by the image display device 10. This is an example of superimposing a virtual image on the outside world. 6 is a display example by the video display device 10. This is an example of image display in which the contrast is weakened by the image processing unit 1008. This is an image display example in which sharpness is weakened by the image processing unit 1008. It is an example of an image that is a target of the image determination processing unit 1009. 11 is another example of a video that is a target of the video determination processing unit 1009. 6 is a timing chart showing an example of light source control stop display operation of the video display unit in the video display device 10. 9 is a timing chart showing an example of an operation of stopping the update of the ferroelectric liquid crystal of the video display unit in the video display device 10. It is a figure which shows one Example of a 2nd sensor. It is a detailed view showing an example of the 2nd sensor. It is an example of an image at the time of initial setting of an image display device. It is a figure which shows the video display range at the time of initial setting of a video display device. 7 is a block diagram showing a video display device 130 according to Example 2. FIG. 9 is a block diagram showing a video display device 140 according to Example 2. FIG. 7 is a block diagram showing a video display device 150 according to a second embodiment. FIG. FIG. 9 is a block diagram showing a video display device 160 according to a third embodiment. FIG. 9 is a block diagram showing an example of a control method of a video display device according to a third embodiment.

Embodiments of the present invention will be described below with reference to the drawings.

Embodiments of the present invention will be described with reference to the accompanying drawings.
1. Outline of Device FIG. 1A is a block diagram showing an image display device that can be worn by a user and displays an image in front of the user's eyes.

The video display device 10 includes a video display unit 1001, a display control unit 1002, a control unit 1003, a first sensor 1004, a second sensor 1005, a motion determination processing unit 1006, a video information source 1007, a video processing unit 1008, and a video determination processing unit. 1009, a storage unit 1010, a frequency determination processing unit 1011 and a power supply unit 1012 are provided.

The video display device 10 transmits the video information acquired from the video information source 1007 to the video display unit 1001 via the video processing unit 1008 to display the video. The image display unit 1001 is typically a display using a liquid crystal element or a mirror array element.

The video information source 1007 appropriately selects the data of the video information stored in the storage device (not shown), performs processing such as decryption and decryption of the data as necessary, and transmits the video information to 1008. Time-series moving images may be transmitted in time series, or still images may be sequentially transmitted.

The control unit 1003 is connected to the controller 1020 outside the video display device 10 by wire or wirelessly. By operating the controller 1020 by the device user 20, various settings related to the power supply On/Off of the image display device 10 and images can be performed. The controller 1020 may be a controller dedicated to the video display device 10, or a dedicated application program may be installed in a smartphone so that the smartphone can be used as the controller. In addition to the controller, the power supply unit 1012 is provided with a switch for turning the power supply on/off.

The display control unit 1002, the control unit 1003, the motion determination processing unit 1006, the video information source 1007, the video processing unit 1008, the video determination processing unit 1009, and the frequency determination processing unit 1011 are mounted on the video display device 10 as independent hardware. To be done. Alternatively, they may be implemented by one or more processors or microprocessors and software or firmware. They may be implemented as a part of functional blocks of an integrated circuit, or may be implemented by a programmable logic device such as an FPGA (Field-Programmable Gate Array).

The storage unit 1010 also need not be mounted as an individual component, and may be mounted as a partial functional block of an integrated circuit.

FIG. 1B is an external view of an example of the video display device 10. The main components are housed in the joint between the lens and the temple, and the image is projected on the lens part, which is a half mirror. The controller 1020 and the main body of the apparatus are connected by a cable.

FIG. 1C is a diagram when the video display device 10 of FIG. 1B is mounted.

FIG. 1D is an external view of another example of the video display device 10. Main components are housed in the joint between the lens and the temple, and the prism serves as the projection unit (2007 in FIG. 2) of the image display unit 1001.

FIG. 1E is a diagram when the video display device 10 of FIG. 1D is mounted.

FIG. 1F is an external view of still another example of the video display device 10. The prism in front of the eyes serves as a projection unit (2007 in FIG. 2) of the image display unit 1001, and other components are dispersedly housed in the helmet and the temple.

2. Display of Image An example of the configuration of the image display unit 1001 is shown in FIG. The video display unit 1001 includes a video signal processing unit 2001, a light source element power supply control unit 2002, a light source element 2003, a light source driver 2004, a modulator 2005, a modulator driver 2006, a projection unit 2007, and a setting control unit 2008.

The video information from the video processing unit 1008 is transmitted to the video signal processing unit 2001, and the video signal processing unit 2001 determines the brightness of the light source, the light source driving timing, and the modulator driving pattern corresponding to the received video information.

Information on the brightness of the light source is transmitted to the light source element power supply control unit 2002. The light source element power control unit 2002 controls the supply voltage to the light source element 2003 according to the received brightness information.
The timing of driving the light source is transmitted to the light source driver 2004. The light source driver 2004 controls the light source element 2003 according to the timing of driving the light source.
The light source element power supply control unit 2002 and the light source driver 2004 may be mounted in the same element.

FIG. 3 shows a detailed configuration diagram of the light source element power supply control unit 2002, the light source element 2003, and the light source driver 2004. The light source element 2003 includes three LEDs of three primary colors of a red LED 3001, a green LED 3002, and a blue LED 3003. The three LEDs are connected in series with current limiting resistors 3004, 3005, and 3006, respectively, and the potentials VLEDr, VLEDg, and VLEDb are applied from the light source element power supply control unit 2002, respectively. The potentials of VLEDr, VLEDg, and VLEDb can be set to arbitrary values by the light source element power control unit 2002, and the light emission of the red LED 3001, the green LED 3002, and the blue LED 3003 can be controlled. The terminals of the current limiting resistors 3004, 3005, and 3006 on the opposite side of the LEDs are connected to the light source driver 2004. The light source driver 2004 controls the light emission amount and light emission time of the red LED 3001, the green LED 3002, and the blue LED 3003, respectively, by changing the potentials of the terminals of CTRLr, CTRLg, and CTRLb, and the modulator 2005 controls all pixels (pixels). ) Is modulated. The display control unit 1002 sends a control signal designating the brightness of the light source to the setting control unit 2008 of the video display unit 1001.

The light source has been described as having a configuration in which each of the three primary color LEDs is provided, but the configuration is not limited to this. The configuration may include one or more white LEDs. Further, it may be a light source other than the LED. Regarding the three primary colors, the light source does not have to be configured to emit only the primary colors, and for example, a configuration may be adopted in which a specific color is taken out by a filter by using a white light source and a dichroic filter or a color wheel.

Returning to the explanation of FIG.
The modulator drive pattern is transmitted to the modulator driver 2006. The modulator driver 2006 drives the modulator 2005 according to the modulator drive pattern.
The modulator 2005 is, for example, a transmissive liquid crystal element, an LCOS (Liquid Crystal On Silicon), a DMD (Digital Mirror Device), or the like.
The modulator 2005 and the modulator driver 2006 may be configured as one element component.
Hereinafter, the modulator 2005 will be described assuming the LCOS system.

The light emitted from the light source element 2003 is modulated by the modulator 2005 and projected on the projection unit 2007.
The projection unit 2007 may be, for example, a reflective object such as a mirror, a scattered object such as a screen, a prism, a half mirror, or a lens. Also, a combination of a plurality of these may be used.

Due to the structure of the projection unit 2007 or the addition of other components, the image display device 10 of the present invention has a form such as opaque goggles that covers the entire field of view of the device user 20 and that the device user 20 visually recognizes the outside world. It is possible that a transmissive type that allows images to be recognized in a part of the field of view is possible. The following description assumes a transmissive form.

In the transmissive mode, the device user 20 visually recognizes an image as shown in FIGS. 7A, 7B, and 7C. That is, the device user 20 can recognize the external world as shown in FIG. 7A and the virtual image 7001 as shown in FIG. 7B by superimposing them as shown in FIG. 7C.

Further, the image display device 10 has a form in which an image is projected to both eyes of the device user 20 and a form in which only one eye is projected. Although it is assumed in FIG. 2 that the image is projected to only one eye, when projecting to both eyes, two projection units 2007 are provided so that the light from the modulator 2005 enters the left and right eyes. To configure. Alternatively, the entire image display unit 1001 may be configured to have two for the right eye and one for the left eye, and different images with parallax may be projected to form a stereoscopic image.

When the device user 20 observes the light from the projection unit 2007, the light according to the input image information can be recognized as an image.
The setting control unit 2008 can receive the control signal and change the setting of the video signal processing unit 2001.

The image display unit 1001 has two or more display methods for displaying an image. The display method is illustrated in FIGS. 4A, 4B, 4C, and 4D. The video information is treated as moving picture information in which, for example, N still images per second are arranged in a predetermined order on average, and N is a positive number of 1 or more. At this time, N is called a frame rate, and is expressed in frames per second (fps) as the number of still images per second. A frame rate of 30 fps or 60 fps is commonly used at present. In the example of FIG. 4, the display count of the video display unit 1001 is changed with respect to the frame rate of the video information.

FIG. 4A illustrates the first display method. In the first display method, the video display unit 1001 changes the display video once per 1/N seconds with respect to the frame rate N. The displayed image is called a frame.

In this embodiment, the field sequential method (color time division method) will be described as an example. That is, with respect to the display of one frame, the image display unit 1001 divides the image information into the primary color components of red, green, and blue, and displays the image of each color component separately within the time obtained by further dividing 1/N second into three. ..

For example, at the beginning of the display period of the frame 1, the display of the modulator 2005, which is the LCOS, is set to correspond to the red component divided from the image information of the frame 1 (1R), and the red LED 3001 is shorter than 1/(3N) seconds. It emits light for a predetermined time (period 1R). Next, the modulator 2005 is set to the setting (1G) corresponding to the green component of the frame 1, and the green LED 3002 emits light for a predetermined time (period 1G) shorter than 1/(3N) seconds. Further, the modulator 2005 is set to the setting (1B) corresponding to the blue component of the frame 1, and the blue LED 3003 emits light for a predetermined time (period 1B) shorter than 1/(3N) seconds. By sequentially displaying each color component at high speed, an observer wearing the apparatus recognizes it as a full-color image in which the three primary color components are mixed.

Such display of each color component is similarly performed on the frames 2 and 3 of the display image. 4A to 4D, the description is given up to the frame 3, but the same processing is repeatedly executed for the subsequent frames.

FIG. 4B illustrates the second display method. In the second display method, as compared with the first display method of FIG. 4A, the display time of each color component of the modulator 2005 for each frame is further divided into two, and 1/N seconds corresponding to the frame rate N of the display video. One color component is displayed in a period divided into six. This second display method is referred to as double-speed driving because the modulator 2005 is driven at a double update speed as compared with the first display method.

Further, by increasing the driving speed of the modulator 2005 and the light source element 2003, a display method of 3× speed, 4× speed, or higher speed can be realized. Changing the driving speed of the modulator 2005 and the light source element 2003 is changing the update cycle of the pixel information of the liquid crystal element or the mirror array element. FIG. 4D shows a timing chart of the 3× speed drive.

FIG. 4C shows a modified example at the time of double speed driving. The driving cycle of the modulator 2005 and the light source element 2003 is the same as the display method of FIG. 4B. By generating frame 1.5 of the intermediate frame from the frames 1 and 2 of the original image information and the images of other frames, and displaying them in the order of frame 1, frame 1.5, and frame 2, it is more possible for the observer. It is recognized as a smooth image. The generation of the intermediate frame is performed by the video processing unit 1008.
For example, as shown in FIG. 4D, the intermediate frame can be generated even at the update speed such as 3× speed or 4× speed that is faster than 2× speed, and the update speed is not limited to 2× speed.

The 1x speed display, the 2x speed display, the 3x speed display, and the 2x speed display, or the generation of the intermediate frame makes it difficult for the device user 20 to perceive color breakup, thereby reducing discomfort and reducing the movement of the image. Is smoother but consumes more power.

The display control unit 1002 transmits a control signal for instructing switching of the display method via the setting control unit 2008 of the video display unit 1001.

The image processing unit 1008 can change the image information input from the image information source 1007 and output the changed image information to the image display unit 1001.
The video information is, for example, information including data of H pixels (H is an integer of 1 or more) and V pixels (V is an integer of 1 or more) of H×V pixels per frame.
Changing the contrast of an image is a process of changing the difference in lightness and darkness of colors, and a value obtained by multiplying the pixel value input from the video information source 1007 by a proportional coefficient higher than 1 is set as the pixel value output to the video display unit 1001. Processing.

The image brightness change is a process of incrementing a pixel value by a designated value to obtain an output pixel value, and adding an arbitrary value to the pixel value input from the video information source 1007 to display an image. This is processing for setting the pixel value to be output to the display unit 1001.
The video processing unit 1008 performs processing such as raising or lowering the contrast of the image and/or raising or lowering the brightness of the image according to the signal from the control unit 1003.

The video processing unit 1008 may change other parameters related to the image, for example, sharpness, saturation, and hue according to the signal from the control unit.
The video processing unit 1008 can be switched so as to send video information to the video display unit 1001 without performing these processes in response to a signal from the control unit 1003.

For example, when the contrast of the image shown in FIG. 8A is weakened, the image shown in FIG. 8B is obtained. On the other hand, if the sharpness of the image shown in FIG. 8A is weakened, the image shown in FIG. 8C is obtained. As will be described later, when the viewpoint position of the device user 20 is not on the image of the display image, the eyestrain of the device user 20 can be reduced by weakening the contrast and sharpness.

3. Control of motion and image of device user The first sensor 1004 is a sensor for detecting rotation of the head of the device user 20, and is, for example, a gyro sensor. The first sensor 1004 outputs a three-dimensional motion vector indicating the movement of the head for a predetermined time. The second sensor 1005 is a line-of-sight sensor for detecting the line-of-sight motion and the position of the gazing point, and outputs a two-dimensional motion vector or a three-dimensional vector indicating the line-of-sight motion of the device user 20 for a predetermined time.

The exercise determination processing unit 1006 determines the exercise status of the device user 20 from the outputs of the first sensor 1004 and the second sensor 1005. Specifically, there are three directions: the movement of the head, the movement of the line of sight, and the direction of the movement of the head and the movement of the line of sight. The process of determining these three items from the sensor output will be described later. Further, the motion determination processing unit 1006 determines whether or not the gazing point is on the virtual image 7001 by the image display device, and the process will also be described later.
The control unit 1003 determines the processing content according to FIG. 5 from the determination result of the motion determination processing unit 1006, sets the display speed of the video display unit 1001 to the display control unit 1002, and processes the video parameters. Instruct the section 1008.

In pattern 1 of FIG. 5, there is no movement of the head of the device user 20, no movement of the line of sight, and the gazing point is on the virtual image. At this time, since the device user 20 is stationary, it is difficult for the device user 20 to vibrate and the device and the device user 20 are displaced from each other. There is also no movement of the eyeball. A phenomenon such as a color break that the device user 20 feels uncomfortable is unlikely to occur. In such a case, since it is not necessary to consider the influence on the discomfort, the control unit 1003 instructs the process A.

The process A is a process in which the display method of the video display unit 1001 is driven at 1× speed, the video processing by the video processing unit 1008 is not performed, and the video from the video information source is passed to the video display unit 1001 without being changed. .. This is the process that consumes the least power.

Pattern 2 in FIG. 5 is different from pattern 1 in that the gazing point detected by the second sensor 1005 is not on the virtual image. At this time, the device user 20 is stopped, the virtual image 7001 is not visually recognized, and the importance of the visibility is lowered. Therefore, the control unit 1003 instructs the process B.

The process B is a process in which the display method of the video display unit 1001 is driven at 1× speed and the video processing unit 1008 lowers the contrast and brightness of the video by a predetermined amount.

Pattern 3, pattern 5, and pattern 8 in FIG. 5 are cases where the movement of the head of the device user 20 is detected or the movement of the line of sight is detected, and the movement is on the gazing point virtual image. Is.
In these patterns, the device user 20 is not stationary, and the device user 20 of the video display device 10 is more likely to feel uncomfortable. Therefore, the need to consider discomfort becomes high. The control unit 1003 for reducing discomfort instructs the process C.

In the process C, the display method of the video display unit 1001 is set to the triple speed with a higher update speed, and the video processing by the video processing unit 1008 is returned to the initial value.

Pattern 4, pattern 6, and pattern 9 in FIG. 5 are cases where the movement of the head of the device user 20 is detected or the movement of the line of sight is detected, and the gazing point is not on the virtual image. Is.
In these patterns, the device user 20 is not stationary, and it becomes more necessary to consider the discomfort of the device user 20, but the device user 20 does not visually recognize the virtual image 6001. Since the importance of sex is low, the control unit 1003 instructs the process D.

In the process D, the display method of the video display unit 1001 is set to the triple speed with a faster update speed, while the video processing unit 1008 lowers the contrast and brightness of the video by a predetermined amount.

In pattern 7 of FIG. 5, the movement of the head of the device user 20 is detected and the movement of the line of sight is detected. If the direction of movement of the head and the direction of movement of the line of sight at this time are substantially the same, the eye movement is likely to be so-called saccade (intermittent movement), and the device user 20 There is a high possibility that the outside world and the virtual image 7001 are out of recognition. Even in such a case, it is necessary to consider the discomfort of the device user 20 in the image, but since the importance of the visibility of the virtual image is low, the control unit 1003 instructs the process D described above.

FIG. 6 is a flowchart for realizing the pattern-based control mode shown in FIG. 5, which is executed by the control unit 1003 and the motion determination processing unit 1006.
When the main body power switch is turned on or when there is an instruction from the device user 20, the process starts from step S010.

In step S020, the control unit 1003 performs initial setting and energization processing of the first sensor 1004 and the second sensor 1005. In the initial processing, the display method of the video display unit 1001 is set to double speed, and various parameters of the video processing unit 1008 are set to prescribed initial values.

In step S030, the motion determination processing unit 1006 acquires the sensor output from the first sensor 1004.
In step S040, the motion determination processing unit 1006 acquires the sensor output from the second sensor 1005.

In step S050, the motion determination processing unit 1006 determines from the output of the first sensor whether the magnitude of the rotational speed of the head is equal to or greater than the specified value A1 (A1 is a positive value), and the determination result is X. .. If the detection result is the specified value A1 or more, X is true ('1'), and if less than A1, X is false ('0').
In step S060, the motion determination processing unit 1006 determines from the output of the second sensor whether the magnitude of the speed of movement of the line of sight is equal to or greater than a specified value S1 (S1 is a positive value), and the determination result is Y. And If the detection result is the specified value S1 or more, Y is true ('1'), and if it is less than S1, Y is false ('0').

In step S070, the motion determination processing unit 1006 determines from the output of the second sensor whether the two-dimensional coordinate of the position of the line of sight (gazing point) is on the virtual image (within a predetermined range). Be Z. When the position of the gazing point is within the predetermined range, Z is true ('1'), and when the position of the gazing point is outside the predetermined range, Z is false ('0').

Please refer to the above X, Y, Z and their values shown in FIG.
In step S080, the motion determination processing unit 1006 performs a conditional branch based on the logical product of the determination result X and the determination result Y. If X·Y=0, the process proceeds to step S110, and if X·Y=1, the process proceeds to step S090.

In step S090, the motion determination processing unit 1006 performs conditional branching based on the exclusive OR (XOR) of the determination result X and the determination result Y. If X XOR Y=1, the process proceeds to step S111, and if X XOR Y=0, the process proceeds to step S100.
In step S110, if Z is true, the process proceeds to step S120, the control unit 1003 instructs the process A, and if Z is false, the process proceeds to step S130 and the control unit 1003 instructs the process B.

In step S100, the motion determination processing unit 1006 compares the moving direction of the head, which is the output of the first sensor 1004, with the moving direction of the line of sight, which is the output of the second sensor 1005, and determines the direction of the motion vector. It is determined whether they substantially match. The determination method will be described later.
In step S111, conditional branching is performed based on the determination result Z. If Z is true, the process proceeds to step S140 and the control unit 1003 instructs the process C. If Z is false, the process proceeds to step S150 and the control unit 1003 instructs the process D.

In step S160, it is determined whether or not this flow is set to be continuously executed. The device user 20 may set or change the setting, or a default may be set. If the setting is valid, the process proceeds to standby step S170, and if the setting is invalid, the process proceeds to end step S180.

In step S170, after waiting for a predetermined time (300 milliseconds to 10 seconds), the process returns to step S030.
In step S180, a power-off process and a sleep setting for the first sensor 1004 and the second sensor 1005 are performed.
The process ends at end step S190.

The video display device 10 has a storage unit 1010. When the display method of the video display unit 1001 is changed or the processing method of the video processing unit 1008 is changed, the control unit 1003 records the history and time of the change in the storage unit 1010.

If the change history of each process recorded in the storage unit 1010 exceeds a predetermined number of times within a predetermined time, for example, if the change history is changed five times in three days, the frequency determination processing unit 1011 causes the video information source 1007. Request to change the setting of the video information according to the change history. The request to change the video information is made at the timing of the ending process of step S180 in FIG. 6, for example, and the change to the video information is to change parameters such as image contrast, sharpness, saturation, hue, and image brightness. ..

4. Line-of-sight sensor An example of the line-of-sight sensor which is the second sensor 1005 will be introduced.
As shown in FIG. 11A, the sensor A element 1101, the sensor B element 1102, and the detection controller 1103 are provided. When the sensor A element 1101 detects a movement, the detection controller 1103 activates the sensor 2 element 1102 and commands the sensor B element to output more precise data.

A more specific structure is shown in FIG. 11B. It is suitable for mounting on a frame portion of a glasses-type device as shown in FIGS. 1B and 1D. The line-of-sight sensor can detect the movement of the eye 1111 of the device user 20, and emits infrared light in a first light projecting unit 1112 and a second light projecting unit 1113, a first light receiver 1114, and a second light receiving unit 1114. The light receiver 1115, the comparator 1116, the first camera 1117 and the second camera 1118, the current controller 1119, the motion detection processor 1120, and the pause controller 1121.

The infrared light emitted from the first light projecting unit 1112 and the second light projecting unit 1113 is projected and reflected by the eye 1111 of the device user 20. The reflected infrared light enters the first light receiver 1114 and the second light receiver 1115. The first light receiver 1114 and the second light receiver 1115 are installed in the left and right directions different from each other with respect to the eye 1111 and the amount of light received changes depending on the positions of the black eye and the white eye. Since the first light receiver 1114 and the second light receiver 1115 are arranged on the left and right of the eye 1111, the change in the amount of light received when the eye 1111 is displaced differs depending on the light receiver. The movement of the eye 1111 can be detected by obtaining a difference by the comparator 1116 from the amount of light received by the first light receiver 1114 and the amount of light received by the second light receiver 1115. This is a detection method called the scleral reflection method.

The motion detector 1120 determines that the motion of the eye 1111 is detected when the output of the comparator 1116 is equal to or larger than a predetermined value, outputs a motion detection signal, and moves to the eye 1111 when the output of the comparator 1116 is smaller than the predetermined value. It is determined that there is no motion, and a motion non-detection signal is output.

Upon receiving the motion detection signal from the motion detector 1120, the pause control unit 1121 sets the first camera 1117 and the second camera 1118 in an image capturing state capable of capturing an image, and those cameras are reflected from the eye 1111. The eye 1111 is imaged with infrared light. From the images captured by the first camera 1117 and the second camera 1118, image processing is performed by an image processing unit (not shown), and more detailed eye movement and viewpoint position are estimated. A dark pupil method, a corneal reflection method, or the like is used for image processing.

When receiving the motion non-detection signal from the motion detector 1120, the pause control unit 1121 stops some functions in the first camera and the second camera to bring them into a pause state in which power consumption is suppressed.
The motion detection processing unit 1120 changes the power supplied to the first light projecting unit 1112 and the second light projecting unit 1113 by sending a signal for motion detection or no motion detection to the current control unit 1119. The current amount in the imaging state is controlled to be larger than the current amount in the resting state.

In general, a camera element consumes more power than a light receiver and also requires a large amount of light for detection. Therefore, the power consumption of the device is suppressed by the present control method in which the camera element is in the image capturing state only when a rough movement is detected by the light receiver, rather than the camera element is always in the image capturing state.
In this embodiment, the case where the number of light projecting units is two has been illustrated, but the number of light projecting units is not limited to this. You may have more light projecting parts. Further, the light emission timing of each light projecting unit may be controlled to be different, and the light receiver or the camera element may acquire data in accordance with the light emission of each light projecting unit.

The number of camera elements and the number of light receivers are two in each example, but the number is not limited to this. Further, the light receiver and the camera element may be configured as one element, for example, some pixels of the camera element may be configured as the light receiver.

5. Correction of deviation of viewpoint and detection of position of gazing point In the initial setting step S020 shown in FIG. 6, initialization processing of the second sensor 1005 may be performed. When the second sensor 1005 is a sensor that detects the line of sight of the device user 20, a gap may occur between the gazing point of the device user 20 and the position of the second sensor each time the device is used.

In order to correct this shift, the motion determination processing unit 1006 displays a diagonal virtual image 1201 in the display area on the image display unit 1001 during the initialization process as shown in FIG. 12A, and the device user 20 looks at the intersection. Display a sentence prompting you to. When the second sensor 1005 detects that the user is still at the position of the gazing point for a predetermined number of seconds, it is determined that the device user 20 is gazing at the intersection of the virtual image 1201, and the position of the gazing point is determined as 2 The reference position p0 (h0, v0) of the sensor 1005 is set.

When the entire image display range of the image display unit 1001 is a quadrangle surrounded by P1 (Hmin, Vmin), P2 (Hmax, Vmin), P3 (Hmin, Vmax), and P4 (Hmax, Vmax) like the virtual image 1201. , H0=(Hmax−Hmin)/2 and v0=(Vmax−Vmin)/2.

In the gazing point determination step S070 of FIG. 6, whether the detection result of the second sensor 1005 is on the image displayed on the image display unit 1001 with reference to the reference position p0 (h0, v0) (whether Z is '1' or not). ) Is determined. As shown in FIG. 12B, when an arbitrary virtual display image 1202 is displayed in a region surrounded by Q1 (Hl, Vd), Q2 (Hl, Vu), Q3 (Hr, Vd), and Q4 (Hr, Vu). (Hmin ≤ Hl ≤ Hmax, Hmin ≤ Hr ≤ Hmax, Vmin ≤ Vd ≤ Vmax, Vmin ≤ Vu ≤ Vmax), and the position of the gazing point is obtained as p(h, v) by the second sensor 1005. At this time, when p(h,v) is within the quadrangle Q1Q2Q3Q4, the motion determination processing unit 1006 determines that the position of the gazing point is on the virtual image, and determines that Z in step S070 is true.

The reference position may be detected at a time specified by the device user 20 by the controller 1020, other than the initial processing. The device user 20 may instruct the second sensor 1005 about the detection timing by intentionally blinking a specific length or a specific order before and after gazing at a specified point.
The virtual display image 1202 does not have to be a quadrangle, but may be another figure such as a triangle or a circle.

6. The direction of the movement of the line of sight of the movement of the head In the motion determination unit 1006, based on the output of the motion vector of the first sensor 1004 and the output of the motion vector of the second sensor 1005, the direction of each of the movement of the head and the movement of the line of sight. Detect if the two match.

It is assumed that the output of the first sensor 1004 can be represented by the three-dimensional vector A ₀ . When the device user 20 is at a distance to recognize the virtual image 7001, and S is a virtual plane including four corners of the virtual image 7001, the orthogonal projection vector of the three-dimensional vector A _{0 with} respect to the plane S is A. Similar vector conversion processing is performed when the output of the second sensor 1005 is a three-dimensional vector.
The distance at which the device user 20 recognizes the virtual image 7001 is determined by an optical configuration such as the projection unit 2007 in the image display unit 1001, and the device user 20 separates the virtual image 7001 by a predetermined distance by the eyeball focus adjustment function. Recognize the position.

When the second sensor 1005 outputs a two-dimensional vector B0, consider the plane T of the three-dimensional space including the detecting axis of the second sensor 1005, converts the two-dimensional vector B0 to a three-dimensional vector B ₁ on the plane T Then, let B be an orthogonal projection vector of the three-dimensional vector B ₁ with respect to the plane S. Similar vector conversion processing may be performed when the detection result of the first sensor 1004 is obtained as a two-dimensional output.

When the moving directions of both detection sensors are expressed as a vector projected onto a single plane, and the moving vector of the output of the first sensor 1004 is A and the moving vector B of the output of the second sensor 1005, the angle formed by the two vectors is expressed. (ANGLE) θ=ANGLE(A−B) absolute value and an arbitrary value α (α is a positive value) are compared to determine a substantial match.

That is, if θ≦α, it is determined that the moving directions substantially match, and if θ>α, it is determined that the moving directions do not match.

7. Modified Example (1) The first sensor 1004 and the second sensor 1005 measure an acceleration sensor, a geomagnetic sensor, a GPS, a camera that captures the user, a camera that captures an external image seen by the user, and the pulse of the user. It may be a sensor, a sensor that measures the blood flow of the user, a clock, or the like. The first sensor 1004 and the second sensor 1005 may each include a filter, an amplifier, a level shifter, and the like. Alternatively, it may be configured to include a comparator and transmit the binary result indicating whether the detection result is higher or lower than the threshold value together with the vector value. Also, one or more conditions are satisfied when the duration of the motion, which is the detection result, exceeds a predetermined time, when the motion exceeds a predetermined speed, or when the motion exceeds a predetermined fluctuation amount. Sometimes, it may be configured to output a signal that a motion is detected.

(2) Instead of the determination process of the motion determination processing unit 1006 using the output of the first sensor 1004 or the second sensor 1005, the image determination processing unit 1009 may determine the feature of the image.
The image determination processing unit 1009 determines whether the image information may cause display discomfort to the device user 20. For example, in the case of an image in which an object continuously moves in the screen as shown in FIG. 9A, it is predicted that the device user 20 follows the moving image by moving the eyeball. Further, for example, as shown in FIG. 9B, when the image includes a content such as a character string that is continuously recognized in the vertical, horizontal, and diagonal directions in the screen, the device user 20 can make an eyeball. Is expected to move the viewpoint along the image.

Such an image can be detected in advance by the video determination processing unit 1009 performing video analysis of the digital image and referring to the amount of difference data between image frames.
In the case of a video in which the viewpoint is predicted to be moved in advance as shown in FIGS. 9A and 9B, the video determination processing unit 1009 outputs a control signal to the control unit 1003 so as to apply the process C.

Further, when the projection unit 2007 of the video display device 10 is a transmissive type, the video determination processing unit 1009 determines whether the video to be displayed is a video that is related to the outside world, for example, an augmented reality (AR) video. You may judge. The video determination processing unit 1009 can determine the type of video from the video metadata and additional information. For example, the virtual image 7001 in FIG. 7B displays information related to the external world in FIG. 7A. When it is determined that the image is related to the external world, the gazing point is located at a position other than the virtual image 7001. Even if it is detected (Z is false), the visibility of the displayed image is high in importance, and therefore the control unit 1003 is instructed to apply the process C of FIG.

(3) As the display method of the video display unit 1001, the display method shown in FIG. 10A may be adopted. In the display method shown in FIG. 10A, the double-speed drive of FIG. 4B is performed, the control of the light source is stopped, and the light emitting periods of the red LED 3001, the green LED 3002, and the blue LED 3003 of the light source element 2003 are made substantially the same. By doing so, the lights from the three LEDs are mixed, the device user 20 recognizes the image as a black and white image, and in principle, no color break occurs. In FIG. 5, it may be applied to the processing C and the processing D in FIG. 5 in which it is highly necessary to consider the discomfort of the image.

Furthermore, when the images of a plurality of consecutive frames in the video information are substantially the same image, the frame update of the modulator 2005 may be stopped. For example, as shown in FIG. 10B, when frame 1, frame 2 and frame 3 are substantially the same image, the modulator 2005 continues to display the same frame 1 for a period corresponding to these frames.

When the image information can be represented as, for example, H×V pixel information (both H and V are positive integers), it means that the number of pixels whose information has been changed between successive frames is substantially the same image. When the number of pixels is sufficiently smaller than H×V, or when the color information of each pixel can be expressed as primary color information of R, G, B (for example, R, G, B are positive integers from 0 to 255), consecutive frames There is a case where the change in the R, G, and B values of each pixel between the two is sufficiently small.

By doing so, the flicker of the image can be reduced. When the modulator 2005 uses a ferroelectric liquid crystal, the same frame is displayed a plurality of times, so power is consumed to erase and redisplay information. Stopping the frame update of the modulator 2005 leads to further reduction of power consumption. It may be applied to the processes C and D of FIG.

(4) The video information source 1007 may be configured to acquire video information from the outside. For example, the receiver may be a receiver that conforms to a video transmission standard such as DVI, HDMI (registered trademark), or DisplayPort, or a receiver that uses a general electric signal transmission method such as SPI, I2C, RS232, or USB. It may be a wired network receiver such as Ethernet (registered trademark), or a wireless network receiver such as a wireless LAN or Bluetooth (registered trademark).
The video information source 1007 may include a decoder that receives compressed information and decompresses it to convert it into video information, or includes a function of receiving and decoding encrypted video information. Good.

(5) The power supply unit 1012 supplies power to the video display device 10. The power supply unit 1012 is a rechargeable battery that can be charged from an external power supply as a power source, a power supply circuit that extracts desired power from a replaceable primary battery, a converter that is connected to an external power supply such as an outlet to extract desired power, and power. It includes one or more of the stabilizing circuits. In addition to the power source, the power supply unit 1012 may have a configuration including an integrated circuit for power control for controlling charging, supplied power, and the like, and observing the power source.

The control unit 1003 acquires information on the remaining power amount of the power source from the power supply unit 1012, and controls the video processing of the video processing unit 1008 to operate only when the remaining power amount exceeds a predetermined value.
The control unit 1003 may be configured to change the display method of the video display unit 1001 so that the display cycle is shortened only when the remaining power amount of the power supply unit 1012 exceeds a predetermined value.
The control unit 1003 may be configured to change the display method of the video display unit 1001 so that the display cycle becomes longer when the remaining power amount of the power supply unit 1012 falls below a predetermined value.

FIG. 13 is an explanatory diagram of the second embodiment. Only the parts different from FIG. 1 will be described.
The first sensor 1304 and the second sensor 1305 are provided separately from the housing of the video display device 130. Both sensors detect the operation of the device user 20, as in the first embodiment.

In this embodiment, information is exchanged between the first sensor 1304 and the video display device 130 via the communication unit 1013. The communication unit 1013 and the first sensor 1304 may be communicated as an electric signal on a conductor that is physically connected, or via wireless communication such as wireless LAN, Bluetooth (registered trademark), or Zigbee (registered trademark). May be done. The first sensor 1304 may include a communication unit (not shown). When using wireless communication, the first sensor 1304 may be powered by a power source different from that of the video display device 130.

Similarly to the first sensor 1305, the second sensor 1305 may exchange information with the communication unit 1013 by the wired communication or the wireless communication. The second sensor 1305 may include a communication unit (not shown). When using wireless communication, power may be supplied from a power source different from that of the video display device 130.
The motion determination processing unit 1006 receives the outputs of the first sensor 1304 and the second sensor 1305 via the communication unit 1013.

14 and 15 are modified examples of this embodiment.
In the modification shown in FIG. 14, the first sensor 1404, the second sensor 1405, and the motion determination processing unit 1406 are provided separately from the housing of the video display device 140. Information is exchanged between the exercise determination processing unit 1406 and the control unit 1003 via the communication unit 1013, and may be performed by the wired communication or the wireless communication. In addition, the first sensor 1404, the second sensor 1405, and the motion determination processing unit 1406 may be included in the same housing. Further, the exercise determination processing unit 1406 may be configured to include a communication unit (not shown).

In the modification shown in FIG. 15, the second sensor 1305 is provided separately from the housing of the video display device 150. Information is exchanged between the second sensor 1305 and the motion determination processing unit 1006 via the communication unit 1013, and may be wired communication or wireless communication as described above.

The first sensor 1304 and the second sensor 1305, which are provided separately, do not have to be attached to the device user 20, and may be any device that can detect the motion of the device user 20's head, the motion of the eyes, and the like. It may be a sensor using image processing. It is assumed that the device user 20 performs a work at a fixed position while watching an image and installs the camera-type first sensor 1304 and the second sensor 1305 on the workbench.
Further, when the device user 20 stands upright or sits in a fixed position and uses the image display device 130, the image display device 140, and the image display device 150, the pressure distribution for measuring the displacement of the center of gravity position below the device user 20. The first sensor 1304 or the second sensor 1305 may be provided by providing a detection sensor such as a measuring instrument.

FIG. 16 is an explanatory diagram of the third embodiment. Only the parts different from FIG. 1 will be described.
In this embodiment, the storage unit 1610 and the frequency determination processing unit 1611 are provided in the server 1601 separately from the video display device 160. The operations of the storage unit 1610 and the frequency determination processing unit 1611 are the same as those of the storage unit 1010 and the frequency determination processing unit 1011 of the first embodiment, respectively.

Information is exchanged between the storage unit 1310 and the control unit 1003 via the communication unit 1013 of the video display device 160 and the communication unit 1612 of the server 1601, and physical communication is performed between the communication units 1013 and 1612. May be communicated as an electric signal on a conducting wire connected to, or may be performed via wireless communication such as wireless LAN, Bluetooth (registered trademark), Zigbee (registered trademark).
Information is also exchanged between the frequency determination processing unit 1611 and the video information source 1007 via the communication unit 1013 and the communication unit 1612, which may be wired communication or wireless communication as described above. ..

Further, FIG. 17 shows a control method in a system including a plurality of video display devices 160. A first video display device 1711, a second video display device 1712, a third video display device 1713, and a fourth video display device 1714, which can communicate with the server 1601 via the network 1730.

The first video display device 1711 is used by the first user 1721, the second video display device 1712 is used by the second user 1722, and the third video display device 1713 is used by the third user 1723. The fourth video display device 1714 is used by the fourth user 1724.

Each of the first video display device 1711, the second video display device 1712, the third video display device 1713, and the fourth video display device 1714 has the same performance as the video display device 160.
The server 1601 includes a storage unit 1610 and a frequency determination processing unit 1611, and includes a first video display device 1711, a second video display device 1712, a third video display device 1713, and a fourth video display device 1714. The history and time when the display method of the video display unit 1001 is changed or when the processing method of the video processing unit 1008 is changed are received from the control unit 1003 via the network 1730.

The server 1601 extracts common information from the change information sent from the respective video display devices, and the frequency determination processing unit 1611 performs common processing of the respective video display devices recorded in the storage unit 1610. When the common processing exceeds a predetermined number of times within a predetermined time, the video information source 1007 is requested to change the video information. The change of the video information is, for example, change of parameters such as image contrast, sharpness, saturation, hue, and image brightness.

The number of video display devices 160 connected to the network 1730 is not limited to the number shown in this embodiment, and it is sufficient that at least one video display device 160 is connected.

Although the above description has been made with reference to the embodiments, it is obvious to those skilled in the art that the present invention is not limited thereto and various changes and modifications can be made within the spirit of the present invention and the scope of the appended claims.

10... Video display device 1001... Video display unit 1002... Display control unit 1003... Control unit 1004... First sensor 1005... Second sensor 1006... Exercise determination processing unit 1007 ... video information source 1008 ... video processing unit 1009 ... video determination processing unit 1010 ... storage unit 1011 ... frequency determination processing unit 1012 ... power supply unit 1013 ... communication unit 1020 ... -Controller 20... Device user 2001... Video signal processing unit 2002... Light source element power supply control unit 2003... Light source element 2004... Light source driver 2005... Modulator 2006... Modulator Driver 2007... Projection unit 2008... Setting control unit 3001... Red LED
3002・・・Green LED
3003...Blue LED
3004, 3005, 3006... Current limiting resistance 7001... Virtual image 1101... Sensor A
1102...Sensor B
1103... Detection control section 1111... Eyes 1112, 1113... Light projecting section 1114, 1115... Photoreceiver 1116... Comparator 1117, 1118... Camera 1119... Current control section 1120 ... Motion detection processing unit 1121... Pause control unit 1201... Virtual image 1202... Virtual display image 130... Image display device 1304... First sensor 1305... Second sensor 140... .. video display device 1404... first sensor 1405... second sensor 1406... motion determination processing unit 150... video display device 160... video display device 1601... server 1610... Storage unit 1611... Frequency determination processing unit 1612... Communication unit 1711... First image display device 1712... Second image display device 1713... Third image display device 1714... Fourth image display device 1721... First user 1722... Second user 1723... Third user 1724... Fourth user 1730... Network

Claims (8)

A video display device that can be worn on a user's head,
One video frame is divided into display periods corresponding to each of a plurality of colors, and a field sequential type video display unit configured to display an image of each color in each of the display periods corresponding to the respective colors.
A control unit for controlling the image display unit,
A first detection unit that detects the movement of the user's head,
A second detection unit that detects the movement of the user's line of sight,
An exercise determination unit that determines the exercise status of the device user by the output of the first detection unit and the output of the second detection unit,
Including,
When the control unit determines that the movement status includes movement of at least one of the head and the line of sight of the user, the control unit determines each display period corresponding to each color. Is divided into a plurality of parts, and the video of each color is controlled to be displayed a plurality of times in the one video frame.
Video display device characterized by. The video display device according to claim 1, wherein the plurality of colors are R, G, B three primary colors, and the one video frame is divided into three display periods corresponding to the respective colors R, G, B. What to do
Video display device characterized by. The image display device according to claim 2, wherein the image display unit includes a light source that emits light of each color of R, G, and B, and a modulator that modulates light from the light source to form an image of each color. The control unit controls the driving of the light source and the modulator according to the number of divisions of the display period corresponding to each of the R, G, and B colors.
Video display device characterized by. The video display device according to claim 3, wherein the modulator is a liquid crystal element or a digital micromirror device.
Video display device characterized by. The image display device according to claim 1, wherein the control unit moves if the motion determination unit determines that there is a motion of at least one of a head and a line of sight of the user as the motion status. An image display device characterized by increasing the contrast or brightness of an image more than when there is no light. The video display device according to claim 1, wherein
The first detection unit is configured to include any one of a gyro sensor, an acceleration sensor, a geomagnetic sensor, GPS, and a camera,
Video display device characterized by. The video display device according to claim 1, wherein
The second detection unit,
At least two floodlights that illuminate different eye positions,
Two light receivers at different positions for receiving the reflected light from the eye of the illuminated light;
A motion detector that detects the movement of the eyeball from the outputs of the two light receivers;
A camera that captures at least one eye that operates while the motion detector detects eye movement.
Video display device characterized by. The video display device according to claim 1, wherein
The movement determination unit, as the movement situation, if the duration of the movement detected by the first detection unit or the second detection unit exceeds a predetermined time, if the movement exceeds a predetermined speed, the movement When any one or more conditions are satisfied when exceeds a predetermined fluctuation amount, it is determined that there is a motion,
Video display device characterized by.