WO2020042638A1

WO2020042638A1 - Head-mounted display apparatus

Info

Publication number: WO2020042638A1
Application number: PCT/CN2019/084001
Authority: WO
Inventors: 姜欣; 黄通兵
Original assignee: 北京七鑫易维信息技术有限公司
Priority date: 2018-08-28
Filing date: 2019-04-24
Publication date: 2020-03-05
Also published as: CN108828779A; CN108828779B

Abstract

Disclosed is a head-mounted display apparatus (10), comprising multiple display screens (11), an optical device (12), a drive device (13) and an eyeball tracking module (14), wherein the multiple display screens (11) include a first display screen (111) and a second display screen (112), a picture on the first display screen (111) is imaged on the second display screen (112) by means of the optical device (12), the size of a formed image (111') is not less than the size of the corresponding area of a resolution vision field of a user's eyes on the first display screen (111), and the drive device (13) is connected to a processor (15). If the position of the geometrical center of the image (111') formed by the picture of the first display screen (111) by means of the optical device (12) is inconsistent with the position of a fixation point, the processor (15) sends a control instruction to the drive device (13), such that the drive device (13) adjusts the position of at least one optical component in the optical device (12) until the position of the geometrical center coincides with the position of the fixation point. When in use, a user pays attention to a high resolution picture on the first display screen (111), and the user will not be aware of the presence of a screen door effect due to the existence of the high resolution picture, thereby greatly improving the user experience of the head-mounted display apparatus (10).

Description

Head-mounted display device

Technical field

Embodiments of the present invention relate to the technical field of head-mounted display devices, and in particular, to a head-mounted display device.

Background technique

With the advent of the Internet of Everything, VR (Virtual Reality, virtual reality) equipment has gradually become the direction of the development of electronic equipment. Virtual reality refers to the reproduction of a specific environment or scene of a real environment by using three-dimensional image technology.

Due to the limitation of the display screen technology and the limitation of the high-resolution picture on the hardware requirements of the existing head-mounted display devices, the existing head-mounted display devices also have a screen window effect, and there is no particularly good method in the prior art. Solve this problem.

Summary of the Invention

At least some embodiments of the present invention provide a head-mounted display device to at least partially solve the problem of the screen window effect of the head-mounted display device in the related art when in use.

In one embodiment of the present invention, a head-mounted display device is provided. The head-mounted display device includes a plurality of display screens, optical devices, driving devices, and eye tracking modules.

The plurality of display screens include a first display screen and a second display screen, and a picture of the first display screen is imaged on the second display screen through an optical device, and the size of the image is not less than the resolution of the user's eyes. The size of the domain in the corresponding area of the first display screen, and the angular resolution of the first display screen is greater than the standard angular resolution;

The eyeball tracking module is connected to a processor, and is configured to obtain position information of a gaze point corresponding to a user's line of sight on the second display screen, and send the position information to the processor;

The driving device is connected to the processor, and is configured to receive a control instruction issued by the processor when the geometric center of the image formed by the screen of the first display screen through the optical device is inconsistent with the position of the gaze point, and Adjusting the position of the optical device according to the control instruction until the geometric center coincides with the position of the fixation point.

A head-mounted display device provided by one embodiment of the present invention includes a plurality of display screens, an optical device, a driving device, and an eye tracking module. The plurality of display screens includes a first display screen and a second display screen. The image of the first display screen is imaged on the second display screen by an optical device, and the size of the image is not less than the size of the user's eyes' resolution field of view in the corresponding area of the first display screen, and makes the first display The angular resolution of the screen is greater than the standard angular resolution, the driving device is configured to adjust the position of at least one optical component in the optical device, and the eye tracking module is configured to obtain a user's line of sight corresponding on the second display screen. The gaze point position information, the driving device is connected to the processor, and is configured to receive when the geometric center of the image formed by the processor through the optical device on the screen of the first display screen is inconsistent with the position of the gaze point A control instruction issued, and adjusting the position of at least one optical component in the optical device according to the control instruction until the geometric center and The gaze points are coincident. When the user uses the above-mentioned head-mounted display device, the user pays attention to the picture of the first display screen. The existence of the high-resolution picture makes the user not perceive the presence of the screen window effect, which greatly improves the head. User experience for wearable display devices.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic structural diagram of a head-mounted display device according to an embodiment of the present invention.

FIG. 2 is a schematic diagram of a connection relationship between a first driving motor and a second driving motor and constituent components in an optical device according to an exemplary embodiment of the present invention.

Fig. 3 is a schematic diagram showing the distribution of pixel units for displaying images on an array substrate and pixel units for acquiring user eyeball image information according to an exemplary embodiment of the present invention.

detailed description

The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present invention, rather than limiting the present invention. It should also be noted that, for the convenience of description, only some parts related to the present invention are shown in the drawings instead of all the structures.

Example one

FIG. 1 is a schematic structural diagram of a head-mounted display device according to an embodiment of the present invention. The head-mounted display device 10 may include a plurality of display screens 11, an optical device 12, a driving device 13, and an eye tracking module 14.

The plurality of display screens 11 includes a first display screen 111 and a second display screen 112, and a picture of the first display screen 111 is imaged on the second display screen 112 through the optical device 12. The size is not less than the size of the corresponding viewing area of the user's eyes in the corresponding area of the first display screen 111, and the angular resolution of the first display screen 111 is greater than the standard angular resolution.

The eyeball tracking module 14 is connected to the processor 15 and is configured to obtain position information of a gaze point corresponding to a user's line of sight on the second display screen 112 and send the position information to the processor 15. The processor 15 may be integrated in the head-mounted display device, or may be provided at a computer end connected to the head-mounted display device. In the solution shown in FIG. 1, the connection relationship between various components is presented, and the specific layout of the processor 15 is not limited.

The driving device 13 is connected to the processor 15 and is configured to receive the geometric center of the image 111 'formed by the processor 15 on the screen of the first display screen 111 through the optical device 12 and the position of the fixation point. A control instruction issued from time to time, and the position of at least one optical component in the optical device 12 is adjusted according to the control instruction until the geometric center coincides with the position of the fixation point.

Generally speaking, only the central part of the image reflected on the retina of the human eye can be clearly distinguished. This part is usually called the resolution field of view, and the angle of view is about 8 degrees to 15 degrees. The part from 15 to 30 degrees of field of view is called the effective field of view. In the effective field of view, users can see the existence and movement of objects. Although they can see clearly without turning their heads, the resolution has decreased. The peripheral part with a field angle of more than 30 degrees is called the induced field of vision. It can only sense the existence of objects and cannot see clearly what objects are.

When the human eye is observing objects or pictures, they are most concerned about distinguishing objects or pictures in the field of view. After a head-mounted display device is designed and manufactured, when a user uses the head-mounted display device, because the human eye is away from the head-mounted device, The distance of the display screen of the conventional display device has been almost determined, and which areas of the display screen are reflected in the resolution field of view of the human eye, so the size of the resolution field of view of the user's eye in the corresponding area of the first display screen can be determined.

In an optional embodiment of the present invention, the first display screen is a special high-resolution display screen, and the size of the special high-resolution display screen is strictly limited. However, this optional embodiment has specific requirements for the size of the first display screen, that is, the size of the image formed by the picture of the first display screen on the second display screen is not smaller than the resolution view field on the first display. The size of the corresponding area of the screen. The purpose of this setting is to make the user pay more attention to the resolution of the viewing area when watching the display screen of the head-mounted display device, and therefore pay more attention to the screen of the first display screen. It is not easy to notice the picture of the second display screen.

It should be noted that for the requirement that the size of the image formed by the first display screen on the second display screen is not smaller than the size of the corresponding area of the first viewing screen in the resolution field of view, it is a comprehensive requirement. This is because the size of the image is related to the size of the first display screen body and the magnification or reduction magnification when the optical device is imaging.

In addition to this, in addition to clearly defining the size of the first display screen, there are also clear requirements for the size of the resolution of the first display screen. In an optional embodiment of the present invention, in order to prevent the user from perceiving the screen effect when using the head-mounted display device, the angular resolution of the first display screen needs to be greater than the standard angular resolution.

Specifically, the angular resolution is a pixel point in a unit angle of the field of view of the human eye, and can be divided into two directions: a horizontal direction and a vertical direction. When the user uses the head-mounted display device of the embodiment of the present invention, the most concerned picture is the image formed by the picture of the first display screen on the second display screen. Therefore, the angular resolution of the first display screen should be It is understood as "the angular resolution of the image of the first display screen". Since the distance between the human eye and the display screen of the head-mounted display device has been determined, the field angle of the image has also been determined. After the resolution of the first display screen is determined, "the angular resolution of the image of the first display screen" has also been determined. In an optional embodiment of the present invention, "the angular resolution of the image of the first display screen is required "Ratio" is greater than the standard angular resolution in both the horizontal and vertical directions. The standard angular resolution is the limit of the screen effect that can be perceived by the human eye. Generally, the standard angular resolution is 60 pixels per degree. .

It should be noted that, in order to reduce the high-resolution requirement of the first display screen in the embodiment of the present invention, it is possible to set the image of the first display screen to be imaged on the second display screen with an optical device smaller than the size of the first display screen body. That is, the image of the first display screen is reduced when the optical device is imaging, so that the size of the image in the corresponding area of the first display screen is not smaller than that of the resolution field of view of the user's eyes, and the first display is satisfied. When the condition that the screen ’s angular resolution is greater than the standard angular resolution, the requirement for the resolution of the first display screen is reduced, that is, a relatively low resolution can be used to meet the requirement.

In an optional embodiment of the present invention, in addition to meeting the above requirements, the head-mounted display device further includes an eyeball tracking module, which is configured to obtain the gaze point position information corresponding to the user's line of sight on the display screen, And sending it to the processor, specifically, a mature and commercialized device for implementing a tracking function in the prior art may be selected.

In an optional embodiment of the present invention, the optical components in the optical device 12 may have different configurations. For example, the optical components include at least one of a plane mirror 121, a lens 122, and a prism 123, and the driving device 13 It is configured to adjust a position state of at least one optical component in the optical device 12.

The optical device usually includes at least one of a plane mirror, a lens, and a prism, and may optionally be a combination of the three to realize imaging of the picture of the first display screen on the second display screen, wherein the picture of the first display screen is displayed on the second display. Compared with the screen of the first display screen, the size of the image formed on the screen can be enlarged, reduced, or even the size remains the same. How to select it depends on those skilled in the art. Choose your own.

The driving device is a device for adjusting the position of at least one optical component in the optical device. Its function is to adjust the position of the optical component in the optical device to realize the image of the first display screen on the second display screen. The adjustment of specific positions, specifically how to determine which optical component to adjust, and how to adjust, can be confirmed by those skilled in the art based on optical path analysis, optical design software simulation calculations, or optical experiments. The embodiments of the present invention are not described in detail, and the driving device can be determined by It is realized by various types of micro motors, and the position of each optical component in the optical device is adjusted by the rotation of the motor.

In an optional embodiment of the present invention, the driving device and the eye-tracking module are respectively connected to the processor, if the geometric center of the image formed by the optical device through the image of the first display screen is inconsistent with the position of the gaze point , The processor sends a control instruction to the driving device, so that the driving device adjusts the position of at least one optical component in the optical device until the geometric center coincides with the position of the fixation point, so that the user When using a head-mounted display device according to an embodiment of the present invention, as the user's vision shifts, the position of the image formed on the second display screen by the optical device through the optical device also changes, and due to the The size is not smaller than the size of the corresponding viewing area of the user's eyes in the corresponding area of the first display screen, and the angular resolution of the first display screen is greater than the standard angular resolution, so that the user is using the head-mounted type of the embodiment of the present invention. When displaying the device, the user is paying attention to the picture of the first display screen. The existence of the high-resolution picture makes the user not perceive the screen window. The existence of the effect greatly improves the user experience of the head-mounted display device.

It should be noted that, in the foregoing embodiments of the present invention, the image formed by the optical screen of the first display screen through the optical device has been described all the time. The second display screen has not been described in detail, and the second display screen may use a conventional display. Screen, such as the more conventional 1080P, 2K resolution display.

In an optional embodiment of the present invention, in consideration of the actual display effect, for a complete picture, except for the part displayed on the screen of the first display screen, the remaining parts are displayed on the second display screen. The image of the display screen is perfectly spliced with the image of the second display screen into a complete picture after being imaged by the optical device, and the user cannot feel the intersection of the two pictures.

Or, for a complete picture, except for the part displayed on the first display screen, the complete picture is still displayed on the second display screen, but when the complete picture is displayed to the user, the The image of the screen obstructs the corresponding part of the second display screen, and what the user is most concerned about when viewing is still the image of the screen of the first display screen.

It should be further explained that the screen of the first display screen of the embodiment of the present invention changes as the user's line of sight changes. The specific change method has been described above, and is not described here.

In summary, a head-mounted display device provided by an embodiment of the present invention includes a plurality of display screens, an optical device, a driving device, an eye tracking module, and a processor. The plurality of display screens include a first display screen and A second display screen, wherein the picture of the first display screen is imaged on the second display screen by an optical device, and the size of the image is not smaller than the size of the user's eyes in the corresponding area of the first display screen And the angular resolution of the first display screen is greater than the standard angular resolution, the driving device is configured to adjust a position of at least one optical component in the optical device, and the eye tracking module is configured to obtain a user's line of sight at the The corresponding gaze point position information on the second display screen, the driving device and the eye tracking module are respectively connected to the processor, and if the picture of the first display screen passes the geometric center of the image formed by the optical device and the gaze The point positions are inconsistent, and the processor sends a control instruction to the driving device, so that the driving device adjusts at least one light in the optical device. Learn the position of the component until the geometric center coincides with the position of the fixation point. When the user uses the above-mentioned head-mounted display device, the user pays attention to the picture of the first display screen. The existence of the high-resolution picture makes the user The presence of the screen effect is not perceived, which greatly improves the user experience of the head-mounted display device.

In an optional embodiment of the present invention, the driving device includes a first driving motor and a second driving motor, the first driving motor is configured to control the translation of the at least one optical component, and the second driving motor is provided To control the rotation of the at least one optical component. Since the optical device includes at least one optical component, the optical device may include at least one first driving motor and at least one second driving motor. Each driving motor separately controls the translation or rotation of an optical component, so that the adjustment of the translation or rotation of each optical component of the optical device can be individually adjusted without being affected by each other.

In an optional embodiment of the present invention, as shown in FIG. 2, an output shaft of the first driving motor 131 is connected to a belt transmission mechanism 132, and the belt transmission mechanism 132 is configured to drive the at least one optical component 133 ( The at least one optical component may include a plane mirror 121, a lens 122, and a prism 123) translation, and an output shaft of the second driving motor 134 is connected to at least two rolling bearings, and one of the rolling bearings 135 is configured to drive the at least one optical component 133 at the first Rotate in one direction (for example: front-rear), and the remaining rolling bearing 136 is configured to drive the at least one optical component 133 to rotate in the second direction (for example, left-right).

In an alternative embodiment of the present invention, as shown in FIG. 2, the bottom of the at least one optical component 133 is a hemispherical shape 133 ′, and is mounted in a fixing component 17 in a head-mounted display device. Hemispherical 133 'matching groove 18, rolling bearing 135 is embedded at the bottom of the groove, rolling bearing 136 is embedded at the side of the groove, the output shaft of the second driving motor 134 is set to drive the rolling bearing 135 forward and backward, and the output of the second driving motor 134 The shaft is configured to drive the rolling bearing 136 to rotate left and right.

In an optional embodiment of the present invention, the output shaft of the first drive motor is connected to a screw mechanism, the screw mechanism drives the at least one optical component to translate, and the output of the first drive motor is driven by the screw mechanism The rotation of the shaft is converted into linear motion. The use of the high-precision control characteristics of the screw mechanism can improve the precision of the drive device controlling the optical components of the optical device.

In an optional embodiment of the present invention, in addition to the aforementioned eye tracking module, which uses the mature and commercialized devices in the prior art to implement the tracking function, the eye tracking module may also be integrated in the second display screen and On the array substrate of at least one of the first display screens, that is, the eye tracking module includes a pixel unit provided on the array substrate of at least one of the second display screen and the first display screen, the pixel unit is configured to obtain a user Eyeball image information, and sending the eyeball image information to the processor, the processor being configured to calculate the gaze point position information of the user based on the eyeball image information. Optionally, the eye-tracking module is integrated on the second display screen, and the second display screen is more convenient to obtain the user's eye-ball image information than the first display screen.

It should be noted that the position of the pixel unit on the array substrate for obtaining the user's eyeball image information and the position relationship of the pixel unit on the array substrate used to display the image in the related art (that is, how the two are arranged) are not relevant. limited. In an alternative embodiment of the present invention, FIG. 3 is a schematic diagram showing the distribution of pixel units for displaying images and pixel units for obtaining user eyeball image information on an array substrate according to an exemplary embodiment of the present invention. As shown in FIG. 3, the pixel unit 23 that acquires the user's eyeball image information and the pixel unit 24 for displaying the image are arranged in a mixed manner. Each pixel unit 23 that acquires the user's eyeball image information is disposed in two adjacent ones for displaying the image. Between the pixel units 24. In this solution, the head-mounted display device may have different presentation modes according to a specific application mode, for example, as a VR device.

Of course, the arrangement of the pixel units 23 and the pixel units 24 for displaying images in FIG. 3 is not limited to this, and those skilled in the art can change the arrangement of the pixel units according to their needs.

By arranging the pixel unit 23 between the pixel units 24 for displaying images in the related art, the display screen of the existing VR display device integrates the function of collecting user eyeball image information, and can not increase the existing display. In the case of the area of the array substrate of the screen, more functions are realized.

Moreover, after the display screen of the head-mounted display device integrates the user's eyeball image information collection function, the processor can obtain the sight information of the user's eyes and the gaze point position information of the user's eyes on the display screen according to the user's eyeball image information. How to obtain it specifically, as an example, the pupil corneal vector reflection method can be adopted.

In an optional embodiment of the present invention, the position of the gaze point of the user's eyes on the display screen can be regarded as the intersection point between the line connecting the center of the human eyeball and the center of the human iris and the plane of the display screen. When the human head is stationary, the position of the center of the eyeball of the human eye is unchanged. At this time, the only moving center is the iris center, which is mapped to the eye diagram. The coordinates of the iris center in the eye diagram and the gaze point can be considered. The positions on the display plane are one-to-one. The fixation point estimation method is based on this. Before starting to determine the fixation point of the human eye, let the user fix several calibration points, obtain the coordinates of the iris center in the eye diagram at the corresponding moment, and calculate the relationship between the fixation point and the center of the iris. The mapping relationship can be used later to realize the calibration of the gaze point.

Optionally, a second-order polynomial relationship between the coordinates of the center of the iris and the coordinates of the fixation point can be considered:

X0 = f (xe, ye) = a ₀ + a ₁ xe + a ₂ ye + a ₃ xeye + a ₄ xe2 + a ₅ ye ² ;

Y0 = f (xe, ye) = b ₀ + b ₁ xe + b ₂ ye + b ₃ xeye + b ₄ xe2 + b ₅ ye ² ;

Among them, (X0, Y0) represents the gaze point coordinates, and (xe, ye) represents the coordinates of the iris center in the eye diagram at the corresponding time. After the calibration starts, the user needs to keep the head still, and then follow the prompts to look at these correction points in turn. The computer will record the coordinates of the gaze point of the human eye and the coordinates of the center of the iris in the picture at the corresponding moment, and then use the least square method to calculate the parameters. a ₀ a ₁ a ₂ a ₃ a ₄ a ₅ b ₀ b ₁ b ₂ b ₂ b ₃ b ₄ b ₅ to obtain a mapping function.

It should be noted that the above determination process of the mapping function is a process under the condition that the human head remains unchanged, but this process is suitable for the VR display device to implement the eye tracking process, because when the user wears a head-mounted display When the device is basically worn, each time the position of the user's eyes and the display screen is relatively fixed. Of course, in the implementation process, the determination of the mapping function can still consider the change of the human head relative to the display screen. The user can wear the head-mounted display device several times to obtain the user's eyeballs at different positions relative to the display screen. A mapping function, and then an interpolation algorithm can be used to obtain the mapping function of the user's head at any position.

After obtaining the mapping function, the pixel unit 23 collects the user's eyeball image information in real time. After image processing, the user's eyeball center coordinate and iris center coordinate can be obtained from the eyeball image information. The connection between the two is the user's eyeball's line of sight, and Query the mapping function to obtain the gaze point position information according to the iris center coordinates.

Note that the above are only the preferred embodiments of the present invention and the applied technical principles. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and those skilled in the art can make various obvious changes, readjustments and substitutions without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the concept of the present invention, more equivalent embodiments may be included, and the present invention The scope is determined by the scope of the appended claims.

Claims

A head-mounted display device, the head-mounted display device includes a plurality of display screens, an optical device, a driving device, and an eye tracking module;

The plurality of display screens include a first display screen and a second display screen, and a picture of the first display screen is imaged on the second display screen through the optical device, and the size of the imaged image is not less than the resolution of the user's eyes. The size of the domain in the corresponding area of the first display screen and the angular resolution of the first display screen is greater than the standard angular resolution;

The eyeball tracking module is connected to a processor, and is configured to obtain position information of a gaze point corresponding to a user's line of sight on the second display screen, and send the position information to the processor;

The driving device is connected to the processor, and is configured to receive a control instruction issued by the processor when the geometric center of the image formed by the screen of the first display screen through the optical device is inconsistent with the position of the gaze point, and Adjusting a position state of at least one optical component in the optical device according to the control instruction until the geometric center coincides with the position of the fixation point.
The head-mounted display device according to claim 1, wherein the optical device is configured to reduce a screen of the first display screen when imaging on the second display screen.
The head-mounted display device according to claim 1, wherein the driving device includes a first driving motor and a second driving motor, the first driving motor is configured to control translation of the at least one optical component, and The second driving motor is configured to control rotation of the at least one optical component.
The head-mounted display device according to claim 3, wherein an output shaft of the first drive motor is connected to a belt transmission mechanism, the belt transmission mechanism is configured to drive the at least one optical component to translate, and the second The output shaft of the driving motor is connected to at least two rolling bearings, one of which is configured to drive the at least one optical component to rotate in the first direction, and the remaining rolling bearing is configured to drive the at least one optical component to rotate in the second direction, The second direction is different from the first direction.
The head-mounted display device according to claim 4, wherein an output shaft of the first drive motor is connected to a screw mechanism, and the screw mechanism is configured to drive the at least one optical component to translate.
The head-mounted display device according to any one of claims 1 to 5, wherein the eye tracking module includes a pixel unit provided on an array substrate of at least one of the second display screen and the first display screen, The pixel unit is configured to acquire user eyeball image information and send the eyeball image information to the processor, and the processor is configured to calculate the gaze point position information according to the eyeball image information.
The head-mounted display device according to any one of claims 1 to 5, wherein the standard angular resolution is 60 pixels / degree.
The head-mounted display device according to claim 6, wherein the processor is configured to determine an iris center coordinate of a user's iris center in a preset coordinate system according to the eyeball image information, and according to the iris center coordinate Query the preset iris center coordinate and gaze point position information mapping query to obtain the gaze point position information that matches the iris center coordinate.
The head-mounted display device according to claim 1, wherein the optical component includes at least one of a flat mirror, a lens, and a prism;

The driving device is configured to adjust a position state of at least one of the optical components according to the control instruction.