TW202036081A - Near-eye display device - Google Patents

Abstract

A near-eye display device including at least one display, at least one lens set, and at least one camera module is provided. The display is configured to emit an image beam and a structured light. The lens set is disposed on paths of the image beam and the structured light, and configured to converge the image beam on an eye of a user, so that the eye observes a virtual image. The structured light irradiates the eye through the lens set to form a light pattern. The camera module is configured to photograph the light pattern. The light pattern photographed by the camera is configured to analyze a distance of the eye.

Description

Near-eye display device

The present invention relates to an optical device, and particularly relates to a near-eye display device.

Near-eye display (near-eye display) can generally be divided into virtual reality display (virtual reality display, VR display) and augmented reality display (augmented reality display, AR display), which have developed vigorously in recent years. It is different from the previous visual experience.

The current virtual reality display software can adjust the image according to the user’s inter-pupillary distance (IPD) and eye relief (eye relief) to display the correct display content, but it requires additional It can be achieved by installing an independent and complete depth sensing module. However, adding an independent and complete depth sensing module will increase the cost. If no additional depth sensing module is added, the image calibration will consume a lot of time and need to be calibrated for different users, and even for different accessories worn by the user (such as prescription glasses or hats). .

In addition, some head mounted displays (HMD) use a single sensor for eye tracking and proximity detection. However, this single sensor is not sufficient to detect accurate depth, and therefore cannot provide the pupillary distance and eye-fitting distance of the eyes.

The present invention provides a near-eye display device, which can reduce the cost without sacrificing the accuracy of human eye sensing.

An embodiment of the present invention provides a near-eye display device, which includes at least one display, at least one lens group, and at least one camera module. The display is used to emit an image beam and a structured light. The lens group is arranged on the transmission path of the image beam and the structured light to converge the image beam on the eyes of a user so that the eyes can observe a virtual image. The structured light is irradiated on the eyes through the lens group to form a light pattern, and the camera module is used to photograph the light pattern, and the light pattern captured by the camera module is used to analyze the distance of the eye.

In the near-eye display device of the embodiment of the present invention, since the image-providing display is used as the structured light emission source, the camera module does not need to use additional light sources to provide structured light. In this way, the cost can be reduced without sacrificing the accuracy of human eye sensing, and the size of the near-eye display device can be reduced.

In order to make the above-mentioned features and advantages of the present invention more comprehensible, the following specific embodiments are described in detail in conjunction with the accompanying drawings.

1 is a schematic diagram of the optical path of a near-eye display device according to an embodiment of the present invention, FIG. 2A is a display screen corresponding to a structured light displayed on the display of the near-eye display device in FIG. 1, and FIG. 2B is the near-eye display device in FIG. The display corresponding to another structured light display. Fig. 3 is a system block diagram of the near-eye display device of Fig. 1. Please refer to FIGS. 1, 2A, 2B and 3, the near-eye display device 100 of this embodiment includes at least one display 110, at least one lens group 120 and at least one camera module 130. In this embodiment, as shown in FIG. 3, two displays 110a and 110b respectively corresponding to the left and right eyes of the user are taken as an example, and FIG. 1 only schematically shows one display 110 as a representative. In addition, in this embodiment, two lens groups 120 respectively corresponding to the user's left eye and right eye are taken as an example, and FIG. 1 only schematically illustrates one lens group 120 as a representative. Furthermore, in this embodiment, as shown in FIG. 3, two camera modules 130a and 130b respectively corresponding to the user's left eye and right eye are taken as an example, and FIG. 1 only schematically shows one camera module. Group 130 is represented.

The display 110 is used for emitting an image beam 111 and a structured light 113. In this embodiment, the display 110 is, for example, a liquid crystal display (LCD), an organic light-emitting diode display (OLED display), or a micro-LED display. Or the liquid-crystal-on-silicon panel (LCOS panel), digital micro-mirror device (DMD) or other spatial light modulator (spatial light modulator, SLM).

The lens group 120 is disposed on the transmission path of the image beam 111 and the structured light 113 to converge the image beam 111 on the user's eye 50 so that the eye can observe a virtual image. The lens group 120 may include one or more lenses. In this embodiment, the lens group 120 includes one lens as an example. The structured light 113 is irradiated on the eye 50 through the lens group 120 to form a light pattern on the eye 50, and the camera module 130 is used to photograph the light pattern. The light pattern captured by the camera module 130 is used to analyze the distance of the eye 50. For example, when the display 110 provides the structured light 113, the display 110 can present a display image as shown in FIG. 2A, and through the projection of the lens group 120, a light pattern corresponding to the display image is formed on the eyes. Compared with the original pattern on the display screen, the light pattern will deform with the unevenness and distance on the surface of the eye. By analyzing the direction of the deformation and the degree of the deformation, the distance of the eye 50 (for example, the eye 50 to the distance of the lens group 120).

The near-eye display device 100 is, for example, a head-mounted display. In the near-eye display device 100 of this embodiment, since the image-providing display 110 is used as the structured light 113 emission source, the camera module 130 does not need to use an additional light source to provide the structured light 113. In this way, the cost can be reduced without sacrificing the accuracy of human eye sensing, and the volume of the near-eye display device 100 can be reduced.

In this embodiment, the near-eye display device 100 may further include a driver 140 for driving the display 110 to display images. Specifically, in a display mode, the driver 140 can drive the display 110 to display a picture corresponding to the virtual image for the user to view, that is, the display 110 emits an image beam 111. On the other hand, in a distance measurement mode, the driver 140 can drive the display 110 to display a screen as shown in FIG. 2A or FIG. 2B, that is, the display 110 emits structured light 113 to form a structure corresponding to the display 110 on the user's eyes 50 The light pattern of the displayed image is taken by the camera module 130. When the driver 140 drives the display 110 to provide structured light 113 or finishes providing structured light 113, the driver 140 is used to send a notification signal S1 to the camera module 130, and the camera module 130 activates the camera function according to the notification signal S1 to capture the structured light 113 The light pattern formed, or the end of the camera function.

In this embodiment, the near-eye display device 100 further includes at least one image processor 160, which is electrically connected to the aforementioned at least one camera module 130, and calculates the distance and the distance of the eye 50 according to the light pattern captured by the camera module 130 The position, for example, can calculate the three-dimensional coordinates of the eye 50 or its pupil in space. The pixels of the camera module 130 can be used to detect visible light or infrared light, and the display 110 can emit visible light or infrared light, that is, the structured light 113 can be visible light or infrared light. In this embodiment, as shown in FIG. 3, two image processors 160 are taken as an example, and the two image processors 160 are electrically connected to the camera module 130a corresponding to the left eye and the camera module 130a corresponding to the right eye. The camera module 130b. In this embodiment, the operation of the image processor 160 corresponding to the left eye can obtain the three-dimensional coordinates of the left eye or its pupil in space, and the operation of the image processor 160 corresponding to the right eye can obtain the right eye or its pupil. Three-dimensional coordinates in space. The image processor 160 is, for example, an image signal processor (ISP) or an image processor (graphics processing unit, GPU).

In this embodiment, the near-eye display device 100 further includes a processor 190, which is electrically connected to the above-mentioned two image processors 160 corresponding to the left eye and the right eye, and according to the two image processors 160 provided The distance and position of the left eye and the distance and position of the right eye (for example, the three-dimensional coordinates of the left eye or its pupil and the three-dimensional coordinates of the right eye or its pupil) are used to calculate the pupil distance and the eye-fitting distance of the user. Among them, the eye-fitting distance refers to the distance between the eye 50 and the lens group 120. According to the calculated pupillary distance and eye-fitting distance of the two eyes, the processor 190 can adjust the display content of the display 110 or adjust the focal length of the lens group 120 to produce a proper and clear image corresponding to the pupillary distance and eye-fitting distance of the two eyes. The virtual image of, and produce correct and good image content.

In this embodiment, the processor 190 is, for example, a central processing unit (CPU), a microprocessor (microprocessor), a digital signal processor (digital signal processor, DSP), a programmable controller, and a programmable controller. A logic device (programmable logic device, PLD) or other similar devices or a combination of these devices is not limited by the present invention. In addition, in this embodiment, the functions of the processor 190 can be implemented as multiple program codes. These program codes are stored in a memory, and the processor 190 executes these codes.

In this embodiment, the display 110 sequentially emits a variety of different structured lights 113 (for example, the structured light 113 corresponding to the picture in FIG. 2A and the structured light 113 corresponding to the picture in FIG. 2B in sequence) to be on the eye 50 A variety of different light patterns are formed (for example, the light pattern corresponding to the screen of FIG. 2A and the light pattern corresponding to the screen of FIG. 2B). The image processor 160 calculates the distance and position of the eye 50 according to various light patterns captured by the camera module 130 (for example, calculates the three-dimensional coordinates of the eye or its pupil in space). When the camera module 130 sequentially shoots a variety of different light patterns, the image processor 160 can calculate the distance and position of the eye 50 more accurately. However, when the requirements for the accuracy of the distance and position of the eyes 50 are relatively low, the display 110 can also emit only one kind of structured light 113, and the image processor 160 can use a light pattern captured by the camera module 130. Calculate the distance and position of the eyes 50.

In FIGS. 2A and 2B, the two different images corresponding to the two different structured lights 113 are based on the stripe images in different directions respectively. This stripe image may include regular or equal width stripes, or Including irregularly arranged or unequal width stripes. In addition, the picture corresponding to the structured light 113 may also be a dot array picture or an irregularly arranged multi-dot picture, and the sizes of multiple dots in the picture may be the same or different. Alternatively, the picture corresponding to the structured light 113 may also be any suitable shape or any suitable arrangement of pictures, as long as it is convenient for the image processor 160 to analyze the distance and position of the eye 50.

In summary, in the near-eye display device of the embodiment of the present invention, since the image-providing display is used as the structured light emission source, the camera module does not need to use additional light sources to provide structured light. In this way, the cost can be reduced without sacrificing the accuracy of human eye sensing, and the size of the near-eye display device can be reduced.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the technical field can make some changes and modifications without departing from the spirit and scope of the present invention. The scope of protection of the present invention shall be determined by the scope of the attached patent application.

50: eyes 100: Near-eye display device 110, 110a, 110b: display 111: image beam 113: structured light 120: lens group 130, 130a, 130b: camera module 140: drive 160: image processor 190: processor S1: Notification signal

FIG. 1 is a schematic diagram of an optical path of a near-eye display device according to an embodiment of the present invention. 2A is a display screen corresponding to a structured light displayed on the display of the near-eye display device in FIG. 1. 2B is a display screen corresponding to another structured light displayed on the display of the near-eye display device in FIG. 1. Fig. 3 is a system block diagram of the near-eye display device of Fig. 1.

50: eyes

100: Near-eye display device

110: display

111: image beam

113: structured light

120: lens group

130: camera module

Claims (6)

A near-eye display device, including: At least one display for emitting an image beam and a structured light; At least one lens group is arranged on the transmission path of the image beam and the structured light, and is used for condensing the image beam on the eyes of a user, so that the eyes can observe a virtual image; and At least one camera module, wherein the structured light is irradiated on the eye through the lens group to form a light pattern, and the camera module is used for photographing the light pattern, and the light pattern photographed by the camera module is used for Analyze the distance of the eye. The near-eye display device described in claim 1 further includes at least one image processor, which is electrically connected to the at least one camera module, and calculates the light pattern according to the light pattern captured by the camera module. The distance and position of the eyes. According to the near-eye display device described in claim 2, wherein the at least one image processor is an image signal processor or an image processor. According to the near-eye display device described in claim 2, wherein the at least one display is two displays, the at least one lens group is two lens groups, the at least one camera module is two camera modules, and the two The two structured lights emitted by a display are irradiated to the left eye and right eye of the user through the two lens groups to form two light patterns respectively, and the two camera modules are used to photograph the two lights respectively Pattern, the at least one image processor is two image processors electrically connected to the two camera modules, and the near-eye display device further includes a processor electrically connected to the two image processors, and according to The distance and position of the left eye and the distance and position of the right eye provided by the two image processors are used to calculate the interpupillary distance and eye-fitting distance of the user. The near-eye display device according to claim 1, wherein the display sequentially emits a plurality of different structured lights to form a plurality of different light patterns on the eye, and the near-eye display device further includes at least one image processor , Electrically connected to the at least one camera module, and calculating the distance and position of the eye according to the multiple different light patterns captured by the camera module. For example, the near-eye display device described in claim 1 further includes a driver, which is electrically connected to the display, and used to drive the display to display images, and provide the image beam and the structured light, wherein when the driver drives When the display provides the structured light or ends the provision of the structured light, the driver is used to send a notification signal to the camera module, and the camera module starts or ends the photographing function according to the notification signal.

Images