TW202136861A - System and method for displaying an object with depths - Google Patents

Abstract

Disclosed are systems and methods for displaying an object. The object displaying system includes a right light signal generator, a left light signal generator, a right combiner, and a left combiner. The right light signal generator generates multiple right light signals for an object. The right combiner receives and redirects the multiple right light signals towards one retina of a viewer to display multiple right pixels of the object. The left light signal generator generates multiple left light signals for the object. The left combiner receives and redirects the multiple left light signals towards the other retina of the viewer to display multiple left pixels of the object. In addition, a first redirected right light signal and a corresponding first redirected left light signal are perceived by the viewer to display a first virtual binocular pixel of the object with a first depth that is related to a first angle between the first redirected right light signal and the corresponding first redirected left light signal.

Description

System and method for displaying an object with depth of field

This case relates to a system and method for displaying objects with depth of field, in particular, by generating a plurality of right light signals and left light signals and redirecting a plurality of right light signals and left light signals to the user's retina to display objects with depth of field Systems and methods.

In traditional Virtual Reality (VR) and Augmented Reality (AR) systems, the stereoscopic technology used is to simultaneously project two parallax images with different viewing angles to the eyes of neighboring users. The left and right display panels are used to generate 3D virtual images. The viewing angle difference between the two parallax images is interpreted by the brain as the depth of field of the image. In fact, the user's eyes are focused (gazing) on the display panel; due to the parallax image, the depth of field of the display panel is different from the depth of field of the image seen by the user. When the focus adjustment of the depth perception is inconsistent with the eye convergence, a vergence-accommodation conflict (VAC) occurs. VAC makes users dizzy or headache. Furthermore, when using parallax images in Mixed Reality (MR), users will not be able to focus on the real object and the virtual image at the same time, so there is a focus rivalry (focal rivalry). Furthermore, displaying the movement of the virtual image through the parallax image technology is a heavy burden for the image processing hardware equipment.

The purpose of this case is to provide a system and method for displaying objects with depth of field in space. Since the depth of field of the object is the same as the position where the user's eyes are gazing, it is possible to avoid vergence-accommodation conflict (VAC) and focus competition (focal rivalry). The object display system includes a right optical signal generator, a right combiner, a left optical signal generator, and a left combiner. The right optical signal generator generates a plurality of right optical signals of the object. The right combiner receives and redirects a plurality of right light signals toward the user's retina to display a plurality of right pixels of the object. The left optical signal generator generates a plurality of left optical signals of the object. The left combiner receives and redirects the plurality of left optical signals toward the other retina of the user to display the plurality of left pixels of the object. In addition, the user perceives the right light signal of the first redirection and the corresponding left light signal of the first redirection, and generates the first virtual binocular pixel of the object with the first depth of field, and the first depth of field is the same as the first redirection. The first angle between the right optical signal of and the corresponding left optical signal of the first redirection is related. In one embodiment, the first depth of field is determined by the first angle between the first redirected right light signal and the corresponding first redirected left light signal light path extension.

When an object is considered to have a plurality of depths of field, for the first virtual binocular pixel of the object, the user perceives the right light signal of the second redirection and the corresponding left light signal of the second redirection, with the second depth of field A second virtual binocular pixel of the object is generated, and the second depth of field is related to a second angle between the second redirected right light signal and the corresponding second redirected left light signal.

Furthermore, the right optical signal of the first redirection is not the parallax of the corresponding left optical signal of the first redirection. In this case, both the right eye and the left eye receive the image of the object from the same viewing angle, instead of the traditional 3D image generation method of parallax between the right eye and the left eye.

In another embodiment, the first redirected right light signal and the corresponding first redirected left light signal have substantially the same height on the retina of the user's eyes.

In another embodiment, the plurality of right optical signals generated by the right optical signal generator are reflected only once before entering the user's retina, and the plurality of left optical signals generated by the left optical signal generator The signal is reflected only once before entering the user's retina.

In one embodiment, the right combiner receives and redirects a plurality of right optical signals toward the user's right retina to display a plurality of right pixels of an object, and the left combiner receives and redirects a plurality of left optical signals toward the user's left The retina displays the plural left pixels of the object. In another embodiment, the right combiner receives and redirects a plurality of left optical signals toward the user's right retina to display a plurality of right pixels of the object, and the left combiner receives and redirects a plurality of right optical signals toward the user The left retina to display the plural left pixels of the object.

In augmented reality (AR) or mixed reality (MR) applications, the right combiner and the left combiner are transparent to ambient light.

In addition to AR and MR applications, the object display system further includes a support structure that can be worn by the user's head. The right optical signal generator, the left optical signal generator, the right combiner, and the left combiner are mounted on the supporting structure. In one embodiment, the support structure wearable by the user's head is glasses. In this case, the supporting structure can be spectacles with or without lenses. The lens can be a lens with power to correct nearsightedness, farsightedness, etc.

In the embodiment of the smart glasses, the right optical signal generator can be mounted on the right temple of the spectacle frame and the left optical signal generator can be mounted on the left temple of the spectacle frame. In addition, the right side combiner can be loaded on the right lens and the left side combiner can be loaded on the left lens. There are many different implementations of loading methods. The combiner can be joined or integrated into the lens by detachable or non-detachable means. In addition, the combiner can be integrally formed with the lens, and the lens includes a lens with power.

This project uses retinal scanning technology to project the right light signal and the left light signal to the user's retina, which is different from the near-eye display that is usually placed quite close to the user's eyes.

Other features or advantages of this case will be explained more clearly in the following narratives. Those with ordinary knowledge in this field can also learn more about this case by implementing this case. The purpose and advantages of this case are represented by the structure and methods indicated in the description, scope of rights and diagrams of this case. The following descriptions are all examples, and are intended to explain the scope of rights.

The vocabulary used in this article is used to describe the details of a specific embodiment of the present invention, and all vocabulary should be interpreted reasonably in the largest category. Certain vocabulary will be specifically emphasized below; any restrictive terms will be defined in the specific examples.

The invention relates to a system and method system for displaying an object with depth of field in space. Since the depth of field of the object is the same as the position where the user's eyes are gazing, it is possible to avoid vergence-accommodation conflict (VAC) and focus competition (focal rivalry). The embodiments described herein involve one or more methods, systems, devices, and computer-readable media storing processor-executable steps to display an object with a depth of field in space. The object display system has a right optical signal generator, a right combiner, a left optical signal generator, and a left combiner. The right optical signal generator generates a plurality of right optical signals of the object. The right combiner receives and redirects a plurality of right light signals toward a plurality of right pixels of a retina display object of the user. The left optical signal generator generates a plurality of left optical signals of the object. The left combiner receives and redirects a plurality of left optical signals toward another user's retina to display a plurality of left pixels of the object. In addition, the user perceives the right light signal of the first redirection and the corresponding left light signal of the first redirection to display the first virtual binocular pixels of the object with the first depth of field, and the first depth of field and the first redirected right light signal The first angle correlation between the signal and the corresponding first redirected left optical signal. In an embodiment, the first depth of field is determined by the first angle between the optical path extension of the first redirected right light signal and the corresponding first redirected left light signal.

The user can perceive the position of the object at multiple depths of field. In addition to the first virtual binocular pixel of the object, the user perceives the right light signal of the second redirection and the corresponding left light signal of the second redirection to display at the second depth of field For the second virtual binocular pixel of the object, the second depth of field is related to the second angle between the second redirected right light signal and the corresponding second redirected left light signal.

Furthermore, the right optical signal of the first redirection is not the parallax of the corresponding left optical signal of the first redirection. The right eye and the left eye receive the image of the object from the same viewing angle, instead of receiving the parallax that traditionally produces 3D images from the right eye view and the left eye view.

In another embodiment, the first redirected right light signal and the corresponding first redirected left light signal have substantially the same height on the retina of the user's eyes.

In another embodiment, the plurality of right optical signals generated by the right optical signal generator are reflected only once before entering the user’s retina, and the plurality of left optical signals generated by the left optical signal generator enter the The user's retina was only reflected once before.

In one embodiment, the right combiner receives and redirects a plurality of right optical signals toward the user's right retina to display a plurality of right pixels of the object, and the left combiner receives and redirects a plurality of left optical signals toward the user's right retina. The left retina displays the plural left pixels of the object. In another embodiment, the right combiner receives and redirects a plurality of left optical signals toward the user's right retina to display a plurality of right pixels of the object, and the left combiner receives and redirects a plurality of right optical signals toward the user The left retina of the person can display a plurality of left pixels of the object.

In augmented reality (AR) or mixed reality (MR) applications, the right combiner and the left combiner are transparent to ambient light.

In AR and MR applications, the object display system further includes a support structure that can be worn on the user's head. The right optical signal generator, the left optical signal generator, the right combiner, and the left combiner are mounted on the supporting structure. In the embodiment, the system in this case is a head wearable device, especially glasses. In this situation, the supporting structure may be a spectacle frame with or without spectacle lenses. The lens can be a power lens to correct nearsightedness or hyperopia...etc.

In the embodiment of the smart glasses, the right optical signal generator can be mounted on the right temple of the spectacle frame and the left optical signal generator can be mounted on the left temple of the spectacle frame. In addition, the right side combiner can be loaded on the right lens and the left side combiner can be loaded on the left lens. There are many different implementations of loading methods. The combiner can be attached to or integrated into the lens by detachable or non-detachable means. In addition, the combiner can be integrally formed with the lens, and the lens includes a lens with power.

As shown in Fig. 1, the object display system includes a right light signal generator 10 for generating a plurality of right light signals (for example, 12 of RLS_1, 14 of RLS_1 and 16 of RLS_3), and a right combiner 20 for receiving and redirecting the plural light signals. The right light signals are directed toward the user’s right retina 54. The left light signal generator 30 is used to generate a plurality of left light signals (such as 32 of LLS_1, 34 of LLS_2, and 36 of LLS_3), and the left combiner 40 is used to receive And redirect the plurality of left light signals toward the left retina 64 of the user. The user has a right eye 50 and includes a right pupil 52 and a right retina 54, and the left eye 60 includes a left pupil 62 and a left retina 64. Generally speaking, depending on the intensity of ambient light, the diameter of a human pupil is between 2 to 8 mm. In a strong light environment, the diameter of an adult's normal pupil varies from 2 to 4mm, while in the dark, the diameter of an adult's normal pupil varies from 4 to 8mm. The plurality of right light signals are redirected by the right combiner 20, pass through the right pupil 52, and are finally received by the right retina 54. The right optical signal RLS_1 is the rightmost optical signal that the user's right eye can see on a certain horizontal plane. The right light signal RLS_2 is the leftmost light signal that the user's right eye can see on a certain horizontal plane. When receiving the redirected right light signal, the user can see a plurality of right pixels of the object in the area A defined by the extension of the redirected right light signal RLS_1 and RLS_2. The area A may be referred to as the field of view (FOV) of the right eye 50. Similarly, a plurality of left light signals are redirected by the left combiner 40, pass through the left pupil 62, and are finally received by the left retina 64. The left light signal LLS_1 is the rightmost light signal that the user's left eye can see on a certain horizontal plane. The left light signal LLS_2 is that the user's left eye can see the leftmost light signal on a certain horizontal plane. When receiving the redirected left light signal, the user can see a plurality of left pixels of the object in the area B defined by the extension of the redirected left light signal LLS_1 and LLS_2. The area B may be referred to as the field of view (FOV) of the left eye 60. When a plurality of right pixels and left pixels are displayed in area C (the area where area A and area B overlap), at least one right light signal showing a right pixel and a corresponding left light signal showing a left pixel are fused to be in the area A virtual binocular pixel with depth of field is displayed in C. The depth of field is related to the angle of the redirected right light signal and the redirected left light signal. This angle is also called the convergent angle.

右瞳孔52與左瞳孔62之間的距離為瞳距(interpupillary distance/IPD)。相同的，第二重新導向的右側光訊號18與對應的第二重新導向的左側光訊號28之間的第二角度為Ɵ2。第二景深D2D1與第二角度Ɵ2相關。尤其，物體的第二虛擬雙目像素的第二景深D2可由第二重新導向的右側光訊號與對應的第二重新導向的左側光訊號光徑延伸之間的第二角度Ɵ2決定。使用者感受到第二虛擬雙目像素74比第一虛擬雙目像素72更遠離使用者(意即，具有較大的景深)，第二角度Ɵ2小於第一角度θ1。As shown in FIGS. 1 and 2, a virtual image of a dinosaur object 70 with multiple depths of field can be seen in the area C in front of the user. The image of the dinosaur object 70 includes a first virtual binocular pixel 72 displayed at the first depth of field D1 and a second virtual binocular pixel 74 displayed at the second depth of field D2. The first angle between the first redirected right optical signal 16' and the corresponding first redirected left optical signal 26' is Ɵ1. The first depth of field D1 is related to the first angle θ1. In particular, the first depth of field D1 of the first virtual binocular pixel of the object can be determined by the first angle θ1 between the light path extension of the first redirected right light signal and the corresponding first redirected left light signal. Therefore, the first depth of field D1 of the first virtual binocular pixel 72 can be calculated by the following formula:

The distance between the right pupil 52 and the left pupil 62 is interpupillary distance (IPD). Similarly, the second angle between the second redirected right optical signal 18 and the corresponding second redirected left optical signal 28 is Ɵ2. The second depth of field D2D1 is related to the second angle Ɵ2. In particular, the second depth of field D2 of the second virtual binocular pixel of the object can be determined by the second angle Ɵ2 between the light path extension of the second redirected right light signal and the corresponding second redirected left light signal. The user feels that the second virtual binocular pixel 74 is farther away from the user than the first virtual binocular pixel 72 (that is, it has a larger depth of field), and the second angle Ɵ2 is smaller than the first angle θ1.

Furthermore, although the right light signal 16' of the redirected RLG_2 and the left light signal 26' of the corresponding redirected LLS_2 together display the first virtual binocular pixel 72 with the first depth of field D1, the redirected right light signal of RLG_2 The signal 16' is not the parallax of the redirected left optical signal 26' of the corresponding LLS_2. Traditionally, since the right eye views the object from a different perspective from the left eye, the parallax between the image received by the right eye and the image received by the left eye is used to allow the user to experience a 3D image with depth of field. However, in the present invention, the right optical signal of the virtual binocular pixel and the corresponding left optical signal display images with the same viewing angle. Therefore, the light intensity of red, blue, and green (RBG) and/or the brightness of the right light signal and the left light signal are approximately the same. In other words, the right pixel is substantially the same as the corresponding left pixel. However, in another embodiment, one or both of the right light signal and the left light signal can be adjusted to present other 3D effects such as shadows. Generally speaking, in the present invention, both the right eye and the left eye receive the image of the object from the same viewing angle, instead of the parallax between the right eye view and the left eye view in the traditional way of generating 3D images. As described above, a plurality of right light signals are generated by the right light signal generator 10, redirected by the right combiner 20, and scanned directly to the right retina to form a right retina image on the right retina. Similarly, a plurality of left optical signals are generated by the left optical signal generator 30, redirected by the left combiner 40, and scanned directly to the left retina to form a left retina image on the left retina. In one embodiment, as shown in FIG. 2, the right retinal image 80 includes 36 right pixels (6x6 matrix) and the left retinal image 90 also includes 36 left pixels (6x6 matrix). In another embodiment, the right retinal image 80 includes 921,600 right pixels (1280x720 matrix) and the left retinal image 90 also includes 921,600 left pixels (1280x720 matrix). The object display system is configured to generate a plurality of right light signals and a corresponding plurality of left light signals, each of which forms a right retina image on the right retina and a left retina image on the left retina. Therefore, due to the image fusion, the user can see a virtual binocular object with a specific depth of field in the area C.

As shown in FIG. 2, the first right optical signal 16 from the right optical signal generator 10 is received by the right combiner 20 and reflected. The first redirected right light signal 16' passes through the right pupil 52 and reaches the user's right retina to display the right pixel R34. The corresponding left optical signal 26 from the left optical signal generator 30 is received by the left combiner 40 and reflected. The first redirected light signal 26' passes through the left pupil 62 and reaches the left retina of the user to display the left pixel L33. Due to image fusion, the user can see virtual binocular objects with multiple depths of field, which is determined by the angle between the multiple redirected right light signals of the same object and the corresponding multiple redirected left light signals . The angle between the redirected right light signal and the corresponding left light signal is determined by the relative horizontal distance between the right pixel and the left pixel. Therefore, the depth of field of the virtual binocular pixel is inversely proportional to the relative horizontal distance between the right pixel forming the virtual binocular pixel and the corresponding left pixel. In other words, the virtual binocular pixel has a deeper depth of field for the user, and the relative horizontal distance between the right pixel and the left pixel forming the virtual binocular pixel is smaller. For example, as shown in FIG. 2, the second virtual binocular pixel 74 has a deeper depth of field (that is, farther from the user) than the first virtual binocular pixel 72 for the user. Therefore, on the retina, the horizontal distance between the second right pixel and the second left pixel is smaller than the horizontal distance between the first right pixel and the first left pixel. In particular, the horizontal distance between the second right pixel R41 and the second left pixel L51 forming the second virtual binocular pixel is a length of 4 pixels. However, the distance between the first right pixel R43 and the first left pixel L33 forming the first virtual binocular pixel is 6-pixel length.

In the embodiment shown in FIG. 3, it illustrates the optical paths of the plurality of right optical signals and the plurality of left optical signals from the optical signal generator to the retina. A plurality of right light signals generated by the right light generator are projected to the right combiner 20 to form a right combiner image (RCI) 82. These plural right light signals are redirected by the right combiner 20 and converge into a small right pupil image (RPI) 84 to pass through the right pupil 52 and finally reach the right retina 54 to form a right retinal image (RRI) 86. RCI, RPI, and RRI include ixj pixels. Each right light signal RLS(i,j) passes through the same corresponding pixel from RCI(i,j) to RPI(i,j) and arrives at RRI(x,y). For example, RLS(5,3) goes from RCI(5,3) to RPI(5,3) and arrives at RRI(2,4). Similarly, a plurality of left light signals generated by the left light generator 30 are projected to the left combiner 40 to form a left combiner image (LCI) 92. These plural left light signals are redirected by the left combiner 40 and converge into a small left pupil image (LPI) 94 to pass through the left pupil 62 and finally reach the left retina 64 to form a right retinal image (LRI) 96. Each of LCI, LPI, and LRI includes ixj pixels. Each left light signal LLS(i,j) passes through the same corresponding pixel from LCI(i,j) to LPI(i,j) and arrives at LRI(x,y). For example, LLS(3,1) goes from LCI(3,1) to LPI(3,1) and arrives at LRI(4,6). Pixel (0,0) is the upper left pixel of each image. The pixels in the retinal image are left and right opposite to the corresponding pixels in the combiner image, and upside down. Depending on the appropriate relative position and angle of the optical signal generator and the combiner, each optical signal has its own optical path from the optical signal generator to the retina. A right light signal that displays a right pixel on the right retina is combined with a left light signal that displays a left pixel on the left retina to form a virtual binocular pixel with a specific depth of field. Therefore, the virtual binocular pixels in a space can be represented by a pair of right pixels and left pixels or a pair of right combiner 20 pixels and left combiner 40 pixels.

The virtual object seen by the user in the area C includes a plurality of virtual binocular pixels. In order to accurately describe the position of the virtual binocular pixel in space, each position in the space is represented by three-dimensional (3D) coordinates, such as XYZ coordinates. In other embodiments, different 3D coordinate systems can also be used. Therefore, each virtual binocular pixel has 3D coordinates—horizontal direction, vertical direction, and depth of field direction. The horizontal direction (or X-axis direction) is the direction along the interpupillary line. The vertical direction (or Y-axis direction) is along the facial mid line and perpendicular to the horizontal direction. The depth of field direction (or Z-axis direction) is orthogonal to the frontal surface and perpendicular to the horizontal and vertical directions.

Figure 4 shows the relationship between pixels in the right combiner image, pixels in the left combiner image, and virtual binocular pixels. As mentioned above, there is a one-to-one relationship between the pixels in the right combiner image and the pixels in the right retinal image (right pixels). The pixels in the left combiner image have a one-to-one relationship with the pixels in the left retina image (left pixels). However, the pixels in the retinal image and the corresponding pixels in the combiner image are left and right, and upside down. For the right retinal image containing 36 (6x6) right pixels and the left retinal image containing 36 (6x6) left pixels, assuming that the light signal is in the FOV of the user's eyes, there are a total of 216 (6x6x6) virtual Binocular pixels (displayed in dot matrix). The light path extension of a redirected right light signal intersects the light path extension of each redirected left light signal in the same row in the image. Similarly, the light path extension of a redirected left light signal intersects the light path extension of each redirected right light signal in the same row in the image. Therefore, there are 36 (6x6) virtual binocular pixels in one layer, and there are a total of 6 layers in space. A small angle between two adjacent lines usually means that the light path extends and intersects and forms a virtual binocular pixel (although it is shown as parallel in FIG. 4). Each retina has a right pixel and a corresponding left pixel (that is, the right retinal image and the left retinal image in the same row) that have approximately the same height and are first fused with each other. Therefore, the right and left pixels in the same row in the retinal image are paired with each other to form a virtual binocular pixel.

As shown in Fig. 5, a look-up table is used to easily find the right pixel and left pixel pair of each virtual binocular pixel. For example, 216 virtual binocular pixels, numbered 1 to 216, consist of 36 (6x6) right pixels and 36 (6x6) left pixels. The first (1st) virtual binocular pixel VBP(1) represents a pair of the right pixel RRI(1,1) and the left pixel LRI(1,1). The second (2nd) virtual binocular pixel VBP(2) represents a pair of the right pixel RRI(2,1) and the left pixel LRI(1,1). The seventh (7th) virtual binocular pixel VBP(7) represents a pair of the right pixel RRI(1,1) and the left pixel LRI(2,1). The 37th (37th) virtual binocular pixel VBP (37) represents a pair of the right pixel RRI(1,2) and the left pixel LRI(1,2). The 216th (216th) virtual binocular pixel VBP (216) represents a pair of the right pixel RRI (6, 6) and the left pixel LRI (6, 6). Therefore, in order to display a specific virtual binocular pixel of an object in space, it is necessary to determine which right pixel and left pixel pair can be used to generate the corresponding right light signal and left light signal. In addition, each row of virtual binocular pixels on the lookup table includes a pointer to a memory address that stores the depth (z) of the VBP and the position (x, y) of the VBP. Other data, such as the size of the VBP ratio, the number of overlapping objects, and the continuous depth of field... etc. can be stored. The ratio size can be the relative size data of the comparison between the specific VBP and the standard VBP. For example, when the object is displayed one meter in front of the user in standard VBP, the scale can be set to 1. Therefore, the ratio of the specific VBP 90 cm in front of the user can be set to 1.2. Similarly, the ratio of the specific VBP 1.5m in front of the user can be set to 0.8. When the object moves from the first depth of field D1 to the second depth of field D2, the scale can be used to determine the size of the displayed object. The number of object overlaps is the number of multiple objects overlapping each other (the object is completely or partially hidden behind another object). The sequence of depth of field provides the sequence data of the depth of field of each overlapping object. For example, take 3 objects overlapping each other as an example. The depth order of the first object in front can be set to 1, and the depth order of the second object hidden behind the first object can be set to 2. When different objects are moving, the number of object overlaps and the order of depth of field can be used to determine which object or part needs to be displayed.

As shown in FIG. 6, a virtual object with multiple depths of field, such as a dinosaur, can be displayed in area C by projecting preset right and left pixels to the user's retina. In one embodiment, the position of the object can be determined by a reference position, and the angle of view of the object is determined by a rotation angle. As shown in FIG. 7, in step 710, an object image is created with a reference position. In one embodiment, the object image can be generated by a 2D or 3D module. The reference position may be at the center of gravity of the object. In step 720, the virtual binocular pixel at the reference position is determined. Through the 3D coordinates of the reference position, the designer can directly determine the number, for example, through GUI software, a pixel closest to the virtual binocular, such as VBP (145). In step 730, a pair of right and left pixels corresponding to the virtual binocular pixels is found. The designer can find the corresponding right pixel and left pixel pair by looking up the table. The designer can also calculate the convergence angle with a predetermined reference position depth of field and find the corresponding right and left pixels, assuming that the reference position is in front of the user's eyes. The designer can move the reference position on the XY plane to the preset X and Y coordinates and find the final corresponding right and left pixels. In step 740, the right light signal and the corresponding left light signal are projected to respectively display the right pixel and the corresponding left pixel as a reference position. After the right pixel and left pixel pair of the virtual binocular pixel corresponding to the reference position is determined, the entire virtual object can be displayed by 2D or 3D module data.

The lookup table is generated by the following steps. The first step is to obtain a personal virtual map based on the user’s IPD. The virtual map is generated by the system during the initialization and calibration process. It defines the boundary of the area C where the user can see objects with depth of field, and the user can see the depth of field. The object of is caused by the fusion of the right retinal image and the left retinal image. In the second step, the convergence angle of each depth of field (each point on the Z-coordinate) in the Z-axis direction is calculated to find the right pixel and the left pixel pair regardless of the X-coordinate and the left retinal image respectively. Y-coordinate position. In the third step, the right pixel and left pixel pair are moved along the X axis to find the X-coordinate and Z-coordinate of each right pixel and left pixel pair at a specific depth of field regardless of the Y-coordinate position. In the fourth step, the right pixel and the left pixel pair are moved along the Y axis to determine the Y-coordinate of each right pixel and left pixel pair. In this way, the 3D coordinate system (such as XYZ) of each right pixel and left pixel pair in the right retinal image and the left retinal image can be determined to generate a look-up table. In addition, the third step and the fourth step can be exchanged.

In another embodiment, the designer can determine each necessary virtual binocular pixel to form a virtual object, and then find each corresponding right pixel and left pixel pair through a look-up table. The right light signal and the left light signal can then be generated. The right retinal image and the left retinal image have the same angle of view. Parallax is not used to present 3D images. Therefore, complicated and time-consuming computer graphics calculations can be avoided. The relative position of objects on the right retinal image and the left retinal image determines the depth of field perceived by the user.

The optical signal generators 10 and 20 can be used later, light emitting diodes ("LED", including mini and micro LEDs), organic light emitting diodes ("OLED"), or super luminescent diodes ("LED"). /"SLD"), liquid crystal on silicon (LcoS), liquid crystal display ("LCD"), or a combination of the above can be used as a light source. In one embodiment, the optical signal generators 10 and 20 are laser beam scanning projectors (LBS projectors), which include light sources such as red lasers, green lasers, and blue lasers, and light color regulators. (Such as bidirectional color combiner and polarization combiner), and two-dimensional (2D) adjustable reflector (such as 2D electromechanical system ("MEMS") mirror). The 2D adjustable reflector can be replaced by two one-dimensional (1D) reflectors, such as two 1D MEMS mirrors. The LBS projector sequentially generates and scans light signals to form a 2D image with a predetermined resolution, for example, each frame has 1280x720 pixels. Therefore, one light signal is generated one pixel at a time and projected toward the combiners 20 and 40 one pixel at a time. In order for the user to see the 2D image with a single eye, the LBS projector must sequentially generate an optical signal for each pixel, such as a 1280x720 optical signal, during the visual duration (for example, 1/18 second). Therefore, the dwell time of each optical signal is about 60.28 nanoseconds.

In another embodiment, the optical signal generators 10 and 30 can be digital light processing projectors ("DLP projectors"), which can generate one 2D color image at a time. Texas Instruments' DLP technology is one of many technologies that can produce DLP projectors. The entire 2D color image frame, for example, may contain 1280x720 pixels, and is projected toward the combiners 20 and 40 at the same time.

The combiners 20 and 40 receive and redirect a plurality of optical signals generated by the optical signal generators 10 and 20. In one embodiment, the combiners 20 and 40 reflect a plurality of light signals so that the redirected light signal and the incident light signal are on the same side of the combiners 20 and 40. In another embodiment, the combiners 20 and 40 refract a plurality of light signals so that the redirected light signal and the incident light signal are on different sides of the combiners 20 and 40. When the combiners 20 and 40 have the function of a refractor, the reflection ratio can vary considerably, such as 20%-80%, which partly depends on the capability of the optical signal generator. Those with ordinary knowledge in this field can determine the appropriate reflection ratio based on the characteristics of the optical signal generator and combiner. In addition, in one embodiment, the opposing surfaces where the light signals of the combiners 20 and 40 are incident may be transparent with respect to the ambient light. Based on different embodiments, the degree of transparency can vary considerably. In AR/MR applications, the transparency is preferably 50%, and in other embodiments 75% is better. In addition, in order to redirect the light signals, the combiners 20 and 40 may converge a plurality of light signals to form a combiner image so that it can pass through the pupil and reach the user's retina.

The combiners 20 and 40 can be made of glass or plastic mirrors and coated with a specific material (for example, metal) to make them partially transparent or partially reflective. The advantage of using a reflective combiner is that it does not use the light guide plate in the prior art to guide the light signal to the user's eyes, which can solve the poor reflection effects, such as multiple shadows, color distortion... etc. The combiners 20 and 40 may be holographic combiners but are not a preferred embodiment because the sparing effect may cause multiple shadows and RGB distortions. In some embodiments, it is necessary to avoid the use of holographic combiners.

In one embodiment, the combiners 20 and 40 have ellipsoidal surfaces. In addition, the optical signal generator and the user's eyes are located at the focal point of the ellipsoid. As shown in FIG. 8, the right combiner 20 has an ellipsoidal surface, the right optical signal generator 10 is located at the right focal point and the user's right eye is located at the left focal point of the ellipsoid. Similarly, the left combiner 40 has an ellipsoidal surface, the left optical signal generator 30 is located at the left focal point and the user's left eye is located at the right focal point of the ellipsoid. Based on the geometric characteristics of the ellipsoid, all light beams projected from one focal point to the surface of the ellipsoid will be reflected to the other focal point. Under such conditions, all light beams projected from the optical signal generator to the combiner on the ellipsoid surface will be reflected to the user's eyes. Therefore, in this embodiment, the FOV can be maximized, comparable to the surface of an ellipsoid. In another embodiment, the combiners 20 and 40 may have a flat surface and a holographic film, which is configured to reflect light in an ellipsoid manner.

The object display system can use the right collimator and the left collimator to converge the light beams of a plurality of optical signals, for example, to cause the moving direction to be aligned with one direction or to make the spatial profile of the light beam smaller. The right collimator may be disposed between the right optical signal generator 10 and the right combiner 20, and the left collimator may be disposed between the left optical signal generator 30 and the left combiner 40. The collimator can be a curved mirror or a lens.

As shown in FIG. 9, the object display system may further include a right optical replicator (optical replicator) and a left optical replicator. The optical replicator can be arranged between the optical signal generators 10 and 30 and the combiners 20 and 40 to replicate incident optical signals. Therefore, the optical replicator can generate a plurality of incident light signals to enlarge the user's eye-box. Optical replicators can be beam splitters, polarizing beam splitters, half-silvered mirrors, partial reflective mirrors, dichroic mirrored prisms, bidirectional color, or dielectric optical coatings. Cloth layer. The optical replicator 120 may include at least two optical elements to replicate incident light signals into at least two objects. Each optical element can be a lens, a reflector, a partial reflector, a diamond mirror, or any combination of the above.

The object display system may further include a control unit, which includes all necessary circuits to control the right optical signal generator 10 and the left optical signal generator 30. The control unit provides electronic signals to the optical signal generator to generate a plurality of optical signals. In one embodiment, the positions and angles of the right optical signal generator 10 and the left optical signal generator 30 can be adjusted to adjust the incident angle of the right optical signal and the left optical signal and the receiving position of the right combiner 20 and the left combiner 40. Such adjustment can be realized by the control unit. The control unit can communicate with an independent video signal provider through a wired or wireless mechanism. Wireless communication includes 4G and 5G telecommunications, WiFi, Bluetooth, near field communication, and the Internet. The control unit may include a processor, a memory, and an input/output interface (I/O interface) to communicate with the video signal provider and the user. The object display system also includes a power supply. The power source can be a battery and/or a wirelessly rechargeable original.

There are at least two options for arranging the optical path from the optical signal generator to the user's retina. The first option mentioned above is that the right optical signal generator 10 generates the right optical signal, and the right optical signal is redirected to the right retina by the right combiner 20; the left optical signal generator 30 generates the left optical signal, and the left optical signal is combined from the left The device 40 is redirected to the left retina. As shown in FIG. 10, the second option is that the right optical signal generator 10 generates the right optical signal, and the right optical signal is redirected to the left retina by the left combiner 40; and the left optical signal generator 30 generates the left optical signal, which generates the left optical signal. The signal is redirected to the right retina by the right combiner 20.

In another embodiment, as shown in FIG. 11, the right combiner 20 and the left combiner 40 can be integrated into an integrated combiner and have a specific curvature for the right optical signal and the left optical signal. In the large combiner of this embodiment, the right light signal generated by the right light signal generator 10 is reflected to the left retina and the left light signal generated by the left light signal generator 30 is reflected to the right retina. By extending the width of the combiner to produce a relatively large reflective surface, the size of the FOV and area C of the binocular fusion can be increased.

The object display system may include a support structure wearable on the user's head to mount the right optical signal generator 10, the left optical signal generator 30, the right combiner 20, and the left combiner 40. The right side combiner 20 and the left side combiner 40 are located in the user's field of vision. Therefore, in this embodiment, the object display system is a head-mounted device (HWD). In particular, as shown in FIG. 12, the object display system can be loaded by glasses, that is, smart glasses. In this situation, the supporting structure can be a spectacle frame with or without lenses. The lens can be a lens with power to correct nearsightedness, farsightedness, etc. The right optical signal generator 10 can be mounted on the right temple 140 of the spectacle frame and the left optical signal generator 30 can be mounted on the left temple 130 of the spectacle frame. The right combiner 20 can be loaded on the right lens and the left combiner 40 can be loaded on the left lens. There are many different implementations of loading methods. The combiner can be attached to or integrated into the lens by detachable or non-detachable means. In addition, the combiner can be integrally formed with the lens, and the lens includes a lens with power. When the supporting structure does not include a lens, the right side combiner 20 and the left side combiner 40 can be directly mounted on the frame or frame.

All the various components of the above-mentioned object display system embodiment can be applied to the HWD. Therefore, HWD, including smart glasses, can be further equipped with other object display system components, such as the control unit, right collimator and left collimator. The right collimator may be located between the right optical signal generator 10 and the right combiner 20, and the left collimator may be located between the left optical signal generator 30 and the left combiner 40. In addition, the combiner can be replaced by a beam splitter and a converging lens. The function of the beam splitter is to reflect the light signal, and the function of the converging lens is to converge the light signal to pass through the pupil and reach the user's retina.

When the object display system is configured in smart glasses, the smart glasses lens can have refractive characteristics at the same time to correct the user's vision and combine the function. Smart glasses may have lenses with power to correct the vision of near-sighted users or long-sighted users. Under these conditions, each lens of the smart glasses may include a refractive unit and a combiner. The diopter unit and the combiner can be connected or manufactured in one piece with different materials. The diopter unit and the combiner can be separately manufactured into two components and assembled together. The two elements can be joined to each other, can also be detached, for example, through a magnetic attraction, or can be permanently joined together. In the above two situations, the combiner is provided on the side of the lens close to the user's eye. If the lens is integrally formed, the combiner forms the inner surface of the lens. If the lens has two parts, the combiner forms the inner surface of the lens. The combiner also allows the environment to penetrate and reflect the light signal generated by the light signal generator to the user's eyes to form a virtual image in the real environment. The combiner is designed with appropriate curvature to reflect and converge all the light signals from the light signal generator into the pupil to the retina of the eye.

In some embodiments, the curvature of a surface of the diopter unit is determined based on the user's refractive power. If the lens is integrally formed, the curvature of the refractive power is the outer surface of the lens. If the lens has two parts, the diopter unit forms the outer part of the lens. In this case, the curvature of the diopter can be the inner surface or the outer surface of the diopter unit. In order to better match the refractive unit and the combiner, in one embodiment, the refractive unit can be classified into 3 groups based on the power—over +3.00 (farsightedness), between -3.0-+3.0, and Xiaoyu -3.0 (Shortsighted). The combiner can be designed according to the category of the diopter unit. In another embodiment, the refractive units can be classified into 5 groups or 10 groups, each group having a smaller range of refractive power. As shown in FIG. 13, the outer surface 210 of the diopter unit 200 is used to provide a curvature with diopter power, and the inner surface 220 of the diopter unit 200 can be designed to have the same curvature as the outer surface 310 of the combiner 300. Therefore, the refractive unit 200 can be combined with the combiner 300 more easily. For example, the inner surface 220 of the diopter unit 200 and the outer surface 310 of the combiner 300 may be the same sphere or ellipsoid surface. In another embodiment, when the curvature of the inner surface 220 of the refractive unit 200 has a degree, the outer surface 310 of the combiner 300 can be designed to have the same or similar curvature of the inner surface 220 of the refractive unit 200 so that the two can be easily combined. However, when the outer surface 310 of the combiner 300 and the inner surface 220 of the refraction unit 200 do not have the same curvature, the outer surface 310 of the combiner 300 and the inner surface 220 of the refraction unit 200 can be made by mechanical mechanisms such as magnets, adhesive materials or bonding members. Combine each other. In addition, an intermediate material may be used to assemble the refractive unit 200 and the combiner 300. Alternatively, the combiner 300 may be coated on the inner surface 320 of the lens.

In addition to static virtual objects in the space frame, the object display system can display moving objects. When the right optical signal generator 10 and the left optical signal generator 30 can generate optical signals at a high speed, such as 30, 60 or higher frames per second, the user can see objects moving smoothly in the image due to the persistence of vision. Various embodiments to display the movement of the virtual object will be described below. Figures 14A-I illustrate examples 1-9 of object movement, respectively. The objects in the right-side combiner image 82 and the left-side combiner image 92 shown in the figure do not accurately reflect the corresponding positions where the right light signal and the left light signal are displayed. In addition, in these examples, the middle point of the line between the pupils of the user is taken as the origin of the XYZ coordinate system. Furthermore, RCI(10,10) and LCI(10,10) are set as the midpoints of the right combiner image and the left combiner image. Similarly, RRI(10,10) and LRI(10,10) are set as the midpoint between the right retinal image and the left retinal image. The (0,0) pixel is the top left pixel of each image.

Example 1 of FIG. 14A illustrates that the virtual object only moves in the X-axis direction (to the right) on the same depth plane, from the first virtual binocular pixel to the second virtual binocular pixel. To achieve this goal, the positions of the right light signal and the corresponding left light signal on the right combiner image and the left combiner image must be moved to the right by the same distance (pixels) in the X-axis direction. Therefore, the right light signal and the corresponding left light signal are moved to the left at the same distance in the X-axis direction at the positions of the right retinal image and the left retinal image forming the virtual object. In other words, the right optical signal from the optical signal generator and the corresponding left optical signal must be projected at different X-coordinate positions on the image of the combiner. However, since the Y-coordinate and Z-coordinate (depth direction) of the virtual object remain unchanged, the right light signal and the corresponding left light signal are projected to the same position (Y-coordinate and Z-coordinate) on the combiner image. For example, when the XYZ coordinates of the virtual object move from (0,0,100) to (10,0,100), the right light signal on the right combiner image moves from RCI(10,10) to RCI(12,10), and the left The left light signal on the combiner image moves from LCI(10,10) to LCI(12,10). Therefore, the right light signal on the right retinal image moves from RRI(10,10) to RRI(8,10), and the left light signal on the left retinal image moves from LRI(10,10) to LRI(8,10) .

Example 2 of FIG. 14B illustrates that the virtual object moves only in the Y-axis direction (downward) on the same depth plane, from the first virtual binocular pixel to the second virtual binocular pixel. In order to achieve this, the positions of the right light signal and the corresponding left light signal on the right combiner image and the left combiner image must be moved down by the same distance (pixels) in the Y-axis direction, respectively. Therefore, the right optical signal and the corresponding left optical signal respectively move upward in the Y-axis direction at the positions of the right retinal image and the left retinal image that form the virtual object. In other words, the right optical signal from the optical signal generator and the corresponding left optical signal must be projected at different Y-coordinate positions on the image of the combiner. However, since the X-coordinate and Z-coordinate (depth direction) of the virtual object remain unchanged, the right light signal and the corresponding left light signal are projected to the same position (X-coordinate and Z-coordinate) on the combiner image. For example, when the XYZ coordinates of the virtual object move from (0,0,100) to (0,-10,100), the right light signal on the right combiner image moves from RCI(10,10) to RCI(10,12), and The left optical signal on the left combiner image moves from LCI(10,10) to LCI(10,12). Therefore, the right light signal on the right retinal image moves from RRI(10,10) to RRI(10,8), and the left light signal on the left retinal image moves from LRI(10,10) to LRI(10,8) .

Example 3 of FIG. 14C illustrates that the virtual object only moves in the Z-axis direction (toward the user), and therefore changes from the original depth-of-field plane to the new depth-of-field plane. To achieve this goal, based on the increased convergence angle between the right optical signal and the corresponding left optical signal optical path extension, the position of the right optical signal and the corresponding left optical signal on the right combiner image and the left combiner image must be at X The axis directions are close to each other. Therefore, the positions of the right light signal and the corresponding left light signal forming the virtual object in the right retinal image and the left retinal image are far away from each other in the X-axis direction. In general, when the virtual object moves closer to the user, the relative distance between the right light signal and the corresponding left light signal position in the combiner image decreases, while the right light signal in the retinal image is between the right light signal position and the corresponding left light signal position The relative distance increases. In other words, the right optical signal from the optical signal generator and the corresponding left optical signal must be projected on two different X-coordinate positions on the combiner image so that they are close to each other. However, since the Y-coordinate of the virtual object remains unchanged, the right optical signal and the corresponding left optical signal are projected to the same Y-coordinate position of the combiner image. For example, when the XYZ coordinates of the virtual object move from (0,0,100) to (0,0,50), the right light signal on the right combiner image moves from RCI(10,10) to RCI(5,10), The left light signal on the left combiner image moves from LCI(10,10) to LCI(15,10). Therefore, the right light signal on the right retinal image moves from RRI(10,10) to RRI(15,10), and the left light signal on the left retinal image moves from LRI(10,10) to LRI(5,10) .

However, in order to move the virtual object closer to the user, the X-coordinate of the virtual object is not in the middle (middle point) of the interpupillary line (in one embodiment, X-coordinate = 0), and the right light signal corresponds to the left light The respective positions of the signal in the right combiner image and the left combiner image must be close to each other based on a ratio. The ratio is the distance between the position of the right light signal in the right combiner image and its left border (near the middle of the eyes), and the distance between the position of the left light signal in the left combiner image and the right border (close to the double The middle of the eye) compared. For example, suppose that the position of the right light signal in the right combiner image is 10 pixels from the left boundary (near the middle of the eyes), and the left light signal in the left combiner image is located 5 pixels away from the right boundary (close to the eyes) Middle), the ratio of the distance from the right position to the middle to the distance from the left position to the middle is 2:1 (10:5). In order to move the object closer to the user, due to the 2:1 ratio, if the right position on the right combiner image and the left position on the left combiner image must be close to each other by a distance of 3 pixels, the right position must move toward the left boundary 2 pixels, and the left position must be moved 1 pixel toward the right.

Example 4 of FIG. 14D illustrates that the virtual object moves in the X-axis direction (rightward) and Y-axis direction (upward) on the same depth plane in space, from the first virtual binocular pixel to the second virtual binocular pixel. To achieve this, the positions of the right light signal and the corresponding left light signal in the right combiner image and the left combiner image must be moved to the right and upward relative to the original position. Therefore, the right light signal and the corresponding left light signal move to the right and upward at the positions of the right retinal image and the left retinal image forming the virtual object relative to the original position. In other words, the right optical signal and the corresponding left optical signal from the optical signal generator must be projected at a new position on the upper right side of the right combiner image and the left combiner image, and the right optical signal and the corresponding left optical signal light The convergence angle between the radial extensions remains unchanged. For example, when the XYZ coordinates of the virtual object move from (0,0,100) to (10,10,100), the right light signal moves from RCI(10,10) to RCI(12,8) on the right combiner image, and the left light The signal is moved from LCI(10,10) to LCI(12,8) in the combiner image on the left. Therefore, the right light signal moves from RRI(10,10) to RRI(8,12) in the right retina image, and the left light signal moves from LRI(10,10) to LRI(8,12) in the left retina image.

Example 5 of FIG. 14E illustrates that the virtual object moves in the Y-axis direction (downward) and Z-axis direction (close to the user) in space, and therefore moves from the original depth plane to the new depth plane. To achieve this goal, the positions of the right light signal and the corresponding left light signal on the right combiner image and left combiner image must move down in the Y-axis direction, and move closer to each other in the X-axis direction, corresponding to the convergence angle. Therefore, the right optical signal and the corresponding left optical signal must move upward in the Y-axis direction and move away from each other in the X-axis direction at the positions of the right and left retinal images forming the virtual object. In other words, the right optical signal from the optical signal generator and the corresponding left optical signal must be projected at different Y-coordinate positions and two different X-coordinate positions (close to each other) of the combiner image. For example, when the XYZ coordinates of a virtual object move from (0,0,100) to (0,-10,50), the right light signal moves from RCI(10,10) to RCI(5,12) on the right combiner image, The left light signal moves from LCI(10,10) to LCI(15,12) in the left combiner image. Therefore, the right light signal moves from RRI(10,10) to RRI(15,8) in the right retina image, and the left light signal moves from LRI(10,10) to LRI(5,8) in the left retina image.

However, since the X-coordinate of the virtual object remains unchanged and the virtual object moves toward the user, the respective positions of the right light signal and the corresponding left light signal on the right combiner image and the left combiner image must be close to each other based on a ratio. The ratio is the distance between the position of the right light signal in the right combiner image and its left border (near the middle of the eyes), and the distance between the position of the left light signal in the left combiner image and the right border (close to the double The middle of the eye) compared. For example, (assuming that the position of the right light signal in the right combiner image is 10 pixels from the left border (near the middle of the eyes), and the left light signal is located at 5 pixels from the right border (close to the eyes) in the left combiner image The ratio of the distance from the right position to the middle to the distance from the left position to the middle is 2:1 (10:5). In order to move the object closer to the user, due to the 2:1 ratio, if the right The right position on the combiner image and the left position on the left combiner image must be close to each other by a distance of 3 pixels, the right position must move 2 pixels toward the left boundary, and the left position must move 1 pixel toward the right.

Example 6 of FIG. 14F illustrates that the virtual object moves in the X-axis direction (to the right) and Z-axis direction (close to the user) in space, and therefore moves from the original depth plane to the new depth plane. To achieve this, the positions of the right light signal and the corresponding left light signal on the right combiner image and left combiner image must move to the right in the X-axis direction, and move closer to each other in the X-axis direction, corresponding to the convergence angle. Therefore, the right light signal and the corresponding left light signal must move to the left in the X-axis direction and move away from each other in the X-axis direction at the positions of the right retinal image and the left retinal image forming the virtual object. In other words, the right optical signal from the optical signal generator and the corresponding left optical signal must be projected at different X-coordinate positions and two different X-coordinate positions (to the right and close to each other) of the combiner image. Since the Y-coordinate of the virtual object remains unchanged, the right light signal and the corresponding left light signal are projected to the same Y-coordinate position of the combiner image. For example, when the XYZ coordinates of the virtual object move from (0,0,100) to (10,0,50), the right light signal on the right combiner image moves from RCI(10,10) to RCI(7,10), The left light signal on the left combiner image moves from LCI(10,10) to LCI(17,10). Therefore, the right light signal on the right retinal image moves from RRI(10,10) to RRI(13,10) and the left light signal on the left retinal image moves from LRI(10,10) to LRI(3,10).

Example 7 of FIG. 14G illustrates that the virtual object moves in the X-axis direction (rightward), Y-axis direction (downward), and Z-axis direction (close to the user) in space, thus moving from the original depth plane to the new depth plane. To achieve this, the positions of the right light signal and the corresponding left light signal on the right combiner image and left combiner image must move to the right in the X-axis direction, move down in the Y-axis direction, and move closer to each other in the X-axis direction , Corresponding to the meeting point. Therefore, the right light signal and the corresponding left light signal must move to the left in the X-axis direction, move up in the Y-axis direction, and move away from each other in the X-axis direction at the positions of the right and left retina images forming the virtual object. In other words, the right optical signal from the optical signal generator and the corresponding left optical signal must be projected on two different X-coordinates (to the right and close to each other) and different Y-coordinate positions of the combiner image. For example, when the XYZ coordinates of the virtual object move from (0,0,100) to (10,-10,50), the right light signal on the right combiner image moves from RCI(10,10) to RCI(7,12) , And the left optical signal on the left combiner image moves from LCI(10,10) to LCI(17,12). Therefore, the right light signal on the right retinal image moves from RRI(10,10) to RRI(13,8), and the left light signal on the left retinal image moves from LRI(10,10) to LRI(3,8) .

Example 8 shown in FIG. 14H illustrates a method of moving a virtual object in the Z-axis direction. The object moves from a depth of field of 1 m to a depth of field of 10 m (away from the user), and therefore moves from the original depth of field to a new depth of field in space. When the space of the area C contains a sufficient number of virtual binocular pixels, the virtual object can move smoothly through many intermediate virtual binocular pixels. In other words, when the right retinal image and the left retinal image contain a sufficient number of right pixels and left pixels, the user can see a large number of virtual binocular pixels in space. In FIG. 14H, the object represented by a dot moves from a first virtual binocular pixel with a depth of 1m to a second virtual binocular pixel with a depth of 10m, passing through different intermediate virtual binocular pixels in the middle. First, the convergence angle between the first redirected right light signal of the first virtual binocular pixel with a depth of field of 1 m and the first redirected left light signal optical path extension is 3.4 degrees.

=60mm/(2*1m)=0.03。如果IPD=60mm，Ɵ=3.4度。

=60mm/(2*1m)=0.03. If IPD=60mm, Ɵ=3.4 degrees.

Second, the convergence angle between the second redirected right light signal of the second virtual binocular pixel with a depth of field of 10 m and the light path extension of the second redirected left light signal is 0.34 degrees.

=60mm/(2*10m)=0.003。如果IPD=60mm，Ɵ=0.34度。

=60mm/(2*10m)=0.003. If IPD=60mm, Ɵ=0.34 degrees.

Third, calculate the intermediate virtual binocular pixels. The number of intermediate virtual binocular pixels can be calculated based on the difference between the convergence angle of the first virtual binocular pixel and the second virtual binocular pixel and the number of pixels in the X-axis direction in each degree of the FOB. The difference between the convergence angle of the first virtual binocular pixels (3.4 degrees) and the convergence angle of the second virtual binocular pixels (0.34 degrees) is 3.06. The number of pixels in the X-axis direction in each degree of FOB is 32, assuming that the total width of the scanned retinal image is 1280 pixels, and its field of view (FOV) covers 40 degrees. Therefore, when a virtual object moves from a first virtual binocular pixel with a depth of field of 1m to a second virtual binocular pixel with a depth of field of 10m, there are about 98 (32x3.06) virtual binocular pixels in between to show this movement. The 98 virtual binocular pixels can be found by the above-mentioned look-up table. Fourth, in this embodiment, 98 intermediate virtual binocular pixels can be used to display movement, such as being divided into 98 small movement steps. The right light signals and the corresponding left light signals of these 98 virtual binocular pixels are respectively generated by the right light signal generator 10 and the left light signal generator 30 and projected to the user's right retina and left retina. Therefore, the user can see that the virtual object moves smoothly from 1m to 10m through 98 intermediate positions.

=60mm/(2*1m)=0.03。IfIPD=60mm，Ɵ=3.4度。Example 9 of FIG. 14I illustrates a method of moving a virtual object from a depth of field of 1 m to a depth of field of 20 cm (closer to the user) in the Z-axis direction, so that it moves from the original depth plane to the new depth plane in space. When the space of the area C contains a sufficient number of virtual binocular pixels, the virtual object can move smoothly through many intermediate virtual binocular pixels. In other words, when the right retinal image and the left retinal image contain a sufficient number of right pixels and left pixels, the user can see a large number of virtual binocular pixels in space. (In Figure 14I, the object represented by a dot moves from the first virtual binocular pixel with a depth of field of 1m to the second virtual binocular pixel with a depth of 20cm, passing through different intermediate virtual binocular pixels in the middle. First, the first virtual binocular pixel with a depth of field of 1m The convergence angle between the first redirected right light signal of a virtual binocular pixel and the first redirected left light signal light path extension is 3.4 degrees.)

=60mm/(2*1m)=0.03. IfIPD=60mm, Ɵ=3.4 degrees.

=60mm/(2*20cm)=0.15。IfIPD=60mm，Ɵ=17度。Second, the convergence angle between the second redirected right light signal of the second virtual binocular pixel with a depth of field of 20 cm and the light path extension of the second redirected left light signal is 17 degrees.

=60mm/(2*20cm)=0.15. IfIPD=60mm, Ɵ=17 degrees.

Third, calculate the intermediate virtual binocular pixels. The number of intermediate virtual binocular pixels can be calculated based on the difference between the convergence angle of the first virtual binocular pixel and the second virtual binocular pixel and the number of pixels in the X-axis direction in each degree of the FOB. The difference between the convergence angle of the first virtual binocular pixels (3.4 degrees) and the convergence angle of the second virtual binocular pixels (17 degrees) is 13.6. The number of pixels in the X-axis direction in each degree of FOB is 32, assuming that the total width of the scanned retinal image is 1280 pixels, and its field of view (FOV) covers 40 degrees. Therefore, when the virtual object moves from the first virtual binocular pixel with a depth of field of 1m to the second virtual binocular pixel with a depth of field of 20cm, there are about 435 (32x13.6) virtual binocular pixels in between to show this movement. These 435 virtual binocular pixels can be found by the above-mentioned look-up table. Fourth, in this embodiment, the movement can be displayed by 435 intermediate virtual binocular pixels, such as being divided into 435 small movement steps. The right light signal and the corresponding left light signal of these 435 virtual binocular pixels are respectively generated by the right light signal generator 10 and the left light signal generator 30 and projected to the user's right retina and left retina. Therefore, the user can see that the virtual object moves smoothly from 1m to 20cm through 435 intermediate positions.

Although many technical features and advantages of the present invention have been described above, the disclosed functions and detailed structures are all exemplary descriptions. Without departing from the spirit of the present invention, the maximum interpretation of the scope of the patent application of the present invention covers improvements obtained by changing the shape, size, and arrangement of components of the present invention based on the teachings of this specification.

The above examples are provided to those with general knowledge in the field to use this case. Various improvements and changes of this embodiment may be obvious to those with ordinary knowledge in the field. Without involving new technical features, the technical ideas and subject matter described herein can be applied to other embodiments. The scope of rights requested in this case is not intended to limit the embodiments described here, and should be interpreted in the broadest category. Other embodiments included in the spirit of this case can also be included in this case. The scope of rights requested in this case also covers other improvements and equivalents.

10: Right optical signal generator 10 110: Left optical replicator 12: Right light signal 12’: Right light signal of the first redirect 120: Right optical replicator 130: left temple 14: Right light signal 14’: Right light signal of the first redirect 140: right temple 16: Right light signal 16’: Right light signal of the first redirect 20: right combiner 20 200: Refractive unit 210: Outer surface 220: inner surface 30: Left optical signal generator 300: Combiner 310: Outer surface 32: Left light signal 32’: Left light signal 320: inner surface 34: left light signal 34’: Left light signal 36: Left light signal 36’: Left light signal 40: Left combiner 40 50: right eye 52: Right pupil 54: right retina 60: left eye 62: Left pupil 64: left retina 70: Dinosaur Object 710: step 72:0 step 72: virtual binocular pixels 730: step 74: The second virtual binocular pixel 740: step 82: Right combiner image 84: Right pupil image 86: Right retina image 92: Left combiner image 94: Left pupil image 96: Right retina image A: area B: area C: area D1: The first depth of field D2: second depth of field θ1: the first angle

Fig. 1 is a schematic diagram illustrating an embodiment of an object display system according to the present invention. 2 is a schematic diagram illustrating the relationship between the virtual binocular pixel and the corresponding right pixel and left pixel pair according to the present invention. 3 is a schematic diagram illustrating the light path from the optical signal generator to the combiner and reaching the user's retina according to the present invention. 4 is a schematic diagram illustrating the virtual binocular pixels formed by the right optical signal and the left optical signal according to the present invention. Figure 5 illustrates a lookup table according to an embodiment of the invention. FIG. 6 is a schematic diagram illustrating the display of objects by different virtual binocular pixels according to the present invention. Fig. 7 is a flowchart illustrating the process of displaying an object according to the present invention. FIG. 8 is a schematic diagram illustrating the position of the optical signal generator relative to the combiner according to the present invention. FIG. 9 is a schematic diagram illustrating an object display system with an optical replicator according to an embodiment of the present invention. FIG. 10 is a schematic diagram illustrating an object display system according to an embodiment of the present invention. Fig. 11 is a schematic diagram illustrating an integrated combiner according to the present invention. Fig. 12 is a schematic diagram illustrating an object display system mounted on glasses according to the present invention. Fig. 13 is a schematic diagram illustrating the refractive unit and the combiner according to the present invention. 14A-I are schematic diagrams illustrating the display of moving objects according to the present invention.

10: Optical signal generator on the right

12: Right light signal

12’: Right light signal of the first redirect

14: Right light signal

14’: Right light signal of the first redirect

16: Right light signal

16’: Right light signal of the first redirect

20: Right combiner

30: Left optical signal generator

32: Left light signal

32’: Left light signal

34: left light signal

34’: Left light signal

36: Left light signal

36’: Left light signal

40: Left combiner

50: right eye

52: Right pupil

54: right retina

60: left eye

62: Left pupil

64: left retina

70: Dinosaur Object

72: virtual binocular pixels

74: The second virtual binocular pixel

A: area

B: area

C: area

Claims (30)

A system for displaying an object with depth of field, including: A right light signal generator for generating a plurality of right light signals of an object; A right combiner, receiving and redirecting the plurality of right light signals toward a user's retina to display the plurality of right pixels of the object; A left optical signal generator for generating a plurality of left optical signals of the object; A left combiner, receiving and redirecting the plurality of left optical signals toward the user's other retina to display the plurality of left pixels of the object; and Wherein, the user perceives a first redirected right light signal and a corresponding first redirected left light signal to display a first virtual binocular pixel of the object with a first depth of field, the first depth of field and the corresponding A first angle correlation between the first redirected right optical signal and the corresponding first redirected left optical signal. The system according to claim 1, wherein the first depth of field is determined by the first angle between the right light signal of the first redirection and the corresponding extension of the left light signal of the first redirection . The system according to claim 1, wherein the first redirected right light signal and the corresponding first redirected left light signal have substantially the same height on the retina of both eyes of the user. The system according to claim 1, wherein the right optical signal of the first redirection is not a parallax corresponding to the left optical signal of the first redirection. The system according to claim 1, wherein the plurality of right optical signals generated by the right optical signal generator are reflected only once before entering the retina of the user, and the left optical signal generator generated The plurality of left light signals are reflected only once before entering the user's retina. The system according to claim 1, wherein the right optical signal generator is a right laser beam scanning projector (LBS projector), and the plurality of right optical signals generated by the right LBS projector enters the use The retina of the user is only reflected once by the right combiner. The left optical signal generator is a left LBS projector, and the plural left optical signals generated by the left LBS projector enter the user’s retina only by This left combiner reflects once. The system according to claim 1, wherein the right side combiner and the left side combiner are transparent to ambient light. The system of claim 1, wherein the user perceives a second redirected right light signal and a corresponding second redirected left light signal to display a second virtual binocular of the object with a second depth of field For pixels, the second depth of field is related to a second angle between the second redirected right light signal and the corresponding second redirected left light signal. The system according to claim 1, wherein the right combiner receives and redirects the plurality of left light signals toward a right retina of the user to display a plurality of right pixels of the object, and the left combiner receives and redirects The plurality of right light signals are redirected toward a left retina of the user to display the plurality of left pixels of the object. The system according to claim 1, wherein the right side combiner and the left side combiner are of an ellipsoid shape, the right optical signal generator is arranged at a focal position of the right combiner, and the left optical signal generator is arranged at A focal position of the left combiner. The system according to claim 1, wherein a right projection angle of the right optical signal generator can be adjusted to change an incident angle of the plurality of right optical signals to the right combiner, and the left optical signal generator A left projection angle can be adjusted to change an incident angle of the plurality of left light signals to the left combiner. The system described in claim 1, further including: A support structure that can be worn on a head of the user; Wherein, the right optical signal generator and the left optical signal generator are mounted on the supporting structure; and Wherein, the right side combiner and the left side combiner are loaded on the supporting structure and arranged in a field of view of the user. The system according to claim 12, wherein the supporting structure is a pair of glasses. The system according to claim 13, wherein a lens of the glasses has a power and is loaded with the right side combiner or the left side combiner. The system according to claim 13, wherein a lens of the glasses has a power and the lens and the right side combiner or the left side combiner are integrally formed. The system according to claim 13, wherein the lens with power and the right side combiner or the left side combiner can be combined with each other and can be detached from each other. The system according to claim 12, wherein the right side combiner and the left side combiner are integrally formed into an integrated combiner. A method of displaying an object with depth of field, including: Generate a plurality of right optical signals of the object from a right optical signal generator; Redirect the plurality of right light signals to a retina of a user; Generate a plurality of left optical signals of the object from a left optical signal generator; Redirect the plurality of left light signals to another retina of the user; Wherein, the user perceives a first redirected right light signal and a corresponding first redirected left light signal to display a first virtual binocular pixel of the object with a first depth of field, the first depth of field and the corresponding A first angle correlation between the first redirected right optical signal and the corresponding first redirected left optical signal. The method according to claim 18, wherein the first depth of field is determined by the first angle between the right light signal of the first redirection and the corresponding extension of the left light signal of the first redirection . The method according to claim 18, wherein the first redirected right light signal and the corresponding first redirected left light signal have substantially the same height on the retina of both eyes of the user. The method according to claim 18, wherein the right optical signal of the first redirection is not a parallax corresponding to the left optical signal of the first redirection. The method according to claim 18, wherein the plurality of right optical signals generated by the right optical signal generator are reflected only once before entering the retina of the user, and the left optical signal generator generates The plurality of left light signals are reflected only once before entering the user's retina. The method according to claim 18, wherein the right optical signal generator is a right laser beam scanning projector (LBS projector), and the plurality of right optical signals generated by the right LBS projector enters the use The retina of the user is only reflected once by the right combiner. The left optical signal generator is a left LBS projector, and the plural left optical signals generated by the left LBS projector enter the user’s retina only by This left combiner reflects once. The method according to claim 18, wherein the right side combiner and the left side combiner are transparent to ambient light. The method according to claim 18, wherein the right combiner receives and redirects the plurality of left light signals toward a right retina of the user to display a plurality of right pixels of the object, and the left combiner receives and redirects The plurality of right light signals are redirected toward a left retina of the user to display the plurality of left pixels of the object. The method according to claim 18, wherein the right side combiner and the left side combiner are of an ellipsoid shape, the right optical signal generator is arranged at a focal position of the right combiner, and the left optical signal generator is arranged at A focal position of the left combiner. The method according to claim 18, wherein a right projection angle of the right optical signal generator can be adjusted to change the incident angle of the plurality of right optical signals to the right combiner, and the left optical signal generator A left projection angle can be adjusted to change an incident angle of the plurality of left light signals to the left combiner. The method according to claim 18, wherein: The right optical signal generator and the left optical signal generator are mounted on a supporting structure, and the supporting structure can be worn on a head of the user; and The right side combiner and the left side combiner are loaded on the supporting structure and arranged in a visual field of the user. The method according to claim 28, wherein the support structure is a pair of glasses, a lens of the glasses has a power and is loaded with the right side combiner or the left side combiner. The method according to claim 28, wherein the right side combiner and the left side combiner are integrally formed into an integrated combiner.

Images