CN114594595B - Glasses control method and device and glasses - Google Patents
Glasses control method and device and glasses Download PDFInfo
- Publication number
- CN114594595B CN114594595B CN202011393843.4A CN202011393843A CN114594595B CN 114594595 B CN114594595 B CN 114594595B CN 202011393843 A CN202011393843 A CN 202011393843A CN 114594595 B CN114594595 B CN 114594595B
- Authority
- CN
- China
- Prior art keywords
- control information
- user
- vision
- information
- light modulator
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/0093—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
Abstract
The application provides a pair of spectacles, comprising: a display, an optical assembly, a spatial light modulator, a controller. The controller is used for sending first control information to the spatial light modulator, and the first control information is used for controlling the spatial light modulator to adjust the phase of an optical signal output by the display; the optical assembly is configured to transmit the optical signal to an eye location of a user. The spatial light modulator arranged in the glasses is controlled by the controller so that the spatial light modulator can adjust the phase of the optical signal output by the display, and the phase of the optical signal at the eye position of the user can be adjusted, so that the glasses are suitable for users with different vision requirements.
Description
Technical Field
The application relates to the field of electronics, in particular to glasses, and a control method and a control device of the glasses.
Background
With the increasing diversification of intelligent wearable devices, intelligent glasses are continuously and rapidly developed. The smart glasses may be applied to near-eye display scenes such as augmented reality (augmented reality, AR) and Virtual Reality (VR).
The smart glasses form a remote virtual image of the image on the display through the optical assembly and project it into the human eye. The intelligent glasses utilizing the VR technology can enable the user to be fully immersed in a virtual synthetic environment, and the user cannot see the real environment. The intelligent glasses using the AR technology can not shade the real environment, and the real environment is displayed in the retina of the user while the retina of the virtual user generated by the intelligent glasses is displayed.
In order to present clear images for users with different vision conditions, the space for accommodating the user glasses can be reserved in the existing intelligent glasses, so that the user can conveniently use virtual reality equipment by carrying the glasses, and the volume of the intelligent glasses is enlarged due to the space for carrying the glasses by the user.
Or, the display can be driven to move through the bevel gear structure so as to adjust the distance between the optical assembly and the screen, thereby adjusting the imaging distance of the screen picture in human eyes, enabling the intelligent glasses to adapt to users with different vision conditions, but the bevel gear structure is complex, and the adjusting precision is poor.
Disclosure of Invention
The application provides glasses, a control method and a control device of the glasses, which can reduce the volume of the glasses while adapting to users with different vision conditions.
In a first aspect, an eyeglass is provided that includes a display, an optical assembly, a spatial light modulator, and a controller. The controller is used for sending first control information to the spatial light modulator, and the first control information is used for controlling the spatial light modulator to adjust the phase of an optical signal output by the display; the optical assembly is configured to transmit the optical signal to an eye location of a user.
The spatial light modulator in the glasses adjusts the phase of the optical signal output by the display according to the control of the controller, so that the glasses can adapt to users with different vision conditions, and the size of the glasses is smaller.
With reference to the first aspect, in some possible implementations, the spatial light modulator transmits the adjusted optical signal to the optical component; the optical assembly is configured to transmit the adjusted optical signal to the eye position of the user.
That is, the spatial light modulator may be disposed after the display and the optical component along the optical path transmission direction. The spatial light modulator is arranged at a position far away from eyes of a user, and light rays in a real environment are transmitted to an area where eyes of the user pass, so that the light rays in the real environment are not influenced.
With reference to the first aspect, in some possible implementations, the controller is configured to: acquiring first vision information of a user of the glasses; and determining first control information corresponding to the first vision information according to the corresponding relation between the vision information and the control information, wherein the vision information comprises the first vision information.
According to the corresponding relation between the vision information and the control information, the first control information corresponding to the first vision information of the current user of the glasses is determined, and the determination mode of the first control information is simple and convenient.
With reference to the first aspect, in some possible implementations, when the controller sequentially sends the plurality of control information to the spatial light modulator, the correspondence is determined according to phase difference values of optical signals transmitted to different positions of the phase difference detector under each control information determined by the phase difference detector located at the eye position of the user, and the different phase difference values correspond to different vision information.
Before the user uses the glasses, the correspondence between the vision information and the control information may be determined. The vision information adapted to each type of control information is determined through the phase difference value of the optical signals transmitted to different positions of the phase difference detector under each type of control information determined by the phase difference detector at the eye position of the user, so that the corresponding relation between the vision information and the control information is more accurate, the glasses are better adapted to the vision conditions of different users, and the user experience is improved.
With reference to the first aspect, in some possible implementations, the glasses further include an eye tracker for acquiring a first line of sight direction of a user of the glasses; the controller is further configured to send, to the spatial light modulator, first control information corresponding to the first vision information in the first line-of-sight direction according to a correspondence relationship between a plurality of vision information and a plurality of control information in each of a plurality of line-of-sight directions including the first line-of-sight direction.
In general, the direction of the user's line of sight is the direction in which the user is focused. According to the corresponding relation between the vision information and the control information in the vision directions, the first control information corresponding to the vision information in the first vision direction of the user is determined, so that clear images are provided for the user in the first vision direction focused by the user, and user experience is improved.
In a second aspect, a method for controlling a pair of glasses is provided, the glasses including a display, an optical assembly, and a spatial light modulator; the optical assembly is for transmitting the optical signal to a user's eye position of the glasses. The method comprises the following steps: generating first control information, wherein the first control information is used for controlling the spatial light modulator to adjust the phase of an optical signal output by the display; and sending the first control information to the spatial light modulator.
With reference to the second aspect, in some possible implementations, the spatial light modulator transmits the adjusted optical signal to the optical component; the optical assembly is configured to transmit the adjusted optical signal to the eye position of the user.
With reference to the second aspect, in some possible implementations, the method further includes: acquiring first vision information of a user of the glasses; the generating the first control information includes: and determining the first control information corresponding to the first vision information according to the corresponding relation between the plurality of vision information and the plurality of control information, wherein the plurality of control information comprises the first control information.
With reference to the second aspect, in some possible implementations, the correspondence is determined according to aberration information of an optical signal transmitted to the eye position of the user under each control information determined by an aberration detector located at the eye position of the user when the controller sequentially sends the plurality of control information to the spatial light modulator, and different aberration information corresponds to different vision information.
With reference to the second aspect, in some possible implementations, the glasses further include an eye tracker for acquiring a first line of sight direction of a user of the glasses; the determining the first control information corresponding to the first vision information according to the correspondence between each vision direction of the plurality of vision information and the plurality of control information includes: and determining that the spatial light modulator transmits first control information corresponding to the first vision information in the first vision direction according to the corresponding relation between the vision information and the control information in the vision directions, wherein the vision directions comprise the first vision direction.
In a third aspect, there is provided a control device for eyeglasses, the eyeglasses comprising: comprising the following steps: a display, an optical assembly, and a spatial light modulator; the optical assembly is for transmitting the optical signal to a user's eye position of the glasses. The control device comprises a memory and a processor; the memory is used for storing programs; when the program is run in the processor, the processor is configured to: generating first control information, wherein the first control information is used for controlling the spatial light modulator to adjust the phase of an optical signal output by the display; and sending the first control information to the spatial light modulator.
With reference to the third aspect, in some possible implementations, the spatial light modulator transmits the adjusted optical signal to the optical component; the optical assembly is configured to transmit the adjusted optical signal to the eye position of the user.
With reference to the third aspect, in some possible implementations, the processor is further configured to: acquiring first vision information of a user of the glasses; and determining the first control information corresponding to the first vision information according to the corresponding relation between the plurality of vision information and the plurality of control information, wherein the plurality of control information comprises the first control information.
With reference to the third aspect, in some possible implementations, the correspondence is determined according to aberration information of an optical signal transmitted to the eye position of the user under each control information determined by an aberration detector located at the eye position of the user when the controller sequentially sends the plurality of control information to the spatial light modulator, and different aberration information corresponds to different vision information.
With reference to the third aspect, in some possible implementations, the glasses further include an eye tracker for acquiring a first line of sight direction of a user of the glasses. The processor is further configured to: and determining that the spatial light modulator transmits first control information corresponding to the first vision information in the first vision direction according to the corresponding relation between the plurality of vision information and the plurality of control information in each of the plurality of vision directions, wherein the plurality of vision directions comprise the first vision direction.
In a fourth aspect, a control device for glasses is provided, including a storage module and a processing module; the storage module is used for storing a program; the processing module is adapted to perform the method of the second aspect when the program is run in the processing module.
It should be understood that the program may also be referred to as program code, computer instructions, program instructions, etc.
In a fifth aspect, there is provided a computer readable storage medium storing a program comprising instructions for performing the steps of the method in the second aspect.
In a sixth aspect, there is provided a computer program product comprising computer instructions which, when executed by a computer, perform the method of the second aspect.
In a seventh aspect, there is provided a chip system comprising at least one processor, which when executed by a program causes the chip system to perform the method of the second aspect.
Optionally, as an implementation manner, the chip system may further include a memory, where the memory stores a program, and the processor is configured to execute the program stored on the memory, and when the program is executed, the processor is configured to perform the method in the second aspect.
The chip system may be a field programmable gate array (field programmable gate array, FPGA) or an application specific integrated circuit (application specific integrated circuit, ASIC).
It should be understood that, in the present application, the method of the second aspect may specifically refer to the method of the second aspect and any implementation manner of the various implementation manners of the second aspect.
Drawings
Fig. 1 is a schematic structural diagram of smart glasses using VR technology.
Fig. 2 is a schematic structural diagram of smart glasses using AR technology.
Fig. 3 is a schematic structural diagram of glasses according to an embodiment of the present application.
Fig. 4 is a schematic diagram of a spatial light modulator for adjusting the phase of incident light.
Fig. 5 is a schematic structural view of another pair of glasses according to an embodiment of the present application.
Fig. 6 is a schematic diagram of an image displayed by the display.
Fig. 7 is a schematic structural view of still another pair of glasses according to an embodiment of the present application.
Fig. 8 is a schematic flowchart of a method for controlling glasses according to an embodiment of the present application.
Fig. 9 is a schematic structural diagram of a glasses control device according to an embodiment of the present application.
Fig. 10 is a schematic structural diagram of another glasses control device according to an embodiment of the present application.
Detailed Description
The technical scheme of the application will be described below with reference to the accompanying drawings.
With the increasing diversification of intelligent wearable devices, intelligent glasses are continuously and rapidly developed. The smart glasses may be applied to near-eye display scenes such as augmented reality (augmented reality, AR) and Virtual Reality (VR).
As shown in fig. 1, smart glasses 100 utilizing VR technology include an optical assembly 120 and a display 110. The optical assembly 120 is used to form a remote virtual image from the image on the display 110 and project it into the human eye.
The intelligent glasses utilizing the VR technology can enable the user to be fully immersed in a virtual synthetic environment, and the user cannot see the real environment.
As shown in fig. 2, smart glasses 200 utilizing AR technology include an optical assembly 220 and a display 210.
The smart glasses 200 do not block the real environment, and light in the real environment is received in the retina of the user while the optical assembly 220 forms a virtual image on the display 210 and projects the virtual image on the retina of the user.
In order to present clear images for users with different vision conditions, the space for accommodating the user glasses can be reserved in the existing intelligent glasses, so that the user can conveniently use virtual reality equipment by carrying the glasses, and the volume of the intelligent glasses is enlarged due to the space for carrying the glasses by the user.
Or, can drive the display through bevel gear structure and remove in order to adjust the distance between optical component and the display to adjust the imaging distance of screen picture in the human eye, make intelligent glasses can adapt to the user of different vision conditions, but bevel gear structure is complicated, and the regulation precision is relatively poor, needs to reserve the space that the display removed in the intelligent glasses, has enlarged intelligent glasses's volume.
In order to solve the above problems, embodiments of the present application provide a pair of glasses, which can reduce the volume of the glasses while adapting to users with different vision conditions.
Fig. 3 is a schematic structural diagram of glasses according to an embodiment of the present application.
Glasses 300 include a display 310, an optical assembly 320, a spatial light modulator 330, and a controller 340.
The controller 340 is configured to control the spatial light modulator 330 to adjust the phase of the optical signal output from the display 310.
The optical assembly 320 is used to transmit the optical signal to the position of the eyes of the user of the glasses 300.
The spatial light modulator 330 in the glasses 300 adjusts the phase of the optical signal output by the display 310 according to the control of the controller 340, so that the glasses 300 can adapt to users with different vision conditions, and the size of the glasses 300 is smaller.
The display 310 may be a liquid crystal on silicon (also referred to as liquid crystal on silicon, liquid crystal on silicon, LCOS) panel, digital light processing (digital light processing, DLP) panel, or the like. LCOS is a very small size matrix liquid crystal display device based on a reflective mode. LCOS matrices are fabricated on silicon chips using complementary metal oxide semiconductor (complementary metal oxide semiconductor, CMOS) technology. DLP projection technology uses digital micromirror wafers (digital micromirror device, DMD) as the primary key processing element to achieve visual digital information display.
Spatial light modulator 330 may be used to adjust the phase of the optical signal output by the display. The phase may also be referred to as a phase.
The spatial light modulator 330 may include a liquid crystal and a driving device. An electric signal is applied to the outside of the liquid crystal, the liquid crystal is tilted, and the refractive index is changed, so that the phase of the incident light can be adjusted.
As shown in fig. 4, similar to the phase modulation effect of the lens, the optical signals output by the display are incident on the spatial light modulator 330, and the optical path differences of the light transmitted in the spatial light modulator 330 at different incident positions are different, that is, the time delays are different, so that the equiphase surfaces of the light waves are bent.
By applying different electrical signals outside the liquid crystal, the liquid crystal can realize different degrees of defocusing, astigmatism and other treatments on incident light. The defocus treatment of the incident light, i.e., the degree of convergence or divergence of the incident light, is adjusted to accommodate a user suffering from hyperopia or myopia. Astigmatic treatment of incident light, i.e. adjusting the difference in the degree of convergence or divergence of incident light at different locations, to accommodate a user suffering from astigmatism.
The first control signal may be an electrical signal applied outside the liquid crystal. Different control signals are applied to the outside of the liquid crystal so that the liquid crystal makes different adjustments to the phase of the incident light.
Alternatively, the spatial light modulator 330 includes a driving device for generating an electrical signal applied to the outside of the liquid crystal. Different control signals correspond to different electrical signals. That is, by the controller 340 transmitting different control signals to the spatial light modulator 330, the driving device of the spatial light modulator 330 may be caused to apply different electrical signals to the outside of the liquid crystal, thereby causing the liquid crystal in the spatial light modulator 330 to differently adjust the phase of the incident light.
The optical assembly 320 may include an optical waveguide. As shown in fig. 5, the optical component 320 is an optical waveguide. The optical waveguide (optical waveguide) is a medium device for guiding light waves to propagate therein, and can be understood as a guiding structure for transmitting optical frequency electromagnetic waves, which is formed by an optically transparent medium (such as quartz glass), and is also called a medium optical waveguide.
Optical waveguides offer great advantages for optimizing the design and aesthetic appearance of a headset. Because the optical signal transmission channel provided by the optical waveguide is provided, the display and other elements can be arranged far away from the glasses and moved to the top or the side face of the forehead, the blocking of the display and other elements to the outside sight is greatly reduced, and the weight distribution is more in accordance with the ergonomics, so that the wearing experience of the equipment is improved.
The optical assembly 320 may also include lenses, mirrors, and the like.
The optical signal emitted from the display 310 may be transmitted to the spatial light modulator 330 through the optical component 320, and then transmitted to the eye of the user after the phase of the optical signal is adjusted by the spatial light modulator 330.
That is, the spatial light modulator 330 may be disposed behind the optical assembly 320 in the optical path transmission direction. Thus, spatial light modulator 330 is positioned closer to the eyes of the user. When the glasses 300 are glasses using AR technology, the spatial light modulator 330 may have an effect on the transmission of light rays in the real environment to the eyes of the user.
In addition, the spatial light modulator 330 is controlled by the controller 340, and the controller 340 is generally disposed close to the spatial light modulator 330 for cost saving. When the glasses 300 are glasses using AR technology, the controller 340 and the spatial light modulator 330 need not block the light rays in the real environment from being transmitted to the eyes of the user, and high requirements are placed on the position setting of the controller 340 and the spatial light modulator 330.
Spatial light modulator 330 may adjust the phase of the optical signal emitted by display 310 and transmit the adjusted optical signal to optical assembly 320. The optical assembly 320 transmits the adjusted optical signal to the position of the user's eye.
That is, the spatial light modulator 330 may be disposed behind the display 310 and the optical assembly 320 in the optical path transmission direction. The spatial light modulator 330 is disposed at a position far from the eyes of the user, and is far from the region through which the light in the real environment is transmitted to the eyes of the user, without affecting the transmission of the light in the real environment to the eyes of the user.
The display may display an image when the user begins to use the glasses 300. The user may cause the controller 340 to sequentially issue a plurality of control information to the spatial light modulator 330. For example, the user may input the switching information by means of a key or the like, and the controller 340 receives the switching information and issues one control information each time the switching information is received.
The plurality of control information is used by spatial light modulator 330 to perform different phase adjustments on the optical signal output by display 310. The user may select a control signal among the plurality of control signals, which makes the user feel the display of the clearest, as a final control signal so that an electric signal externally loaded to the liquid crystal of the spatial light modulator 330 corresponds to the control signal. For example, the user may stop inputting the switching information when the display is the clearest, and the control information sent when the controller 340 receives the last switching information is the first control information.
The user may also input first vision information for indicating the vision condition of the user, such as the degree of myopia or hyperopia, the degree of astigmatism, etc.
The controller 340 may determine first control information corresponding to the first vision information according to correspondence between the plurality of vision information and the plurality of control information, and transmit the first control information to the spatial light modulator 330.
As shown in fig. 5, an aberration detector may be provided at the eye position of the user to determine correspondence between a plurality of vision information and a plurality of control information.
The correspondence is determined according to aberration information of the optical signal transmitted to the eye position of the user under each control information determined by the aberration detector located at the eye position of the user when the controller 340 sequentially transmits the plurality of control information to the spatial light modulator 330, and different aberration information corresponds to different vision information.
The display 310 may display the image shown in fig. 6. In the image, only the position of the center of the image is a white bright point, and the other positions are black. The controller 340 may sequentially transmit the plurality of control information to the spatial light modulator 330. The spatial light modulator performs different phase adjustments on the optical signal output by the display 310, controlled by different control information. The aberration detector receives images transmitted by the optical assembly 320 to the eye position of the user and determines the difference between the phases of the optical signals received at the respective positions. The difference between the phases of the optical signals received at the respective positions of the aberration detector is different when the controller 340 transmits different control information. The sphere, cylinder, astigmatism axis, etc. corresponding to the control signal can be determined from the differences between the phases of the optical signals received at the respective positions of the aberration detector. Aberrations include sphere, cylinder, astigmatism axis, etc. Visual information can be understood as aberrations of the human eye. Thus, correspondence between the plurality of vision information and the plurality of control information is determined.
The optical signal output by spatial light modulator 330 may pass through all or some of the optical elements in optical assembly 320 that may adjust the phase of the optical signal output by spatial light modulator 330 during transmission to the eye position of the user.
By determining the correspondence between the plurality of vision information and the plurality of control information, the glasses 300 can be better adapted to users with different vision conditions.
The correspondence between the plurality of visual acuity information and the plurality of control information may also be represented by the correspondence between the plurality of control information and the plurality of aberration adjustment manners of the spatial light modulator 330 and the correspondence between the aberration adjustment manners of the spatial light modulator 330 and the visual acuity information.
The spatial light modulator 330 may be used to implement a plurality of aberration adjustment modes, each of which is implemented by one type of control information. When the controller 340 sends out a control message, the spatial light modulator 330 adjusts the output light signal of the display 310 according to the aberration adjustment method a corresponding to the control message. The optical signal adjusted by the aberration adjustment mode a is transmitted to the aberration detector located at the eye position of the user through the optical component 320. The aberration detector determines the aberration B of the optical signal receiving the information.
The aberration adjustment modes a and B can be expressed by means of a matrix or a vector, and values in the matrix or the vector respectively represent sphere power, cylinder power, astigmatism axis and the like. In the aberration adjustment methods a and B, the values at the same position have the same meaning. For example, the first position in the vector of the aberration adjustment method a represents the sphere power, and the first position in the vector of the aberration B also represents the sphere power.
In some embodiments, the relationship between aberration adjustment, a, and aberration, B, can be expressed as: b=c×a. Wherein, C is a relation matrix between the aberration adjustment mode and the aberration.
When the user uses the glasses 300, the aberration B can be understood as visual information of the user, including sphere power, cylinder power, astigmatism axis, etc., which are required by the user.
Based on the correspondence between the plurality of control information and the plurality of aberration adjustment manners of the spatial light modulator 330 and the relationship matrix between the aberration adjustment manners and the aberrations, the correspondence between the plurality of vision information and the plurality of control information can be determined
When the first vision information of the user is known, the required aberration adjustment mode can be determined according to the inverse matrix of the relation matrix C. The first control information corresponding to the first vision information may be determined according to the correspondence between the plurality of control information and the plurality of aberration adjustment manners of the spatial light modulator 330.
When the user uses the glasses 300, the directions of transmitting pixels at different positions in the image displayed by the display 310 to the positions of eyes of the user are different, and the transmission paths are also different to some extent. In general, the center position of the image displayed on the display 310 when the user of the glasses 300 looks straight ahead is located at the center of the user's field of view. That is, the center position of the image displayed on the display 310 corresponds to the direction of the front head-up of the user.
In general, the direction of the user's sight is the direction in which the user focuses, and the direction of the user's sight may not be the direction of the front head-up. In order to enable the glasses 300 to obtain a clear image when the user focuses on various directions, correspondence between a plurality of vision information and a plurality of control information in a plurality of viewing directions may be determined.
The display 310 may also display an image in which the white bright point is located at other positions than the center of the image, and the other positions than the white bright point are black. Different positions of the white bright spot correspond to different directions of the user's line of sight. The controller 340 may sequentially transmit the plurality of control information to the spatial light modulator 330 while the white spot is located at each position of the image. The spatial light modulator performs different phase adjustments on the optical signal output by the display 310, controlled by different control information. The aberration detector receives images transmitted by the optical assembly 320 to the eye position of the user and determines the difference between the phases of the optical signals received at the respective positions. So that correspondence between the plurality of vision information and the plurality of control information in each of the plurality of vision directions can be determined.
The glasses 300 may also include an eye tracker. The eye tracker is used for acquiring a first sight line direction of a user of the glasses. The eye tracker can determine the position of the eye by its reflex and determine the orientation of the eye, i.e. the direction of the user's line of sight.
The controller 340 is further configured to send, to the spatial light modulator 330, first control information corresponding to the first vision information in the first viewing direction according to correspondence between the plurality of vision information and the plurality of control information in each of the plurality of viewing directions.
It should be appreciated that the plurality of gaze directions includes the first gaze direction and the plurality of vision information in the first gaze direction includes the first vision information in the first gaze direction.
According to the corresponding relation between the vision information and the control information in the vision directions, the first control information corresponding to the vision information in the first vision direction of the user is determined, so that clear images are provided for the user in the first vision direction focused by the user, and user experience is improved.
Further, the controller 340 may also be configured to determine a duration of the first line of sight direction of the user. The controller 340 may transmit first control information corresponding to the first vision information in the first viewing direction to the spatial light modulator 330 when the holding time period of the first viewing direction of the user exceeds a preset time period.
That is, a clear image can be provided to the user in the first line of sight direction in which the user is focused when the user is looking in the first line of sight direction.
As shown in fig. 7, in the glasses 300, the eye tracker 370 may be disposed at an area out of the user's vision range, not blocking the user's vision. For example, may be disposed on an optical waveguide of optical assembly 320.
The glasses provided by the embodiment of the present application are described above with reference to fig. 1 to 7, and the control method and the control device for the glasses provided by the embodiment of the present application are described below with reference to fig. 8 to 10. It should be understood that the description of the glasses corresponds to the description of the control method and the control device of the glasses, and thus, a part not described in detail may be referred to the previous embodiments.
Fig. 8 is a schematic flowchart of a method for controlling glasses according to an embodiment of the present application.
The glasses include: comprising the following steps: display, optical assembly, spatial light modulator. The optical assembly is for transmitting the optical signal to a user's eye position of the glasses.
At S810, first control information is generated, where the first control information is used to control the spatial light modulator to adjust a phase of an optical signal output by the display.
At S820, the first control information is transmitted to the spatial light modulator.
Optionally, the spatial light modulator transmits the adjusted optical signal to the optical component.
The optical assembly is configured to transmit the adjusted optical signal to the eye position of the user.
Optionally, the method further comprises: first vision information of a user of the glasses is acquired.
The generating the first control information includes: and determining the first control information corresponding to the first vision information according to the corresponding relation between the plurality of vision information and the plurality of control information, wherein the plurality of control information comprises the first control information.
Optionally, when the controller sequentially sends the plurality of control information to the spatial light modulator, the correspondence is determined according to aberration information of the optical signal transmitted to the eye position of the user under each control information determined by the aberration detector located at the eye position of the user, and different aberration information corresponds to different vision information.
Optionally, the glasses further comprise an eye tracker for acquiring a first line of sight direction of a user of the glasses.
The determining the first control information corresponding to the first vision information according to the correspondence between the plurality of vision information and the plurality of control information includes: and determining that the spatial light modulator transmits first control information corresponding to the first vision information in the first vision direction according to the corresponding relation between the plurality of vision information and the plurality of control information in each of the plurality of vision directions, wherein the plurality of vision directions comprise the first vision direction.
Fig. 9 is a schematic structural diagram of a glasses control device according to an embodiment of the present application.
The glasses include: comprising the following steps: display, optical assembly, spatial light modulator. The optical assembly is for transmitting the optical signal to a user's eye position of the glasses. In particular, reference may be made to the description of fig. 3 to 7.
It should be appreciated that the control device 2000 may be the controller 340 described in fig. 3. Alternatively, the control device 2000 may be located within the glasses or independent of the glasses.
The control device 2000 includes a memory module 2010 and a processing module 2020; the storage module is used for storing a program; the processing module 2020 is configured to perform the control method described above when the program is run in the processing module 2020.
Specifically, the processing module 2020 is configured to generate first control information, where the first control information is used to control the spatial light modulator to adjust a phase of an optical signal output by the display.
The processing module 2020 is further configured to send the first control information to the spatial light modulator.
Optionally, the spatial light modulator transmits the adjusted optical signal to the optical component.
The optical assembly is configured to transmit the adjusted optical signal to the eye position of the user.
Optionally, the processing module 2020 is further configured to obtain first vision information of a user of the glasses.
The processing module 2020 is further configured to determine, according to a correspondence between a plurality of vision information and a plurality of control information, the first control information corresponding to the first vision information, where the plurality of control information includes the first control information.
Optionally, when the controller sequentially sends the plurality of control information to the spatial light modulator, the correspondence is determined according to aberration information of the optical signal transmitted to the eye position of the user under each control information determined by the aberration detector located at the eye position of the user, and different aberration information corresponds to different vision information.
Optionally, the glasses further comprise an eye tracker for acquiring a first line of sight direction of a user of the glasses.
The processing module 2020 is further configured to determine, according to a correspondence between a plurality of vision information and a plurality of control information in each of a plurality of vision directions, that the spatial light modulator transmits first control information corresponding to the first vision information in the first vision direction, where the plurality of vision directions includes the first vision direction.
Fig. 10 is a schematic structural diagram of a glasses control device according to an embodiment of the present application.
The glasses include: comprising the following steps: display, optical assembly, spatial light modulator. The optical assembly is for transmitting the optical signal to a user's eye position of the glasses.
The control device 3000 includes a memory 3010 and a processor 3020; the memory is used for storing programs; when the program is run in the processor 3020, the processor 3020 is configured to execute the control method described above.
Specifically, the processor 3020 is configured to generate first control information, where the first control information is used to control the spatial light modulator to adjust the phase of the optical signal output by the display.
Processor 3020 is also used to send the first control information to the spatial light modulator.
Optionally, the spatial light modulator transmits the adjusted optical signal to the optical component.
The optical assembly is configured to transmit the adjusted optical signal to the eye position of the user.
Optionally, the processor 3020 is further configured to obtain the first vision information of the user of the glasses.
The processor 3020 is further configured to determine the first control information corresponding to the first vision information according to a correspondence between a plurality of vision information and a plurality of control information, where the plurality of control information includes the first control information.
Optionally, when the controller sequentially sends the plurality of control information to the spatial light modulator, the correspondence is determined according to aberration information of the optical signal transmitted to the eye position of the user under each control information determined by the aberration detector located at the eye position of the user, and different aberration information corresponds to different vision information.
Optionally, the glasses further comprise an eye tracker for acquiring a first line of sight direction of a user of the glasses.
The processor 3020 is further configured to determine, according to correspondence between a plurality of visual information and a plurality of control information in each of a plurality of visual directions, that the spatial light modulator transmits first control information corresponding to the first visual information in the first visual direction, the plurality of visual directions including the first visual direction.
It is to be appreciated that the processor in embodiments of the application may be a central processing unit (central processing unit, CPU), but may also be other general purpose processors, digital signal processors (digital signal processor, DSP), application specific integrated circuits (application specific integrated circuit, ASIC), off-the-shelf programmable gate arrays (field programmable gate array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
It should also be appreciated that the memory in embodiments of the present application may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The nonvolatile memory may be a read-only memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an electrically Erasable EPROM (EEPROM), or a flash memory. The volatile memory may be random access memory (random access memory, RAM) which acts as an external cache. By way of example but not limitation, many forms of random access memory (random access memory, RAM) are available, such as Static RAM (SRAM), dynamic Random Access Memory (DRAM), synchronous Dynamic Random Access Memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced Synchronous Dynamic Random Access Memory (ESDRAM), synchronous Link DRAM (SLDRAM), and direct memory bus RAM (DR RAM).
The descriptions of the processes corresponding to the drawings have emphasis, and the descriptions of other processes may be referred to for the parts of a certain process that are not described in detail.
The embodiment of the application also provides a computer readable storage medium, which is characterized in that the computer readable storage medium has program instructions, which when executed directly or indirectly, cause the method in the foregoing to be implemented.
In an embodiment of the present application, there is also provided a computer program product containing instructions which, when run on a computing device, cause the computing device to perform the method hereinbefore described or cause the computing device to carry out the functions of the apparatus hereinbefore described.
An embodiment of the present application also provides a chip system, where the chip system includes at least one processor, and where program instructions, when executed in the at least one processor, cause the method in the foregoing to be implemented.
The above embodiments may be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented in software, the above-described embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions described in accordance with embodiments of the present application are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more sets of available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium may be a solid state disk.
It should be understood that the term "and/or" is merely an association relationship describing the associated object, and means that three relationships may exist, for example, a and/or B may mean: a alone, a and B together, and B alone, wherein A, B may be singular or plural. In addition, the character "/" herein generally indicates that the associated object is an "or" relationship, but may also indicate an "and/or" relationship, and may be understood by referring to the context.
In the present application, "at least one" means one or more, and "a plurality" means two or more. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one (one) of a, b or c may represent: a, b, c, a-b, a-c, b-c or a-b-c, wherein a, b, c can be single or multiple.
It should be understood that, in various embodiments of the present application, the sequence numbers of the foregoing processes do not mean the order of execution, and the order of execution of the processes should be determined by the functions and internal logic thereof, and should not constitute any limitation on the implementation process of the embodiments of the present application.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.
In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (11)
1. An augmented reality glasses, comprising: a display, an optical assembly, a spatial light modulator, a controller, and an eye tracker;
the eye tracker is used for acquiring a first sight line direction of a user of the augmented reality glasses;
the controller is configured to obtain first vision information of a user of the augmented reality glasses, determine a holding duration of a first sight line direction of the user, and determine first control information corresponding to the first vision information according to a correspondence between a plurality of vision information and a plurality of control information when the holding duration exceeds a preset duration, where the plurality of vision information includes the first vision information, send the first control information to the spatial light modulator, and the first control information is configured to control the spatial light modulator to adjust a phase of an optical signal output by the display;
The optical assembly is for transmitting the optical signal to the user's eye position.
2. The augmented reality glasses according to claim 1, wherein,
the spatial light modulator transmits the adjusted optical signal to the optical component;
the optical assembly is configured to transmit the adjusted optical signal to the eye position of the user.
3. The augmented reality glasses according to claim 1 or 2, wherein the correspondence is determined according to a phase difference value of an optical signal transmitted to different positions of the aberration detector under each control information determined by the aberration detector located at the eye position of the user when the controller sequentially transmits the plurality of control information to the spatial light modulator, the different phase difference values corresponding to different vision information.
4. The augmented reality glasses according to claim 1 or 2, wherein,
the controller is further configured to send, to the spatial light modulator, first control information corresponding to the first vision information in the first line-of-sight direction according to a correspondence relationship between a plurality of vision information and a plurality of control information in each of a plurality of line-of-sight directions including the first line-of-sight direction.
5. A method of controlling augmented reality glasses, the glasses comprising: a display, an optical assembly, a spatial light modulator, and an eye tracker; the optical assembly is used for transmitting optical signals to eye positions of a user of the augmented reality glasses;
the method comprises the following steps:
acquiring a first line of sight direction of a user of the augmented reality glasses;
acquiring first vision information of a user of the augmented reality glasses;
determining first control information corresponding to the first vision information according to the corresponding relation between the plurality of vision information and the plurality of control information, wherein the plurality of control information comprises the first control information, and the first control information is used for controlling the spatial light modulator to adjust the phase of the optical signal output by the display;
and determining a holding time length of the first sight line direction of the user, and sending the first control information to the spatial light modulator under the condition that the holding time length exceeds a preset time length.
6. The method of claim 5, wherein the step of determining the position of the probe is performed,
the spatial light modulator transmits the adjusted optical signal to the optical component;
the optical assembly is configured to transmit the adjusted optical signal to the eye position of the user.
7. The method according to claim 5 or 6, wherein the correspondence is determined based on aberration information of the optical signal transmitted to the eye position of the user under each control information determined by the aberration detector located at the eye position of the user when the controller sequentially transmits the plurality of control information to the spatial light modulator, and different aberration information corresponds to different vision information.
8. The method according to claim 5 or 6, wherein the determining the first control information corresponding to the first vision information according to the correspondence between the plurality of vision information and the plurality of control information includes: and determining that the spatial light modulator transmits first control information corresponding to the first vision information in the first vision direction according to the corresponding relation between the plurality of vision information and the plurality of control information in each of the plurality of vision directions, wherein the plurality of vision directions comprise the first vision direction.
9. A control device of augmented reality glasses, which is characterized by comprising a memory and a processor; the memory is used for storing programs; the processor is for the method of any of claims 5-8 when the program is run in the processor.
10. A computer-readable storage medium storing computer instructions that, when executed by a computing device, cause the computing device to perform the method of any of claims 5-8.
11. A chip comprising a processor and a data interface, the processor reading instructions stored on a memory via the data interface to perform the method of any of claims 5-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011393843.4A CN114594595B (en) | 2020-12-03 | 2020-12-03 | Glasses control method and device and glasses |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011393843.4A CN114594595B (en) | 2020-12-03 | 2020-12-03 | Glasses control method and device and glasses |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114594595A CN114594595A (en) | 2022-06-07 |
| CN114594595B true CN114594595B (en) | 2023-10-20 |
Family
ID=81812860
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011393843.4A Active CN114594595B (en) | 2020-12-03 | 2020-12-03 | Glasses control method and device and glasses |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114594595B (en) |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005303843A (en) * | 2004-04-14 | 2005-10-27 | Olympus Corp | Display device and image pickup device |
| CN104407438A (en) * | 2014-10-20 | 2015-03-11 | 深圳市亿思达科技集团有限公司 | Intelligent glasses |
| CN105589196A (en) * | 2014-10-20 | 2016-05-18 | 深圳市亿思达科技集团有限公司 | Multifunctional intelligent glasses for correcting eyesight |
| CN105652444A (en) * | 2016-01-18 | 2016-06-08 | 北京国承万通信息科技有限公司 | Display device |
| CN106199958A (en) * | 2015-05-07 | 2016-12-07 | 尚立光电股份有限公司 | Automatic focusing head-mounted display device |
| CN205844631U (en) * | 2016-06-07 | 2016-12-28 | 腾讯科技（深圳）有限公司 | A kind of virtual reality glasses and display device thereof |
| KR101704442B1 (en) * | 2016-11-04 | 2017-02-09 | 한국프라임제약주식회사 | An Eyesight measurement system using a virtual reality device |
| CN106444028A (en) * | 2016-09-27 | 2017-02-22 | 成都虚拟世界科技有限公司 | Near-to-eye display system and virtual reality equipment |
| TW201710740A (en) * | 2015-09-14 | 2017-03-16 | 源奇科技股份有限公司 | Head-mounted display with vision compensation function |
| EP3179289A1 (en) * | 2015-12-08 | 2017-06-14 | Oculus VR, LLC | Focus adjusting virtual reality headset |
| CN107582237A (en) * | 2017-09-28 | 2018-01-16 | 深圳市华星光电技术有限公司 | Wearable display device and vision correction procedure |
| CN108051917A (en) * | 2017-12-08 | 2018-05-18 | 深圳创维新世界科技有限公司 | Augmented reality display optical system and augmented reality display methods |
| CN108287409A (en) * | 2018-04-26 | 2018-07-17 | 北京枭龙科技有限公司 | The nearly eye display optical system of Clairvoyant type |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10025060B2 (en) * | 2015-12-08 | 2018-07-17 | Oculus Vr, Llc | Focus adjusting virtual reality headset |
| US11874530B2 (en) * | 2017-05-17 | 2024-01-16 | Apple Inc. | Head-mounted display device with vision correction |
| EP3698212A4 (en) * | 2017-10-16 | 2021-07-14 | Optica Amuka (A.A.) Ltd. | Spectacles with electrically-tunable lenses controllable by an external system |
-
2020
- 2020-12-03 CN CN202011393843.4A patent/CN114594595B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005303843A (en) * | 2004-04-14 | 2005-10-27 | Olympus Corp | Display device and image pickup device |
| CN104407438A (en) * | 2014-10-20 | 2015-03-11 | 深圳市亿思达科技集团有限公司 | Intelligent glasses |
| CN105589196A (en) * | 2014-10-20 | 2016-05-18 | 深圳市亿思达科技集团有限公司 | Multifunctional intelligent glasses for correcting eyesight |
| CN106199958A (en) * | 2015-05-07 | 2016-12-07 | 尚立光电股份有限公司 | Automatic focusing head-mounted display device |
| TW201710740A (en) * | 2015-09-14 | 2017-03-16 | 源奇科技股份有限公司 | Head-mounted display with vision compensation function |
| EP3179289A1 (en) * | 2015-12-08 | 2017-06-14 | Oculus VR, LLC | Focus adjusting virtual reality headset |
| CN105652444A (en) * | 2016-01-18 | 2016-06-08 | 北京国承万通信息科技有限公司 | Display device |
| CN205844631U (en) * | 2016-06-07 | 2016-12-28 | 腾讯科技（深圳）有限公司 | A kind of virtual reality glasses and display device thereof |
| CN106444028A (en) * | 2016-09-27 | 2017-02-22 | 成都虚拟世界科技有限公司 | Near-to-eye display system and virtual reality equipment |
| KR101704442B1 (en) * | 2016-11-04 | 2017-02-09 | 한국프라임제약주식회사 | An Eyesight measurement system using a virtual reality device |
| CN107582237A (en) * | 2017-09-28 | 2018-01-16 | 深圳市华星光电技术有限公司 | Wearable display device and vision correction procedure |
| CN108051917A (en) * | 2017-12-08 | 2018-05-18 | 深圳创维新世界科技有限公司 | Augmented reality display optical system and augmented reality display methods |
| CN108287409A (en) * | 2018-04-26 | 2018-07-17 | 北京枭龙科技有限公司 | The nearly eye display optical system of Clairvoyant type |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114594595A (en) | 2022-06-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10319154B1 (en) | Methods, systems, and computer readable media for dynamic vision correction for in-focus viewing of real and virtual objects | |
| Chakravarthula et al. | Focusar: Auto-focus augmented reality eyeglasses for both real world and virtual imagery | |
| US10539789B2 (en) | Eye projection system | |
| US11150476B2 (en) | Method for providing a display unit for an electronic information device | |
| US10222621B2 (en) | Head-mounted display apparatus | |
| CN108490611B (en) | Method and device for adjusting refraction of augmented reality equipment and augmented reality equipment | |
| US20180284437A1 (en) | Methods for near-to-eye displays exploiting optical focus and depth information extraction | |
| JP2010540005A (en) | Methods and devices for prevention and treatment of myopia and fatigue | |
| US11054639B2 (en) | Eye projection system | |
| KR102511897B1 (en) | How to Determine Corrective Optical Functions for Virtual Images | |
| US11221487B2 (en) | Method and device of field sequential imaging for large field of view augmented/virtual reality | |
| EP3739375A1 (en) | Stereoscopic glasses, method for designing spectacle lens used in same, and method for observing stereoscopic image | |
| Itoh et al. | Computational phase-modulated eyeglasses | |
| US11256128B2 (en) | Display device | |
| Sutton et al. | Computational glasses: Vision augmentations using computational near-eye optics and displays | |
| CN114594595B (en) | Glasses control method and device and glasses | |
| KR102543452B1 (en) | Augmented Reality Optical System for Correcting the Wearer's Vision Universally | |
| CN113960800B (en) | Augmented reality device, diopter adjustment method thereof, and storage medium | |
| US11513359B1 (en) | Lens | |
| Chakravarthula | [DC] auto-focus augmented reality eyeglasses for both real world and virtual imagery | |
| KR102476727B1 (en) | Augmented Reality Optical System to Correct Weares's Vision | |
| CN114879360A (en) | Head-mounted display device | |
| EP4100782A1 (en) | Variable optical correction using spatially varying polarizers | |
| KR20210083777A (en) | Augmented Reality Optical Device and System with Minimized Exposure of Augmented Reality Image | |
| TWM560606U (en) | System for relieving eye convergence while viewing an electronic image on a display |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |