CN110596895B - Near-to-eye display device and projection feedback method - Google Patents
Near-to-eye display device and projection feedback method Download PDFInfo
- Publication number
- CN110596895B CN110596895B CN201910733697.6A CN201910733697A CN110596895B CN 110596895 B CN110596895 B CN 110596895B CN 201910733697 A CN201910733697 A CN 201910733697A CN 110596895 B CN110596895 B CN 110596895B
- Authority
- CN
- China
- Prior art keywords
- image
- feedback
- preset object
- optical
- scanning device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B26/00—Optical devices or arrangements for the control of light using movable or deformable optical elements
- G02B26/08—Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
- G02B26/10—Scanning systems
- G02B26/103—Scanning systems having movable or deformable optical fibres, light guides or waveguides as scanning elements
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B26/00—Optical devices or arrangements for the control of light using movable or deformable optical elements
- G02B26/08—Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
- G02B26/10—Scanning systems
- G02B26/106—Scanning systems having diffraction gratings as scanning elements, e.g. holographic scanners
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- Mechanical Optical Scanning Systems (AREA)
Abstract
The invention discloses a near-eye display device and a projection feedback method, wherein the near-eye display device comprises an image light source and a feedback light source; an optical scanning device for scanning the outgoing display image light and the feedback image light; an optical waveguide; the image acquisition unit is used for acquiring feedback image light penetrating through the optical waveguide and generating an acquired image corresponding to the feedback image; a readable storage medium having a program stored thereon, the program when executed by a processor implementing the steps of: comparing the features of the preset object in the acquired image with the features of the preset object in a standard projection image to obtain a comparison result, and adjusting a driving signal of the optical scanning device based on the comparison result. The scheme is used for monitoring the motion trail of the optical fiber in the optical fiber scanning imaging system, and further can provide a feedback signal for compensation control according to the motion trail of the optical fiber, so that the technical problem of projection quality degradation is solved.
Description
Technical Field
The invention relates to the field of near-eye display, in particular to a near-eye display device and a projection feedback method.
Background
The imaging principle of the fiber scanning display technology (FSD) is that a scanning optical fiber is driven by an optical fiber scanner to perform a motion of a predetermined two-dimensional scanning track, and light emitted from a light source is modulated, i.e., light corresponding to each pixel point of an image to be displayed is modulated, and then the light corresponding to each pixel point of the image to be displayed is projected onto a projection plane one by one through the scanning optical fiber, thereby forming a projection picture.
The optical fiber scanner is used as an image source, is easily influenced by factors such as temperature, humidity, vibration, driving fluctuation, aging fatigue and the like in the environment, and is difficult to stably scan and emit light according to a driving signal, so that the actual motion track and state of the optical fiber scanner deviate from an ideal mode, and the projection quality is degraded in the long-time working process.
Disclosure of Invention
The invention aims to provide a near-eye display device and a projection feedback method, which are used for monitoring the motion trail of an optical fiber in an optical fiber scanning imaging system, and further can provide a feedback signal for compensation control according to the motion trail of the optical fiber, so that the technical problem of projection quality degradation is solved.
In order to achieve the above object, a first aspect of an embodiment of the present invention provides a near-eye display device, including an image light source and a feedback light source, where the image light source is configured to emit display image light corresponding to an image to be displayed, and the feedback light source is configured to emit feedback image light corresponding to a feedback image, and the feedback image includes a preset object;
the optical scanning device is simultaneously connected with the image light source and the feedback light source and is used for scanning and emitting the display image light and the feedback image light;
an optical waveguide including a coupling-in unit for coupling light emitted from the optical scanning device partially into the optical waveguide and partially through the optical waveguide;
the image acquisition unit and the optical scanning device are arranged on two sides of the optical waveguide, the image acquisition unit is arranged facing the optical scanning device and is used for acquiring feedback image light penetrating through the optical waveguide and generating an acquired image corresponding to the feedback image;
a readable storage medium having a program stored thereon, the program when executed by a processor implementing the steps of:
comparing the features of the preset object in the acquired image with the features of the preset object in a standard projection image to obtain a comparison result, and adjusting a driving signal of the optical scanning device based on the comparison result.
Optionally, the feature comprises a size and/or a contour of the preset object in the acquired/standard projection image.
Optionally, the optical scanning device is an optical fiber scanner, and the program, when executed by the processor, is configured to implement the steps of comparing the features of the preset object in the acquired image with the features of the preset object in the standard projection image, obtaining a comparison result, and adjusting the driving signal of the optical scanning device based on the comparison result, specifically including the following steps:
calculating a size difference between the size of the preset object in the acquired image and the size of the preset object in a standard projection image;
and adjusting the amplitude of the driving signal of the optical fiber scanner based on the size difference.
Optionally, the optical scanning device is an optical fiber scanner, and the program, when executed by the processor, is configured to implement the steps of comparing the features of the preset object in the acquired image with the features of the preset object in the standard projection image, obtaining a comparison result, and adjusting the driving signal of the optical scanning device based on the comparison result, specifically including the following steps:
comparing whether the contour of the preset object in the acquired image is the same as the contour of the preset object in the standard projection image or not to obtain a comparison result;
and when the comparison results are different, adjusting the phase of the driving signal of the optical fiber scanner to enable the phase of the driving signal of the optical fiber scanner to be synchronous with the phase of the modulation signal of the image light source.
Optionally, the preset object is a character, a character or a pattern.
Optionally, the image acquisition unit is configured to acquire feedback image light transmitted through the optical waveguide in real time to generate the acquired image; or
The image acquisition unit is used for acquiring feedback image light penetrating through the optical waveguide when the near-eye display device is started to generate the acquired image; or
The image acquisition unit is used for acquiring feedback image light penetrating through the optical waveguide at intervals of preset duration after the near-eye display device is started, and generating the acquired image.
Optionally, the coupling-in unit is a coupling-in grating, the first-order diffracted light emitted from the coupling-in grating is coupled into the optical waveguide, and the zero-order diffracted light emitted from the coupling-in grating passes through the optical waveguide.
Optionally, the coupling-in unit is a coupling-in grating, and an incident angle of the feedback image light emitted by the optical scanning device with respect to the coupling-in grating is greater than a diffraction angle bandwidth of the coupling-in grating.
Optionally, the feedback light source is an infrared light source; the image acquisition unit is an infrared image acquisition unit.
A second aspect of the embodiments of the present invention provides a projection feedback method, which is applied to a near-eye display device, where the near-eye display device includes an image light source and a feedback light source, the image light source is used to emit display image light corresponding to an image to be displayed, the feedback light source is used to emit feedback image light corresponding to a feedback image, and the feedback image includes a preset object; the optical scanning device is simultaneously connected with the image light source and the feedback light source and is used for scanning and emitting the display image light and the feedback image light; an optical waveguide including a coupling-in unit for coupling light emitted from the optical scanning device partially into the optical waveguide and partially through the optical waveguide; the image acquisition unit and the optical scanning device are arranged on two sides of the optical waveguide, and the image acquisition unit is arranged facing the optical scanning device; the method comprises the following steps:
acquiring feedback image light penetrating through the optical waveguide through the image acquisition unit to generate an acquired image corresponding to the feedback image;
comparing the features of the preset object in the acquired image with the features of the preset object in a standard projection image to obtain a comparison result, and adjusting a driving signal of the optical scanning device based on the comparison result.
Optionally, the feature comprises a size and/or a contour of the preset object in the acquired/standard projection image.
Optionally, comparing the features of the preset object in the acquired image with the features of the preset object in the standard projection image to obtain a comparison result, and adjusting the driving signal of the optical scanning device based on the comparison result, including the following steps:
calculating a size difference between the size of the preset object in the acquired image and the size of the preset object in a standard projection image;
and adjusting the amplitude of the driving signal of the optical fiber scanner based on the size difference.
Optionally, comparing the features of the preset object in the acquired image with the features of the preset object in the standard projection image to obtain a comparison result, and adjusting the driving signal of the optical scanning device based on the comparison result, including the following steps:
comparing whether the contour of the preset object in the acquired image is the same as the contour of the preset object in the standard projection image or not to obtain a comparison result;
and when the comparison structures are different, adjusting the phase of the driving signal of the optical fiber scanner to enable the phase of the driving signal of the optical fiber scanner to be synchronous with the phase of the modulation signal of the image light source.
Optionally, the preset object is a character, a character or a pattern.
Optionally, the acquiring, by the image acquiring unit, feedback image light penetrating through the optical waveguide to generate an acquired image corresponding to the feedback image includes:
acquiring feedback image light penetrating through the optical waveguide in real time through the image acquisition unit to generate an acquired image; or
When the near-eye display device is started, the image acquisition unit acquires feedback image light penetrating through the optical waveguide to generate an acquired image; or
After the near-to-eye display device is started, feedback image light penetrating through the optical waveguide is collected through the image collecting unit at preset intervals to generate the collected image.
One or more technical solutions in the embodiments of the present invention have at least the following technical effects or advantages:
in the scheme of the embodiment of the invention, the feedback image light emitted by the optical scanning device is collected through the image collecting unit to generate a collected image, the characteristics of the preset object in the collected image are compared with the characteristics of the preset object in the standard projection image to obtain a comparison result, and the driving signal of the optical scanning device is adjusted based on the comparison result.
In addition, in the scheme of the embodiment of the invention, the image acquisition unit is arranged facing the optical scanning device to acquire the light emitted by the feedback light source so as to monitor the motion track of the optical fiber, the structure is simple, the vibration states of the optical scanning device and the optical fiber are not influenced, and the detection feedback is carried out by the light transmitted out of the waveguide by utilizing the property of the waveguide coupling-in unit, so that the utilization rate of the imaging light is not reduced because the light is not used for imaging.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without inventive exercise:
FIGS. 1A and 1B are schematic diagrams of a fiber scanning imaging system according to an embodiment of the present invention;
fig. 2A is a schematic diagram of a possible structure of smart glasses according to an embodiment of the present invention;
fig. 2B is a schematic diagram of another possible structure of smart glasses according to an embodiment of the present invention;
fig. 3A and 3B are schematic structural diagrams of a near-eye display device according to an embodiment of the present invention;
fig. 3C is a block diagram of a near-eye display device according to an embodiment of the invention;
fig. 4 is a schematic flowchart of a projection feedback method according to an embodiment of the present invention;
FIGS. 5A-5F are schematic diagrams of default objects provided in accordance with embodiments of the present invention;
FIG. 6 is a schematic flow chart of a projection feedback method according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of another possible near-eye display device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1A and fig. 1B, fig. 1A and fig. 1B are schematic diagrams of an optical fiber scanning imaging system according to an embodiment of the present invention. The optical fiber scanning imaging system mainly comprises: the device comprises a processor, a scanning driving circuit, a light source module, a light source modulation module, an optical fiber scanner 11, a light source beam combining module 12 and an optical fiber 13. The working principle of the optical fiber scanning imaging system is as follows: the processor drives the optical fiber scanner 11 by sending an electric control signal to the scan driving circuit, and at the same time, the processor controls the light emitting condition of the light source module by sending an electric control signal to the light source modulation module. The signal transmission among the processor, the scanning driving circuit and the light source modulation module can be performed through an electronic input/output device, the light source modulation module outputs a light source modulation signal according to a received control signal to modulate light emitting units (such as a laser/a light emitting diode, and the like, and red, green, blue, and RGB three-color laser shown in fig. 1A) with multiple colors in the light source module, light generated by the light emitting units with each color in the light source module is combined by a light source combining module 12 to generate light corresponding to each pixel point in an image one by one, light beams generated by the light source combining module 12 are guided into an optical fiber scanner 11 through an optical fiber 13, and simultaneously, the scanning driving circuit outputs a scanning driving signal according to the received control signal to control the optical fiber 13 in the optical fiber scanner 11 to perform two-dimensional scanning tracks (such as spiral scanning and grid type scanning) in a predetermined manner, Lissajous scanning), and then the optical system amplifies and projects the light of each pixel point emitted from the optical fiber 13 onto a projection surface to form an image. In the embodiment of the invention, the light emitted by the optical fiber scanning imaging system directly enters human eyes.
Referring to fig. 2A, which is a schematic view of a possible structure of an intelligent glasses according to an embodiment of the present invention, an optical fiber scanner 20 is disposed at a temple of the intelligent glasses, as shown in fig. 2A, the optical fiber scanner 20 is disposed at a connecting end of the temple close to a frame, a light source module 22 is disposed at a terminal end of the temple, the optical fiber scanner 20 is connected to the light source module 22 through an optical fiber 21, an optical system 23 is disposed on an emergent light path of the optical fiber scanner 20, and light emitted from the optical system 23 is guided into a human eye 25 through an optical waveguide 24.
Referring to fig. 2B, which is a schematic view of another possible structure of the smart glasses according to the embodiment of the present invention, the optical fiber scanner 20 is disposed at a frame of the smart glasses, the optical fiber scanner 20 is connected to the light source module 22 through an optical fiber 21, the optical system 23 is disposed on an emergent light path of the optical fiber scanner 20, and light emitted from the optical system 23 is guided into a human eye 25 through an optical waveguide 24.
Referring to fig. 3A to 3C, fig. 3A and 3B are schematic structural diagrams of a near-eye display device according to an embodiment of the present invention, and fig. 3C is a schematic block diagram of the near-eye display device according to the embodiment of the present invention, where the near-eye display device includes an image light source 30 and a feedback light source 31, the image light source 30 emits display image light corresponding to an image to be displayed, the feedback light source 31 emits feedback image light corresponding to a feedback image, and the feedback image includes a preset object; and an optical scanning device 32 connected to the image light source 30 and the feedback light source 31, wherein the optical scanning device 32 is configured to scan and emit the display image light and the feedback image light. The image light source 30 may be a RGB three-color laser or other types of light sources, and the feedback light source 31 may be an invisible light source, such as: infrared laser light source or ultraviolet laser light source, etc. The optical scanning device 32 may be a fiber scanner, or may be a MEMS (micro electro Mechanical Systems) scanning mirror or the like. The preset object is a character, a pattern, or the like, which is not limited in the present invention.
And an optical waveguide 33 including a coupling-in unit 330, wherein the coupling-in unit 330 is configured to couple light emitted from the optical scanning device 32 into the optical waveguide 33 partially as shown by a solid line in fig. 3A and partially transmit through the optical waveguide 33 as shown by a dotted line in fig. 3A.
The optical scanning device 32 and the image acquisition unit 34 are arranged on two sides of the optical waveguide 33, the image acquisition unit 34 faces the optical scanning device 32, and the image acquisition unit 34 is used for acquiring feedback image light penetrating through the optical waveguide 33 and generating an acquired image corresponding to the feedback image. The image capturing unit 34 may be implemented by a device having an imaging function, such as a camera or a video camera. Readable storage medium 35 includes, but is not limited to, disk storage, CD-ROM (Compact disk Read-Only Memory), optical storage, and the like. The readable storage medium 35 has stored thereon a program which, when executed by the processor 36, performs the steps of: the features of the preset object in the acquired image and the features of the preset object in the standard projection image are compared to obtain a comparison result, and the driving signal of the optical scanning device 32 is adjusted based on the comparison result.
In the scheme, the comparison result can reflect the motion trail of the end face of the optical fiber, so that the motion trail of the optical fiber in the optical fiber scanning imaging system can be monitored, and further, a feedback signal can be provided according to the motion trail of the optical fiber for compensation control, so that the technical problem of projection quality degradation is solved. In addition, in the scheme of the embodiment of the invention, the image acquisition unit 34 is arranged facing the optical scanning device 32 to acquire the light emitted by the feedback light source 31 so as to monitor the motion track of the optical fiber, the structure is simple, the vibration states of the optical scanning device 32 and the optical fiber are not influenced, and the utilization rate of the imaging light is not reduced.
In an embodiment of the present invention, the feature of the preset object in the acquired image includes a size and/or a contour of the preset object in the acquired image, and correspondingly, the feature of the preset object in the standard projection image includes a size and/or a contour of the preset object in the standard acquired image. The standard projection image may be a projection image obtained through parameter simulation calculation, or may be a projection image of the near-eye display device in an ideal non-interference environment, which is not limited in the present invention. In the embodiment of the present invention, the comparison of the size and/or the profile of the preset object is performed because when the amplitude of the optical fiber scanner changes, the size of the preset object in the captured feedback image changes due to the change of the motion trajectory of the end face of the optical fiber along with the amplitude change of the optical fiber scanner, and therefore, the size characteristic can be detected to reflect the amplitude change of the optical fiber scanner and the change of the motion trajectory of the end face of the optical fiber. When the amplitude of the optical fiber scanner changes little or unchanged and the phase of the optical fiber scanner changes (generally, the scanning phase changes in the horizontal direction), the vertical line in the preset object profile will be split, and one vertical line will be split into two vertical lines, so that the profile characteristic can be used as a characteristic to detect, and the phase change of the optical fiber scanner can be reflected.
In this embodiment of the present invention, the feedback light source 31 may be an infrared laser light source, and the infrared laser light source may modulate and output a feedback image at any part of the display screen, where the feedback image includes a preset object, and the preset object is a character, a pattern, or the like. In the embodiment of the present invention, it is assumed that the preset object is a "farm" character. Correspondingly, the camera which is used for detecting the motion trail of the optical fiber end face of the optical fiber scanner works in an infrared mode, and the requirement on the camera for small-field shooting is lower, so that enough sampling precision can be realized without high pixel density, and the camera can only shoot the 'field' character area; alternatively, an optical element group (similar to a magnifier) with a magnifying function is arranged in front of the camera, so that the character area of the 'farmland' is magnified in front of the camera. Therefore, the pixels of the whole camera are used for shooting a part of the projection picture, so that the shooting result can be more accurate.
Next, the scheme in the embodiment of the present invention will be described by taking the optical scanning device 32 as an optical fiber scanner and the preset object as a "field" character.
In a possible embodiment, as shown in fig. 4, the program stored on the readable storage medium when executed by the processor for implementing the steps of comparing the feature of the preset object in the acquired image with the feature of the preset object in the standard projection image to obtain a comparison result, and adjusting the driving signal of the optical scanning device based on the comparison result specifically includes the following steps.
In the embodiment of the present invention, the near-eye display device may store the corresponding relationship between the size difference and the adjustment amplitude of the driving signal in advance, and in step 402, the adjustment may be performed directly according to the corresponding relationship between the size difference and the adjustment amplitude of the driving signal. Alternatively, a driving signal adjustment amplitude value may be stored in the near-eye display device in advance, and in step 402, the adjustment amplitude value may be adjusted directly according to the driving signal.
In the embodiment of the present invention, when the size difference calculated in step 401 is not zero, the amplitude of the driving signal of the optical fiber scanner may be adjusted, and of course, a difference threshold may also be preset, and when the size difference calculated in step 401 is greater than or equal to the difference threshold, the amplitude of the driving signal of the optical fiber scanner is adjusted, and no matter which way is used to determine the adjustment amplitude of the driving signal, step 401 and step 402 may be repeatedly executed until the size difference between the preset objects in the ideal state and the actual state is controlled to be zero or the size difference is controlled to be below the difference threshold, so as to eliminate the amplitude variation of the optical fiber scanner, or control the amplitude variation of the optical fiber scanner to be below the amplitude threshold.
In the embodiment of the present invention, in order to improve the resolution of an image, multiple scanners are often required to be spliced, and as shown in fig. 5E and 5F, the projection images of two fiber scanners are spliced. In the manner described above, the first fiber scanner projects a "farm" character at any location, and the second fiber scanner projects another "farm" character in the vicinity of the "farm" character. In this way, the camera can simultaneously feed back the track conditions of the two scanners by only shooting the double-field character area (the thickened frame area), so that the motion conditions of the scanning optical fibers of the optical fiber scanners are fed back through one camera.
In another possible embodiment, as shown in fig. 6, the program stored on the readable storage medium is executed by a processor to compare the feature of the preset object in the acquired image with the feature of the preset object in a standard projection image, obtain a comparison result, and adjust the driving signal of the fiber scanner based on the comparison result, and specifically includes the following steps.
In the embodiment of the present invention, the feature of the preset object may be other features besides the size and the outline of the preset object, which is not limited in the present invention.
In the embodiment of the present invention, the coupling unit 330 is configured to couple part of the light emitted from the optical scanning device 32 into the optical waveguide 33 and part of the light passes through the optical waveguide 33. In one possible embodiment, as shown in fig. 3A and 3B, the image capturing unit 34 is disposed opposite to the optical waveguide 33 of the optical scanning device 32 along the exit optical path of the optical scanning device 32, the coupling-in unit 330 is a coupling-in grating, the exit light of the optical scanning device 32 is collimated by the optical system 37 and then enters the optical waveguide 33 via the coupling-in grating to propagate as +1/-1 st diffraction light (shown by a solid line in fig. 3A and 3B), but the 0 th diffraction light (shown by a dotted line in fig. 3A and 3B) of the exit light of the optical scanning device 32 directly enters the image capturing unit 34. It should be noted that, in fig. 3A, the 0 th order diffracted light is projected into the camera, and the three light rays represent three fields of view, which are not parallel light, and are shown in fig. 3A only for indication and do not represent actual light path traces.
In another possible implementation manner, as shown in fig. 7, which is a schematic diagram of another possible arrangement manner of the image capturing unit in the embodiment of the present invention, wherein the incoupling unit 330 is an incoupling grating, and since the incoupling grating has a diffraction angle bandwidth (generally, around 35 °), and the incoupling grating has little diffraction effect on the incident light with an incident angle exceeding the diffraction angle bandwidth, the incoupling grating directly transmits the incoupling grating. Therefore, the incident angle of the light emitted from the optical scanning device 32 with respect to the coupling grating may be slightly larger than the diffraction angle bandwidth, for example, 35 °, the designed incident angle is 0 ° - (35+ n) °, and n may be a relatively small value such as 1, 2, and the like. Thus 0-35 of the incident light is propagated by the incoupling waveguide and the remaining n of the incident light (as indicated by the dashed lines in fig. 7) is transmitted through the incoupling grating into the corresponding camera head.
It should be noted that, in the embodiment corresponding to fig. 3A and 3B, the feedback light source may modulate and output the feedback image at any part of the display screen, and for the embodiment corresponding to fig. 7, the feedback image may be projected only by the incident light at the above-mentioned remaining n °, that is, the projection of the feedback image is realized by the display area with fixed edge, and further the detection and correction of the optical fiber motion trajectory are performed, so that a camera with a large field angle FOV is not required, which is favorable for reducing the cost and the requirement for hardware model selection.
In the embodiment of the invention, the image shot by the camera is used for feeding back the motion track condition of the scanning optical fiber in real time or non-real time, so that the motion track of the scanning optical fiber is corrected in real time or non-real time, and the display effect of a projected image is improved.
In the embodiment of the invention, if the motion track of the optical fiber in the optical fiber scanning imaging system needs to be monitored in real time, the feedback image light penetrating through the optical waveguide can be collected in real time through the image collecting unit to generate the collected image, then the characteristics of the preset object in the collected image and the standard projection image are compared, and a feedback signal is provided for the optical scanning device to be used for real-time compensation control based on the comparison result.
In another possible embodiment, the correction may also be non-real-time, and the feedback image light transmitted through the optical waveguide may be collected by the image collecting unit at the time of starting the near-eye display device to generate the collected image, and then, the characteristics of the preset object in the collected image and the standard projection image are compared, and based on the comparison result, a feedback signal is provided to the optical scanning device for compensation control; or after the near-eye display device is started, at a preset time interval, acquiring feedback image light penetrating through the optical waveguide through the image acquisition unit to generate the acquired image, comparing the characteristics of the preset object in the acquired image and the standard projection image, and providing a feedback signal for the optical scanning device for compensation control based on the comparison result.
In the embodiment of the invention, the following purposes are mainly realized through the detection of the camera: 1. scanning amplitude variation caused by performance degradation and fluctuation of the optical scanning device; 2. the problem of cross coupling of two driving shafts of a scanning track caused by strong nonlinearity; 3. in response to the problem of image display position mismatch caused by phase drift (causing blurring of the projected image). Because the problems are all accompanied by obvious image display characteristics, detection, identification and correction can be conveniently carried out. According to the practical engineering experience, the optical scanner can be ensured to generate harmonic response all the time through the process, and the method is also the basic premise of the scheme.
Based on the same inventive concept, an embodiment of the present invention further provides a projection feedback method, which is applied to a near-eye display device, where the near-eye display device includes an image light source and a feedback light source, the image light source is used to emit display image light corresponding to an image to be displayed, the feedback light source is used to emit feedback image light corresponding to a feedback image, and the feedback image includes a preset object; the optical scanning device is simultaneously connected with the image light source and the feedback light source and is used for scanning and emitting the display image light and the feedback image light; an optical waveguide including a coupling-in unit for coupling light emitted from the optical scanning device partially into the optical waveguide and partially through the optical waveguide; the image acquisition unit and the optical scanning device are arranged on two sides of the optical waveguide, and the image acquisition unit is arranged facing the optical scanning device; the method comprises the following steps:
acquiring feedback image light penetrating through the optical waveguide through the image acquisition unit to generate an acquired image corresponding to the feedback image;
comparing the features of the preset object in the acquired image with the features of the preset object in a standard projection image to obtain a comparison result, and adjusting a driving signal of the optical scanning device based on the comparison result.
Optionally, the feature comprises a size and/or a contour of the preset object in the acquired/standard projection image.
Optionally, comparing the features of the preset object in the acquired image with the features of the preset object in the standard projection image to obtain a comparison result, and adjusting the driving signal of the optical scanning device based on the comparison result, including the following steps:
calculating a size difference between the size of the preset object in the acquired image and the size of the preset object in a standard projection image;
and adjusting the amplitude of the driving signal of the optical fiber scanner based on the size difference.
Optionally, comparing the features of the preset object in the acquired image with the features of the preset object in the standard projection image to obtain a comparison result, and adjusting the driving signal of the optical scanning device based on the comparison result, including the following steps:
comparing whether the contour of the preset object in the acquired image is the same as the contour of the preset object in the standard projection image or not to obtain a comparison result;
and when the comparison structures are different, adjusting the phase of the driving signal of the optical fiber scanner to enable the phase of the driving signal of the optical fiber scanner to be synchronous with the phase of the modulation signal of the image light source.
Optionally, the preset object is a character, a character or a pattern.
Optionally, the acquiring, by the image acquiring unit, feedback image light penetrating through the optical waveguide to generate an acquired image corresponding to the feedback image includes:
acquiring feedback image light penetrating through the optical waveguide in real time through the image acquisition unit to generate an acquired image; or
When the near-eye display device is started, the image acquisition unit acquires feedback image light penetrating through the optical waveguide to generate an acquired image; or
After the near-to-eye display device is started, feedback image light penetrating through the optical waveguide is collected through the image collecting unit at preset intervals to generate the collected image.
Various modifications and specific examples of the near-eye display device in the embodiments of fig. 1A to fig. 7 are also applicable to the projection feedback method in the embodiments of the present invention, and the implementation method of the projection feedback method in the embodiments of the present invention is clear to those skilled in the art from the detailed description of the near-eye display device, so for the brevity of the description, detailed descriptions are omitted here.
One or more technical solutions in the embodiments of the present invention have at least the following technical effects or advantages:
in the scheme of the embodiment of the invention, the feedback image light emitted by the optical scanning device is collected through the image collecting unit to generate the collected image, the characteristics of the preset object in the collected image are compared with the characteristics of the preset object in the standard projection image to obtain a comparison result, and the driving signal of the optical scanning device is adjusted based on the comparison result.
In addition, in the scheme of the embodiment of the invention, the movement track of the optical fiber is monitored by arranging the image acquisition unit facing the optical scanning device to acquire the light emitted by the feedback light source, the structure is simple, the vibration states of the optical scanning device and the optical fiber are not influenced, and the utilization rate of the imaging light is not reduced.
All of the features disclosed in this specification, or all of the steps in any method or process so disclosed, may be combined in any combination, except combinations of features and/or steps that are mutually exclusive.
Any feature disclosed in this specification (including any accompanying claims, abstract and drawings), may be replaced by alternative features serving equivalent or similar purposes, unless expressly stated otherwise. That is, unless expressly stated otherwise, each feature is only an example of a generic series of equivalent or similar features.
The invention is not limited to the foregoing embodiments. The invention extends to any novel feature or any novel combination of features disclosed in this specification and any novel method or process steps or any novel combination of features disclosed.
Claims (13)
1. The near-eye display device is characterized by comprising an image light source and a feedback light source, wherein the image light source is used for emitting display image light corresponding to an image to be displayed, the feedback light source is used for emitting feedback image light corresponding to a feedback image, and the feedback image comprises a preset object;
the optical scanning device is simultaneously connected with the image light source and the feedback light source and is used for scanning and emitting the display image light and the feedback image light;
an optical waveguide including a coupling-in unit for coupling light emitted from the optical scanning device partially into the optical waveguide and partially through the optical waveguide;
the image acquisition unit and the optical scanning device are arranged on two sides of the optical waveguide, the image acquisition unit is arranged facing the optical scanning device and is used for acquiring feedback image light penetrating through the optical waveguide and generating an acquired image corresponding to the feedback image;
a readable storage medium having a program stored thereon, the program when executed by a processor implementing the steps of:
comparing the features of the preset object in the acquired image with the features of the preset object in a standard projection image to obtain a comparison result, and adjusting a driving signal of the optical scanning device based on the comparison result; wherein the features comprise dimensions and contours of the pre-set object in the acquired image and standard projection image, the dimensions reflecting amplitude variations of the optical scanning device, and the contours reflecting phase variations of the optical scanning device.
2. The near-eye display device according to claim 1, wherein the optical scanning device is an optical fiber scanner, and the program when executed by the processor performs the steps of comparing the feature of the preset object in the captured image with the feature of the preset object in a standard projection image, obtaining a comparison result, and adjusting the driving signal of the optical scanning device based on the comparison result, specifically comprising the steps of:
calculating a size difference between the size of the preset object in the acquired image and the size of the preset object in a standard projection image;
and adjusting the amplitude of the driving signal of the optical fiber scanner based on the size difference.
3. The near-eye display device according to claim 1, wherein the optical scanning device is an optical fiber scanner, and the program when executed by the processor performs the steps of comparing the feature of the preset object in the captured image with the feature of the preset object in a standard projection image, obtaining a comparison result, and adjusting the driving signal of the optical scanning device based on the comparison result, specifically comprising the steps of:
comparing whether the contour of the preset object in the acquired image is the same as the contour of the preset object in the standard projection image or not to obtain a comparison result;
and when the comparison results are different, adjusting the phase of the driving signal of the optical fiber scanner to enable the phase of the driving signal of the optical fiber scanner to be synchronous with the phase of the modulation signal of the image light source.
4. The near-eye display device of any one of claims 1-3 wherein the predetermined object is a word, character, or pattern.
5. The near-eye display device of claim 1, wherein the image capture unit is configured to capture feedback image light transmitted through the optical waveguide in real time to generate the captured image; or
The image acquisition unit is used for acquiring feedback image light penetrating through the optical waveguide when the near-eye display device is started to generate the acquired image; or
The image acquisition unit is used for acquiring feedback image light penetrating through the optical waveguide at intervals of preset duration after the near-eye display device is started, and generating the acquired image.
6. The near-eye display device of claim 1, wherein the incoupling unit is an incoupling grating, wherein the first order diffracted light emitted from the incoupling grating is coupled into the optical waveguide, and the zero order diffracted light emitted from the incoupling grating is transmitted through the optical waveguide.
7. The near-eye display device of claim 1, wherein the incoupling unit is an incoupling grating, and an incident angle of the feedback image light emitted by the optical scanning device with respect to the incoupling grating is larger than a diffraction angle bandwidth of the incoupling grating.
8. The near-eye display device of claim 1, wherein the feedback light source is an infrared light source; the image acquisition unit is an infrared image acquisition unit.
9. A projection feedback method is applied to a near-eye display device, and is characterized in that the near-eye display device comprises an image light source and a feedback light source, wherein the image light source is used for emitting display image light corresponding to an image to be displayed, the feedback light source is used for emitting feedback image light corresponding to a feedback image, and the feedback image comprises a preset object; the optical scanning device is simultaneously connected with the image light source and the feedback light source and is used for scanning and emitting the display image light and the feedback image light; an optical waveguide including a coupling-in unit for coupling light emitted from the optical scanning device partially into the optical waveguide and partially through the optical waveguide; the image acquisition unit and the optical scanning device are arranged on two sides of the optical waveguide, and the image acquisition unit is arranged facing the optical scanning device; the method comprises the following steps:
acquiring feedback image light penetrating through the optical waveguide through the image acquisition unit to generate an acquired image corresponding to the feedback image;
comparing the features of the preset object in the acquired image with the features of the preset object in a standard projection image to obtain a comparison result, and adjusting a driving signal of the optical scanning device based on the comparison result; wherein the features comprise dimensions and contours of the pre-set object in the acquired image and standard projection image, the dimensions reflecting amplitude variations of the optical scanning device, and the contours reflecting phase variations of the optical scanning device.
10. The method as claimed in claim 9, wherein the optical scanning device is a fiber scanner, the features of the preset object in the acquired image and the features of the preset object in a standard projection image are compared to obtain a comparison result, and the driving signal of the optical scanning device is adjusted based on the comparison result, comprising the steps of:
calculating a size difference between the size of the preset object in the acquired image and the size of the preset object in a standard projection image;
and adjusting the amplitude of the driving signal of the optical fiber scanner based on the size difference.
11. The method as claimed in claim 9, wherein the optical scanning device is a fiber scanner, the features of the preset object in the acquired image and the features of the preset object in a standard projection image are compared to obtain a comparison result, and the driving signal of the optical scanning device is adjusted based on the comparison result, comprising the steps of:
comparing whether the contour of the preset object in the acquired image is the same as the contour of the preset object in the standard projection image or not to obtain a comparison result;
and when the comparison structures are different, adjusting the phase of the driving signal of the optical fiber scanner to enable the phase of the driving signal of the optical fiber scanner to be synchronous with the phase of the modulation signal of the image light source.
12. The method of claim 9, wherein the predetermined object is a letter, a character or a pattern.
13. The method according to claim 9, wherein the acquiring, by the image acquisition unit, the feedback image light transmitted through the optical waveguide to generate the acquired image corresponding to the feedback image includes:
acquiring feedback image light penetrating through the optical waveguide in real time through the image acquisition unit to generate an acquired image; or
When the near-eye display device is started, the image acquisition unit acquires feedback image light penetrating through the optical waveguide to generate an acquired image; or
After the near-to-eye display device is started, feedback image light penetrating through the optical waveguide is collected through the image collecting unit at preset intervals to generate the collected image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910733697.6A CN110596895B (en) | 2019-08-09 | 2019-08-09 | Near-to-eye display device and projection feedback method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910733697.6A CN110596895B (en) | 2019-08-09 | 2019-08-09 | Near-to-eye display device and projection feedback method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110596895A CN110596895A (en) | 2019-12-20 |
| CN110596895B true CN110596895B (en) | 2021-10-29 |
Family
ID=68853835
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910733697.6A Active CN110596895B (en) | 2019-08-09 | 2019-08-09 | Near-to-eye display device and projection feedback method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110596895B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113747129A (en) * | 2020-05-28 | 2021-12-03 | 成都理想境界科技有限公司 | Time delay correction method and scanning display device |
| CN113741123B (en) * | 2020-05-28 | 2022-12-30 | 成都理想境界科技有限公司 | Projection correction method and optical fiber scanning imaging equipment |
| CN114967109B (en) * | 2021-02-25 | 2024-01-05 | 成都理想境界科技有限公司 | Image correction method and optical fiber scanning imaging system |
| CN113554150B (en) * | 2021-07-02 | 2023-07-18 | 中国计量大学 | Single-fiber scanner light spot track detection method and system based on deep learning |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105700297A (en) * | 2014-11-27 | 2016-06-22 | 上海微电子装备有限公司 | Amplitude monitoring system, focus leveling device and defocusing amount detection method |
| CN105739225A (en) * | 2014-12-24 | 2016-07-06 | Prysm公司 | Scanning Beam Display System |
| JP2016526177A (en) * | 2013-05-02 | 2016-09-01 | マイクロビジョン，インク． | High efficiency laser modulation |
| CN107390362A (en) * | 2017-08-20 | 2017-11-24 | 华中科技大学 | A kind of fiber spiral scanner and its braking method and application |
| CN107870427A (en) * | 2016-09-26 | 2018-04-03 | 精工爱普生株式会社 | Head-mount type display unit |
| CN107924103A (en) * | 2015-06-15 | 2018-04-17 | 奇跃公司 | Display system with the optical element for interior coupling multiplexing light stream |
| KR20180066166A (en) * | 2015-10-05 | 2018-06-18 | 매직 립, 인코포레이티드 | Microlens collimator for scanning optical fiber in virtual / augmented reality system |
| CN207689753U (en) * | 2017-10-31 | 2018-08-03 | 成都理想境界科技有限公司 | A kind of optical fiber scanning projection arrangement and projection device |
| CN108431583A (en) * | 2015-12-22 | 2018-08-21 | 微视公司 | System and method for the feedback control in scanning projector |
| CN108549188A (en) * | 2018-04-24 | 2018-09-18 | 歌尔股份有限公司 | The position phase method of adjustment of laser scanning projection's instrument |
| CN108803007A (en) * | 2017-10-31 | 2018-11-13 | 成都理想境界科技有限公司 | One mode control method and laser scanning device |
| CN109031656A (en) * | 2018-07-27 | 2018-12-18 | 成都理想境界科技有限公司 | A kind of optical fiber scanning projection arrangement, equipment and detection method |
| JP2019502142A (en) * | 2015-10-06 | 2019-01-24 | マジック リープ， インコーポレイテッドＭａｇｉｃ Ｌｅａｐ，Ｉｎｃ． | Virtual / augmented reality system with inverted angle grating |
| US10228565B1 (en) * | 2016-05-27 | 2019-03-12 | Facebook Technologies, Llc | Variable focus waveguide display |
| CN109613790A (en) * | 2018-11-19 | 2019-04-12 | 成都理想境界科技有限公司 | A kind of laser projection optics module and near-eye display device |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7283250B2 (en) * | 2004-01-16 | 2007-10-16 | Veeco Instruments, Inc. | Measurement of object deformation with optical profiler |
| DE102005002190B4 (en) * | 2005-01-17 | 2007-04-12 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Scanner and method for operating a scanner |
| US8757812B2 (en) * | 2008-05-19 | 2014-06-24 | University of Washington UW TechTransfer—Invention Licensing | Scanning laser projection display devices and methods for projecting one or more images onto a surface with a light-scanning optical fiber |
| CN207689757U (en) * | 2017-12-28 | 2018-08-03 | 成都理想境界科技有限公司 | A kind of optical fiber scanning projection device |
-
2019
- 2019-08-09 CN CN201910733697.6A patent/CN110596895B/en active Active
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2016526177A (en) * | 2013-05-02 | 2016-09-01 | マイクロビジョン，インク． | High efficiency laser modulation |
| CN105700297A (en) * | 2014-11-27 | 2016-06-22 | 上海微电子装备有限公司 | Amplitude monitoring system, focus leveling device and defocusing amount detection method |
| CN105739225A (en) * | 2014-12-24 | 2016-07-06 | Prysm公司 | Scanning Beam Display System |
| CN107924103A (en) * | 2015-06-15 | 2018-04-17 | 奇跃公司 | Display system with the optical element for interior coupling multiplexing light stream |
| KR20180066166A (en) * | 2015-10-05 | 2018-06-18 | 매직 립, 인코포레이티드 | Microlens collimator for scanning optical fiber in virtual / augmented reality system |
| JP2019502142A (en) * | 2015-10-06 | 2019-01-24 | マジック リープ， インコーポレイテッドＭａｇｉｃ Ｌｅａｐ，Ｉｎｃ． | Virtual / augmented reality system with inverted angle grating |
| CN108431583A (en) * | 2015-12-22 | 2018-08-21 | 微视公司 | System and method for the feedback control in scanning projector |
| JP2019509507A (en) * | 2015-12-22 | 2019-04-04 | マイクロビジョン，インク． | System and method for feedback control in a scanning projector |
| US10228565B1 (en) * | 2016-05-27 | 2019-03-12 | Facebook Technologies, Llc | Variable focus waveguide display |
| CN107870427A (en) * | 2016-09-26 | 2018-04-03 | 精工爱普生株式会社 | Head-mount type display unit |
| CN107390362A (en) * | 2017-08-20 | 2017-11-24 | 华中科技大学 | A kind of fiber spiral scanner and its braking method and application |
| CN207689753U (en) * | 2017-10-31 | 2018-08-03 | 成都理想境界科技有限公司 | A kind of optical fiber scanning projection arrangement and projection device |
| CN108803007A (en) * | 2017-10-31 | 2018-11-13 | 成都理想境界科技有限公司 | One mode control method and laser scanning device |
| CN108549188A (en) * | 2018-04-24 | 2018-09-18 | 歌尔股份有限公司 | The position phase method of adjustment of laser scanning projection's instrument |
| CN109031656A (en) * | 2018-07-27 | 2018-12-18 | 成都理想境界科技有限公司 | A kind of optical fiber scanning projection arrangement, equipment and detection method |
| CN109613790A (en) * | 2018-11-19 | 2019-04-12 | 成都理想境界科技有限公司 | A kind of laser projection optics module and near-eye display device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110596895A (en) | 2019-12-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110596895B (en) | Near-to-eye display device and projection feedback method | |
| US5982555A (en) | Virtual retinal display with eye tracking | |
| US6560028B2 (en) | Virtual retinal display with eye tracking | |
| CN1042386C (en) | Video projection system | |
| CN107637071B (en) | Projection device, projection method, projection module, electronic device, and program | |
| CN102648431B (en) | Image projection device, image protection method, distance measuring device and distance measuring method | |
| JP7221362B2 (en) | Double Mixed Light Irradiation Field Device | |
| EP1006857B1 (en) | Point source scanning apparatus and method | |
| US7023621B2 (en) | Device and method for the laser projection of high-resolution images onto the retina of the eye, superimposed with the image content of the field of vision | |
| US20020171810A1 (en) | System and method for displaying/projecting a color image | |
| CN103676145A (en) | Image display apparatus and optical component | |
| US10114215B1 (en) | Scanning laser devices with reduced exit pupil disparity | |
| US6639719B2 (en) | System and method for using multiple beams to respectively scan multiple regions of an image | |
| WO2016203992A1 (en) | Projection device, projection method, projection module, electronic device, and program | |
| US7180556B2 (en) | System and method for capturing, transmitting, and displaying an image | |
| CN110891533A (en) | Eye projection system and method with focus management | |
| US7180555B2 (en) | System and method for producing an image with a screen using erase (off) and image (on) light sources | |
| US20110176068A1 (en) | Closed Loop Feedback for Electronic Beam Alignment | |
| JP2017021131A (en) | Optical scanner, image display device, object device and optical scanning method | |
| CN114967143B (en) | Near-to-eye display device | |
| CN109348210A (en) | Image source mould group, near-eye display system, control method and near-eye display device | |
| CN112130321B (en) | Waveguide module, near-to-eye display module based on waveguide and near-to-eye display equipment | |
| CN103116219A (en) | Scanning projection device with detecting function and detection methods thereof | |
| CN216561219U (en) | Light adjusting structure, display module and projection display equipment | |
| CN116068752A (en) | Scanning driving method and scanning device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |