GB2556121A

GB2556121A - Optical device

Info

Publication number: GB2556121A
Application number: GB1619727.9A
Authority: GB
Inventors: Fares Maha; Valera Salim
Original assignee: Wave Optics Ltd
Current assignee: Wave Optics Ltd
Priority date: 2016-11-22
Filing date: 2016-11-22
Publication date: 2018-05-23
Also published as: GB201619727D0

Abstract

An optical device 2 for use in an augmented reality system comprises a projector 10 having at least three light sources, each with a different colour so as to define a colour gamut, and a waveguide 5 with input and output diffractive optical elements. The input diffractive optical element receives light from the projector and couples it into the waveguide. The output diffractive optical element couples light out of the waveguide towards a viewer. The optical device further comprises a sensor 18 which receives three colours of light from the viewers external environment, and a control unit which controls the energy consumption of each light source based on the light received by the sensor. The control unit may adjust the colour output, drive current or luminance of the light sources, and may have a power saving mode. The optical device may be a headset having the sensor mounted on its frame. The projector may be powered by battery 14.

Description

(71) Applicant(s):

Wave Optics Ltd

Park Drive, Milton Park, ABINGDON, Oxfordshire, OX14 4RY, United Kingdom (72) Inventor(s):

Maha Fares Salim Valera (56) Documents Cited:

WO 2016/011367 A2 US 20160187652 A1 US 20130127980 A1 (58) Field of Search:

INT CL G02B, G06T Other: EPODOC, WPI

US 20160274361 A1 US 20150146301 A1 (74) Agent and/or Address for Service:

Gill Jennings & Every LLP

The Broadgate Tower, 20 Primrose Street, LONDON, EC2A 2ES, United Kingdom (54) Title of the Invention: Optical device

Abstract Title: Augmented reality optical device including external environment light sensor (57) An optical device 2 for use in an augmented reality system comprises a projector 10 having at least three light sources, each with a different colour so as to define a colour gamut, and a waveguide 5 with input and output diffractive optical elements. The input diffractive optical element receives light from the projector and couples it into the waveguide. The output diffractive optical element couples light out of the waveguide towards a viewer. The optical device further comprises a sensor 18 which receives three colours of light from the viewer’s external environment, and a control unit which controls the energy consumption of each light source based on the light received by the sensor. The control unit may adjust the colour output, drive current or luminance of the light sources, and may have a power saving mode. The optical device may be a headset having the sensor mounted on its frame. The projector may be powered by battery 14.

At least one drawing originally filed was informal and the print reproduced here is taken from a later filed formal copy.

1/3

10 17

FIG.1

FIG.2

2/3

10 17

3/3

10 17 y

FIG.5

OPTICAL DEVICE

The present invention relates to an optical device for use in an augmented reality display. In particular, the invention relates to an optical device that enables a power saving mode.

An augmented reality display allows a user to view their surroundings as well as projected images. In military or transportation applications the projected images can be overlaid on the real world perceived by the user. Other applications for these displays include video games and wearable devices, such as glasses.

In a normal set-up for an augmented reality display a transparent display screen is provided in front of a user so that they can continue to see the physical world. The display screen is typically a glass waveguide, and a projector is provided to one side. Light from the projector is coupled into the waveguide by a diffraction grating. The projected light is totally internally reflected within the waveguide. The light is then coupled out of the waveguide by another diffraction grating so that it can be viewed by a user. The projector can provide information and/or images that augment a user's view of the physical world.

A colour projector can be provided. The projector can include LED or OLED pixel elements with at least three light emitters which are typically red, green and blue (RGB). The choices of colour for the light emitters define the colour gamut of the projector. By appropriate mixing, any colour can be achieved within the colour gamut defined by the colour of the RGB light emitters.

Figure 1 is a ClE colour space showing RGB points in a LED projector. The RGB points define a triangle within the CIE colour space. Appropriate mixing of colours from the RGB LEDs can achieve any colour defined within the triangle. A white point W is provided by an even mixture of red, green and blue.

Figure 2 is another CIE colour space showing RGB points for a different LED projector. In this arrangement the RGB points define a smaller triangle within the

CIE colour space. The smaller triangular area of the colour gamut means that a smaller number of colours are available for output in the projected light.

It has been found that projectors for use in AR displays can be power inefficient in certain conditions. Efficient battery use is important for AR displays, and therefore an object of the present invention is to provide an AR device that can achieve efficient energy management through the control of projected light.

According to an aspect of the invention there is provided an optical device for use in an augmented reality system, the optical device comprising: a waveguide; a projector having at least three light sources each with a different colour for defining a colour gamut; an input diffractive optical element configured to receive light from the projector and to couple it into the waveguide; an output diffractive optical element configured to couple light out of the waveguide towards a viewer; a sensor for receiving light from the viewer’s external environment in three respective colours; and a control unit for the projector, wherein the control unit is configured to control the energy consumption of each light source based on the light received by the sensor in the three respective colours.

In this way, the power demands of each light source can be shifted according to environmental lighting conditions in order to optimise the energy usage of the projector. Preferably the energy usage of the projector can be reduced when the colour gamut that can be perceived from the external environment is smaller than the colour gamut than can be achieved by the light sources when operated at full power. This can also improve image perception of the display from the user eye/brain processing perspective.

In certain modes of operation it may be acceptable for the augmented reality system to have reduced freedom with regards projected colours. Specifically, it may be sufficient for the augmented reality system only to project colours that can be perceived by the eye from the external environment. In certain lighting conditions the external environment may have a reduced colour gamut, at least as perceived by the human eye, in comparison to a well-lit outdoor environment; this is particularly so in low lighting conditions and in indoor environments. It these situations it may be possible to reduce the energy used by the projector.

Preferably the control unit is configured to adjust the colour output by the light sources in dependence on the light received by the sensor in the respective colours. Thus, the control unit may be dependent on the chrominance and/or luminance of light received by the sensor from the outside world. The shift in colour of the light sources may facilitate energy saving. Preferably the colours of the light sources are modified in order to provide a better match with the colours received by the sensor in the different colours. The relative output of the light sources may be adjusted in order to provide an improved match with the relative brightness received in the different filters by the sensor.

In low lighting conditions the limits of the human eye mean that a user may be less capable of distinguishing colours. The control unit may be configured to adjust the drive currents supplied to the light sources to make available only those colours that are distinguishable by the user in the ambient lighting conditions.

Preferably the control unit is configured to adjust the drive current supplied to the light sources in dependence on the light received by the sensor in respective colours. The energy consumption of the light sources may be dependent on the drive current with which they are supplied. If the brightness in a particular colour is low in the external environment then preferably the drive current for the corresponding light source is reduced. In this way, the colours that may be provided to the display in projected light may match the colours that are visible in the external environment.

The control unit may be operable to adjust the luminance of each of the three light sources by modulating the drive current. Pulse width modulation may be used effectively to dim the light sources. This may be used to facilitate mixing between the light sources in order to obtain a desired colour within the colour gamut of the projector. One alternative dimming technique involves the use of a variable resistor that can be used to control the voltage drop on the light source. Preferred dimming techniques are those that help to optimise the usage of energy within the projector.

In one arrangement the three light sources are LEDs. LEDs, including OLEDs, are considered to be particularly desirable light sources for use in an augmented reality system. However, other light sources could be used where they can be controlled to adjust their colour output and energy consumption. One alternative light source may be a laser.

Preferably the sensor comprises filters for each of the three respective colours. The sensor preferably detects the external environment’s brightness in three respective colours, such as red, green and blue. The properties of the filters may be substantially matched to the properties of the light sources when driven at full power. The colour gamut of the projector is preferably shifted in dependence on the colour gamut of the external environment, as defined by the brightness of the three respective colours detected by the sensor.

The control unit can preferably be activated in a power saving mode. A switch may be provided on the augmented reality system to enable the power saving mode. The power saving mode may not be appropriate when the augmented reality application demands the use of colours that do cannot be perceived by the user from the external environment. In these circumstances it may be appropriate to deactivate the control unit so that the light sources have a broader range of use.

The optical device may be a headset having a frame and the sensor is mounted on the frame. The sensor is preferably arranged to point away from a viewer, in use. The frame may include legs that extend towards the user and a bridge that extends across the user’s face. The sensor may be mounted in either of the legs or the bridge. A wide variety of headset designs are possible within the scope of the present disclosure.

The device may include a battery for supplying electrical current to the projector. The control unit advantageously conserves battery resources by driving the light sources at colours that are consistent with those found in the external environment. Of course, in alternative arrangements any type of electrical power source could be used.

According to another aspect of the invention there is provided a method of controlling an optical device in an augmented reality system, the method comprising the steps of: providing a waveguide, an input diffractive optical element configured to receive light from a projector and to couple it into the waveguide, and an output diffractive optical element configured to couple light out of the waveguide towards a viewer; sensing light from the viewer’s external environment in three respective colours, using a sensor; providing a projector having at least three light sources each with a different colour for defining a colour gamut; and controlling the energy consumption of each light source in the projector based on the light sensed by the sensor in the three respective colours.

Embodiments of the invention are now described, by way of example, with reference to the drawings, in which:

Figure 1 is a Cl E colour chart showing the colour gamut that can be achieved by a first selection of RGB LEDs in an AR projector;

Figure 2 is a is a ClE colour chart showing the colour gamut that can be achieved by a second selection of RGB LEDs in an AR projector;

Figure 3 is a perspective view of an augmented reality device in an embodiment of the present invention;

Figure 4 is a schematic view of an augmented reality device in an embodiment of the present invention; and

Figure 5 is a Cl E colour chart showing the colour gamut that can be achieved by RGB LEDs in an AR projector in an embodiment of the invention;

An augmented reality headset 2 is now described with reference to Figures 3 and 4. The headset 2 comprises legs 4 that can fit on the side of a user’s head and left and right eyepieces 6, 8 are provided in a transparent waveguide 5 which rests across the user’s face. A projector 10 is provided above a nose bridge 12. An input grating 14 is provided in the nose bridge 12 for coupling light from the projector 10 into the waveguide 5. The input projector 14 is configured to couple light in mutually opposite directions towards the left and right eyepieces 6, 8 respectively. The left and right eyepieces 6, 8 comprise respective output diffraction gratings that are configured to couple light out of the waveguide 5 and towards the user. In this way, augmented reality images can be provided to the user’s eyes and can be overlaid on light received from the physical world.

The headset 2 comprises a rechargeable battery 14 which is configured to supply power to the projector 10. Efficient use of battery resources are considered important if the headset 2 is to have a reasonable life span. A button 16 is provided on one of the legs 4 which can be used to enable a power saving mode.

The headset 2 comprises a sensor 18 mounted in a corner of the waveguide 5. The sensor 18 is arranged to point outwards from the user’s perspective so that it can receive light from the user’s environment. The sensor 18 comprises red, green and blue filter segments 20, 22, 24 for receiving light in these respective wavebands. In this way, the sensor 18 can determine the brightness in the user’s environment in each of these three colours.

The projector 10 includes LED or OLED pixel elements with three light emitters which are typically red, green and blue (RGB). The colours of the light emitters define the colour gamut of the projector 10. Any colour can be achieved within the colour gamut defined by the LED colours in the projector 10. In some embodiments there may be more than three light emitters which may advantageously enable a larger colour gamut to be achieved by the projector 10.

The headset 2 also comprises a control unit 26 for the projector 10. The control unit 26 is configured to receive inputs from the sensor 18 and can control the power consumption of the projector 10 based on the brightness conditions in the external environment.

In use the sensor 18 is configured to measure luminosity in the red, green and blue colour bands. The measured luminosities are sent to the control unit 26. The control unit 26 then calculates an adjustment ΔΙ_Κ, ΔΙ₀, ΔΙ_Β to the drive current for the RGB light emitters in the projector 10 in dependence on the luminosities in the different colour bands.

In good external lighting conditions the luminosities in each of the three colour bands are high. In these circumstances the control unit 26 will calculate a zero adjustment (or only a small adjustment) to the RGB drive currents. Thus, the light emitters are driven at their normal operating current. This provides an output from the projector 10 that has the maximum achievable colour gamut using the available LEDs.

Figure 5 is a CIE colour space showing colours that can be achieved from the projector 10 when there is zero modification to the drive currents by the control unit 26. The colour gamut is defined between the points R^G^B,. These points lie on the monochromatic line that extends around the periphery of the colour chart. Π,,Θ,,Β, define a triangle within the CIE colour space. Appropriate mixing of colours from the RGB LEDs can achieve any colour defined within R^G^B,. In bright external conditions colours may be found within the entire CIE colour space. This includes colours to be found inside the R^G^B, triangle, as well as colours from outside. Thus, the projector 10 can provide a subset of the colours that can be found in the external environment.

In poor external lighting conditions it is possible that the luminosities in all of the three colour bands are low. These conditions may be experienced in low lighting or in indoor environments. In these circumstances the control unit 26 may be configured to calculate a non-zero adjustment to the RGB drive currents in inverse proportion to the luminosity detected in each of the RGB colour bands. Thus, the light emitters are provided with a drive current that is less than their normal operating current. For appropriate light emitters, including LEDs, OLEDs and lasers, this causes a shift in the central wavelength. In addition, it means that the power requirements of the light emitters are reduced.

Figure 5 shows a colour gamut that is defined between the points R₂,G₂,B₂ such as may be achieved when the drive current is modified for each of the RGB light emitters. The R₂,G₂,B₂ triangle has a smaller area than the R₁,G₁,B₁ triangle, which means that a smaller colour gamut is possible from the projector 10. In poor lighting conditions the colour gamut that is experienced by a user from the external environment can be significantly reduced. In certain augmented reality applications it is not considered necessary for the projector 10 to provide colours that cannot be perceived from the external environment. In these applications it is desirable only to project augmented reality colours that are consistent with the external environment. Therefore, the drive currents to the RGB light emitters in the projector 10 can be modified in order to provide a better match between the user’s perception of the external environment colour gamut and the R₂,G₂,B₂ triangle. This can provide an adequate colour gamut for augmented reality. In addition, by reducing the drive currents supplied to the light emitter it is possible to reduce the energy usage of the projector 10 and conserve the lifetime of the battery 14.

The control unit 26 can be enabled by selecting the power saving mode button 16 on one of the legs 4 of the device 2. In certain augmented reality applications it may be desirable to project augmented reality colours that cannot be found in the external environment. These colours can be provided by de-selecting the power saving mode, if required.

The projector 10 can adjust the relative lighting of each of the RGB light emitters using mixing to select the required colour within the available colour gamut. Pulse width modulation (PWM) may be employed to control the brightness of each light emitter.

Claims

1. An optical device for use in an augmented reality system, the optical device comprising:

a waveguide;

a projector having at least three light sources each with a different colour for defining a colour gamut;

an input diffractive optical element configured to receive light from the projector and to couple it into the waveguide;

an output diffractive optical element configured to couple light out of the waveguide towards a viewer;

a sensor for receiving light from the viewer’s external environment in three respective colours; and a control unit for the projector, wherein the control unit is configured to control the energy consumption of each light source based on the light received by the sensor in the three respective colours.

2. The optical device of claim 1, wherein the control unit is configured to adjust the colour output by the light sources in dependence on the light received by the sensor in the respective colours.

3. The optical device of claim 1, wherein the control unit is configured to adjust the drive current supplied to the light sources in dependence on the light received by the sensor in respective colours.

4. The optical device of claim 3, wherein the control unit is operable to adjust the luminance of each of the three light sources by modulating the drive current.

5. The optical device of any of the preceding claims, wherein the three light sources are LEDs.

6. The optical device of any of the preceding claims, wherein the sensor comprises filters for each of the three respective colours.

7. The optical device of any of the preceding claims, wherein the control unit can be activated in a power saving mode.

5

8. The optical device of any of the preceding claims, wherein the optical device is a headset having a frame and the sensor is mounted on the frame.

9. The optical device of any of the preceding claims, further comprising a battery for supplying electrical current to the projector.

10. A method of controlling an optical device in an augmented reality system, the method comprising the steps of:

providing a waveguide, an input diffractive optical element configured to receive light from a projector and to couple it into the waveguide, and an output

15 diffractive optical element configured to couple light out of the waveguide towards a viewer;

sensing light from the viewer’s external environment in three respective colours, using a sensor;

providing a projector having at least three light sources each with a different 20 colour for defining a colour gamut; and controlling the energy consumption of each light source in the projector based on the light sensed by the sensor in the three respective colours.

Intellectual

Property

Office

Application No: GB1619727.9 Examiner: Rob Valkass