EP3447756A1

EP3447756A1 - Method for sensing light being incident on an electronic device

Info

Publication number: EP3447756A1
Application number: EP17187115.5A
Authority: EP
Inventors: Steven Li
Original assignee: Ams AG
Current assignee: Ams Osram AG
Priority date: 2017-08-21
Filing date: 2017-08-21
Publication date: 2019-02-27

Abstract

A method is suggested for sensing light being incident on an electronic device. The electronic device comprises a display (11) and a light sensor arrangement (12) mounted behind the display (11) such as to receive incident light through the display. The method comprises periodically switching the display (11) on and off, wherein a frame (FR) is defined by an on-state (ON) of the display (11). In a first frame (FR1) a display brightness is set to a first level (DL1). Then, a first frame count (FC1) is determined by integrating the incident light by means of the light sensor arrangement (12) during the first frame (FR1). In a second frame (FR2) the display brightness is set to a second level (DL2) and a second frame count (FC2) is generated by integrating the incident light by means of the light sensor arrangement (12) during the second frame (FR2). Finally, an ambient light level (AL) is determined depending on the first frame count (FC1) and the second frame count (FC2).

Description

The invention relates to a method for light sensing, in particular for light sensing of ambient light incident on a display.
Light sensors, such as ambient light sensors (ALS) and color sensors, are widely used in mobile devices such as mobile phones, smartphones, computers and tablets. ALS and color sensors provide information about the level and color of ambient light, respectively, which can be used to support display control such as a backlight LED power circuit. For example, the backlight of a LCD panel in a smartphone accounts for around 40% of the device's total power consumption. Thus, there is a great benefit to be gained from adjusting its brightness in response to changes in ambient light levels. In a relatively dark environment, the brightness of the display can be reduced to save power. This is also easier on the user's eyes, and so improves the user experience at the same time.
As smartphone bezels keep getting narrower, often there is not enough place to mount a light sensor to measure the illuminance and correlated color temperature (CCT) of the environment under the touch panel. Customers demand placing the light sensor directly under the display panel to measure the illuminance and CCT of the environment. One of the challenges of measuring the ambient light is to remove the stray light generated by the display. Furthermore, the light sensor often has additional functionality such as proximity detection as thus should be placed close to the display in order to function correctly. These and other constraints limit the freedom for the design engineer to optimize the design of the light sensor.
It is an object to provide a method for light sensing that allows for an improved detection of light sources such as ambient light in order to facilitate for reliable display control.
This objective is achieved by the subject matter of the independent claim. Further developments and embodiments are described in dependent claims.
It is to be understood that any feature described hereinafter in relation to any one embodiment may be used alone, or in combination with other features described hereinafter, and may also be used in combination with one or more features of any other of the embodiments, or any combination of any other of the embodiments, unless explicitly described as an alternative. Furthermore, equivalents and modifications not described below may also be employed without departing from the scope of the method for light sensing as defined in the accompanying claims.
The following relates an improved concept for sensing light being incident on an electronic device. The improved concept employs ways to cancel out stray light originating from a display by modulating the brightness the display then cancel out the stray light by a mathematical operation.
In at least one embodiment a method for sensing light employs an electronic device. The electronic device comprises a display and a light sensor arrangement mounted behind the display. Light being incident on the display may traverse through the display and can be received by the light sensor arrangement.
The term "behind" denotes that the light sensor arrangement is placed on or into a body of the electronic device and is covered by the display. In this configuration, light which is incident on the electronic device traverses through the display and eventually reaches the light sensor arrangement. Light detected by the light sensor arrangement basically comprises contributions from at least two sources. First, ambient light from the devices environment, e.g. sun light, room lighting, light emitted by other devices etc., may strike the display and be incident on the light sensor arrangement. Second, the display emits light, e.g. by means of active pixels or by backlighting etc., which may be guided or reflected back towards the light sensor arrangement.
Different types of displays may be used with the proposed method such as a flat-panel display including liquid crystal displays (LCDs), liquid crystal displays with light-emitting diode (LED) backlighting, plasma panels, electroluminescent panels or displays based on organic light-emitting diodes (OLEDs). The light sensor arrangement can be implemented by various types of light sensors including ambient light sensors and/or color sensors. The term "arrangement" indicates an integrated circuit which may have various on-chip components for signal acquisition or (pre-) processing. The light sensor arrangement may have a single or a number of light sensors which are packaged into a common integrated circuit or spread over different locations of the electronic device.
The method for sensing light being incident on the electronic device comprises the following steps.
The display is periodically switched on and off. A period is defined by a succession of an on-state and an off-state of the display. A frame is defined by an on-state of the display. A frame may display a certain content of frame, i.e. a video frame, a single image, such as icons of applications of a computer or mobile device or photograph, etc. In general a frame may have a characteristic brightness distribution. For example, the display shows a certain content for the duration of a single frame, i.e. for the duration of an on-state. During a given frame the display may irradiate with a constant distribution of light intensity, i.e. light emission and/or color may be known for each location on the display.
In a first frame a display brightness is set to a first level. A first frame count is determined by integrating the incident light by means of the light sensor arrangement during the first frame. In a second frame the display brightness is set to a second level. For example, the second level can be higher or lower than the first level leading to a higher or lower display brightness. A second frame count is determined by integrating the incident light by means of the light sensor arrangement during the second frame. Finally, an ambient light level is determined depending on the first and the second frame counts. In fact, the ambient light level is a function of the first and the second frame counts. In general, a content of the second frame can be different compared to the first frame or both contents can be the same. It could be easier for the implementation if the contents of the two frames are the same in some cases.
The term "display brightness" can be considered a parameter to alter characteristic properties of the display, in particular, display emission. For example, by setting the display brightness to a certain level, the luminance is adjusted. Luminance relates to an amount of light that passes through, is emitted or reflected from a particular area of the display, and falls within a given solid angle. Thus, the display brightness can be adjusted locally, e.g. by altering luminance of an area of the display and a given solid angle. Alternatively, display brightness can be adjusted globally, e.g. by altering luminance of the whole area of the display and a given solid angle. However, the term "display brightness" can also relate to a color or color information to be displayed. For example, by setting the display brightness to a certain level, the color or correlated color temperature, CCT or Tcp, of an active element or area of the display and/or a display backlight is adjusted. The display brightness may be given by arithmetic mean color coordinates such as red, green, and blue, RGB, or any other color coordinates.
The ambient light level can be determined, and accounted for. In particular, the ambient light level can even be determined when the light sensor arrangement resides behind the active display. For example, other solutions may suffer from interference and superposition with the light emitted by the display itself. The proposed method, however, allows for canceling out stray light caused by the display. At the same time adjusting display brightness between the frames can be done in a way that the end user cannot notice there is any change. Measurement of the various signal counts is performed an activated display (on-state). This way the frame counts represents ambient light samples from the display and ambient light. The first and second frame counts differ from each other in a known way and, thus, can be combined to extract the ambient light level. Thus, the method can be used with various devices effectively reducing the impact of device to device variation and aging.
In at least one embodiment the first frame and the second frame define successive on states of the display. For example, first frame and the second frame are successive in time. In general, however, there may be intermediate frames or the first frame and the second frame follow each other immediately in time. The succession of frames can be determined by a refresh rate or a frame rate. These rates are a measure of how many frames may be displayed by the display per time, e.g. the refresh rate or frame rate lies between 40 HZ to 100 Hz. For an OLED display the refresh rate may be 60 Hz.
In at least one embodiment the periodical switching of the display depends on a control signal. For example, the control signal may be provided by a power circuit of the electronic device, such as a backlight LED power circuit. The control signal may be pulsed in the sense that it comprises consecutive high and low levels. In turn, the display is turned on or off depending on whether the control signal is at a high and low level. The control signal may be independent, i.e. not controlled by the display or a display driver.
A first sensor signal is generated by integrating the incident light by means of the light sensor arrangement. Integration lasts for a first integration time. For example, the light sensor arrangement comprises means to collect incident light by integrating the signal generated by the light sensor, such as integrators of various types, light-to-frequency or light-to-voltage converters.
The first frame count is determined from the first sensor signal and, thus, is indicative of an amount of integrated incident light during an on-state. For example, the sensor signal collected or integrated within the first integration time can be translated into the first frame count and can be interpreted as a measure of intensity. Similarly, the second frame count is determined from a second sensor signal and, thus, is indicative of an amount of integrated incident light during an on-state.
In at least one embodiment generating the first and/or second sensor signal is synchronized to the control signal. This way it may be assured that the first and the second frame counts are determined during an on-state only.
In at least one embodiment the first frame and the second frame are arranged to follow each other in time such that a change in ambient light level can be assumed constant in value. For example, ambient light can be assumed constant on the timescale of 1 to 100 milliseconds. For the sake of explanation consider the ambient light not to change on a scale of 20 milliseconds. Then, a frame rate or succession of the first and second frames can be in the frequency range of 100 Hz to 120 Hz. Furthermore, a normal frequency of an AC power supply, e.g. of the electronic device or ambient light sources such as room lighting, typically is operated and 50 Hz to 60 Hz. Thus, the frame rate can be adjusted to cancel out variations caused by the AC power supply and still assume constant ambient lighting between successive frames.
In at least one embodiment the first frame and the second frame immediately follow each other in time and are only separated by an off-state of the display. Alternatively, the first frame and the second frame follow each other in time with the number of off states of the display in between. Whether one or the other alternative is possible or implemented may depend on the variations or expected variations in ambient light level. For example, the electronic device provides means, such as a control unit or microprocessor, to adjust a delay on number of intermediate off states between successive first and second frames.
In at least one embodiment the ambient light level is determined from the difference of the first frame count can be expressed in terms of the second frame count or vice versa. In order to establish such expressions the knowledge about the first and second brightness levels can be employed, respectively.
In at least one embodiment the first frame count, denoted FC1, is determined by a sum of a first ambient light level, denoted AL1, and the display brightness at the first level, denoted DL1. The sum can be expressed as FC1 = AL1 + DL1. Similarly, the second frame count, denoted FC2, is determined by a sum of a second ambient light level, denoted AL2, and the display brightness at the second level, denoted DL2.
In at least one embodiment the first frame count and the second frame count differ by of brightness factor, denoted p%. For example, the second level of display brightness can be higher or lower than the first level of display brightness leading to a higher or lower display brightness. A difference in display brightness may be expressed by the brightness factor, e.g. as DL1 = p₁%·DL2 or DL2 = p₂%·DL1, respectively. Furthermore, the above expressions may only hold if ambient light levels do not change between the first and second frames, such that it may be assumed AL1 = AL2.
In at least one embodiment the frames of the display have a duration which is determined by a frame rate of the display. In particular, the frame rate is 50 to 100 Hz and/or the duration of a frame, including the first and second frames, recites between 1 to 20 milliseconds. The integration time for integrating the incident light by means of the light sensor arrangement depends on the frame rate. In particular, the integration time is set to 1 to 20 milliseconds.
For example, the refresh rate of an OLED display is about 60 Hz, so each frame takes about 16.7 milliseconds. The normal frequency of an AC power supply resides in the range of 50Hz to 60Hz. Light intensity of the ambient light, e.g. indoor light, can vary at a frequency of 100 Hz or 120 Hz. If the light sensor arranged is set to integrate the incident light for 10 milliseconds, then variations caused by the AC power supply can be cancelled out to a higher degree. Further, it can be assumed that the ambient light will not change in level within 20 milliseconds.
In at least one embodiment the light sensor arrangement comprises a light sensor. The ambient light level can be used to adjust the display brightness.
In at least one embodiment the light sensor arrangement comprises a color light sensor. The ambient light level can be used to adjust a display color, in particular, a color temperature value of the display. For example, the color light sensor can be used to adjust both display brightness and display color.
In at least one embodiment to consecutive frames are used as first and second frames to determine the ambient light level, respectively. Alternatively, a number of first frames and a number of second frames are combined to determine the ambient light level. For example, the number of first frames can be accumulated within a certain time period or several frames may be combined together into a single first frame, e.g. as mean or median frame. The same may be done for the number of second frames. Whether such an approach is practical may be determined on the temporal changes in ambient light level.
In at least one embodiment the frames used to determine the ambient light level are inserted into a stream of content frames. For example, the first and second frames are intermediate frames which comprise constant and predetermined brightness distributions and, thus, establish a sensor signal or frame counts that can be correlated to the predetermined conditions. At typical display refresh rates or frame rates the first and second frames, as intermediate may not be distinguishable for an end user. The proposed method may not interfere with the end user experience.
In at least one embodiment the display brightness at the first and/or second level is set for a region of interest. The light sensor arrangement integrates the incident light locally from the region of interest. For example, any frame of the display may be used as first or second frame to determine the ambient light level. The region of interest can be arranged to have a predetermined or known brightness and the light sensor arrangement determines the corresponding frames counts. This way only a small part of the display may be used to determine the ambient light level. The small part may not be apparent for an end user such that the proposed method may not interfere with the end user experience. Alternatively, the display brightness at the first and/or second level is set for the whole display. The light sensor arrangement integrates the incident light globally for the whole display, e.g. for first/second frames as intermediate frames. Furthermore, the first/second frames can be recorded at brightness levels that appear the same to the end user or at differences in level that do not interfere with the end user experience.
In the following, the principle presented above is described in further detail with respect to drawings, in which exemplary embodiments are presented.
In the exemplary embodiments and Figures below, similar or identical elements may each be provided with the same reference numerals. The elements illustrated in the drawings and their size relationships among one another, however, should not be regarded as true to scale. Rather individual elements, such as layers, components, and regions, may be exaggerated to enable better illustration or improved understanding.

Figure 1: shows an example method for sensing light, and
Figure 2: shows an example embodiment of an electronic device with a display and light sensor arrangement.

Figure 1 shows an example method for sensing light. The drawing shows a first frame FR1 and a second frame FR2 which closely follow each other in time, e.g. are two consecutive frames of the display 11. A frame is defined by an on-state ON of the display 11. The frames are displayed by means of an electronic device. The electronic device comprises a display 11 and a light sensor arrangement 12 mounted behind the display 11 such as to receive incident light through the display 11 (see Figure 2).
The display 11 can be periodically switched on and off depending on a control signal CS, e.g. a PWM signal provided by control electronics of display driver circuitry (not shown). A period PD is determined by a picture refresh signal PR or by a refresh rate of the display 11 which is a number of times in a second that a display hardware updates an image. The refresh rate is known from the specifications of the display or can be set by the user.
A frame is defined by an on-state ON of the display 11. Furthermore, a duration of an off-state OFF and an on-state ON is determined by a duty cycle DC of the display 11. The duty cycle DC is a fraction of one period PD in which a signal or system is active. The period PD thus corresponds to the time it takes for display 11, or control signal CS, to complete an on-and-off cycle, i.e. a succession of one on-and one off-state ON, OFF. The duty cycle DC typically is either known or can be set by the user.
Furthermore, the display 11 can be adjusted in brightness. For example, the first frame FR1 and the second frame FR2 comprise the same content, i.e. have the same or similar brightness distribution. However, considered as a whole, or globally, the two frames FR1 and FR2 have a different brightness level. In this example, the first frame FR1 has a first level of brightness L1 and the second frame FR2 has a second level of brightness L2. For the sake of explanation assume that the second level L2 is 10% smaller than the first level L1.
The light sensor arrangement is set to generate a sensor signal which is indicative of light incident on the display and of light emitted by the display or its backlighting. The sensor signal is generated by integrating the light which reaches the light sensor arrangement through the display. A first sensor signal is generated for the duration of the first frame FR1. A second sensor signal is generated for the duration of the second frame FR2. Correspondingly, the light sensor arrangement integrates the incident light for a first and second integration time T1, T2, respectively. For example, the integration time is set to 10 ms and a frame has a duration of about 16.7 ms.
These specific values derive from the following considerations. A refresh rate an OLED display is 60 Hz, so each frame takes 16.7 ms. The normal frequency of AC power is 50Hz or 60Hz, so the light intensity of indoor light can vary at a frequency of 100 Hz or 120 Hz. Here, we will take 100 Hz as an example. If the light sensor arrangement integrates the light in a frame for 10 ms, then it can cancel out the variation caused by the AC power supply.
In the following assume that the ambient light will not change within 20 ms. Then the first frame FR1 results in a first frame count of FC1. The consecutive second frame FR2, having same content, is the same as the first frame FR1 except for a different brightness level. Thus, the second frame FR2 results in a second frame count of FC2. Assume here that the brightness level L2 of the second frame is 10% lower than the brightness level of the second frame L2. Furthermore, the first and second frame counts are recorded within 20 ms such that we can assume a first ambient light level AL1 at the time of the first frame FR1 to be the same in value than a second ambient light level AL2 at the time of the second frame FR2, i.e. AL1 = AL2.
In other words, the first frame count, denoted FC1, is determined by a sum of a first ambient light level, denoted AL1, and the display brightness at the first level, denoted DL1. The sum can be expressed as FC1 = AL1 + DL1. Similarly, the second frame count, denoted FC2, is determined by a sum of a second ambient light level, denoted AL2, and the display brightness at the second level, denoted DL2. The ambient light level AL (= AL1 = AL2) can be determined from a difference of the first and the second frame counts FC1, FC2. Given the parameters above it follows that FC2 = 0.9·FC1. Thus, then the ambient light level AL = 10·FC2 - 9·FC1, assuming AL1 = AL2. Thus, with this approach, the stray light caused by the display is significantly reduced or even cancelled out. The determined ambient light level can be used for further purposes, including adjusting brightness and/or color of the display.
Figure 2 shows an exemplary embodiment of an electronic device with a display and light sensor arrangement. The electronic device has a surface 13 and the display 11 is arranged on the surface. The display 11 comprises a panel of pixels 14, a backlight panel 15 and a reflector 16. The light sensor arrangement 12 is arranged behind the display 11 (with respect to the main direction of emission).
During operation of the display several light sources may contribute to the sensor signal generated by the light sensor arrangement. A fraction of the emitted light EL emitted by the display may be reflected back towards the light sensor arrangement as indicated by arrows in the drawing. The emitted light may partly stem from the pixels arranged in panel 14 and/or from LEDs used for backlighting the display. Furthermore, ambient light from different sources may strike the display 11 and traverse towards the light sensor arrangement 12 with gives rise to a contribution to the sensor signal. The electronic device depicted in Figure 2 illustrates one possible embodiment. Other types of displays and arrangements are possible without restricting the scope of the proposed concept.

Reference numerals

11: display
12: light sensor arrangement
13: surface
14: pixels
15: backlight panel
16: reflector
AL: ambient light level
AL1: first ambient light level
AL2: second ambient light level
CS: control signal
DL1: display brightness level
DL2: display brightness level
EL: fraction of emitted light
FC1: first frame count
FC2: second frame count
FR: frame
FR1: first frame
FR2: second frame
INT1: first sensor signal
INT2: second sensor signal
LED: light emitting diode
OFF: off state
ON: on state
RL: reflected light
T1: first integration time
T2: second integration time

Claims

Method for sensing light being incident on an electronic device, the electronic device comprising a display (11) and a light sensor arrangement (12) mounted behind the display (11) such as to receive incident light through the display (11), the method comprising the steps of:
- periodically switching the display (11) on and off,
wherein a frame (FR) is defined by an on-state (ON) of the display (11),

- in a first frame (FR1), setting a display brightness to a first level (DL1),

- determining a first frame count (FC1) by integrating the incident light by means of the light sensor arrangement (12) during the first frame (FR1),

- in a second frame (FR2), setting the display brightness to a second level (DL2),

- generating a second frame count (FC2) by integrating the incident light by means of the light sensor arrangement (12) during the second frame (FR2), and

- determining an ambient light level (AL) depending on the first frame count (FC1) and the second frame count (FC2).
Method according to claim 1, wherein the first frame (F1) and the second frame (FR2) define successive on-states (ON) of the display (11).
Method according to claim 1 or 2, wherein
- the periodical switching of the display (11) depends on a control signal (CS),

- a first sensor signal (INT1) is generated by integrating the incident light by means of the light sensor arrangement (12) for a first integration time (T1) and the first frame count (FC1) is determined from the first sensor signal (INT1), and

- a second sensor signal (INT2) is generated by integrating the incident light by means of the light sensor arrangement (12) for a second integration time (T2) and the second frame count (FC2) is determined from the second sensor signal (INT2).
Method according to claim 3, wherein generating the first and/or second sensor signal is synchronized to the control signal (CS).
Method according to one of claims 1 to 4, wherein the first frame (FR1) and the second frame (FR2) are arranged to follow each other in time such that a change in ambient light level (AL) can be assumed constant in value.
Method according to claim 5, wherein
- the first frame (FR1) and the second frame (FR2) immediately follow each other in time and are only separated by an off-state (OFF) of the display (11) or

- the first frame (F1) and the second frame (FR2) follow each other time with a number of off-states (OFF) of the display (11) in-between.
Method according to one of claims 1 to 6, wherein the ambient light level (AL) is determined from a difference of the first frame count (FC1) and the second frame count (FC2).
Method according to claim 7, wherein
- the first frame count (FC1), denoted FC1, is determined by a sum of a first ambient light level (AL1), denoted AL1, and the display brightness at the first level (L1), denoted DL1, such that FC1 = AL1 + DL1, and

- the second frame count (FC2), denoted FC2, is determined by a sum of a second ambient light level (AL2), denoted AL2, and the display brightness at the second level (L2), denoted DL2, such that FC2 = AL2 + DL2.
Method according to claim 8, wherein the first frame count (FC1) and the second frame count (FC2) differ by a brightness factor (p%), denoted p%, such that DL2 = p%·DL1 and assuming AL1 = AL2.
Method according to one of claims 1 to 9, wherein
- the frames of the display (11) have a duration determined by a frame rate of the display, in particular, the frame rate is 50 to 100 Hz and/or the duration of a frame is 1 to 20 ms; and/or

- the integration time for integrating incident light by means of the light sensor arrangement depends on the frame rate, in particular, the integration time is set to 1 to 20 ms.
Method according to one of claims 1 to 10, wherein the light sensor arrangement (12) comprises a light sensor and the ambient light level (AL) is used to adjust the display brightness.
Method according to one of claims 1 to 10, wherein the light sensor arrangement (12) comprises a color light sensor and the ambient light level (AL) is used to adjust a display color, in particular, a color temperature value of the display (11) .
Method according to one of claims 1 to 12, wherein
- two consecutive frames are used as first and second frame to determine the ambient light level (AL), or

- a number of first frames and a number of second frames are combined to determine the ambient light level (AL).
Method according to one of claims 1 to 13, wherein the the frames used to determine the ambient light level (AL) are inserted into a stream of content frames.
Method according to one of claims 1 to 14, wherein
- the display brightness at the first and/or second level (L1, L2) is set for a region of interest and the light sensor arrangement (12) integrates incident light locally from the region of interest, or

- the display brightness at the first and/or second level (L1, L2) is set for the whole display (11) and the light sensor arrangement (12) integrates incident light globally for the whole display (11).