EP3503081B1

EP3503081B1 - A method, apparatus and computer program for encoding visible light communication information in an image frame

Info

Publication number: EP3503081B1
Application number: EP17210157.8A
Authority: EP
Inventors: Anil ÍKIZLER
Original assignee: Vestel Elektronik Sanayi ve Ticaret AS
Current assignee: Vestel Elektronik Sanayi ve Ticaret AS
Priority date: 2017-12-22
Filing date: 2017-12-22
Publication date: 2023-05-31
Anticipated expiration: 2037-12-22
Also published as: TR201722689A2; EP3503081A1

Description

Technical Field

The present disclosure relates to a method, apparatus and computer program for encoding visible light communication information in an image frame to be displayed at a display screen.

Background

Visible light communication is a data communication technique which uses visible light (ranging from around 390nm to 700nm) to convey data. Visible light communication is possible over a multitude of electronic items that comprise a screen and that can be imaged using a camera. Examples of such electronic items include televisions, personal computers, smartphones, and smart watches, and display screens or display screens generally. In one example, a video stream can be encoded with visible light information, thereby transmitting the visible light information when the encoded video is output at the screen of an electronic device.
US2010225673A1 discloses a method of presenting an image on a display device having colour channel dependent light emission comprising receiving an image input signal including a plurality of three-component input pixel signals; selecting a reduction colour component; calculating a reduction factor for each input pixel signal dependent upon a distance metric between the input pixel signal and the selected reduction colour component; selecting a respective saturation adjustment factor for each colour component of each pixel signal; producing an image output signal having four colour components from the image input signal using the reduction factors and saturation adjustment factors to adjust the luminance and colour saturation, respectively, of the image input signal; providing a four-channel display device having colour channel dependent light emission; and applying the image output signal to the display device to cause it to present an image corresponding to the image output signal.
US2017178565A1 discloses a head-mounted display worn which includes a right display unit and a left display unit, an image display section for making a right eye of the user visually recognize an image based on light emitted by the right display unit and making a left eye user visually recognize an image based on light emitted by the left display unit, temperature sensors for detecting respective temperatures of the right display unit and the left display unit, and a control section for controlling the luminance of at least either one of the light display unit and the left display unit based on the temperatures detected by the temperature sensors.
WO2014096944A1 discloses a method and electronic device for selecting a waveform to release data towards a device driver. The method includes receiving data for display on a heat-affected display area controlled by the device driver, obtaining at least one temperature measurement (e.g., from one or more thermistors), selecting a waveform from a plurality of waveforms considering a temperature value and releasing the data towards the device driver under the selected waveform.
US8890773B1 discloses an LED light and communication system which includes Visible Light Communication Transceiver Glasses having at least one projector, lense(s), and optical transceiver.

Summary

According to a first aspect disclosed herein, there is provided a method of encoding visible light communication information in an image frame to add the information to content in the image frame to be displayed at a display screen, the method comprising:

using circuitry for measuring the temperature of at least one portion of a display screen, measuring the temperature of at least one portion of the display screen during use; and
using a processor, encoding the visible light communication information in the image frame to add the visible light communication information to content in the image frame so as to be displayed at the at least one portion of the display screen, the encoding being carried out by setting at least one of the luminosity and the hue of the pixels at the at least one portion of the display screen, wherein the value that is set for said at least one of the luminosity and the hue of the pixels at the at least one portion of the display screen for the encoding of the visible light communication information in the image frame is dependent on the measured temperature of the at least one portion of the display screen.

Examples described herein allow for temperature changes that may occur in the display screen during normal usage and can also allow for changes that may occur over time from manufacture to actual usage. Examples described herein take into account the actual temperature of one or more portions of the display screen when determining how to encode the visible light communication information in the image frame. This leads to a better, more faithful presentation of the visible light communication information in the image frame and reduces the error rate when the visible light communication information is read.
In an example, the method comprises measuring the temperature of the at least one portion of the display screen only after the display screen has been on for longer than a threshold time period.
In an example the method comprises:

measuring an initial temperature of a plurality of portions of the display screen during manufacture of the display screen;
storing data concerning the initial temperatures of the portions of the display screen;
comparing the initial temperature of at least one portion of the display screen at manufacture and the measured temperature of at least one portion of the display screen during field use; and
encoding the visible light communication information in the image frame dependent on the measured temperature of the at least one portion of the display screen during field use if the difference between the stored initial temperature of the at least one portion of the display screen at manufacture and the measured temperature of the at least one portion of the display screen during field use differs by more than a threshold.

The initial temperature of the plurality of portions of the display screen may be measured during manufacture only after the display screen has been on for longer than a threshold time period.
In an example setting the hue comprises adjusting one or more of the red, green and blue values associated with the pixels at the at least one portion of the display screen.
In an example each of the red, green and blue values associated with the pixels at the at least one portion of the display screen can be adjusted independently of each other.
According to a second aspect disclosed herein, there is provided an apparatus configured to encode visible light communication information in an image frame to add the information to content in the image frame, the system comprising:

circuitry for measuring the temperature of at least one portion of a display screen during use; and
a processor constructed and arranged to encode visible light communication information in an image frame to add the visible light communication information to content in the image frame so as to be displayed at the at least one portion of the display screen, the encoding being carried out by setting at least one of the luminosity and the hue of the pixels at the at least one portion of the display screen, wherein the value that is set for said at least one of the luminosity and the hue of the pixels at the at least one portion of the display screen for the encoding of the visible light communication information in the image frame is dependent on the measured temperature of the at least one portion of the display screen.

In an example the circuitry is arranged to measure the temperature only after the display screen has been on for longer than a threshold time period.
In an example the apparatus comprises:

a data store for storing initial temperatures of a plurality of portions of the display screen measured during manufacture of the display screen; and
In an example, the apparatus is arranged such that each of the red, green and blue values associated with the pixels at the at least one portion of the display screen can be adjusted independently of each other.

According to a third aspect disclosed herein, there is provided a computer program for encoding information in an image frame displayed by a display screen, the computer program comprising instructions such that when the computer program is executed on a computing device, the computing device is arranged to:
encode information in an image frame to add the information to content in the image frame so as to be displayed, using visible light, at at least one portion of the display screen, the encoding being carried out by setting at least one of the luminosity and the hue of the pixels at the at least one portion of the display screen, wherein the value that is set for said at least one of the luminosity and the hue of the pixels at the at least one portion of the display screen is dependent on a measured temperature of the at least one portion of the display screen.

Brief Description of the Drawings

To assist understanding of the present disclosure and to show how embodiments may be put into effect, reference is made by way of example to the accompanying drawings in which:

Figure 1 shows schematically an example system for setting properties of at least one portion of a display screen;
Figure 2 shows schematically an example device for outputting visible light communication encoded content.

Detailed Description

As mentioned previously, visible light communication information can be added to content which is to be displayed on a screen. Display screens are used in many different types of consumer apparatus including for example television screens,

the processor being configured to:
compare the initial temperature of at least one portion of the display screen at manufacture and the measured temperature of at least one portion of the display screen during field use; and
encode the visible light communication information in the image frame dependent on the measured temperature of the at least one portion of the display screen during field use if the difference between the stored initial temperature of the at least one portion of the display screen at manufacture and the measured temperature of the at least one portion of the display screen during field use differs by more than a threshold.

In an example, the apparatus is arranged such that the hue of the pixels are set by adjusting one or more of the red, green and blue values associated with the pixels at the at least one portion of the display screen.
In an example, the apparatus is arranged such that each of the red, green and blue values associated with the pixels at the at least one portion of the display screen can be adjusted independently of each other.
According to a third aspect disclosed herein, there is provided a computer program for encoding visible light communication information in an image frame to add the visible light communication information to content in the image frame displayed by a display screen, the computer program comprising instructions such that when the computer program is executed on a computing device which has circuitry for measuring the temperature of at least one portion of the display screen during use, the computing device is arranged to:
encode visible light communication information in an image frame to add the visible light communication information to content in the image frame so as to be displayed at at least one portion of the display screen, the encoding being carried out by setting at least one of the luminosity and the hue of the pixels at the at least one portion of the display screen, wherein the value that is set for said at least one of the luminosity and the hue of the pixels at the at least one portion of the display screen for the encoding of the visible light communication information in the image frame is dependent on the measured temperature of the at least one portion of the display screen.

Brief Description of the Drawings

Detailed Description

As mentioned previously, visible light communication information can be added to content which is to be displayed on a screen. Display screens are used in many different types of consumer apparatus including for example television screens, the display screen is susceptible to distortion. Therefore, incorrect visible light communication information, if any at all in some cases, may be transmitted from the display screen to a recipient device. Examples described herein adapt the appearance of the visible light communication as encoded, having taken into consideration the temperature of the corresponding portion of the display screen at which the visible light communication is to be displayed.
Figure 1 shows an example system 100 for determining the temperature of one or more portions of a display screen, and to set at least one of the luminosity and hue values used for encoding visible light communication information in the pixels associated with the one or more portions of the display screen based on the measured temperature(s). The system 100 carries out a temperature check 102 to determine the temperature of one or more portions of the display screen when image/video information to belbeing displayed at the display screen is encoded with visible light communication information. The temperature check 102 may for example determine the temperature of the entire display screen as a whole or separately of one or more portions of the display screen. The temperature check 102 may in one example be circuitry arranged to measure field use temperature 104 of the at least one portion of the display screen when an image/video to be displayed at the display screen is encoded with visible light communication information. The measure field use temperature 104 of the at least one portion of the display screen is used to determine the luminosity and/or hue of the at least one portion of the display screen that is to be used when encoding visible light communication information in an image frame for display at the display screen.
In one example, determining the hue of the display screen may comprise determining the RGB (red, green and blue) values for pixels or groups of pixels associated with the at least one portion of the display screen. RGB pixels of a display screen typically comprise 3 sub-pixels: red, green and blue (and optionally others, in other schemes, such as RGB and yellow, RGB and white, etc.). The output of at least one of the RGB sub-pixels may be modified to take into account the current temperature of the display screen at that part of the display screen. The values used for each of the sub-pixels may be independently increased or decreased as necessary. The luminosity and/hue of the display screen may be determined by a display modifier 106.
In one example, the temperature check 102 determines whether the temperature of the display screen in use is above or below a threshold value. If the temperature is below the threshold value, pre-defined default luminosity and/or hue settings are used at the display screen when visible light communication information is encoded in an image frame displayed at the display screen or at different portions of the display screen. If the temperature is above the threshold value, the luminosity and/or hue settings are modified such that the visible light information encoded in the image/video is not distorted by the temperature of the display screen or different temperatures at different portions of the display screen. This may, for example, involved increasing/decreasing the luminosity and/or hue values of the display screen. In another example, the threshold value may be a temperature range. In this example if at least one portion of the display screen falls within the threshold temperature range, the luminosity and/or hue of the at least one portion of the display screen are kept at the default predetermined value(s) when encoding the visible light information. Conversely, if the temperature of the display screen is outside the threshold temperature range, the luminosity and/or hue of the at least one portion of the display screen are modified such that the visible light information encoded in the image/video is not distorted by the temperature of the display screen or different temperatures at different portions of the display screen.
In one example, the temperature check 102 determines whether the measure field use temperature 104 of the display screen in use is above or below an initial temperature reading(s) 108 of the device. The initial temperature reading(s) 108 may be for one or more portions of the display screen arranged to display visible light communication information encoded in an image frame. Optionally, the initial temperature reading(s) 108 is/are stored at an initial temperature reading data store or "look-up" table 110.
The initial temperature reading(s) 108 of the display screen may be taken after the display screen has been functioning for a pre-determined period of time. In one example, after manufacture, the display screen is arranged to function for a pre-determined manufacture "warm-up" time. The warm-up time is required to allow all components of and relating to the display screen to reach a normal functioning temperature. Once the display screen has been functioning for the warm-up time, an initial temperature reading(s) 108 of at least one portion of the display screen is taken. The initial temperature reading(s) 108 may be for each pixel or even sub-pixel (e.g. the RGB sub-pixels) of the display screen.
Subsequently, the display screen will be used by end users in the field. The display screen is turned on and allowed to function for a field use "warm-up" time. The field use warm up time may be the same as or different from the manufacture warm up time. Having different warm-up times may be particularly important in devices exposed to extreme temperatures or wide temperature ranges, such as for example so-called "signage" display screens. Again, the field use warm-up time allows all components of and relating to the display screen reach a normal functioning temperature. Once the display screen has been functioning for the field use warm-up time, the field use temperature 104 of at least one portion of the display screen is determined when visible light communication information is to be added to an image/video for display at the display screen.
In an example, the temperature check 102 is arranged to compare the field use temperature reading 104 for the at least one portion of the display screen and the initial temperature reading 108 for the at least one portion of the display screen when visible light communication information is to be added to an image/video for display at the display screen. Optionally, the temperature check 102 receives the initial temperature reading 108 from the data store/"look-up" table 110.
In an example, when the field use temperature 104 of the at least one portion of the display screen is determined to be below the initial temperature reading 108, pre-defined default luminosity and/or hue values are applied/maintained to the at least one portion of the display device when visible light communication information is to be added to an image/video for display at the display screen. Conversely, when the field use temperature 104 of at least one portion of the display screen is determined to be above the initial temperature reading 108 when visible light communication information is to be added to an image/video for display at the display screen, the display modifier 106 of the visible light communication coder 104 is arranged to modify the settings of the display screen to prevent unwanted modifications from occurring to the visible light communication information that is to be encoded to content being displayed at the display device.
In one example, when an image/video for display at the display screen is arranged to be encoded with visible light communication information, the temperature check 102 may be arranged to determine the field use temperature 104 of the display screen at regular intervals of time. Alternatively or additionally, the temperature check 102 may be triggered to measure the temperature of the display screen once the display screen reaches a threshold temperature.
In a specific example to illustrate this, a display screen is arranged to display an image/video encoded with visible light communication information. After the display screen has been functioning over a field usage warm up time, a field use temperature reading 104 may be taken to determine the temperature of at least one portion of the display screen. In this example, the field use temperature reading 104 is compared with an initial temperature 108 of the at least one portion of the display screen. The initial temperature 108 may be stored in a data store or "look-up table" 110. In this example, the temperature check 102 determines that the field use temperature 104 is different from the initial temperature 108, the difference being above a predefined threshold value.
In one example, the VLC (visible light communication) coder 112 determines the location of the at least one portion of the display screen exhibiting the highly different field use temperature 104 on the display screen. Initially, in a specific example for illustrative purposes, the portion of the display screen exhibits an RGB value of 200, 150, 200. Upon determining that the portion of the display screen is exhibiting a highly different field use temperature 104, a display modifier 106 of the VLC coder 112 is arranged to set new luminosity and/or hue values of the display screen. In this example, the RGB value of the at least one portion of the display screen is set at 180,170, 210. New luminosity and/or hue values to be applied to a display screen may be stored at a data store or "look-up" table, or calculated on-the-fly.
When the image/video of this specific example is encoded with visible light communication information, the hue of the at least one portion of the display screen in this example is amended such that the visible light communication information transmitted from the display screen is correct and unaffected by the temperature of the display screen. The VLC coder 112 accommodates non-linearities arising from different temperatures of portions of the display, thereby ensuring the visible light communication information encoded in the image/video is able to be received correctly by a receiver.
Figure 2 shows an example device comprising a display device 200 for displaying an image/video that has been encoded with visible light communication information. Examples of the device 200 include television sets and so-called signage apparatus. The device 200 comprises a display screen 202, the luminosity and/or hue of at least one part of which is determined by the temperature of the display screen 202 when in use, when the image/video is encoded with visible light communication information. The device 200 is in this example arranged to receive image/video information from a set-top box 204. In other examples, the image/video information may be from a data storage, for examples from a wired connection such as an Ethernet network connection or an HDMI connection, etc., or a wireless connection, such as a WiFi connection. In an example, the image/video information may be received from a data storage of a device (e.g. a mobile phone or tablet computer or other computing device). The image/video information may, in another example, be from another source, such as a DVD or Blu Ray or the like player, or a terrestrial, satellite or cable broadcast. The received image/video information will be encoded with visible light communication information, to be displayed at the display screen 202. An image/video be encoded with visible light communication information by a component of the display device 200, or by a separate component or device that is able to communication with the display device 200.
It will be understood that the processor or processing system or circuitry referred to herein may in practice be provided by a single chip or integrated circuit or plural chips or integrated circuits, optionally provided as a chipset, an application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), digital signal processor (DSP), graphics processing units (GPUs), etc. The chip or chips may comprise circuitry (as well as possibly firmware) for embodying at least one or more of a data processor or processors, a digital signal processor or processors, baseband circuitry and radio frequency circuitry, which are configurable so as to operate in accordance with the exemplary embodiments. In this regard, the exemplary embodiments may be implemented at least in part by computer software stored in (non-transitory) memory and executable by the processor, or by hardware, or by a combination of tangibly stored software and hardware (and tangibly stored firmware).
Reference is made herein to data storage for storing data. This may be provided by a single device or by plural devices. Suitable devices include for example a hard disk and non-volatile semiconductor memory.
Although at least some aspects of the embodiments described herein with reference to the drawings comprise computer processes performed in processing systems or processors, the invention also extends to computer programs, particularly computer programs on or in a carrier, adapted for putting the invention into practice. The program may be in the form of non-transitory source code, object code, a code intermediate source and object code such as in partially compiled form, or in any other non-transitory form suitable for use in the implementation of processes according to the invention. The carrier may be any entity or device capable of carrying the program. For example, the carrier may comprise a storage medium, such as a solid-state drive (SSD) or other semiconductor-based RAM; a ROM, for example a CD ROM or a semiconductor ROM; a magnetic recording medium, for example a floppy disk or hard disk; optical memory devices in general; etc.
The examples described herein are to be understood as illustrative examples of embodiments of the invention. Further embodiments and examples are envisaged. Any feature described in relation to any one example or embodiment may be used alone or in combination with other features. In addition, any feature described in relation to any one example or embodiment may also be used in combination with one or more features of any other of the examples or embodiments, or any combination of any other of the examples or embodiments. Furthermore, equivalents and modifications not described herein may also be employed within the scope of the invention, which is defined in the claims.

Claims

A method of encoding visible light communication information in an image frame to add the visible light communication information to content in the image frame to be displayed at a display screen (202), the method comprising:
using circuitry for measuring the temperature (104) of at least one portion of a display screen (202), measuring the temperature (104) of at least one portion of the display screen (202) during use; and

using a processor, encoding the visible light communication information in the image frame to add the visible light communication information to content in the image frame so as to be displayed at the at least one portion of the display screen (202), the encoding being carried out by setting at least one of the luminosity and the hue of the pixels at the at least one portion of the display screen (202), wherein the value that is set for said at least one of the luminosity and the hue of the pixels at the at least one portion of the display screen (202) for the encoding of the visible light communication information in the image frame is dependent on the measured temperature (104) of the at least one portion of the display screen (202).
A method according to claim 1, comprising:
measuring the temperature (104) of the at least one portion of the display screen (202) only after the display screen (202) has been on for longer than a threshold time period.
A method according to claim 1 or claim 2, comprising:
measuring an initial temperature (108) of a plurality of portions of the display screen (202) during manufacture of the display screen (202);

storing data concerning the initial temperature (108) of the portions of the display screen (202);

comparing the initial temperature (108) of at least one portion of the display screen (202) at manufacture and the measured temperature (104) of at least one portion of the display screen (202) during field use; and

encoding the visible light communication information in the image frame dependent on the measured temperature (104) of the at least one portion of the display screen (202) during field use if the difference between the stored initial temperature (108) of the at least one portion of the display screen (202) at manufacture and the measured temperature (104) of the at least one portion of the display screen (202) during field use differs by more than a threshold.
A method according to any of claims 1 to 3, wherein setting the hue comprises adjusting one or more of the red, green and blue values associated with the pixels at the at least one portion of the display screen (202).
A method according claim 4, wherein each of the red, green and blue values associated with the pixels at the at least one portion of the display screen (202) can be adjusted independently of each other.
An apparatus configured to encode visible light communication information in an image frame to add the visible light communication information to content in the image frame, the apparatus comprising:
circuitry for measuring the temperature (104) of at least one portion of a display screen (202) during use; and

a processor constructed and arranged to encode visible light communication information in an image frame to add the visible light communication information to content in the image frame so as to be displayed at the at least one portion of the display screen (202), the encoding being carried out by setting at least one of the luminosity and the hue of the pixels at the at least one portion of the display screen (202), wherein the value that is set for said at least one of the luminosity and the hue of the pixels at the at least one portion of the display screen (202) for the encoding of the visible light communication information in the image frame is dependent on the measured temperature (104) of the at least one portion of the display screen (202).
An apparatus according to claim 6, wherein the circuitry is arranged to measure the temperature (104) only after the display screen (202) has been on for longer than a threshold time period.
An apparatus according to claim 6 or claim 7, comprising:
a data store (110) for storing initial temperature (108)s of a plurality of portions of the display screen (202) measured during manufacture of the display screen (202); and

the processor being configured to:
compare the initial temperature (108) of at least one portion of the display screen (202) at manufacture and the measured temperature (104) of at least one portion of the display screen (202) during field use; and

encode the visible light communication information in the image frame dependent on the measured temperature (104) of the at least one portion of the display screen (202) during field use if the difference between the stored initial temperature (108) of the at least one portion of the display screen (202) at manufacture and the measured temperature (104) of the at least one portion of the display screen (202) during field use differs by more than a threshold.
An apparatus according to any of claims 6 to 8, arranged such that the hue of the pixels are set by adjusting one or more of the red, green and blue values associated with the pixels at the at least one portion of the display screen (202).
An apparatus according to claim 9, arranged such that each of the red, green and blue values associated with the pixels at the at least one portion of the display screen (202) can be adjusted independently of each other.
A computer program for encoding visible light communication information in an image frame to add the visible light communication information to content in the image frame displayed by a display screen (202), the computer program comprising instructions to cause the apparatus of claim 6 to execute the steps of the method of claim 1.