EP3435366B1 - Method of controlling a backlight unit, a backlight unit and a display device and camera system - Google Patents

Method of controlling a backlight unit, a backlight unit and a display device and camera system Download PDF

Info

Publication number
EP3435366B1
EP3435366B1 EP17183145.6A EP17183145A EP3435366B1 EP 3435366 B1 EP3435366 B1 EP 3435366B1 EP 17183145 A EP17183145 A EP 17183145A EP 3435366 B1 EP3435366 B1 EP 3435366B1
Authority
EP
European Patent Office
Prior art keywords
light source
light
display device
backlight unit
faulty
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP17183145.6A
Other languages
German (de)
French (fr)
Other versions
EP3435366A1 (en
Inventor
Kadir YILDIZ
Alper Sait ER
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Vestel Elektronik Sanayi ve Ticaret AS
Original Assignee
Vestel Elektronik Sanayi ve Ticaret AS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Vestel Elektronik Sanayi ve Ticaret AS filed Critical Vestel Elektronik Sanayi ve Ticaret AS
Priority to EP17183145.6A priority Critical patent/EP3435366B1/en
Priority to TR2017/11138A priority patent/TR201711138A2/en
Publication of EP3435366A1 publication Critical patent/EP3435366A1/en
Application granted granted Critical
Publication of EP3435366B1 publication Critical patent/EP3435366B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G3/00Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
    • G09G3/20Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters
    • G09G3/34Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters by control of light from an independent source
    • G09G3/3406Control of illumination source
    • G09G3/342Control of illumination source using several illumination sources separately controlled corresponding to different display panel areas, e.g. along one dimension such as lines
    • G09G3/3426Control of illumination source using several illumination sources separately controlled corresponding to different display panel areas, e.g. along one dimension such as lines the different display panel areas being distributed in two dimensions, e.g. matrix
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00Control of display operating conditions
    • G09G2320/04Maintaining the quality of display appearance
    • G09G2320/043Preventing or counteracting the effects of ageing
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00Control of display operating conditions
    • G09G2320/06Adjustment of display parameters
    • G09G2320/0693Calibration of display systems
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2330/00Aspects of power supply; Aspects of display protection and defect management
    • G09G2330/08Fault-tolerant or redundant circuits, or circuits in which repair of defects is prepared
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2360/00Aspects of the architecture of display systems
    • G09G2360/14Detecting light within display terminals, e.g. using a single or a plurality of photosensors
    • G09G2360/145Detecting light within display terminals, e.g. using a single or a plurality of photosensors the light originating from the display screen

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Liquid Crystal Display Device Control (AREA)
  • Control Of Indicators Other Than Cathode Ray Tubes (AREA)
  • Circuit Arrangement For Electric Light Sources In General (AREA)
  • Lighting Device Outwards From Vehicle And Optical Signal (AREA)

Description

    Technical Field
  • The present disclosure relates to method of controlling a backlight unit of a display device, a backlight unit for a display device and a system comprising a display device and a camera.
  • Background
  • Display devices are used in a large variety of contexts for displaying various forms of content. The quality of an image displayed by a display device can depend on a large number of factors. One problem that persists in many known display devices is the problem of a non-uniform distribution of light being used to generate the displayed image. This problem may occur, for example, as a result of one or more light sources in a backlight unit of the display device becoming faulty. A faulty light source can cause a pixel on the screen to appear brighter or darker than intended, thereby affecting the quality of the image that is displayed at the display device.
  • WO2011028335A1 discloses that LEDs (light emitting diodes) near or surrounding a non-functional or sub-par LED are adjusted to compensate the sub-par LED. The compensation may include illumination (e.g., increased illumination of nearby LEDs to compensate for under illumination of a non-functional LED) or a shift in hue of the nearby LEDs to compensate for an opposite shift in hue of the sub-par LED.
  • Summary
  • According to a first aspect disclosed herein, there is provided a method of controlling a backlight unit of a display device, the backlight unit comprising a plurality of light sources for illuminating a screen of the display device, the method comprising: detecting a faulty light source amongst the plurality of light sources; characterised by: at least some of the light sources comprising an actuator for controllably moving the light source within the backlight unit; and characterised by the method comprising: operating an actuator of the backlight unit to move at least one other light source that is a nearest neighbour to the faulty light source from a first position to a second position and adjusting the intensity of light that is output by the at least one other light source at the second position to allow the at least one other light source to compensate for the faulty light source.
  • This allows a more uniform distribution of light being output by the display device to be maintained.
  • In an example, the faulty light source outputs no light or outputs light at an intensity that is lower than a desired intensity; and the second position of the at least one other light source is closer to the position of the faulty light source than the first position.
  • In an example, the method comprises increasing the intensity of light that is output by the at least one other light source at the second position.
  • In an example, the method comprises decreasing the intensity of light that is output by one or more light sources that are not nearest neighbours to the faulty light source.
  • In an example, the method comprises receiving image data indicative of the distribution of light output by the display device, and wherein the step of detecting the faulty light source is based on the received image data.
  • According to a second aspect disclosed herein, there is provided a backlight unit for a display device, the backlight unit comprising: a plurality of light sources for illuminating a screen of the display device; characterised by: at least some of the light sources comprising an actuator for controllably moving the light source within the backlight unit, the backlight unit being arranged such that if one of the light sources is detected by a controller as being faulty, at least one of the other light sources that is a nearest neighbour to the faulty light source is caused by the controller controlling the actuator of the light source to move from a first position to a second position and the intensity of light output by the at least one other light source is caused by the controller to be adjusted from a first intensity to a second intensity, so that the at least one other light source compensates for the faulty light source.
  • According to a third aspect disclosed herein, there is provided a display device, the display device comprising: a screen, a backlight unit as described above; and a controller for controllably moving the at least one other light source from a first position to a second position and for adjusting the intensity of light output by the at least one other light source.
  • According to a fourth aspect disclosed herein, there is provided a system comprising: a display device configured to display an image, the display device comprising a screen, a backlight unit as described above, and a controller; a camera device configured to capture an image of the light output by the display device and transmit the captured image as image data to the controller; wherein the controller is configured to: receive the image data from the camera device; detect if one of the light sources of the backlight unit is faulty based on the received image data; and cause at least one other light source to move from a first position to a second position to allow the other light source to compensate for the faulty light source.
  • In an example, the camera device is configured to capture a subsequent image of the light output by the display device, the subsequent image being captured when the at least one other light source is at the second position, and transmit the subsequent image as image data to the controller; and wherein the controller is configured to determine, based on the received image data, whether the distribution of light output by the display device is of a desired uniformity, and if not, at least one of (i) cause the at least one other light source to move to a third position and (ii) adjust the intensity of light output by the at least one other light source from a first intensity to a second intensity, so that the at least one other light source compensates for the faulty light source.
  • Brief Description of the Drawings
  • To assist understanding of the present disclosure and to show how embodiments may be put into effect, reference is made by way of example to the accompanying drawings in which:
    • Figure 1 shows schematically a side view of a display device having a direct-lit backlight;
    • Figure 2 shows schematically an example of a display device having an edge-lit backlight;
    • Figure 3A shows schematically a portion of a display device in which the light sources of the backlight unit are organised in a first arrangement;
    • Figure 3B shows schematically a portion of a display device in which the light sources of the backlight unit are organised in a second arrangement;
    • Figure 3C shows schematically different regions of a screen that are illuminated by a light source in the first arrangement, and the same light source in the second arrangement; and
    • Figure 4 shows schematically a system for controlling the distribution of light received at a screen of a display device.
    Detailed Description
  • Many types of display device employ light sources for generating an image. Examples include, for example, television screens or monitors, computer displays or monitors, and displays for other computing devices, including smartphones, tablet computers, laptop computers, etc. These display devices tend to comprise a backlight and a screen, where the backlight generates an image by illuminating the screen. Typically, a display device employs either a direct-lit backlight or an edge-lit backlight.
  • Figure 1 shows schematically a side view of a display device 10 having a direct-lit backlight 11. As is known, the direct-lit backlight 11 has plural light sources 12 for emitting light. The light sources 12 may be for example LEDs (light emitting diodes). The light sources 12 are arranged typically in a regular array on a reflector panel 13. The light sources 12 may be associated with strips, and a group of light sources may be attached to the reflector panel 13 by fixing the respective strip to the reflector panel. The light sources 12 emit light which is directed through a diffuser 14 to a display panel 15. The diffuser 14 helps to reduce glare that can otherwise occur. The combination of the light sources, reflector panel and diffuser may be collectively referred to as a backlight unit. Alternatively, the term backlight unit may be used to refer to the combination of the light sources and the reflector panel only. The display panel 15 is formed of or includes a number of display elements 16 (which are also often referred to as "pixels" as they typically correspond to pixels in the image that is displayed). The display elements 16 are controllable so as to selectively transmit or prevent light from the light sources 12 passing through the display panel 15. The display elements 16 may be for example LCDs (liquid crystal display devices). In a display device 10 having a direct-lit backlight 11, generally there is a light source 12 for each display element 16.
  • Referring now to Figure 2, there is shown another example of a display device 20. This example has an edge-lit backlight 21. That is, there is at least one light source 22 which is arranged at or towards an edge of the display device 20. Commonly, there are light sources 22 arranged around each of the four edges of the display device 20. The light sources 22 are typically elongate and may be for example cold-cathode fluorescent lamps. In other examples, the light sources 22 located at the edges are plural LEDs or other individual light sources arranged along the edges of the display device 20. The light sources 22 emit light into a light guide 23 which is mounted in front of a reflector 24. The light guide 23 directs the light through a diffuser 25 into a display panel 26. Similarly to the example of Figure 1, the display panel 26 may have plural individually controllable display elements 27. The display elements 27 may be for example LCDs. As above, the term backlight unit may be used to describe the combination of the light sources, the reflector and/or the diffuser.
  • In these and other examples, in order to increase the brightness that is ultimately output by the display device, typically the electrical power supplied to the light sources 12, 22 is increased. Here it is mentioned that display devices that use LEDs as their backlights typically rapidly switch the LEDs on and off during normal operation. Pulse width modulation (PWM) is typically used to control the LEDs. To increase the brightness, the on time of duty cycle of the PWM is increased.
  • In known display devices such as those described above, each light source in the backlight unit is arranged such that, when each light source is emitting light at the same intensity, a uniform distribution of light is received at the screen. It will be understood that references herein to a "uniform distribution of light on the screen" means for example when the light sources are operating to produce a maximum light intensity. In practice during normal operation of the display device and backlight unit, at any particular point in time one or more specific light sources may be controlled to produce less than a maximum light intensity because the corresponding part of the image to be displayed is less bright or is dim at that location.
  • During operation, one or more of the light sources that make up the backlight unit of the display device may become faulty. A light source may be said to have become faulty when it is unable to emit light at a desired intensity or is no longer able to emit any light at all. For example, a light source may be associated with a nominal light intensity value, and the intensity of light output by the faulty light source may be less than the nominal intensity value and may even be zero.
  • A region of the front screen that is illuminated by the faulty light source, or that should have been illuminated by the faulty light source, may appear darker than it should as a result of less light being received at that region. This in turn will cause a corresponding region in the displayed image to appear darker than it should.
  • The present disclosure provides a method, backlight unit, display device and system through which the distribution of light received at a screen of the display device may be controlled. An example of the method is described below in relation to Figures 3A and 3B.
  • Figure 3A schematically shows portion of a display device 30 with a backlight unit 31 having 25 light sources arranged in a regular five by five array. It will be appreciated that, whilst a regular array of five by five light sources is shown for simplicity, the backlight unit 31 typically has many more such light sources. This example backlight unit 31 is a direct-lit backlight unit.
  • In Figure 3A, the backlight unit of the display device 30 is shown at an initial time t1. A faulty light source 32 is indicated in black at the centre of the array. Light sources that are nearest neighbours to the faulty light source 32 are indicated at 33, 34, 35 and 36 respectively. A light source is said to be a nearest neighbour to the faulty light source 32 if it is located physically closest to the faulty light source 32. For example, in Figure 3A, light sources 33, 34, 35 and 36 are equidistant from the faulty light source 32 and are each considered as nearest neighbours. If the array were, for example, a rectangular array, the nearest neighbours may include the light sources that are closest in the X-direction and the light sources that are closest in the Y-direction. In Figure 3A, each light of the nearest neighbour light sources 33, 34, 35 and 36 are shown located at a first position in the array.
  • In a first step of the method, light source 32 is identified as faulty. This may be achieved, for example, by first causing the display device to display a test image. The test image may be correspond to a plain white image, where each light source is emitting light at e.g. a maximum intensity. The displayed image may include one or more pixels that appear darker than expected. These one or more pixels correspond to the pixels that are receiving less or no illumination from the faulty light source 32. A mapping between the light sources in the backlight unit and the pixels of the screen may be known in advance. This mapping may be used to identify light source 32 as the light source that is causing the dark pixel(s) to appear in the displayed test image.
  • In a second step of the method, at least one of the other light sources in the backlight unit 31 is moved from a first position to a second position. The second position may be closer to the position of the faulty light source than the first position. This enables the moved light source to re-distribute some of the light towards the region of the screen that is receiving less or no light from the faulty light source 32. In this way, the moved light source is said to compensate for the faulty light source 32.
  • In some examples, all of the light sources that are nearest neighbours to the faulty light source are each moved to a position that is closer to the position of the faulty light source. The second position of the nearest neighbour light sources may be such that each nearest neighbour light source is still equidistant from the faulty light source 32. This is shown in Figure 3B. If the array is, for example, a rectangular array, the nearest neighbours in the X-direction may be moved closer to the faulty light source 32 by the same distance. Similarly, the nearest neighbours in the Y-direction may be moved closer to the position of the faulty light source 32 by the same distance. It will appreciated that light sources that are not a nearest neighbour to the faulty light source 32 may also be moved to a position that is closer to the position of the faulty light source 32. Generally, any light sources that are moved from a first position to a second position will remain at the second position, unless it is determined that a further, different position would better compensate for the faulty light source 32 (an example of which is described further below).
  • Figure 3B shows an example in which the four light sources that are nearest neighbours to the faulty light source 32 have each been moved to a second, different position. The backlight unit 31 shown in Figure 3B corresponds to the backlight unit 31 shown in Figure 3A, but at a later time t2. The light output by each of the light sources 33, 34, 35 and 36 is used to compensate for the darker region that would otherwise appear on the screen as a result of the faulty light source 32.
  • In addition to moving one or more of the other light sources to a different position in the array, the intensity of light that is output by that moved light source may be adjusted. For example, if a light source is moved from a first position to a second position, then the region of the front screen that was illuminated by the light source at the first position will receive less light when the light source is at the second position. To compensate for this, the intensity of light output by the moved light source may be increased. This is illustrated in Figure 3C.
  • Figure 3C shows the different regions of the screen that are illuminated by light source 33 at the first and second positions in the array. As can be seen in Figure 3C, when light source 33 is at the first position, a first region 330a of the screen is illuminated by light source 33. When light source 33 is at the second position, a second region 330b of the screen is illuminated by the light source. The second region 330b does not entirely overlap with the first region 330a. Hence, there will be a region on the screen that is receiving less light from the light source that has been moved to the second position. Generally, this region will be located in a direction that is opposite to the direction in which the light source has been moved.
  • The intensity of the light that is output by other light sources that are not nearest neighbours to the faulty light source 32, and which may or may not have also been moved depending on the embodiment, may also be adjusted. For example, other light sources within the vicinity of the moved light source may also have their intensity increased. This allows the other light sources to compensate for any dark regions that might occur as a result of another light source having been moved closer to the faulty light source 32.
  • In a further (optional) step of the method, the intensity of light output by one or more light sources that are not nearest neighbours to the faulty light source 32 is decreased (i.e. dimmed). The dimming of these other light sources may be controlled so as to counteract any excessively bright regions that would otherwise occur as a result of the change in position and/or intensity of the one or more moved light sources. Figure 3A shows an example of a light source 37 that might be selected for dimming.
  • In some examples, a faulty light source 32 may be emitting less light than is desired, but enough to generate an image of acceptable quality on the screen of the display device 30. In this situation, the intensity of light output by other non-faulty light sources may be reduced so that the lower intensity level of the faulty light source 32 is less noticeable.
  • It will be appreciated that, by configuring the light sources as described above, the distribution of light received at the screen of the display device can be maintained as uniform, or within an acceptable level of uniformity. Advantageously, this means that any darker regions that would otherwise appear in the displayed image can be compensated for. A user therefore does not have to wait for the faulty light source 32 to be replaced or for the backlight unit 31 as a whole to be replaced in order to view images of a desired quality.
  • Figure 4 schematically shows an example of a system 400 for compensating for faulty light sources. The system 400 comprises a display device 40 and a camera device 41. The camera device 41 may be configured to couple to the display device 40 by way of, for example, a Bluetooth or WiFi connection. This coupling is indicated as communication line 42 in Figure 4.
  • The display device 40 comprises a screen 43, a backlight unit (not shown) and a controller 44. The screen 43 of the display device 40 may be illuminated by a direct-lit backlight. The light sources in the backlight unit may be arranged accordingly. The controller 44 may be separate from the display device 40 and its backlight unit or may be integrated within the display device 40 and its backlight unit.
  • The backlight unit comprises a plurality of actuators 45 for controlling the position of one or more light sources in the backlight unit. (Only one actuator 45 is shown in the drawing and is shown schematically as a motor. Examples of actuators 45 will be discussed below.) The controller 44 is communicatively coupled to each of the actuators 45. Communication line 46 represents the connection between the controller and the actuators 45. The actuators 45 are coupled to one or more light sources, as indicated by communication line 47. The controller 44 is also coupled to each of the light sources in the backlight unit via communication line 48. Communication line 48 enables the intensity of light output by different light sources in the backlight unit to be controlled.
  • The position of a light source in the backlight unit is controlled by controlling the actuator associated with that light source. In some examples, each light source is associated with a respective actuator, such that each light source in the backlight unit has an individually controllable position. A light source may be associated with two actuators. For example, a first actuator may be used to control the position of the light source in a horizontal ("X") direction and a second actuator may be used to control the position of the light source in a vertical ("Y") direction.
  • In other examples, only some, which may be a majority, of the light sources are associated with respective actuators. In yet other examples, groups of light sources may be associated with a single respective actuator such that the positions of the light sources in a respective group are controlled by the associated actuator. In all examples, at least two of the light sources have an individually controllable position.
  • In the system shown in Figure 4, the controller 44 is communicatively coupled to each of the plurality of actuators 45. However, in some embodiments, a light source and the associated actuator may considered as a single unit. Hence it may be possible to control both the position and intensity of light output by a light source via a single connection between the controller 44 and the light source.
  • An example of an actuator that may be used move the light sources in the backlight unit is a servo motor. Other examples of actuators 45 which may be used for this purpose include microelectromechanical systems (MEMS) and nano-machines. The plurality of actuators 45 present in the display device may comprise a plurality of the same type of actuators or may comprise two or more different types of actuator.
  • The controller 44 may be operable to individually adjust the intensity of light that is output by one or more light sources in the backlight unit. In normal use, this may be used to perform e.g. local dimming. However, in the present disclosure, this may also be used to compensate for any unwanted dark pixels that appear in a displayed image. In some examples, the intensity of light output by each of the light sources in the backlight unit is individually controllable. This may involve, for example, adjusting the on time of duty cycle of the PWM for each light source. The controller 44 is configured to determine which of the light sources are to compensate for the faulty light source, and the intensity of light that is to be output by those light sources.
  • The camera device 41 is configured to capture an image of the display device 40 when it is displaying a test image. As mentioned previously, the test image may correspond to a blank white image. The blank white image may be generated by the light sources in the backlight unit emitting light at a maximum intensity. The camera device 41 may be configured to capture images of the test image periodically, for example, every few seconds or so. The capturing of these images may be triggered, for example, in response to determining that the display device 40 is displaying the test image. For example, the display device may be configured to transmit a triggering signal to the camera device as soon as the display device has begun displaying the test image.
  • The camera device 41 is located in front of the screen 43 of the display device 40 such that any captured images will include the whole screen 43 of the display device 40. In an example, the camera device 41 captures an image of the screen 43 only (i.e. no background). The camera device 41, may, for example, be fixed to a wall in front of the display device 40. In other examples, the camera device 41 may be part of another device that a user already owns, such as a smartphone, tablet, personal computer, etc. If the camera device 41 is hand-held, the user may be required to hold the camera device 41 in front of the screen 43 and perform the capturing of images manually. A user may be guided through this process by downloading a software application to their user device, the application being provided by the company that manufactured the display device 40.
  • The camera device 41 includes a transmitter for transmitting the captured image as image data to a receiver at the display device 40. The image data may include, for example, a grayscale image of the test image being displayed at the screen 43. The pixels of the grayscale image provide an indication of the brightness of each of the pixels of the screen 43 of the display device. The received image data may be received at an input of the controller 44.
  • The controller 44 comprises image analysis software for processing the image data received from the camera device 41. This software is stored in memory at the controller 44 and is executed on one or more of processors at the controller 44. The image analysis software may be used to filter out parts of the image that do not correspond to the screen 43 of the display device 40 (e.g. background). In some examples, this may be performed as a preliminary step at the camera device 41. If the captured images are of the screen 43 only, then this step may be omitted.
  • The controller 44 uses the image analysis software to identify any regions in the displayed test image that appear darker than expected. This may involve, for example, identifying any pixels in the captured image that have a brightness that is lower than a desired brightness. The desired brightness may be known in advance. For example, the controller 44 may be pre-programmed with a brightness value (or brightness range) that would be expected for a display device that is displaying the test image at the maximum brightness. Alternatively, the desired brightness may be determined based on the average brightness of the captured image. For example, any pixels with a brightness that deviates significantly from the average brightness may be identified as being associated with faulty light sources. Here, a deviation may be considered significant if it is larger than a threshold value of deviation.
  • The controller 44 may also be configured to determine a mapping between the pixels in the captured image and the light sources in the backlight unit of the display device 40. This mapping may be performed, for example, based on knowledge of the positions of the light sources in the backlight unit and their maximum operating intensity. This information may be stored, for example, in memory at the controller 44. The distance of the camera device 41 from the display device 40 may also be used to facilitate the mapping between the pixels of the captured image and the light sources in the backlight unit.
  • In some examples, the mapping between the pixels of the captured image and the light sources in the backlight unit may be predetermined. This may occur if for example the camera device 41 is always located at the same distance from the screen 43 and only captures an image of the screen 43. In such an example, the mapping operation above does not need to be calculated for each image that is received at the controller 44.
  • The controller 44 uses the image analysis and mapping described above to detect if any of the light sources are faulty. A light source is detected as faulty if its location in the backlight unit corresponds to one or more darker pixels in the captured image. It will be appreciated that the number of light sources that are detected as faulty will depend on the number of dark pixels that appear in the captured image.
  • Having identified a light source as faulty, the controller 44 identifies at least one other light source that is to be moved from a first position to a second position in the backlight unit of the display device. As described previously, the at least one other light source may be a nearest neighbour to the faulty light source 32. In some examples, light sources that are not nearest neighbours may also identified for moving.
  • The controller 44 determines a new position (i.e. a second position) to which the at least one other light source is to be moved. This may involve determining how far, and in what direction, the at least one other light source is to be moved. The controller transmits this information to the actuator(s) associated with the at least one other light source. The actuator then moves the at least one other light source by the corresponding amount in the corresponding direction. Thus, the at least one other light source is moved to the desired location in the backlight unit. It will be appreciated that, whilst this is happening, light sources that have not been selected for moving will remain at their current positions in the backlight unit.
  • As described previously, the at least one other light source may be moved to a position that is closer to the position of the faulty light source 32. Generally, the position to which the at least one other light source is moved may depend on the brightness of the pixels associated with the faulty light source 32. For example, in the case that the faulty light source 32 is still emitting some light, but not enough light, then the darker the pixels, the closer the position of the at least one other light source to the position of the faulty light source 32.
  • The controller 44 may also be configured to adjust the intensity of light that is output by the at least one other light source. For example, the controller 44 may cause the moved light source to output light at a higher intensity. This is so that any regions of the screen 43 that were previously illuminated by the moved light source, are still illuminated by the moved light source at its new position. Here, the intensity is higher relative to the intensity of light that was output by the light source at its previous position.
  • The controller 44 may also adjust the intensity of light that is output by light sources that have not been moved to a new position. For example, the controller 44 may be configured to identify the light sources that were nearest neighbours to the moved light source when it was at its previous position. The controller 44 may cause these light sources to output light at a higher intensity, to compensate for the light source that has been moved closer to the faulty light source 32.
  • The controller 44 may also cause dimming of light sources located further from the faulty light source 32. This is for the same reasons as described in relation to Figures 3A to 3C.
  • The camera device 41 may be configured to capture a subsequent image of the display device 40 when it is displaying the test image. This test image corresponds to the test image that is generated when at least one of the other light sources has been moved to a different position in the backlight unit. The intensity of light output by the display devices may also have been adjusted as described above. The controller 44 may be configured to receive this image (as image data) and determine whether the distribution of light output by the display device 40 is of a desired uniformity. Again, this may involve determining whether the pixels in the captured image are of a satisfactory brightness. If it is determined that the distribution of light output by the display device 40 is not of the desired uniformity, the controller 44 may be configured to re-adjust the position of one or more light sources in the backlight unit. This may involve re-adjusting the position of any light sources that were moved previously to yet different positions. In another example, this may involve moving one or more light sources that were not previously moved, to new positions.
  • The controller 44 may also be configured to further adjust the intensity of light that is output by one or more of the light sources in the backlight unit. This may involve adjusting the intensity of light that is output by light sources that had their intensities adjusted previously. It may also involve adjusting the intensity of light that is output by light sources that did not have their intensities adjusted previously.
  • The controller 44 may be configured to re-adjust the positions of the light sources in the backlight unit until the light output by the display device 40 is of a desired uniformity. The controller 44 may also be configured to re-adjust the brightness of the light sources in the backlight unit until the desired uniformity is obtained. The desired uniformity may have been said to have been obtained when all, or a majority of the pixels in the received image data are of a satisfactory brightness. The controller 44 may be required to iterate through a number of cycles of receiving image data and adjusting the positons and/or intensities of the light sources until the desired uniformity is achieved.
  • If the desired uniformity cannot be obtained, the controller 44 may be configured to select the configuration (i.e. positions and brightness) of light sources for which the uniformity was detected as being greatest.
  • In the examples described above, the faulty light source 32 has been described as outputting light at an intensity that is lower than a desired intensity. It will be appreciated that the same system may be adapted to compensate for a faulty light source 32 that is outputting light at an intensity that is higher (i.e. is brighter) than desired. In such a situation, one or more light sources may be moved away from the faulty light source 32. These light sources may correspond to the nearest neighbours of the faulty light source 32. The light sources that have been moved away may also have their intensities reduced, so as to compensate for an excessively bright region that might otherwise arise on the screen 43.
  • It will be understood that any processors or processing systems or circuitry referred to herein may in practice be provided by a single chip or integrated circuit or plural chips or integrated circuits, optionally provided as a chipset, an application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), digital signal processor (DSP), graphics processing units (GPUs), etc. The chip or chips may comprise circuitry (as well as possibly firmware) for embodying at least one or more of a data processor or processors and a digital signal processor or processors, which are configurable so as to operate in accordance with the exemplary embodiments. In this regard, the exemplary embodiments may be implemented at least in part by computer software stored in (non-transitory) memory and executable by the processor, or by hardware, or by a combination of tangibly stored software and hardware (and tangibly stored firmware).
  • The examples described herein are to be understood as illustrative examples of embodiments of the invention. Further embodiments and examples are envisaged.

Claims (9)

  1. A method of controlling a backlight unit (31) of a display device (30), the backlight unit (31) comprising a plurality of light sources for illuminating a screen (43) of the display device (30), the method comprising:
    detecting a faulty light source (32) amongst the plurality of light sources;
    characterised by:
    at least some of the light sources comprising an actuator (45) for controllably moving the light source within the backlight unit (31);
    and characterised by the method comprising:
    operating an actuator (45) of the backlight unit (31) to move at least one other light source (33-36) that is a nearest neighbour to the faulty light source (32) from a first position to a second position and adjusting the intensity of light that is output by the at least one other light source (33-36) at the second position to allow the at least one other light source (33-36) to compensate for the faulty light source (32).
  2. A method according to claim 1, wherein the faulty light source (32) outputs no light or outputs light at an intensity that is lower than a desired intensity; and
    wherein the second position of the at least one other light source (33-36) is closer to the position of the faulty light source (32) than the first position.
  3. A method according to claim 2, comprising increasing the intensity of light that is output by the at least one other light source (33-36) at the second position.
  4. A method according to claim 2, comprising decreasing the intensity of light that is output by one or more light sources (37) that are not nearest neighbours to the faulty light source (32).
  5. A method according to any of claims 1 to 4, comprising receiving image data indicative of the distribution of light output by the display device (30), and wherein the step of detecting the faulty light source (32) is based on the received image data.
  6. A backlight unit (31) for a display device (40), the backlight unit (31) comprising:
    a plurality of light sources for illuminating a screen (43) of the display device (40); characterised by:
    at least some of the light sources comprising an actuator (45) for controllably moving the light source within the backlight unit (31),
    the backlight unit (31) being arranged such that if one (32) of the light sources is detected by a controller (44) as being faulty, at least one of the other light sources (33-36) that is a nearest neighbour to the faulty light source (32) is caused by the controller (44) controlling the actuator (45) of the light source (33-36) to move from a first position to a second position and the intensity of light output by the at least one other light source (33-36) is caused by the controller (44) to be adjusted from a first intensity to a second intensity, so that the at least one other light source compensates for the faulty light source (32).
  7. A display device (40), the display device (40) comprising:
    a screen (43);
    a backlight unit (31) according to claim 6 for illuminating the screen; and
    a controller (44) for controllably moving the at least one other light source (33-36) from a first position to a second position and for adjusting the intensity of light output by the at least one other light source (33-36).
  8. A system (400) comprising:
    a display device (40) configured to display an image, the display device (40) comprising a screen (43), a backlight unit (31) according to claim 6, and a controller (44); and
    a camera device (41) configured to capture an image of the light output by the display device (40) and transmit the captured image as image data to the controller (44);
    wherein the controller (44) is configured to:
    receive the image data from the camera device (41);
    detect if one of the light sources of the backlight unit (31) is faulty based on the received image data; and
    cause at least one other light source (33-36) to move from a first position to a second position to allow the other light source (33-36) to compensate for the faulty light source (32).
  9. A system (400) according to claim 8, wherein the camera device (41) is configured to capture a subsequent image of the light output by the display device (40), the subsequent image being captured when the at least one other light source (33-36) is at the second position, and transmit the subsequent image as image data to the controller (44); and
    wherein the controller (44) is configured to determine, based on the received image data, whether the distribution of light output by the display device (40) is of a desired uniformity, and if not, at least one of (i) cause the at least one other light source to move to a third position and (ii) adjust the intensity of light output by the at least one other light source (33-36) from a first intensity to a second intensity, so that the at least one other light source (33-36) compensates for the faulty light source (32).
EP17183145.6A 2017-07-25 2017-07-25 Method of controlling a backlight unit, a backlight unit and a display device and camera system Active EP3435366B1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP17183145.6A EP3435366B1 (en) 2017-07-25 2017-07-25 Method of controlling a backlight unit, a backlight unit and a display device and camera system
TR2017/11138A TR201711138A2 (en) 2017-07-25 2017-07-28 Method of controlling a backlight unit, a backlight unit and a display device and camera system.

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP17183145.6A EP3435366B1 (en) 2017-07-25 2017-07-25 Method of controlling a backlight unit, a backlight unit and a display device and camera system

Publications (2)

Publication Number Publication Date
EP3435366A1 EP3435366A1 (en) 2019-01-30
EP3435366B1 true EP3435366B1 (en) 2021-08-25

Family

ID=59399370

Family Applications (1)

Application Number Title Priority Date Filing Date
EP17183145.6A Active EP3435366B1 (en) 2017-07-25 2017-07-25 Method of controlling a backlight unit, a backlight unit and a display device and camera system

Country Status (2)

Country Link
EP (1) EP3435366B1 (en)
TR (1) TR201711138A2 (en)

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7086736B2 (en) * 2004-01-20 2006-08-08 Hewlett-Packard Development Company, L.P. Display system with sequential color and wobble device
WO2011028335A1 (en) * 2009-09-02 2011-03-10 Dolby Laboratories Licensing Corporation Compensation for sub-par lighting in displays
US20120147293A1 (en) * 2010-06-11 2012-06-14 Amercan Panel Corporation Redundant backlight for liquid crystal displays

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
TR201711138A2 (en) 2019-02-21
EP3435366A1 (en) 2019-01-30

Similar Documents

Publication Publication Date Title
KR100831369B1 (en) Backlight apparatus for display device and method of adjusting brightness for the same
US20170047022A1 (en) Display apparatus and control method thereof
EP3644303B1 (en) Display apparatus and method for driving same
US9529562B2 (en) Image display apparatus, image output apparatus, and control methods therefor
CN103456275B (en) The backlight driver of liquid crystal indicator and driving method thereof
KR101435466B1 (en) Display apparatus and method for scanning a backlight thereof
KR101182245B1 (en) Display apparatus and control method thereof
US9349328B2 (en) Backlight apparatus, method for controlling the same, and image display apparatus
US20110249034A1 (en) Luminance Control Apparatus and Luminance Control Method
KR101462066B1 (en) Compensation for sub-par lighting in displays
JP2017015768A (en) Display unit and method of controlling the same
US20140368558A1 (en) Image display apparatus and method of controlling same
US10705377B2 (en) Liquid crystal display device and method of controlling the same
US9966012B2 (en) Image display apparatus and control method thereof
EP2923353B1 (en) Automatic color adjustment on led video screen
US9786216B2 (en) Display apparatus, light-emitting device, and control method of display apparatus
US11749214B2 (en) Display device
US11605356B2 (en) Driving display apparatus and method acquiring current duty to drive backlight unit based on excluding text area in input image
EP3435366B1 (en) Method of controlling a backlight unit, a backlight unit and a display device and camera system
KR101654082B1 (en) Method for Control Brightness of Unit Module of Electronic Display and Apparatus therof
US20140055510A1 (en) Display apparatus and control method thereof
JP2012053387A (en) Backlight device, display device equipped with the backlight device, and illuminating device
US20090167655A1 (en) Light emission controller and liquid crystal display apparatus including light emission controller
WO2010020949A1 (en) Video processing with spatially-based backlight control
JP6917759B2 (en) Video generation system, video signal generator, and display device

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20190730

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20191211

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20210324

RIN1 Information on inventor provided before grant (corrected)

Inventor name: ER, ALPER SAIT

Inventor name: YILDIZ, KADIR

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602017044601

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Ref country code: AT

Ref legal event code: REF

Ref document number: 1424601

Country of ref document: AT

Kind code of ref document: T

Effective date: 20210915

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20210825

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1424601

Country of ref document: AT

Kind code of ref document: T

Effective date: 20210825

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210825

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211125

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210825

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211227

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211125

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210825

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210825

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210825

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210825

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210825

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210825

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210825

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20211126

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210825

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210825

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602017044601

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210825

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210825

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210825

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210825

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210825

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210825

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210825

26N No opposition filed

Effective date: 20220527

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210825

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210825

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20220731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220725

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220725

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: TR

Payment date: 20230725

Year of fee payment: 7

Ref country code: GB

Payment date: 20230721

Year of fee payment: 7

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20230719

Year of fee payment: 7

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20170725

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210825

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210825