EP2477176A1 - Backlight compensation table approximation for displays - Google Patents

Backlight compensation table approximation for displays Download PDF

Info

Publication number
EP2477176A1
EP2477176A1 EP12151013A EP12151013A EP2477176A1 EP 2477176 A1 EP2477176 A1 EP 2477176A1 EP 12151013 A EP12151013 A EP 12151013A EP 12151013 A EP12151013 A EP 12151013A EP 2477176 A1 EP2477176 A1 EP 2477176A1
Authority
EP
European Patent Office
Prior art keywords
matrix
display
backlight
size
approximation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP12151013A
Other languages
German (de)
French (fr)
Other versions
EP2477176B1 (en
Inventor
Emrah Tasli
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Vestel Elektronik Sanayi ve Ticaret AS
Original Assignee
Vestel Elektronik Sanayi ve Ticaret AS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Vestel Elektronik Sanayi ve Ticaret AS filed Critical Vestel Elektronik Sanayi ve Ticaret AS
Publication of EP2477176A1 publication Critical patent/EP2477176A1/en
Application granted granted Critical
Publication of EP2477176B1 publication Critical patent/EP2477176B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G3/00Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
    • G09G3/20Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters
    • G09G3/34Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters by control of light from an independent source
    • G09G3/3406Control of illumination source
    • G09G3/342Control of illumination source using several illumination sources separately controlled corresponding to different display panel areas, e.g. along one dimension such as lines
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G3/00Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
    • G09G3/20Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters
    • G09G3/34Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix no fixed position being assigned to or needed to be assigned to the individual characters or partial characters by control of light from an independent source
    • G09G3/3406Control of illumination source
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00Control of display operating conditions
    • G09G2320/02Improving the quality of display appearance
    • G09G2320/0233Improving the luminance or brightness uniformity across the screen
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00Control of display operating conditions
    • G09G2320/02Improving the quality of display appearance
    • G09G2320/0285Improving the quality of display appearance using tables for spatial correction of display data

Definitions

  • the present invention is related to a backlight compensation table approximation method and system for displays.
  • LCDs Liquid Crystal Displays
  • Backlights may be positioned both under the thin film layer of the display, which is called as direct-lit displays, or along the edges of the display, which is called as edge-lit displays.
  • the direct-lit displays employ uniformly distributed light elements along the display. Hence, the display is uniformly illuminated by the backlight.
  • the edge-lit displays employ a limited number of backlights, which are positioned along the edges of the display. Since the light intensity has maximum value at a zero distance to the source and decreases as the distance increases, the display cannot be illuminated uniformly.
  • edge-lit displays Another drawback with the edge-lit displays are that since the backlight cannot be used for illumination of a definite region of the display, it is not possible to use local dimming methods as in the direct-lit displays. Instead, a combined illumination of backlights should be considered and to obtain uniformity, again complex operations and large memory space is required.
  • edge-lit displays For the stated drawbacks of the edge-lit displays, the market is focused on direct-lit displays, which serves for quality picture but the production cost is relatively much higher than edge-lit displays.
  • WO 2007/027539 A1 patent document discloses another solution for backlight compensation in which a concave transflective panel to distribute emitted light uniformly is utilized. This method is well known and widely used in edge-lit displays. However the fact that the invention can be used with only one backlight source limits the scope of the solution. Furthermore it is known that trans-reflective panels do not provide a complete solution for uniformity since a perfectly light distribution surface cannot be obtained.
  • US 7717602 B2 patent document discloses another solution for backlight non-uniformity compensation. Stated solution comprises utilizing extra backlight elements in the display and controlling these backlight elements to compensate backlight non-uniformities.
  • all backlight sources known in the prior art provide a point spread light beam, extra backlight sources add new non-uniformities and it is not possible to fully compensate non-uniformities using the solution of this patent document.
  • Present invention comprises a backlight compensation table approximation method by utilizing singular value decomposition of a matrix and approximation of matrix via determining a rank number and reducing matrix sizes using this rank number.
  • the invention also discloses utilization of this approximation in displays to obtain backlight uniformity along the whole display.
  • the approximation value is utilized to adjust transmission values of display pixels and compensate backlight deviation all over the display.
  • a backlight compensation table approximation method for a display which comprises at least one backlight source and pixels of m rows and n columns and wherein a compensation table of said backlight source with respect to m rows and n columns of pixels as a table M is known, said method comprising the steps of;
  • a backlight compensation method for a display which comprises at least one backlight source and pixels of m rows and n columns the method comprising the steps of;
  • a local dimming method for a display which comprises a plurality of backlight sources and pixels of m rows and n columns, the method comprising the steps of;
  • a display comprising at least which comprises at least one backlight source and pixels of m rows and n columns, further comprising;
  • a display which comprises at least one backlight source and pixels of m rows and n columns, further comprising;
  • the object of the invention is to provide a backlight compensation table approximation method for displays.
  • Another object of the invention is to provide a backlight compensation method and system for compensation of backlight non-uniformity on a display.
  • Another object of the invention is to provide a backlight compensation method and system which reduces the required memory for a compensation table.
  • Another object of the invention is to provide a backlight compensation method and system which enables utilization of local dimming in edge-lit displays.
  • Edge-lit displays suffer from backlight intensity non-uniformity, which degrades perceived image quality on the display.
  • the solution of the state of the art is to compensate non-uniformity by adjusting display thin film layer by utilizing a compensation table.
  • the aim of the method and system of present invention is to reduce the memory required by approximation of the compensation table by singular value decomposition and a mean square error (MSE) criterion.
  • MSE mean square error
  • a matrix M having a size of m x n can be represented using singular value decomposition as shown in the equation (1).
  • M U ⁇ V *
  • U is an m x m unitary matrix
  • is an m x n diagonal matrix with nonnegative real numbers on the diagonal
  • V* denotes the conjugate transpose of V, which is an n x n unitary matrix.
  • V can be assumed real and hence conjugate can be replaced by transpose.
  • Equation (1) can be decomposed into two equations (2, 3) to realize the point spread function.
  • the method of the present invention utilizes an approximate model matrix M' for matrix M using an R th order approximation.
  • the approximation is based on minimizing the frobenius norm of the difference between the matrix M and M'.
  • ⁇ ' is obtained by leaving the largest R values of the matrix ⁇ and setting all other values to zero. This reduces the size of matrices which are composed to determine M'.
  • Figure 1 shows light intensity of a point light source (e.g. light emitting diode - LED) with respect to horizontal position.
  • Figure 2 shows light intensity of the same source with respect to vertical position.
  • the highly correlation characteristic of intensity distribution enables approximation of the compensation table easily.
  • the rank number R is determined by comparing approximate and real values using mean square error (MSE) criterion.
  • MSE criterion gives true information of a practical perception error.
  • Figure 3 shows MSE of approximation of present invention with respect to rank number R. As it can be seen on the figure, the MSE of the approximation decreases exponentially with the increasing rank number. Determination of rank number by means of visual observation as well as mathematical results is also important. Application of method and visual observation states similar results with mathematical results and those clearly state that, a rank number 5 is high enough to approximate compensation table adequately and no visual difference can be observed in visual observation.
  • the method of the present invention comprises the steps of;
  • the approximation technique of present invention can be used for backlight compensation in a display device comprising at least a backlight source and pixels of m rows and n columns. Said method comprises the steps of;
  • Present invention further comprises a display having a display resolution m x n, utilizing the method of the invention in determining backlight compensation to obtain uniformity.
  • the display comprises
  • the display of the present invention utilizes the approximation method of the invention and adjusts pixels of the display by calculating a compensation value for each light source of the display. As a result, the whole display is compensated regarding light intensity non-uniformity.
  • the compensation value may be calculated for each refresh of the pixels of the display. If the display device has adequate temporary memory to store the whole approximate matrix, the calculated compensation values may be stored in the temporary memory to avoid repeated compensation calculations. As long as light intensity parameters are not changed by means of other parameters of the display, for instance a brightness adjustment by the user, it may not be necessary to calculate compensation values repeatedly.
  • backlight control based enhancements are possible since it is possible to determine uniformity around the whole display.
  • backlight control based enhancements such as local dimming or boosting backlight to increase contrast and picture dynamics can be only applied in devices with direct-lit displays.
  • local dimming or boosting backlight to increase contrast and picture dynamics
  • the compensation method of the present invention is applied for each backlight source of the display, it is merely a matter of calculation and prediction to use local dimming since the result of a combination of different illumination levels on any pixel can be calculated and a desired local dimming can be determined in accordance with compensation calculations.
  • the rest of the display is also affected since no locality exists.
  • the applicability of local dimming using the method of the invention enables utilization of local dimming in not only direct-lit displays but also in edge-lit displays.
  • the method can be used in direct-lit displays to reduce the number of backlight sources but maintain the effectiveness of local dimming.
  • Present invention further comprises a display device comprising a plurality of backlight sources and a plurality of predefined regions on the display comprising;
  • the approximation technique reduces the required memory size significantly, it is possible to store the matrices used for approximation separately for each backlight source. However, if the backlight sources have equivalent light intensity characteristics, it is possible to use same matrices for approximation of these equivalent backlight sources.
  • Basic matrix operations are useful such as exchanging row indexes and column indexes, moving row indexes and column indexes, reversal of row indexes and/or column indexes, and limitation of row index and/or column index.
  • Present invention enables illumination uniformity and fully illumination control in both direct-lit and edge-lit displays independently from the number and positions of the backlight sources.

Abstract

Present invention comprises an approximation method for backlight compensation tables and method and system for backlight compensation of displays. The invention further comprises method and system for local dimming in displays utilizing the approximation of the present invention. Within the methods and systems of the present invention, it is possible to determine uniform backlight over an entire display by generating an approximate compensation table. The invention enables to further control backlight sources of both edge-lit and direct-lit displays for image enhancement operations such as local dimming for backlight sources.

Description

    Technical Field
  • The present invention is related to a backlight compensation table approximation method and system for displays.
  • Prior Art
  • LCDs (Liquid Crystal Displays) employ a plurality of backlights to illuminate display. Backlights may be positioned both under the thin film layer of the display, which is called as direct-lit displays, or along the edges of the display, which is called as edge-lit displays.
  • The direct-lit displays employ uniformly distributed light elements along the display. Hence, the display is uniformly illuminated by the backlight.
  • However, the edge-lit displays employ a limited number of backlights, which are positioned along the edges of the display. Since the light intensity has maximum value at a zero distance to the source and decreases as the distance increases, the display cannot be illuminated uniformly.
  • To obtain a uniform illumination, several light reflective or light transmitting elements are used within the display. However, those elements are not adequate to obtain uniformity, either. Therefore a further compensation for non-uniformity compensation is required.
  • Wide investigations on compensation methods led to the conclusion that, it is possible to compensate light distribution non-uniformity by adjusting thin film layer to altering light transmission. However, in practice, a compensation data requires complex calculations and furthermore large memory space to store compensation data.
  • For instance, a full HD (High Definition) (1920 pixels by 1080 pixels) display, having 16 bits colour resolution requires a compensation table of 1920 x 1080 x 16 = 33177600 bits which is approximately 4 megabytes, for a single light source. Due to this large memory requirement, compensation is not considered as a practical solution and a fully uniform back-light cannot be realized.
  • Another drawback with the edge-lit displays are that since the backlight cannot be used for illumination of a definite region of the display, it is not possible to use local dimming methods as in the direct-lit displays. Instead, a combined illumination of backlights should be considered and to obtain uniformity, again complex operations and large memory space is required.
  • For the stated drawbacks of the edge-lit displays, the market is focused on direct-lit displays, which serves for quality picture but the production cost is relatively much higher than edge-lit displays.
  • Several solutions are proposed regarding the stated problems. One of these solutions is described in WO 2010/096269 A1 patent document in which a compensation panel is employed within the display and an image acquiring device is positioned in front of the display to detect backlight non-uniformities. The compensation panel is adjusted by a system controller based on the acquired non-uniformities. The proposed solution may compensate for non-uniformities. However utilization of a compensation panel and image acquiring device brings in both implementation difficulty and significant cost increase.
  • WO 2007/027539 A1 patent document discloses another solution for backlight compensation in which a concave transflective panel to distribute emitted light uniformly is utilized. This method is well known and widely used in edge-lit displays. However the fact that the invention can be used with only one backlight source limits the scope of the solution. Furthermore it is known that trans-reflective panels do not provide a complete solution for uniformity since a perfectly light distribution surface cannot be obtained.
  • US 7717602 B2 patent document discloses another solution for backlight non-uniformity compensation. Stated solution comprises utilizing extra backlight elements in the display and controlling these backlight elements to compensate backlight non-uniformities. However, since all backlight sources known in the prior art provide a point spread light beam, extra backlight sources add new non-uniformities and it is not possible to fully compensate non-uniformities using the solution of this patent document.
  • Obviously, a method for eliminating implementation difficulties of edge-lit displays and determining high display quality is required.
  • Brief Description of the Invention
  • Present invention comprises a backlight compensation table approximation method by utilizing singular value decomposition of a matrix and approximation of matrix via determining a rank number and reducing matrix sizes using this rank number.
  • Since the distribution of light along a vertical and horizontal axis is highly correlated, a successful approximation can be done for a compensation table and a bulky compensation table is compressed up to a ratio of 99%.
  • The invention also discloses utilization of this approximation in displays to obtain backlight uniformity along the whole display. The approximation value is utilized to adjust transmission values of display pixels and compensate backlight deviation all over the display.
  • Furthermore, within the teaching of the present invention, method and system are proposed for local dimming for both edge-lit and direct-lit displays wherein backlight uniformity is maintained while effective and smooth local dimming is achieved.
  • The scope of protection of the present invention therefore comprises;
  • A backlight compensation table approximation method for a display which comprises at least one backlight source and pixels of m rows and n columns and wherein a compensation table of said backlight source with respect to m rows and n columns of pixels as a table M is known, said method comprising the steps of;
    • Determining a rank number R;
    • Defining said m x n compensation table using singular value decomposition
      equation (1) M = UΣV *
      Figure imgb0001

      wherein U denotes an m x m unitary matrix, ∑ is an m x n diagonal matrix with nonnegative real numbers on the diagonal and V* denotes the conjugate transpose of V, which is an n x n unitary matrix;
    • Obtaining a matrix ∑' by leaving largest R values of ∑ and setting all other values to zero;
    • Obtaining a matrix ∑" diagonals of which are singular, by reducing the size of ∑'to R x R;
    • Obtaining an m x R matrix U" by reducing U such that columns of U" are left singular vectors of U;
    • Obtaining an R x n matrix V" by reducing V such that columns of V" are right singular vectors of V;
    • Defining a matrix M' as an approximation of M by the approximation equation (5) as; = UʺΣʺVʺ T
      Figure imgb0002
  • A backlight compensation method for a display which comprises at least one backlight source and pixels of m rows and n columns the method comprising the steps of;
    • Storing a first matrix U" having size of m x R, a second matrix ∑" having size of R x R and third matrix V"T having size of R x n for at least one backlight source of the display, wherein R denotes an approximation rank number;
    • calculating a compensation value for at least one pixel by using an approximation equation (5); = UʺΣʺVʺ T
      Figure imgb0003
    • adjusting said pixel's light transmission value using said calculated compensation value,
  • A local dimming method for a display which comprises a plurality of backlight sources and pixels of m rows and n columns, the method comprising the steps of;
    • Storing a first matrix U" having size of m x R, a second matrix ∑" having size of R x R and third matrix V"T having size of R x n for at least one backlight source of the display, wherein R denotes an approximation rank number;
    • Processing an input image to determine local dimming levels for each predefined region of the display;
    • Adjusting backlight driving level of at least one backlight source to dim at least a portion of the display including at least one predefined region that is determined to have local dimming;
    • Calculating a compensation value for at least one pixel by approximating illumination level originating from each backlight source using the equation (5)
    = UʺΣʺVʺ T
    Figure imgb0004
    and summing all approximated illumination levels;
    • Adjusting said pixel's light transmission value using said calculated compensation value,
  • A display comprising at least which comprises at least one backlight source and pixels of m rows and n columns, further comprising;
    • Means for storing a first matrix U" having size of m x R a second matrix ∑" having size of R x R and a third matrix V"T having size of R x n for at least one backlight source of the display wherein R denotes an approximation rank number;
    • Means for calculating a compensation value by using an approximation equation (5);
    - = UʺΣʺVʺ T
    Figure imgb0005
    and
    • Means for adjusting said pixels' light transmission value using said calculated compensation value.
    and
  • A display which comprises at least one backlight source and pixels of m rows and n columns, further comprising;
    • Means for storing a first matrix U" having size of m x R, a second matrix ∑" having size of R x R and third matrix V"T having size of R x n for at least one backlight source of the display wherein R denotes an approximation rank number;
    • Means for processing input image to determine local dimming levels for each predefined region of the display;
    • Means for adjusting backlight driving levels of backlight sources to dim at least a portion of the display including at least one predefined region that is determined to have local dimming;
    • Means for calculating a compensation value for at least one pixel by approximating illumination level originating from each backlight source using the equation (5) = UʺΣʺVʺ T
      Figure imgb0006
    and summing all approximated illumination levels; and
    • Means for adjusting said pixel's light transmission value using said calculated compensation value.
    Object of the Invention
  • The object of the invention is to provide a backlight compensation table approximation method for displays.
  • Another object of the invention is to provide a backlight compensation method and system for compensation of backlight non-uniformity on a display.
  • Another object of the invention is to provide a backlight compensation method and system which reduces the required memory for a compensation table.
  • Another object of the invention is to provide a backlight compensation method and system which enables utilization of local dimming in edge-lit displays.
  • Brief Description of the Drawings
    • Figure 1 shows horizontal light intensity distribution of an edge-lit display comprising a single backlight source with respect to horizontal position.
    • Figure 2 shows vertical light intensity distribution of an edge-lit display comprising a single backlight source with respect to vertical position.
    • Figure 3 shows mean square error of method of present invention with respect to a rank parameter of the present invention
    Detailed Description of the Invention
  • Edge-lit displays suffer from backlight intensity non-uniformity, which degrades perceived image quality on the display. To compensate this non-uniformity, the solution of the state of the art is to compensate non-uniformity by adjusting display thin film layer by utilizing a compensation table.
  • However for a full high definition display, having 1920 columns and 1080 rows of pixels with 16 bit resolution, a single backlight source requires a compensation table consuming approximately 4 megabytes of memory, which is large.
  • The aim of the method and system of present invention is to reduce the memory required by approximation of the compensation table by singular value decomposition and a mean square error (MSE) criterion.
  • A matrix M, having a size of m x n can be represented using singular value decomposition as shown in the equation (1). M = UΣV *
    Figure imgb0007
  • In the equation (1), U is an m x m unitary matrix, ∑ is an m x n diagonal matrix with nonnegative real numbers on the diagonal and V* denotes the conjugate transpose of V, which is an n x n unitary matrix. For the matrix of concern, which is the compensation table, V can be assumed real and hence conjugate can be replaced by transpose.
  • The equation (1) can be decomposed into two equations (2, 3) to realize the point spread function. =
    Figure imgb0008
    M = UʹV T
    Figure imgb0009
  • The method of the present invention utilizes an approximate model matrix M' for matrix M using an Rth order approximation. The approximation is based on minimizing the frobenius norm of the difference between the matrix M and M'.
  • The approximation matrix M' is decomposed as in equation (4). = UΣʹV T
    Figure imgb0010
  • Here ∑' is obtained by leaving the largest R values of the matrix ∑ and setting all other values to zero. This reduces the size of matrices which are composed to determine M'.
  • Since all but R values of ∑' are zero, M' is expressed with a new decomposition equation (5). = UʺΣʺVʺ T
    Figure imgb0011
  • In the equation (5), matrices U" V"T are re-defined since size of ∑' is reduced. For the equation (5), U" becomes an m x R matrix reduced from U such that columns of U" are left singular vectors of U. Likewise, V"T is reduced from VT to an n x R matrix such that rows of V"T are right singular vectors of VT. Since ∑' is obtained by leaving the largest R values of the matrix ∑ and setting all other values to zero, ∑" is defined as an R x R matrix diagonals of which are singular. The resultant equation (5) compose the matrix M' having equal size to the M which is m x n. However, since ∑ matrix is reduced and U and VT are reduced to conform to the equations, the values of M are approximated up to an error. In other words, an Rth order approximation is done by forming M' matrix.
  • The stated approximation method can be successfully used to determine the compensation table using singular value decomposition. Since an m x n table is reduced to multiplication of three matrices having sizes of m x R, R x R and R x n, practically speaking, a full definition display 16 bit compensation table, which is stated to consume 4 megabytes when uncompressed, is reduced to approximately 16 x (1920 + 1080) x 5 = 240000 bits which is approximately 29 Kilobytes. The compression ratio reaches up to 99%. While the table size is reduced on one hand, since the reconstruction of the table requires additional R 16 bit multiplications and R-1 32 bit additions, the computational processing power is required on the other hand. However, reduction of table size becomes very advantageous with almost no drawback since displays employ power microprocessors which are suitable to handle complex mathematical operations in order to use image enhancement algorithms.
  • The utilization of singular value decomposition and an Rth order approximation can be used since the vertical intensity distribution and horizontal intensity distribution of a light source is highly correlated. Figure 1 shows light intensity of a point light source (e.g. light emitting diode - LED) with respect to horizontal position. Figure 2 shows light intensity of the same source with respect to vertical position. The highly correlation characteristic of intensity distribution enables approximation of the compensation table easily.
  • In the application of the method of present invention, it is an important issue to determine rank number R to have an adequate approximation for compensation table. In an embodiment of the invention, the rank number R is determined by comparing approximate and real values using mean square error (MSE) criterion. MSE criterion gives true information of a practical perception error. Figure 3 shows MSE of approximation of present invention with respect to rank number R. As it can be seen on the figure, the MSE of the approximation decreases exponentially with the increasing rank number. Determination of rank number by means of visual observation as well as mathematical results is also important. Application of method and visual observation states similar results with mathematical results and those clearly state that, a rank number 5 is high enough to approximate compensation table adequately and no visual difference can be observed in visual observation.
  • Conclusively, the method of the present invention comprises the steps of;
    • Determining a rank number R;
    • Defining the m x n compensation table using singular value decomposition equation (1) M = UΣV *
      Figure imgb0012
      wherein U denotes an m x m unitary matrix, ∑ is an m x n diagonal matrix with nonnegative real numbers on the diagonal and V* denotes the conjugate transpose of V, which is an n x n unitary matrix;
    • Obtaining a matrix ∑' by leaving largest R values of ∑ and setting all other values to zero.
    • Obtaining a matrix ∑" diagonals of which are singular, by reducing the size of ∑' to R x R
    • Obtaining an m x R matrix U" by reducing U such that columns of U" are left singular vectors of U
    • Obtaining an R x n matrix V" by reducing V such that columns of V" are right singular vectors of V
    • Defining a matrix M' as an approximation of M by the approximation equation (5)
      as; = UʺΣʺVʺ T
      Figure imgb0013
  • The approximation technique of present invention can be used for backlight compensation in a display device comprising at least a backlight source and pixels of m rows and n columns. Said method comprises the steps of;
    • Storing a first matrix U" having size of m x R, a second matrix ∑" having size of R x R and third matrix V"T having size of R x n are stored for at least one backlight source of the display, wherein R denotes an approximation rank number;
    • calculating a compensation value for at least one pixel by using an approximation equation (5); = UʺΣʺVʺ T
      Figure imgb0014
    • adjusting said pixel's light transmission value using said calculated compensation
      value.
  • Present invention further comprises a display having a display resolution m x n, utilizing the method of the invention in determining backlight compensation to obtain uniformity. The display comprises
    • Means for storing a first matrix U" having size of m x R, a second matrix ∑" having size of R x R and a third matrix V"T having size of R x n for at least one backlight source of the display wherein R denotes an approximation rank number;
    • Means for calculating a compensation value for at least one pixel by using an approximation equation (5); = UʺΣʺVʺ T
      Figure imgb0015
    • Means for adjusting a pixel's light transmission value using said calculated compensation value.
  • The display of the present invention utilizes the approximation method of the invention and adjusts pixels of the display by calculating a compensation value for each light source of the display. As a result, the whole display is compensated regarding light intensity non-uniformity. The compensation value may be calculated for each refresh of the pixels of the display. If the display device has adequate temporary memory to store the whole approximate matrix, the calculated compensation values may be stored in the temporary memory to avoid repeated compensation calculations. As long as light intensity parameters are not changed by means of other parameters of the display, for instance a brightness adjustment by the user, it may not be necessary to calculate compensation values repeatedly.
  • Now within the teachings of the present invention, backlight control based enhancements are possible since it is possible to determine uniformity around the whole display. In the state of the art, backlight control based enhancements such as local dimming or boosting backlight to increase contrast and picture dynamics can be only applied in devices with direct-lit displays. Within the known methods in the state of the art, it is almost impossible to utilize local dimming techniques since the locality principle does not apply to edge-lit displays, that is; each light source illuminates a wide area on the display and it is almost impossible to determine a correct illumination for regions of the display and maintain illumination uniformity.
  • However, if the compensation method of the present invention is applied for each backlight source of the display, it is merely a matter of calculation and prediction to use local dimming since the result of a combination of different illumination levels on any pixel can be calculated and a desired local dimming can be determined in accordance with compensation calculations.
  • Within the local dimming method of the present invention, following steps are followed;
    • A first matrix U" having size of m x R, a second matrix ∑" having size of R x R and third matrix V"T having size of R x n are stored for at least one backlight source of the display;
    • An input image is processed to determine local dimming levels for each predefined region of the display;
    • Backlight driving levels of backlight sources are adjusted to dim at least a portion of the display including at least one predefined region that is determined to have local dimming;
    • A compensation value for at least one pixel is calculated by approximating illumination level originating from each backlight source using the equation (5) = UʺΣʺVʺ T
      Figure imgb0016

      and summing all approximated illumination levels;
    • Said pixel's light transmission value is adjusted using said calculated compensation value.
  • Within the techniques in the state of the art, while dimming a light source to obtain local dimming, the rest of the display is also affected since no locality exists. However within the method of the present invention, it is possible to dim a backlight source and compensate degradation of image quality on the rest of the display by increasing the intensities of non-dimmed light sources to balance light intensity and obtain uniformity by calculating combined compensation values using the approximation method of the present invention. The applicability of local dimming using the method of the invention enables utilization of local dimming in not only direct-lit displays but also in edge-lit displays. Furthermore, the method can be used in direct-lit displays to reduce the number of backlight sources but maintain the effectiveness of local dimming.
  • Present invention further comprises a display device comprising a plurality of backlight sources and a plurality of predefined regions on the display comprising;
    • Means for storing a first matrix U" having size of m x R, a second matrix ∑" having size of R x R and third matrix V"T having size of R x n for at least one backlight source of the display wherein R denotes an approximation rank number;
    • Means for processing an input image to determine local dimming levels for each predefined region of the display;
    • Means for adjusting backlight driving levels of backlight sources to dim at least a portion of the display at least including at least one predefined region that is determined to have local dimming;
    • Means for calculating a compensation value for at least one pixel by approximating illumination level originating from each backlight source using the equation (5) = UʺΣʺVʺ T
      Figure imgb0017

      and summing all approximated illumination levels; and
    • Means for adjusting said pixel's light transmission value using said calculated compensation value.
  • In the direct-lit displays wherein the local dimming known from the state of the art is used, another problem, which arises from the fact that each light source illuminates a certain region of the display, that is a locality exists, is possible occurrence of backlight transition discrepancies, since the boundary of each light source is discrete and if a region is boosted while a neighbouring one is dimmed, a huge light intensity difference may occur between these neighbouring regions and the perceived image creates visual disturbance. For this possible problem, further local dimming compensation techniques are used to obtain possible discrepancy regions and correct them by re-adjusting dimming levels.
  • However, since the necessity to obtain a discrete locality for backlights is eliminated and the continuous intensity deviation nature of the light is not disturbed within the method and system of the present invention, a smooth light transition is obtained on the whole display and the possibility of stated discrepancy disappears.
  • Since the approximation technique reduces the required memory size significantly, it is possible to store the matrices used for approximation separately for each backlight source. However, if the backlight sources have equivalent light intensity characteristics, it is possible to use same matrices for approximation of these equivalent backlight sources. Basic matrix operations are useful such as exchanging row indexes and column indexes, moving row indexes and column indexes, reversal of row indexes and/or column indexes, and limitation of row index and/or column index.
  • Present invention enables illumination uniformity and fully illumination control in both direct-lit and edge-lit displays independently from the number and positions of the backlight sources.

Claims (6)

  1. A backlight compensation table approximation method for a display which comprises at least one backlight source and pixels of m rows and n columns and wherein a compensation table of said backlight source with respect to m rows and n columns of pixels as a table M is known, characterized in that said method comprises the steps of;
    - Determining a rank number R;
    - Defining said m x n compensation table using singular value decomposition
    equation (1) M = UΣV *
    Figure imgb0018
    wherein U denotes an m x m unitary matrix, ∑ is an m x n diagonal matrix with nonnegative real numbers on the diagonal and V* denotes the conjugate transpose of V, which is an n x n unitary matrix;
    - Obtaining a matrix ∑' by leaving largest R values of ∑ and setting all other values to zero.
    - Obtaining a matrix ∑" diagonals of which are singular, by reducing the size of
    ∑ to R x R
    - Obtaining an m x R matrix U" by reducing U such that columns of U" are left singular vectors of U
    - Obtaining an R x n matrix V" by reducing V such that columns of V" are right singular vectors of V
    - Defining a matrix M' as an approximation of M by an approximation equation (5); = UʺΣʺVʺ T
    Figure imgb0019
  2. A method according to claim 1 characterized in that said determining a rank number R comprises comparing an approximate value from M' and a real value from M using mean square error criterion.
  3. A backlight compensation method for a display which comprises at least one backlight source and pixels of m rows and n columns characterized in that the method comprises the steps of;
    - Storing a first matrix U" having size of m x R, a second matrix ∑" having size of R x R and third matrix V"T having size of R x n for at least one backlight source of the display, wherein R denotes an approximation rank number;
    - calculating a compensation value for at least one pixel by using an approximation equation (5); = UʺΣʺVʺ T
    Figure imgb0020
    - adjusting at least one pixel's light transmission value using said calculated compensation value.
  4. A local dimming method for a display which comprises a plurality of backlight sources and pixels of m rows and n columns characterized in that the method comprises the steps of;
    - Storing a first matrix U" having size of m x R, a second matrix ∑" having size of R x R and third matrix V"T having size of R x n for at least one backlight source of the display, wherein R denotes an approximation rank number;
    - Processing an input image to determine local dimming levels for each predefined region of the display;
    - Adjusting backlight driving level of at least one backlight source to dim at least a portion of the display at least including at least one predefined region that is determined to have local dimming;
    - Calculating a compensation value for at least one pixel by approximating illumination level originating from each backlight source using an equation (5) = UʺΣʺVʺ T
    Figure imgb0021

    and summing all approximated illumination levels;
    - Adjusting said pixel's light transmission value using said calculated compensation value.
  5. A display which comprises at least one backlight source and pixels of m rows and n columns, characterized in that it further comprises;
    - Means for storing a first matrix U" having size of m x R a second matrix ∑" having size of R x R and a third matrix V"T having size of R x n for at least one backlight source of the display wherein R denotes an approximation rank number;
    - Means for calculating a compensation value by using an approximation equation (5); - = UʺΣʺVʺ T
    Figure imgb0022

    and
    - Means for adjusting at least one pixel's light transmission value using said calculated compensation value.
  6. A display which comprises at least one backlight source and pixels of m rows and n columns, characterized in that it further comprises;
    - Means for storing a first matrix U" having size of m x R, a second matrix ∑" having size of R x R and third matrix V"T having size of R x n for at least one backlight source of the display wherein R denotes an approximation rank number;
    - Means for processing input image to determine local dimming levels for each predefined region of the display;
    - Means for adjusting backlight driving levels of backlight sources to dim at least a portion of the display at least including at least one predefined region that is determined to have local dimming;
    - Means for calculating a compensation value for at least one pixel by approximating illumination level originating from each backlight source using the equation (5) = UʺΣʺVʺ T
    Figure imgb0023

    and summing all approximated illumination levels; and
    - Means for adjusting said pixel's light transmission value using said calculated compensation value.
EP12151013.5A 2011-01-14 2012-01-13 Backlight compensation table approximation for displays Active EP2477176B1 (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
TR2011/00389A TR201100389A2 (en) 2011-01-14 2011-01-14 Backlight compensation table estimation for displays

Publications (2)

Publication Number Publication Date
EP2477176A1 true EP2477176A1 (en) 2012-07-18
EP2477176B1 EP2477176B1 (en) 2015-06-03

Family

ID=45557860

Family Applications (1)

Application Number Title Priority Date Filing Date
EP12151013.5A Active EP2477176B1 (en) 2011-01-14 2012-01-13 Backlight compensation table approximation for displays

Country Status (3)

Country Link
EP (1) EP2477176B1 (en)
ES (1) ES2546520T3 (en)
TR (1) TR201100389A2 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103871373A (en) * 2014-04-01 2014-06-18 合肥工业大学 Local dynamic dimming method used for edge-type LED backlight liquid crystal display
CN110599965A (en) * 2019-08-09 2019-12-20 深圳市美芒科技有限公司 Image display method, image display device, terminal device, and readable storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007027539A1 (en) 2005-08-27 2007-03-08 3M Innovative Properties Company Edge-lit backlight having light recycling cavity with concave transflector
EP2154674A1 (en) * 2008-08-14 2010-02-17 Samsung Electronics Co., Ltd. Display apparatus and brightness correcting method
US7717602B2 (en) 2004-12-13 2010-05-18 Lg Display Co., Ltd. External electrode fluorescent lamp, method of fabricating the same and liquid crystal display device having the same
WO2010096269A1 (en) 2009-02-19 2010-08-26 Zoran Corporation Systems and methods for display device backlight compensation

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7717602B2 (en) 2004-12-13 2010-05-18 Lg Display Co., Ltd. External electrode fluorescent lamp, method of fabricating the same and liquid crystal display device having the same
WO2007027539A1 (en) 2005-08-27 2007-03-08 3M Innovative Properties Company Edge-lit backlight having light recycling cavity with concave transflector
EP2154674A1 (en) * 2008-08-14 2010-02-17 Samsung Electronics Co., Ltd. Display apparatus and brightness correcting method
WO2010096269A1 (en) 2009-02-19 2010-08-26 Zoran Corporation Systems and methods for display device backlight compensation

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
"Singular value decomposition", 10 January 2011 (2011-01-10), XP002671786, Retrieved from the Internet <URL:http://en.wikipedia.org/w/index.php?title=Singular_value_decomposition&oldid=407097783> [retrieved on 20120319] *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103871373A (en) * 2014-04-01 2014-06-18 合肥工业大学 Local dynamic dimming method used for edge-type LED backlight liquid crystal display
CN103871373B (en) * 2014-04-01 2015-12-09 合肥工业大学 For the regional dynamics light-dimming method of side entering-type LED backlight liquid crystal display
CN110599965A (en) * 2019-08-09 2019-12-20 深圳市美芒科技有限公司 Image display method, image display device, terminal device, and readable storage medium

Also Published As

Publication number Publication date
TR201100389A2 (en) 2012-08-22
EP2477176B1 (en) 2015-06-03
ES2546520T3 (en) 2015-09-24

Similar Documents

Publication Publication Date Title
US9489725B2 (en) Method and device for expanding a dynamic range of display device
CN109064979B (en) Image display processing method and device, display device and storage medium
US10571762B2 (en) High dynamic range displays using filterless LCD(s) for increasing contrast and resolution
US9135864B2 (en) Systems and methods for accurately representing high contrast imagery on high dynamic range display systems
US9418600B2 (en) Apparatus for controlling a display and method thereof
US8400392B2 (en) Apparatus and method for controlling backlight and liquid crystal display
US9595229B2 (en) Local dimming method and liquid crystal display
CN102165513B (en) Systems and methods for applying adaptive gamma in image processing for high brightness and high dynamic range displays
CN102314844B (en) Liquid crystal display device and method for local dimming driving of the same
CN109887468B (en) Liquid crystal display dynamic dimming method based on image local features
US20120224121A1 (en) High Dynamic Range Displays Using Filterless LCD(s) For Increasing Contrast And Resolution
EP2126787B1 (en) Multiple modulator displays and related methods
US20140049571A1 (en) Dual LCD Display with Color Correction to Compensate for Varying Achromatic LCD Panel Drive Conditions
CN101286300A (en) Display apparatus and method for adjusting brightness thereof
JP2010049125A (en) Image display apparatus
US20110115829A1 (en) Image display apparatus and control apparatus thereof
JP5404409B2 (en) Liquid crystal display system and method
US9047828B2 (en) Liquid crystal display device including signal controllers for driving panel areas and method for driving thereof
EP2477176B1 (en) Backlight compensation table approximation for displays
US8564528B1 (en) LCD image compensation for LED backlighting
US20120056905A1 (en) Image display apparatus and information processing apparatus
KR20160092125A (en) Display apparatus
CN114420059B (en) Backlight control method and device and display equipment
US9001021B2 (en) Display device and driving method thereof
KR101779294B1 (en) Apparatus and method for boosting a backlight based on image characteristics

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

17P Request for examination filed

Effective date: 20130107

17Q First examination report despatched

Effective date: 20140721

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20150211

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 730273

Country of ref document: AT

Kind code of ref document: T

Effective date: 20150715

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602012007661

Country of ref document: DE

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2546520

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20150924

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 730273

Country of ref document: AT

Kind code of ref document: T

Effective date: 20150603

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 5

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150603

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150603

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150903

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150603

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20150603

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150904

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150603

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150603

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150903

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150603

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150603

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150603

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150603

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150603

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151006

Ref country code: RO

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20150603

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151003

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602012007661

Country of ref document: DE

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150603

26N No opposition filed

Effective date: 20160304

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160131

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150603

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160113

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150603

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160131

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160131

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 6

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150603

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160113

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150603

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150603

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150603

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 7

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20120113

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150603

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150603

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150603

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160131

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150603

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20210122

Year of fee payment: 10

Ref country code: IT

Payment date: 20210121

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20210120

Year of fee payment: 10

Ref country code: ES

Payment date: 20210326

Year of fee payment: 10

Ref country code: GB

Payment date: 20210121

Year of fee payment: 10

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: TR

Payment date: 20220107

Year of fee payment: 11

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602012007661

Country of ref document: DE

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20220113

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220113

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220802

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220131

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220113

REG Reference to a national code

Ref country code: ES

Ref legal event code: FD2A

Effective date: 20230331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220114