JP5474264B2 - Display device - Google Patents

Description

  The present invention relates to a display device in which an illuminance sensor is arranged in a display area of a display.

  In recent years, techniques for arranging an illuminance sensor in a screen display area (pixel arrangement area) of a display have been proposed. For example, Patent Document 1 discloses a circuit in which a photosensor is arranged in a pixel circuit to electrically reduce noise. Patent Document 2 discloses an area sensor in which a photosensor and a light emitting pixel are arranged in close contact. Patent Document 3 discloses a technique for forming an illuminance sensor in a pixel area.

In these patent documents 1 to 3, the illuminance sensor is arranged in a liquid crystal pixel (on the TFT substrate) array. Thus, a sensor can be formed simultaneously with the TFT formation, and sensor signal wiring can be performed together with image signal wiring to each pixel.
For example, Patent Document 4 discloses a touch panel display device in which a photosensor is provided for each pixel as a technique for using an illuminance sensor in a pixel area for a touch panel.

International Publication No. 2010/038512 JP 2001-292276 A JP 2006-251806 A JP 2010-104889 A

  However, in the conventional techniques as shown in Patent Documents 1 to 4, when measuring incident light from the outside of the display using these techniques, internal light (light from pixels around the sensor in the display) is also emitted from the display itself. If the sensor enters the sensor at the same time and the sensor sensitivity is increased, there is a problem that the measurement result varies depending on the displayed contents. Therefore, when the display screen is turned on, it is impossible to measure a very fine change in incident light like an illuminance sensor installed in a dark place that is not in the screen.

  The present invention has been made to solve the above-described problems. In the configuration in which the illuminance sensor is arranged in the display area of the display, the external light reaching the display is considered in consideration of internal light due to light emission of the display itself. An object of the present invention is to provide a display device capable of accurately measuring the amount of light only from the light source.

  In order to achieve the above object, according to the present invention, in a display device having a display area, a pixel and an illuminance sensor arranged in the display area, and a frame for storing one frame of a video signal displayed on the screen in the display area A buffer, a table that stores in advance the internal light emission illuminance by the pixel itself in association with the pixel in the display area, and the illuminance of only ambient light around the illuminance sensor from the illuminance value detected by the illuminance sensor A sensor output correction calculation unit for calculating a value, the sensor output correction calculation unit reads a pixel address and a pixel value from a frame stored in the frame buffer, and reads the read pixel address and pixel value from the table Is obtained, the illuminance value detected by the illuminance sensor is obtained, and the obtained illuminance is obtained. From by subtracting the internal luminous intensity obtained from the table, and calculates the illuminance value of the external light only near the illuminance sensor.

  According to the display device of the present invention, in the configuration in which the illuminance sensor is arranged in the display area of the display, only the external light reaching the display is accurately measured in consideration of the internal light due to the light emission of the display itself. Can do.

BRIEF DESCRIPTION OF THE DRAWINGS It is a whole block diagram of the display apparatus in Embodiment 1 of this invention. It is a block diagram explaining the internal process in an arithmetic processing part. It is explanatory drawing which shows a display frame and the timing of each process. It is a figure which shows an example of arrangement | positioning of the pixel P and the illumination intensity sensor S in a display area. It is a flowchart which shows the processing operation in an arithmetic processing part.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
Embodiment 1 FIG.
FIG. 1 is an overall configuration diagram of a display device according to Embodiment 1 of the present invention. The display device 1 is a liquid crystal display in which an illuminance sensor is arranged in a display area (pixel arrangement area), and an LCD (Liquid Crystal Display) as in a normal display device having no illuminance sensor inside. A liquid crystal display) module 10 and a display substrate 20 are included.

In the LCD module 10, an illuminance sensor S (not shown in FIG. 1, see FIG. 4) is disposed in the display area 11, and the built-in sensor driver 12 is controlled by the sensor controller 13. 11 ambient illuminances can be acquired. The method for forming the illuminance sensor in the display area is a known technique as disclosed in, for example, Patent Documents 1 and 2, and the description thereof is omitted here.
The ambient illuminance acquired from the illuminance sensor S by controlling the sensor driver 12 is converted into a digital value by a sensor ADC (Analog Digital Converter) 14 and connected to the display substrate 20 as a sensor output result. At this time, the dedicated line of the sensor output result is wired to the display substrate 20 together with a video signal connection line (in many cases, FPC (Flexible Printed Circuits)) similar to that of a normal liquid crystal module. Therefore, the connection between the display substrate 20 and the LCD module 10 includes a dedicated line for the sensor output result in addition to the normal video signal line.

  On the display substrate 20, an LCD controller 21 that transmits a video signal and a synchronization signal to the LCD module 10, an arithmetic processing unit 22 that performs sensor output processing such as various video processes and calculations in the previous stage of the LCD controller 21, and its arithmetic operation A video receiver 23 for sending a video signal to the processing unit 22 and a microcomputer 24 are included. Note that the LCD controller 21, the arithmetic processing unit 22, and the video receiver 23 may each be configured by individual ICs, or the LCD controller 21 and the arithmetic processing unit 22 may be configured as one IC. All may be processed by one IC (ASIC: Application Specific Integrated Circuit). Further, the microcomputer 24 may control ICs of various video lines. A broken line shown in FIG. 1 indicates a control signal line from the microcomputer 24.

The video source supplied to the video receiver 23 is supplied from the outside of the display substrate 20, and various digital / analog formats can be considered. Further, as an external device that supplies a video source, a signal input from a tuner, an external storage device, or another video device can be considered.
The video receiver 23 is adapted to the format of the supplied video source, is arranged on the display substrate 20, and performs conversion into a video format suitable for video processing on the display substrate 20. The converted video / control signal is sent to the arithmetic processing unit 22 and the LCD controller 21 on the display substrate 20 and displayed on the LCD module 10. Note that video conversion and various types of video processing can be processed by software using an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), or the like.

Next, processing of output from the video receiver 23 and sensor output from the illuminance sensor S will be described. FIG. 2 is a block diagram for explaining internal processing in the arithmetic processing unit 22.
As shown in FIG. 2, the arithmetic processing unit 22 internally includes a video processing unit 31, a frame buffer 32, a sensor output correction arithmetic unit 33, and an LUT (Look Up Table) 34. The frame buffer 32 and the LUT 34 may be provided in an external storage device such as a DRAM (Dynamic Random Access Memory) or a flash memory. fast.

  The video signal converted by the video receiver 23 is input to the video processing unit 31. Generally, RGB 8-bit video and vertical / horizontal synchronization signals are input. In some cases, video and synchronization signals are transmitted using a high-speed serial bus with a reduced number of signals. The video processing unit 31 corrects the input video into a form preferable for the display device. Gamma correction, contrast adjustment, chromaticity adjustment, response speed correction, etc. are common. Note that this correction function may be incorporated into the video receiver 23 or the LCD controller 21.

  The input video from the video receiver 23 is transmitted to the video processing unit 31 and also transmitted to the frame buffer 32 for storage. The frame buffer 32 stores a maximum of one frame of video. The size to be stored varies depending on the sensor arrangement and the range of sensor correction, but is a maximum of one frame. The reason why the image for one frame is stored in the frame buffer 32 in this way is to use it for the correction calculation. That is, since the acquired value of the illuminance sensor S is a value acquired when displaying a frame that is one frame past the video signal from the video receiver 23, the same frame is subtracted during the correction calculation. For this purpose, the past frames are stored for one frame.

The output value of the illuminance sensor S is input to the sensor output correction calculation unit 33. Further, the video signal from the video receiver 23 stored in the frame buffer 32 is also input to the sensor output correction calculation unit 33. The video signal from the video receiver 23 input here is a frame (fn) one frame before the frame (fn + 1) input to the video processing unit 31.
The sensor output correction calculation unit 33 subtracts the internal light emission illuminance corresponding to the amount of light emitted from the pixels of the liquid crystal display (display area) itself from the raw output value from the illuminance sensor S (input value to the calculation unit). Since this subtraction is performed using the LUT 34 created by prior measurement, the read value from the LUT 34 is also input to the sensor output correction calculation unit 33. The correction calculation and creation of the LUT 34 will be described later.

  The output value after the correction calculation is output from the sensor output correction calculation unit 33 to the microcomputer 24. The output value after the correction calculation is an illuminance value of only light from the outside of the display, which eliminates the influence of whatever image the display displays. By using the output value after this correction calculation for various purposes (backlight brightness adjustment, day / night determination, outdoor / indoor determination, object proximity determination such as finger), the output value from the illuminance sensor in the display area is used as it is. Unlike the conventional method using the value of, the illuminance of only the outside light when the image is continuously displayed can be accurately acquired.

Next, input of an image delayed by one frame to the sensor output correction calculation unit 33 using the frame buffer 32 will be described with reference to FIG. FIG. 3 is an explanatory diagram showing the display frame and the timing of each process.
In FIG. 3, fn means the nth frame, and fn + 1 means the nth next (n + 1) th frame. The horizontal axis represents time.

  The display frame valid period indicates, for example, what number of images are currently displayed when 60 moving images are included per second. This figure shows a state in which the nth fn is displayed at the first timing, the (n + 1) th fn + 1 is displayed at the next timing, and the (n + 2) th fn + 2 is displayed at the next timing. Here, when the signal level is HIGH, it indicates that the one image is being displayed, and the LOW time (blanking period during which no image is displayed) for several mm seconds before moving to the next image. I understand that there is.

  When the display frame is nth, the nth image being displayed is also stored in the frame buffer 32. Therefore, the previous display frame fn−1 and the currently displayed display frame fn are mixed in the frame buffer 32. When the transmission of fn is completed and the blanking period (period in which no video is displayed) is entered, the frame buffer 32 completely stores only fn. In the display area, fn is displayed. At this moment (in the drawing, the output value of the illuminance sensor S in the display area is read out (at the timing of the position described as “illuminance readout at the time of fn display”) and delivered to the sensor output correction calculation unit 33.

As a result, the output value of the illuminance sensor S during fn display is input to the sensor output correction calculation unit 33. As shown in FIG. 2, the sensor output correction calculation unit 33 also captures the value of the fn video signal in the frame buffer 32.
The sensor output correction calculation unit 33 performs correction calculation using the output value from the illuminance sensor S during fn display and the fn video value stored in the frame buffer 32. Specific correction calculation will be described later.

  Here, if the correction is not completed within the blanking period, fn + 1 is started to be overwritten in the frame buffer 32, so that the correction calculation that is a process using the video of fn cannot be continued. Such a case can be dealt with by preparing two frame buffers. However, it is not necessary to prepare two frame buffers as long as the processing capability to finish the correction within this blanking period is provided. In the present invention, the output value from the illuminance sensor S is read when the blanking period starts, and the correction calculation is performed and terminated within the blanking period.

Next, correction calculation of the illuminance sensor output value will be described.
FIG. 4 is a diagram illustrating an example of the arrangement of the pixels P and the illuminance sensor S in the display area. A broken line arrow indicates a path (internal light) through which the light of the pixel P in the display area reaches the illuminance sensor S. That is, not only light from the outside of the display (external light) but also internal light from surrounding pixels P reaches the illuminance sensor S. Moreover, this internal light varies depending on the displayed image. An object of the present invention is to accurately measure only the illuminance of external light by removing the influence of internal light that varies depending on the displayed image.

  In the correction calculation, the amount of internal light is subtracted from the output value of the illuminance sensor S (external light + internal light). The internal light amount is calculated based on the pixel values around the illuminance sensor S taken out from the frame buffer 32. The extent to which the peripheral pixels are used may be freely selected according to the calculation / memory resources of the system. The example of FIG. 4 shows a case where up to two pixels are taken into consideration in the vertical and horizontal directions. Further, it can be said that the intensity of the internal light from each pixel P is independently linear. That is, the output value (the illuminance value of the pixel P2-1) detected by the illuminance sensor S when only the pixel address P2-1 is lit with a certain pixel value (usually between 0 and 255) is represented by S (P2 −1), the internal light value when selecting up to two pixels in the upper, lower, left, and right directions as the correction range (pixel consideration range) is a value S (P2−) obtained by adding the illuminance values of all the pixels in the range. 1) + S (P2-2) +... + S (P2-12) + S (P1-1) + S (P1-2) + S (P1-3) + S (P1-4).

On the other hand, in the LUT 34, all the values from S (P1-1) to S (P2-12) measured in advance are stored in the range of pixel values 0 to 255 for each of RGB.
The sensor output correction calculation unit 33 uses each pixel (pixel address) P1-1 to P2-12 and the pixel value (0 to 255) taken out from the frame buffer 32 as an index, and outputs the sensor output value S (P1-1) from the LUT 34. -S (P2-12) is taken out. Then, all the internal light values for the pixel consideration range optimally determined for the system (upper, lower, left and right in the example of FIG. 4) are added, and the value is used as the internal light component (internal light emission illuminance) in the frame. Thereafter, by subtracting the previously calculated internal light (internal light emission illuminance) from the output value (external light + internal light) of the original illuminance sensor S, the illuminance of only the external light can be obtained. Since this process is performed every frame, the illuminance of pure external light can be obtained regardless of what image is displayed in each frame.

Next, a method for creating the LUT 34 in advance and LUT reference will be described.
First, the surrounding pixels of the illuminance sensor S are sequentially turned on one by one in a dark room where the outside light can be reduced to zero, and the sensor output value at that time is recorded. At this time, a pixel value of 0 to 255 is measured for each pixel. That is, the pixel here indicates each of RGB and is generally called a sub-pixel, but it is measured and recorded one by one regardless of whether the pixel is red, green, or blue. To do. The values measured in this way are arranged in a table and stored in the LUT 34. For example, if the sensor output value when the pixel address P2-1 is lit at the pixel value 128 (others are extinguished) is 50, the LUT 34 will display “pixel address P2-1, pixel value 128, sensor output value (internal light emission). Illuminance) 50 "is stored as one data.
When data is extracted with reference to this LUT 34, the index is the pixel address and the pixel value, and the extracted element is the sensor output value (internal light emission illuminance) at that time. For example, when the sensor output correction calculation unit 33 accesses the LUT 34 at the index of the pixel address P2-1 and the pixel value 128, the LUT 34 returns the sensor output value 50 as the internal light emission illuminance.

  This LUT creation and LUT reference can be simplified using approximate calculation. When the above method is implemented as it is, the LUT 34 has a very large size, and a large amount of references must be performed. Therefore, as a simplification method, for example, it is used that the internal light illuminance is inversely proportional to the distance between the sensor and the pixel. After measuring the illuminance of the innermost pixels P1-1 to P1-4 with respect to the illuminance sensor S, the portion of the pixels outside the illuminance sensor S is the distance that is far from the value of P1-1 to P1-4. It is determined by multiplying by a coefficient. As a result, the LUT size can be reduced and the number of LUT references can be reduced. As another approach, instead of measuring all 256 illuminance values of pixel values 0 to 255 for a pixel at one position, illuminance is measured only for some representative pixel values, and the others are linear It can also be obtained by approximation. For example, with respect to a certain pixel, only three types of illuminances of pixel values 0, 128, and 255 are measured, and the illuminance therebetween can be obtained by linear interpolation. Even when this method is used, the LUT size and the number of LUT references can be reduced.

FIG. 5 is a flowchart showing the processing operation in the arithmetic processing unit 22. Although basically the same as the contents described in FIGS. 2 and 3, a processing flow after the arithmetic processing unit 22 receives a video input will be described with reference to FIG. 5.
First, when receiving a video input from the video receiver 23, the video processing unit 31 performs video processing (step ST1), and the video signal is transmitted to the LCD module 10 via the LCD controller 21 and displayed on the display area 11 as a screen display. (Step ST2). On the other hand, video input signals from the video receiver 23 are sequentially stored in the frame buffer 32 (step ST3). By repeating this while the video input signal continues (in the case of YES in step ST4), screen display and storage in the frame buffer are continued.

  Thereafter, when the video input of one frame is finished and the blanking period starts (in the case of NO in step ST4), the sensor output correction calculation unit 33 uses the illuminance sensor S in the display area 11 displayed on the screen. The output value (illuminance value) is read and acquired (step ST5), and at the same time, the pixel address and pixel value are also read and acquired from the frame stored in the frame buffer 32 (step ST6). Then, referring to the LUT 34, a correction calculation is performed to subtract the internal light component (internal light emission illuminance) from the pixel itself from the output value (illuminance value) from the illuminance sensor S (step ST7), and the calculated illuminance value (external light Minute) is output to the microcomputer 24 or the like (step ST8).

  In this way, by subtracting the illuminance value (internal illuminance) for the internal light emitted by the pixels in the display area from the output value (illuminance value) detected by the illuminance sensor S arranged in the display area. Only the external light around the display area (pixel) can be measured with high accuracy. Thereby, for example, the backlight can be adjusted efficiently. In addition, for example, by applying this display device to a touch panel and embedding a large number of sensors in the display area, the touch position can be accurately determined because the proximity of a finger or the like to the touch panel or the illuminance at the contact point decreases (darkens). It can also be detected.

  As described above, according to the first embodiment, in the configuration in which the illuminance sensor is arranged in the display area of the display, the pixels in the display area are determined from the illuminance value (external light + internal light) detected by the illuminance sensor. Since the internal illuminance of the light emitted by itself is subtracted, only the external light reaching the display can be accurately measured in consideration of the internal light due to the light emitted from the display itself.

  In the present invention, any constituent element of the embodiment can be modified or any constituent element of the embodiment can be omitted within the scope of the invention.

  As described above, the display device according to the present invention can be applied to backlight adjustment and a touch panel.

  DESCRIPTION OF SYMBOLS 1 Display apparatus, 10 LCD module, 11 Display area (pixel arrangement area, sensor arrangement area), 12 Sensor driver, 13 Sensor controller, 14 Sensor ADC, 15 Source driver, 16 Gate driver, 20 Display substrate, 21 LCD controller, 22 Arithmetic processing unit, 23 video receiver, 24 microcomputer, 31 video processing unit, 32 frame buffer, 33 sensor output correction calculation unit, 34 LUT, P1-1 to P1-4, P2-1 to P2-12 pixel, S illuminance sensor .

Claims (2)

In a display device having a display area,
A pixel and an illuminance sensor arranged in the display area;
A frame buffer for storing one frame of a video signal to be displayed on the screen in the display area;
In association with the pixels in the display area, a table that stores in advance the internal light emission illuminance by the pixels themselves;
A sensor output correction calculation unit that calculates an illuminance value of only external light around the illuminance sensor from an illuminance value detected by the illuminance sensor;
The sensor output correction calculator is
The pixel address and the pixel value are read from the frame stored in the frame buffer, and the internal emission illuminance corresponding to the read pixel address and the pixel value is acquired from the table,
Obtain the illuminance value detected by the illuminance sensor,
An illuminance value of only the external light around the illuminance sensor is calculated by subtracting the internal light emission illuminance acquired from the table from the acquired illuminance value. The acquisition of the illuminance value from the illuminance sensor by the sensor output correction calculation unit and the calculation of the illuminance value of only the ambient light around the illuminance sensor are performed during a blanking period in which no video signal is displayed in the display area. The display device according to claim 1.

Images