JP2011048196A - Display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To partially correct luminance on a display screen according to illuminance of external light. <P>SOLUTION: A data display/sensor device 100a includes: a built-in sensor liquid crystal panel 301 where a display/sensor unit, which is composed of one or a plurality of pixels and a light sensor for detecting illuminance of external light, is arranged planarly; a correction unit 615 for a displayed data, which corrects luminance of the pixels according to illuminance of light detected by the light sensor in each of the display/sensor unit; and a driving circuit 304 for a liquid crystal panel, which lights the pixels by using illuminance corrected by luminance correction means. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a display device capable of displaying an image and detecting external light.

  2. Description of the Related Art Conventionally, in a display device having a display screen such as a CRT (cathode ray tube) or a liquid crystal panel, a technique for adjusting luminance and color tone according to external light has been widely used. As technologies relating to adjustment of luminance and color tone in a display device, there are technologies disclosed in Patent Documents 1 to 5.

  Patent Document 1 discloses a CRT display device that controls the brightness of a CRT tube surface according to the brightness of external light. Patent Document 2 discloses an organic electroluminescence device capable of automatically adjusting the screen brightness in accordance with the brightness of external light. Patent Document 3 discloses a color display device that automatically corrects and controls light emission luminance and color temperature of a display.

  Note that Patent Document 4 discloses a technique for improving image quality by improving color reproducibility in a reflection mode in a display device that performs display in a reflection mode using external light.

  Patent Document 5 discloses a technique for outputting a shadow of a subject obtained by imaging with a camera to a display unit.

Japanese Patent Laid-Open No. 3-177888 (publication date: August 1, 1991) JP 2002-297096 A (publication date: October 9, 2002) JP 2002-72992 A (publication date: March 12, 2002) JP 2007-140260 A (publication date: June 7, 2007) JP 63-132278 A (publication date: June 4, 1988)

  However, all of the conventional techniques are techniques for adjusting the overall luminance and color tone of the display screen in the display device depending on the position on the screen in accordance with the illuminance of outside light. Therefore, when the luminance or the like is adjusted according to the illuminance of outside light using the conventional technology, the entire display screen is displayed with uniform brightness.

  However, the illuminance of outside light is not necessarily uniform and may vary depending on the position on the display screen.

  For example, in a large-screen display device installed outdoors, a part of the display screen may become a shadow such as a building. In addition, for example, in an art museum or a museum, when a viewer appreciates a picture displayed on a display screen of a display device, a part of the display screen becomes a shadow of the viewer, or a spotlight is part of the display screen. May be illuminated. In such a case, in the related art, the brightness is adjusted so that the entire display screen is brightened or darkened even though the illuminance of outside light is not uniform.

  In other words, the conventional technique has a problem in that the brightness of only the portion darkened by the shadow reflected on the display screen cannot be lowered because the overall brightness of the display screen is uniformly adjusted. Similarly, there is a problem that it is not possible to increase the brightness of only the portion that is brightened by the light irradiated on the display screen.

  For example, when viewing a picture displayed on the display screen, the shadow of the viewer and the light of the spotlight are not displayed as in the case of viewing an actual picture. Therefore, in this case, there is a problem that it is not possible to provide the viewer with an atmosphere that allows the viewer to appreciate the actual painting.

  Similarly, when a display screen such as a touch panel or PDA (personal digital assistance) is operated with a finger or a pen, when a part of the display screen becomes a shadow such as a finger or a pen, brightness adjustment using conventional technology is performed. Then, the entire display screen becomes dark. That is, unlike the case where a character or the like is written on the paper with a pen or the hardware switch is pressed with a finger, the shadow of the finger or the pen is not displayed. Therefore, in this case, there is a problem that it is impossible to provide the operator with a feeling that a physical object such as paper or a hardware switch is being operated.

  In order to solve the above problem, it is necessary to adjust the luminance and color tone according to the external light for each portion of the display screen. Such techniques are disclosed in Patent Documents 1 to 5 and the like. Absent.

  The present invention has been made in view of the above problems, and an object thereof is to provide a display device that partially corrects the luminance of a display screen in accordance with the illuminance of external light. In particular, an object of the present invention is to provide a display device that lowers the luminance of only a portion of the display screen where the illuminance of external light is low and increases the luminance of only a portion of the illuminance of external light that is large.

  In order to solve the above-described problems, a display device according to the present invention includes a display panel in which a display / sensor unit including one or a plurality of picture elements and an optical sensor that detects the illuminance of external light is arranged in a planar shape. A display device comprising, for each display / sensor unit, luminance correction means for correcting the luminance of a pixel according to the illuminance of light detected by the optical sensor, and luminance corrected by the luminance correction means, The display control means for lighting the picture element is provided.

  According to said structure, a pixel can be lighted by the brightness | luminance corrected according to the illumination intensity of the light which the optical sensor detected for every display / sensor unit.

  Therefore, there is an effect that the luminance of the display panel can be partially corrected according to the illuminance of outside light.

  Furthermore, the display device according to the present invention may be configured such that, when the illuminance of the light detected by the photosensor is smaller than a reference value, the luminance correction unit lowers the luminance of the picture element according to the illuminance.

  According to said structure, when the illumination intensity of external light is smaller than a reference value, a brightness | luminance can be lowered | hung according to illumination intensity and a picture element can be lit.

  Therefore, it is possible to reduce the luminance of a portion of the display panel where the illuminance of outside light is small.

  Furthermore, the display device according to the present invention may be configured such that when the illuminance of the light detected by the photosensor is greater than a reference value, the luminance correction means increases the luminance of the picture element according to the illuminance.

  According to said structure, when the illumination intensity of external light is larger than a reference value, a brightness | luminance can be raised according to illumination intensity and a picture element can be lighted.

  Therefore, the effect that the brightness | luminance of the part with large illuminance of external light can be raised among display panels is produced.

  Furthermore, the display device according to the present invention may be configured such that the luminance correction unit corrects the luminance of the picture element by multiplying a correction factor according to the illuminance of the light detected by the optical sensor.

  According to said structure, the brightness | luminance of a picture element is correct | amended by multiplying the correction factor according to the illumination intensity of external light.

  Therefore, there is an effect that the luminance of the picture element can be corrected appropriately by appropriately determining the correction rate according to the illuminance of the external light. For example, by setting the correction rate to a value that monotonously increases in a broad sense with respect to the illuminance of external light, the luminance of the picture element can be corrected to be lower as the illuminance of external light is lower.

  Furthermore, the display device according to the present invention further includes one or a plurality of illuminance sensors having spectral sensitivity characteristics that are sensitive to visible light, and each of the display / sensor units includes one of the illuminance sensors. The brightness correction means is associated with the display / sensor unit to which the optical sensor belongs and the illuminance of light detected by the illuminance sensor detected by the illuminance sensor associated with the display / sensor unit. An adjustment may be made to reduce an error caused by light other than light, and the luminance of the picture element may be corrected for each display / sensor unit according to the illuminance of the light after the adjustment.

  According to said structure, while using the illumination intensity of the light which the illumination intensity sensor detected, while adjusting the illumination intensity of the light which the optical sensor which belongs to the display / sensor unit matched with the said illumination intensity sensor detected, a display / sensor unit Every time, the picture element can be turned on with the brightness corrected in accordance with the adjusted illuminance.

  Therefore, the adjusted illuminance becomes an illuminance close to the sensitivity (visual sensitivity) of the human eye, so that the luminance of the display panel can be partially corrected according to the illuminance close to the visual sensitivity.

  Therefore, there is an effect that the luminance of the display panel can be corrected more appropriately.

  Further, in the display device according to the present invention, the picture element includes pixels for displaying red, green, and blue, respectively, and further has a spectral sensitivity characteristic that is sensitive to visible light, and has an illuminance of external light. One or a plurality of illuminance sensors that detect each of red, green, and blue colors are provided, each of the display / sensor units is associated with one of the illuminance sensors, and the luminance correction unit includes In accordance with the ratio of the illuminance of each color of the light detected by the illuminance sensor associated with the display / sensor unit to which the optical sensor belongs, the illuminance of the light detected by the optical sensor is converted into the illuminance for each color. The luminance of the pixels of each color included in the picture element may be corrected for each display / sensor unit according to the illuminance for each color after the conversion.

  According to said structure, the light which the optical sensor which belongs to the display / sensor unit matched with the said illuminance sensor detected using the illuminance for every red, green, and blue color of the external light which the illuminance sensor detected Can be turned on for each display / sensor unit with luminance corrected according to the illuminance for each color of the conversion.

  Therefore, since the illuminance of the light detected by the optical sensor can be converted into illuminance for each color, the brightness of the display panel can be partially corrected for each color according to the illuminance for each color component of external light. There is an effect that can be done.

  In order to solve the above-described problem, a display device according to the present invention includes a picture element including pixels for displaying red, green, and blue, and an optical sensor that detects external light of red, green, and blue, respectively. The display / sensor unit comprising a display panel in which the display / sensor unit is arranged in a planar shape, wherein each display / sensor unit has a corresponding color according to the illuminance of each color of light detected by the photosensor. It is characterized by comprising brightness correction means for correcting the brightness of each pixel, and display control means for lighting the pixel with the brightness of each color pixel corrected by the brightness correction means.

  According to the above configuration, for each display / sensor unit, the pixel of the corresponding color is corrected according to the illuminance of the light of each color detected by the optical sensor, and the pixel is corrected with the luminance of the pixel of each corrected color. Can be lit.

  Therefore, the luminance of the display panel can be partially corrected for each color according to the illuminance for each color component of the external light.

  Furthermore, in the display device according to the present invention, when the luminance correction unit detects that the illuminance of the light of each color detected by the photosensor is smaller than a reference value, the luminance of the corresponding color pixel is decreased according to the illuminance. It is good.

  According to said structure, when the illumination intensity for every color component of external light is smaller than a reference value, according to illumination intensity, the brightness | luminance of the pixel of a corresponding color can be lowered | hung and a picture element can be lighted.

  Therefore, there is an effect that the luminance of the corresponding color in the portion of the display panel where the illuminance for each color component of the external light is small can be lowered.

  Furthermore, in the display device according to the present invention, when the illuminance of the light of each color detected by the photosensor is greater than a reference value, the luminance correction unit increases the luminance of the corresponding color pixel according to the illuminance. It is good.

  According to said structure, when the illumination intensity for every color component of external light is larger than a reference value, according to illumination intensity, the brightness | luminance of the pixel of a corresponding color can be raised and a picture element can be lighted.

  Therefore, there is an effect that the luminance of the corresponding color pixel in the portion of the display panel where the illuminance for each color component of the external light is large can be increased.

  Furthermore, the display device according to the present invention has a configuration in which the luminance correction unit corrects the luminance of the corresponding color pixel by multiplying the correction factor according to the illuminance of the light of each color detected by the optical sensor. Also good.

  According to said structure, the brightness | luminance of a pixel is correct | amended by multiplying the correction factor according to the illumination intensity of the light of the corresponding color which the optical sensor detected.

  Therefore, by appropriately determining the correction rate according to the illuminance for each color component, it is possible to appropriately correct the luminance of the corresponding color pixel. For example, by setting the correction rate to a value that monotonously increases in a broad sense with respect to the illuminance for each color component, the luminance of the corresponding color pixel can be corrected to be lower as the illuminance for each color component of the external light is lower. it can.

  Furthermore, the display device according to the present invention further includes one or a plurality of illuminance sensors having spectral sensitivity characteristics that are sensitive to visible light, and each of the display / sensor units includes one of the illuminance sensors. The brightness correction means associated with each other uses the illuminance of light detected by the illuminance sensor associated with the display / sensor unit to which the optical sensor belongs, to determine the illuminance of light of each color detected by the optical sensor. The adjustment is made so as to reduce the error caused by the light other than the visible light, and the luminance of the pixel of the corresponding color is corrected for each display / sensor unit according to the illuminance of the light of each color after the adjustment. It is good also as a structure.

  According to the above configuration, the illuminance of light detected by the illuminance sensor is used to adjust the illuminance of light of each color detected by the optical sensor belonging to the display / sensor unit associated with the illuminance sensor. For each sensor unit, the pixel of the corresponding color is corrected according to the adjusted illuminance, and the picture element can be turned on with the corrected luminance.

  Therefore, since the illuminance after the adjustment is illuminance close to the sensitivity (visual sensitivity) of the human eye, the luminance of the display panel can be partially corrected for each color according to the illuminance close to the visual sensitivity. .

  Therefore, there is an effect that the luminance of the display panel can be corrected more appropriately.

  Furthermore, in the display device according to the present invention, each of the display / sensor units may include one picture element and one optical sensor.

  According to said structure, according to the illumination intensity of the light which one optical sensor detected for every display / sensor unit, the brightness | luminance of one corresponding picture element can be correct | amended.

  Therefore, there is an effect that the luminance can be individually corrected for each picture element.

  As described above, the display device according to the present invention includes a display panel in which a display / sensor unit including one or a plurality of picture elements and an optical sensor that detects the illuminance of external light is arranged in a planar shape. For each display / sensor unit, a luminance correction unit that corrects the luminance of the pixel according to the illuminance of the light detected by the optical sensor, and a luminance that is corrected by the luminance correction unit. Display control means for lighting.

  Therefore, there is an effect that the luminance of the display panel can be partially corrected according to the illuminance of outside light.

  In the display device according to the present invention, the display / sensor unit including the picture element including the pixels for displaying red, green, and blue, and the optical sensor for detecting the red, green, and blue external light is flat. A display device having a display panel arranged in a shape, wherein each display / sensor unit corrects the brightness of a pixel of a corresponding color according to the illuminance of each color of light detected by the photosensor. Correction means, and display control means for lighting the pixels with the brightness of each color pixel corrected by the brightness correction means.

  Therefore, the luminance of the display panel can be partially corrected for each color according to the illuminance for each color component of the external light.

It is a block diagram which shows the detailed structure of the data display / sensor apparatus which concerns on one Embodiment of this invention. FIG. 2A is a schematic diagram showing a state in which a viewer appreciates a picture displayed by a data display / sensor device according to an embodiment of the present invention. FIG. 2B is a schematic diagram showing a state in which the data display / sensor device according to the embodiment of the present invention is operated using a stylus pen. It is a schematic diagram which shows a mode that the spotlight is irradiated toward the image of the picture displayed on the data display / sensor apparatus which concerns on one Embodiment of this invention. It is a figure which shows typically the cross section of the liquid crystal panel with a built-in sensor with which the data display / sensor apparatus which concerns on one Embodiment of this invention is provided. FIG. 5A is a schematic diagram showing a state in which external light is detected by the sensor built-in liquid crystal panel provided in the data display / sensor device according to the embodiment of the present invention. FIG. 5B is a schematic diagram showing a state in which a shadow image is detected by detecting the shadow image with a sensor built-in liquid crystal panel included in the data display / sensor device according to the embodiment of the present invention. It is a block diagram which shows the structure of the liquid crystal panel with a sensor with which the data display / sensor apparatus which concerns on one Embodiment of this invention is provided, and the structure of its peripheral circuit. It is a block diagram which shows the other structure of the liquid crystal panel with a sensor with which the data display / sensor apparatus which concerns on one Embodiment of this invention is provided, and the structure of its peripheral circuit. FIG. 8A is a graph illustrating an example of conversion information stored in the illuminance / luminance correction rate storage unit included in the data display / sensor device according to the embodiment of the present invention. FIG. 8B is a diagram illustrating a graph illustrating another example of the conversion information stored in the illuminance / luminance correction rate storage unit included in the data display / sensor device according to the embodiment of the present invention. It is a figure which shows the structural example of the display / sensor unit with which the liquid crystal panel with a sensor with which the data display / sensor apparatus which concerns on one Embodiment of this invention is equipped is arranged. It is a figure which shows the other structural example of the display / sensor unit arranged in the liquid crystal panel with a built-in sensor with which the data display / sensor apparatus which concerns on one Embodiment of this invention is equipped. 6 is a flowchart showing a flow of processing for correcting the luminance of an image displayed on a sensor-embedded liquid crystal panel in accordance with the illuminance of outside light in the data display / sensor device according to the embodiment of the present invention. 5 is a flowchart showing a flow of processing for correcting the color tone of an image displayed on the sensor-embedded liquid crystal panel in accordance with the illuminances of R, G, and B of outside light in the data display / sensor device according to the embodiment of the present invention. is there. It is a block diagram which shows the detailed structure of the data display / sensor apparatus which concerns on other one Embodiment of this invention. 14 is a flowchart showing a flow of processing for correcting the luminance of an image displayed on the sensor built-in liquid crystal panel in accordance with the illuminance of outside light in the data display / sensor device shown in FIG. 13. It is a block diagram which shows the detailed structure of the data display / sensor apparatus which concerns on another one Embodiment of this invention. FIG. 16A is a graph illustrating an example of conversion information stored in the illuminance / luminance correction value storage unit included in the data display / sensor device illustrated in FIG. 15. FIG. 16B is a diagram illustrating a graph illustrating another example of conversion information stored in the illuminance / luminance correction value storage unit included in the data display / sensor device illustrated in FIG. 15. FIG. 16 is a diagram showing an example of an arrangement of a display / sensor unit and a backlight in a sensor built-in liquid crystal panel included in the data display / sensor device shown in FIG. 15. 16 is a flowchart showing a flow of processing for correcting the luminance of an image displayed on the sensor built-in liquid crystal panel in accordance with the illuminance of outside light in the data display / sensor device shown in FIG. FIG. 19A is a diagram showing a state in which an illuminance sensor is arranged in the data display / sensor device shown in FIG. FIG. 19B is a diagram showing another state in which the illuminance sensor is arranged in the data display / sensor device shown in FIG. It is a block diagram which shows the detailed structure of the data display / sensor apparatus which concerns on another one Embodiment of this invention. It is a circuit diagram which shows the structure of the illumination intensity sensor with which the data display / sensor apparatus shown in FIG. 20 is provided. FIG. 21 is a flowchart showing a flow of processing for correcting the luminance of an image displayed on the sensor-equipped liquid crystal panel in accordance with the illuminance of outside light in the data display / sensor device shown in FIG. 20. It is a block diagram which shows the detailed structure of the data display / sensor apparatus which concerns on another one Embodiment of this invention. It is a circuit diagram which shows the structure of the illumination intensity sensor with which the data display / sensor apparatus shown in FIG. 23 is provided. 24 is a flowchart showing a flow of processing for correcting the luminance of an image displayed on the sensor built-in liquid crystal panel in accordance with the illuminance of external light in the data display / sensor device shown in FIG. It is a figure which shows a mode that the illumination intensity sensor was arrange | positioned to the data display / sensor apparatus shown in FIG. 20 or FIG. It is a schematic diagram which shows a response | compatibility with the light extinction period of a backlight, and the period which detects external light with an optical sensor in the data display / sensor apparatus which concerns on each embodiment of this invention.

[Outline of data display / sensor device]
A data display / sensor device (display device) 100 according to an embodiment of the present invention includes a display / sensor comprising one or more picture elements and a sensor that detects the illuminance of external light (hereinafter referred to as an optical sensor). The display device includes a sensor built-in liquid crystal panel (display panel) 301 (described later) in which sensor units are arranged in a plane.

  The data display / sensor device 100 is used as a general term for data display / sensor devices 100a to 100e described later. In addition, the picture element indicates a dot composed of a pixel that displays red, a pixel that displays green, and a pixel that displays blue.

  Hereinafter, red, green, and blue are abbreviated as R, G, and B, respectively. The display / sensor unit is also referred to as a unit UT. The sensor built-in liquid crystal panel 301 is also simply referred to as a panel 301.

  The feature of the data display / sensor device 100 is that, roughly, the luminance of an image displayed on the panel 301 is partially corrected in accordance with the illuminance of external light detected by the optical sensor. Specifically, for each unit UT, the data display / sensor device 100 corrects the luminance of each picture element belonging to the unit UT according to the illuminance of external light detected by the optical sensor belonging to the unit UT, and The luminance of the image displayed on the panel 301 is corrected by lighting each picture element with the corrected luminance.

  Here, “correct the luminance of the picture element according to the illuminance of the external light detected by the optical sensor” specifically means that the illuminance of the external light detected by the optical sensor is based on a predetermined reference value. When it is small, the correction is made to lower the luminance of the picture element. That is, in the display area of the panel 301, the area where the illuminance of outside light is smaller than the predetermined reference value is displayed dark with the brightness lowered. Here, the reference value may be a preset value, or may be a value calculated as appropriate based on the illuminance of external light.

  In addition, when the illuminance of the external light detected by the optical sensor is larger than the reference value, the luminance of the picture element may be corrected to be increased. That is, of the display area of the panel 301, an area where the illuminance of outside light is larger than a predetermined reference value may be displayed brightly with increased brightness.

  Furthermore, when the illuminance of external light R, G, and B can be detected by the optical sensor, the data display / sensor device 100 can detect the R, G, and external light detected by the optical sensor for each unit UT. The luminances of the R, G, and B pixels included in the picture element are corrected according to the illuminance of B. As a result, the color tone of the image displayed on the panel 301 can be corrected for each part so as to have the color of the external light detected by the optical sensor.

〔Concrete example〕
Next, a specific example will be described in which the data display / sensor device 100 corrects the luminance of the image displayed on the panel 301 for each part in accordance with the illuminance of external light detected by the optical sensor.

(Specific example 1)
In the first specific example, when a part of the panel 301 is a shadow of the object, an area (hereinafter referred to as a shadow area) of the image displayed on the panel 301 is a shadow of the object. It is an example which correct | amends a brightness | luminance. The target object is assumed to be, for example, a person in the vicinity of the data display / sensor device 100, a finger of a user who operates the data display / sensor device 100, or a stylus pen used by the user for the operation. .

  Since the shadow area is dark, the illuminance of external light detected by the optical sensor belonging to the unit UT included in the shadow area is small. When the illuminance of external light detected by the optical sensor is smaller than a predetermined reference value, the data display / sensor device 100 corrects the luminance of the picture elements belonging to the unit UT included in the shadow area to be lowered. . That is, the image displayed in the shadow area is displayed darkly. Thereby, it is possible to make the image displayed on the panel 301 appear as if the shadow of the object is reflected.

  The specific example described above will be described with reference to FIG. FIG. 2A is a schematic diagram showing a state in which the viewer P1 appreciates a painting in an art museum or a museum. It is assumed that when the viewer P1 views a picture image displayed on the panel 301 of the data display / sensor device 100, light is emitted from the back of the viewer P1 toward the panel 301. It is assumed that a part of the display area of the panel 301 is a shadow of the viewer P1 (shadow area SA1 in FIG. 2A). In this case, the data display / sensor device 100 corrects the luminance of the picture element belonging to the unit UT included in the shadow area SA1 that is the shadow of the viewer P1 in the display area of the panel 301, and displays the image darkly. (See FIG. 2 (a)). Thereby, it is possible to make it appear as if the shadow of the viewer P1 is reflected in the painting on the display surface. Therefore, it is possible to provide the viewer P1 with an atmosphere as if he / she is viewing an actual picture. In addition, not only the shadow area SA1 but also the shadow area SA2 that becomes the shadow of the other object P2 is displayed in a dark manner with the brightness lowered.

  Next, FIG. 2B is a schematic diagram showing how an information processing apparatus such as a PDA is operated using the stylus pen P3. It is assumed that light such as an electric light is emitted toward the panel 301 when an input operation or the like is performed on the panel 301 of the data display / sensor device 100 with the stylus pen P3. A part of the display area of the panel 301 is assumed to be a shadow of the stylus pen P3 (shadow area SA3 in FIG. 2B). In this case, the data display / sensor device 100 corrects the luminance of the picture element belonging to the unit UT included in the shadow area SA3 to be a shadow of the stylus pen P3 in the display area of the panel 301, and displays it darkly. (See FIG. 2 (b)). Thereby, it can be made to appear as if the shadow of the stylus pen P3 is reflected on the display surface. Therefore, for example, it is possible to provide the operator with an atmosphere in which characters are written on real paper. That is, it is possible to provide the operator with an operational feeling for the real thing.

(Specific example 2)
In the second specific example, when light such as a spotlight is irradiated on a part of the display area of the panel 301, an area (such as a spotlight or the like irradiated from the image displayed on the panel 301). This is an example of correcting the brightness of the irradiation area. Since the irradiation area is brightly illuminated by the spotlight or the like, the illuminance of outside light detected by the optical sensor belonging to the unit UT included in the irradiation area is increased.

  When the illuminance of the external light detected by the optical sensor becomes larger than a predetermined reference value, the data display / sensor device 100 corrects the luminance of the picture elements belonging to the unit UT included in the irradiation area to be increased. . That is, the image displayed in the irradiation area is displayed brightly. As a result, the spotlight or the like can be seen as reflected in the image displayed on the panel 301.

  Furthermore, when the illuminance of R, G, and B of external light can be detected by the optical sensor, the data display / sensor device 100 determines the luminance of each color pixel included in the picture element belonging to the unit UT included in the irradiation region. Is corrected according to the illuminance of light of each color detected by the optical sensor. By correcting in this way, the color tone of the image displayed in the irradiation area can be corrected. Therefore, for example, when the color of a spotlight or the like is yellowish, among the images displayed on the panel 301, the color tone of the irradiation area is corrected to a color tone that is more yellowish than the color tone of the original image. be able to.

  The specific example described above will be described with reference to FIG. FIG. 3 is a schematic diagram showing a state in which the spotlight SL is irradiated toward an image of a picture displayed on the panel 301 of the data display / sensor device 100 in an art museum or a museum. In this case, the data display / sensor device 100 detects the luminance of the irradiation area LA irradiated with the spotlight SL in the display area of the panel 301 by the optical sensor belonging to the unit UT included in the irradiation area LA. Correction is performed according to the illuminance of R, G, and B of external light. As a result, the image displayed in the irradiation area LA can be displayed in a color tone to which the color of the spotlight SL is added (see FIG. 3). Thereby, it can be made to appear as if the spotlight is reflected on the display surface. Therefore, also in this case, it is possible to provide the viewer with an atmosphere in which a real picture is being appreciated.

[Embodiment 1]
An embodiment of the present invention will be described below with reference to FIGS. 1 and 4 to 12. Below, the outline | summary of the liquid crystal panel 301 with a sensor is demonstrated first. Hereinafter, instead of “detecting the illuminance of light”, it may be expressed as “detecting light”.

(Outline of LCD panel with built-in sensor)
The panel 301 provided in the data display / sensor device 100 is a liquid crystal panel capable of detecting the illuminance of external light in addition to displaying an image. The device used for display is not limited to a liquid crystal panel, and may be an organic EL (Electro Luminescence) panel or the like.

  The structure of the panel 301 will be described with reference to FIG. FIG. 4 is a diagram schematically showing a cross section of the panel 301. Note that the panel 301 described here is an example, and any structure can be used as long as the display surface and the detection surface are shared.

  As illustrated, the panel 301 includes an active matrix substrate 51A disposed on the back surface side and a counter substrate 51B disposed on the front surface side, and has a structure in which a liquid crystal layer 52 is sandwiched between these substrates. . The active matrix substrate 51A is provided with a pixel electrode 56, a data signal line 57, an optical sensor circuit 32 (not shown) (hereinafter also simply referred to as an optical sensor), an alignment film 58, a polarizing plate 59, and the like. The counter substrate 51B is provided with color filters 53r (red), 53g (green), 53b (blue), a light shielding film 54, a counter electrode 55, an alignment film 58, a polarizing plate 59, and the like. A backlight 307 (317) is provided on the back surface of the panel 301.

  The photodiode 36 included in the photosensor circuit 32 is provided in the vicinity of the pixel electrode 56 provided with the blue color filter 53b, but is not limited to this configuration. It may be provided in the vicinity of the pixel electrode 56 provided with the red color filter 53r, or may be provided in the vicinity of the pixel electrode 56 provided with the green color filter 53g. Further, the pixel electrode 56 provided with the red color filter 53r, the pixel electrode 56 provided with the green color filter 53g, and the pixel electrode 56 provided with the blue color filter 53b may be provided.

  Note that the photodiode 36 is assumed to be a photodiode that detects light in a wide wavelength range from outside Shiba light to infrared light, such as a silicon (Si) photodiode.

  Next, how the panel 301 detects external light will be described with reference to FIG. FIG. 5 is a schematic diagram showing how the panel 301 detects external light. As shown in FIG. 5A, the optical sensor circuit 32 including the photodiode 36 can detect the external light 61 transmitted through the counter substrate 51B and the like. In addition, as shown in FIG. 5B, when there is an object 62 such as a pen, the incidence of the external light 61 is hindered, so that the amount of light detected by the optical sensor circuit 32 is reduced. Thereby, a shadow image of the object 62 can be detected.

  Note that the photodiode 36 may detect reflected light (reflected image) in which light emitted from the backlight 307 (317) is reflected by the object. Moreover, it is also possible to detect both a shadow image and a reflected image at the same time by using the above two types of detection methods in combination.

(Data display / sensor configuration)
Next, the configuration of the main part of the data display / sensor device 100a will be described with reference to FIG. FIG. 1 is a block diagram showing a main configuration of the data display / sensor device 100a. As shown in the figure, the data display / sensor device 100a includes a display / light sensor unit 300, a circuit control unit 600a, a main control unit 800, a storage unit 901, a primary storage unit 902, an operation unit 903, an external communication unit 907, An audio output unit 908 and an audio input unit 909 are provided.

  The display / light sensor unit 300 is a so-called liquid crystal display device with a built-in light sensor. The display / light sensor unit 300 includes a panel 301, a backlight 307, and a peripheral circuit 309 for driving them.

  The panel 301 includes a plurality of pixel circuits 31 and photosensor circuits 32 arranged in a matrix. The detailed configuration of the panel 301 will be described later.

  The peripheral circuit 309 includes a liquid crystal panel drive circuit (display control means) 304, an optical sensor drive circuit 305, a signal conversion circuit 306, and a backlight drive circuit 308.

  The liquid crystal panel drive circuit 304 outputs the control signal (G) and the data signal (S) in accordance with the timing control signal (TC1) and the data signal (D) from the display control unit 601 of the circuit control unit 600a, and the pixel circuit 31. It is a circuit which drives. Details of the driving method of the pixel circuit 31 will be described later.

  The optical sensor driving circuit 305 is a circuit that drives the optical sensor circuit 32 by applying a voltage to the signal line (R) in accordance with the timing control signal (TC2) from the sensor control unit 602 of the circuit control unit 600a. Details of the driving method of the optical sensor circuit 32 will be described later.

  The signal conversion circuit 306 converts the sensor output signal (SS) output from the optical sensor circuit 32 into a digital signal (DS), and converts the converted signal into a luminance correction rate determination unit (luminance correction unit) of the circuit control unit 600a. ) A circuit for transmitting to 611a. In the following, “the value of the digital signal DS obtained by converting the voltage value output via the sensor output signal SS according to the illuminance of the light detected by the optical sensor circuit 32 by the signal conversion circuit 306”. Is simply expressed as “illuminance of light detected by the optical sensor”.

  The backlight 307 includes a plurality of white LEDs (Light Emitting Diodes) and is disposed on the back surface of the panel 301. When a power supply voltage is applied from the backlight driving circuit 308, the backlight 307 is turned on and irradiates the panel 301 with light. Note that the backlight 307 is not limited to white LEDs, and may include LEDs of other colors. The backlight 307 may include, for example, a cold cathode fluorescent lamp (CCFL) instead of the LED.

  The backlight drive circuit 308 applies a voltage to the backlight 307 when the control signal (BK) from the backlight control unit 603 of the circuit control unit 600a is at a high level, and conversely, from the backlight control unit 603. When the control signal is at a low level, no voltage is applied to the backlight 307.

  Next, the circuit control unit 600a will be described. The circuit control unit 600 a has a function as a device driver that controls the peripheral circuit 309 of the display / light sensor unit 300. The configuration of the circuit control unit 600a will be described later.

  Next, the main control unit 800 controls the operation of each unit included in the data display / sensor device 100. The main control unit 800 reads out various programs stored in the storage unit 901, controls each unit of the data display / sensor device 100, and realizes various functions provided in the data display / sensor device 100. The main control unit 800 reads the program stored in the storage unit 901 into a primary storage unit 902 configured by, for example, a RAM (Random Access Memory) and executes the program.

  A program executed by the main control unit 800 transmits display data to the circuit control unit 600 a in order to display display data on the panel 301.

  The circuit control unit 600a and the main control unit 800 can be configured by a CPU (Central Processing Unit) and a memory, respectively. Further, the circuit control unit 600a may be configured by a circuit such as an ASIC (application specific integrate circuit).

  Next, as shown in the figure, the storage unit 901 stores programs and data executed by the main control unit 800, and is realized by a nonvolatile memory such as a flash memory. The program executed by the main control unit 800 may be separated into an application-specific program and a general-purpose program that can be shared by each application.

  Next, the operation unit 903 receives an input operation of the user of the data display / sensor device 100. The operation unit 903 includes input devices such as a switch, a remote controller, a mouse, and a keyboard, for example. Then, the operation unit 903 generates a control signal corresponding to the user's input operation of the data display / sensor device 100, and transmits the generated control signal to the main control unit 800.

  As an example of the switch, a hardware switch such as a power switch 905 that switches power on and off and a user switch 906 to which a predetermined function is assigned in advance is assumed.

  In addition, the data display / sensor device 100 includes an external communication unit 907 for communicating with an external device by wireless / wired communication, an audio output unit 908 such as a speaker for outputting audio, and a microphone for inputting an audio signal. A voice input unit 909 such as the above may be provided as appropriate.

(Configuration of sensor built-in liquid crystal panel)
Next, the configuration of the panel 301 and the configuration of the peripheral circuit 309 of the panel 301 will be described with reference to FIG. FIG. 6 is a block diagram showing the main part of the display / light sensor unit 300, particularly the configuration of the panel 301 and the configuration of the peripheral circuit 309.

  The panel 301 includes a pixel circuit 31 for setting light transmittance (brightness) and an optical sensor circuit 32 that outputs a voltage corresponding to the illuminance of light received by the panel 301. The pixel circuit 31 is used as a general term for the R pixel circuit 31r, the G pixel circuit 31g, and the B pixel circuit 31b corresponding to the red, green, and blue color filters, respectively.

  The m pixel circuits 31 are arranged in the column direction (vertical direction) on the panel 301 and 3n in the row direction (horizontal direction). A set of the R pixel circuit 31r, the G pixel circuit 31g, and the B pixel circuit 31b is continuously arranged in the row direction (lateral direction). This set forms one picture element.

  In order to set the light transmittance of the pixel circuit 31, first, the high level voltage (TFT 33 is turned on) to the scanning signal line Gi connected to the gate terminal of the TFT (Thin Film Transistor) 33 included in the pixel circuit 31. Voltage). Thereafter, a predetermined voltage is applied to the data signal line SRj connected to the source terminal of the TFT 33 of the R pixel circuit 31r. Similarly, the light transmittance is also set for the G pixel circuit 31g and the B pixel circuit 31b. Then, an image is displayed on the panel 301 by setting these light transmittances.

  Next, in the configuration shown in FIG. 6, one photosensor circuit 32 is arranged for each picture element. One pixel may be arranged in the vicinity of each of the R pixel circuit 31r, the G pixel circuit 31g, and the B pixel circuit 31b. One may be arranged for each of a plurality of adjacent picture elements.

  In order for the optical sensor circuit 32 to output a voltage corresponding to the illuminance of light, first, the sensor readout line RWi connected to one electrode of the capacitor 35 and the anode terminal of the photodiode 36 are connected. A predetermined voltage is applied to the sensor reset line RSi. In this state, when light is incident on the photodiode 36, a current corresponding to the amount of incident light flows through the photodiode 36. Then, according to the current, the voltage at the connection point (hereinafter referred to as connection node V) between the other electrode of the capacitor 35 and the cathode terminal of the photodiode 36 decreases. When the power supply voltage VDD is applied to the voltage application line SDj connected to the drain terminal of the sensor preamplifier 37, the voltage at the connection node V is amplified and output from the source terminal of the sensor preamplifier 37 to the sensing data output line SPj. The output voltage is a voltage according to the illuminance of the light detected by the optical sensor circuit 32.

  Next, the liquid crystal panel drive circuit 304, the optical sensor drive circuit 305, and the sensor output amplifier 44, which are peripheral circuits of the panel 301, will be described.

  The liquid crystal panel drive circuit 304 is a circuit for driving the pixel circuit 31, and includes a scanning signal line drive circuit 3041 and a data signal line drive circuit 3042.

  The scanning signal line driving circuit 3041 sequentially selects one scanning signal line from the scanning signal lines G1 to Gm for each line time based on the timing control signal TC1 received from the display control unit 601, and A high level voltage is applied to the selected scanning signal line, and a low level voltage is applied to the other scanning signal lines.

  Based on the data signal D (DR, DG, and DB) received from the display control unit 601, the data signal line driver circuit 3042 generates a predetermined voltage corresponding to display data for one row for each line time. The data signal lines SR1 to SRn, SG1 to SGn, and SB1 to SBn are applied (line sequential method). Note that the data signal line driver circuit 3042 may be driven by a dot sequential method.

  The optical sensor driving circuit 305 is a circuit for driving the optical sensor circuit 32. Based on the timing control signal TC2 received from the sensor control unit 602, the optical sensor driving circuit 305 selects a predetermined sensor readout signal line from the sensor readout signal lines RW1 to RWm for each line time. A read voltage is applied, and a voltage other than a predetermined read voltage is applied to the other sensor read signal lines. Similarly, based on the timing control signal TC2, a predetermined reset voltage is applied to the sensor reset signal line selected from the sensor reset signal lines RS1 to RSm for each line time, and the others. A voltage other than a predetermined reset voltage is applied to the sensor reset signal line.

  The sensing data output signal lines SP1 to SPn are connected to the sensor output amplifier 44. The sensor output amplifier 44 amplifies the voltage of the sensing data output signal lines SP1 to SPn and outputs the amplified signal to the signal conversion circuit 306 as a sensor output signal SS (SS1 to SSn).

  As described above, one photosensor circuit 32 may be disposed in the vicinity of each of the R pixel circuit 31r, the G pixel circuit 31g, and the B pixel circuit 31b. With reference to FIG. 7, a configuration of the panel 301 in which one photosensor circuit 32 is arranged in the vicinity of each of the R pixel circuit 31r, the G pixel circuit 31g, and the B pixel circuit 31b of the panel 301 will be described. FIG. 7 is a diagram illustrating a configuration of the panel 301 in which one optical sensor circuit 32 is arranged in the vicinity of each of the R pixel circuit 31r, the G pixel circuit 31g, and the B pixel circuit 31b of the panel 301.

  In this case, the optical sensor circuit 32 disposed in the vicinity of the R pixel circuit 31r detects the external light of R, and the optical sensor circuit 32 disposed in the vicinity of the G pixel circuit 31g transmits the external light of G. The light sensor circuit 32 that detects and detects the B external light is disposed in the vicinity of the B pixel circuit 31b.

  As shown in the figure, the sensing data output signal lines SP1 to SP3n from the optical sensor circuits 32 are output to the signal conversion circuit 306 as sensor output signals SS (SS1 to SS3n) via the sensor output amplifier 44. Is done.

  Therefore, the signal conversion circuit 306 can obtain a digital signal DS representing the illuminance of each of R, G, and B light for each picture element.

(Configuration of circuit controller)
Next, the configuration of the circuit control unit 600a will be described. As shown in FIG. 1, the circuit control unit 600a includes a display control unit 601, a sensor control unit 602, a backlight control unit 603, a display data storage unit 604, a luminance correction rate determination unit 611a, and an illuminance / luminance correction rate storage unit 612. , And a unit information storage unit 614.

  The display control unit 601 receives display data for displaying an image on the panel 301 from the main control unit 800, and sends the display data to the liquid crystal panel drive circuit 304 of the display / light sensor unit 300 based on the received display data. By transmitting the timing control signal TC1 and the data signal D, an image is displayed on the panel 301.

  Note that the display control unit 601 primarily stores display data received from the main control unit 800 in the display data storage unit 604. Then, a data signal D is generated based on the display data that is primarily stored. The display data storage unit 604 is, for example, a video random access memory (VRAM).

  The display control unit 601 further includes a display data correction unit (luminance correction unit) 615. The display data correction unit 615 will be described later.

  Next, the sensor control unit 602 transmits the timing control signal TC2 to the optical sensor driving circuit 305 of the display / optical sensor unit 300 to drive the optical sensor every predetermined period. The predetermined cycle may be any cycle as long as it is a cycle longer than the time required to drive all the optical sensors of the panel 301.

  Further, the sensor control unit 602 instructs the backlight control unit 603 to turn off the backlight 307 while driving the optical sensor.

  The backlight control unit 603 transmits a control signal BK to the backlight drive circuit 308 of the display / light sensor unit 300 to drive the backlight 307.

  Next, the luminance correction rate determination unit 611a first receives the illuminance of light detected by the optical sensor from the signal conversion circuit 306 of the display / optical sensor unit 300.

  Here, in the display area of the panel 301, when the brightness of an area where the illuminance of external light is smaller than the reference value STL is reduced and displayed dark, the brightness correction factor determination unit 611a determines that each of the received illuminances is a reference value. It is determined whether it is smaller than STL. When the received illuminance is smaller than the reference value STL, a luminance correction rate (hereinafter referred to as a correction rate CR) corresponding to the illuminance is determined.

  On the other hand, when the brightness of an area where the illuminance of outside light is larger than the reference value STH is increased in the display area of the panel 301 and the brightness is displayed brightly, the brightness correction factor determination unit 611a determines that each of the received illuminances is the reference value STH. Determine if greater than. When the received illuminance is larger than the reference value STH, a correction rate CR corresponding to the illuminance is determined.

  The data display / sensor device 100a operates to (1) reduce the luminance of the area where the illuminance of outside light is smaller than the reference value STL in the display area of the panel 301, or to display it darkly (2 ) Among the display areas of the panel 301, whether to operate to increase the luminance of an area where the illuminance of external light is larger than the reference value STH and to display brightly may be determined in advance. Either one may be selected as appropriate.

  Further, the reference value STL and the reference value STH may be values appropriately determined according to the characteristics of each element constituting the panel 301, or may be values that can be set by the user. Further, it may be a value calculated as appropriate based on the illuminance of external light. For example, the average value of the illuminance of outside light detected by all the optical sensors may be used.

  The correction rate CR is a value indicating the correction rate of the luminance of the picture element. For example, if the correction rate CR is 50%, the luminance of the picture element is corrected to 50%.

  Note that the luminance correction rate determination unit 611a uses the conversion information (described later) stored in the illuminance / luminance correction rate storage unit 612 to uniquely specify the correction rate CR from the illuminance, according to the illuminance. A correction factor CR is determined.

  Subsequently, after determining the correction rate CR corresponding to each of the illuminances, the luminance correction rate determination unit 611a identifies a pixel for which the luminance is corrected by multiplying the correction rate CR by each of the correction rates CR. To do.

  For this purpose, the luminance correction rate determination unit 611a first identifies which light sensor detects the illuminance of light before conversion into the correction rate CR. That is, the correspondence relationship between the correction rate CR and the optical sensor is specified. Note that each of the illuminances obtained from the signal conversion circuit 306 is the illuminance of the light detected by which optical sensor, based on which optical sensor is driven by the optical sensor driving circuit 305. Can be identified.

  Next, the luminance correction rate determination unit 611a refers to the unit information storage unit 614, and identifies, for each photosensor, a picture element belonging to the unit UT to which the photosensor belongs. That is, the correspondence relationship between the optical sensor and the picture element is specified.

  Then, the luminance correction rate determination unit 611a associates the correction rate CR with the picture element based on the correspondence between the specified correction rate CR and the optical sensor and the correspondence between the specified optical sensor and the pixel. By performing the above, for each correction factor CR, the pixel for which the luminance is to be corrected is specified by multiplying the correction factor CR.

  Next, the illuminance / luminance correction rate storage unit 612 stores the conversion information. The conversion information may be any information as long as the information can be uniquely specified from the illuminance to the correction factor CR. For example, a correspondence table (table) in which the illuminance and the correction rate CR are associated with each other may be used, or a mathematical formula capable of calculating the correction rate CR from the illuminance by a predetermined calculation.

  Note that the conversion information stored in the illuminance / luminance correction rate storage unit 612 may be set in advance when the data display / sensor device 100a is shipped from the factory, or may be registered by the user. In addition, the illuminance / luminance correction rate storage unit 612 is realized by, for example, a nonvolatile memory such as a RAM or a flash memory.

  Here, when the illuminance of light detected by the optical sensor is smaller than the reference value STL, the correction rate CR increases monotonously in a broad sense with respect to the illuminance smaller than the reference value STL. It is assumed to be a value. That is, it is assumed that the smaller the illuminance, the smaller the correction rate CR.

  Note that when the illuminance of the light detected by the optical sensor is larger than the reference value STH, the correction rate CR is a value that monotonously increases in a broad sense with respect to the illuminance greater than the reference value STH. Suppose that That is, it is assumed that the correction rate CR is larger as the illuminance is higher.

  Here, an example of the conversion information stored in the illuminance / luminance correction rate storage unit 612 will be described with reference to FIG. FIG. 8 is a diagram schematically illustrating an example of conversion information stored in the illuminance / luminance correction rate storage unit 612.

  FIG. 8A is an example of conversion information used when correction is performed so that the luminance of the picture element is lowered when the illuminance of light detected by the optical sensor is smaller than the reference value STL. As shown in the figure, the correction rate CR is a value that monotonously increases in a broad sense with respect to the illuminance smaller than the reference value STL of 700 lux (lx), and the correction rate CR can be uniquely specified from the illuminance. it can. In this example, when the illuminance is equal to or higher than the reference value STL, the value of the correction rate CR is always 100%. That is, the luminance is not corrected when the illuminance is equal to or higher than the reference value STL.

  FIG. 8B is an example of conversion information used for correcting to increase the luminance of the picture element when the illuminance of the light detected by the optical sensor is larger than the reference value STH. As shown in the figure, the correction rate CR is a value that monotonously increases in a broad sense for an illuminance greater than the reference value STH of 300 lux, and the correction rate CR can be uniquely identified from the illuminance. In this example, when the illuminance is equal to or less than the reference value STH, the value of the correction rate CR is always 100%. That is, the luminance is not corrected when the illuminance is less than or equal to the reference value STH.

  The unit information storage unit 614 will be described again with reference to FIG. The unit information storage unit 614 stores, for each unit UT, information that can identify a picture element belonging to the unit UT and information that can identify an optical sensor that belongs to the unit UT in association with each other. The information that can identify the picture element and the optical sensor may be any information as long as the information can be identified by the picture element and the optical sensor. For example, the optical sensor and the identifier of the picture element. Instead of the identifier, information (such as coordinates) that can specify the position on the panel 301 may be stored.

  The data structure of the unit information storage unit 614 will be described later. The unit information storage unit 614 is realized by a nonvolatile memory such as a RAM or a flash memory, for example.

  Next, the display data correction unit 615 uses the correction rate CR determined by the luminance correction rate determination unit 611a before temporarily storing the display data received from the main control unit 800 in the display data storage unit 604. Correct the displayed data.

  Specifically, the display data correction unit 615 corrects the luminance of each picture element of the image determined by the luminance correction rate determination unit 611a when the image is displayed on the panel 301 based on the received display data. Correct by multiplying by the rate CR. That is, when the luminance of the pixel before correction is expressed as LB and the luminance of the pixel after correction is expressed as LN, the luminance of each pixel is corrected so that LN = LB × CR.

  The display control unit 601 generates the data signal D based on the corrected display data. As a result, an image in which the luminance of the picture element is corrected is displayed on the panel 301.

(Configuration of display / sensor unit and data structure example of unit information storage unit)
A configuration example of the display / sensor unit arranged on the panel 301 will be described with reference to FIGS. 9 and 10. 9 and 10 are diagrams showing a configuration example of the display / sensor unit arranged on the panel 301. FIG.

  First, the configuration example shown in FIG. 9 will be described. This figure is a view of the panel 301 as viewed from the top, and shows an example of a configuration in which each display / sensor unit (UT11, UT12, UT21, UT22,...) Is composed of four picture elements and one photosensor. . For example, the unit UT11 includes four picture elements indicated by L11, L12, L21, and L22, and one photosensor indicated by S11. Similarly, the unit UT12 includes four picture elements indicated by L13, L14, L23, and L24, and one photosensor indicated by S12. The same applies to the unit UT21 and the unit UT22.

  Next, another configuration example shown in FIG. 10 will be described. This figure is a view of the panel 301 as viewed from the surface. Each display / sensor unit (UT′11, UT′12, UT′21, UT′22,...) Has one picture element and one optical sensor. The example of a structure which consists of is shown. For example, the unit UT′11 includes a picture element indicated by L′ 11 and an optical sensor indicated by S′11. Similarly, the unit UT′12 includes a picture element indicated by L′ 12 and an optical sensor indicated by S′12. The same applies to the unit UT'21 and the unit UT'22.

  Next, an example of the data structure of the unit information storage unit 614 that stores unit information will be described. When the display / sensor unit has a configuration including four picture elements and one optical sensor as shown in FIG. 9, the unit information storage unit 614 has a data structure as shown in Table 1 below. Can do. Table 1 is a table showing another example of the data structure of the unit information storage unit 614.

  In the example shown in Table 1, the unit information storage unit 614 stores a set of optical sensors and picture elements belonging to the unit UT for each unit UT. In the first record, information on the unit UT11 is stored. The unit UT11 is a display / display composed of a photosensor identified by S11 and four picture elements identified by L11, L12, L21, and L22. It is stored that it is a sensor unit.

  Further, when the display / sensor unit has a configuration including one picture element and one photosensor as shown in FIG. 10, the unit information storage unit 614 has a data structure as shown in Table 2 below. can do. Table 2 is a table showing another example of the data structure of the unit information storage unit 614.

  In the example shown in Table 2, information relating to the unit UT′11 is stored in the first record, and is a display composed of a photosensor identified by S′11 and a picture element identified by L′ 11. / It is stored that it is a sensor unit.

(Flow of processing to correct brightness)
Next, the flow of processing for correcting the luminance of the image displayed on the panel 301 in the data display / sensor device 100a will be described with reference to FIG. FIG. 11 is a flowchart showing a flow of processing for correcting the luminance of an image displayed on the panel 301 in the data display / sensor device 100a.

  First, when external light is detected (scanned) by the optical sensor (step S11), the sensor output amplifier 44 amplifies the voltage output from the optical sensor (step S12). Then, the amplified voltage is output to the signal conversion circuit 306, and the signal conversion circuit 306 converts it into a digital signal DS (step S13).

  Next, the luminance correction rate determination unit 611a determines a correction rate CR corresponding to each of the illuminances converted by the signal conversion circuit 306 (step S14). Subsequently, the luminance correction rate determination unit 611a specifies a pixel for which the luminance is corrected by multiplying the correction rate CR by each of the correction rates CR (step S15).

  Thereafter, the display data correction unit 615 determines the luminance for each picture element of the image displayed on the panel 301 based on the display data received from the main control unit 800 as the correction rate for each pixel determined by the luminance correction rate determination unit 611a. The display data is updated so as to be corrected to a value multiplied by CR (step S16). Thereafter, the display control unit 601 displays an image on the panel 301 via the liquid crystal panel drive circuit 304 based on the corrected display data (step S17).

  By repeatedly performing the above processing (steps S11 to S17), the luminance of the image displayed in the changed area can be corrected according to the change in the illuminance of the external light.

(Modification)
As described with reference to FIG. 7, when the illuminance of R, G, and B of outside light can be detected by the optical sensor, R, G, B depending on the detected illuminance of each color of light. Since the tone of the image displayed on the panel 301 can be corrected by correcting the luminance of the pixels, this configuration will be described.

  As described with reference to FIG. 7, sensing data output signal lines SP1 to SP3n from the optical sensor circuit 32 are output to the signal conversion circuit 306 as sensor output signals SS (SS1 to SS3n) via the sensor output amplifier 44. Then, the signal conversion circuit 306 can obtain the digital signals DS corresponding to the illuminances of the R, G, and B lights, respectively.

  In this case, the luminance correction rate determination unit 611a uses the conversion information stored in the illuminance / luminance correction rate storage unit 612 to correct the R correction rate CR according to each of the R, G, and B illuminances. , G correction rate CR and B correction rate CR are determined. Note that one type of conversion information may be stored in the illuminance / brightness correction rate storage unit 612, or different conversion information may be stored for each of R, G, and B.

  Thereafter, the luminance correction rate determination unit 611a identifies the picture element to which the pixel whose luminance is to be corrected belongs by multiplying each of the R, G, and B correction rates CR. Since this specifying method is the same as the method already described, description thereof is omitted here.

  Thereafter, when the display data correction unit 615 displays the image on the panel 301 based on the display data, the luminance correction rate determination unit 611a determines the luminance of each color pixel belonging to each pixel of the image R, Correction is performed by multiplying the correction factors CR for G and B.

  That is, the luminances of the R, G, and B pixels of the pixel before correction are expressed as LBr, LBg, and LBb, respectively, and the luminances of the R, G, and B pixels of the corrected pixel are respectively LNr, LNg , LNb, and R, G, and B correction factors CR are expressed as CRr, CRr, and CRb, respectively, so that LNr = LBr × CRr, LNg = LBg × CRg, and LNb = LBb × CRb. In addition, the luminance of each pixel is corrected.

  In this way, the luminance of the R, G, and B pixels is corrected, and as a result, the color tone of the image can be corrected.

  Next, a flow of processing for correcting the color tone of an image displayed on the panel 301 according to the illuminances of R, G, and B of external light will be described with reference to FIG. FIG. 12 is a flowchart showing a flow of processing for correcting the color tone of an image displayed on the panel 301 in accordance with the illuminances of R, G, and B of outside light.

  First, when external light of R, G, and B is detected (scanned) by the optical sensor (step S21), the sensor output amplifier 44 amplifies the voltage output from the optical sensor (step S22). The amplified sensor output signal SS is output to the signal conversion circuit 306, and the signal conversion circuit 306 converts it into a digital signal DS for each color (step S23).

  Next, the luminance correction rate determination unit 611a determines R, G, and B correction rates CR corresponding to the digital signals DS of the respective colors converted by the signal conversion circuit 306 (step S24). ). Subsequently, the luminance correction rate determination unit 611a specifies the picture element to which the pixel whose luminance is to be corrected belongs by multiplying the correction rate CR for each of the R, G, and B correction rates CR. (Step S25).

  Thereafter, the display data correction unit 615 uses the luminance correction factor determination unit 611a to determine the luminance of each color pixel included in each picture element of the image to be displayed based on the display data received from the main control unit 800. The display data is updated so as to be corrected to a value obtained by multiplying the correction factors CR for G and B (step S26). Thereafter, the display control unit 601 displays an image on the panel 301 via the liquid crystal panel drive circuit 304 based on the corrected display data (step S27).

  By repeating the above processing (steps S21 to S27), the color tone can be corrected by correcting the luminance of the image displayed in the changed area according to the change in the illuminance of the external light. .

(Other configuration of circuit control unit)
In the circuit control unit 600a described above, the luminance of the picture element is corrected by multiplying the correction factor CR. However, the method of correcting the luminance of the picture element is not limited to this, and other methods are possible. You may correct | amend by. A data display / sensor device 100b including a circuit control unit 600b that corrects the luminance after correction of each pixel using a correction palette described later will be described with reference to FIG.

  FIG. 13 is a block diagram showing the configuration of the data display / sensor device 100b. As shown in the figure, the data display / sensor device 100b includes the same members as the data display / sensor device 100a except that the circuit control unit 600a is replaced with a circuit control unit 600b.

  Here, in the panel 301, as described with reference to FIG. 7, one photosensor circuit 32 is disposed in the vicinity of each of the R pixel circuit 31r, the G pixel circuit 31g, and the B pixel circuit 31b. It is assumed that the illuminances of R, G, and B of external light can be detected by the optical sensor.

  The circuit control unit 600b includes a display control unit 601b, a sensor control unit 602, a backlight control unit 603, a display data storage unit 604, a correction palette calculation unit 621, a standard palette storage unit 622, and a unit information storage unit 614. Yes. Note that the sensor control unit 602, the backlight control unit 603, the display data storage unit 604, and the unit information storage unit 614 are as described above, and thus the description thereof is omitted here.

  First, the correction palette calculating unit 621 first converts the illuminances of the R, G, and B light detected by the optical sensor from the signal conversion circuit 306 of the display / optical sensor unit 300 to predetermined gradations (hereinafter referred to as N). (Represented as gradation). Hereinafter, the set of values after the conversion is referred to as “scan data”.

  Note that N is, for example, 256 (that is, 256 gradations), but is not limited to this gradation. Further, when the scan data is expressed as (R0, G0, B0), for example, the value (R0, G0, B0) = (50, 100, 0) can be taken.

  Subsequently, the correction palette calculation unit 621 includes a set of scan data and N gradation values of R, G, and B, which are reference values, stored in advance in the standard palette storage unit 622 (hereinafter referred to as a standard color palette SCP). Are used to calculate a set of R, G, and B values called “correction palette”. The correction palette indicates the degree to which the scan data deviates from the standard color palette SCP that is the reference value.

  At this time, the correction palette calculation unit 621 (1) when the brightness of an area where the illuminance of external light is low is reduced and displayed darkly in the display area of the panel 301 (hereinafter also referred to as dark area enhancement mode), (2) Among the display areas of the panel 301, different processing is performed when the brightness of an area where the illuminance of outside light is large is increased and displayed brightly (hereinafter also referred to as bright area enhancement mode). In which mode the data display / sensor device 100b operates, either one may be determined in advance, or one may be appropriately selected.

  When the data display / sensor device 100b operates in the dark region enhancement mode, the correction palette calculation unit 621 calculates a correction palette by subtracting scan data from the standard color palette SCP. That is, when the standard color palette SCP is represented as (Rs, Gs, Bs) and the correction palette is represented as (Rx, Gx, Bx), (Rx, Gx, Bx) = (Rs−R0, Gs−G0, Bs-B0).

  On the other hand, when the data display / sensor device 100b operates in the bright area enhancement mode, the correction palette calculation unit 621 calculates the correction palette by subtracting the standard color palette SCP from the scan data. That is, (Rx, Gx, Bx) = (R0−Rs, G0−Gs, B0−Bs).

  Since Rx, Gx, and Bx are values of N gradations, they are integer values of 0 or more and N or less. Therefore, when Rx becomes negative as a result of the calculation, the value of Rx is set to a predetermined value. For example, it is set to 0. The same applies to Gx and Bx.

  Subsequently, the correction palette calculation unit 621 specifies, for each of the correction palettes, a picture element whose luminance is to be corrected using the correction palette.

  For this purpose, the correction pallet calculation unit 621 first identifies which optical sensor has detected each of the illuminances before being converted into the scan data from which the correction pallet is derived. That is, the correspondence relationship between the correction pallet and the optical sensor is specified. Note that each of the illuminances obtained from the signal conversion circuit 306 indicates which light sensor has detected the light intensity based on which light sensor is driven by the optical sensor driving circuit 305. be able to.

  Next, the correction palette calculating unit 621 identifies the picture element included in the unit UT to which the optical sensor belongs for each optical sensor by referring to the unit information storage unit 614. That is, the correspondence relationship between the optical sensor and the picture element is specified.

  Then, the correction palette calculation unit 621 associates the correction palette with the picture element based on the correspondence between the specified correction palette and the optical sensor and the correspondence between the specified optical sensor and the pixel. Thus, for each of the correction palettes, the picture element whose luminance is to be corrected is specified using the correction palette.

  Next, the standard palette storage unit 622 stores the standard color palette SCP. The standard color palette SCP may be any value as long as it is a set of R, G, and B values serving as reference values. For example, (Rs, Gs, Bs) = (100, 100, 100), but is not limited to this value. The standard color palette SCP may be set in advance at the time of factory shipment or the like, or may be registered by the user.

  Note that the standard palette storage unit 622 is realized by a nonvolatile memory such as a RAM or a flash memory, for example.

  Next, the display control unit 601b is obtained by replacing the display data correction unit 615 in the display control unit 601 with a display data correction unit (luminance correction unit) 625.

  The display data correction unit 625 performs different processing in the dark region enhancement mode and the bright region enhancement mode. When the data display / sensor device 100b operates in the dark region enhancement mode, the display data correction unit 625 displays the display data image on the panel 301 based on the display data temporarily stored in the display data storage unit 604. The luminance is corrected by subtracting the correction palette calculated by the correction palette calculating unit 621 from the luminance of the R, G, and B pixels of each pixel. That is, the luminance of the R, G, and B pixels of the pixel before correction is expressed as (Rb, Gb, Bb), and the luminance of the R, G, and B pixels of the corrected pixel is (Ry, Gy, By) and the correction palette value (Rx, Gx, Bx), (Ry, Gy, By) = (Rb−Rx, Gb−Gx, Bb−Bx) The luminance of the pixel belonging to the picture element is corrected.

  On the other hand, when the data display / sensor device 100b operates in the bright region enhancement mode, the display data correction unit 625 corrects the luminance of the R, G, and B pixels of each picture element by adding a correction palette. . That is, the luminance of the pixels belonging to each picture element is corrected so that (Ry, Gy, By) = (Rb + Rx, Gb + Gx, Bb + Bx).

  The display control unit 601b generates a data signal D based on the corrected display data. As a result, an image in a state where the luminance is corrected is displayed on the panel 301.

(Explanation using specific values)
Processing for correcting the luminance of each color pixel using the correction palette will be described using specific values.

(Example of correcting the brightness of dark parts to dark)
First, the case where the data display / sensor device 100b operates in the dark region enhancement mode will be described.

  For example, it is assumed that the standard color palette SCP (Rs, Gs, Bs) is set to (100, 100, 100). Then, it is assumed that (50, 100, 10) is obtained as scan data (R0, G0, B0) from the illuminance of external light detected by the optical sensor. The scan data indicates that the illuminance of external light detected by the optical sensor is the largest in green, the second largest in red, and the second largest in blue light. That is, the illuminance of external light detected by the optical sensor is larger in the order of G, R, and B.

  In this case, the correction palette is a value obtained by subtracting the scan data from the standard color palette SCP (50, 0, 90). From this value, it can be seen that the R value of the scan data is 50 smaller than the reference value, the B value is 100 smaller than the reference value, and the G value is 10 smaller than the reference value.

  Next, assuming that the luminance (Rb, Gb, Bb) before correction of R, G, B pixels of the picture element in the unit UT to which the optical sensor belongs is (150, 120, 100). The luminance (Ry, Gy, By) after correcting by subtracting the palette is (100, 120, 10). In this case, when comparing before and after correction, the luminance of the R pixel decreases from 150 to 100, the luminance of the G pixel does not change, and the luminance of the B pixel decreases from 100 to 10. Accordingly, the luminance of the picture element as a whole is smaller than that before the correction, so that the picture element is displayed darker than before the correction.

(Example of correcting the brightness of bright areas brightly)
Next, the case where the data display / sensor device 100b operates in the bright region enhancement mode will be described.

  Also in this case, it is assumed that the standard color palette SCP (Rs, Gs, Bs) is set to (100, 100, 100). Then, it is assumed that (250, 150, 200) is obtained as scan data (R0, G0, B0) from the illuminance of external light detected by the optical sensor. The scan data indicates that the illuminance of the external light detected by the optical sensor is the largest in red, the second largest in blue, and the second largest in green. That is, the illuminance of external light detected by the optical sensor is larger in the order of R, B, and G.

  In this case, the correction palette is (150, 50, 100). From this value, it can be seen that the R, G, and B values of the scan data are 150, 50, and 100 larger than the reference value, respectively.

  Next, assuming that the luminance (Rb, Gb, Bb) before correction of the R, G, B pixels of the picture element in the unit UT to which the optical sensor belongs is (50, 100, 100). The luminance (Ry, Gy, By) after correcting by adding the palette is (200, 150, 200). In this case, when comparing before and after correction, the luminance of the R pixel increases from 50 to 200, the luminance of the G pixel increases from 100 to 150, and the luminance of the B pixel increases from 100 to 200. . Therefore, since the luminance of the picture element as a whole is larger than that before correction, the picture element is displayed brighter than before correction.

(Flow of processing to correct brightness)
Next, the flow of processing for correcting the luminance of the image displayed on the panel 301 in the data display / sensor device 100b will be described with reference to FIG. FIG. 14 is a flowchart showing a flow of processing for correcting the luminance of an image displayed on the panel 301 in the data display / sensor device 100b. Since steps S31 to S33 are the same as steps S21 to S23, description thereof is omitted.

  After step S33, the correction palette calculation unit 621 converts each of the illuminances converted in step S33 into scan data (step S34), and further calculates a correction palette using the standard color palette SCP (step S35). . Then, the correction palette calculation unit 621 specifies, for each of the correction palettes, a picture element whose luminance is to be corrected using the correction palette value (step S36).

  After that, the display data correction unit 625 corrects the correction palette calculation unit 621 calculated the luminance of each color pixel included in the picture element of the image to be displayed based on the display data temporarily stored in the display data storage unit 604. The display data is updated by correcting using the palette (step S37). Thereafter, the display control unit 601b displays an image on the panel 301 via the liquid crystal panel drive circuit 304 based on the corrected display data (step S38).

  By repeatedly performing the above processing (steps S31 to S38), the luminance of the image displayed in the changed area can be corrected according to the change in the illuminance of the external light.

(Modification)
In the above description, the illuminance of R, G, and B external light can be detected by the optical sensor, and the scan data, the standard color palette SCP, and the correction palette are R, G, and B, respectively. It has been described that the brightness of pixels of each color is corrected using a correction palette, assuming that it is a set of values.

  In contrast, a configuration in which the optical sensor does not detect the illuminance of external light for each color (for example, a configuration in which one optical sensor is arranged for each pixel as described with reference to FIG. 6). Even in this case, the luminance of the picture element can be corrected in the same manner as the processing flow described above, and will be described below. However, in this case, the scan data, the standard color palette SCP, and the correction palette are not single sets of R, G, and B values, but single values.

  In this case, the correction palette calculating unit 621 obtains the illuminance of external light detected by the optical sensor from the signal conversion circuit 306 of the display / optical sensor unit 300 for each optical sensor. Then, each of the obtained illuminances is converted into a value of N gradation and used as scan data. Subsequently, a correction palette is calculated based on the scan data and the standard color palette SCP as described above.

  Then, after the correction palette calculation unit 621 specifies a picture element whose luminance is to be corrected using the correction palette, the display data correction unit 625 is based on the display data temporarily stored in the display data storage unit 604. Then, the luminance of each picture element when the display data image is displayed on the panel 301 is corrected in the same manner as described above using the correction palette.

  Finally, the display control unit 601b generates a data signal D based on the corrected display data. As a result, an image in a state where the luminance is corrected is displayed on the panel 301.

  As described above, even if the optical sensor is configured not to detect the illuminance of external light for each color, the luminance of the image displayed in the changed area is corrected according to the change in the illuminance of external light. be able to.

[Embodiment 2]
In the first embodiment, the display data itself is corrected in correcting the luminance of the image displayed on the panel 301. However, the display data is displayed on the panel 301 by adjusting the luminance of light emitted from the backlight. The brightness of the image to be corrected may be corrected. Therefore, in this embodiment, a mode in which the luminance of an image displayed on the panel 301 is corrected by adjusting the luminance of light emitted from the backlight will be described.

  An embodiment of the present invention will be described below with reference to FIGS. For convenience of explanation, members having the same functions as those shown in the first embodiment are denoted by the same reference numerals, and description thereof is omitted. Further, the description of the same processing as that described in Embodiment 1 is omitted.

(Data display / sensor configuration)
With reference to FIG. 15, the configuration of the main part of the data display / sensor device 100c will be described. FIG. 15 is a block diagram showing a main configuration of the data display / sensor device 100c. As shown in the figure, the data display / sensor device 100c includes a display / light sensor unit 310, a circuit control unit 600c, a main control unit 800, a storage unit 901, a primary storage unit 902, an operation unit 903, an external communication unit 907, An audio output unit 908 and an audio input unit 909 are provided.

  The display / light sensor unit 310 is a liquid crystal display device with a built-in light sensor. In the display / light sensor unit 310, the backlight 307 of the display / light sensor unit 300 is replaced with a plurality of backlights 317, and the backlight driving circuit 308 of the display / light sensor unit 300 is replaced with a backlight driving circuit 318. The display / light sensor unit 300 is provided with the same members except for the point replaced with. Note that the liquid crystal panel drive circuit 304, the optical sensor drive circuit 305, the signal conversion circuit 306, and the backlight drive circuit 318 are referred to as a peripheral circuit 319.

  The backlight 317 includes a plurality of white LEDs, and is arranged in a plane on the back surface of the panel 301. Each backlight 317 is turned on when a voltage is applied from the backlight driving circuit 318, and irradiates the panel 301 with light.

  Each unit UT and one of the backlights 317 are associated with each other in advance, and each of the backlights 317 emits light to an area where the associated unit UT is provided. It shall be.

  In addition, each of the backlights 317 can individually control the luminance, and emits light having luminance corresponding to the magnitude of the voltage applied from the backlight driving circuit 318. Specifically, as the applied voltage is smaller, light with reduced brightness is irradiated.

  As described above, each of the backlights 317 can irradiate light having different luminance to the area where the associated unit UT is provided, so that the luminance of the image displayed on the panel 301 can be increased. , Can be partially corrected.

  Next, the backlight drive circuit 318 sets the luminance of the value instructed by the control signal BK to the backlight 317 instructed by the control signal BK from the backlight control unit 603c of the circuit control unit 600c. Apply the appropriate voltage.

  Next, the circuit control unit 600c will be described. The circuit control unit 600 c has a function as a device driver that controls the peripheral circuit 319 of the display / light sensor unit 310. The circuit control unit 600c includes a display control unit 601c, a sensor control unit 602, a backlight control unit 603c, a display data storage unit 604, a luminance correction value determination unit 611c, an illuminance / luminance correction value storage unit 612c, and a unit information storage unit. 616.

  The display control unit 601c has the same function as the display control unit 601 except that the display data correction unit 615 is not provided.

  The backlight control unit 603 c transmits a control signal BK to the backlight drive circuit 318 of the display / light sensor unit 310 to drive each backlight 317. Here, the backlight control unit 603c determines the luminance of the light irradiated to each backlight 317. Specifically, after the luminance correction value determination unit 611c specifies the backlight 317 whose luminance is to be corrected, the backlight control unit 603c determines the luminance of the light irradiated to each backlight 317 as the correction value CV. It is determined to correct to a value obtained by adding (described later). That is, when the luminance before correction is expressed as LB and the luminance after correction is expressed as LN, LN = LB + CV.

  Then, the backlight control unit 603c transmits to the backlight drive circuit 318 a control signal BK indicating the corrected luminance of the light irradiated to each backlight 317.

  Next, the luminance correction value determination unit 611c first receives the illuminance of the light detected by the optical sensor from the signal conversion circuit 306 of the display / light sensor unit 310.

  Here, in the display area of the panel 301, when the brightness of an area where the illuminance of external light is smaller than the reference value STL is reduced and displayed dark, the brightness correction value determination unit 611c determines that each of the received illuminances is a reference value. It is determined whether it is smaller than STL. When the received illuminance is smaller than the reference value STL, a luminance correction value (hereinafter referred to as a correction value CV) corresponding to the illuminance is determined.

  When the brightness of an area where the illuminance of outside light is larger than the reference value STH is increased in the display area of the panel 301 and the brightness is displayed brightly, the brightness correction value determination unit 611c determines that each of the received illuminances is the reference value STH. Determine if greater than. If the received illuminance is greater than the reference value STH, a correction value CV corresponding to the illuminance is determined.

  Note that the luminance correction value determination unit 611c determines a correction value CV corresponding to the illuminance using the conversion information stored in the illuminance / luminance correction value storage unit 612c.

  Then, the luminance correction value determination unit 611c determines a backlight 317 whose luminance is to be corrected based on the correction value CV after determining the correction value CV corresponding to each illuminance.

  For this purpose, the luminance correction value determination unit 611c first identifies which light sensor is the illuminance of the light detected before the conversion to the correction value CV. That is, the correspondence relationship between the correction value CV and the optical sensor is specified. Note that each of the illuminances obtained from the signal conversion circuit 306 is the illuminance of the light detected by which optical sensor, based on which optical sensor is driven by the optical sensor driving circuit 305. Can be identified.

  Next, the luminance correction value determination unit 611c identifies the backlight 317 associated with the unit UT to which the optical sensor belongs for each optical sensor by referring to the unit information storage unit 616. That is, the correspondence relationship between the optical sensor and the backlight 317 is specified.

  Then, the luminance correction value determination unit 611c determines whether the correction value CV and the backlight 317 are based on the correspondence between the specified correction value CV and the optical sensor and the correspondence between the specified optical sensor and the backlight 317. By performing the association, the backlight 317 whose luminance is to be corrected is specified based on each of the correction values CV.

  Next, the illuminance / luminance correction value storage unit 612c stores the conversion information. The conversion information may be any information as long as it can be uniquely specified from the illuminance to the correction value CV. For example, it may be a correspondence table (table) in which the illuminance and the correction value CV are associated with each other, or may be a mathematical formula that can calculate the correction value CV from the illuminance by a predetermined calculation.

  Note that the conversion information stored in the illuminance / luminance correction value storage unit 612c may be set in advance when the data display / sensor device 100c is shipped from the factory, or may be registered by the user. In addition, the illuminance / luminance correction value storage unit 612c is realized by, for example, a nonvolatile memory such as a RAM or a flash memory.

  Here, an example of the conversion information stored in the illuminance / luminance correction value storage unit 612c will be described with reference to FIG. FIG. 16 is a diagram schematically illustrating an example of conversion information stored in the illuminance / luminance correction value storage unit 612c.

  FIG. 16A is an example of conversion information used for correcting to increase the luminance of a picture element when the illuminance of light detected by the optical sensor is smaller than the reference value STL. In the example shown in the figure, the correction value CV is 200 when the illuminance is smaller than the reference value STL, which is 700 lux (lx). In this example, when the illuminance is smaller than the reference value STL, the correction value CV is a constant value. However, the correction value CV may be changed according to the illuminance. Further, when the illuminance is equal to or higher than the reference value STL, the correction value CV is always 0. That is, the luminance is not corrected when the illuminance is equal to or higher than the reference value STL.

  FIG. 16B is an example of conversion information used when correction is performed so that the luminance of the picture element is lowered when the illuminance of the light detected by the optical sensor is greater than the reference value STH. In the example shown in the figure, when the illuminance is larger than the reference value STH, which is 300 lux, the correction value CV is −200. In this example, when the illuminance is larger than the reference value STH in this example, the correction value CV is a constant value. However, the correction value CV may be changed according to the illuminance. Further, when the illuminance is equal to or less than the reference value STH, the correction value CV is always 0. That is, the luminance is not corrected when the illuminance is less than or equal to the reference value STH.

  The unit information storage unit 616 will be described again with reference to FIG. The unit information storage unit 616 stores information that can identify the backlight 317 associated with each unit UT in addition to the information stored in the unit information storage unit 614. Instead of the identification information of the backlight 317, information (coordinates or the like) that can specify the position where the backlight 317 is disposed may be stored.

  The unit information storage unit 616 is realized by, for example, a nonvolatile memory such as a RAM or a flash memory.

  An example of the data structure of the unit information storage unit 616 will be described. When the display / sensor unit has a configuration composed of one picture element and one optical sensor as shown in FIG. 10, the unit information storage unit 616 has a data structure as shown in Table 3 below, for example. be able to. Table 3 is a table showing an example of the data structure of the unit information storage unit 616.

  In the example shown in Table 3, the unit information storage unit 616 stores, for each unit UT, a set of picture elements and photosensors belonging to the unit UT, and a backlight 317 associated with the unit UT. In the first record, information on the unit UT identified by UT′11 is stored, and the unit UT identified by UT′11 is an optical sensor identified by S′11, and L′ 11. It is stored that it is a display / sensor unit composed of picture elements identified by. Further, it is stored that the unit UT identified by UT′11 is associated with the backlight 317 identified by BK1.

  In the example shown in Table 3, the backlight 317 identified by BK1 is associated with the unit UT identified by UT′11, UT′12, UT′21, and UT′22.

(Example of backlight arrangement)
An arrangement example of the unit UT and the backlight 317 will be described with reference to FIG. FIG. 17 is a diagram illustrating an example of the arrangement of the unit UT and the backlight 317.

  Here, an arrangement example of the unit UT and the backlight 317 associated with each other as shown in Table 3 will be described. In this case, as shown in FIG. 17, a backlight 317 identified by BK1 is disposed on the back surface of the unit UT identified by UT′11, UT′12, UT′21, and UT′22, and , UT′13, UT′14, UT′23, and the back of the unit UT identified by UT′24, a backlight 317 identified by BK2 is disposed.

  And the backlight 317 shown by BK1 irradiates the area | region where unit UT identified by UT'11, UT'12, UT'21, and UT'22 is arrange | positioned among the panels 301. To do. Similarly, the backlight 317 indicated by BK2 is provided in an area where the unit UT identified by UT′13, UT′14, UT′23, and UT′24 is disposed in the panel 301. Irradiate light.

(Flow of processing to correct brightness)
Next, the flow of processing for correcting the luminance of the image displayed on the panel 301 in the data display / sensor device 100c will be described with reference to FIG. FIG. 18 is a flowchart showing a flow of processing for correcting the luminance of an image displayed on the panel 301 in the data display / sensor device 100c. Since steps S41 to S43 are the same as steps S11 to S13, the description thereof is omitted.

  After step S43, the luminance correction value determination unit 611c converts each of the illuminances obtained in step S43 into a correction value CV (step S44). Then, the backlight 317 whose luminance is to be corrected is specified based on each of the correction values CV (step S45).

  Then, the backlight control unit 603c determines to correct the luminance of the light applied to each backlight 317 to a value obtained by adding the correction value CV (step S46). Then, the backlight control unit 603 c transmits a control signal BK including the corrected luminance of the light irradiated to each backlight 317 to the backlight driving circuit 318.

  Thereafter, the backlight drive circuit 318 applies a voltage corresponding to the luminance instructed by the control signal BK to the backlight 317 instructed by the control signal BK. And the backlight 317 irradiates the light of the brightness | luminance according to the applied voltage (step S47). Thereby, the brightness of the image displayed on the panel 301 is partially corrected.

  By repeatedly performing the above processing (steps S41 to S47), the luminance of the image displayed in the changed area can be corrected according to the change in the illuminance of the external light.

  In the circuit control unit 600c described above, the luminance of the light irradiated on the backlight 317 is corrected by adding a correction value. However, the method for correcting the luminance of the light irradiated on the backlight 317 is not limited thereto. It is not limited, and you may correct | amend by another method.

[Embodiment 3]
The optical sensor included in the panel 301 detects light in a wide wavelength range from ultraviolet light to infrared light. For this reason, the illuminance of light detected by the optical sensor may differ from the sensitivity (visual sensitivity) of the human eye. Therefore, in this embodiment, in order to obtain an illuminance closer to the visual sensitivity, detection is performed by an illuminance sensor having a spectral sensitivity characteristic that is sensitive to visible light (that is, having a spectral sensitivity characteristic close to the visual sensitivity characteristic). A mode in which the luminance of the picture element is corrected after adjusting the illuminance of the light detected by the optical sensor using the illuminance of the detected light will be described.

  One embodiment of the present invention will be described below with reference to FIGS. For convenience of explanation, members having the same functions as those shown in the first and second embodiments are denoted by the same reference numerals, and description thereof is omitted. The description of the same processing as that described in the first and second embodiments is omitted.

(Outline of illumination sensor placement and illumination adjustment)
First, an example in which an illuminance sensor is arranged in the data display / sensor device 100d according to the present embodiment will be described with reference to FIG. FIG. 19 is a diagram showing a state in which an illuminance sensor is arranged in the data display / sensor device 100d.

  The example shown in FIG. 6A is an example in which one illuminance sensor (VS) is arranged in the vicinity of the panel 301. In this case, the data display / sensor device 100d adjusts the illuminance of light detected by all the optical sensors of the panel 301 using the illuminance of light detected by the illuminance sensor VS.

  On the other hand, the example shown in FIG. 5B is an example in which a plurality of illuminance sensors are arranged in the vicinity of the panel 301. In this example, six illuminance sensors (VS1 to VS6) are arranged. In this case, the data display / sensor device 100d uses the illuminance of light detected by the illuminance sensor to adjust the illuminance of light detected by the optical sensor belonging to the unit UT associated with the illuminance sensor in advance.

  For example, as shown in FIG. 5B, the area in the panel 301 is divided into six areas (PA1 to PA6) that are the same as the number of illuminance sensors. Then, it is assumed that the illuminance sensor VS1 close to the area PA1 and each unit UT belonging to the area PA1 are associated in advance. Similarly, it is assumed that the illuminance sensor VS2 adjacent to the area PA2 and the units UT belonging to the area PA2 are associated in advance. Similarly, the other illuminance sensors VS3 to VS6 are associated with the units UT belonging to the areas PA3 to PA6 in advance.

  In this case, the illuminance of the light detected by the optical sensor belonging to each unit UT associated with the illuminance sensor VS1 is adjusted using the illuminance of the light detected by the illuminance sensor VS1. The same applies to other optical sensors. That is, using the illuminance of light detected by the illuminance sensors VS2 to VS6, the illuminance of light detected by the optical sensors belonging to each unit UT associated with the illuminance sensors VS2 to VS6 is adjusted.

(Configuration of data display / sensor device)
Next, the configuration of the main part of the data display / sensor device 100d will be described with reference to FIG. FIG. 20 is a block diagram showing a main configuration of the data display / sensor device 100d. As shown in the figure, the data display / sensor device 100d includes a display / light sensor unit 300, an illuminance sensor circuit 320 (hereinafter also simply referred to as an illuminance sensor), a signal conversion circuit 325, a circuit control unit 600d, and a main control. Unit 800, storage unit 901, primary storage unit 902, operation unit 903, external communication unit 907, audio output unit 908, and audio input unit 909.

  As will be described later, the illuminance sensor circuit 320 includes a photodiode 322, detects the illuminance of outside light, and outputs a voltage corresponding to the detected illuminance of the light as a sensor output signal SS '. The configuration of the illuminance sensor circuit 320 will be described later.

  Next, the signal conversion circuit 325 is a circuit that converts the sensor output signal SS ′ into a digital signal DS ′ and transmits the converted signal to the luminance correction rate determination unit (luminance correction unit) 611d of the circuit control unit 600d. is there. When there are a plurality of illuminance sensors, the sensor output signal SS 'obtained from each illuminance sensor is converted into a digital signal DS', and the converted signal is transmitted to the luminance correction rate determination unit 611d.

  Hereinafter, “the value of the digital signal DS ′ obtained by converting the voltage value output via the sensor output signal SS ′ according to the illuminance of the light detected by the illuminance sensor circuit 320 by the signal conversion circuit 325”. This is simply expressed as “illuminance of light detected by an illuminance sensor”.

  Next, the circuit control unit 600d replaces the luminance correction rate determination unit 611a of the circuit control unit 600a with the luminance correction rate determination unit 611d, and the unit information storage unit 617 replaces the unit information storage unit 614 of the circuit control unit 600a. Except for the point replaced by, the same members as the circuit control unit 600a are provided.

  First, the luminance correction rate determination unit 611d receives the illuminance (DS) of light detected by the optical sensor from the signal conversion circuit 306 of the display / optical sensor unit 300. On the other hand, the illuminance (DS ′) of light detected by the illuminance sensor is received from the signal conversion circuit 325.

  Then, each of the illuminance (DS) of the light detected by the optical sensor is adjusted using the illuminance (DS ′) of the light detected by the illuminance sensor. In order to perform the adjustment, the luminance correction rate determination unit 611d includes an illuminance adjustment unit 618.

  First, the illuminance adjustment unit 618 refers to the unit information storage unit 617 and identifies, for each illuminance sensor, an optical sensor belonging to the unit UT associated with the illuminance sensor. And it adjusts using the illumination intensity of the light which the illumination sensor detected about each of the illumination intensity of the light which the specified optical sensor detected.

  Here, the adjustment content is not particularly limited. For example, (A) the illuminance of light detected by the optical sensor is expressed as ds, and (B) the adjusted value of ds is expressed as dsc. (C) The illuminance of light detected by the illuminance sensor is expressed as ds', and (D) the average of the illuminance of light detected by each of the optical sensors belonging to the unit UT associated with the illuminance sensor is expressed as dsa. In this case, the illuminance adjustment unit 618 performs calculation according to the following mathematical formula (1) to calculate the value of dsc.

dsc = ds × ds ′ ÷ dsa (1)
Here, the value of (ds ′ ÷ dsa) on the right side of the formula (1) is a value indicating the relative relationship of ds. Therefore, the result of multiplying (ds ′ ÷ dsa) by ds is the illuminance of the optical sensor portion.

  In the image displayed on the panel 301, when the brightness of an area where the illuminance of outside light is smaller than the reference value STL is reduced and displayed dark, the brightness correction rate determination unit 611d has the illuminance adjusted by the illuminance adjustment unit 618. It is determined whether each is smaller than the reference value STL. When the received illuminance is smaller than the reference value STL, the correction factor CR corresponding to the illuminance is determined.

  Note that, in the image displayed on the panel 301, when the brightness of an area where the illuminance of external light is larger than the reference value STH is increased and displayed brightly, the brightness correction rate determination unit 611d has the illuminance adjusted by the illuminance adjustment unit 618. It is determined whether each is greater than the reference value STH. When the received illuminance is greater than the reference value STH, a correction rate CR corresponding to the illuminance is determined.

  Note that the luminance correction rate determination unit 611d determines the correction rate CR according to the illuminance using the conversion information stored in the illuminance / luminance correction rate storage unit 612, similarly to the luminance correction rate determination unit 611a.

  Then, similarly to the luminance correction rate determination unit 611a, the luminance correction rate determination unit 611d determines the correction rate CR corresponding to each illuminance, and then multiplies each correction rate CR by the correction rate CR to obtain the luminance. Specify the pixel to be corrected. Since the specifying method is the same as the method performed by the luminance correction rate determining unit 611a, the description thereof is omitted here. However, by referring to the unit information storage unit 617, the picture element included in the unit UT to which the optical sensor belongs is specified for each optical sensor.

  Next, in addition to the information stored by the unit information storage unit 614, the unit information storage unit 617 further stores information that can identify the illuminance sensor associated with each unit UT.

  An example of the data structure of the unit information storage unit 617 will be described. The unit information storage unit 617 can have a data structure as shown in Table 4 below. Table 4 is a table showing an example of the data structure of the unit information storage unit 617.

  In the example shown in Table 4, the unit information storage unit 617 stores, for each unit UT, a set of picture elements and optical sensors belonging to the unit UT, and an illuminance sensor associated with the unit UT. In the first record, information related to the unit UT11 is stored. The unit UT11 is a unit composed of a photosensor identified by S11 and four picture elements identified by L11, L12, L21, and L22. It is stored that the unit UT11 is associated with the illuminance sensor VS1. In the example shown in Table 4, the illuminance sensor VS1 is associated with the unit UT11 and the unit UT12.

(Configuration of illuminance sensor)
The configuration of the illuminance sensor circuit 320 will be described with reference to FIG. FIG. 21 is a circuit diagram showing a configuration of the illuminance sensor circuit 320. In order for the illuminance sensor circuit 320 to output a voltage corresponding to the illuminance of external light, first, the readout line RLV connected to one electrode of the capacitor 321 and the anode terminal of the photodiode 322 are connected. A predetermined voltage is applied to the reset line RLS. In this state, when light is incident on the photodiode 322, a current corresponding to the amount of incident light flows through the photodiode 322. In response to the current, the illuminance sensor circuit 320 outputs a voltage at a connection point between the other electrode of the capacitor 321 and the cathode terminal of the photodiode 322 (hereinafter, connection node W). A sensor output signal SS ′ is obtained by amplifying the voltage output from the illuminance sensor circuit 320 by the sensor output amplifier 323.

  Note that the photodiode 322 included in the illuminance sensor circuit 320 is, for example, a photo diode that is difficult to detect light in the wavelength region of the outside light such as gallium arsenide (GaAs) photodiode or gallium arsenide phosphorus (GaAsP) photodiode. It is a diode. Therefore, the illuminance sensor circuit 320 has a spectral sensitivity characteristic that is sensitive to visible light (that is, a spectral sensitivity characteristic that is close to the visual sensitivity characteristic).

(Flow of processing to correct brightness)
Next, the flow of processing for correcting the luminance of the image displayed on the panel 301 in the data display / sensor device 100d will be described with reference to FIG. FIG. 22 is a flowchart showing a flow of processing for correcting the luminance of an image displayed on the panel 301 in the data display / sensor device 100d.

  Steps S51 to S53 are the same as steps S11 to S13, respectively, and a description thereof will be omitted.

  On the other hand, when ambient light is detected by the illuminance sensor (step S56), the sensor output amplifier 323 amplifies the voltage output from the illuminance sensor (step S57). Then, the amplified voltage is output to the signal conversion circuit 325, and is converted into a digital signal DS 'by the signal conversion circuit 325 (step S58).

  Next, the illuminance adjustment unit 618 of the luminance correction rate determination unit 611d uses the illuminance represented by the digital signal DS ′ obtained in step S58 as the illuminance represented by the digital signal DS obtained in step S53. It adjusts using (step S61). Then, the luminance correction rate determination unit 611d determines a correction rate CR corresponding to each adjusted illuminance (step S62). Subsequently, for each of the correction factors CR, a pixel for which the luminance is corrected by multiplying the correction factor CR is specified (step S63).

  Thereafter, the display data correction unit 615 multiplies the luminance for each picture element of the image to be displayed based on the display data received from the main control unit 800 by the correction factor CR for each picture element determined by the luminance correction rate determination unit 611a. The display data is updated so that the corrected value is corrected (step S64). Thereafter, the display control unit 601 displays an image on the panel 301 via the liquid crystal panel drive circuit 304 based on the corrected display data (step S65).

  By repeatedly performing the above processing (steps S51 to S65), the luminance of the image displayed in the changed area can be more appropriately corrected according to the change in the illuminance of the external light of the panel 301. .

(Modification 1)
As described with reference to FIG. 7, when the illuminance of R, G, and B of external light can be detected by the optical sensor, the illuminance sensor detects the illuminance of the light detected by the illuminance sensor. The illuminance of R, G, and B of the light detected by the optical sensor belonging to each unit UT can be adjusted. Since the process after the adjustment is the same as described above, the description thereof is omitted here.

(Modification 2)
When the illuminance sensor can detect the illuminance of external light R, G, and B, the data display / sensor device 100d uses the illuminance of the R, G, and B light detected by the illuminance sensor, The illuminance of the light detected by the optical sensor is converted to the illuminance of R, G, B light, and the luminance of the R, G, B pixels is corrected according to the illuminance of the converted R, G, B light. By doing so, the color tone of the image can be corrected, so this configuration will be described below. The illuminance sensor that can detect R, G, and B of external light is also referred to as a color illuminance sensor below.

(Configuration of data display / sensor device)
Next, the configuration of the main part of the data display / sensor device 100e will be described with reference to FIG. FIG. 23 is a block diagram showing a main configuration of the data display / sensor device 100e. As shown in the figure, the data display / sensor device 100e includes a display / light sensor unit 300, an illuminance sensor circuit 330 (hereinafter also simply referred to as a color illuminance sensor), a signal conversion circuit 325e, a circuit control unit 600e, A control unit 800, a storage unit 901, a primary storage unit 902, an operation unit 903, an external communication unit 907, an audio output unit 908, and an audio input unit 909 are provided.

  The illuminance sensor circuit 330 is a color illuminance sensor. As will be described later, each of the color filters 324r (red), 324g (green), and 324b (blue) includes a photodiode 322, and detects the illuminance for each color component of the external light. The configuration of the illuminance sensor circuit 330 will be described later.

  The signal conversion circuit 325e converts the sensor output signal SS ′ obtained from the illuminance sensor circuit 330 into a digital signal DS ′, and sends the converted signal to the luminance correction rate determination unit (luminance correction unit) 611e of the circuit control unit 600e. It is a circuit to transmit. Here, the converted digital signal DS ′ represents the illuminance of each of R, G, and B of the light detected by the color illuminance sensor. That is, the signal conversion circuit 325 transmits the R, G, and B illuminances of the light detected by the color illuminance sensor to the luminance correction rate determination unit 611e. When there are a plurality of color illuminance sensors, the sensor output signal SS 'obtained from each color illuminance sensor is converted into a digital signal DS', and the converted signal is transmitted to the luminance correction rate determination unit 611e.

  Hereinafter, “the value of the digital signal DS ′ obtained by converting the voltage value output via the sensor output signal SS ′ according to the illuminance of the light detected by the illuminance sensor circuit 330 by the signal conversion circuit 325 e”. This is simply expressed as “illuminance of light detected by the color illuminance sensor”.

  Next, the circuit control unit 600e includes the same members as the circuit control unit 600d except that the luminance correction rate determination unit 611d is replaced with a luminance correction rate determination unit 611e.

  First, the luminance correction rate determining unit 611e receives the illuminance (DS) of light detected by the optical sensor from the signal conversion circuit 306 of the display / optical sensor unit 300. On the other hand, the R, G, B illuminance (DS ′) of the light detected by the color illuminance sensor is received from the signal conversion circuit 325. Then, the illuminance (DS) of the light detected by the optical sensor is converted into R illuminance, G illuminance, and B illuminance. In order to perform the conversion, the luminance correction rate determination unit 611e includes an illuminance conversion unit 619.

  First, the illuminance conversion unit 619 refers to the unit information storage unit 617 and identifies, for each illuminance sensor, an optical sensor belonging to the unit UT associated with the illuminance sensor. Then, the illuminance of the light detected by the specified light sensor is converted into the illuminance for each color according to the ratio of the illuminance for each color component of the light detected by the color illuminance sensor.

  Specifically, (A) the illuminance of light R detected by the illuminance sensor is dsr ′, the illuminance of light G detected by the illuminance sensor is dsg ′, and the illuminance of light B detected by the illuminance sensor is dsb ′. (B) The illuminance of the light detected by each of the optical sensors belonging to the unit UT associated with the illuminance sensor is denoted as ds, and (C) a value obtained by converting ds to the illuminance of R, ds When the value converted into the illuminance of G and the value converted from the ds into the illuminance of B are expressed as dsr, dsg, and dsb, respectively, the illuminance conversion unit 619 performs calculation according to the following formulas (2) to (4). , Dsr, dsg, dsb are calculated.

dsr =
ds × dsr ′ ÷ (dsr ′ × C1 + dsg ′ × C2 + dsb ′ × C3) (2)
dsg =
ds × dsg ′ ÷ (dsr ′ × C1 + dsg ′ × C2 + dsb ′ × C3) (3)
dsb =
ds × dsb ′ ÷ (dsr ′ × C1 + dsg ′ × C2 + dsb ′ × C3) (4)
Here, each of C1, C2, and C3 is a coefficient that satisfies C1 + C2 + C3 = 1. For example, C1 = 0.299, C2 = 0.487, and C3 = 0.114.

  In the image displayed on the panel 301, when the luminance of the area where the illuminance of external light is lower than the reference value STL is lowered and displayed dark, the luminance correction rate determination unit 611e is converted by the illuminance conversion unit 619. It is determined whether each of the R, G, and B illuminances is smaller than the reference value STL. When the received illuminance is smaller than the reference value STL, the correction factor CR corresponding to the illuminance is determined.

  Note that, in the image displayed on the panel 301, when the brightness of an area where the illuminance of external light is larger than the reference value STH is increased and displayed brightly, the brightness correction rate determination unit 611e is the one after being converted by the illuminance conversion unit 619. It is determined whether each of the R, G, and B illuminances is greater than the reference value STH. When the received illuminance is greater than the reference value STH, a correction rate CR corresponding to the illuminance is determined.

  Note that the luminance correction rate determination unit 611e determines the correction rate CR according to the illuminance using the conversion information stored in the illuminance / luminance correction rate storage unit 612, similarly to the luminance correction rate determination unit 611a.

  Then, similarly to the luminance correction rate determination unit 611a, the luminance correction rate determination unit 611e converts each of the illuminances to the correction rate CR, and then multiplies each correction rate CR by the correction rate CR to correct the luminance. Identify the target picture element. Since the specifying method is the same as the method performed by the luminance correction rate determining unit 611a, the description thereof is omitted here. However, by referring to the unit information storage unit 617, the picture element included in the unit UT to which the optical sensor belongs is specified for each optical sensor.

(Configuration of color illuminance sensor)
The configuration of the illuminance sensor circuit 330 will be described with reference to FIG. FIG. 24 is a diagram illustrating a configuration of the illuminance sensor circuit 330. As shown in the figure, the illuminance sensor circuit 330 is provided with color filters 324r (red), 324g (green), and 324b (blue), and the external light 61 that has passed through each color filter is received by the respective photodiodes 322. It is the structure which injects into.

  Therefore, the illuminance sensor circuit 330 can output voltages corresponding to the illuminances of R, G, and B of external light (SSr, SSg, and SSb, respectively). Note that the output voltages amplified by the respective sensor output amplifiers 323 connected to SSr, SSg, and SSb are collectively referred to as a sensor output signal SS ′.

  Note that, as described above, the photodiode 322 is a photodiode that is difficult to detect light in the wavelength region of outside Shiba light or infrared light. Therefore, the illuminance sensor circuit 330 has a spectral sensitivity characteristic that is sensitive to visible light (that is, has a spectral sensitivity characteristic close to the visual sensitivity characteristic).

(Flow of processing to correct brightness)
Next, the flow of processing for correcting the luminance of the image displayed on the panel 301 in the data display / sensor device 100e will be described with reference to FIG. FIG. 25 is a flowchart showing a flow of processing for correcting the luminance of an image displayed on the panel 301 in the data display / sensor device 100e.

  Steps S71 to S73 are the same as steps S51 to S53, respectively, and thus description thereof is omitted.

  On the other hand, when R, G, and B external light is detected by the color illuminance sensor (step S76), the sensor output amplifier 323 amplifies the voltage output from the color illuminance sensor (step S77). Then, the amplified sensor output signal SS 'is output to the signal conversion circuit 325, and is converted into a digital signal DS' by the signal conversion circuit 325 (step S78).

  Next, the illuminance conversion unit 619 of the luminance correction rate determination unit 611e uses the illuminance represented by the digital signal DS ′ obtained in step S78 as the illuminance represented by the digital signal DS obtained in step S73. And converted into R, G, and B illuminance (step S81). Then, the luminance correction rate determination unit 611e determines a correction rate CR corresponding to each of the converted R, G, and B illuminances (step S82). Then, for each of the correction rates CR, the pixel for which the luminance is corrected by multiplying the correction rate CR is specified (step S83).

  Thereafter, the display data correction unit 615 is a value obtained by multiplying the luminance for each pixel of the image to be displayed based on the display data received from the main control unit 800 by the correction rate CR for each pixel determined by the luminance correction rate determination unit 611e. The display data is updated so as to be corrected to (step S84). Thereafter, the display control unit 601 displays an image on the panel 301 via the liquid crystal panel drive circuit 304 based on the corrected display data (step S85).

  By repeatedly performing the above processing (steps S71 to S85), the luminance of the image displayed in the changed area can be more appropriately corrected according to the change in the illuminance of the external light of the panel 301. .

(Other adjustment methods)
Various adjustment methods using the illuminance sensor other than the above can be considered. An example of another adjustment method will be described with reference to FIG. FIG. 26 is a diagram illustrating a state where six illuminance sensors (VS_1 to VS_6) are arranged on the periphery of the panel 301. FIG.

  Here, the coordinates of the upper left corner of the panel 301 in the figure are (0, 0), and the coordinates of the lower right corner are (1, 1). The illuminance sensor VS1 is at coordinates (0,0), the illuminance sensor VS2 is at coordinates (0.5,0), the illuminance sensor VS3 is at coordinates (1,0), and the illuminance sensor VS4 is at coordinates (0,1). In addition, the illuminance sensor VS5 is arranged at the coordinates (0.5, 1), and the illuminance sensor VS6 is arranged at the coordinates (1, 1).

  The illuminance of light detected by the illuminance sensor VS_i (i = 1 to 6) is expressed as ds_i. Moreover, the average of the illumination intensity of the light which the optical sensor arrange | positioned in the area | region near the illumination intensity sensor VS_i detected is described as sa_i.

  The region in the vicinity of the illuminance sensor VS_1 is, for example, a region in the rectangle RC1 having coordinates (0, 0) and coordinates (0.1, 0.1) as vertices. Further, the vicinity of the illuminance sensor VS2 is, for example, a region in the rectangle RC2 having coordinates (0.45, 0) and coordinates (0.55, 0.1) as vertices. Further, the vicinity of the illuminance sensor VS_3 is, for example, an area in the rectangle RC3 having coordinates (0.9, 0) and coordinates (1, 0.1) as vertices. Further, the vicinity of the illuminance sensor VS_4 is, for example, a region in the rectangle RC4 having the coordinates (0, 0.9) and the coordinates (0.1, 1) as vertices. The vicinity of the illuminance sensor VS_5 is, for example, a region in the rectangle RC5 having coordinates (0.45, 0.9) and coordinates (0.55, 1) as vertices. Further, the vicinity of the illuminance sensor VS_6 is, for example, a region in the rectangle RC6 having coordinates (0.9, 0.9) and coordinates (1, 1) as vertices.

  At this time, the correction value α_i of the illuminance sensor VS_i is set to α_i = ds_i ÷ sa_i.

  Next, the distance dxy_i between the point Z whose coordinates are (x, y) on the panel 301 and the illuminance sensor VS_i are respectively expressed by the following equations (5) to (10).

  Therefore, the illuminance correction value αxy at the point Z can be obtained by the following mathematical formula (11).

  However, when (x, y) = (0, 0), αxy = α_1, and when (x, y) = (0.5, 0), αxy = α_2, (x, y) = (1 , 0), αxy = α_3, (x, y) = (0, 1), αxy = α_4, and (x, y) = (0.5, 1), αxy = α_5. , (X, y) = (1, 1), αxy = α_6.

  If the illuminance of the light detected by the optical sensor at the point Z is ssxy, the corrected illuminance dsxy can be obtained by the following formula (12).

dsxy = ssxy × αxy (12)
[Additional information]
(Reduction of noise included in illuminance)
As described above, when the optical sensor circuit 32 is driven, the backlight 307 (317) is turned off. However, when an image is displayed on the panel 301, it is desirable that the period during which the backlight 307 (317) is turned off is shorter in order to suppress flickering of the image and the like. Therefore, the period in which the illuminance of external light is detected by the optical sensor when an image is displayed is a short period in which flicker cannot be visually recognized.

  Noise may be included in the illuminance of light detected in such a short period. When the luminance correction described above is performed using the illuminance including noise, the corrected image may include noise. A method for reducing noise included in the corrected image will be described below.

  With reference to FIG. 27, the correspondence between the backlight extinguishing period and the period in which external light is detected by the optical sensor will be described. FIG. 27 is a schematic diagram illustrating a correspondence between a backlight extinguishing period and a period in which external light is detected by an optical sensor. As shown in the figure, outside light is detected by an optical sensor during periods Ti, Ti + 1, Ti + 2, and Ti + 3, which are backlight off (off) periods.

  At this time, the illuminance detected in the periods Ti, Ti + 1, Ti + 2, and Ti + 3 is sequentially stored in a memory or the like, and thereafter, an average value of these four values is calculated, and the calculated value is used as the luminance. The illuminance used for correction. A total value or an intermediate value may be calculated.

  In this way, for each optical sensor, it is a case where noise is included in any of the illuminances by obtaining one of the most recently detected total value, intermediate value, average value, etc. However, the influence of the noise can be reduced.

(Correction direction)
In the above description, the data display / sensor device 100 describes a mode of correcting so that an area where the outside light is dark is displayed with a reduced brightness, and an area where the outside light is bright is displayed with a higher brightness. It has been explained that the correction may be made as described above.

  However, the direction of correction may be reversed. In other words, the area where the outside light is dark may be corrected so that the brightness is increased and displayed brightly. In this case, when the illuminance of external light detected by the optical sensor is smaller than the reference value STL, the luminance of the picture element is increased. In addition, in a region where the outside light is bright, the luminance may be lowered and displayed dark. In this case, when the illuminance of outside light detected by the optical sensor is larger than the reference value STH, the luminance of the picture element is lowered.

  Similarly, when the illuminance of external light R, G, and B can be detected by the optical sensor, the data display / sensor device 100 detects the color tone of the image displayed on the panel 301 by the external light detected by the optical sensor. You may correct | amend for every part so that the color tone of may be reduced.

[Additional Notes]
In the above description, the configuration including the circuit control unit 600 has been described as the configuration of the data display / sensor device 100. However, the configuration is not limited to this configuration. For example, the function of the circuit control unit 600 may be provided in the main control unit 800, or may be provided in the display / light sensor unit 300.

  In addition, the circuit control unit 600 and the main control unit 800 of the data display / sensor device 100 may be configured by hardware logic, or may be realized by software using a CPU as follows.

  That is, the circuit control unit 600 and the main control unit 800 include a CPU (central processing unit) that executes instructions of a control program that realizes each function, a ROM (read only memory) that stores the program, and a RAM that expands the program. (Random access memory), a storage device (recording medium) such as a memory for storing the program and various data. An object of the present invention is a record in which a program code (execution format program, intermediate code program, source program) of a control program for the data display / sensor device 100, which is software for realizing the above-described functions, is recorded so as to be readable by a computer. This can also be achieved by supplying a medium to the data display / sensor device 100 and reading and executing the program code recorded on the recording medium by the computer (or CPU or MPU).

  Examples of the recording medium include a tape system such as a magnetic tape and a cassette tape, a magnetic disk such as a floppy (registered trademark) disk / hard disk, and an optical disk such as a CD-ROM / MO / MD / DVD / CD-R. Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM / flash ROM.

  Further, the data display / sensor device 100 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Also, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), IEEE802.11 radio, HDR, mobile phone network, satellite line, terrestrial digital network, and the like can also be used. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. Is also included in the technical scope of the present invention.

  The present invention can be applied to an apparatus including a display panel capable of displaying data such as an image and detecting the illuminance of external light. In particular, the present invention can be preferably applied to a display device used indoors or outdoors that is illuminated.

31 Pixel circuit (picture element)
31r R pixel circuit (pixel)
31g G pixel circuit (pixel)
31b B pixel circuit (pixel)
32 Optical sensor circuit (optical sensor)
100, 100a to 100e Data display / sensor device (display device)
301 LCD panel with built-in sensor (display panel)
304 Liquid crystal panel drive circuit (display control means)
320, 330 Illuminance sensor circuit (illuminance sensor)
601, 601 b, 601 c Display control unit (display control means)
611a, 611d, 611e Luminance correction rate determination unit (luminance correction means)
615, 625 Display data correction unit (luminance correction means)
UT unit (display / sensor unit)
STH, STL standard value

Claims (12)

A display device comprising a display panel in which a display / sensor unit comprising one or a plurality of picture elements and an optical sensor for detecting the illuminance of external light is arranged in a plane,
Brightness correction means for correcting the brightness of the picture element according to the illuminance of light detected by the optical sensor for each display / sensor unit;
A display device comprising: display control means for lighting a picture element with the brightness corrected by the brightness correction means.   2. The display device according to claim 1, wherein when the illuminance of the light detected by the optical sensor is smaller than a reference value, the luminance correction unit decreases the luminance of the picture element according to the illuminance.   2. The display device according to claim 1, wherein when the illuminance of the light detected by the optical sensor is greater than a reference value, the luminance correction unit increases the luminance of the picture element according to the illuminance. The brightness correction means is
4. The display device according to claim 1, wherein the luminance of the picture element is corrected by multiplying a correction factor corresponding to the illuminance of light detected by the optical sensor. 5. Further comprising one or more illuminance sensors having spectral sensitivity characteristics sensitive to visible light;
Each of the display / sensor units is associated with one of the illuminance sensors,
The brightness correction means is
Using the illuminance of light detected by the illuminance sensor associated with the display / sensor unit to which the optical sensor belongs, the illuminance of light detected by the optical sensor is reduced by errors other than visible light. And adjust it to
5. The display device according to claim 1, wherein the luminance of the picture element is corrected for each display / sensor unit in accordance with the illuminance of the light after the adjustment. The picture element includes pixels that display red, green, and blue, respectively.
Furthermore, it has a spectral sensitivity characteristic that is sensitive to visible light, and includes one or a plurality of illuminance sensors that detect the illuminance of external light for each color of red, green, and blue,
Each of the display / sensor units is associated with one of the illuminance sensors,
The brightness correction means is
In accordance with the ratio of the illuminance of each color of light detected by the illuminance sensor associated with the display / sensor unit to which the optical sensor belongs, the illuminance of the light detected by the optical sensor is converted into illuminance for each color. With
2. The display device according to claim 1, wherein brightness of pixels of each color included in a picture element is corrected for each display / sensor unit in accordance with the illuminance for each color after the conversion. A display including a display panel in which display / sensor units each including a picture element including pixels for displaying red, green, and blue and an optical sensor that detects external light of red, green, and blue are arranged in a plane. A device,
Brightness correction means for correcting the brightness of the corresponding color pixel in accordance with the illuminance of the light of each color detected by the photosensor for each display / sensor unit;
A display device comprising: display control means for turning on the pixels with the brightness of each color pixel corrected by the brightness correction means.   8. The luminance correction unit according to claim 7, wherein when the illuminance of each color of light detected by the photosensor is smaller than a reference value, the luminance of the corresponding color pixel is lowered according to the illuminance. Display device.   8. The luminance correction unit according to claim 7, wherein when the illuminance of each color of light detected by the photosensor is larger than a reference value, the luminance of the corresponding color pixel is increased according to the illuminance. Display device. The brightness correction means is
10. The luminance of a pixel of a corresponding color is corrected by multiplying a correction rate according to the illuminance of light of each color detected by the optical sensor. Display device. Further comprising one or more illuminance sensors having spectral sensitivity characteristics sensitive to visible light;
Each of the display / sensor units is associated with one of the illuminance sensors,
The brightness correction means is
Using the illuminance of light detected by the illuminance sensor associated with the display / sensor unit to which the optical sensor belongs, the illuminance of light of each color detected by the optical sensor is caused by light other than visible light. And adjust to decrease
11. The display according to claim 7, wherein the brightness of the pixel of the corresponding color is corrected for each display / sensor unit according to the illuminance of the light of each color after the adjustment. apparatus.   The display device according to claim 1, wherein each of the display / sensor units includes one picture element and one photosensor.

Images