JP2007018458A - Display unit, sensor signal correction method, and imaging unit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a display unit, a sensor signal correction method, and an imaging unit, capable of suppressing fluctuations and variations in a sensor signal, generated by a photosensor device provided in a pixel portion. <P>SOLUTION: The display unit includes a photosensor device and a display device. A portion of a plurality of pixel portions P, mutually having an equivalent structure, is light-shielded by a light-shielding film 8. The sensor signal, generated by the photosensor device included in each of the plurality of pixel portions, is read out under the control of a sensor controller 61, so as to be supplied to a correction section 63. In the correction section 63, the sensor signal read out from the pixel portion that is not light shielded by the light-shielding film 8 is corrected, according to the sensor signal read out from the pixel portion being light shielded by the light-shielding film 8. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a display device provided with a photosensor element in a display area, a method for correcting a sensor signal generated by the photosensor element, and an imaging device for correcting a sensor signal generated by the photosensor element. The present invention relates to a liquid crystal display device in which an optical sensor element is provided in a display area.

  A liquid crystal display device in which an optical sensor element is provided in a display area is known (see, for example, Patent Document 1). By providing an optical sensor element in the display area, it is possible to realize a function related to light detection, such as a scanner and a touch panel, in addition to a normal display function.

As a photosensor element provided in the display region, a thin film transistor (TFT) element having the same structure as that used for driving a liquid crystal is used in the liquid crystal display device described in Patent Document 1. This simplifies the structure of the pixel compared to the case where light is detected using a photodiode or the like, and simplifies the manufacturing process because a special process for forming the optical sensor element is not necessary. Can do.
JP 7-325319 A

  However, the sensor signal generated in the optical sensor element changes not only due to the light to be detected but also due to various factors. For example, when light is detected using a TFT element as in the liquid crystal display device described in Patent Document 1, the dark current flowing through the TFT element causes variations and fluctuations in sensor signals.

  The present invention has been made in view of such circumstances, and a first object thereof is to provide a display device capable of suppressing variations and fluctuations in sensor signals generated by optical sensor elements provided in a pixel portion. A second object is to provide a method for correcting variations and fluctuations in sensor signals generated by an optical sensor element in a display device in which an optical sensor element is provided in a pixel portion. A third object is to provide an imaging apparatus capable of suppressing variations and fluctuations in sensor signals generated by an optical sensor element for imaging.

  A display device according to a first aspect of the present invention includes a plurality of pixel portions each including an optical sensor element and a display element, and having a structure equivalent to each other, and a light-shielding film that shields part of the plurality of pixel portions. A readout unit that reads out a sensor signal generated by an optical sensor element included in each of the plurality of pixel units, and a pixel in which a sensor signal read out from the pixel unit that is not shielded from light by the light shielding film is shielded by the light shielding film A correction unit that performs correction based on the sensor signal read from the unit.

According to the first aspect, some of the plurality of pixel portions including the optical sensor element and the display element and having the same structure are shielded from light by the light shielding film. Sensor signals generated by the optical sensor elements included in each of the plurality of pixel units are read by the reading unit and supplied to the correction unit. In the correction unit, the sensor signal read from the pixel portion not shielded by the light shielding film is corrected based on the sensor signal read from the pixel portion shielded by the light shielding film.
The sensor signal read from the light-shielded pixel portion is generated due to a factor different from that of light incident in a state where the light is not shielded. Since this sensor signal is generated in a pixel unit having a structure equivalent to another pixel unit that is not shielded from light, a signal component that is generated due to the different factors included in the sensor signal of the other pixel unit that is not shielded from light Has a certain correlation. Therefore, the correction unit corrects the influence of the signal component generated due to a factor different from the incident light included in the sensor signal read from the pixel unit that is not shielded from light.

The correction unit reads from one sensor signal read from the light-shielded pixel unit or an average value of a plurality of sensor signals read from the plurality of light-shielded pixel units, and the non-light-shielded pixel unit. The difference from the sensor signal may be calculated.
Thereby, a signal component of a factor different from the incident light, which is added to the sensor signal read from the pixel portion that is not shielded from light, is corrected.

  The plurality of pixel units may be arranged in a matrix. In this case, the light shielding film may shield at least a part of the pixel portion located at the periphery of the matrix.

The display device according to the first aspect may include a display control unit that controls the luminance of the display element.
The display control unit may set the luminance of the display element included in the pixel unit shielded by the light shielding film to a maximum value. Alternatively, the luminance of the display element included in the pixel portion shielded by the light shielding film is set to the same value as the luminance of the display element included in the pixel portion adjacent to the pixel portion and not shielded by the light shielding film. May be. Alternatively, the luminance of the display element included in the pixel portion shielded by the light shielding film may be set to the average value of the luminance of the display element included in the pixel portion not shielded by the light shielding film.

  A second aspect of the present invention corrects a sensor signal generated by the optical sensor element in a display device that includes a plurality of pixel portions each having an equivalent structure, each including an optical sensor element and a display element. The correction method includes a first step of reading sensor signals generated by the optical sensor elements of the plurality of pixel units, and a sensor of the plurality of pixel units read out in the first step. A second step of correcting the sensor signal of the pixel portion not shielded by the light shielding film based on the sensor signal of the pixel portion shielded by the light shielding film among the signals.

  According to a third aspect of the present invention, there is provided a display device including a plurality of pixel portions each including a photosensor element and a display element, having an equivalent structure, and arranged in a matrix. The present invention relates to a method for correcting a generated sensor signal, and this correction method selects each row of the matrix in order and reads a sensor signal generated by a photosensor element of a pixel unit belonging to the selected row. When the sensor signal is read from one row of the matrix in the first step, the pixel portion in which the sensor signal of the pixel portion that is not shielded by the light shielding film in the one row is shielded by the light shielding film. And a second step of correcting based on the sensor signal.

  An imaging apparatus according to a fourth aspect of the present invention includes a plurality of pixel units each including an optical sensor element and having an equivalent structure, a light shielding film that shields a part of the plurality of pixel units, and the plurality of the plurality of pixel units. A readout unit that reads out a sensor signal generated by an optical sensor element included in each of the pixel units, and a sensor signal that is read out from a pixel unit that is not shielded by the light shielding film from the pixel unit that is shielded from light by the light shielding film. And a correction unit that performs correction based on the sensor signal.

  According to the present invention, incident light is corrected by correcting sensor signals read from other pixel portions based on sensor signals generated by factors different from incident light in a part of a plurality of pixel portions having an equivalent structure. Variations and fluctuations in sensor signals caused by factors different from light can be suppressed.

FIG. 1 is a diagram illustrating an example of a configuration of a display device according to an embodiment of the present invention.
As shown in FIG. 1, for example, the display device according to the present embodiment includes a pixel array 1, a scanning line driving unit 2, a signal line driving unit 3, a sensor control line driving unit 4, a sensor output unit 5, A control unit 6 and a storage unit 7 are included.

  The pixel array 1 has a plurality of pixel portions P having the same structure. The plurality of pixel portions P are arranged in a matrix, for example, as shown in FIG. The pixel units P belonging to the same row are connected to a common scanning line driven by the scanning line driving unit 2 and to a common sensor control line driven by the sensor control line driving unit 4. The pixel units P belonging to the same column are connected to a common pixel signal line driven by the signal line driving unit 3 and to a common sensor signal line from which a signal is read out by the sensor output unit 5.

Each pixel portion P includes a display element and a photosensor element.
A display element is a liquid crystal display element comprised, for example using a liquid crystal. The liquid crystal display element changes the light transmittance by changing the orientation of liquid crystal molecules sandwiched between two transparent substrates in accordance with the voltage supplied from the signal line driver 3. One of the two substrates sandwiching the liquid crystal is provided with a circuit for turning on and off the voltage applied to the liquid crystal in accordance with the drive signal from the scanning line driving unit 2, and this circuit is constituted by, for example, a TFT element.
The optical sensor element converts light incident from the outside into electricity and outputs it as a sensor signal. The optical sensor element is composed of, for example, a TFT element or a photodiode.

FIG. 3 is a diagram illustrating an example of the configuration of the pixel portion P.
The pixel portion P illustrated in the example of FIG. 3 includes a liquid crystal display element 21 and a photosensor element 22.

  The liquid crystal display element 21 shown in FIG. 3 includes a transistor Q1 for turning on and off the voltage from the signal line S1 applied to the liquid crystal LC, and an auxiliary capacitor for holding a voltage applied to the liquid crystal LC when the transistor Q1 is off. C1.

The transistor Q1 is constituted by, for example, a TFT element.
The gate of the transistor Q1 is connected to the scanning line G1 of the scanning line driving unit 2, and the source thereof is connected to the pixel signal line S1. The drain of the transistor Q1 is connected to an electrode (pixel electrode) for applying a voltage from the signal line S1 to the liquid crystal LC, and an auxiliary capacitor C1 is connected between the pixel electrode and the auxiliary capacitor line CS. The storage capacitor line CS is connected to a common electrode formed on the substrate facing the pixel electrode with the liquid crystal LC interposed therebetween.

  When a pulse voltage is applied to the scanning line G1 by the scanning line driving unit 2, the transistor Q1 connected thereto is turned on. When the transistor Q1 is turned on, a display pixel signal output from the signal line driver 3 is applied to the pixel electrode through the signal line S1, and the capacitor C1 is charged. After the charging, when the transistor Q1 is turned off, the voltage corresponding to the pixel signal is held in the capacitor C1, so that the voltage corresponding to the pixel signal is stored in the liquid crystal LC until the next scanning line G1 is driven. Is applied.

  The photosensor element 22 shown in FIG. 3 includes a transistor Q2 through which a current corresponding to light incident from the outside flows, a capacitor C2 charged according to the current flowing through the transistor Q2, and a voltage charged into the capacitor C2. The transistor Q4 for driving the sensor signal line S2, the transistor Q5 for turning on / off the connection between the sensor signal line S2 and the transistor Q4, and the transistor Q3 for resetting the voltage of the capacitor C2. The transistors Q2 to Q5 are configured by TFT elements, for example, similarly to the transistor Q1.

The transistor Q2 has a gate connected to the bias supply line BA, a drain connected to the power supply line VDD, and a source connected to the reference potential line VSS via the capacitor C2. The transistor Q2 is turned off by the bias voltage supplied from the bias supply line BA.
A transistor Q3 is connected to both ends of the capacitor C2. The gate of the transistor Q3 is connected to a reset line RST that is one of the sensor control lines.
The gate of the transistor Q4 is connected to the connection point between the capacitor Q2 and the transistor Q2, its drain is connected to the power supply line VDD, and its source is connected to the sensor signal line S2 via the transistor Q5.
The gate of the transistor Q5 is connected to the scanning line G2, which is one of the sensor control lines.

When detecting incident light, first, a pulse voltage is applied to the reset line RST by the sensor control line driver 4, and the transistor Q3 is turned on. As a result, the electric charge accumulated in the capacitor C2 is discharged, and the voltage of the capacitor C2 is reset to zero. After the reset, when the transistor Q3 is turned off, the leakage current or the off-current of the transistor Q2 flows into the capacitor C2. Since these currents change according to the incident light from the outside, the charging voltage of the capacitor C2 changes according to the incident light.
When a pulse voltage is applied to the scanning line G2 by the sensor control line driving unit 4, the transistor Q5 connected thereto is turned on. When the transistor Q5 is turned on, the source of the transistor Q4 and the sensor signal line S2 are connected. The transistor Q4 forms a source follower circuit, and a current flows from the power supply line VDD to the source so that a source voltage corresponding to the gate voltage is generated. Therefore, the voltage of the sensor signal line S2 corresponds to the charging voltage of the capacitor C2. A voltage is generated. That is, a voltage corresponding to the incident light is output to the sensor signal line S2.

Note that a part of the pixel portion P constituting the pixel array 1 is shielded by the light shielding film 8, and the incident light from the outside is shielded.
For example, as illustrated in FIG. 1, the light shielding film 8 shields a pixel portion located at the periphery of the matrix. In the example of FIG. 1, the pixel portions in one row or one column located at the upper end, lower end, left end, and right end of the matrix are shielded from light.

Returning to the description of FIG.
The scanning line driving unit 2 sequentially drives the scanning lines provided for each row of the pixel units P arranged in a matrix according to the control of the control unit 6. When the scanning line of a certain row is driven by the scanning line driving unit 2, the pixel signal supplied from the pixel signal line is taken in each pixel unit P belonging to this row, and the luminance of the display element is set.

  The signal line driving unit 3 drives the pixel signal lines provided for each column of the pixel units P arranged in a matrix according to the image signal supplied from the control unit 6. When a scanning line in a certain row is driven by the scanning line driving unit 2, the signal line driving unit 3 supplies a pixel signal to each pixel unit P belonging to this row through the pixel signal line.

  The sensor control line drive unit 4 sequentially drives the sensor control lines provided for each row of the pixel units P arranged in a matrix according to the control of the control unit 6. The sensor control line is a wiring for supplying a control signal to the optical sensor element. For example, in the case of the optical sensor element 22 shown in FIG. 3, the scanning line G2 and the reset line RST correspond to the sensor control line. When a control signal is supplied to the pixel unit P in a row with the sensor control line driving unit 4, sensor signals generated in the photosensor elements of the pixel units P belonging to this row are output to the sensor signal lines.

The sensor output unit 5 takes in a sensor signal from an optical sensor element output to a sensor signal line provided for each column of the pixel units P arranged in a matrix and outputs the sensor signal to the control unit 6.
The sensor output unit 5 is, for example, a sample hold circuit that holds a sensor signal output from the pixel unit P of each row through a sensor signal line, a shift circuit that sequentially extracts the held sensor signals, and a shift circuit that sequentially extracts the sensor signals. An analog-to-digital conversion circuit that converts the sensor signal into a digital signal. The sensor signal that is digitized and output from the sensor output unit 5 is referred to as pixel data in the following description.

  The control unit 6 performs various controls related to an operation for writing a display pixel signal to each pixel unit P constituting the pixel array 1 and an operation for reading a sensor signal from each pixel unit P, and also reads from the pixel unit P. The pixel data (sensor signal) is corrected. The control unit 6 includes, for example, a timing generation circuit that defines driving timings of scanning lines and signal lines, a sequencer circuit that controls an operation sequence, a computer that performs various processes and operations, and the like.

  For example, as illustrated in FIG. 1, the control unit 6 includes a sensor control unit 61, a display control unit 62, and a correction unit 63.

  The sensor control unit 61 controls the operation of the optical sensor element of the pixel unit P (generation of sensor signals, reading of sensor signals, etc.). That is, a series of operations for selecting each row of the pixel portions P arranged in a matrix, reading the sensor signals from the pixel portions P belonging to the selected row, and outputting them as pixel data from the sensor output portion 5 are performed appropriately. As described above, the sensor control line driving unit 4 and the sensor output unit 5 are controlled.

  The correction unit 63 converts pixel data read from a pixel part P that is not shielded by the light shielding film 8 (hereinafter referred to as a non-shielded pixel part) into a pixel part P that is shielded by the light shielding film 8 (hereinafter referred to as a light shielding pixel part). Correction is performed based on the pixel data read out from (1).

  For example, the dark current flowing through the transistor Q2 in the optical sensor element 22 shown in FIG. 3 varies from individual to individual depending on the manufacturing conditions, and fluctuates due to environmental factors such as temperature. This dark current variation or fluctuation is added directly to the pixel data value. Therefore, the correction unit 63 calculates the difference between the pixel data read from the light-shielded pixel unit (or the average value thereof) and the pixel data read from the non-light-shielded pixel unit, so that the dark current included in the latter pixel data is calculated. Remove the components.

  The display control unit 62 controls an operation of writing a display pixel signal to the pixel unit P. That is, the scanning line driving unit 2 and the signal are selected so that a series of operations for selecting each row of the pixel units P arranged in a matrix and writing a desired pixel signal to the pixel units P belonging to the selected row are appropriately performed. The line drive unit 3 is controlled.

In addition, the display control unit 62 sets the luminance of the display element included in the light-shielding pixel unit to an appropriate value that allows the correction unit 63 to obtain a good correction result.
For example, the display control unit 62 sets the luminance of the display element included in the light-shielding pixel unit to the maximum value (that is, white). Thereby, the influence of the light leaking from the display element to the photosensor element can be made to appear strongly in the sensor signal of the light-shielding pixel portion.
Alternatively, the display control unit 62 may set the luminance of the display element included in the light shielding pixel unit to the same value as the luminance of the display element included in the non-light shielding pixel unit adjacent to the light shielding pixel unit. Thereby, the influence which the light which leaks from a display element to an optical sensor element has on a sensor signal can be approximated in two adjacent pixel portions.
Alternatively, the display control unit 62 may set the luminance of the display elements included in the light-shielding pixel unit to the average value of the luminances of the display elements included in some or all of the non-light-shielding pixel units of the pixel array 1. Thereby, the influence which the light which leaks from a display element to an optical sensor element has on a sensor signal can be approximated as a whole in the pixel array 1.

  The storage unit 7 stores image data to be displayed on the pixel array 1 and image data captured on the pixel array 1. Further, for example, data used for the processing of the control unit 6 such as data used for correction processing in the correction unit 63 and data of the processing result are stored.

  Here, correction processing of pixel data in the display device having the above-described configuration will be described with some examples.

(First example of correction processing)
In the first example of the correction process, the two pixel portions located at both ends of the pixel portions P arranged in the row direction are shielded from light, and the pixel data of the non-light-shielded pixel portions belonging to the same row is used. Correct.

FIG. 4 is a diagram illustrating an array of pixel units in the row direction in the pixel array 1.
In the example of FIG. 4, the pixel portion in the i-th row and the j-th column is represented as “P (i, j)”, and the pixel data read there is represented as “D (i, j)”.
In the i-th row, '2N + 2' pixel units are arranged. The pixel portions P (i, 0) and P (i, 2N + 1) at both ends indicated by diagonal lines in the i-th row are shielded by the light-shielding film 8.

The graph at the left end and the right end in FIG. 4 shows how the pixel data of the light-shielding pixel portion changes according to the position in the column direction. That is, the leftmost graph in FIG. 4 shows how the pixel data D (y, 0) read from the light-shielding pixel portion in the 0th column changes according to the row number y. The rightmost graph in FIG. 4 shows how pixel data D (y, 2N + 1) read from the light-shielding pixel portion in the (2N + 1) th column changes according to the row number y.
As shown in this graph, the light-shielding pixel portion generates pixel data having a value closer to that of the light-shielding pixel portion formed closer to the position. This is because the characteristics of circuit elements (TFT elements, etc.) constituting the optical sensor element are approximated as they are closer to each other on the substrate.
In this example, in consideration of such a tendency of the pixel data, the pixel data of the non-light-shielded pixel portion is corrected using the pixel data read by the light-shielded pixel portion as close as possible.

  That is, when the pixel portions are arranged as shown in FIG. 4, the correction unit 63 in this example uses the pixel data D (i, j) of the non-light-shielded pixel portion P (i, j) as the non-light-shielded pixel. Correction is performed based on the pixel data of either the light-shielding pixel portion P (i, 0) or P (i, 2N + 1) located at a position close to the portion P (i, j).

That is, the correction unit 63 performs correction as follows according to the column number j.
When the column number j is “1 ≦ j ≦ N”, the correction unit 63 corrects the pixel data D (i, j) based on the pixel data D (i, 0). For example, the difference between the pixel data D (i, j) and the pixel data D (i, 0) is calculated.
When the column number j is “N + 1 ≦ j ≦ 2N”, the correction unit 63 corrects the pixel data D (i, j) based on the pixel data D (i, 2N + 1). For example, the difference between the pixel data D (i, j) and the pixel data D (i, 2N + 1) is calculated.

  FIG. 5 is a flowchart illustrating an example of the correction process described above. FIG. 6 is a diagram illustrating the flow of data when the correction process shown in FIG. 5 is executed.

Step ST101:
When the sensor control line driving unit 4 drives the sensor control line in the i-th row in accordance with the control of the sensor control unit 61, each pixel unit belonging to the i-th row outputs a sensor signal to the sensor signal line. The unit 5 outputs the pixel data of the i-th row.

Step ST102:
The correction unit 63 corrects the pixel data of the non-light-shielding pixel portion located between the pixel data of the light-shielding pixel portions at both ends among the pixel data of the i-th row output from the sensor output unit 5.
That is, the pixel data read from the non-light-shielded pixel portion located near the light-shielded pixel portion P (i, 0) is corrected by the pixel data D (i, 0), and the light-shielded pixel portion P (i, 2N + 1) is corrected. The pixel data read from the non-light-shielded pixel portion at a close position is corrected by the pixel data D (i, 2N + 1).
The correction unit 63 stores the corrected image data for one row in the storage unit 7.

Steps ST103 and ST104:
When the correction process for one row is completed, the sensor control unit 61 determines whether pixel data of all areas of the pixel array has been read. When the pixel data of all the areas are read, the process for one frame is finished, and when not read yet, the processes of steps ST101 and ST102 are repeated in the next (i + 1) th row.

  As described above, in the correction process of this example, the pixel data reading process and the correction process proceed in parallel. Therefore, when the processing for one frame is completed, the corrected image data of one frame is stored in the storage unit 7.

(Second example of correction processing)
Next, a second example of the correction process will be described.
In the correction processing of this example, a predetermined correction function using the position in the row direction as a variable is determined according to the pixel data of the two light-shielding pixel portions located at both ends of the row, and the non-light-shielded pixel is determined using this correction function. Part of the pixel data is corrected.

Also in this example, it is assumed that the pixel portions are arranged as shown in FIG.
The correction unit 63 determines a correction function Fi (x) for the i-th row expressed by the following equation, for example, using the column number x (= 0,..., 2N + 1) representing the position in the row direction as a variable.

[Equation 1]
Fi (x) = Ai × x + Bi (1)

  In the equation (1), the slope Ai and the intercept Bi are expressed by the following equations.

[Equation 2]
Ai = (D2-D1) / (2N + 1) (2)
Bi = D1 (3)
However,
D1 = D (i, 0);
D2 = D (i, 2N + 1);

The correction function Fi (x) shown in Expression (1) is a linear function that interpolates the value of the pixel data in the middle of the row according to the pixel data D1 and D2 read in the light-shielding pixel portions at both ends of the row.
The correction unit 63 corrects the pixel data D (i, j) of the non-light-shielding pixel unit based on the value of the correction function Fi (x). For example, the difference between the pixel data D (i, j) and the correction function Fi (x) is calculated.

  The correction process in this example may be the same as that shown in FIG. That is, when the pixel data of the i-th row is output from the sensor output unit 5 in step ST102, the slope Ai and the intercept Bi represented by the equations (2) and (3) according to the pixel data D1 and D2 at both ends thereof. And the correction function Fi (x) is determined. Then, using the determined correction function Fi (x), the pixel data D (i, j) of the non-light-shielded pixel portion is corrected. If this correction processing is performed by, for example, a logic circuit, the correction result can be written in the storage unit 7 every time pixel data for one row is output from the sensor output unit 5.

  However, when the delay of the correction process by the logic circuit as described above is so large that it cannot be ignored as compared with the period in which the pixel data for one row is output in the sensor output unit 5, the signal is output from the sensor output unit 5. It is also possible to reduce the influence of the delay of the correction processing by temporarily holding the pixel data for one row in the line memory or the like.

FIG. 7 is a flowchart showing an example of such correction processing. In the flowchart shown in FIG. 7, steps ST105 and ST106 are inserted between steps ST101 and ST102 in the flowchart shown in FIG.
In step ST105, the sensor control unit 61 temporarily stores pixel data for one row output from the sensor output unit 5 in a line memory (not shown). In parallel with this, the correction unit 63 calculates the slope Ai and the intercept Bi of the correction function Fi (x) according to the pixel data D1 and D2 output from the sensor output unit 5, and the calculation result is stored in the register ( Temporarily stored in (not shown).
In step ST102, the value of the correction function Fi (x) of each column (x = 1,..., 2N) is calculated using the slope Ai and the intercept Bi stored in the register, and based on this, the pixel of the non-light-shielding pixel portion is calculated. The data D (i, j) is corrected (for example, the difference between the two is obtained).
As described above, since the correction process is divided into a plurality of steps and the delay of the correction process in each step can be shortened, the correction process can be performed without extending the output period of the pixel data in the sensor output unit 5. It can be carried out.

(Third example of correction processing)
Next, a third example of the correction process will be described.
In this example, light-shielding pixel portions are arranged at both ends of a row and columns, respectively, and pixel data of a non-light-shielding pixel portion is corrected using those pixel data.

FIG. 8 is a diagram illustrating the arrangement of the pixel portions in the row direction and the column direction in the pixel array 1.
In the example of FIG. 8, as in FIG. 4, the pixel portion in the i-th row and j-th column is represented as 'P (i, j)', and the pixel data read out there is represented as 'D (i, j)'. Yes.
In the i-th row, '2N + 2' pixel units are arranged. The pixel portions P (i, 0) and P (i, 2N + 1) at both ends indicated by diagonal lines in the i-th row are shielded by the light-shielding film 8.
In the j-th column, '2M + 2' pixel units are arranged. The pixel portions P (0, j) and P (2M + 1, j) at both ends indicated by oblique lines in the j-th column are shielded from light by the light shielding film 8.

The graphs at the left end and the right end in FIG. 8 show how the pixel data of the light-shielding pixel portion changes according to the position in the column direction, and are the same as those shown in FIG.
The upper and lower graphs in FIG. 8 illustrate how the pixel data of the light-shielding pixel portion changes according to the position in the row direction. The graph at the upper end of FIG. 8 shows how the pixel data D (0, x) read from the light-shielding pixel portion in the 0th row changes according to the column number x. The graph at the bottom of FIG. 8 shows how the pixel data D (2M + 1, x) read from the light-shielding pixel portion in the (2M + 1) th row changes according to the column number x.

When the pixel units are arranged as shown in FIG. 8, the correction unit 63 in this example is
The pixel data D (i, j) of the non-light-shielding pixel portion P (i, j) is converted to the light-shielding pixel portion P (i, 0) or P at a position close to the non-light-shielding pixel portion P (i, j). Any one of the pixel data of (i, 2N + 1) and any one of the light-shielded pixel portion P (0, j) or P (2M + 1, j) located at a distance close to the non-light-shielded pixel portion P (i, j) Correction is performed based on the average value of the pixel data.

That is, the correction unit 63 of this example performs correction as follows according to the row number i and the column number j.
When the row number i is “1 ≦ i ≦ M” and the column number j is “1 ≦ j ≦ N”, the correction unit 63 sets the average value of the pixel data D (0, j) and D (i, 0). Based on this, the pixel data D (i, j) is corrected.
When the row number i is “1 ≦ i ≦ M” and the column number j is “N + 1 ≦ j ≦ 2N”, the correction unit 63 sets the average value of the pixel data D (0, j) and D (i, 2N + 1). Based on this, the pixel data D (i, j) is corrected.
When the row number i is “M + 1 ≦ i ≦ 2M” and the column number j is “1 ≦ j ≦ N”, the correction unit 63 sets the average value of the pixel data D (2M + 1, j) and D (i, 0). Based on this, the pixel data D (i, j) is corrected.
When the row number i is “M + 1 ≦ i ≦ 2M” and the column number j is “N + 1 ≦ j ≦ 2N”, the correction unit 63 sets the average value of the pixel data D (2M + 1, j) and D (i, 2N + 1). Based on this, the pixel data D (i, j) is corrected.

  FIG. 9 is a flowchart illustrating an example of the correction process described above. FIG. 10 is a diagram illustrating the flow of data when the correction process shown in FIG. 9 is executed.

Steps ST201 and ST202:
The sensor control unit 61 controls the sensor control unit line driving unit 4 and the sensor output unit 5 to read out pixel data of all areas of the pixel array 1 and writes them in the storage unit 7.

Step ST203:
The correction unit 63 reads out pixel data of the light-shielding pixel unit from the pixel data of all areas of the pixel array 1 stored in the storage unit 7. Then, the above-described average value corresponding to the row number i and the column number j is calculated and stored in the storage unit 7 as correction data.

Step ST204:
The correction unit 63 reads the correction data calculated in step ST203 and the pixel data stored in step ST202 from the storage unit 7, respectively, and corrects the pixel data of the non-light-shielding pixel unit. Then, the corrected pixel data is written back to the storage unit 7.

  As described above, in the correction processing of this example, the pixel data of all areas are temporarily stored in the storage unit 7 and then the pixel data of the light-shielding pixel unit is corrected.

(Fourth example of correction processing)
In the correction processing of this example, a predetermined correction function using the position in the row direction as a variable is determined according to the pixel data of the two light-shielding pixel portions positioned at both ends of the row, and two positions positioned at both ends of the column are determined. A predetermined correction function with the position in the column direction as a variable is determined according to the pixel data of the light-shielding pixel portion. Then, the pixel data of the non-light-shielded pixel portion is corrected using the average value of the two correction functions.

Also in this example, it is assumed that the pixel portions are arranged as shown in FIG.
The correction unit 63 determines, for example, the correction function Fi (x) for the i-th row represented by the above equation (1) using the column number x (= 0,..., 2N + 1) representing the position in the row direction as a variable. .
Further, the correction unit 63 determines the correction function Gj (y) of the j-th column expressed by the following equation using the row number y (= 0,..., 2M + 1) representing the position in the column direction as a variable.

[Equation 3]
Gj (y) = Ci * y + Di (4)

  In the equation (4), the slope Ci and the intercept Di are expressed by the following equations.

[Equation 4]
Ci = (D4-D3) / (2M + 1) (5)
Di = D3 (6)
However,
D3 = D (0, j);
D4 = D (2M + 1, j);

  The correction function Gj (y) shown in Expression (4) is a linear function that interpolates the value of pixel data in the middle of the column according to the pixel data D3 and D4 read out at the light-shielding pixel portions at both ends of the column.

  The correction unit 63 corrects the pixel data D (i, j) of the non-light-shielding pixel unit based on the average value of the correction functions Fi (j) and Gj (i). For example, the difference between the pixel data D (i, j) and the average value is calculated.

  The correction process in this example may be the same as that shown in FIG. That is, in step ST203, a correction function for each row and each column is determined, a function value corresponding to row number i and column number j is calculated, and an average value of both is calculated. Then, the calculated average value of the function is stored in the storage unit 7 as correction data. In step ST204, the pixel data of the non-light-shielded pixel portion is corrected based on the correction data stored in the storage unit 7.

As described above, according to the present embodiment, a part of the plurality of pixel portions P including the optical sensor element and the display element and having the same structure is shielded by the light shielding film 8. Sensor signals generated by the optical sensor elements included in each of the plurality of pixel units are read according to the control of the sensor control unit 61 and supplied to the correction unit 63. In the correction unit 63, the sensor signal read from the pixel portion not shielded by the light shielding film 8 is corrected based on the sensor signal read from the pixel portion shielded by the light shielding film 8.
The sensor signal read from the light-shielding pixel unit is generated by a factor different from light incident in a state where light is not shielded, such as a dark current of the TFT element. Since the light-shielding pixel portion has the same structure as the non-light-shielding pixel portion, the sensor signal generated in the light-shielding pixel portion depends on the above factors (dark current, etc.) included in the sensor signal of the non-light-shielding pixel portion. The generated signal component has a certain correlation.
Therefore, by correcting the sensor signal read from the non-light-shielded pixel portion based on the sensor signal read from the light-shielded pixel portion, the influence of the signal component generated by a factor different from the incident light included in the latter sensor signal is reduced. In addition, variations and fluctuations can be suppressed.

  As mentioned above, although one Embodiment of this invention was described, this invention is not limited only to said form, Various variations are included.

  In FIG. 1, the upper end, the lower end, the left end, and the right end of the matrix of the pixel array 1 are shielded from light by the light shielding film 8, respectively, but the present invention is not limited to this. For example, when the first example or the second example of the correction processing described above is employed, only the left and right end columns of the matrix may be shielded from light.

  In the above-described embodiment, a liquid crystal display element is cited as an example of the display element. However, the present invention is not limited to this, and the present invention can be realized by using other various display elements such as an EL element.

  In the above-described embodiment, an example in which a display element and a photosensor element are incorporated in each of the pixel portions constituting the pixel array 1 is shown, but the present invention is not limited to this. For example, a pixel portion including only a display element that does not include a photosensor element may be mixed.

  In the above-described embodiment, a display device provided with a photosensor element in the display area is taken as an example, but the present invention is not limited to the display device. That is, when a display element is deleted from the pixel portion in the above display device, it is possible to configure an imaging device that captures an image with the optical sensor element. Even in such an imaging apparatus, it is possible to achieve the same effect as the above display apparatus.

It is a figure which shows an example of a structure of the display apparatus which concerns on embodiment of this invention. It is a figure which shows an example of a structure of a pixel array. It is a figure which shows an example of a structure of a pixel part. It is the figure which illustrated the arrangement | sequence of the pixel part of the row direction in a pixel array. It is a 1st flowchart which shows an example of a correction process. FIG. 6 is a diagram illustrating a data flow when the correction process illustrated in FIG. 5 is executed. It is a 2nd flowchart which shows an example of a correction process. It is the figure which illustrated the arrangement | sequence of the pixel part of the row direction and column direction in a pixel array. It is a 3rd flowchart which shows an example of a correction process. FIG. 10 is a diagram illustrating a data flow when the correction process illustrated in FIG. 9 is executed.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Pixel array, 2 ... Scan line drive part, 3 ... Signal line drive part, 4 ... Sensor control line drive part, 5 ... Sensor output part, 6 ... Control part, 61 ... Sensor control part, 62 ... Display control part, 63: correction unit, 7: storage unit, 8: light shielding film, P: pixel unit, Q1 to Q5: thin film transistor, C1: auxiliary capacitance, C2: capacitor, LC: liquid crystal

Claims (13)

A plurality of pixel portions each including an optical sensor element and a display element and having an equivalent structure;
A light shielding film that shields part of the plurality of pixel portions;
A reading unit that reads a sensor signal generated by an optical sensor element included in each of the plurality of pixel units;
A correction unit that corrects a sensor signal read from a pixel portion not shielded by the light shielding film based on a sensor signal read from the pixel portion shielded by the light shielding film. The correction unit reads from one sensor signal read from the light-shielded pixel unit or an average value of a plurality of sensor signals read from the plurality of light-shielded pixel units, and the non-light-shielded pixel unit. Calculate the difference from the sensor signal
The display device according to claim 1. The plurality of pixel portions are arranged in a matrix,
The light-shielding film shields at least a part of the pixel portion located at the periphery of the matrix;
The display device according to claim 1. The light shielding film shields the first pixel portion and the second pixel portion located at both ends of each row of the matrix,
The readout unit sequentially selects each row of the matrix, reads out sensor signals of pixel units belonging to the selected row,
When the reading unit reads a sensor signal from one row of the matrix in the reading unit, the correction unit reads a sensor signal read from a pixel unit that is not shielded in the one row to the pixel unit in the one row. Correction is performed based on a sensor signal read from either the first pixel unit or the second pixel unit that is close.
The display device according to claim 3. The light shielding film shields the first pixel portion and the second pixel portion located at both ends of each row of the matrix,
The readout unit sequentially selects each row of the matrix, reads out sensor signals of pixel units belonging to the selected row,
When a sensor signal is read from one row of the matrix in the reading unit, the correction unit, according to the sensor signal read from the first pixel unit and the second pixel unit belonging to the one row, The first correction function is determined with the position in the row direction in one row as a variable, and the sensor signal of the pixel portion that is not shielded from light in the row is determined according to the row direction position of the pixel portion. Correcting based on the value of the first correction function;
The display device according to claim 3. The light shielding film shields the first pixel portion and the second pixel portion located at both ends of each row of the matrix, and the third pixel portion and the fourth pixel portion located at both ends of each column of the matrix, respectively.
The correction unit reads a sensor signal read from one pixel unit that belongs to one row and one column of the matrix and is not shielded from light from the first pixel unit close to the one pixel unit in the one row. Sensor signal read from either one pixel portion or the first pixel portion, and read from either the third pixel portion or the fourth pixel portion close to the one pixel portion in the one column Correct based on the average value with the sensor signal,
The display device according to claim 3. The light shielding film shields the first pixel portion and the second pixel portion located at both ends of each row of the matrix, and the third pixel portion and the fourth pixel portion located at both ends of each column of the matrix, respectively.
The correction unit performs a first correction using a position in the row direction in the one row as a variable in accordance with sensor signals read from the first pixel unit and the second pixel unit belonging to one row of the matrix. A second correction function that determines a function and uses the position in the column direction in the one column as a variable in accordance with sensor signals of the third pixel unit and the fourth pixel unit belonging to one column of the matrix And the sensor signal read from one pixel portion that belongs to the one row and one column and is not shielded from light is determined according to the position in the row direction of the one pixel portion. Correction based on the average value of the value of the function and the value of the second correction function determined according to the position of the one pixel portion in the column direction,
The display device according to claim 3. A display control unit that sets the luminance of the display element included in the pixel unit shielded by the light-shielding film to a maximum value;
The display device according to claim 1. Display control for setting the luminance of the display element included in the pixel portion shielded by the light shielding film to the same value as the luminance of the display element adjacent to the pixel portion and not shielded by the light shielding film Having a part,
The display device according to claim 1. A display control unit that sets the luminance of the display element included in the pixel portion shielded by the light shielding film to an average value of the luminance of the display element included in the pixel portion not shielded by the light shielding film;
The display device according to claim 1. In a display device that includes a plurality of pixel portions each having an optical sensor element and a display element, and having a structure equivalent to each other, a method for correcting a sensor signal generated by the optical sensor element,
A first step of reading out sensor signals generated by the optical sensor elements of the plurality of pixel units;
Of the sensor signals of the plurality of pixel portions read out in the first step, the sensor signals of the pixel portions not shielded by the light shielding film are obtained based on the sensor signals of the pixel portions shielded by the light shielding film. A sensor signal correction method comprising: a second step of correcting. A method of correcting a sensor signal generated by an optical sensor element in a display device that includes an optical sensor element and a display element, each having an equivalent structure, and having a plurality of pixel portions arranged in a matrix. There,
A first step of sequentially selecting each row of the matrix and reading a sensor signal generated by a photosensor element of a pixel unit belonging to the selected row;
When the sensor signal is read from one row of the matrix in the first step, the sensor signal of the pixel portion that is not shielded by the light shielding film in the one row is converted to the sensor of the pixel portion that is shielded by the light shielding film. A sensor signal correction method comprising: a second step of correcting based on the signal. A plurality of pixel portions each including an optical sensor element and having an equivalent structure;
A light shielding film that shields part of the plurality of pixel portions;
A reading unit that reads a sensor signal generated by an optical sensor element included in each of the plurality of pixel units;
An image pickup apparatus comprising: a correction unit that corrects a sensor signal read from a pixel portion that is not shielded by the light shielding film based on a sensor signal read from the pixel portion shielded by the light shielding film.

Images