KR20190108374A - Image sensing apparatus and display apparatus using the same - Google Patents

Abstract

According to an embodiment of the present invention, an image sensing device comprises: an image sensor photographing a subject by varying the exposure time to generate and output at least one high-level exposure frame and at least one low-level exposure frame; and an optical deviation compensation unit compensating pixel values of the at least one high-level exposure frame and the at least one low-level exposure frame, and coupling the at least one high-level exposure frame and the at least one low-level exposure frame with the compensated pixel values so as to generate one final image frame.

Description

Image sensing device and display device using the same {IMAGE SENSING APPARATUS AND DISPLAY APPARATUS USING THE SAME}

The present invention relates to an image sensing device, and more particularly, to an image sensing device capable of more accurately sensing an image of a subject even when light from a light source is irradiated and reflected unevenly on the subject.

An image sensor is a device that converts an optical image into an electrical signal. Recently, with the development of the computer industry and the communication industry, the demand for improved image sensors in various fields such as digital cameras, camcorders, personal communication systems (PCS), gaming devices, security cameras, medical micro cameras, robots, etc. is increasing. have.

Image sensors are also used to recognize fingerprints.

An object of the present invention is to provide an image sensing device capable of more accurately sensing a subject even if the light from the light source is not uniformly irradiated on the surface of the subject and the amount of light reflected from the subject varies depending on the position in the subject.

An image sensing device according to an embodiment of the present invention includes an image sensor for photographing a subject at different exposure times to generate and output at least one high exposure frame and at least one low exposure frame; And correcting pixel values of the at least one high exposure frame and the at least one low exposure frame, and combining the at least one high exposure frame with the corrected pixel values and the at least one low exposure frame to form one final image frame. It may include a light deviation compensation unit to generate.

According to an embodiment of the present invention, a display apparatus for displaying an image image, a main body for supporting the display panel, and an image for sensing an image of the subject by sensing light reflected from a subject located above the display panel And a sensing device, wherein the image sensing device is positioned below the display panel and photographs the subject at different exposure times to generate and output at least one high exposure frame and at least one low exposure frame. ; And correcting pixel values of the at least one high exposure frame and the at least one low exposure frame, and combining the at least one high exposure frame with the corrected pixel values and the at least one low exposure frame to form one final image frame. It may include a light deviation compensation unit to generate.

An image sensing device according to another embodiment of the present invention includes an image sensor photographing a subject to generate and output a plurality of raw image frames; And an optical deviation compensator configured to correct each pixel of the plurality of raw frames, and generate a final image frame by summing a plurality of raw frames whose pixel values are corrected.

In accordance with another aspect of the present invention, a display apparatus includes a display panel configured to display an image image; A main body supporting the display panel; And an image sensing device configured to sense light reflected from a subject located above the display panel and sense an image of the subject, wherein the image sensing device photographs the subject to generate and output a plurality of raw image frames. sensor; And an optical deviation compensator configured to correct each pixel of the plurality of raw frames, and generate a final image frame by summing a plurality of raw frames whose pixel values are corrected.

Technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following descriptions.

According to the present invention, even if the light from the light source is not uniformly irradiated on the surface of the subject, the amount of light reflected from the subject varies depending on the position in the subject, thereby making it possible to more accurately sense the subject.

1 is a configuration diagram briefly showing the configuration of an image sensing device according to an embodiment of the present invention.
2 is a diagram illustrating an installation position of an image sensor in the case where the image sensing device of the present invention is applied to a display device.
3 is a view showing an example of the amount of light (pixel value) according to the distance from a light source in a high exposure frame and a low exposure frame.
4 is a view for explaining a method for frame correction.
5 is a circuit diagram illustrating a structure of a pixel according to an exemplary embodiment of the image sensor of FIG. 1.
6 is a diagram illustrating a timing diagram for the driving signals of FIG. 5.
7 illustrates a timing diagram for the driving signals of FIG. 5.
8 is a diagram illustrating a timing diagram for the driving signals of FIG. 5.
9 is a diagram illustrating a timing diagram for the driving signals of FIG. 5.
10 is a circuit diagram illustrating a structure of a pixel according to another embodiment of the image sensor of FIG. 1.
FIG. 11 is a timing diagram for describing an operation of an isolation transistor of FIG. 10. FIG.
12 illustrates the structure and operation of a photodiode according to an embodiment of the present invention.
FIG. 13 is a graph showing a relationship between an accumulation time and an output voltage in the structure of the photodiode of FIG. 12.
14 is a view showing a metal wall and a metal shielding structure according to an embodiment of the present invention.
15 is a schematic view showing the configuration of an image sensing device according to another embodiment of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail with reference to the accompanying drawings. In adding reference numerals to the components of each drawing, it should be noted that the same reference numerals are assigned to the same components as much as possible even though they are shown in different drawings. In addition, in describing the embodiments of the present invention, when it is determined that a detailed description of a related well-known configuration or function interferes with the understanding of the embodiments of the present invention, the detailed description thereof will be omitted.

1 is a configuration diagram briefly illustrating a configuration of an image sensing device according to an embodiment of the present invention.

The image sensing device of FIG. 1 may include an image sensor 100 and an optical deviation compensator 200.

The image sensor 100 detects the strength and weakness of the optical image reflected from the subject, and generates and outputs an image frame obtained by converting the intensity into digital image data. In particular, the image sensor 100 of the present embodiment integrates the exposure time (integration) for the same subject in order to eliminate the gradation effect due to the position of the light source (for example, an infrared LED light source) for irradiating light onto the subject (for example, a finger fingerprint). By photographing at different times, the raw input signal (high exposure frame FRAME_L) of high exposure and the low exposure frame (FRAME_S) of low exposure are generated and output. For example, the image sensor 100 generates a high exposure frame FRAME_L by photographing the subject by first increasing the exposure time while the light source is irradiating the subject, and then photographing the subject again by shortening the exposure time to reduce the low exposure frame. Create (FRAME_S). The image sensor 100 may be a monochrome image sensor for fingerprint sensing in which a color filter is not formed on the pixel array. In addition, the image sensor 100 of the present embodiment may be installed below the display panel so as to overlap the display panel (eg, the OLED panel of the mobile phone) of a specific device (eg, a smartphone).

The optical deviation compensator 200 corrects and combines the high exposure frame FRAME_L and the low exposure frame FRAME_S output from the image sensor 100 to generate the final image frame FRAME_COM. For example, the optical deviation compensator 200 may remove the high exposure frame such that the pixel value (light amount) of the high exposure frame FRAME_L and the low exposure frame FRAME_S is saturated to the highest or lowest value. The FRAME_L and the low exposure frame FRAME_S are corrected, and the corrected high exposure frame and the low exposure frame are combined to generate one image frame FRAME_COM with compensation for optical deviation.

FIG. 2 is a diagram illustrating an installation position of an image sensor when the image sensing device of the present invention is applied to a display device. Here, (a) is a view showing a cross-sectional view of the display device, (b) is a view showing a plan view of the display device. In addition, FIG. 3 is a diagram exemplarily illustrating the amount of light (pixel value) according to a distance from a light source in a high exposure frame and a low exposure frame.

2 and 3, a method of compensating for the optical deviation in the image sensing device according to an exemplary embodiment of the present invention will be described in more detail.

The display device of FIG. 2 may be, for example, a smartphone. The display device may include a main body 10, a display panel 20, a glass substrate 30, and an image sensor 100.

The main body 10 is a structure for supporting the display device. The display panel 20 displays an image image generated by driving of the display device, and has fine holes formed therein. Such display panel 20 may comprise an OLED panel. The glass substrate 30 is positioned above the display panel 20 to protect the display panel 20.

The image sensor 100 is used for sensing a fingerprint, and is disposed below the display panel 20 to overlap a portion of the display panel 20. The image sensor 100 detects light reflected from a fingerprint after being emitted from a light source (not shown) as high exposure and low exposure, and generates and outputs a high exposure frame and a low exposure frame, each of which is converted into digital image data. do.

In general, in the sensing device for fingerprint recognition, the light source is located at the rear side of the image sensor so that the light from the light source is evenly irradiated onto the fingerprint.

However, as in the present exemplary embodiment, when the image sensor 100 is disposed at the rear side of the display panel 20 to overlap the display panel 20, the light source for fingerprint recognition is used because of the display panel 20. It is difficult to arrange | position to the back side of the image sensor 100. FIG. Accordingly, the light source for fingerprint recognition may be disposed outside the display panel 20, for example.

In this case, the subject (fingerprint) is located where the image sensor 100 is installed on the upper side of the display panel 20. Since the light source is located outside the display panel 20, light from the light source is irradiated obliquely to the fingerprint. . As a result, much light is irradiated to the area close to the light source in the fingerprint, while less light is irradiated to the opposite area, and shadows may be generated due to such light deviation.

If the deviation of light is large, it is impossible to properly sense an image of an area where too much light is irradiated and an area where too little light is irradiated. Therefore, in the present embodiment, after photographing the same subject twice with high exposure and low exposure, the image is sensed using the low exposure image for the area to which the light is irradiated too much, and for the area to which the light is irradiated too low. A method of sensing an image using a high exposure image is used.

To this end, when a sensing command is applied, the image sensor 100 captures a different exposure time for the same subject (fingerprint) to capture a high exposure frame FRAME_L, which is a raw input signal of a high integration. A low exposure frame FRAME_S, which is a raw input signal of a short integration and a short exposure, is generated. That is, the image sensor 100 continuously photographs the same subject twice in high exposure and low exposure.

In this case, when the exposure is high, the amount of light flowing into the image sensor 100 increases as a whole, and too much light flows in the area close to the light source. On the other hand, during low exposure, the amount of light flowing into the image sensor 100 becomes smaller as a whole and too little light is introduced into the area far from the light source.

Accordingly, as shown in FIG. 3, in the high exposure frame, pixels corresponding to an area close to the light source have too much light so that pixel values are saturated to a predetermined maximum value. In addition, in the low exposure frame, pixels corresponding to an area far from the light source have too little light so that the pixel values are saturated to the lowest value.

In order to remove this saturation region, the optical deviation compensator 200 corrects respective pixel values of the high exposure frame FRAME_L and the low exposure frame FRAME_S. To this end, the optical deviation compensator 200 sets the pixel values of the high exposure frame FRAME_L and the low exposure frame FRAME_S to average values (average pixel values) of pixels within a predetermined range including the corresponding pixels in the corresponding pixel values. Correct by subtracting.

For example, the corrected pixel value {(Long_AC) i, j} for the (i, j) th pixel in the high exposure frame FRAME_L is a pixel value of the corresponding pixel (i, j) as shown in Equation 1 below. {(Long) i, j} may be obtained by subtracting the average value {Average (Long) i, j} of n × n (n is a natural number) pixels centering on the pixel (i, j). In this case, n may be “5” as shown in FIG. 4A, but is not limited thereto.

The optical deviation compensator 200 corrects the entire pixel values of the high exposure frame FRAME_L in this manner. In this case, when the pixel to be corrected is an outer pixel in the pixel array, for example, the outermost pixel as shown in FIG. The average can be calculated. That is, in FIG. 4B, an average value {Average (Long) i, j} for correcting the outermost pixels i and j may be an average value of 3 × 5 pixels.

In this case, pixel values of an area in which only pixels saturated to the highest value are collected in the high exposure frame FRAME_L are corrected to “0”. For example, if the (i, j) th pixel is saturated and all 5x5 pixels around it are also saturated, the value {(Long) i, j} of the (i, j) th pixel and 5 around it Since the average value {Average (Long) i, j} of the x5 pixels is the same, the corrected pixel value {(Long_AC) i, j} becomes "0".

In the case of a region containing many saturated pixels, each pixel in the region is corrected to a very small value.

The optical deviation compensator 200 corrects each pixel for the low exposure frame FRAME_S in the same manner. That is, the corrected pixel value {(Short_AC) i, j} for the (i, j) th pixel in the low exposure frame FRAME_S is the pixel value of the corresponding pixel (i, j) as shown in Equation 2 below. {(Short) i, j} may be obtained by subtracting the average value {Average (Short) i, j} of 5x5 pixels centered on the pixel (i, j).

As a result, in the low exposure frame FRAME_S, the pixel values of the region where the pixels saturated to the lowest value are collected are corrected to “0”.

However, as shown in FIG. 3, the region corrected to "0" in the high exposure frame FRAME_L has a value in the low exposure frame FRAME_S, and the region corrected to "0" in the low exposure frame FRAME_S is high. It has a value in the exposure frame FRAME_L.

Therefore, the optical deviation compensator 200 combines the corrected high exposure frame and the low exposure frame to generate one image frame FRAME_COM. In this case, the optical deviation compensator 200 may add an offset value of a predetermined size after combining the corrected high exposure frame and the low exposure frame.

That is, the final combined output value {(Combined output) i, j} may be expressed by Equation 3 below.

As a method of combining the high exposure frame and the low exposure frame, a method such as a method of generating a high dynamic range (HDR) image by combining the high exposure image frame and the low exposure image frame may be used.

 In the above-described embodiment, as a correction method, a method of subtracting an average value of pixels within a predetermined range from the value of the pixel to be corrected around the pixel is used, but another method may be used. For example, the corrected pixel value {(Long_AC) i, j or (Short_AC) i, j} for the (i, j) th pixel is the value of the correction target pixel (i, j) as shown in Equation 4 below. In {(Long) i, j or (Short) i, j} the smallest value {Min (Long) i, j or Min (Short) i, j} of the pixels within a certain range around the pixel Can be subtracted.

That is, when all pixels in a certain area are saturated, the pixel values in the corresponding area are the same, so even if the minimum value is subtracted, the corrected pixel value {(Long_AC) i, j or (Short_AC) i, j} is "0". do. However, when the minimum value is subtracted, a larger value can be obtained for an area where the correction value is not “0” compared with the case where the average value is subtracted, thereby maximizing the values of the ridge and valley of the fingerprint. There is an advantage to use.

Even in the case of the correction using the minimum value, the method of generating one image frame by combining the corrected high exposure frame and the low exposure frame may be performed in the same manner as in the case of using the above-described average value.

Although the optical deviation compensation method using the average value or the minimum value in the above-described embodiment has been described in the case of generating one final image frame by correcting and combining one high exposure frame and one low exposure frame, a plurality of high exposure frames are described. And a plurality of low exposure frames may be used to generate one final image frame.

Depending on the person, the status of the fingerprint may be different. Some fingerprints are worn out a lot. In this case, by photographing the same fingerprint continuously and adding the values together, it is possible to increase the amount of light difference between the ridge and the valley of the fingerprint to achieve more accurate sensing.

For example, the method of forming one image frame using the above-described high exposure frame and one low exposure frame may be repeatedly performed a plurality of times, and the results may be summed to generate one final image frame. In this case, the image sensor 100 may continuously generate and output a plurality of frame pairs in which one high exposure frame and one low exposure frame are paired. The optical deviation compensator 200 may correct and combine the high exposure frame and the low exposure frame for each frame pair according to the above-described method, and then generate one final image frame by summing the plurality of combined image frames.

Alternatively, a plurality of high exposure frames and low exposure frames may be generated and corrected and then combined together. In this case, the image sensor 100 continuously generates a plurality of high exposure frames and low exposure frames. The optical deviation compensator 200 corrects each of the plurality of high exposure frames and the low exposure frames according to the above-described correction method using the average value or the minimum value, and then, the plurality of high exposure frames and the low exposure pixels of which pixel values are corrected. The frames can be summed to produce one final image frame.

5 is a circuit diagram illustrating a structure of a pixel according to an exemplary embodiment of the image sensor of FIG. 1.

Referring to FIG. 5, the image sensor 100 includes a photodiode PD, a reset transistor P1, a transfer transistor P2, a floating diffusion region FD, a selection transistor P3, a conversion transistor N1, and a current source. (Is).

The photodiode PD is connected between ground and the node ND1, the reset transistor P1 is connected between the power supply voltage supply line VDD and the node ND1, and the transfer transistor P2 is connected to the node ND1 and the node ND2. The floating diffusion region FD is connected between the node ND2 and the ground. The selection transistor P3, the conversion transistor N1 and the current source Is are connected in series between the power supply voltage supply line VDD and the ground, and the gate of the conversion transistor N1 is connected to the node ND2. The voltage at the node ND3, which is connected to the conversion transistor N1 and the current source Is, becomes the output voltage Vout.

The photodiode PD detects an image signal by converting an optical signal into an electrical signal. The photodiode PD is an example of a photoelectric conversion element, and may be composed of at least one of a photo transistor, a photo gate, and a pinned photo diode (PPD).

The floating diffusion region FD accumulates charges generated in the photodiode PD. The floating diffusion region FD may include a junction capacitor Cj and an additional capacitor Cm, as shown in FIG. 5. Junction capacitor Cj represents a capacitor having a PN junction structure. The additional capacitor Cm is a capacitor additionally connected to the junction capacitor Cj. For example, the additional capacitor Cm may be a metal-insulator-metal (MIM) or a MOS capacitor.

In the case of the global shutter method, since a pixel corresponding to a row which is read out later, for example, a lower row, has to store charge in the floating diffusion region FD for a long time, the possibility of charge leakage increases. Accordingly, the output voltage of the pixel corresponding to the lower row may drop, causing gradation in the image or generating a fixed pattern noise (FPN). MIM capacitors can reduce gradation or FPN due to low charge leakage. In addition, the MIM capacitor can be used as a frame buffer because of low charge leakage.

The reset transistor P1 initializes the voltage of the photodiode PD, that is, the voltage of the node ND1 based on the reset signal RX /, and the voltage of the floating diffusion region FD together with the transfer transistor P2 described later, That is, the voltage of the node ND2 is initialized. In the present embodiment, the reset transistor P1 may be a PMOS transistor and may be turned on when the reset signal RX / is at a low level to reset the voltage of the photodiode PD to the power supply voltage VDD. Accordingly, since the gate-source voltage drop of the reset transistor P1 does not occur, the node ND1 can be reset to the power supply voltage VDD, thereby reducing the FPN.

The transfer transistor P2 connects the photodiode PD and the floating diffusion region FD based on the transfer control signal TX /. Accordingly, in the reset operation, the voltage of the node ND2 becomes equal to the voltage of the node ND1 reset to the power supply voltage VDD. In addition, charge sharing occurs between the photodiode PD and the floating diffusion region FD while charge accumulation is performed in the photodiode PD or after charge accumulation ends. In the present exemplary embodiment, the transfer transistor P2 may be a PMOS transistor, and may be turned on when the transfer control signal TX / is at a low level to transfer charges accumulated in the photodiode PD to the floating diffusion region FD. . In this embodiment, since the gate-source voltage drop of the transfer transistor P2 does not occur, the voltage at the node ND1 can be made the same as the voltage at the node ND2, thereby reducing the FPN.

The selection transistor P3 drives the conversion transistor N1 based on the selection control signal LS /. In the present embodiment, the selection transistor P3 may be a PMOS transistor, and similarly to the reset transistor P1 and the transfer transistor P2, the selection transistor P3 may reduce the FPN.

The conversion transistor N1 generates the output voltage Vout at the node ND3 according to the charge amount of the node ND2. The output voltage Vout may be output as an image signal in a correlated double sampling (CDS) unit.

6 is a diagram illustrating a timing diagram for the driving signals of FIG. 5.

In FIG. 6, the value in the right parenthesis of the driving signals RX /, TX /, LS / indicates a row of a pixel to which the driving signals RX /, TX /, LS / are applied. For example, RX / (n) represents the reset signal RX / applied to the pixel corresponding to the nth row, and RX / (n + 1) represents the reset applied to the pixel corresponding to the n + 1th row. Indicates the signal RX /. In addition, it is assumed that the reset signal RX /, the transfer control signal TX /, and the selection signal LS / according to the present embodiment are low enable signals that are enabled at the low level.

First, driving signals applied to the pixel corresponding to the nth row will be described.

5 and 6, the reset signal {RX / (n)} of the nth row and the transfer control signal TX / (n) of the nth row become low level during T0 to T1. As a result, the reset transistor P1 and the transfer transistor P2 are turned on, and the voltages of the nodes ND1 and ND2 are reset to the power supply voltage VDD.

In T2 to T3, the reset signal {RX / (n)} of the nth row and the transfer control signal TX / (n)} of the nth row become a low level again. After T1, since there is a time interval until T3 where charge accumulation occurs in the photodiode PD, variations in the voltage of the node ND1 or the node ND2 may occur. By performing the reset operation once more before the exposure time T3, it is possible to ensure that the voltages of the nodes ND1 and ND2 become the power supply voltage VDD.

T3 to T5 are integration times of the photodiode PD. Accordingly, charges due to photoelectric conversion are generated in the photodiode PD and accumulate inside the photodiode PD.

In this embodiment, the image sensor 100 may obtain a high exposure frame and a low exposure frame by controlling the exposure time by adjusting T3 or T3 and T2.

The exposure of the photodiode PD is terminated at T5, and the transfer control signal TX / (n) becomes low during T4 to T5. Accordingly, the transfer transistor P2 is turned on so that the charge accumulated in the photodiode PD is shared with the floating diffusion region FD.

At T5, the transfer control signal TX / (n) transitions to a high level, thereby ending charge sharing.

At T6, the selection signal LS / (n) is at a low level, whereby the selection transistor P3 and the conversion transistor N1 are driven to output the output voltage Vout. The output voltage Vout at this time is denoted as signal voltage Vsig.

The output voltage Vout is read out by the CDS at T7, and the output voltage Vout read out is denoted as the signal voltage Vsig (n).

During the period T8 to T9, the reset signal RX / (n) and the transfer control signal TX / (n) become low level, and reset the voltages of the nodes ND1 and ND2 to the power supply voltage VDD.

At T9, the reset signal RX / (n) and the transfer control signal TX / (n) transition to the high level to turn off the reset transistor P1 and the transfer transistor P2.

At T10, the output voltage Vout is read out by the CDS, and at this time, the output voltage Vout read out is referred to as the reference voltage Vref (n).

Although not shown, the CDS generates an image signal for the nth row based on the difference between the signal voltage Vsig (n) and the reference voltage Vref (n).

Next, driving signals applied to the pixel corresponding to the n + 1th row will be described.

The timing diagrams of the driving signals applied to the pixels corresponding to the n + 1th rows applied during T0 to T5 are the same as the timing diagrams of the driving signals applied to the pixels corresponding to the nth row.

5 and 6, reset signals {RX / (n + 1) of the n + 1th row and transfer control signals of the n + 1th row {TX / (n +) during T0 to T1 and T2 to T3. 1)} becomes a low level, and a reset operation is performed. The transfer control signal {TX / (n + 1)} becomes a low level during T4 to T5 to perform a charge sharing operation.

After the read operation for the nth row is terminated during T6 to T11, the select signal {LS / (n + 1)} becomes low at T12, and thus the select transistor of the pixel corresponding to the n + 1th row. P3 and the conversion transistor N1 are driven to output the output voltage Vout.

In T13, the output voltage Vout of the pixel corresponding to the n + 1th row is read out by the CDS, and the output voltage Vout at this time is denoted as Vsig (n + 1).

The reset signal {RX / (n + 1)} and the transfer control signal {TX / (n + 1)} of the pixel corresponding to the n + 1th row become the low level during the T14 to T15 voltages of the nodes ND1 and ND2. This power supply voltage is reset to VDD.

At T15, the reset signal {RX / (n + 1)} and the transfer control signal {TX / (n + 1)} are transitioned to a high level to reset the reset transistor P1 and the transfer transistor of the pixel corresponding to the n + 1th row. Turn off (P2).

The output voltage Vout is read out by the CDS at T16, and the output voltage at this time is denoted as the reference voltage Vref (n + 1).

The transition of the selection signal LS / (n + 1) to the high level at T17 ends the read operation for the pixel corresponding to the n + 1th row.

Referring to FIG. 6, in the image sensor 100 according to the exemplary embodiment of the present invention, the operation during T0 to T5, that is, the reset operation of the photodiode PD, the exposure operation of the photodiode, and the charge sharing operation are all low. Is performed simultaneously. The read operation for each row is sequentially performed. That is, a read operation on the n-th row is performed during T6 to T11, and a read operation on the n + 1-th row is performed during T12 to T17. In other words, the image sensor according to the present embodiment operates in a global shutter method.

In the case of the global shutter method, since the read operation is sequentially performed from the previous row to the subsequent row after the exposure operation is performed at the same time, leakage of charge occurs in the pixel corresponding to the later row, so that FPN and Gradients may occur. According to the embodiment of the present invention, since the MIM capacitor with less leakage of charge is used, the FPN can be reduced even when using the global shutter method.

FIG. 7 is a diagram illustrating a timing diagram for the driving signals of FIG. 5.

Referring to FIG. 7, the reset signals {RX / (n), RX / (n + 1)} and the transfer control signals {TX / (n), TX / (n + 1)} go low during T0 to T1. After the transition from T1 to the high level, the high level is maintained for the reset period. In the present embodiment, the operation during T2 to T3 is the same as the operation during T1 to T4, so the operation during T2 to T3 is omitted.

In this embodiment, the image sensor 100 may obtain a high exposure frame and a low exposure frame by controlling the exposure time by adjusting T1.

FIG. 8 is a diagram illustrating a timing diagram for driving signals of FIG. 5.

Referring to FIG. 8, the reset signal {RX / (n)}, the transfer control signal {TX / (n)}, and the pixel corresponding to the n + 1th row applied to the pixel corresponding to the nth row during T0 to T3. The reset signal {RX / (n)} and the transfer control signal TX / (n)} applied to the low level. That is, in this embodiment, the reset time increases from T0 to T3 as compared with the reset time from T0 to T1 in FIG. 7. In this case, the image sensor 100 may obtain a high exposure frame and a low exposure frame by controlling the exposure time by adjusting T3.

9 is a diagram illustrating a timing diagram for the driving signals of FIG. 5.

Referring to FIG. 9, operations of T0 to T3 are the same as in FIG. 9.

At T3, the reset signals {RX / (n), RX / (n + 1)} transition to a high level, and the transfer control signals {TX / (n), TX / (n + 1)} remain at a low level. . Accordingly, during the exposure time of T3 to T4, the transfer transistor P2 of the nth row and the n + 1th row is turned on, and charge sharing occurs between the photodiode PD and the floating diffusion region FD. . This is different from the exposure operation of the photodiode PD during T3 to T4 in FIG. 9 and the charge sharing during T4 to T5 thereafter. Although FIG. 10 illustrates that the transfer control signals TX / (n) and TX / (n + 1) are different when the reset operation of FIG. 9 is performed, the reset operation of FIGS. 7 and 8 is performed. This may also apply.

10 is a circuit diagram illustrating a structure of a pixel according to another exemplary embodiment of the image sensor of FIG. 1.

Referring to FIG. 10, the pixel further includes an isolation transistor N2 between the node ND1 and the node ND4 to which the photodiode PD is connected, as compared with the pixel of FIG. 5.

The gate of the isolation transistor N2 is connected to the power supply voltage supply line VDD. Since the isolation transistor N2 is connected between the photodiode PD and the node ND1, for example, the photodiode PD and the node ND1 are connected by metal lines, for example, the photodiode PD and the node ND1. Parasitic capacitance between ND1 can be reduced.

FIG. 11 is a timing diagram for describing an operation of the isolation transistor N2 of FIG. 10.

In FIG. 11, the reset signal RX / and the transfer control signal TX / are applied according to the embodiment of FIG. 9. In FIG. 11, V1 represents the voltage of the node ND1, and V4 represents the voltage of the node ND4.

10 and 11, a low level reset signal RX / and a transfer control signal TX / are applied before T3. Accordingly, the reset transistor P1 is turned on to reset the voltage V1 of the node ND1 to the power supply voltage VDD. When the gate-source voltage difference of the isolation transistor N2 is referred to as Vth, the voltage V4 of the node ND4 becomes VDD-Vth.

As the photodiode PD begins to be exposed at T3, charge is accumulated at the node ND4. At this time, the isolation transistor N2 operates in the saturation mode. Therefore, the charge accumulated in the source of the isolation transistor N2 moves to the drain of the isolation transistor N2 to reduce the voltage V1 of the node ND1.

When the voltage V1 of the node ND1 becomes VDD-Vth at Ta, and becomes equal to the voltage V4 of the node ND4, the isolation transistor N2 starts to operate in the linear mode. Therefore, the electric potentials of the node ND4 and the node ND1 decrease together with the charges accumulated in the node ND4.

Subsequent readout operations are the same as in FIG. 9 and will be omitted.

As such, even when the isolation transistor N2 is inserted, when the exposure time has elapsed (that is, after Ta), the voltage V1 of the node ND1 reflects the voltage of the photodiode PD, that is, the voltage V4 of the node ND4. On the other hand, parasitic capacitance may occur due to the reason that the node ND4 to which the photodiode PD is connected to the node ND1 is connected to the metal line. According to the present embodiment, the parasitic capacitance can be reduced by inserting the isolation transistor N2 between the node ND4 and the node ND1.

12 is a view showing the structure and operation of a photodiode according to an embodiment of the present invention. Here, (a) is a figure which shows the cross section of the photodiode PD of FIG. 5 or 10, (b) and (c) is a figure for demonstrating operation | movement of the photodiode PD.

Referring to FIG. 12A, the photodiode PD includes a P-type substrate 810, a first PDN region 820 and a second PDN formed on the P-type substrate 810. It may include a PDN region 830, a photo diode p-type (PDP) region 840, and a contact 850. The contact 850 may be connected to the node ND1 to which the reset transistor P1 and the transfer transistor P2 are connected through the metal line.

In the present embodiment, the second PDN region 830 may be a region having a higher doping concentration than the first PDN region 820, that is, an n + region. For example, the doping concentration of the second PDN region 830 may be 1E15, and the doping concentration of the first PDN region 820 may be 1E12. As such, the pin voltages of the first PDN region 820 and the second PDN region 830 may be set by adjusting the doping concentrations of the first PDN region 820 and the second PDN region 830. For example, the pin voltage of the first PDN region 820 may be lower than the power supply voltage, and the pin voltage of the second PDN region 830 may be greater than or equal to the power supply voltage VDD.

In this embodiment, the area of the second PDN area 830 may be smaller than the area of the first PDN area 820. For example, the photodiode PD may have an area of 50 μm × 50 μm, and the second PDN region 830 may have an area of 1 μm × 1 μm.

In FIGS. 12B and 12C, the pin voltage Vpin1 of the first PDN region 820 is 0.5V, the power supply voltage VDD is 3.0V, and the pin voltage Vpin2 of the second PDN region 830 is 3.0V or more. Assume that

When the power supply voltage VDD of 3.0V is applied to the contact 850 in the reset state, the voltage of the first PDN region 820 is 0.5V, the pin voltage Vpin1, and the voltage of the second PDN region 830 is the power supply voltage VDD. 3.0V.

The charge generated in the photodiode PD accumulates in the second PDN region 830 having a high voltage, as indicated by the gray region of FIG. 12B, and as a result, the voltage of the second PDN region 830 gradually increases. Lowers. After the voltage of the second PDN region 830 reaches the pin voltage Vpin1 of the first PDN region 820, that is, 0.5V, the generated charge is indicated by the gray region of FIG. 820 and the entire second PDN region 830.

As described above, in the period (1) in which the output voltage of the photodiode (PD) is VDD to Vpin1, it operates with a capacitance corresponding to the area of the second PDN region 830, and in the period (2) in the Vpin1 to 0V range, the first PDN It operates with a capacitance corresponding to the area of the region 820 and the second PDN region 830, that is, the entire area of the photodiode PD. Accordingly, the photodiode PD according to the present embodiment adjusts the pin voltage Vpin1 of the first PDN region 820 by adjusting the doping concentration of the first PDN region 820, and accordingly the capacitance of the photodiode PD. Can be adjusted.

FIG. 13 is a graph illustrating a relationship between an accumulation time and an output voltage in the structure of the photodiode of FIG. 12.

In FIG. 13, section 1 corresponds to section 1 of FIG. 12B, that is, a section in which the voltage of the photodiode is 3.0 to 0.5V, and section 2 in FIG. 13 corresponds to FIG. It corresponds to the section (2) of, i.e., the section where the voltage of the photodiode is 0.5 to 0V.

Referring to FIG. 13, the slope of the section 1 in which the output voltage of the photodiode PD is VDD to Vpin1 is greater than the slope of the section 2 of the output voltage of Vphoto1 to 0V of the photodiode PD. That is, it can be seen that the capacitance of the interval (1) is smaller than the capacitance of the interval (2).

14 illustrates a metal wall and a metal shielding structure according to an embodiment of the present invention.

Referring to FIG. 14, a pixel includes a reset transistor P1, a transfer transistor P2, a floating diffusion region FD, a selection transistor P3, a conversion transistor N1, a current source Is, a photodiode PD, Metal walls and metal shielding. In the pixel, the reset transistor P1, the transfer transistor P2, the floating diffusion region FD, the selection transistor P3, the conversion transistor N1, and the current source Is are the same as in FIG. 5, and the photodiode PD Since the structure is the same as in FIG. 12A, description thereof will be omitted.

The metal wall has a structure that surrounds the photodiode PD. This can block light coming from the side and improve optical crosstalk.

The metal shielding surrounds the side and the upper surface of the portion except for the photodiode PD, that is, the reset transistor P1, the transfer transistor P2, the selection transistor P3, the conversion transistor N1, and the floating diffusion region FD. Has a structure. Accordingly, even when the charge is stored in the floating diffusion region FD during the global shutter method, the junction region of the transistors P1, P2, P3, and N1 does not react to light, thereby preventing distortion of data. You can prevent it.

The pixel according to the present embodiment includes a metal line connecting the photodiode PD to the node ND1. In other words, the pixel has a structure in which the photodiode PD and the node ND1 are separated by metal lines. Accordingly, the metal wall and the metal shield may pass through the metal shielding sidewalls.

15 is a configuration diagram briefly illustrating a configuration of an image sensing device according to another embodiment of the present invention.

The image sensing device of FIG. 15 may include an image sensor 300 and a light deviation compensator 400.

The image sensor 300 detects the strength and weakness of the optical image reflected from the subject (fingerprint), and generates and outputs a raw image frame FRAME_SOURCE converted into digital image data. In this case, the image sensor 300 may continuously generate and output a plurality of raw image frames FRAME_SOURCE by performing continuous photographing on the same fingerprint. The image sensor 100 may be a monochrome image sensor for fingerprint sensing in which a color filter is not formed on the pixel array. In addition, the image sensor 300 may include the structure of FIG. 5 or 10 described above.

The optical deviation compensator 400 corrects each pixel of the raw image frame FRAME_SOURCE output from the image sensor 300 to generate a final image frame FRAME_F. In this case, the optical deviation corrector 400 may correct each pixel value of the raw image frame FRAME_SOURCE by subtracting an average value (average pixel value) or minimum value of pixels within a predetermined range including the pixel from the corresponding pixel value. Can be. For example, the optical deviation compensator 400 corrects each pixel value of the high exposure image frame or the low exposure image frame in the optical deviation compensator 200 of FIG. 1 as described above. Can be corrected. In particular, when the image sensor 300 continuously outputs a plurality of raw image frames FRAME_SOURCE, the optical deviation compensator 400 continuously corrects each raw image frame FRAME_SOURCE and then corrects the raw image frame. By summating these, one final image frame FRAME_F may be generated.

The image sensing device may also be applied to the display device of FIG. 2.

The above description is merely illustrative of the technical idea of the present invention, and those skilled in the art to which the present invention pertains may make various modifications and changes without departing from the essential characteristics of the present invention.

Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the present invention.

Claims (20)

An image sensor photographing a subject at different exposure times to generate and output at least one high exposure frame and at least one low exposure frame; And
Correcting pixel values of the at least one high exposure frame and the at least one low exposure frame, and combining the at least one high exposure frame with the corrected pixel values and the at least one low exposure frame to generate one final image frame An image sensing device comprising a light deviation compensation unit. The image sensor of claim 1, wherein the image sensor
To generate and output one high and one low exposure frame for the same subject,
The optical deviation compensator
For each pixel of the one high exposure frame and the one low exposure frame, correcting a value obtained by subtracting the average value or the minimum value of pixels within a predetermined range including the pixel from the value of the pixel to the value of the pixel. An image sensing device. The image sensor of claim 1, wherein the image sensor
For the same subject, frame pairs of one high exposure frame and one low exposure frame are continuously generated and output.
The optical deviation compensator
For each frame pair, for each pixel in the high and low exposure frames of that frame pair, the value of that pixel is subtracted from the value of that pixel minus the average or minimum of the pixels within a range that includes that pixel. Image sensing device, characterized in that for correcting. The method of claim 3, wherein the optical deviation compensation unit
And a corrected high exposure frame and a corrected low exposure frame for each frame pair, and add the combined image frames to generate the final image frame. The image sensor of claim 1, wherein the image sensor
Generates and outputs a plurality of high exposure frames and a plurality of low exposure frames,
The optical deviation compensator
For each pixel of the plurality of high exposure frames and the plurality of low exposure frames, correcting a value obtained by subtracting an average value or a minimum value of pixels within a predetermined range including the pixel from the value of the corresponding pixel to the value of the corresponding pixel. An image sensing device. The method of claim 5, wherein the image sensor
And combining and adding the corrected plurality of high exposure frames and the corrected plurality of low exposure frames to generate the final image frame. The optical deviation compensator according to any one of claims 2, 3 and 5, wherein the optical deviation compensator
And for each pixel of each high exposure frame and each low exposure frame, a mean value or a minimum value of n ㅧ n (n is a natural number) pixels centered on the pixel from the corresponding pixel value is subtracted. The method of claim 1, wherein the optical deviation compensation unit
And a predetermined offset value is added to the final image frame. A display panel displaying an image image;
A main body supporting the display panel; And
And an image sensing device configured to sense light reflected from a subject located above the display panel and sense an image of the subject.
The image sensing device
An image sensor positioned below the display panel and configured to photograph the subject at different exposure times to generate and output at least one high exposure frame and at least one low exposure frame; And
Correcting pixel values of the at least one high exposure frame and the at least one low exposure frame, and combining the at least one high exposure frame with the corrected pixel values and the at least one low exposure frame to generate one final image frame Display device including a light deviation compensation unit. The method of claim 9, wherein the image sensor
To generate and output one high and one low exposure frame for the same subject,
The optical deviation compensator
For each pixel of the one high exposure frame and the one low exposure frame, correcting a value obtained by subtracting the average value or the minimum value of pixels within a predetermined range including the pixel from the value of the pixel to the value of the pixel. Display device characterized in that. The method of claim 9, wherein the image sensor
For the same subject, frame pairs of one high exposure frame and one low exposure frame are continuously generated and output.
The optical deviation compensator
For each frame pair, for each pixel in the high and low exposure frames of that frame pair, the value of that pixel is subtracted from the value of that pixel minus the average or minimum of the pixels within a range that includes that pixel. Display device, characterized in that for correcting. The method of claim 11, wherein the optical deviation compensation unit
And a corrected high exposure frame and a corrected low exposure frame for each frame pair, and add the combined image frames to generate the final image frame. The method of claim 9, wherein the image sensor
Generates and outputs a plurality of high exposure frames and a plurality of low exposure frames,
The optical deviation compensator
For each pixel of the plurality of high exposure frames and the plurality of low exposure frames, correcting a value obtained by subtracting an average value or a minimum value of pixels within a predetermined range including the pixel from the value of the corresponding pixel to the value of the corresponding pixel. Display device characterized in that. The method of claim 13, wherein the image sensor
And a plurality of corrected high exposure frames and a corrected plurality of low exposure frames to generate the final image frame. The method according to any one of claims 10, 12 and 14,
The optical deviation compensator
And for each pixel of each high exposure frame and each low exposure frame, a mean value or a minimum value of n ㅧ n (n is a natural number) pixels centered on the pixel from the corresponding pixel value is subtracted. 10. The method of claim 9, wherein the optical deviation compensator
And a predetermined offset value is added to the final image frame. An image sensor photographing a subject to generate and output a plurality of raw image frames; And
And an optical deviation compensator configured to correct each pixel of the plurality of raw frames, and generate a final image frame by summing a plurality of raw frames whose pixel values are corrected. 18. The method of claim 17, wherein the optical deviation compensator
And for each pixel of each image frame, a value obtained by subtracting an average value or a minimum value of pixels within a predetermined range including the corresponding pixel from the value of the corresponding pixel to the value of the corresponding pixel. A display panel displaying an image image;
A main body supporting the display panel; And
And an image sensing device configured to sense light reflected from a subject located above the display panel and sense an image of the subject.
The image sensing device
An image sensor photographing a subject to generate and output a plurality of raw image frames; And
And an optical deviation compensator configured to correct each pixel of the plurality of raw frames, and generate a final image frame by summing a plurality of raw frames whose pixel values are corrected. 20. The method of claim 19, wherein the optical deviation compensator
And for each pixel of each image frame, a value obtained by subtracting an average value or a minimum value of pixels within a predetermined range including the pixel from the value of the corresponding pixel to the value of the corresponding pixel.

Images