KR102292136B1 - Image sensor including a pixel having photoelectric conversion elements and image processing device having the image sensor - Google Patents

Abstract

An image sensor according to an embodiment of the present invention includes a pixel array including pixels, and each of the pixels includes photoelectric conversion elements that operate independently to detect a phase difference for autofocus. The image sensor further includes an exposure time control circuit for independently controlling an exposure time of each of the photoelectric conversion elements included in each of the pixels.

Description

An image sensor including a pixel including photoelectric conversion elements, and an image processing apparatus including the same

An embodiment according to a concept of the present invention relates to an image sensor, and more particularly, to an image sensor capable of performing phase difference auto focus at the same time as color imaging, and a data processing system including the same.

PAF may also mean phase detection auto focus or phase difference auto focus.

In photography, dynamic range is the range between the maximum measurable light intensity and the minimum measurable light intensity. The degree of variation in light intensity depends on the device used as the capture device, which determines the overall performance of the dynamic range of the imaging sensor.

Wide dynamic range (WDR) is also referred to as high dynamic range (HDR). WDR technology increases the dynamic range of an imaging sensor either by physically increasing pixel performance or by digitally applying multiple exposure times to each pixel.

An ideal WDR sensor is a sensor with high full well capacity (FWC). FWC is defined as the maximum number of electrons in an incident signal that can be accommodated without saturation during readout. As the FWC increases, the dynamic range of the image sensor increases.

In a digital single lens reflex (SLR) camera, in order to reduce the space occupied by a phase difference auto-focus module, the camera sensor includes pixels capable of directly detecting the phase difference. Thus, a DSLR camera is capable of autofocus. This technology is being applied to mirrorless DSLRs.

A conventional pixel for detecting a phase difference shields a portion of a photodiode with a metal or the like, and detects only light incident to a portion not covered by the photodiode. The conventional method of detecting a phase difference using a occluded pixel and a non-occluded pixel, that is, two pixels, has a problem in that the quality of a color image deteriorates due to the two pixels operating irregularly.

The technical problem to be achieved by the present invention is to configure two or four photoelectric conversion elements for each pixel in order to solve the above-mentioned problem, that is, the problem of poor quality of a color image to detect a phase difference signal in the entire area. An image sensor and an image processing apparatus including the same are provided.

An image sensor according to an embodiment of the present invention includes a pixel array including pixels, wherein each of the pixels is independently controlled to detect a phase difference for auto-focus. include minors.

The image sensor further includes an exposure time control circuit for independently controlling an exposure time of each of the photoelectric conversion elements included in each of the pixels.

In some embodiments, when the pixels are arranged in rows, the exposure time control circuit may use a row address for any one of the rows, and the photoelectric conversion element included in each of the pixels Each of the exposure times can be controlled independently.

In some embodiments, when the pixels are arranged in rows, the exposure time control circuit may adjust the exposure time of each of the photoelectric conversion elements included in each of the pixels based on binning condition data. can be controlled independently.

The pixels include a first pixel including a first photoelectric conversion element and a second photoelectric conversion element, and a second pixel including a third photoelectric conversion element and a fourth photoelectric conversion element, wherein the image sensor controls the first control output a first control signal for controlling the first exposure time of the first photoelectric conversion element through a line, and a second control for controlling a second exposure time of the second photoelectric conversion element through a second control line output a signal, output a third control signal for controlling a third exposure time of the third photoelectric conversion element through a third control line, and output a fourth exposure of the fourth photoelectric conversion element through a fourth control line and an exposure time control circuit outputting a fourth control signal for controlling time, wherein the first exposure time, the second exposure time, the third exposure time, and the fourth exposure time are the exposure control circuit independently controlled by

In some embodiments, when the first pixel and the second pixel are arranged in the same row or in the same column, the first exposure time and the third exposure time are the same, and the second exposure time and the second exposure time are the same. The fourth exposure time may be the same, and the first exposure time may be longer than the second exposure time.

In some embodiments, when the first pixel and the second pixel are arranged in the same column, the first exposure time and the fourth exposure time are the same, and the second exposure time and the third exposure time are the same and the first exposure time may be longer than the second exposure time.

In some embodiments, the pixels include a first pixel including a first photoelectric conversion element and a second photoelectric conversion element, and a second pixel including a third photoelectric conversion element and a fourth photoelectric conversion element, and the first When the pixel and the second pixel are arranged in the same row, the first photoelectric conversion element and the second photoelectric conversion element share a first floating diffusion region through corresponding transfer gates, and the third photoelectric conversion element and the third photoelectric conversion element The fourth photoelectric conversion element shares a second floating diffusion region different from the first floating diffusion region through corresponding transfer gates.

In some embodiments, the pixels include a first pixel including a first photoelectric conversion element and a second photoelectric conversion element, and a second pixel including a third photoelectric conversion element and a fourth photoelectric conversion element, and the first When the pixel and the second pixel are arranged in the same column, the first pixel and the second pixel share one floating diffusion area.

A first deep trench isolation (DTI) structure is formed between two corresponding pixels among the pixels, and a second DTI structure is formed between two corresponding photoelectric conversion elements among the photoelectric conversion elements included in each of the pixels. is formed

Each of the pixels further includes a color filter formed over the photoelectric conversion elements included in each pixel, and a microlens formed over the color filter.

A data processing system according to an embodiment of the present invention includes an image sensor and a controller for controlling the operation of the image sensor. The image sensor includes a pixel array including pixels, each of which includes independently controlled photoelectric conversion elements to detect a phase difference for auto focus.

The image sensor further includes an exposure time control circuit for independently controlling an exposure time of each of the photoelectric conversion elements included in each of the pixels.

The photoelectric conversion elements included in each of the pixels include a first photoelectric conversion element controlled with a relatively long-exposure time and a second photoelectric conversion element controlled with a relatively short-exposure time, and a second photoelectric conversion element from among the pixels. The first photoelectric conversion element included in one pixel and the first photoelectric conversion element included in a second pixel disposed adjacent to the first pixel among the pixels are arranged in a diagonal direction.

The image sensor further includes an exposure time control circuit for independently controlling an exposure time of each of the photoelectric conversion elements included in each of the pixels, wherein the photoelectric conversion elements have a relatively long exposure time. a first photoelectric conversion element controlled and a second photoelectric conversion element controlled with a relatively short-exposure time, wherein the pixel signal generated by the first photoelectric conversion element included in each of the pixels controls the exposure time They are output in parallel according to the control of the circuit.

The image processing system includes an analog-to-digital converter that converts pixel signals output from the photoelectric conversion elements included in each of the pixels into digital signals, and a pre-image signal processor that generates color information from the digital signals. and a phase difference processing circuit generating phase difference data corresponding to the phase difference from the digital signals and compressing the generated data.

A first deep trench isolation (DTI) structure is formed between two corresponding pixels among the pixels, and a second DTI structure is formed between two corresponding photoelectric conversion elements among the photoelectric conversion elements included in each of the pixels. is formed

The image sensor according to an embodiment of the present invention includes a plurality of photoelectric conversion elements independently controlled for each pixel, and can independently control the exposure time or integration time of each of the plurality of photoelectric conversion elements included in each pixel. have.

The image sensor including a pixel array has an effect of uniformly detecting phase difference signals in the entire area of the pixel array using a plurality of photoelectric conversion elements included in each pixel disposed in the pixel array.

The image quality of the color image generated by the image sensor capable of processing uniformly detected phase difference signals is improved.

In addition, since the reliability of the phase difference signals detected by the image sensor is increased and the spatial resolution of the image sensor is increased, the autofocus performance of the image sensor is improved.

In order to more fully understand the drawings recited in the Detailed Description of the Invention, a detailed description of each drawing is provided.
1 shows a pixel array of an image sensor including a plurality of pixels.
FIG. 2 shows a portion of the pixel array shown in FIG. 1 .
3 shows a portion of the pixel array shown in FIG. 1 .
FIG. 4 shows a portion of the pixel array shown in FIG. 1 .
FIG. 5 shows a portion of the pixel array shown in FIG. 1 .
FIG. 6 shows a portion of the pixel array shown in FIG. 1 .
7 shows a cross-sectional view of pixels each comprising two photodiodes.
8 shows a cross-sectional view of a pixel comprising four photodiodes.
9 shows a circuit diagram of a pixel comprising two photodiodes, for example a PAF pixel.
10 shows a circuit diagram of a pixel comprising four photodiodes, for example a PAF pixel.
11 is a block diagram of an image sensor including the pixel array of FIG. 1 .
12 shows another block diagram of an image sensor including the pixel array of FIG. 1 .
13 is a block diagram of a data processing system including the pixel array shown in FIG. 1 according to an embodiment;
FIG. 14 is a schematic block diagram of the image signal processor shown in FIG. 13 .
FIG. 15 shows exposure times for one field and the amount of accumulated exposure light in the CMOS image sensor shown in FIG. 13 .
16 is an input/output luminance characteristic of a long-exposure image signal and a short-exposure image for explaining a process of combining a long-exposure image signal and a short-exposure image signal. Indicates input/output luminance characteristics of a short-exposure image signal.
17 is a block diagram for explaining a method of outputting color data and depth data in synchronization from output signals of a multi-diode PAF sensor.
18 is a conceptual diagram of transmission gate control lines required for each row to control operation of pixels according to an embodiment of the present invention.
19 shows a circuit diagram of pixels arranged in the same column.
FIG. 20 is a timing diagram for explaining the operation of the pixels shown in FIG. 19 .
21 illustrates an embodiment of an arrangement of pixels for wide dynamic range (WDR).
22 shows another embodiment of an arrangement of pixels for wide dynamic range (WDR).
23 is a conceptual diagram for explaining a method of reading out pixel signals according to an embodiment of the present invention.
FIG. 24 is a circuit diagram for explaining a method of reading out the pixel signals shown in FIG. 23 .
25 is a conceptual diagram for explaining an operation of a timing generator operating under normal operating conditions.
26 is a conceptual diagram for explaining the operation of the timing generator when binning in units of two rows.
27 is a conceptual diagram for explaining the operation of the timing generator when binning in units of three rows.
28 is a block diagram illustrating another embodiment of a data processing system including the pixels shown in FIG. 1 or 18 .
29 is a flowchart for explaining the operation of the data processing system shown in FIG.
30 is a block diagram illustrating another embodiment of a data processing system including the pixels illustrated in FIG. 1 or 18 .

Specific structural or functional descriptions of the embodiments according to the concept of the present invention disclosed in this specification are only exemplified for the purpose of explaining the embodiments according to the concept of the present invention, and the embodiments according to the concept of the present invention are It may be implemented in various forms and is not limited to the embodiments described herein.

Since the embodiments according to the concept of the present invention may have various changes and may have various forms, the embodiments will be illustrated in the drawings and described in detail herein. However, this is not intended to limit the embodiments according to the concept of the present invention to specific disclosed forms, and includes all modifications, equivalents, or substitutes included in the spirit and scope of the present invention.

Terms such as first or second may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one element from another, for example without departing from the scope of the inventive concept, a first element may be termed a second element and similarly a second element A component may also be referred to as a first component.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but it is understood that other components may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle. Other expressions describing the relationship between components, such as "between" and "immediately between" or "neighboring to" and "directly adjacent to", etc., should be interpreted similarly.

The terms used herein are used only to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present specification, terms such as "comprise" or "have" are intended to designate that a feature, number, step, operation, component, part, or combination thereof described herein exists, but one or more other features It is to be understood that it does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present specification. does not

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 shows a pixel array of an image sensor including a plurality of pixels. Each of the plurality of pixels R, G, and B included in the pixel array 100 may include a plurality of photodiodes.

The pixel array 100 may be included in a portable electronic device. The portable electronic device includes a laptop computer, a cellular phone or mobile phone, a smart phone, a tablet PC, a digital camera, a camcorder, and a mobile internet device. device (MID)), a wearable computer, an internet of things (IoT) device, or an internet of everything (IoE) device.

Each of the photodiodes included in the pixel array 100 is an example of a photoelectric conversion device, and each of the photodiodes is replaced with a phototransistor, a photogate, or a pinned-photodiode. can be

Each of the plurality of photodiodes included in each pixel may independently capture light or an image.

In FIG. 1 , R means a red pixel, G means a green pixel, and B means a blue pixel. A corresponding microlens may be formed on each of the pixels R, G, and B. The pixel array 100 may implement WDR or HDR without loss of resolution. The structure of each pixel R, G, and B will be described with reference to FIGS. 7 and 8 .

FIG. 2 shows a portion 110A of the pixel array 100 shown in FIG. 1 . Each pixel R, G, and B may include two photodiodes L and S that operate independently of each other.

In FIG. 2 , L denotes a first photodiode, and S denotes a second photodiode. For example, L may be a photodiode capable of generating a long-exposure image signal, and S may be a photodiode capable of generating a short-exposure image signal. It may be a diode (photodiode).

Each pixel G and R arranged in each row Row1 and Row3 includes two photodiodes L and S.

Each pixel B and G disposed in each row Row2 and Row4 includes two photodiodes L and S.

The exposure time or integration time of each photodiode L and S included in each pixel R, G, and B can be controlled differently and independently by the row driver. can

In FIG. 2, for convenience of explanation, each pixel R, G, and B is illustrated as including two photodiodes L and S implemented on the left and right, but according to an embodiment, each pixel R, G and B) may include two photodiodes L and S implemented above and below.

For example, a gate of a transfer transistor connected to each photodiode L of each pixel R, G, and B disposed in each row Row1 - Row4 is connected to the corresponding first transfer line (or the first transfer line). 1 metal line; a gate of a transfer transistor connected to each photodiode S of each pixel R, G, and B disposed in each row Row1-Row4, which may be connected to LINE1. may be connected to a corresponding second transmission line (or a second metal line; LINE2).

FIG. 3 shows a portion 110B of the pixel array 100 shown in FIG. 1 . Each pixel R. G, and B comprises two photodiodes L and S that operate independently of each other.

The positions of the two photodiodes L and S included in the rows Row3 and Row4 of FIG. 3 and the positions of the two photodiodes L and S included in the rows Row3 and Row4 of FIG. 2 . The positions are opposite to each other.

2 and 3 , the positions of the photodiodes L and S included in each pixel R, G, and B may be variously changed according to design specifications.

For example, a gate of a transfer transistor connected to each photodiode L of each pixel R, G, and B disposed in each row Row1 - Row4 is connected to the corresponding first transfer line (or the first transfer line). A gate of a transfer transistor that may be connected to one metal line: LINE1 and connected to each photodiode S of each pixel R, G, and B has a corresponding second transfer line (or second transfer line). It may be connected to a metal line (LINE2).

FIG. 4 shows a portion 120A of the pixel array 100 shown in FIG. 1 . Each pixel R. G, and B comprises four photodiodes L1, L2, S1, and S2 that operate independently of each other.

According to an embodiment, the exposure time or integration time of each photodiode L1 , L2 , S1 , and S2 included in each pixel R, G, and B may be determined by a row driver. ) can be independently controlled differently.

According to another embodiment, the exposure time or integration time of each photodiode L1 and L2 included in each pixel R, G, and B may be equally controlled by the row driver, and each pixel R , G, and B), the exposure time or integration of each of the photodiodes S1 and S2 may be equally controlled by the row driver.

The exposure time or integration of each of the photodiodes L1 and L2 may be set longer than the exposure time or integration time of each of the photodiodes S1 and S2.

The physical characteristics of each of the photodiodes L1 and L2 may be implemented to be the same or different from each other. In addition, the physical characteristics of each of the photodiodes S1 and S2 may be implemented to be the same or different from each other.

L1 denotes a first photodiode, S1 denotes a second photodiode, L2 denotes a third photodiode, and S2 denotes a fourth photodiode.

For example, each of L1 and L2 may be a photodiode capable of generating a long-exposure image signal, and each of S1 and S2 may be capable of generating a short-exposure image signal. It may be a capable photodiode.

Each pixel G and R disposed in the row Row1 includes four photodiodes L1 , L2 , S1 , and S2 . Each pixel B and G arranged in Row2 includes four photodiodes L1, L2, S1, and S2.

Each pixel R, G, and B comprises two photodiodes L1 and L2 capable of generating a long-exposure image signal and two photodiodes L1 and L2 capable of generating a short-exposure image signal. S1 and S2). In this case, the implementation positions of each of the photodiodes L1 , L2 , S1 , and S2 may be variously changed according to design specifications.

For example, the gate of each transfer transistor connected to each photodiode L1 and L2 of each pixel R, G, and B disposed in each row Row1 and Row2 is connected to a corresponding first transfer line (or first transfer line). The gate of each transfer transistor may be connected to a metal line LINE1 and connected to each photodiode S1 and S2 of each pixel R, G, and B is a corresponding second transfer line (or second metal line) ; can be connected to LINE2).

FIG. 5 shows a portion 120B of the pixel array 100 shown in FIG. 1 . Each pixel R. G, and B includes four photodiodes L1, L2, L3, and S1 that operate independently of each other.

That is, each pixel R, G, and B includes three photodiodes L1, L2, and L3 each capable of generating a long-exposure image signal and capable of generating a short-exposure image signal. Includes one photodiode (S1). In this case, the implementation positions of each of the photodiodes L1 , L2 , L3 , and S1 may be variously changed according to design specifications.

According to an embodiment, the exposure time or integration time of each photodiode L1 , L2 , L3 , and S1 included in each pixel R, G, and B may be differently and independently controlled by the row driver. .

According to another embodiment, the exposure time or integration time of each photodiode L1 , L2 , and L3 included in each pixel R, G, and B may be equally controlled by the row driver. The exposure time or integration time of each photodiode L1 , L2 , and L3 included in each pixel R, G, and B may be set longer than the exposure time or integration time of the photodiode S1 .

The physical characteristics of each photodiode L1 , L2 , and L3 may be implemented to be the same or different from each other.

L1 denotes a first photodiode, L2 denotes a second photodiode, L3 denotes a third photodiode, and S1 denotes a fourth photodiode.

For example, each of L1, L2, and L3 may be a photodiode capable of generating a long-exposure image signal, and S1 may be a photodiode capable of generating a short-exposure image signal.

Each pixel G and R disposed in the row Row1 includes four photodiodes L1 , L2 , L3 , and S1 . Each pixel B and G disposed in the row Row2 includes four photodiodes L1, L2, L3, and S1.

For example, as shown in Fig. 5, the gate of each transfer transistor connected to each photodiode L1, L2, and L3 of each pixel R, G, and B disposed in each row Row1 and Row2 is The gate of the transfer transistor which may be connected to the corresponding first transfer line (or the first metal line; LINE1) and which is connected to the photodiode S1 of each pixel R, G, and B is the corresponding second transfer line. (or a second metal line; LINE2).

FIG. 6 shows a portion 120C of the pixel array 100 shown in FIG. 1 . Each pixel R. G, and B includes four photodiodes S1, S2, S3, and L1 that operate independently of each other.

That is, each pixel R, G, and B comprises one photodiode L1 capable of generating a long-exposure image signal, and three photodiodes each capable of generating a short-exposure image signal. includes S1 , S2 , and S3 . In this case, the implementation positions of each of the photodiodes S1 , S2 , S3 , and L1 may be variously changed according to design specifications.

According to an embodiment, the exposure time or integration time of each photodiode S1 , S2 , S3 , and L1 included in each pixel R, G, and B may be differently and independently controlled by the row driver. .

According to another embodiment, the exposure time or integration time of each photodiode S1 , S2 , and S3 included in each pixel R, G, and B may be equally controlled by the row driver.

The physical characteristics of each photodiode S1 , S2 , and S3 may be implemented to be the same or different from each other.

S1 denotes a first photodiode, S2 denotes a second photodiode, S3 denotes a third photodiode, and L1 denotes a fourth photodiode. For example, L1 may be a photodiode capable of generating a long-exposure image signal, and each of S1, S2, and S3 may be a photodiode capable of generating a short-exposure image signal.

Each pixel G and R arranged in the row Row1 includes four photodiodes S1 , S2 , S3 , and L1 . Each pixel B and G disposed in Row2 includes four photodiodes S1 , S2 , S3 , and L1 .

For example, as shown in Fig. 6, the gate of each transfer transistor connected to each photodiode S1, S2, and S3 of each pixel R, G, and B disposed in each row Row1 and Row2 is The gate of the transfer transistor connected to the corresponding first transfer line (or first metal line; LINE1) and connected to the photodiode L1 of each pixel (R, G, and B) is connected to the corresponding second transfer line. (or a second metal line; LINE2).

7 shows a cross-sectional view of a pixel including two photodiodes PD1 and PD2. The pixel may refer to R, G, or B of FIG. 1 .

The photodiode PD1 or PD1' may be either a photodiode capable of generating a long-exposure image signal or a photodiode capable of generating a short-exposure image signal, and the photodiode PD2 or PD2' may be The other may be a photodiode capable of generating a long-exposure image signal and a photodiode capable of generating a short-exposure image signal.

Two photodiodes (PD1 and PD2, and PD1' and PD2') are formed in a silicon substrate, and deep trench isolation (DTI) is formed with the two photodiodes (PD1 and PD2, and PD1'). PD2'). For example, an in-pixel DTI is formed between the two photodiodes PD1 and PD2, and PD1' and PD2', and an inter-pixel DTI is formed between the pixels. can

In the circuit region formed between the two photodiodes PD1 and PD2, or PD1' and PD2' and the color filter, metal wiring, multi-layer wiring, or wiring layers ( wiring layers) may be formed. A lens buffer or a planarization layer may be formed between the microlens and the color filter.

8 shows a cross-sectional view of a pixel including four photodiodes PD1-PD4. The pixel may refer to R, G, or B of FIG. 1 .

4 and 8 , PD1 is any one of L1, S1, L2, and S2, PD2 is another one of L1, S1, L2, and S2, and PD3 is any one of L1, S1, L2, and S2. Another one, and PD4 may be the other one of L1, S1, L2, and S2.

5 and 8 , PD1 is any one of L1, L2, L3, and S1, PD2 is another one of L1, L2, L3, and S1, and PD3 is any one of L1, L2, L3, and S1. Another one, and PD4 may be the other one of L1, L2, L3, and S1.

6 and 8 , PD1 is any one of S1, S2, S3, and L1, PD2 is another one of S1, S2, S3, and L1, and PD3 is any one of S1, S2, S3, and L1. Another one, and PD4 may be the other one of S1, S2, S3, and L2.

Four photodiodes PD1-PD4 are formed inside a silicon substrate, and a corresponding DTI, for example, between two corresponding photodiodes PD1 and PD2, PD2 and PD3, and PD3 and PD4. , an in-pixel DTI may be formed. An inter-pixel DTI may be formed between the pixels.

Metal wiring, multi-layer wiring, or wiring layers may be formed in the circuit region formed between the four photodiodes PD1-PD4 and the color filter. have. A lens buffer or a planarization layer may be formed between the microlens and the color filter.

9 shows a circuit diagram of a pixel including two photodiodes PD1 and PD2, for example a PAF pixel. 2, 3, 7, and 9 , a pixel includes two photodiodes PD1 and PD2, two transfer transistors TX1 and TX2, a reset transistor RX, and a source. a source follower (SF), and a selection transistor (SX).

Each of the control signals TG1, TG2, RS, and SEL capable of controlling each of the transistors TX1, TX2, RX, and SX may be output from the row driver. The output signal of the selection transistor SX is supplied to the column line.

In FIG. 9 , pixels in which a floating diffusion region (FD) is shared are shown for convenience of explanation. However, according to the designer's intention, pixels that distinguish between long-exposure and short-exposure can be divided into one floating diffusion region. It may not be shared by each photodiode PD1 and PD2 in the region FD.

10 shows a circuit diagram of a pixel including four photodiodes PD1-PD4, for example a PAF pixel. 4, 5, 6, 8, and 10 , a pixel includes four photodiodes PD1-PD4, four transfer transistors TX1 to TX4, a reset transistor RX, and a source. a follower SF, and a selection transistor SX.

Each of the control signals TG1 to TG4, RS, and SEL capable of controlling each of the transistors TX1-TX2, RX, and SX may be output from the row driver. The output signal of the selection transistor SX is supplied to the column line,

In FIG. 10 , pixels in which the floating diffusion region FD is shared are illustrated for convenience of explanation. However, according to the intention of the designer, pixels that distinguish between long-exposure and short-exposure are defined as one floating diffusion region FD. may not be shared by each photodiode PD1-PD4.

11 is a block diagram of an image sensor including the pixel array of FIG. 1 .

The structure of each PAF pixel P included in the pixel array is substantially the same as that of the pixel described with reference to FIGS. 2 to 8 . PAF pixel P stands for R, G, or B.

The output signal of each PAF pixel P implemented in the odd-numbered rows Row1, Row3, ... is transmitted to the bottom analog-digital converter. The digital signals output from the bottom analog-to-digital converter may be stored in a corresponding memory (or buffer).

The output signal of each PAF pixel P implemented in the even-numbered rows Row2, Row4, ... is transmitted to an upper analog-to-digital converter. Digital signals output from the upper analog-to-digital converter may be stored in a corresponding memory (or buffer).

As shown in FIG. 11 , when each pixel P includes a plurality of photodiodes, N (N is 2) capable of controlling the exposure time or integration time of the plurality of photodiodes included in each pixel P N transmission lines (N lines) capable of transmitting more than a natural number) of control signals may be implemented.

12 shows another block diagram of an image sensor including the pixel array of FIG. 1 .

The structure of each PAF pixel P included in the pixel array is substantially the same as that of the pixel described with reference to FIGS. 2 to 8 . PAF pixel P stands for R, G, or B.

The output signal of each PAF pixel P implemented in the odd-numbered rows Row1, Row3, ... is transmitted to a first analog-digital converter. The digital signals output from the first analog-to-digital converter may be stored in a corresponding memory (or buffer). The memory (or buffer) may output image data.

The output signal of each PAF pixel P implemented in the even-numbered rows Row2, Row4, ... is transmitted to a second analog-digital converter. The digital signals output from the second analog-to-digital converter may be stored in a corresponding memory (or buffer). The memory (or buffer) may output image data.

As shown in FIG. 12 , when each pixel P includes a plurality of photodiodes, N (N is 2) capable of controlling the exposure time or integration time of the plurality of photodiodes implemented in each pixel P N transmission lines (N lines) capable of transmitting more than a natural number) of control signals may be implemented.

13 is a block diagram of a data processing system 500 including the pixel array 100 shown in FIG. 1 according to an embodiment.

1 to 10 and 16 , the data processing system 500 may be implemented as the aforementioned portable electronic device.

The data processing system 500 includes an optical lens 503 , a CMOS image sensor 505 , a digital signal processor (DSP) 600 , and a display 640 .

The CMOS image sensor 505 may generate image data IDATA of the subject 501 incident through the optical lens 503 . The image data IDATA is data corresponding to pixel signals output from the plurality of photodiodes P.

The CMOS image sensor 505 includes a pixel array 100 , a row driver 520 , a readout circuit 525 , a timing generator 530 , a control register block 550 , a reference signal generator 560 , and a buffer 570 . ) is included.

The pixel array 100 includes a plurality of pixels P. The pixel P of the CMOS image sensor 505 may be manufactured using a CMOS manufacturing process. As described with reference to FIGS. 1 to 10 , each of the plurality of pixels P may include photodiodes.

The pixel array 100 includes pixels P arranged in a matrix form. The pixels P transmit pixel signals to column lines.

The row driver 520 drives control signals for controlling the operation of each of the pixels P to the pixel array 100 according to the control of the timing generator 530 .

The row driver 520 may perform a function of a control signal generator capable of generating control signals. For example, the control signals include the control signals RS, TG1, TG2, and SEL shown in FIG. 9 or include the control signals RS, TG1 to TG4, and SEL shown in FIG. 10 .

The timing generator 530 controls operations of the row driver 520 , the readout circuit 525 , and the reference signal generator 560 according to the control of the control register block 550 .

The readout circuit 525 includes an analog-to-digital converter 526 for each column and a memory 527 for each column. According to an embodiment, the analog-to-digital converter 526 may perform a function of correlated double sampling (CDS).

The readout circuit 525 outputs a digital image signal corresponding to the pixel signal output from each pixel P.

The control register block 550 controls operations of the timing generator 530 , the reference signal generator 560 , and the buffer 570 under the control of the DSP 600 .

The buffer 570 transmits image data IDATA corresponding to the plurality of digital image signals output from the readout circuit 525 to the DSP 600 . The image data IDATA includes first image data corresponding to long-exposure image signals and second image data corresponding to short-exposure image signals.

The DSP 600 includes an image signal processor (ISP) 610 , a sensor controller 620 , and an interface 630 .

The ISP 610 controls the sensor controller 620 that controls the control register block 550 , and the interface 630 .

According to an embodiment, the CMOS image sensor 505 and the DSP 600 may be implemented in one package, for example, a multi-chip package (MCP).

In FIG. 13 , the CMOS image sensor 505 and the ISP 610 are illustrated in a separate form, but the ISP 610 may be implemented as a part of the CMOS image sensor 505 .

The ISP 610 processes the image data IDATA transmitted from the buffer 570 and transmits the processed image data to the interface 630 . For example, the ISP 610 may interpolate the image data IDATA corresponding to the pixel signals output from the pixels P and generate interpolated image data.

The sensor controller 620 may generate various control signals for controlling the control register block 550 according to the control of the ISP 610 .

The interface 630 may transmit image data processed by the ISP 610 , for example, interpolated image data to the display 640 .

The display 640 may display the interpolated image data output from the interface 630 . The display 640 may be implemented as a thin film transistor-liquid crystal display (TFT-LCD), a light emitting diode (LED) display, an organic LED (OLED) display, an active-matrix OLED (AMOLED) display, or a flexible display. .

14 is a schematic block diagram of the image signal processor shown in FIG. 13 , and FIG. 15 is exposure times and accumulated exposure light during one field in the CMOS image sensor of FIG. 13 . 16 shows the input/output luminance characteristics of the long-exposure image signal for explaining the combining process of the long-exposure image signal and the short-exposure image signal. Indicates input/output luminance characteristics of a short-exposure image signal.

Referring to FIG. 14 , the ISP 610 of FIG. 13 may include a reconstruction circuit 200 and a dynamic range compression circuit 220 . An operation method of the reconfiguration circuit 200 will be described with reference to FIGS. 15 and 16 .

First, referring to FIG. 15A , T2 seconds-long exposure and T3 seconds short-exposure formed according to a field period of T1 seconds (eg, 1/60 seconds) (T3 seconds short-exposure) is performed. Depending on the embodiment, the long-exposure time and the short-exposure time may vary.

In order to perform long-exposure and short-exposure, a long-exposure image signal and a short-exposure image signal are obtained depending on the number of rows in one field period. In order to combine the long-exposure image signal and the short-exposure image signal, the captured image data is generated depending on the number of rows in one field.

Combining the long-exposure image signal and the short-exposure image signal may be performed in the reconstruction circuit 200 of FIG. 14 . The combining process in the reconfiguration circuit 200 may be described with reference to FIG. 16 .

According to an embodiment, the input image data INPUT may include long-exposure image signals corresponding to the first image data and short-exposure image signals corresponding to the second image data.

In the combining process by the reconstruction circuit 200, the combined signal or combined image is converted to signals (or images) at the switching points indicated by the luminance threshold indicated by the dashed line. ) can be created by switching

For example, a corresponding long-exposure image signal is applied to a pixel signal having a luminance level lower than the luminance level of the switching point, and a corresponding short-exposure image signal is applied to a pixel signal having a luminance level higher than the luminance level of the switching point. do.

Level matching between two images is performed by multiplying a short-exposure image signal by an exposure ratio or gain. For example, the exposure ratio or gain may be determined according to the ratio of the long-exposure image signal to the short-exposure image signal.

When the exposure ratio between the long-exposure image signal and the short-exposure image signal is K:1, the exposure of the short-exposure image signal is 1/K, the exposure of the long-exposure image signal is 1/K. The luminance level of the long-exposure image signal is K times greater than the luminance level of the short-exposure image signal. Thus, multiplying the short-exposure image signal by a gain K can match the two levels.

In this way, the short-exposure image signal is multiplied by a factor of K. As a result, a combined image having the characteristics of the Long Exposure Signal and the characteristics of the Combined Signal is generated.

That is, the reconstruction circuit 200 combines the input image data INPUT as described with reference to FIG. 16 , and outputs the combined image OUTPUT1 . The reconstruction circuit 200 linearly converts a short-exposure image signal obtained through short-exposure (ie, a short-exposure image) and a long-exposure image signal obtained through a long-exposure (ie, a long-exposure image) can perform the function of summing with .

The reconstruction circuit 200 generates a linear image OUTPU1 by multiplying the short-exposure image by the exposure ratio, and then linearly sums the image generated as a result of the multiplication and the long-exposure image. For example, the first image data corresponding to the long-exposure image signals is M-bits (eg, 14-bits) and the second image data corresponding to the short-exposure image signals is M-bits (eg, 14-bits) ), the first image data and the second image data overlap in a certain section, and the overlapped combined image OUTPUT1 becomes smaller than 2*M bits. For example, the overlapped combined image OUTPUT1 may be 14-bits. Here, the number of bits means the number of bits of image data corresponding to each pixel signal output from each pixel.

The dynamic range compression circuit 220 lowers the number of bits (eg, 14-bits) of the overlapped combined image OUTPUT1 to a bit (eg, 10-bits) that meets the display or output standard, and has the lowered number of bits Output the image (OUTPUT2). For example, the dynamic range compression circuit 220 reduces the number of bits of the combined image OUTPUT1 by using a curve such as gamma implemented by a local method or a global method and reduces the number of bits. Output the output image (OUTPUT2) with For example, the dynamic range compression circuit 220 may perform a function of compressing the dynamic range of the combined image OUTPUT1 .

FIG. 15(b) is for explaining a rolling shutter method. In FIG. 15(b) , long-exposure and short-exposure may be overlapped. T2 and T3 may depend on the number of rows.

17 is a block diagram for explaining a method of outputting color data and depth data from output signals of a multi-diode PAF sensor in synchronization. The circuit shown in FIG. 17 may be included as part of the CMOS image sensor 505 of FIG. 13 .

The output signal of the multi-photodiode PAF sensor 300 may mean a phase difference signal, that is, a signal output from a plurality of photoelectric conversion elements (eg, photodiodes) included in a pixel.

The color data processing circuit 310 may perform image data enhancement processing using signals LDATA and SDATA output from the photoelectric conversion elements. The processing may include pre-processing. The pre-processing may correct a problem occurring in the manufacturing process of the CMOS image sensor before image processing the main color (eg, RGB data). The correction may include lens shading correction and/or bad pixel correction and/or the like.

The processing may include main color image processing. The main color image processing may include interpolation, noise reduction, edge enhancement, color correction, and/or gamma processing.

The color data processing circuit 310 may perform at least one of pre-processing and main color image processing.

The PAF data processing circuit 320 performs processing for depth data enhancement in units of PAF pixels. The PAF data processing circuit 320 refers to phase difference auto focus data processing, and uses the signals LDATA and SDATA output from the photodiodes included in each pixel to obtain disparity data ( It may perform a function of converting disparity data or depth data. For example, the disparity data may refer to data about an image of one point acquired through multi-diodes. The PAF data processing circuit 320 may perform a series of processing, such as noise reduction, to obtain disparity data or depth data.

Each of the circuits 310 and 320 may be implemented in a pipeline structure.

The color data processed by the color data processing circuit 310 and the depth data processed by the PAF data processing circuit 320 may be output in synchronization with each other or sequentially output in a line according to embodiments. can

According to an embodiment, when the color data and the depth data are synchronized with each other, the color data and the depth data may be output in synchronization with each other. According to another embodiment, after color data is output, depth data is output, or after depth data is output, color data is output.

According to another embodiment, color data and depth data may be alternately output. According to another embodiment, a method of mixing and outputting color data and depth data may be used.

According to embodiments, when the multi-diode PAF sensor 300 is implemented in the first chip, the color data processing circuit 310 and the PAF data processing circuit 320 may be implemented in the second chip. For example, the color data processing circuit 310 and the PAF data processing circuit 320 may be implemented in an ISP or may be implemented in an application processor or a system on chip (SoC). The circuit illustrated in FIG. 17 may output depth map data using full PAF pixels.

18 is a conceptual diagram of transfer gate control lines required for each row to control operation of pixels according to an embodiment of the present invention, and FIG. 19 is a circuit diagram of pixels arranged in the same column.

1, 7, 18, and 19 , each pixel R, G, and B may include two photoelectric conversion elements. For example, the first green pixel may include two photoelectric conversion elements GrPD1 and GrPD2, the red pixel may include two photoelectric conversion elements RPD1 and RPD2, and the blue pixel may include two photoelectric conversion elements. The conversion elements BPD1 and BPD2 may be included, and the second green pixel may include two photoelectric conversion elements GbPD1 and GbPD2.

Each pixel R, G, and B described herein means a PAF pixel capable of performing a phase detection auto focus operation or a phase difference auto focus operation.

As shown in FIG. 18 , four transfer gate control lines may be disposed per row.

The first control signal TA1 may control the transfer gate TXa1 connected to the photoelectric conversion element GrPD1 , and the second control signal TA2 may control the transfer gate TXa2 connected to the photoelectric conversion element GrPD2 . , the third control signal TA3 may control a transfer gate connected to the photoelectric conversion element RPD1 , and the fourth control signal TA4 may control a transfer gate connected to the photoelectric conversion element RPD2 . can control The exposure time control circuit, for example, the control signals TA1 to TA4 output from the row driver 520 of FIG. 13 independently sets the exposure time of each of the two photoelectric conversion elements included in each of the pixels arranged in the first row. can be controlled with

The fifth control signal TB1 may control the transfer gate TXb1 connected to the photoelectric conversion element BPD1 , and the sixth control signal TB2 may be the transfer gate TXb2 connected to the photoelectric conversion element BPD2 . , the seventh control signal TB3 may control a transfer gate connected to the photoelectric conversion element GrPD1 , and the eighth control signal TB4 may control a transfer gate connected to the photoelectric conversion element GrPD2 . can control The exposure time control circuit, for example, the control signals TB1 to TB4 output from the row driver 520 of FIG. 13 independently sets the exposure time of each of the two photoelectric conversion elements included in each of the pixels arranged in the second row. can be controlled with

Charge accumulation and charge transfer of each of the plurality of photoelectric conversion elements included in each pixel may be controlled according to a corresponding control signal supplied to a gate of a corresponding transfer transistor.

In example embodiments, the first control signal TA1 may be supplied to the transfer gate connected to the photoelectric conversion element GrPD2 and the second control signal TA2 may be supplied to the transfer gate connected to the photoelectric conversion element GrPD1 It is possible to change the design of the pixel array so that it can be In addition, the third control signal TA3 may be supplied to the transfer gate connected to the photoelectric conversion element RPD2 and the fourth control signal TA4 may be supplied to the transfer gate connected to the photoelectric conversion element RPD1. Design changes to the pixel array are possible.

19 , assuming that pixel A means a first green pixel disposed in a first row and pixel B is a blue pixel disposed in a second row, the first green pixel disposed in the same column and the blue pixel may share a floating diffusion region (or a floating diffusion node; FD). That is, the four photoelectric conversion elements GrPD1 , GrPD2 , BPD1 , and BPD2 may share the floating diffusion region FD through four transfer gates.

FIG. 20 is a timing diagram for explaining the operation of the pixels shown in FIG. 19 .

18 to 20 , in one horizontal section, two shutter sections STX1 and STX2 exist and one readout section READ exists. In the first shutter period STX1 , the first photoelectric conversion element GrPD1 of the pixel A disposed in the first row corresponding to the address ADD1 is reset through the transfer gate TXa1 and the reset transistor RX. That is, the first control signal TA1 is supplied to the gate of the transfer transistor TXa1 and the reset signal RST is supplied to the gate of the reset transistor RX.

In the first shutter period STX1 , a reset operation is not performed on the corresponding photoelectric conversion element.

In the read-out period READ, the floating diffusion region of the pixel disposed in the row corresponding to the address ADD3 different from the address ADD1 is reset. Thereafter, the second control signal TA2 is supplied to the gate of the transfer transistor TXa2 connected to the second photoelectric conversion element GrPD2 of the pixel A arranged in the first row. Accordingly, the charges accumulated in the second photoelectric conversion element GrPD2 are transferred to the floating diffusion region FD through the transfer gate TXa2, and in response to the charges transferred to the floating diffusion region FD, the source follower SF ) operates, and the signal output from the source follower SF is transmitted to the column line through the selection transistor SX.

VPIX refers to the operating voltage supplied to the reset transistor (RX) and the source follower (SF).

21 illustrates an embodiment of an arrangement of pixels for wide dynamic range (WDR). 18, 19, and 21 , L denotes a photoelectric conversion element capable of generating a long-exposure image signal, and S denotes a photoelectric conversion element capable of generating a short-exposure image signal .

Each control signal TA1 , TA3 , TB1 , and TB3 supplied to the gate of each transfer transistor connected to each photoelectric conversion element GrPD1 , RPD1 , BPD1 , and GbPD1 corresponds to a relatively long-exposure time. However, each control signal TA2, TA4, TB2, and TB4 supplied to the gate of each transfer transistor connected to each photoelectric conversion element GrPD2, RPD2, BPD2, and GbPD2 corresponds to a relatively short-exposure time. .

22 shows another embodiment of an arrangement of pixels for wide dynamic range (WDR). 18, 19, and 22 , L denotes a photoelectric conversion element capable of generating a long-exposure image signal, and S denotes a photoelectric conversion element capable of generating a short-exposure image signal .

Each control signal TA1 , TA3 , TB1 , and TB3 supplied to the gate of each transfer transistor connected to each photoelectric conversion element GrPD1 , RPD1 , BPD2 , and GbPD2 corresponds to a relatively long-exposure time. However, each control signal TA2, TA4, TB2, and TB4 supplied to the gate of each transfer transistor connected to each of the photoelectric conversion elements GrPD2, RPD2, BPD1, and GbPD1 corresponds to a relatively short-exposure time. .

In the case of binning, pixel signals corresponding to charges output from the photoelectric conversion devices corresponding to the same exposure-time must be exhausted, so that the row driver 520 controls each of the control signals TA1 to TA4 to enable the binning. , and TB1 to TB4) should be controlled. Accordingly, the row driver 520 needs to control the respective control signals TA1 to TA4 and TB1 to TB4 related to the exposure time according to the performance of the PAF and/or operating conditions such as binning.

18 to 22, whether the control signal supplied to the gate of the transfer transistor connected to the photoelectric conversion element is a signal corresponding to a relatively long-exposure time or a signal corresponding to a relatively short-exposure time There is an effect that pixels arranged in different spatially different positions according to perception can be controlled with different exposure times.

Here, L and S are relative, and when a control signal corresponding to a relatively short-exposure time is supplied to the gate of the transfer transistor connected to L, the L will generate charges corresponding to the relatively short-exposure image signal. can In addition, when a control signal corresponding to a relatively long-exposure time is supplied to the gate of the transfer transistor connected to S, the S may generate charges corresponding to the relatively long-exposure image signal.

As described with reference to FIGS. 18 to 22 , since the control signal corresponding to the long-exposure time is supplied to only one of the two photoelectric conversion elements included in each pixel, the pixels described with reference to FIGS. 18 to 22 are Since the Nyquist spatial frequency corresponding to the pixel signals output from The resolution of the image sensor including pixels described with reference to FIGS. 18 to 22 is the same as the resolution of the image sensor including Bayer patterns.

23 is a conceptual diagram illustrating a method of reading out pixel signals according to an embodiment of the present invention, and FIG. 24 is a circuit diagram illustrating a method of reading out the pixel signals shown in FIG. 23 .

For convenience of description, it is assumed that image signals are output in the order of (1) to (4) view.

(1) When the control signals output from the row driver 520 are supplied to the gates of the transfer transistors connected to the photoelectric conversion elements arranged at different positions at a time point, the pixels including the photoelectric conversion elements are displayed in the corresponding column. Pixel signals may be output to the comparators CP1 to CP4 in parallel through the lines COL1 to COL4.

Each of the comparators CP1 to CP4 implemented in the readout circuit 525A according to the embodiment of the readout circuit 525 of FIG. 13 compares the ramp signal VRAMP and the pixel signals, respectively, and compares each of the comparison signals. can be printed out. The ramp signal VRAMP may be output from a reference signal generator, for example, the ramp signal generator 560 .

For example, (1) long-exposure green image signals from pixels drawn at the time may be output simultaneously or in parallel. (2) Short-exposure green image signals from pixels drawn at the time can be output simultaneously or in parallel. (3) Long-exposure red image signals from red pixels and long-exposure blue image signals from blue pixels may be output simultaneously or in parallel at a time point. (4) Short-exposure red image signals from red pixels and short-exposure blue image signals from blue pixels may be output simultaneously or in parallel at a time point.

25 is a conceptual diagram for explaining the operation of the timing generator operating under normal operating conditions, FIG. 26 is a conceptual diagram for explaining the operation of the timing generator when binning in units of two rows, and FIG. 27 is three rows It is a conceptual diagram for explaining the operation of the timing generator when binning in units of fields.

13 and 18 to 27 , the timing generator 640 is configured to use one of the two photoelectric conversion elements included in each pixel for a long-exposure time so that the phase difference detection performance is not degraded under WDR operating conditions. The row driver 520 may be controlled to supply a control signal corresponding to .

The timing generator 640 may control the exposure of each of the pixels in a binning condition to be described with reference to FIGS. 27, 28, and 29 .

27 is a WDR pattern corresponding to exposure control for normal operating conditions. 28 is a WDR pattern corresponding to exposure control in the case of binning by two rows. In this case, two rows are read at the same time. 29 is a WDR pattern corresponding to exposure control in the case of binning by 3 rows. In this case, three rows are simultaneously read.

28 is a block diagram illustrating another embodiment of a data processing system including the pixels illustrated in FIG. 1 or 18 .

The data processing system 600 includes a pixel array 100 , an exposure time control circuit 610 , an analog-to-digital converter 630 , a pre-ISP 650 , a phase difference processing circuit 660 , and an output interface. (670).

The pixel array 100 includes a plurality of pixels. Each of the plurality of pixels may refer to the pixel described with reference to FIGS. 1 through 27 . That is, each pixel may include two or more photoelectric conversion elements that can be controlled independently.

The exposure time control circuit 610 may independently control an exposure time of each of the photoelectric conversion elements for each pixel. The exposure time control circuit 610 may include a row driver 520 and a timing generator 530 .

29 is a flowchart for explaining the operation of the data processing system shown in FIG. 28;

28 and 29 , according to embodiments, the timing generator 530 checks the current row address, generates first exposure time control signals based on the result of the check, and controls the generated first exposure time. Signals may be output to the row driver 520 (S110). That is, the first exposure time control signals may be generated based on only the current row address.

The row driver 520 generates long-sequence and/or short-sequence control signals in response to the first exposure time control signals. The long-sequence means a sequence for controlling a long-exposure time, and the short-sequence means a sequence for controlling a short-exposure time.

According to other embodiments, the timing generator 530 checks binning condition data corresponding to the binning-conditions, generates first exposure time control signals based on the result of the check, and generates the first exposure time control signal. may be output to the row driver 520 (S120). That is, the first exposure time control signals may be generated based on only the binning condition data. In this case, the binning condition data may be data corresponding to the number of rows to be used in the binning operation as described with reference to FIGS. 25, 26, and 27 .

According to still other embodiments, the timing generator 530 generates and generates first exposure time control signals based on a result of checking the current row address ( S110 ) and a result of checking the binning condition data ( S120 ) of the binning condition data. The first exposure time control signals may be output to the row driver 520 .

The photoelectric conversion elements implemented for each pixel, for example, photodiodes are reset according to the long-sequence and/or the short-sequence ( S130 ).

The photoelectric conversion elements implemented for each pixel transfer the accumulated charges according to a long-sequence and/or a short-sequence to a corresponding floating diffusion region through transfer transistors. That is, the pixels output the pixel signals to the analog-to-digital converter 630 through the column lines after the exposure time has elapsed ( S140 ).

The analog-to-digital converter 630 converts the pixel signals output from the pixels into digital signals, and transmits the digital signals to the pre-ISP 650 .

The pre-ISP 650 performs lens shading correction and/or bad pixel correction on digital signals, generates color information according to the result of the execution, and outputs the generated color information to the output interface 670 . can

In addition, the pre-ISP 650 may output digital signals or signals corresponding to the result of the execution to the phase difference processing circuit 660 .

The phase difference processing circuit 660 may compress the digital signals transmitted from the pre-ISP 650 or signals corresponding to the result of the execution, and output phase difference information corresponding to the compression result to the output interface 670 . have.

For example, the phase difference processing circuit 660 converts the digital signals transmitted from the pre-ISP 650 into Y values (eg, phase values), and based on the converted Y values, digital signals corresponding to long-exposure image signals. Parity between the signals and digital signals corresponding to the short-exposure image signal may be detected and phase difference information corresponding to a result of the detection may be generated.

For example, the phase difference processing circuit 660 calculates an average of digital signals corresponding to pixel signals output from pixels included in M (M is a natural number equal to or greater than 2) rows in the Y-axis direction, and 1/ M-compressed phase difference information may be generated.

The output interface 670 may output the color information output from the pre-ISP 650 and the phase difference information output from the phase difference processing circuit 660 . The output interface 670 may determine an output order of the color information and the phase difference information.

30 is a block diagram illustrating another embodiment of a data processing system including the pixels shown in FIG. 1 or 18 .

If from Fig. 1, see FIG. 30, data processing system 700 may be implemented in a data processing system that may use or support the ^{MIPI ® (mobile industry processor interface)} .

The data processing system 700 may be implemented as a portable electronic device. The portable electronic device may refer to a mobile computing device. The mobile computing device may mean a laptop computer, mobile phone, smart phone, tablet PC, digital camera, camcorder, MID, wearable computer, IoT device, or IoE device.

The data processing system 700 may include an application processor (AP) 710 , an image sensor 505 , and a display 730 .

A camera serial interface (CSI) host 713 implemented in the AP 710 may serially communicate with the CSI device 706 of the image sensor 505 through CSI. According to an embodiment, the CSI host 713 may include a deserializer (DES), and the CSI device 706 may include a serializer (SER).

The image sensor 505 may include a plurality of pixels (or a plurality of active pixel sensors) as described with reference to FIGS. 1 to 13 and 18 to 28 . Each of the plurality of pixels may include a plurality of photoelectric conversion elements that can be independently controlled. The structure of each of the plurality of pixels is the same as described with reference to FIG. 7 or FIG. 8 . The image sensor 505 may be implemented as a front side illuminated (FSI) CMOS image sensor or a back side illuminated (BSI) CMOS image sensor.

In addition, the image sensor 505 may include an exposure time control circuit that can independently control the exposure time of each of the plurality of photoelectric conversion elements included in each pixel according to the row address and/or binning condition data, for example, a row driver ( 520) may be included.

A display serial interface (DSI) host 711 implemented in the AP 710 may serially communicate with the DSI device 731 of the display 730 through the DSI. According to an embodiment, the DSI host 711 may include a serializer SER and the DSI device 731 may include a deserializer DES.

For example, image data output from the image sensor 505 may be transmitted to the AP 710 through CSI. The AP 710 may process the image data and transmit the processed image data to the display 730 through DSI.

The data processing system 700 may further include an RF chip 740 capable of communicating with the AP 710 . A physical layer (PHY) 715 of the AP 700 and a physical layer (PHY) 741 of the RF chip 740 may exchange data according to MIPI DigRF.

The CPU 717 may control the operation of each of the DSI host 711 , the CSI host 713 , and the PHY 715 , and may include one or more cores.

The AP 710 may be implemented as an integrated circuit or a system on chip (SoC), and may refer to a processor or a host capable of controlling the operation of the image sensor 505 .

The data processing system 700 includes a GPS receiver 750 , a data storage device 753 including a volatile memory 751 such as dynamic random access memory (DRAM), a nonvolatile memory such as a flash-based memory, a microphone 755 . ), or a speaker 757 . The data storage device 753 may be implemented as an external memory detachable from the AP 710 . In addition, the data storage device 753 may be implemented as a universal flash storage (UFS), a multimedia card (MMC), an embedded MMC (eMMC ^TM ), or a memory card. .

The data processing system 700 may include at least one communication protocol, for example, worldwide interoperability for microwave access (WiMAX) 759, Wireless LAN (WLAN) 761, ultra-wideband (UWB) 763, and/or LTE ^TM (long term evolution; 765) may be used to communicate with an external device.

According to an embodiment, the data processing system 700 may further include a near-field communication (NFC) module, a Wi-Fi module, and/or a Bluetooth module.

Although the present invention has been described with reference to the embodiment shown in the drawings, this is merely exemplary, and those skilled in the art will understand that various modifications and equivalent other embodiments are possible therefrom. Accordingly, the true technical protection scope of the present invention should be determined by the technical spirit of the appended claims.

100: pixel array
520: low driver
530: timing generator
610: exposure time control circuit

Claims (20)

A pixel array comprising pixels arranged in a plurality of rows and a plurality of columns, each of the pixels being independently controlled to detect a phase difference for auto-focus a pixel array comprising elements; and
Based on binning condition data corresponding to the number of rows to be used for a binning operation among the plurality of rows, the exposure time of each of the photoelectric conversion elements included in each of the pixels is independently controlled Image sensor including exposure time control circuitry. delete According to claim 1,
The exposure time control circuit may independently set the exposure time of each of the photoelectric conversion elements included in each of the pixels disposed in the first row by using a row address for a first row among the plurality of rows. image sensor controlled by delete According to claim 1,
The pixels are
a first pixel including a first photoelectric conversion element and a second photoelectric conversion element; and
a second pixel including a third photoelectric conversion element and a fourth photoelectric conversion element;
The image sensor is
outputting a first control signal for controlling a first exposure time of the first photoelectric conversion element through a first control line, and controlling a second exposure time of the second photoelectric conversion element through a second control line output a second control signal, output a third control signal for controlling a third exposure time of the third photoelectric conversion element through a third control line, and output a third control signal for controlling a third exposure time of the third photoelectric conversion element through a fourth control line Further comprising an exposure time control circuit for outputting a fourth control signal for controlling the fourth exposure time,
The first exposure time, the second exposure time, the third exposure time, and the fourth exposure time are independently controlled by the exposure time control circuit. 6. The method of claim 5,
the first pixel and the second pixel are disposed in a first row of the plurality of rows or a first column of the plurality of columns;
The first exposure time and the third exposure time are the same,
The second exposure time and the fourth exposure time are the same,
The first exposure time is longer than the second exposure time. 6. The method of claim 5,
the first pixel and the second pixel are disposed in a first column of the plurality of columns;
The first exposure time and the fourth exposure time are the same,
The second exposure time and the third exposure time are the same,
The first exposure time is longer than the second exposure time. According to claim 1,
The pixels are
a first pixel including a first photoelectric conversion element and a second photoelectric conversion element; and
a second pixel including a third photoelectric conversion element and a fourth photoelectric conversion element;
the first pixel and the second pixel are disposed in a first row among the plurality of rows;
The first photoelectric conversion element and the second photoelectric conversion element share a first floating diffusion region through corresponding transfer gates,
The third photoelectric conversion element and the fourth photoelectric conversion element share a second floating diffusion area different from the first floating diffusion area through corresponding transfer gates. According to claim 1,
The pixels are
a first pixel including a first photoelectric conversion element and a second photoelectric conversion element; and
a second pixel including a third photoelectric conversion element and a fourth photoelectric conversion element;
the first pixel and the second pixel are disposed in a first column of the plurality of columns;
and the first pixel and the second pixel share one floating diffusion area. According to claim 1,
A first deep trench isolation (DTI) structure is formed between two corresponding pixels among the pixels;
An image sensor in which a second DTI structure is formed between two corresponding photoelectric conversion elements among the photoelectric conversion elements included in each of the pixels. 11. The method of claim 10,
Each of the pixels,
a color filter formed over the photoelectric conversion elements included in each pixel; and
The image sensor further comprising a microlens formed on the color filter. image sensor; and
A controller for controlling the operation of the image sensor,
The image sensor is
A pixel array comprising pixels arranged in a plurality of rows and a plurality of columns, each pixel comprising photoelectric conversion elements independently controlled to detect a phase difference for autofocus;
Based on binning condition data corresponding to the number of rows to be used for a binning operation among the plurality of rows, the exposure time of each of the photoelectric conversion elements included in each of the pixels is independently controlled A data processing system comprising exposure time control circuitry. delete 13. The method of claim 12,
The exposure time control circuit may independently set an exposure time of each of the photoelectric conversion elements included in each of the pixels disposed in the first row by using a row address for a first row among the plurality of rows. Controlling data processing system. delete 13. The method of claim 12,
The photoelectric conversion elements included in each of the pixels include a first photoelectric conversion element controlled with a relatively long-exposure time and a second photoelectric conversion element controlled with a relatively short-exposure time,
The first photoelectric conversion element included in a first pixel among the pixels and the first photoelectric conversion element included in a second pixel disposed adjacent to the first pixel among the pixels are arranged in a diagonal direction. system. 13. The method of claim 12,
The photoelectric conversion elements include a first photoelectric conversion element controlled with a relatively long-exposure time and a second photoelectric conversion element controlled with a relatively short-exposure time,
The pixel signal generated by the first photoelectric conversion element included in each of the pixels is output in parallel under the control of the exposure time control circuit. 13. The method of claim 12,
an analog-to-digital converter for converting pixel signals output from the photoelectric conversion elements included in each of the pixels into digital signals;
a pre-image signal processor for generating color information from the digital signals; and
and a phase difference processing circuit for generating phase difference data corresponding to the phase difference from the digital signals and compressing the generated data. 13. The method of claim 12,
A first deep trench isolation (DTI) structure is formed between two corresponding pixels among the pixels;
A data processing system in which a second DTI structure is formed between two corresponding photoelectric conversion elements among the photoelectric conversion elements included in each of the pixels. 20. The method of claim 19,
Each of the pixels,
a color filter formed on the photoelectric conversion elements included in each pixel; and
and a microlens formed over the color filter.

Images