JP2020108176A - Imaging apparatus and imaging method - Google Patents

Abstract

To provide an imaging apparatus and an imaging method.SOLUTION: The imaging apparatus comprises a pixel array including a normal pixel to use for image generation and a focus detection pixel to use for focal adjustment; a frame memory for recording therein a main-line pixel signal read out of the normal pixel; an input unit to which a main-line pixel signal recorded in the frame memory and a focal pixel signal read out of the focal detection pixel are input; and a lens drive instruction unit for giving an instruction to drive a focus lens based on the focal pixel signal, during the input of the main-line pixel signal.SELECTED DRAWING: Figure 4

Description

The present disclosure relates to an imaging device and an imaging method.

In recent years, digital still cameras and digital video cameras equipped with imaging devices such as CCD (Charge Coupled Device) and CMOS (Complementary Metal-Oxide Semiconductor) sensors have become widespread. Although a digital still camera captures a still image and a digital video camera captures a moving image, there are digital still cameras capable of capturing a moving image and digital video cameras capable of capturing a still image. In the following, the digital still camera and the digital video camera may be simply referred to as “digital camera” unless otherwise distinguished. Further, hereinafter, a still image and a moving image may be simply referred to as “image” unless otherwise distinguished.

The digital camera as described above often has a function of autofocus processing for automatically performing focus adjustment, as disclosed in, for example, Japanese Patent Laid-Open No. 2004-242242. In the auto-focus process, for example, the distance between the digital camera and the subject is measured, and the lens is driven based on the measured distance to properly adjust the focus, thereby obtaining a focused (focused) image. It becomes possible to acquire.

JP, 2013-223054, A

In the above autofocus processing, it has been desired to reduce the time (latency) required from the measurement of the distance to the reflection of the result (for example, completion of driving the lens).

Therefore, the present disclosure proposes a new and improved imaging apparatus and imaging method capable of reducing the latency associated with autofocus.

According to the present disclosure, a pixel array including normal pixels used for image generation and focus detection pixels used for focus adjustment, a frame memory in which main line pixel signals read from the normal pixels are recorded, and the frame memory An input unit to which the main line pixel signal recorded in and the focus pixel signal read from the focus detection pixel are independently input, and based on the focus pixel signal during input of the main line pixel signal. An image pickup apparatus is provided that includes a lens drive instruction unit that issues a drive instruction for a focus lens.

Further, according to the present disclosure, there is provided an imaging method executed by an imaging device having a pixel array including a normal pixel used for image generation and a focus detection pixel used for focus adjustment, wherein the main line read from the normal pixel is used. The pixel signal is recorded in a frame memory, the main line pixel signal recorded in the frame memory and the focus pixel signal read out from the focus detection pixel are independently input, and An image pickup method is provided, which comprises: issuing an instruction to drive a focus lens based on the focus pixel signal while inputting a main line pixel signal.

As described above, according to the present disclosure, it is possible to reduce the latency associated with autofocus.

Note that the above effects are not necessarily limited, and in addition to or in place of the above effects, any of the effects shown in this specification, or other effects that can be grasped from this specification. May be played.

FIG. 8 is an explanatory diagram illustrating a usage scene of the imaging device according to the embodiment of the present disclosure. It is an explanatory view for explaining an example of a schematic structure of an imaging device concerning a comparative example of the embodiment. FIG. 8 is an explanatory diagram for explaining a flow of an autofocus process in continuous shooting using the image pickup apparatus according to the comparative example of the embodiment. It is an explanatory view for explaining the outline of the imaging device concerning the embodiment. It is explanatory drawing which shows an example of a structure of the imaging device which concerns on the embodiment. FIG. 3 is an explanatory diagram illustrating an example of a pixel arrangement of a pixel array section included in the image sensor according to the same embodiment. It is explanatory drawing which shows the example of the phase contrast pixel which concerns on the same embodiment. FIG. 3 is a block diagram showing a partial configuration example of the image sensor according to the same embodiment. FIG. 3 is an explanatory diagram illustrating an example of a configuration of a chip of the image sensor according to the same embodiment. 6 is a schematic time chart for explaining an operation example 1 according to the embodiment. 9 is a schematic time chart for explaining an operation example 2 according to the same embodiment.

Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and the drawings, constituent elements having substantially the same functional configuration are designated by the same reference numerals, and a duplicate description will be omitted.

The description will be given in the following order.
<<1. Introduction >>
<1-1. Background>
<1-2. Overview>
<<2. Configuration >>
<2-1. Configuration of imaging device>
<2-2. Image sensor configuration>
<<3. Operation example>
<3-1. Operation example 1>
<3-2. Operation example 2>
<<4. Modification>>
<4-1. Modification 1>
<4-2. Modification 2>
<<5. Conclusion >>

<<1. Introduction >>
<1-1. Background>
Before describing the imaging device according to the embodiment of the present disclosure, the background that led to the creation of the imaging device according to the embodiment of the present disclosure will be described with reference to the drawings.

FIG. 1 is an explanatory diagram illustrating a usage scene of the imaging device 1 according to an embodiment of the present disclosure. The imaging device 1 shown in FIG. 1 is, for example, a digital camera. As shown in FIG. 1, the image pickup apparatus 1 includes a focus lens 11 for focus adjustment, and the focus lens 11 is drive-controlled according to a distance between a subject and the image pickup apparatus 1 to automatically perform focus adjustment (autofocus, AF) is performed.

Here, with reference to FIGS. 2 and 3, an autofocus process in an example of a conventional image pickup apparatus will be described as a comparative example. FIG. 2 is an explanatory diagram for describing an example of a schematic configuration of an imaging device according to a comparative example.

The image sensor 800 illustrated in FIG. 2 is an image sensor that is provided in the image capturing apparatus according to the comparative example and captures a subject to obtain digital data of the captured image. The image sensor 800 may be an image sensor such as a CMOS (Complementary Metal-Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor. The image sensor 800 has a pixel array unit 811 in which a plurality of pixels are arranged in a matrix (array). Note that the image sensor 800 generally includes circuits other than the pixel array unit 811, but in the example illustrated in FIG. 2, circuits other than the pixel array unit 811 are omitted for the sake of clarity. There is.

The image processing LSI 900 is an LSI (Large Scale Integration) that performs so-called image processing on a pixel signal (image signal) supplied based on a pixel signal from each pixel of the pixel array unit 811. Examples of image processing include black level correction, color mixture correction, defect correction, demosaic processing, matrix processing, gamma correction, and YC conversion. The image processing unit 910 schematically shows the image processing function realized by the image processing LSI 900a. Note that the image processing LSI 900 may include a configuration for executing a function other than the image processing, but in the example shown in FIG. 2, the configuration other than the image processing unit 910 is shown for the sake of clarity. Is omitted.

Further, reference numeral n0 schematically indicates a signal flow (stream) between the image sensor 800 and the image processing LSI 900.

That is, in the image pickup apparatus according to the comparative example shown in FIG. 2, the image sensor 800 photoelectrically converts light incident through an optical system element (not shown) and A/D-converts the pixel value of each pixel. A pixel signal indicating a captured image of the subject is generated. Then, the image sensor 800 outputs the generated pixel signal as a stream n0 to the image processing unit 910 of the image processing LSI 900.

The image processing LSI 900 acquires the pixel signal output from the image sensor 800 as the stream n0, performs image processing on the acquired pixel signal, and measures the distance between the object and the imaging device (object distance). The imaging device according to the comparative example performs focus adjustment (autofocus) by driving and controlling the focus lens included in the imaging device based on the measured subject distance. For example, the image pickup apparatus according to the comparative example may execute autofocus by the image plane phase difference method. That is, the pixel array unit 811 according to the comparative example includes phase difference pixels for performing focus adjustment by the phase difference method, and the image processing unit 910 determines the subject based on the pixel signal (focus pixel signal) applied to the phase difference pixels. The distance is measured, and the focus lens is drive-controlled based on the subject distance.

Next, referring to FIG. 3, in the image pickup apparatus shown in FIG. 2, an image of a subject is exposed (imaged) by image sensor 800, pixel signals representing the exposed image are read out to image processing LSI 900, and a lens is used. An example of the process flow until the drive control is completed will be described. FIG. 3 is an explanatory diagram for explaining a flow of an autofocus process in continuous shooting using the image pickup apparatus according to the comparative example, and a case where the image pickup apparatus according to the comparative example electronically controls the exposure period of each pixel. 2 shows an example of a schematic time chart of.

In FIG. 3, the horizontal axis represents the time direction, and the vertical axis in pixel/output control represents the row direction of the pixel array unit 811. Further, reference numerals d910, d911, d912, and d913 schematically indicate the exposure period of each pixel of the pixel array unit 811. Note that the exposure period d910 indicates the exposure period of the image captured before the exposure period d911.

Here, the flow of a series of processes when autofocus is executed in the image pickup apparatus according to the comparative example will be described focusing on the exposure period d911. First, as indicated by reference numeral d901, resetting of pixel signals accumulated in each pixel is sequentially performed in units of rows, and when reset of each pixel is completed, light is promptly incident on the pixel and exposure of the pixel is performed. Is started. After that, as indicated by reference numeral d921, when the exposure of each pixel is completed, the reading of the pixel signal from the pixel and the output to the image processing unit 910 are immediately started. That is, for each pixel, the period T92 between the reference symbols d901 and d921 corresponds to the exposure time of the pixel, and the period T93 from the start to the end of the reference symbol d921 is the time required to read and output all pixels. Equivalent to. As described above, in the example shown in FIG. 3, the timing of starting and ending the exposure of each pixel, and the timing of reading and outputting each pixel are different for each row.

When the pixel signals from all the pixels are output to the image processing unit 910, the image processing unit 910 causes the phase difference detection (detection) based on the focus pixel signal corresponding to the phase difference pixel in the period indicated by reference numeral d951 in FIG. ), the object distance is measured (ranging). Subsequently, during a period indicated by reference numeral d961 in FIG. 3, an AF calculation is performed to calculate a lens drive amount based on the measured subject distance and the lens position in the exposure period d911. Further, during the period indicated by reference numeral d971 in FIG. 3, drive control of the focus lens included in the imaging device according to the comparative example is performed. As a result, the autofocus executed based on the image pickup in the exposure period d911 shown in FIG. 3 is reflected in the image pickup in the exposure period d913 shown in FIG. That is, in the example of FIG. 3, the time (latency) required from the start of exposure for distance measurement to the reflection of autofocus is twice the shooting interval T91.

As shown in FIG. 1, when an image capturing apparatus is used to capture a moving subject in a sports scene or the like, latency associated with autofocus is important. For example, if the subject moves and the distance between the subject and the imaging device changes from the exposure for distance measurement to the completion of the drive control of the focus lens, an out-of-focus image is acquired. There is. Therefore, it is desirable that the latency required for autofocus is small.

Therefore, the present embodiment has been created by focusing on the above circumstances. According to this embodiment, it is possible to reduce the latency required for autofocus. The outline of the image pickup apparatus according to this embodiment having such an effect will be described below.

<1-2. Overview>
FIG. 4 is an explanatory diagram for describing the outline of the imaging device according to the embodiment of the present disclosure. Note that FIG. 4 shows a schematic configuration of the imaging device 1 (imaging control device) according to the present embodiment, focusing on the image sensor 100 and the image processing LSI 200, and the other configurations are not shown. doing.

As shown in FIG. 4, the image sensor 100 according to the present embodiment differs from the image sensor 800 according to the comparative example shown in FIG. 2 in that it includes a frame memory 190. Reference numerals n1 and n2 schematically show signal flows (streams) between the image sensor 100 and the image processing LSI 200.

Further, the image pickup apparatus 1 according to the present embodiment performs autofocus by the image plane phase difference method and electronically adjusts the exposure period of each pixel, as in the comparative example described with reference to FIGS. 2 and 3. Control. That is, the pixel array unit 111 included in the image sensor 100 according to the present embodiment illustrated in FIG. 4 includes a phase difference pixel (focus detection pixel) for performing focus adjustment by a phase difference method and a normal pixel used for image generation. Including, the exposure period of each pixel is electronically controlled.

In the image pickup apparatus according to the present embodiment shown in FIG. 4, the image sensor 100 photoelectrically converts light incident through an optical system element (not shown), and A/D-converts the pixel value of each pixel to obtain an image of a subject. An image signal indicating a captured image is generated (reading out). At this time, the image sensor 100 reads out the focus image signal from at least the phase difference pixel among the plurality of pixels forming the pixel array unit 111, and outputs it to the image processing LSI 200 as a stream n1.

Further, the image sensor 100 reads out the main line pixel signal from at least a normal pixel among the plurality of pixels forming the pixel array unit 111, and temporarily records the main line pixel signal in the frame memory 190. Then, the image sensor 100 outputs the main line pixel signal recorded in the frame memory 190 to the image processing LSI 200 as a stream n2.

With such a configuration, the image sensor 100, for example, sets the focus image signals from the phase difference pixels as the stream n1 and the main line pixel signals (images) from the normal pixels as the stream n2, and images each pixel signal independently from each other. It becomes possible to output to the processing LSI 200. In addition, although the example in which the focus image signal from the phase difference pixel is output as the stream n1 has been described above, the present technology is not limited to this example. For example, it is possible to output a pixel signal read from a part of the normal pixels used for image generation as the stream n1, and such an example will be described later as a modified example.

Since the image sensor 100 can temporarily hold pixel signals from all pixels in the frame memory 190, for example, the stream n1 and the stream n2 are output to the image processing LSI 200 in parallel at the same timing. No need. That is, the image sensor 100 can also output the stream n2 after outputting the stream n1. Of course, it goes without saying that the image sensor 100 may output the stream n1 and the stream n2 in parallel to the image processing LSI 200.

Therefore, for example, the image processing LSI 200 can perform the phase difference detection process based on the focus pixel signal (stream n1) output prior to the main line pixel signal (stream n2) from the image sensor 100. With this configuration, the image pickup apparatus 1 according to the present embodiment can reduce the latency associated with autofocus.

In addition, as another example, the image processing LSI 200 parallelizes the phase difference detection processing based on the focus pixel signal previously output as the stream n1 from the image sensor 100, and the main line pixel signal output as the stream n2. It is possible to perform an acquisition.

The image sensor 100 and the image processing LSI 200 do not necessarily have to be provided in the same housing. In that case, the device provided with the image sensor 100 corresponds to an example of an “imaging control device”.

The outline of the imaging device 1 according to the present embodiment has been described above with reference to FIG. Subsequently, in the following, the configuration and operation of the imaging device 1 according to the present embodiment will be sequentially described in more detail.

<<2. Configuration >>
Hereinafter, first, a configuration example of the image pickup apparatus 1 according to the present embodiment will be described with reference to FIG. 5, and then a configuration example of the image sensor 100 included in the image pickup apparatus 1 will be described with reference to FIGS.

<2-1. Configuration of imaging device>
FIG. 5 is an explanatory diagram showing an example of the configuration of the image pickup apparatus 1 according to the present embodiment, and shows an example of the case where the image sensor 100 and the image processing LSI 200 described above are provided in the same housing. The imaging device 1 shown in FIG. 5 is a device that images a subject and outputs an image of the subject as an electrical signal.

As shown in FIG. 5, the image pickup apparatus 1 includes a focus lens 11 (hereinafter may be simply referred to as a lens), an image sensor 100, a drive control unit 12, an operation unit 13, an image processing unit 210, a codec processing unit 14, and a recording unit. It has a unit 15 and a display unit 16.

The focus lens 11 is controlled by the drive control unit 12, adjusts the focus to the subject, collects light from a focused position, and supplies the light to the image sensor 100.

The image sensor 100 is an image sensor that images a subject, and is controlled by the drive control unit 12 to photoelectrically convert incident light and A/D-convert the pixel value of each pixel to read and output a pixel signal. To do. Here, as described above, the image sensor 100 reads out the focus pixel signal from the phase difference pixel (focus detection pixel) used for focus adjustment and reads out the main line pixel signal from the normal pixel used for image generation. , Independently. Further, the image sensor 100 outputs the focus pixel signal and the main line pixel signal to the image processing unit 210 independently of each other.

The drive control unit 12 is based on a signal corresponding to a user's operation input input by the operation unit 13, the focus lens 11, the image sensor 100, the image processing unit 210, the codec processing unit 14, the recording unit 15, and the display unit. The drive of 16 is controlled and each part is made to perform the process regarding imaging. In particular, the drive control unit 12 uses a focus pixel signal input prior to the main pixel signal so that the lens drive is performed during the input of the main pixel signal from the image sensor 100 to the image processing unit 210. It has a function as a lens drive instructing unit that gives a drive instruction. For example, the drive control unit 12 receives the result of the phase difference detection performed by the image processing unit 210 based on the focus pixel signal, calculates the lens drive amount by AF calculation, and gives a lens drive instruction. Note that the image processing unit 210 may issue a lens drive instruction as a lens drive instruction unit.

The operation unit 13 is composed of, for example, a jog dial (trademark), a key, a button, a touch panel, or the like, receives an operation input by the user, and supplies a signal corresponding to the operation input to the drive control unit 12.

The image processing unit 210 receives a pixel signal from the image sensor 100 and performs image processing. For example, the image processing unit 210 has a function as an input unit into which the focus pixel signal output from the image sensor 100 and the main line pixel signal are input independently of each other. Further, the image processing unit 210 performs phase difference detection based on the focus pixel signal, and provides the drive control unit 12 with the result of the phase difference detection. Further, the image processing unit 210 performs various image processing such as black level correction, color mixture correction, defect correction, demosaic processing, matrix processing, gamma correction, and YC conversion. The content of this image processing is arbitrary, and processing other than the above may be performed. Further, the image processing unit 210 supplies the image signal subjected to the image processing to the codec processing unit 14 and the display unit 16.

The codec processing unit 14 performs an encoding process of a predetermined method on the image signal from the image processing unit 210 and supplies the image data obtained as a result of the encoding process to the recording unit 15.

The recording unit 15 records the image data supplied from the codec processing unit 14. The image data recorded in the recording unit 15 is supplied to the display unit 16 by being read by the image processing unit 210 as necessary, and the corresponding image is displayed.

The display unit 16 is configured as, for example, a liquid crystal display or the like, and displays the image of the subject based on the image signal from the image processing unit 210.

<2-2. Image sensor configuration>
The example of the configuration of the imaging device 1 according to the present embodiment has been described above with reference to FIG. Next, a configuration example of the image sensor 100 included in the imaging device 1 will be described with reference to FIGS.

(Pixel arrangement)
First, with reference to FIG. 6, regarding an example of the pixel arrangement of the pixel array unit 111 included in the image sensor 100 according to the present embodiment described with reference to FIG. 4, particularly, the arrangement of the phase difference pixels included in the pixel array unit 111. Pay attention to the explanation. FIG. 6 is an explanatory diagram for explaining an example of the pixel arrangement of the pixel array unit 111 included in the image sensor 100 according to this embodiment. In the pixel array section 111, in addition to normal pixels of red, blue, and green, phase difference pixels 121a and 121b used for focus adjustment are periodically arranged at positions shown as A and B in FIG.

FIG. 7 is an explanatory diagram showing an example of the phase difference pixel according to the present embodiment. The phase difference pixels 121a and 121b arranged in the pixel array unit 111 may be, for example, a left opening pixel 121a and a right opening pixel 121b as shown in the upper part of FIG. As shown in FIG. 7, in the left opening pixel 121a and the right opening pixel 121b, the on-chip lenses 1211a and 1211b are covered by the oblique metal 1212a and 1212b. Further, a left opening 1213a is provided on the left side of the oblique light metal 1212a included in the left opening pixel 121a, and the left opening pixel 121a can receive only light incident on the area of the left opening 1213a. Similarly, the right opening portion 1213b is provided on the right side of the oblique light metal 1212b included in the right opening pixel 121b, and the right opening pixel 121b can receive only the light incident on the area of the right opening portion 1213b. Therefore, the left opening pixel 121a and the right opening pixel 121b respectively receive the light fluxes that have passed through different optical paths of the taking lens.

The left opening pixel 121a and the right opening pixel 121b are set as a pair of pixels, and the image processing unit 210 evaluates the degree of deviation between the two images in the focus pixel signal corresponding to the pair of pixels, thereby performing the phase difference detection process. Is done.

Note that, in the above, the example in which the phase difference pixels arranged in the pixel array unit 111 are the paired pixels including the left opening pixel and the right opening pixel has been described, but the present technology is not limited to this example. For example, as shown in the middle of FIG. 7, the phase difference pixels arranged in the pixel array unit 111 may be a pair of pixels including an upper opening pixel 121c and a lower opening pixel 121d. The upper opening pixel 121c and the lower opening pixel 121d are similar to the left opening pixel 121a and the right opening pixel 121b, and the on-chip lenses 1211c and 1211d, the oblique metal 1212c and 1212d, the upper opening 1213c, and the lower opening. And a portion 1213d.

Further, the phase difference pixels arranged in the pixel array section 111 may be photodiode divided pixels. In this case, for example, one of the left and right photodiode divided pixels 121e shown in the lower left of FIG. 7 or the upper and lower photodiode divided pixels 121f shown in the lower right of FIG. 7 may be arranged as the phase difference pixel. The left and right photodiode divided pixels 121e include two photodiodes (a left photodiode 1214a and a right photodiode 1214b) that can independently receive light with respect to one on-chip lens 1211e. Similarly, the upper and lower photodiode divided pixels 121f include two photodiodes (an upper photodiode 1214c and a lower photodiode 1214d) that can independently receive light with respect to one on-chip lens 1211f. The image processing unit 210 can perform the phase difference detection process by evaluating the degree of deviation between the two images in the focus pixel signal corresponding to any one of the photodiode divided pixels.

(Configuration related to read/output)
The example of the pixel arrangement of the pixel array unit 111 included in the image sensor 100 according to the present embodiment has been described above. Next, with reference to FIG. 8, regarding an example of a configuration related to reading and output in the image sensor 100, in particular, light incident through an optical system element (not shown) is photoelectrically converted, and the pixel value of each pixel is A/ A description will be given focusing on the configuration in which the pixel signal is read out and output by D conversion.

FIG. 8 is a block diagram showing a partial configuration example of the image sensor 100 according to the present embodiment. The image sensor 100 shown in FIG. 8 is an image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor that captures an image of a subject and obtains digital data of the captured image.

As shown in FIG. 8, the image sensor 100 includes a control unit 101, a pixel array unit 111, a selection unit 112, an A/D conversion unit (ADC (Analog Digital Converter)) 113, and a constant current circuit unit 114.

The control unit 101 controls each unit of the image sensor 100 to execute a process related to reading of image data (pixel signal). In particular, the control unit 101 according to the present embodiment performs reading of focus pixel signals from the phase difference pixels (focus detection pixels) used for focus adjustment and reading of main line pixel signals from normal pixels used for image generation. , Control the pixels to be done independently.

Note that the reading of the focus pixel signal and the reading of the main line pixel signal are independently performed means that the reading of the focus pixel signal and the reading of the main line pixel signal are performed without being limited to each other. Means For example, when the readout of the focus pixel signal and the readout of the main line pixel signal are performed in different periods, or when the start timings of the readout of the respective pixel signals are different but the readout periods overlap, the respective pixel signals are arranged in parallel. The case where it is read is also included in the above.

The control unit 101 according to the present embodiment also has a function as an exposure control unit that controls the exposure of the focus detection pixels and the exposure of the pixels used for image generation. For example, the control unit 101 exerts a function as an exposure control unit, and controls the exposure of the phase difference pixel and the normal pixel so that the exposure time of the phase difference pixel (focus detection pixel) is different from the exposure time of the normal pixel. The exposure may be controlled. For example, the control unit 101 may control the exposure of the phase difference pixel and the exposure of the normal pixel so that the exposure time of the phase difference pixel is longer than the exposure time of the normal pixel. With such a configuration, it is possible to receive more light even in a phase difference pixel having a light receiving area narrower than that of a normal pixel, and for example, there is an effect that the AF accuracy in a dark area is improved. The control unit 101 may control the exposure of the phase difference pixel and the exposure of the normal pixel so that the exposure time of the phase difference pixel is shorter than the exposure time of the normal pixel. According to such a configuration, when performing AF on a high-luminance subject, there is an effect that the influence of the overflow of the pixel value on the AF accuracy is reduced.

As described with reference to FIGS. 6 and 7, the pixel array unit 111 is a pixel region in which pixel configurations having photoelectric conversion elements such as photodiodes are arranged in a matrix. Under control of the control unit 101, the pixel array unit 111 receives light of a subject at each pixel, photoelectrically converts the incident light to accumulate charges, and at predetermined timing, accumulates charges accumulated in each pixel. Output as an analog pixel signal.

A pixel 121 and a pixel 122 represent two vertically adjacent pixels in a pixel group arranged in the pixel array section 111. The pixel 121 and the pixel 122 are pixels in consecutive rows in the same column. For example, the pixel 121 may be a phase difference pixel used for focus adjustment described with reference to FIGS. 6 and 7, and the pixel 122 may be a normal pixel used for image generation. In the case of the example in FIG. 8, as shown in the pixel 121 and the pixel 122, a photoelectric conversion element and four transistors are used in the circuit of each pixel. The circuit configuration of each pixel is arbitrary and may be other than the example shown in FIG.

In a general pixel array, an output line of an analog pixel signal is provided for each column. In the case of the pixel array unit 111, two (two systems) output lines are provided for each column. Circuits of pixels in one column are alternately connected to these two output lines every other row. For example, the circuits of the pixels in the odd-numbered rows from the top are connected to one output line, and the circuits of the pixels in the even-numbered rows are connected to the other output line. In the case of the example in FIG. 8, the circuit of the pixel 121 is connected to the first output line (VSL1), and the circuit of the pixel 122 is connected to the second output line (VSL2).

Note that, in FIG. 8, only one column of output lines is shown for convenience of description, but in reality, two output lines are similarly provided for each column. The circuit of the pixel in the column is connected to each output line every other row.

The selection unit 112 has a switch that connects each output line of the pixel array unit 111 to the input of the ADC 113, and is controlled by the control unit 101 to control the connection between the pixel array unit 111 and the ADC 113. That is, the analog pixel signal read from the pixel array unit 111 is supplied to the ADC 113 via the selection unit 112.

The selection unit 112 includes a switch 131, a switch 132, and a switch 133. The switch 131 (selection SW) controls connection of two output lines corresponding to the same column. For example, when the switch 131 is turned on, the first output line (VSL1) and the second output line (VSL2) are connected, and when the switch 131 is turned off, the switch 131 is disconnected.

Although details will be described later, in the image sensor 100, one ADC is provided for each output line (column ADC). Therefore, assuming that both the switch 132 and the switch 133 are in the ON state, when the switch 131 is in the ON state, two output lines of the same column are connected, and thus a circuit of one pixel is connected to two ADCs. Will be. On the contrary, when the switch 131 is turned off, the two output lines of the same column are disconnected and the circuit of one pixel is connected to one ADC. That is, the switch 131 selects the number of ADCs (column ADCs) to which the signal of one pixel is output.

Although details will be described later, the image sensor 100 performs image processing on more diverse pixel signals according to the number of ADCs by controlling the number of ADCs to which the switch 131 outputs analog pixel signals. It can be output to the LSI 200. That is, the image sensor 100 can realize more various data outputs.

The switch 132 controls the connection between the first output line (VSL1) corresponding to the pixel 121 and the ADC corresponding to the output line. When the switch 132 is turned on, the first output line is connected to one input of the comparator of the corresponding ADC. Also, when they are turned off, they are disconnected.

The switch 133 controls connection between the second output line (VSL2) corresponding to the pixel 122 and the ADC corresponding to the output line. When the switch 133 is turned on, the second output line is connected to one input of the comparator of the corresponding ADC. Also, when they are turned off, they are disconnected.

Under the control of the control unit 101, the selection unit 112 can control the number of ADCs (column ADCs) to which signals of one pixel are output by switching the states of the switches 131 to 133. ..

Note that the switch 132 and the switch 133 (either one or both) may be omitted, and each output line and the ADC corresponding to the output line may be always connected. However, by enabling these switches to control connection/disconnection, the range of selection of the number of ADCs (column ADCs) that are the output destinations of the signals of one pixel is expanded. That is, by providing these switches, the image sensor 100 can output more various pixel signals.

Although only the configuration for the output line for one column is shown in FIG. 8, in actuality, the selection unit 112 has the same configuration (switch 131 to switch) as shown in FIG. 8 for each column. 133). That is, the selection unit 112 performs the same connection control as described above on each column according to the control of the control unit 101.

The ADC 113 A/D-converts the analog pixel signals supplied from the pixel array unit 111 via the respective output lines, and outputs them as pixel signals (digital data). The ADC 113 has an ADC (column ADC) for each output line from the pixel array unit 111. That is, the ADC 113 has a plurality of column ADCs. The column ADC corresponding to one output line is a single slope type ADC having a comparator, a D/A converter (DAC (Digital Analog Converter)), and a counter.

The comparator compares the DAC output with the signal value of the analog pixel signal. The counter increments the count value (digital value) until the analog pixel signal and the DAC output become equal. The comparator stops the counter when the DAC output reaches the signal value. Thereafter, the signals digitized by the counters 1 and 2 are output from the DATA 1 and DATA 2 to the outside of the image sensor 100.

The counter returns the count value to the initial value (for example, 0) after outputting the data for the next A/D conversion.

The ADC 113 has two systems of column ADCs for each column. For example, a comparator 141 (COMP1), a DAC 142 (DAC1), and a counter 143 (counter 1) are provided for the first output line (VSL1), and a comparison is made for the second output line (VSL2). A device 151 (COMP2), a DAC 152 (DAC2), and a counter 153 (counter 2) are provided. Although illustration is omitted, the ADC 113 has the same configuration for output lines of other columns.

However, the DAC can be shared among these configurations. The sharing of the DAC is performed for each system. That is, the DACs of the same system in each column are shared. In the case of the example in FIG. 8, the DAC corresponding to the first output line (VSL1) of each column is shared as the DAC 142, and the DAC corresponding to the second output line (VSL2) of each column is shared as the DAC 152. ing. The comparator and the counter are provided for each output line system.

The constant current circuit unit 114 is a constant current circuit connected to each output line, and is driven by being controlled by the control unit 101. The circuit of the constant current circuit unit 114 is composed of, for example, a MOS (Metal Oxide Semiconductor) transistor or the like. Although this circuit configuration is arbitrary, in FIG. 8, a MOS transistor 161 (LOAD1) is provided for the first output line (VSL1) and a second output line (VSL2) is provided for convenience of description. Therefore, a MOS transistor 162 (LOAD2) is provided.

The control unit 101 receives a request from the outside, such as a user, selects the read mode, controls the selection unit 112, and controls the connection to the output line. The control unit 101 also controls the drive of the column ADC according to the selected read mode. Further, in addition to the column ADC, the control unit 101 controls the driving of the constant current circuit unit 114 as necessary, and controls the driving of the pixel array unit 111 such as the reading rate and timing. ..

That is, the control unit 101 can operate not only the control of the selection unit 112 but also each unit other than the selection unit 112 in more various modes. Therefore, the image sensor 100 can output more various pixel signals.

The number of each unit shown in FIG. 8 is arbitrary as long as it is not insufficient. For example, three or more output lines may be provided for each column. The number of pixel signals output from the ADC 132 in parallel or the number of ADCs 132 themselves shown in FIG. 8 may be increased to increase the number of pixel signals output in parallel to the outside.

Next, the configuration of the chip of the image sensor 100 according to the present embodiment will be described with reference to FIG. FIG. 9 is an explanatory diagram for describing an example of the configuration of the chip of the image sensor 100 according to the present embodiment. As described with reference to FIG. 8, providing a plurality of ADCs for each column may increase the chip size and increase the cost. Therefore, in the image sensor 100 according to this embodiment, the chips may be stacked as shown in FIG.

In the example shown in FIG. 9, the image sensor 100 includes a pixel chip 100-1 in which the pixel array section 111 is mainly formed, a peripheral circuit chip 100-2 in which an output circuit 180, a peripheral circuit, an ADC 113, and the like are formed, And a pad (PAD). The output line and the drive line of the pixel array section 111 of the pixel chip 100-1 are connected to the circuit of the peripheral circuit chip 100-2 via a through via (VIA). The frame memory 190 shown in FIG. 4 may be provided in the output circuit 180 or a peripheral circuit, for example. The output circuit 180 functions as an output unit that outputs the focus pixel signal and the main line pixel signal read out as described above independently of each other.

With the above configuration, the chip size can be reduced and the cost can be reduced. Further, since the space of the wiring layer can be provided, the wiring can be easily routed. Further, by forming a plurality of chips, each chip can be optimized. For example, in pixel chips, the height can be reduced with fewer wiring layers to prevent deterioration of quantum efficiency due to optical reflection from the wiring layers, and in peripheral circuit chips, optimization such as inter-wiring coupling countermeasures is possible. Therefore, it is possible to realize a multilayer wiring layer. For example, the wiring layer of the peripheral circuit chip can be made more multilayer than the wiring layer of the pixel chip.

In the case of a back-illuminated image sensor, optical reflection does not occur due to the wiring layers, but by suppressing an increase in the number of unnecessary wiring layers, it is possible to suppress an increase in the number of wiring processes and reduce costs. Can be realized.

Further, since the peripheral circuit chip area has a chip area equivalent to that of the pixel chip area, a plurality of ADCs can be mounted in the peripheral circuit area without increasing the total area of the laminated chips.

Note that it goes without saying that the imaging control device (imaging device) to which the present technology is applied is not limited to the above-described configuration and may have another configuration.

As described above, the image sensor 100 according to the present embodiment photoelectrically converts the light incident through the optical system element (not shown) and A/D-converts the pixel value of each pixel to generate a pixel signal. The generated pixel signals are read from DATA1 and DATA2 shown in FIG. Then, the output circuit 180 outputs, for example, the focus pixel signal output from either DATA1 or DATA2 (here, DATA1) to the image processing LSI 200 as the stream n1 shown in FIG.

Further, the image sensor 100 temporarily records the main line pixel signal read from DATA2 in the frame memory 190. Then, the image sensor 100 sequentially reads the main line pixel signals recorded in the frame memory 190, and the output circuit 180 outputs the main line pixel signals read from the frame memory 190 to the image processing LSI 200 as a stream n2. Note that the subject of control relating to input/output of pixel signals to/from the frame memory 190 and output of pixel signals from the output circuit 180 is not particularly limited. For example, the control unit 101 described above may perform the control, or a control unit may be provided separately from the control unit 101. Further, the control may be performed according to the control from the control unit provided outside the image sensor 100 (for example, the image processing LSI 200 or the like).

Further, the position where the frame memory 190 is provided and the number of the frame memories 190 are not particularly limited as long as the pixel signals acquired by the respective pixels forming the pixel array unit 111 can be temporarily held. For example, by providing a plurality of frame memories 190, it is possible to execute input/output of pixel signals to and from each of the plurality of frame memories 190 in parallel. Therefore, it is possible to mitigate the decrease in processing speed depending on the speed of inputting and outputting the pixel signal to and from the frame memory 190.

Further, as another example, a cache for temporarily holding a pixel signal acquired by the pixel may be provided on the pixel circuit of each pixel forming the pixel array unit 111. With such a configuration, it is possible to more flexibly control the execution timing of the processing related to the input/output of pixel signals between each pixel and the frame memory 190 and the processing related to the reading of each pixel signal from the frame memory 190. Is possible. Further, since the number of pins for writing the pixel signal from each pixel circuit to the frame memory 190 increases, it is possible to further widen the band of the bus between the pixel circuit and the frame memory 190.

<<3. Operation example>
The configuration examples of the imaging device 1 according to the present embodiment and the image sensor 100 included in the imaging device 1 have been described above with reference to FIGS. Subsequently, hereinafter, as an operation example of the image pickup apparatus 1 according to the present embodiment, an example in which the focus pixel signal is read and output once for one focus adjustment, and one focus adjustment is performed. An example of reading and outputting the focus pixel signal a plurality of times will be sequentially described with reference to FIGS.

<3-1. Operation example 1>
FIG. 10 is a schematic time chart for explaining an operation example 1 in which the focus pixel signal is read and output once for one focus adjustment in the image pickup apparatus 1 according to the present embodiment. ..

FIG. 10 shows pixel control relating to exposure/readout processing in each pixel, output control relating to transmission processing of pixel signals from the output circuit 180 to the image processing LSI 200, and phase difference detection relating to focus adjustment processing based on the pixel signals. , AF calculation, and lens drive control are shown. Pixel control 1 and output control 1 are pixel control and output control related to normal pixels used for image generation, and pixel control 2 and output control 2 are pixel control and output related to phase difference pixels used for focus adjustment. Control. Also, in FIG. 10, the horizontal axis indicates the time direction. Further, the vertical axis in the pixel control 1, the pixel control 2, the output control 1, and the output control 2 indicates the position in the row direction of the pixel that is the output source of the target pixel signal.

A period T11 shown in FIG. 10 indicates a shooting interval in the image sensor 100 according to this embodiment. A period T12 shows the exposure time of the normal pixel according to this embodiment, and a period T13 shows the exposure time of the phase difference pixel according to this embodiment. In the present embodiment, the period T12 which is the exposure time of the normal pixel and the period T13 which is the exposure time of the phase difference pixel may be different. For example, the control unit 101 exerts a function as an exposure control unit so that the period T13, which is the exposure time of the phase difference pixel (focus detection pixel), is longer than the period T12, which is the exposure time of the normal pixel, The exposure of the phase difference pixel and the exposure of the normal pixel may be controlled. According to this configuration, as described above, there is an effect that the AF accuracy in the dark area is improved.

Further, reference numerals d210 to d212 indicate an output process in which the output circuit 180 outputs the main line pixel signal read from the normal pixel by the image sensor 100 and recorded in the frame memory 190 in each of the exposure periods d110 to d112. .. Further, reference numerals d221 and d222 indicate output processing in which the output circuit 180 outputs the focus pixel signal read from the phase difference pixel by the image sensor 100 in each of the exposure periods d121 and d122. Note that in the output processing d210 to d212, the output circuit 180 outputs the main line pixel signal as the stream n2 shown in FIG. 4, and in the output processing d221 and d222, the output circuit 180 outputs the focus pixel signal as the stream n1 shown in FIG. To do.

Reference numerals d311 and d312 are phase difference detection processing in which the image processing unit 210 to which the focus pixel signal is input in the output processing d221 and d222 (receives the output of the output circuit 180) measures (distances) the subject distance. Is shown. Reference numerals d321 and d322 are AF calculations in which the drive control unit 12 calculates the lens drive amount based on the subject distance measured in the phase difference detection processing d221 and d222 and the lens position in the exposure periods d121 and d122. The process is shown. Further, reference numerals d331 and d332 represent lens drive processing from the drive control unit 12 issuing a lens drive instruction to the completion of lens drive based on the lens drive amount calculated in the AF calculation processing d321 and d322. ing. In the following, the phase difference detection process, the AF calculation process, and the lens drive process may be collectively referred to as a series of AF-related processes.

As shown in FIG. 10, after the normal pixels are exposed and the main line pixel signals are read (recorded in the frame memory 190) in the exposure periods d110 to d112, the output process d210 to d212 of the main line pixel signals read from the frame memory 190. Is done. On the other hand, after the exposure of the phase difference pixels and the reading of the focus pixel signal in the exposure periods d121 and d122, the output processing d221 and d222 of the read focus pixel signal is performed.

Here, in the present embodiment, the number of phase difference pixels used for focus adjustment is smaller than the number of normal pixels, and the focus pixel signal is output without passing through the frame memory 190. The processing time of d221 and d222 is shorter than that of the output processing d210 to d212. Therefore, for example, immediately after the A/D conversion (reading) and output processing d221 of the phase difference pixel, the A/D conversion of the normal pixel and the recording in the frame memory 190 are performed, so that the lens drive processing d331 is performed by the normal pixel. Is completed during the input/output of (output processing d211). That is, it is possible to complete the series of AF-related processing d311, d321, and d331 by the start of the exposure period d112 of the normal pixel. Therefore, in the operation example shown in FIG. 10, the time (latency) required from the start of exposure for distance measurement to the reflection of autofocus is less than or equal to the total of the shooting interval T11 and the exposure period d121 of the phase difference pixel. Is. Since the exposure period d121 of the phase difference pixel is shorter than the shooting interval, this operation example can reduce the latency required for autofocusing as compared with the comparative example described with reference to FIG.

Although the example of FIG. 10 illustrates an example in which the lens driving process d331 is completed by the start of the exposure period d122 of the phase difference pixel, the present operation example is not limited to this example. For example, when the lens driving process d331 is not completed by the start of the exposure period d122 of the phase difference pixel, the average position and the center position of the lens in the exposure period d122 are used as the representative lens position, and the AF calculation is performed. The process d322 may be performed.

<3-2. Operation example 2>
As above, as the operation example of the image pickup apparatus 1 according to the present embodiment, the operation example 1 in which the focus pixel signal is read and output once for the focus adjustment once is described with reference to FIG. 10. Next, an operation example 2 in which the readout and output of the focus pixel signal are performed a plurality of times for one focus adjustment will be described with reference to FIG. 11. FIG. 11 is a schematic time chart for explaining an operation example 2 in which the focus pixel signal is read and output a plurality of times for one focus adjustment in the image pickup apparatus 1 according to the present embodiment.

FIG. 11 shows pixel control related to exposure/readout processing in each pixel, output control related to transmission processing of pixel signals from the output circuit 180 to the image processing LSI 200, and phase difference detection related to focus adjustment processing based on the pixel signals. , AF calculation, and lens drive control are shown. Note that, similar to FIG. 10, the pixel control 1 and the output control 1 in FIG. 11 are the pixel control and the output control related to the normal pixels used for image generation, and the pixel control 2 and the output control 2 are the positions used for the focus adjustment. These are pixel control and output control related to the phase difference pixel. Further, as in FIG. 10, the horizontal axis in FIG. 11 indicates the time direction. Further, as in FIG. 10, in FIG. 11, the vertical axis in the pixel control 1, the pixel control 2, the output control 1, and the output control 2 indicates the position in the row direction of the pixel that is the output source of the target pixel signal. There is.

Reference numerals T21 and T22 shown in FIG. 11 represent an example of a vertical synchronization signal in the image sensor 100 according to the present embodiment. Further, reference signs t1 and t2 are times at which the shooting process is started, and a period T21 is a shooting interval from time t1 to t2 (an interval at which normal pixels are exposed). For example, the period T21, which is the shooting interval, is about 1/20 [s], and the corresponding frame rate is about 20 [fps]. The period T22 is about 1/120 [s], and the corresponding frame rate is about 120 [fps].

Reference numerals d151 and d152 shown in FIG. 11 are the same as the exposure periods d111 and d112 described with reference to FIG. Reference numerals d250 to d252 are the same as the output processes d210 to d212 described with reference to FIG. The output process d210 shows the output process corresponding to the exposure/readout of the normal pixels performed before the exposure period d111.

Reference numerals d161 to d165 shown in FIG. 11 are the same as the exposure periods d121 and d122 described with reference to FIG. 10, but in the example shown in FIG. 11, a plurality of times are used for one focus adjustment. Exposure and reading (exposure period) are performed multiple times. Reference numerals d261 to d265 are similar to the output processes d221 and d222 described with reference to FIG. 10, but in the example shown in FIG. 11, a plurality of output processes are performed for one focus adjustment. Be seen.

Reference numerals d351 to d355 shown in FIG. 11 are similar to the phase difference detection processing d311 and d312 described with reference to FIG. 10, but in the example shown in FIG. 11, they correspond to the exposure periods d161 to d165. Phase difference detection processing is performed a plurality of times for one focus adjustment. Similarly, reference numerals d361 to d365 are similar to the AF calculation processes d321 and d322 described with reference to FIG. 10, but in the example shown in FIG. 11, a plurality of AF calculations are performed for one focus adjustment. Processing is performed.

Further, reference numerals d371 and d372 shown in FIG. 11 are similar to the lens driving process d331 described with reference to FIG. However, in the example shown in FIG. 11, the result of four AF calculations by the AF calculation processes d361 to d364 is input for one lens drive process (for example, the lens drive process d371). In this operation example, the drive control unit 12 gives a lens drive instruction using the latest AF calculation result every time the AF calculation process is performed even during the lens drive, so that the lens drive control can be performed with higher accuracy. Will be possible. As a result, the accuracy of autofocus is improved.

Focusing on the focus adjustment related to the lens driving process d371 shown in FIG. 11, the time from the AF calculating process d364 to the completion of the lens driving process d371 is from the AF calculating process d321 to the lens driving process d331 shown in FIG. Less than time to complete. This is because, in the example of FIG. 11, when the lens is driven using the result of the AF calculation process d364, the result of the AF calculation process d363, which is the immediately preceding AF calculation process, is already input and reflected in the lens drive process. This is because the required lens drive amount tends to be small. Here, from the start of the exposure period d164 for distance measurement to the start time of the exposure period d152 in which autofocus is reflected, it can be considered as the latency for autofocus. Therefore, in this operation example, it is possible to further reduce the latency required for autofocus.

As described above, in the present operation example shown in FIG. 11, the focus pixel signal is output multiple times during the period in which the main line pixel signal is output once. Since the image sensor 100 according to the present embodiment includes the frame memory 190 and the frame memory 190 can record the main pixel signal, the main pixel signal read from the frame memory 190 is input or output during the input/output as described above. It is possible to perform exposure processing and output processing of the phase difference pixels. Therefore, the focus pixel signal output process, the phase difference detection process, and the AF calculation process can be performed a plurality of times for one focus adjustment, and it is possible to reduce the latency related to the autofocus and to perform the autofocus. The accuracy of is improved.

Since the exposure periods d162, d163, d164 and the lens driving process d371 overlap as shown in FIG. 11, a representative lens position in each exposure period is used as described above, and the AF calculation process d362 is performed. , D363, d364 may be performed.

<<4. Modification>>
The embodiment of the present disclosure has been described above. Hereinafter, some modified examples of the present embodiment will be described. It should be noted that each of the modified examples described below may be applied to this embodiment alone, or may be applied to this embodiment in combination. Moreover, each modification may be applied instead of the configuration described in the present embodiment, or may be additionally applied to the configuration described in the present embodiment.

<4-1. Modification 1>
In the above embodiment, the focus detection pixel used for focus adjustment has been described as an example in which the focus detection pixel is a phase difference pixel for performing focus adjustment by the phase difference method, but the present technology is not limited to the above example. For example, the focus detection pixel may be a pixel for performing focus adjustment by a contrast method in which a lens position where the contrast of an image becomes high is a focus position. When the contrast method is used for focus adjustment, the focus detection pixel may be a normal pixel that can also be used for image generation. In this case, as in Modification 2 described below, some pixels or some lines of the normal pixels may be controlled independently of the pixels other than the above. Further, the present technology can be applied even when a hybrid method in which the above two methods of the phase difference method and the contrast method are combined is used as the focus adjustment method.

<4-2. Modification 2>
In the above-described embodiment, the control unit independently reads out the focus pixel signal from the focus detection pixel previously arranged in the pixel array unit and reads out the main line pixel signal from the normal pixel for image generation. Although the example of controlling the pixels has been described above, the present technology is not limited to the above. In the present technology, any pixel may be read out independently (prior to).

For example, the pixels read out in advance are not limited to focus detection pixels, and among the normal pixels, reading out and outputting from some pixels or some lines are performed in advance, and pixels other than the above are It is also possible to control such that the data is recorded in the frame memory and then read and output. By performing the above control, for example, the image processing LSI analyzes the image based on the pixel signals output from some pixels in advance, or the display unit displays a low-resolution preview image (so-called through image). May be displayed. With such a configuration, it is possible to shorten the time from the photographing to the analysis and display processing, as compared with the case where the image analysis and the preview image display are performed after all the pixels are read out.

Further, since any pixel can be independently read out, when the pixel read out in advance is the focus detection pixel used for focus adjustment, the arrangement of the focus detection pixels need not be predetermined. Absent. For example, the focus detection pixel may be identified based on image processing. In this case, the control unit controls the pixels such that the reading of the focus pixel signal from the focus detection pixel specified based on the image processing and the reading of the main line pixel signal are performed independently. The image processing may be performed by, for example, an image processing unit included in the image processing LSI, and the image sensor that receives the image processing result may specify the focus detection pixel. According to such a configuration, only the necessary focus detection pixels specified based on the image processing need to be read and output in advance, so that the processing time for reading and outputting the focus pixel signals can be further shortened. May be possible. Further, according to this configuration, the focus detection pixel is specified based on the result of the image processing according to the purpose of the user, so that the image is captured in a state in which the user wants to focus on the area desired by the image capturing apparatus. It is possible to do.

For example, the image processing described above includes a subject tracking processing, and the focus detection pixel specified based on the image processing described above is a pixel corresponding to the object region (detection frame for focus adjustment) specified by the object tracking processing. May be With this configuration, it is possible to perform shooting while focusing on a specific subject within the shooting range of the imaging device.

For example, the above image processing includes object detection processing, and the focus detection pixel specified based on the above image processing is a pixel corresponding to the object region (detection frame for focus adjustment) specified by the object detection processing. May be The detected object may be, for example, a person, a face, a car, or the like. With such a configuration, it is possible to perform shooting while focusing on a specific object within the shooting range of the imaging device.

<<5. Conclusion >>
As described above, according to the embodiment of the present disclosure, by reading and outputting the focus pixel signal from the focus detection pixel used for focus adjustment in advance of the main line pixel signal from the normal pixel, It is possible to reduce the latency required for autofocus. Further, the image sensor according to the present embodiment is provided with the frame memory, so that it is possible to output the focus pixel signal a plurality of times during the period in which the main line pixel signal is output once, thereby improving the accuracy of autofocus. At the same time, it is possible to further reduce the latency.

Although the preferred embodiments of the present disclosure have been described above in detail with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field of the present disclosure can come up with various changes or modifications within the scope of the technical idea described in the claims. It is understood that the above also naturally belongs to the technical scope of the present disclosure.

For example, in the above embodiment, an example has been described in which the imaging device electronically controls the exposure period of each pixel, but the present technology is not limited to such an example. For example, the present technology may be applied to an imaging device that includes a so-called focal plane shutter that uses a mechanical front curtain and a rear curtain, and that controls the exposure period of each pixel by moving the front curtain and the rear curtain.

In the above embodiment, the example in which the pixel control of the phase difference pixel is performed immediately before the pixel control of the normal pixel (the exposure period in which the exposure and the reading are performed) has been described, but the present technology is not limited to such an example. For example, in the example of FIG. 10, the pixel control of the phase difference pixel (exposure period d121) is performed immediately before the pixel control of the normal pixel (exposure period d111), but the pixel control of the phase difference pixel is performed after the pixel control of the normal pixel. Pixel control may be performed. In particular, when the lens drive processing based on the pixel control of the phase difference pixel performed after the pixel control of the normal pixel can be completed before the pixel control of the normal pixel performed next to the pixel control of the normal pixel, The latency associated with autofocus can be smaller.

Further, each processing in the operation example of the above embodiment does not necessarily have to be processed in time series in the order described as the time chart. For example, each processing in the operation example of the above-described embodiment may be processed in an order different from the order described as the time chart or may be processed in parallel.

Further, the effects described in the present specification are merely illustrative or exemplary, and are not limitative. That is, the technique according to the present disclosure may have other effects that are apparent to those skilled in the art from the description of the present specification, in addition to or instead of the above effects.

The following configurations also belong to the technical scope of the present disclosure.
(1)
A pixel array including normal pixels used for image generation and focus detection pixels used for focus adjustment;
A frame memory in which main line pixel signals read from the normal pixels are recorded,
An input unit into which the main line pixel signal recorded in the frame memory and the focus pixel signal read from the focus detection pixel are independently input;
A lens drive instructing unit for instructing drive of the focus lens based on the focus pixel signal during input of the main line pixel signal,
An imaging device including.
(2)
The imaging device according to (1), wherein the focus pixel signal is output a plurality of times during a period in which the main line pixel signal is output once.
(3)
The imaging device according to (1) or (2), further including an output unit that outputs the main line pixel signal recorded in the frame memory.
(4)
The imaging device according to any one of (1) to (3), further including an exposure controller that controls exposure of the focus detection pixels and exposure of pixels used for the image generation.
(5)
The exposure control unit controls the exposure of the focus detection pixels and the exposure of the pixels used for the image generation so that the exposure time of the focus detection pixels and the exposure time of the pixels used for the image generation are different. The imaging device according to (4).
(6)
The exposure control unit controls the exposure of the focus detection pixel and the exposure of the pixel used for the image generation such that the exposure time of the focus detection pixel is longer than the exposure time of the pixel used for the image generation. The imaging device according to (5).
(7)
(1) further comprising a control unit that controls the pixels so that reading of the focus pixel signal from the focus detection pixel specified based on image processing and reading of the main line pixel signal are performed independently. The imaging device according to any one of (6) to (6).
(8)
The imaging according to (7), wherein the image processing includes a subject tracking process, and the focus detection pixel specified based on the image processing is a pixel corresponding to a subject region specified by the subject tracking process. apparatus.
(9)
The image processing includes object detection processing, and the focus detection pixel specified based on the image processing is a pixel corresponding to the object region specified by the object detection processing, (7) or (8) The imaging device according to.
(10)
The imaging device according to any one of (1) to (9), wherein the focus detection pixel is a pixel for performing focus adjustment by a phase difference method.
(11)
The image pickup device according to (7), wherein the control unit controls so that two or more identical pixel signals are read from one pixel.
(12)
An imaging method executed by an imaging device having a pixel array including a normal pixel used for image generation and a focus detection pixel used for focus adjustment,
Main line pixel signals read from the normal pixels are recorded in a frame memory,
The main line pixel signal recorded in the frame memory and the focus pixel signal read from the focus detection pixel are independently input;
Performing a focus lens drive instruction based on the focus pixel signal during input of the main line pixel signal,
An imaging method including:.

DESCRIPTION OF SYMBOLS 1 Imaging device 11 Focus lens 12 Drive control part 13 Operation part 14 Codec processing part 15 Recording part 16 Display part 100 Image sensor 100-1 Pixel chip 100-2 Peripheral circuit chip 101 Control part 111 Pixel array part 180 Output circuit 190 Frame memory 210 Image processing unit

Claims (12)

A pixel array including normal pixels used for image generation and focus detection pixels used for focus adjustment;
A frame memory in which main line pixel signals read from the normal pixels are recorded,
An input unit into which the main line pixel signal recorded in the frame memory and the focus pixel signal read from the focus detection pixel are independently input;
A lens drive instructing unit for instructing drive of the focus lens based on the focus pixel signal during input of the main line pixel signal,
An imaging device including. The imaging device according to claim 1, wherein the focus pixel signal is output a plurality of times during a period in which the main line pixel signal is output once. The image pickup apparatus according to claim 1, further comprising an output unit that outputs the main line pixel signal recorded in the frame memory. The image pickup apparatus according to claim 1, further comprising an exposure control unit that controls exposure of the focus detection pixels and exposure of pixels used for the image generation. The exposure control unit controls the exposure of the focus detection pixels and the exposure of the pixels used for the image generation so that the exposure time of the focus detection pixels and the exposure time of the pixels used for the image generation are different. The image pickup apparatus according to claim 4. The exposure control unit controls the exposure of the focus detection pixels and the exposure of the pixels used for the image generation so that the exposure time of the focus detection pixels becomes longer than the exposure time of the pixels used for the image generation. The image pickup apparatus according to claim 5. The control unit that controls a pixel so that reading of the focus pixel signal from the focus detection pixel specified based on image processing and reading of the main pixel signal are performed independently of each other. The image pickup apparatus according to claim 6. The imaging according to claim 7, wherein the image processing includes a subject tracking process, and the focus detection pixel specified based on the image processing is a pixel corresponding to a subject region specified by the subject tracking process. apparatus. The image processing includes object detection processing, and the focus detection pixel specified based on the image processing is a pixel corresponding to an object region specified by the object detection processing. Imaging device. The image pickup apparatus according to claim 1, wherein the focus detection pixel is a pixel for performing focus adjustment by a phase difference method. The image pickup apparatus according to claim 7, wherein the control unit controls so that two or more identical pixel signals are read from one pixel. An imaging method executed by an imaging device having a pixel array including a normal pixel used for image generation and a focus detection pixel used for focus adjustment,
Main line pixel signals read from the normal pixels are recorded in a frame memory,
The main line pixel signal recorded in the frame memory and the focus pixel signal read from the focus detection pixel are independently input;
Performing a focus lens drive instruction based on the focus pixel signal during input of the main line pixel signal,
An imaging method including:.