JP2013258546A - Image pick-up device and signal processing method and electronic apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate an image signal of still image in which occurrence of signal deterioration or aliasing is suppressed in parallel with generation of an image signal of moving image.SOLUTION: A vertical drive section 122 drives each unit pixel of a pixel arrangement part 121 so as to perform charge storage operation in a storage period. A timing control section 125 controls driving operation of unit pixels so that a first signal based on the charges stored in the storage period is read from a first pixel group constituting a moving image in a plurality of pixels when capturing a moving image, and so that a first signal based on the charges stored in the storage period is read from a first pixel group, and a second signal based on the charges stored in the storage period is read from a second pixel group excepting the first pixel group from a plurality of pixels when capturing a still image in parallel with a moving image. The order of reading a second signal from a second pixel group is changed depending on the importance. An image signal generation processing section 127 generates an image signal of still image by compounding the first and second signals thus read.

Description

  This technique relates to an imaging device, a signal processing method, and an electronic apparatus. Specifically, it is possible to generate an image signal of a still image in which signal degradation and generation of a false signal are suppressed in parallel with generation of a moving image signal.

  2. Description of the Related Art Conventionally, electronic devices such as digital cameras and digital video cameras that can acquire still images and moving images using an imaging device such as a CMOS image sensor have become widespread. For example, in Patent Document 1, the first signal is read from all of the first pixel group in the first frame period. Further, in each of a plurality of consecutive frame periods starting from the first frame period, the second signal is read out from some different pixels in the second pixel group. By synthesizing the first signal and the second signal, an image signal for a still image for one frame is generated.

JP 2010-004175 A

  When a still image signal is generated in parallel with the generation of a moving image signal, signal degradation or a false signal is generated depending on the subject and imaging conditions. For example, when the subject has high luminance, signal deterioration or a false signal occurs in the subject or the surrounding area. In addition, if the amount of light is suppressed in order to suppress signal deterioration and false signals, there may be a problem in capturing moving images. For example, if the aperture value is changed during readout of a still image, the depth of field changes when a moving image is captured. Furthermore, since the brightness of the moving image also changes, it is necessary to change the exposure time and the gain in order to maintain the optimum brightness, which hinders the image quality during the moving image. Further, in the imaging device, it is known that signal deterioration due to a dark signal or the like occurs as the time held in the charge holding unit is longer, and the signal deterioration due to the dark signal or the like varies depending on temperature.

  Accordingly, an object of the present technology is to provide an imaging device, a signal processing method, and an electronic device that can generate a still image signal in which signal degradation and generation of a false signal are suppressed in parallel with generation of a moving image signal. And

  A first aspect of the technology includes a pixel array unit in which a plurality of pixels are arrayed in a row direction and a column direction, a drive unit that drives the plurality of pixels to perform a charge accumulation operation in an accumulation period, When capturing a moving image, when reading a first signal based on the charge accumulated during the accumulation period from the first pixel group constituting the moving image of the pixel array unit, and capturing a still image in parallel with the moving image A first signal based on the charge accumulated in the accumulation period from the first pixel group, and a charge accumulated in the accumulation period from the second pixel group excluding the first pixel group in the pixel array unit. And a readout unit that reads out the second signal based on the second pixel group and changes a readout order of the second signal from the second pixel group.

  In this technique, a plurality of pixels in the pixel array unit arranged in the row direction and the column direction are driven by the driving unit so as to perform the charge accumulation operation in the accumulation period. Here, when capturing a moving image, the first signal based on the charge accumulated in the accumulation period is read from the first pixel group constituting the moving image in a plurality of pixels, and a still image is captured in parallel with the moving image. The first signal based on the charge accumulated from the first pixel group during the accumulation period and the second signal based on the charge accumulated during the accumulation period from the second pixel group excluding the first pixel group from the plurality of pixels. Signal reading is performed by the reading unit. In addition, the reading order of the second signal from the second pixel group is performed in descending order of priority. The priority is set based on subject information, for example, information indicating the brightness of the subject, information indicating the distance to the subject, temperature distribution information obtained in advance, and user setting information. Further, an image signal for one frame of the moving image is generated from the read first signal, and a still image signal is generated by synthesizing the read first signal and the second signal. The

  According to a second aspect of the present technology, when a plurality of pixels arranged in a row direction and a column direction are driven to perform a charge accumulation operation in an accumulation period, and when a moving image is captured, the plurality of pixels In the case where the first signal based on the electric charge accumulated in the accumulation period is read from the first pixel group constituting the moving image and the still image is picked up in parallel with the moving image, the first pixel group during the accumulation period Reading out a first signal based on the accumulated charge and a second signal based on the charge accumulated in the accumulation period from a second pixel group obtained by removing the first pixel group from the plurality of pixels; A signal processing method including a step of changing a reading order of the second signal from the second pixel group.

  A third aspect of the technology includes a pixel array unit in which a plurality of pixels are arrayed in a row direction and a column direction, a drive unit that drives the plurality of pixels so as to perform a charge accumulation operation in an accumulation period, When capturing a moving image, when reading a first signal based on the charge accumulated during the accumulation period from the first pixel group constituting the moving image of the pixel array unit, and capturing a still image in parallel with the moving image A first signal based on the charge accumulated in the accumulation period from the first pixel group, and a charge accumulated in the accumulation period from the second pixel group excluding the first pixel group in the pixel array unit. A readout unit that reads out the second signal based on the second pixel group and changes a readout order of the second signal from the second pixel group; and a frame of a moving image from the read-out first signal Generating an image signal and A generating unit that combines the extracted first signal and the second signal to generate a still image signal, and a reading setting unit that sets the reading order of the second signal from the second pixel group It is in an electronic device provided with.

  According to this technique, a plurality of pixels arranged in the row direction and the column direction are driven so as to perform the charge accumulation operation in the accumulation period. Here, when capturing a moving image, the first signal based on the charge accumulated in the accumulation period is read from the first pixel group constituting the moving image in a plurality of pixels, and a still image is captured in parallel with the moving image. The first signal based on the charge accumulated from the first pixel group during the accumulation period and the second signal based on the charge accumulated during the accumulation period from the second pixel group excluding the first pixel group from the plurality of pixels. A signal is read out. In addition, the reading order of the second signal from the second pixel group can be changed. Further, an image signal for one frame of the moving image is generated from the read first signal, and a still image signal is generated by synthesizing the read first signal and the second signal. The Therefore, it is possible to seamlessly capture still images while capturing moving images. Furthermore, it is possible to generate still images with good image quality by increasing the priority of pixels that are likely to cause signal degradation and generation of false signals, and reading the second signal from the second pixel group in descending order of priority. become.

It is a figure which shows the structure of the electronic device using an imaging device. It is the figure which illustrated the composition of the CMOS (Complementary Metal Oxide Semiconductor) image sensor. It is the figure which illustrated the circuit structure of the unit pixel. It is a figure for demonstrating operation | movement of an electronic device. FIG. 6 is a diagram illustrating a part of a signal supplied from a vertical driving unit 122 to a pixel array unit 121 in a frame F1. It is a figure which shows the line from which a pixel signal is read at the time of the production | generation of a moving image. FIG. 6 is a diagram illustrating a part of a signal supplied from a vertical driving unit 122 to a pixel array unit 121 in a frame F1. It is a flowchart which shows operation | movement of an electronic device.

Hereinafter, embodiments for carrying out the present technology will be described. The description will be given in the following order.
1. 1. Configuration of electronic device 2. Configuration of imaging unit Electronic device operation

<1. Configuration of electronic equipment>
FIG. 1 shows a configuration of an electronic device, for example, a camera using the imaging device of the present technology. The electronic device 10 includes a lens unit 11, an imaging unit 12, a signal processing unit 13, a memory unit 14, a display unit 15, a recording / playback processing unit 16, an imaging control unit 17, a lens control unit 18, and a user interface (I / F) unit. 19. A system control unit 20 is provided.

  The lens unit 11 is configured using a focus lens, a zoom lens, or the like. The lens unit 11 forms a subject optical image on the imaging surface of the imaging unit 12.

  The imaging unit 12 is configured using an imaging device. The imaging unit 12 performs photoelectric conversion, generates an image signal corresponding to the subject optical image, and outputs the image signal to the signal processing unit 13. In addition, the imaging unit 12 performs a noise removal process on the generated image signal, a gain adjustment that makes the signal level of the image signal a desired signal level, a process of converting an analog image signal into a digital image signal, and the like. .

  The signal processing unit 13 performs camera process processing or the like on the digital image signal output from the imaging unit 12. For example, the signal processing unit 13 performs nonlinear processing such as gamma correction and knee correction, color correction processing, and contour enhancement processing on the image signal. The signal processing unit 13 is connected to a memory unit 14 for storage. The signal processing unit 13 performs camera process processing and the like using the memory unit 14.

  The display unit 15 is configured using a liquid crystal display element or an organic EL (Electro Luminescence) display element. The display unit 15 displays a captured image such as a moving image or a still image based on the image signal output from the signal processing unit 13. Further, the display unit 15 performs menu display for setting operations, functions, and the like of the electronic device 10 based on a control signal from the system control unit 20, information display indicating the operation and setting state of the electronic device 10, and the like.

  The recording / playback processing unit 16 records the image signal output from the signal processing unit 13 on a recording medium. Also, the image signal recorded on the recording medium is read and output to an external device. The recording medium may be detachable like a memory card, an optical disk, a magnetic tape or the like, or may be a fixed type hard disk device or a semiconductor memory module. Further, the recording / playback processing unit 16 may be provided with an encoder or a decoder to perform compression encoding or expansion decoding of the image signal, and record the encoded signal on the recording medium.

  The imaging control unit 17 controls the operation of the imaging unit 12 to generate a still image signal in parallel with a moving image signal or a moving image signal. When generating an image signal of a still image, the imaging control unit 17 generates control information for setting the priority of a line from which the pixel signal is read and supplies the control information to the imaging unit 12, thereby Control is performed so that the imaging unit 12 reads pixel signals from a high line.

  The lens control unit 18 performs drive control of the focus lens of the lens unit 11 so that a focused moving image or still image can be obtained. Further, the angle of view is adjusted by driving the zoom lens of the lens unit 11 in accordance with a user operation.

  The user interface unit 19 includes operation switches, operation buttons, and the like. The user interface unit 19 generates an operation signal corresponding to the user operation and outputs it to the system control unit 20.

  The system control unit 20 includes, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). The CPU reads and executes the control program stored in the ROM as necessary. The ROM stores in advance programs executed by the CPU, data necessary for various processes, and the like. The RAM is a memory used as a so-called work area that temporarily stores intermediate results of processing. The ROM or RAM stores information such as various setting parameters and correction data. The system control unit 20 controls each unit according to an operation signal from the user interface unit 19 and causes the electronic device 10 to perform an operation corresponding to the user operation.

<2. Configuration of imaging unit>
FIG. 2 illustrates a configuration of an imaging device used in the imaging unit 12, for example, a complementary metal oxide semiconductor (CMOS) image sensor.

  The CMOS image sensor 120 includes a pixel array unit 121, a vertical driving unit 122, a column processing unit 123, a horizontal driving unit 124, a timing control unit 125, and a constant current source 126. Further, the CMOS image sensor 120 includes an image signal generation processing unit 127 and an output interface (I / F) unit 128.

  In the pixel array unit 121, unit pixels having photoelectric conversion elements, for example, photodiodes, that generate and store charges (light reception signals) according to incident light are two-dimensionally arranged in a matrix. The unit pixel may be simply referred to as “pixel”.

  The vertical drive unit 122 includes a shift register, an address decoder, and the like, and drives each pixel of the pixel array unit 121 so as to perform a charge accumulation operation in the accumulation period based on a control signal from the timing control unit 125. . Further, the vertical drive unit 122 reads out the pixel signal based on the control signal from the timing control unit 125. Further, the vertical drive unit 122 is configured to have two scanning systems, a reading scanning system and a sweeping scanning system, although illustration of a specific configuration thereof is omitted.

  The readout scanning system selectively scans the unit pixels of the pixel array unit 121 sequentially in units of rows in order to read out signals from the unit pixels. The sweep-out scanning system performs sweep-out scanning with respect to a readout row in which readout scanning is performed by the readout scanning system.

  By the sweep scanning by the sweep scanning system, unnecessary charges are swept from the photoelectric conversion elements of the unit pixels in the readout row. A so-called electronic shutter operation is performed by sweeping out unnecessary charges by the sweep-out scanning system. Here, the electronic shutter operation refers to an operation in which the photoelectric charge of the photoelectric conversion element is discarded and a new exposure is started (photocharge accumulation is started).

  The signal read by the reading operation by the reading scanning system corresponds to the amount of light incident after the immediately preceding reading operation or electronic shutter operation. The period from the read timing by the previous read operation or the sweep timing by the electronic shutter operation to the read timing by the current read operation is the photocharge accumulation time (exposure time) in the unit pixel.

  The pixel signal output from each unit pixel of the line selectively scanned by the vertical driving unit 122 is supplied to the column processing unit 123. The constant current source 126 supplies a bias current to each pixel and is arranged in each pixel column. The column processing unit 123 performs predetermined signal processing on the pixel signal (light reception signal) output from each unit pixel of the selection line for each pixel column of the pixel array unit 121.

  Specifically, the column processing unit 123 performs noise removal processing such as CDS (Correlated Double Sampling) processing as signal processing. The CDS processing by the column processing unit 123 removes fixed pattern noise unique to the pixel such as reset noise and threshold variation of the amplification transistor. The column processing unit 123 performs analog-digital conversion, converts an analog pixel signal into a digital pixel signal, and outputs the digital pixel signal.

  The horizontal drive unit 124 includes a shift register, an address decoder, and the like, and sequentially selects unit circuits corresponding to the pixel columns of the column processing unit 123. By the selective scanning by the horizontal drive unit 124, the pixel signals subjected to signal processing by the column processing unit 123 are sequentially output to the image signal generation processing unit 127.

  The timing control unit 125 includes a timing generator that generates various timing signals, and drives the vertical driving unit 122, the column processing unit 123, the horizontal driving unit 124, and the like based on the various timing signals generated by the timing generator. Take control. For example, when capturing a moving image, the timing control unit 125 reads the first signal based on the charge accumulated in the accumulation period from the first pixel group constituting the moving image of the pixel array unit 121, and in parallel with the moving image. When capturing a still image, the first signal based on the charge accumulated from the first pixel group during the accumulation period and the second pixel group excluding the first pixel group in the pixel array unit 121 are accumulated during the accumulation period. The second signal based on the charged charge is read out. In addition, the timing control unit 125 can change the reading order of the second signal from the second pixel group. For example, the timing control unit 125 controls the operation of the vertical drive unit 122 so as to read pixel signals from a line with high priority based on the control information from the imaging control unit 17.

  The image signal generation processing unit 127 performs black level offset processing by subtracting a reset level (corresponding to a black level) pixel signal from the read pixel signal. Further, the image signal generation processing unit 127 performs digital clamp processing, gain adjustment, and the like, and outputs the processed pixel signal to the output interface unit 128. In addition, when generating a moving image, the image signal generation processing unit 127 adjusts the number of pixels in the horizontal direction, and performs pixel thinning on pixel signals read out in units of lines from the first pixel group constituting the moving image. Let the number of pixels in the horizontal direction be the number of pixels in the moving image. The image signal generation processing unit 127 outputs the generated moving image signal to the output interface unit 128. Further, when a still image is generated in parallel with the moving image, the image signal generation processing unit 127 stores a pixel signal after pixel signal processing in which pixel thinning of the first pixel group is not performed. Also, the image signal generation processing unit 127 generates a still image signal by rearranging the pixel signals that have not been subjected to pixel thinning of the first pixel group line and the pixel signals of the second pixel group line in the order of the line numbers. To do. The image signal generation processing unit 127 outputs the generated still image signal to the output interface unit 128.

  The output interface unit 128 outputs the moving image signal and the still image signal supplied from the image signal generation processing unit 127.

  FIG. 3 illustrates a circuit configuration of a unit pixel of the imaging unit 12. In FIG. 3, a circuit configuration within a range surrounded by a dotted line is a circuit configuration example of the unit pixel 300. The unit pixel 300 includes a photodiode 301, a discharge transistor 302, transfer gates 303 and 305 including transistors, a reset transistor 307, an amplification transistor 308, a selection transistor 309, a pixel memory unit 304, and a floating diffusion region 306.

  The photodiode 301 which is a photoelectric conversion element has an anode electrode grounded and a cathode electrode connected to the source electrode of the discharge transistor 302 and the drain electrode of the transfer gate 303.

  The drain transistor 302 has a gate electrode connected to the vertical drive unit 122, a source electrode connected to the cathode of the photodiode 301 and the drain electrode of the transfer gate 303, and a drain electrode connected to the power supply VDDofd.

  The transfer gate 303 has a drain electrode connected to the anode of the photodiode 301 and the source electrode of the discharge transistor 302, and a source electrode connected to one electrode of the pixel memory unit 304 and the source electrode of the transfer gate 305. The transfer gate 303 has a gate electrode connected to the vertical drive unit 122, and the other electrode of the pixel memory unit 304 is grounded.

  The transfer gate 305 has a drain electrode connected to the source electrode of the transfer gate 303 and one electrode of the pixel memory unit 304. The transfer gate 305 has a source electrode connected to the source electrode of the reset transistor 307, the gate electrode of the amplification transistor 308, and one electrode of the floating diffusion region 306. Further, the transfer gate 305 has a gate electrode connected to the vertical drive unit 122 and the other electrode of the floating diffusion region 306 is grounded.

  The reset transistor 307 has a gate electrode connected to the vertical drive unit 122, and a source electrode connected to the source electrode of the transfer gate 305, one electrode of the floating diffusion region 306, and the gate electrode of the amplification transistor 308. The reset transistor 307 has a drain electrode connected to the power supply VDDrst.

  The amplification transistor 308 has a gate electrode connected to the source electrode of the transfer gate 305, one electrode of the floating diffusion region 306, and the source electrode of the reset transistor 307, and a drain electrode connected to the power supply VDDamp. The amplification transistor 308 has a source electrode connected to the drain electrode of the selection transistor 309.

  The selection transistor 309 has a gate electrode connected to the vertical drive unit 122, a drain electrode connected to the source electrode of the amplification transistor 308, and a source electrode connected to the vertical signal line VSL.

  In the unit pixel 300, the discharge transistor 302 is turned on when the discharge pulse LOFG is supplied from the vertical driving unit 122, and discharges the charge in the photodiode 301. In this way, by turning on the discharge transistor 302, excessive light is incident on the photodiode 301 and is saturated with the charge, and the charge exceeding the saturation charge amount is prevented from overflowing to the periphery.

  The transfer gate 303 is turned on when the transfer pulse LTRG is supplied from the vertical driving unit 122, and transfers the charge accumulated in the photodiode 301 to the pixel memory unit 304.

  The transfer gate 305 is turned on when the transfer pulse LROG is supplied from the vertical driving unit 122, and transfers the charge accumulated in the pixel memory unit 304 to the floating diffusion region 306.

  The reset transistor 307 is turned on when the reset pulse LRST is supplied from the vertical driving unit 122 and discharges the charge held in the floating diffusion region 306 to reset the pixel.

  The amplification transistor 308 amplifies and outputs a light reception signal that is a charging voltage of the floating diffusion region 306 applied to the gate electrode.

  The selection transistor 309 is turned on when the selection pulse LSEL is supplied from the vertical driving unit 122, and outputs the amplified light reception signal to the vertical signal line VSL. Note that a constant current source 126 is connected to the vertical signal line VSL, and the value of the current flowing through the vertical signal line VSL is kept constant.

  Note that the imaging apparatus is not limited to the configuration shown in FIG. For example, the synthesis processing performed by the image signal generation processing unit 127 may be performed by the signal processing unit 13 of the electronic device 10. Alternatively, the readout order of the second signal from the second pixel group may be set by the imaging control unit 17 and the result may be supplied to the timing control unit 125. Further, the imaging apparatus may be a single device by integrating each part into an integrated circuit.

<3. Operation of electronic equipment>
Next, the operation of the electronic device will be described. When capturing a moving image, the electronic device 10 performs thinning-out reading so as to satisfy the number of lines and the frame rate necessary for the moving image, and generates a moving image signal. In addition, the electronic device 10 generates an image signal of a still image using a portion thinned out in a moving image when capturing a still image. In order to simplify the description, the pixel array unit 121 of the imaging unit 12 has 24 pixels in the horizontal direction and 18 lines in the vertical direction.

  When the electronic device 10 generates an image signal of a moving image, the electric charge accumulated in the photodiode 301 in each unit pixel in the line is stored in the pixel memory for each line of the first pixel group used for generating the moving image in each frame. Forward to the unit 304. In addition, the electronic device 10 transfers the charge accumulated in the pixel memory unit 304 to the floating diffusion region 306 and transmits a light reception signal corresponding to the charge accumulated in the floating diffusion region 306 for each line of the first pixel group. Are sequentially output to the vertical signal line VSL. In this way, the electronic device 10 reads out the pixel signal for each line of the first pixel group, and generates a pixel signal whose vertical direction is the number of moving image lines. Furthermore, the electronic device 10 performs pixel thinning for each line with respect to the pixel signal whose vertical direction is the number of lines of the moving image, and the moving image image in which the horizontal direction has the desired number of pixels and the vertical direction has the desired number of lines. Generate a signal.

  When the electronic device 10 generates image signals of a moving image and a still image, the electronic device 10 transfers charges accumulated in the photodiode 301 of each unit pixel in the pixel array unit 121 to the pixel memory unit 304.

  When generating an image signal of a moving image, the electronic device 10 transfers the charge accumulated in the pixel memory unit 304 to the floating diffusion region 306 as described above. In addition, the electronic device 10 sequentially outputs a light reception signal corresponding to the electric charge accumulated in the floating diffusion region 306 to the vertical signal line VSL for each line, and generates a pixel signal whose vertical direction is the number of moving image lines. To do. Furthermore, the electronic device 10 performs pixel thinning for each line on the generated pixel signal, and generates a moving image signal having a desired number of pixels in the horizontal direction and a desired number of lines in the vertical direction.

  When generating an image signal of a still image, the electronic device 10 uses a period from the end of moving image reading to the start of the next moving image reading, for example, using a blanking period for each line of the second pixel group. Read out the received light signal. That is, for each line of the second pixel group, the electronic device 10 transfers the charge accumulated in the pixel memory unit 304 in each unit pixel in the line to the floating diffusion region 306 and accumulates it in the floating diffusion region 306. The received light signal corresponding to the charged electric charges is sequentially output to the vertical signal line VSL for each line. Further, the electronic device 10 reads the pixel signal of the first pixel group line (pixel signal before thinning) read out during the generation of the moving image and the second pixel group line read out using the blanking period. Are rearranged in the order of line numbers, and a still image signal having a larger number of pixels than a moving image is generated. In addition, the electronic device 10 sets a priority for the line of the second pixel group based on one or a plurality of priority factors, and reads out the received light signal from the high priority line, that is, reads out the pixel signal. .

  FIG. 4 is a diagram for explaining the operation of the electronic apparatus. FIG. 4 illustrates the case where frames F1 to F7 generate moving image signals, and generate still image signals based on the charges accumulated during the exposure period in frame F2.

  For example, the electronic device 10 supplies the transfer pulse LTRG to the transfer gate 303 of each unit pixel in the readout target line at the head of the frame to turn it on, so that the charge accumulated in the photodiode 301 during the exposure period. Is transferred to the pixel memory unit 304. Note that the readout target line corresponds to the line of the first pixel group when generating the image signal of the moving image, and the line of the first pixel group and the second pixel group when generating the image signal of the moving image and the still image. Equivalent to.

  FIG. 5 shows a part of signals supplied from the vertical drive unit 122 to the pixel array unit 121 in the frame F1. In the frame F1, the electronic device 10 transmits a transfer pulse LTRG to the transfer gate 303 of each unit pixel in the first pixel group line (first line, fourth line, seventh line,...) At the head of the frame. Then, the charge accumulated in the photodiode 301 during the exposure period is transferred to the pixel memory unit 304.

  The electronic device 10 sequentially selects the lines of the first pixel group as shown in FIG. 4 and reads out the pixel signals of the selected lines. For example, as shown in FIG. 5, the pixel is reset by supplying a reset pulse LRST to the reset transistor 307 of each unit pixel in the first line that is a line of the first pixel group. Thereafter, by supplying a transfer pulse LROG to the transfer gate 305 of each unit pixel in the first line, the charge accumulated in the pixel memory unit 304 is transferred to the floating diffusion region 306. Further, by supplying the selection pulse LSEL to the selection transistor 309 of each unit pixel in the first line, the light reception signal corresponding to the electric charge accumulated in the floating diffusion region 306 is vertically transmitted in each unit pixel in the first line. Output to the signal line VSL. In this way, the pixel signal of the first line is read out. The electronic device 10 reads the pixel signal of the fourth line in the same manner after reading the pixel signal of the first line. Similarly, by sequentially selecting the lines of the first pixel group and reading the pixel signals, the pixel signals of the lines of the first pixel group are read and the vertical direction is set as shown in FIG. Pixel signals for the number of lines of moving images can be generated.

  Further, as shown in FIG. 5, the electronic device 10 discharges the charges in the photodiode 301 by supplying the discharge pulse LOFG to the discharge transistor 302 of each unit pixel in the first pixel group line immediately before the start of exposure. Then, the exposure operation is performed.

  The electronic device 10 performs the processing as described above to generate a frame image “F1m” of the moving image. In addition, the electronic device 10 performs the same process on the frame F2 and generates a frame image “F2m” of the moving image.

  Next, a case where a moving image and a still image are generated will be described. FIG. 7 shows a part of the signal supplied from the vertical drive unit 122 to the pixel array unit 121 in the frame F3.

  In the frame F3, the electronic device 10 starts generating a still image signal based on the charge accumulated in the exposure period in the frame F2. The electronic device 10 includes a first pixel group line (first line, fourth line, seventh line,...) And a second pixel group line (second line, third line, fifth line) at the head of the frame. , Sixth line,...), A transfer pulse LTRG is supplied to the transfer gate 303 of each unit pixel, and the charge accumulated in the photodiode 301 during the exposure period is transferred to the pixel memory unit 304.

  The electronic device 10 sequentially selects the lines of the first pixel group as shown in FIG. 4 and reads out the pixel signals of the selected lines. That is, as shown in FIG. 7, the pixel is reset by supplying the reset pulse LRST to the reset transistor 307 of each unit pixel in the first line which is the line of the first pixel group. Thereafter, by supplying a transfer pulse LROG to the transfer gate 305 of each unit pixel in the first line, the charge accumulated in the pixel memory unit 304 is transferred to the floating diffusion region 306. Further, by supplying the selection pulse LSEL to the selection transistor 309 of each unit pixel in the first line, the light reception signal corresponding to the electric charge accumulated in the floating diffusion region 306 is vertically transmitted in each unit pixel in the first line. Output to the signal line VSL. In this way, the pixel signal of the first line is read out. The electronic device 10 reads the pixel signal of the fourth line in the same manner after reading the pixel signal of the first line. Similarly, pixel signals corresponding to the number of lines of moving images can be generated by sequentially selecting the lines of the first pixel group and reading out the pixel signals.

  Further, as shown in FIG. 7, the electronic device 10 discharges the charge in the photodiode 301 by supplying the discharge pulse LOFG to the discharge transistor 302 of each unit pixel in the first pixel group line immediately before the start of exposure. Then, the exposure operation is performed. The electronic device 10 performs the processing as described above to generate the frame images “F3m”, “F4m”,.

  Furthermore, the electronic device 10 reads the pixel signal of the line selected from the second pixel group using a period from the end of the moving image reading to the start of the next moving image reading. In addition, the electronic device 10 selects a line of the second pixel group so that a still image with good image quality can be generated. For example, when the subject has high luminance, if the transfer gates 303 and 305 and the like malfunction due to high luminance light, signal deterioration and false signals are generated due to discharge of charges stored in the pixel memory unit 304. In addition, as the time for which the pixel memory unit 304 holds charges is longer, signal degradation due to a dark signal or the like occurs. Since such signal degradation depends on temperature, the degree of signal degradation is large when the imaging unit 12 is provided close to a component or circuit that generates a large amount of heat.

  Therefore, the electronic device 10 sets the priority of the reading order with respect to the lines of the second pixel group, and increases the priority of the lines including the subject with high brightness, the lines in the high temperature region, and the like. By reading out the pixel signal from the line with a high degree, a still image with good image quality can be generated. In addition, since the importance of the subject close to the subject far away is often higher, the priority of the subject close to the subject is increased.

  The priority is set using a detection value relating to one or more priority factors. The priority factor includes subject information, for example, information on the luminance of the subject (luminance information), information on the distance to the subject (distance information), information on the positional relationship with a component or circuit that generates a large amount of heat (temperature distribution information), Information (user setting information) indicating the set priority is used.

  When luminance information is used as the priority factor, the electronic device 10 generates a detection value by detecting the luminance of the subject with the signal processing unit 13 when capturing a moving image, for example. When distance information is used as the priority factor, the electronic device 10 generates a detection value by measuring the distance to the subject using a distance measuring sensor or the like. When temperature-related information is used as a priority factor, a detection value is generated in advance according to the heat generation status of components and circuits that are close to each other at the development and design stage of the electronic device.

  When calculating the priority from a plurality of priority factors for each line, the electronic device 10 performs weighting for each priority factor. For example, the detection value of the priority factor (PFa) on the n line is “Xna”, and the detection value of the priority factor (PFb) on the n line is “Xnb”. Further, the weighting value of the priority factor (PFa) on the n line is “Yna”, and the weighting value of the priority factor (PFb) on the n line is “Ynb”. The detection value is a large value as the brightness increases, a large value as the distance is shortened, and a large value as the temperature is increased. Furthermore, the weighting value is increased as the weighting is increased.

  The electronic device 10 calculates Equation (1) using the detection value and the weighting value corresponding to the priority factor, and calculates the priority “Zn” of the n line.

Zn = (Xna × Yna) + (Xnb × Ynb) (1)
The electronic device 10 compares the priority calculated for each line of the second pixel group, selects the lines in descending order of priority in the second pixel group, and selects the pixel signal of the selected line. Is read out.

  For example, the priority is 5th line> 6th line> 3rd line> 2nd line> 8th line> 9th line> 11th line> 12th line> 14th line> 15th line> 17th line> 17th line It is assumed that the order decreases in the order of 18 lines. In this case, as shown in FIG. 4, the lines of the second pixel group are selected in order of priority, and the pixel signals of the selected lines are read out. That is, as shown in FIG. 7, the pixel is reset by supplying the reset pulse LRST to the reset transistor 307 of each unit pixel in the fifth line which is the highest priority line in the second pixel group. Thereafter, by supplying a transfer pulse LROG to the transfer gate 305 of each unit pixel in the fifth line, the charge accumulated in the pixel memory unit 304 is transferred to the floating diffusion region 306. Further, by supplying the selection pulse LSEL to the selection transistor 309 of each unit pixel in the fifth line, the light reception signal corresponding to the electric charge accumulated in the floating diffusion region 306 is vertically applied to each unit pixel in the fifth line. Output to the signal line VSL.

  Next, the pixel is reset by supplying the reset pulse LRST to the reset transistor 307 of each unit pixel in the sixth line. Thereafter, by supplying a transfer pulse LROG to the transfer gate 305 of each unit pixel in the sixth line, the charges accumulated in the pixel memory unit 304 are transferred to the floating diffusion region 306. Further, by supplying the selection pulse LSEL to the selection transistor 309 of each unit pixel in the sixth line, the light reception signal corresponding to the electric charge accumulated in the floating diffusion region 306 is vertically transmitted in each unit pixel in the sixth line. Output to the signal line VSL.

  Similarly, the lines of the second pixel group are sequentially selected in descending order of priority, and pixel signals are read out. If the readout of the pixel signals of the second pixel group line is not completed within the period of the frame F3, the pixel signals of the remaining lines of the second pixel group are read out in the next frame. In FIG. 4, pixel signals of the eighth, ninth, eleventh, and twelfth lines that are the lines of the second pixel group are read out in the frame F4, and the fourteenth, fifteenth, seventeenth, and the lines of the second pixel group are displayed in the frame F5. 18 line pixel signals are read out.

  The number of frames from which the pixel signals of the lines of the second pixel group are read out is set according to, for example, the number of moving image readout lines, the blanking period, the number of lines output in a still image, and the like.

  Further, when selecting the line of the second pixel group according to the priority, the pixel signals supplied to the column processing unit 123 are not in line order. Therefore, the image signal generation processing unit 127 rearranges the pixel signals in line units selected according to the priority into the pixel signals in the line order based on the priority calculated for each line. Further, the image signal generation processing unit 127 has more pixels than moving images by rearranging the pixel signals of the first pixel group line and the image signals of the second pixel group line rearranged in the line order. An image signal of a still image is generated.

  When the processing as described above is performed, the pixel signals of the lines of the second pixel group can be read during the periods of the frames F3 to F5. Therefore, the pixel signals of the lines of the first pixel group read out in the frame F3 and the pixel signals of the lines of the second pixel group read out in the periods of the frames F3 to F5 are rearranged in line order, so that FIG. As shown in FIG. 5, a still image frame image “SP” having more pixels than a moving image can be generated. Furthermore, since pixel signals are read out from high priority lines, it is possible to reduce signal deterioration and color false signal generation compared to reading out pixel signals in line order, and still images with good image quality can be obtained. Can be generated.

  FIG. 8 is a flowchart showing the operation of the electronic device described above. In step ST1, the electronic device 10 determines whether a still image is generated. If a still image generation instruction has not been issued during the generation of a moving image, the electronic device 10 proceeds to step ST2, and if a still image generation instruction has been issued, for example, the user interface unit 19 performs a shutter operation. If yes, the process proceeds to step ST5.

  In step ST2, the electronic device 10 performs charge transfer in the first pixel group. In the unit pixel of the first pixel group, the electronic device 10 transfers the charge generated by the photodiode 301 to the pixel memory unit 304 and proceeds to step ST3.

  In step ST3, the electronic device 10 reads a signal for each line from the first pixel group. For each line in the first pixel group, the electronic device 10 reads out a pixel signal corresponding to the electric charge accumulated in the pixel memory unit 304 by a unit pixel in the line. Further, the electronic device 10 performs signal processing such as noise removal and black level adjustment on the signal read from the first pixel group for each line, and proceeds to step ST4.

  In step ST4, the electronic device 10 generates an image signal of one frame of moving image. The electronic device 10 performs pixel thinning on the signal read for each line from the first pixel group, generates an image signal of one frame of the moving image, and returns to step ST1.

  When a still image generation instruction is issued and the process proceeds from step ST1 to step ST5, the electronic device 10 performs charge transfer in the first and second pixel groups. In the unit pixels of the first and second pixel groups, the electronic device 10 transfers the charge generated by the photodiode 301 to the pixel memory unit 304 and proceeds to step ST6.

  In step ST6, the electronic device 10 reads a signal for each line from the first pixel group. For each line in the first pixel group, the electronic device 10 reads out a pixel signal corresponding to the electric charge accumulated in the pixel memory unit 304 by a unit pixel in the line. Further, the electronic device 10 performs signal processing such as noise removal and black level adjustment on the signal read from the first pixel group for each line, and proceeds to step ST7.

  In step ST7, the electronic device 10 stores the pixel signal. The electronic device 10 stores the pixel signal read from the first pixel group in order to generate the image signal of the still image, and proceeds to step ST8.

  In step ST8, the electronic device 10 generates an image signal of one frame of moving images. The electronic device 10 performs pixel thinning on the signal read for each line from the first pixel group, generates an image signal of one frame of the moving image, and proceeds to step ST9.

  In step ST <b> 9, the electronic device 10 performs signal readout for each line from the second pixel group in descending order of importance. The electronic device 10 reads out a pixel signal corresponding to the electric charge accumulated in the pixel memory unit 304 by a unit pixel in one line from a line having high importance in the second pixel group. Further, the electronic device 10 performs signal processing such as noise removal and black level adjustment on the signal read from the second pixel group, and proceeds to step ST10.

  In step ST10, the electronic device 10 determines whether the signal reading of the second pixel group is completed. The electronic device 10 proceeds to step ST11 when there remains a line from which signals are not read in the second pixel group, and proceeds to step ST14 when signal reading is completed for all lines of the second pixel group. .

  In step ST11, the electronic device 10 determines whether it is the next frame. The electronic device 10 proceeds to step ST12 when the next frame period is completed before the signal reading of one line from the second pixel group is completed, and the signal of one line is output from the second pixel group before the next frame period. When the reading is completed, the process returns to step ST9.

  In step ST12, the electronic device 10 performs charge transfer using the first pixel group. In the unit pixel of the first pixel group, the electronic device 10 transfers the charge generated by the photodiode 301 to the pixel memory unit 304, and proceeds to step ST13.

  In step ST13, the electronic device 10 reads a signal for each line from the first pixel group. For each line in the first pixel group, the electronic device 10 reads out a pixel signal corresponding to the electric charge accumulated in the pixel memory unit 304 by a unit pixel in the line. In addition, the electronic device 10 performs signal processing such as noise removal and black level adjustment on the signal read from the first pixel group for each line, and returns to step ST8.

  When it is determined in step ST10 that signal readout has been completed for all the lines of the second pixel group and the process proceeds to step ST14, the electronic device 10 generates a still image signal. The electronic device 10 rearranges the stored pixel signals of the first pixel group in line units and the line-by-line pixel signals read out from the second pixel group in the order of the most important lines in the order of the line numbers. The image signal is generated and the process returns to step ST1.

  By performing the processing as described above, a still image signal can be generated in parallel with the generation of the moving image signal. In addition, since the pixel signal is read from a line with high priority in the generation of a still image, for example, the importance of a line close to a line with high luminance or a component with a large amount of heat generation is increased. In this way, signal degradation and generation of false signals are reduced as compared with the case where pixel signals are read in line order, and a still image with good image quality can be generated. In addition, since signal degradation and generation of false signals can be reduced, there is no need to install a circuit that corrects signal degradation or false signals in electronic devices, shortening processing time, increasing power consumption due to increased correction circuits, and increasing power consumption It is possible to prevent a temperature rise due to.

  Note that the present technology should not be construed as being limited to the embodiments of the technology described above. The embodiments of this technology disclose the present technology in the form of examples, and it is obvious that those skilled in the art can make modifications and substitutions of the embodiments without departing from the gist of the present technology. In other words, in order to determine the gist of the present technology, the claims should be taken into consideration.

In addition, the imaging device of this technique can also take the following structures.
(1) a pixel array unit in which a plurality of pixels are arrayed in the row direction and the column direction;
A driving unit that drives the plurality of pixels to perform a charge accumulation operation in an accumulation period;
When capturing a moving image, when reading a first signal based on the charge accumulated during the accumulation period from the first pixel group constituting the moving image of the pixel array unit, and capturing a still image in parallel with the moving image A first signal based on the charge accumulated in the accumulation period from the first pixel group, and a charge accumulated in the accumulation period from the second pixel group excluding the first pixel group in the pixel array unit. A readout unit that performs readout of the second signal based on the second pixel group, and enables the readout order of the second signal from the second pixel group to be changed;
An imaging apparatus comprising:
(2) The readout unit sets a priority for the second pixel group, and reads out a second signal based on charges accumulated in the accumulation period from a pixel having a high priority (1). The imaging device described in 1.
(3) The imaging device according to (2), wherein the reading unit sets the priority according to subject information.
(4) The imaging apparatus according to (3), wherein the subject information is information indicating luminance of the subject.
(5) The imaging device according to (3) or (4), wherein the subject information is information indicating a distance to the subject.
(6) The imaging device according to any one of (2) to (5), wherein the reading unit sets the priority according to temperature distribution information obtained in advance.
(7) The imaging device according to any one of (2) to (6), wherein the reading unit sets the priority based on user setting information.
(8) An image signal for one frame of a moving image is generated from the read first signal, and a still image signal is generated by combining the read first signal and the second signal. The imaging device according to any one of (2) to (7), further including an image signal generation processing unit to be generated.

  In the imaging apparatus, the signal processing method, and the electronic apparatus according to this technique, a plurality of pixels arranged in the row direction and the column direction are driven so as to perform the charge accumulation operation in the accumulation period. Here, when capturing a moving image, the first signal based on the charge accumulated in the accumulation period is read from the first pixel group constituting the moving image in a plurality of pixels, and a still image is captured in parallel with the moving image. The first signal based on the charge accumulated from the first pixel group during the accumulation period and the second signal based on the charge accumulated during the accumulation period from the second pixel group excluding the first pixel group from the plurality of pixels. A signal is read out. In addition, the reading order of the second signal from the second pixel group can be changed. Further, an image signal for one frame of the moving image is generated from the read first signal, and a still image signal is generated by synthesizing the read first signal and the second signal. The Therefore, it is possible to seamlessly capture still images while capturing moving images. Furthermore, it is possible to generate still images with good image quality by increasing the priority of pixels that are likely to cause signal degradation and generation of false signals, and reading the second signal from the second pixel group in descending order of priority. become. For this reason, it is suitable for an electronic device capable of shooting a subject, for example, an electronic device such as a camera, a mobile phone, or a mobile terminal device.

  DESCRIPTION OF SYMBOLS 10 ... Electronic device, 11 ... Lens part, 12 ... Imaging part, 13 ... Signal processing part, 14 ... Memory part, 15 ... Display part, 16 ... Recording / reproducing process 17, imaging control unit, 18 lens control unit, 19 user interface unit, 20 system control unit, 120 CMOS image sensor, 121 pixel array unit, 122: Vertical drive unit, 123: Column processing unit, 124: Horizontal drive unit, 125 ... Timing control unit, 126 ... Constant current source, 127 ... Image signal generation processing unit, 128: output interface unit, 300: unit pixel, 301: photodiode, 302 ... discharge transistor, 303, 305 ... transfer gate, 304 ... pixel memory unit, 305 ... Transfer gate, 06 ... floating diffusion region, 307 ... reset transistor, 308 ... amplifier transistor, 309 ... selection transistor

Claims (10)

A pixel array section in which a plurality of pixels are arrayed in the row direction and the column direction;
A driving unit that drives the plurality of pixels to perform a charge accumulation operation in an accumulation period;
When capturing a moving image, when reading a first signal based on the charge accumulated during the accumulation period from the first pixel group constituting the moving image of the pixel array unit, and capturing a still image in parallel with the moving image A first signal based on the charge accumulated in the accumulation period from the first pixel group, and a charge accumulated in the accumulation period from the second pixel group excluding the first pixel group in the pixel array unit. A readout unit that performs readout of the second signal based on the second pixel group, and enables the readout order of the second signal from the second pixel group to be changed;
An imaging apparatus comprising: 2. The imaging according to claim 1, wherein the readout unit sets a priority for the second pixel group, and reads out a second signal based on charges accumulated in the accumulation period from a pixel having a high priority. apparatus. The imaging apparatus according to claim 2, wherein the reading unit sets the priority according to subject information. The imaging device according to claim 3, wherein the subject information is information indicating luminance of the subject. The imaging device according to claim 3, wherein the subject information is information indicating a distance to the subject. The imaging device according to claim 2, wherein the reading unit sets the priority according to temperature distribution information obtained in advance. The imaging apparatus according to claim 2, wherein the reading unit sets the priority based on user setting information. An image that generates an image signal for one frame of a moving image from the read first signal and combines the read first signal and the second signal to generate a still image signal. The imaging apparatus according to claim 1, further comprising a signal generation processing unit. Driving a plurality of pixels arranged in a row direction and a column direction so as to perform a charge accumulation operation in an accumulation period;
When capturing a moving image, the first signal based on the charge accumulated during the accumulation period is read from the first pixel group constituting the moving image in the plurality of pixels, and a still image is captured in parallel with the moving image. A first signal based on the charge accumulated in the accumulation period from the first pixel group, and a charge accumulated in the accumulation period from the second pixel group excluding the first pixel group from the plurality of pixels. A signal processing method including: reading a second signal based on the second signal group, and changing a reading order of the second signal from the second pixel group. A pixel array section in which a plurality of pixels are arrayed in a row direction and a column direction; a drive section that drives the plurality of pixels so as to perform a charge accumulation operation in an accumulation period;
When capturing a moving image, when reading a first signal based on the charge accumulated during the accumulation period from the first pixel group constituting the moving image of the pixel array unit, and capturing a still image in parallel with the moving image A first signal based on the charge accumulated in the accumulation period from the first pixel group, and a charge accumulated in the accumulation period from the second pixel group excluding the first pixel group in the pixel array unit. A readout unit that performs readout of the second signal based on the second pixel group, and enables the readout order of the second signal from the second pixel group to be changed;
An image that generates an image signal for one frame of a moving image from the read first signal and combines the read first signal and the second signal to generate a still image signal. A signal generation processing unit;
An electronic apparatus comprising: a read setting unit that sets a read order of the second signal from the second pixel group.

Images