JP2009171545A - Image pickup system, method for driving image pickup elements, program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image pickup apparatus capable of obtaining an image, in which a refresh cycle of the apparatus is shorter, at high speed while a constant refresh cycle is maintained, when image signals are read from different image regions, and to provide a driving method thereof. <P>SOLUTION: An image pickup system includes an image pickup element with an image pickup region in which a plurality of pixels are arranged in a matrix, and a controller configured to control reading of signals from the pixels. The controller divides a first frame period in which a first image is read from the image pickup element into a plurality of divided frame periods, including first and second divided frame periods. When the number of pixels included in the first image is larger than the number of pixels included in a second image, a second frame period required for reading all signals from the pixels included in the second image is inserted between the first and second divided frame periods. Further, a refresh cycle of the second image is shorter than a refresh cycle of the first image. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a method for driving an image sensor.

  As an imaging apparatus capable of obtaining an image signal from different imaging regions of one imaging element and forming an image, for example, there are Patent Document 1 and Patent Document 2 below.

  Patent Document 1 includes a skip mode in which pixels are thinned and scanned as an operation for reading a signal from an image sensor, and a block mode in which some (predetermined range) pixels are scanned without being thinned. It is described that image data obtained by scanning in this mode is read out one screen at a time. Further, it is described that the images obtained in the skip mode and the block mode are not limited to being read alternately on one screen, and the ratio of the scanning time of both may be changed.

  Patent Document 2 reads out a partial area of a pixel array in which pixels are arranged two-dimensionally and reads out signals of all pixels included in the area without thinning out another partial area. Is described. In order to acquire the thinned image of the entire pixel array, only the signal of the pixel at the position corresponding to the thinning scanning out of the signal of the region from which all the pixels are read is read from the buffer memory at the subsequent stage of the image sensor.

Patent Document 3 discloses a technique for realizing autofocus processing at a cycle faster than the cycle of the monitoring image while displaying the monitoring image. When there are a plurality of distance measurement frames as shown in FIG. 8B of Patent Document 3, a sequence as shown in FIG. 12 of Patent Document 3 is performed. Specifically, it is disclosed to repeat the operation of reading all of a plurality of distance measurement frames after reading a part of a monitoring image.
JP 09-214836 A JP 2000-032318 A JP 2007-150643 A

  For example, as in Patent Document 1, when the entire screen is thinned and read from the pixels at a low density, and all pixels are read from a part of the region of interest, the total number of pixels of the image sensor increases. The time for reading the entire image tends to be longer. Therefore, as described in Patent Document 1, when images in the skip mode and images in the block mode are alternately output, the update speed of the image in the block mode, which is a partial image that can be read at a high speed, updates the entire image in the skip mode. Slowed by the speed. In applications where the angle of view and focus adjustment are performed simultaneously, there is a need to acquire an image of the region of interest with high definition and to increase the update speed. It is conceivable that an image of a certain area cannot be obtained at a sufficient update speed.

  Further, when the ratio of switching between the skip mode and the block mode is changed, that is, in this case, when the partial image of interest is acquired continuously for a plurality of screens (a plurality of frames) and then the operation for acquiring the entire image is repeated, the partial image The average update period of becomes shorter. However, since the update interval between two partial images acquired with the entire image sandwiched between them, that is, the update cycle is different from the update interval between consecutive partial images acquired without sandwiching the entire image, The problem of non-uniform movement occurs.

  In addition, when the area to be read out as a partial image is large and the entire image is greatly thinned out and read at an extremely low density, the time required to read out the partial image becomes longer than the time required to read out the entire image. The update cycle of the entire image becomes longer depending on the reading speed. In this case, the problem described above also occurs when the ratio of reading out the entire image and the partial image is changed. In other words, there is a problem in that an image having a short read time is slowed down by an update speed of an image having a long read time.

  In the invention of Patent Document 2, since different scanning is performed on different areas within one frame, the update speed of the whole image and the partial image is determined by the total readout time of one frame. In this case, in order to shorten the update interval of the image of interest, it is necessary to improve both update speeds, but it is difficult.

  In the invention of Patent Document 3, if each of a period for reading a part of a monitoring image and a period for reading all of a plurality of distance measurement frames is considered as a subframe, the following problems can be considered. For example, when the monitoring image is divided into four as shown in FIG. 12 of Patent Document 3, a period of 8 subframes is required to update the monitoring image. On the other hand, a period of two subframes is required to update the ranging frame. Therefore, the deviation of the exposure timing becomes large between the divided portions of the monitoring image. This becomes more prominent as the number of divided monitoring images increases.

  The present invention has been made in view of the above-described problems. When reading out image signals of a plurality of different image areas, an image whose update interval is desired to be shortened can be obtained uniformly and at high speed. It is an object of the present invention to provide an imaging apparatus capable of performing the above and a driving method thereof.

  In order to solve the above problems, the imaging system according to the first aspect of the present invention includes an imaging device having an imaging region in which a plurality of pixels are arranged in a matrix, and a control unit that controls readout of signals from the pixels. And the control means divides a first frame period for reading a first image from the image sensor into a plurality of divided frame periods including first and second divided frame periods, and A signal related to the second image between the first divided frame period and the second divided frame period when the number of pixels related to the image is larger than the number of pixels related to the second image A second frame period, which is a time required to read all of the first image, is provided, and the update cycle of the second image is shorter than the update cycle of the first image.

  The image sensor driving method according to the second aspect of the present invention is a driving method for reading out signals related to the first and second images from an image sensor having an imaging region in which a plurality of pixels are arranged in a matrix. When the number of pixels related to the first image is larger than the number of pixels related to the second image, a first frame period in which all signals related to the first image are read out is defined as the first and second divided frames. Dividing into a plurality of divided frame periods including a period, and providing a second frame period for reading all signals related to the second image between the first divided frame period and the second divided frame period, and The update cycle of the second image is shorter than the update cycle of the first image.

  According to the present invention, when reading image signals of a plurality of different image areas, an image whose update interval is desired to be shortened can be obtained at a shorter and more uniform update interval than before, resulting in an unnatural moving image. Can be suppressed.

  First, the outline of the present invention will be described with reference to FIG. Note that reading a signal from a pixel means outputting a signal to the outside of the imaging element based on the signal output from the pixel unless otherwise specified in the present specification.

  FIG. 1A is a diagram schematically illustrating an imaging region of an image sensor according to the present invention, that is, an effective pixel region of the image sensor. The entire area 11 represents an area including pixels in which an operation of outputting signals at a first density from the entire imaging area 10 which is an effective pixel area of the imaging element is performed. Further, the partial image 12 is an area including pixels in which an operation of outputting a signal at a second density from a part of the imaging area 10 is performed. Although the partial image 12 overlaps with the entire area 11, the pixel from which a signal is read is described here as being different between the entire area 11 and the partial image 12.

  In FIG. 1A, the time required to output a signal for one screen of the entire area 11 (one frame period) is the time required to output a signal for one screen of the partial image 12 (one frame period). Is longer than the update interval of the image of the partial image 12 (partial image). In the present invention, in order to shorten the update interval of the image on the side for which a high update speed is required, the image on the side for which the update speed is not further required is divided into a plurality of divided images, and the divided image and another divided image are divided. While performing an operation of outputting a signal from the corresponding pixel, an operation of outputting a signal from the pixel corresponding to the image on the side where the update speed is required is performed.

  For example, as shown in FIG. 1B, the entire area 11 is divided into divided images 11a and 11b. FIG. 1C schematically shows the time for outputting signals from the pixels corresponding to the divided images 11a and 11b and the partial images. The horizontal axis represents time. In this way, while outputting the divided image and another divided image, by acquiring signals from the pixels of the partial image 12 for one screen, the image of the region where the image is desired to be acquired at a higher update rate is updated constantly. It can be acquired at intervals and at a high update rate, that is, with a short update cycle.

(First embodiment)
In this embodiment, an example in which an image of the entire area is divided into two divided images will be described.

  FIG. 2 schematically shows the effective pixel region 20 of the image sensor corresponding to the imaging region 10, and the vertical shift register 22 and the horizontal shift register 23 that control the operation of reading signals from the pixels in the effective pixel region 20. . In the effective pixel region 20, pixels that generate an electrical signal according to incident light are arranged in a matrix. The vertical shift register 22 and the horizontal shift register 23 scan the effective pixel area 20 in the vertical direction and the horizontal direction, respectively, and change the scanning method based on control by an external system control circuit unit (control means) described later. Is configured to be possible.

  For the sake of simplicity, FIG. 2 shows an example in which the effective pixel region 20 includes unit pixels 21 of 32 rows × 48 columns. Although not shown in the figure, the image pickup device generally has an optical black pixel that is used for correction and is shielded so that light does not enter, around the effective pixel region.

  In FIG. 2, all the thinned-out read pixels 24 are shaded, and the pixels related to the first image from which the read operation is performed at the first density from the entire effective pixel region 20 are illustrated. Also, hatched lines indicate the partial readout pixels 25, which indicate pixels related to the second image in which readout operation is performed at a second density from a part of the effective pixel region 20. The total thinned readout pixel 24 corresponds to the entire region 11 in FIG. 1, and the partial readout pixel 25 corresponds to the partial image 12 in FIG.

  What is formed based on the signal read from the entire thinned-out readout pixel 24 is a thinned image (entirely thinned-out image) having a low density, that is, a low resolution obtained from the entire effective pixel region 20, and a partial readout pixel. A thinned image (partial image) obtained from a part of the effective pixel region 20 and having a higher density than the whole thinned image, that is, a high resolution, is formed based on the signal read from 25. Here, the time required to read out signals from all the thinned-out readout pixels 24 and the time required to read out signals from all of the partial readout pixels 25 are defined as one frame period. Here, the time required to read out the signal related to the whole thinned image for one screen is defined as the first frame period, and the time required to read out the signal related to the partial image is referred to as the second frame period. The pixels related to the second image are selected at a higher density than the pixels related to the first image. In FIG. 2, it is assumed that no signal is read from blank pixels in this embodiment.

  FIG. 3 is a timing chart showing an example of timing for performing an operation of reading a signal from the pixels related to the first and second images in the effective pixel region 20 shown in FIG. FIG. 3 shows the timing for performing the operation of reading out signals from the pixels in each row in the effective pixel region 20, and the operation of outputting the signals from the pixels in the corresponding row is performed during a period when the pulse is at a high level. Shall.

  Here, the time required to read signals from the portions included in rows 1 to 16 and the portions included in rows 17 to 32 in the entire thinned readout pixel 24 is defined as divided frame periods 1 and 2, respectively. Here, the divided frame period 1 is the first divided frame period, and the divided frame period 2 is the second divided frame period. After the divided frame period 1 is finished and before the divided frame period 2 is started, a frame period for reading a signal from the partial readout pixel 25 is inserted. That is, the control unit performs an operation of reading the signal by dividing the entire image thinned out for one screen into a plurality of divided images, and the divided frame period and the divided frame period in which the signal related to each divided image is read. In the meantime, an operation of reading a signal from a pixel constituting a partial image for one screen is performed.

  The timing shown in FIG. 3 will be described more specifically. When performing the above-described operation, the vertical shift register 22 scans the row 1 and then scans the rows 5, 9, and 13, and the rows with the thinned-out readout pixels 24 every third row. The horizontal shift register 23 scans the column 1, the column 5, the column 9,..., The column 45 of the row after the scanning of each row is finished and before the next row is scanned. Thereby, the signal from the pixel which concerns on the divided image 1 is output among the signals from the whole thinning-out readout pixel 24 (low-resolution thinning-out readout period (A)). As described above, this is the divided frame period 1.

  Next, the vertical shift register 22 is reset, and the process returns to the first row. Then, the lines 1, 5, and 9 are skipped, and the lines 2 to 4, the lines 6 to 8, and the line 10 are scanned. Again, the horizontal shift register 23 scans columns 2 to 4, columns 6 to 8, column 10, and column 11 of the row, and a signal from the partial readout pixel 25 for one screen is output (high resolution portion). Read period (1)). This is the second frame period for reading out the partial image.

  In the low resolution thinning readout period (B) (divided frame period 2) subsequent to the high resolution partial readout period (1), the entire thinned readout pixel 24 related to the divided image 2 is read in the same manner as the low resolution thinning readout period (A). Is output from the image sensor. Thereby, the whole thinned image is obtained for one screen together with the signal output from the image sensor in the low resolution thinning readout period (A).

  In the subsequent high resolution partial readout period (2), signals are read again from the pixels from which signals were read out in the high resolution partial readout period (1). That is, the partial image is updated for two screens during the period for updating the entire thinned image for one screen. Since the above operation is repeated thereafter, the entire thinned image and the partial image are updated at a ratio of 1: 2. That is, the update period of the second image is shorter than the update period of the first image.

  Note that signals read from the overall thinned readout pixels 24 related to the divided images 1 and 2 and output from the image sensor are synthesized by, for example, a signal processing circuit unit described later, and are reproduced / displayed later as one overall thinned image. Displayed in the section.

  The accumulation time of each pixel when the operation according to this embodiment is performed will be described. Each pixel in the effective pixel region 20 is configured to accumulate electric charge according to incident light, and a signal corresponding to the accumulated electric charge amount is output from the image sensor, and the magnitude of the signal depends on the length of the accumulation time. Changes. In FIG. 3, the accumulation time A indicates the accumulation time of the entire thinned readout pixel 24 in the row 1. When a signal is read from the pixel in the low resolution thinning readout period (A), a signal corresponding to the charge accumulated in the pixel so far is output, and from that time, the signal is read out in the low resolution thinning readout period (A ′). The time until is the accumulation time.

  On the other hand, when attention is paid to the pixel in the row 2 of the partial readout pixels 25, a signal corresponding to the charge accumulated in the pixels so far in the high resolution partial readout period (1) is output. The time until the signal is output in 2) is the accumulation time (accumulation time (2)). The period indicated by the accumulation time (1) is also the same length as the accumulation time (2).

  The accumulation time A and the accumulation times (1) and (2) indicate the maximum accumulation time that can be taken by each pixel. However, if it is desired to set an accumulation time shorter than this, an arbitrary value in the middle of the accumulation time can be obtained. The operation of resetting the pixels can be performed at the timing of. More specifically, a signal for resetting the pixel is input to the pixel from a timing control circuit unit described later.

  Next, an outline of an imaging device and an imaging system including the imaging device according to the imaging apparatus according to the present embodiment will be described with reference to FIG.

  The imaging system 100 includes, for example, an optical unit 110, an imaging device 120, a signal processing circuit unit 130, a recording / communication unit 140, a timing control circuit unit 150, a system control circuit unit 160, and a reproduction / display unit 170.

  The optical unit 110 forms an image of the subject by forming light from the subject on the pixel portion of the image sensor 120 in which a plurality of pixels are two-dimensionally arranged. The pixel portion includes the above-described effective pixel region. The image sensor 120 outputs a signal corresponding to the light imaged on the pixel unit at a timing based on the signal from the timing control circuit unit 150.

  The signal output from the image sensor 120 is input to the signal processing circuit unit 130, and the signal processing circuit unit 130 performs processing such as AD conversion on the input electric signal according to a method determined by a program or the like. . A signal obtained by processing in the signal processing circuit unit is sent to the recording / communication unit 140. The recording / communication unit 140 sends a signal for forming an image to the reproduction / display unit 170 and causes the reproduction / display unit 170 to reproduce and display a moving image or a still image. The recording communication unit also receives the signal from the signal processing circuit unit 130 and communicates with the system control circuit unit 160, and also performs an operation of recording a signal for forming an image on a recording medium (not shown). .

  FIG. 14 shows an example schematically showing the playback / display unit 170. The figure shows a monitor as the reproduction / display unit 170, and the entire thinned image ALL is shown on the left side. Of the whole thinned-out image ALL, the portions to which the symbols A and B are attached correspond to the partial images. An enlarged image of each partial image A and B is displayed on the right side of the monitor. Here, the entire image ALL and each partial image are displayed on one monitor, but each image may be displayed on a different monitor, or an arbitrary plurality of images may be displayed on one monitor.

  For example, when an imaging system is used as a monitoring system, it is required to display in detail a region requiring attention while roughly capturing a wide range as an entire image. In particular, in the monitoring system, it is required to update the image at a high frame rate in addition to extracting the target portion with high definition. Therefore, it is preferable to apply the present invention.

  The system control circuit unit 160 controls the overall operation of the imaging system, and controls the driving of the optical unit 110, the timing control circuit unit 150, the recording / communication unit 140, and the reproduction / display unit 170. In addition, the system control circuit unit 160 includes a storage device (not shown) that is a recording medium, for example, and a program necessary for controlling the operation of the imaging system is recorded therein.

  The timing control circuit unit 150 controls the drive timing of the image sensor 120 and the signal processing circuit unit 130 based on control by the system control circuit unit 160 which is a control unit.

  In the system control circuit unit 160, for example, a program such as how to determine the entire area and the partial image described above may be recorded. In this case, prior to the start of the operation, the entire area, the partial image area, and the density of pixels from which signals are read are determined, and which is divided into divided images to be output. The system control circuit unit 160 may be configured to accept a user operation through an interface unit (not shown). When the present imaging system is used as the monitoring system described above, the user can specify a region to be extracted as a high-definition partial image.

  In addition to the driving method exemplified in this specification, the control unit can also cope with progressive scanning that sequentially scans pixels in the effective pixel region from the first row and interlace scanning that scans every other row. It may be configured. Furthermore, without acquiring a signal related to the partial image in this embodiment, scanning is performed by thinning out pixels from only the entire imaging region, or only a region corresponding to the partial image is acquired without acquiring a signal related to the entire image. It may be configured to be compatible with a scanning method such as scanning. The imaging system described here can be applied to any of the embodiments described later.

  A configuration example of each pixel of the image sensor 120 will be described with reference to FIGS. FIG. 5 is a schematic diagram of a pixel. The pixel PIX1 includes a photodiode PD that photoelectrically converts light from a subject and accumulates charges, and a pixel amplifier MSFM that amplifies a signal corresponding to the charges accumulated in the photodiode PD. In addition, the transfer switch MTX for controlling the transfer of the charge accumulated in the photodiode to the gate of the pixel amplifier MSFM, the reset switch MRES for setting the gate of the pixel amplifier MSFM to a predetermined potential, and the pixel PIX1 And a selection switch MSEL for selecting a state. When the selection switch MSEL becomes conductive, the pixel amplifier MSFM configures a load switch MRV, which is a constant current source load provided on the vertical signal line Vh, and a source follower circuit. At this time, a potential corresponding to the gate potential of the pixel amplifier MSFM of the pixel PIX1 is output to the vertical signal line Vh as a signal of the pixel PIX1.

  FIG. 6 shows another configuration example of the pixel. FIG. 6 shows a pixel of a type in which a plurality of photodiodes PDn and a plurality of transfer switches TXn corresponding thereto share one reset switch MRES, one pixel amplifier MSFM, and one selection switch MSEL. In the figure, a configuration for three pixels is shown. According to such a configuration, the ratio of the reset switch MRES, the pixel amplifier MSFM, and the selection switch MSEL to one photodiode is reduced, and the aperture ratio for the photodiode PD can be improved.

  Although omitted in this specification, an image sensor generally has a configuration for sampling a signal output to a vertical signal line and a CDS (Correlated Double Sampling) circuit for removing noise. When a row from which a signal is to be extracted is selected by the vertical shift register, the signal output on the vertical signal line is sampled by the above circuit, and the signal sampled by the above circuit is scanned by the horizontal shift register sequentially. Output to the outside of the image sensor.

  According to the embodiment described above, when it is required to shorten the update interval of the partial image, the update interval of the partial image can be obtained with a uniform update interval that is shorter than the conventional one. It can suppress natural video.

  Although the whole screen is divided into two here, the number of divisions may be three or more, and the whole image may be divided based on the required partial image update speed.

(Second Embodiment)
Next, a second embodiment according to the present invention will be described with reference to FIGS. This embodiment is different from the first embodiment in that there are a plurality of partial images. Also in this embodiment, the case where the output time of the signal related to the partial image is regulated by the output time of the signal related to the entire area in the conventional driving method will be described as an example.

  FIG. 7A is a diagram schematically showing an effective pixel area which is an imaging area of the imaging element in the present embodiment. The entire area 71 represents an area including pixels in which an operation of outputting signals at the first density from the entire imaging area 70 that is an effective pixel portion of the imaging element is performed. The partial images 72 and 73 are regions including pixels in which an operation for outputting signals at a second density from a part of the imaging region 70 is performed. Although the partial images 72 and 73 overlap with the entire area 71, the pixel from which the signal is read is described here as being different between the entire area 71 and the partial images 72 and 73.

  Here, FIG. 7B is a diagram schematically illustrating an imaging surface when the entire region 71 is divided into three divided images 1 to 3 in a band shape. FIG. 7C schematically shows a sequence in the case where signals from the pixels corresponding to the regions of the divided images 1 to 3 and the partial images 1 and 2, that is, the divided images, are output from the image sensor. Here, a signal related to the divided image 1, a signal related to the partial image 1, a signal related to the divided image 2, a signal related to the partial image, and a signal related to the divided image 3 are output.

  FIG. 8 schematically shows an effective pixel area 80 of the image sensor corresponding to the imaging area 70, and a vertical shift register 82 and a horizontal shift register 83 which are control means for controlling the operation of reading signals from the pixels in the effective pixel area 80. It shows. The vertical shift register 82 and the horizontal shift register 83 scan the effective pixel region 80 in the vertical direction and the horizontal direction, respectively, and can be configured to change the scanning method based on the control by the external system control circuit unit described above. Has been.

  For the sake of simplicity, FIG. 8 shows an example in which the effective pixel region 80 includes unit pixels 81 of 32 rows × 48 columns. In addition, although not shown in the drawing, as in FIG. 2, pixels used for signal correction such as an optical black pixel may be arranged around the effective pixel region 80.

  In FIG. 8, all the thinned-out read pixels 84 are shaded, and the pixels related to the first image in which the read operation is performed from the entire effective pixel region 20 at the first density are shown. Also, hatched lines indicate the partial readout pixels 85 and 86, which indicate the pixels related to the second image in which the readout operation is performed at a second density from a part of the effective pixel region 80. The partial readout pixel 85 corresponds to the partial image 1 in FIG. 7A, and the partial readout pixel 86 corresponds to the partial image 2 in FIG. The whole thinned readout pixel 84 corresponds to the whole area 71 in FIG.

  What is formed based on the signal read from the whole thinned-out read pixel 84 is a thinned image (whole thinned image) having a low density obtained from the entire effective pixel region 80, that is, a low resolution, and is a partial read pixel. Based on the signals read from 85 and 86, a thinned image (partial image) having a higher density than the whole thinned image, that is, having a higher resolution, obtained from a part of the effective pixel region 80 is formed. Here, the time required to read out the signal related to the whole thinned image for one screen is defined as the first frame period, and the time required to read out the signal related to the partial image is referred to as the second frame period. Here, the whole thinned readout pixel 84 is selected at a lower density than the partial readout pixels 85 and 86. In FIG. 8, it is assumed that a blank pixel is not read out in this embodiment.

  FIG. 9 illustrates an operation sequence according to the present embodiment, and it is assumed that signals from pixels in the row are read during a period in which the pulse is at a high level.

  Here, of the whole thinned-out readout pixels 84 that are pixels related to the first image, a portion included in row 1 to row 11, a portion included in row 12 to row 22, and a portion included in row 23 to row 32 These are divided images 1, 2, and 3, respectively. After the divided frame period 1 in which the signal from the pixel of the entire thinned-out readout pixel 84 related to the divided image 1 is finished, before the divided frame period 2 in which the signal from the entire thinned-out readout pixel 84 related to the divided image 2 is read out, A second frame period for reading a signal from the read pixel 85 is inserted. In addition, after the divided frame period 2 in which the signal related to the divided image 2 is read and before the divided frame period 3 in which the signal from the entire thinned-out read pixel 84 related to the divided image 3 is read, the signal from the partial read pixel 86 is received. A second frame period for reading is inserted. Further, after the operation of outputting the signal from the pixel of the entire thinned-out readout pixel 84 related to the divided image 2 is completed, the partial readout is performed before the frame period 3 in which the signal from the entire thinned-out readout pixel 84 related to the divided image 3 is read out. An operation of outputting a signal from the pixel 85 is performed. Thereafter, an operation of alternately reading signals from pixels corresponding to one screen of a partial image is performed between a divided frame period in which signals from pixels related to two consecutive divided images are read. .

  The timing shown in FIG. 9 will be described more specifically. When performing the above-described operation, the vertical shift register 82 scans the row 1, then scans the rows 5 and 9 and every third row with the thinned-out read pixels 24, and then the horizontal shift register 83. Scan column 1, column 5, column 9... Thereby, the signal from the pixel which concerns on the divided image 1 is output among the signals from the whole thinning-out readout pixel 84 (low-resolution thinning-out readout period (A)).

  Next, the vertical shift register 82 is reset, and the process returns to the first row. Then, row 1, row 5, row 9 are skipped and rows 2 to 4, rows 6 to 8, and row 10 are scanned, and then the horizontal shift register 83 is moved to columns 2 to 4 and columns 6 to 8. By scanning the columns 10 and 11, a signal from the partial readout pixel 85 for one screen is output (high resolution partial readout period (1)).

  In the low resolution thinning readout period (B) following the high resolution partial readout period (1), the signal from the whole thinning readout pixel 84 related to the divided image 2 is transmitted from the image sensor in the same manner as in the low resolution thinning readout period (A). Is output.

  In the subsequent high-resolution partial readout period (2), the vertical shift register 82 scans the rows 18 to 20, the rows 22 to 24, and the rows 26, and then the horizontal shift register 83 is the columns 18 to 20, and the columns. By scanning the columns 22 to 24, 26 and 27, signals from the partial readout pixels 86 for one screen are output.

  In the low resolution thinning readout period (C), a signal from the whole thinning readout pixel 84 related to the divided image 3 is output from the image sensor. Thereby, the whole thinned image is obtained for one screen together with the signal output from the image sensor in the low resolution thinning readout period (A). Thereafter, a second frame period for reading out signals from pixels corresponding to one screen of a partial image is between the divided frame period for reading signals from pixels related to two consecutive divided images. Inserted alternately. In this example, the partial images 1 and 2 are updated three times each while the entire thinned image is updated for two screens.

  Here, for example, the signal output as the divided image is combined as an image representing the entire imaging surface by the signal processing circuit unit 130 and displayed on the reproduction / display unit 170. The partial image is also displayed on the reproduction / display unit 170.

  By repeating the above operation, the images related to the partial images 1 and 2 can be displayed uniformly and at a short update interval. Although the case where there are two partial images has been described as an example here, the number of partial images is not limited to two.

  In this embodiment, the partial images 1 and 2 are alternately read in this order, but the reverse order may be used.

  Next, the number of partial images existing in the imaging plane and the number of divided images will be described. Here, it is assumed that the partial image is an image of interest and is to be acquired at a short update interval. If n partial images exist in the imaging surface (n is a natural number), the entire thinned image corresponding to the entire area needs to be divided into n + 1 or more divided images. That is, when there are n second images, it is necessary to divide and read the first frame period required to read the first image into n + 1 or more divided frame periods. This is because if the number of divided images is n or less, the update interval of the partial image for which the update interval is to be shortened is equal to or longer than the update interval of the whole thinned image.

  As described above, FIG. 7C schematically shows a sequence according to this embodiment. On the other hand, FIG. 7D schematically shows a sequence to which the sequence disclosed in Patent Document 3 is applied. As is clear from a comparison between FIG. 7C and FIG. 7D, in the sequence according to Patent Document 3, it takes a long time to update the entire image once. Furthermore, since the time required to update the entire image becomes longer, the difference in timing for accumulating signals in the entire image for one screen becomes larger. On the other hand, according to the sequence according to the present invention, the update cycle of the entire thinned image can be shortened, and as a result, the difference in timing for accumulating signals in the entire image for one screen is reduced. It becomes possible.

  According to the present embodiment described above, when it is required to shorten the update interval of the partial image, the update interval of the partial image is shorter than before, and an unnatural moving image is obtained with a uniform update interval. Can be suppressed. Further, even when there are n partial images of interest, where n is a natural number, by reading out the entire thinned image divided into n + 1 or more divided frame periods, the update interval of the partial image of interest is shorter than before, And it can obtain at a uniform update interval.

(Third embodiment)
Next, a third embodiment according to the present invention will be described with reference to FIGS. In this embodiment, signals from pixels in the entire area are alternately output in odd and even rows of an image like interlace scanning.

  FIG. 10A is a diagram schematically showing an effective pixel area which is an imaging area of the imaging element in the present embodiment, and there is one partial image. In the second embodiment, the whole area is handled as a divided image divided vertically, but here, among the pixels related to the whole area selected by thinning out, only the divided image 101f that outputs only the odd-numbered lines and the even-numbered lines are used. It is divided into the divided 101g to be output and handled.

  The entire area 101 is an area including pixels in which an operation of outputting signals at a first density from the entire imaging area 100 that is an effective pixel portion of the imaging element is performed. The partial image 102 is an area including pixels in which an operation of outputting a signal at a second density from a part of the imaging area 100 is performed, and corresponds to the partial image. Although the partial image 102 overlaps with the entire area 101, the pixel from which a signal is read will be described here as being different between the entire area 101 and the partial image 102.

  FIG. 10B is a diagram schematically illustrating an imaging surface when the entire area 101 is divided into two divided images 101f and 101g. Here, the pixels related to the entire area 101 have different timings for reading signals from the pixels in the odd and even rows in the area, and the density of the pixels read in each period is higher than the density of the pixels related to the original entire area 101. Low. The fact that the density of pixels from which signals are read is reduced is schematically shown by making the density of hatched lines for the divided images 101f and 101g lower than the hatched lines for the entire area 101 before division.

  Although partial readout pixels 85 and 86 are illustrated in FIG. 8, in this embodiment, the description will be given assuming that the partial readout pixels 86 do not exist. The other points are the same as those described in the second embodiment.

  FIG. 11 illustrates an operation sequence according to the present embodiment, and it is assumed that an operation for outputting a signal from a pixel in the row is performed during a period in which the pulse is at a high level.

  Here, out of the whole thinned-out readout pixels 84, the odd-numbered rows, that is, the pixels in rows 1, 9, 17, and 25 are set as the divided image 1, and the even-numbered rows, that is, the pixels in rows 5, 13, 21, and 29 are set as the divided image 2. And After completing the frame period 1 in which the signal from the entire thinned-out readout pixel 84 related to the divided image 1 is read and before the frame period 2 in which the signal from the entire thinned-out read pixel 84 related to the divided image 2 is read out, the partial readout pixel 85 A frame period for outputting the above signal is inserted. That is, an operation of outputting a signal by dividing an entire image thinned out for one screen into a plurality of divided images, and for one screen between a frame period for reading a signal related to each divided image. A frame period in which a signal is read from the pixels constituting the partial image is inserted.

  The timing shown in FIG. 11 will be described more specifically.

  First, in the low resolution thinning readout period (A), which is the first divided frame period, out of the rows 1, 5, 9, 13, 17,... , 9, 17 and 25, signals from the pixels are output.

  Next, in the high-resolution partial readout period (1) that is the second frame period, out of the pixels related to the partial image 102, the pixels excluding the rows related to the entire region, that is, the rows 2, 3, 4, 6, 7 , 8 and 10 are output from the pixel rows.

  Next, in the low resolution decimation readout period (B), which is the second divided frame period, the even-numbered row among the rows 1, 5, 9, 13, 17,. The signals from the pixels according to rows 5, 13, 21, and 29 are output.

  Next, in the high-resolution partial readout period (2), among the pixels related to the partial image 102, the pixels excluding the rows related to the entire region, that is, the rows 2, 3, 4, 6, 7, 8, and 10 A signal from the pixel row is output.

  By repeating the above-described operation, it is possible to obtain an image related to a partial image at a higher update speed, which is conventionally regulated by the update speed of the entire area. In this embodiment, the partial image is updated for two screens while the entire thinned image is updated for one screen.

  According to the present embodiment described above, when it is required to shorten the update interval of the partial image, the update interval of the partial image can be obtained with a uniform update interval that is shorter than the conventional one. It can suppress natural video. In the first and second embodiments, since the entire thinned image is divided into strips, an entire image cannot be obtained with only one divided image, whereas in this embodiment, the entire thinned image is divided into odd rows. Since signals are read from corresponding pixels separately for the eyes and even-numbered rows, the entire image can be obtained even if only one of the divided images is output.

  In addition, although the case where there is one partial image is illustrated here, the number of partial images may be two or more. However, in that case, for the reason described in the second embodiment, the number of divisions of the entire area needs to be n + 1 or more with respect to the number n of partial images.

(Other 1)
In any of the above-described embodiments, assuming that the number of pixels related to the whole thinned image is larger than the number of pixels related to the partial image, the frame period for reading the signal related to the whole thinned image is divided into a plurality of divided frame periods. A frame period for outputting a signal related to the partial image for one screen is inserted into the frame. However, when the density of the pixels related to the whole thinned image is low, the signal from the pixels related to the whole thinned image is output rather than the time required to output the signal from the pixels related to the high density partial image. May take longer. In addition, there may be a desire to shorten the update interval of the entire thinned image. In such a case, the whole thinned image may be handled as the second image, and the partial image may be handled as the first image. That is, a signal from a pixel related to a partial image can be divided and read out in a plurality of divided frame periods, and a signal from a pixel related to a whole thinned-out image can be read out for one screen. As a result, when it is required to shorten the update interval of the whole thinned image, the update interval of the whole thinned image can be obtained with a uniform update interval that is shorter than the conventional one, resulting in an unnatural moving image. Can be suppressed.

  Further, it can be understood from the embodiment that the number of pixels per row in at least one of the entire thinned image or the partial image is smaller than the number of pixels per row in the imaging region.

(Other 2)
In the second embodiment, the frame period for outputting a signal related to the entire thinned image is divided into three divided frame periods. However, the divided images 1 and 2 and the divided image 3 include the number of rows of pixels included, That is, the number of pixels is different. For this reason, when all the divided image regions are driven in the same manner, the low resolution thinning readout period (C) is shorter than the low resolution thinning readout periods (A) and (B). Therefore, if the same driving is simply performed, it is considered that the update interval of the partial images becomes slightly non-uniform.

  As a solution to this, it is conceivable to change the drive frequency of the shift register, for example. By increasing the drive frequency of the shift register when scanning the pixels related to the divided image having the larger number of pixels, the lengths of the low resolution thinning readout periods can be made uniform, and the update intervals of the partial images can be made equal. The length of each low resolution thinning readout period may be made uniform by lowering the drive frequency of the shift register when scanning the pixels related to the divided image having the smaller number of pixels.

  As another solution, for example, a waiting time may be provided between a period during which a divided image with a small number of pixels is scanned and a continuous high-resolution partial readout period. Thereby, the length of the low-resolution thinning readout period for scanning the divided image with a large number of pixels is aligned with the length of the low-resolution thinning readout period for scanning the divided image with a small number of pixels, and the update intervals of the partial images are made equal.

  As described above, even when the image of the imaging region cannot be divided into divided images having the same number of pixels, the update intervals of the partial images can be made equal.

(Other 3)
In the above-described embodiment, pixels that read signals every several pixels are set in both the vertical direction (column direction) and the horizontal direction (row direction). On the other hand, for example, as shown in FIG. 12, the vertical direction may be selected every several pixels, and all the pixels in the horizontal direction may be selected.

  In this case, if the output signal is output as it is, the aspect ratio of the original imaging surface changes in the vertical direction and the horizontal direction. Therefore, the signals from the pixels in the horizontal direction are added and averaged so as to maintain the aspect ratio of the original imaging surface. For example, if pixels that read signals every 4 pixels in the vertical direction are selected, the signals in the horizontal direction are summed or averaged for 5 pixels to form one signal. According to this example, compared to the case where the horizontal direction is selected every several pixels, the effect of suppressing the moire in the horizontal direction of the obtained image and reducing the deterioration of the image quality can be obtained.

(Other 4)
First, an example of a configuration that can be taken by the pixels of the image sensor has been described with reference to FIGS. 5 and 6. Here, a configuration example in which one pixel can be selected is shown.

  The pixel shown in FIG. 13 is different from the pixel shown in FIG. 5 in that the second transfer means on the path connecting the transfer switch MTX, which is the first transfer means, and the gate portion (floating diffusion) of the pixel amplifier MSFM. The transfer switch MVX is further provided. According to this configuration, the transfer switch MTX is arbitrarily input by random access by inputting the signal φTX from the vertical shift register and the signal φVX from the horizontal shift register, for example, while the selection switch MSEL of the pixel is turned on. It is possible to output a signal from the pixel to the vertical signal line Vh.

  In the pixel having such a configuration, when resetting the photodiode PD, the transfer switches MTX and MVX and the reset switch MRES are made conductive so that the charge accumulated in the photodiode PD passes through these switches. Discharged. When the above-described CDS circuit (not shown) is included, a signal appearing on the vertical signal line Vh immediately after the photodiode PD is reset is held as a reset level. The CDS circuit further holds a signal appearing on the vertical signal line Vh by the charge transferred from the photodiode PD to the floating diffusion, and outputs a difference from the reset level. As a result, the influence due to the characteristic variation of the pixel amplifier is reduced and the noise is reduced. The signals φSEL and φTX may be a common signal.

  According to this embodiment, for example, when a signal is read from a pixel corresponding to the entire thinned image, the drive frequency of the shift register can be lowered, and power consumption can be reduced. In addition, since signals are read out only from the pixels used to form the entire thinned image, other pixels can be used as pixels constituting the partial image, and deterioration of the image quality of the partial image can be suppressed. .

(Other 5)
In the embodiment described above, since the pixels constituting the entire thinned image and the pixels constituting the partial image are different, there are discontinuous portions in the obtained image. For example, when it is necessary to acquire a partial image with high definition, the discontinuity of the image may be a problem. It is also conceivable that the aspect ratio of the obtained image differs from that on the imaging surface depending on how the pixels are selected. On the other hand, in the signal processing circuit unit provided in the subsequent stage of the image sensor, interpolation processing can be performed on the portion corresponding to the missing pixel based on the signal of the adjacent pixel.

(Other 6)
For example, in applications such as surveillance cameras, indoors and outdoors may enter the imaging plane at the same time. In the daytime, outdoor images often have higher luminance than indoor images, and there are areas with significantly different luminance in the imaging surface. In such a case, for example, the region of the image related to the outdoors is a partial image, and the gain for the signal is different between the period for reading the signal from the pixel for the partial image and the period for reading the signal from the pixel for the entire thinned image. It is possible. By reducing the gain for signals from pixels related to high-intensity partial images and setting a higher gain for signals from pixels related to whole thinned-out images, the range of brightness that can recognize the subject is expanded. it can.

  Specific implementation means will be described. In a MOS type imaging device, a configuration having an amplifier on a vertical signal line is known, and the gain can be switched here. It is also conceivable to perform processing outside the imaging device, for example, with a signal processing circuit unit. However, amplifying the signal at a position closer to the pixel is advantageous in that the influence of noise can be reduced.

(Other 7)
In each embodiment, for the sake of simplicity, the range of the imaging area corresponding to the entire thinned image and the partial image has been described as being fixed. However, a means for setting this may be provided.

  For example, when an area centered on a moving subject is used as a partial image in an application such as a surveillance camera, the area of the partial image is made to follow the movement of the subject according to a program stored in the system control circuit unit 160, for example. It is possible.

  Further, instead of automatically tracking, the operator may set the area. In this case, it is conceivable to set the range of the image and the density of the pixel from which the signal is read out by a range selection unit (not shown).

(Other 8)
The imaging system and the imaging element driving method described so far can be realized by operating a program stored in a storage device such as a RAM or a ROM of a computer.

  More specifically, the program is recorded on a recording medium such as a CD-ROM or provided to a computer via a transmission medium. In addition to CD-ROM, there are various recording media such as a flexible disk, a hard disk, a magnetic tape, a magneto-optical disk, and a nonvolatile memory. As the transmission medium, a computer network system such as a WAN such as a LAN or the Internet can be used, and even a wired line using an optical fiber or an electric wire is transmitted via a wireless line using radio waves or the like.

  In addition to the implementation of the supplied program by the computer, the functions of the imaging system may be implemented in cooperation with the computer operating system, other application software, and the like. In addition, the functions of the imaging system may be realized by performing all or part of the processing of the supplied program by a function expansion board or function expansion unit of a computer.

A diagram briefly explaining the concept of the present invention The figure which shows the effective pixel area | region which concerns on the 1st Embodiment of this invention. 1 is a timing chart showing a driving method according to a first embodiment of the present invention. 1 is a schematic diagram of an imaging system according to an embodiment of the present invention. A diagram showing a configuration example of a pixel A diagram showing a configuration example of a pixel The figure explaining briefly the concept of the 2nd Embodiment of this invention. The figure which shows the effective pixel area | region which concerns on the 2nd Embodiment of this invention. Timing chart showing a driving method according to a second embodiment of the present invention The figure explaining briefly the concept of the 3rd Embodiment of this invention. Timing chart showing the driving method of the third embodiment of the present invention The figure which shows the effective pixel area | region which concerns on another embodiment of this invention. A diagram showing a configuration example of a pixel The figure which represents typically the reproduction | regeneration / display part which concerns on the 1st Embodiment of this invention

Explanation of symbols

10, 70, 100 Imaging area 11, 71, 101 Whole area 11a Divided image 1
11b Split image 2
12, 102 Partial image 71a Divided image 1
71b Split image 2
71c Split image 3
72 Partial image 1
73 Partial image 2
101f Split image 1
101g split image 2
PD photodiode MTX transfer switch MRES reset switch MSFM pixel amplifier MSEL selection switch MRV constant current source MVX transfer switch Vh vertical signal line

Claims (21)

An imaging device having an imaging region in which a plurality of pixels are arranged in a matrix;
Control means for controlling readout of signals from the pixels,
The control means divides a first frame period for reading a first image from the image sensor into a plurality of divided frame periods including first and second divided frame periods,
When the number of the pixels related to the first image is larger than the number of the pixels related to the second image,
Providing a second frame period which is a time required to read out all signals related to the second image between the first divided frame period and the second divided frame period;
Furthermore, the update period of the second image is shorter than the update period of the first image. When n is a natural number and there are n second images,
The imaging system according to claim 1, wherein the first frame period is divided into n + 1 or more divided frame periods. If there are multiple second images that are part of the imaging area,
The imaging system according to claim 1, wherein one second frame period is performed between the first divided frame period and the second divided frame period. When the first divided frame period is shorter than the second divided frame period,
The control means provides a waiting time for the first divided frame period so as to be equal to the second divided frame period together with the first divided frame period. 4. The imaging system according to any one of items 3. When the number of pixels read out in the first divided frame period is smaller than the number of pixels read out in the second divided frame period,
The frequency at which the control means scans the pixels read out during the first divided frame period is lower than the frequency at which the control means scans the pixels read out during the second divided frame period. 5. The imaging system according to any one of 4.   The imaging system according to claim 1, wherein the divided frame periods have an equal length. The imaging system according to claim 1, further comprising range selection means for setting a range in the imaging area corresponding to the first and second images. The first image corresponds to the entire imaging region,
The imaging system according to claim 1, wherein the second image corresponds to a part of the imaging region.   The imaging system according to claim 8, wherein a density of pixels related to the first image is lower than a density of pixels related to the second image. The first image corresponds to a part of the imaging region,
The second image corresponds to the entire imaging area,
Furthermore, the density of the pixel which concerns on a said 1st image is higher than the density of the pixel which concerns on a said 2nd image, The imaging system of any one of Claim 1 thru | or 7 characterized by the above-mentioned. A driving method for reading out signals related to first and second images from an imaging device having an imaging region in which a plurality of pixels are arranged in a matrix,
When the number of pixels related to the first image is larger than the number of pixels related to the second image,
Dividing a first frame period in which all signals related to the first image are read out into a plurality of divided frame periods including first and second divided frame periods;
Providing a second frame period for reading all signals related to the second image between the first divided frame period and the second divided frame period;
Furthermore, the update period of the said 2nd image is made shorter than the update period of the said 1st image, The drive method of the image pick-up element characterized by the above-mentioned. When n is a natural number and there are n second images,
The image sensor driving method according to claim 11, wherein the first frame period is divided into n + 1 or more divided frame periods. If there are multiple second images that are part of the effective pixels,
The image sensor driving method according to claim 11, wherein one second frame period is provided between the first divided frame period and the second divided frame period. When the first divided frame period is shorter than the second divided frame period,
The standby time is provided for the first divided frame period so as to be equal to the second divided frame period together with the first divided frame period. The drive method of an image pick-up element given in the paragraph. When the number of pixels read out in the first divided frame period is smaller than the number of pixels read out in the second divided frame period,
15. The frequency of scanning pixels read out in the first divided frame period is lower than the frequency of scanning pixels read out in the second divided frame period. 15. Imaging system.   The image sensor driving method according to claim 11, wherein the divided frame periods have equal lengths. The first image corresponds to the entire imaging region,
The image sensor driving method according to claim 11, wherein the second image corresponds to a part of the imaging region.   The image sensor driving method according to claim 17, wherein a density of pixels related to the first image is lower than a density of pixels related to the second image. The first image corresponds to a part of the imaging region,
The image sensor driving method according to claim 11, wherein the second image corresponds to the entire imaging region.   The program which makes an image pick-up element perform the drive method of any one of Claim 11 thru | or 19.   A recording medium on which the program according to claim 20 is written.

Images