JP2007329905A - Image pickup apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain an image pickup apparatus that enables high frame rate moving images to be obtained, and further, in which the degradation of an image resolution can be prevented, and random noise contained in images can be suppressed. <P>SOLUTION: The image pickup apparatus comprises: a CCD image sensor 2 that includes a plurality of photoelectric conversion elements and is operable in a first drive mode for reading a low resolution image based on an output of the photoelectric conversion elements or a second drive mode for reading a higher resolution image than the low resolution image; a CCD drive unit 3 that drives the CCD image sensor 2; a frame memory 9 that stores an image read by the CCD image sensor 2 in the first drive mode; an image enlarging portion 8 that enlarges the image size of the image stored in the frame memory 9; and a noise reduction unit that performs noise reduction on the image read by the CCD image sensor 2 in the second drive mode based on the image enlarged by the image enlarging portion 8. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an imaging apparatus having a function of suppressing random noise generated in an imaging element or the like.

  A CCD image sensor, which is an example of an imaging device, includes a photoelectric conversion element arranged in a matrix, a transfer unit that transfers charges output from the photoelectric conversion element, and an image obtained by amplifying the voltage of the charge transferred by the transfer unit. And an amplifier for outputting a signal.

  Currently, image sensors that are widely used in digital still cameras and video cameras have been reduced in size and increased in pixel count, so that the amount of charge that can be stored in one photoelectric conversion element has decreased. When the amount of stored charge decreases, the sensitivity of the image decreases, but the sensitivity is increased by increasing the amplification in the amplifier. However, when the amplification degree in the amplifier is increased, noise components included in the charges output from the photoelectric conversion elements are also amplified. The amplified noise component appears as random noise on the image and deteriorates the S / N ratio of the image. Hereinafter, a technique for suppressing random noise will be described.

[1.2 Low-pass filter processing using a dimensional filter]
Various techniques have already been proposed as techniques for suppressing random noise. For example, Patent Document 1 discloses a method of suppressing random noise by performing low-pass filter processing using a two-dimensional filter.

  FIG. 16 shows a configuration of a random noise suppression circuit using a two-dimensional filter. When the image data is input from the input terminal 41, the synchronization unit 42 synchronizes the target pixel and 8 pixels (peripheral pixels) adjacent to the target pixel in the diagonal direction. The image data synchronized by the synchronization unit 42 is output to the subtraction units 431 to 438.

FIG. 17 is a schematic diagram illustrating the concept of synchronization in the synchronization unit 42, and schematically illustrates a two-dimensional arrangement of pixels in the image sensor. When the target pixel is a _{i, j} , the peripheral pixels a _{i−1, j−1} , a _{i−1, j} , a _{i−1, j + 1} , a _{i, j−1} , a _{i, j Since +1} , a _{i + 1, j−1} , a _{i + 1, j} , and a _{i + 1, j + 1} are not read out simultaneously with the target pixel, all the peripheral pixels are compared to the target pixel. After being read, these are synchronized and output to the subtracting units 431 to 438.

  In FIG. 16, the subtraction units 431 to 438 calculate the difference value between the target pixel and the surrounding pixels and output the difference value to the correlation detection units 441 to 448. The correlation detection units 441 to 448 detect the presence / absence of correlation between the target pixel and the surrounding pixels and output the detected correlation to the counting unit 45. For example, if the absolute value of the difference value output from the subtracting units 431 to 438 is smaller than the threshold value, it is determined that “there is a correlation”. In the following description, a pixel that is determined to be correlated with the target pixel among the surrounding pixels is referred to as a “correlated pixel”, and a pixel that is determined to have no correlation with the target pixel is referred to as a “non-correlated pixel”. ".

  The counting unit 45 specifies a correlation pixel and notifies the selection unit 46 of information on the specified correlation pixel. Further, the counting unit 45 outputs the number of correlated pixels to the numerical value generating unit 49.

  The selection unit 46 replaces the value of the non-correlated pixel among the outputs of the synchronization unit 42 with “0” and outputs the result to the addition unit 47.

  The adder 47 adds the outputs of the selector 46 and outputs the result to the divider 48.

  The numerical value generation unit 49 adds “1” to the number of correlation pixels output from the counting unit 45 and outputs the number after the addition to the division unit 48.

  The division unit 48 divides the output of the addition unit 47 by the number output from the numerical value generation unit 49 and outputs the result to the output terminal 50. Here, since the output of the numerical value generation unit 49 is the number of pixels added by the addition unit 47, the output of the division unit 48 is an addition average value of the target pixel and the correlation pixel.

  Through the above processing, an output in which random noise having an amplitude smaller than the threshold set by the correlation detection units 441 to 448 is suppressed is obtained. Further, if the edge of the subject is larger than the threshold set by the correlation detection units 441 to 448, the low-pass filter process using the two-dimensional filter is not performed, so that the edge of the image can be prevented from being blurred.

[2. Pixel mixed readout method)
Patent Document 2 discloses a pixel mixture reading method that reads after adding pixel values on an image sensor.

  FIG. 18 is a schematic diagram illustrating an example of a combination of pixels when each pixel is read by the pixel mixture reading method. In the pixel mixture reading method, the pixels 61 to 66 (target pixel) are read out after adding pixels (a total of 8 pixels) that are two pixels apart in the vertical and horizontal diagonal directions. For example, when the pixel 61 is the target pixel, eight pixels of the peripheral pixels 61 a to 61 h are added to the pixel 61 and then read out as a pixel signal of the pixel 61. The pixels 62 to 66 are similarly processed and read out as pixel signals.

  By such a pixel mixture readout method, an image signal in which random noise is suppressed can be obtained. Further, since pixels are thinned out and read out from the imaging device, there is an advantage that the number of pixels to be read is reduced and reading can be performed at high speed.

[3. (Circular noise reduction method)
Patent Document 3 discloses a cyclic noise reduction method using a frame memory.

  FIG. 19 is a block diagram illustrating an example of a cyclic noise reduction circuit using a frame memory. The subtractor 75 calculates a difference value between the image data input from the input terminal 71 and the image data of the previous frame read from the frame memory 79 and outputs the difference value to the attenuation unit 77. The attenuator 77 attenuates the image data output from the subtractor 75 by nonlinear processing.

  FIG. 20 is a diagram illustrating the input / output characteristics of the attenuation unit 77. When the absolute value of the difference value between the image data input from the input terminal 71 and the image data of the previous frame is smaller than the threshold Th, the attenuation unit 77 attenuates the output by ½. Further, when the absolute value of the difference value exceeds the threshold value Th, the output of the attenuation unit 77 is further attenuated. Further, when the absolute value of the difference value exceeds the threshold Th2, the output of the attenuation unit 77 becomes zero.

  The image data output from the attenuation unit 77 is input to the subtractor 76. The subtractor 76 subtracts the image data output from the attenuation unit 77 from the image data input to the input terminal 71. The image data output from the subtractor 76 is output from the output terminal 72 and written to the frame memory 79.

  That is, when the absolute value of the difference value between the image data input from the input terminal 71 and the image data of the previous frame is small, it can be determined that the difference value is random noise. Therefore, by subtracting the random noise component output from the attenuation unit 77 from the image data input to the input terminal 71 by the subtractor 76, the random noise included in the image data input from the input terminal 71 is reduced. Can be suppressed.

  Further, when the absolute value of the difference value between the image data input from the input terminal 71 and the image data one frame before is large, it can be determined that the subject is moving. Therefore, the image data input to the input terminal 71 is output from the output terminal 72 as it is, thereby preventing afterimages from occurring in a portion where movement has occurred.

By repeating the above processing for input image data of a plurality of frames, image data in which random noise is suppressed can be obtained.
Japanese Patent Laid-Open No. 11-41491 JP 2006-14075 A JP 2001-45334 A

  However, the low-pass filter processing using a two-dimensional filter as disclosed in Patent Document 1 and the pixel mixed readout method disclosed in Patent Document 2 have a problem that the resolution deteriorates.

  In addition, the random noise suppression effect by the low-pass filter processing using a two-dimensional filter is determined by the number of taps of the filter, and therefore the number of taps needs to be increased to obtain a large effect. When the number of taps of the filter is increased, there is a problem that the circuit scale increases.

  In addition, in the cyclic noise reduction method disclosed in Patent Document 3, when a still image is shot with an imaging device having a large number of pixels, the number of frames per unit time that can be read from the imaging device is reduced, and the moving image There is a problem that the quality of the product is degraded.

  An object of the present invention is to realize an imaging apparatus that can suppress random noise, obtain a moving image with a high frame rate, and obtain a high-resolution still image. Another object of the present invention is to realize such an imaging apparatus with a simple configuration.

  The imaging device of the present invention includes a plurality of photoelectric conversion elements, and reads out a low-resolution image based on an output of the photoelectric conversion element, or at least reads an image having a higher resolution than the low-resolution image. An image sensor that can operate in a second drive mode, a drive unit that drives the image sensor, a frame memory that stores image data read from the image sensor in the first drive mode, and the frame memory An image enlarging unit for enlarging the image data stored in the image data, and noise reduction of image data read out from the image sensor in the second drive mode based on the image data enlarged by the image enlarging unit And a noise reduction unit that performs processing.

  According to the imaging apparatus of the present invention, random noise can be suppressed, a moving image with a high frame rate can be obtained, and a high-resolution still image can be obtained. Moreover, such an imaging device can be realized with a simple configuration.

(Embodiment 1)
[1. Configuration of imaging device]
FIG. 1 is a block diagram of a configuration of the imaging apparatus according to the first embodiment. The imaging apparatus includes a lens 1, a CCD image sensor 2, a CCD drive unit 3, an analog-digital conversion unit 4 (hereinafter referred to as an A / D conversion unit), a release switch 10, a microcomputer 11 (hereinafter referred to as a microcomputer), An image memory 12 and a noise reduction circuit 14 are provided.

  The CCD image sensor 2 includes a large number of photoelectric conversion elements in a matrix, and converts an optical image incident through the lens 1 into an electrical image and outputs it. Further, the CCD image sensor 2 can operate in a pixel mixture drive mode (first drive mode) or an all-pixel readout mode (second drive mode). The pixel mixture drive mode is a mode in which the values of a plurality of pixels arranged in the horizontal direction and the vertical direction are added and output, and the pixel is substantially thinned out to reduce the resolution and output. The all-pixel reading mode is a mode in which pixel signals are output from all photoelectric conversion elements without thinning out pixel signals output from the photoelectric conversion elements.

  The CCD drive unit 3 can drive the CCD image sensor 2 by a control signal output from the microcomputer 11. Specifically, the CCD image sensor 2 is controlled to operate in the pixel mixture drive mode or the all-pixel readout mode, or the exposure operation is controlled.

  The microcomputer 11 (control unit) instructs the CCD drive unit 3 to switch between the pixel mixture drive mode and the all-pixel readout mode. In the pixel mixing drive mode, the pixel signals are read out after being mixed in the combination shown in FIG. 18 described above, so the number of pixels in the horizontal and vertical directions is reduced to 1/3 compared to the all-pixel reading mode. The read image signal is read out. Further, the microcomputer 11 can specify a magnification for the image enlargement unit 8. The microcomputer 11 controls the operations of the CCD drive unit 3 and the image enlargement unit 8 by a control signal output from the release switch 10.

  The noise reduction circuit 14 includes a first subtracter 5, a second subtracter 6, an attenuation unit 7, an image enlargement unit 8, and a frame memory 9. The noise reduction circuit 14 is for reducing noise components in the image data output from the A / D conversion unit 4.

  The image memory 12 stores the image data output from the noise reduction circuit 14. An image based on the image data stored in the image memory 12 can be displayed on a monitor (not shown) mounted on the imaging device.

  In the following description, the magnification at the time of enlargement or reduction of the image data is based on the size (1 ×) of the image output when the CCD image sensor 2 is operated in the pixel mixture drive mode.

  Next, the operation will be described.

  An image signal (analog signal) output from the CCD image sensor 2 is digitized by the A / D converter 4. The image data output from the A / D converter 4 is input to the first subtracter 5 and the second subtracter 6. The first subtracter 5 subtracts the image data (image one frame before) output from the image enlargement unit 8 from the image data (current image) output from the A / D conversion unit 4, and the difference Is output to the attenuation unit 7. The attenuation unit 7 performs non-linear processing based on the characteristics shown in FIG. 20, for example, and attenuates image data. The image data output from the attenuation unit 7 is input to the second subtracter 6. The second subtracter 6 subtracts the image data output from the attenuation unit 7 from the image data output from the A / D conversion unit 4. The image data output from the second subtracter 6 is written to the image memory 12 and to the frame memory 9.

  The image data written in the frame memory 9 is read after one frame has passed and is input to the image enlargement unit 8. Under the control of the microcomputer 11, the image enlarging unit 8 performs duplication processing of the same pixel or linear interpolation processing from a plurality of pixels, and enlarges the image size of the image data read from the frame memory 9. The image data enlarged by the image enlargement unit 8 is input to the first subtracter 5. Thereafter, the same operation as described above is performed.

  Note that when the magnification of the image enlargement unit 8 is set to 1, the image data read from the frame memory 9 is output as it is, so that it is the same as the cyclic noise reduction circuit shown in FIG. It becomes operation. The first subtracter 5, the second subtracter 6, and the attenuation unit 7 are examples of the noise reduction unit of the present invention.

[2. (Operation during shooting)
FIG. 2 shows the state of each part when the moving image is displayed on the monitor (monitor state) and when a still image is taken in the imaging apparatus of the present embodiment. FIG. 2A shows a read state of the pixel signal in the CCD image sensor 2, and the pixel signal is read during the High period. FIG. 2B shows the driving state of the CCD image sensor 2. FIG. 2C shows the magnification set in the image enlargement unit 8.

  First, as shown in a period 81 in FIG. 2, the microcomputer 11 controls the CCD drive unit 3 to operate the CCD image sensor 2 in the pixel mixture drive mode while the release switch 10 is not operated. Further, the microcomputer 11 sets the magnification of the image enlargement unit 8 to 1.

  The CCD image sensor 2 outputs an image signal for one frame at a predetermined period. An image signal output from the CCD image sensor 2 is input to the A / D converter 4. The A / D converter 4 digitizes the image signal and outputs image data. The image data output from the A / D converter 4 is written to the image memory 12 via the second subtracter 6. Images based on the image data written in the image memory 12 are sequentially output to a monitor (not shown). Thereby, a moving image (so-called “through image”) is displayed on the monitor. At this time, since the CCD image sensor 2 operates in the pixel mixture drive mode, it outputs an image signal (frame) in which pixels are thinned out in the horizontal and vertical directions. Therefore, since the size of the image per frame is small, the frame rate can be increased, and a smoothly moving moving image can be displayed on the monitor.

  Here, the image data output from the A / D converter 4 is subjected to random noise suppression processing in the second subtractor 6 and is written in the frame memory 9. After one frame has elapsed, the image data read from the frame memory 9 is input to the first subtracter 5 via the image enlargement unit 8. The image enlargement unit 8 outputs the image data read from the frame memory 9 without changing the image size because the magnification is set to 1. The first subtracter 5 subtracts the image data (image size is 1 time) output from the image enlargement unit 8 from the image data (image size is 1 time) output from the A / D conversion unit 4. The difference image data is output. The image data output from the first subtracter 5 is input to the attenuation unit 7. The attenuation unit 7 attenuates the image data based on the characteristics shown in FIG. The second subtracter 6 subtracts the image data output from the attenuation unit 7 from the image data output from the A / D conversion unit 4. As a result, the second subtracter 6 outputs image data in which random noise is suppressed.

  Next, as shown in a period 82, when the release switch 10 is operated, the microcomputer 11 controls the CCD driving unit 3 to cause the CCD image sensor 2 to perform an exposure operation. After the period 82 ends, the microcomputer 11 controls the CCD drive unit 3 to stop the exposure operation of the CCD image sensor 2.

  Next, as shown in a period 83, the microcomputer 11 controls the CCD driving unit 3 to operate the CCD image sensor 2 in the all-pixel reading mode. The CCD drive unit 3 operates the CCD image sensor 2 in the all pixel readout mode for one frame period. Further, the microcomputer 11 sets the magnification of the image enlargement unit 8 to 3 times.

  An image signal output from the CCD image sensor 2 operating in the all-pixel readout mode is digitized by the A / D conversion unit 4 and image data is output. The image data output from the A / D converter 4 is input to the first subtracter 5 and the second subtracter 6.

  On the other hand, image data for one frame before exposure (image size is 1 time) is written in the frame memory 9. After the exposure shown in the period 82, the image data read from the frame memory 9 is enlarged three times in the image enlargement unit 8. The image data (image size is 3 times) output from the image enlargement unit 8 is input to the first subtracter 5.

  The first subtracter 5 subtracts the image data enlarged by the image enlargement unit 8 from the image data output from the A / D conversion unit 4 and outputs the image data as the difference. The image data output from the first subtracter 5 is input to the attenuation unit 7. The size of the image that has been enlarged three times in the image enlargement unit 8 is the same size as the image size of the image data read from the CCD image sensor 2, so that the first subtracter 5 has an appropriate size. Difference image data can be obtained.

  The attenuator 7 attenuates image data (difference values) by nonlinear processing based on the characteristics shown in FIG. The image data output from the attenuation unit 7 is input to the second subtracter 6.

  The second subtracter 6 subtracts the image data output from the attenuation unit 7 from the image data output from the A / D conversion unit 4. The image data output from the second subtracter 6 is written into the image memory 12. Based on the image data written in the image memory 12, a display control unit (not shown) displays a still image on a monitor (not shown).

  Here, in the image data (difference value) of the difference between the output of the image enlargement unit 8 and the output of the A / D conversion unit 4, the difference value that appears in a portion where the temporal change of the image is small is considered as a random noise component. . If the amplitude of the random noise component is small, the level of the difference value falls within the range of -Th2 to Th2 shown in FIG. Therefore, in the second subtracter 6, the random noise component is subtracted from the image data output from the A / D conversion unit 4, and image data in which the random noise is suppressed can be obtained.

  On the other hand, if the amplitude of the difference value input to the attenuation unit 7 is outside the range of the threshold Th2 to -Th2 shown in FIG. 20, the level of the difference value output from the attenuation unit 7 is zero. This is because it is not a noise component but a portion where the temporal change of the image is large. Therefore, since the image data input from the A / D converter 4 is output as it is from the second subtracter 6, it is possible to prevent the occurrence of afterimages in the image.

  When the random noise suppression process is completed, the state returns to the period 81 again. That is, the moving image is displayed on the monitor.

[3. Effects of the embodiment, etc.]
According to the present embodiment, since a moving image is displayed based on the image data read in the pixel mixture driving mode, a moving image with a high frame rate can be obtained.

  In the pixel mixture drive mode, by performing noise reduction processing based on the image enlarged by the image enlargement unit 8, it is possible to suppress random noise without reducing the resolution.

  Further, by performing random noise reduction processing on an image read in the all-pixel reading mode, a high-quality still image can be obtained without reducing the resolution.

  Further, the CCD image sensor 2 is configured to mix and output pixel signals output from a plurality of photoelectric conversion elements in the pixel mixing drive mode, so that the operation in the all-pixel reading mode in which the pixel signals are not mixed is performed. Compared to the above, since an image in which random noise is further suppressed is obtained, a larger random noise suppression effect can be obtained when noise reduction processing is performed on an image read in the all-pixel readout mode.

  In the present embodiment, the magnification of the image enlarging unit 8 is 1 or 3 times, but is not limited to this magnification. The magnification may be determined according to the number of pixels to be thinned out during the pixel mixing operation in the CCD image sensor 2.

(Embodiment 2)
FIG. 3 is a block diagram of a configuration of the imaging apparatus according to the second embodiment. In FIG. 3, the same components as those shown in FIG. The configuration shown in FIG. 3 is obtained by adding a two-dimensional filter 13 to the configuration shown in FIG. The imaging apparatus according to the present embodiment operates based on the timing chart shown in FIG.

  The two-dimensional filter 13 (filter unit) is a filter having the configuration shown in FIG. 16 described above, and adaptively performs low-pass filter processing based on the correlation between the target pixel and surrounding pixels (eight pixels).

  Next, the operation will be described.

  In a period 81 in FIG. 2, image data in which the pixels are mixed by the CCD image sensor 2 and the pixels are thinned by 1/3 in the horizontal direction and the vertical direction is input to the two-dimensional filter 13. The two-dimensional filter 13 adaptively performs low-pass filter processing on the image data based on the correlation between the target pixel and surrounding pixels (8 pixels). For example, the low-pass filter process is performed based on the correlation between the target pixel 61 and the peripheral pixels 61a to 61h in FIG. Image data output from the two-dimensional filter 13 is input to the first subtracter 5 and the second subtractor 6. Subsequent operations are the same as those described in the first embodiment, and thus description thereof is omitted.

  According to the present embodiment, a larger random noise suppression effect can be obtained as compared with the case where the two-dimensional filter processing is not performed. In particular, when the CCD image sensor 2 is operated in the pixel mixture drive mode, the number of photoelectric conversion elements to be two-dimensionally filtered is increased without increasing the number of taps of the two-dimensional filter 13. A large random noise suppression effect can be obtained with a small circuit scale.

  That is, if the random noise suppression effect is to be enhanced only by the two-dimensional filter 13, the number of filter taps must be increased, and the circuit scale increases in accordance with the increased number of taps. The image pickup apparatus according to the present embodiment realizes a random noise suppression effect with reference to 81 pixels with a small circuit scale by pixel mixture of 9 pixels and two-dimensional filter processing of 9 pixels.

(Embodiment 3)
[1. Configuration of imaging device]
FIG. 4 is a block diagram of a configuration of the imaging apparatus according to the third embodiment. In FIG. 4, the same components as those shown in FIG. 1 are assigned the same reference numerals and detailed description thereof is omitted. The configuration shown in FIG. 4 is a configuration in which a thinning unit 15 is added to the configuration shown in FIG. 1 and a CCD image sensor 21 is provided in place of the CCD image sensor 2. In addition, the imaging apparatus according to the present embodiment operates based on the timing chart shown in FIG. However, in the present embodiment, the operation of the thinning unit 15 is shown in FIG.

  Unlike the CCD image sensor 2 of the first embodiment, the CCD image sensor 21 is configured by an image sensor that operates only in the all-pixel readout mode.

  The thinning unit 15 operates in one of the pixel mixture driving mode and the all-pixel reading mode. The pixel mixture drive mode is a mode in which pixels are thinned out and output by adding pixel values in the horizontal direction and the vertical direction to the image data output from the A / D conversion unit 4. The processing in this mode is the same processing as the pixel mixture driving mode in the CCD image sensor 2 described in the first embodiment. The all-pixel readout mode is a mode in which the image data output from the A / D converter 4 is output as it is. Switching of the mode in the thinning unit 15 is controlled by the microcomputer 11.

[2. (Operation during shooting)
First, as shown in a period 81 in FIG. 2, while the release switch 10 is not operated, the microcomputer 11 controls the thinning unit 15 to operate in the pixel mixture drive mode. An image signal output from the CCD image sensor 21 (an image signal read from all pixels) is digitized by the A / D conversion unit 4 to output image data.

  The thinning unit 15 performs pixel mixing processing on the image data output from the A / D conversion unit 4. Specifically, in the image data output from the A / D conversion unit 4, the pixel data in the horizontal direction and the vertical direction are added to generate substantially thinned image data. That is, low-resolution image data is output from the thinning unit 15. The image data output from the thinning unit 15 is input to the first subtracter 5 and the second subtracter 6. Subsequent operations are the same as those in the first embodiment, and are therefore omitted.

  Thus, by operating the thinning unit 15 in the pixel mixture drive mode in the period 81, it is possible to reduce the image size per frame and obtain a moving image with a high frame rate.

  Next, as shown in a period 82, when the release switch 10 is operated, the microcomputer 11 controls the CCD driving unit 3 to cause the CCD image sensor 21 to perform an exposure operation. After the period 82 ends, the microcomputer 11 controls the CCD drive unit 3 to stop the exposure operation of the CCD image sensor 21.

  Next, as shown in a period 83, when the exposure operation in the CCD image sensor 21 is completed, the microcomputer 11 controls the thinning unit 15 to operate in the all-pixel reading mode. Accordingly, the thinning unit 15 outputs the image data output from the CCD image sensor 21 and input via the A / D conversion unit 4 to the first subtracter 5 and the second subtractor 6 as they are. That is, high-resolution image data is output from the thinning unit 15. Subsequent operations are the same as the operations described in the first embodiment, and are therefore omitted.

[3. Effects of the embodiment, etc.]
According to the present embodiment, it is possible to obtain image data (moving image) having a high frame rate by suppressing the random noise by including the thinning unit 15 and performing the thinning process of the image data in a state before exposure. can do. Also, high-resolution image data (still images) can be obtained by not performing the thinning process after exposure.

  The CCD image sensor 21 that operates only in one operation mode (all pixel readout mode in this embodiment) is less expensive than a CCD image sensor that can selectively operate in a plurality of operation modes. The cost can be reduced as a whole.

  In the embodiment, as the operation in the pixel mixture driving mode, the combination of pixels shown in FIG. 18 (peripheral pixels that are two pixels apart in the up, down, left, and right directions around the target pixel) is used in the horizontal direction and the vertical direction. Although the thinning rate is 1/3 in the direction, the same effect can be obtained by changing the magnification of the image enlargement unit 8 according to the thinning rate even in other combinations and other thinning rates.

(Embodiment 4)
[1. Configuration of imaging device]
FIG. 5 is a block diagram illustrating a configuration of the imaging apparatus according to the fourth embodiment. In FIG. 5, the same components as those shown in FIG. The configuration shown in FIG. 5 is a configuration in which an image reduction unit 16 is further provided in the configuration shown in FIG.

  The image reduction unit 16 can perform processing to reduce the image size of the image data output from the A / D conversion unit 4. As a reduction processing method, for example, there is a method of substantially performing a thinning process by adding pixel values in the horizontal direction and vertical direction of an image. That is, the resolution of the image reduced by the image reduction unit 16 is low. The image reduction unit 16 is controlled by the microcomputer 11.

[2. (Operation during shooting)
FIG. 6 shows the state of each part when a moving image is displayed on the monitor (monitor state) and when a still image is taken in the imaging apparatus of the present embodiment. FIG. 6A shows the readout state of the pixel signal in the CCD image sensor 2, and the pixel signal is read out during the High period. FIG. 6B shows the driving state of the CCD image sensor 2. FIG. 6C shows the magnification in the image enlargement unit 8. FIG. 6D shows the magnification in the image reduction unit 16.

  As shown in FIG. 6, in the period 81, the microcomputer 11 controls the CCD drive unit 3 to operate the CCD image sensor 2 in the pixel mixture drive mode. Further, the microcomputer 11 sets the magnification of the image enlargement unit 8 to 1 and sets the magnification of the image reduction unit 16 to 1.

  The CCD image sensor 2 adds the pixel values in the horizontal direction and the vertical direction, and substantially performs a thinning process. That is, the CCD image sensor 2 outputs a low resolution image signal. From the CCD image sensor 2, an image signal for one frame is output at a predetermined cycle and input to the A / D converter 4.

  The A / D conversion unit 4 digitizes the image signal output from the CCD image sensor 2 and outputs the image data to the second subtractor 6 and the image reduction unit 16.

  The second subtracter 6 suppresses random noise in the image data output from the A / D conversion unit 4 and writes it in the image memory 12. Images based on the image data written in the image memory 12 are sequentially output to a monitor (not shown) by a display control unit (not shown), and a moving image is displayed on the monitor.

  On the other hand, the image reduction unit 16 outputs the input image data without changing the image size because the magnification is set to 1. Since the CCD image sensor 2 operates in the pixel mixture drive mode, a moving image having a small image size per frame and a high frame rate can be obtained.

  Here, the image data output from the A / D converter 4 is subjected to random noise suppression processing in the second subtractor 6 and is written in the frame memory 9. After one frame has passed, the image data read from the frame memory 9 is input to the first subtracter 5.

  The first subtracter 5 subtracts the image data (image size is 1 time) output from the frame memory 9 from the image data (image size is 1 time) output from the image reduction unit 16, and calculates the difference. Output image data.

  The attenuator 7 attenuates the image data output from the first subtractor 5 by nonlinear processing based on the characteristics shown in FIG. The image data output from the attenuation unit 7 is input to the image enlargement unit 8.

  Since the magnification is set to 1, the image enlargement unit 8 outputs the image data output from the attenuation unit 7 without changing the image size.

  The second subtracter 6 subtracts the image data output from the image enlargement unit 8 from the image data output from the A / D conversion unit 4. As a result, the second subtracter 6 outputs image data in which random noise is suppressed.

  Next, as shown in a period 82, when the release switch 10 is operated, the microcomputer 11 controls the CCD driving unit 3 to cause the CCD image sensor 2 to perform an exposure operation. After the period 82 ends, the microcomputer 11 controls the CCD drive unit 3 to stop the exposure operation of the CCD image sensor 2.

  Next, as shown in a period 83, the microcomputer 11 controls the CCD driving unit 3 to operate the CCD image sensor 2 in the all-pixel reading mode. Further, the microcomputer 11 sets the magnification of the image enlargement unit 8 to N times. Further, the microcomputer 11 sets the magnification of the image reduction unit 16 to 1 / N times. Note that N may be a value larger than “1”, and in this embodiment, N = 3.

  An image signal (image size is N times) output from the CCD image sensor 2 operating in the all-pixel readout mode is digitized by the A / D converter 4, and image data is output from the A / D converter 4. The The image data output from the A / D conversion unit 4 is input to the second subtracter 6 and the image reduction unit 16. The image reduction unit 16 reduces the size of the image data output from the A / D conversion unit 4 to 1 / N times. That is, the image data output from the image reduction unit 16 is an image having a size of 1 time.

  On the other hand, image data for one frame before exposure (image size is 1 time) is written in the frame memory 9. After the exposure shown in the period 82 is completed, the image data read from the frame memory 9 is input to the first subtracter 5.

  The first subtracter 5 subtracts the image data (image size is 1 time) output from the frame memory 9 from the image data (image size is 1 time) output from the image reduction unit 16, and the difference Output image data. The image data output from the first subtracter 5 is input to the attenuation unit 7.

  The attenuator 7 attenuates the image data output from the first subtracter 5 by nonlinear processing based on the characteristics shown in FIG. The image data output from the attenuation unit 7 is input to the image enlargement unit 8.

  The image enlargement unit 8 enlarges the image data (image size is 1 time) output from the attenuation unit 7 N times and outputs the result.

  The second subtracter 6 subtracts the image data (image size is N times) output from the image enlargement unit 8 from the image data (image size is N times) output from the A / D conversion unit 4. . The image data output from the second subtracter 6 is written into the image memory 12. Based on the image data written in the image memory 12, a display control unit (not shown) displays a still image on a monitor (not shown).

  When the random noise suppression process is completed, the state returns to the period 81 again. That is, the moving image is displayed on the monitor.

[3. Side effects that occur during random noise suppression processing)
When the enlarged image data is compared with the image data read from the CCD image sensor in the all-pixel read mode to obtain difference data, a false signal may be generated at the edge portion in the image. There is sex.

  FIG. 7A is a timing chart showing an operation when obtaining difference data by comparing the image data output from the CCD image sensor and the enlarged image data. FIG. 7B is a timing chart illustrating an operation when obtaining difference data in the imaging apparatus according to the present embodiment. 7A and 7B, t1 is an edge portion of the image, and TH is a threshold value in the attenuation unit 7. In the following explanation, random noise components are ignored.

  When the first subtracter 5 subtracts the enlarged image data (FIG. 7A (b)) from the input image data (FIG. 7A (a)), the image data shown in FIG. 7A (c) is obtained. can get. Next, the attenuation unit 7 attenuates the image data shown in FIG. 7A (c) with reference to the threshold value TH, and outputs the image data shown in FIG. 7A (d). Next, the second subtracter 6 subtracts the image data shown in FIG. 7A (d) from the image data shown in FIG. 7A (a), thereby outputting the image data shown in FIG. 7A (e). . As described above, when the difference data is obtained using the enlarged image, a false signal F is generated near the edge of the image as shown in FIG. 7A (e). This is because the first subtracter 5 subtracts two images having different resolutions.

  That is, the image data shown in FIG. 7A (a) has a high resolution because no enlargement processing or the like is performed. On the other hand, since the image data shown in FIG. 7A (b) is image data that has been enlarged by replication processing of the same pixel or linear interpolation processing from a plurality of pixels, the size of the image is increased. Is the state before enlargement processing (low resolution). Therefore, since the first subtracter 5 performs subtraction processing between the low resolution image and the high resolution image, the first subtracter 5 has a difference with a large amplitude as shown in FIG. 7A (c). Data is output. As a result, image data on which the false signal F is superimposed is output as shown in FIG. 7A (e).

  On the other hand, in the imaging apparatus according to the present embodiment, since the image enlargement process is not performed before the first subtracter 5, the first subtracter 5 performs the subtraction process on the image having the same resolution. become. That is, the first subtracter 5 in FIG. 5 includes the image data output from the image reduction unit 16 (FIG. 7B (a)) and the image data output from the frame memory 9 (FIG. 7B (b)). Is entered. The first subtracter 5 subtracts both pieces of image data and outputs the image data shown in FIG. 7B (c). Here, the image data output from the first subtracter 5 has a value close to zero. Next, the image data shown in FIG. 7B (c) is input to the second subtracter 6 via the attenuation unit 7 and the image enlargement unit 8. The second subtracter 6 subtracts the image data shown in FIG. 7B (d) from the image data shown in FIG. 7B (a), and outputs the image data shown in FIG. 7B (e). As shown in FIG. 7B (e), the false signal F as shown in FIG. 7A (e) is not superimposed on the image data output from the second subtracter 6.

  In the present embodiment, when the CCD image sensor 2 is operating in the all-pixel reading mode, the image reducing unit 16 reduces the image size of the image data. The image data input to the image reduction unit 16 is substantially thinned out by adding the pixel values in the horizontal and vertical directions, so that the resolution is lowered. The resolution of the image data at this time is equivalent to the resolution of the image data output from the frame memory 9. Therefore, as shown in FIG. 7B, image data on which no false signal is superimposed can be obtained. If the reduction processing method of the image reduction unit 16 is exactly the same as the pixel mixture processing method in the CCD image sensor 2 at the time of pixel mixture driving, the false signal due to the difference in resolution can be minimized.

  Note that the configuration described in the present embodiment is an example of a configuration that can eliminate a side effect that occurs when random noise is suppressed.

[4. Effects of the embodiment, etc.]
According to the present embodiment, when the CCD image sensor 2 is operating in the all-pixel readout mode, the image reduction is performed to reduce the image size by reducing the resolution of the image data output from the A / D conversion unit 4. By providing the unit 16, the resolution of the image data output from the image reduction unit 16 and the image data output from the frame memory 9 can be made equal. Therefore, it is possible to prevent the generation of a false signal in the image data output from the second subtracter 6 and improve the image quality.

  In addition, since the first subtracter 5 performs the subtraction process on the low resolution image data, the amount of calculation in the first subtracter 5 can be reduced, and the image data can be read out at high speed. Therefore, a moving image having a high frame rate can be obtained in the monitor state before exposure.

  The CCD image sensor 2 may use an image sensor that can be driven only in the all-pixel readout mode. FIG. 8 shows a configuration provided with a CCD image sensor 21 that can be driven only in the all-pixel readout mode, instead of the CCD image sensor 2 in FIG. Further, the thinning unit 15 that can operate in either the pixel mixed readout mode or the all-pixel readout mode is further provided. The imaging apparatus illustrated in FIG. 8 operates based on the timing chart illustrated in FIG. However, FIG. 6B shows the operation of the thinning unit 15. The pixel mixture drive mode is a mode in which low-resolution image data is output by adding horizontal and vertical pixel values to the image data output from the A / D converter 4. The all-pixel readout mode is a mode in which the image data output from the A / D converter 4 is output as it is. Switching of the mode in the thinning unit 15 is controlled by the microcomputer 11.

  First, as shown in a period 81 in FIG. 6, while the release switch 10 is not operated, the microcomputer 11 controls the thinning unit 15 to operate in the pixel mixture drive mode. An image signal output from the CCD image sensor 21 (an image signal read from all pixels) is digitized by the A / D conversion unit 4 to output image data.

  The thinning unit 15 performs pixel mixing processing on the image data output from the A / D conversion unit 4. Specifically, in the image data output from the A / D conversion unit 4, the pixel data in the horizontal direction and the vertical direction are added to generate substantially thinned image data. That is, low-resolution image data is output from the thinning unit 15. The image data output from the thinning unit 15 is input to the image reduction unit 16 and the second subtracter 6. Subsequent operations are the same as those of the imaging apparatus shown in FIG.

  Thus, by operating the thinning unit 15 in the pixel mixture drive mode in the period 81, it is possible to reduce the image size per frame and obtain a moving image with a high frame rate.

  Next, as shown in a period 82, when the release switch 10 is operated, the microcomputer 11 controls the CCD driving unit 3 to cause the CCD image sensor 21 to perform an exposure operation. After the period 82 ends, the microcomputer 11 controls the CCD drive unit 3 to stop the exposure operation of the CCD image sensor 21.

  Next, as shown in a period 83, when the exposure operation in the CCD image sensor 21 is completed, the microcomputer 11 controls the thinning unit 15 to operate in the all-pixel reading mode. Accordingly, the thinning unit 15 outputs the image data output from the CCD image sensor 21 and input via the A / D conversion unit 4 to the image reduction unit 16 and the second subtractor 6 as they are. That is, high-resolution image data is output from the thinning unit 15. Subsequent operations are the same as those of the imaging apparatus shown in FIG.

  According to the imaging apparatus shown in FIG. 8, the CCD image sensor 21 that operates only in one operation mode (all pixel readout mode in this embodiment) is a CCD image sensor that can selectively operate in a plurality of operation modes. Since it is cheaper than that, the cost of the entire apparatus can be reduced.

(Embodiment 5)
[1. Configuration of imaging device]
FIG. 9 is a block diagram illustrating a configuration of the imaging device according to the fifth embodiment. In FIG. 9, the same components as those shown in FIG. 1 are given the same reference numerals and detailed description thereof is omitted. The configuration shown in FIG. 9 is a configuration in which a two-dimensional filter 17 is further added to the configuration shown in FIG.

  The two-dimensional filter 17 (filter unit) removes high-frequency components from the image data output from the A / D conversion unit 4. Therefore, the image based on the image data output from the two-dimensional filter 17 has an edge portion.

[2. (Operation during shooting)
FIG. 10 shows a state of each part when a moving image is displayed on the monitor (monitor state) and when a still image is taken in the imaging apparatus according to the present embodiment. FIG. 10A shows the readout state of the pixel signal in the CCD image sensor 2, and the pixel signal is read out during the High period. FIG. 10B shows the driving state of the CCD image sensor 2. FIG. 10C shows the magnification in the image enlargement unit 8. FIG. 10D shows the operating state of the two-dimensional filter 17.

  As shown in FIG. 10, in the period 81, the microcomputer 11 controls the CCD drive unit 3 to operate the CCD image sensor 2 in the pixel mixture drive mode, and sets the magnification of the image enlargement unit 8 to 1. The operation of the two-dimensional filter 17 is turned off.

  The CCD image sensor 2 adds the values of the pixels in the horizontal direction and the vertical direction and performs a process of thinning out the pixels substantially. From the CCD image sensor 2, an image signal for one frame is output at a predetermined cycle and input to the A / D converter 4.

  The A / D converter 4 digitizes the image signal output from the CCD image sensor 2 and outputs the image data to the second subtractor 6 and the two-dimensional filter 17.

  The second subtracter 6 suppresses random noise in the image data output from the A / D converter 4 and writes the image data in the image memory 12. An image based on the image data written in the image memory 12 is output to a monitor (not shown) by a display control unit (not shown), and a moving image is displayed on the monitor.

  On the other hand, since the two-dimensional filter 17 is in the off state, the input image data is output as it is. Since the CCD image sensor 2 operates in the pixel mixture drive mode, a moving image having a small image size per frame and a high frame rate can be obtained.

  Here, the image data output from the A / D converter 4 is subjected to random noise suppression processing in the second subtractor 6 and is written in the frame memory 9. After one frame has elapsed, the image data read from the frame memory 9 is input to the image enlargement unit 8.

  The image enlargement unit 8 outputs the image data output from the frame memory 9 to the first subtractor 5 as it is because the magnification is set to 1.

  The first subtracter 5 is based on the image data (image size is 1 time) output from the two-dimensional filter 17. The image data output from the image enlargement unit 8 (image size is 1 time) is subtracted, and difference image data is output. The image data output from the first subtracter 5 is input to the attenuation unit 7.

  The attenuator 7 attenuates the image data output from the first subtractor 5 by nonlinear processing based on the characteristics shown in FIG. The image data output from the attenuation unit 7 is input to the second subtracter 6.

  The second subtracter 6 subtracts the image data output from the attenuation unit 7 from the image data output from the A / D conversion unit 4. Thereby, the second subtracter 6 can output image data in which random noise is suppressed.

  Next, as shown in a period 82, when the release switch 10 is operated, the microcomputer 11 controls the CCD driving unit 3 to cause the CCD image sensor 2 to perform an exposure operation. After the period 82 ends, the microcomputer 11 controls the CCD drive unit 3 to stop the exposure operation of the CCD image sensor 2.

  Next, as shown in a period 83, the microcomputer 11 controls the CCD driving unit 3 to operate the CCD image sensor 2 in the all-pixel reading mode. Further, the microcomputer 11 sets the magnification of the image enlargement unit 8 to N times (N = 3 in the present embodiment). Further, the microcomputer 11 turns on the operation of the two-dimensional filter 17.

  An image signal (image size is N times) output from the CCD image sensor 2 operating in the all-pixel readout mode is digitized by the A / D converter 4 and image data is output. Image data output from the A / D converter 4 is input to the second subtractor 6 and the two-dimensional filter 17.

  The two-dimensional filter 17 removes high frequency components of the image data output from the A / D conversion unit 4. As a result, the image data output from the two-dimensional filter 17 is an image in which the edge portion is blurred.

  On the other hand, image data for one frame before exposure (image size is 1 time) is written in the frame memory 9. After the exposure shown in the period 82 is completed, the image data read from the frame memory 9 is input to the image enlargement unit 8.

  The image enlarging unit 8 performs an enlarging process on the image data output from the frame memory 9 by a duplication process of the same pixel or a linear interpolation process from a plurality of pixels. Since the image data output from the image enlargement unit 8 has been enlarged by signal processing, the level of the high-frequency component is low and the image has an edge portion. The image data output from the image enlargement unit 8 is input to the first subtracter 5.

  The first subtracter 5 subtracts the image data (image size is N times) output from the image enlargement unit 8 from the image data (image size is N times) output from the two-dimensional filter 17, Output the difference image data. That is, the first subtracter 5 subtracts image data from which high frequency components have been removed. The image data output from the first subtracter 5 is input to the attenuation unit 7.

  The attenuator 7 attenuates the image data output from the first subtracter 5 by nonlinear processing based on the characteristics shown in FIG. The image data output from the attenuation unit 7 is input to the second subtracter 6.

  The second subtracter 6 subtracts the image data output from the attenuation unit 7 from the image data output from the A / D conversion unit 4. The image data output from the second subtracter 6 is written into the image memory 12. Based on the image data written in the image memory 12, a display control unit (not shown) displays a still image on a monitor (not shown).

  When the random noise suppression process is completed, the state returns to the period 81 again. That is, the moving image is displayed on the monitor.

[3. Effects of the embodiment, etc.]
According to the present embodiment, since the two-dimensional filter 17 that removes the high-frequency component of the image data input to the first subtracter 5 is provided, the first subtracter 5 is a two-dimensional filter that has a high-frequency component. The difference data is generated by subtracting the image data in which the high frequency component output from the image enlargement unit 8 is reduced from the removed image data. Therefore, generation of a false signal as described in Embodiment 4 can be prevented, and the image quality can be improved.

  That is, when the subtraction process is performed between the image data from which the high frequency component has been removed and the image data from which the high frequency component has not been removed, the difference value becomes large as shown in FIG. 7A (c). Therefore, a false signal F as shown in FIG. 7A (e) is generated. On the other hand, in the present embodiment, since the image data from which the high-frequency components are removed as shown in FIG. 7A (b) is subtracted, the difference value is low. Therefore, as shown in FIG. 7B (e), it is possible to obtain image data in which no false signal is generated.

  The CCD image sensor 2 may use an image sensor that can be driven only in the all-pixel readout mode. FIG. 11 shows a configuration provided with a CCD image sensor 21 that can be driven only in the all-pixel readout mode, instead of the CCD image sensor 2 in FIG. Further, the thinning unit 15 that can operate in either the pixel mixed readout mode or the all-pixel readout mode is further provided. The imaging device illustrated in FIG. 11 operates based on the timing chart illustrated in FIG. However, FIG. 10B shows the operation of the thinning unit 15. The pixel mixture drive mode is a mode in which low-resolution image data is output by adding horizontal and vertical pixel values to the image data output from the A / D converter 4. The all-pixel readout mode is a mode in which the image data output from the A / D converter 4 is output as it is. Switching of the mode in the thinning unit 15 is controlled by the microcomputer 11.

  First, as shown in a period 81 in FIG. 10, while the release switch 10 is not operated, the microcomputer 11 controls the thinning unit 15 to operate in the pixel mixture drive mode. An image signal output from the CCD image sensor 21 (an image signal read from all pixels) is digitized by the A / D conversion unit 4 to output image data.

  The thinning unit 15 performs pixel mixing processing on the image data output from the A / D conversion unit 4. Specifically, in the image data output from the A / D conversion unit 4, the pixel data in the horizontal direction and the vertical direction are added to generate substantially thinned image data. That is, low-resolution image data is output from the thinning unit 15. The image data output from the thinning unit 15 is input to the two-dimensional filter 17 and the second subtracter 6. Subsequent operations are the same as those of the imaging apparatus shown in FIG.

  Thus, by operating the thinning unit 15 in the pixel mixture drive mode in the period 81, it is possible to reduce the image size per frame and obtain a moving image with a high frame rate.

  Next, as shown in a period 82, when the release switch 10 is operated, the microcomputer 11 controls the CCD driving unit 3 to cause the CCD image sensor 21 to perform an exposure operation. After the period 82 ends, the microcomputer 11 controls the CCD drive unit 3 to stop the exposure operation of the CCD image sensor 21.

  Next, as shown in a period 83, when the exposure operation in the CCD image sensor 21 is completed, the microcomputer 11 controls the thinning unit 15 to operate in the all-pixel reading mode. Accordingly, the thinning unit 15 outputs the image data output from the CCD image sensor 21 and input via the A / D conversion unit 4 to the two-dimensional filter 17 and the second subtractor 6 as they are. That is, high-resolution image data is output from the thinning unit 15. Subsequent operations are the same as those of the imaging apparatus shown in FIG.

  According to the imaging apparatus shown in FIG. 11, the CCD image sensor 21 that operates only in one operation mode (all pixel readout mode in this embodiment) is a CCD image sensor that can selectively operate in a plurality of operation modes. Since it is cheaper than that, the cost of the entire apparatus can be reduced.

(Embodiment 6)
[1. Configuration of imaging device]
FIG. 12 is a block diagram illustrating a configuration of the imaging device according to the sixth embodiment. In FIG. 12, the same components as those shown in FIG. 5 are given the same numbers, and detailed description thereof is omitted. The configuration illustrated in FIG. 12 is a configuration in which a two-dimensional filter 18, a motion detection unit 19, and a switching unit 20 are further provided in the configuration illustrated in FIG.

  The two-dimensional filter 18 (filter unit) is a filter having the configuration shown in FIG. 16 described above, and adaptively performs low-pass filter processing based on the correlation between the target pixel and surrounding pixels (8 pixels).

  The motion detection unit 19 detects image motion based on the image data output from the image enlargement unit 8. Specifically, the coefficient K is output according to the absolute value of the image data (difference value) output from the image enlargement unit 8. FIG. 13 shows the relationship between the difference value and the coefficient K. As shown in FIG. 13, if the absolute value of the difference value is S1 or less, it is determined that there is no image motion, and a coefficient K = 0 is output. If the absolute value of the difference value is S2 or more, the coefficient K = 1 is output. If the absolute value of the difference value is in the range of S1 to S2, the coefficient K changes in the range of 0 to 1 according to the difference value.

  In the switching unit 20, the two-dimensional filter 18 is connected to the A terminal, and the second subtracter 6 is connected to the B terminal. Image data that has been subjected to filter processing (two-dimensional noise reduction processing) by the two-dimensional filter 18 is input to the A terminal. Image data that has been subjected to random noise suppression processing (three-dimensional noise reduction processing) by the noise reduction circuit 14 is input to the B terminal. In addition, the switching unit 20 adds the image data output from the two-dimensional filter 18 and the image data output from the second subtractor 6 at a predetermined ratio by the coefficient K output from the motion detection unit 19. Output. In the present embodiment, when the coefficient K output from the motion detector 19 is “1”, the image data input to the A terminal is output to the image memory 12. When the coefficient K is “0”, the image data input to the B terminal is output to the image memory 12. If 0 <K <1, the image data input to the A terminal and the image data input to the B terminal are added at a predetermined ratio and output to the image memory 12. FIG. 14 shows a specific configuration of the switching unit 20, and detailed operation will be described later.

[2. (Operation during shooting)
As shown in FIG. 12, the microcomputer 11 controls the CCD drive unit 3 to operate the CCD image sensor 2 in the pixel mixture drive mode before the exposure. Further, the microcomputer 11 sets the magnification of the image enlargement unit 8 to 1 and sets the magnification of the image reduction unit 16 to 1.

  The CCD image sensor 2 adds the values of the pixels in the horizontal direction and the vertical direction and performs a process of thinning out the pixels substantially. From the CCD image sensor 2, an image signal for one frame is output at a predetermined cycle and input to the A / D converter 4. The A / D converter 4 digitizes the image signal output from the CCD image sensor 2 and outputs image data. Image data output from the A / D conversion unit 4 is input to the second subtractor 6, the image reduction unit 16, and the two-dimensional filter 18.

  The second subtracter 6 suppresses random noise in the image data output from the A / D converter 4 and writes the image data in the frame memory 9. After one frame has passed, the image data read from the frame memory 9 is input to the first subtracter 5.

  The image reduction unit 16 outputs the image data without changing the image size of the image data output from the A / D conversion unit 4 because the magnification is set to 1. The image data output from the image reduction unit 16 is input to the first subtracter 5.

  The first subtracter 5 subtracts the image data (image size is 1 time) output from the frame memory 9 from the image data (image size is 1 time) output from the image reduction unit 16, and calculates the difference. Output image data. The image data output from the first subtracter 5 is input to the image enlargement unit 8.

  Since the magnification is set to 1, the image enlargement unit 8 outputs the image data output from the first subtracter 5 without changing the image size. Image data output from the image enlargement unit 8 is input to the attenuation unit 7 and the motion detection unit 19.

  The attenuating unit 7 attenuates the image data output from the image enlarging unit 8 by nonlinear processing based on the characteristics shown in FIG. The image data output from the attenuation unit 7 is input to the second subtracter 6.

  The second subtracter 6 subtracts the image data output from the attenuation unit 7 from the image data output from the A / D conversion unit 4. Thereby, the second subtracter 6 can output image data in which random noise is suppressed.

  The motion detection unit 19 outputs a coefficient K based on the image data output from the image enlargement unit 8. Specifically, a coefficient K corresponding to the value (difference value) of the image data is determined and output based on the characteristics shown in FIG. When the imaging apparatus is in a state before exposure (monitor state), there is a lot of image movement, and therefore the coefficient K continues to output “1”. The output coefficient K is input to the switching unit 20.

  Based on the coefficient K output from the motion detection unit 19, the switching unit 20 adds the image data input to the A terminal and the image data input to the B terminal at a predetermined ratio. Since the coefficient K = 1 is continuously output when the imaging apparatus is in a state before exposure, the switching unit 20 outputs image data input to the A terminal. As a result, the image data output from the A / D converter 4 is filtered by the two-dimensional filter 18 and written to the image memory 12 via the switching unit 20. Based on the image data written in the image memory 12, a display control unit (not shown) displays a moving image on a monitor (not shown).

  Next, when the release switch 10 is operated, the microcomputer 11 controls the CCD drive unit 3 to cause the CCD image sensor 2 to perform an exposure operation. After a predetermined time has elapsed, the microcomputer 11 controls the CCD drive unit 3 to stop the exposure operation of the CCD image sensor 2.

  Next, the microcomputer 11 controls the CCD drive unit 3 to operate the CCD image sensor 2 in the all-pixel reading mode. Further, the microcomputer 11 sets the magnification of the image enlargement unit 8 to N times. Further, the microcomputer 11 sets the magnification of the image reduction unit 16 to 1 / N times. In this embodiment, N = 3.

  An image signal (image size is N times) output from the CCD image sensor 2 operating in the all-pixel readout mode is digitized by the A / D converter 4 and image data is output. Image data output from the A / D conversion unit 4 is input to the second subtractor 6, the image reduction unit 16, and the two-dimensional filter 18. The image reduction unit 16 reduces the image size of the image data output from the A / D conversion unit 4 to 1 / N times. That is, the image data output from the image reduction unit 16 is an image having a size of 1 time. The image data output from the image reduction unit 16 is input to the first subtracter 5.

  On the other hand, image data for one frame before exposure (image size is 1 time) is written in the frame memory 9. After the exposure is completed, the image data read from the frame memory 9 is input to the first subtracter 5.

  The first subtracter 5 subtracts the image data (image size is 1 time) output from the frame memory 9 from the image data (image size is 1 time) output from the image reduction unit 16, and the difference Output image data. The image data output from the first subtracter 5 is input to the image enlargement unit 8.

  The image enlargement unit 8 enlarges the image data output from the first subtracter 5 N times and outputs the result. Image data output from the image enlargement unit 8 is input to the attenuation unit 7 and the motion detection unit 19.

  The attenuating unit 7 attenuates the image data output from the image enlarging unit 8 by nonlinear processing based on the characteristics shown in FIG. The image data output from the attenuation unit 7 is input to the second subtraction unit 6.

  The second subtracter 6 subtracts the image data output from the attenuation unit 7 from the image data output from the A / D conversion unit 4. The image data output from the second subtracter 6 is input to the switching unit 20.

  The motion detection unit 19 determines and outputs a coefficient K based on the image data (difference value) output from the image enlargement unit 8. The coefficient K is determined based on the characteristics shown in FIG. That is, if there is a lot of image movement, the difference value of the image data increases, and the coefficient K increases (the maximum value is “1”). On the other hand, if the image motion is small, the difference value of the image data is small, and the coefficient K is small (the minimum value is “0”). The coefficient K output from the motion detector 19 is input to the switching unit 20.

  The switching unit 20 adds the image data input to the A terminal and the image data input to the B terminal at a predetermined ratio based on the coefficient K output from the motion detection unit 19, and the added image data Is written in the image memory 12. Based on the image data written in the image memory 12, a display control unit (not shown) displays a still image on a monitor (not shown).

  When the random noise suppression process is completed, the state before the exposure is restored. That is, the moving image is displayed on the monitor.

  FIG. 14 shows a specific configuration of the switching unit 20. The switching unit 20 includes an A terminal 201, a B terminal 202, a K terminal 203, a first multiplier 204, a second multiplier 205, a reference input terminal 206, a third subtractor 207, an adder 208, and an output terminal. 209. In FIG. 14, the A terminal 201 is connected to the two-dimensional filter 18. Image data input to the A terminal 201 is input to the first multiplier 204. The B terminal 202 is connected to the second subtracter 6. The image data input to the B terminal 202 is input to the second multiplier 205. The coefficient K input to the K terminal 203 is input to the first multiplier 204 and the third subtracter 207. The third subtractor 207 performs a subtraction process (1-K) with the coefficient K and the reference value (“1” in the present embodiment) input to the reference input terminal 206. The value output from the third subtractor 207 is input to the second multiplier 205. The image data output from the first multiplier 204 and the image data output from the second multiplier 205 are input to the adder 208. The image data output from the adder 208 is output to the outside via the output terminal 209.

  For example, when the image data output from the two-dimensional filter is input to the A terminal 201 and the image data output from the noise reduction circuit 14 is input to the B terminal 202, the coefficient K = 0 is set to the K terminal 203. When input, the output of the first multiplier 204 becomes “0”. Further, the value “1” is output from the third subtractor 207 and input to the second multiplier 205. Therefore, only the image data output from the second multiplier 205 is input to the adder 208, and the image data input to the B terminal 202 is output from the output terminal 209.

  When the coefficient K = 1 is input to the K terminal 203, the image data output from the first multiplier 204 is input to the adder 208. The value “0” is output from the third subtractor 207 and input to the second multiplier 205. As a result, the output of the second multiplier 205 becomes “0”. Therefore, only the image data output from the first multiplier 204 is input to the adder 208, and the image data input to the A terminal 201 is output from the output terminal 209.

  If the coefficient K input to the K terminal 203 is a predetermined value between 0 and 1, image data output from the first multiplier 204 and output from the second multiplier 205 according to the value. The ratio with the image data to be changed changes. For example, when the coefficient K = 0.8 is input, the image data output from the first multiplier 204 is 80% and the image data output from the second multiplier 205 is 20%. Input to the adder 208.

[3. Effects of the embodiment, etc.]
According to the present embodiment, the motion detection unit 19 detects the motion of the image, and when the motion is large, the two-dimensional filter 18 performs the filter processing. When the motion is small, the noise reduction circuit 14 performs the random noise suppression processing. By adopting a configuration for performing the above, it is possible to prevent the deterioration of the S / N ratio when there is a lot of image movement. In other words, if the processing by the noise reduction circuit 14 is performed when there is a lot of image movement, the correlation between frames is low, so that correction is more than necessary, or no correction is applied at a portion where the movement is large, and the image quality deteriorates. There is a possibility. In the present embodiment, when the movement of the image is large, the filter processing is preferentially operated by the two-dimensional filter, so that deterioration of the image quality can be suppressed.

  In the present embodiment, image motion detection by the motion detection unit 19 and switching operation in the switching unit 20 are performed in both the pre-exposure state and the post-exposure state. What is necessary is just to set it as the structure which operates the detection part 19 and the switch part 20. FIG.

  Further, in the present embodiment, as shown in FIG. 13, the coefficient K output from the motion detection unit 19 is linearly changed between 0 and 1, but the binary values “0” and “1” are used. Also good. Thereby, the switching unit 20 can select and output only one of the image data output from the two-dimensional filter 18 and the image data output from the noise reduction circuit 14.

  In the present embodiment, the motion detection uses the difference data output from the first subtracter 5, but the present invention is not limited to this.

  Moreover, although Embodiment 6 is combined with the structure described in Embodiment 5, it can also be combined with any of the structures described in Embodiments 1-4.

  The CCD image sensor 2 may use an image sensor that can be driven only in the all-pixel readout mode. FIG. 15 shows a configuration provided with a CCD image sensor 21 that can be driven only in the all-pixel readout mode, instead of the CCD image sensor 2 in FIG. Further, the thinning unit 15 that can operate in either the pixel mixed readout mode or the all-pixel readout mode is further provided. The imaging apparatus shown in FIG. 15 operates based on the timing chart shown in FIG. However, FIG. 2B shows the operation of the thinning unit 15. The pixel mixture drive mode is a mode in which low-resolution image data is output by adding horizontal and vertical pixel values to the image data output from the A / D converter 4. The all-pixel readout mode is a mode in which the image data output from the A / D converter 4 is output as it is. Switching of the mode in the thinning unit 15 is controlled by the microcomputer 11.

  First, as shown in a period 81 in FIG. 2, while the release switch 10 is not operated, the microcomputer 11 controls the thinning unit 15 to operate in the pixel mixture drive mode. An image signal output from the CCD image sensor 21 (an image signal read from all pixels) is digitized by the A / D conversion unit 4 to output image data.

  The thinning unit 15 performs pixel mixing processing on the image data output from the A / D conversion unit 4. Specifically, in the image data output from the A / D conversion unit 4, the pixel data in the horizontal direction and the vertical direction are added to generate substantially thinned image data. That is, low-resolution image data is output from the thinning unit 15. The image data output from the thinning unit 15 is input to the two-dimensional filter 18, the image reducing unit 16, and the second subtracter 6. Subsequent operations are the same as those of the imaging apparatus shown in FIG.

  Thus, by operating the thinning unit 15 in the pixel mixture drive mode in the period 81, it is possible to reduce the image size per frame and obtain a moving image with a high frame rate.

  Next, as shown in a period 82, when the release switch 10 is operated, the microcomputer 11 controls the CCD driving unit 3 to cause the CCD image sensor 21 to perform an exposure operation. After the period 82 ends, the microcomputer 11 controls the CCD drive unit 3 to stop the exposure operation of the CCD image sensor 21.

  Next, as shown in a period 83, when the exposure operation in the CCD image sensor 21 is completed, the microcomputer 11 controls the thinning unit 15 to operate in the all-pixel reading mode. Thereby, the thinning unit 15 outputs the image data output from the CCD image sensor 21 and input via the A / D conversion unit 4 to the two-dimensional filter 18, the image reduction unit 16 and the second subtractor 6 as they are. To do. That is, high-resolution image data is output from the thinning unit 15. Subsequent operations are the same as those of the imaging apparatus shown in FIG.

  According to the imaging apparatus shown in FIG. 15, the CCD image sensor 21 that operates only in one operation mode (all pixel readout mode in the present embodiment) is a CCD image sensor that can selectively operate in a plurality of operation modes. Since it is cheaper than that, the cost of the entire apparatus can be reduced.

[Appendix 1]
The imaging apparatus of the present invention includes a plurality of photoelectric conversion elements, and a first driving mode for reading a low resolution image based on an output of the photoelectric conversion elements, or a first read mode for reading an image having a higher resolution than the low resolution image. An image sensor that can operate in two drive modes; a drive unit that drives the image sensor; a frame memory that stores image data read from the image sensor in the first drive mode; and the frame memory An image enlarging unit for enlarging the image size of the stored image data, and noise reduction of the image data read from the image sensor in the second drive mode based on the image data enlarged by the image enlarging unit And a noise reduction unit that performs processing.

  With this configuration, the image read out in the first drive mode is stored in the frame memory to perform high-speed reading, and the image read out in the second drive mode is stored in the frame memory. By performing noise reduction processing based on an enlarged image, it is possible to suppress random noise without causing resolution degradation that is strong against subject movement.

  The CCD image sensor 2 is an example of an image sensor. The CCD drive unit 3 is an example of a drive unit. The pixel mixture drive mode is an example of a first drive mode. The all-pixel readout mode is an example of a second drive mode.

[Appendix 2]
The imaging device of the present invention may be configured such that the imaging element mixes and outputs the outputs of the plurality of photoelectric conversion elements in the first drive mode.

  With this configuration, an image in which random noise is further suppressed is obtained in the first drive mode. Therefore, when performing noise reduction processing on an image read in the second drive mode, a larger random noise is obtained. A suppression effect is obtained.

[Appendix 3]
An image pickup apparatus according to the present invention includes an image pickup element including a plurality of photoelectric conversion elements, a drive unit that drives the image pickup element, and a first drive that generates a low-resolution image based on an image output from the image pickup element. Mode, or a thinning unit operable in a second driving mode for generating an image having a higher resolution than the low-resolution image, and a frame memory for storing image data output from the thinning unit in the first driving mode And an image enlarging unit for enlarging the image size of the image data stored in the frame memory, and noise reduction processing for the image read from the image sensor based on the image enlarged by the image enlarging unit A noise reduction unit.

  With this configuration, an inexpensive image sensor that operates only in the second drive mode can be used, so that the cost of the imaging apparatus can be reduced.

[Appendix 4]
The imaging device of the present invention may be configured such that the thinning unit mixes and outputs output signals of the plurality of photoelectric conversion elements read from the imaging element.

  With this configuration, an image in which random noise is further suppressed is obtained in the first drive mode. Therefore, when performing noise reduction processing on image data read from the image sensor, a larger random noise suppression effect is obtained. Is obtained.

[Appendix 5]
The imaging apparatus of the present invention may further include a filter unit that performs two-dimensional filter processing on the image data read from the thinning unit.

  With this configuration, an image in which random noise is further suppressed is obtained as compared with the case where the two-dimensional filter processing is not performed. Therefore, when performing noise reduction processing on image data read from the image sensor, A large random noise suppression effect can be obtained. In particular, since the number of photoelectric conversion elements to be two-dimensionally filtered is increased without increasing the number of taps of the two-dimensional filter, a large random noise suppression effect can be obtained with a small circuit scale.

  The two-dimensional filter 13 is an example of a filter unit.

[Appendix 6]
The imaging apparatus of the present invention includes a plurality of photoelectric conversion elements, and a first driving mode for reading a low resolution image based on an output of the photoelectric conversion elements, or a first read mode for reading an image having a higher resolution than the low resolution image. An image sensor operable in two drive modes, a drive unit that drives the image sensor, a frame memory that stores image data read from the image sensor in the first drive mode, and the second In the drive mode, an image reduction unit that reduces the image size of the image data read from the image sensor, and the image data reduced by the image reduction unit, the image sensor reads the image data in the second drive mode. A noise reduction unit that performs noise reduction processing on the read image data, and the noise reduction unit is output from the image reduction unit in the second drive mode. The image data output from the frame memory is subtracted from the image data, the difference image data is enlarged, and the enlarged image data and the image data output from the image sensor are subtracted. It is.

  With this configuration, it is possible to suppress the generation of false signals during random noise suppression processing. Therefore, the image quality of the image can be improved.

[Appendix 7]
An image pickup apparatus according to the present invention includes an image pickup element including a plurality of photoelectric conversion elements, a drive unit that drives the image pickup element, and a first drive that generates a low-resolution image based on an image output from the image pickup element. Mode, or a thinning unit operable in a second driving mode for generating an image having a higher resolution than the low-resolution image, and a frame for storing image data output from the thinning unit in the first driving mode Based on the memory, an image reduction unit that reduces the image size of the image data read from the thinning unit in the second drive mode, and the image data reduced by the image reduction unit A noise reduction unit that performs a noise reduction process on image data read out from the unit in the second drive mode, and the noise reduction unit is configured in the second drive mode, The image data output from the frame memory is subtracted from the image data output from the image reduction unit, the image data as the difference is enlarged, and the enlarged image data and the image output from the decimation unit Subtraction processing is performed on the data.

  With this configuration, it is possible to suppress the generation of false signals during random noise suppression processing. Therefore, the image quality of the image can be improved. In addition, since an inexpensive image sensor that operates only in the second drive mode can be used, the cost of the image pickup apparatus can be reduced.

[Appendix 8]
The imaging apparatus of the present invention includes a plurality of photoelectric conversion elements, and a first driving mode for reading a low resolution image based on an output of the photoelectric conversion elements, or a first read mode for reading an image having a higher resolution than the low resolution image. An image sensor operable in two drive modes, a drive unit that drives the image sensor, a frame memory that stores image data read from the image sensor in the first drive mode, and the second In the drive mode, an image enlarging unit that enlarges the size of the image data output from the frame memory, and a filter that performs low-pass filter processing on the image data read from the image sensor in the second drive mode A first subtractor for subtracting the image data output from the image enlargement unit from the image data output from the filter unit, and the imaging From the image data output from a child, in which a second subtracter for subtracting the image data output from the first subtractor.

  With this configuration, it is possible to suppress the generation of false signals during random noise suppression processing. Therefore, the image quality of the image can be improved.

  The two-dimensional filter 17 is an example of a filter unit.

[Appendix 9]
An image pickup apparatus according to the present invention includes an image pickup element including a plurality of photoelectric conversion elements, a drive unit that drives the image pickup element, and a first drive that generates a low-resolution image based on an image output from the image pickup element. A thinning unit operable in a second driving mode that generates an image having a higher resolution than the low-resolution image, or image data read from the thinning unit in the first driving mode. A frame memory; an image enlarging unit for enlarging an image size of image data output from the frame memory in the second drive mode; and image data read out from the thinning unit in the second drive mode. A filter unit that performs low-pass filter processing, and subtracts image data output from the image enlargement unit from image data output from the filter unit That a first subtractor, from the image data outputted from the thinning unit, in which a second subtracter for subtracting the image data output from the first subtractor.

  With this configuration, it is possible to suppress the generation of false signals during random noise suppression processing. Therefore, the image quality of the image can be improved. In addition, since an inexpensive image sensor that operates only in the second drive mode can be used, the cost of the image pickup apparatus can be reduced.

[Appendix 10]
The imaging apparatus of the present invention includes a plurality of photoelectric conversion elements, and a first driving mode for reading a low resolution image based on an output of the photoelectric conversion elements, or a first read mode for reading an image having a higher resolution than the low resolution image. An image sensor that can operate in two drive modes; a drive unit that drives the image sensor; a filter unit that performs two-dimensional filter processing on image data read from the image sensor; and the first from the image sensor. A frame memory for storing image data read in the driving mode, an image reducing unit for reducing the image size of the image data output from the image sensor in the second driving mode, and the image reducing unit. A first subtracter for subtracting the image data output from the frame memory from the output image data; and the first drive unit in the second drive mode. An image enlarging unit for enlarging the image size of the image data output from the subtractor, and a second subtractor for subtracting the image data output from the image enlarging unit from the image data output from the image sensor. A motion detection unit that detects a motion of an image based on the image data output from the image enlargement unit; an image data output from the filter unit based on a detection result in the motion detection unit; and the second And a switching unit that adds and outputs the image data output from the subtractor at a predetermined ratio.

  With this configuration, when the image movement is large, the filter unit operates the two-dimensional noise reduction process, and when the image movement is small, the three-dimensional noise reduction process is operated. Degradation of the S / N ratio can be prevented.

  The two-dimensional filter 18 is an example of a filter unit.

[Appendix 11]
An image pickup apparatus according to the present invention includes an image pickup element including a plurality of photoelectric conversion elements, a drive unit that drives the image pickup element, and a first drive that generates a low-resolution image based on an image output from the image pickup element. A thinning unit operable in a mode or a second drive mode that generates an image having a higher resolution than the low-resolution image, and a filter unit that performs two-dimensional filtering on the image data read from the thinning unit A frame memory that stores image data read from the thinning unit in the first driving mode, and an image that reduces the image size of the image data output from the thinning unit in the second driving mode. A reduction unit; a first subtracter that subtracts image data output from the frame memory from image data output from the image reduction unit; and the second drive. In this mode, image data output from the image enlargement unit is selected from an image enlargement unit that enlarges the image size of the image data output from the first subtractor, and image data output from the thinning unit. From the second subtractor for performing the subtraction process, the motion detection unit for detecting the motion of the image based on the image data output from the image enlargement unit, and the filter unit based on the detection result in the motion detection unit. A switching unit is provided that adds the output image data and the image data output from the second subtractor at a predetermined ratio and outputs the result.

  With this configuration, when the image movement is large, the filter unit operates the two-dimensional noise reduction process, and when the image movement is small, the three-dimensional noise reduction process is operated. Degradation of the S / N ratio can be prevented. In addition, since an inexpensive image sensor that operates only in the second drive mode can be used, the cost of the image pickup apparatus can be reduced.

  Since the present invention can reduce random noise included in an image, the present invention is useful for an electronic apparatus including an imaging device such as a digital still camera, a video camera, a surveillance camera, and a camera-equipped mobile phone terminal.

1 is a block diagram of an imaging apparatus according to a first embodiment. Timing chart of imaging apparatus according to first and second embodiments Block diagram of an imaging apparatus according to the second embodiment Block diagram of an imaging apparatus according to the third embodiment Block diagram of an imaging apparatus according to the fourth embodiment Timing chart of imaging apparatus according to embodiment 4 Waveform diagram showing how side effects occur in random noise suppression processing Waveform diagrams of respective parts of the imaging apparatus according to the fourth and fifth embodiments FIG. 9 is a block diagram of another example of the imaging apparatus according to the fourth embodiment. Block diagram of an imaging apparatus according to the fifth embodiment Timing chart of imaging apparatus according to embodiment 5 FIG. 9 is a block diagram of another example of the imaging apparatus according to the fifth embodiment. Block diagram of an imaging apparatus according to the sixth embodiment FIG. 10 is a characteristic diagram showing the relationship between the difference value and the coefficient in the motion detection unit according to the sixth embodiment. FIG. 6 is a logic circuit diagram of a switching unit according to the sixth embodiment. FIG. 10 is a block diagram of another example of the imaging apparatus according to the sixth embodiment. Block diagram of two-dimensional filter processing unit Schematic diagram showing the concept of synchronization of the two-dimensional filter processing unit Schematic diagram showing the combination of pixels in the pixel mixing drive mode of the imaging device Block diagram showing an example of a cyclic noise reduction circuit Diagram showing input / output characteristics of attenuation section

Explanation of symbols

2, 21 CCD image sensor 3 CCD drive unit 5 1st subtractor 6 2nd subtractor 7 Attenuation unit 8 Image enlargement unit 9 Frame memory 13, 17, 18 Two-dimensional filter 14 Noise reduction circuit 15 Thinning-out unit 16 Image reduction Unit 19 motion detection unit 20 switching unit

Claims (12)

A plurality of photoelectric conversion elements are provided, and can be operated in a first drive mode for reading a low-resolution image based on an output of the photoelectric conversion element, or in a second drive mode for reading an image having a higher resolution than the low-resolution image An image sensor,
A drive unit for driving the image sensor;
A frame memory for storing image data read from the image sensor in the first drive mode;
An image enlarging unit for enlarging the image size of the image data stored in the frame memory;
An image pickup apparatus comprising: a noise reduction unit that performs a noise reduction process on image data read from the image pickup device in the second drive mode based on the image data enlarged by the image enlargement unit.   The imaging device according to claim 1, wherein the imaging device mixes and outputs outputs of the plurality of photoelectric conversion elements in the first drive mode.   The imaging apparatus according to claim 1, further comprising a filter unit that performs two-dimensional filtering on the image data read from the imaging element. An imaging device including a plurality of photoelectric conversion elements;
A drive unit for driving the image sensor;
A thinning unit operable in a first drive mode for generating a low-resolution image based on an image output from the image sensor, or in a second drive mode for generating an image having a higher resolution than the low-resolution image; ,
A frame memory for storing image data output from the thinning unit in the first drive mode;
An image enlarging unit for enlarging the image size of the image data stored in the frame memory;
An image pickup apparatus comprising: a noise reduction unit that performs noise reduction processing on image data read from the image pickup device based on image data enlarged by the image enlargement unit.   The imaging apparatus according to claim 4, wherein the thinning unit mixes and outputs outputs of the plurality of photoelectric conversion elements read from the imaging element.   The imaging apparatus according to claim 4, further comprising a filter unit that performs two-dimensional filter processing on the image data read from the thinning unit. A plurality of photoelectric conversion elements are provided, and can be operated in a first drive mode for reading a low-resolution image based on an output of the photoelectric conversion element, or in a second drive mode for reading an image having a higher resolution than the low-resolution image An image sensor,
A drive unit for driving the image sensor;
A frame memory for storing image data read from the image sensor in the first drive mode;
An image reduction unit for reducing the image size of the image data read from the image sensor in the second drive mode;
A noise reduction unit that performs noise reduction processing of image data read out from the image sensor in the second drive mode based on the image data reduced by the image reduction unit;
The noise reduction unit is
In the second drive mode, the image data output from the frame memory is subtracted from the image data output from the image reduction unit, the image data that is the difference is enlarged, and the enlarged image data and An imaging apparatus that performs a subtraction process on image data output from the imaging element. An imaging device including a plurality of photoelectric conversion elements;
A drive unit for driving the image sensor;
A thinning unit operable in a first drive mode for generating a low-resolution image based on an image output from the image sensor, or in a second drive mode for generating an image having a higher resolution than the low-resolution image; ,
A frame memory for storing image data output from the thinning unit in the first drive mode;
An image reduction unit for reducing the image size of the image data read from the thinning unit in the second drive mode;
A noise reduction unit that performs a noise reduction process on the image data read from the thinning unit in the second drive mode based on the image data reduced by the image reduction unit;
The noise reduction unit is
In the second drive mode, the image data output from the frame memory is subtracted from the image data output from the image reduction unit, the image data that is the difference is enlarged, and the enlarged image data and An imaging apparatus that performs a subtraction process on image data output from the thinning unit. A plurality of photoelectric conversion elements are provided, and can be operated in a first drive mode for reading a low-resolution image based on an output of the photoelectric conversion element, or in a second drive mode for reading an image having a higher resolution than the low-resolution image An image sensor,
A drive unit for driving the image sensor;
A frame memory for storing image data read from the image sensor in the first drive mode;
An image enlarging unit for enlarging the image size of the image data output from the frame memory in the second drive mode;
A filter unit that performs low-pass filter processing on image data read from the image sensor in the second drive mode;
A first subtractor for subtracting image data output from the image enlargement unit from image data output from the filter unit;
An image pickup apparatus, comprising: a second subtracter that subtracts image data output from the first subtracter from image data output from the image sensor. An imaging device including a plurality of photoelectric conversion elements;
A drive unit for driving the image sensor;
A thinning unit operable in a first drive mode for generating a low-resolution image based on an image output from the image sensor, or in a second drive mode for generating an image having a higher resolution than the low-resolution image; ,
A frame memory for storing image data read from the thinning unit in the first drive mode;
An image enlarging unit for enlarging the image size of the image data output from the frame memory in the second drive mode;
A filter unit that performs low-pass filter processing on the image data read in the second driving mode from the thinning unit;
A first subtractor for subtracting image data output from the image enlargement unit from image data output from the filter unit;
An image pickup apparatus comprising: a second subtracter that subtracts image data output from the first subtracter from image data output from the thinning unit. A plurality of photoelectric conversion elements are provided, and can be operated in a first drive mode for reading a low-resolution image based on an output of the photoelectric conversion element, or in a second drive mode for reading an image having a higher resolution than the low-resolution image An image sensor,
A drive unit for driving the image sensor;
A filter unit that performs two-dimensional filter processing on the image data read from the image sensor;
A frame memory for storing image data read from the image sensor in the first drive mode;
An image reduction unit that reduces the image size of the image data output from the image sensor in the second drive mode;
A first subtractor for subtracting image data output from the frame memory from image data output from the image reduction unit;
An image enlarging unit for enlarging the image size of the image data output from the first subtracter in the second drive mode;
A second subtracter for subtracting the image data output from the image enlargement unit from the image data output from the image sensor;
A motion detection unit that detects the motion of the image based on the image data output from the image enlargement unit;
A switching unit that adds the image data output from the filter unit and the image data output from the second subtractor at a predetermined rate based on the detection result in the motion detection unit, and outputs the result. An imaging device. An imaging device including a plurality of photoelectric conversion elements;
A drive unit for driving the image sensor;
A thinning unit operable in a first drive mode for generating a low-resolution image based on an image output from the image sensor, or in a second drive mode for generating an image having a higher resolution than the low-resolution image; ,
A filter unit that performs two-dimensional filter processing on the image data read from the thinning unit;
A frame memory for storing image data read from the thinning unit in the first drive mode;
An image reduction unit for reducing the image size of the image data output from the thinning unit in the second drive mode;
A first subtractor for subtracting image data output from the frame memory from image data output from the image reduction unit;
An image enlarging unit for enlarging the image size of the image data output from the first subtracter in the second drive mode;
A second subtractor for subtracting the image data output from the image enlargement unit from the image data output from the thinning unit;
A motion detection unit that detects the motion of the image based on the image data output from the image enlargement unit;
A switching unit that adds the image data output from the filter unit and the image data output from the second subtractor at a predetermined rate based on the detection result in the motion detection unit, and outputs the result. An imaging device.

Images