JP4769634B2 - Imaging device and noise removal method in imaging device - Google Patents

Description

  The present invention relates to a technique for reducing noise and camera shake that occur when an imaging device such as a digital camera mainly captures a low-luminance subject.

  Generally, in a digital camera that obtains a still image using an image sensor, the subject brightness is low, and when shooting is performed without using a flash or the like, the charge accumulation time of the image sensor is increased (that is, the shutter speed is decreased). The brightness of the captured image is adjusted by increasing the amplification factor (generally ISO sensitivity) of the output signal from the image sensor.

  However, when the shutter speed is slowed down, the position of the image formed on the image sensor relatively moves due to the movement of the photographer during imaging, and so-called camera shake occurs. On the other hand, when the sensitivity is increased, there is a problem that noise is conspicuous because an output signal with a small S / N is amplified with a high amplification factor because the amount of charge originally stored in each pixel of the image sensor is small.

  As countermeasures, exposure is performed n times with n times the shutter speed and n times sensitivity during normal shooting (n is an integer of 2 or more), and camera shake is electronically corrected from n exposure data, and noise is averaged. (Patent Document 1) to obtain a single still image that is reduced by the conversion, or multiply the shutter speed n times, similarly n exposures, and add the obtained n exposure data while considering the blur amount Attempts have been made to obtain a still image with an exposure amount of n times by combining without degrading the S / N (Patent Document 2).

In addition to these countermeasures, there has been an attempt to reduce the generated noise by noise removal processing by performing single exposure with n times the sensitivity alone.
JP 2004-266648 A (page 4-8, FIG. 1-7) JP 2000-224470 A (pages 5-8, 2-7)

  However, the above-described method of performing multiple exposures involves the problem that the transfer amount of exposure data and the storage capacity increase n times because exposure is performed n times to obtain n times the exposure amount.

  In addition, in the method of performing noise removal processing after single exposure with n times the sensitivity, when the S / N of the exposure data is remarkably deteriorated, it becomes difficult to separate the noise and the signal, and a sufficient noise removal effect is obtained. There is a problem that it cannot be obtained.

  The present invention has been made in order to solve the above-described problems. Even when the subject brightness is low, the present invention does not require a flash or a camera shake detection sensor, and suppresses the occurrence of camera shake and noise. An object of the present invention is to provide an imaging apparatus capable of obtaining a still image with an exposure amount and reducing the storage capacity. Moreover, it aims at providing the noise removal method in an imaging device.

An image pickup apparatus according to the present invention includes an image sensor that converts received light into an electrical signal, an amplifier that amplifies the electrical signal, an A / D converter that converts the amplified signal into a digital signal, the image sensor, A signal processing unit that controls the amplifier and processes the digital signal by the A / D converter, and the signal processing unit includes:
Means for acquiring A exposure data by performing imaging with A exposure in which the exposure time of the image sensor is set relatively short and the amplification factor of the amplifier is set relatively high;
Means for obtaining B exposure data by performing imaging with B exposure for setting a relatively long exposure time of the image sensor and a relatively low amplification factor of the amplifier;
And a means for removing noise from the A exposure data by correction using the B exposure data. In this configuration, the signal processing unit is preferably a DSP (Digital Signal Processor) or an image processing LSI.

  In the A exposure, the exposure time in the image sensor is set to be short and the amplification factor in the amplifier at the subsequent stage of the image sensor is set to be high to perform imaging. A exposure has a high shutter speed and high sensitivity. In the A exposure, since the shutter speed is fast, there is little camera shake, but at low luminance, the image tends to be dark and noise tends to increase.

  On the other hand, B exposure is performed by setting the exposure time in the image sensor to be long and setting the amplification factor in the amplifier to be low. B exposure has a low shutter speed and low sensitivity. In B exposure, since the exposure time is long, the image is bright even at low luminance, and the S / N is good, but the camera speed tends to increase due to the slow shutter speed.

  When generating still image data, both A exposure and B exposure are performed, the pixel value of the A exposure data is corrected using the pixel value of the B exposure data, and noise is removed from the A exposure data. As a result, a still image with small camera shake and good S / N can be obtained. Even when the subject brightness is low, it is possible to obtain a still image with an optimum exposure amount while suppressing camera shake and noise by performing exposure twice or less without requiring a flash or a camera shake detection sensor. Since the number of exposures is small, the amount of exposure data is small, which is suitable for high-speed processing, and requires less storage capacity.

  In the imaging apparatus having the above-described configuration, the signal processing unit further includes means for calculating a similarity of the B exposure data to the A exposure data for each small region in the noise removal processing, and according to the similarity Thus, there is a mode in which the degree of contribution of the B exposure data to the correction to the A exposure data is adjusted.

  The degree of correction, that is, the degree of contribution to the A exposure data by the B exposure data for noise removal differs depending on the similarity, regardless of the S / N of the pixel value. Therefore, the higher the similarity is, the higher the contribution is set, and conversely, the lower the similarity is, the lower the contribution is set. As a result, it is possible to enhance the correction of the B exposure data to the A exposure data, that is, the noise removal effect on the A exposure data.

  Moreover, it is preferable that the signal processing unit in the above configuration calculates the similarity after performing blur correction of the B exposure data based on the A exposure data in the noise removal processing.

  It is considered that the B exposure data with a slow shutter speed contains more blurring elements than the A exposure data with a fast shutter speed. Therefore, prior to correcting the A exposure data by the B exposure data, blur correction of the B exposure data is performed based on the A exposure data, and the similarity is corrected. The correction contribution from the B exposure data to the A exposure data is adjusted according to the corrected similarity. As a result, the effect of removing noise from the A exposure data by the B exposure data can be further enhanced.

  Further, the signal processing unit configured as described above performs the blur correction of the B exposure data based on the A exposure data only under a condition where the similarity of the B exposure data to the A exposure data is not more than a predetermined value. There is.

  Even if there is a high possibility that the B exposure data contains more elements of camera shake, this is not always the case. Depending on the situation, camera shake may be sufficiently small in the B exposure data. Therefore, the degree of similarity of the B exposure data with respect to the A exposure data is examined, and the blur correction of the B exposure data based on the A exposure data is performed only when the similarity is equal to or less than a predetermined value. When the degree of similarity is relatively high, the blur correction is omitted, so that the speed can be increased.

  Further, the signal processing unit having the above-described configuration may determine the contribution of the A exposure data to the blur correction in the blur correction of the B exposure data based on the A exposure data. There is an aspect in which adjustment is performed according to the average luminance.

  With this configuration, it is possible to reduce the adverse effect on the blur correction of the B exposure data based on the A exposure data, for example, by reducing the contribution of the blur correction to a low luminance region where dark current noise becomes remarkable. .

  Further, in the above configuration, there is a mode in which the image sensor is single, and the signal processing unit performs the B exposure after performing the A exposure using the single image sensor. Alternatively, there may be a plurality of image sensors, and the signal processing unit may perform the A exposure and the B exposure simultaneously using the plurality of image sensors. When multiple image sensors are installed and A exposure and B exposure are performed simultaneously, signal processing by the signal processing unit can be started immediately after the end of B exposure with a slow shutter speed, thus shortening the processing time. become.

  Further, in the above configuration, the image sensor is configured in an XY address type, and an array of image sensors that can control the reset timing of accumulated charges in line units, and amplifies signals from the array of image sensors. There is an aspect in which the A exposure and the B exposure are performed at the same time while adjusting the exposure time and the amplification factor for each line.

  In this configuration, the A exposure and the B exposure can be performed simultaneously while using a single image sensor, and the processing time can be shortened.

  Further, in the above configuration, the signal processing unit includes a line memory that stores data for the A exposure and the B exposure for a required number of lines, and performs the noise removal processing on the line memory. is there. With this configuration, the memory capacity required for noise removal processing can be greatly reduced.

In addition, the noise removal method in the imaging apparatus according to the present invention includes:
With respect to exposure data obtained from an image sensor, capturing A exposure data by performing imaging with A exposure that sets the shutter speed of the image sensor at high speed and high sensitivity; and
Capturing B exposure data by performing imaging with B exposure for setting the shutter speed of the image sensor at a low speed and low sensitivity; and
Performing noise removal on the A exposure data by correction using the B exposure data ;
Calculating the similarity of the B exposure data to the A exposure data for each small area in the noise removal process,
In the noise removal process, after the blur correction of the B exposure data is performed based on the A exposure data, the similarity is calculated, and the A exposure data is corrected by the B exposure data according to the similarity. It adjusts the degree of contribution .

  In the noise removal method in the imaging apparatus, the sensitivity is preferably set to an amplification factor of an amplifier that amplifies a signal from the image sensor.

  According to the present invention, even when the subject brightness is low, it is possible to obtain a still image with an optimal exposure amount while suppressing camera shake and noise with two or less exposures without requiring a flash or a camera shake detection sensor. it can.

  Hereinafter, an imaging apparatus according to an embodiment of the present invention will be described with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram showing a schematic configuration of an imaging apparatus according to Embodiment 1 of the present invention. In FIG. 1, 1 is a lens for capturing light from outside the imaging apparatus, 2 is a mechanical shutter for allowing light incident from the lens 1 to pass through for a specified period (1 / shutter speed), and 3 is incident light. An image sensor for accumulating charges as a result of photoelectric conversion and outputting them as an analog signal, 4 an amplifier for amplifying the analog signal by a specified magnification (sensitivity), and 5 an A / D for converting the analog signal into a digital signal A converter 6 is a signal processing unit (DSP or signal processing LSI) for performing various signal processing on the digital signal, and 7 is a transfer pulse for causing the image sensor 3 to transfer charges, and a charge accumulation time. A timing generator (TG) 8 for supplying a driving pulse such as an electronic shutter pulse to be determined, A frame memory for temporarily storing the tall signal as data, 9 is a recording medium for storing data in the frame memory 8 finally.

  Next, the operation of the imaging apparatus of the present embodiment configured as described above will be described with reference to the flowchart of FIG.

  First, in step S1, A exposure is performed to acquire A exposure data Ea. That is, in the A exposure, the signal processing unit 6 designates a high-speed shutter (short exposure time) for the mechanical shutter 2 and the timing generator 7, and the timing generator 7 further shortens the charge accumulation time for the image sensor 3. Give an electronic shutter pulse. Further, the signal processing unit 6 designates high sensitivity (high amplification factor) for the amplifier 4. As a result, exposure with a high shutter speed and high sensitivity is performed, and the signal processing unit 6 obtains A exposure data Ea and stores it in the frame memory 8.

  Next, in step S2, B exposure is executed to acquire B exposure data Eb. That is, in B exposure, the signal processing unit 6 designates a low-speed shutter (long exposure time) for the mechanical shutter 2 and the timing generator 7, and the timing generator 7 further increases the charge accumulation time for the image sensor 3. Give an electronic shutter pulse. The signal processing unit 6 designates low sensitivity (low amplification factor) for the amplifier 4. Thus, exposure is performed with a low shutter speed and low sensitivity, and the signal processing unit 6 obtains B exposure data Eb and stores it in the frame memory 8.

  As for the order of A exposure and B exposure, as shown in FIG. 9A, first, A exposure with a high shutter speed is performed, and then B exposure with a low shutter speed is performed. This is because the image finally stored in the recording medium 9 is obtained by removing noise from the A exposure data Ea. In order to obtain a still image at the timing intended by the photographer, the A exposure data Ea is used. This is because it is desirable to expose first.

  However, when a plurality of image sensors 3 are used, as shown in FIG. 9B, A exposure and B exposure can be performed simultaneously. In this case, the signal processing by the signal processing unit 6 can be started immediately after the B exposure with a slow shutter speed is completed, and the photographing time can be shortened.

  Next, in step S3, the signal processing unit 6 divides the exposure data Ea and B exposure data Eb on the frame memory 8 into small areas as shown in FIGS. 3 (a) and 3 (b).

  Next, in step S4, a loop process is set for each of the small areas of the A exposure data Ea and the B exposure data Eb.

In step S5, the similarity S between the A exposure data Ea and the B exposure data Eb is obtained for each small area. First, for each small area, an average value M _E1 of pixel values of A exposure data Ea and an average value M _E2 of pixel values of B exposure data Eb are obtained. The similarity S is defined so as to be higher in a region where the difference between M _E1 and M _E2 is smaller. The calculation formula of the similarity S is shown below.

S = (2n-1)-| M _E1 -M _E2 |, (n is the number of bits of exposure data)
As a result, the B exposure data Eb with respect to the A exposure data Ea has an area Q1 in which the similarity S is slightly low due to camera shake or the like, an area Q2 in which the similarity S is extremely low due to subject blur or a new subject entering the screen, It is divided into a region Q3 where the influence of blur is not so noticeable and the similarity S is high. If the similarity S is expressed as an image, it is as shown in FIG. Since the similarity S only needs to be referred to for each small area, a frame memory for storing the similarity S corresponding to the entire exposure data is not necessary.

  Next, in step S6, it is determined whether or not the similarity S obtained above is less than a threshold value THs. If this determination result is No and the similarity S is equal to or greater than the threshold value THs, the signal processing unit 6 proceeds to step S9, assuming that there is no problem even if the B exposure data Eb is used as a reference for noise removal processing on the A exposure data Ea. Then, the following noise removal processing is performed.

  4A and 4B are enlarged views of A exposure data Ea and B exposure data Eb on the frame memory 8 and a part thereof. First, the pixel value of the pixel p1 to be processed on the A exposure data Ea is Pa, the pixel value of the pixel p2 at the corresponding coordinate on the B exposure data Eb is Pb, and the difference ΔP between the pixel value Pa and the pixel value Pb is as follows: It is calculated by the following formula.

ΔP = | Pa−Pb |
Next, the addition average value Pm of the pixel value Pa and the surrounding eight pixels pc is obtained by the following equation.

Pm = Σ {Pa, Pc} / 9
Here, since the pixel value Pb is usually better in S / N than the pixel value Pa from the exposure condition, when the difference ΔP is large, the pixel value Pa is highly likely to be noise, and conversely, the difference ΔP is small. In this case, there is a high possibility that the pixel value Pa is a normal value. Therefore, as for the pixel value Pa ′ after noise removal, as shown in the graph of FIG. 5, when the difference ΔP is less than the threshold value TH, the value of the pixel value Pa is used as it is. On the other hand, when the difference ΔP is equal to or greater than the threshold value TH, the pixel value Pa ′ is processed so as to approach the value of the addition average value Pm with the correction gradient G and smooth the noise. This is expressed as follows.

Pa ′ = Pa + G (ΔP−TH),
(However, Pa ≦ Pa ′ ≦ Pm or Pm ≦ Pa ′ ≦ Pa)
G = G ′ (Pm−Pa), (G ′ is the correction slope gain)
However, as indicated by three diagonal lines, the difference ΔP increases as the similarity S decreases (the pixel value Pb is deviated from the pixel value Pa) regardless of the S / N of the pixel value Pa, and the similarity The higher the S, the smaller the difference ΔP, and a deviation occurs from the ideal correction inclination. Therefore, the correction inclination gain G ′ is defined as follows to correct the deviation.

G ′ = αS, (α is a constant)
Accordingly, the correction gradient G increases as the similarity S increases, and the contribution of the term of the difference ΔP composed of the pixel values Pb to the pixel value Pa ′ increases, that is, the contribution of the B exposure data Eb to noise removal increases. Get higher. That is, the higher the similarity S, the greater the noise removal effect. Then, the pixel value Pa ′ is calculated while shifting the coordinates from the beginning to the end of the small area, and the noise-removed data Ea ′ is obtained and stored in the frame memory 8.

  By performing the noise removal processing from step S6 to step S9, the S / N is remarkably poor, and the similarity S can be obtained even when it is not possible to determine whether the signal is noise or signal from a single exposure data. If it is relatively large, a sufficient noise removal effect can be obtained without going through steps S7 → S8. As Pm, a result such as a median filter or a nearest value filter may be used in addition to a simple addition average.

  On the other hand, if the similarity S is less than the threshold value THs, the process proceeds to step S7 before correcting the A exposure data Ea by the B exposure data Eb to remove noise, and the B exposure is performed based on the A exposure data Ea. The blur correction of the data Eb is performed, and then the similarity S is calculated again in step S8. FIG. 6 shows a procedure for blur correction of the B exposure data Eb based on the A exposure data Ea.

  First, in step S11, as shown in FIG. 7, the A edge component D1 and the B edge component D2 are extracted for each small region from the A exposure data Ea and the B exposure data Eb by HPF (high pass filter). Note that edge component extraction is not particularly limited, such as a Sobel filter or a Laplacian filter. Further, since the edge component only needs to be referenced for each small area, a frame memory for storing the edge component corresponding to the entire exposure data is not necessary.

  Next, in step S12, the motion vector V of the B exposure data Eb with respect to the A exposure data Ea is obtained from the difference in the peak position d2 of the B edge component D2 with reference to the peak position d1 of the A edge component D1. When the peak position d1 of the A edge component D1 is (xa, ya) and the peak position d2 of the B edge component D2 is (xb, yb), the motion vector V is defined as follows.

V = (xb−xa, yb−ya)
However, the accuracy of the motion vector V may be lowered in a region where the noise level of the A exposure data Ea is high. In general, the noise level of exposure data varies depending on the luminance.

  Therefore, in step S13, the motion vector V is corrected according to the average luminance Y in the small area of the A exposure data Ea, thereby adjusting the degree of contribution of the A exposure data Ea to the blur correction of the B exposure data Eb.

  The calculation formula of the corrected motion vector V ′ is shown below.

V '= g0V + (g1V-g0V) .Y / Y1 (when Y <Y1)
V ′ = g1V + (g2V−g1V) · (Y−Y1) / (Y2−Y1) (when Y1 ≦ Y <Y2)
V ′ = g2V + (g3V−g2V) · (Y−Y2) / (Y3−Y2) (when Y2 ≦ Y <Y3)
V ′ = g3V (when Y3 ≦ Y)
However, g0, g1, g2, and g3 are constants of 1 or less.

  As a result, the corrected motion vector V ′ can be changed in magnitude according to the average luminance Y as shown in the graph of FIG. 8, and the corrected motion vector V ′ becomes lower in the low luminance region where dark current noise becomes conspicuous. By reducing ′, etc., it is possible to reduce the adverse effect on blur correction of the B exposure data Eb based on the A exposure data Ea.

  Next, in step S14, a recovery filter is calculated from the corrected motion vector V ′ obtained above.

  Next, in step S15, blur correction is performed by applying a recovery filter to the B exposure data Eb to obtain B exposure data Eb 'after blur correction.

  In step S8, the similarity S is calculated again with respect to the B exposure data Eb ′ after blur correction.

  The calculation of the recovery filter is not particularly limited with respect to the method of generating the recovery filter based on the correlation of images after performing motion vector correction used in MPEG or the like. Further, a motion vector may be directly obtained from exposure data without extracting an edge component, or blur correction may be performed using an inverse filter or a Wiener filter.

  Finally, proceeding to step S9, based on the updated similarity S, noise removal processing is performed in the same manner as when the similarity S is equal to or higher than THs, and A exposure data Ea ′ is obtained and written into the frame memory 8.

  The above processing is executed for the entire exposure data while switching the target small area, and finally the A exposure data Ea ′ corresponding to the entire exposure data stored in the frame memory 8 is stored in the recording medium 9. Thereby, it is possible to obtain a still image with an optimum exposure amount while suppressing the occurrence of camera shake and noise.

(Embodiment 2)
The image pickup apparatus according to the second embodiment of the present invention is configured to perform A exposure and B exposure simultaneously while using a single image sensor.

  The image sensor 3 is configured as an XY address type, and an array of image pickup elements whose stored charge reset timing can be controlled in units of lines, and amplifies signals from the array of image sensors, and the amplification factor is changed in units of lines. An image sensor configured to have a possible amplifying unit is employed. A exposure and B exposure are performed simultaneously while adjusting the exposure time (shutter speed) and amplification factor (sensitivity) for each line.

  FIG. 10 shows the structure of the image sensor 3 according to the second embodiment of the present invention. An area sensor 52 is composed of a plurality of image sensors 51. When the charge read timing reference signal RD is given, the vertical shift register 53 sequentially shifts with time to send charge read timing signals RD0, RD1,... For each line of the area sensor 52 to the charge read control unit 56. Is configured to do.

  When the reset shift register 54 is supplied with the charge reset timing reference signal RT, the reset shift register 54 sequentially shifts with the passage of time, so that the charge reset timing signals RT0, RT1,. It is configured to send out.

  The reset mask control unit 55 performs control to individually mask the charge reset timing signals RT0, RT1,... From the reset shift register 54, and to permit or prohibit transmission of the signals to the charge read control unit 56.

  The charge readout control unit 56 receives the charge readout timing signal and the charge reset timing signal as input, and controls the readout and sweeping out of the charge of each area sensor 52 (the accumulated charge of the image sensor is made zero). When the charge reset timing signal is input, charge is accumulated in the image sensor with the period from the charge reset timing signal to the charge read timing signal as the exposure time. When the charge reset timing signal is not input, the charge read Control is performed so that charges are accumulated with the exposure time from the timing signal to the next charge readout timing signal. Thereafter, the charges exposed and accumulated in line units are vertically transferred to the horizontal shift register 57.

  The horizontal shift register 57 transfers the transferred charges horizontally one pixel at a time, and the transferred charges are amplified by the amplifying unit 59 and converted into a voltage signal.

  The amplification factor controller 58 receives the high amplification factor AGh and the low amplification factor AG1, receives the line number being transferred from the charge readout controller 56, and based on the line number, sends the amplification factor AG ( AGh or AGl) is set.

  A method of simultaneously performing A exposure and B exposure in units of lines using the image sensor 3 having the above configuration will be described below.

  First, as shown in FIG. 11, since the primary color image sensor generally has a Bayer array composed of R, Gr, Gb, and B image sensors, A exposure and B exposure are switched every two lines. I will do it. Then, the reset mask control unit 55 performs setting so as to permit the transmission of the charge reset timing signal in the 4n, 4n + 1 line (n is an integer) and prohibit the transmission of the charge reset timing signal in the 4n + 2, 4n + 3 line. . Further, the amplification factor control unit 58 performs setting so that the amplification factor AG = AGh in the 4n, 4n + 1 line, and the amplification factor AG = AG1, in the 4n + 2, 4n + 3 line.

By the above, as shown in FIG. 12, the exposure time T _S becomes from the line 0,1 charge reset timing reference signal RT0, RT1 to charge readout timing reference signal RD0, RD1, the line 2,3 RT2, the RT3 RD2 , T _L up to RD3. At this time, the mechanical shutter 2 may be operated in accordance with the longer exposure time _TL . The amplification factors are AG = AGh for lines 0 and 1, and AG = AG1 for lines 2 and 3.

  Therefore, as shown in FIG. 9C, A exposure and B exposure can be performed simultaneously in units of four lines. However, the exposure start timing of each line is not completely simultaneous because it is shifted by the transfer time by transferring the charge in line units. However, even if a shift corresponding to the transfer time is taken into account, it is possible to shorten the photographing time as compared with the case where the single image sensor 3 is used in the first embodiment.

  In addition, since the A exposure and the B exposure are switched every two lines, the vertical resolution of the still image obtained is naturally half that of the first embodiment. However, in the present embodiment, the manufacturing cost and power consumption can be suppressed as compared with the case where a plurality of image sensors 3 are used.

  Note that since the vertical phases of A exposure and B exposure are shifted by two pixels, calculation of similarity S, blur correction, and noise removal processing cannot be performed as they are, but this is dealt with by performing interpolation processing as shown in FIG. . Paying attention to R in the Bayer array, the noise removal processing range is 61 for the pixel value Pa in A exposure and the pixel value Pb in B exposure in noise removal processing, and small in similarity S calculation and blur correction. The area is 62, and the bold pixels in the figure required for each process are obtained by averaging the upper and lower pixels.

  Furthermore, a line memory for storing A exposure data Ea and B exposure data Eb for the minimum number of lines necessary for noise removal processing (including preprocessing similarity S calculation and blur correction) is provided. The amount of use of the frame memory 8 can be reduced by performing the removal process.

  In the first embodiment, the A exposure data Ea and the B exposure data Eb are obtained in units of frames (for one screen) and in time series. Therefore, a memory for storing exposure data must also be provided in units of frames. However, in this embodiment, since the A exposure data Ea and the B exposure data Eb are obtained alternately every two lines, it is possible to have a memory for storing the exposure data in units of two lines. For example, in the case of the processing shown in FIG. 13, a line memory that stores data of a total of 14 lines × 9 pixels of A exposure data Ea and B exposure data Eb is included, a noise removal processing range 61, and a similarity S calculation. In addition, by moving the small region 62 in blur correction around the pixel value Pa as the process proceeds, the process from exposure to noise removal is pipelined in units of 14 lines as shown in the timing chart of FIG. Is possible. Therefore, this embodiment can significantly reduce the amount of memory required as compared with the first embodiment.

  The present invention relates to a technique for reducing noise and camera shake that occurs when shooting a low-luminance subject. Since a S / N is high and a still image free of camera shake can be obtained without using a flash or the like, a digital camera or the like can be obtained. This is useful for reducing the size and improving the image quality.

1 is a block diagram showing a schematic configuration of an imaging apparatus according to Embodiment 1 of the present invention. The flowchart which shows operation | movement of the imaging device in Embodiment 1 of this invention. Conceptual diagram of similarity in Embodiment 1 of the present invention Conceptual diagram of the noise removal processing range in the first embodiment of the present invention FIG. 5 is a characteristic diagram regarding noise removal processing according to the first embodiment of the present invention. Flowchart of shake correction in Embodiment 1 of the present invention Conceptual diagram of motion vector in Embodiment 1 of the present invention Characteristic chart relating to motion vector correction in Embodiment 1 of the present invention Timing chart of processing in Embodiments 1 and 2 of the present invention The block diagram which shows the structure of the image sensor used in Embodiment 2 of this invention. Corresponding diagram of Bayer arrangement, A exposure, and B exposure in Embodiment 2 of the present invention Relationship diagram between exposure time and amplification factor for each line in Embodiment 2 of the present invention Conceptual diagram of pixel interpolation in Embodiment 2 of the present invention Timing chart of pipeline processing in embodiment 2 of the present invention

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Lens 2 Mechanical shutter 3 Image sensor 4 Amplifier 5 A / D converter 6 Signal processing part 7 Timing generator (TG)
8 Frame memory 9 Recording medium 51 Image sensor 52 Area sensor 53 Vertical shift register 54 Reset shift register 55 Reset mask control unit 56 Charge readout control unit 57 Horizontal shift register 58 Amplification rate control unit 59 Amplification unit Q1 Similarity S is slightly low Area Q2 Area where similarity S is extremely low Q3 Area where similarity S is high

Claims (9)

An image sensor that converts received light into an electrical signal;
An amplifier for amplifying the electrical signal;
An A / D converter that converts the amplified signal into a digital signal;
A signal processing unit for controlling the image sensor and the amplifier and processing the digital signal by the A / D converter;
The signal processing unit
Means for acquiring A exposure data by performing imaging with A exposure in which the exposure time of the image sensor is set relatively short and the amplification factor of the amplifier is set relatively high;
Means for obtaining B exposure data by performing imaging with B exposure for setting a relatively long exposure time of the image sensor and a relatively low amplification factor of the amplifier;
Means for performing noise removal on the A exposure data by correction using the B exposure data ;
Means for calculating the similarity of the B exposure data to the A exposure data for each small area in the noise removal process;
In the noise removal process, after the blur correction of the B exposure data is performed based on the A exposure data, the similarity is calculated, and the A exposure data is corrected by the B exposure data according to the similarity. An imaging device that adjusts the degree of contribution . 2. The imaging according to claim 1 , wherein the signal processing unit performs blur correction of the B exposure data based on the A exposure data only under a condition in which a similarity of the B exposure data to the A exposure data is a predetermined value or less. apparatus. In the blur correction of the B exposure data based on the A exposure data, the signal processing unit determines a contribution degree to the blur correction by the A exposure data according to a pixel value of the A exposure data or an average luminance of a small area. the imaging apparatus according to claim 1 or claim 2 to adjust. The image sensor is a single, the signal processing unit, by using the single image sensor, according to claim 1 for the B exposure after performing the A exposure to claim 3 Imaging device. The image sensor is a plurality of the signal processing unit uses the plurality of image sensors, image pickup apparatus according to claim 1 for the B exposure and the A exposure at the same time claim 3. The image sensor is configured in an XY address type, and an image pickup element array in which reset timing of accumulated charges can be controlled in line units, and a signal from the image pickup element array are amplified, and an amplification factor thereof in line units. The imaging according to any one of claims 1 to 3 , wherein the imaging unit is configured to include a changeable amplification unit, and the A exposure and the B exposure are performed simultaneously while adjusting an exposure time and an amplification factor for each line. apparatus. The imaging apparatus according to claim 6 , wherein the signal processing unit includes a line memory that stores data for the A exposure and the B exposure for a required number of lines, and performs the noise removal processing on the line memory. . With respect to exposure data obtained from an image sensor, capturing A exposure data by performing imaging with A exposure that sets the shutter speed of the image sensor at high speed and high sensitivity; and
Capturing B exposure data by performing imaging with B exposure for setting the shutter speed of the image sensor at a low speed and low sensitivity; and
Performing noise removal on the A exposure data by correction using the B exposure data ;
Calculating the similarity of the B exposure data to the A exposure data for each small area in the noise removal process,
In the noise removal process, after the blur correction of the B exposure data is performed based on the A exposure data, the similarity is calculated, and the A exposure data is corrected by the B exposure data according to the similarity. For removing noise in an imaging apparatus that adjusts the contribution to the image. The noise removal method for an imaging apparatus according to claim 8 , wherein the sensitivity is an amplification factor of an amplifier that amplifies a signal from the image sensor.

Images