JP2007013270A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus making a photographed image low in noise and high in image quality. <P>SOLUTION: A photographing condition setting section 120 sets photographing parameters for determining photographing conditions. A read pixel discrimination section 122 sets the number of pixels of the photographed image after pixel mixed reading on the basis of the photographing parameters set by the photographing condition setting section 120. A pixel signal of the photographed image acquired by an imaging section 50 is given to a first image processing section 111, wherein the pixel signal is subjected to pixel mixture read processing. In this case, since the pixel mixed reading is executed so that the number of the photographed image after the pixel mixed reading is equal to the number of pixels set by the read pixel discrimination section 122, the image signal with the sensitivity in response to the photographing state is acquired. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an imaging apparatus that electronically records an image of a subject.

As a technique for reducing noise caused by insufficient exposure, for example, there is a technique disclosed in Japanese Patent Laid-Open No. 10-174000 (Patent Document 1). Japanese Patent Application Laid-Open No. H10-228561 discloses an invention for obtaining an image with appropriate brightness by increasing the gain of AGC (Automatic Gain Control). Further, in the invention disclosed in Patent Document 1, when the subject is dark and proper brightness cannot be obtained even by increasing the AGC gain, the shutter speed is increased and long exposure is performed. We are trying to reduce it.
Further, as a technique for realizing noise reduction using a plurality of images, there is an invention disclosed in, for example, International Publication No. 97/05745 (Patent Document 2). In this patent document 2, by providing a filter circuit for removing random noise in a bit rate encoder that compresses and encodes a plurality of captured video data, in conjunction with increasing the gain of the captured video data, A technique for improving filtering characteristics and reducing random noise is disclosed. In the case of video data such as MPEG, an adaptive filter is added to the compression coding circuit, or a circuit that performs filtering processing and noise removal processing is connected as a preprocessing circuit for the compression coding circuit. It is possible to remove noise.

  Further, for example, Japanese Patent Laid-Open No. 2001-008087 (Patent Document 3) discloses an invention that handles a plurality of images. The invention disclosed in Patent Document 3 realizes a kind of bracket photography in which a plurality of frames are photographed by appropriately changing the shutter speed, focus position, and exposure, depending on the situation at the time of photographing. In addition, by adaptively switching between single shooting and continuous shooting, mistakes during shooting can be reduced without placing a burden on the user.

Furthermore, a technique for generating a high-quality image from a plurality of images has been proposed. For example, Japanese Patent No. 28828138 (Patent Document 4) discloses a method of generating a high-resolution image using a plurality of low-resolution images having positional deviation.
In addition, a technique for generating a high-quality image by pixel mixed readout has been proposed. For example, Japanese Patent Laid-Open No. 2000-253415 (Patent Document 5) discloses that a CCD has an exposure time longer than a predetermined time. A method is disclosed in which charges read from a plurality of pixels are mixed to generate a low noise image.

Japanese Patent Laid-Open No. 10-174000 International Publication No. 97/05745 Pamphlet JP 2001-008087 A Japanese Patent No. 2828138 JP 2000-253415 A

In the invention disclosed in Patent Document 3, it is determined whether or not to perform continuous shooting based on shooting conditions and information on a process for obtaining shooting conditions. However, in the present invention, the number of continuous shots at the time of continuous shooting is fixedly determined with respect to shooting conditions, and optimization of the number of continuous shots is not considered. In addition, when the charges read from the plurality of pixels of the CCD are mixed, the shooting situation is not fully considered. For this reason, it is insufficient to obtain a high-quality image according to the shooting situation.
In addition, according to the inventions disclosed in Patent Document 4 and Patent Document 5, high image quality can be achieved, but the specific processing content of high image quality is sufficiently related to the shooting situation. Since this is not taken into consideration, there is a problem that a high-quality image cannot be obtained according to the shooting situation.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide an imaging apparatus capable of achieving high image quality.

In order to solve the above problems, the present invention employs the following means.
The present invention relates to an imaging apparatus that captures a subject and electronically records an image of the subject, a photographing parameter setting unit that sets photographing parameters that determine photographing conditions, and the number of pixels of the photographed image after pixel mixture reading. Is set to a number corresponding to the shooting parameter, an image pickup means for shooting the subject, and an image signal from the image pickup means so as to be the number of pixels set by the pixel number set means. An image pickup apparatus including a pixel mixture readout unit that performs pixel mixture readout is provided.

According to such a configuration, shooting parameters that determine shooting conditions are set by the shooting parameter setting means, and the number according to the shooting parameters set by the shooting parameter setting means is set by the pixel number setting means. The number of pixels of the captured image after pixel mixture readout is set. On the other hand, a photographed image is acquired by photographing a subject by the imaging means, and pixel mixture readout is performed on the photographed image by the pixel mixture readout means. In this case, the pixel mixture reading means performs pixel mixture reading so that the number of pixels after the pixel mixture reading becomes the number of pixels set by the pixel number setting means. Here, “pixel mixture readout” means that, for example, in the readout of signals from a Bayer array CCD image sensor or the like, by adding and reading out signals of a plurality of pixels of the same color channel, the resolution of the image is reduced. This is a system that can read out image signals with multiple sensitivities.
As described above, according to the imaging apparatus of the present invention, the number of pixels of the captured image after the pixel mixture readout can be set to a number according to the imaging parameter, so that the sensitivity of the captured image changes according to the imaging situation. Can be made. This makes it possible to obtain a low noise and high quality image.

  The imaging parameter setting means corresponds to, for example, the imaging condition setting unit 120 shown in FIG. The pixel number setting means corresponds to, for example, the readout pixel determination unit 122 shown in FIG. The imaging unit corresponds to, for example, the imaging unit 50 illustrated in FIG. 1 and the imaging control unit 110 that controls the imaging unit 50. The imaging unit 50 includes, for example, a lens system 100 including a diaphragm 101, a spectral half mirror system 102, a shutter 103, a low-pass filter 104, a CCD imaging device 105, an A / D conversion circuit 106, and the like. The pixel reading means is provided in the first image processing unit 111 shown in FIG. 1, for example. In addition, the said specific structure is an example and each component of this invention is not limited to the component which concerns on below-mentioned embodiment.

  The imaging apparatus of the present invention includes an exposure amount calculation unit that calculates an exposure amount from the shooting parameter, and the pixel number setting unit is configured to perform the exposure when the exposure amount is smaller than a preset second threshold value. It is preferable to set the number of pixels of the photographed image after the pixel mixture readout according to the amount.

  According to such a configuration, the exposure amount is calculated from the photographing parameter by the exposure amount calculation unit, and when the exposure amount is smaller than the preset second threshold value, the pixel number setting unit calculates the exposure amount. The number of pixels corresponding to the amount is set. This makes it possible to set the number of pixels of the captured image after pixel mixture readout to the number of pixels corresponding to the exposure amount. The exposure amount can be calculated from, for example, the shutter speed and the illuminance of the subject. Specifically, the exposure amount Hm can be calculated by the product of the illuminance of the subject and the shutter speed (= exposure time). The number of pixels is set so as to decrease stepwise as the exposure amount decreases, for example.

  In the imaging apparatus of the present invention, the shooting parameter setting unit includes a sensitivity value setting unit that sets a sensitivity value as the shooting parameter, and the pixel number setting unit includes a second specified value in which the sensitivity value is set in advance. In the above case, it is preferable to set the number of pixels of the photographed image after the pixel mixture reading according to the sensitivity value.

  According to such a configuration, when the sensitivity value set by the sensitivity value setting unit included in the imaging parameter setting unit is equal to or more than the second predetermined value set in advance, the pixel number setting unit performs the sensitivity value setting. The number of pixels corresponding to is set. This makes it possible to set the number of pixels of the captured image after pixel mixture readout to the number of pixels corresponding to the sensitivity value. For example, the number of pixels is set to decrease stepwise as the sensitivity value increases.

  According to the present invention, it is possible to change the sensitivity of a captured image in accordance with the shooting situation, so that it is possible to obtain a high-quality image.

Hereinafter, an embodiment in which an imaging apparatus according to the present invention is applied to an electronic still camera will be described with reference to the drawings.
First, before the specific description of the electronic still camera according to the embodiment of the present invention, the effect of improving the SN ratio by the addition averaging process and the image mixed readout will be described.
First, in the present embodiment, “pixel mixed readout” means, for example, as shown in FIG. 26, in readout of signals from an image sensor (for example, CCD) having a Bayer array, a plurality of adjacent pixels of the same color channel ( For example, it is a signal readout method that can reduce the resolution but can increase the sensitivity by a factor of 4 by mixing signals of 4 pixels).

  FIG. 24 shows an example of the result of random noise reduction using the after-mentioned averaging method (see FIG. 14). In FIG. 24, the vertical axis represents the SN ratio, and the horizontal axis represents the number of images used for the averaging method. FIG. 24 shows the SN ratio in the case where 1 to 100 images are used and averaged. In addition, as a plurality of images, an image to which random noise having a normal distribution probability distribution is added in advance is employed.

  FIG. 25 is a diagram illustrating an example of a result of reducing random noise in the image obtained by performing the pixel mixture reading of FIG. 26 described above. In FIG. 25, the vertical axis indicates the SN ratio, and the horizontal axis indicates σ (sigma: the larger σ is, the more random noise is added to the evaluation image). In FIG. 25, a broken line (before) indicates characteristics before pixel mixture in the evaluation image, and a solid line (after) indicates restoration processing (super solution of a single image) for further increasing the resolution after pixel mixture reading. The characteristic of the image which performed (image) was shown. Note that random noise having a normal distribution probability distribution is added in advance to the evaluation image.

  The S / N ratio shown in FIG. 24 and FIG. 25 is obtained by converting the square root of the value obtained by averaging the average value of all pixels in an image to which random noise has been added in advance and the square of the deviation from each pixel in the result image into decibels. It is. The calculation procedure is shown in the following (1) to (5).

  From the result of the SN ratio in FIG. 24, it can be seen that the use of a plurality of averaging methods has the effect of reducing random noise, and the random noise is reduced as the number of pixels used is increased. Further, from the result of the SN ratio in FIG. 25, it can be seen that there is an effect of reducing random noise by performing pixel reconstruction after performing pixel mixed readout.

[First Embodiment]
Next, an electronic still camera according to the first embodiment of the present invention will be described. FIG. 1 is a block diagram showing a schematic configuration of an electronic still camera according to the first embodiment of the present invention. As shown in FIG. 1, the electronic still camera according to the present embodiment includes, for example, an imaging unit 50, an A / D conversion circuit 106, an AE photosensor 107, an imaging control unit 110, a first image processing unit 111, and an image. Buffer 115, compression unit 116, memory card I / F unit 117, memory card 118, operation display unit 119, imaging condition setting unit (imaging parameter setting unit) 120, continuous shooting determination unit (frame number setting unit) 121, readout A pixel determination unit (pixel number setting means) 122, a switching unit 200, a continuous shooting buffer 201, and a second image processing unit 202 are provided. The imaging unit 50 includes a lens system 100 including a diaphragm 101, a spectral half mirror system 102, a shutter 103, a low-pass filter 104, a CCD imaging device 105, and an AF motor 108.

  The lens system 100, the spectral half mirror system 102, the shutter 103, the low-pass filter 104, and the CCD image sensor 105 that contain the stop 101 are arranged along the optical axis. The lens system 100 is connected to an AF motor 108 for moving a part of the lens system during focusing. The light beam that has passed through the lens system 100 is branched into two hands by the spectral half mirror system 102, one light beam is guided to the CCD image sensor 105 via the shutter 103 and the low-pass filter 104, and the other is the photo for AE. It is guided to the sensor 107. The CCD image sensor 105 according to this embodiment is premised on a single-plate CCD image sensor. The output from the CCD image sensor 105 is input to the A / D conversion circuit 106. An output from the A / D conversion circuit 106 is input to the imaging condition setting unit 120 and the first image processing unit 111.

  The output of the first image processing unit 111 is transferred to the image buffer 115 or the continuous shooting buffer 201 via the switching unit 200. Here, the switching unit 200 transfers the signal from the first image processing unit 111 to the image buffer 115 or the continuous shooting buffer by switching the connection destination based on the signal from the continuous shooting determination unit 121. The output of the image buffer 115 is input to a compression unit 116 that performs compression such as JPEG, and an operation display unit 119 including a liquid crystal display unit and mode setting buttons. The output from the compression unit 116 is input to the removable memory card 118 via the memory card I / F unit 117.

  Signals are input from the AE photosensor 107 and the A / D conversion circuit 106 to the imaging condition setting unit (imaging parameter setting means) 120. The output signal of the imaging condition setting unit 120 is input to the continuous shooting determination unit 121 and the readout pixel determination unit 122. A signal from the continuous shooting determination unit 121 is output to the imaging control unit 110 and the switching unit 200. The output of the readout pixel determination unit 122 is input to the imaging control unit 110.

  The output of the continuous shooting buffer 201 is input to the second image processing unit 202 and the operation display unit 119. The second image processing unit 202 includes a motion estimation unit 202a, a super-resolution processing unit 202b, and an addition average processing unit 202c. The output of the second image processing unit 202 is input to the compression unit 116.

Next, FIG. 2 shows a schematic external configuration diagram of the electronic still camera according to the present embodiment. FIG. 2 is an external configuration diagram of the electronic still camera according to the present embodiment.
As shown in FIG. 2, the electronic still camera of this embodiment includes a camera body Z1, a mode selection dial Z2 provided in the camera body Z1, a release switch Z3, a shooting number mode switch Z4, and a pixel mixture shooting. It has a mode switch Z7 and a liquid crystal display panel Z6.

Next, the operation of the electronic still camera according to the present embodiment having the configuration shown in FIGS. 1 and 2 will be described with reference to the drawings.
FIG. 3 is a flowchart showing an outline of processing performed by the electronic still camera according to the present embodiment.
First, in pre-shooting, shooting parameters are set by shooting parameter setting processing in step B01 in FIG. Next, in step B02, the shooting mode is set by the shooting mode setting process. Here, the electronic still camera according to the present embodiment includes a “normal number shooting mode” in which one image is shot by operating the release switch Z3 (see FIG. 2) once for the main shooting, By operating the release switch Z3 for shooting once, “continuous shooting mode” in which a plurality of images are continuously shot, “pixel mixture reading shooting mode” in which pixel mixture reading is performed, and pixel mixture reading is not performed. There are four shooting modes, “normal readout shooting mode”. In the “pixel mixture readout photographing mode”, as described above, in the readout of signals from the Bayer-array CCD image sensor 105, the signals of a plurality of pixels of the same color channel are added and read out, whereby the image is read out. Although the resolution is lowered, a method of reading out the image signal with multiple times of sensitivity is adopted. On the other hand, in the “normal pixel mode”, a method of reading out signals for each pixel in reading out signals from the Bayer array CCD image sensor 105 without performing pixel mixed reading is adopted.

  In the subsequent step B03, the image is shot in the shooting mode set in step B02. In step B04, the captured image signal is stored in the image buffer 115 or the continuous shooting buffer 201. In step B05, a predetermined process is performed on the image stored in the image buffer 115 or the continuous shooting buffer 201, and the process ends. In step B05, for example, the second image processing unit 202 performs high image quality processing or addition averaging processing on the image stored in the continuous shooting buffer 201.

Next, each processing content described above will be specifically described.
First, in the electronic still camera shown in FIG. 1, when the release switch Z3 (see FIG. 2) is half-pressed by the user or the power switch (not shown) is turned on, the photographing control unit 110 detects the aperture. 101, the shutter 103, and the AF motor 108 are controlled to perform pre-photographing.

  In pre-photographing, a signal from the CCD image sensor 105 is converted into a digital signal by the A / D conversion circuit 106 and transferred to the first image processing unit 111. The first image processing unit 111 performs known white balance, enhancement processing, interpolation processing, and the like on the image signal to generate a three-plate image signal. The three-plate image signal generated in this way is output to the image buffer 115 via the switching unit 200.

  Subsequently, in pre-imaging, the imaging condition setting unit 120 sets imaging conditions (including imaging parameters) for the main imaging using a known technique, and the imaging conditions that have been set are determined by the imaging control unit 110 and the continuous shooting determination unit 121. And transferred to the readout pixel determination unit 122. Here, the imaging condition is, for example, a combination of elements required for shooting such as shutter speed, aperture value, in-focus position, ISO sensitivity, etc., that is, combinations of setting values for shooting parameters. For details of these setting methods, It will be described later.

The continuous shooting determination unit 121 sets a shooting mode related to the number of shooting frames based on the shooting conditions set by the shooting condition setting unit 120, and sets the shooting mode information about the set shooting mode and the switching unit 200. Forward to. On the other hand, the read pixel determination unit 122 sets a shooting mode related to the number of pixels in the pixel mixture reading based on the shooting conditions set by the shooting condition setting unit 120, and sets information on the set shooting mode. Forward to.
The imaging control unit 110 controls the imaging unit 50 based on the imaging conditions set by the imaging condition setting obtaining unit 120 and the imaging modes set by the continuous shooting determination unit 121 and the readout pixel determination unit 122. Specifically, the diaphragm 101, the AF motor 108, the shutter 103, and the like constituting the imaging unit 50 are controlled.

Next, imaging conditions set by the imaging condition setting unit 120 will be described.
Shooting parameters such as a shutter speed and an aperture value relating to the exposure amount are set based on the measurement result of the luminance of the subject input to the AE photosensor 107. The area to be measured for light quantity can be switched by using an aperture function (not shown) disposed in front of the AE photosensor 107. In the above light quantity measurement, spot, center weight, average photometry and the like are measured. In the electronic still camera according to the present embodiment, the combination of the shutter speed and the aperture value can be appropriately selected from “automatic exposure method”, “shutter speed priority method”, and “aperture priority method”. .

  The “automatic exposure method” is a method for automatically setting the shutter speed and the aperture value from a plurality of combinations of predetermined shutter speed and aperture value. The “shutter speed priority method” is a method for automatically setting an aperture value in accordance with the shutter speed set by the user. The “aperture priority method” is a method for automatically setting the shutter speed in accordance with the aperture value set by the user. The setting of the shutter speed and the aperture value will be described later in detail.

  The in-focus position is obtained by converting an output signal from the CCD image sensor 105 into a digital signal by the A / D conversion circuit 106, calculating a luminance signal from the image in a single plate state, and calculating from the edge intensity in the luminance signal. Ask. That is, the focus position of the lens system 100 is changed stepwise by the AF motor 108 to estimate the focus position at which the edge intensity becomes maximum and set it.

  The ISO sensitivity setting varies depending on the sensitivity mode setting in the electronic still camera. In the electronic still camera, when the sensitivity mode is set to the manual sensitivity mode, the setting is performed by the user. When the sensitivity mode of the electronic still camera is the automatic sensitivity mode, the sensitivity is determined based on the light amount of the subject measured by the AE photosensor 107. Specifically, when the amount of light measured by the AE photosensor 107 is small, the ISO sensitivity is set high, and when the amount of light is large, the ISO sensitivity is set low. Note that the ISO sensitivity in the present embodiment is a value representing the degree of electrical amplification (gain-up) for the signal from the CCD image sensor 105, and the greater the value, the higher the degree of electrical amplification.

  Next, shooting parameter setting processing executed by the imaging condition setting unit 120 will be specifically described with reference to FIG. This shooting parameter setting process corresponds to step B01 in the flowchart shown in FIG. In the present embodiment, a case will be described in which exposure conditions, specifically, an aperture value, a shutter speed, and ISO sensitivity are set as shooting parameters.

  First, in step C01 of FIG. 4, it is determined which mode is selected from the three automatic exposure modes of “aperture priority mode”, “shutter speed priority mode”, and “automation mode”. As a result, if the “aperture priority mode” is selected, the process proceeds to step C02, and the value set by the user is set as the aperture value. Subsequently, in step C04, the illuminance of the subject is detected, and in the subsequent step C05, the shutter speed at which appropriate exposure is obtained is calculated from the aperture value set in step C02 and the illuminance detected in step S04. And set it as the shutter speed. Subsequently, ISO sensitivity is set in step C06. The ISO sensitivity is set so that the user can select a mode. Here, when the manual mode is selected, the setting value set by the user is set as the ISO sensitivity. On the other hand, when the automatic mode is selected, an appropriate value is determined from the illuminance of the subject detected in step C04. Determine the sensitivity value. Then, when the ISO sensitivity is set, the process is finished.

  On the other hand, if the “shutter speed priority mode” is selected in step C01 described above, the process proceeds to step C03, and the value set by the user is set as the shutter speed. Subsequently, in step C04, the illuminance of the subject is detected, and in the subsequent step C05, an aperture value for obtaining an appropriate exposure is calculated from the shutter speed set in step C03 and the illuminance detected in step C04. To set the aperture value. Then, in the same manner as described above, the ISO sensitivity is set in step C06, and the process ends.

  If the “automation mode” is selected in step C01 described above, the process proceeds to step C04, and the illuminance of the subject is detected. In step C05, an aperture value and a shutter speed are calculated from the illuminance detected in step C04 so that an appropriate exposure is obtained, and each is set. In step C06, the ISO sensitivity is set, and the process ends.

Next, the shooting mode setting process executed in the continuous shooting determination unit 121 and the read image determination unit 112 after the shooting parameter setting process described above is performed by the imaging condition setting unit 120 will be described with reference to FIG. . This shooting mode setting process corresponds to step B02 in the flowchart shown in FIG.
When the release switch is turned on in step A01 in FIG. 5, the process proceeds to step A02, and it is determined whether the number-of-photographing mode switch Z4 (see FIG. 2) is ON or OFF. As a result, if the number-of-shooting mode switch Z4 is OFF, the process proceeds to step A10, where the number of shots is determined to be one, and in step A11, “normal number shooting mode” is set as the shooting mode. Set and move to step A12.

On the other hand, if the number-of-shots mode switch is ON in step A02, the process proceeds to step A03, and the exposure amount is calculated from the shutter speed and the illuminance of the subject. The exposure amount can be calculated by the product of the illuminance on the imaging surface (CCD imaging device) and the shutter speed (= exposure time). When Hm is the exposure amount (lx × s), E is the illuminance (unit is lx) on the imaging surface (CCD imaging device), and T is the shutter speed (unit is s), the following equation (6) is obtained. E is a value that is inversely proportional to the square of the aperture value (F value) and proportional to the luminance of the subject.
Hm = E × T (6)

  Subsequently, in step A04, it is determined whether or not the exposure amount calculated in step A03 is lower than the first threshold value for setting the number of shots (see FIG. 6). As a result, if the exposure amount is greater than or equal to the first threshold value, the process proceeds to step A08, where the number of shots is determined to be one, and in step A09, “normal number shooting mode” is set as the shooting mode. After setting, the process proceeds to Step A12 described later.

  On the other hand, if the exposure amount is less than the first threshold value in step A04, the process proceeds to step A05, and the number of shots (number of frames) is determined to be a number corresponding to the exposure amount. Thereby, the exposure amount and the number of shots are linked. Regarding the number of shots, an appropriate number is determined in advance, and that number is used. For example, as shown in FIG. 6, the relationship between the exposure amount and the number of shots is set so that the number of shots increases step by step as the exposure amount decreases. A table in which the exposure amount and the number of shots are associated with each other is stored in the electronic still camera, and the number of shots is determined based on this relationship.

Subsequently, in step A06, it is determined whether or not the number of shots determined in step A05 is two or more. As a result, if the number of shots is two or more, the process proceeds to step A07, the shooting mode is set to “continuous shooting mode”, and the process proceeds to step A12.
On the other hand, if the number of shots determined in step A06 is one, the process proceeds to step A09, the shooting mode is set to “normal number shooting mode”, and the process proceeds to step A12. Note that the processing in steps A01 to A11 described above is executed by the continuous shooting determination unit 121 shown in FIG.

  Subsequently, at step A12, it is determined whether the pixel mixture readout photographing mode changeover switch Z7 (see FIG. 2) is ON or OFF. As a result, if the pixel mixture readout photographing mode changeover switch Z7 is OFF, the process proceeds to step A19, where it is determined that the normal readout photographing is performed, and in step 20, “normal readout photographing mode” is set as the photographing mode. Then, in step A21, the set shooting modes, the number of shots, and the number of pixels after the pixel mixture reading are displayed on the liquid crystal display panel Z6 (see FIG. 2), and this processing is terminated.

  On the other hand, if the pixel mixture readout photographing mode changeover switch Z7 (see FIG. 2) is ON in step A12, the process proceeds to step A13, where the exposure amount is determined from the shutter speed, aperture value, and subject brightness. Is calculated. Note that the exposure amount calculated in step A03 described above may be used instead of the exposure amount calculated here.

  Subsequently, it is determined whether or not the exposure amount calculated in step A13 is lower than a second threshold value for determining the reading method (see FIG. 7). As a result, if the exposure amount is greater than or equal to the second threshold value, the process proceeds to step A17, where it is determined that the image is normally read, and in step A18, after the shooting mode is set to “normal read shooting mode”, Subsequently, in step A21, each set shooting mode, the number of shots, and the number of pixels after pixel mixture reading are displayed on the liquid crystal display panel Z6 (see FIG. 2), and this processing is terminated.

  On the other hand, when the exposure amount is less than the second threshold value in step A14 described above, the process proceeds to step A15, and the number of pixels after the pixel mixture reading according to the exposure amount is determined. Thereby, the exposure amount, the number of shots, and the number of pixels after pixel mixture reading are linked. Here, with respect to the number of pixels after pixel mixture readout, an appropriate number of pixels after readout is determined in advance, and the number of pixels may be used.

  For example, as shown in FIG. 7, the relationship between the exposure amount and the number of pixels after pixel mixture readout is set so that the number of pixels after pixel mixture readout gradually decreases as the exposure amount decreases. Such a table in which the exposure amount is associated with the number of pixels after pixel mixture reading is stored in the electronic still camera, and the number of pixels after pixel mixture reading is determined based on this relationship.

When the number of pixels corresponding to the exposure amount is determined in this way, the shooting mode is set to “pixel mixture readout shooting mode” in the subsequent step A16, and each shooting mode set in the subsequent step A21. Then, the number of shots and the number of pixels after the pixel mixture reading are displayed on the liquid crystal display panel Z6, and this processing ends.
Note that the processing of step A12 to step A20 described above is executed in the readout pixel determination unit 122 shown in FIG.

  Note that the above-described shooting mode setting process can be performed not only according to the exposure amount as described above but also according to the ISO sensitivity value. Hereinafter, the procedure of the shooting mode setting process according to the ISO sensitivity will be described with reference to FIG. In FIG. 8, the same processing numbers as those in FIG. 5 are denoted by the same step numbers, and the description thereof is omitted.

  In this process, steps A30 and A31 of FIG. 8 are employed instead of steps A03 to A05 shown in FIG. Specifically, if it is determined in step A02 in FIG. 8 that the number-of-shooting mode switch is ON, in step A30, the ISO sensitivity is equal to or higher than the first specified value for setting the number of shots (see FIG. 9). It is determined whether or not. As a result, if the ISO sensitivity is less than the first specified value, the process proceeds to step A08, where the number of shots is determined to be one, and in step A09, “normal number shooting mode” is set as the shooting mode. After setting, the process proceeds to Step A12 described later.

  On the other hand, if the ISO sensitivity is equal to or higher than the first specified value in step A30, the process proceeds to step A31, and the number of shots corresponding to the ISO sensitivity is determined. Thereby, the ISO sensitivity and the number of shots are linked. Regarding the number of shots, an appropriate number is determined in advance, and that number is used. For example, as shown in FIG. 9, the relationship between the ISO sensitivity and the number of shots is set so that the number of shots increases stepwise as the ISO sensitivity increases. Such a table in which the ISO sensitivity is associated with the number of shots is stored in the electronic still camera, and the number of shots is determined based on this relationship.

  Further, in this process, steps A32 and A33 of FIG. 8 are employed instead of steps A13 to A15 shown in FIG. Specifically, when it is determined in step A12 in FIG. 8 that the pixel mixture readout photographing mode switch is ON, in step A32, the ISO sensitivity is equal to or higher than the second specified value for determining the readout method (see FIG. 10). It is determined whether or not. As a result, if the ISO sensitivity is less than the second specified value, the process proceeds to step A17, where it is determined that the image is normally read, and in step A18, “normal read image capturing mode” is set as the image capturing mode. .

  On the other hand, when the ISO sensitivity is equal to or higher than the second specified value in step A32, the process proceeds to step A33, and the number of pixels after the pixel mixture reading according to the ISO sensitivity is determined. Thereby, the ISO sensitivity and the number of pixels after the pixel mixture reading are linked. Here, with respect to the number of pixels after pixel mixture readout, an appropriate number of pixels after readout is determined in advance, and the number of pixels may be used.

  The relationship between the ISO sensitivity and the number of pixels after pixel mixture readout is set so that the number of pixels after pixel mixture readout gradually decreases as the ISO sensitivity increases as shown in FIG. Such a table in which the ISO sensitivity is associated with the number of pixels after pixel mixture readout is stored in the electronic still camera, and the number of pixels after pixel mixture readout is determined based on this relationship.

  Next, when the release switch Z3 (see FIG. 2) is completely pressed by the user in the pre-shooting state in which the shooting conditions and the shooting modes are set as described above, the main shooting is performed. This actual shooting is controlled by the imaging control unit 110 based on the shooting parameters set by the shooting condition setting unit 120 shown in FIG. 1 and the shooting modes set by the continuous shooting determination unit 121 and the readout pixel determination unit 122. Originally done. As a result, the subject is photographed at the set shutter speed, aperture value, and ISO sensitivity, and the image signal of the photographed image is sent to the image buffer 115 or the continuous shooting buffer 201 via the first image processing unit 111. Is output. At this time, if the shooting mode is set to “normal number shooting mode”, an image signal of one shot image is supplied to the first image processing unit 111. On the other hand, when the shooting mode is set to “continuous shooting mode”, the image signal of the number of shot images set by the continuous shooting determination unit 121 is supplied to the continuous shooting buffer 201.

  The image signal supplied to the image buffer 115 is output to the compression unit 116, and after being subjected to a compression process such as known JPEG, a memory card 118 that can be attached and detached via the memory card I / F unit 117. Etc. are saved.

On the other hand, the image signal input to the continuous shooting buffer 201 is supplied to the second image processing unit 202, where high image quality processing is performed.
Hereinafter, the procedure of high image quality processing performed by the second image processing unit 202, specifically, motion estimation and addition averaging processing will be described with reference to FIGS.
The motion estimation unit 202a in the second image processing unit 202 uses the image signals of a plurality of images shot in the continuous shooting mode and input to the continuous shooting buffer 201, and uses the image signals between the frames in each image (frame). Perform motion estimation. FIG. 11 shows an example of a motion estimation processing procedure.

  First, in step S1 of FIG. 11, one image serving as a reference for motion estimation is read. In step S2, the reference image is deformed by a plurality of movements. In step S3, one reference image for estimating the motion between the standard images is read. In step S4, a similarity value between each image sequence obtained by deforming a plurality of standard images and the reference image is calculated. In step S5, a discrete similarity map as shown in FIG. 12 is created using the relationship between the deformed motion parameter and the calculated similarity value. In step S6, by complementing the discrete similarity map created in step S5, the extreme value of the similarity map is searched to obtain the extreme value of the similarity map. Here, the deformation motion determined by the extreme value becomes the motion estimation value. Examples of methods for searching for extreme values in the similarity map include parabolic fitting and spline interpolation.

  Subsequently, in step S7, it is determined whether or not motion estimation has been performed on all reference images. As a result, when motion estimation is not performed for all reference images, the process proceeds to step S8, the frame number of the reference image is incremented by 1, and the process returns to step S3 described above. As a result, the processes in steps S3 to S8 are repeated until motion estimation is performed on all reference images. If it is determined in step S7 that motion estimation has been performed on all target reference images, the process ends.

Next, an example of a specific method for performing motion estimation by parabolic fitting will be described with reference to FIG. In FIG. 12, the vertical axis represents the square deviation, and the horizontal axis represents the deformation motion parameter. In this figure, it can be said that the smaller the square deviation on the vertical axis is, the higher the similarity is.
Here, the plurality of deformations of the reference image in step S2 in FIG. 11 are, for example, 19 reference images with the motion parameter of ± 1 pixel in the horizontal, vertical, and rotation directions (8 out of 27 patterns are the same). (Deformation pattern). Further, in step S5 of FIG. 11, when considering the motion parameter of the horizontal axis shown in FIG. 12 as a combination of the horizontal direction, the vertical direction, and the rotation direction, (−1, +1, −1) Each discrete similarity value of (-1, +1, 0) (-1, +1, +1) is plotted. If each deformation direction is considered to be different, the negative direction is (−1), (0), (+1), and the horizontal direction, the vertical direction, and the rotation direction are plotted separately.

  FIG. 13 shows an approximation of the reference image to the standard image. Each reference image is a value obtained by inverting the sign of the motion estimation value, and approximates the reference image by deforming the image. After that, when the addition averaging method shown in FIG. 14 is used, an image is generated as a new pixel value by adding and averaging the corresponding pixels of all the frames in all images approximate to the reference image at once. On the other hand, when the addition averaging method shown in FIG. 15 is used, an addition average image is generated using a value obtained by adding and averaging the corresponding pixel positions in all images approximate to the reference image for each frame. Note that the addition averaging method shown in FIG. 14 may have a tendency that noise is less likely to remain than the addition averaging method shown in FIG.

  Then, the addition average image generated in this way is output from the second image processing unit 202 to the compression unit 116, and after being subjected to compression processing such as known JPEG, the memory card I / F unit 117. And stored in the memory card 118.

  As described above, according to the electronic still camera according to the present embodiment, the continuous shooting determination unit 121 performs the main shooting based on the shooting conditions (shooting parameters) set by the shooting condition setting unit 120. The number of captured images at the time is set, and the number of pixels after the pixel mixture readout is performed by the readout pixel determination unit 122 is set. Thus, since the number of frames of the captured image and the number of pixels in the pixel mixture readout are set according to the imaging conditions, by processing the image quality of the captured image captured based on these settings, It is possible to acquire a high-quality image with reduced noise. The point that noise is reduced by pixel mixture readout and the point that a low noise image can be obtained by generating an image by averaging the plurality of captured images are shown in FIGS. 24 and 25 described above. That's right.

In the first embodiment described above, after the image quality enhancement processing is performed by the second image processing unit 202, the compression is performed by the compression unit 116. However, this is an example. Without being output to the memory card I / F unit 117 or the like, the above-described image quality improvement processing can be performed by an external device having the same function as the second image processing unit 202. As another example, the second image processing unit 202 does not compress the image after processing, but based on the previously compressed image, the second image processing unit 202 performs high image quality processing. It is good also as a structure which is performed.
In addition, the image processing performed in the second image processing unit 202 is not limited to the above-described processing procedure, and it is possible to adopt a known high image quality processing or the like.

In the above embodiment, the exposure amount is calculated in the shooting mode setting process, and the number of shot images is set based on the exposure amount. However, the shot image can be set as follows. is there.
For example, in the shooting parameter setting process performed by the shooting condition setting unit 120, as shown in FIG. 4, in the aperture priority mode, as shown in Step C02 of FIG. A shutter speed is set based on the illuminance of the subject (step C05).
At this time, if the set shutter speed is slower than the predetermined speed, camera shake is likely to occur. Therefore, the shutter speed is reset to a predetermined speed or higher, and the reset shutter speed is set to the reset speed. The number of captured images is set to a plurality of corresponding frame numbers. At this time, the faster the reset shutter speed, the larger the number of captured images.
On the other hand, in the shutter speed priority mode, as shown in step C03 in FIG. 4, the aperture value is set based on the shutter speed set by the user and the illuminance of the subject (step C05). In this case, when the aperture is set to the open end state, it can be estimated that the exposure amount is small. Therefore, the number of captured images is set to a plurality of frames according to the shutter speed set by the user. At this time, the faster the shutter speed set by the user, the greater the number of captured images.

  As described above, the number of captured images is set to a plurality of frames according to the shutter speed reset in the aperture priority mode, or the plurality of frames is captured according to the shutter speed set in the shutter speed priority mode. When the number of images is set, it is impossible to adjust the number according to the exposure amount. However, the number of shots can be set without making such adjustment, and it is necessary to calculate the exposure amount. There is an advantage that processing can be reduced in that there is no point.

[Second Embodiment]
Next, an imaging device according to a second embodiment of the present invention will be described with reference to the drawings. In the first embodiment described above, the setting of the number of frames by the continuous shooting determination unit 121 and the setting of the number of read pixels by the read pixel determination unit 122 are performed in the shooting mode setting process. In the imaging apparatus in FIG. 5, only the frame number setting by the continuous shooting determination unit 121 is performed, and the pixel number setting process by the readout pixel determination unit 122 is omitted. This simplifies the processing, reduces the processing load, and shortens the processing time.
FIGS. 16 and 17 show an example of the shooting mode setting process executed by the continuous shooting determination unit 121 according to the present embodiment. In FIG. 16, the same processing steps as those in FIG. 5 are given the same step numbers. In FIG. 17, the same processing steps as those in FIG. 8 are given the same step numbers.

[Third Embodiment]
Next, an imaging device according to a third embodiment of the present invention will be described with reference to the drawings. In the first embodiment described above, the setting of the number of frames by the continuous shooting determination unit 121 and the setting of the number of read pixels by the read pixel determination unit 122 are performed in the shooting mode setting process. In the imaging apparatus in FIG. 5, only the number of pixels is set by the readout pixel determination unit 122, and the setting of the number of frames by the processing continuous shooting determination unit 121 is omitted. This simplifies the processing, reduces the processing load, and shortens the processing time.
FIGS. 18 and 19 show an example of the shooting mode setting process executed by the readout pixel determination unit 122 according to the present embodiment. In FIG. 18, the same processing steps as those in FIG. 5 are given the same step numbers. In FIG. 19, the same processing steps as those in FIG. 8 are denoted by the same step numbers.

[Fourth Embodiment]
Next, an imaging device according to a fourth embodiment of the present invention will be described with reference to the drawings. In the first embodiment described above, in the shooting mode setting process, first, the number of frames is set by the continuous shooting determination unit 121, and then the number of read pixels is set by the read pixel determination unit 122. In the imaging apparatus according to the present embodiment, first, the readout pixels are set by the readout pixel determination unit 122, and then the number of frames is set by the continuous shooting determination unit 121.
20 and 21 show an example of the shooting mode setting process executed by the continuous shooting determination unit 121 and the readout pixel determination unit 122 according to this embodiment. In FIG. 20, the same processing steps as those in FIG. 5 are given the same step numbers. In FIG. 21, the same processing steps as those in FIG. 8 are given the same step numbers.

[Fifth Embodiment]
Next, an imaging device according to a fifth embodiment of the present invention will be described with reference to the drawings. The imaging apparatus according to the present embodiment has substantially the same configuration as the imaging apparatus according to the first embodiment described above, but the image quality enhancement process executed by the second image processing unit 202 is different.
Hereinafter, referring to the drawing, the image quality enhancement processing procedure according to the present embodiment, specifically, the image quality enhancement processing (super-resolution) for restoring a high-resolution image using a plurality of images will be described. I will explain.

  First, as shown in FIG. 22, in step S <b> 11, a plurality of low resolution images n (n ≧ 1) for use in high resolution image estimation are read from the continuous shooting buffer 201. Subsequently, in S12, an arbitrary one of the plurality of low-resolution images read in step S11 is assumed to be a target frame, and an initial high-resolution image is created by performing complement processing. This step can be omitted in some cases. Subsequently, in step S13, a target frame and other frames obtained in advance by some motion estimation method (for example, a method for obtaining a motion estimation value by the motion estimation unit 202a as described in the first embodiment above). The positional relationship between the images is clarified by the motion between the images.

In subsequent step S14, an optical transfer function (OTF: Optical)
A point spreading function (PSF) in consideration of imaging characteristics such as transfer function and CCD aperture is obtained. The PSF uses, for example, a Gauss function. In step S15, the evaluation function f (z) is minimized based on the information obtained in steps S13 and S14. However, f (z) becomes a form like the following (7) Formula.

In the above equation (7), y is a low-resolution image, z is a high-resolution image, A is an inter-image motion (for example, a motion estimation value obtained by the motion estimation unit 202a in FIG. 1), an imaging system including PSF and the like. It is an image transformation matrix that represents. g (z) includes a constraint term in consideration of image smoothness and color correlation. λ is a weighting factor. For example, the steepest descent method is used to minimize the evaluation function.
Subsequently, in step S16, it is determined whether or not f (z) obtained in step S15 is minimized. As a result, if not yet minimized, the process proceeds to step S17, the high-resolution image z is updated, the process returns to step S15, and steps S15 to S17 are repeated until the high-resolution image z is minimized. Do. In step S16, when f (z) obtained in step S15 is minimized, the processing is terminated and a high-resolution image z is obtained.

Next, FIG. 23 shows an example of a hardware configuration for executing the processing procedure shown in FIG.
As shown in FIG. 23, the high-resolution image estimation processing unit includes an initial image storage unit 1201, a convolution integration unit 1202, a PSF data holding unit 1203, an image comparison unit 1204, a multiplication unit 1205, a pasting addition unit 1206, and an accumulation addition. A unit 1207, an update image generation unit 1208, an image storage unit 1209, an iterative calculation determination unit 1210, and an iterative determination value holding unit 1211.

  In such a configuration, the interpolated enlarged image from the continuous shooting buffer 201 (see FIG. 1) is given to the initial image storage unit 1201. This image is the initial image. This initial image is given to the convolution integration unit 1202 and is convolved with the PSF data given from the PSF data holding unit 1203. The PSF data here is given in consideration of the motion of each frame. The initial image data is simultaneously sent to the image storage unit 1209 and stored therein. The image data subjected to the convolution operation by the convolution integration unit 1202 is sent to the image comparison unit 1204 and based on the motion (motion estimation value) for each frame obtained by the motion estimation unit 202a (see FIG. 1). It is compared with the captured image provided from the continuous shooting buffer 201 (see FIG. 1) at an appropriate coordinate position.

  The compared residuals are sent to the multiplier 1205 and multiplied by the value for each pixel of the PSF data given from the PSF data holding unit 1203. The calculation result is sent to the pasting and adding unit 1206 and placed at the corresponding coordinate position. Here, the image data from the multiplying unit 1205 is shifted little by little while having an overlap, so the overlapping portions are added. When the addition of data for one photographed image is completed, the data is sent to the accumulation / addition unit 1207. The accumulation / addition unit 1207 accumulates data sequentially sent until processing for the number of frames is completed, and sequentially adds image data for each frame in accordance with the estimated motion.

  The added image data is sent to the updated image generation unit 1208. At the same time, the image data stored in the image storage unit 1209 is given to the update image generation unit 1208, and the two image data are weighted and added to generate update image data. The generated update image data is given to the iterative calculation determination unit 1210, and it is determined whether or not to repeat the calculation based on the iteration determination value given from the iteration determination value holding unit 1211. When repeating the calculation, the data is sent to the convolution integrator 1202 and the above-described series of processing is repeated. On the other hand, if not repeated, the generated image data is output as a high resolution image. By performing the above series of processing, the image output from the iterative calculation determination unit 1210 has a higher resolution than the captured image.

  Further, the PSF data held by the PSF data holding unit 1203 needs to be calculated at an appropriate coordinate position at the time of convolution integration, so that a motion for each frame is given from the motion estimation unit. .

  Note that the imaging apparatus according to the first to fourth embodiments described above is premised on processing by hardware, but is not necessarily limited to such a configuration. For example, a configuration is possible in which the signal from the CCD 105 is processed as raw data as unprocessed by software separately.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes design changes and the like within a scope not departing from the gist of the present invention.

It is a block diagram showing a schematic structure of an electronic still camera concerning a 1st embodiment of the present invention. 1 is an external configuration diagram of an electronic still camera according to a first embodiment of the present invention. It is the flowchart which showed the outline of the process performed with the electronic still camera which concerns on the 1st Embodiment of this invention. 4 is a flowchart showing a flow of imaging parameter setting processing shown in FIG. 3. 4 is a flowchart illustrating a flow of a shooting mode setting process illustrated in FIG. 3. It is a figure which shows an example of the table with which the exposure amount and the imaging | photography number of sheets were matched. It is a figure which shows an example of the table with which the exposure amount and the pixel number after pixel mixing reading were matched. It is a figure which shows the other example of the flowchart shown in FIG. It is a figure which shows an example of the table with which ISO sensitivity and the imaging | photography number of sheets were matched. It is a figure which shows an example of the table with which the ISO sensitivity and the pixel number after pixel mixture reading were matched. It is a flowchart which shows an example of the process sequence of the motion estimation performed with the electronic still camera which concerns on the 1st Embodiment of this invention. It is a figure which shows the conceptual diagram of the optimal similarity estimation in motion estimation. It is an approximate figure to a standard picture in reference picture modification using a motion estimation value. It is explanatory drawing for demonstrating an example of the addition average method of a reference | standard image and an approximate image. It is explanatory drawing for demonstrating the other example of the addition average method of a reference | standard image and an approximate image. It is a flowchart which shows an example of the imaging | photography mode setting process implemented with the electronic still camera which concerns on the 2nd Embodiment of this invention. It is a figure which shows the other example of the flowchart shown in FIG. It is a flowchart which shows an example of the imaging | photography mode setting process implemented with the electronic still camera which concerns on the 3rd Embodiment of this invention. It is a figure which shows the other example of the flowchart shown in FIG. It is a flowchart which shows an example of the imaging | photography mode setting process implemented with the electronic still camera which concerns on the 4th Embodiment of this invention. It is a figure which shows the other example of the flowchart shown in FIG. It is a flowchart which shows the process sequence of the high resolution image estimation implemented with the electronic still camera which concerns on the 5th Embodiment of this invention. An example of a hardware configuration for realizing the processing procedure shown in FIG. 22 will be described. It is the figure which showed the S / N ratio experiment result for every several sheets by the addition average method. It is the figure which showed the S / N ratio experiment result in the image restoration by pixel mixing readout. It is explanatory drawing for demonstrating the pixel mixing readout in 4 pixels adjacent to the same color channel.

Explanation of symbols

50 Imaging unit 101 Aperture 100 Lens system 102 Spectral half mirror system 103 Shutter 104 Low-pass filter 105 CCD imaging device 106 A / D conversion circuit 107 AE photo sensor 108 AF motor 110 Imaging control unit 111 First image processing unit 115 Image buffer 116 Compression Unit 117 Memory Card I / F Unit 118 Memory Card 119 Operation Display Unit 120 Imaging Condition Setting Unit 121 Continuous Shooting Determination Unit 122 Read Pixel Determination Unit 200 Switching Unit 201 Continuous Shooting Buffer 202 Second Image Processing Unit

Claims (3)

In an imaging device that photographs a subject and electronically records an image of the subject,
Shooting parameter setting means for setting shooting parameters for determining shooting conditions;
A pixel number setting means for setting the number of pixels of the photographed image after the pixel mixture readout to a number according to the photographing parameter;
Imaging means for photographing the subject;
An image pickup apparatus comprising: a pixel mixture readout unit that performs pixel mixture readout on an image signal from the image pickup unit so that the number of pixels set by the pixel number setting unit is obtained. Exposure amount calculating means for calculating an exposure amount from the photographing parameters;
The pixel number setting means sets the number of pixels of the photographed image after the pixel mixture reading according to the exposure amount when the exposure amount is smaller than a preset second threshold value. The imaging device described. The shooting parameter setting means includes sensitivity value setting means for setting a sensitivity value as the shooting parameter,
The pixel number setting unit sets the number of pixels of the photographed image after the pixel mixture reading in accordance with the sensitivity value when the sensitivity value is equal to or greater than a second predetermined value set in advance. The imaging device described.

Images