JP2009164723A - Imaging apparatus and imaging control method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus achieving high-resolution photographing, wide dynamic range and high quality image and reducing the load of a signal processing. <P>SOLUTION: A first CCD 18 with a color filter 36 and a second CCD 21 with no color filter are simultaneously exposed to light. The second CCD 21 is exposed to light with sensitivity lower than the first CCD 18. A digital signal processing circuit 25 calculates a brightness signal Y1 of the first CCD 18 and a brightness signal Y2 of the second CCD 21 from respective image data after gamma conversion by different gamma curves. The brightness signals Y1 and Y2 are mixed at a predetermined ratio to calculate a brightness signal Y3, further calculating color-difference signals Cb2 and Cr2 from the brightness signal Y3. A wide dynamic range image is generated from the brightness signal Y3 and the color-difference signals Cb2 and Cr2. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image pickup apparatus and an image pickup control method, and more particularly to an image pickup apparatus and an image pickup control method for obtaining an image with a wide dynamic range from image pickup data simultaneously picked up by two image pickup elements.

  In today's camera photographing processing, an image with a high resolution and a wide dynamic range is required. As a general technique for obtaining such an image, a technique is known in which a plurality of images having different exposure amounts are photographed in the same scene, and these multiple images are combined to obtain a wide dynamic range image.

As another technique, Patent Document 1 discloses a device that provides a plurality of optical systems for one imaging surface, changes sensitivity for each imaging area, and performs a synthesis process to perform a wide dynamic range process. Is described. According to this technique, it is possible to realize an imaging system with a wide dynamic range, and it is possible to simultaneously perform photographing with a plurality of optical systems, and thus it is possible to deal with a moving subject.
JP 2002-171430 A

  However, the method of photographing a plurality of images with different exposure amounts in the same scene has a drawback that a large amount of memory area is required and processing time is required after the photographing. In addition, the apparatus described in Patent Document 1 has the disadvantage that the resolution is impaired and the phase difference of each area data needs to be adjusted. In addition, there is a problem that the influence of false colors generated during the color interpolation process increases due to the synthesis process.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide an imaging apparatus and an imaging control method that realize high-resolution, wide dynamic range, high-quality imaging and reduce the load caused by signal processing. And

  In order to achieve the above object, the imaging apparatus according to claim 1, a spectroscopic unit that splits incident light from a subject incident on an imaging optical system into first subject light and second subject light, and the first A color imaging device that receives a subject light of the first color, a first imaging means that acquires a first image signal of three primary colors, and a monochrome imaging device that receives the second subject light, Second imaging means for obtaining a second image signal having a wider dynamic range than the image signal of the first image signal, means for calculating a first luminance signal for each pixel based on the first image signal, and the second The third luminance signal having a wide dynamic range is calculated by mixing the first luminance information and the second luminance signal with the means for calculating the second luminance signal for each pixel based on the image signal of Means, and the first image signal and the third luminance signal Zui characterized by comprising a means for calculating a color difference signal for each pixel.

  Thereby, high-resolution and wide dynamic range shooting can be realized, and the load caused by signal processing can be reduced.

  According to a second aspect of the present invention, in the imaging apparatus according to the first aspect, the spectroscopic unit performs the spectroscopic analysis so that the amount of light of the second subject light is smaller than the amount of light of the first subject light. And

  Thereby, an image signal with a wide dynamic range can be acquired.

  According to a third aspect of the present invention, in the imaging apparatus according to the first aspect, the first imaging unit is set to have higher imaging sensitivity than the second imaging unit.

  Thereby, an image signal with a wide dynamic range can be acquired.

  According to a fourth aspect of the present invention, in the image pickup apparatus according to the first aspect, the color image pickup element of the first image pickup unit includes a high sensitivity element, and the monochrome image pickup element of the second image pickup unit includes a low sensitivity element. It is characterized by having.

  Thereby, an image signal with a wide dynamic range can be acquired.

  5. The image pickup apparatus according to claim 1, further comprising gradation conversion processing means for performing gradation conversion processing different between the first image data and the second image data. The means for calculating the first luminance signal for each pixel based on the first image signal includes the first luminance signal for each pixel based on the first image signal subjected to gradation conversion processing by the gradation conversion processing means. Means for calculating one luminance signal and calculating a second luminance signal for each pixel based on the second image signal; the second image signal that has been subjected to gradation conversion processing by the gradation conversion processing means; The second luminance signal for each pixel is calculated based on the above.

  As a result, it is possible to acquire a wide dynamic range image with appropriate gradation.

  6. The image pickup apparatus according to claim 1, further comprising an automatic focusing unit that moves a focus lens to a focusing position based on the second image data. And

  Thereby, it is possible to focus accurately in a dark scene or the like.

  In order to achieve the above object, an imaging control method according to claim 7, wherein a spectroscopic step of dividing incident light from a subject incident on an imaging optical system into first subject light and second subject light; The first image signal is obtained by a first imaging step of acquiring a first image signal of three primary colors by a color imaging device that receives one subject light, and by a monochrome imaging device that receives the second subject light. A second imaging step of acquiring a second image signal having a wider dynamic range, a step of calculating a first luminance signal for each pixel based on the first image signal, and the second image signal Calculating a second luminance signal for each pixel based on the above, and calculating a third luminance signal having a wide dynamic range by mixing the first luminance information and the second luminance signal; The first image signal and the three luminance signals; Based characterized by comprising a step of calculating a color difference signal for each pixel.

  Thereby, high-resolution and wide dynamic range shooting can be realized, and the load caused by signal processing can be reduced.

  According to the present invention, it is possible to provide an imaging apparatus and an imaging control method that realize high-resolution, wide dynamic range, high-quality imaging and reduce the load caused by signal processing.

  The best mode for carrying out the present invention will be described below with reference to the accompanying drawings.

<First Embodiment>
FIG. 1 is a block diagram showing an electrical configuration of a digital camera 10 to which the present invention is applied.

  As shown in the figure, the digital camera 10 of the present embodiment includes a CPU 11, an operation unit 12, a zoom lens motor driver 13, a focus lens motor driver 14, a zoom lens 15, a focus lens 16, a spectroscope 17, 1 CCD 18, first analog signal processor 19, first A / D converter 20, second CCD 21, second analog signal processor 22, second A / D converter 23, image input controller 24, image signal processing circuit 25, compression / decompression processing circuit 26, display circuit 27, LCD monitor 28, bus 29, AE detection circuit 30, AF detection circuit 31, ROM 32, memory 33, media controller 34, recording medium 35, etc. Configured.

  Each unit operates under the control of the CPU 11, and the CPU 11 controls each unit of the digital camera 10 by executing a predetermined control program based on an input from the operation unit 12.

  In addition to the control program executed by the CPU 11, various data necessary for control are recorded in the ROM 32. The CPU 11 controls each unit of the digital camera 10 by reading the control program recorded in the ROM 32 into the memory 33 and sequentially executing the program.

  The memory 33 is used as a program execution processing area, a temporary storage area for image data, and various work areas.

  In addition to the release button, the operation unit 12 includes general camera operation means such as a power switch and a shooting mode dial, and outputs a signal corresponding to the operation to the CPU 11. A release button (not shown) includes a switch S1 that is turned on when half-pressed to prepare for photographing such as focus lock and photometry, and a switch S2 that is turned on when fully pressed to capture an image.

  The zoom lens 15 is driven by the zoom lens motor driver 13 to move back and forth on the optical axis of the focus lens 16. The CPU 11 controls the movement of the zoom lens 15 via the zoom lens motor driver 13 to perform zooming.

  The focus lens 16 is driven by the focus lens motor driver 14 to move back and forth on the optical axis of the zoom lens 15. The CPU 11 controls the movement of the focus lens 16 via the focus lens motor driver 14 to perform focusing.

  The spectroscopic unit 17 is disposed behind the focus lens 16 and is an optical member such as a prism that divides subject light transmitted through the focus lens 16 into two equal parts.

  The first CCD 18 receives one of the subject lights divided by the spectroscopic unit 17. In the first CCD 18, one element corresponds to one pixel of the screen. As shown in FIG. 2A, a large number of light receiving elements are two-dimensionally arranged on the light receiving surface, and red (R), green (G), and blue (B) corresponding to each light receiving element. Primary color filters 36 are arranged in a predetermined arrangement structure.

  The other subject light divided by the spectroscopic unit 17 is received by the second CCD 21. The second CCD 21 has the same number of elements and arrangement as the first CCD 18 and can capture the same subject image as the first CCD 18, but as shown in FIG. The difference is that no color filter is arranged on the surface of the light receiving element. The second CCD 21 is set to have a lower element sensitivity than the first CCD 18 in order to cope with a wide dynamic range process. FIG. 3 shows the relationship between the input light amount and the output value of the first CCD 18 and the second CCD 21. In the example of FIG. 3, the second CCD 21 has half the sensitivity of the first CCD 18.

  Thus, the subject light divided by the spectroscopic unit 17 forms an image on the light receiving surfaces of the first CCD 18 and the second CCD 21 and is converted into an electric signal by each light receiving element.

  Note that the first CCD 18 and the second CCD 21 may use other types of imaging elements such as MOS type instead of the CCD.

  The output of the image signal is started when the digital camera 10 is set to the shooting mode. That is, when the digital camera 10 is set to the photographing mode, output of an image signal is started to display a through image on the LCD monitor 28. The output of the image signal for the through image is temporarily stopped when the instruction for the main photographing is given, and is started again when the main photographing is finished.

  The image signal output from the first CCD 18 is an analog signal, and this analog image signal is taken into the first analog signal processing unit 19. The first analog signal processing unit 19 includes a correlated double sampling circuit (CDS) and an automatic gain control circuit (AGC). The CDS removes noise contained in the image signal, and the AGC amplifies the noise-removed image signal with a predetermined gain. The analog image signal subjected to the required signal processing by the first analog signal processing unit 19 is taken into the first A / D converter 20.

  The first A / D converter 20 converts the captured analog image signal into a digital image signal having a gradation width of a predetermined bit. This image signal is so-called RAW data, and has a gradation value indicating the density of R, G, and B for each pixel.

  The second analog signal processing unit 22 and the second A / D converter 23 have the same internal configuration as the first analog signal processing unit 19 and the first A / D converter 20. The analog image signal output from the second CCD 21 is subjected to necessary signal processing by the second analog signal processing unit 22 and then has a predetermined bit gradation width in the second A / D converter 23. Converted into a digital image signal. Since the second CCD 21 is not provided with a color filter, the image signal output from the second CCD 21 has no R, G, B information, but luminance information can be sampled in all pixels. Highly accurate and wide dynamic luminance information can be obtained.

  The image input controller 24 has a built-in line buffer having a predetermined capacity, and accumulates image signals for one frame output from the first A / D converter 20 and the second A / D converter 23. The image signal for one frame accumulated in the image input controller 24 is stored in the memory 33 via the bus 29. The memory 33 can store image signals for two frames or more, and can store both image data photographed simultaneously by the first CCD 18 and the second CCD 21.

  In addition to the CPU 11, memory 33, and image input controller 24, the digital signal processing circuit 25, compression / decompression processing circuit 26, display circuit 27, media controller 34, AE detection circuit 30, AF detection circuit 31, etc. are connected to the bus 29. These are configured to be able to transmit and receive information to and from each other via a bus 29.

  The AE detection circuit 30 takes in R, G, and B image signals, which are output signals of the first A / D converter 20 stored in the memory 33 via the image input controller 24, in accordance with a command from the CPU 11, and outputs AE. The integrated value required for control is calculated. The CPU 11 calculates a luminance value from the integrated value and obtains an exposure value from the luminance value. Further, the aperture value and the shutter speed are determined from the exposure value according to a predetermined program diagram.

  The AF detection circuit 31 takes in the output signal of the second A / D converter 23 stored in the memory 33 via the image input controller 24 in accordance with a command from the CPU 11 and performs focus evaluation necessary for AF (Automatic Focus) control. Calculate the value. The AF detection circuit 31 includes a focus area extraction unit that extracts a signal within a predetermined focus area set on the screen, and an integration unit that integrates absolute value data within the focus area. The absolute value data in the focus area is output to the CPU 11 as a focus evaluation value. During the AF control, the CPU 11 searches for a position where the focus evaluation value output from the AF detection circuit 31 is maximized, and moves the focus lens 16 to that position, thereby performing focusing on the main subject.

  Since the second CCD 21 does not have a color filter, sampling is possible for the entire visible light region. Therefore, it is possible to focus accurately in a dark scene or the like as compared with the case where the focus evaluation value is calculated using the output signal of the first CCD 18 provided with the color filter 36. In addition, there is an advantage that the S / N ratio is improved and the influence of noise is reduced.

  The AF detection circuit 31 may calculate a focus evaluation value based on R, G, and B image signals that are output values of the first A / D converter 20. In this case, a high-pass filter that passes only high-frequency components of the G signal and an absolute value processing unit are required.

  The digital signal processing circuit 25 performs predetermined signal processing on the R, G, and B color image signals captured in a dot-sequential manner, and generates an image signal (Y / C) composed of a luminance signal Y and color difference signals Cr, Cb. Signal). As shown in FIG. 4, the digital signal processing circuit 25 takes in the white balance gain calculation circuit 40 that calculates the gain value for white balance gain adjustment from the integrated value data, and the image signals of R, G, and B colors. An offset correction circuit 41 that performs offset processing, a gain correction circuit 42 that captures an output signal of the offset correction circuit 41 and performs white balance adjustment using the gain value calculated by the gain calculation circuit 40, and an output from the gain correction circuit 42 And a gamma correction circuit 43 that performs gamma conversion using a predetermined γ value for the received signal. The digital signal processing circuit 25 further interpolates the RGB color signals output from the gamma correction circuit 43 to obtain RGB three-color signals at the respective pixel positions, and from the RGB signals after the RGB interpolation calculations. An RGB / YC conversion circuit 45 for obtaining the luminance signal Y and the color difference signals Cr and Cb, a noise filter 46 for reducing noise from the luminance signal Y and the color difference signals Cr and Cb, and a contour for the luminance signal Y after noise reduction. A contour correction circuit 47 that performs correction, a color difference matrix circuit 48 that performs color tone correction by multiplying the color difference signals Cr and Cb after noise reduction by a color difference matrix (C-MTX), and a light source type that incorporates integrated value data And a light source type determination circuit 49 for outputting a color difference matrix selection signal to the color difference matrix circuit 48.

  The RGB interpolation calculation unit 44 is not necessary if the image sensor is a three-plate color image sensor, but the first CCD 18 used in the present embodiment is a single-plate solid-state image sensor, Since only one color signal of R, G, and B is output, in a pixel that does not output, that is, in a pixel that outputs R, how much the G and B color signals are at this pixel position. It is obtained by interpolation calculation from the G and B signals.

  The light source type determination circuit 49 divides one captured image into, for example, 8 × 8 = 64 areas, obtains the values of ΣR, ΣG, and ΣB of the signal charges in each divided area from the integrated values, and obtains ΣR / ΣG data and A set of ΣB / ΣG data is obtained, and these 64 sets of data are plotted in a two-dimensional space spanned by the R / G axis and the B / G axis, and a photographing light source type is detected from the shape of the distribution. The color difference matrix circuit 48 is provided with a plurality of types of color difference matrices corresponding to the light sources. The color difference matrix to be used is switched according to the light source type obtained by the light source type determination circuit 49, and the color difference matrix (C-MTX after this switching) is switched. ) Is multiplied by the input color difference signals Cr and Cb, and the color difference signals Cr ′ and Cb ′ subjected to color correction are output.

  As described above, the digital signal processing circuit 25 performs predetermined signal processing on the R, G, and B image signals captured in a dot-sequential manner, and outputs an image signal (Y that includes the luminance signal Y and the color difference signals Cr and Cb). / C signal).

  The compression / decompression processing circuit 26 performs compression processing of a predetermined format (for example, JPEG) on the image signal (Y / C signal) composed of the input luminance signal Y and color difference signals Cr and Cb in accordance with a compression command from the CPU 11. Generate compressed image data. Further, in accordance with a decompression command from the CPU 11, the input compressed image data is subjected to decompression processing in a predetermined format to generate uncompressed image data.

  The display circuit 27 controls display on the LCD monitor 28 in accordance with a command from the CPU 11.

  The media controller 34 controls reading / writing of data with respect to the recording medium 35 in accordance with a command from the CPU 11. The recording medium 35 may be detachable from the camera body such as a memory card, or may be built in the camera body. In the case of detachable, a card slot is provided in the camera body, and the card slot is used by being loaded.

  Next, wide dynamic range imaging of the digital camera 10 will be described with reference to FIG. FIG. 5 is a flowchart showing the operation of the digital camera 10 in the wide dynamic range shooting mode.

When the digital camera 10 is set to the wide dynamic range shooting mode and the release button is fully pressed, the first CCD 18 and the second CCD 21 are simultaneously exposed (steps S1 and S2). After the exposure is completed, the data of the first CCD 18 and the second CCD 21 are taken in (steps S3 and S4). The data of the first CCD 18 is subjected to predetermined processing by the first analog signal processing unit 19 and then converted into a digital image signal by the first A / D converter 20. The data of the second CCD 21 is subjected to predetermined processing by the second analog signal processing unit 22 and then converted to a digital image signal by the second A / D converter 23. The image input controller 24 stores these image signals in the memory 33.

  Next, signal processing is performed on the image data captured by the first CCD 18, that is, the output signal of the first A / D converter 20, in the digital signal processing circuit 25 (steps S5 to S7). In addition to the output data of the first A / D converter 20 from the memory 33, the digital signal processing circuit 25 receives the integrated value data calculated by the AE detection circuit 30. With respect to these data, the gamma correction circuit 43 performs gamma conversion using the gamma curve shown in FIG. 7A (step S5), and the RGB interpolation calculation unit 44 performs an interpolation calculation to calculate the RGB three colors at each pixel position. A signal is calculated (step S6). Further, the RGB / YC conversion circuit 45 calculates the luminance signal Y and the color difference signals Cr and Cb from the RGB signal after the RGB interpolation calculation, and the noise filter 46 generates noise from the luminance signal Y and the color difference signals Cr and Cb. To reduce. The contour correction circuit 47 performs contour correction on the luminance signal Y1 after noise reduction, and the color difference matrix circuit 48 multiplies the color difference signals Cr and Cb after noise reduction by the color difference matrix (C-MTX). Perform color correction.

  As described above, the luminance signal Y1 and the color difference signals Cr1 and Cb1 are generated from the output signal of the first A / D converter 20. FIG. 6A is an image of the luminance signal Y1 and the color difference signals Cr1 and Cb1 of each pixel of the first CCD 18.

  Next, digital signal processing is performed on the image data captured by the second CCD 21, that is, the output signal of the second A / D converter 23. For the output signal of the second A / D converter 23, the value obtained by gamma conversion (step S8) using the gamma curve shown in FIG. 7B in the gamma correction circuit 43 becomes the luminance signal Y2. FIG. 6B is an image of the luminance signal Y2 of each pixel of the second CCD 21. FIG.

  As shown in FIGS. 7A and 7B, in the gamma conversion process performed by the gamma correction circuit 43, the process for the first CCD 18 and the process for the second CCD 21 are used. The gamma curve is different.

  Next, the luminance signals Y1 and Y2 are mixed at a predetermined mixing ratio to generate the luminance signal Y3 (step S9). This mixing ratio can be adjusted by a user operation, and an image with a wider dynamic range can be obtained as the mixing ratio of the luminance signal Y2 is increased. Further, when obtaining a sharp image, the mixing ratio of the luminance signal Y1 may be increased. Note that the photographing sensitivity and the gamma curve may be changed according to a preset mixing ratio.

  Next, a color difference signal is calculated according to the generated luminance signal Y3 (step S10). The color difference signals Cb2 and Cr2 are generated according to the following formula.

[Equation 1]
Cb2 = B−Y3
[Equation 2]
Cr2 = R−Y3
Here, B and R are image signals of B and R colors, which are output values of the first A / D converter 20, stored in the memory 33.

  Note that the color difference signals Cb2 and Cr2 may be calculated by correcting the color difference signals Cb1 and Cr1 according to the generated luminance signal Y3. FIG. 8 is a diagram showing how the luminance signal Y3 and the color difference signals Cr2 and Cb2 are generated from the luminance signals Y1 and Y2 and the color difference signals Cr1 and Cb1.

  An image is generated based on the Cb2, Cr2, and Y3 signals calculated in this way (step S11).

  As described above, two CCDs are provided, and with respect to one of the CCDs, luminance data with a wide dynamic range is acquired by taking a low sensitivity image by removing the color filter. Further, by processing the luminance data in combination with RAW data of a CCD having a normal color filter, it is possible to perform high-quality shooting with a wide dynamic range without overexposure and underexposure. In addition, the color difference data can be adjusted according to the change in the luminance signal, whereby an output result with high image quality can be obtained. Further, by using a common optical system for the two CCD surfaces, it is possible to reduce the number of parts and the size.

  In the present embodiment, the subject light is divided into two equal parts in the spectroscope 17, the same amount of light is received by the first CCD 18 and the second CCD 21, and the element sensitivity of the first CCD 18 is the element sensitivity of the second CCD 21. Although the configuration is higher, the first CCD 18 and the second CCD 21 may have the same element sensitivity, and the light amount of the first CCD 18 may be increased in the spectroscope 17. For example, instead of receiving the same amount of light with the sensitivity ratio of the first CCD 18 and the second CCD 21 being 2: 1, the spectroscope 17 is configured to have the same sensitivity and the received light amount is 2: 1. May be. In addition, the spectroscope 17 may divide the subject light into two equal parts, and an ND filter may be inserted in the optical path of the second CCD 21 to reduce the amount of light.

FIG. 1 is a block diagram showing an electrical configuration of a digital camera 10 to which the present invention is applied. FIG. 2 is a diagram showing the arrangement of the elements of the first CCD 18 and the second CCD 21 and the color filter 36. FIG. 3 is a diagram showing the relationship between the input light amount and the output value of the first CCD 18 and the second CCD 21. FIG. 4 is a diagram showing an internal configuration of the digital signal processing circuit 25. FIG. 5 is a flowchart showing the operation of the digital camera 10 in the wide dynamic range shooting mode. FIG. 6 is an image of the luminance signal and the color difference signal of each pixel of the first CCD 18 and the second CCD 21. FIG. 7 is a diagram showing a gamma curve used in the gamma correction circuit 43. FIG. 8 is a diagram showing how the luminance signal Y3 and the color difference signals Cr2 and Cb2 are generated from the luminance signals Y1 and Y2 and the color difference signals Cr1 and Cb1.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Digital camera, 11 ... CPU, 17 ... Spectroscope, 18 ... 1st CCD, 19 ... 1st analog signal processing circuit, 20 ... 1st A / D converter, 21 ... 2nd CCD, 22 2nd analog signal processing circuit, 23 ... 2nd A / D converter, 24 ... Image input controller, 25 ... Digital signal processing circuit, 30 ... AE detection circuit, 31 ... AF detection circuit, 42 ... Gain correction circuit 43 ... Gamma correction circuit, 44 ... RGB interpolation circuit, 45 ... RGB / YC conversion circuit

Claims (7)

Spectroscopic means for dividing incident light from a subject incident on the imaging optical system into first subject light and second subject light;
A first imaging means having a color imaging element for receiving the first subject light, and acquiring a first image signal of three primary colors;
A second imaging means having a monochrome imaging element for receiving the second subject light, and acquiring a second image signal having a wider dynamic range than the first image signal;
Means for calculating a first luminance signal for each pixel based on the first image signal;
Means for calculating a second luminance signal for each pixel based on the second image signal;
Means for mixing the first luminance information and the second luminance signal to calculate a third luminance signal having a wide dynamic range;
Means for calculating a color difference signal for each pixel based on the first image signal and the three luminance signals;
An imaging apparatus comprising:   The imaging apparatus according to claim 1, wherein the spectroscopic unit performs spectroscopy so that a light amount of the second subject light is smaller than a light amount of the first subject light.   The imaging apparatus according to claim 1, wherein the first imaging unit is set to have higher imaging sensitivity than the second imaging unit.   The image pickup apparatus according to claim 1, wherein the color image pickup element of the first image pickup unit includes a high sensitivity element, and the monochrome image pickup element of the second image pickup unit includes a low sensitivity element. Gradation conversion processing means for performing gradation conversion processing different between the first image data and the second image data;
The means for calculating the first luminance signal for each pixel based on the first image signal is the first for each pixel based on the first image signal subjected to gradation conversion processing by the gradation conversion processing means. And calculating a second luminance signal for each pixel based on the second image signal. The means for calculating the second luminance signal for each pixel based on the second image signal The imaging device according to claim 1, wherein a second luminance signal for each pixel is calculated based on the image.   The imaging apparatus according to claim 1, further comprising an automatic focusing unit that moves a focus lens to a focusing position based on the second image data. A spectroscopic step of dividing incident light from a subject incident on the imaging optical system into first subject light and second subject light;
A first imaging step of acquiring a first image signal of three primary colors by a color imaging element that receives the first subject light;
A second imaging step of acquiring a second image signal having a wider dynamic range than the first image signal by a monochrome imaging element that receives the second subject light;
Calculating a first luminance signal for each pixel based on the first image signal;
Calculating a second luminance signal for each pixel based on the second image signal;
Mixing the first luminance information and the second luminance signal to calculate a third luminance signal having a wide dynamic range;
Calculating a color difference signal for each pixel based on the first image signal and the third luminance signal;
An imaging control method comprising:

Images