JP2023084577A - Imaging device, imaging method and program - Google Patents

Abstract

To provide an imaging device, an imaging method and a program that acquire a photographed image having on-axis chromatic aberrations more suppressed.SOLUTION: An imaging device 10 comprises: an imaging optical system 12; an imaging element 16 which acquires a photographed image, a memory 8 which stores chromatic aberration correction information corresponding to an image height of the photographed image in a sagittal direction and a tangential direction of the photographed image for correcting on-axis chromatic aberrations of the imaging optical system 12; a drive device 20 which moves the imaging optical system 12 in an optical-axis direction; and a processor 40. The processor 40 acquires a correction object area, reads the corresponding chromatic aberration correction information out of the memory based upon the image height in the correction object area, and determines an imaging position corresponding to a wavelength band of the imaging element based upon the chromatic aberration correction information.SELECTED DRAWING: Figure 1

Description

The present invention relates to an imaging device, an imaging method, and a program.

In general, light incident on an imaging optical system has light of a plurality of different wavelengths. In such a case, the imaging optical system forms images corresponding to the wavelengths of incident light at different positions in the optical axis direction of the imaging optical system (axial chromatic aberration).

In the technique described in Patent Literature 1, longitudinal chromatic aberration is suppressed by changing the relative distance between the imaging optical system and the imaging element, performing multiple shots, and synthesizing the obtained multiple images. Techniques are described for acquiring an image of the

JP 2008-85774 A

One embodiment according to the technology of the present disclosure is to provide an imaging device, an imaging method, and a program capable of acquiring a captured image with further suppressed longitudinal chromatic aberration.

An imaging apparatus according to one aspect of the present invention includes an imaging optical system, an imaging device that acquires a captured image, chromatic aberration correction information for correcting axial chromatic aberration of the imaging optical system, and a sagittal direction in the captured image. and a memory for storing chromatic aberration correction information according to the image height of a captured image in the tangential direction, a driving device for moving the imaging optical system in the optical axis direction, and a processor, wherein the processor is , acquires a correction target area, reads corresponding chromatic aberration correction information from the memory based on the image height of the correction target area, and determines an imaging position according to the wavelength band of the image sensor based on the chromatic aberration correction information.

Preferably, the processor causes the driving device to move the lens group of the imaging optical system to the imaging position, and causes the imaging element to acquire the captured image.

Preferably, the wavelength band has a first color wavelength band, a second color wavelength band, and a third color wavelength band, and the processor determines the imaging position based on the chromatic aberration correction information. A first lens position that is the position of the lens group when imaging the wavelength band of the first color, a second lens position that is the position of the lens group when imaging the wavelength band of the second color, and a third lens position, which is the position of the lens group when imaging the wavelength band of the third color, is determined, and the first lens position, which is the captured image of the wavelength band of the first color at the first lens position A captured image, a second captured image that is a captured image in the second color wavelength band at the second lens position, and a third captured image that is a captured image in the third color wavelength band at the third lens position An image is captured by the imaging device.

An imaging apparatus according to another aspect of the present invention is an imaging apparatus comprising an imaging optical system, an imaging element for acquiring a captured image, a driving device for moving the imaging optical system in an optical axis direction, and a processor. , the processor acquires a correction target area, scans the lens group with a driving device, acquires contrast information in the sagittal direction and the tangential direction of the wavelength band in the correction target area, and performs imaging based on the contrast information. An imaging position is determined according to the wavelength band of the device.

Preferably, the processor causes the driving device to move the imaging optical system to the imaging position, and causes the imaging element to acquire the captured image.

Preferably, the wavelength band has a first color wavelength band, a second color wavelength band, and a third color wavelength band, and the processor is the imaging position based on the contrast information. A first lens position that is the position of the lens group when imaging the wavelength band of one color, a second lens position that is the position of the lens group when imaging the wavelength band of the second color, and A third lens position, which is the position of the lens group in the case of imaging the wavelength band of the third color, is determined, and the first imaging is the captured image of the wavelength band of the first color at the first lens position. an image, a second captured image that is a captured image in the second color wavelength band at the second lens position, and a third captured image that is a captured image in the third color wavelength band at the third lens position is acquired by the imaging device.

Preferably, the wavelength band relates to spectral characteristics of the imaging device.

Preferably, the processor acquires the captured image for each spectral characteristic and generates another captured image from the captured image.

Preferably, the processor calculates the contrast in the sagittal direction and the contrast in the tangential direction of the correction target area, calculates the area contrast ratio of the contrast in the sagittal direction and the contrast in the tangential direction, and performs imaging based on the area contrast ratio. Determine position.

Preferably, the processor determines the imaging position by weighting differently in the sagittal direction and the tangential direction based on the area contrast ratio.

Preferably, the processor receives input of the correction target area based on an area selection instruction from the user.

Preferably, the processor detects a contrast area including a portion having a contrast equal to or greater than a first threshold in the captured image, and receives input as the correction target area.

Preferably, the processor performs face detection processing and accepts input of the face detection area as the correction target area.

Preferably, the processor performs subject detection processing and accepts input of the subject detection area as the correction target area.

Preferably, the processor detects a luminance area including a portion having luminance equal to or higher than the second threshold in the captured image, and receives input as the correction target area.

Preferably, the first lens position, the second lens position, and the third lens position are obtained based on the focal length obtained by the contrast autofocus method of the area to be corrected.

Preferably, the first color is green, the second color is red and the third color is blue.

An image pickup method according to another aspect of the present invention includes an image pickup optical system, an image sensor for acquiring a photographed image, chromatic aberration correction information for correcting longitudinal chromatic aberration of the image pickup optical system, and a sagittal direction in the photographed image. and a memory for storing chromatic aberration correction information according to the image height of a captured image in the tangential direction, a driving device for moving the imaging optical system in the optical axis direction, and a processor. , the processor acquires a correction target area; reads out chromatic aberration correction information corresponding to the correction target area from the memory based on the image height of the correction target area; and determining an imaging position.

An imaging method according to another aspect of the present invention is an imaging method for an imaging apparatus including an imaging optical system, an imaging element for acquiring a captured image, a driving device for moving the imaging optical system in an optical axis direction, and a processor. wherein the processor acquires a correction target area, scans the lens group with a driving device, and acquires contrast information in the sagittal direction and the tangential direction of the wavelength band in the correction target area; and determining an imaging position according to the wavelength band of the imaging element based on the contrast information.

A program, which is another aspect of the present invention, includes an imaging optical system, an imaging device for acquiring a captured image, chromatic aberration correction information for correcting axial chromatic aberration of the imaging optical system, and the sagittal direction and the A program for causing an image capturing apparatus having a memory for storing chromatic aberration correction information corresponding to an image height in a captured image in the tangential direction, a driving device for moving an image capturing optical system in the optical axis direction, and a processor to execute an image capturing method. wherein the processor acquires a correction target area, reads chromatic aberration correction information corresponding to the correction target area from the memory based on the image height of the correction target area, and based on the chromatic aberration correction information, determines the wavelength band of the image sensor. and a step of determining an imaging position in accordance with.

A program, which is another aspect of the present invention, provides an imaging method for an imaging apparatus including an imaging optical system, an imaging element for acquiring a captured image, a driving device for moving the imaging optical system in an optical axis direction, and a processor. A program to be executed, comprising a step of acquiring a correction target area by a processor, and causing a driving device to scan a lens group to acquire contrast information of a wavelength band in a sagittal direction and a tangential direction in the correction target area. and determining an imaging position according to the wavelength band of the imaging element based on the contrast information.

FIG. 1 is a block diagram showing the internal configuration of an imaging device. FIG. 2 is a block diagram showing main functions implemented by a processor. FIG. 3 is a diagram for explaining the sagittal direction and tangential direction of the correction target area. FIG. 4 is a diagram for explaining longitudinal chromatic aberration at image height, sagittal image plane, and tangential image plane. FIG. 5 is a diagram illustrating an example of chromatic aberration correction information. FIG. 6 is a diagram for explaining the movement of the lens group and acquisition of captured images in each wavelength band. FIG. 7 is a diagram illustrating generation of a completed composite image from a multicolor mosaic image. FIG. 8 is a flow chart showing an imaging method using the imaging device. FIG. 9 is a functional block diagram showing functions realized by a processor. FIG. 10 is a diagram illustrating acquisition of contrast information. FIG. 11 is a flow chart showing an imaging method using an imaging device. FIG. 12 is a diagram illustrating specific example 1 of the correction target area. FIG. 13 is a diagram illustrating specific example 2 of the correction target area. FIG. 14 is a diagram illustrating specific example 3 of the correction target area. FIG. 15 is a diagram illustrating specific example 4 of the correction target area. FIG. 16 is a diagram illustrating specific example 5 of the correction target area.

Preferred embodiments of an imaging apparatus, an imaging method, and a program according to the present invention will be described below with reference to the accompanying drawings.

<First Embodiment>
FIG. 1 is a block diagram showing the internal configuration of the imaging apparatus of the present invention.

The imaging device 10 records captured images in a memory card 54 , and the operation of the entire device is centrally controlled by a processor (CPU: Central Processing Unit) 40 .

The imaging device 10 is provided with an operation unit 38 such as a shutter button, a power/mode switch, a mode dial, a cross operation button, and the like. A signal (command) from the operation unit 38 is input to the processor 40 . The processor 40 controls each circuit of the imaging device 10 based on the input signal, and performs drive control of the shutter 14, drive control of the imaging device 16, lens drive control by the lens drive device (driving device) 20, aperture drive control, and imaging. It performs operation control, image processing control, image data recording/playback control, display control of the image monitor 30, and the like.

The imaging optical system 12 is composed of, for example, an interchangeable lens device. The imaging optical system 12 is appropriately exchanged depending on the application and user's preference, and is attached to the camera body (not shown) of the imaging apparatus 10 . The imaging optical system 12 and the camera body are electrically connected, and the imaging optical system 12 and the camera body can communicate with each other. The imaging optical system 12 has a lens side memory 8 . The lens-side memory 8 stores parameters related to the lens group 18 of the imaging optical system 12 . Further, the lens-side memory 8 stores chromatic aberration correction information, which will be described later.

The imaging optical system 12 includes a lens group 18 , and the lens group 18 is moved to the object side or the imaging element 16 side on the optical axis via the lens driving device 20 under the control of the processor 40 . In this example, the lens group 18 is moved to change the distance between the lens group 18 and the imaging element 16, but application of the present invention is not limited to this. For example, the distance between the lens group 18 and the image sensor 16 may be changed by moving the image sensor 16 .

The luminous flux that has passed through the imaging optical system 12 forms an image on an imaging element 16, which is a CMOS (Complementary Metal-Oxide Semiconductor) type color image sensor. However, due to the influence of optical axis chromatic aberration, the position where the image is formed shifts back and forth in the optical axis direction according to the wavelength band of the light that constitutes the luminous flux. Note that this wavelength band relates to the spectral characteristics of the imaging device 16 . The imaging device 16 is not limited to the CMOS type, but may be another type of image sensor such as a CCD (Charge Coupled Device) type, an organic imaging device, or an imaging device having a three-layer laminated sensor structure with one color per layer. .

A lens driving device (driving device) 20 of the imaging optical system 12 is controlled by a processor 40 . The lens driving device 20 moves the lens group 18 of the imaging optical system 12 to the object side and the imaging element 16 side in the optical axis (L) direction. Therefore, the processor 40 can move the imaging position (or image plane position) of the light flux of each wavelength band by moving the lens group 18 with the lens driving device 20 .

The imaging element 16 has a large number of light receiving elements (for example, photodiodes) arranged two-dimensionally, and the subject image formed on the light receiving surface of each light receiving element is a signal voltage (or charge) corresponding to the amount of incident light. ) (photoelectric conversion), converted to a digital signal via an A/D (Analog/Digital) converter in the imaging device 16, and output.

An image signal (image data) read out from the image sensor 16 when shooting a moving image or still image is temporarily stored in a memory (SDRAM (Synchronous Dynamic Random Access Memory) 48 via the image input controller 22 ).

A flash memory 47 stores various parameters and tables used for camera control programs, image processing, and the like.

Shutter 14 is controlled by processor 40 . When the shutter button, which is part of the operation unit 38, is pressed, the processor 40 controls the opening and closing of the shutter 14 to perform exposure for a time corresponding to the preset shutter speed.

The image processing unit 24 reads out unprocessed image data acquired via the image input controller 22 when shooting a moving image or still image and temporarily stored in the memory 48 . The image processing unit 24 performs offset processing, pixel interpolation processing, white balance correction, gain control processing including sensitivity correction, gamma correction processing, demosaic processing (synchronization processing), and luminance and color difference signal generation processing on the read image data. , edge enhancement processing, color correction, and the like. Image data processed by the image processing unit 24 and processed as a live view image is input to a VRAM (Video RAM Random Access Memory) 50 .

The image data read from the VRAM 50 is encoded by the video encoder 28 and output to the image monitor 30 provided on the back of the camera. As a result, a live view image showing the subject image is displayed on the image monitor 30 .

Image data processed by the image processing unit 24 and processed as still images or moving images for recording are stored in the memory 48 again.

The compression/decompression processing unit 26 performs compression processing on the luminance data (Y) and the color difference data (Cb) and (Cr) that are processed by the image processing unit 24 and stored in the memory 48 when recording a still image or moving image. Apply. Compressed compressed image data is recorded in the memory card 54 via the media controller 52 .

Further, the compression/decompression processing unit 26 decompresses the compressed image data obtained from the memory card 54 via the media controller 52 in the reproduction mode. The media controller 52 performs recording and reading of compressed image data on the memory card 54 .

FIG. 2 is a block diagram showing main functions implemented by the processor 40. As shown in FIG.

The processor 40 implements the functions of a correction target area acquisition section 40A, a contrast information acquisition section 40B, a chromatic aberration correction information readout section 40C, an imaging position determination section 40D, a drive device control section 40E, and an image acquisition section 40F.

40 A of correction|amendment object area acquisition parts acquire a correction|amendment object area. 40 A of correction|amendment object area acquisition parts can acquire a correction|amendment object area by various aspects. For example, the correction target area acquisition unit 40A recognizes an AF (Auto Focus) area selected by the user, an area where the subject has a high contrast, an area where a person's face is detected, and the subject in the captured image (or live view image). It is possible to acquire the area where the brightness is high or the area where the luminance is high as the correction target area. The correction target area acquisition unit 40A may acquire the correction target area by an input corresponding to the operation of the operation unit 38 by the user, or the correction target area may be acquired by the correction target area acquisition unit 40A performing preset detection processing. You can get the area. A specific example of the correction target area will be described later.

The contrast information acquisition unit 40B acquires an area contrast ratio, which is the ratio of the contrast in the sagittal direction and the contrast in the tangential direction of the correction target area. First, the contrast information acquisition unit 40B acquires the contrast in the sagittal direction of the correction target area and the contrast in the tangential direction of the correction target area using a known technique. After that, the contrast information acquisition unit 40B acquires an area contrast ratio, which is the ratio of the acquired contrast in the sagittal direction of the correction target area to the contrast in the tangential direction of the correction target area.

FIG. 3 is a diagram for explaining the sagittal direction and tangential direction of the correction target area.

FIG. 3 shows a photographed image 100, and the sagittal direction (indicated by symbol S) and the tangential direction (indicated by symbol T) of the photographed image 100 are shown. A correction target area 102 is also shown in the captured image 100 . The correction target area 102 also has a sagittal direction (indicated by symbol S) and a tangential direction (indicated by symbol T) based on the sagittal direction and tangential direction in the captured image 100 . The contrast information acquisition unit 40B acquires the contrast in the sagittal direction by acquiring the difference between the brightness of the brightest part and the brightness of the darkest part in the correction target area 102 in the sagittal direction. Further, the contrast information acquisition unit 40B acquires the contrast in the tangential direction by acquiring the difference between the brightness of the brightest part and the brightness of the darkest part in the correction target area 102 in the tangential direction. Then, the contrast information acquisition unit 40B acquires an area contrast ratio, which is the ratio of the contrast in the sagittal direction and the contrast in the tangential direction.

A chromatic aberration correction information reading unit 40C (see FIG. 2) reads chromatic aberration correction information stored in the lens side memory 8. FIG. The chromatic aberration correction information is information for correcting axial chromatic aberration of the imaging optical system 12 . Here, longitudinal chromatic aberration varies depending on the image height of the captured image. In addition, since the characteristics of the lens (or lens group) are different between the sagittal image plane and the tangential image plane, it is necessary to correct different longitudinal chromatic aberrations even if the image height is the same. Therefore, the chromatic aberration correction information stored in the lens-side memory 8 has information for correcting axial chromatic aberration according to the image height in the sagittal image plane, the tangential image plane, and the captured image. In addition, since the chromatic aberration correction information differs depending on each imaging optical system 12 , it is preferable to prepare the chromatic aberration correction information for each imaging optical system 12 .

The chromatic aberration correction information reading unit 40C reads the corresponding chromatic aberration correction information from the memory based on the image height of the correction target area. Here, the image height of the correction target area is, for example, the image height corresponding to the center of the correction target area.

FIG. 4 is a diagram for explaining longitudinal chromatic aberration at image height, sagittal image plane, and tangential image plane. In the following example, an image of light in three color wavelength bands will be described, but application of the present invention is not limited to this. For example, the present invention can be applied to light having wavelength bands of two colors.

In FIG. 4, the horizontal axis indicates the position of the image plane, and the vertical axis indicates the image height. The position of the light image of the green wavelength band (first color wavelength band) on the sagittal image plane is indicated by G(S), and the position on the tangential image plane is indicated by G(T). The position of the sagittal image plane of the light image of the red wavelength band (second color wavelength band) is indicated by R(S), and the position of the tangential image plane is indicated by R(T). Also, the position of the sagittal image plane of the light image of the blue wavelength band (the wavelength band of the third color) is indicated by B(S), and the position of the tangential image plane is indicated by B(T).

As shown in FIG. 4, on the optical axis L where the image height is 0, the image of the light in the red wavelength band (indicated by R) is centered on the image of the light in the green wavelength band (indicated by G). and the positions of the image planes of the light images of the blue wavelength band (indicated by symbol B) are different. Further, as the image height increases, images are formed at different positions on the sagittal image plane and the tangential image plane. The chromatic aberration correction information described below has information about the positions of the image planes of the light of the wavelength bands of each color on the image height, the sagittal image plane, and the tangential image plane.

FIG. 5 is a diagram illustrating an example of chromatic aberration correction information stored in the lens side memory 8. As shown in FIG.

Chromatic aberration correction information is, for example, 0 to 10 steps depending on the position of the zoom lens, 0 to 10 steps depending on the aperture size of the diaphragm, 0 to 10 steps depending on the focal length, and 0 to 10 steps depending on the image height. The position of the sagittal image plane and the position of the tangential image plane are divided into 10 levels and are shown with respect to the image of light in the green wavelength band, the image of light in the red wavelength band, and the image of light in the blue wavelength band. .

For example, the image plane position of the sagittal image plane with a zoom of 10, an aperture of 10, a focal length of 10, and an image height of 0 is α0(R) for the image of light in the red wavelength band, and α0(R) for the image of light in the green wavelength band. G), and α0(B) for the image of light in the blue wavelength band. Further, for example, the image plane position of the tangential image plane with a zoom of 10, an aperture of 10, and a focal length of 10 is β0(R) for the image of light in the red wavelength band, and β0(G ), and β0(B) in the image of light in the blue wavelength band. Here, when the image height is 0, the sagittal image plane and the tangential image plane are equal, so α0(R)=β0(R), α0( G)=β0(G), α0(B)=β0(B). Note that FIG. 5 shows image plane positions at image heights 0 to 10 at zoom 10, diaphragm 10, and focal length 10, and other information in the chromatic aberration correction information is omitted.

The imaging position determining section 40D (see FIG. 2) determines the imaging position based on the chromatic aberration correction information read by the chromatic aberration correction information reading section 40C. Specifically, the imaging position determination unit 40D determines the imaging position based on the area contrast ratio and the read chromatic aberration correction information. For example, the image plane position corresponding to the image height is read from the chromatic aberration correction information reading unit 40C, and the imaging position determination unit 40D determines the position of the sagittal image plane and the position of the tangential image plane based on the area contrast ratio. Let the position where the weighted average is performed be the imaging position. In this way, the imaging position determination unit 40D determines the imaging position of the light image in the green wavelength band (first lens position), the imaging position of the light image in the red wavelength band (second lens position), and blue wavelength band light image imaging positions (third lens positions). Note that a photographed image (first photographed image) for G pixels (green pixels) is acquired at the imaging position of the light image in the green wavelength band, and at the imaging position of the light image in the red wavelength band, A captured image (second captured image) for R pixels (red pixels) is acquired, and a captured image (third captured image) for B pixels (blue pixels) is obtained at the imaging position of the light image in the blue wavelength band. captured image) is acquired.

A specific example of determination of the imaging position performed by the imaging position determination unit 40D will be described below.

(Specific example 1)
In this example, the contrast information acquisition unit 40B acquires the area contrast ratio of the correction target area as 2:3 (sagittal direction: tangential direction). Further, the chromatic aberration correction information reading unit 40C reads the image plane positions (α10(R), α10(G), α10(B), β10(R ), β10(G), β10(B)) (see FIG. 5). In such a case, based on the following equations 1 to 3, the imaging position X1 for acquiring the captured image for G pixels, the imaging position X2 for acquiring the captured image for R pixels, and the captured image for B pixels are determined. An imaging position X3 to be acquired is determined.

X1=(α10(G)×2+β10(G)×3)/5 (Formula 1)
X2=(α10(R)×2+β10(R)×3)/5 (Formula 2)
X3=(α10(B)×2+β10(B)×3)/5 (Formula 3)
Here, the imaging position refers to the relative positional relationship between the position of the lens group 18 and the imaging element 16, and in the present embodiment, the imaging position is changed by moving the lens group 18. FIG.

(Specific example 2)
In this example, an imaging position X1 for obtaining a captured image for G pixels is determined by AF control, MF (Manual Focus) control, or the like. In this case, the imaging position X1 for acquiring the captured image for G pixels is set to 0, the imaging position X2 for acquiring the captured image for R pixels, and the imaging position X3 for acquiring the captured image for B pixels are α10(G). , β10(G), and determined based on the following equations 1a to 3a. Note that the area contrast ratio of the correction target area (2:3 (sagittal direction: tangential direction), and the image plane positions of the sagittal image plane and the tangential image plane at an image height of 10 read by the chromatic aberration correction information reading unit 40C are Same as 1.

X1=0 (formula 1a)
X2=((α10(R)-α10(G))×2+(β10(R)-β10(G))×3)/5 (Formula 2a)
X3=((α10(B)-α10(G))×2+(β10(B)-β10(G))×3)/5 (Formula 3a)
The driving device control section 40E (see FIG. 2) moves the lens group 18 via the lens driving device 20 to the imaging position determined by the imaging position determining section 40D. The driving device control unit 40E, for example, sequentially shifts the lens group 18 to a photographing position for photographing a photographed image for G pixels, a photographing position for photographing a photographed image for R pixels, and a photographing position for photographing a photographed image for B pixels. to move.

The image acquisition unit 40F causes the image pickup device 16 to acquire a captured image after the driving device control unit 40E moves the lens group 18 to the image pickup position. For example, the image acquisition unit 40F causes the image pickup element 16 to acquire the captured image for the G pixels after the lens group 18 has completed the movement to the imaging position for the captured image for the G pixels. Further, for example, the image acquisition unit 40F causes the image pickup device 16 to acquire the captured image for the R pixel after the lens group 18 has completed the movement to the imaging position for the captured image for the R pixel. Further, for example, the image acquisition unit 40F causes the image pickup device 16 to acquire the captured image for the B pixels after the lens group 18 has completed the movement to the imaging position of the captured image for the B pixels.

FIG. 6 is a diagram for explaining the movement of the lens group 18 and acquisition of captured images in each wavelength band.

FIG. 6A is a diagram illustrating acquisition of a photographed image for G pixels. The driving device control unit 40E moves the lens group 18 to the imaging position of the image of the light in the green wavelength band (the imaging position of the captured image for the G pixels). After that, the image acquiring unit 40F acquires the photographed image 110 for G pixels.

FIG. 6B is a diagram illustrating acquisition of a photographed image for R pixels. The driving device control unit 40E moves the lens group 18 to the imaging position of the light image of the red wavelength band (the imaging position of the captured image for the R pixel). After that, the image acquiring unit 40F acquires the photographed image 112 for R pixels.

FIG. 6C is a diagram illustrating acquisition of a photographed image for B pixels. The driving device control unit 40E moves the lens group 18 to the imaging position of the image of the light in the blue wavelength band (the imaging position of the captured image for the B pixels). After that, the image acquiring unit 40F acquires the photographed image 114 for B pixels.

The image acquisition unit 40F uses the obtained G pixels of the captured image 110 for G pixels, the R pixels of the captured image 112 for R pixels, and the B pixels of the captured image 114 for B pixels to obtain one multi-pixel image. Generate a color mosaic image (another captured image). Then, the image acquisition unit 40F acquires a completed composite image by causing the image processing unit 24 to perform demosaic processing on the multicolor mosaic image.

FIG. 7 is a diagram illustrating generation of a completed composite image from a multicolor mosaic image.

The multicolor mosaic image 116 includes G pixels of the captured image 110 for G pixels (indicated as “1” in the figure), R pixels of the captured image 112 for R pixels (indicated as “2” in the figure), B It is composed of B pixels (denoted as “3” in the figure) of the captured image 114 for pixels. In this example, the color filters arranged in the imaging device 16 are arranged in a Bayer array. Then, the image acquisition unit 40</b>F demosaicizes the multicolor mosaic image by the image processing unit 24 to acquire the completed composite image 118 . The completed composite image 118 obtained in this manner is an image in which axial chromatic aberration has been corrected along the contrast ratio of the correction target area.

FIG. 8 is a flow chart showing an imaging method using the imaging device 10. As shown in FIG. In this imaging method, each step is executed by the processor 40 executing a program stored in the flash memory 47 .

First, the correction target area acquisition unit 40A acquires a correction target area (step S100). For example, the correction target area acquiring unit 40A acquires the area detected by the face detection process, which is the position where the subject's face is captured by the face detection process performed by the imaging device 10, as the correction target area. In this example, the face detected in the correction target area is automatically focused (autofocused). Next, the contrast information acquisition unit 40B calculates the contrast in the sagittal direction and the contrast in the tangential direction in the correction target area (step S101). After that, the contrast information acquiring unit 40B acquires an area contrast ratio, which is the ratio of the contrast in the sagittal direction and the contrast in the tangential direction (step S102).

Thereafter, the chromatic aberration correction information reading unit 40C reads the image plane positions of the sagittal image plane and the tangential image plane according to the image height from the lens side memory 8 as chromatic aberration correction information. Then, based on the area contrast ratio and the acquired chromatic aberration correction information (the positions of the sagittal image plane and the tangential image plane), the imaging position determining unit 40D determines the imaging position of the captured image for G pixels, the captured image for R pixels, and so on. and the imaging position of the captured image for B pixels are determined (step S103). Specifically, the imaging position determination unit 40D performs weighting by the area contrast ratio, and determines a position obtained by weighted averaging the image plane positions of the sagittal image plane and the tangential image plane as the imaging position.

After that, the driving device control section 40E moves the lens group 18 of the imaging optical system 12 to each imaging position determined by the imaging position determining section 40D. First, the driving device control unit 40E moves the lens group 18 to the imaging position of the captured image for G pixels (step S104). After that, the image acquisition unit 40F causes the image capturing device 10 to perform image capturing, and captures a captured image for G pixels (step S105). After that, the driving device control unit 40E moves the lens group 18 to the imaging position of the captured image for the R pixel (step S106). After that, the image acquisition unit 40F causes the image capturing device 10 to perform image capturing to capture a captured image for R pixels (step S107). After that, the driving device control unit 40E moves the lens group 18 to the imaging position of the captured image for the B pixels (step S108). After that, the image acquisition unit 40F causes the image capturing device 10 to perform image capturing and capture a captured image for B pixels (step S109). Note that in this case, the captured image for G pixels, the captured image for R pixels, and the captured image for B pixels are composed of RAW data.

After that, the image acquisition unit 40F uses the G pixels of the captured image for the G pixels, the R pixels of the captured image for the R pixels, and the B pixels of the captured image for the B pixels to obtain one multicolor mosaic image. Generate (step S110). After that, the image acquisition unit 40F acquires the completed composite image 118 by causing the image processing unit 24 to perform demosaic processing on the multicolor mosaic image (step S111).

In the above example, the area detected by the face detection process acquired as the correction target area is automatically focused, but the present invention is not limited to this. The focus when acquiring the captured image for G pixels, the captured image for R pixels, and the captured image for B pixels can be adjusted automatically or manually in various forms.

As described above, in the present embodiment, the imaging positions of the captured image for G pixels, the captured image for R pixels, and the captured image for B pixels are determined by the contrast in the sagittal direction and the contrast in the tangential direction of the correction target area. is determined based on the area contrast ratio, which is the ratio of the contrast of Then, a multicolor mosaic image is generated using the obtained captured image for G pixels, captured image for R pixels, and captured image for B pixels, and the multicolor mosaic image is demosaiced to complete a combined image. is generated. As a result, the present embodiment can obtain a complete composite image in which axial chromatic aberration is appropriately suppressed according to the captured image.

<Second embodiment>
Next, a second embodiment will be described.

FIG. 9 is a functional block diagram showing functions realized by the processor 40 of this embodiment.

The processor 40 of the present embodiment implements the functions of a correction target area acquisition section 40A, a contrast AF control section 40G, an imaging position determination section 40D, a driving device control section 40E, and an image acquisition section 40F. 2 are assigned the same reference numerals, and the description thereof will be omitted.

The contrast AF control unit 40G causes the imaging device 10 to perform AF in the correction target area by the contrast autofocus method. Then, the contrast AF control section 40G acquires contrast information in the correction target area. Specifically, the contrast AF control unit 40G scans the lens group 18 from infinity to a close distance by the driving device, measures the contrast in the sagittal direction and the contrast in the tangential direction of the correction target area, and obtains the contrast information. do.

FIG. 10 is a diagram for explaining acquisition of contrast information performed under the control of the contrast AF control section 40G.

In FIG. 10 , the vertical axis indicates the contrast, and the horizontal axis indicates the position of the lens group 18 . FIG. 10 also shows the contrast in the sagittal direction and the contrast in the tangential direction in the area to be corrected, obtained by scanning the lens group 18 from infinity to close range. Specifically, the sagittal contrast of the green wavelength band light image is indicated by line 122A, and the tangential contrast of the green wavelength band light image is indicated by dashed line 122B. The sagittal contrast of the red wavelength band light image is indicated by line 120A, and the tangential contrast of the red wavelength band light image is indicated by dashed line 120B. The sagittal contrast of the image of light in the blue wavelength band is indicated by line 124A, and the tangential contrast of the image of light in the blue wavelength band is indicated by dashed line 124B. FIG. 10 also shows the contrast peak position of each color. Specifically, the peak position of the contrast (line 122A) in the sagittal direction of the image of the light in the green wavelength band is (P1(S), L1(S)), and the tangential direction of the image of the light in the green wavelength band is The peak position of the directional contrast (dotted line 122B) is (P1(T), L1(T)). Also, the peak position of the contrast (line 120A) in the sagittal direction of the light image of the red wavelength band is (P2(S), L2(S)), and the tangential direction of the light image of the red wavelength band is The peak position of the contrast (dotted line 120B) is indicated by (P2(T), L2(T)). Also, the peak position of the contrast (line 124A) in the sagittal direction of the image of the light in the blue wavelength band is (P3(S), L3(S)), and the tangential direction of the image of the light in the blue wavelength band is The peak position of the contrast (dotted line 124B) is indicated by (P3(T), L3(T)).

As shown in FIG. 10, the contrast AF control unit 40G scans the lens group 18 to obtain the contrast of each color for each position of the lens group 18, and obtains the contrast peak position.

Then, the contrast AF control unit 40G acquires an area contrast ratio, which is the ratio of the contrast in the sagittal direction and the contrast in the tangential direction, from the contrast peak position. Specifically, in the case shown in FIG. 10, the area contrast ratio (sagittal direction: tangential direction) of the green wavelength band is L1(S):L1(T), and the area contrast ratio of the red wavelength band is (sagittal direction: tangential direction) becomes L2(S):L2(T), and the area contrast ratio (sagittal direction: tangential direction) of the blue wavelength band becomes L3(S):L3(T). . Thus, in this embodiment, the area contrast ratio for the green wavelength band, the area contrast ratio for the red wavelength band, and the area contrast ratio for the blue wavelength band are obtained.

The imaging position determination unit 40D determines the captured image for the G pixel based on the peak position (position of the lens group 18) of the wavelength band of each color acquired by the contrast AF control unit 40G and the area contrast ratio of the wavelength band of each color. , the imaging positions of the captured image for R pixels and the captured image for B pixels are weighted and averaged.

The imaging position determination unit 40D, for example, determines the imaging position Y1 of the captured image for G pixels, the imaging position Y2 of the captured image for R pixels, and the imaging position Y3 of the captured image for B pixels according to the following equations (4) to ( 6) can be determined.

Y1=(P1(S)×L1(S)+P1(T)×L1(T))/(L1(S)+L1(T)) (Formula 4)
Y2=(P2(S)×L2(S)+P2(T)×L2(T))/(L2(S)+L2(T)) (Formula 5)
Y3=(P3(S)×L3(S)+P3(T)×L3(T))/(L3(S)+L3(T)) (Formula 6)
FIG. 11 is a flow chart showing an imaging method using the imaging apparatus 10 of this embodiment. In this imaging method, each step is executed by the processor 40 executing a program stored in the flash memory 47 .

First, the correction target area acquiring unit 40A acquires a correction target area (step S200). For example, the correction target area acquiring unit 40A acquires the area detected by the face detection process, which is the position where the face of the subject is captured by the face detection process, as the correction target area. Next, the contrast AF control unit 40G causes the driving device to scan the lens group 18 to perform contrast AF in the correction target area (step S201). The contrast AF control unit 40G executes contrast AF in the correction target area to calculate the contrast in the sagittal direction and the tangential direction for each position of the lens group 18 (step S202). After that, the correction target area acquisition unit 40A acquires an area contrast ratio, which is the ratio of the contrast in the sagittal direction and the contrast in the tangential direction (step S203).

Next, the imaging position determination unit 40D determines the imaging position from the average (weighted average) weighted based on the area contrast ratio (step S204). Specifically, the imaging position determining unit 40D determines the imaging position of the captured image for the G pixels, the imaging position of the captured image for the R pixels, the imaging position of the captured image for the B pixels, and the It is determined by weighted averaging the imaging position of the image.

After that, the driving device control section 40E moves the lens group 18 of the imaging optical system 12 to each imaging position determined by the imaging position determination section 40D. First, the drive device control unit 40E moves the lens group 18 to the imaging position of the captured image for G pixels (step S205). After that, the image acquisition unit 40F executes shooting and shoots a shot image for G pixels (step S206). After that, the driving device control unit 40E moves the lens group 18 to the imaging position of the captured image for the R pixel (step S207). After that, the image acquisition unit 40F executes image capturing and captures a captured image for R pixels (step S208). The driving device control unit 40E moves the lens group 18 to the imaging position for the B pixel (step S209). After that, the image acquisition unit 40F executes image capturing to capture a captured image for B pixels (step S210).

Next, the image acquisition unit 40F creates one multicolor mosaic image using the G pixels of the captured image for G pixels, the R pixels of the captured image for R pixels, and the B pixels of the captured image for B pixels. is generated (step S211). After that, the image acquiring unit 40F acquires a composite image by demosaicing the multicolor mosaic image (step S212).

As described above, in the present embodiment, the imaging positions of the captured image for G pixels, the captured image for R pixels, and the captured image for B pixels are determined by the contrast in the sagittal direction and the contrast in the tangential direction of the correction target area. is determined based on the area contrast ratio, which is the ratio of the contrast of Then, a multicolor mosaic image is generated using the obtained captured image for G pixels, captured image for R pixels, and captured image for B pixels, and the multicolor mosaic image is demosaiced to complete a combined image. is generated. As a result, the present embodiment can obtain a complete composite image in which axial chromatic aberration is appropriately suppressed according to the captured image. Further, in the present embodiment, even when using the imaging optical system 12 in which the chromatic aberration correction information described in the first embodiment is not stored in the lens-side memory 8, a complete composition in which longitudinal chromatic aberration is suppressed is obtained. Images can be acquired.

<Example of correction target area>
Next, a specific example of the correction target area will be described. The correction target area acquisition unit 40A can acquire the correction target area in various modes. A specific example of acquisition of the correction target area by the correction target area acquisition unit 40A will be described below.

FIG. 12 is a diagram illustrating specific example 1 of the correction target area.

In this example, the correction target area acquisition unit 40A receives an input of the correction target area based on the area selection instruction from the user. As shown in FIG. 12, the user selects AF area 132 on image monitor 30 . For example, the user selects the AF area 132 via the operation section 38 . The correction target area acquisition unit 40A acquires the AF area 132 as the correction target area based on the input signal from the operation unit 38. FIG. In this way, by acquiring the area selected by the user as the correction target area, it is possible to correct axial chromatic aberration in accordance with the user's intention.

FIG. 13 is a diagram illustrating specific example 2 of the correction target area.

In this example, the correction target area acquisition unit 40A receives an input as the correction target area, which is the outline of the subject. The contour portion of the subject is a portion with high contrast, and longitudinal chromatic aberration appears remarkably at the portion with high contrast.

In FIG. 13, a subject 130 is shown. The processor 40 detects the contour portion 134 by detecting a contrast area having a contrast equal to or greater than a preset first threshold in the captured image (live view image). Then, the correction target area acquiring unit 40A acquires the detected contour part 134 as the correction target area. By obtaining the contour portion 134 (contrast area) as a correction target area in this way, it is possible to obtain an image in which longitudinal chromatic aberration is suppressed in the contour portion of the subject. Note that the first threshold is preferably determined in advance from the viewpoint of detecting the contour of the subject.

FIG. 14 is a diagram illustrating specific example 3 of the correction target area.

FIG. 14 shows a face detection area 136 that is displayed after face detection processing of a main subject (person) 138 is performed by the processor 40 . The face detection area 136 is detected by the processor 40 in the captured image (live view image) using a known face detection technique. Thereafter, the correction target area acquisition unit 40A accepts the detected face detection area 136 as the correction target area. In this example, the correction target area acquiring unit 40A receives an input as a correction target area, which is a face detection area subjected to face detection processing. As a result, longitudinal chromatic aberration can be corrected with priority given to the main subject.

FIG. 15 is a diagram illustrating specific example 4 of the correction target area.

In this example, the correction target area acquiring unit 40A receives an input as the correction target area of the subject detection area in which the subject detection processing has been performed.

In FIG. 15, subject detection processing for a main subject 142 is performed by the processor 40, and a subject detection area 140 is shown. The subject detection area 140 is detected by the processor 40 in the captured image (live view image) by a known subject detection technique. After that, the correction target area acquiring unit 40A accepts the detected subject detection area 140 as the correction target area. By making the subject detection area 140 the area to be corrected in this way, axial chromatic aberration in the subject can be suppressed. Note that the subject to be detected may be singular or plural. When there are a plurality of areas to be corrected, averaging or weighted averaging is performed.

FIG. 16 is a diagram illustrating specific example 5 of the correction target area.

In this example, the correction target area obtaining unit 40A receives an input of a brightness area including a portion having brightness equal to or higher than the second threshold in the captured image as the correction target area. Since the axial chromatic aberration is conspicuous in the contour portion with a large luminance difference, obtaining the contour portion with a large luminance difference as the correction target area effectively suppresses the axial chromatic aberration.

In FIG. 16, a subject (tree) is shown with a light 146 of high brightness. The processor 40 detects a brightness area 144 having a brightness equal to or higher than a preset second threshold in the captured image (live view image). Then, the correction target area acquisition unit 40A acquires the detected brightness area 144 as the correction target area. By obtaining the luminance area 144 as a correction target area in this way, it is possible to obtain a complete composite image in which axial chromatic aberration is suppressed in areas with high luminance.

In the above embodiment, the processing units (correction target area acquiring unit 40A, contrast information acquiring unit 40B, chromatic aberration correction information reading unit 40C, imaging position determining unit 40D, driving device control unit 40E, image acquiring unit 40F) that execute various processes , Contrast AF control unit 40G) (processing unit) has a hardware structure of various processors as shown below. For various processors, the circuit configuration can be changed after manufacturing such as CPU (Central Processing Unit), which is a general-purpose processor that executes software (program) and functions as various processing units, FPGA (Field Programmable Gate Array), etc. Programmable Logic Device (PLD), which is a processor, ASIC (Application Specific Integrated Circuit), etc. be

One processing unit may be composed of one of these various processors, or composed of two or more processors of the same type or different types (for example, a plurality of FPGAs, or a combination of a CPU and an FPGA). may Also, a plurality of processing units may be configured by one processor. As an example of configuring a plurality of processing units in a single processor, first, as represented by a computer such as a client or server, a single processor is configured by combining one or more CPUs and software. There is a form in which a processor functions as multiple processing units. Secondly, as typified by System On Chip (SoC), etc., there is a form of using a processor that realizes the function of the entire system including a plurality of processing units with a single IC (Integrated Circuit) chip. be. In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.

Further, the hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor elements.

Each configuration and function described above can be appropriately realized by arbitrary hardware, software, or a combination of both. For example, a program that causes a computer to execute the above-described processing steps (procedures), a computer-readable recording medium (non-temporary recording medium) recording such a program, or a computer capable of installing such a program However, the present invention can be applied.

Although the examples of the present invention have been described above, it goes without saying that the present invention is not limited to the above-described embodiments, and that various modifications are possible without departing from the scope of the present invention.

8: Lens side memory 10: Aperture 12: Imaging optical system 14: Shutter 16: Imaging element 18: Lens group 20: Lens driving device 22: Image input controller 24: Image processing unit 26: Compression/decompression processing unit 28: Video encoder 30 : Image monitor 38 : Operation unit 40 : Processor 47 : Flash memory 48 : Memory 52 : Media controller 54 : Memory card

Claims (21)

An imaging optical system, an imaging element for acquiring a captured image, and chromatic aberration correction information for correcting axial chromatic aberration of the imaging optical system, wherein the sagittal direction and the tangential direction in the captured image in the captured image An imaging apparatus comprising: a memory for storing chromatic aberration correction information according to image height; a driving device for moving the imaging optical system in an optical axis direction; and a processor,
The processor
Acquire the correction target area,
reading the corresponding chromatic aberration correction information from the memory based on the image height of the correction target area;
determining an imaging position according to the wavelength band of the imaging element based on the chromatic aberration correction information;
Imaging device. The processor
moving the lens group of the imaging optical system to the imaging position by the driving device;
2. The imaging apparatus according to claim 1, wherein said imaging device acquires said photographed image. the wavelength band has a first color wavelength band, a second color wavelength band, and a third color wavelength band;
The processor
based on the chromatic aberration correction information; determining a second lens position, which is the position of the lens group when imaging, and a third lens position, which is the position of the lens group when imaging the wavelength band of the third color;
a first captured image that is the captured image in the first color wavelength band at the first lens position; a second captured image that is the captured image in the second color wavelength band at the second lens position; and a third captured image, which is the captured image of the third color wavelength band at the third lens position.
The imaging device according to claim 2. An imaging apparatus comprising an imaging optical system, an imaging element for acquiring a captured image, a driving device for moving the imaging optical system in an optical axis direction, and a processor,
The processor
Acquire the correction target area,
Scanning the lens group with the driving device to obtain contrast information in the sagittal direction and the tangential direction of the wavelength band in the correction target area,
Determining an imaging position according to the wavelength band of the imaging element based on the contrast information;
Imaging device. The processor
moving the imaging optical system to the imaging position by the driving device;
5. The imaging apparatus according to claim 4, wherein said imaging device acquires said photographed image. the wavelength band has a first color wavelength band, a second color wavelength band, and a third color wavelength band;
The processor
Based on the contrast information, a first lens position, which is the position of the lens group when imaging the wavelength band of the first color, which is the imaging position, and the wavelength band of the second color. determining a second lens position, which is the position of the lens group when performing imaging, and a third lens position, which is the position of the lens group when imaging the wavelength band of the third color;
a first captured image that is the captured image in the first color wavelength band at the first lens position; a second captured image that is the captured image in the second color wavelength band at the second lens position; and a third captured image, which is the captured image of the third color wavelength band at the third lens position.
The imaging device according to claim 5. The imaging apparatus according to any one of claims 1 to 6, wherein the wavelength band relates to spectral characteristics of the imaging device. 8. The imaging apparatus according to claim 7, wherein the processor acquires the captured image for each spectral characteristic and generates another captured image from the captured image. The processor
calculating the contrast in the sagittal direction and the contrast in the tangential direction of the correction target area, calculating the area contrast ratio of the contrast in the sagittal direction and the contrast in the tangential direction;
The imaging apparatus according to any one of claims 1 to 8, wherein the imaging position is determined based on the area contrast ratio. The processor
10. The imaging apparatus according to claim 9, wherein the imaging position is determined by weighting differently in the sagittal direction and the tangential direction based on the area contrast ratio. The imaging apparatus according to any one of claims 1 to 10, wherein the processor receives input of the correction target area based on an area selection instruction from a user. 12. The processor according to any one of claims 1 to 11, wherein the processor detects a contrast area including a portion having a contrast equal to or greater than a first threshold in the captured image, and receives input of the contrast area as the correction target area. The imaging device described. 13. The imaging apparatus according to any one of claims 1 to 12, wherein the processor performs face detection processing and receives input of a face detection area as the correction target area. 14. The imaging apparatus according to any one of claims 1 to 13, wherein the processor performs subject detection processing and receives an input of a subject detection area as the correction target area. 15. The processor according to any one of claims 1 to 14, wherein the processor detects a brightness area including a portion having a brightness equal to or higher than a second threshold in the captured image, and receives input of the brightness area as the correction target area. The imaging device described. 7. The first lens position, the second lens position, and the third lens position according to claim 3 or 6, wherein the focal length of the correction target area obtained by a contrast autofocus method is obtained. imaging device. 7. The imaging device according to claim 3, wherein said first color is green, said second color is red, and said third color is blue. An imaging optical system, an imaging element for acquiring a captured image, and chromatic aberration correction information for correcting axial chromatic aberration of the imaging optical system, wherein the sagittal direction and the tangential direction in the captured image in the captured image An imaging method for an imaging device comprising: a memory for storing chromatic aberration correction information according to image height; a driving device for moving the imaging optical system in an optical axis direction; and a processor,
the processor
a step of obtaining a correction target area;
reading out the corresponding chromatic aberration correction information from the memory based on the image height of the correction target area;
determining an imaging position according to the wavelength band of the imaging element based on the chromatic aberration correction information;
imaging method. An image capturing method for an image capturing apparatus comprising: an image capturing optical system; an image sensor for obtaining a captured image; a driving device for moving the image capturing optical system in an optical axis direction; and a processor,
the processor
a step of obtaining a correction target area;
a step of scanning the lens group by the driving device to obtain contrast information in the sagittal direction and the tangential direction of the wavelength band in the correction target area;
determining an imaging position corresponding to the wavelength band of the imaging element based on the contrast information;
imaging method. An imaging optical system, an imaging element for acquiring a captured image, and chromatic aberration correction information for correcting axial chromatic aberration of the imaging optical system, wherein the sagittal direction and the tangential direction in the captured image in the captured image A program for causing an imaging apparatus comprising a memory for storing chromatic aberration correction information corresponding to image height, a driving device for moving the imaging optical system in the optical axis direction, and a processor to execute an imaging method,
to the processor;
a step of obtaining a correction target area;
reading out the corresponding chromatic aberration correction information from the memory based on the image height of the correction target area;
determining an imaging position according to the wavelength band of the imaging element based on the chromatic aberration correction information;
program to run. A program for causing an imaging apparatus comprising an imaging optical system, an imaging element for acquiring a captured image, a driving device for moving the imaging optical system in an optical axis direction, and a processor to execute an imaging method,
to the processor;
a step of obtaining a correction target area;
a step of scanning the lens group by the driving device to obtain contrast information in the sagittal direction and the tangential direction of the wavelength band in the correction target area;
determining an imaging position corresponding to the wavelength band of the imaging element based on the contrast information;
program to run.

Images