JP5272699B2 - Image processing apparatus, imaging apparatus, program, and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a means for correcting aberrations such as axial chromatic aberration and field curvature aberration. <P>SOLUTION: An image processing apparatus includes an image reading part and a corrected image generation part. The image reading part receives a plurality of shot images respectively shot by changing at least one of the parameters such as a focal point distance of shooting lens, a shooting distance, and a lens location of shooting lens. The corrected image generation part receives image information from each of shot images and combines a plurality of pieces of image information to generate a corrected image with shooting lens aberrations corrected. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to an image processing device, an imaging device, a program, and an image processing method.

It is known that the lens has aberrations such as aberrations caused by monochromatic light (Sydel's five aberrations) and chromatic aberrations due to differences in color wavelengths. Various techniques for suppressing the influence of the above-described aberration have been proposed for images captured by an electronic camera. As an example, in Patent Document 1, a plurality of images obtained by changing the relative distance between the lens and the image sensor in the optical axis direction are used, and the field curvature aberration (the focus position changes according to the image height). Thus, there is disclosed a technique for correcting aberrations in which the image surface is curved) and axial chromatic aberration (aberration caused by the fact that the imaging position on the optical axis differs for each color wavelength).
JP 2008-85774 A

  By the way, in the above prior art, aberration correction is performed by adjusting the relative distance between the lens and the image sensor in the optical axis direction. However, there is still a demand for means for correcting axial chromatic aberration and curvature of field by other methods. Has been.

  Therefore, an object of the present invention is to provide means for correcting aberrations such as longitudinal chromatic aberration and curvature of field aberration.

An image processing apparatus according to one aspect includes an image reading unit, an image processing unit, and a corrected image generation unit. The image reading unit obtains a plurality of captured images respectively captured by changing at least one parameter of the focal length of the photographing lens, the photographing distance, and the arrangement of a part of the lens group of the photographing lens. The image processing unit executes image processing of the captured image. The corrected image generation unit acquires image information from each captured image, and generates a corrected image in which the aberration of the photographing lens is corrected by combining a plurality of pieces of image information. The amount of change in shooting conditions in a plurality of captured images is determined based on the type of shooting lens, the focal length of the shooting lens, and the shooting distance. The captured image includes information on a plurality of color components, and the plurality of captured images are images that are captured by changing parameters according to the shift of the imaging position of the photographing lens in the light of the wavelength corresponding to each color component. is there. The image processing unit includes a magnification adjustment process that adjusts a difference in photographing magnification between captured images, a distortion aberration adjustment process that adjusts a difference in distortion aberration between captured images, and a shake that corrects subject blur between the captured images. At least one of a correction process and a brightness adjustment process for adjusting a difference in image brightness between captured images is executed. The corrected image generation unit acquires image information corresponding to different color components from each captured image, and generates a corrected image in which the longitudinal chromatic aberration is corrected.

An image processing apparatus according to one aspect includes an image reading unit, an image processing unit, and a corrected image generation unit. The image reading unit obtains a plurality of captured images respectively captured by changing at least one parameter of the focal length of the photographing lens, the photographing distance, and the arrangement of a part of the lens group of the photographing lens. The image processing unit executes image processing of the captured image. The corrected image generation unit acquires image information from each captured image, and generates a corrected image in which the aberration of the photographing lens is corrected by combining a plurality of pieces of image information. The amount of change in shooting conditions in a plurality of captured images is determined based on the type of shooting lens, the focal length of the shooting lens, the shooting distance, and the aperture value of the shooting lens. The image processing unit includes a magnification adjustment process that adjusts a difference in photographing magnification between captured images, a distortion aberration adjustment process that adjusts a difference in distortion aberration between captured images, and a blur that corrects blurring of a subject between the captured images. At least one of a correction process and a brightness adjustment process for adjusting a difference in image brightness between captured images is executed. The corrected image generation unit acquires image information from positions where the image heights differ between the captured images, and generates a corrected image in which the field curvature aberration is corrected.

An image processing apparatus according to one aspect includes an image reading unit, an image processing unit, and a corrected image generation unit. The image reading unit acquires a plurality of captured images that are bracketed and photographed by changing a focal length of the photographing lens. When the brightness differs between the captured images, the image processing unit executes a brightness adjustment process for adjusting a difference in image brightness between the captured images. The corrected image generation unit acquires image information from each captured image, and generates a corrected image in which the aberration of the photographing lens is corrected by combining a plurality of pieces of image information. The image processing unit includes a first captured image formed on the imaging surface with the main wavelength of the R pixel, a second captured image formed on the imaging surface with the main wavelength of the G pixel, and an imaging surface with the main wavelength of the B pixel. The difference in image brightness between the third captured images to be formed is adjusted.

  Here, an imaging apparatus including the image processing apparatus according to the one aspect, a program that causes a computer to function as the image processing apparatus according to the first aspect, and operations of the image processing apparatus and the imaging apparatus according to a method category. What is expressed is also effective as a specific embodiment of the present invention.

In the present invention, a corrected image in which the aberration of the photographing lens is corrected can be generated.

<Description of One Embodiment>
FIG. 1 is a schematic diagram illustrating a configuration example of an electronic camera system according to an embodiment. An electronic camera system according to one embodiment includes a lens unit 1 and a camera body 2 as an image processing apparatus. The lens unit 1 is replaceably mounted on the camera body 2 via a mount (not shown) having electrical contacts. Then, when the lens unit 1 and the camera body 2 are connected, the electrical connection by the electrical contact is established.

  First, the configuration of the lens unit 1 will be described. The lens unit 1 accommodates a photographing lens 11, a lens driving unit 12, a diaphragm 13, a diaphragm driving unit 14, a lens memory 15, and a lens microcomputer 16. The lens driving unit 12, the aperture driving unit 14, and the lens memory 15 are each connected to a lens microcomputer 16.

  The photographing lens 11 includes a plurality of lenses including a zoom lens and a focusing lens. The lens position of the taking lens 11 is adjusted by the lens driving unit 12 in the optical axis direction. The lens driving unit 12 has a built-in encoder for detecting the lens position.

  In FIG. 1, the photographic lens 11 is illustrated as a single lens for simplicity, but the actual photographic lens 11 has a plurality of lens groups, and the lens driving unit 12 arbitrarily selects one of them. It is possible to move.

  The diaphragm 13 adjusts the amount of light per unit time incident on the camera body 2. The aperture amount (aperture value) of the diaphragm 13 is adjusted by the diaphragm driver 14.

  The lens memory 15 is a non-volatile storage medium. The lens memory 15 stores table data indicating the correspondence between the lens position, the focal length of the photographing lens 11 and the photographing distance.

  The lens microcomputer 16 acquires the lens position of the taking lens 11 by the encoder of the lens driving unit 12. Further, the lens microcomputer 16 refers to the table data in the lens memory 15 to obtain the focal length of the photographing lens 11 and the photographing distance at the time of photographing from the above lens position. Then, the lens microcomputer 16 communicates with the camera body 2 through the electrical contacts described above, and information such as the type of the photographing lens 11, the focal length of the photographing lens 11, the photographing distance, the aperture value, and the like. Output to.

  Next, the configuration of the camera body 2 will be described. The camera body 2 includes an image sensor 21, an AFE 22, a CPU 23, a first memory 24, a second memory 25, a media I / F 26, and a release button 27. Note that the AFE 22, the first memory 24, the second memory 25, the media I / F 26, and the release button 27 are connected to the CPU 23, respectively.

  The imaging element 21 captures an image formed by the light flux that has passed through the photographing lens 11 and generates an image signal of the captured image. A plurality of light receiving elements are arranged in a matrix on the imaging surface of the imaging element 21. In addition, red (R), green (G), and blue (B) color filters are arranged in each light receiving element of the imaging element 21 in accordance with a known Bayer array. Therefore, each light receiving element of the imaging element 21 outputs an image signal corresponding to each color by color separation in the color filter. Note that the output of the image sensor 21 is connected to the AFE 22.

  Further, the image sensor 21 in the shooting mode captures a still image for recording (main image) in response to a full pressing operation of the release button 27. Further, the imaging element 21 in the imaging mode continuously captures images for observation (through images) at predetermined intervals even during imaging standby. Here, the data of the through image acquired in time series is used for various arithmetic processes by the CPU 23.

  The AFE 22 is an analog front end circuit that performs analog signal processing on the output of the image sensor 21. The AFE 22 performs correlated double sampling, image signal gain adjustment, and image signal A / D conversion.

  The CPU 23 is a processor that performs overall control of the camera body 2. For example, the CPU 23 executes known AF (autofocus) calculation processing and AE (automatic exposure) calculation processing using the through image data, respectively. The CPU 23 functions as the image processing unit 28 and the corrected image generation unit 29 by executing the program.

  The image processing unit 28 performs various types of image processing (color interpolation processing, gradation conversion processing, contour enhancement processing, white balance adjustment, etc.) on the digital image signal. Further, the image processing unit 28 performs magnification adjustment processing, distortion aberration adjustment processing, blur correction processing, and brightness adjustment processing in the aberration correction mode that is one of the photographing modes (magnification adjustment processing, distortion aberration adjustment processing, blurring). The details of the correction process and the brightness adjustment process will be described later).

  In the aberration correction mode, the corrected image generation unit 29 acquires image information from a plurality of captured images with different shooting conditions, and generates a corrected image in which the aberration of the photographing lens 11 is corrected by combining the plurality of image information. Note that, in the aberration correction mode in one embodiment, there are two types of submodes: a first mode for correcting axial chromatic aberration and a second mode for correcting field curvature aberration.

  The first memory 24 is a buffer memory that temporarily stores image data in the pre-process and post-process of image processing by the CPU 23. For example, the first memory 24 is configured by an SDRAM that is a volatile storage medium.

  The second memory 25 is composed of a nonvolatile storage medium such as a flash memory. The second memory 25 stores a program executed by the CPU 23. An operation example in the aberration correction mode by this program will be described later.

  The second memory 25 stores axial chromatic aberration data and field curvature aberration data of the lens unit 1 that can be attached to the camera body 2. Of course, the axial chromatic aberration data and the field curvature aberration data can be stored in the second memory 25 for each of the plurality of lens units 1 for each type of lens unit 1.

  Here, the axial chromatic aberration data is data for obtaining the focal length (fr, fg, fb) of the photographing lens 11 when photographing each image in the bracketing photographing executed in the first mode. It is. The field curvature aberration data is data for determining the focal length of the photographing lens 11 when photographing each image in the bracketing photographing performed in the second mode (axial chromatic aberration data). The specific contents of the field curvature aberration data will be described later).

  The parameters of the longitudinal chromatic aberration data and the field curvature aberration data are determined by the optical design of the photographing lens 11 and are different for each type of the photographing lens 11. The axial chromatic aberration data and the field curvature aberration data are prepared in advance by the manufacturer of the lens unit 1.

  The media I / F 26 has a connector to which the storage medium 3 can be detachably attached. The media I / F 26 writes / reads data to / from the storage medium 3 attached to the connector. The storage medium 3 is composed of a hard disk, a memory card incorporating a semiconductor memory, or the like. In FIG. 1, a memory card is illustrated as an example of the storage medium 3.

  The release button 27 receives from the user an instruction input for starting an AF operation before shooting by a half-press operation and an instruction input for starting an imaging operation by a full-press operation.

<Operation example 1 of aberration correction mode>
Next, an operation example of the aberration correction mode (first mode) in one embodiment will be described with reference to the flowchart of FIG.

  The in-focus positions of the R, G, and B pixels of the electronic camera are different depending on the axial chromatic aberration of the photographing lens 11. For example, when the camera body 2 focuses on the main subject, an image of the main subject with a reference wavelength (for example, d line) is formed on the imaging surface of the image sensor 21. At this time, the main wavelength (λr, λg, λb) images of the R, G, B pixels are not formed on the imaging surface (focal position of the photographing lens 11) due to the influence of axial chromatic aberration, and dfr, dfg, respectively. , Dfb away from the imaging surface (see FIG. 3).

  For this reason, even when focusing is performed so that an arbitrary wavelength is in focus at the time of shooting, an image shot under one shooting condition may cause color blur such as red or blue due to axial chromatic aberration. Even if the photographic lens 11 is designed so that the axial chromatic aberration is as small as possible, in reality, it is difficult to suppress all aberrations to zero. Depending on the type of shooting lens 11 and shooting conditions, there is a problem that axial chromatic aberration is conspicuous in the shot image.

  Therefore, in the operation example of FIG. 2, the CPU 23 generates a corrected image in which the axial chromatic aberration is corrected using a plurality of picked-up images captured by changing the power of the photographing lens 11 (reciprocal of the focal length). To do. The series of processes shown in FIG. 2 is started when the CPU 23 executes a program in response to a half-press operation of the release button 27.

  Step S101: The CPU 23 executes AF calculation processing and AE calculation processing. At this time, the CPU 23 communicates with the lens microcomputer 16 to acquire information on the shooting conditions. The information on the photographing conditions in S101 includes information on the type of the photographing lens 11, the focal length of the photographing lens 11, and the photographing distance. Note that the camera body 2 in S101 is in a state where an image of the main subject at the reference wavelength is formed on the imaging surface of the imaging device 21.

  Step S102: The CPU 23 uses the axial chromatic aberration data in the second memory 25 to determine the shooting conditions for each image in bracketing shooting.

  As described above, the CPU 23 changes the focal length f and performs bracketing shooting, so that images of the main wavelengths (λr, λg, λb) of the R, G, and B pixels are formed on the imaging surface. Need to get an image. Therefore, the CPU 23 in S102 uses the focal length (fr, fg, fb) of the photographing lens 11 when the above-mentioned main wavelengths (λr, λg, λb) are imaged on the imaging surface as photographing conditions for bracketing photographing. Ask. Note that the CPU 23 in S102 applies the axial chromatic aberration data corresponding to the photographing lens 11 attached to the camera body 2 by using the type information (S101) of the photographing lens 11. In addition, the main wavelengths (λr, λg, λb) of the R, G, and B pixels are the peak wavelength of the spectral sensitivity of each pixel, the central wavelength of the spectral sensitivity of each pixel, and the spectral sensitivity of each pixel. A combined centroid wavelength or the like can be used.

  Here, the axial chromatic aberration data may indicate fr, fg, and fb in the form of a function (such as a coefficient value of the function) of the focal length of the photographing lens 11 and the photographing distance. In this case, the CPU 23 can obtain the values of fr, fg, and fb, which are shooting conditions for bracketing shooting, by substituting the focal length and shooting distance values acquired in S101 into the above functions.

  Further, the axial chromatic aberration data may be stored in a table format with values of fr, fg, and fb with respect to the combination of the focal length and the photographing distance of the photographing lens 11. In this case, the CPU 23 may read the values of fr, fg, and fb corresponding to the combination of the focal length and the shooting distance acquired in S101 from the axial chromatic aberration data. If the values of fr, fg, and fb corresponding to the focal length and shooting distance in S101 are not in the axial chromatic aberration data, the CPU 23 selects a plurality of data with conditions close to the focal length and shooting distance in S101 as samples. And CPU23 should just obtain | require the value of fr, fg, fb by interpolation from these samples.

  Step S103: The CPU 23 determines whether or not the release button 27 has been fully pressed. If the above requirement is satisfied (YES side), the CPU 23 shifts the process to S104. On the other hand, when the above requirement is not satisfied (NO side), the CPU 23 waits for a full pressing operation of the release button 27.

  Step S104: The CPU 23 sequentially sets the focal length of the taking lens 11 to fr, fg, and fb, and drives the image sensor 21 to execute bracketing shooting. The data of each captured image output from the image sensor 21 by the bracketing shooting is subjected to predetermined image processing by the image processing unit 28 via the AFE 22, and then is in a state before being subjected to color interpolation. 24 is buffered.

  Further, the CPU 23 performs shooting by matching the exposure conditions of the respective images during the bracketing shooting. Moreover, in order to suppress the influence of subject blurring and illumination change, the CPU 23 preferably performs continuous shooting (continuous shooting) at high speed.

  Of the images acquired by the bracketing shooting in S104, an image shot at the focal length fr (an image formed on the imaging surface at the main wavelength λr of the R pixel) is denoted as Ir. Also, an image photographed at the focal length fg (image formed on the imaging surface at the main wavelength λg of the G pixel) is denoted as Ig. In addition, an image photographed at the focal length fb (image formed on the imaging surface at the main wavelength λb of the B pixel) is denoted as Ib.

  Step S105: The image processing unit 28 executes a magnification adjustment process for adjusting a difference in photographing magnification among the images Ir, Ig, and Ib.

  When bracketing photographing is performed by changing the focal length of the photographing lens 11, the magnification βr in the image Ir, the magnification βg in the image Ig, and the magnification in the image Ib due to the difference in photographing conditions and the influence of lateral chromatic aberration. The magnification βb is different from each other. Therefore, the image processing unit 28 in S105 scales the image Ir to βg / βr and scales the image Ib to βg / βb. Thereby, the shift | offset | difference of the pattern by the magnification difference of image Ir, Ig, Ib is suppressed.

  Here, the acquisition of βg / βr and βg / βb is performed as follows. For example, a table showing the correspondence between each value of βg / βr, βg / βb, the type of the photographing lens 11, the focal length of the photographing lens 11, and the photographing distance is stored in the second memory 25 in advance, and image processing is performed. The unit 28 may acquire the values of βg / βr and βg / βb from the above table using the information on the photographing conditions in S101. If there is no value corresponding to the shooting condition in S101 in the above table, the image processing unit 28 selects a plurality of data having conditions close to the shooting condition in S101 as samples. Then, the image processing unit 28 may obtain the values of βg / βr and βg / βb from these samples by interpolation. FIG. 4 is a schematic diagram showing a difference in photographing magnification due to a difference in wavelength.

  Alternatively, the values of βg / βr and βg / βb may be stored in the second memory 25 in the form of a function (such as a coefficient value of the function) of the focal length of the photographing lens 11 and the photographing distance. In this case, the CPU 23 can obtain the values of βg / βr and βg / βb by substituting the photographing condition of S101 into the above function.

  Note that if the difference in shooting magnification between images is sufficiently small (for example, if the pattern shift due to the shooting magnification is less than one pixel), the image processing unit 28 may omit the process of S105. .

  Step S106: The image processing unit 28 performs a distortion aberration adjustment process for adjusting the distortion aberration difference among the images Ir, Ig, and Ib.

  When bracketing photographing is performed by changing the focal length of the photographing lens 11, the distortion aberration dr in the image Ir and the distortion aberration dg in the image Ig according to the difference in photographing conditions and the difference in distortion due to color. And the distortion aberration db in the image Ib are different from each other. Therefore, the image processing unit 28 in S106 corrects, for example, the distortion aberration of the image Ir and the distortion aberration of the image Ib to be equal to the distortion aberration of the image Ig, respectively. Thereby, the shift | offset | difference of the pattern by the difference in the distortion aberration of image Ir, Ig, Ib is suppressed.

  Note that when the difference in distortion between images is sufficiently small (for example, when the picture shift due to distortion is less than one pixel), the image processing unit 28 may omit the process of S106. .

  Step S107: The image processing unit 28 executes a shake correction process for correcting the shake of the subject between the images Ir, Ig, and Ib.

  Even when the above bracketing shooting is performed by high-speed continuous shooting, there is a case where the pattern is shifted between images due to shooting blurring or the like. Therefore, the image processing unit 28 in S107 performs alignment between the images Ir, Ig, and Ib by blur correction processing.

  For example, the image processing unit 28 performs edge extraction processing on each of the images Ir, Ig, and Ib. Then, the image processing unit 28 aligns the images by performing known matching between the images using the edges or corners of the images.

  Step S108: The image processing unit 28 executes brightness adjustment processing for adjusting the difference in image brightness between the images Ir, Ig, and Ib.

  In the above bracketing shooting, the brightness may be different between images due to a change in peripheral dimming due to a difference in shooting conditions or a change in illumination state between images. Therefore, the image processing unit 28 in S108 compares the brightness of the subject at the corresponding position between the images Ir, Ig, and Ib. When the brightness differs at corresponding positions between the images, the image processing unit 28 corrects the gain of the image Ir or Ib based on the brightness of the image Ig, for example. Thereby, the brightness of the same part can be match | combined with image Ir, Ig, and Ib.

  Step S109: The corrected image generation unit 29 acquires image information corresponding to each color of the color filter from the images Ir, Ig, and Ib.

  Specifically, the corrected image generation unit 29 extracts only image information of R pixels from the image Ir. In addition, the corrected image generation unit 29 extracts only image information of G pixels from the image Ig. Further, the corrected image generation unit 29 extracts only the image information of the B pixel from the image Ib (see FIG. 5). Therefore, the image information of each color pixel has information on pixel positions different from each other in a mosaic pattern conforming to the Bayer arrangement.

  Step S110: The corrected image generation unit 29 generates corrected image data by combining the image information of each color acquired in S109 (see FIG. 5).

  Here, the image information of the R, G, and B pixels is generated from an image acquired under the condition that the dominant wavelength of each color forms an image on the imaging surface. Therefore, the corrected image generation unit 29 can obtain a corrected image for one frame in which the axial chromatic aberration is corrected by combining these pieces of image information.

  Then, the image processing unit 28 performs various types of image processing including color interpolation on the corrected image data. Thereafter, the corrected image data is finally recorded on the storage medium 3 mounted on the media I / F 26 under the control of the CPU 23. The CPU 23 may reproduce and display the corrected image on a monitor (not shown). This is the end of the description of the flowchart of FIG.

  As described above, the electronic camera system according to the embodiment can acquire a corrected image in which the axial chromatic aberration is corrected by using a plurality of captured images obtained by bracketing imaging with the focal length of the imaging lens 11 adjusted.

<Example 2 of aberration correction processing>
Next, an operation example of the aberration correction mode (second mode) in one embodiment will be described with reference to the flowchart of FIG.

  Due to the field curvature aberration of the photographing lens 11, when photographing is performed under photographing conditions in which focusing is performed at the center of the optical axis, the peripheral portion of the image is out of focus (see FIG. 7). On the other hand, when shooting is performed under shooting conditions that focus on the periphery of the image, the image is out of focus at the center of the image. Even if the photographic lens 11 is designed so as to reduce the field curvature aberration as much as possible, it is difficult to suppress all aberrations to zero. Then, depending on the type of shooting lens 11 and shooting conditions, there has been a problem that field curvature aberration is conspicuous in the shot image.

  Therefore, in the operation example of FIG. 6, the CPU 23 uses the plurality of captured images obtained by bracketing imaging while changing the power (reciprocal of the focal length) of the imaging lens 11 to correct the corrected image in which the field curvature aberration is corrected. Generate. The series of processing shown in FIG. 6 is started when the CPU 23 executes a program in response to a half-press operation of the release button 27.

  Step S201: The CPU 23 executes AF calculation processing and AE calculation processing. At this time, the CPU 23 communicates with the lens microcomputer 16 to acquire information on the shooting conditions. The information on the photographing conditions in S101 includes information on the type of the photographing lens 11, the focal length of the photographing lens 11, the photographing distance, and the aperture value.

  Step S202: The CPU 23 uses the curvature of field data of the second memory 25 to determine the shooting conditions for each image in bracketing shooting.

  In this second mode, the CPU 23 performs bracketing shooting by changing the focal length f, whereby a plurality of images having different image heights (distances from points corresponding to the optical axis center) formed on the imaging surface are different. Need to get. Therefore, the CPU 23 in S202 obtains the focal length of the photographing lens 11 in bracketing photographing so that the image heights formed on the imaging surface are different in consideration of the curvature of field of the photographing lens 11. FIG. 8 schematically shows an example in which the curvature of field at a predetermined image height is corrected by adjusting the focal length f.

  The CPU 23 in S202 applies field curvature aberration data corresponding to the photographic lens 11 attached to the camera body 2 by using the type information (S201) of the photographic lens 11.

  Here, the field curvature aberration data is stored in the form of a function (such as a coefficient value of the function) of the focal length, the photographing distance, and the aperture value of the photographing lens 11 and a correspondence relationship with the image height formed on the imaging surface. You may do. In this case, the CPU 23 can obtain the position of the image height that forms an image on the imaging surface by substituting the focal length, the shooting distance, and the aperture value acquired in S201 into the above functions. Further, if the image height to be imaged on the imaging surface is designated, the CPU 23 can also obtain a focal length value corresponding to an arbitrary image height from the above function.

  Therefore, the CPU 23 in S202 may specify image heights at arbitrary intervals according to the number of times of bracketing shooting, and obtain a focal length value corresponding to each image height.

  Note that the field curvature aberration data may be a table in which the image height value formed on the imaging surface is stored in a table format with respect to the combination of the focal length, the imaging distance, and the aperture value of the imaging lens 11. . In this case as well, as in the case of the above function, the CPU 23 can obtain the value of the focal length applied in bracketing shooting.

  Step S203: The CPU 23 determines whether or not the release button 27 has been fully pressed. When the above requirement is satisfied (YES side), the CPU 23 shifts the process to S204. On the other hand, when the above requirement is not satisfied (NO side), the CPU 23 waits for a full pressing operation of the release button 27.

  Step S204: The CPU 23 sequentially sets the focal length of the photographing lens 11 to the value obtained in S202, and drives the image sensor 21 to execute a predetermined number of bracketing photographings. Data of each captured image output from the image sensor 21 by bracketing shooting is subjected to predetermined image processing by the image processing unit 28 via the AFE 22 and then buffered in the first memory 24.

  Further, the CPU 23 performs shooting by matching the exposure conditions of the respective images during the bracketing shooting. Further, it is preferable that the CPU 23 performs continuous shooting at high speed in order to suppress the influence of subject blurring and illumination change. For simplicity, the images acquired by the bracketing shooting in S204 are denoted as images I1 to In.

  Step S205: The image processing unit 28 executes a magnification adjustment process for adjusting a difference in photographing magnification between the images I1 to In. The contents of the magnification adjustment processing in S205 are almost the same as the processing in S105 in the first mode, and therefore a duplicate description is omitted.

  Step S206: The image processing unit 28 executes a distortion aberration adjustment process for adjusting the difference in distortion between the images I1 to In. Since the content of the distortion aberration adjustment process in S206 is almost the same as the process in S106 of the first mode, a duplicate description is omitted.

  Step S207: The image processing unit 28 executes a shake correction process for correcting the shake of the subject between the images I1 to In. The details of the blur correction process in S207 are almost the same as the process in S107 in the first mode, and a duplicate description is omitted.

  Step S208: The image processing unit 28 executes brightness adjustment processing for adjusting the difference in image brightness between the images I1 to In. Since the content of the brightness adjustment process in S208 is almost the same as the process in S108 of the first mode, a duplicate description is omitted.

  Step S209: The corrected image generation unit 29 generates image information from the images I1 to In. Specifically, the corrected image generation unit 29 generates image information by cutting out the range of the image height where the field curvature aberration can be regarded as 0 in each of the images I1 to In into a circular shape or an annular shape.

  Step S210: The corrected image generation unit 29 generates corrected image data by combining the image information acquired in S209. At this time, the corrected image generation unit 29 performs image composition by extracting pixel values at each position of the corrected image from the image information of the photographing condition in which the field curvature aberration at each image height can be regarded as zero. Note that, for a portion where there is no image information of a photographing condition in which the field curvature aberration can be regarded as 0, the corrected image generation unit 29 estimates a pixel value at the portion using a plurality of pieces of image information photographed under a close photographing condition. do it.

  Here, the image information acquired in S209 is composed of images in a range of image height where the field curvature aberration can be regarded as zero. Therefore, the corrected image generation unit 29 can obtain a corrected image for one frame in which the field curvature aberration is corrected by combining these pieces of image information.

  Then, the image processing unit 28 performs various types of image processing on the corrected image data. Thereafter, the corrected image data is finally recorded on the storage medium 3 mounted on the media I / F 26 under the control of the CPU 23. The CPU 23 may reproduce and display the corrected image on a monitor (not shown). Above, description of the flowchart of FIG. 6 is complete | finished.

  As described above, the electronic camera system according to the embodiment can acquire a corrected image in which the field curvature aberration is corrected by using a plurality of captured images that are bracketed by adjusting the focal length of the photographic lens 11. .

<Modification of one embodiment>
In the aberration correction mode operation example in FIGS. 2 and 6 described above, the example in which only the focal length of the photographing lens is changed during bracketing photographing has been described. However, the CPU 23 changes the position of the photographic lens 11 or the photographic distance parameter by moving one of the lens groups constituting the photographic lens 11 during bracketing photographing in the aberration correction mode. Also good. As described above, even when the position of the photographing lens 11 or the parameter of the photographing distance is changed, substantially the same effect as in the case of the operation example shown in FIG. 2 or FIG. 6 can be obtained.

  For example, in the first mode of the aberration correction mode, when bracketing shooting is performed by changing the position of the shooting lens 11 or the parameter of the shooting distance, the CPU 23 first sets the amount of axial chromatic aberration (dfr, dfg, dfb) is obtained. Next, the CPU 23 may determine the movement amounts of the photographing lens 11 corresponding to the amount of axial chromatic aberration, and perform bracketing photographing using the movement amounts of these lenses.

  Similarly, in the second mode of the aberration correction mode, when bracketing photographing is performed by changing the position of the photographing lens 11 or the parameter of the photographing distance, the CPU 23 uses the field curvature aberration data to set an arbitrary image height. What is necessary is just to obtain | require the corresponding imaging distance and the position of a focus lens.

  Here, when adjusting an imaging distance or a lens position by driving an arbitrary lens group of the imaging lens 11, if the lens group to be driven is an inner lens, when shooting a plurality of images with different imaging conditions In addition, the electronic camera system can suppress the rotation of the hood and filter attached to the lens unit 1.

  In addition, when the lens group composed of a plurality of lenses is moved when adjusting the shooting distance or the like, the correction of aberrations becomes easier in terms of optical design than when a single lens is moved.

  Furthermore, the electronic camera system of one embodiment may be configured to shift the position of the image sensor 21 in the optical axis direction and adjust the shooting conditions for bracketing shooting.

  Further, the CPU 23 may execute a combination of the change of the focal length of the photographing lens 11 and the change of the position of the photographing lens 11 or the photographing distance at the time of the bracketing photographing. In this case, even in a case where it is difficult to correct aberrations with one parameter, the CPU 23 can acquire each captured image under more appropriate shooting conditions.

<Description of other embodiments>
FIG. 9 is a block diagram illustrating a configuration example of an image processing apparatus according to another embodiment. In another embodiment, the function of the image processing apparatus is realized by causing a computer to execute an image processing program. Therefore, in the configuration of the other embodiment, substantially the same effect as the electronic camera system of the above-described one embodiment can be obtained.

  A computer 31 constituting the image processing apparatus includes a data reading unit 32, a storage device 33, a CPU 34, a memory 35, an input / output I / F 36, and a bus 37. The data reading unit 32, the storage device 33, the CPU 34, the memory 35, and the input / output I / F 36 are connected to each other via a bus 37. Furthermore, an input device 38 (keyboard, pointing device, etc.) and a monitor 39 are connected to the computer 31 via an input / output I / F 36. The input / output I / F 36 accepts various inputs from the input device 38 and outputs display data to the monitor 39.

  The data reading unit 32 is used when reading captured image data and an image processing program from the outside. For example, the data reading unit 32 is a reading device (such as a reading device for an optical disk, a magnetic disk, or a magneto-optical disk) that acquires data from a removable storage medium, or an external device (camera body) according to a known communication standard. 2) and other communication devices (USB interface, LAN module, wireless LAN module, etc.).

  The storage device 33 stores the above-described image processing program and various data (such as axial chromatic aberration data and field curvature aberration data) necessary for executing the program. The storage device 33 can also record captured image data acquired from the data reading unit 32. Note that the storage device 33 in another embodiment is configured by a hard disk, a nonvolatile semiconductor memory, or the like.

  The CPU 34 is a processor that comprehensively controls the operation of each part of the computer 31. In another embodiment, the CPU 34 executes the image processing program, so that the operations of the CPU 23, the image processing unit 28, and the corrected image generation unit 29 in one embodiment are realized in software. The memory 35 temporarily stores the calculation result of the image processing program. The memory 35 is composed of, for example, a volatile SDRAM.

  Here, when the operation of the aberration correction mode described in one embodiment is realized by the computer 31 according to another embodiment, a plurality of images photographed under a condition in which the axial chromatic aberration is corrected, or images that are different from each other. A plurality of images having a high field curvature aberration of 0 must be taken in advance with an electronic camera. Then, the computer 31 can perform the same processing as that of the first embodiment by reading an image file in which each data of the captured image described above is associated with information on the shooting condition from the data reading unit 32. It becomes.

<Supplementary items of the embodiment>
(1) In the above embodiment, an example of an electronic camera system with interchangeable lenses has been described. However, the present invention can naturally be applied to a so-called lens-integrated electronic camera.

  (2) If the image pickup device 21 of the electronic camera is configured to be addressable and readable (such as CMOS), the image pickup device 21 generates image information when shooting the images Ir, Ig, and Ib in the first mode. Only a signal of a necessary color may be output.

  (3) In the one embodiment described above, the reference wavelength may be the main wavelength of any pixel.

  (4) The image processing apparatus described in the above embodiment is merely an example, and when implementing the present invention, all combinations of the configurations and the processing order shown in the above embodiment and the modification can be taken. For example, the image processing apparatus may appropriately change the execution order of the magnification adjustment process, the distortion aberration adjustment process, the shake correction process, and the brightness adjustment process in the aberration correction mode. In addition, the image processing apparatus generates a corrected image in which both the axial chromatic aberration and the field curvature aberration are corrected by combining the processing in the first mode shown in FIG. 2 and the processing in the second mode shown in FIG. May be.

  (5) The image processing apparatus that executes the program of the present invention is not limited to the examples of the camera body 2 and the computer 31 of the above-described embodiment, and is widely applicable to electronic devices such as a mobile phone with a camera and an image viewer. it can.

  (6) In each of the above-described embodiments, the example in which the functions of the image processing unit 28 and the corrected image generation unit 29 are implemented by software has been described. However, these processing may be implemented by hardware using an ASIC. Of course it doesn't matter.

  From the above detailed description, features and advantages of the embodiments will become apparent. It is intended that the scope of the claims extend to the features and advantages of the embodiments as described above without departing from the spirit and scope of the right. Further, any person having ordinary knowledge in the technical field should be able to easily come up with any improvements and modifications, and there is no intention to limit the scope of the embodiments having the invention to those described above. It is also possible to use appropriate improvements and equivalents within the scope disclosed in.

1 is a schematic diagram illustrating a configuration example of an electronic camera system according to an embodiment. 6 is a flowchart showing an operation example of an aberration correction mode (first mode) in one embodiment. Explanatory drawing showing an example of axial chromatic aberration Schematic diagram showing the difference in shooting magnification due to the difference in wavelength Schematic diagram showing a method of generating a corrected image in the first mode 6 is a flowchart showing an operation example of an aberration correction mode (second mode) in one embodiment. Explanatory drawing showing an example of field curvature aberration Schematic diagram showing an example of correcting field curvature at a predetermined image height by adjusting the focal length f Schematic diagram showing a configuration example of an image processing apparatus according to another embodiment

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Lens unit, 2 ... Camera body, 11 ... Shooting lens, 12 ... Lens drive part, 13 ... Aperture, 16 ... Lens microcomputer, 21 ... Image sensor, 23 ... CPU, 25 ... 2nd memory, 28 ... Image processing part , 29 ... corrected image generation unit, 31 ... computer

Claims (10)

An image reading unit that obtains a plurality of captured images each captured by changing at least one parameter of a focal length of the photographing lens, a photographing distance, and an arrangement of a part of the lens group of the photographing lens;
An image processing unit for performing image processing of the captured image;
A correction image generation unit that obtains image information from each of the captured images, and generates a correction image in which the aberration of the photographing lens is corrected by combining a plurality of the image information ,
The amount of change in shooting conditions in the plurality of captured images is determined based on the type of the shooting lens, the focal length of the shooting lens, and the shooting distance .
The captured image includes information on a plurality of color components,
The plurality of captured images are images captured by changing the parameter according to a shift in an imaging position of the photographing lens in light having a wavelength corresponding to each of the color components,
The image processing unit includes a magnification adjustment process that adjusts a difference in photographing magnification between the captured images, a distortion aberration adjustment process that adjusts a difference in distortion aberration between the captured images, and a subject blur between the captured images. At least one of a shake correction process for correcting the brightness and a brightness adjustment process for adjusting a difference in image brightness between the captured images,
The image processing apparatus, wherein the correction image generation unit acquires the image information corresponding to the different color components from each captured image, and generates the correction image in which axial chromatic aberration is corrected . An image reading unit that obtains a plurality of captured images each captured by changing at least one parameter of a focal length of the photographing lens, a photographing distance, and an arrangement of a part of the lens group of the photographing lens;
An image processing unit for performing image processing of the captured image;
A correction image generation unit that obtains image information from each of the captured images, and generates a correction image in which the aberration of the photographing lens is corrected by combining a plurality of the image information ,
The amount of change in shooting conditions in the plurality of captured images is determined based on the type of the shooting lens, the focal length of the shooting lens, the shooting distance, and the aperture value of the shooting lens ,
The image processing unit includes a magnification adjustment process that adjusts a difference in photographing magnification between the captured images, a distortion aberration adjustment process that adjusts a difference in distortion aberration between the captured images, and a subject blur between the captured images. At least one of a shake correction process for correcting the brightness and a brightness adjustment process for adjusting a difference in image brightness between the captured images,
The correction image generation unit acquires the image information from a position where the image heights differ between the captured images, and generates the correction image in which the field curvature aberration is corrected. . An image reading unit that acquires a plurality of captured images taken by bracketing by changing the focal length of the photographing lens;
An image processing unit that executes a brightness adjustment process for adjusting a difference in image brightness between the captured images when the brightness differs between the captured images;
A correction image generation unit that obtains image information from each of the captured images, and generates a correction image in which the aberration of the photographing lens is corrected by combining a plurality of the image information,
The image processing unit includes: a first captured image that forms an image on the imaging surface with the main wavelength of the R pixel; a second captured image that forms an image on the imaging surface with the main wavelength of the G pixel; an imaging surface with the main wavelength of the B pixel An image processing apparatus that adjusts a difference in image brightness between third captured images formed on the image. The image processing apparatus according to claim 3 .
The amount of change in shooting condition in the plurality of captured images is determined based on at least one of the type of the shooting lens, the focal length of the shooting lens, the shooting distance, and the aperture value of the shooting lens. Image processing device. The image processing apparatus according to claim 3 or 4,
The captured image includes information on a plurality of color components,
The plurality of captured images are images captured by changing the focal length according to a shift in the imaging position of the photographing lens in light having a wavelength corresponding to each of the color components,
The image processing apparatus, wherein the correction image generation unit acquires the image information corresponding to the different color components from each captured image, and generates the correction image in which axial chromatic aberration is corrected . The image processing apparatus according to claim 3 or 4,
The correction image generation unit acquires the image information from a position where the image heights differ between the captured images, and generates the correction image in which the field curvature aberration is corrected. . The image processing apparatus according to any one of claims 3 to 6,
The image processing unit includes a magnification adjustment process that adjusts a difference in photographing magnification between the captured images, a distortion aberration adjustment process that adjusts a difference in distortion aberration between the captured images, and a subject blur between the captured images. An image processing apparatus that further executes at least one of a blur correction process for correcting the image. An image reading process for acquiring a plurality of captured images taken by bracketing by changing the focal length of the taking lens ;
When the brightness differs between the captured images, a first captured image that forms an image on the imaging surface with the main wavelength of the R pixel, a second captured image that forms an image on the imaging surface with the main wavelength of the G pixel, and a B pixel Image processing for performing brightness adjustment processing for adjusting a difference in image brightness between the third captured images formed on the imaging surface at the principal wavelength of
Correction image generation processing for acquiring image information from each of the captured images and generating a correction image in which the aberration of the photographing lens is corrected by combining a plurality of the image information;
A program that causes a computer to execute. An imaging step of imaging a plurality of captured images taken by bracketing by changing the focal length of the imaging lens;
When the brightness differs between the captured images, a first captured image that forms an image on the imaging surface with the main wavelength of the R pixel, a second captured image that forms an image on the imaging surface with the main wavelength of the G pixel, and a B pixel An image processing step for performing image processing for performing brightness adjustment processing for adjusting a difference in image brightness between the third captured images formed on the imaging surface at the principal wavelength of
A corrected image generation step of acquiring image information from each of the captured images and generating a corrected image in which the aberration of the photographing lens is corrected by combining a plurality of the image information;
An image processing method comprising: An imaging unit for imaging an image formed by a light beam that has passed through the taking lens;
The image processing apparatus according to any one of claims 1 to 3,
An imaging apparatus comprising:

Images