JP5034117B2 - Imaging apparatus, imaging method, and program - Google Patents

Description

  The present invention relates to an imaging apparatus, an imaging method, and a program. In particular, the present invention relates to an imaging device and an imaging method for capturing an image, and a program for the imaging device.

Optical masks that make the optical transfer function essentially constant within a certain range from the focal position are known (see, for example, Patent Documents 1 and 2).
US Pat. No. 5,748,371 US Patent Application Publication No. 2002/0118457

  According to the optical mask described in the above patent document, the optical transfer function can be made essentially constant within a certain range from the focal position. However, even within that range, the optical transfer function slightly changes according to the distance. In addition, it is difficult to make the optical transfer function constant for a short-distance object such as a macro mode and a distant object such as a landscape when a landscape image is captured. If the same restoration process is performed on such a subject, the subject image may not be restored properly.

  In order to solve the above-described problem, according to a first aspect of the present invention, there is provided an imaging device that captures a captured image of a subject from each position within a predetermined positional relationship with the imaging device. An optical system in which the light is received by the light receiving unit of the imaging device with substantially the same spread, and the optical transfer function is different for light from different positions within a predetermined positional relationship with the imaging device, and optical transmission A processing parameter storage unit that stores processing parameters that can correct the influence of a function on a captured image in association with a condition relating to a positional relationship between an object and an optical system that should be corrected when the processing parameters are corrected. And a positional information acquisition unit that acquires positional information indicating the positional relationship between the subject and the optical system, and a process that is associated with a condition that the positional relationship indicated by the positional information acquired by the positional information acquisition unit matches And a processing parameter selection section for selecting the processing parameters parameter storing portion is storing.

  The optical system receives light from each position within a predetermined distance range from the optical system by the light receiving unit with substantially the same spread, and from different positions within a predetermined distance range from the optical system. An optical transfer function is different for light, and the processing parameter storage unit is an optical system that should be adapted when a processing parameter that can correct the influence of the optical transfer function on a captured image is corrected by the processing parameter. The position information acquisition unit acquires distance information indicating the distance between the subject and the optical system, and the processing parameter selection unit acquires the distance information acquired by the position information acquisition unit. The processing parameter stored in the processing parameter storage unit may be selected in association with the condition that the distance indicated by is suitable.

  An image processing unit that corrects the influence of the optical transfer function on the captured image using the processing parameter selected by the processing parameter selection unit may be further provided. The optical system may cause the light receiving unit to receive light from a position within a range of a predetermined distance from the optical system with substantially the same spread due to lateral aberration.

  The position information acquisition unit acquires an operation mode for handling a captured image obtained by imaging a subject at a short distance or an operation mode for handling a captured image obtained by imaging a subject at a long distance, and a processing parameter storage unit applies to the captured image by an optical transfer function. A processing parameter capable of correcting the influence is stored in association with the operation mode of the imaging device when the imaging device handles a captured image to be corrected with the processing parameter, and the processing parameter selection unit acquires position information The processing parameter stored in the processing parameter storage unit may be selected in association with the operation mode acquired by the unit.

  According to the second aspect of the present invention, the light from each position within the range of a predetermined positional relationship with the imaging device that captures the captured image of the subject is received by the light receiving unit of the imaging device with substantially the same spread. An imaging method for imaging using an optical system having a different optical transfer function for light from different positions within a predetermined positional relationship with the imaging device, and A processing parameter storage stage for storing processing parameters capable of correcting the influence of the processing parameters in association with conditions relating to the positional relationship between the subject and the optical system to be adapted when the processing parameters are corrected, and the subject and the optical Processing is performed in association with a condition in which the positional information acquisition stage for acquiring positional information indicating the positional relation between the system and the positional relation indicated by the positional information acquired in the positional information acquisition stage is suitable. And a processing parameter selection step of selecting the processing parameter stored in the parameter storage step.

  According to the third aspect of the present invention, the light from each position within the range of a predetermined positional relationship with the imaging device is received by the light receiving unit of the imaging device with substantially the same spread, and the imaging device is determined in advance. A program for an imaging apparatus that has an optical system having different optical transfer functions for light from different positions within a specified positional relationship, and that captures a captured image of a subject. Processing parameter storage that stores processing parameters that can correct the influence of the transfer function on the captured image in association with conditions regarding the positional relationship between the subject and the optical system that should be matched when the processing parameters are corrected. A positional information acquisition unit that acquires positional information indicating a positional relationship between the subject and the optical system, and a process that is performed in association with a condition that the positional relationship indicated by the positional information acquired by the positional information acquisition unit is suitable Parameter storage section to function as a processing parameter selection unit that selects the processing parameters are stored.

  The above summary of the invention does not enumerate all the necessary features of the present invention, and sub-combinations of these feature groups can also be the invention.

  As is apparent from the above description, according to the present invention, the subject image can be properly restored.

  Hereinafter, the present invention will be described through embodiments of the invention. However, the following embodiments do not limit the invention according to the scope of claims, and all combinations of features described in the embodiments are included. It is not necessarily essential for the solution of the invention.

  FIG. 1 shows an example of the configuration of an imaging apparatus 100 according to an embodiment. The imaging apparatus 100 includes an imaging unit 105, an image processing unit 140, a display unit 150, an image recording unit 155, an optical characteristic specifying unit 160, an optical characteristic storage unit 165, a frequency characteristic calculation unit 170, an optical characteristic storage unit 165, and a processing parameter selection. A unit 180, a position information acquisition unit 190, a distance measurement unit 192, a captured image analysis unit 194, and an object region specifying unit 196.

  The imaging unit 105 forms an image based on the optical system 110 that forms light, the light receiving unit 120 that receives the light formed by the optical system 110, and the amount of light from the optical system 110 received by the light receiving unit 120. An image generation unit 130 for generation is included. The imaging device 100 captures an image of a subject and generates a captured image.

  The imaging unit 105 captures a captured image of the subject through the optical system 110. Specifically, the light receiving unit 120 includes a plurality of light receiving elements arranged two-dimensionally. The image generation unit 130 generates a captured image of the subject by A / D converting the amount of light received by each of the plurality of light receiving elements. Then, the image processing unit 140 corrects the captured image generated by the image generation unit 130 based on the A / D converted received light amount value, the position of each light receiving element, and the optical transfer function of the optical system 110. To generate a restored image. Then, the display unit 150 displays the restored image or the captured image generated by the processing by the image processing unit 140. The display unit 150 may be a display that allows the user to browse images.

  The optical system 110 has different optical transfer functions depending on the distance to the subject. Although the optical characteristics of the optical system 110 will be described in detail later, the optical transfer function of the optical system 110 is not significantly affected by the defocus. Therefore, the image processing unit 140 performs a restoration process on the captured image using a more appropriate one of several representative optical transfer functions of the optical system 110 corresponding to the distance to the subject.

  The optical system 110 causes the light receiving unit 120 of the imaging apparatus 100 to receive the light from each position within the range of a predetermined positional relationship with the imaging apparatus 100 with substantially the same spread. The optical system 110 has an optical transfer function different from that of light from different positions within a predetermined positional relationship with the imaging apparatus 100. The processing parameter storage unit 185 relates to the positional relationship between the subject and the optical system 110 that should be matched when processing parameters that can correct the influence of the optical transfer function on the captured image are corrected using the processing parameters. Store in association with the condition. Then, the position information acquisition unit 190 acquires position information indicating the positional relationship between the subject and the optical system 110. Then, the processing parameter selection unit 180 selects a processing parameter stored in the processing parameter storage unit 185 in association with a condition in which the positional relationship indicated by the positional information acquired by the positional information acquisition unit 190 matches.

  The positional relationship may be a distance between the optical system 110 and the subject. That is, the optical system 110 causes the light receiving unit 120 to receive light from each position within a predetermined distance from the optical system 110 with substantially the same spread. The optical system 110 has an optical transfer function that differs from light from different positions within a predetermined distance from the optical system 110. Then, the processing parameter storage unit 185 corresponds to the condition regarding the distance from the optical system 110 to be adapted when the processing parameter capable of correcting the influence of the optical transfer function on the captured image is corrected with the processing parameter. And store. The position information acquisition unit 190 acquires distance information indicating the distance between the subject and the optical system 110. Then, the processing parameter selection unit 180 selects the processing parameter stored in the processing parameter storage unit 185 in association with the condition that the distance indicated by the distance information acquired by the position information acquisition unit 190 is suitable.

  The positional relationship may include a distance between the optical axis and the subject in a plane perpendicular to the optical axis, in addition to the distance between the optical system 110 and the subject. In other words, the optical system 110 is within the range of a predetermined distance from the optical system 110, and is optical with respect to light from a position where the distance in the optical axis direction from the optical system 110 and the distance from the optical axis are different. The transfer function is different. The distance from the optical axis may be the length of a perpendicular line from the subject position to the extension line of the optical axis. Then, the processing parameter storage unit 185 is an image region that should be adapted when the processing parameter that can correct the influence of the optical transfer function on each of the plurality of image regions in the captured image is corrected with the processing parameter. The image from the optical axis point, which is the point on the captured image corresponding to the optical axis, that indicates the distance in the optical axis direction of the subject that is being imaged and the distance from the optical axis of the subject that is being imaged in the image area Stored in association with the conditions related to the distance of the area. Then, the position information acquisition unit 190 acquires the distance between the imaged subject and the optical system 110 and the distance from the optical axis point as position information for each of the plurality of image regions. Then, the processing parameter selection unit 180 associates the distance between the optical system 110 acquired by the position information acquisition unit 190 and the distance from the optical axis point with respect to each of the plurality of image regions. A processing parameter stored in the processing parameter storage unit 185 is selected.

  In addition to the distance between the optical system 110 and the subject and the distance between the optical axis and the subject in the plane perpendicular to the optical axis, the positional relationship is determined around the intersection of the vertical plane and the optical axis. An angle may be included. That is, the optical system 110 is within a predetermined distance range from the optical system 110, and relates to the distance in the optical axis direction from the optical system 110, the distance from the optical axis, and the intersection of the optical axis and the object plane. The optical transfer function is different for light from different positions. In addition, the processing parameter storage unit 185 is an image region that should be adapted when the processing parameter that can correct the influence of the optical transfer function on each of the plurality of image regions in the captured image is corrected with the processing parameter. Indicates the position of the image area with respect to the optical axis, indicating the distance in the optical axis direction of the subject being imaged, the distance of the image area from the optical axis point, and the angle with respect to the intersection of the subject being imaged in the image area Stored in association with the angle-related conditions. Then, for each of the plurality of image areas, the position information acquisition unit 190 uses the distance between the imaged subject and the optical system 110, the distance from the optical axis point, and the angle related to the optical axis point as position information. get. The processing parameter selection unit 180 adapts the distance to the optical system 110 acquired by the position information acquisition unit 190, the distance from the optical axis point, and the angle related to the optical axis point for each of the plurality of image regions. The processing parameter stored in the processing parameter storage unit 185 is selected in association with the condition to be processed.

  As described above, the optical system 110 is within a predetermined distance range from the optical system 110, and the distance of the optical system 110 from the optical system 110 in the optical axis direction and the positional relationship of the optical system 110 with respect to the optical axis are the same. For light from different positions, the optical transfer function is different. In addition, the processing parameter storage unit 185 is an image region that should be adapted when the processing parameter that can correct the influence of the optical transfer function on each of the plurality of image regions in the captured image is corrected with the processing parameter. Are stored in association with the conditions regarding the distance in the optical axis direction of the subject being imaged and the position of the image area in the captured image indicating the positional relationship of the optical system 110 with respect to the optical axis. The position information acquisition unit 190 acquires, as position information, the distance between the imaged subject and the optical system 110 and the position of the image area in the captured image for each of the plurality of image areas. Then, the processing parameter selection unit 180 stores processing parameters in association with the distance between the optical system 110 acquired by the position information acquisition unit 190 and the condition of the position of the image region for each of the plurality of image regions. The processing parameter stored in the unit 185 is selected. Then, the image processing unit 140 corrects the influence of the optical transfer function on each of the plurality of image regions by using the processing parameter selected by the processing parameter selection unit 180 for each of the plurality of image regions.

  The optical system 110 may cause the light receiving unit 120 to receive light from a position within a predetermined distance range from the optical system 110 with substantially the same spread due to lateral aberration. The image processing unit 140 corrects the influence of the optical transfer function on the captured image using the processing parameter selected by the processing parameter selection unit 180.

  In addition, the position information acquisition unit 190 acquires the distance to the subject measured by the distance measurement unit 192. The distance measuring unit 192 may be a laser distance sensor that acquires a distance to a subject with a laser. Then, the processing parameter storage unit 185 associates the processing parameter that can correct the influence of the optical transfer function on the captured image with the condition regarding the distance to the subject that should be matched when the processing parameter is corrected. Store. Then, the processing parameter selection unit 180 selects the processing parameter stored in the processing parameter storage unit 185 in association with the condition that the distance acquired by the distance measuring unit 192 is suitable.

  The distance measuring unit 192 may measure the distance to the subject imaged in each of the plurality of image areas. Then, the position information acquisition unit 190 may acquire the distance measured by the distance measurement unit 192 and the position of the image region in the captured image as position information for each of the plurality of image regions. Then, the processing parameter selection unit 180 adapts the distance from the optical system 110 acquired by the position information acquisition unit 190 and the distance from the optical axis point to the position of the image region for each of the plurality of image regions. The processing parameter stored in the processing parameter storage unit 185 may be selected in association with the condition.

  The captured image analysis unit 194 analyzes the image content of the captured image. Then, the position information acquisition unit 190 acquires the analysis result by the captured image analysis unit 194. Then, the processing parameter storage unit 185 associates the processing parameter that can correct the influence of the optical transfer function on the captured image with the condition relating to the image content of the captured image that should be adapted when the processing parameter is corrected. Stored. Then, the processing parameter selection unit 180 selects a processing parameter stored in the processing parameter storage unit 185 in association with a condition that the analysis result acquired by the position information acquisition unit 190 matches.

  As described above, the imaging apparatus 100 has been described based on the image content. However, the optical transfer function can be directly specified from the image content of the captured image. Specifically, the optical characteristic storage unit 165 stores the spatial frequency characteristics of each of the plurality of optical transfer functions of the optical system 110. The frequency characteristic calculation unit 170 calculates the spatial frequency characteristic of the captured image. The optical characteristic specifying unit 160 is an optical transfer function having a spatial frequency characteristic that matches the spatial frequency characteristic calculated by the frequency characteristic calculation unit 170 among a plurality of optical transfer functions. Identified as a transfer function.

  The frequency characteristic calculation unit 170 calculates a spatial frequency characteristic for each partial region of the captured image. Then, the optical characteristic specifying unit 160 determines, for each partial region, an optical transfer function having a spatial frequency characteristic that matches the spatial frequency characteristic calculated by the frequency characteristic calculating unit 170 for each partial region among the plurality of optical transfer functions. Identify. Then, the image processing unit 140 performs image processing for correcting the optical transfer function specified for each partial region by the optical characteristic specifying unit 160 for each partial region of the captured image.

  More specifically, the object area specifying unit 196 specifies a plurality of object areas in the captured image. Then, the frequency characteristic calculator 170 calculates a spatial frequency characteristic for each object region. Then, the optical characteristic specifying unit 160 selects, for each object region, an optical transfer function having a spatial frequency characteristic that matches the spatial frequency characteristic calculated by the frequency characteristic calculating unit 170 for each object region among the plurality of optical transfer functions. Identify. Then, the image processing unit 140 performs image processing for correcting the optical transfer function specified for each object region by the optical characteristic specifying unit 160 for each object region of the captured image.

  The optical characteristic storage unit 165 stores a plurality of optical transfer functions for the subject existing at the distance in association with the distance from the optical system 110 to the subject. Then, the image recording unit 155 records the distance information indicating the distance stored in the optical property storage unit 165 in association with the optical transfer function specified by the optical property specifying unit 160 along with the captured image. Good.

  FIG. 2 schematically shows an example of the optical characteristics of the optical system 110. The optical system 110 forms an image of the light beam 210 incident from the height h1 at a position Z210 on the optical system 110 side of the light receiving unit 120 in the optical axis direction, and the light beam 210 is at a position lower than the optical axis by Δy1 in the light receiving unit 120. Received light. Thus, the lateral aberration of the light ray 210 by the optical system 110 becomes a negative value. Further, the optical system 110 forms an image of a light ray incident from a height h2 higher than h1 at a position Z220 on the optical system 110 side further than the position Z210 in the optical axis direction, and the light ray 220 is Δy1 from the optical axis in the light receiving unit 120. Light is received at a position lower by Δy2 from the lower optical axis.

  On the other hand, the optical system 110 causes light incident from h3 higher than h2 to be received by the light receiving unit 120 at a position higher by Δy3 from the optical axis. That is, according to the optical system 110, the value of lateral aberration decreases as the incident height increases from the optical axis, but the lateral aberration reaches a maximum value at a certain height. When the incident height is further increased, the lateral aberration is increased, the lateral aberration is 0 at a certain height, and the lateral aberration is more than 0 when the incident height is further increased.

  In the light receiving unit 120, a substantially planar light receiving surface is formed by a plurality of light receiving elements. The light receiving surface of the light receiving unit 120 is provided substantially perpendicular to the optical axis of the optical system 110. The light receiving element may be a CCD image sensor or a MOS image sensor.

  The optical characteristics of the optical system 110 have been described qualitatively with reference to FIG. The schematic diagram of the optical system 110 and the light receiving unit 120 shown in FIG. 2 is drawn for the purpose of facilitating understanding of the optical characteristics of the optical system 110, and is not drawn according to an actual scale. Should be noted.

  FIG. 3 shows an example of the structure of the optical system 110. The optical system 110 includes a plurality of optical elements 310, 320, 330, 350, 360, and 370, and a diaphragm 340. In addition, the light receiving surface of the light receiving unit 120 is indicated by an image surface 380 in the drawing. In the drawing, three principal rays 300, 301, and 302 are drawn on the optical system 110. In the following FIGS. 4, 5, and 6, optical characteristics of the optical system 110 with respect to light having a wavelength of 0.5876 nm indicated by principal rays 300, 301, and 302 are shown. Hereinafter, optical data of the optical elements 310, 320, 330, 350, 360, and 370 will be described.

  The optical element 310 has a refractive index of 1.664445663 and a thickness of 1.997163 mm. Further, the curvature and diameter on the object side of the optical element 310 are 15.20834 mm and 13.47915 mm, respectively, and the curvature and diameter on the image plane 380 side are 8.153898 mm and 10.99605 mm, respectively. In addition, the thickness in description of this figure shows the length of the optical axis direction of an optical element.

  The optical element 320 is provided on the image plane 380 side from the optical element 310 by a distance of 5.193977 mm in the optical axis direction. The distance here refers to the distance between the surface of the optical element 310 on the image plane 380 side and the surface of the optical element 320 on the object side, and the distance in the following description of this figure also has the same meaning. . The optical element 320 has a refractive index of 1.922885059 and a thickness of 8.880505 mm. The curvature and diameter on the object side of the optical element 320 are 38.38834 mm and 9.300722 mm, respectively, and the curvature and diameter on the image plane 380 side are −28.17275 mm and 6.105449 mm, respectively.

  The optical element 330 is provided in contact with the optical element 320. The optical element 330 has a refractive index of 1.46449858 and a thickness of 1.9997mm. The curvature and diameter of the optical element 330 on the image plane 380 side are 10.8814 mm and 4.69059 mm, respectively. The diaphragm 340 is provided at a distance of 1.245339 mm in the optical axis direction from the optical element 330 to the image plane 380 side, and has a diameter of 4.432406 mm.

  The optical element 350 is provided at a distance of 4.864987 mm in the optical axis direction from the stop 340 to the image plane 380 side. The optical element 350 has a refractive index of 2.02033350 and a thickness of 10.00014 mm. Further, the curvature and diameter on the object side of the optical element 350 are −443.0356 mm and 8.913335 mm, respectively, and the curvature and diameter on the image plane 380 side are −17.46664 mm and 13.00595 mm, respectively.

  The optical element 360 is provided in contact with the optical element 350. The optical element 360 has a refractive index of 1.50012841 and a thickness of 10.13764 mm. Further, the curvature and the diameter on the image plane 380 side of the optical element 360 are −23.90391 mm and 16.52799 mm, respectively.

  The optical element 370 is provided at a distance of 5.136917 mm in the optical axis direction from the optical element 360 to the image plane 380 side. The optical element 370 has a refractive index of 2.02033350 and a thickness of 9.916248 mm. Further, the curvature and diameter on the object side of the optical element 370 are 15.68482 mm and 18.15194 mm, respectively, and the curvature and diameter on the image plane 380 side are 25.22519 mm and 13.3875 mm, respectively. The image plane 380 is located at a distance of 7.73001 mm from the optical element 370.

FIG. 4 shows the lateral aberration characteristics of the optical system 110. As illustrated, the optical system 110 has substantially the same lateral aberration characteristics in the X direction and the Y direction over a plurality of image heights. Such lateral aberration characteristics can be obtained by a design method for making the lateral aberration characteristics substantially the same. In this design method, the lateral aberration characteristic is given by a cubic function. Specifically, Δy = ax ³ −ab ² x in which the coefficients a and b are constants is given as a function indicating the lateral aberration characteristics, and the target value at the time of convergence is, for example, 5 × 10 ⁻⁴ and the coefficient b. 10 is determined.

  In this design method, as will be described later, the target value of the x coordinate where the lateral aberration becomes an extreme value is determined by the coefficient b, and the target value of the extreme value is further determined by the coefficient a. The magnitude of the extreme value is specified by a coefficient b according to the interval between the light receiving elements of the light receiving unit 120 so that the magnitude of the lateral aberration on the image plane 380 is at least larger than the interval between the light receiving elements of the light receiving unit 120. It's okay. Then, the lateral aberration characteristics are calculated for the light beams of each image height while changing the parameters of the respective optical elements of the optical system 110. Then, the parameter of each optical element is changed until the error between the calculated lateral aberration characteristic and the predetermined lateral aberration characteristic becomes smaller than a predetermined value. In addition, as a function which shows the characteristic of a lateral aberration, if it is a function which is symmetrical about an origin and has an extreme value, such as a sine function, it will not be restricted to said cubic formula.

  As shown in FIG. 2, the magnitude of the lateral aberration by the optical system 110 is 0 when x = 0, and when the change of the lateral aberration when the x coordinate changes in the positive direction, the lateral aberration is the extreme value. The size increases until it reaches. Further, the lateral aberration curve is substantially symmetric with respect to the origin. As described above, the relationship between the incident position, which is the position on the entrance pupil of the optical system 110, and the lateral aberration with respect to the light ray incident on the incident position is as follows: the first incident position separated from the optical axis by the first distance; The longer the distance from the optical axis to the incident position in the range between, the greater the magnitude of the lateral aberration, which is substantially symmetric about the optical axis. The relationship between the incident position, which is the position on the entrance pupil of the optical system 110, and the lateral aberration with respect to the light beam incident on the incident position is continuous with respect to the distance from the optical axis to the incident position.

  Further, as shown in FIG. 2, when the lateral aberration value further increases from the x coordinate at which the lateral aberration value is an extreme value, the magnitude of the lateral aberration starts to decrease. As the x coordinate further increases, the lateral aberration increases after it becomes zero. Thus, the magnitude of the differential value of lateral aberration with respect to the incident position at the first incident position is smaller than the magnitude of the differential value of lateral aberration with respect to the incident position in the vicinity of the optical axis. More specifically, the differential value of lateral aberration with respect to the incident position at the first incident position is zero. Then, in a range between the second incident position separated from the optical axis by a second distance longer than the first distance and the first incident position, the longer the distance from the optical axis to the incident position, the greater the magnitude of lateral aberration. small.

  The lateral aberration with respect to the light ray incident on the second incident position is zero. Further, in the range between the third incident position separated from the optical axis by a third distance longer than the second distance and the second incident position, the longer the distance from the optical axis to the incident position, the greater the magnitude of lateral aberration. large. Further, as described in the above design method, the lateral aberration of the optical system 110 is represented by a cubic function related to the distance from the optical axis to the incident position. In addition, the lateral aberration of the optical system 110 may be expressed by a sine function related to the distance from the optical axis to the incident position.

  FIG. 5 shows the MTF characteristics of the optical system 110. In the present MTF diagram, the horizontal axis indicates the defocus amount in the optical axis direction from the image plane 380, and the vertical axis indicates the MTF value. FIG. 6 shows an example of a spot diagram by the optical system 110.

  Referring to FIG. 5, it can be seen that the optical system 110 has substantially the same distribution of MTF values for a plurality of image heights as well as for sagittal rays and meridional rays. It can also be seen that the optical system 110 has an MTF value equal to or greater than a predetermined value (for example, 0.2) over a wide range of defocus amounts. FIG. 6 shows spot diagrams obtained by the optical system 110 arranged in the horizontal direction with respect to the defocus amount and in the vertical direction with respect to the image height. Referring to FIG. 6, it can also be seen that the spot diagram shows substantially the same spread for a predetermined range of defocus amount and image height. As described above, the optical system 110 causes the light receiving unit 120 to receive light from a subject located within a predetermined distance range from the optical system 110 with substantially the same spread due to lateral aberration. As illustrated, the spread of light by the optical system 110 is substantially the same over a predetermined defocus amount and image height range, so that the image processing unit 140 passes through the optical system 110 and the light receiving unit 120 receives light. An image obtained from the obtained light can be restored by substantially the same inverse filter.

  While the spread of light by the optical system 110 is substantially the same, when the spot diagram of FIG. 6 is observed in detail, it can be seen that the spot diagram is slightly different with respect to defocusing. Specifically, the spot diagram has a shape having a “core” when the defocus is positive, and has an “annular” or “doughnut” shape when the defocus is negative. The spot diagram in this figure is a result obtained with respect to a plurality of defocuses with respect to light from one point, but the change of the spot diagram with respect to the distance to the subject has the same tendency. For example, for light from a subject in the vicinity of the optical system 110, the light beam spreads to the surroundings due to the effect of lateral aberration on the light passing through the periphery of the optical system 110 and the effect that the subject is located near the optical system 110. , The spot diagram becomes "annular" shape. On the other hand, since the optical system 110 has small lateral aberration at the center, light close to paraxial light from a distant subject is narrowed down by the optical system 110, and the spot diagram has a shape having a “core”.

FIG. 7 shows an example of the response of the optical system 110 to the point light source and the arrangement of the light receiving elements included in the light receiving unit 120. As described above, the optical system 110 has substantially the same spread for a plurality of image heights. Therefore, if the position of the light receiving unit 120 is within a predetermined defocus amount range, the half-value widths δ0, δ1, and δ2 of the response of the optical system 110 to the point light source can be made substantially the same as illustrated. Note that the width of the defocus amount at which the half-value widths δ0, δ1, and δ2 can be made substantially the same is controlled by determining the target value of the x coordinate where the lateral aberration is an extreme value in the above design method. Can do. For example, in the lateral aberration characteristic Δy = ax ³ −ab ² x, control is performed with the value of the coefficient b.

The light receiving unit 120 includes a plurality of light receiving elements 701 to 704, 711 to 714, 721 to 724,. The light receiving elements of the light receiving unit 120 are arranged at equal intervals with a pitch of Px in the x direction and Py in the y direction. In this case, the optical system 110 may be designed by designating the lateral aberration characteristics of the optical system 110 such that the half widths δ0, δ1, and δ2 of the responses are larger than the pixel pitches Px and Py. Specifically, when the lateral aberration characteristic Δy = ax ³ −ab ² x is given by the above design method, the coefficient a is set so that the half-value widths δ0, δ1, and δ2 of the response are larger than any of Px and Py. Set the target value. Thus, when designing the optical system 110, the target value of the coefficient b is determined according to the allowable width of the defocus amount, and the target value of the coefficient a is determined according to the pixel pitch of the light receiving element of the light receiving unit 120. .

  Note that when the imaging apparatus 100 captures a color image, adjacent light receiving elements may receive light of different color wavelengths. In this case, the pixel pitch may be a distance between light receiving elements that receive light having a wavelength indicating the same color. For example, the light receiving elements 701, 703, 712, 714, 721, and 723 receive light having a wavelength indicating green, the light receiving elements 702, 704, 722, and 724 receive light having a wavelength indicating blue, and the light receiving elements When 711 and 713 receive light having a wavelength indicating red, the pixel pitch is a distance between the center position of the light receiving element 701 and the center position of the light receiving element 721 in the y direction, and the light receiving element 701 in the x direction. Between the center position of the light receiving element 703 and the center position of the light receiving element 703.

  Note that the light receiving unit 120 may be provided between an intersection where a light beam incident in parallel to the optical axis from the first incident position intersects the optical axis and a paraxial image point of the optical system 110. Specifically, the light receiving unit 120 may be provided near the midpoint between the intersection point and the paraxial image point of the optical system 110.

  The display unit 150 may display an image obtained according to the amount of light received by the light receiving unit 120. Specifically, the display unit 150 displays the image generated by the image generation unit 130. When the number of pixels of the display unit 150 is smaller than the number of light receiving elements of the optical system 110, the display unit 150 thins out the pixels without spatially averaging the image generated by the image generation unit 130. May be displayed. Further, the imaging apparatus 100 does not have a focus control device that changes the focal length of the optical system 110, and the focal length of the optical system 110 may be fixed. Further, the imaging apparatus 100 may not have an optical low-pass filter that spatially disperses light from the subject.

  FIG. 8 shows an example of data stored in the processing parameter storage unit 185 in a table format. The processing parameter storage unit 185 stores information indicating frequency filters (for example, filters A and B) used for the restoration process in association with conditions regarding the operation mode, distance, image feature amount, and spatial frequency characteristics. Specifically, the operation mode may be an operation mode indicating whether or not the imaging apparatus 100 performs short-distance shooting. For example, the position information acquisition unit 190 acquires, as position information, a short-distance imaging mode, which is an operation mode for performing short-distance shooting, when an instruction to start a barcode reader software that analyzes a barcode is received from a user. To do. On the other hand, when receiving an instruction to shoot a landscape from the user, the position information acquisition unit 190 acquires a long-distance imaging mode, which is an operation mode for performing long-distance shooting, as position information.

  The operation mode is a mode in which the imaging apparatus 100 operates when the imaging apparatus 100 handles a captured image. For example, the imaging apparatus 100 processes the captured image, displays the captured image, or displays the captured image. This mode may be a mode in which the imaging apparatus 100 operates when saving or outputting a captured image to the outside. That is, the operation mode may indicate a software operation in the imaging apparatus 100. For example, when the operation mode is set, the imaging apparatus 100 may not adjust the optical characteristics such as the focal length of the optical system 110 according to the operation mode. For example, when an instruction to start barcode reader software is received from the user, the imaging apparatus 100 processes the captured image according to instructions of a program included in the barcode reader software.

  Note that when the user gives an instruction to shoot a landscape through a shooting menu or the like, the position information acquisition unit 190 may acquire the long-distance imaging mode as position information according to the instruction. In addition, the position information acquisition unit 190 may acquire the long-distance imaging mode as the position information based on the position information where the imaging device 100 exists. For example, the imaging device 100 acquires latitude and longitude information of a position where the imaging device 100 exists from GPS information. If the position indicated by the latitude and longitude information is included in an area in which a landscape is characteristic, such as a mountain or sea, the position information acquisition unit 190 The long-distance imaging mode may be acquired as position information.

  The operation mode includes an imaging mode manually selected by the user, for example, a macro mode. When the imaging apparatus 100 determines that flash photography should be performed, the position information acquisition unit 190 may acquire an imaging mode other than the short-distance imaging mode as position information. In this way, the position information acquisition unit 190 acquires the operation modes of the imaging apparatus 100 such as the short-distance imaging mode, the long-distance imaging mode, the normal imaging mode, and the macro mode as position information. Then, the processing parameter selection unit 180 selects the filter A or B stored in the processing parameter storage unit 185 in association with these operation modes. For example, when the operation mode is the short-distance imaging mode, the processing parameter selection unit 180 selects the filter A.

  Further, the distance stored in the processing parameter storage unit 185 may include a range of distances to the subject. In this case, the position information acquisition unit 190 acquires the distance from the distance measuring unit 192 to the subject. Then, the processing parameter selection unit 180 selects a filter stored in the processing parameter storage unit 185 in association with the range including the distance acquired by the position information acquisition unit 190. For example, if the distance to the subject is 20 cm, the processing parameter selection unit 180 selects the filter A.

  The image feature quantity stored in the processing parameter storage unit 185 may include a color feature quantity. In addition, the color feature amount stored in the processing parameter storage unit 185 may be, for example, information on the black-and-white ratio. For example, the processing parameter storage unit 185 may store a condition relating to an image feature amount that “the ratio of black is 35% to 60%”. In addition, the image feature amount may be the number of color components. For example, the processing parameter storage unit 185 may further store a condition relating to an image feature amount that “the ratio of colors created by a combination of two specific colors including black is greater than 90%”.

  For example, barcodes are often printed in black on a plain background. The black ratio is expected to be about 50% on average. Also, the barcode area is expected to occupy nearly 100% of the total area of the two colors of white and black. However, it is expected that the barcode image included in the captured image is a blurred image due to the optical characteristics of the optical system 110. For this reason, it is expected that the barcode image included in the captured image includes a color created by a combination of two color intensities. Therefore, it is expected that the barcode image includes, for example, a color created from a combination of black and white in an area ratio exceeding 90%.

  Therefore, the captured image analysis unit 194 calculates the ratio of black and the ratio of colors created from the combination of two colors in the captured image. Then, the position information acquisition unit 190 acquires the calculation result by the captured image analysis unit 194 as position information. Then, the processing parameter selection unit 180 selects a filter stored in the processing parameter storage unit 185 in association with a condition in which the ratio occupied by black and the ratio occupied by a color created from a combination of two colors are matched.

  As described above, the processing parameter storage unit 185 is an imaging device when the imaging device 100 handles a processing parameter that can correct the influence of the optical transfer function on the captured image, and the captured image to be corrected by the processing parameter. Stored in association with 100 operation modes. Then, the position information acquisition unit 190 acquires, as position information, an operation mode that handles a captured image obtained by capturing a subject at a short distance or an operation mode that handles a captured image obtained by capturing a subject at a long distance. Then, the processing parameter selection unit 180 selects the processing parameter stored in the processing parameter storage unit 185 in association with the operation mode acquired by the position information acquisition unit 190.

  In addition, the processing parameter storage unit 185 sets the processing parameter that can correct the influence of the optical transfer function on the captured image as the imaging mode of the imaging apparatus 100 when capturing the captured image that should be corrected with the processing parameter. Stored in association. Then, the position information acquisition unit 190 acquires, as position information, an imaging mode indicating that a subject at a short distance should be imaged. Then, the processing parameter selection unit 180 selects a processing parameter stored in the processing parameter storage unit 185 in association with the imaging mode acquired by the position information acquisition unit 190.

  As described above, the position information acquisition unit 190 acquires, as position information, an instruction indicating the positional relationship between the subject and the optical system 110, such as an instruction indicating an operation mode or an instruction indicating an imaging mode. To do. The instruction includes an instruction to cause the imaging apparatus 100 to perform a software operation, such as an instruction to start the barcode reader software as described above. In addition, the processing parameter storage unit 185 is acquired when capturing a captured image that should be suitable for correcting the processing parameter that can correct the influence of the optical transfer function on the captured image with the processing parameter. Stored in association with the conditions related to the contents of instructions. The contents of the instruction can be exemplified by the above-described operation mode or imaging mode. Then, as described above, the processing parameter selection unit 180 may select the processing parameter stored in the processing parameter storage unit 185 in association with the condition that the content of the instruction acquired by the position information acquisition unit 190 is suitable. .

  Further, the spatial frequency characteristics stored in the processing parameter storage unit 185 may include the spatial frequency characteristics of the captured image. For example, the optical characteristic storage unit 165 may store information (OTFID) for identifying an optical transfer function that most closely matches the spatial frequency characteristic of the captured image as a condition regarding the spatial frequency characteristic. The operation when the processing parameter selection unit 180 selects a filter based on the spatial frequency characteristic will be described with reference to FIG.

  FIG. 9 shows an example of a plurality of image areas in the captured image. An entire image area 900 that is an image area of the entire captured image includes a first image area 901 including an optical axis point that is a point corresponding to the optical axis in the captured image, a plurality of second image areas 902a to 902d, and a plurality of third areas. The image areas 903a to 903h and a plurality of fourth image areas 904a to 904l are formed. In the following description, the second image area 902a to d, the third image area 903a to h, and the fourth image area 904a to h are respectively referred to as the second image area 902, the third image area 903, and the fourth image area. The region 904 is collectively referred to.

  The second image area 902 is located in the periphery of the first image area 901, the third image area 903 is located in the periphery of the second image area 902, and the fourth image area 904 is the third image. It is located at the periphery of the region 903. The second image area 902 is adjacent to the first image area 901 and the third image area 903, and the third image area 903 is adjacent to the second image area 902 and the fourth image area 904. Specifically, the first image region 901 is an image region whose radius from the optical axis point is up to r1, the second image region 902 is an image region whose radius from the optical axis point is r1 to r2, and the third The image region 903 is an image region having a radius from r2 to r3 from the optical axis point, and the fourth image region 904 is an image region having a radius from the optical axis point larger than r3.

  Then, the processing parameter storage unit 185 associates each of the first image area 901, the plurality of second image areas 902, the plurality of third image areas 903, and the plurality of fourth image areas 904 with each other. The filter described in relation to FIG. 8 is stored. In this way, the processing parameter storage unit 185 associates each of the plurality of image areas divided in the angular direction around the optical axis point and the radial direction with the optical axis point as the origin, A filter for correcting the image in the image area is stored.

  As shown in the figure, the image area showing the entire captured image is divided into more image areas in the angular direction as the distance from the optical axis point is larger. For example, an image region having radii r1 to r2 from the optical axis point is formed from four second image regions 902, and an image region having radii r2 to r3 from the optical axis point is formed from eight third image regions 903. Thus, an image region having a radius larger than r3 from the optical axis point is formed from twelve fourth image regions 904. As described above, the processing parameter storage unit 185 associates the information for identifying each of the image regions divided more in the angular direction as the distance from the optical axis point is larger, and the filter for correcting the image of each image region. May be stored.

  Then, the processing parameter selection unit 180 identifies each of the first image area 901, the plurality of second image areas 902, the plurality of third image areas 903, and the plurality of fourth image areas 904. Among the filters stored in the processing parameter storage unit 185 in association with the information, a filter corresponding to the distance to the subject indicated by the position information acquired by the position information acquisition unit 190 is selected. For example, the distance measuring unit 192 calculates the distance to the subject imaged in each of the first image area 901, the plurality of second image areas 902, the plurality of third image areas 903, and the plurality of fourth image areas 904. You may measure the distance. Then, the processing parameter selection unit 180 identifies each of the first image area 901, the plurality of second image areas 902, the plurality of third image areas 903, and the plurality of fourth image areas 904. Of the filters stored in the processing parameter storage unit 185 in association with the information, a filter corresponding to the distance measured by the distance measuring unit 192 may be selected. Then, the image processing unit 140 selects the first image region 901, the plurality of second image regions 902, the plurality of third image regions 903, and the plurality of fourth image regions 904 by the processing parameter selection unit 180. Correct the image using a filter.

  As described above, the processing parameter storage unit 185 associates each of the plurality of image regions with a condition relating to a three-dimensional positional relationship between the subject imaged in each of the plurality of image regions and the optical system 110. A filter such as an inverse filter for the optical transfer function of the optical system 110 with respect to the light from the position is stored. Then, the position information acquisition unit 190 acquires position information indicating a three-dimensional positional relationship between the imaged subject and the optical system 110 for each of the plurality of image regions. Then, the processing parameter storage unit 185 stores the processing parameter selection unit 180 in association with the condition that the three-dimensional positional relationship indicated by the positional information acquired by the positional information acquisition unit 190 is matched for each of the plurality of image regions. Select the processing parameter.

  As described above, according to the imaging apparatus 100, each image of the plurality of image areas is used, for example, by using an inverse filter for the optical transfer function with respect to light from the position of the subject imaged in each of the plurality of image areas. Can be corrected. For this reason, the image of the subject imaged in each image area can be properly restored.

  FIG. 10 shows an example of a block configuration of the optical characteristic specifying unit 160. The optical characteristic specifying unit 160 includes a frequency component difference calculating unit 172, a total value calculating unit 174, and an optical transfer function selecting unit 176. FIG. 11 illustrates an example of a restoration processing flow in which the imaging apparatus 100 restores a captured image. This restoration processing flow shows an example of a processing flow for performing restoration processing on a captured image by estimating the optical transfer function of the optical system 110 from the image content of the captured image. The operation of each component of the optical characteristic specifying unit 160 will be described along the restoration processing flow of this drawing.

  The frequency component difference calculation unit 172 acquires the captured image generated by the image generation unit 130 (S1000). Then, the object area specifying unit 196 extracts a plurality of areas where the object exists (referred to as a plurality of object areas) from the captured image by edge extraction or the like (S1002). Note that since the subject image is blurred by the optical system 110, it may be difficult to extract an edge from the captured image. In such a case, as will be described in a later step, the object area specifying unit 196 performs a restoration process on the captured image using a representative filter, and extracts an edge from the obtained restored image. An object area may be specified.

  Next, a restored image is generated for each object area by performing the following processing from S1004 to S1014 for each of the plurality of object areas specified by the object area specifying unit 196. First, the frequency characteristic calculation unit 170 calculates the value of the spatial frequency component for each object region of the captured image (S1004). Specifically, the frequency characteristic calculation unit 170 calculates the value of the spatial frequency component for each object region by performing Fourier transform for each object region of the captured image.

  Then, the frequency component difference calculation unit 172 calculates, for each of the plurality of optical transfer functions, the difference between the value of the spatial frequency component calculated by the frequency characteristic calculation unit 170 and the value of the spatial frequency component indicated by the optical transfer function. It is calculated every time (S1006). Then, the total value calculation unit 174 calculates a total value obtained by summing the magnitudes of the differences calculated by the frequency component difference calculation unit 172 over the frequencies for each of the plurality of optical transfer functions (S1008). Then, the optical transfer function selection unit 176 selects an optical transfer function having a smaller total value calculated by the total value calculation unit 174 as an optical transfer function of the optical system 110 among the plurality of optical transfer functions. (S1010). Specifically, the optical transfer function selection unit 176 selects OTFID that is identification information for identifying an optical transfer function having a smaller total value calculated by the total value calculation unit 174.

  Then, the processing parameter selection unit 180 selects the filter (filter A or filter B) stored in the processing parameter storage unit 185 in association with the OTFID for each object area (S1012). Then, the image processing unit 140 performs filtering in the spatial frequency domain for each object region using the selected filter and performs inverse Fourier transform to obtain a restored image of each object region (S1014). Then, the image processing unit 140 generates one restored image by combining the obtained restored images of the object regions.

  In S1012 and S1014, the restoration process using the frequency domain filter has been described. However, S1012 and S1014 are a convolution restoration process that is substantially equivalent to the filtering process in the frequency domain as described above. A restored image can also be generated. That is, the processing parameter storage unit 185 stores a convolution filter in association with the OTFID that identifies a plurality of optical transfer functions. Then, the processing parameter selection unit 180 selects the convolution filter stored in the processing parameter storage unit 185 in association with the OTFID selected in S1010. Then, the image processing unit 140 generates a restored image from the captured image by the convolution process using the convolution filter selected by the processing parameter selection unit 180.

  In the above example, the optical transfer function of the optical system 110 is specified for each object area specified by the object area specifying unit 196, and the image processing unit 140 is set for each object area according to the specified optical transfer function. A restored image was generated. In addition, the optical characteristic specifying unit 160 may specify an optical transfer function for each predetermined region. Then, the image processing unit 140 may perform a restoration process for each predetermined region based on the specified optical transfer function. In addition, for the image area area for which the object area specifying unit 196 could not specify the object area, the optical characteristic specifying unit 160 specifies the optical transfer function for each predetermined area, and the image processing unit 140 performs the restoration process. May be applied.

  As described above, the frequency characteristic calculation unit 170 calculates the frequency distribution of the spatial frequency component of the captured image. Then, the optical characteristic specifying unit 160 calculates an optical transfer function having a frequency distribution of the spatial frequency component that more closely matches the frequency distribution of the spatial frequency component calculated by the frequency characteristic calculation unit 170 among the plurality of optical transfer functions, It is specified as an optical transfer function of the optical system 110.

  FIG. 12 shows an example of the frequency characteristic indicated by the optical transfer function and the spatial frequency characteristic of the image. The frequency component distributions 1110a and 1110b show the frequency characteristics of two different types of optical transfer functions as light and shade information. In addition, an image obtained by Fourier transform of an image transmitted by an optical transfer function having frequency characteristics indicated by frequency component distributions 1110a and 1110b is indicated by grayscale information by frequency component distributions 1120a and 1120b, respectively. Although the frequency component distributions 1120a and 1120b include a frequency component derived from the image signal and a frequency component derived from the edge of the image, the difference due to the difference in the optical transfer function can be identified. The optical characteristic specifying unit 160 can easily specify the optical transfer function of the optical system 110 by extracting the difference that different optical transfer functions give to the captured image in the frequency domain.

  FIG. 13 shows an example of data stored in the optical characteristic storage unit 165 in a table format. The optical characteristic storage unit 165 stores frequency component data indicating the value of the spatial frequency component indicated by each of the plurality of optical transfer functions, OTFID, and distance. The filter A corresponds to an inverse filter of OTF data (A), and the filter B corresponds to an inverse filter of OTF data (B). The optical characteristic storage unit 165 stores a representative distance to the subject that is an optical transfer function indicated by the OTF data. The image recording unit 155 reads the distance stored in the optical property storage unit 165 in association with the OTFID specified by the optical property specifying unit 160. Then, the image recording unit 155 attaches the distance to the captured image or the restored image and outputs it to the outside.

  As described above, the optical characteristic storage unit 165 stores a plurality of optical transfer functions that differ depending on the positional relationship between the optical system 110 and the subject. Specifically, the optical characteristic storage unit 165 stores a plurality of optical transfer functions that differ depending on the distance from the optical system 110 to the subject. More specifically, the optical characteristic storage unit 165 has a spatial frequency characteristic of each of a plurality of optical transfer functions with respect to a subject at a plurality of positions whose distance from the optical system 110 is within a predetermined distance range, for example, a space. Stores the frequency distribution of frequency components.

  FIG. 14 schematically illustrates an example of the positional relationship among the optical system 110, the light receiving unit 120, and the subject when the image capturing unit 105 performs tilt shooting. As illustrated, the optical axis of the optical system 110 intersects the light receiving surface of the light receiving unit 120 at an angle smaller than a right angle. In this way, by causing the light receiving surface of the light receiving unit 120 and the optical axis to intersect with each other without being orthogonal, the enlargement ratio for the subject 1381 located far from the optical system 110 is larger than the enlargement ratio for the subject 1383 closer to the optical system 110. growing. For this reason, it is possible to reduce the possibility that the image of a distant subject becomes too small.

  When tilting is performed in this way, the distance range captured by the imaging unit 105 can be expanded, and the optical transfer function of the optical system 110 may not be considered to be substantially the same over the distance range. There is. Therefore, in one embodiment, the processing parameter storage unit 185 assigns a different filter for each image area.

  In general, the distance to the subject (for example, l1, l0, and l2) varies depending on the imaging direction of the imaging unit 105. Further, the distance to the subject varies depending on the space configuration of the imaging target space. Therefore, the processing parameter storage unit 185 sets the processing parameters that can correct the influence of the optical transfer function to each of the plurality of image regions in the captured image, as the direction of the optical axis of the optical system 110 and the plurality of images in the captured image. It is desirable to store the image area in association with the conditions relating to the respective positions of the image area. Then, the position information acquisition unit 190 acquires the direction of the optical axis of the optical system 110 and the positions of the plurality of image areas in the captured image as position information. The processing parameter selection unit 180 then stores the processing parameters stored in the processing parameter storage unit 185 in association with the conditions in which the orientation of the optical axis acquired by the position information acquisition unit 190 and the respective positions of the plurality of image regions match. May be selected.

  Further, the distance to the subject (for example, l1 and l2) also varies depending on the angle θ0 formed by the optical axis of the optical system 110 and the light receiving surface of the light receiving unit 120. Therefore, the processing parameter storage unit 185 determines processing parameters that can correct the influence of the optical transfer function on each of the plurality of image regions in the captured image as the optical axis of the optical system 110 and the light receiving surface of the light receiving unit 120. Are stored in association with a condition in which the angle formed by and the respective positions of the plurality of image areas in the captured image match. Then, the position information acquisition unit 190 acquires, as position information, the angle formed by the optical axis of the optical system 110 and the light receiving surface of the light receiving unit 120 and the positions of the plurality of image regions in the captured image. Then, the processing parameter selection unit 180 selects the processing parameter stored in the processing parameter storage unit 185 in association with the condition that the corner acquired by the position information acquisition unit 190 and the position of each of the plurality of image areas match. It's okay.

  More generally, the distance to the subject changes according to the three-dimensional positional relationship between the optical system 110 and the light receiving unit 120. Therefore, the processing parameter storage unit 185 determines a processing parameter that can correct the influence of the optical transfer function for each of the plurality of image regions in the captured image in a three-dimensional manner between the optical system 110 and the light receiving unit 120. May be stored in association with conditions that match the position of each of the plurality of image areas in the captured image. Note that the positional relationship may include, for example, angles θ1 and θ2 formed by a line connecting the diagonal point of the rectangular light receiving surface of the light receiving unit 120 and the principal point in addition to the above θ0. Then, the processing parameter storage unit 185 sets the processing parameter that can correct the influence of the optical transfer function to each of the plurality of image regions in the captured image in the positional relationship between the optical system 110 and the light receiving unit 120. Store in association. Then, the position information acquisition unit 190 acquires the positional relationship between the optical system 110 and the light receiving unit 120 and the respective positions of the plurality of image areas in the captured image as position information. Then, the processing parameter selection unit 180 selects the processing parameter stored in the processing parameter storage unit 185 in association with the positional relationship acquired by the position information acquisition unit 190 and the condition in which each position of the plurality of image areas matches. You can do it.

  FIG. 15 shows a state in which filters are assigned to a plurality of image regions of the captured image 1400, respectively. As shown in this figure, filter C, filter D, filter E, and filter F are assigned to image regions 1401, 1402, 1403, and 1404. Note that the filter C, the filter D, the filter E, and the filter F may be an inverse filter of an optical transfer function for a subject located further away in order.

  As described above, the processing parameter storage unit 185 determines the processing parameter that can correct the influence of the optical transfer function on each of the plurality of image regions in the captured image, as the processing parameter for each of the plurality of image regions in the captured image. Store it in association with the condition that the position matches. And the position information acquisition part 190 acquires each position of the several image area | region in a captured image as position information. Then, the processing parameter selection unit 180 selects a processing parameter stored in the processing parameter storage unit 185 in association with a condition in which each position of the plurality of image regions acquired by the position information acquisition unit 190 is suitable.

  FIG. 16 shows an example of a hardware configuration when the imaging apparatus 100 is configured by an electronic information processing apparatus such as a personal computer. The imaging apparatus 100 includes a CPU peripheral part, an input / output part, and a legacy input / output part. The CPU peripheral section includes a CPU 1505, a RAM 1520, a graphic controller 1575, and a display device 1580 that are connected to each other by a host controller 1582. The input / output unit includes a communication interface 1530, a hard disk drive 1540, and a CD-ROM drive 1560 that are connected to the host controller 1582 by the input / output controller 1584. The legacy input / output unit includes a ROM 1510, a flexible disk drive 1550, and an input / output chip 1570 connected to the input / output controller 1584.

  The host controller 1582 connects the RAM 1520, the CPU 1505 that accesses the RAM 1520 at a high transfer rate, and the graphic controller 1575. The CPU 1505 operates based on programs stored in the ROM 1510 and the RAM 1520 to control each unit. The graphic controller 1575 acquires image data generated by the CPU 1505 or the like on a frame buffer provided in the RAM 1520 and displays the image data on the display device 1580. Alternatively, the graphic controller 1575 may include a frame buffer that stores image data generated by the CPU 1505 or the like.

  The input / output controller 1584 connects the host controller 1582 to the hard disk drive 1540, the communication interface 1530, and the CD-ROM drive 1560, which are relatively high-speed input / output devices. The hard disk drive 1540 stores programs and data used by the CPU 1505. The communication interface 1530 is connected to the network communication device 1598 to transmit / receive programs or data. The CD-ROM drive 1560 reads a program or data from the CD-ROM 1595 and provides it to the hard disk drive 1540 and the communication interface 1530 via the RAM 1520.

  The input / output controller 1584 is connected to the ROM 1510, the flexible disk drive 1550, and the relatively low-speed input / output device of the input / output chip 1570. The ROM 1510 stores a boot program executed when the imaging apparatus 100 is started up, a program depending on the hardware of the imaging apparatus 100, and the like. The flexible disk drive 1550 reads a program or data from the flexible disk 1590 and provides it to the hard disk drive 1540 and the communication interface 1530 via the RAM 1520. The input / output chip 1570 connects various input / output devices via the flexible disk drive 1550 or a parallel port, serial port, keyboard port, mouse port, and the like.

  A program executed by the CPU 1505 is stored in a recording medium such as the flexible disk 1590, the CD-ROM 1595, or an IC card and provided by the user. The program stored in the recording medium may be compressed or uncompressed. The program is installed in the hard disk drive 1540 from the recording medium, read into the RAM 1520, and executed by the CPU 1505. The program executed by the CPU 1505 causes the imaging apparatus 100 to be connected to the imaging unit 105, the image processing unit 140, the display unit 150, the image recording unit 155, the optical characteristic specifying unit 160, and the optical characteristics described with reference to FIGS. Storage unit 165, frequency characteristic calculation unit 170, frequency component difference calculation unit 172, total value calculation unit 174, optical transfer function selection unit 176, processing parameter selection unit 180, processing parameter storage unit 185, position information acquisition unit 190, measurement It functions as a distance unit 192, a captured image analysis unit 194, and an object area specifying unit 196.

  The program shown above may be stored in an external storage medium. As the storage medium, in addition to the flexible disk 1590 and the CD-ROM 1595, an optical recording medium such as a DVD or PD, a magneto-optical recording medium such as an MD, a tape medium, a semiconductor memory such as an IC card, or the like can be used. Further, a storage device such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet may be used as a recording medium, and a program via the network may be provided to the imaging device 100.

  In the above description, the embodiment of the present invention has been described using the imaging device 100. However, in another embodiment, the imaging device 100 may be implemented as an image processing device that specifies the optical transfer function of the optical system 110. The image processing device specifies an optical transfer function of the imaging device 100 based on a captured image captured by the imaging device 100. That is, the image processing apparatus includes the image processing unit 140, the display unit 150, the image recording unit 155, the optical characteristic specifying unit 160, the optical characteristic storage unit 165, the frequency characteristic calculation unit 170, the frequency among the components included in the imaging device 100. Component difference calculation unit 172, total value calculation unit 174, optical transfer function selection unit 176, processing parameter selection unit 180, processing parameter storage unit 185, position information acquisition unit 190, captured image analysis unit 194, and object region specification unit 196 May be provided. Note that the image processing apparatus may be provided in a separate device from the imaging apparatus 100. Further, as in the above-described embodiment, the imaging apparatus 100 may be mounted.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

1 is a diagram illustrating an example of a configuration of an imaging apparatus 100. FIG. It is a figure which shows typically an example of the optical characteristic of the optical system 110. FIG. 2 is a diagram illustrating an example of a structure of an optical system 110. FIG. It is a figure which shows the lateral aberration characteristic of the optical system. It is a figure which shows the MTF characteristic of the optical system. It is a figure which shows an example of the spot diagram by the optical system. It is a figure which shows an example of the response of the optical system 110, and the arrangement | sequence of a light receiving element. It is a figure which shows an example of the data which the process parameter storage part 185 stores. It is a figure which shows an example of the several image area | region in a captured image. 5 is a diagram illustrating an example of a block configuration of an optical characteristic specifying unit 160. FIG. FIG. 10 is a diagram illustrating an example of a restoration processing flow by the imaging apparatus. It is a figure which shows an example of an optical transfer function and the spatial frequency characteristic of an image. It is a figure which shows an example of the data which the optical characteristic storage part 165 has stored. It is a figure which shows an example of the positional relationship between the optical system 110, the light-receiving part 120, and a to-be-photographed object in tilt photography. FIG. 10 is a diagram showing filters assigned to a plurality of image areas of a captured image 1400. 2 is a diagram illustrating an example of a hardware configuration of the imaging apparatus 100. FIG.

Explanation of symbols

100 imaging device 105 imaging unit 110 optical system 120 light receiving unit 130 image generating unit 140 image processing unit 150 display unit 155 image recording unit 160 optical characteristic specifying unit 165 optical characteristic storage unit 170 frequency characteristic calculating unit 172 frequency component difference calculating unit 174 total Value calculation unit 176 Optical transfer function selection unit 180 Processing parameter selection unit 185 Processing parameter storage unit 190 Position information acquisition unit 192 Distance measurement unit 194 Captured image analysis unit 196 Object region specification unit

Claims (13)

An imaging device that captures a captured image of a subject,
The light from each position within the range of a predetermined positional relationship with the imaging device is received by the light receiving unit of the imaging device with substantially the same spread, and the range of the positional relationship with the imaging device is predetermined. An optical system having a different optical transfer function for light from different positions in
A processing parameter capable of correcting the influence of the optical transfer function on the captured image is associated with a condition relating to a positional relationship between the subject and the optical system to be adapted when the processing parameter is corrected. A processing parameter storage unit to store;
A position information acquisition unit that acquires position information indicating a positional relationship between a subject and the optical system;
A processing parameter selection unit that selects a processing parameter stored in the processing parameter storage unit in association with a condition that the positional relationship indicated by the positional information acquired by the positional information acquisition unit is suitable;
With
The processing parameter storage unit sets a processing parameter capable of correcting the influence of the optical transfer function to each of the plurality of image regions in the captured image, as a condition regarding each position of the plurality of image regions in the captured image. Stored in association with
The position information acquisition unit acquires the position of each of a plurality of image areas in the captured image as the position information,
The processing parameter selection unit selects a processing parameter stored in the processing parameter storage unit in association with a condition in which each position of the plurality of image regions acquired by the position information acquisition unit is compatible. > Imaging device.   The processing parameter storage unit sets a processing parameter capable of correcting the influence of the optical transfer function to each of a plurality of image regions in the captured image as a plurality of orientations of the optical axis of the optical system and the captured image. In association with the conditions related to the position of each image area,
  The position information acquisition unit acquires, as the position information, the orientation of the optical axis of the optical system and the positions of a plurality of image regions in the captured image,
  The processing parameter selection unit stores the processing parameter stored in the processing parameter storage unit in association with a condition in which the direction of the optical axis acquired by the position information acquisition unit and the position of each of the plurality of image regions match. select
The imaging device according to claim 1.   The optical axis of the optical system intersects with the light receiving surface of the light receiving unit at an angle smaller than a right angle,
  The processing parameter storage unit includes a processing parameter capable of correcting the influence of the optical transfer function on each of a plurality of image regions in the captured image, and an optical axis of the optical system and a light receiving surface of the light receiving unit. Are stored in association with the angle formed by and the conditions regarding the position of each of the plurality of image areas in the captured image,
  The position information acquisition unit acquires, as the position information, an angle formed by an optical axis of the optical system and a light receiving surface of the light receiving unit and each position of a plurality of image regions in the captured image,
  The processing parameter selection unit selects a processing parameter stored in the processing parameter storage unit in association with a condition in which the corners acquired by the position information acquisition unit and the positions of the plurality of image regions match each other.
The imaging device according to claim 1. An imaging device that captures a captured image of a subject,
The light from each position within the range of a predetermined positional relationship with the imaging device is received by the light receiving unit of the imaging device with substantially the same spread, and the range of the positional relationship with the imaging device is predetermined. An optical system having a different optical transfer function for light from different positions in
A processing parameter capable of correcting the influence of the optical transfer function on the captured image is associated with a condition relating to a positional relationship between the subject and the optical system to be adapted when the processing parameter is corrected. A processing parameter storage unit to store;
A position information acquisition unit that acquires position information indicating a positional relationship between a subject and the optical system;
A processing parameter selection unit that selects a processing parameter stored in the processing parameter storage unit in association with a condition that the positional relationship indicated by the positional information acquired by the positional information acquisition unit is suitable;
With
The optical system causes the light-receiving unit to receive light from each position within a predetermined distance range from the optical system with substantially the same spread, and within the predetermined distance range from the optical system. The optical transfer function is different for light from a position where the optical system has a distance from the optical system in the optical axis direction and a positional relationship with respect to the optical axis of the optical system is different,
The processing parameter storage unit is an image region that should be adapted when a processing parameter capable of correcting the influence of the optical transfer function on each of a plurality of image regions in the captured image is corrected with the processing parameter. The distance in the optical axis direction of the subject being imaged and the condition relating to the position of the image area in the captured image indicating the positional relationship with respect to the optical axis of the optical system,
The position information acquisition unit acquires, as the position information, the distance between the object being imaged and the optical system, and the position of the image area in the captured image, for each of a plurality of image areas.
The processing parameter selection unit is configured to associate the distance between the optical system acquired by the position information acquisition unit and the condition of the position of the image region with respect to each of a plurality of image regions. There <br/> imaging device for selecting the processing parameters are stored. The optical system is within the range of the predetermined distance from the optical system, and the light in the optical axis direction from the optical system and the light from the position where the distance from the optical axis is different, The optical transfer function is different,
The processing parameter storage unit is an image region that should be adapted when a processing parameter capable of correcting the influence of the optical transfer function on each of a plurality of image regions in the captured image is corrected with the processing parameter. An optical axis that is a point on the captured image corresponding to the optical axis that indicates the distance in the optical axis direction of the subject that is being imaged and the distance from the optical axis of the subject that is imaged in the image area Store it in association with the condition related to the distance of the image area from the point,
The position information acquisition unit acquires, as the position information, the distance between the object being imaged and the optical system, and the distance from the optical axis point, for each of a plurality of image regions,
The processing parameter selection unit is associated with a condition in which a distance from the optical system acquired by the position information acquisition unit and a distance from the optical axis point match each of a plurality of image regions. The imaging apparatus according to claim 4, wherein the processing parameter stored in the processing parameter storage unit is selected. The optical system is within the predetermined distance range from the optical system, the distance in the optical axis direction from the optical system, the distance from the optical axis, and the optical axis and the object plane The optical transfer function is different for light from different positions with respect to the intersection point,
The processing parameter storage unit is an image region that should be adapted when a processing parameter capable of correcting the influence of the optical transfer function on each of a plurality of image regions in the captured image is corrected with the processing parameter. The image area related to the optical axis point, which indicates the distance in the optical axis direction of the subject imaged in the optical axis, the distance of the image area from the optical axis point, and the angle related to the intersection of the subject imaged in the image area Stored in association with the conditions related to the angle indicating the position of
The position information acquisition unit calculates, for each of a plurality of image areas, a distance between the object being imaged and the optical system, a distance from the optical axis point, and an angle with respect to the optical axis point. Get as
The processing parameter selection unit adapts a distance from the optical system acquired by the position information acquisition unit, a distance from the optical axis point, and an angle related to the optical axis point for each of a plurality of image regions. The imaging apparatus according to claim 5, wherein a processing parameter stored in the processing parameter storage unit is selected in association with a condition to be performed. 7. The optical system according to claim 4 , wherein the optical system causes the light receiving unit to receive light from a position within a predetermined distance range from the optical system with substantially the same spread due to lateral aberration. 8. Imaging device. A distance measuring unit for measuring the distance to the subject imaged in each of the plurality of image areas;
The imaging apparatus according to claim 4 , wherein the position information acquisition unit acquires, as the position information, the distance measured by the distance measurement unit and the position of the image area in the captured image for each of a plurality of image areas.   An image processing unit that corrects the influence of the optical transfer function on each of the plurality of image regions in the captured image using the processing parameter selected by the processing parameter selection unit for each of the plurality of image regions.
The imaging device according to any one of claims 1 to 8, further comprising:   A program for an imaging apparatus having the optical system, which causes a computer to function as the imaging apparatus according to claim 1.   Light from each position within the range of a predetermined positional relationship with the imaging device that captures the captured image of the subject is received by the light receiving unit of the imaging device with substantially the same spread, and the imaging device and the predetermined An imaging method for imaging using optical systems having different optical transfer functions for light from different positions within a range of positional relations,
  A processing parameter capable of correcting the influence of the optical transfer function on the captured image is associated with a condition relating to a positional relationship between the subject and the optical system to be adapted when the processing parameter is corrected. A processing parameter storage stage to store;
  A position information acquisition step of acquiring position information indicating a positional relationship between a subject and the optical system;
  A processing parameter selection step of selecting a processing parameter stored in the processing parameter storage step in association with a condition in which the positional relationship indicated by the positional information acquired in the positional information acquisition step is suitable;
With
  In the processing parameter storage step, a processing parameter capable of correcting the influence of the optical transfer function on each of the plurality of image regions in the captured image is a condition relating to the position of each of the plurality of image regions in the captured image. Stored in association with
  The position information acquisition step acquires each position of a plurality of image areas in the captured image as the position information,
  In the processing parameter selection step, the processing parameter stored in the processing parameter storage step is selected in association with a condition that the respective positions of the plurality of image regions acquired in the position information acquisition step match.
Imaging method.   Light from each position within the range of a predetermined positional relationship with the imaging device that captures the captured image of the subject is received by the light receiving unit of the imaging device with substantially the same spread, and the imaging device and the predetermined An imaging method for imaging using optical systems having different optical transfer functions for light from different positions within a range of positional relations,
  A processing parameter capable of correcting the influence of the optical transfer function on the captured image is associated with a condition relating to a positional relationship between the subject and the optical system to be adapted when the processing parameter is corrected. A processing parameter storage stage to store;
  A position information acquisition step of acquiring position information indicating a positional relationship between a subject and the optical system;
  A processing parameter selection step of selecting a processing parameter stored in the processing parameter storage step in association with a condition in which the positional relationship indicated by the positional information acquired in the positional information acquisition step is suitable;
With
  The optical system causes the light-receiving unit to receive light from each position within a predetermined distance range from the optical system with substantially the same spread, and within the predetermined distance range from the optical system. The optical transfer function is different for light from a position where the optical system has a distance from the optical system in the optical axis direction and a positional relationship with respect to the optical axis of the optical system is different,
  In the processing parameter storing step, an image region that should be adapted when a processing parameter that can correct the influence of the optical transfer function on each of a plurality of image regions in the captured image is corrected with the processing parameter. The distance in the optical axis direction of the subject being imaged and the condition relating to the position of the image area in the captured image indicating the positional relationship with respect to the optical axis of the optical system,
  In the position information acquisition step, for each of a plurality of image areas, the distance between the subject being imaged and the optical system, and the position of the image area in the captured image are acquired as the position information,
  The processing parameter selection step stores the processing parameter for each of a plurality of image areas in association with the distance between the optical system acquired in the position information acquisition step and the condition of the position of the image area. Select the processing parameters stored in the stage
Imaging method.   An image processing step for correcting the influence of the optical transfer function on each of the plurality of image regions in the captured image using the processing parameter selected in the processing parameter selection step for each of the plurality of image regions.
The imaging method according to claim 11 or 12, further comprising:

Images