JP5523194B2 - Imaging apparatus and image processing method - Google Patents

Description

  The present invention relates to a small and inexpensive image pickup apparatus in which a photographing lens can be replaced.

  In a conventional single-lens reflex camera, a half mirror is placed in the preparation state to divide the light beam incident from the photographic lens into a light beam that goes to the optical viewfinder and a light beam that goes to the imaging surface. It was common to evacuate. In such a single-lens reflex camera, when an image is taken using an optical viewfinder, the incident light flux from the taking lens is divided into a light flux to the optical viewfinder and a light flux to the imaging surface. It is necessary to shoot a movie through a mirror. However, in this case, a double image is generated due to the internal reflection of the half mirror, and the image quality may be deteriorated.

  Therefore, Patent Document 1 discloses a technique for reducing the influence of image deterioration due to internal reflection of the half mirror by configuring the half mirror with a thin film mirror or a glass mirror of 0.2 mm or less.

JP 2008-052246 A

  However, in patent document 1, since the thin film mirror which is a delicate optical component, or a glass mirror of 0.2 mm or less is used, handling is difficult. Further, the thin film mirror itself is expensive, and it is difficult to provide a small and inexpensive camera if a mechanism for holding the thin film mirror or the glass mirror of 0.2 mm or less is maintained while maintaining the surface accuracy.

  Therefore, the present invention provides a small and inexpensive imaging apparatus while suppressing image deterioration caused by internal reflection of a half mirror.

An imaging apparatus according to an aspect of the present invention is fixedly disposed obliquely with respect to an optical axis of an interchangeable lens, reflects an optical element that reflects a part of an incident light beam and transmits another part, and reflects the reflected light by the optical element A finder means configured to be able to observe a subject using a first light beam, an image pickup means for receiving a second light beam transmitted through the optical element and outputting image data, and the output from the image pickup means the image data, the incident light beam to have a image processing means for correcting the degradation caused by internal reflection in the optical element,
When the interchangeable lens and the optical element are projected on a plane parallel to the optical axis of the interchangeable lens in a state where the interchangeable lens is attached to the imaging device, at least a part of the interchangeable lens and the optical element is It is characterized by overlapping.

An image processing method according to another aspect of the present invention includes an optical element that is fixedly disposed obliquely with respect to the optical axis of an interchangeable lens, reflects a part of an incident light beam, and transmits another part. A finder unit configured to be able to observe a subject using the reflected first light beam, and an imaging unit which receives the second light beam transmitted through the optical element and outputs image data; When the interchangeable lens and the optical element are projected onto a plane parallel to the optical axis of the interchangeable lens in a state where the lens is attached to the imaging device, at least a part of the interchangeable lens and the optical element overlap. An image processing method for an imaging apparatus, comprising: acquiring image data from the imaging means; and generating the incident light beam by internally reflecting the acquired image data with the optical element. And correcting the degradation that is characterized by having a.

  Other objects and features of the present invention are illustrated in the following examples.

  ADVANTAGE OF THE INVENTION According to this invention, a small and cheap imaging device can be provided, suppressing the image degradation which generate | occur | produces by the internal reflection of a half mirror.

1 is a central sectional view of a single-lens reflex camera in Embodiment 1. FIG. FIG. 4 is a diagram illustrating aberrations that occur in the half mirror in Example 1. It is a figure showing refraction of the light beam by the half mirror in Example 1. FIG. 3 is a diagram illustrating aberration at a peripheral image height in Example 1. It is a figure showing the ghost light beam which generate | occur | produces by the half mirror internal surface reflection in Example 1. FIG. It is a figure showing the state of refraction and reflection of half mirror inner surface reflection in Example 1. FIG. FIG. 3 is a diagram illustrating a half-mirror inner surface reflection ghost beam at a peripheral image height in the first embodiment. FIG. 3 is a diagram illustrating a photographing range by an image sensor in the first embodiment. 2 is a block diagram illustrating an electrical configuration of a single-lens reflex camera in Embodiment 1. FIG. 3 is a flowchart illustrating a moving image shooting sequence of the single-lens reflex camera according to the first embodiment. 6 is a flowchart illustrating a one-frame process in the moving image shooting sequence according to the first embodiment. FIG. 3 is a diagram illustrating a state where an imaging range is divided according to the first exemplary embodiment. 7 is a flowchart illustrating a correction subroutine for aberration and internal reflection ghost in the moving image shooting sequence according to the first exemplary embodiment. 6 is a flowchart illustrating a transfer function generation subroutine in the moving image shooting sequence according to the first embodiment. 6 is a central cross-sectional view of a single-lens reflex camera according to Embodiment 2. FIG. FIG. 10 is a diagram illustrating a state where an imaging range in Example 2 is divided. It is explanatory drawing of allocation of the degradation function of Example 3. FIG.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In each figure, the same members are denoted by the same reference numerals, and redundant description is omitted.

  First, the image pickup apparatus according to Embodiment 1 of the present invention will be described. In this embodiment, the present invention is applied to a single-lens reflex digital camera (imaging device) to which an interchangeable lens can be attached and detached. FIG. 1 is a central sectional view showing an optical system of a single-lens reflex camera. In FIG. 1, reference numeral 100 denotes a photographing lens (interchangeable lens) mounted on the camera 200 so as to be replaceable, 101 denotes a lens unit including a focus lens group and a zoom lens group, and L denotes an optical axis of the photographing lens 100.

  Reference numeral 200 denotes a camera (camera body). An imaging element 201 (imaging means) including a photoelectric conversion element including an optical low-pass filter, an infrared cut filter, a CMOS sensor, and the like is disposed in the vicinity of a planned imaging surface of the photographing lens 100. Be placed. A half mirror 202 (optical element) formed of a parallel flat glass plate or the like is fixedly disposed obliquely with respect to the optical axis L of the photographing lens 100 between the photographing lens 100 and the imaging element 201. The half mirror 202 is a semi-transparent mirror that reflects part of the incident light beam and transmits the other part. Therefore, the incident light beam that has passed through the photographing lens 100 is separated into reflected light (first light beam) guided to the upper optical finder (finder means) and transmitted light (second light beam) incident on the image sensor 201. The

  Reference numerals 203 to 205 shown in FIG. The optical viewfinder is configured so that the subject can be observed using the reflected light reflected by the half mirror 202. Specifically, the reflected light of the half mirror 202 forms an image on the mat surface of the focus plate 203 having a mat surface and a Fresnel surface, and is guided to the photographer's eyes through the pentaprism 204 and the eyepiece optical system 205. . On the other hand, the transmitted light is received by the image sensor 201, and the intensity signal of the received light becomes an electronic image through AD conversion, an image processing unit, and the like. Thus, the image sensor 201 receives the transmitted light that has passed through the half mirror 202 and outputs image data. Note that the optical axis L that has passed through the half mirror 202 becomes an optical axis L ′ that is offset downward in the drawing by refraction at the half mirror 202. Therefore, the center of the image sensor 201 is arranged so as to coincide with the optical axis L ′.

  In general, when a flat optical element (half mirror 202) such as a glass plate is disposed obliquely between the photographing lens 100 and the image sensor 201, a large aberration occurs in an image obtained by the image sensor 201. This is because each light beam that has passed through the lens has a different position in the optical path length direction that passes through the optical element, and the aberration also varies depending on the position. Therefore, in a conventional single-lens reflex camera, it is general that the half mirror 202 is retracted out of the photographing light beam by a quick return mechanism during photographing.

  On the other hand, in the present embodiment, as shown in FIG. 1, the half mirror 202 is arranged to be inclined, and a light beam transmitted through the half mirror 202 is used at the time of photographing. Although aberration and internal reflection ghost are generated in the image obtained by the image sensor 201, this is image degradation without axial symmetry unlike the optical aberration of the photographing lens 100. In order to reduce the internal reflection ghost, it is conceivable to deposit a multilayer film for preventing reflection on the half mirror 202. However, the deposition of the multilayer film is very expensive and expensive. Therefore, in this embodiment, an asymmetric aberration and an internal reflection ghost are corrected by the image processing circuit 413 to form a good electronic image at low cost. Details of this image processing method will be described later.

  Conventionally, when shooting a moving picture with a single-lens reflex camera, the half mirror 202 is retracted out of the photographing light flux in order to guide the photographing light flux to the image sensor 201 and to prevent aberration caused by transmission through the half mirror 202. It was common to do. For this reason, the photographing light flux is not guided to the optical viewfinder, and an electronic viewfinder is used. In the electronic viewfinder, the image processing operation takes time in order to create an accumulation / read-out operation or an electronic image in the image sensor 201, so that a time delay occurs in the displayed image. For this reason, it is difficult to follow a fast moving body.

  On the other hand, in the configuration of this embodiment, it is possible to observe the subject with the optical viewfinder while taking an electronic image with the image sensor 201. For this reason, the optical viewfinder can be used even when shooting a moving image, and it is possible to follow a moving body that moves faster than the conventional electronic viewfinder of a camera. Further, the half mirror 202 is fixedly arranged. For this reason, unlike the conventional single-lens reflex camera, it is not necessary to use a quick return mechanism for retracting the half mirror 202 out of the photographing light beam during photographing. Therefore, a complicated mechanical configuration used for the quick return mechanism is not necessary, and it is possible to reduce the cost and the size. Furthermore, according to the configuration of the present embodiment, it is possible to achieve noise reduction during moving image shooting.

  FIG. 2A is an explanatory diagram of aberrations generated in the half mirror 202, and only necessary portions are extracted from FIG. In FIG. 2A, light rays 110 a and 110 b emit one point on the optical axis L on the object side, pass through the lens unit 101, and form the outermost light flux that forms an image near the center image height of the image sensor 201. Only the rays are shown. Note that light rays on the object side across the lens unit 101 are omitted. The light beams 110 a and 110 b are refracted by a predetermined angle on the incident surface of the half mirror 202 according to the well-known Snell's law, and travel straight through the half mirror 202. Thereafter, the light is refracted by a predetermined angle again on the exit surface of the half mirror 202 in accordance with Snell's law. At this time, the angle of the light ray incident on the half mirror 202 is equal to the angle of the light ray emitted. As is well known, this is the principle of refraction of light on a parallel plate. The same applies to the optical axis L. That is, the light beam that has passed through the half mirror 202 does not change its angle, and only a parallel shift occurs.

  FIG. 3 is a detailed explanatory view of the refraction of the light beam, and shows the refraction of the light of the parallel plate in the air. In FIG. 3, reference numeral 210 denotes a parallel plate, 120 denotes a light beam incident on the parallel plate 210, and its traveling direction is indicated by an arrow. The light beam 120 enters the incident surface 210a of the parallel plate 210 at an incident angle φ, and the light beam 120 is refracted based on Snell's law. After refraction, the light travels straight through the parallel plate 210 at an angle φ ′. Thereafter, the light beam 120 is refracted again by the exit surface 210b of the parallel plate 210 and exits at the same angle φ as the incident angle φ. The emitted light is shifted in parallel by d with respect to the incident light. Here, assuming that the thickness of the parallel plate is t, the refractive index of air is n, and the refractive index of the parallel plate 210 is n ′, the parallel shift amount d is expressed by the following equation (1).

From the above equation (1), the parallel shift amount d increases as the incident angle φ increases.

  Therefore, in FIG. 2A, when the parallel shift amounts of the light beam 110a, the optical axis L, and the light beam 110b are da, dl, and db, respectively, the relationship that dl is larger than da and db is larger than dl (da <dl < db) is obtained. This is because the angles at which the light beam 110a, the optical axis L, and the light beam 110b are incident on the half mirror 202 are different. The incident angle is such that the optical axis L is larger than the light beam 110 a and the light beam 110 b is larger than the optical axis L. At this time, the position where the light beam 110a intersects the optical axis L ′ and the position where the light beam 110b intersects the optical axis L ′ do not coincide with each other due to the above-described parallel shift, and the light beam 110b is closer to the right side in the figure than the light beam 110a. Crosses the optical axis L. This is the cause of the aberration caused by the half mirror 202.

  In FIG. 2, the description is made using only two light beams. However, in practice, an image is formed by a large number of light beams, and the positions where the light beams intersect on the optical axis L ′ are not all coincident in the cross section of FIG. 2. . FIG. 2B is an enlarged view of the vicinity of the central image height C of the image sensor 201 of FIG. 1, and shows the light rays that form an image. In FIG. 2B, 201a is the vicinity of the light receiving surface of the image sensor 201, that is, the planned image formation surface of the photographic lens 100, and a large number of light beams are gathered on the planned image determination surface 201a. However, due to the influence of the parallel shift by the half mirror 202 described in FIG. 2B, the positions where all the light beams intersect on the optical axis L ′ are different, and the image does not converge at one point.

  Since actual light rays are also present in the depth direction of the paper, more complicated aberration occurs. However, the half mirror 202 is arranged to be inclined with respect to the optical axis L only in the cross section of FIG. For this reason, the aberration has symmetry in the depth direction of the drawing. Further, the aberration situation varies depending on the refractive index of the parallel plate 210, that is, the half mirror 202, from the equation (1). Therefore, the situation of aberration varies depending on the wavelength. Further, from the equation (1), since the angle of the incident light beam varies depending on the image height, the situation of aberration also varies depending on the image height.

  FIG. 4 is an explanatory diagram of the aberration of light rays reaching the peripheral image height. In FIG. 4, light rays 111 a and 111 b indicate light rays that are imaged at the upper image height of the image sensor 201, and light rays 112 a and 112 b indicate light rays that are imaged at the lower image height of the image sensor 201. Since each light beam is incident on the half mirror 202 at different angles, the amount of parallel shift is also different from the principle described with reference to FIG. 3, and the light beams 111a and 111b and the light beams 112a and 112b have different aberrations. Further, since the parallel shift amount of the light beam formed at the upper image height and the lower image height is different from the light beam formed near the central image height of the image sensor 201 described with reference to FIG. Asymmetrical aberrations occur in the vertical direction. In the depth direction of the paper, the aberration has symmetry even in the peripheral image height for the reason described above. Also in the peripheral image height, the aberration situation varies depending on the wavelength. The actual state of aberration can be observed in detail by calculating a spot diagram, MTF, etc. using well-known optical simulation software.

  FIG. 5A is an explanatory diagram of ghost rays generated by internal reflection of the half mirror 202, and only necessary portions are extracted from FIG. In addition, the half mirror 202 is drawn thick to make it easier to see the path of the light beam. In FIG. 5A, light rays 113a and 113b emit one point on the object-side optical axis L, pass only through the lens unit 101, and form only near the center image height of the image sensor 201. The rays of light are shown. Note that light rays on the object side across the lens unit 101 are omitted. The light beams 113 a and 113 b are refracted by a predetermined angle on the incident surface of the half mirror 202 according to the well-known Snell's law and travel straight through the half mirror 202. On the exit surface of the half mirror 202, most of the light beam is transmitted as it is, and refracted by a predetermined angle according to Snell's law. This is the path of the light beam that is emitted without internal reflection as described in FIG.

  On the other hand, there is a reflected light beam on the exit surface of the half mirror 202. The light beam reflected by the exit surface of the half mirror 202 travels straight through the half mirror 202 and is separated into a reflected component and a transmitted component on the incident surface of the half mirror 202. The light beam reflected by the incident surface of the half mirror 202 travels straight through the half mirror 202 and reaches the exit surface of the half mirror 202 again. Of the light beam that has reached the exit surface of the half mirror 202, the transmitted component is refracted by a predetermined angle on the exit surface of the half mirror 202 again and exits according to Snell's law. In this way, it is reflected on the incident surface of the half mirror 202 after being reflected on the exit surface of the half mirror 202, passes through the exit surface of the half mirror 202 that has reached again, and reaches the image sensor 201. It is a ghost beam by internal reflection. At this time, the angle of the light ray incident on the half mirror 202 is equal to the angle of the light ray emitted. As is well known, this is a path of a ghost ray due to internal reflection of light on a parallel plate. The same applies to the optical axis L. That is, the light beam that has passed through the half mirror 202 does not change its angle, and only a parallel shift occurs.

  FIG. 6 is a detailed explanatory view of refraction and reflection of a ghost ray by internal reflection of the half mirror 202. In FIG. 6, reference numeral 210 denotes a parallel plate, 120 denotes a light beam incident on the parallel plate 210, and its traveling direction is indicated by an arrow. The light beam 120 enters the incident surface 210a of the parallel plate 210 at an incident angle φ, and the light beam 120 is refracted based on Snell's law. After refraction, the light travels straight through the parallel plate 210 at an angle φ ′. Thereafter, the light beam 120 is incident on the exit surface 210b of the parallel plate 210 at an angle φ ′ and is reflected at the angle φ ′. The reflected light beam 120 is incident on the incident surface 210a of the parallel plate 210 at an angle φ ′ and reflected at an angle φ ′. Again, the light beam 120 reaching the exit surface 210b of the parallel plate 210 is refracted and exits at the same angle φ as the incident angle φ. The emitted light is shifted in parallel by g with respect to the incident light. Here, when the plate thickness of the parallel plate is t, the refractive index of air is n, the refractive index of the parallel plate 210 is n ′, and the parallel shift amount when transmitted through the parallel plate is d, the light is emitted without being reflected internally. The difference e in the parallel shift amount between when the light is emitted and when it is reflected from the inner surface is expressed by the following equation (2).

Further, the parallel shift amount g when the inner surface is reflected by the parallel plate is expressed by the following formula (3) from the formulas (1) and (2).

From the above equation (3), it can be seen that the parallel shift amount g varies depending on the incident angle φ.

  Therefore, in FIG. 5A, assuming that the parallel shift amounts of the light beam 113a, the optical axis L, and the light beam 113b are ga, gl, and gb, respectively, gl is smaller than ga and gb is smaller than gl from Equation (3). The relationship (ga> gl> gb) is obtained. This is because the angles at which the light beam 113a, the optical axis L, and the light beam 113b are incident on the half mirror 202 are different. The incident angle is greater on the optical axis L than on the light beam 113a, and on the light beam 113b. It becomes a big relationship. At this time, the position at which the light beam 113a intersects with the optical axis L ″ and the position at which the light beam 113b intersects with the optical axis L ″ do not coincide with each other due to the effect of the parallel shift. Intersects with the optical axis L.

  The light beam emitted without being internally reflected by the half mirror 202 is shifted downward in the drawing with respect to the incident light beam as described with reference to FIG. On the other hand, the light beam emitted after being internally reflected by the half mirror 202 is shifted upward in the figure with respect to the incident light beam as described with reference to FIG. That is, the light beam that has passed through the half mirror 202 is separated into two light beams: a light beam that is shifted downward without being internally reflected and a light beam that is internally reflected and shifted upward. This is the cause of the double image due to the internal reflection of the half mirror 202.

  In FIG. 5, the description is made using only two light beams. However, in practice, an image is formed by a large number of light beams, and the positions where the light beams intersect on the optical axis L ″ are all coincident in the cross section of FIG. do not do. FIG. 5B is an enlarged view of the vicinity of the central image height C of the image sensor 201 of FIG. 1, and shows the light rays that form an image. In FIG. 5B, 201a indicates the vicinity of the light receiving surface of the image sensor 201, that is, the planned image formation surface of the photographic lens 100, and a large number of light rays are collected on the planned image determination surface 201a. However, due to the influence of the parallel shift by the half mirror 202 described with reference to FIG. 5A, the positions where all the light beams intersect on the optical axis L ″ are different and complete imaging is not performed.

  In addition, since actual light rays are also present in the depth direction of the paper, more complicated aberrations and internal reflection ghosts are generated, but the half mirror 202 is arranged with an inclination with respect to the optical axis L only in the cross section of FIG. Has been. For this reason, there are symmetrical aberrations and internal reflection ghosts in the depth direction of the drawing. Further, the aberration and the internal reflection ghost have different aberrations depending on the refractive index of the parallel flat plate 210, that is, the half mirror 202, from the equation (3). Therefore, the situation of aberration and internal reflection ghost also differs depending on the wavelength. Furthermore, from the equation (3), since the angle of the incident light beam varies depending on the image height, the situation of the aberration and the internal reflection ghost also varies depending on the image height.

  FIG. 7 is an explanatory diagram of an internally reflected ghost ray of a ray that reaches the peripheral image height. In FIG. 7, light beams 114 a and 114 b indicate light beams that are formed at the upper image height of the image sensor 201, and light beams 115 a and 115 b indicate light beams that are formed at the lower image height of the image sensor 201. Since each light beam is incident on the half mirror 202 at a different angle, the parallel shift amount is also different as described with reference to FIG. 6, and different aberrations and internal reflection ghosts are generated between the light beams 114a and 114b and the light beams 115a and 115b. It has occurred. Further, the parallel shift amount of the light beam formed at the upper image height and the lower image height is different from the light beam formed near the central image height of the image sensor 201 described with reference to FIG. For this reason, asymmetrical aberrations and internal reflection ghosts occur in the vertical direction of the image sensor 201 in FIG. In the depth direction of the paper, for the reasons described above, the aberration and the internal reflection ghost have symmetry even in the peripheral image height. Also, in the peripheral image height, the situation of the aberration and the internal reflection ghost varies depending on the wavelength. The actual aberration and internal reflection ghost situation can be observed in detail by calculating a spot diagram, MTF, etc. using well-known optical simulation software.

  From the above, in the image obtained by the light beam that has passed through the half mirror 202 as in the present embodiment, the following aberrations (1) to (3) and the internal reflection ghost are generated. . (1) A double image of a light beam emitted without internal reflection and a light beam emitted with internal reflection is generated. (2) Both light beams are symmetrical in the depth direction of the paper and have asymmetrical aberrations in the vertical direction. (3) The aberration varies depending on the image height and wavelength.

  Therefore, in this embodiment, the aberration and the internal reflection ghost are restored by the image processing circuit 413 to generate an image without deterioration due to the aberration. Hereinafter, the basic concept of aberration restoration will be described. FIG. 8 is a diagram illustrating a shooting range by the image sensor 201. In FIG. 8, reference numeral 220 denotes a shooting range, and the XY coordinates are defined as shown in the figure with the center of the shooting range as the origin. Then, the light amount distribution of the image before deterioration due to the aberration of the photographing lens 100 and the half mirror 202 and the internal reflection ghost at an arbitrary coordinate point (x, y) in the photographing range 220 is defined as o (x, y). Further, if the light quantity distribution of the image after degradation due to the aberration of the photographing lens 100 and the half mirror 202 corresponding thereto is i (x, y), i (x, y) is expressed by the following equation (4).

Here, h (x, y) is a transfer function representing a state in which an image is deteriorated by the photographing lens 100 and the half mirror 202. Expression (4) is expressed as a deteriorated image model expression including not only the aberration by the half mirror 202 and the internal reflection ghost but also the aberration of the photographing lens 100. Also, since Equation (4) is a convolution integral of o (x, y) and h (x, y), if Fourier transformation is performed, a simpler product form is obtained as in Equation (5) below. Can be represented.

Here, I (u, v) is i (x, y), O (u, v) is o (x, y), and H (u, v) is h (x, y). U and v are spatial frequency variables. Therefore, the image O (u, v) before the deterioration from the equation (5) is represented by the following equation (6).

Here, 1 / H (u, v) is generally called an inverse filter. By storing the inverse filter in advance in the nonvolatile memory 431 of the camera 200, it is possible to restore the image before the deterioration due to the aberration of the half mirror 202 and the internal reflection ghost. Note that actual image restoration processing is greatly affected by noise and the like, and thus is not limited to an inverse filter, and a known Wiener filter, a linear iterative correction method, a non-linear iterative correction method, or the like may be used. Further, the aberration and the internal reflection ghost caused by the half mirror 202 are Y-axis symmetric in FIG. Therefore, by preparing only half of the shooting range 220 as a restoration filter, it is possible to restore the entire image. Furthermore, since the filter takes into consideration not only the aberration of the half mirror 202 and the internal reflection ghost but also the aberration of the photographing lens 100, it is possible to restore the deterioration due to the aberration of the photographing lens.

  As described above, the image before deterioration due to aberration is restored on the half mirror 202. Since the aberration and the internal reflection ghost due to the half mirror 202 differ depending on the wavelength, the actual restoration by the image processing circuit 413 is performed using image data acquired for each RGB color. Details will be described later.

  FIG. 9 is a configuration diagram of the imaging apparatus in the present embodiment. FIG. 9 shows a camera system including a camera 200 and a photographing lens 100 that is interchangeably attached to the camera 200. In this embodiment, any of the camera 200, the photographing lens 100, or the camera system configured by the camera 200 and the photographing lens 100 can be applied as the imaging device. The present embodiment is not limited to this, and can also be applied to an image pickup apparatus in which a camera and a taking lens are configured integrally.

  With reference to FIG. 9, the camera 200 side will be described in detail first. Reference numeral 401 denotes a shutter for controlling the exposure amount, 201 denotes an image sensor that converts an optical image into an electric signal, and 412 denotes an A / D converter that converts an analog signal of the image sensor 201 into a digital signal (image data). A timing generation circuit 403 supplies a clock signal and a control signal to the image sensor 201, the A / D converter 412, and the D / A converter 404, and is controlled by the memory control circuit 407 and the system control circuit 440.

  Reference numeral 413 denotes an image processing circuit (image processing means) that performs predetermined pixel interpolation processing and color conversion processing on the image data from the A / D converter 412 or the image data from the memory control circuit 407. The aberration correction (image processing method) in this embodiment is performed by, for example, the image processing circuit 413. The image processing circuit 413 can also perform predetermined arithmetic processing using image data. Based on the obtained calculation result, the system control circuit 440 performs processing such as AF (autofocus) processing and AE (automatic exposure) processing on a focus control unit 454 and an aperture control unit 455 on the photographing lens 100 side described later. I do. Further, the image processing circuit 413 also performs TTL AWB (auto white balance) processing based on the obtained calculation result.

  A face detection unit 409 corresponds to a human face as a target image to be detected. A predetermined face detection operation is performed on the image data from the image processing circuit 413 or the image data from the memory control circuit 407. The memory control circuit 407 controls the A / D converter 412, the timing generation circuit 403, the image processing circuit 413, the image display memory 408, the D / A converter 404, the face detection unit 409, the memory 410, and the compression / decompression circuit 411. . Data from the A / D converter 412 is written into the image display memory 408 or the memory 410 via the image processing circuit 413 and the memory control circuit 407, or data from the A / D converter 412 is directly written through the memory control circuit 407. It is.

  Reference numeral 405 denotes an image display unit composed of a liquid crystal monitor or the like, which displays image data written in the image display memory 408 via the D / A converter 404. If the image data captured using the image display unit 405 is sequentially displayed, an electronic viewfinder function can be realized. Further, the image display unit 405 can arbitrarily turn on / off the display according to an instruction from the system control circuit 440. When the display is turned off, the power consumption of the camera 200 can be significantly reduced. it can.

  Reference numeral 410 denotes a memory for storing captured still images and moving images, and has a sufficient storage capacity for storing a predetermined number of still images and a moving image for a predetermined time. Thus, even in continuous shooting or panoramic shooting in which a plurality of still images are continuously shot, high-speed and large-capacity image writing can be performed on the memory 410. The memory 410 can also be used as a work area for the system control circuit 440.

  A compression / decompression circuit 411 compresses / decompresses image data by applied discrete cosine transform (ADCT) or the like, reads an image stored in the memory 410 and performs compression processing or decompression processing, and stores the processed image data in the memory 410. Write to. A shutter control unit 420 controls the shutter 401 in cooperation with a diaphragm control unit 455 that controls the diaphragm 452 on the photographing lens 100 side based on photometric information from the photometric unit 422. Reference numeral 423 denotes an interface (I / F) for electrically connecting the camera 200 and the photographing lens 100, and reference numeral 443 denotes a connector for electrically connecting the camera 200 to the photographing lens 100.

  Reference numeral 421 denotes an AF unit for performing AF processing. By using the well-known contrast method AF that performs focus control of the photographic lens 100 while evaluating the contrast component of the image data obtained by the image sensor 201, the focus state of the photographic lens 100 on the image sensor 201 is measured. Yes. A photometric unit 422 performs AE processing. The exposure state of the image can be measured by causing the light beam that has passed through the photographing lens 100 to enter the photometric unit 422 via the camera mount 442, the half mirror 202, and a photometric lens (not shown). The photometry unit 422 also has an EF processing function in cooperation with the flash unit 424. Similar to the AF unit 421, the system control circuit 440 controls the shutter control unit 420 and the aperture control unit 455 of the photographing lens 100 based on the calculation result obtained by calculating the image data of the image sensor 201 by the image processing circuit 413. It is also possible to perform AE control. A flash unit 424 has an AF auxiliary light projecting function and a flash light control function.

  A system control circuit 440 controls the entire camera 200, and a memory 429 stores constants, variables, programs, and the like for operating the system control circuit 440. Reference numeral 430 denotes a display unit such as a liquid crystal display device and a speaker that displays an operation state, a message, and the like using characters, images, sounds, and the like according to program execution in the system control circuit 440. One or a plurality of locations are provided near the operation unit of the camera 200 so as to be easily visible, and are configured by a combination of, for example, an LCD, an LED, a sound generation element, and the like. In addition, the display unit 430 is partially installed in the optical viewfinder 445. Details of the optical viewfinder 445 will be described later. Among the display contents of the display unit 430, what is displayed on the LCD or the like includes information on the number of shots such as the number of recorded images and the number of remaining shots, information on shooting conditions such as shutter speed, aperture value, exposure correction, and flash. There is. In addition, the remaining battery level, date and time are also displayed. Information displayed in the optical viewfinder 445 includes information such as an in-focus state, a camera shake warning, a flash charging state, a shutter speed, an aperture value, and exposure correction.

  Reference numeral 431 denotes an electrically erasable / recordable nonvolatile memory such as an EEPROM. Reference numerals 433, 434, 435, 436, 437, and 438 denote operation units for inputting various operation instructions of the system control circuit 440. One or a plurality of switches, dials, touch panels, pointing by line-of-sight detection, voice recognition devices, and the like. Consists of A dial mode switch 433 can be set by switching various modes such as power off, auto shooting mode, manual shooting mode, panoramic shooting mode, macro shooting mode, playback mode, multi-screen playback / erase mode, and PC connection mode. it can. Reference numeral 434 denotes a shutter switch SW1, which is turned on when a shutter button (not shown) is half-pressed and instructs to start operations such as AF processing, AE processing, AWB processing, and EF processing.

  Reference numeral 435 denotes a shutter switch SW2, which is turned on when a shutter button (not shown) is fully pressed, and instructs the start of a series of processing related to photographing. The process related to photographing is an exposure process, a development process, a storage process, and the like. In the exposure process, image data is written into the memory 410 via the A / D converter 412 and the memory control circuit 407 from the signal read from the image sensor 201. In the development processing, development is performed using calculation in the image processing circuit 413 and the memory control circuit 407. In the recording process, image data is read from the memory 410, compressed by the compression / decompression circuit 411, and written to the recording medium 485 or 491. An image display ON / OFF switch 436 can set ON / OFF of the image display unit 405. With this function, when photographing is performed using the optical finder 445, it is possible to save power by cutting off the current supply to the image display unit 405 including a liquid crystal monitor or the like. Reference numeral 437 denotes a quick review ON / OFF switch, which can set ON / OFF of a quick review function for automatically reproducing image data taken immediately after photographing. An operation unit 438 includes various buttons and a touch panel. The various buttons include a menu button, a flash setting button, a single shooting / continuous shooting / self-timer switching button, a selection moving button, a shooting image quality selection button, an exposure correction button, a date / time setting button, and the like. Further, the operation unit 438 includes a face detection mode setting button for setting whether or not to perform face detection by the face detection unit 409.

  Reference numeral 425 denotes a power supply control unit, which includes a battery detection circuit, a DC / DC converter, a switch circuit that switches a block to be energized, and the like. The power supply control unit 425 detects the presence / absence of a battery, the type of battery, and the remaining battery level, controls the DC / DC converter based on the detection result and an instruction from the system control circuit 440, and requires a necessary voltage. It is supplied to each part including the recording medium for a period. Reference numerals 426 and 427 denote connectors, and reference numeral 428 denotes a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCD battery, NiMH battery or Li-Ion battery, an AC adapter, or the like.

  Reference numerals 480 and 486 denote interfaces with recording media such as a memory card and a hard disk, and reference numerals 481 and 487 denote connectors for connecting with recording media such as a memory card and a hard disk. Reference numeral 439 denotes a recording medium attachment / detachment detection unit that detects whether a recording medium is attached to the connector 481 or 487.

  In the present embodiment, the description is given assuming that there are two systems of interfaces and connectors for attaching the recording medium. However, the present embodiment is not limited to this, and the interface and connector to which the recording medium is attached may have a single or plural number of systems. Moreover, it is good also as a structure provided with combining the interface and connector of a different standard. Further, the interface and the connector may be configured using a PCMCIA card or the like that conforms to the standard. In addition, by connecting various communication cards such as LAN cards and modem cards to the interface and connector, image data and management information attached to the image data can be transferred to and from other peripheral devices such as computers and printers. Can do.

  Reference numeral 445 denotes an optical viewfinder as a viewfinder means, which mainly includes a focus plate 203, a pentaprism 204, and an eyepiece optical system 205. The light beam that has passed through the photographing lens 100 can be guided through the camera mount 442 and the half mirror 202 and observed as an optical image. As a result, it is possible to perform shooting using only the optical viewfinder 445 without using the electronic viewfinder function of the image display unit 405.

  A communication unit 432 has various communication functions such as RS232C, USB, IEEE1394, P1284, SCSI, modem, LAN, and wireless communication. Reference numeral 446 denotes a connector for connecting the camera 200 to another device by the communication unit 432 or an antenna in the case of wireless communication. The connector 443 has a function of supplying control signals, status signals, data signals, and the like between the camera 200 and the photographing lens 100 and supplying currents of various voltages. Further, the connector 443 may be configured to transmit not only electrical communication but also optical communication, voice communication, and the like.

  Reference numeral 202 denotes an entire surface half mirror that reflects the speed of light passing through the photographing lens 100 to the optical finder 445 and guides the light beam that has passed through the half mirror 202 to the image sensor 201. Further, a part of the speed of light transmitted through the half mirror 202 may be guided to the AF unit 421 via an AF sub mirror (not shown). In this embodiment, the half mirror 202 is a full-surface half mirror without a movable mechanism, but may be a quick return mirror.

  Reference numerals 485 and 491 denote recording media such as a memory card and a hard disk. The recording media 485 and 491 are provided with recording units 484 and 490 constituted by a semiconductor memory, a magnetic disk, and the like, interfaces 483 and 489 with the camera 200, and connectors 482 and 488 for connecting with the camera 200.

  Next, the photographing lens 100 will be described in detail. Reference numeral 458 denotes a lens mount that mechanically couples the photographing lens 100 to the camera 200 and is attached to the camera 200 via the camera mount 442 so as to be replaceable. The camera mount 442 and the lens mount 458 include various functions for electrically connecting the photographing lens 100 to the camera 200. Reference numeral 101 denotes a lens unit, and 452 denotes an aperture. The lens unit 101 includes a focus lens that adjusts the focus of the subject.

  Reference numeral 459 denotes a connector for electrically connecting the photographing lens 100 to the camera 200, and reference numeral 457 denotes an interface (I / F) for connecting to the connector 443 of the camera 200. The connector 459 has a function of transmitting and supplying a control signal, a status signal, a data signal, and the like between the camera 200 and the photographing lens 100, and supplying or supplying currents of various voltages. Note that the connector 459 may be configured to transmit not only electrical communication but also optical communication, voice communication, and the like.

  Reference numeral 453 denotes a zoom control unit that controls zooming of the lens unit 101, and reference numeral 454 denotes a focus control unit that controls the focus lens operation of the lens unit 101. Note that if the photographing lens 100 is a single focus lens type without a zoom function, the zoom control unit 453 may be omitted. Reference numeral 455 denotes an aperture control unit that controls the aperture 452 in cooperation with the shutter control unit 420 that controls the shutter 401 based on photometry information from the photometry unit 422.

  A lens control unit 456 controls the entire photographing lens 100. The lens control unit 456 has a memory function for storing constants, variables, programs, and the like for taking lens operations. In addition, it also has a non-volatile memory function for storing identification information such as a number unique to the photographing lens, management information, function information such as an open aperture value, minimum aperture value, and focal length, and current and past set values. The above is the configuration of the imaging apparatus in the present embodiment.

  10 to 14 are a flowchart of the operation of the image pickup apparatus in the present embodiment and an explanatory diagram thereof. FIG. 10 is a main flow of the image pickup apparatus in the present embodiment. The main flow operation shown in FIG. 10 is performed by the system control circuit 440. When the photographer turns on a power switch provided on the operation unit 438 of the camera (step S600), the system control circuit 440 confirms the operation of each actuator and the image sensor 201 in the camera. Further, the system control circuit 440 initializes the memory contents and the execution program, and executes the shooting preparation operation (step S601).

  In step S602, preparation processing for performing moving image shooting is executed. The moving image shooting preparation processing includes lens communication processing, shutter opening processing, preview image display start processing, focus adjustment processing, exposure control processing, and the like. In the lens communication process, lens communication is performed with the lens control unit 456 in the photographing lens 100 via the connectors 443 and 459. The operation of the photographic lens 100 is confirmed by lens communication, the memory contents of the photographic lens 100 and the execution program are initialized, and a preparatory operation is executed. Further, various characteristic data of the photographing lens 100 necessary for focus detection and imaging are acquired and stored in the memory 429.

  In the shutter opening process, the system control circuit 440 drives the shutter 140 via the shutter control unit 420 to make the shutter open. With this operation, an open F number photographing light beam that has passed through the photographing lens 100 reaches the image sensor 201. In the preview image display start process, the image data written in the image display memory 408 for preview is displayed via the D / A converter 404. The image is displayed on an image display unit 405 provided on the back of the camera. The photographer looks at the electronic image to determine the composition at the time of photographing.

  In the focus adjustment process, the focus lens operation of the lens unit 101 is controlled via the focus control unit 454, and the focus is adjusted so that the focus state of the photographing lens 100 is in focus. The AF evaluation value is acquired while performing a so-called wobbling drive that reciprocates the focus lens of the lens unit 101 in the optical axis direction. If the magnitude of the change in the AF evaluation value due to the reciprocating movement of the focus lens is within a predetermined amount, it is determined that the photographing lens 100 is in focus, and the reciprocating movement is performed at the focus lens position as it is. When the fluctuation of the AF evaluation value is larger than a predetermined amount, it is determined that the focus is not in focus, and the focus lens position is moved in the direction of the focus position while being driven by wobbling. Here, the AF evaluation value is contrast information of image data. As contrast information, the absolute sum of differences between adjacent pixel outputs of image data is often used.

  In the exposure control process, an exposure condition is set based on the pixel output value obtained from the image sensor 201. The exposure control is performed by a combination of the sensitivity of the image sensor 201, the accumulation time of the image sensor 201, and the F number of the photographing lens 100. By this operation, the image sensor 201 can obtain an appropriate pixel output by performing a photoelectric conversion process on the subject image by the photographing light flux having a predetermined F number with a predetermined sensitivity and a predetermined accumulation time.

  In step S603, it is determined whether a moving image shooting start operation has been performed from the operation unit 438. If the moving image shooting start operation has not been performed, the process proceeds to step S607, and it is determined whether or not the power switch has been turned off. If a moving image shooting start operation has been performed, the process proceeds to step S604. In step S604, a one-frame processing subroutine is executed to perform a shooting operation for one frame in moving image shooting. In step S605, it is determined whether or not a moving image shooting end operation has been performed from the operation unit 438. If the moving image shooting end operation has not been performed, the process returns to step S604 to continue moving image shooting. When the moving image shooting end operation is performed, the process proceeds to step S606, and the moving image shooting end process is executed. In step S606, processing for ending moving image shooting is performed. The moving image shooting end process includes a preview image display end process, a shutter closing process, and a lens communication process.

  In the preview image display end process, the operation of displaying the image data on the image display unit 405 for preview is ended. In the shutter closing process, the shutter 140 is driven via the shutter control unit 420 to close the shutter. By this operation, the photographing light flux that has passed through the photographing lens 100 does not reach the image sensor 201. In the lens communication process, lens communication is performed with the lens control unit 456 in the photographing lens 100 via the connectors 443 and 459. An operation instruction is given to the photographing lens 100 to end moving image shooting through lens communication, and the memory contents and the execution program of the photographing lens 100 are shifted to the moving image shooting preparation state. In step S607, it is determined whether or not the power switch is turned off. If the off operation has not been performed, the process returns to step S602. When an off operation is performed, a series of operations are terminated.

  FIG. 11 is an operation flow of the one-frame processing subroutine (step S604). The system control circuit 440 controls the one-frame processing subroutine. When step S604 of the main routine is executed, a one-frame processing subroutine S604 is called, and in step S650, the focus is adjusted so that the focus state of the photographic lens 100 is in focus. The AF evaluation value is acquired while performing a so-called wobbling drive that reciprocates the focus lens position of the lens unit 101 in the optical axis direction. When the magnitude of the fluctuation of the AF evaluation value due to the reciprocating movement of the focus lens position is within a predetermined amount, it is determined that the photographing lens 100 is in an in-focus state, and the reciprocating movement is performed with the focus lens position as it is. When the fluctuation of the AF evaluation value is larger than a predetermined amount, it is determined that the focus is not in focus, and the focus lens position is moved in the direction of the focus position while being driven by wobbling.

  In step S651, exposure control is performed based on the pixel output value obtained from the image sensor 201. The exposure control is performed by a combination of the sensitivity of the image sensor 201, the accumulation time of the image sensor 201, and the F number of the photographing lens 100. By this operation, the image sensor 201 can obtain an appropriate pixel output by performing a photoelectric conversion process on the subject image by the photographing light flux having a predetermined F number with a predetermined sensitivity and a predetermined accumulation time. In step S652, the accumulation operation of the image sensor 201 is performed. Charge accumulation is performed based on the accumulation time set in the exposure control in step S651.

  In step S653, image data obtained by accumulating charges is read. Since it is necessary to read out image data at a high frame rate such as 30 frames per second or 60 frames per second during moving image shooting, the number of pixels is limited. Therefore, at the time of reading the image data in step S653, thinning-out reading and addition reading are performed so that the number of pixels that can be read at the frame rate for the moving image is accommodated. In step S654, the image processing circuit 413 performs image processing such as image γ correction, color conversion, and edge enhancement.

  In step S655, the characteristic information of the camera 200 (camera body) is recorded in the memory 429 in correspondence with the image data recorded in step S658. Here, the characteristic information of the camera 200 includes the position and thickness information of the half mirror 202, the refractive index and Abbe number information of the half mirror 202, the light receiving sensitivity distribution information of the image sensor 201, and the vignetting information of the photographing light beam in the camera 200. Is included. Further, the characteristic information includes distance information from the mounting surface of the camera 200 and the photographing lens 100 to the image sensor 201, manufacturing error information, and the like. Since the light reception sensitivity distribution information of the imaging pixels of the imaging element 201 is determined by the on-chip microlens and the photoelectric conversion unit, these pieces of information may be recorded.

  In step S656, the characteristic information of the photographing lens 100 is recorded in the memory 429 in correspondence with the image data recorded in step S658. Here, the characteristic information of the photographing lens 100 includes exit pupil information, frame information, F number information at the time of photographing, aberration information, manufacturing error information, and the like. In step S657, an aberration and internal reflection ghost correction subroutine is executed. It corrects image degradation caused by both the aberration caused by transmission through the half mirror 202 and the double image caused by internal reflection of the half mirror 202. The correction process is performed by the image processing circuit 413. In step S658, images are sequentially recorded on the recording medium 485 or the recording medium 491 with respect to the image data subjected to aberration correction by the image processing circuit 413. That is, the image data stored in the recording medium 485 or the recording medium 491 is stored with the aberration corrected.

  In the image pickup apparatus of the present embodiment, the transfer function is estimated based on the photographing lens characteristic information and the camera body characteristic information for the image data in which the aberration and the internal reflection ghost are generated by the half mirror 202, and the inverse transformation is performed. Aberration correction processing is performed by the image processing circuit 413. Therefore, the image data recorded in step S658 is an image in which the aberration by the half mirror 202 and the internal reflection ghost are corrected. As a result, an imaging apparatus capable of capturing high-quality moving images while looking at the optical viewfinder 445 can be realized with an inexpensive configuration.

In step S659, the recorded image is displayed on the image display unit 405 provided on the back of the camera. The photographer can visually confirm the electronic image and determine the composition at the time of photographing, and can confirm the recorded image.
When step S659 ends, step S604 ends the one-frame processing subroutine and returns to the main routine.

FIG. 12 shows a state in which transfer function generation and correction processing are sequentially performed on the divided imaging ranges in the aberration and internal reflection ghost correction subroutine (step S657). The right direction on the paper surface is the + X direction, and the upward direction on the paper surface is the + Y direction. Reference numeral 220 denotes an imaging range obtained from the image sensor 201. Area (m, n) (m = 1 to 5-1, n = 1 to 5) is a divided region obtained by dividing the imaging range 220 by 5 × 5 = 25. The lower left area is Area (1,1), the right area (+ X direction) is Area (2,1), the upper area (+ Y direction) is Area (1,2), etc. M increases by one as it goes in the (+ X direction), and n increases by one as it goes up (in the + Y direction). The central area is Area (3, 3), and the upper right end area is Area (5, 5). (Definition of coordinate system)
The aberration caused by the half mirror 202 being inclined at an angle of 45 ° in the photographing optical path differs depending on the position in the XY direction in the image data. Therefore, as shown in FIG. 12, the imaging range 220 is divided into 5 × 5 = 25, a transfer function is defined for each region, and aberration correction is performed based on the transfer function. As described with reference to FIGS. 2 to 7, the transfer function defined here has a shape that is symmetric in the depth direction of the paper and asymmetric in the vertical direction. Thereby, aberration correction suitable for the position in the image data can be performed.

  Of the divided areas, Area (1, 1) to Area (5, 5), first, for Area (1, 1), the transfer function definition, the aberration, and the internal reflection ghost are corrected. When Area (1, 1) is completed, the area is then moved to the right (+ X direction), and transfer function definition, aberration, and internal reflection ghost correction are performed for Area (1, 2). When reaching the right end, the area is moved to Area (2, 1) which is the upper left one, and the transfer function definition, aberration, and internal reflection ghost are corrected. In this way, the region and the internal reflection ghost are corrected by moving the region one by one so as to scan in the X and Y directions, and the aberration and the internal reflection ghost are corrected for the entire region of the imaging range 220.

  The aberration and the internal reflection ghost corrected in the aberration and internal reflection ghost correction subroutine (step S657) are asymmetrical in the vertical direction but symmetrical in the horizontal direction. Therefore, as the transfer function defined for each divided region, the transfer function at a symmetrical position in the left-right direction can be the same shape reversed left and right. For example, Area (1, 5) and Area (5, 5) can be substituted with the same shape reversed left and right. As a result, the number of transfer function data to be defined can be reduced. In this embodiment, 5 × 5 25 divisions have been described in consideration of the processing speed. However, it may be divided into a larger number so as to correct aberrations and internal reflection ghosts with high accuracy.

  FIG. 13 is a flowchart of the aberration and internal reflection ghost correction subroutine (step S657). The image processing circuit 413 also performs a series of operations of a correction subroutine for aberration and internal reflection ghost. In step S700, conversion information indicating the content of the conversion process performed in the conversion process in step S654 at the time of image data capture is acquired.

  In step S701, a conversion method to be applied to the image data is determined before performing the correction process. In order to ensure linearity, which is a precondition for the image restoration processing algorithm, a method of converting image data is determined so that the exposure value and the pixel value are in a proportional relationship. For example, when the image processing circuit 413 performs gamma correction in the conversion process of step S654 when capturing image data, in step S701, inverse conversion of conversion by gamma correction is set. As a result, it is possible to reproduce the image data before conversion, and to acquire image data having linearity. Similarly, when the image processing circuit 413 performs color correction in the conversion process of step S654 at the time of image data capture, in step S701, reverse conversion of conversion by color conversion is set. Thereby, it is possible to acquire image data having linearity. As described above, in step 701, a conversion method corresponding to the inverse conversion of the conversion process performed by the image processing circuit 413 in the conversion process in step S654 at the time of capturing image data is determined.

  Since the present embodiment is described on the assumption that aberration correction is performed on image data that has been subjected to conversion processing that does not maintain linearity, such pre-correction conversion is performed. However, when image data with linearity can be acquired, aberration correction may be performed without performing such pre-correction conversion.

  In step S702, image data is acquired. In step S703, the acquired image data is converted according to the conversion method determined in step S701. In step S704, an imaging range (correction target area) for correcting the aberration and the internal reflection ghost is set to Area (1, 1) that is the initial position. In step S705, a transfer function generation subroutine is executed to generate a transfer function in the set area. Here, a transfer function for each RGB is generated. Further, the transfer function generated here has a vertically asymmetric shape as described with reference to FIGS.

  In step S706, the aberration and internal reflection ghost correction processing is performed on the image data converted in step S703 based on the transfer function generated in step S705 as described in equation (6). Since the transfer function differs for each RGB, the aberration and the internal reflection ghost correction by the inverse transformation process are performed for each RGB signal. Here, the correction processing of the aberration and the internal reflection ghost is performed by the image restoration processing described in Expression (6). Thereby, an image in which the aberration of the predetermined subject and the internal reflection ghost are corrected can be obtained. An image restoration method by performing the inverse transformation process of the transfer function is disclosed in, for example, Japanese Patent Application Laid-Open No. 2000-20691, and the description thereof is omitted.

  In step S707, it is determined whether or not the shooting range for correcting the aberration and the internal reflection ghost has reached Area (5, 5). When reaching Area (5, 5), it is determined that the correction of all areas has been completed, the aberration and internal reflection ghost correction subroutine (step S657) is terminated, and the process returns to the one-frame processing subroutine of step S604. If the area has not reached Area (5, 5), the process proceeds to step S708 to move the correction target area to the next area.

  In step S708, the region is moved to the right (+ X direction). When reaching the right end, the region is moved to the left end one level up. As described above, correction is performed by moving the regions one by one so as to scan in the X and Y directions, thereby sequentially correcting the aberration and the internal reflection ghost for the entire region of the imaging range 220. When step S708 is completed, the process returns to step S705 to generate a transfer function in the moved area.

  FIG. 14 is a flow of the transfer function generation subroutine (step S705). A transfer function is generated in the divided area Area (m, n) (m = 1 to 5-1, n = 1 to 5) obtained by dividing the imaging range 220 by 5 × 5 = 25. The system control circuit 440 performs a series of operations of the transfer function generation subroutine. In step S750, the characteristic information of the camera 200 (camera body) recorded in the memory 429 in step S655 is acquired. In step S751, the characteristic information of the photographing lens 100 recorded in the memory 429 in step S656 is acquired. In step S752, a parameter used when defining a transfer function is acquired. The transfer function is determined by the light transfer characteristic between the photographing lens 100 and the image sensor 201. The light transfer characteristics are different for each RGB color. Further, it varies depending on factors such as the characteristic information of the camera 200, the characteristic information of the photographing lens 100, the position in the XY direction in the image data, and the subject distance. Therefore, table data in which these factors are associated with parameters used when defining the transfer function is stored in the nonvolatile memory 431 in advance. When step S752 is executed, the system control circuit 440 acquires parameters used when defining the transfer function for each RGB color from the nonvolatile memory 431 based on these factors.

  In step S753, a transfer function for the R signal is defined based on the parameter acquired in step S752. This transfer function is considered to be an optical transfer function characteristic at the R wavelength from the photographing lens 100 to the image sensor 201. In step S754, a transfer function for the G signal is defined based on the parameter acquired in step S752. This transfer function is considered to be an optical transfer function characteristic at the G wavelength from the photographing lens 100 to the image sensor 201.

  In step S755, a transfer function for the B signal is defined based on the parameter acquired in step S752. This transfer function is considered to be an optical transfer function characteristic at the B wavelength from the photographing lens 100 to the image sensor 201. When step S755 ends, the step S705 transfer function generation subroutine ends, and the process returns to the aberration correction subroutine (step S657).

  According to the present embodiment, it is possible to realize a digital single-lens reflex camera (imaging device) capable of shooting a moving image with an optical viewfinder with a small and inexpensive configuration. In addition, since the aberration and the internal reflection ghost generated by the optical element are corrected, a high-quality moving image can be obtained.

  In step S657, the image processing circuit 413 according to the present exemplary embodiment reflects the aberration caused by the incident light beam passing through the half mirror 202 and the internal reflection by the half mirror 202 with respect to the image data output from the image sensor 201. Both of the deterioration caused by this are corrected. However, the present embodiment is not limited to this, and the image processing circuit 413 may be configured to correct only one of such aberration and deterioration.

  Next, an image pickup apparatus in Embodiment 2 of the present invention will be described. The present embodiment is a modification of the first embodiment, and a camera system is configured by attaching a short flange back photographing lens to a camera. FIG. 15 is a central cross-sectional view showing an optical system of a single-lens reflex camera in the present embodiment. In FIG. 15, members similar to those in FIG. 1 are denoted by the same symbols, and description thereof is omitted. Reference numeral 300 denotes a photographing lens that is attached to the camera 200 in an exchangeable manner. Reference numeral 301 denotes a lens unit including a focus lens group and a zoom lens group, and L denotes an optical axis of the photographing lens 100. FB is the distance from the rear end of the photographic lens 300 to the image sensor 201, so-called flange back, which is shorter than that in the first embodiment.

  Further, in a state where the photographic lens 300 is attached to the camera 200, the rear side of the photographic lens 300 and the tip of the half mirror 202 overlap each other by an amount indicated by D in the direction parallel to the optical axis L. That is, when the photographing lens 300 and the half mirror 202 are projected on a plane parallel to the optical axis L of the photographing lens 300, at least a part of the photographing lens 300 and the half mirror 202 overlap each other. Thus, since the half mirror 202 of the camera 200 is fixed and arranged without rotating, the photographing lens 300 can be arranged more inside the camera 200. Therefore, according to the present embodiment, the entire size including the photographing lens 300 and the camera 200 can be reduced.

  In the present embodiment, the case where the photographing lens 300 and the half mirror 202 partially overlap each other has been described. However, even when they do not overlap, the size can be reduced as compared with the first embodiment. On the other hand, in the case of the short flange back, the position of the exit pupil of the photographic lens 300 is close to the half mirror 202. Therefore, the aberration caused by the half mirror 202 and the change due to the image height of the internal reflection ghost more greatly occur. Therefore, in this embodiment, when the transfer function generation and correction processing are sequentially performed on the divided photographing ranges, the photographing ranges are divided more finely.

  FIG. 16 is a diagram showing a state in which transfer function generation and correction processing are sequentially performed on the divided imaging ranges in the aberration and internal reflection ghost correction subroutine (step S657) of the present embodiment. Components equivalent to those in FIG. 12 are denoted by the same reference numerals, and description thereof is omitted. In the first embodiment, the photographing range 220 is divided into 5 × 5 = 25. On the other hand, in this embodiment, 7 × 7 = 49 divisions are made. As a result, the short flange back enables good correction even when the aberration caused by the half mirror 202 and the change in the image height of the internal reflection ghost are large.

  Next, Embodiment 3 of the present invention will be described. The present embodiment is a modification of the first embodiment, and is an example in which the correction procedure for aberration and internal reflection ghost is partially changed. FIG. 17 is a diagram illustrating assignment of deterioration functions in the present embodiment. In the first embodiment, aberration correction occurring in each of the photographing lens 100 and the half mirror 202 is expressed as a transfer function without being distinguished. On the other hand, in this embodiment, it is possible to individually correct the generated aberration by using the transfer functions of the photographing lens 100 and the half mirror 202, respectively. Only the parts different from the first embodiment will be described below.

In the present embodiment, the light amount distribution I _l (u, v) after being deteriorated by the photographing lens 100 is expressed by the following equation (7).

Here, H _l (u, v) is a transfer function representing a state in which an image is deteriorated by the photographing lens 100. Further, the light amount distribution I _m (u, v) after deterioration due to transmission through the half mirror 202 is expressed by the following equation (8).

Here, H _m (u, v) is a transfer function representing a state in which the image is degraded by the half mirror 202. From Expressions (7) and (8), the light quantity distribution I _m (u, v) considering both the photographic lens 100 and the degradation caused by the half mirror 202 can be expressed as the following Expression (9).

Here, H _m (u, v) is a function unique to the camera 200. On the other hand, H _l (u, v) is a function determined by the taking lens 100. For this reason, in a camera system (imaging device) using an interchangeable photographic lens, the light amount distribution I _m (u, v) changes every time the photographic lens changes. By reading the inverse filter 1 / H _m (u, v) stored in advance in the nonvolatile memory 431 and the like, and reading the inverse filter 1 / H _l (u, v) from the photographing lens 100 via the I / F 423. The aberration can be corrected.

  The image processing circuit 413 according to the present exemplary embodiment independently corrects aberrations caused by the incident light flux passing through the photographing lens 100 with respect to the image data output from the image sensor 201. For this reason, according to the present embodiment, the deterioration function can be changed according to the interchangeable photographic lens, and a high-quality moving image can be obtained.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

DESCRIPTION OF SYMBOLS 100 Shooting lens 200 Camera 201 Image pick-up element 202 Half mirror 413 Image processing circuit 445 Optical finder

Claims (5)

An optical element that is fixedly disposed obliquely with respect to the optical axis of the interchangeable lens, reflects a part of the incident light beam, and transmits the other part;
Finder means configured to be able to observe a subject using the first light beam reflected by the optical element;
Imaging means for receiving the second light flux transmitted through the optical element and outputting image data;
On the image data output from the imaging means, the incident light beam is closed and an image processing means for correcting the degradation caused by internal reflection in the optical element,
When the interchangeable lens and the optical element are projected on a plane parallel to the optical axis of the interchangeable lens in a state where the interchangeable lens is attached to the imaging device, at least a part of the interchangeable lens and the optical element is An imaging device characterized by overlapping .   The imaging apparatus according to claim 1, wherein the image processing unit further corrects an aberration caused by the incident light beam passing through the optical element.   The imaging apparatus according to claim 1, wherein the image processing unit further corrects an aberration caused by the incident light beam passing through the interchangeable lens. An optical element that is fixedly disposed obliquely with respect to the optical axis of the interchangeable lens, reflects a part of the incident light beam, and transmits the other part;
Finder means configured to be able to observe a subject using the first light beam reflected by the optical element;
Image pickup means for receiving the second light flux transmitted through the optical element and outputting image data;
When the interchangeable lens and the optical element are projected on a plane parallel to the optical axis of the interchangeable lens in a state where the interchangeable lens is attached to the imaging device, at least a part of the interchangeable lens and the optical element is An image processing method for overlapping imaging devices,
Obtaining image data from the imaging means;
Correcting the deterioration caused by the internal reflection of the incident light beam by the optical element with respect to the acquired image data. Setting a region to be corrected out of the pre-Symbol image data,
Generating a transfer function in the area to be corrected ,
Wherein the step of correcting, based on the correction target region, based of the transfer function, to claim 4 wherein the incident light beam is characterized by a Turkey to correct the degradation caused by internal reflection in the optical element The image processing method as described .