JP2006311472A - Imaging apparatus, imaging system, and imaging method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus, imaging system and imaging method which enable a single apparatus to restore an image obtained by a plurality of optical systems. <P>SOLUTION: The imaging system includes: a plurality of imaging apparatuses 100A-100C, each having an optical system 110, a phase plate (wavefront coding optical element) 120, an imaging element 130 for imaging an object aberration image which has passed through the optical system 110 and the phase plate 120, an imaging apparatus side storage unit 140 for storing coefficient specifying information for specifying a convolution coefficient, and a transmission device 150 for transmitting the object aberration image data obtained from the imaging apparatus 130 and the coefficient specifying information stored in the storage unit 140; and a processing device 200 for acquiring a coefficient from the plurality of prestored convolution coefficients, according to the coefficient specifying information in accordance with the zoom position or zoom amount transmitted from the imaging apparatus 100 and generating object image data having no dispersion from the dispersion image signal supplied from the imaging element 130. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and an imaging method, such as a digital still camera, a mobile phone camera, a mobile information terminal camera, and the like that use an imaging device and have a zoom optical system.

In response to the digitization of information, which has been rapidly developing in recent years, the response in the video field is also remarkable.
In particular, as symbolized by a digital camera, a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) sensor, which is a solid-state image sensor, is used in most cases instead of a conventional film.

  As described above, an imaging lens device using a CCD or CMOS sensor as an imaging element is for taking an image of a subject optically by an optical system and extracting it as an electrical signal by the imaging element. In addition to a digital still camera, It is used for a video camera, a digital video unit, a personal computer, a mobile phone, a personal digital assistant (PDA) and the like.

FIG. 17 is a diagram schematically illustrating a configuration and a light flux state of a general imaging lens device.
The imaging lens device 1 includes an optical system 2 and an imaging element 3 such as a CCD or CMOS sensor.
In the optical system, the object side lenses 21 and 22, the diaphragm 23, and the imaging lens 24 are sequentially arranged from the object side (OBJS) toward the image sensor 3 side.

In the imaging lens device 1, as shown in FIG. 17, the best focus surface is matched with the imaging device surface.
FIGS. 18A to 18C show spot images on the light receiving surface of the imaging element 3 of the imaging lens device 1.

In addition, an imaging apparatus has been proposed in which a light beam is regularly dispersed by a phase front (wavefront coding optical element) and restored by digital processing to enable imaging with a deep depth of field (for example, non-patent literature). 1, 2, and patent documents 1 to 5).
"Wavefront Coding; jointly optimized optical and digital imaging systems", Edward R. Dowski, Jr., Robert H. Cormack, Scott D. Sarama. "Wavefront Coding; A modern method of achieving high performance and / or low cost imaging systems", Edward R. Dowski, Jr., Gregory E. Johnson. USP 6,021,005 USP 6,642,504 USP 6,525,302 USP 6,069,738 JP 2003-235794 A

In the imaging devices proposed in the above-mentioned documents, all of them are based on the assumption that the PSF (Point-Spread-Function) when the above-described phase plate is inserted into a normal optical system is constant, When the PSF changes, it is extremely difficult to realize an image with a deep depth of field by convolution using a subsequent kernel.
Therefore, apart from a single-focus lens, a zoom lens has a great problem in adopting it due to the high accuracy of the optical design and the associated cost increase.
In other words, in the conventional imaging apparatus, proper convolution calculation cannot be performed, and astigmatism and coma that cause a shift of a spot (SPOT) image at the time of wide or tele (Tele). Therefore, an optical design that eliminates various aberrations such as zoom chromatic aberration is required.
However, the optical design that eliminates these aberrations increases the difficulty of optical design, causing problems such as an increase in design man-hours, an increase in cost, and an increase in the size of the lens.
As a result, one phase plate and one image restoration are required for one optical system, and it is impossible to control a plurality of optical systems by one image restoration. However, there is a big problem in adopting a device that requires a plurality of optical systems due to an increase in cost amplifiers and systems.

  It is an object of the present invention to obtain a high-definition image quality, simplify an optical system, reduce costs, and perform lens design without worrying about the zoom position or zoom amount. Another object of the present invention is to provide an imaging apparatus, an imaging system, and an imaging method that can perform image restoration by high-accuracy calculation and that can perform image restoration of a plurality of optical systems with one apparatus.

  An image pickup system according to a first aspect of the present invention includes an image pickup device capable of picking up an object aberration image that has passed through an optical system and a light wavefront modulation device, and an image pickup device-side storage that stores coefficient specifying information for specifying a convolution coefficient. Means, and transmission means for transmitting the subject aberration image data obtained from the imaging element and the coefficient specifying information stored in the storage means, and the transmission means of the imaging apparatus. Receiving means for receiving the subject aberration image data and the coefficient specifying information, a processing apparatus side storing means for storing a plurality of convolution coefficients, and the processing apparatus side storing based on the coefficient specifying information received by the receiving means. A coefficient selection means for selecting a convolution coefficient from the means, and the subject aberration image data received by the receiving means Having a processing unit including conversion means for generating a no aberration object image data, the by converting the selected the one convolution coefficient by the means.

  An image pickup apparatus according to a second aspect of the present invention includes an image pickup element that can pick up an object aberration image that has passed through an optical system and a light wavefront modulation element, and an image pickup apparatus side storage that stores coefficient specifying information for specifying a convolution coefficient. And transmission means for transmitting subject aberration image data obtained from the image sensor and the coefficient specifying information stored in the storage means.

  According to a third aspect of the present invention, there is provided an imaging apparatus capable of selectively mounting a plurality of lenses, and capable of imaging a subject aberration image that has passed through at least one of the plurality of lenses and the light wavefront modulation element. An image sensor, convolution coefficient specifying information acquiring means for acquiring coefficient specifying information for specifying a convolution coefficient corresponding to the mounted lens, subject aberration image data obtained from the image sensor, and the convolution coefficient acquiring means; Transmitting means for transmitting the coefficient specifying information acquired in step (1).

  An image pickup apparatus according to a fourth aspect of the present invention is an image pickup device that can pick up an object aberration image that has passed through a zoom optical system and a light wavefront modulation device, and a zoom amount of the zoom optical system. Imaging apparatus side storage means for storing a plurality of coefficient specifying information capable of specifying convolution coefficients, zoom amount detecting means for detecting a zoom amount of a zoom optical system, and the zoom amount detecting means Coefficient specifying information acquisition means for acquiring one coefficient specifying information from the imaging device side storage means, subject aberration image data obtained from the image sensor, and acquisition of the coefficient specifying information based on the zoom amount detected by Transmitting means for transmitting the coefficient specifying information acquired by the means.

  An imaging apparatus according to a fifth aspect of the present invention is an imaging device capable of imaging a subject aberration image that has passed through an optical system and a light wavefront modulation device, and subject distance information acquisition means for acquiring information corresponding to the distance to the subject. And information corresponding to the distance to the subject acquired by the imaging device-side storage means for storing a plurality of coefficient specifying information capable of specifying a convolution coefficient according to the distance to the subject, and the subject distance information acquiring means. Based on the coefficient specifying information acquiring means for acquiring one coefficient specifying information from the imaging device side storage means, the subject aberration image data obtained from the imaging element, and the coefficient specifying information acquired by the coefficient specifying information acquiring means Transmitting means for transmitting.

  According to a sixth aspect of the present invention, there is provided an imaging method in which a subject aberration image that has passed through an optical system and a light wavefront modulation element is picked up by an imaging device, subject aberration image data, and coefficient specifying information for specifying a convolution coefficient. Transmitting the subject aberration image data and the coefficient specifying information, and selecting a convolution coefficient from a plurality of convolution coefficients based on the received coefficient specifying information. And a step of generating subject image data having no aberration by converting the received subject aberration image data with the one convolution coefficient selected by the coefficient selection unit.

  According to the present invention, high-definition image quality can be obtained, and restoration of a plurality of optical systems can be restored with a single device.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

  FIG. 1 is a configuration diagram of an imaging system according to the present embodiment.

The imaging system 10 includes imaging devices 100A, 100B, and 100C and a processing device 200 that are a plurality (three in the present embodiment) of optical systems.
Then, the imaging system 10 transmits the captured image data of each of the imaging apparatuses 100A, 100B, and 100C, the type of the optical system, the zoom amount and approximate subject distance information at the time of image capture, and coefficient specifying information that characterizes the convolution coefficient wirelessly or It is configured to be able to transmit to the processing device 200 by wire.

  As shown in FIG. 1, each of the imaging devices 100 </ b> A, 100 </ b> B, and 100 </ b> C basically has an imaging system capable of imaging, for example, an object aberration image that has passed through the optical system 110, the phase plate 120, and the optical system 110 and the phase plate 120. The element 130, the imaging device side storage unit 140 that stores coefficient specifying information for specifying the convolution coefficient, the subject aberration image data obtained from the imaging element 130, and the coefficient specifying information stored in the storage unit 140 are transmitted. The transmitter 150 is a main component.

  FIG. 2 is a block diagram showing a more specific configuration of the imaging apparatus according to the present embodiment. The imaging apparatus 100 in FIG. 2 is further provided with a zoom information detection apparatus 160 as a zoom amount detection unit in addition to the configuration of each imaging apparatus in FIG.

The zoom optical system 110 optically captures an image of the imaging target object (subject) OBJ.
The image pickup device 130 is a CCD or CMOS sensor that forms an image captured by the zoom optical system 110 including the phase plate 120 and outputs the primary image information of the image formation as the primary image signal FIM of the electrical signal to the transmission device 150. Become. In FIG. 2, the imaging element 130 is described as a CCD as an example.
The transmission device 150 functions as a coefficient identification information acquisition unit that acquires one coefficient identification information from the storage unit 140 based on the amount of zoom detected by the zoom information detection apparatus 160, and is obtained from the image sensor 130. The subject aberration image data and the acquired coefficient specifying information are transmitted to the processing device 200 wirelessly or by wire.

  FIG. 3 is a diagram schematically illustrating a configuration example of the optical system of the zoom optical system 110 according to the present embodiment.

The zoom optical system 110 in FIG. 3 is disposed between an object side lens 111 disposed on the object side OBJS, an image forming lens 112 for forming an image on the image sensor 130, and between the object side lens 111 and the image forming lens 112. An optical wavefront modulation element (wavefront forming optical element: Wavefront) made of, for example, a phase plate (Cubic Phase Plate) having a three-dimensional curved surface, which deforms the wavefront of the image formation on the light receiving surface of the imaging element 130 by the imaging lens 112. Coding Optical Element) group 113 is included. A stop (not shown) is disposed between the object side lens 111 and the imaging lens 112.
In the present embodiment, the case where the phase plate is used has been described. However, the optical wavefront modulation element of the present invention may be any element that deforms the wavefront, and an optical element whose thickness changes. (For example, the above-described third-order phase plate), an optical element whose refractive index changes (for example, a gradient index wavefront modulation lens), an optical element whose thickness and refractive index change by coding on the lens surface (for example, wavefront modulation) A light wavefront modulation element such as a hybrid lens) or a liquid crystal element capable of modulating the phase distribution of light (for example, a liquid crystal spatial phase modulation element) may be used.

The zoom optical system 110 in FIG. 3 is an example in which an optical phase plate 120a is inserted into a 3 × zoom used in a digital camera.
The phase plate 120 shown in the figure is an optical lens that regularly disperses the light beam converged by the optical system. By inserting this phase plate, an image that does not fit anywhere on the image sensor 130 is realized.
In other words, the phase plate 120 forms a deep light beam (which plays a central role in image formation) and a flare (blurred portion).
Means for restoring the regularly dispersed image to a focused image by digital processing is called a wavefront aberration controlling optical system (WFCO), and this processing is performed on the processing apparatus 200 side which is a transmission destination. To do.

  FIG. 4 is a diagram showing a spot image when the zoom optical system 110 not including the phase plate is wide. FIG. 5 is a diagram illustrating a spot image at the time of tele (tele) of the zoom optical system 110 that does not include a phase plate. FIG. 6 is a diagram showing a spot image on the infinite side of the zoom optical system 110 including the phase plate. FIG. 7 is a diagram showing a spot image on the near side of the zoom optical system 110 including the phase plate.

Basically, as shown in FIGS. 4 and 5, spot images of light passing through an optical lens system that does not include a phase plate show different spot images when the zoom optical system is wide and tele.
Naturally, as shown in FIGS. 6 and 7, the spot image that has passed through the phase plate affected by the spot image is also different on the infinite side and the near side.
In such an optical system having different spot images at the zoom position, the H function described later is different.

The conventional apparatus cannot perform an appropriate convolution calculation, and requires an optical design that eliminates astigmatism, coma aberration, zoom chromatic aberration, and the like that cause the deviation of the spot image. Optical design that eliminates these aberrations increases the difficulty of optical design, causing problems such as an increase in design man-hours, an increase in cost, and an increase in lens size.
Therefore, in the present embodiment, as shown in FIG. 2, when the imaging device (camera) 100 enters the shooting state, the zoom position or zoom amount is read from the zoom information detection device 160 and optically transmitted from the transmission device 150. Coefficient specifying information for specifying the type of system, the zoom amount at the time of image capture, and the convolution coefficient is transmitted to the processing device 200 wirelessly or by wire.

  The processing device 200 acquires one coefficient from a plurality of convolution coefficients stored in advance based on the specific information in accordance with the transmitted zoom position or zoom amount, and distributes the variance from the dispersed image signal from the image sensor 130. No subject image data is generated.

  In the present embodiment, as described above, dispersion refers to the formation of an image that does not fit anywhere on the image sensor 130 by inserting the phase plate 120a. It plays a central role in image formation) and a phenomenon of forming flare (blurred portion), and includes the same meaning as aberration because of the behavior of the image being dispersed to form a blurred portion. Therefore, in this embodiment, it may be described as aberration.

  As illustrated in FIG. 1, the processing device 200 includes a reception unit 210, a reception information control unit 220, a convolution device 230, a kernel / numerical operation coefficient storage register 240, and an image processing operation processor 250.

  FIG. 8 is a chart showing a processing outline of the processing apparatus 200 on the receiving side.

In the processing device 200, the image processing arithmetic processor 250 sequentially drives the imaging devices 100A to 100C as the plurality of optical systems. At that time, the reception unit 210 controls the reception timing of transmission information from each of the imaging devices 100A to 100C so that the image transfer captured by each of the imaging devices 100A to 100C does not overlap in the reception unit 210.
Next, the single optical system instructed to capture an image, for example, the imaging apparatus 100A captures the image information and transmits the image information to the transmitting apparatus 150.
The transmitter 150 to which the image information has been transmitted, while synchronizing with the receiving unit 210, includes image information, the type of optical system (AorBorC), zoom information at the time of image capture, approximate subject distance information at the time of image capture, etc. Then, information (coefficient specifying information) that is a factor for determining the optimum value in the numerical operation storage register 240 is added, and the data is transmitted to the receiving unit 210 wirelessly or by wire.
Receiving unit 210 that has received the information transfers information that is a factor in determining the optimum value to received information control unit 220 (ST1 to ST3).
The reception information control unit 220 sets the optimum value in the convolution device 30 from the kernel and numerical operation storage register 240 (ST4).
The convolution device 230 performs optimum restoration of the image using the captured image information, the kernel which is the optimum value set, and the numerical operation value (ST5).

Here, the basic principle of WFCO will be described.
As shown in FIG. 9, the image f of the subject enters the WFCO optical system H, thereby generating a g image.
This can be expressed by the following equation.

(Equation 1)
g = H * f
Here, * represents convolution.

  In order to obtain a subject from the generated image, the following processing is required.

(Equation 2)
f = H ^-1 * g

Here, the kernel size and calculation coefficient regarding the function H will be described.
Let each zoom position (zoom position) be Zpn, Zpn-1,.
The H function is defined as Hn, Hn-1,.
Since each spot is different, each H function is as follows.

  The difference in the number of rows and / or the number of columns in this matrix is the kernel size, and each number is the operation coefficient.

As described above, when a phase plate as a light wavefront modulation element is applied to an imaging apparatus provided in a zoom optical system, the spot image generated differs depending on the zoom position of the zoom optical system. For this reason, when a convolution calculation is performed on a defocus image (spot image) obtained from the phase plate by a DSP or the like at a later stage, different convolution calculations are required depending on the zoom position in order to obtain an appropriate focused image. Become.
Therefore, in the present embodiment, the zoom information detection device 160 is provided, and is configured to perform an appropriate convolution calculation according to the zoom position and obtain an appropriate focused image regardless of the zoom position.

As described above, a proper convolution calculation in the processing apparatus 200 may be configured to store the convolution calculation coefficient in the register 240.
In addition to this configuration, the following configuration can be employed.

  A configuration in which a correction coefficient is stored in advance in the register 240 in accordance with each zoom position, a calculation coefficient is corrected using the correction coefficient, and an appropriate convolution calculation is performed using the corrected calculation coefficient. Then, the kernel 240 and the convolution calculation coefficient itself are stored in the register 240 in advance, the convolution calculation is performed using the stored kernel size and calculation coefficient, and the calculation coefficient corresponding to the zoom position is stored in the register 302 as a function in advance. It is possible to adopt a configuration in which the calculation coefficient is obtained from this function based on the zoom position, and the convolution calculation is performed using the calculated calculation coefficient.

In the present embodiment, it is possible to adopt WFCO and obtain high-definition image quality, simplify the optical system, and reduce costs.
Hereinafter, this feature will be described.

10A to 10C show spot images on the light receiving surface of the imaging element 130 of the imaging apparatus 100. FIG.
10A shows a case where the focal point is shifted by 0.2 mm (Defocus = 0.2 mm), FIG. 10B shows a case where the focal point is a focal point (Best focus), and FIG. 9C shows a case where the focal point is shifted by −0.2 mm. In this case, each spot image is shown (Defocus = −0.2 mm).
As can be seen from FIGS. 10A to 10C, in the imaging lens device 200 according to the present embodiment, the wavefront forming optical element group 113 including the phase plate 120 causes a deep light beam (a central part of image formation). Play a role) and flare (blurred part) is formed.

  As described above, the primary image FIM formed in the imaging apparatus 100 of the present embodiment has a light beam condition with a very deep depth.

FIGS. 11A and 11B are diagrams for explaining a modulation transfer function (MTF) of a primary image formed by the imaging lens device according to the present embodiment. FIG. 11A is a diagram showing a spot image on the light receiving surface of the imaging element of the imaging lens device, and FIG. 11B shows the MTF characteristics with respect to the spatial frequency.
In the present embodiment, the high-definition final image is left to the correction processing of the image processing apparatus 300 including a digital signal processor (Digital Signal Processor), for example, as shown in FIGS. 11A and 11B. The MTF of the primary image is essentially a low value.

  The processing device 200 is configured by, for example, a DSP, and receives a primary image FIM from the imaging device 100 as described above and performs a predetermined correction process or the like that raises the MTF at the spatial frequency of the primary image so as to achieve high definition. A final image FNLIM is formed.

The MTF correction processing of the processing apparatus 200 is, for example, post-processing such as edge enhancement and chroma enhancement using the spatial frequency as a parameter for the MTF of the primary image which is essentially a low value, as shown by a curve A in FIG. Thus, the correction is performed so as to approach (reach) the characteristic indicated by the curve B in FIG.
The characteristic indicated by the curve B in FIG. 12 is a characteristic obtained when the wavefront is not deformed without using the wavefront forming optical element as in the present embodiment, for example.
It should be noted that all corrections in the present embodiment are based on spatial frequency parameters.

In this embodiment, as shown in FIG. 12, in order to achieve the MTF characteristic curve B to be finally realized with respect to the MTF characteristic curve A with respect to the optically obtained spatial frequency, each spatial frequency is changed to each spatial frequency. On the other hand, the original image (primary image) is corrected by applying strength such as edge enhancement.
For example, in the case of the MTF characteristic of FIG. 12, the curve of edge enhancement with respect to the spatial frequency is as shown in FIG.

  That is, a desired MTF characteristic curve B is virtually realized by performing correction by weakening edge enhancement on the low frequency side and high frequency side within a predetermined spatial frequency band and strengthening edge enhancement in the intermediate frequency region. To do.

As described above, the imaging system 10 according to the embodiment includes the plurality of imaging devices 100 including the optical system 110 that forms the primary image, and the processing device 200 that forms the primary image into a high-definition final image. In the system system, the wavefront of the imaging is deformed by providing a new optical element for wavefront shaping, or by providing an optical element such as glass or plastic that has been molded for wavefront shaping. In this image forming system, such a wavefront is imaged on an imaging surface (light-receiving surface) of an imaging device 130 made of a CCD or a CMOS sensor, and the imaged primary image is obtained through a processing device 200.
In the present embodiment, the primary image obtained by the imaging apparatus 100 has a light flux condition with a very deep depth. For this reason, the MTF of the primary image is essentially a low value, and the processing device 200 corrects the MTF.

Here, the imaging process in the imaging apparatus 100 according to the present embodiment will be considered in terms of wave optics.
A spherical wave diverging from one of the object points becomes a convergent wave after passing through the imaging optical system. At that time, aberration occurs if the imaging optical system is not an ideal optical system. The wavefront is not a spherical surface but a complicated shape. Wavefront optics lies between geometric optics and wave optics, which is convenient when dealing with wavefront phenomena.
When dealing with the wave optical MTF on the imaging plane, the wavefront information at the exit pupil position of the imaging optical system is important.
The MTF is calculated by Fourier transform of the wave optical intensity distribution at the imaging point. The wave optical intensity distribution is obtained by squaring the wave optical amplitude distribution, and the wave optical amplitude distribution is obtained from the Fourier transform of the pupil function in the exit pupil.
Further, since the pupil function is exactly from the wavefront information (wavefront aberration) at the exit pupil position itself, if the wavefront aberration can be precisely calculated through the optical system 110, the MTF can be calculated.

Therefore, if the wavefront information at the exit pupil position is subjected to various processes by a predetermined method, the MTF value on the imaging plane can be arbitrarily changed.
In this embodiment, the wavefront shape is mainly changed by the wavefront forming optical element, but the target wavefront is formed by increasing or decreasing the phase (phase, optical path length along the light beam). .
Then, if the desired wavefront is formed, the exiting light flux from the exit pupil is made up of dense and sparse portions of the light, as can be seen from the geometric optical spot images shown in FIGS. It is formed.
The MTF in the luminous flux state shows a low value at a low spatial frequency and a characteristic that the resolving power is managed up to a high spatial frequency.
That is, if this MTF value is low (or such a spot image state in terms of geometrical optics), the phenomenon of aliasing will not occur.
That is, a low-pass filter is not necessary.
Then, the flare-like image that causes the MTF value to be lowered may be removed by the image processing apparatus 200 including a DSP or the like at the subsequent stage. Thereby, the MTF value is significantly improved.

As described above, according to the present embodiment, the optical system 110, the phase plate 120 as a light wavefront modulation element, the imaging element 130 capable of capturing an object aberration image that has passed through the optical system 110 and the phase plate 120, An imaging device-side storage unit 140 that stores coefficient specifying information for specifying a convolution coefficient, and a transmission device 150 that transmits subject aberration image data obtained from the imaging element 130 and coefficient specifying information stored in the storage unit 140; , As a main component, and one coefficient from a plurality of convolution coefficients stored in advance based on coefficient specifying information corresponding to the zoom position or zoom amount transmitted from the image pickup apparatus 100 To generate subject image data having no dispersion from the dispersion image signal from the image sensor 130. Since it has bets, it is possible to perform image reconstruction of a plurality of optical systems in a single device.
Here, the imaging devices 100A to 100C select appropriate coefficient characteristic information from the storage unit 140 in accordance with the respective zoom positions or zoom amounts, and perform transmission.
In addition, it is possible to design a lens without worrying about the zoom position, and it is possible to restore an image by accurate convolution. Therefore, in any zoom lens, there is an advantage that it is possible to provide a focused image without driving a lens that is difficult, expensive, and does not require an enlarged optical lens. .
The imaging apparatus 100 according to the present embodiment can be used for a WFCO of a zoom lens in consideration of the small size, light weight, and cost of consumer equipment such as a digital camera and a camcorder.
Here, the imaging devices 100A to 100C select coefficient specifying information corresponding to the phase plate from the storage unit 140 and transmit the coefficient specifying information to the processing device 200, and the zoom information corresponding to the zoom position and zoom amount of each imaging device is the same. You may make it transmit to the processing apparatus 200 using the transmitter 150. FIG. In this case, the processing apparatus 200 acquires one convolution coefficient from the received coefficient specifying information and zoom information, and performs image restoration.

In the present embodiment, the imaging device 100 having a wavefront forming optical element that deforms the wavefront of the imaging on the light receiving surface of the imaging device 130 by the imaging lens 112 and the primary image FIM by the imaging device 100 are received. In addition, since it has the processing device 200 that forms a high-definition final image FNLIM by performing a predetermined correction process or the like for raising the MTF at the spatial frequency of the primary image, it becomes possible to obtain a high-definition image quality. There is an advantage.
In addition, the configuration of the optical system 110 of the imaging apparatus 100 can be simplified, manufacturing becomes easy, and cost can be reduced.

By the way, when a CCD or CMOS sensor is used as an image sensor, there is a resolution limit determined by the pixel pitch, and if the resolution of the optical system exceeds the limit resolution, a phenomenon such as aliasing occurs, which adversely affects the final image. It is a well-known fact that
In order to improve image quality, it is desirable to increase the contrast as much as possible, but this requires a high-performance lens system.

However, as described above, aliasing occurs when a CCD or CMOS sensor is used as an image sensor.
Currently, in order to avoid the occurrence of aliasing, the imaging lens apparatus uses a low-pass filter made of a uniaxial crystal system to avoid the occurrence of aliasing.
The use of a low-pass filter in this way is correct in principle, but the low-pass filter itself is made of crystal, so it is expensive and difficult to manage. Moreover, there is a disadvantage that the use of the optical system makes the optical system more complicated.

As described above, in order to form a high-definition image, the optical system must be complicated in the conventional imaging lens apparatus in spite of the demand for higher-definition image due to the trend of the times. . If it is complicated, manufacturing becomes difficult, and if an expensive low-pass filter is used, the cost increases.
However, according to the present embodiment, the occurrence of aliasing can be avoided without using a low-pass filter, and high-definition image quality can be obtained.

  In the present embodiment, the example in which the wavefront forming optical element of the optical system 210 is disposed closer to the object side lens than the stop has been described. The effect of this can be obtained.

  Further, the lens constituting the optical system 110 is not limited to the example of FIG. 3, and various aspects of the present invention are possible.

  In the above embodiment, the case where the image pickup apparatus 100 is provided with the zoom amount detection device as an example in FIG. 2 has been described as an example, but as shown in FIG. 15, for example, the object approximate distance information detection device 171 and the operation switch 172 It is also possible to provide a shooting mode setting unit 170 including the above and select specific information based on distance information.

In this case, there are a plurality of shooting modes, for example, a normal shooting mode (portrait), a macro shooting mode (close-up), and a distant view shooting mode (infinity). The operation switch 172 can be selected and input.
For example, as shown in FIG. 15, the operation switch 172 includes changeover switches 301 a, 301 b, and 301 c provided on the lower side of the liquid crystal screen 303 on the back side of the camera (imaging device).
The changeover switch 301a is a switch for selecting and inputting a distant view shooting mode (infinity), the changeover switch 301b is a switch for selecting and inputting a normal shooting mode (portrait), and the changeover switch 301c is a macro shooting. This is a switch for selecting and inputting the mode (nearest).
Note that the mode switching method may be a touch panel type as well as a switch method as shown in FIG. 15, or a mode for switching the object distance may be selected from the menu screen.

The object approximate distance information detection device 171 as subject distance information generation means generates information corresponding to the distance to the subject based on the input information of the operation switch 172 and supplies the information to the transmission device 140.
Based on the transmission information of the transmission device 140, the processing device 200 converts the dispersed image signal from the image sensor 130 of the imaging device 100 into an image signal having no dispersion. At this time, the detection result of the object approximate distance information detection device 171 In response to the coefficient specifying information selected according to the above, different conversion processing is performed according to the set photographing mode.
For example, the processing device 200 includes a normal conversion process in the normal shooting mode, a macro conversion process corresponding to a macro shooting mode in which aberration is reduced on the near side compared to the normal conversion process, and a far side compared to the normal conversion process. A distant view conversion process corresponding to a distant view photographing mode for reducing aberration is selectively executed according to the photographing mode.
Here, the imaging devices 100A to 100C transmit coefficient specifying information corresponding to the phase plate, and also transmit information corresponding to the distance to the subject, and the processing device 200 uses one of these two types of information as one convolution. The image coefficient may be acquired to restore the image.

Also in this case, the same effect as described above can be obtained.
In the above-described embodiment, the zoom amount detection device and the object approximate distance information detection device are described as examples. However, the embodiment may not include these. In this case, in FIG. 1, based on only the coefficient specifying information transmitted from the imaging apparatus 100, one coefficient is acquired from the convolution coefficient stored in advance in the processing apparatus 200, and subject image data without dispersion is obtained. Is generated. Also in this case, the same effect as described above can be obtained.

In this embodiment, a plurality of optical systems are provided, and the captured image information is transmitted to the processing apparatus 200 side together with the coefficient specifying information. For example, as shown in FIG. 16, the optical system 110-1 110-2, a desired optical system may be selected in order, and a subject image that has passed through each optical system may be input to a single image sensor 130.
Also in this case, the same effect as described above can be obtained.

It is a block diagram of the imaging system which concerns on this embodiment. It is a block which shows the specific structural example of the imaging device which concerns on this embodiment. It is a figure which shows typically the structural example of the zoom optical system of the imaging lens apparatus which concerns on this embodiment. It is a figure which shows the spot image at the time of the wide (Wide) of the zoom optical system which does not include a phase plate. It is a figure which shows the spot image at the time of tele (Tele) of the zoom optical system which does not include a phase plate. It is a figure which shows the spot image of the infinite side of the zoom optical system containing a phase plate. It is a figure which shows the spot image of the near side of the zoom optical system containing a phase plate. It is a flowchart which shows the process outline | summary of the processing apparatus of this embodiment. It is a figure for demonstrating the principle of WFCO. It is a figure which shows the spot image in the light-receiving surface of the image pick-up element of the image pick-up lens apparatus which concerns on this embodiment, Comprising: (A) is a case where a focus shifts by 0.2 mm (Defocus = 0.2 mm), (B) is a result. In the case of a focus (Best focus), (C) is a figure which shows each spot image when a focus shifts by -0.2 mm (Defocus = -0.2 mm). It is a figure for demonstrating MTF of the primary image formed with the imaging lens apparatus which concerns on this embodiment, Comprising: (A) is a figure which shows the spot image in the light-receiving surface of the image pick-up element of an imaging lens apparatus, ( B) shows the MTF characteristic with respect to the spatial frequency. It is a figure for demonstrating the MTF correction process in the processing apparatus which concerns on this embodiment. It is a figure for demonstrating concretely the MTF correction process in the processing apparatus which concerns on this embodiment. It is a block diagram which shows the other example of the imaging device which concerns on this embodiment. It is a figure which shows the structural example of the operation switch of FIG. It is a block diagram which shows the other example of an imaging system. It is a figure which shows typically the structure and light beam state of a general imaging lens apparatus. FIG. 18A is a diagram showing a spot image on the light receiving surface of the imaging element of the imaging lens device of FIG. 17, where FIG. 17A shows a case where the focal point is shifted by 0.2 mm (Defocus = 0.2 mm), and FIG. In the case (Best focus), (C) is a diagram showing each spot image when the focal point is shifted by -0.2 mm (Defocus = -0.2 mm).

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 ... Imaging device, 110 ... Zoom optical system, 111 ... Object side lens, 112 ... Imaging lens, 113 ... Optical element for wavefront formation, 120 ... Phase plate (light wavefront modulation element), 130 ... Imaging element, 140 ... Memory , 150 ... transmitting device, 160 ... zoom information detecting device, 200 ... processing device, 210 ... receiving unit, 220 ... received information control unit, 230 ... convolution device, 240 ... kernel, numerical calculation coefficient storage register, 250 ... image Processing arithmetic processor.

Claims (6)

An imaging device capable of imaging a subject aberration image that has passed through an optical system and a light wavefront modulation device;
Imaging device side storage means for storing coefficient specifying information for specifying a convolution coefficient;
Transmitting means for transmitting subject aberration image data obtained from the image sensor and the coefficient specifying information stored in the storage means;
An imaging device including:
Receiving means for receiving the subject aberration image data and the coefficient specifying information transmitted by the transmitting means of the imaging device;
A processor-side storage means for storing a plurality of convolution coefficients;
Coefficient selecting means for selecting a convolution coefficient from the processing device side storage means based on the coefficient specifying information received by the receiving means;
Conversion means for generating subject image data without aberration by converting the subject aberration image data received by the receiving means with the one convolution coefficient selected by the coefficient selection means;
A processing device comprising:
An imaging system. An imaging device capable of imaging a subject aberration image that has passed through an optical system and a light wavefront modulation device;
Imaging device side storage means for storing coefficient specifying information for specifying a convolution coefficient;
Transmitting means for transmitting subject aberration image data obtained from the image sensor and the coefficient specifying information stored in the storage means;
An imaging apparatus including: An imaging device capable of selectively mounting a plurality of lenses,
An imaging device capable of imaging an object aberration image that has passed through at least one of the plurality of lenses and the light wavefront modulation device;
A convolution coefficient specifying information acquiring means for acquiring coefficient specifying information for specifying a convolution coefficient corresponding to the mounted lens;
Transmitting means for transmitting subject aberration image data obtained from the image sensor and the coefficient specifying information acquired by the convolution coefficient acquiring means;
An imaging apparatus including: An image pickup device capable of picking up a subject aberration image that has passed through a zoom optical system and a light wavefront modulation device;
An imaging device-side storage unit that stores a plurality of coefficient specifying information capable of specifying a convolution coefficient according to a zoom amount of the zoom optical system;
A zoom amount detecting means for detecting a zoom amount of the zoom optical system;
Coefficient specifying information acquisition means for acquiring one coefficient specifying information from the imaging device side storage means based on the zoom amount detected by the zoom amount detection means;
Transmitting means for transmitting subject aberration image data obtained from the image sensor and the coefficient specifying information acquired by the coefficient specifying information acquiring means;
An imaging apparatus including: An imaging device capable of imaging a subject aberration image that has passed through an optical system and a light wavefront modulation device;
Subject distance information acquisition means for acquiring information corresponding to the distance to the subject;
Imaging device side storage means for storing a plurality of coefficient specifying information capable of specifying a convolution coefficient according to the distance to the subject;
Coefficient specifying information acquisition means for acquiring one coefficient specifying information from the imaging device side storage means based on information corresponding to the distance to the subject acquired by the subject distance information acquisition means;
Transmitting means for transmitting subject aberration image data obtained from the image sensor and the coefficient specifying information acquired by the coefficient specifying information acquiring means;
An imaging apparatus including: Capturing an object aberration image that has passed through the optical system and the light wavefront modulation element with an imaging element;
Transmitting subject aberration image data and coefficient specifying information for specifying a convolution coefficient;
Receiving the transmitted subject aberration image data and the coefficient specifying information;
Coefficient selecting means for selecting a convolution coefficient from a plurality of convolution coefficients based on the received coefficient specifying information;
Transforming the received subject aberration image data with the one convolution coefficient selected by the coefficient selection means to generate subject image data without aberration;
An imaging method including:

Images