JP2007060647A - Imaging apparatus and imaging method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus and its method which can assure image quality after processing, without changing loading of an operation process, and moreover, which can also acquire certain image of blurs. <P>SOLUTION: The imaging apparatus has an imaging lens device 200 which images an object variance image, which has passed an optical system and a phase plate (light wave surface modulation element), an image processing apparatus 500 which forms an image signal which does not have variance from a distributed image signal from an imaging element 200, a photographic information formation portion 300 which forms a focal length, a result of multipoint ranging, information, corresponding to a distance to an object, zoom information, or photographing mode information and supplies to an image processing apparatus 500, and a selecting switching portion 400 which inserts or evacuates alternatively the desired phase plate of a plurality of the phase plates on an optical path. The image processing apparatus 500 forms the image signal, which does not have dispersion from the image signal, based on information formed by the photographic information formation portion 300. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an imaging apparatus such as a digital still camera, a camera mounted on a mobile phone, and a camera mounted on a portable information terminal, and a method thereof, using an imaging element and including an optical wavefront modulation element such as an optical system and a phase plate.

In response to the digitization of information, which has been rapidly developing in recent years, the response in the video field is also remarkable.
In particular, as symbolized by a digital camera, a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) sensor, which is a solid-state image sensor, is used in most cases instead of a conventional film.

  As described above, an imaging lens device using a CCD or CMOS sensor as an imaging element is for taking an image of a subject optically by an optical system and extracting it as an electrical signal by the imaging element. In addition to a digital still camera, It is used for a video camera, a digital video unit, a personal computer, a mobile phone, a personal digital assistant (PDA) and the like.

FIG. 24 is a diagram schematically illustrating a configuration and a light flux state of a general imaging lens device.
The imaging lens device 1 includes an optical system 2 and an imaging element 3 such as a CCD or CMOS sensor.
In the optical system, the object side lenses 21 and 22, the diaphragm 23, and the imaging lens 24 are sequentially arranged from the object side (OBJS) toward the image sensor 3 side.

In the imaging lens device 1, as shown in FIG. 24, the best focus surface is matched with the imaging element surface.
25A to 25C show spot images on the light receiving surface of the imaging element 3 of the imaging lens device 1.

In addition, an imaging apparatus has been proposed in which a light beam is regularly dispersed by a phase front (wavefront coding optical element) and restored by digital processing to enable imaging with a deep depth of field (for example, non-patent literature). 1, 2, and patent documents 1 to 5).
In addition, a depth-of-field expansion system has been proposed in which the phase modulation amount of the liquid crystal spatial phase modulator is variable according to the amount of expansion of the field.
"Wavefront Coding; jointly optimized optical and digital imaging systems", Edward R. Dowski, Jr., Robert H. Cormack, Scott D. Sarama. "Wavefront Coding; A modern method of achieving high performance and / or low cost imaging systems", Edward R. Dowski, Jr., Gregory E. Johnson. USP 6,021,005 USP 6,642,504 USP 6,525,302 USP 6,069,738 JP 2003-235794 A

In the imaging devices proposed in the above-mentioned documents and the like, all of them are based on the assumption that the PSF (Point-Spread-Function) when the above-described phase plate is inserted into the normal optical system is constant. When the PSF changes, it is extremely difficult to realize an image having a deep depth of field by convolution using a kernel thereafter.
Therefore, even with a single focal point lens, a regular (non-changing) PSF cannot be realized with a normal optical system in which the spot image changes depending on the object distance. There is a big problem in adopting due to the high accuracy of design and the accompanying cost increase.

In addition, in an optical system including a plurality of optical systems, for example, an optical system equipped with an optical zoom mechanism such as a digital camera, a technique such as expanding the depth of field by the above-described phase plate and subsequent signal processing, etc. When applied, the optical design must be designed so that the PSF has the same size and shape in each optical system, but this increases the design difficulty and increases the accuracy of the lens and the associated costs. It will cause a big problem to adopt because of up.
Further, although it is possible to change the focusing range with one phase modulation element to some extent by changing the convolution process, it is not preferable in view of the load on the convolution process and the image quality after the process.

  Furthermore, in the imaging apparatus proposed in each of the above-mentioned documents, only one depth of field can be obtained with one phase plate (light wavefront modulation element) and one restoration process. This is fine if you want an image that is in focus over the entire shooting range, but you cannot control the depth of field as in a typical optical system to get a blur.

  In general cameras, when performing macro photography, the lens is switched to the macro area and then focused by autofocus (hereinafter referred to as AF). Alternatively, it is also known that the depth of field is expanded by reducing the F value to achieve a fixed focus.

By the way, the problem inherent to macro photography is that it is difficult to focus on the target part due to slight unevenness of the subject due to the very short distance, or the camera's front and rear displacement due to handheld photography. The depth is very shallow, making it difficult to focus on the entire screen.
This can be achieved by combining AF for macro photography and fixing means such as a tripod to achieve optimum focusing and focusing on a target subject, but cannot focus on the entire screen. There is also a problem that fixing means such as a tripod is troublesome.

  As a means for solving this problem, a method of widening the depth of field by reducing the F value of the lens is widely used, but it becomes darker by reducing the F value, and the shutter speed is slow to maintain the exposure. Therefore, problems such as camera shake also occur.

  On the other hand, in contrast to the theory of traditional optical systems, the technique of blurring an image by actively inserting an optical phase plate and restoring it by image processing using a blur restoration digital filter reduces the depth of field without reducing the F value. Although there is a merit that can be expanded, there is no perspective in normal landscape painting and portrait photography, and it is difficult to apply to photography that pursues hobbies and artistry due to slight noise unique to image processing was there.

The first object of the present invention is to simplify the optical system, reduce costs, perform lens design without worrying about the object distance and defocus range, and perform highly accurate calculations. It is an object of the present invention to provide an imaging apparatus and method capable of restoring an image by the above-mentioned method, ensuring the image quality after processing without changing the processing load, and obtaining a blurred image.
A second object of the present invention is to provide an imaging apparatus and method capable of easily performing macro photography as a record without failure while enabling artistic photography.

  In order to achieve the above object, an imaging apparatus according to a first aspect of the present invention includes an optical system and at least one light wavefront modulation element, and an object dispersion image that has passed through the optical system or the optical system and the light wavefront modulation element. An image pickup device for picking up an image, and an image processing means including a conversion means for generating a non-dispersion image signal from the subject dispersion image signal from the image pickup device. The light wavefront modulation element can be inserted into and retracted from the optical path. It is.

  Preferably, it has a plurality of light wavefront modulation elements, and selection switching means for selectively inserting and retracting each of the plurality of light wavefront modulation elements on the optical path.

  Preferably, the selection switching unit selectively switches between insertion and withdrawal of a desired light wavefront modulation element on the optical path in accordance with a focal length.

  Preferably, the selection switching means selectively switches between insertion and withdrawal of a desired light wavefront modulation element on the optical path according to the result of multipoint distance measurement.

  Preferably, the optical system includes a zoom function, and the selection switching unit selectively switches insertion and withdrawal of a desired light wavefront modulation element on the optical path in accordance with zoom information.

  Preferably, the selection switching means selectively switches between insertion and withdrawal of a desired light wavefront modulation element on the optical path in accordance with information corresponding to the distance to the subject.

  Preferably, the selection switching unit selectively switches between insertion and withdrawal of a desired light wavefront modulation element on the optical path in accordance with a photographing mode.

  Preferably, the imaging apparatus inserts the light wavefront modulation element into the optical path in the macro mode / macro state, and retracts the light wavefront modulation element from the optical path in the non-macro mode / non-macro state.

  Preferably, the imaging apparatus includes a lens barrel capable of moving the optical system to a macro area that enables close-up photography, and when the distance measurement result is determined to be a short distance, the optical system is taken. Move the lens to a fixed focus macro lens configuration.

  Preferably, the imaging apparatus includes a lens barrel capable of moving the optical system to a macro area that enables close-up photography, and when the macro mode is selected, the taking lens of the optical system is fixed-focused. Move to the macro lens configuration.

  Preferably, the imaging apparatus has a mechanism that mechanically moves the lens to a macro position.

  Preferably, means for inserting the optical wavefront modulation element into the optical path of the optical system when in the macro state and retracting the optical wavefront modulation element from the optical path of the optical system when the macro state is released The image processing means, when the light wavefront modulation element is inserted in a macro state, the conversion means generates an image signal having no dispersion from the subject dispersion image signal, and is released from the macro state, When the light wavefront modulation element is retracted, the conversion means is not processed.

  Preferably, there is a control means for releasing the macro state when the user turns off the power.

  Preferably, the imaging apparatus includes a selection unit that selects whether or not to insert an optical wavefront modulation element into the optical path in the macro mode.

  Preferably, the macro lens has a function of selecting a mode for inserting the light wavefront modulation element and a mode for retracting the light wavefront modulation element, and in the macro photography in which the light wavefront modulation element is inserted, the focusing lens is at a predetermined position. The focusing lens performs a distance measuring operation in macro photography in which the light wavefront modulation element is retracted without performing a distance measuring operation, and has a control unit that performs an auto focus (AF) operation by driving the focusing lens.

  An imaging method according to a second aspect of the present invention includes a first step of generating information relating to imaging, and a predetermined light wavefront modulation element being inserted into or retracted from the optical path based on the information generated by the information generation step. An object dispersion image that has passed through the second step, the optical system, and the position where the light wavefront modulation element can be arranged in the state where the light wavefront modulation element is selectively set in the second step is captured by the image sensor. And a fourth step of converting the dispersed image signal based on the information generated by the second step to generate an image signal without dispersion.

According to the present invention, there is an advantage that lens design can be performed without worrying about an object distance and a defocus range, and that an image can be restored by calculation such as accurate convolution.
Further, according to the present invention, it is possible to easily perform macro shooting as a record without failure while allowing artistic shooting.
Further, according to the present invention, the optical system can be simplified and the cost can be reduced.
In addition, the image quality after processing can be ensured without changing the processing load.
In addition, it is possible to obtain a blurred image like a conventional optical system while using a depth-expanding optical system for the purpose of obtaining a focused image by an optical wavefront modulation element and image processing.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

<First Embodiment>
FIG. 1 is a block diagram showing an imaging apparatus according to the first embodiment of the present invention.

  The imaging apparatus 100 according to the first embodiment includes an imaging lens apparatus 200 having an optical system having a zoom function (hereinafter referred to as a zoom optical system), a shooting information generation unit 300 that generates information related to shooting, and a light wavefront. A modulation element selection switching unit 400 and an image processing device (signal processing unit) 500 are included as main components.

  The imaging lens device 200 optically captures an image of an imaging target object (subject) OBJ, and an image captured by the zoom optical system 210 is formed. The image sensor 220 includes a CCD or a CMOS sensor that is output to the image processing apparatus 300 as the next image signal FIM. In FIG. 1, the imaging element 220 is described as a CCD as an example.

  FIG. 2 is a diagram schematically illustrating a configuration example of the optical system of the zoom optical system 210 according to the present embodiment.

The zoom optical system 210 in FIG. 2 is disposed between an object side lens 211 disposed on the object side OBJS, an image forming lens 212 for forming an image on the image sensor 220, and between the object side lens 211 and the image forming lens 212. An optical wavefront modulation element (wavefront forming optical element) composed of a plurality of phase plates (Cubic Phase Plate) having, for example, a three-dimensional curved surface, which deforms the wavefront of image formation on the light receiving surface of the image sensor 220 by the imaging lens 212 : Wavefront Coding Optical Element) group 213. A stop (not shown) is disposed between the object side lens 211 and the imaging lens 212.
In the present embodiment, the case where the phase plate is used has been described. However, the optical wavefront modulation element of the present invention may be any element that deforms the wavefront, and an optical element whose thickness changes. (For example, the above-described third-order phase plate), an optical element whose refractive index changes (for example, a gradient index wavefront modulation lens), an optical element whose thickness and refractive index change by coding on the lens surface (for example, wavefront modulation) A light wavefront modulation element such as a hybrid lens or a liquid crystal element (for example, a liquid crystal spatial phase modulation element) capable of modulating the phase distribution of light may be used.

The zoom optical system 210 in FIG. 2 is an example in which an optical phase plate 213a is inserted into a 3 × zoom used in a digital camera.
The phase plate 213a shown in the figure is an optical lens that regularly splits the light beam converged by the optical system. By inserting this phase plate, an image that does not fit anywhere on the image sensor 220 is realized.
In other words, the phase plate 213a forms a deep luminous flux (which plays a central role in image formation) and a flare (blurred portion).
Means for restoring the regularly dispersed image to a focused image by digital processing is called a wavefront aberration control optical system, and this processing is performed in the image processing apparatus 300.

  FIG. 3 is a diagram showing a spot image on the infinite side of the zoom optical system 210 that does not include a phase plate. FIG. 4 is a diagram showing a spot image on the near side of the zoom optical system 210 that does not include a phase plate. FIG. 5 is a diagram showing a spot image on the infinite side of the zoom optical system 210 including the phase plate. FIG. 6 is a diagram showing a spot image on the near side of the zoom optical system 210 including the phase plate.

Basically, the spot image of the light passing through the optical lens system not including the phase plate is different depending on whether the object distance is on the close side or on the infinite side, as shown in FIGS. A spot image is shown.
As described above, in an optical system having spot images different in object distance, the H function described later is different.
Naturally, as shown in FIGS. 5 and 6, the spot image passing through the phase plate influenced by the spot image also becomes a different spot image on the near side and the infinite side.

In such an optical system having different spot images at the object position, the conventional apparatus cannot perform an appropriate convolution operation, and astigmatism, coma aberration, spherical aberration, etc. that cause this spot image shift. An optical design that eliminates each aberration is required. However, the optical design that eliminates these aberrations increases the difficulty of optical design, causing problems such as an increase in design man-hours, an increase in cost, and an increase in the size of the lens.
Therefore, in the present embodiment, as illustrated in FIG. 1, when the imaging apparatus (camera) 100 enters a shooting state, information related to shooting is generated in the shooting information generation unit 300 and supplied to the image processing apparatus 500. .

Examples of the information related to shooting generated by the shooting information generation unit 300 include lens focal length, multi-point distance measurement, zoom information of the zoom optical system 210, approximate distance information of the object distance of the subject, or shooting mode information. be able to.
Information relating to shooting generated by the shooting information generation unit 300 is used for switching the filter of the convolution calculation process of the image processing apparatus 500.

  The selection switching unit 400 is formed integrally with, for example, a plurality of phase plates as light wavefront modulation elements, and each of the plurality of phase plates is selected according to the above-described information relating to imaging generated by the imaging information generation unit 300. The optical system 210 is selectively inserted into or retracted from the optical path.

  Note that, here, an example is shown in which a plurality of phase plates as light wavefront modulation elements are selectively inserted into or retracted from the optical path of the optical system 210 according to the state relating to imaging. For example, the present invention can also be applied to a case where the phase plate is selectively inserted into or retracted from the optical path of the optical system 210 according to the photographing mode. This example will be described later in detail as an embodiment including macro shooting in macro mode and lens driving.

  7 and 8 are diagrams illustrating a configuration example of the selection switching unit 400 of the present embodiment.

  The selection switching unit 400A of FIG. 7 has a plurality of (3 or 4 in this example) phase plates 213a-1 to 213a-4 having different modulation characteristics in a locus concentric with a rotationally movable part 401 having a rotation center. The desired phase plate 213 a-1 (˜−) is arranged according to the information generated by the imaging information generation unit 300 by engaging with the gear 402 attached to the edge of the component 401 and the rotation shaft of the motor 402. 4) is inserted (positioned) in the optical path including the optical axis center AX or retracted. Thereby, the light wavefront modulation is realized according to the photographing information.

  The selection switching unit 400B in FIG. 8 has a plurality of (1 or 2 in this example) phase plates 213a-1 and 213a-2 with different modulation characteristics arranged in parallel in a direction perpendicular to the optical axis center on the plate-like component 404. Arranged along a locus on a concentric circle, meshed with one edge of the component 404 and a gear 406 attached to the rotating shaft of the motor 405, and in accordance with information generated by the imaging information generating unit 300 The phase plate 213a-1 (, -2) is inserted (positioned) into or retracted from the optical path including the optical axis center AX. Thereby, the light wavefront modulation is realized according to the photographing information.

In the examples of FIGS. 7 and 8, the phase plate is arranged at all the arrangement positions of the phase plates of the components 401 and 404. For example, a portion where no phase plate is arranged is provided, and the light wavefront modulation is performed according to the information. It is also possible to make it enter into the image pick-up element 220, without giving.
When such a configuration is adopted, for example, when the shooting information is information related to the shooting mode, and the macro (proximity) mode, for example, the phase plate is inserted (positioned) in the optical path, and the image is slightly blurred. In this case, a mode in which a portion where the phase plate is not arranged is inserted (positioned) in the optical path is possible.
That is, it is possible to obtain a blurred image like a conventional optical system while using a depth expanding optical system for the purpose of obtaining a focused image by an optical wavefront modulation element and image processing.

It is also possible to simply insert or retract an optical wavefront modulation element such as a phase plate in the optical path so that the optical wavefront modulation element is not disposed in the optical path or the optical wavefront modulation processing is not performed.
Furthermore, it is also possible to fix the light wavefront modulation element corresponding to one phase plate on the optical path and insert or retract the light wavefront modulation element such as another phase plate in the optical path.

  In the above, only a plurality of phase plates that are considered to be easy to control and highly effective in space are illustrated as one part. However, a plurality of light wavefront modulation elements (phase plates, etc.) are not included in one part. ) Can also be adopted.

  In the case where the information related to shooting generated by the shooting information generation unit 300 is focal length information, for example, the configuration is as follows.

FIG. 9 is a block diagram of a processing system in the case where information related to shooting generated by the shooting information generation unit 300 is focal length information. FIG. 10 is a simple flowchart for switching to the light wavefront modulation element and the restoration process according to the focal length.
In this case, the imaging information generation unit 300 stores lens information for each focal length, information on a plurality of light wavefront modulation elements (phase plates in the present embodiment), and information on restoration processing in a storage device. .
When the focal point is moved, the focal point information is acquired, the stored information is extracted from the storage device, and the selection switching unit 400 switches to an appropriate state.
After that, image processing is performed by switching the filter of the convolution operation unit of the selected image processing apparatus 500 for the signal of the image sensor 220 obtained by photographing, and the image is restored to a focused image (ST1 to ST4).

FIG. 11 is a block diagram of a processing system in the case where the information related to shooting generated by the shooting information generation unit 300 is the depth of field that is the result of multipoint distance measurement. FIG. 12 is a simple flowchart of switching to the light wavefront modulation element and the restoration process according to the focal length according to the depth of field.
With one optical wavefront modulation element and one restoration process, the depth of field is in a certain range. In addition, it is possible to adjust the depth of field by changing the coefficient of the restoration process, but at the same time, the influence of noise included at the time of shooting also changes, making it difficult to restore an appropriate image.
Therefore, this problem can be solved by performing switching and restoration processing of the light wavefront modulation element according to the desired depth of field.

In this case, the imaging information generation unit 300 stores lens information, information on a plurality of phase modulation elements, and information on restoration processing in a storage device.
The distance measurement result executed at the time of photographing, for example, information input by the user for the near point side (first point) and the far point side (second point) is acquired and the necessary depth of field is obtained. Calculate and store in memory.
Based on the result, the stored optical wavefront modulation element is compared with the depth of field obtained by the restoration process, and the appropriate combination of the optical wavefront modulation element and the restoration process is switched.
After that, image processing is performed by switching the filter of the convolution operation unit of the selected image processing apparatus 500 for the signal of the image sensor 220 obtained by shooting, and the image is restored to a focused image (ST11 to ST17).

As described above, the present invention can also be applied to a camera or the like provided with multipoint ranging. There are various methods for ranging, depending on the device, but distance information is extracted and calculated from multiple ranging areas in the field of view, and the depth of field is determined. To do.
Here, the depth of field is calculated from the first distance measurement information on the short distance side and the second distance measurement information on the long distance side, and the optimum phase modulation element and restoration process are selected based on the result. An example is shown.
This means can be used with a single focal point or multiple focal points, and may be based on multi-point distance measurement in calculating the depth of field.

In the above, examples of the information related to photographing generated by the photographing information generation unit 300 include the focal length of the lens and the result of multipoint ranging. However, the zoom information of the zoom optical system 210, the approximate distance information of the object distance of the subject, Alternatively, control can be performed in the same manner as described above using the shooting mode information.
In the case of the approximate distance information of the object distance of the subject, for example, a body approximate distance information detection device 400 including an AF sensor or the like is provided.

  When the shooting information generation unit 300 generates shooting mode information, the shooting mode selected by the operation switch and input when the imaging device (camera) 100 enters the shooting state (in this embodiment, for example, normal The approximate distance of the object distance of the subject corresponding to the photographing mode, the distant view photographing mode, the macro photographing mode, and the blurred mode) is read from the object approximate distance information detection device and supplied to the image processing device 500.

  The image processing apparatus 500 supplies the lens focal length, multipoint distance measurement, zoom information of the zoom optical system 210, approximate distance information of the object distance of the subject, or shooting mode information supplied from the shooting information generation unit 300. Based on the function of generating an image signal having no dispersion from the dispersion image signal from the image sensor 220 and the phase plate as the light wavefront modulation element in the retracted state, and a predetermined image for the image signal not subjected to the light wavefront modulation action Has a processing function.

  In the present embodiment, as described above, dispersion means that, by inserting the phase plate 213a, an image that does not fit anywhere on the image sensor 220 is formed, and the phase plate 213a has a deep light flux ( It plays a central role in image formation) and a phenomenon of forming flare (blurred portion), and includes the same meaning as aberration because of the behavior of the image being dispersed to form a blurred portion. Therefore, in this embodiment, it may be described as aberration.

  FIG. 13 is a block diagram illustrating a configuration example of the image processing apparatus 500 that generates an image signal having no dispersion from the dispersion image signal from the image sensor 220.

  As illustrated in FIG. 13, the image processing apparatus 500 includes a convolution apparatus 501, a kernel / numerical arithmetic coefficient storage register 502, and an image processing arithmetic processor 503.

  In this image processing apparatus 500, the image processing arithmetic processor 503 that obtains information related to photographing supplied from the photographing information generation unit 300 uses the kernel size and the arithmetic coefficient used in appropriate computation for the object separation position. An appropriate calculation is performed by a convolution device 501 that stores the value in the kernel and numerical value calculation coefficient storage register 502, uses the value, and switches the filter according to information related to shooting supplied from the shooting information generation unit 300. And restore the image.

Here, the basic principle of the wavefront aberration control optical system will be described.
As shown in FIG. 14, the image f of the subject enters the wavefront aberration control optical system optical system H, thereby generating a g image.
This can be expressed by the following equation.

(Equation 1)
g = H * f
Here, * represents convolution.

  In order to obtain a subject from the generated image, the following processing is required.

(Equation 2)
f = H ^-1 * g

Here, the kernel size and calculation coefficient regarding the function H will be described.
The individual object approximate distances are AFPn, AFPn-1,..., And the individual zoom positions (zoom positions) are Zpn, Zpn-1,.
The H function is defined as Hn, Hn-1,.
Since each spot is different, each H function is as follows.

  The difference in the number of rows and / or the number of columns in this matrix is the Carney size, and each number is the calculation coefficient.

As described above, in the case of an imaging device including a phase plate (Wavefront Coding optical element) as an optical wavefront modulation element, an image signal without proper aberrations by image processing within the predetermined focal length range However, if it is outside the predetermined focal length range, there is a limit to the correction of the image processing, so that only an object outside the above range has an image signal with aberration.
On the other hand, by performing image processing in which no aberration occurs within a predetermined narrow range, it is possible to bring out a blur to an image outside the predetermined narrow range.
In the present embodiment, the focal length of the lens, the result of multipoint ranging, the zoom information of the zoom optical system 210, the approximate distance information of the object distance of the subject, or the shooting mode information supplied from the shooting information generation unit 300. In response to this, different image correction processes are performed.

The image processing is performed by convolution calculation. To achieve this, for example, one type of convolution calculation coefficient is stored in common, and a correction coefficient is stored in advance according to the focal length, The correction coefficient is used to correct the calculation coefficient, and an appropriate convolution calculation can be performed using the corrected calculation coefficient.
In addition to this configuration, the following configuration can be employed.

  For example, the kernel size and the convolution calculation coefficient itself are stored in advance according to the focal length, the convolution calculation is performed using the stored kernel size and the calculation coefficient, and the calculation coefficient according to the focal length is previously stored as a function. It is possible to adopt a configuration in which a calculation coefficient is obtained from this function based on the focal length and the convolution calculation is performed using the calculated calculation coefficient.

  Further, in addition to the processing according to the focal length of the lens and the result of the multipoint distance measurement described above, the following configuration can be taken in association with the configuration of FIG.

At least two or more conversion coefficients corresponding to the aberration caused by the phase plate 213a are stored in advance in the register 502 as the conversion coefficient storage means according to the subject distance. A coefficient for the image processing arithmetic processor 503 to select a conversion coefficient corresponding to the distance from the register 502 to the subject based on the information generated by the object approximate distance information detecting device as the subject distance information generating means of the photographing information generating unit 300 It functions as a selection means.
Then, the convolution device 501 as the conversion unit converts the image signal based on the conversion coefficient selected by the image processing arithmetic processor 503 as the coefficient selection unit and the supply information of the imaging information generation unit 300.

Alternatively, as described above, the image processing calculation processor 503 as the conversion coefficient calculation unit calculates the conversion coefficient based on the information generated by the object approximate distance information detection device as the subject distance information generation unit, and stores it in the register 502. To do.
Then, the convolution device 501 as the conversion means converts the image signal by the conversion coefficient obtained by the image processing arithmetic processor 503 as the conversion coefficient calculation means and stored in the register 502 and the supply information of the imaging information generation unit 300. Do.

Alternatively, zoom information supplied from the photographing information generation unit 300 is input, and at least one correction value corresponding to the zoom position or zoom amount of the zoom optical system 210 is stored in advance in the register 502 serving as a correction value storage unit. This correction value includes the kernel size of the subject aberration image.
A conversion coefficient corresponding to an aberration caused by a desired phase plate 213a is stored in advance in a register 502 that also functions as a second conversion coefficient storage unit.
Then, based on photographing information such as distance information, focal length information, and multi-point distance measurement results, the image processing arithmetic processor 503 serving as a correction value selection unit determines the distance from the register 502 serving as a correction value storage unit to the subject. Select the correct correction value.
The convolution device 501 as the conversion means selects the conversion coefficient obtained from the register 502 as the second conversion coefficient storage means and the supply information of the imaging information generation unit 300, and the image processing arithmetic processor 503 as the correction value selection means. The image signal is converted based on the corrected value.

  When the shooting information is shooting mode information, for example, the image processing apparatus 500 performs a normal conversion process in the normal shooting mode and a macro conversion corresponding to a macro shooting mode in which aberration is reduced closer to the normal conversion process. The processing and the distant view conversion process corresponding to the distant view photographing mode in which aberration is reduced on the far side compared to the normal conversion process are selectively executed according to the photographing mode.

In the present embodiment, a wavefront aberration control optical system can be adopted to obtain a high-definition image quality, and the optical system can be simplified and the cost can be reduced.
Hereinafter, this feature will be described.

FIGS. 15A to 15C show spot images on the light receiving surface of the imaging element 220 of the imaging lens device 200. FIG.
15A shows a case where the focal point is shifted by 0.2 mm (Defocus = 0.2 mm), FIG. 15B shows a case where the focal point is a focal point (Best focus), and FIG. 15C shows a case where the focal point is shifted by −0.2 mm. In this case, each spot image is shown (Defocus = −0.2 mm).
As can be seen from FIGS. 15A to 15C, in the imaging lens device 200 according to the present embodiment, a light beam having a deep depth (a central part of image formation) is formed by the wavefront forming optical element group 213 including the phase plate 213a. Play a role) and flare (blurred part) is formed.

  As described above, the primary image FIM formed in the imaging lens device 200 of the present embodiment has a light beam condition with a very deep depth.

FIGS. 16A and 16B are diagrams for explaining a modulation transfer function (MTF) of a primary image formed by the imaging lens device according to the present embodiment. FIG. 16A is a diagram showing a spot image on the light receiving surface of the imaging element of the imaging lens device, and FIG. 16B shows the MTF characteristics with respect to the spatial frequency.
In the present embodiment, the high-definition final image is left to the correction process of the image processing apparatus 500 including a digital signal processor, for example, as shown in FIGS. 16A and 16B. The MTF of the primary image is essentially a low value.

  The image processing apparatus 500 is configured by a DSP, for example, and receives a primary image FIM from the imaging lens apparatus 200 as described above and performs a predetermined correction process or the like that raises the MTF at the spatial frequency of the primary image. A fine final image FNLIM is formed.

The MTF correction processing of the image processing apparatus 500 is performed after edge enhancement, chroma enhancement, etc., using the MTF of the primary image, which is essentially a low value, as shown by a curve A in FIG. In the process, correction is performed so as to approach (reach) the characteristic indicated by the curve B in FIG.
The characteristic indicated by the curve B in FIG. 17 is a characteristic obtained when the wavefront is not deformed without using the wavefront forming optical element as in the present embodiment, for example.
It should be noted that all corrections in the present embodiment are based on spatial frequency parameters.

In the present embodiment, as shown in FIG. 17, in order to achieve the MTF characteristic curve B to be finally realized with respect to the MTF characteristic curve A with respect to the optically obtained spatial frequency, each spatial frequency is changed to each spatial frequency. On the other hand, the original image (primary image) is corrected by applying strength such as edge enhancement.
For example, in the case of the MTF characteristic of FIG. 17, the curve of edge enhancement with respect to the spatial frequency is as shown in FIG.

  That is, a desired MTF characteristic curve B is virtually realized by performing correction by weakening edge enhancement on the low frequency side and high frequency side within a predetermined spatial frequency band and strengthening edge enhancement in the intermediate frequency region. To do.

As described above, the imaging apparatus 100 according to the embodiment includes the imaging lens apparatus 200 including the optical system 210 that forms the primary image, and the image processing apparatus 500 that forms the primary image into a high-definition final image. In the optical path of the optical system, by selectively disposing an optical element for wavefront shaping, or by selectively providing a surface of an optical element such as glass or plastic that is molded for wavefront shaping, The wavefront of the imaging is deformed, such a wavefront is imaged on the imaging surface (light receiving surface) of the imaging element 220 composed of a CCD or a CMOS sensor, and the primary image formed through the image processing apparatus 500 is a high-definition image. Can be formed.
In the present embodiment, the primary image obtained by the imaging lens device 200 has a light beam condition with a very deep depth. For this reason, the MTF of the primary image is essentially a low value, and the MTF is corrected by the image processing apparatus 500.

Here, the imaging process in the imaging lens apparatus 200 in the present embodiment will be considered in terms of wave optics.
A spherical wave diverging from one of the object points becomes a convergent wave after passing through the imaging optical system. At that time, aberration occurs if the imaging optical system is not an ideal optical system. The wavefront is not a spherical surface but a complicated shape. Wavefront optics lies between geometric optics and wave optics, which is convenient when dealing with wavefront phenomena.
When dealing with the wave optical MTF on the imaging plane, the wavefront information at the exit pupil position of the imaging optical system is important.
The MTF is calculated by Fourier transform of the wave optical intensity distribution at the imaging point. The wave optical intensity distribution is obtained by squaring the wave optical amplitude distribution, and the wave optical amplitude distribution is obtained from the Fourier transform of the pupil function in the exit pupil.
Further, since the pupil function is exactly from the wavefront information (wavefront aberration) at the exit pupil position itself, if the wavefront aberration can be strictly numerically calculated through the optical system 210, the MTF can be calculated.

Accordingly, if the wavefront information at the exit pupil position is modified by a predetermined method, the MTF value on the imaging plane can be arbitrarily changed.
In this embodiment, the wavefront shape is mainly changed by the wavefront forming optical element, but the target wavefront is formed by increasing or decreasing the phase (phase, optical path length along the light beam). .
Then, if the desired wavefront is formed, the exiting light flux from the exit pupil is made up of dense and sparse portions of the light, as can be seen from the geometric optical spot images shown in FIGS. It is formed.
The MTF in the luminous flux state shows a low value at a low spatial frequency and a characteristic that the resolving power is managed up to a high spatial frequency.
That is, if this MTF value is low (or such a spot image state in terms of geometrical optics), the phenomenon of aliasing will not occur.
That is, a low-pass filter is not necessary.
Then, the flare-like image that causes the MTF value to be lowered may be removed by the image processing apparatus 500 including a DSP or the like at the subsequent stage. Thereby, the MTF value is significantly improved.

  Next, the response of the MTF of this embodiment and the conventional optical system will be considered.

FIG. 19 is a diagram showing the MTF response when the object is at the focal position and when the object is out of the focal position in the case of the conventional optical system.
FIG. 20 is a diagram showing the MTF response when the object is at the focal position and when the object is out of the focal position in the optical system of the present embodiment having the light wavefront modulation element.
FIG. 21 is a diagram illustrating a response of the MTF after data restoration of the imaging apparatus according to the present embodiment.

As can be seen from the figure, in the case of an optical system having a light wavefront modulation element, even when the object deviates from the focal position, the change in the MTF response is smaller than the optical diameter in which the light wavefront modulation element is not inserted.
The response of the MTF is improved by processing the image formed by this optical system using a convolution filter.

As described above, according to the first embodiment, the imaging lens device 200 that captures the subject image that has passed through the optical system or the subject dispersion image that has passed through the optical system and the phase plate (light wavefront modulation element); , An image processing device 500 that generates an image signal having no dispersion from the dispersion image signal from the image sensor 200, and information corresponding to the distance to the subject, zoom information, or shooting mode information as a result of focal length and multipoint distance measurement. And a selection switching unit 400 that selectively inserts or retracts a desired phase plate among a plurality of phase plates on the optical path according to the imaging information. The image processing apparatus 500 generates a non-dispersed image signal from the dispersed image signal based on the information generated by the photographing information generating unit 300, and thus changes the processing load. Can be secured also image quality after processing without.
In addition, it is possible to obtain a blurred image like a conventional optical system while using a depth-expanding optical system for the purpose of obtaining a focused image by an optical wavefront modulation element and image processing.
In addition, there is an advantage that the lens can be designed without worrying about the object distance and the defocus range, and the image can be restored by calculation such as convolution with high accuracy.
Further, according to the present invention, the optical system can be simplified and the cost can be reduced.
The imaging apparatus 100 according to the present embodiment can be used for a zoom lens wavefront aberration control optical system in consideration of the small size, light weight, and cost of consumer devices such as digital cameras and camcorders.
Furthermore, fingerprint authentication, vein authentication, iris authentication, and the like are possible, and it can be used for a biometric authentication device that has different authentication conditions depending on individual differences or performs two or more authentications.

In the present embodiment, the imaging lens device 200 having a wavefront forming optical element that deforms the wavefront of the imaging on the light receiving surface of the imaging device 220 by the imaging lens 212, and the primary image FIM by the imaging lens device 200. The image processing apparatus 500 that forms a high-definition final image FNLIM by performing a so-called predetermined correction process for raising the MTF at the spatial frequency of the primary image and the like, thereby obtaining a high-definition image quality. There is an advantage that it becomes possible.
In addition, the configuration of the optical system 210 of the imaging lens device 200 can be simplified, manufacturing becomes easy, and cost can be reduced.

By the way, when a CCD or CMOS sensor is used as an image sensor, there is a resolution limit determined by the pixel pitch, and if the resolution of the optical system exceeds the limit resolution, a phenomenon such as aliasing occurs, which adversely affects the final image. It is a well-known fact that
In order to improve image quality, it is desirable to increase the contrast as much as possible, but this requires a high-performance lens system.

However, as described above, aliasing occurs when a CCD or CMOS sensor is used as an image sensor.
Currently, in order to avoid the occurrence of aliasing, the imaging lens apparatus uses a low-pass filter made of a uniaxial crystal system to avoid the occurrence of aliasing.
The use of a low-pass filter in this way is correct in principle, but the low-pass filter itself is made of crystal, so it is expensive and difficult to manage. Moreover, there is a disadvantage that the use of the optical system makes the optical system more complicated.

As described above, in order to form a high-definition image, the optical system must be complicated in the conventional imaging lens apparatus in spite of the demand for higher-definition image due to the trend of the times. . If it is complicated, manufacturing becomes difficult, and if an expensive low-pass filter is used, the cost increases.
However, according to the present embodiment, the occurrence of aliasing can be avoided without using a low-pass filter, and high-definition image quality can be obtained.

  In the present embodiment, the example in which the wavefront forming optical element of the optical system 210 is disposed closer to the object side lens than the stop has been described. The effect of this can be obtained.

  Further, the lenses constituting the optical system 210 are not limited to the example of FIG. 2, and various aspects of the present invention are possible.

In the first embodiment, an example has been described in which one or more phase plates as light wavefront modulation elements are selectively inserted or withdrawn from the optical path of the optical system 210 according to the state relating to imaging. The present invention can also be applied to a case where a phase plate is selectively inserted into or retracted from the optical path of the optical system 210 in accordance with information relating to imaging, for example, the imaging mode.
Hereinafter, second to fifth embodiments including macro shooting in macro mode and lens driving will be described.

There is a problem that it is difficult to focus on the target part due to slight unevenness of the subject in macro shooting or the camera's front-back displacement due to handheld shooting, or it is difficult to focus on the whole screen due to the very shallow depth . On the other hand, in normal distance shooting, there is a demand for a camera that makes a work with the intention of drawing with a bright lens.
In the embodiment described below, the optical phase plate (light wavefront modulation element) and the blur restoration filter process are linked, but basically the optical phase plate (light wavefront modulation element) is retracted from the taking lens in the normal shooting distance mode. In the macro mode, an optical phase plate (light wavefront modulation element) is loaded to perform blur restoration filter processing.

Second Embodiment
FIG. 22 is a block diagram showing an imaging apparatus according to the second embodiment of the present invention.

  An imaging apparatus 600 according to the second embodiment has an AF (autofocus) function, and as shown in FIG. 22, an optical system 610, an imaging element 620, a taking lens driving unit 630, an optical wavefront modulation element (optical phase). Board) drive unit 640, analog front end unit (AFE) 650, timing generator 660, camera signal processing unit 670, external memory 680, LCD monitor 690 as an image monitoring device, operation unit 700, and system control unit (MPU) 710 have.

The optical system 610 supplies an image obtained by photographing the subject object OBJ to the image sensor 620.
The optical system 610 of this embodiment includes a lens barrel 6101 that can move the taking lens configuration to a so-called macro area that enables close-up photography, includes a focus lens 611 and a zoom optical system 612, and also drives an optical phase plate. The light wavefront modulation element is selectively included by the portion 640, and is formed so that the amount of defocusing is substantially constant at the in-focus position and the distance before and after the in-focus position.

As the optical wavefront modulation element, for example, an optical phase plate 613 is applied.
Here, since the optical phase plate has a function of dispersing the light flux, the image to be connected on the imaging surface of the image sensor 620 by the imaging optical system (taking lens) 610 including the optical phase plate 613 is blurred without being focused. It becomes a state.
However, this blurred state is not a blur caused by simple defocusing, but is the most concentrated state (similar to a focused state) as a spot diagram, and is not in focus. . This is a result of spreading by the function of dispersing the light beam of the optical phase plate 613 as described above.
Thus, since the focusing lens is moved according to the distance of the subject, the blurring state is the smallest and is maintained almost unchanged. Further, this blur shape becomes a blur in accordance with a point spread function PSF (Point Spread Function) which is a response of a designed point light source.
By the way, the optical phase plate 613 is designed so that the change in the blur shape is insensitive to the defocus amount. Therefore, blur images of objects existing at different distances around the subject distance are defocused. Does not change much.
As described above, the image formed on the imaging element surface through the taking lens is blurred as designed by the optical phase plate 613.

The image pickup device 620 forms an image captured by the optical system 610, and outputs the formed primary image information as a primary image signal FIM of an electric signal to the camera signal processing unit 670 via the analog front end unit 650. It consists of a CCD or CMOS sensor.
In FIG. 1, the image sensor 620 is described as a CCD as an example.

  The optical phase plate driving unit 640 selectively inserts the optical phase plate 613 into the optical path of the taking lens configuration when the lens configuration is in the macro state under the instruction of the MPU 710. 613 is selectively retracted (escaped) from the optical path of the taking lens structure.

  The taking lens driving unit 630 moves the focus lens 611 in the taking lens configuration 6101 to a so-called macro area that enables close-up photography under the instruction of the system control device 710.

The analog front end unit 650 includes an analog / digital (A / D) converter 651.
The A / D converter 651 converts an analog signal input from the CCD into a digital signal, and outputs digital data having a Bayer array, for example, to the camera signal processing unit 670.

  The timing generator 660 generates the drive timing of the CCD of the image sensor 620 under the control of the system control device 710.

As shown in FIG. 22, the camera signal processing unit 670 includes a RAW buffer 671, an RGB conversion unit 672, a YUB conversion unit 673, a blur restoration filter processing unit 674, a Y′UV processing unit 675, and a JPEG engine 676.
In the present embodiment, the camera signal processing unit 670 performs blur restoration processing of the blur restoration filter processing unit 674 because the optical phase plate 613 is inserted into the optical system in the macro state under the control of the system control device 710. Do.
On the other hand, the camera signal processing unit 670 performs the blur restoration process of the blur restoration filter processing unit 674 because the optical phase plate 613 is retracted (escaped) from the optical system in the non-macro state under the control of the system control device 710. Perform bypassed image processing.
Since the process of the blur restoration filter processing unit 674 is performed in the same manner as the process described in the first embodiment, the details thereof are omitted here.
Note that this process is an example, and for example, it is possible to perform a blur restoration process by inserting an optical phase plate 613 having different characteristics even in a non-macro state using the mechanism of FIG.

  The memory 680 stores captured images that have undergone predetermined image processing by the camera signal processing unit 670.

  An LCD monitor 690 serving as an image monitoring device displays a through image or a captured image that has undergone predetermined image processing by the camera signal processing unit 670, for example.

  The operation unit 700 includes an input switch that instructs the system control device 710 to perform predetermined function control. The operation unit 700 includes switches such as a shutter button, a release button, an enlarge button, a movement key, a movement key, a movement key right, and a movement key left.

  The system control device 710 performs exposure control and has operation inputs of the operation unit 700 and the like, determines the operation of the entire system according to those inputs, and takes a lens driving unit 630, an optical phase plate driving unit 640, It controls the timing generator 660, the camera signal processing unit 670, etc., and controls arbitration control of the entire system.

For example, in the macro mode, the system control device 710 controls the taking lens driving unit 630 and the optical phase plate driving unit 640 to move the taking lens to the macro region, and moves the optical phase plate 613 in the optical path of the optical system 610. And the camera signal processing unit 670 is controlled to perform the blur restoration filter processing unit 674.
For example, in the non-macro mode, the system control device 710 controls the taking lens driving unit 630 and the optical phase plate driving unit 640 to move the taking lens from the macro area to a predetermined area, and the optical phase plate 613 is optically operated. The camera 610 is retracted from the optical path of the system 610, and the camera signal processing unit 670 is controlled to bypass the processing of the blur restoration filter processing unit 674.

  When it is determined that the distance measurement result is a short distance, the system control device 710 determines that the macro mode is set, and the taking lens driving unit 630 automatically moves the taking lens to the configuration of the fixed focus macro lens. To control.

  Next, the operation of the second embodiment will be described.

First, a normal distance shooting state that is not close will be described.
An image (light ray) of the subject OBJ is formed on the image sensor 620 through the taking lens in the lens frame 6101.
When capturing image data, control data is set from the camera signal processing unit 670 to the timing generator 660, and the pixel signal FIM is discharged from the image sensor 620 according to the drive waveform of the timing generator 660.
The pixel signal FIM is converted into digital data by an internal AD converter 651 through an AFE (analog front end) circuit 650. Next, since this digital data is in a Bayer array, it is temporarily stored in the RAW buffer 661 and converted into an R plane, a G plane, and a B plane by interpolation processing by the RGB conversion unit 662. Further, it is converted into luminance and color difference signals by YUV conversion in the YUV conversion unit 663.
Since the optical phase plate 613 is retracted (escaped) from the optical path of the optical system 610, the blur restoration digital filter processing for the output after YUV conversion is passed in conjunction with it.
That is, Y ′ is the same as Y. The through image is displayed on the LCD monitor 690 by the Y′UV signal.

Now, when the release button or the like of the operation unit 700 is pressed, the system control device 710 sends a command to the camera signal processing unit 670 so as to acquire distance measurement data.
The camera signal processing unit 670 performs band pass processing (not shown) from the Y′UV signal to remove low frequency frequencies and integrate them to generate distance measurement data. While monitoring the distance measurement data, the system control device 710 measures the distance by the contrast method (so-called hill climbing method) while driving the focusing lens by the taking lens driving unit 630. The focusing lens position where the distance measurement contrast data is the highest (peak position of the mountain) is in focus.
When it is determined that a focus peak is present on the short distance side from the normal distance shooting area, the system control device 710 controls the taking lens driving unit 630 to move the focusing lens to a predetermined position for close-up shooting, and further takes. The optical phase plate 613 evacuated from the optical path of the lens optical system is loaded (inserted) into the optical system 610 of the taking lens.
By this optical phase plate 613, the point spread function (PSF) is blurred over a plurality of pixels. However, this optical phase plate is not greatly affected by the blurred state even if the distance changes, so the blurred state does not change greatly even if different subject distances exist in the screen.
This blur image data is processed through the same path as described above until YUV conversion, but in conjunction with the YUV conversion, the blur restoration filter processing unit 664 performs a blur restoration filter process, and the blur image is an in-focus image. To be restored.
At this time, as described above, since the shape of the PSF does not change greatly with respect to the distance (because the distance sensitivity is designed to be dull), it seems that the wide range is in focus when the blurred image is restored. can do.
The restored luminance signal Y ′ is displayed on the LCD monitor 680, and JPEG file compression is performed by the JPEG engine 666 to record the image file in the external memory.

According to the second embodiment, since it is possible to shoot without using an optical phase plate at a normal shooting distance, the blur is linear with respect to the defocus amount. The degree of in-focus becomes natural and you can shoot pictures with high artistic quality.
It is possible to use it with high hobby that allows you to create your favorite drawing with the lens aperture value, which is a common technique.
On the other hand, in macro photography, an optical phase plate is inserted and designed to be insensitive to changes in the PSF shape with respect to the defocus amount. In this way, an image that is in focus throughout the entire area can be obtained even if the object has irregularities. Therefore, it is possible to provide macro photography suitable for recording use. Further, in general macro photography, blur due to a deviation in the front-rear direction of the distance from the subject cannot be ignored in hand-held photography. Further, in the conventional method of reducing the lens aperture value, the shutter speed becomes slow and there is a problem of camera shake subject shake.
However, by using the blur restoration filter processing by inserting an optical phase plate in the macro shooting of the present embodiment, it is possible to absorb blur caused by some camera front and back at the time of shooting, and it is not necessary to reduce the lens aperture value. Therefore, it is possible to perform hand-held macro photography that was difficult in the past.

  In addition, when the user performs close-up shooting, the camera automatically determines and switches to the macro mode, and since the focus is fixed with an optical phase plate, it is possible to shoot with emphasis on deep recording and macro switching. There is no need for operation and there is no need for distance measurement in the macro area, so there is almost no release time lag. Accordingly, it is possible to greatly reduce the operation time from when the camera is pointed at a short distance to when an image is recorded.

<Third Embodiment>
Next, a third embodiment will be described. The system configuration of the third embodiment is the same as that shown in FIG.

In the third embodiment, the system controller 710 causes the taking lens driving unit 630 to move the taking lens to the configuration of the fixed focus macro lens when the user selects the macro mode using the operation unit 700. Control.
Further, the system control device 710 has a function of canceling the macro state when the user turns off the power.

In this case, when the imaging apparatus 600 is in the normal distance shooting mode, the user presses a macro button (not shown) of the operation unit 700.
The system control device 710 detects the operation of the macro button by the user, controls the taking lens driving unit 630, moves the focusing lens 611 of the taking lens to a predetermined position for close-up photography, and further takes the optical of the taking lens. The optical phase plate 613 evacuated from the optical path of the system 610 is loaded (inserted) into the optical system of the taking lens. By this optical phase plate 613, the point spread function (PSF) is blurred over a plurality of pixels. The optical phase plate 613 is loaded and the blur restoration filter processing is performed after YUV conversion, and the following processing is the same as described above.

Next, when the user presses a macro button (not shown) of the operation unit 700 in the close-up mode, the system control device 710 detects the user's operation of the macro button and moves the optical phase plate driving unit 640. Then, the optical phase plate 613 loaded (inserted) in the optical system 610 of the taking lens is retracted from the optical system 613 of the taking lens, and the taking lens driving hand 630 is controlled to focus the focusing lens 611 of the taking lens. Is moved to the normal distance shooting area.
Since the camera operation associated with the subsequent release is processed without passing through the blur restoration filter as described above, the photographed image is linearly blurred with respect to the defocus amount, resulting in an image of a natural description with a taste. .

Next, when the user presses a power button (not shown) of the operation unit 700 in the close-up mode, the system control device 710 detects the operation of the power button by the user, and sets the optical phase plate driving unit 640. The optical phase plate 613 loaded in the taking lens optical system 610 is controlled to be retracted from the taking lens optical system, and the taking lens driving unit 630 is used to move the taking lens focusing lens 611 to the normal distance photographing region. . Furthermore, the lens is retracted to the lens retracted state according to the design specifications.
Next, for example, when the user presses the power button of the operation unit 700, the macro mode is canceled and the normal photographing state is set. Therefore, the optical phase plate 613 is not loaded in the taking lens system, and the YUV signal for the post-process is The blur restoration filter process is passed.

According to the third embodiment, when the user selects the macro mode, since the focus is fixed with the optical phase plate, it is possible to set and perform shooting with emphasis on deep recording in advance. The release time lag can be almost eliminated.
In addition, when returning from the macro photography with the optical phase plate to the normal distance photography mode, the optical phase plate is retracted so that the blur restoration filter processing is not performed. It is possible to return to the shooting that made the best use of.
Also, when the user turns off the power button, the optical phase plate is retracted from the taking lens optical system, and the focusing lens is moved to the normal distance shooting mode. It is possible to prevent the failure of the camera from being used, and to use the normal shooting mode most likely in the next shooting immediately.

<Fourth embodiment>
Next, a fourth embodiment will be described.
FIG. 23 is a block configuration diagram showing an imaging apparatus according to the fourth embodiment of the present invention.

The imaging apparatus 600A according to the fourth embodiment is different from the imaging apparatus 600 of FIG. 22 according to the second and third embodiments in that the user mechanically moves the lens to the macro position. There is in being.
The imaging apparatus 600 </ b> A includes a macro lever 720, an interlocking mechanism unit 730, and a phase plate position detection unit 740.

First, a normal distance shooting state that is not close will be described.
In this case, the macro lever 720 is at a normal distance position, and the optical phase plate 613 is mechanically (mechanically) retracted from the taking lens system by the interlocking mechanism 730. The phase plate position detector 740 detects the position of the optical phase plate 613 and transmits the retracted state to the system controller 710A.
An image (light ray) of the subject OBJ is formed on the image sensor 620 through the taking lens in the lens frame 6101.
When capturing image data, control data is set from the camera signal processing unit 670 to the timing generator 660, and the pixel signal FIM is discharged from the image sensor 620 according to the drive waveform of the timing generator 660.
The pixel signal FIM is converted into digital data by an internal AD converter 651 through an AFE (analog front end) circuit 650. Next, since this digital data is in a Bayer array, it is temporarily stored in the RAW buffer 661 and converted into an R plane, a G plane, and a B plane by interpolation processing by the RGB conversion unit 662. Further, it is converted into luminance and color difference signals by YUV conversion in the YUV conversion unit 663.
The system controller 710A is notified from the phase plate position detector 740 that the optical phase plate 613 has escaped from the taking lens, and this is transmitted to the camera signal processor 670.
Since the optical phase plate 613 is retracted (escaped) from the optical path of the optical system 610, the blur restoration digital filter processing for the output after YUV conversion is passed in conjunction with it.
That is, Y ′ is the same as Y. The through image is displayed on the LCD monitor 690 by the Y′UV signal.

  Here, when the release button or the like of the operation unit 700 is pressed, the image data is generated up to Y′UV in the above-described flow (Y ′ here is the same as Y), and the JPEG engine 666 performs JPEG file compression. The image file is recorded in the external memory 680.

Next, when the user moves the macro lever 720 to the macro side, the optical phase plate 613 evacuated from the optical system 610 of the taking lens is loaded (inserted) into the taking lens system by the interlocking mechanism 730. By this optical phase plate 613, the point spread function (PSF) is blurred over a plurality of pixels. However, this optical phase plate is not greatly affected by the blurred state even if the distance changes, so the blurred state does not change greatly even if different subject distances exist in the screen. At this time, the phase plate position detector 740 transmits to the system controller 710A that the optical phase plate 613 has been loaded.
When the release button or the like of the operation unit 700 is pressed in this state (macro shooting mode), the blurred image data is processed through the same path until the YUV conversion, but the blur restoration filter is interlocked after the YUV conversion. The blurred image is restored to an in-focus image. At this time, as described above, since the shape of the PSF does not change greatly with respect to the distance (because the distance sensitivity is designed to be dull), it seems that the wide range is in focus when the blurred image is restored. can do.
The restored Y′UV is displayed on the LCD monitor 690, and the JPEG engine 666 performs JPEG file compression to record the image file in the external memory 680.

  According to the fourth embodiment, since the optical phase plate is interlocked with a mechanism for moving the lens position between normal and macro by the user mechanically, particularly in a camera with a mobile phone that does not have an AF mechanism. A bright lens can solve the problem of narrow focus range in close-up photography.

<Fifth Embodiment>
Next, a fifth embodiment will be described. The system configuration of the fifth embodiment is the same as that of FIG.

The imaging apparatus 600A according to the fifth embodiment is different from the imaging apparatus 600 of FIG. 22 according to the second and third embodiments in that it selects whether to insert or retract an optical phase plate in the macro mode. It has a function.
More specifically, the fifth embodiment has a function of selecting a mode for loading the optical phase plate 613 and a mode for retracting the optical phase plate in macro photography, and the optical phase plate 613 is loaded. In macro photography, the focusing lens 612 is fixed at a predetermined position and does not perform distance measurement. In macro photography with the optical phase plate 613 retracted, the focusing lens 612 performs distance measurement and drives the focusing lens to perform AF operation. To do.

In this case, the user previously sets whether or not to use the optical phase plate 613 in the case of the macro area in the camera setting mode by the LCD monitor display corresponding to the operation unit 700.
When the phase plate is used during macro photography (recording mode), as described above, the optical phase plate 613 is loaded into the taking lens system when the macro photography is switched, and the focus lens is set at a predetermined fixed position. . The Y′UV signal of the focused image is generated from the post-process YUV signal through the blur restoration filter processing. Since this image is generated based on PSF whose shape change is insensitive to the defocus amount, the depth of field becomes deep. Also, since the AF operation is not performed, the release time lag is greatly shortened.

On the other hand, when the optical phase plate is not used during macro shooting in the camera setting mode (AF macro mode), when the user presses the macro button, the system control device 710 detects the macro button operation by operating the operation unit 700. Then, the focus lens is moved to the macro region via the taking lens driving unit 630.
Since the optical phase plate 613 is not used in the camera setting mode set in advance, the optical phase plate 613 is kept retracted from the taking lens system.
When the release button is pressed, the subject image (light ray) imaged on the image sensor 620 through the taking lens in the lens frame sets control data from the camera signal processing unit 670 to the timing generator 660 and drives the timing generator 660. A pixel signal is discharged from the image sensor 620 by the waveform.
The pixel signal FIM is converted into digital data by an internal AD converter 651 through an AFE (analog front end) circuit 650. Next, since this digital data is in a Bayer array, it is temporarily stored in the RAW buffer 661 and converted into an R plane, a G plane, and a B plane by interpolation processing by the RGB conversion unit 662. Further, it is converted into luminance and color difference signals by YUV conversion in the YUV conversion unit 663.

Since the optical phase plate 613 has escaped (withdrawn) from the optical system 610 of the taking lens, the output after the YUV conversion passes through the blur restoration digital filter processing in conjunction with it. That is, Y ′ is the same as Y. The through image is displayed on the LCD monitor 690 by the Y′UV signal.
At the same time, the system control device 710 sends a command to the camera signal processing unit 670 so as to acquire distance measurement data. The camera signal processing unit 670 performs band pass processing (not shown) from the Y′UV signal to remove low frequency frequencies and integrate them to generate distance measurement data. While monitoring the distance measurement data, the system control device 710 controls the taking lens driving unit 630 to drive the focusing lens 612 and performs distance measurement by a contrast method (so-called hill climbing method). The focusing lens position where the distance measurement contrast data is highest (the peak position of the mountain) is in focus. When in focus, the Y′UV signal is JPEG file compressed by the JPEG engine 666 and an image file is recorded in the external memory 680.

According to the fifth embodiment, since it is possible to select whether or not the optical phase plate 613 is loaded at the time of macro photography, in the macro photography for recording, the optical phase plate is loaded and a blurred image restoration filter is used. The depth of field can be expanded.
On the other hand, in AF macro shooting, the optical phase plate is retracted and is not used for shooting, and the blurred image restoration filter is not used. Therefore, the blur effect can be intentionally used, and it is slightly generated by restored digital processing. Because it is not affected by noise, you can enjoy highly artistic shooting.
It also has a function for selecting a mode for loading an optical phase plate and a mode for retracting the optical phase plate in macro photography. In macro photography with an optical phase plate 613, the focusing lens is fixed at a predetermined position for measurement. In macro photography where the optical phase plate is retracted without performing the distance operation, the focusing lens performs the distance measurement operation and performs the AF operation by driving the focusing lens, so if the optical phase plate is loaded, the lens Since there is no release time lag because there is no need to move the lens, and there is no release time lag, it is convenient to use for recording. When the optical phase plate is retracted, AF operation is performed, so artistry is possible even in the macro area. High-quality photography can be performed.

It is a block block diagram which shows the imaging device which concerns on the 1st Embodiment of this invention. It is a figure which shows typically the structural example of the zoom optical system of the imaging lens apparatus which concerns on this embodiment. It is a figure which shows the spot image of the infinite side of the zoom optical system which does not include a phase plate. FIG. 4 is a diagram showing a spot image on the close side of a zoom optical system that does not include a phase plate. It is a figure which shows the spot image of the infinite side of the zoom optical system containing a phase plate. It is a figure which shows the spot image of the near side of the zoom optical system containing a phase plate. It is a figure which shows the 1st structural example of the selection switching part of this embodiment. It is a figure which shows the 2nd structural example of the selection switching part of this embodiment. It is a block diagram of a processing system when the information regarding imaging | photography which a imaging | photography information generation part produces | generates is focal distance information. It is a simple flowchart of switching to a light wavefront modulation element and a restoration process according to a focal distance. It is a block diagram of a processing system in case the information regarding the imaging | photography which a imaging | photography information generation part produces | generates the depth of field which is a result of multipoint ranging. It is a simple flowchart of switching to an optical wavefront modulation element and a restoration process according to a focal length according to a depth of field. It is a block diagram which shows the specific structural example of the image processing apparatus of this embodiment. It is a figure for demonstrating the principle of a wavefront aberration control optical system system. It is a figure which shows the spot image in the light-receiving surface of the image pick-up element of the image pick-up lens apparatus which concerns on this embodiment, Comprising: (A) is a case where a focus shifts by 0.2 mm (Defocus = 0.2 mm), (B) is a result. In the case of a focus (Best focus), (C) is a figure which shows each spot image when a focus shifts by -0.2 mm (Defocus = -0.2 mm). It is a figure for demonstrating MTF of the primary image formed with the imaging lens apparatus which concerns on this embodiment, Comprising: (A) is a figure which shows the spot image in the light-receiving surface of the image pick-up element of an imaging lens apparatus, ( B) shows the MTF characteristic with respect to the spatial frequency. It is a figure for demonstrating the MTF correction process in the image processing apparatus which concerns on this embodiment. It is a figure for demonstrating concretely the MTF correction process in the image processing apparatus which concerns on this embodiment. It is a figure which shows the response (response) of MTF when an object exists in a focus position in the case of the conventional optical system, and when it remove | deviated from the focus position. It is a figure which shows the response of MTF when an object exists in a focus position in the case of the optical system of this embodiment which has a light wavefront modulation element, and remove | deviates from a focus position. It is a figure which shows the response of MTF after the data restoration of the imaging device which concerns on this embodiment. It is a block block diagram which shows the imaging device which concerns on 2nd, 3rd, 5th embodiment of this invention. It is a block block diagram which shows the imaging device which concerns on the 4th Embodiment of this invention. It is a figure which shows typically the structure and light beam state of a general imaging lens apparatus. FIG. 25A is a diagram showing a spot image on the light receiving surface of the imaging element of the imaging lens device of FIG. 24, where FIG. 24A shows a case where the focal point is shifted by 0.2 mm (Defocus = 0.2 mm), and FIG. In the case (Best focus), (C) is a diagram showing each spot image when the focal point is shifted by -0.2 mm (Defocus = -0.2 mm).

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 ... Imaging device, 200 ... Imaging lens device, 211 ... Object side lens, 212 ... Imaging lens, 213 ... Optical element for wavefront formation, 213a ... Phase plate (light wavefront) (Modulation element), 300 ... photographing information generation unit, 400 ... selection switching unit, 500 ... image processing device, 501 ... convolution device, 502 ... kernel, numerical operation coefficient storage register, 503 ... Image processing arithmetic processor, 600, 600A ... Imaging device, 610 ... Optical system, 620 ... Imaging element, 630 ... Taking lens driving unit, 640 ... Light wavefront modulation element ( Optical phase plate) drive unit, 650... Analog front end unit (AFE), 660... Timing generator, 670... Camera signal processing unit, 680. 690 ... LCD monitor, 700 ... operation unit, 710, 710A ... system control unit (MPU), 720 ... macro lever, 730 ... interlocking mechanism, 740 ... phase plate position detection Department.

Claims (16)

An optical system and at least one light wavefront modulation element;
An imaging device that captures a subject dispersion image that has passed through the optical system or the optical system and the light wavefront modulation device;
Image processing means including conversion means for generating an image signal having no dispersion from the subject dispersion image signal from the imaging device,
The optical wavefront modulation element can be inserted into and retracted from an optical path. Having a plurality of light wavefront modulation elements,
The imaging apparatus according to claim 1, further comprising a selection switching unit that selectively inserts and retracts each of the plurality of light wavefront modulation elements on the optical path. The imaging apparatus according to claim 2, wherein the selection switching unit selectively switches insertion and withdrawal of a desired light wavefront modulation element on the optical path according to a focal length. The imaging apparatus according to claim 2, wherein the selection switching unit selectively switches insertion and withdrawal of a desired light wavefront modulation element on the optical path according to a result of multipoint ranging. The optical system includes a zoom function,
The imaging apparatus according to claim 2, wherein the selection switching unit selectively switches insertion and retraction of a desired light wavefront modulation element on the optical path according to zoom information. The imaging apparatus according to claim 2, wherein the selection switching unit selectively switches insertion and withdrawal of a desired light wavefront modulation element on the optical path according to information corresponding to a distance to a subject. The imaging apparatus according to claim 2, wherein the selection switching unit selectively switches between insertion and withdrawal of a desired light wavefront modulation element on the optical path according to a photographing mode. 8. The imaging device inserts an optical wavefront modulation element on the optical path when in a macro mode / macro state, and retracts the optical wavefront modulation element from the optical path when in a non-macro mode / non-macro state. The imaging device according to claim 1. The imaging device
It has a lens barrel that can move the optical system to a macro area that enables close-up photography,
The imaging apparatus according to claim 8, wherein when the distance measurement result is determined to be a short distance, the taking lens of the optical system is moved to a fixed focus macro lens configuration. The imaging device
It has a lens barrel that can move the optical system to a macro area that enables close-up photography,
The imaging apparatus according to claim 8, wherein when the macro mode is selected, the taking lens of the optical system is moved to a configuration of a fixed focus macro lens. The imaging device
The imaging apparatus according to claim 8, further comprising a mechanism that mechanically moves the lens to a macro position. The image processing means includes
Means for inserting the optical wavefront modulation element into the optical path of the optical system when in a macro state, and retracting the optical wavefront modulation element from the optical path of the optical system when the macro state is released;
The image processing means includes
When the light wavefront modulation element is inserted in a macro state, the conversion unit generates an image signal having less dispersion than the subject dispersion image signal,
The imaging apparatus according to any one of claims 8 to 10, wherein when the light wavefront modulation element is released from a macro state and the light wavefront modulation element is retracted, the processing of the conversion unit is not performed. The imaging apparatus according to claim 11, further comprising a control unit that cancels the macro state when the user turns off the power. The imaging apparatus according to claim 1, wherein the imaging apparatus includes a selection unit that selects whether or not to insert an optical wavefront modulation element into the optical path in the macro mode. With a function of selecting a mode for inserting a light wavefront modulation element and a mode for retracting the light wavefront modulation element in macro photography,
In macro photography where the light wavefront modulation element is inserted, the focusing lens is fixed at a predetermined position and distance measurement is not performed. In macro photography where the light wavefront modulation element is retracted, the focusing lens performs distance measurement operation and focusing. The imaging apparatus according to claim 13, further comprising control means for driving a lens to perform an autofocus (AF) operation. A first step of generating information relating to shooting;
A second step of inserting or retracting a predetermined light wavefront modulation element on the optical path based on the information generated by the information generation step;
A third step of picking up an object dispersion image that has passed through the optical system and a position where the light wavefront modulation element can be arranged in a state in which the light wavefront modulation element is selectively set in the second step with an image pickup element; ,
And a fourth step of converting the dispersed image signal based on the information generated in the second step to generate an image signal without dispersion.

Images