WO2017191717A1

WO2017191717A1 - Control device and imaging device

Info

Publication number: WO2017191717A1
Application number: PCT/JP2017/012907
Authority: WO
Inventors: 翔平野口
Original assignee: ソニー株式会社
Priority date: 2016-05-06
Filing date: 2017-03-29
Publication date: 2017-11-09
Also published as: JPWO2017191717A1; US20190132562A1; US10972710B2

Abstract

A control device according to one embodiment of the present invention controls an imaging device provided with: a variable optical low pass filter; and an imaging element which generates image data from incident light which has passed through the variable optical low pass filter. The control device is provided with a first derivation unit and a second derivation unit. The first derivation unit derives resolution deterioration amounts and false colour generation amounts on the basis of a plurality of pieces of image data generated by the imaging element as a result of driving the imaging element while changing the effect of the variable optical low pass filter. The second derivation unit derives set values for the variable optical low pass filter on the basis of the resolution deterioration amounts and the false colour generation amounts derived by the first derivation unit.

Description

Control device and imaging device

The present disclosure relates to a control device and an imaging device.

When a Bayer coding imager is used in a normal single-panel digital camera, it is necessary to restore lost information by performing demosaic processing on image data obtained by the imager. However, in principle, it is difficult to completely derive the lost information, so that a reduction in resolution and the occurrence of artifacts cannot be avoided.

Therefore, it is conceivable to provide a low-pass filter between the lens and the imager. In this case, the false color that is one of the artifacts can be reduced by adjusting the effect of the low-pass filter. As a low-pass filter for reducing moire, for example, an optical low-pass filter described in Patent Document 1 below can be used.

JP 2006-08045 A

By the way, in the invention described in Patent Document 1, an image is acquired while reducing the effect of the low-pass filter until moire occurs, and an image obtained immediately before the occurrence of moire is selected from the acquired images. Is done. However, the invention described in Patent Document 1 has a problem that the image quality is deteriorated due to factors other than moire because an image on which the effect of the low-pass filter is strongly applied is easily selected. It is desirable to provide a control device and an imaging device capable of acquiring an image with balanced image quality when using a low-pass filter.

The control device according to an embodiment of the present disclosure sets the low-pass characteristic of the low-pass filter based on a change in resolution and a change in false color in a plurality of captured image data according to a change in the low-pass characteristic of the low-pass filter. A control unit for setting a value is provided.

The imaging apparatus according to an embodiment of the present disclosure includes an imaging element that generates image data from light incident via a low-pass filter. The imaging apparatus further sets a setting value of the low-pass characteristic of the low-pass filter based on a change in resolution and a change in false color in a plurality of image data that is captured in accordance with a change in the low-pass characteristic of the low-pass filter. It has.

In the control device and the imaging device according to the embodiment of the present disclosure, the low-pass characteristics of the low-pass filter based on the change in resolution and the change in false color in the plurality of imaged image data according to the change in the low-pass characteristics of the low-pass filter Is set. As a result, a low-pass filter setting value suitable for obtaining image data with a balanced image quality according to the user's purpose is obtained.

According to the control device and the imaging device of the embodiment of the present disclosure, it is possible to obtain a setting value of a low-pass filter suitable for obtaining image data with a balanced image quality according to a user's purpose. When the low-pass filter is used, an image with a balanced image quality can be acquired. In addition, the effect of this indication is not necessarily limited to the effect described here, Any effect described in this specification may be sufficient.

It is a figure showing an example of the schematic structure of the imaging device concerning one embodiment of this indication. It is a figure showing an example of schematic structure of the light-receiving surface in the imaging device of FIG. It is a figure showing an example of the data memorize | stored in the memory part of FIG. It is a figure showing an example of schematic structure of variable optical LPF (Low * Pass * Filter) of FIG. FIG. 5 is a diagram illustrating an example of a polarization conversion efficiency curve (VT curve) of the liquid crystal layer in FIG. 4. It is a figure showing an example of an effect | action of the variable optical LPF of FIG. It is a figure showing an example of an effect | action of the variable optical LPF of FIG. It is a figure showing an example of an effect | action of the variable optical LPF of FIG. FIG. 7 is a diagram illustrating an example of MTF (Modulation / Transfer / Function) in FIGS. 6A to 6C. It is a figure showing an example of evaluation data. FIG. 2 is a diagram illustrating an example of a schematic configuration of an image processing unit in FIG. 1. It is a figure showing an example of the imaging procedure in the imaging device of FIG. It is a figure showing an example of evaluation data. It is a figure showing an example of the imaging procedure in the imaging device of FIG. It is a figure showing an example of evaluation data. It is a figure showing an example of the imaging procedure in the imaging device of FIG. It is a figure showing an example of a histogram. It is a figure showing an example of the imaging procedure in the imaging device of FIG. It is a figure showing an example of the data memorize | stored in the memory part of FIG. It is a figure showing an example of the generation | occurrence | production position of the false color to which correction coefficient (alpha), (beta) is applied. It is a figure showing an example of the generation | occurrence | production position of the false color to which threshold value Th6, Th7 is applied. It is a figure showing an example of the imaging procedure in the imaging device of FIG. It is a figure showing an example of evaluation data. It is a figure showing a mode that a part of image data is expanded. It is a figure showing a mode that a part of image data is displayed by zebra. It is a figure showing a mode that a part of image data is edge-emphasized displayed. It is a figure showing an example of the imaging procedure in the imaging device of FIG.

Hereinafter, modes for carrying out the present disclosure will be described in detail with reference to the drawings. The description will be given in the following order.

1. Embodiment 2. FIG. Modified example

<1. Embodiment>
[Constitution]
FIG. 1 illustrates an example of a schematic configuration of an imaging apparatus 1 according to an embodiment of the present disclosure. The imaging device 1 includes, for example, an imaging optical system 10, a lens driving unit 20, an LPF driving unit 30, an imaging element 40, and an image processing unit 50. The imaging apparatus 1 further includes, for example, a display panel 60, a memory unit 70, a control unit 80, and an operation unit 90.

(Imaging optical system 10)
The imaging optical system 10 includes, for example, a variable optical LPF (Low Pass Filter) 11 and a lens 12. The lens 12 forms an optical subject image on the image sensor 40. The lens 12 has a plurality of lenses, and is driven by the lens driving unit 20 so that at least one lens can be moved. Thereby, the lens 12 can perform optical focus adjustment and zoom adjustment. The variable optical LPF 11 removes a high spatial frequency component contained in light, and is driven by the LPF driving unit 30 to change a cutoff frequency fc, which is one of low-pass characteristics. . The imaging optical system 10 may be configured integrally with the imaging device 40 or may be configured separately from the imaging device 40. In addition, the variable optical LPF 11 in the imaging optical system 10 may be configured integrally with the imaging device 40 or may be configured separately from the imaging device 40. A specific configuration of the variable optical LPF 11 and a modulation method of the cutoff frequency fc will be described in detail later.

(Lens drive unit 20, LPF drive unit 30)
The lens drive unit 20 drives at least one lens of the lens 12 for optical zoom magnification, focus adjustment, and the like in accordance with instructions from the control unit 80. The LPF drive unit 30 adjusts the effect of the variable optical LPF 11 by performing control to change the low-pass characteristic (cut-off frequency fc) of the variable optical LPF 11 in accordance with an instruction from the control unit 80. The “effect of the variable optical LPF 11” refers to a reduction in a component having a spatial frequency higher than the Nyquist frequency contained in light. For example, the LPF driving unit 30 adjusts the cutoff frequency fc of the variable optical LPF 11 by applying a predetermined voltage V (constant frequency) between the electrodes of the variable optical LPF 11.

(Image sensor 40)
The imaging element 40 generates image data by converting a subject image formed on the light receiving surface 40A via the lens 12 and the variable optical LPF 11 into an electrical signal by photoelectric conversion. The imaging device 40 is configured by, for example, a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

The imaging element 40 has a light receiving surface 40A in which a plurality of photoelectric conversion elements 40B are two-dimensionally arranged at a predetermined interval, for example, as shown in FIG. The image sensor 40 further includes a color filter array 40C on the light receiving surface 40A, for example. FIG. 2 illustrates a state in which the color filter array 40C has a Bayer array in which R, G, G, and B of a 2 × 2 matrix are arranged in a matrix. The color filter array 40C may have an array different from the Bayer array. The image sensor 40 generates image data based on light incident via the variable optical LPF 11. The image sensor 40 generates color image data by spatially sampling light incident through the lens 12 and the variable optical LPF 11, for example. The image data has a color signal of each color included in the color filter array 40C for each pixel.

(Image processing unit 50)
The image processing unit 50 performs image processing such as white balance, demosaicing, gradation conversion, color conversion, and noise reduction on the image data generated by the image sensor 40. The image processing unit 50 performs processing such as converting image data into display data suitable for display on the display panel 60, and converting image data into data suitable for recording in the memory unit 70. ing. The image processing in the image processing unit 50 will be described in detail later.

(Display panel 60, memory unit 70, operation unit 90)
The display panel 60 is configured by a liquid crystal panel, for example. The display panel 60 displays display data input from the image processing unit 50. The memory unit 70 can store photographed image data and various programs. The memory unit 70 is configured by, for example, a nonvolatile memory, and is configured by, for example, an EEPROM (Electrically Erasable Programmable Read-Only Memory), a flash memory, a resistance change type memory, or the like. The memory unit 70 may be an external memory that can be attached to and detached from the imaging apparatus 1. For example, as illustrated in FIG. 3, the memory unit 70 stores various data generated by the image processing unit 50 and various data input from the operation unit 90. FIG. 3 shows an example of data stored in the memory unit 70. Examples of data stored in the memory unit 70 include image data 71, saturation data 72, evaluation data 73, and a set value 74 as shown in FIG. The image data 71, the saturation data 72, the evaluation data 73, and the set value 74 will be described in detail later. The operation unit 90 receives an instruction from the user, and includes, for example, an operation button, a shutter button, an operation dial, a keyboard, and a touch panel.

(Control unit 80)
The control unit 80 is a processor that controls the lens driving unit 20, the LPF driving unit 30, the image sensor 40, the image processing unit 50, the display panel 60, and the memory unit 70. The control unit 80 controls the lens driving unit 20 to adjust the optical zoom magnification and focus of the lens 12. The control unit 80 adjusts the effect (cut-off frequency fc) of the variable optical LPF 11 by controlling the LPF driving unit 30. The controller 80 further drives the image sensor 40 to generate image data by the image sensor 40 and to output the generated image data to the image processor 50. The control unit 80 controls the image processing unit 50 to cause the image processing unit 50 to perform the above-described image processing, and to store various data obtained as a result of the above-described image processing in the memory unit 70 and the display panel 60. To output. The control unit 80 controls the lens driving unit 20, the LPF driving unit 30, the image sensor 40, the image processing unit 50, and the display panel 60 according to various data input from the operation unit 90, or is input from the operation unit 90. Various types of data are stored in the memory unit 70.

(Variable optical LPF11)
Next, the variable optical LPF 11 will be described in detail. FIG. 4 illustrates an example of a schematic configuration of the variable optical LPF 11. The variable optical LPF 11 removes high spatial frequency components contained in light. The variable optical LPF 11 is driven by the LPF driving unit 30 to change the effect (cut-off frequency fc) of the variable optical LPF 11. The variable optical LPF 11 is configured to change the cutoff frequency fc by, for example, a peak value modulation method. The peak value modulation method will be described later in detail.

The variable optical LPF 11 includes a pair of

birefringent plates

111 and 115 having birefringence, and a liquid crystal layer 113 disposed between the pair of

birefringent plates

111 and 115. The variable optical LPF 11 further includes

electrodes

112 and 114 that apply an electric field to the liquid crystal layer 113. Note that the variable optical LPF 11 may include, for example, an alignment film that regulates the alignment of the liquid crystal layer 113. The

electrodes

112 and 114 are arranged to face each other with the liquid crystal layer 113 interposed therebetween. Each of the

electrodes

112 and 114 is composed of one sheet-like electrode. Note that at least one of the electrode 112 and the electrode 114 may be composed of a plurality of partial electrodes.

The

electrodes

112 and 114 are translucent conductive films such as ITO (Indium Tin Oxide), for example. The

electrodes

112 and 114 may be, for example, a light-transmitting inorganic conductive film, a light-transmitting organic conductive film, or a light-transmitting metal oxide film. The birefringent plate 111 is disposed on the light incident side of the variable optical LPF 11, and for example, the outer surface of the birefringent plate 111 is a light incident surface 110A. The incident light L1 is light that enters the light incident surface 110A from the subject side. For example, the birefringent plate 111 is disposed so that the optical axis of the incident light L1 is parallel to the normal line of the birefringent plate 111 (or the light incident surface 110A). The birefringent plate 115 is disposed on the light exit side of the variable optical LPF 11, and for example, the outer surface of the birefringent plate 115 is a light exit surface 110B. The transmitted light L2 of the variable optical LPF 11 is light emitted to the outside from the light emitting surface 110B. The birefringent plate 111, the electrode 112, the liquid crystal layer 113, the electrode 114, and the birefringent plate 115 are stacked in this order from the light incident side.

The

birefringent plates

111 and 115 are birefringent and have a uniaxial crystal structure. The

birefringent plates

111 and 115 have a function of separating ps of circularly polarized light using birefringence. The

birefringent plates

111 and 115 are made of, for example, quartz, calcite, or lithium niobate.

In the

birefringent plates

111 and 115, for example, the image separation directions are opposite to each other. The optical axis AX1 of the birefringent plate 111 and the optical axis AX2 of the birefringent plate 115 intersect each other in a plane parallel to the normal line of the light incident surface 110A. The angle formed by the optical axis AX1 and the optical axis AX2 is, for example, 90 °. Further, the optical axes AX1 and AX2 obliquely intersect the normal line of the light incident surface 110A. The angle formed by the optical axis AX1 and the normal line of the light incident surface 110A is, for example, less than 90 ° counterclockwise with respect to the normal line of the light incident surface 110A, for example, 45 °. . The angle formed by the optical axis AX2 and the normal line of the light incident surface 110A is, for example, greater than 90 ° and smaller than 180 ° counterclockwise with respect to the normal line of the light incident surface 110A. ° (= 180-45 °).

FIG. 5 shows an example of a polarization conversion efficiency curve (VT curve) of the liquid crystal layer 113. In FIG. 5, the horizontal axis is the voltage V (frequency constant) applied between the

electrodes

112 and 114. In FIG. 5, the vertical axis represents the polarization conversion efficiency T. The polarization conversion efficiency T is obtained by multiplying a value obtained by dividing the phase difference given to linearly polarized light by 90 degrees by 100. A polarization conversion efficiency T of 0% means that no phase difference is given to linearly polarized light. For example, it means that linearly polarized light has passed through the medium without changing its polarization direction. Yes. A polarization conversion efficiency T of 100% means that a phase difference of 90 degrees is given to linearly polarized light. For example, a medium obtained by converting p-polarized light to s-polarized light or s-polarized light to p-polarized light. Indicates that it has passed through. A polarization conversion efficiency T of 50% means that a phase difference of 45 degrees is given to linearly polarized light. For example, p-polarized light or s-polarized light is converted into circularly polarized light and transmitted through the medium. pointing.

The liquid crystal layer 113 controls polarization based on the electric field generated by the voltage between the

electrodes

112 and 114. In the liquid crystal layer 113, as shown in FIG. 5, when the voltage V1 is applied between the

electrodes

112 and 114, the polarization conversion efficiency T becomes T2, and the voltage V2 (V1 <V2) is applied between the

electrodes

112 and 114. Then, the polarization conversion efficiency T becomes T1. T2 is 100% and T1 is 0%. Further, in the liquid crystal layer 113, as shown in FIG. 5, when the voltage V3 (V1 <V3 <V2) is applied between the

electrodes

112 and 114, the polarization conversion efficiency T becomes T3. T3 is a value larger than 0% and smaller than 100%. FIG. 5 illustrates a case where the voltage V3 is a voltage when T3 is 50%. Here, the voltage V1 is a voltage equal to or lower than the voltage at the falling position of the polarization conversion efficiency curve. Specifically, in the polarization conversion efficiency curve, the voltage in a section where the polarization conversion efficiency is saturated near the maximum value. pointing. The voltage V2 is a voltage equal to or higher than the voltage at the rising position of the polarization conversion efficiency curve, and specifically refers to a voltage in a section where the polarization conversion efficiency is saturated near the minimum value in the polarization conversion efficiency curve. The voltage V3 is a voltage (intermediate voltage) between the voltage at the falling position of the polarization conversion efficiency curve and the voltage at the rising position of the polarization conversion efficiency curve.

As described above, the liquid crystal layer 113 controls polarization. Examples of the liquid crystal having the polarization conversion efficiency curve as described above include a TN (TwistedistNematic) liquid crystal. The TN liquid crystal is composed of a chiral nematic liquid crystal and has an optical rotation that rotates the polarization direction of light passing therethrough along with the rotation of the nematic liquid crystal.

Next, the optical action of the variable optical LPF 11 will be described. 6A, 6B, and 6C illustrate an example of the action of the variable optical LPF 11. FIG. In FIG. 6A, the voltage V between the

electrodes

112 and 114 is the voltage V1. In FIG. 6B, the voltage V between the

electrodes

112 and 114 is the voltage V2. In FIG. 6C, the voltage V between the

electrodes

112 and 114 is the voltage V3.

(When V = V1 (FIG. 6A))
When the circularly polarized incident light L1 enters the birefringent plate 111, the incident light L1 is separated into p-polarized light and s-polarized light with a separation width d1 due to the birefringence of the birefringent plate 111. When the polarization component that vibrates perpendicularly to the optical axis AX1 of the birefringent plate 111 is an s-polarized component included in the incident light L1, the separated s-polarized light is birefringent in the birefringent plate 111. The light travels straight without being affected and exits from the back surface of the birefringent plate 111. The p-polarized component included in the incident light L1 vibrates in a direction orthogonal to the vibration direction of the s-polarized light, and therefore travels diagonally in the birefringent plate 111 due to the influence of birefringence, and the back surface of the birefringent plate 111 Among them, the light is refracted at a position shifted by the separation width d1 and emitted from the back surface of the birefringent plate 111. Accordingly, the birefringent plate 111 separates the incident light L1 into the p-polarized transmitted light L2 and the s-polarized transmitted light L2 with the separation width d1.

When the p-polarized light separated by the birefringent plate 111 is incident on the liquid crystal layer 113 having a polarization conversion efficiency of T2, the p-polarized light is converted into s-polarized light and travels straight in the liquid crystal layer 113. The light is emitted from the back surface. When the s-polarized light separated by the birefringent plate 111 is incident on the liquid crystal layer 113 having a polarization conversion efficiency of T2, the s-polarized light is converted to p-polarized light and travels straight in the liquid crystal layer 113. The light is emitted from the back surface. Therefore, the liquid crystal layer 113 performs ps conversion on the p-polarized light and the s-polarized light separated by the birefringent plate 111 while keeping the separation width constant.

When s-polarized light and p-polarized light transmitted through the liquid crystal layer 113 are incident on the birefringent plate 115, the separation width of the s-polarized light and p-polarized light changes depending on the birefringence of the birefringent plate 115. When the polarization component that vibrates perpendicularly to the optical axis AX2 of the birefringent plate 115 is s-polarized light, the s-polarized light travels straight in the birefringent plate 115 without being affected by birefringence, and the birefringent plate 115. The light is emitted from the back surface. Since the p-polarized light vibrates in a direction orthogonal to the vibration direction of the s-polarized light, the birefringent plate 115 is influenced by the birefringence and proceeds obliquely in a direction opposite to the image separation direction on the birefringent plate 111. . Further, the p-polarized light is refracted at a position shifted by the separation width d 2 among the back surface of the birefringent plate 115 and is emitted from the back surface of the birefringent plate 115. Therefore, the birefringent plate 115 separates the s-polarized light and p-polarized light transmitted through the liquid crystal layer 113 into s-polarized transmitted light L2 and p-polarized transmitted light L2 with a separation width (d1 + d2).

(When V = V2 (FIG. 6B))
The action of the birefringent plate 111 on the incident light L1 is the same as described above. Therefore, the operation of the liquid crystal layer 113 and the birefringent plate 115 will be described below. When the p-polarized light and the s-polarized light separated by the birefringent plate 111 enter the liquid crystal layer 113 having the polarization conversion efficiency T1, the p-polarized light and the s-polarized light are not subjected to polarization conversion by the liquid crystal layer 113, and the liquid crystal layer The light travels straight through 113 and exits from the back surface of the liquid crystal layer 113. Therefore, the liquid crystal layer 113 does not have an optical effect on p-polarized light and s-polarized light separated by the birefringent plate 111.

When s-polarized light and p-polarized light transmitted through the liquid crystal layer 113 are incident on the birefringent plate 115, the separation width of the s-polarized light and p-polarized light changes depending on the birefringence of the birefringent plate 115. When the polarization component that vibrates perpendicularly to the optical axis AX2 of the birefringent plate 115 is s-polarized light, the s-polarized light travels straight in the birefringent plate 115 without being affected by birefringence, and the birefringent plate 115. The light is emitted from the back surface. Since the p-polarized light vibrates in a direction orthogonal to the vibration direction of the s-polarized light, the birefringent plate 115 is influenced by the birefringence and proceeds obliquely in a direction opposite to the image separation direction on the birefringent plate 111. . Further, the p-polarized light is refracted at a position shifted by the separation width d 2 among the back surface of the birefringent plate 115 and is emitted from the back surface of the birefringent plate 115. Accordingly, the birefringent plate 115 separates the s-polarized light and the p-polarized light transmitted through the liquid crystal layer 113 into s-polarized transmitted light L2 and p-polarized transmitted light L2 with a separation width (| d1-d2 |). To do. Here, when d1 = d2, the s-polarized transmitted light L2 and the p-polarized transmitted light L2 are emitted from the same location on the back surface of the birefringent plate 115. Therefore, in this case, the birefringent plate 115 makes the light obtained by synthesizing the s-polarized light and the p-polarized light transmitted through the liquid crystal layer 113 with each other.

(When V = V3 (FIG. 6C))
The action of the birefringent plate 111 on the incident light L1 is the same as described above. Therefore, the operation of the liquid crystal layer 113 and the birefringent plate 115 will be described below. When the p-polarized light separated by the birefringent plate 111 enters the liquid crystal layer 113 having a polarization conversion efficiency of T3 (= 50%), the p-polarized light is converted into circularly polarized light and travels straight through the liquid crystal layer 113. Then, the light is emitted from the back surface of the liquid crystal layer 113. When the s-polarized light separated by the birefringent plate 111 is incident on the liquid crystal layer 113 whose polarization conversion efficiency is T3 (= 50%), the s-polarized light is also converted into circularly polarized light and travels straight through the liquid crystal layer 113. Then, the light is emitted from the back surface of the liquid crystal layer 113. Therefore, the liquid crystal layer 113 converts the p-polarized light and the s-polarized light separated by the birefringent plate 111 into circularly polarized light while keeping the separation width constant.

When the circularly polarized light emitted from the liquid crystal layer 113 enters the birefringent plate 115, it is separated into p-polarized light and s-polarized light with a separation width d2 due to the birefringence of the birefringent plate 115. When the polarization component that vibrates perpendicularly to the optical axis AX2 of the birefringent plate 115 is s-polarized light, the s-polarized light travels straight in the birefringent plate 115 without being affected by birefringence, and the birefringent plate 115. The light is emitted from the back surface. Since the p-polarized light vibrates in a direction orthogonal to the vibration direction of the s-polarized light, the birefringent plate 115 is influenced by the birefringence and proceeds obliquely in a direction opposite to the image separation direction on the birefringent plate 111. . Further, the p-polarized light is refracted at a position shifted by the separation width d 2 among the back surface of the birefringent plate 115 and is emitted from the back surface of the birefringent plate 115. Accordingly, the birefringent plate 115 converts the circularly polarized light converted from the p-polarized light by the liquid crystal layer 113 and the circularly polarized light converted from the s-polarized light by the liquid crystal layer 113, respectively, into the s-polarized transmitted light L2 with a separation width d2. And p-polarized transmitted light L2.

Here, when d1 = d2, the p-polarized light separated from the circularly polarized light converted from the p-polarized light in the liquid crystal layer 113 and the s-polarized light separated from the circularly polarized light converted from the s-polarized light in the liquid crystal layer 113 are Out of the back surface of the birefringent plate 115, the light is emitted from the same location. In this case, circularly polarized transmitted light L 2 is emitted from the back surface of the birefringent plate 115. Therefore, in this case, the birefringent plate 115 separates the two circularly polarized lights emitted from the liquid crystal layer 113 into p-polarized transmitted light L2 and s-polarized transmitted light L2 with a separation width (d2 + d2). At the same time, the p-polarized light and the s-polarized light that have been once separated are combined with the p-polarized light and the s-polarized light at a position between the p-polarized transmitted light L2 and the s-polarized transmitted light L2.

Next, the point image intensity distribution of the transmitted light of the variable optical LPF 11 will be described. When the voltage V 2 is applied between the

electrodes

112 and 114, the variable optical LPF 11 generates one peak p 1 in the point image intensity distribution of the transmitted light of the variable optical LPF 11. The peak p1 is formed by one transmitted light L2 emitted from the birefringent plate 115. The variable optical LPF 11 causes two peaks p2 and p3 in the point image intensity distribution of the transmitted light of the variable optical LPF 11 when the voltage V1 is applied between the

electrodes

112 and 114. The two peaks p2 and p3 are formed by the two transmitted lights L2 emitted from the birefringent plate 115.

The variable optical LPF 11 generates three peaks p1, p2, and p3 in the point image intensity distribution of the transmitted light of the variable optical LPF 11 when the voltage V3 is applied between the

electrodes

112 and 114 and d1 = d2. . The three peaks p1, p2, and p3 are formed by the three transmitted lights L2 emitted from the birefringent plate 115. The variable optical LPF 11 has four peaks p1, p2, p3, and p4 in the point image intensity distribution of the transmitted light of the variable optical LPF 11 when the voltage V3 is applied between the

electrodes

112 and 114 and d1 ≠ d2. Cause it to occur. The four peaks p1, p2, p3, and p4 are formed by the four transmitted lights L2 emitted from the birefringent plate 115.

As described above, the variable optical LPF 11 has three peaks p1 to p3 or four peaks p1 in the point image intensity distribution of the transmitted light of the variable optical LPF 11 when the voltage V3 is applied between the

electrodes

112 and 114. Produces ~ p4. Here, in the variable optical LPF 11, when the voltage V3 applied between the

electrodes

112 and 114 changes, the values of the three peaks p1 to p3 or the four peaks p1 to p4 change. That is, in the variable optical LPF 11, when the voltage V3 applied between the

electrodes

112 and 114 changes, the point image intensity distribution of the transmitted light changes.

Thus, the variable optical LPF 11 changes the point image intensity distribution of the transmitted light by changing the magnitude of the voltage V applied between the

electrodes

112 and 114. Here, the peak value (peak height) of the three peaks p1 to p3 and the peak value (peak height) of the four peaks p1 to p4 are the magnitude of the voltage V applied between the

electrodes

112 and 114. It depends on. On the other hand, the peak positions of the three peaks p1 to p3 and the peak positions of the four peaks p1 to p4 are determined by the separation widths d1 and d2. The separation widths d1 and d2 are constant regardless of the magnitude of the voltage V3 applied between the

electrodes

112 and 114. Therefore, the peak positions of the three peaks p1 to p3 and the peak positions of the four peaks p1 to p4 are constant regardless of the magnitude of the voltage V3 applied between the

electrodes

112 and 114.

Next, the relationship between the point image intensity distribution of the transmitted light and the cutoff frequency fc will be described. FIG. 7 shows an example of the MTF (Modulation Transfer Function) of FIGS. 6A to 6C. The horizontal axis is the spatial frequency, and the vertical axis is the normalized contrast. In FIG. 6B, since the variable optical LPF 11 does not have a light beam separation effect, the MTF in FIG. 6B matches the MTF of a lens (for example, the lens 13 or the like) disposed in front of the variable optical LPF 11. In FIG. 6A, the distance between peaks is wider than the distance between peaks in FIG. 6C, and the light beam separation effect is the largest. For this reason, the cutoff frequency fc1 of the MTF in FIG. 6A is smaller than the cutoff frequency fc2 of the MTF in FIG. 6C.

In FIG. 6C, the separation width is equal to the separation width in FIG. 6A, but the number of peaks is larger than the number of peaks in FIG. 6A, and the distance between peaks is the distance between peaks in FIG. 6A. It is narrower than. Therefore, in FIG. 6C, the light beam separation effect is weaker than the light beam separation effect in FIG. 6A, so that the MTF cutoff frequency fc2 in FIG. 6C is larger than the MTF cutoff frequency fc1 in FIG. 6A. . The cut-off frequency fc2 of the MTF in FIG. 6C varies depending on the magnitude of the voltage V3 applied between the

electrodes

112 and 114, and can take any frequency greater than the cut-off frequency fc1 of the MTF in FIG. 6A. Therefore, the variable optical LPF 11 changes the magnitude of the voltage V applied between the

electrodes

112 and 114 to change the cutoff frequency fc to an arbitrary value equal to or higher than the cutoff frequency when the light beam separation effect is maximized. Can be set.

(Detailed description of the image processing unit 50 and the memory unit 70)
Next, the image processing unit 50 and the memory unit 70 will be described in detail.

As described above, the memory unit 70 stores the image data 71, the saturation data 72, the evaluation data 73, and the set value 74.

The image data 71 is a plurality of image data obtained by the imaging device 40, and includes, for example, image data I ₁ described later, a plurality of image data I ₂ described later, and image data I described later. The image data I ₁ is image data having no false color or image data having few false colors. Image data having no false color or image data having few false colors is obtained, for example, when the effect of the variable optical LPF 11 is maximized or almost maximized. That is, image data having no false color or image data having less false color is obtained, for example, when the cut-off frequency fc of the variable optical LPF 11 is minimized or substantially minimized. The plurality of image data I ₂ is image data obtained when a setting value different from the setting value used to obtain the image data I ₁ is set in the variable optical LPF 11. The image data I is image data acquired by the image sensor 40 when an appropriate setting value (setting value 74) is set for the variable optical LPF 11. The setting value 74 is a setting value of the variable optical LPF 11 that matches the user's purpose, and is a setting value that is set for the variable optical LPF 11 when obtaining the image data I. The set value 74 is obtained by executing image processing in the image processing unit 50.

The saturation data 72 is data relating to saturation obtained from the image data I ₁ . The saturation data 72 is, for example, two-dimensional data associated with each predetermined unit dot of the image data I ₁ . The evaluation data 73 is data for evaluating the resolution and the false color.

The resolution is an index indicating how fine details can be identified. “High resolution” means that a captured image has a fineness that allows a finer one to be identified. “Resolution is low” means that the photographed image has no fineness and is blurred. “Resolution is degraded” means that the photographed image has no original fineness and is more blurred than the original photographed image. When a captured image including a high spatial frequency is passed through a low-pass filter, the high-frequency component of the captured image is reduced, and the resolution of the captured image is reduced (deteriorated). That is, the deterioration in resolution corresponds to a decrease in spatial frequency. By the way, the false color is a phenomenon in which an original color appears in an image. This phenomenon occurs due to aliasing (folding of high-frequency components into a low-frequency region) when a captured image includes a high-frequency component exceeding the Nyquist frequency because each color is spatially sampled. Therefore, for example, high-frequency components exceeding the Nyquist frequency included in the captured image are reduced by the low-pass filter, so that the resolution of the captured image is reduced (deteriorated), while the occurrence of false colors in the captured image is suppressed.

The evaluation data 73 is derived based on the image data I ₁ and the plurality of image data I ₂ . Specifically, the evaluation data 73 is derived based on the image data I ₁ , the plurality of image data I _2, and the saturation data 33. The evaluation data 73 is derived using, for example, the following formula (1).
D = f (I ₁ , I ₂ ) + ΣC · G (I ₁ , I ₂ ) (1)

Here, D is evaluation data for resolution and false color. f (I ₁ , I ₂ ) is a function for deriving evaluation data (evaluation data D 1) regarding a change (degradation) in resolution between the _two image data I ₁ and I ₂ obtained by the image sensor 40. It means that the larger the value of the evaluation data D1, the lower the resolution. The evaluation data D1 corresponds to a specific example of “first evaluation data” of the present disclosure. f (I ₁ , I ₂ ) is a function for deriving the evaluation data D1 based on the spatial frequencies of the _two image data I ₁ and I ₂ . f (I ₁ , I ₂ ) is a function for deriving the evaluation data D1 based on the difference between the frequency spectrum of the image data I _{1 and} the frequency spectrum of the image data I ₂ , for example. For example, f (I ₁ , I ₂ ) extracts the power at the frequency at which the effect of the variable optical LPF 11 is greatest from the difference between the frequency spectrum of the image data I _{1 and} the frequency spectrum of the image data I _2. This is a function for deriving evaluation data D1 based on the extracted power. Evaluation data D1 is derived from the first term on the right side of Equation (1). C is saturation data 72. _{ΣC · G (I 1, I} 2) , the two image data I _1, based on the grayscale data of I _2, the evaluation data (assessment of false color change between two image data I _1, I ₂ This is a function for deriving data D2). A larger value of ΣC · G (I ₁ , I ₂ ) means that a false color is generated in a wider range. The evaluation data D2 corresponds to a specific example of “second evaluation data” of the present disclosure. ΣC · G (I ₁ , I ₂ ) is a function derived as evaluation data D 2 based on, for example, the difference between the gradation data of the image data I _{1 and} the gradation data of the image data I ₂ . ΣC · G (I ₁ , I ₂ ) is, for example, the difference between the gradation data of the image data I _{1 and} the gradation data of the image data I ₂ , and C (saturation data) obtained from the image data I _1. 72) and a function for deriving the evaluation data D2. ΣC · G (I ₁ , I ₂ ) is, for example, the sum of the difference between the gradation data of the image data I _{1 and} the gradation data of the image data I ₂ multiplied by C as the evaluation data D 2 A function to derive. From the second term on the right side of Equation (1), evaluation data D2 in consideration of the saturation of the subject is derived.

FIG. 8 is a graph illustrating evaluation data D1 obtained from the first term on the right side of Equation (1) and evaluation data D2 obtained from the second term on the right side of Equation (1). In the evaluation data D1 obtained from the first term on the right side of Expression (1), as the effect of the variable optical LPF 11 becomes stronger, the resolution deteriorates abruptly at first, and thereafter, the change in resolution gradually becomes gentler. I understand that. In addition, in the evaluation data D2 obtained from the second term on the right side of Expression (1), it can be seen that the false color range gradually decreases as the effect of the variable optical FPF 11 increases.

(Image processing unit 50)
FIG. 9 illustrates an example of functional blocks of the image processing unit 50. The image processing unit 50 performs predetermined processing on the image data output from the image sensor 40. The image processing unit 50, for example, the effect (or the effect of the variable optical LPF 11 on the basis of a change in resolution and a change in false color in a plurality of captured image data in accordance with a change in the effect (or low-pass characteristics) of the variable optical LPF 11 A low-pass characteristic) set value 74 is set. An example of the low-pass characteristic is a cutoff frequency fc. The image processing unit 50 includes, for example, a preprocessing circuit 51, an image processing circuit 52, a display processing circuit 53, a compression / decompression circuit 54, and a memory control circuit 55.

The pre-processing circuit 51 performs optical correction processing such as shading correction on the image data output from the image sensor 40. The image processing circuit 52 performs various processes described later on the corrected image data output from the preprocessing circuit 51. The image processing circuit 52 further outputs, for example, image data acquired from the image sensor 40 to the display processing circuit 53. The image processing circuit 52 further outputs, for example, image data acquired from the image sensor 40 to the compression / decompression circuit 54. The image processing circuit 52 will be described later in detail.

The display processing circuit 53 generates an image signal to be displayed on the display panel 60 from the image data received from the image processing circuit 52, and sends the image signal to the display panel 60. The compression / decompression circuit 54 performs compression encoding processing on the still image data received from the image processing circuit 52 by a still image encoding method such as JPEG (JointJPhotographic Experts Group). The compression / decompression circuit 54 performs compression coding processing on the moving image data received from the image processing circuit 52 by a moving image coding method such as MPEG (Moving Picture Experts など Group). . The memory control circuit 55 controls writing and reading of data with respect to the memory unit 70.

Next, an imaging procedure in the imaging apparatus 1 will be described.

FIG. 10 illustrates an example of an imaging procedure in the imaging apparatus 1. First, the control unit 80 prepares for operation (step S101). The operation preparation refers to preparations required when the image data I is output from the image sensor 40, for example, setting AF (autofocus) conditions and the like. Specifically, when detecting a preparation instruction from the user (for example, half-pressing the shutter button), the control unit 80 instructs the lens driving unit 20 and the LPF driving unit 30 to prepare for operation such as AF. Then, the lens driving unit 20 performs operation preparation for the lens 12 before outputting the image data I _{1 in} accordance with an instruction from the control unit 80. For example, the lens driving unit 20 sets a focus condition of the lens 12 to a predetermined value. At this time, the control unit 80 causes the lens driving unit 20 to prepare for operation such as AF after the variable optical LPF 11 is not optically operated. The LPF driving unit 30 performs operation preparation for the variable optical LPF 11 before outputting the image data I _{1 in} accordance with an instruction from the control unit 80. The LPF driving unit 30 applies a voltage V2 between the

electrodes

112 and 114, for example. At this time, the polarization conversion efficiency T of the variable optical LPF 11 is T1.

Next, the control unit 80 generates a change instruction for changing the effect of the variable optical LPF 11 at a pitch larger than the set minimum resolution of the effect of the variable optical LPF 11, and gives the change instruction to the LPF driving unit 30. That is, the control unit 80 generates a change instruction for changing the cutoff frequency fc of the variable optical LPF 11 at a pitch larger than the set minimum resolution of the cutoff frequency fc of the variable optical LPF 11, and outputs the change instruction to the LPF driving unit 30. To do. Then, for example, the LPF driving unit 30 gradually changes the cutoff frequency fc of the variable optical LPF 11 by changing the voltage V (constant frequency) applied between the electrodes of the variable optical LPF 11 at a pitch larger than the set minimum resolution. Let At this time, the control unit 80 generates an imaging instruction synchronized with a change in the effect (cut-off frequency fc) of the variable optical LPF 11, and gives the imaging instruction to the imaging element 40. Then, the imaging device 40 performs imaging in synchronization with a change in the effect of the variable optical LPF 11 (change in the cutoff frequency fc) in accordance with an instruction from the control unit 80. As a result, the imaging device 40 generates a plurality of imaging data different from each other in the effect (cutoff frequency fc) of the variable optical LPF 11 and outputs the imaging data to the image processing unit 50.

Hereinafter, a more specific method for acquiring a plurality of imaging data will be described.

First, the control unit 80 instructs the LPF driving unit 30 to maximize or substantially maximize the effect of the variable optical LPF 11 within the range in which the effect of the variable optical LPF 11 can be changed (step S102). In other words, the control unit 80 instructs the LPF driving unit 30 so that the cut-off frequency fc of the variable optical LPF 11 is minimized or almost within the changeable range of the cut-off frequency fc of the variable optical LPF 11. Then, the LPF driving unit 30 maximizes or substantially maximizes the effect of the variable optical LPF 11 within the range in which the effect of the variable optical LPF 11 can be changed in accordance with an instruction from the control unit 80. In other words, the LPF driving unit 30 minimizes or substantially minimizes the cutoff frequency fc of the variable optical LPF 11 within a changeable range of the cutoff frequency fc of the variable optical LPF 11. For example, the LPF driving unit 30 applies a voltage V1 between the

electrodes

112 and 114, or a voltage slightly larger than the voltage V1. As a result, the polarization conversion efficiency T of the variable optical LPF 11 becomes T2 (maximum) or a value close to T2.

Subsequently, the control unit 80 instructs the image sensor 40 to acquire the image data I ₁ (step S).
103). Specifically, the control unit 80 acquires the image data I ₁ when the effect of the variable optical LPF 11 is the maximum or almost the maximum within the changeable range of the effect of the variable optical LPF 11. 40. That is, the control unit 80 acquires the image data I ₁ when the cut-off frequency fc of the variable optical LPF 11 is the minimum or almost the minimum within the changeable range of the cut-off frequency fc of the variable optical LPF 11. To the image sensor 40. Then, the image sensor 40 discretely samples the light incident through the variable optical LPF 11 in which the effect of the variable optical LPF 11 is maximized or substantially maximized by the light receiving surface 40A, thereby obtaining color image data I _1. To get. That is, the image pickup device 40 discretely samples the light incident through the variable optical LPF 11 with the cut-off frequency fc of the variable optical LPF 11 being minimum or almost minimum by the light receiving surface 40A, thereby obtaining a color image. Data I ₁ is acquired. For example, the image sensor 40 discretely samples light incident via the variable optical LPF 11 having the maximum or almost maximum polarization conversion efficiency T by the light receiving surface 40A, thereby obtaining the color image data I ₁ . get.

The image data I ₁ is an image generated by the image sensor 40 by driving the image sensor 40 when the effect of the variable optical LPF 11 is maximized or substantially maximized within a range in which the effect of the variable optical LPF 11 can be changed. It is data. That is, the image data I ₁ is obtained by driving the image sensor 40 when the cut-off frequency fc of the variable optical LPF 11 is minimized or almost within the changeable range of the cut-off frequency fc of the variable optical LPF 11. This is image data generated by the image sensor 40. The image data I ₁ corresponds to a specific example of “first image data” of the present disclosure. The image sensor 40 outputs the acquired image data I ₁ to the image processing unit 50. Next, the image processing unit 50 analyzes the acquired image data I ₁ to derive data relating to the saturation (saturation data 33) in the image data I ₁ (step S104).

Next, the control unit 80 instructs the LPF driving unit 30 to change the effect of the variable optical LPF 11 (step S105). Specifically, the control unit 80 instructs the LPF driving unit 30 so that the effect of the variable optical LPF 11 is smaller than the previous effect of the variable optical LPF 11. That is, the control unit 80 instructs the LPF driving unit 30 so that the cutoff frequency fc of the variable optical LPF 11 is higher than the previous cutoff frequency fc of the variable optical LPF 11. Then, the LPF driving unit 30 makes the effect of the variable optical LPF 11 smaller than the previous effect of the variable optical LPF 11 in accordance with an instruction from the control unit 80. That is, the LPF driving unit 30 sets the cutoff frequency fc of the variable optical LPF 11 to be higher than the previous cutoff frequency fc of the variable optical LPF 11. For example, the LPF driving unit 30 applies a voltage higher than the previous voltage as the voltage V 3 between the

electrodes

112 and 114. As a result, the polarization conversion efficiency T of the variable optical LPF 11 is a magnitude between T2 and T1, and is smaller than the previous time.

Subsequently, the control unit 80 instructs the image sensor 40 to acquire the image data I ₂ (step S106). The image data I ₂ corresponds to a specific example of “second image data” of the present disclosure. Specifically, the control unit 80 instructs the image sensor 40 to acquire the image data I ₂ when a setting value different from the setting value used to obtain the image data I ₁ is set in the variable optical LPF 11. . Then, the imaging device 40 performs color sampling by spatially sampling light incident through the variable optical LPF 11 in which the effect of the variable optical LPF 11 is smaller than the previous effect of the variable optical LPF 11 on the light receiving surface 40A. Image data I ₂ is acquired. That is, the image sensor 40 spatially receives light incident through the variable optical LPF 11 in which the cutoff frequency fc of the variable optical LPF 11 is higher than the previous cutoff frequency fc of the variable optical LPF 11 on the light receiving surface 40A. The color image data I ₂ is obtained by sampling the image data. For example, the image pickup device 40 spatially samples light incident through the variable optical LPF 11 having a polarization conversion efficiency T between T2 and T1, and spatially samples the light on the light receiving surface 40A. Data I ₂ is acquired. The image sensor 40 outputs the acquired image data I ₂ to the image processing unit 50.

Next, the control unit 80 instructs the image processing unit 50 to derive an appropriate low-pass characteristic (set value 74). Then, the image processing unit 50 sets an appropriate low-pass characteristic (set value) based on a change in resolution and a change in false color in the plurality of image data (image data I ₁ and image data I ₂ ) obtained by the image sensor 40. 74) is derived.

The image processing unit 50 first derives two evaluation data D1 and D2 based on a plurality of image data (image data I ₁ and image data I ₂ ) (step S107).

The image processing unit 50 derives evaluation data D1 for a change in resolution based on the spatial frequency of the plurality of image data (image data I ₁ and image data I ₂ ). For example, the image processing unit 50 derives the evaluation data D1 based on the difference between the frequency spectrum of the image data I _{1 and} the frequency spectrum of the image data I ₂ . The image processing unit 50 derives the evaluation data D1 by, for example, applying the image data I ₁ and the image data I ₂ to the first term on the right side of Expression (1).

The image processing unit 50 derives evaluation data D2 for a change in false color based on the gradation data of a plurality of image data (image data I ₁ and image data I ₂ ). For example, the image processing unit 50 derives the evaluation data D2 based on the difference between the gradation data of the image data I _{1 and} the gradation data of the image data I ₂ . The image processing unit 50 evaluates the evaluation data D2 based on, for example, the difference between the gradation data of the image data I _{1 and} the gradation data of the image data I ₂ and the saturation data 33 obtained from the image data I _1. Is derived. The image processing unit 50 derives the evaluation data D2 by, for example, applying the image data I ₁ , the image data I _2, and the saturation data 33 to the second term on the right side of Expression (1).

Next, the image processing unit 50 determines appropriateness of the effect of the variable optical LPF 11 based on the obtained two evaluation data D1 and D2 (step S108). That is, the image processing unit 50 determines whether or not the obtained two evaluation data D1 and D2 satisfy a desired standard according to the purpose. The “desired reference according to the purpose” varies depending on the shooting mode, the subject, the scene, and the like. The image processing unit 50, for example, minimizes the total value (evaluation data D) of the two newly obtained evaluation data D1 and D2 this time in the region where the effect (cutoff frequency fc) of the variable optical LPF 11 can be controlled. It is determined whether it becomes a value.

When the image processing unit 50 determines that the effect of the variable optical LPF 11 is appropriate, the low-pass characteristic of the variable optical LPF 11 at that time is set as the set value 74. Specifically, when the image processing unit 50 determines that the effect of the variable optical LPF 11 is appropriate, the setting value of the variable optical LPF 11 at that time is set as the setting value 74. The image processing unit 50 stores the setting value 74 in the memory unit 70. For example, when the image processing unit 50 determines that the total value (evaluation data D) of the two evaluation data D1 and D2 newly obtained this time is the minimum value in a region where the effect of the variable optical LPF 11 can be controlled. The setting value of the variable optical LPF 11 at that time is set as a setting value 74.

When it is determined that the effect of the variable optical LPF 11 is not appropriate, the image processing unit 50 returns to step S105, changes the effect (cutoff frequency fc) of the variable optical LPF 11, and sequentially performs steps after step S106. ,carry out. In this way, the image processing unit 50 derives the set value 74 of the variable optical LPF 11 based on the derived two evaluation data D1 and D2 by repeatedly performing Steps S105 to S108. That is, the image processing unit 50 has a plurality of pieces of image data (image data I) captured with different low-pass characteristics (cut-off frequency fc) in accordance with a preparation instruction from the user (for example, half-pressing the shutter button). The set value 74 of the variable optical LPF 11 is derived based on _one and a plurality of image data I ₂ ). The image processing unit 50 notifies the control unit 80 that the setting value 74 has been obtained.

The control unit 80 sets the variable optical LPF 11 when detecting an imaging instruction (for example, pressing the shutter button) from the user after the notification that the setting value 74 is obtained is input from the image processing unit 50. The LPF driving unit 30 is instructed to set the value 74. Specifically, when detecting an imaging instruction (for example, pressing of a shutter button) from the user, the control unit 80 reads the setting value 74 from the memory unit 70 and outputs the read setting value 74 to the LPF driving unit 30. At the same time, the LPF driving unit 30 is instructed to set the variable optical LPF 11 to the set value 74. Then, the LPF drive unit 30 sets the variable optical LPF 11 to the set value 74 in accordance with an instruction from the control unit 80.

The control unit 80 further instructs the image sensor 40 to acquire the image data I. Then, the image sensor 40 acquires the image data I in accordance with an instruction from the control unit 80 (step S109). That is, the imaging device 40 acquires color image data I by discretely sampling the light incident through the variable optical LPF 11 having the set value 74 set by the light receiving surface 40A. The image sensor 40 outputs the acquired image data I to the image processing unit 50. The image processing unit 50 performs predetermined processing on the image data I, and then outputs the processed image data I to the memory unit 70 and also to the display panel 60. Then, the memory unit 70 stores the image data I input from the image processing unit 50, and the display unit 60 displays the image data I input from the image processing unit 50 (step S110).

In the imaging apparatus 1, the above-described operation preparation may be performed manually by a user. In the imaging apparatus 1, when the focus condition of the lens 13 is changed while the image data I ₂ is continuously acquired while the effect of the variable optical LPF 11 is changed, the image data I Redo may be performed from acquisition of ₁ . The instruction from the user may be performed by a method other than “half-pressing the shutter button”. For example, the above-described instruction from the user may be performed by pressing a button (an operation button other than the shutter button) attached to the main body of the imaging apparatus 1 after the focus condition of the lens 13 is set. In addition, for example, the above-described instruction from the user may be performed by turning an operation dial attached to the main body of the imaging apparatus 1 after the focus condition of the lens 13 is set.

By the way, as described above, the “desired reference according to the purpose” varies depending on the shooting mode, the subject, the scene, and the like. Thus, “desired criteria according to purpose” will be exemplified below.

(If you want to reduce resolution degradation without a large increase in false color)
For example, as shown in FIG. 8, when there is a range (range α) in which the change of the evaluation data D2 relative to the effect of the variable optical LPF 11 is relatively gentle (range α), the control unit 80 includes the range α within the range α. The effect of the variable optical LPF 11 when the evaluation data D2 is the smallest (white circle in FIG. 8) may be set as the set value 74. That is, when there is a range (range α) in which the change of the evaluation data D2 relative to the cutoff frequency fc of the variable optical LPF 11 is relatively gentle (range α), the control unit 80 determines that the evaluation data D2 is within the range α. The cut-off frequency fc of the variable optical LPF 11 when it becomes the smallest (white circle in FIG. 8) may be set as the set value 74. In such a case, resolution degradation can be reduced without a large increase in false color.

(If you want to increase the processing speed)
In the above embodiment, the second term (evaluation data D2) on the right side of the equation (1) may be a stepped profile as shown in FIG. 11, for example, depending on the characteristics of the subject and the lens 13. FIG. 11 shows an example of the evaluation data D1 and D2. In this case, for example, as illustrated in FIG. 11, the control unit 80 divides the controllable region of the effect (cutoff frequency fc) of the variable optical LPF 11 into a plurality of regions R, and the effect of the variable optical LPF 11 ( The LPF driving unit 30 may be instructed to sequentially change the cutoff frequency fc) to a value set for each divided region R. In this case, the LPF driving unit 30 sequentially sets the effect (cutoff frequency fc) of the variable optical LPF 11 to a value set for each region R in accordance with an instruction from the control unit 80. Here, each region R has a width larger than the set minimum resolution of the effect (cutoff frequency fc) of the variable optical LPF 11. Accordingly, the LPF driving unit 30 sequentially changes the effect (cutoff frequency fc) of the variable optical LPF 11 at a pitch larger than the set minimum resolution of the effect (cutoff frequency fc) of the variable optical LPF 11.

The control unit 80 further instructs the imaging device 40 to perform imaging in synchronization with the setting of the effect (cutoff frequency fc) of the variable optical LPF 11. Then, the imaging element 40 acquires image data I ₁ and a plurality of image data I _2, the acquired image data I ₁ and a plurality of image data I _2, and outputs to the image processing unit 50. The image processing unit 50 derives two evaluation data D1 and D2 based on the input image data I ₁ and the plurality of image data I ₂ .

The image processing unit 50 further starts to increase the false color of the setting value of the variable optical LPF 11 in the region R (for example, the region R1) where the total value (evaluation data D) of the two evaluation data D1 and D2 is minimum. A value (for example, k in FIG. 11) may be set. In this case, the number of times of actually changing the effect (cutoff frequency fc) of the variable optical LPF 11 can be reduced as compared with the above embodiment. As a result, the processing speed is increased.

(If the person whose face is detected has hair)
When a subject to be photographed has a hair, a false color may occur in the hair. Since the false color generated in the subject is conspicuous, it is preferable to make such a false color as small as possible. In this case, the image processing unit 50 may set the setting value 74 based on a change in false color in the face area included in the plurality of image data (image data I ₁ and image data I ₂ ). .

FIG. 12 illustrates an example of an imaging procedure when a person whose face is detected has hair. The image processing unit 50 may perform face detection on the image data I ₁ when acquiring the image data I ₁ from the image sensor 40 (FIG. 12, step S111). Further, when the image processing unit 50 detects a face as a result of performing face detection on the image data I ₁ in step S111, it may be detected whether the person whose face is detected has hair. (FIG. 12, step S112).

As a result, if the person whose face is detected has hair, the image processing unit 50 may perform threshold processing on the evaluation data D2, for example. For example, the image processing unit 50 may determine, as the setting value 74, a setting value that weakens the effect of the variable optical LPF 11 in a range where the evaluation data D2 falls below a predetermined threshold Th1 (FIG. 13). FIG. 13 shows an example of the evaluation data D1 and D2 together with various threshold values. At this time, the predetermined threshold value Th1 is a threshold value for face detection, and is a value suitable for excluding false colors generated in the hair of a person who is a subject, for example. For example, the image processing unit 50 may perform threshold processing not only on the evaluation data D2 but also on the evaluation data D1. For example, in the range where the evaluation data D2 is lower than the predetermined threshold Th1 and the evaluation data D1 is lower than the predetermined threshold Th2, the control unit 80 sets a setting value that makes the effect of the variable optical LPF 11 the weakest. The setting value 74 may be determined (FIG. 13).

If the person whose face is detected has no hair, the image processing unit 50 may determine the set value 74 by the same method as described above. When the person whose face is detected has hair, the image processing unit 50 may determine the setting value 74 in this way.

(In macro shooting mode)
FIG. 14 illustrates an example of an imaging procedure in the macro imaging mode. When the photographing mode is the macro photographing mode, the user often tries to photograph an object close to it. At this time, in particular, when the user intends to shoot a product, it is often not allowed to generate a false color for the product to be photographed. In such a case, it is preferable for the user himself / herself to determine the set value 74 after actually confirming the presence or absence of a false color. Specifically, the image processing unit 50 sets the effect (cut-off frequency fc) of the variable optical LPF 11 when one image data I ₂ selected by the user from the plurality of image data I ₂ is captured. It is preferable to set to 74.

When the image pickup device 40 is in the macro shooting mode, when an image pickup instruction (for example, pressing of the shutter button) from the user is detected after step S101 ends, the image processing unit 50 Process. After step S102 to step S107 are sequentially performed, the image processing unit 50 determines in step S108 whether, for example, the evaluation data D2 is below a predetermined threshold Th3. At this time, the predetermined threshold value Th3 is a threshold value for the macro shooting mode, and is a value suitable for excluding false colors generated in an object that is a subject in the macro shooting mode, for example.

As a result, when the evaluation data D2 falls below the predetermined threshold Th3 (FIG. 13), the image processing unit 50 acquires the set value of the variable optical LPF 11 corresponding to the evaluation data D2 at that time as the appropriate value candidate 35a. Further, the image data I ₂ corresponding to the appropriate value candidate 35a is stored in the memory unit 70 together with the appropriate value candidate 35a (FIG. 14, step S113). If the evaluation data regarding the change in the false color does not fall below the predetermined threshold Th1, the image processing unit 50 returns to step S105.

Note that the image processing unit 50 may perform threshold processing not only on the evaluation data D2 but also on the evaluation data D1. For example, the image processing unit 50 may determine whether the evaluation data D2 is below a predetermined threshold Th3 and whether the evaluation data D1 is below a predetermined threshold Th4 (FIG. 13). As a result, when the evaluation data D2 is lower than the predetermined threshold Th3 and the evaluation data D1 is lower than the predetermined threshold Th4, the image processing unit 50 sets the set value of the variable optical LPF 11 corresponding to the evaluation data D2. You may acquire as the appropriate value candidate 35a. When the evaluation data D2 does not fall below the predetermined threshold Th3, or when the evaluation data D1 does not fall below the predetermined threshold Th4, the image processing unit 50 may return to step S105.

After performing step S113, the image processing unit 50 determines whether or not the effect of the variable optical LPF 11 has been changed (FIG. 14, step S114). The image processing unit 50 repeatedly executes steps S105 to S108 and S113 until the change of the effect of the variable optical LPF 11 is completed. That is, the image processing unit 50 acquires a value corresponding to the evaluation data D2 at that time as the appropriate value candidate 35a every time the evaluation data regarding the change in false color falls below the predetermined threshold Th3. As a result, when the change of the effect of the variable optical LPF 11 is completed, the image processing unit 50 requests the user to select one image data I ₂ from the plurality of image data I ₂ corresponding to each appropriate value candidate 35a. .

Specifically, the image processing unit 50 outputs the image data I ₂ of one from among the plurality of image data I ₂ corresponding to each proper value candidate 35a on the display unit 60. Then, the display unit 60 displays the image data I ₂ input from the image processing unit 50 (FIG. 14, step S115). The image processing unit 50 sequentially outputs a plurality of image data I ₂ corresponding to each appropriate value candidate 35 a to the display unit 60. The image processing unit 50 displays a display every time a display instruction from the user (for example, depression of an operation button attached to the main body of the imaging apparatus 1 or rotation of an operation dial attached to the main body of the imaging apparatus 1) is detected. image data I ₂ to be displayed on the section 60 is so switched to another image data I _2, and outputs a plurality of image data I ₂ on the display unit 60. The display unit 60 replaces the image data I ₂ to be displayed every time the image data I ₂ is input from the image processing unit 50.

At this time, for example, the image processing unit 50 may output a numerical value (for example, a pixel-converted area of the false color generation region) and a histogram together with the image data I ₂ to the display unit 60. Then, the display unit 60 displays numerical values and histograms input from the image processing unit 50. For example, as shown in FIG. 15, the histogram has saturation (for example, Cb, Cr on the YCbCr space) on the vertical and horizontal axes, and each region is shaded according to the number of pixels in which a false color is generated. It may be something that expresses.

When a selection instruction from the user (for example, pressing of another operation button attached to the main body of the imaging apparatus 1) is detected, the image processing unit 50 displays the selection on the display unit 60 when the detection is detected. The appropriate value candidate 35a corresponding to the image data I ₂ is determined as the set value 74. In this way, the image processing unit 50 causes the user to select appropriate image data I ₂ (that is, an appropriate value as the variable optical LPF 11) (FIG. 14, step S116).

When the shooting mode is the macro shooting mode, the image processing unit 50 sets, for example, a setting value (cut-off value) that weakens the effect of the variable optical LPF 11 in a range where the evaluation data D2 falls below a predetermined threshold Th3. A setting value at which the frequency fc is maximized) may be determined as the setting value 74. For example, the image processing unit 50 may perform threshold processing not only on the evaluation data D2 but also on the evaluation data D1. For example, the image processing unit 50 has a setting value that weakens the effect of the variable optical LPF 11 in a range where the evaluation data D2 is below the predetermined threshold Th3 and the evaluation data D1 is below the predetermined threshold Th4. The set value (the set value at which the cut-off frequency fc is maximized) may be determined as the set value 74.

(When processing according to the false color occurrence position is required)
When there is a mode for performing processing according to the position where the false color is generated as the shooting mode (referred to as “false color processing mode” for convenience), the image processing unit 50 uses the false color processing mode. In step S101, the false color generation position may be detected after step S101 is performed. If a false color has occurred in a fairly large area of the entire screen, or if a false color has occurred in the center of the screen, if the false color remains, the remaining false color may stand out. It is thought that there is. Also, if there are false colors at the four corners of the screen, it may be possible to unnecessarily degrade the resolution by completely erasing them. In such a case, it is preferable that the image processing unit 50 adjusts the effect of the variable optical LPF 11 according to the generation position of the false color. Specifically, the image processing unit 50, based resolution and change and false color change in a plurality of image data I _1, I _3, in the false color location included in the plurality of image data I _1, I ₃ It is preferable to set the set value 74.

FIG. 16 shows an example of an imaging procedure in the false color processing mode. First, the image processing unit 50 detects a false color occurrence position after steps S101 to S104 are performed. Specifically, first, the control unit 80 minimizes or substantially minimizes the effect of the variable optical LPF 11 (step S117). For example, the control unit 80 instructs the LPF driving unit 30 so that the effect of the variable optical LPF 11 is minimized or substantially minimized (so that the cutoff frequency fc is maximized or substantially maximized). Then, the LPF driving unit 30 minimizes or substantially minimizes the effect of the variable optical LPF 11 in accordance with an instruction from the control unit 80. That is, the LPF drive unit 30 maximizes or substantially maximizes the cutoff frequency fc in accordance with an instruction from the control unit 80. For example, the LPF driving unit 30 applies a voltage V2 or a voltage slightly smaller than the voltage V2 between the

electrodes

112 and 114. As a result, the polarization conversion efficiency T of the variable optical LPF 11 is T1 (minimum) or almost T1.

Subsequently, the image processing unit 50 acquires the image data I ₃ when the effect of the variable optical LPF 11 is minimized or substantially minimized (step S118). Specifically, the control unit 80 instructs the imaging device 40 to perform imaging when the effect of the variable optical LPF 11 is minimized or almost minimized. That is, the control unit 80 instructs the imaging device 40 to perform imaging when the cutoff frequency fc is maximum or almost maximum. Then, the image pickup device 40 discretely samples the light incident through the variable optical LPF 11 in which the effect of the variable optical LPF 11 is minimized or substantially minimized by the light receiving surface 40A, so that the color image data I _{3 is obtained.} To get. That is, the image sensor 40 discretely samples the light incident via the variable optical LPF 11 having the maximum or almost maximum cutoff frequency fc by the light receiving surface 40A, thereby obtaining the color image data I ₃ . get. For example, the image sensor 40 discretely samples the light incident through the variable optical LPF 11 having the polarization conversion efficiency T that is minimum or substantially minimum at the light receiving surface 40A, thereby obtaining the color image data I ₃ . get. At this time, there is a high possibility that the false color remaining in the image data I ₃ without being removed by the variable optical LPF 11 exists. The image sensor 40 outputs the acquired image data I ₃ to the image processing unit 50. As a result, the image processing unit 50 acquires the image data I ₃ .

Next, the image processing unit 50 detects the position of the false color included in the image data I ₃ (step S119). Specifically, the image processing unit 50 detects the position of a false color included in the image data I ₃ using the image data I ₁ , the image data I _3, and the saturation data 33. The image processing unit 50 calculates, for example, the difference between the gradations of the two image data I ₁ and I ₃ for each dot, and multiplies the difference image obtained thereby by C, thereby obtaining a false color determination image. Is generated. Next, the image processing unit 50 determines whether or not each dot value of the false color determination image exceeds a predetermined threshold Th5. As a result, when any value of each dot of the false color determination image exceeds the predetermined threshold Th5, the image processing unit 50 acquires and acquires the coordinates of the dot exceeding the predetermined threshold Th5. The coordinates are stored in the memory unit 70 as a false color generation position (false color generation position 35) included in the image data I ₃ (FIG. 17). FIG. 17 shows an example of data stored in the memory unit 70. The image processing unit 50 may use the image data I ₂ instead of the image data I ₃ . Specifically, the image processing unit 50 may detect a false color position in the image data I ₂ . The image processing unit 50 may detect a false color position in the image data I ₂ and I ₃ .

Next, when steps S105 to S108 are performed, the image processing unit 50 determines that the evaluation data D2 obtained in step S108 has a desired reference corresponding to the false color generation position (false color generation position 35). It is determined whether it is satisfied.

When the false color generation position 35 is the screen center or a large area including the screen center, the image processing unit 50, for example, with respect to the evaluation data D2, the false color generation position 35 is the screen center. A certain correction coefficient α may be multiplied (FIG. 18). When the false color generation position 35 is the four corners of the screen, for example, the image processing unit 50 may multiply the evaluation data D2 by the correction coefficient β when the false color generation position 35 is the four corners of the screen. Good (FIG. 18). For example, the correction coefficient β is smaller than the correction coefficient α. FIG. 18 shows an example of the false color occurrence position to which the correction coefficients α and β are applied. In FIG. 18, the correction coefficients α and β are represented in correspondence with the false color occurrence position.

When the false color generation position 35 is the center of the screen or is a large area including the screen center, the image processing unit 50 determines that the evaluation data D2 is, for example, the false color generation position 35 is the center of the screen. It may be determined whether or not the threshold value Th6 is below a certain threshold (FIG. 19). When the false color generation position 35 is at the four corners of the screen, the evaluation data D2 indicates whether, for example, the evaluation data D2 is equal to or less than a threshold Th7 when the false color generation position 35 is at the four corners of the screen. You may determine (FIG. 19). The threshold value Th6 is, for example, a value smaller than the threshold value Th7. FIG. 19 illustrates an example of the false color occurrence position to which the thresholds Th6 and Th7 are applied. In FIG. 19, threshold values Th6 and Th7 are shown corresponding to the occurrence positions of false colors.

When the evaluation data D2 satisfies a desired standard corresponding to the false color generation position 35, the image processing unit 50 sets the value corresponding to the evaluation data D2 at that time as the set value 74 and stores it in the memory unit 70. To do.

Note that the image processing unit 50 may detect a position where resolution degradation occurs in the image data I ₃ instead of the false color or together with the false color. At this time, the image processing unit 50 may use the image data I ₂ instead of the image data I ₃ . Specifically, the image processing unit 50 may detect a position where resolution degradation occurs in the image data I ₂ instead of the false color or together with the false color. The image processing unit 50 may detect a position where resolution degradation occurs in the image data I ₂ and I ₃ instead of the false color or together with the false color. In step S 108, the image processing unit 50 determines whether or not the evaluation data D 1 satisfies a desired criterion according to the position where resolution degradation occurs (resolution degradation position).

When the resolution degradation position is the screen center or a large area including the screen center, the image processing unit 50 corrects, for example, the evaluation data D1 when the resolution degradation position is the screen center. The coefficient γ may be multiplied. When the resolution degradation position is at the four corners of the screen, the image processing unit 50 may multiply the evaluation data D1 by the correction coefficient δ when the resolution degradation position is at the four corners of the screen, for example. For example, the correction coefficient δ is smaller than the correction coefficient γ.

When the resolution deterioration position is the center of the screen or a large area including the screen center, the image processing unit 50 uses, for example, a threshold when the evaluation data D1 is the resolution deterioration position of the screen center. You may determine whether it is Th8 or less. When the resolution degradation position is at the four corners of the screen, for example, the image processing unit 50 determines whether or not the evaluation data D1 is equal to or less than the threshold Th9 when the resolution degradation position is at the four corners of the screen. Also good. The threshold value Th8 is smaller than the threshold value Th9, for example.

When the evaluation data D1 satisfies a desired standard according to the resolution degradation position, the control unit 80 sets the value corresponding to the evaluation data D1 at that time as the set value 74 and stores it in the memory unit 70.

[effect]
Next, the effect of the imaging device 1 will be described.

Therefore, it is conceivable to provide a low-pass filter between the lens and the imager. In this case, the false color that is one of the artifacts can be reduced by adjusting the effect of the low-pass filter. However, the resolution further decreases in exchange for the reduction of false colors. Although the effect of the low-pass filter can be adjusted so that the resolution does not decrease excessively, in such a case, the effect of reducing false colors is reduced. Thus, in the low-pass filter, there is a trade-off relationship between an increase in resolution and a reduction in false color. For this reason, when a low pass filter is set by focusing on only one of the resolution and the false color, an image with a strong effect of the low pass filter is selected, and the resolution deteriorates more than necessary. There was a problem.

On the other hand, in the present embodiment, evaluation data D1 and evaluation data D2 are derived based on a plurality of image data (image data I ₁ and a plurality of image data I ₂ ) acquired while changing the effect of the variable optical LPF 11. . Further, the set value 74 of the variable optical LPF 11 is derived based on the derived evaluation data D1 and evaluation data D2. Thereby, it is possible to obtain the image data I in which the image quality is balanced according to the purpose of the user.

In the present embodiment, the evaluation data D1 is generated based on the difference between the frequency spectra of the two image data I ₁ and I ₂ , and the difference between the gradations of the two image data I ₁ and I ₂ is calculated. When the evaluation data D2 is generated based on the image data I, it is possible to obtain the image data I in which the image quality is balanced with higher accuracy.

In the present embodiment, when the value when the total value (evaluation data D) of the evaluation data D1 and the evaluation data D2 is the smallest is set to the set value 74, the image quality can be balanced by a simple method. Image data I can be obtained.

Further, in the present embodiment, for example, as shown in FIG. 8, when there is a range α in which the change of the evaluation data D2 with respect to the effect (cutoff frequency fc) of the variable optical LPF 11 is relatively gentle, When the value at which the effect (cutoff frequency fc) of the variable optical LPF 11 is the lowest in the range α is set to the set value 74, a simple method can be used without causing a large change in false color. The resolution can be reduced.

Further, in the present embodiment, for example, as shown in FIG. 11, an area where the effect of the variable optical LPF 11 can be controlled is divided into a plurality of areas R, and the evaluation data D1 and the evaluation are obtained for each divided area R. When the data D2 is derived, the number of times of actually changing the effect of the variable optical LPF 11 can be reduced as compared with the above embodiment. As a result, it is possible to obtain image data I with a balanced image quality while increasing the processing speed.

In the present embodiment, when face detection is performed on the image data I _{1 and} whether or not the face detected person has hair is detected, the evaluation data D2 is a threshold for face detection. In the range below Th1, a value that makes the effect of the variable optical LPF 11 the weakest is set as the set value 74 of the variable optical LPF 11. Thereby, the image data I with few false colors of the hair of the subject can be obtained. Further, when the value that weakens the effect of the variable optical LPF 11 is set as the set value 74 of the variable optical LPF 11 in a range where the evaluation data D1 is less than the predetermined threshold Th2, the image data I with balanced image quality is set. Can be obtained.

In the present embodiment, when the shooting mode is the macro shooting mode, each time the evaluation data D2 falls below the threshold value Th3 for the macro shooting mode, the value corresponding to the evaluation data D2 at that time is an appropriate value. by the candidate 35a, the user can select one image data I ₂ from the plurality of image data I ₂ corresponding to each proper value candidates 35a. As a result, it is possible to obtain image data I with a balanced image quality according to the purpose of the user.

In the present embodiment, by detecting the false color position included in the image data I ₃ obtained when the effect of the variable optical LPF 11 is minimized or almost minimized, the evaluation data D2 is set to the false color position. A value when the corresponding standard is satisfied can be set as the set value 74 of the variable optical LPF 11. Thereby, for example, even if a false color is generated in a considerably large area on the entire screen or a false color is generated in the center of the screen, the image data I in which the false color is not conspicuous is obtained. Obtainable. Further, for example, even when there are false colors at the four corners of the screen, it is possible to obtain image data I in which the resolution is not unnecessarily deteriorated and the false colors are not conspicuous.

In the present embodiment, when the evaluation data D2 is derived based on the difference between the gradation data of the _two image data I ₁ and I ₂ and the saturation data 33 obtained from the image data I ₁ , It is possible to obtain the image data I in which the image quality is balanced with higher accuracy.

In the present embodiment, the image data I ₁ is generated by the image sensor 40 by driving the image sensor 40 when the effect of the variable optical LPF 11 is maximized or substantially maximized. As a result, the evaluation data D1 and the evaluation data D2 are derived based on the image data I ₁ having no false color or almost no false color and the image data I ₂ having the false color. The acquired image data I can be obtained.

In the present embodiment, when the variable optical LPF 11 is set as the shutter button is half-depressed, the variable optical LPF 11 is changed together with an AF (autofocus) condition or an iris 14 condition setting instruction. A setting instruction for the optical LPF 11 can be issued. As a result, it is possible to obtain image data I with a balanced image quality without increasing the work load on the user.

<2. Modification>
Next, a modification of the imaging device 1 according to the above embodiment will be described.

[Modification A]
In the above embodiment, the image processing unit 50 maximizes or substantially maximizes the effect of the variable optical LPF 11 in step S102. However, the image processing unit 50 may set the effect of the variable optical LPF 11 to an arbitrary value in step S102. In this case, the image processing unit 50 performs the process of erasing the false color on the area where the false color may occur in the acquired image data I ₁ , and then derives the saturation data 33. Also good. In such a case, it is possible to shorten the time required to obtain an appropriate set value for the variable optical LPF 11 compared to when the effect of the variable optical LPF 11 is maximized or substantially maximized. is there.

[Modification B]
In the embodiment and the modification thereof, the variable optical LPF 11 is configured to change the cutoff frequency fc by voltage control. However, in the embodiment and the modification thereof, the control unit 80 may change the cutoff frequency fc of the variable optical LPF 11 by frequency control. Further, in the above-described embodiment and its modification, the imaging apparatus 1 uses an optical LPF that changes the cutoff frequency fc in accordance with the magnitude of the physically applied vibration instead of the variable optical LPF 11. Also good. That is, the variable optical LPF according to the present disclosure may be configured so that the cut-off frequency fc can be adjusted by voltage change, frequency change, or vibration amplitude change. Such an optical LPF may be used.

[Modification C]
In the above embodiment, the control unit 80 automatically changes the effect of the variable optical LPF 11. However, the user may manually change the effect of the variable optical LPF 11.

FIG. 20 shows an example of an imaging procedure in this modification. First, the image processing unit 50 determines whether or not the change of the effect of the variable optical LPF 11 is completed after Steps S101 to S107 are performed (Step S120). Steps S105 to S107 are repeatedly executed until the change of the effect of the variable optical LPF 11 is completed. As a result, when the change of the effect of the variable optical LPF 11 is completed, the image processing unit 50 derives an appropriate value candidate k (k1, k2, k3,...) Of the variable optical LPF 11 corresponding to the position indicated by the white circle in FIG. (Step S121). FIG. 21 shows an example of the evaluation data D1 and D2. Specifically, when there are a plurality of ranges in which the change (cutoff frequency fc) of the evaluation data D2 with respect to the effect of the variable optical LPF 11 is relatively gentle, the image processing unit 50 has an end portion for each range. A value corresponding to the position of (the end portion with the smaller effect of the variable optical LPF 11) is set as an appropriate value candidate k (k1, k2, k3,...) Of the variable optical LPF 11. Each appropriate value candidate k (k1, k2, k3,...) Is set to the set value (cut-off frequency fc) of the variable optical LPF 11 when the effect of the variable optical LPF 11 is weakened within the range in which the evaluation data D2 does not increase. This corresponds to the setting value of the variable optical LPF 11 when it is increased).

Next, the image processing unit 50 requests the user to select one image data I ₂ from among a plurality of image data I ₂ corresponding to each appropriate value candidate k (k1, k2, k3,...). To do. Specifically, the image processing unit 50, outputs the image data I ₂ from one of a plurality of image data I ₂ corresponding to the display unit 60 on the appropriate value candidate k (k1, k2, k3, ...) To do. Then, the display unit 60 displays the image data I ₂ input from the control unit 80 (step S122). The image processing unit 50 sequentially outputs a plurality of image data I ₂ corresponding to each appropriate value candidate k to the display unit 60. The image processing unit 50 displays a display every time a display instruction from the user (for example, depression of an operation button attached to the main body of the imaging apparatus 1 or rotation of an operation dial attached to the main body of the imaging apparatus 1) is detected. image data I ₂ (image data I ₂ corresponding to the proper value candidate k) such that replaced with another image data I ₂ corresponding to the proper value candidate k that appears in section 60, displays a plurality of image data I ₂ To the unit 60. The display unit 60 replaces the image data I ₂ to be displayed every time the image data I ₂ is input from the image processing unit 50. When a selection instruction from the user (for example, pressing of another operation button attached to the main body of the imaging apparatus 1) is detected, the image processing unit 50 displays the selection on the display unit 60 when the detection is detected. The appropriate value candidate k corresponding to the image data I ₂ is determined as the set value 74. In this way, the image processing unit 50 causes the user to select appropriate image data I ₂ (or appropriate value 35 appropriate to the user) (step S123).

In this modification, for example, every time the user presses an operation button attached to the main body of the imaging apparatus 1 or every time an operation dial attached to the main body of the imaging apparatus 1 is turned, the image processing unit 50 sets the set value of the variable optical LPF 11. One set value k among a plurality of set values k can be set. Thereby, the trouble of the manual setting by a user can be reduced. Furthermore, it is possible to obtain image data I having a balance between the resolution and the false color as intended by the user.

In this modification, the image processing unit 50 generates, in each image data I ₂ , an emphasis (specifically enlarged) position of a false color included in the plurality of image data I ₂ , and displays it on the display unit 60. It may be output. The “emphasis” refers to distinguishing a target region from other regions (for example, making it more conspicuous than other regions). At this time, for example, when the set value of the variable optical LPF 11 transitions to a value (set value k) corresponding to the white circle in FIG. A false color change (or occurrence) may be detected based on a plurality of image data I ₂ photographed at different frequencies fc). The image processing unit 50 further emphasizes (specifically enlarges) a region (region 61) in which the false color is changed (or generated) due to the transition in each image data I ₂ (see FIG. 22). May be generated and output to the display unit 60. FIG. 22 shows a state in which a part of the image data I ₂ is enlarged. In FIG. 22, an area 61 corresponds to an area displayed at a magnification larger than the magnification of the area other than the area 61 in the image data I ₂ . For example, when the set value of the variable optical LPF 11 transitions to a value corresponding to the white circle in FIG. 21 (set value k) in response to the user's request, the display unit 60 causes the transition in the image data I ₂ . The region (region 61) where the false color has changed (or has occurred) is highlighted (specifically, enlarged display). In this case, even when the display unit 60 is small and it is difficult to visually recognize the false color change, it is possible to intuitively know that the false color has changed and where the false color has occurred. . As a result, the effect of the variable optical LPF 11 can be easily adjusted.

The image processing unit 50 does not need to perform enlargement for all the locations where the false color has changed (or has occurred) in the image data I ₂ . The image processing unit 50, for example, may be output to the display unit 60 to an enlarged portion evaluation data D2 in the image data I ₂ was greatest (region 61).

Further, in this modification, the image processing unit 50, in each image data I _2, to produce what emphasizes the position of the false color included in the plurality of image data I _2, may be output to the display unit 60 . At this time, for example, when the set value of the variable optical LPF 11 transitions to a value (set value k) corresponding to the white circle in FIG. A false color change (or occurrence) may be detected based on a plurality of image data I ₂ photographed at different frequencies fc). The image processing unit 50 further, for example, in each image data I _2, the false color is changed by the transition (or development) were emphasized area (area 62) (specifically, zebra treatment) were those (see FIG. 23 ) May be generated and output to the display unit 60. Zebra processing refers to superimposing a zebra pattern on an image. FIG. 23 shows a state in which a part of the image data I ₂ is zebra processed. FIG. 23 illustrates a state in which the region 62 is distributed from the center to the lower end of the image data I ₂ . For example, when the set value of the variable optical LPF 11 transitions to a value corresponding to the white circle in FIG. 21 (set value k) in response to a user request, the display unit 60 displays a false color in the image data I _2. The changed (or generated) area (area 62) is highlighted (specifically, zebra display). In this case, even when the display unit 60 is small and it is difficult to visually recognize the false color, it is possible to intuitively know that the false color has changed and where the false color has occurred. As a result, the effect of the variable optical LPF 11 can be easily adjusted.

The image processing unit 50 does not need to perform zebra processing on all locations where the false color changes (or occurs) in the image data I ₂ . The image processing unit 50, for example, may be output to the display unit 60 that places the evaluation data D2 in the image data I ₂ was greatest (the region 62) and Zebra process.

Further, in this modification, the image processing unit 50 emphasizes (specifically highlights) the position where the resolution degradation is included in the plurality of image data I ₂ in each image data I ₂ . It may be generated and output to the display unit 60. For example, when the set value of the variable optical LPF 11 transitions to a value corresponding to the white circle in FIG. 21 (set value k) in response to a user request, the image processing unit 50 has a low-pass characteristic (cutoff frequency fc). A change in resolution may be detected based on a plurality of image data I ₂ photographed with different values. The image processing unit 50 further generates, for example, the image data I ₂ that emphasizes (specifically highlights) the edge (part 63) of the area whose resolution has changed due to the transition (see FIG. 24). Then, it may be output to the display unit 60. Highlighting, for example, changes in the brightness of the image data I _2, change the color saturation of the image data I _2, or made by superimposition of the color signal for the image data I _2. Incidentally, FIG. 24 illustrates a state in which part of the image data I ₂ (part 63) are emphasized. FIG. 24 illustrates a state in which the portion 63 is configured by a plurality of line segments. For example, when the setting value of the variable optical LPF 11 changes to a value corresponding to the white circle in FIG. 21 (setting value k) in response to a user request, the display unit 60 changes the resolution of the image data I _2. The edge (part 63) of the region is highlighted (specifically, highlighted). In such a case, even when the display unit 60 is small and it is difficult to visually recognize the resolution degradation, it is possible to intuitively know the location where the resolution degradation has occurred. As a result, the effect of the variable optical LPF 11 can be easily adjusted.

The image processing unit 50 does not need to perform highlighting on all edges of the area where the resolution has changed (or has occurred) in the image data I ₂ . The image processing unit 50, for example, may be output to the display unit 60 that the largest was place by the evaluation data D1 in the image data I ₂ a (section 63) to highlight.

In the present modification, the image processing unit 50 may perform a plurality of types of enhancement on each image data I ₂ . For example, in each image data I ₂ , the image processing unit 50 generates a result of emphasizing the position of the false color included in the plurality of image data I ₂ and the position where the resolution is deteriorated, and outputs it to the display unit 60. May be. For example, when the set value of the variable optical LPF 11 transitions to a value corresponding to the white circle in FIG. 21 (set value k) in response to a user request, the image processing unit 50 has a low-pass characteristic (cutoff frequency fc). _On the basis of a plurality of image data I ₂ photographed with different values, a false color change (or occurrence) and a resolution change may be detected.

For example, in each image data I ₂ , the image processing unit 50 emphasizes (specifically enlarges) the region 61 in which the false color has changed (or has been generated) due to the transition, and also the region whose resolution has changed due to the transition. An edge (part 63) highlighted (specifically, highlighted) (see FIG. 25) may be generated and output to the display unit 60. At this time, the image processing unit 50 may change the display color of the region 61 and the display color of the portion 63 from each other. For example, when the set value of the variable optical LPF 11 transitions to a value corresponding to the white circle in FIG. 21 (set value k) in response to a user request, the display unit 60 displays a false color in the image data I _2. The changed (or generated) area 61 is emphasized (specifically enlarged), and the edge (part 63) of the area where the resolution has changed is highlighted (specifically highlighted). In such a case, even when the display unit 60 is small and it is difficult to visually recognize the resolution degradation, it is possible to intuitively know the location where the resolution degradation has occurred. As a result, the effect of the variable optical LPF 11 can be easily adjusted.

For example, in each image data I ₂ , the image processing unit 50 emphasizes (specifically, zebra processing) the region 62 in which the false color has changed (or has been generated) due to the transition, and the region in which the resolution has changed due to the transition. The edge (part 63) of the image (specifically highlighted) (see FIG. 26) may be generated and output to the display unit 60. For example, when the set value of the variable optical LPF 11 transitions to a value corresponding to the white circle in FIG. 21 (set value k) in response to a user request, the display unit 60 displays a false color in the image data I _2. The changed (or generated) area 62 is emphasized (specifically, zebra processing), and the edge (part 63) of the area where the resolution has changed is highlighted (specifically highlighted). In such a case, even when the display unit 60 is small and it is difficult to visually recognize the resolution degradation, it is possible to intuitively know the location where the resolution degradation has occurred. As a result, the effect of the variable optical LPF 11 can be easily adjusted.

Incidentally, the image processing unit 50, or enlarged area 61 of the image data I _2, the regions 62 of the image data I ₂ to or Zebra process, the image processing unit 50, in advance, false color The occurrence location must be specified. Therefore, for example, as shown in FIG. 25, the image processing unit 50 needs to perform steps S117 to S119 before performing step S105 in the series of steps of FIG. FIG. 25 illustrates an example of an imaging procedure. Since the image processing unit 50 can specify the generation position of the false color by performing steps S117 to S119, it is possible to enlarge the region 61 and perform the zebra processing of the region 62.

[Modification D]
In the above embodiment, step S117 calculates the value (evaluation data D1) obtained from the first term on the right side of equation (1) and the value (evaluation data D2) obtained from the second term on the right side of equation (1). Based on the added value (evaluation data D), the effect (set value) of the variable optical LPF 11 is automatically set. However, the image processing unit 50 may automatically set the effect (setting value) of the variable optical LPF 11 based on one of the evaluation data D1 and the evaluation data D2. In such a case, the effect (set value) of the variable optical LPF 11 can be adjusted according to various purposes.

[Modification E]
In the embodiment and the modification thereof, a low pass filter having a fixed cutoff frequency may be provided instead of the variable optical LPF 11. In the above-described embodiment and its modifications, the variable optical LPF 11 may be omitted. In these cases, the imaging apparatus 1 may include a drive unit that vibrates the light receiving surface 40A of the imaging element 40 in the in-plane direction. At this time, a device including a drive unit that vibrates the imaging element 40 and the light receiving surface 40A functions as a so-called imager shift type low-pass filter. As described above, even when the imager shift type low-pass filter is provided, the image data I having a balanced image quality according to the purpose of the user can be obtained as in the above embodiment.

[Modification F]
In the embodiment and the modification thereof, a low pass filter having a fixed cutoff frequency may be provided instead of the variable optical LPF 11. In the above-described embodiment and its modifications, the variable optical LPF 11 may be omitted. In these cases, the lens driving unit 20 may drive the lens 12 in a plane parallel to a plane orthogonal to the optical axis of the lens 12. At this time, the device including the lens 12 and the lens driving unit 20 functions as a so-called lens shift type low-pass filter. As described above, even when the lens shift type low-pass filter is provided, the image data I in which the image quality is balanced according to the purpose of the user can be obtained as in the above embodiment.

[Modification G]
In the embodiment and the modification thereof, the imaging device 1 can be applied to an in-vehicle camera, a monitoring camera, a medical camera (endoscopic camera), and the like in addition to a normal camera.

As described above, the present disclosure has been described with reference to the embodiment and its modifications. However, the present disclosure is not limited to the above-described embodiment and the like, and various modifications are possible. In addition, the effect described in this specification is an illustration to the last. The effects of the present disclosure are not limited to the effects described in this specification. The present disclosure may have effects other than those described in this specification.

For example, this indication can take the following composition.
(1)
A control apparatus comprising: a control unit configured to set a setting value of a low-pass characteristic of the low-pass filter based on a change in resolution and a change in false color in a plurality of captured image data according to a change in the low-pass characteristic of the low-pass filter.
(2)
The control unit includes a spatial frequency of first image data that is one of the plurality of image data, and a spatial frequency of second image data other than the first image data among the plurality of image data. The control device according to (1), wherein first evaluation data regarding a change in resolution is derived based on the first evaluation data.
(3)
The control unit according to (2), wherein the control unit derives second evaluation data regarding a change in false color based on a difference between the gradation data of the first image data and the gradation data of the second image data. apparatus.
(4)
The control device according to (3), wherein the control unit sets the set value based on the first evaluation data and the second evaluation data.
(5)
When there is a range in which the change of the second evaluation data with respect to the low-pass characteristics of the variable optical low-pass filter is relatively gentle, the control unit has the smallest first evaluation data in the range. The control device according to (3), wherein the low-pass characteristic of the low-pass filter is set to the set value.
(6)
The control device according to any one of (1) to (5), wherein the control unit sets the setting value based on a false color change in a face area included in the plurality of image data.
(7)
The control unit sets the low-pass characteristic of the low-pass filter when the image data selected by the user from among the plurality of image data is imaged to the set value (1) to (5) The control apparatus as described in any one.
(8)
The control unit sets the setting value based on a change in resolution and a change in false color in the plurality of image data and a position of a false color included in the plurality of image data. (1) to (5) The control apparatus as described in any one of these.
(9)
The control unit according to any one of (1) to (5), wherein the control unit generates a plurality of the image data obtained by enlarging the positions of false colors included in the plurality of image data.
(10)
The control unit according to any one of (1) to (5), wherein a plurality of the image data are generated by highlighting the positions of false colors included in the plurality of image data.
(11)
The said control part produces | generates what highlighted the position where the resolution degradation is included among several said image data among several said image data. (1) thru | or (5) Control device.
(12)
The control unit is configured to determine the second evaluation data based on a difference between the gradation data of the first image data and the gradation data of the second image data, and saturation data obtained from the first image data. The control device according to (3).
(13)
The control device according to (3), wherein the first image data is image data when the second evaluation data is minimized or substantially minimized within a changeable range of a low-pass characteristic of the low-pass filter.
(14)
An image sensor that generates image data from light incident via a low-pass filter;
A controller configured to set a setting value of the low-pass characteristic of the low-pass filter based on a change in resolution and a change in false color in a plurality of image data captured in accordance with a change in the low-pass characteristic of the low-pass filter. apparatus.
(15)
The control unit generates a change instruction for changing the low-pass characteristic of the low-pass filter at a pitch larger than a set minimum resolution of the low-pass characteristic of the low-pass filter, and at the same time, an imaging instruction synchronized with the change of the low-pass characteristic of the low-pass filter The imaging device according to (14).
(16)
A shutter button,
The imaging device according to (15), wherein the control unit generates the change instruction and the imaging instruction when the shutter button is half-depressed.

This application claims priority on the basis of Japanese Patent Application No. 2016-092993 filed on May 6, 2016 at the Japan Patent Office. The entire contents of this application are incorporated herein by reference. This is incorporated into the application.

Those skilled in the art will envision various modifications, combinations, subcombinations, and changes, depending on design requirements and other factors, which are within the scope of the appended claims and their equivalents. It is understood that

Claims

A control apparatus comprising: a control unit configured to set a setting value of a low-pass characteristic of the low-pass filter based on a change in resolution and a change in false color in a plurality of captured image data according to a change in the low-pass characteristic of the low-pass filter.
The control unit includes a spatial frequency of first image data that is one of the plurality of image data, and a spatial frequency of second image data other than the first image data among the plurality of image data. The control device according to claim 1, wherein first evaluation data for a change in resolution is derived based on the first evaluation data.
3. The control according to claim 2, wherein the control unit derives second evaluation data for a change in false color based on a difference between the gradation data of the first image data and the gradation data of the second image data. apparatus.
The control device according to claim 3, wherein the control unit sets the set value based on the first evaluation data and the second evaluation data.
When there is a range in which the change of the second evaluation data with respect to the low-pass characteristics of the variable optical low-pass filter is relatively gentle, the control unit has the smallest first evaluation data in the range. The control device according to claim 3, wherein the low-pass characteristic of the low-pass filter is set to the set value.
The control device according to claim 1, wherein the control unit sets the setting value based on a change in false color in a face area included in the plurality of image data.
The control device according to claim 1, wherein the control unit sets a low-pass characteristic of the low-pass filter when the image data selected by a user among a plurality of the image data is captured to the set value.
The control according to claim 1, wherein the control unit sets the setting value based on a change in resolution and a change in false color in the plurality of image data and a position of a false color included in the plurality of image data. apparatus.
The control device according to claim 1, wherein the control unit generates a plurality of the image data obtained by enlarging the positions of false colors included in the plurality of image data.
The control device according to claim 1, wherein the control unit generates a plurality of the image data in which false color positions included in the plurality of image data are highlighted.
The control device according to claim 1, wherein the control unit generates, among the plurality of image data, a highlight display of a position where resolution degradation is included, which is included in the plurality of image data.
The control unit is configured to determine the second evaluation data based on a difference between the gradation data of the first image data and the gradation data of the second image data, and saturation data obtained from the first image data. The control device according to claim 3.
The control device according to claim 3, wherein the first image data is image data when the second evaluation data is minimized or substantially minimized within a range in which the low-pass characteristics of the low-pass filter can be changed.
An image sensor that generates image data from light incident via a low-pass filter;
A controller configured to set a setting value of the low-pass characteristic of the low-pass filter based on a change in resolution and a change in false color in a plurality of image data captured in accordance with a change in the low-pass characteristic of the low-pass filter. apparatus.
The control unit generates a change instruction for changing the low-pass characteristic of the low-pass filter at a pitch larger than a set minimum resolution of the low-pass characteristic of the low-pass filter, and at the same time, an imaging instruction synchronized with the change of the low-pass characteristic of the low-pass filter The imaging device according to claim 14.
A shutter button,
The imaging device according to claim 15, wherein the control unit generates the change instruction and the imaging instruction when the shutter button is half-depressed.