JP6824757B2 - Imaging device, its control method, and control program - Google Patents

Description

The present invention relates to an image pickup device, a control method thereof, and a control program, and more particularly to an image pickup device using a solid-state image sensor having sensitivity to visible light and infrared light.

In general, an imaging device such as a surveillance camera is required to acquire a clear subject image even in low illuminance such as at night. As an image pickup device capable of acquiring a clear subject image even in low illuminance, for example, an image pickup device in which a solid-state image sensor (hereinafter, simply referred to as an image sensor) having sensitivity to infrared light as well as visible light is used. Is known (see Patent Document 1).

Further, in an image pickup device using an image pickup device, an image pickup device in which an optical low-pass filter is arranged on the light incident side of the image pickup element is known in order to suppress moire caused by a folded component. Further, also in the image pickup apparatus described in Patent Document 1, an optical low-pass filter is arranged on the light incident side of the image pickup device so as to obtain an image in which moire is suppressed.

Japanese Unexamined Patent Publication No. 2005-999208

By the way, in the imaging apparatus described in Patent Document 1, the same optical low-pass filter is inserted in both the photographing mode for acquiring an image photographed by visible light and the photographing mode for acquiring an image photographed by infrared light. It has gained.

However, when the same optical low-pass filter is used regardless of the shooting mode, it is difficult to obtain a high-quality image in all shooting modes depending on the imaging optical system and the image sensor.

Therefore, an object of the present invention is to provide an image pickup apparatus capable of obtaining a high-quality image in all shooting modes, a control method thereof, and a control program.

In order to achieve the above object, the imaging device according to the present invention includes an imaging element in which an optical image is formed via an imaging optical system and an image signal corresponding to the optical image is output, and the imaging optical system. An image pickup apparatus including an optical filter unit arranged between the image pickup element, wherein the optical filter unit is a first compound refraction member arranged in order from the side of the imaging optical system, an infrared cut. The infrared cut filter and the first phase member are imaged with the imaging optics in a first imaging mode having a filter, a first phase member, and a second double refraction member to obtain the image signal by visible light. A driving means for removing the infrared cut filter and the first phase member from the optical path of the imaging optical system in a second photographing mode in which the image signal is obtained by visible light and infrared light by inserting into the optical path of the system. It is characterized by having a control means for controlling the above.

According to the present invention, it is possible to obtain a high-quality image in all shooting modes such as a shooting mode in which shooting is acquired by visible light and a shooting mode in which an image is acquired by infrared light.

It is a block diagram which shows the structure about the example of the image pickup apparatus by embodiment of this invention. It is a figure for demonstrating the structure of the optical filter unit shown in FIG. It is a figure for demonstrating the operation of the optical low-pass filter in the 2nd shooting mode in the optical filter unit shown in FIG. It is a figure for demonstrating the point image intensity distribution of the light emitted from the optical filter unit shown in FIG. It is a figure which shows the point image intensity distribution when the phase difference is changed from 0 degree to 360 degree in the optical filter unit shown in FIG. It is a figure for demonstrating the sampling position in the image sensor shown in FIG. It is a figure for demonstrating another example of a sampling position in the image sensor shown in FIG. It is a figure for demonstrating another example of the image pickup apparatus according to embodiment of this invention.

An example of the image pickup apparatus according to the embodiment of the present invention will be described below with reference to the drawings.

<Imaging device>
FIG. 1 is a block diagram showing a configuration of an example of an image pickup apparatus according to an embodiment of the present invention.

The illustrated image pickup device 100 is, for example, a surveillance camera, and includes an imaging optical system 101, a solid-state imaging device (hereinafter, simply referred to as an imaging element) 102, a control unit 103, and an optical filter unit 104. An optical image is formed on the image pickup device 102 via the imaging optical system 101, and the image pickup device 102 outputs an image signal corresponding to the optical image. The optical filter unit 104 is arranged on the light incident side of the image sensor 102.

The image pickup device 102 is made of Si, InGaAs, an organic semiconductor, or the like having sensitivity to both visible light and infrared light. The image pickup apparatus 100 includes a first shooting mode in which a captured image is acquired by visible light and a second photographing mode in which a captured image is acquired by visible light and infrared light.

The control unit 103 drives the image sensor 102, reads out a signal from the image sensor 102, generates an image (photographed image) according to the output of the image sensor 102, and controls the operation of the optical filter 104.

<Optical filter unit>
FIG. 2 is a diagram for explaining the configuration of the optical filter unit shown in FIG. 2 (a) is a diagram showing the configuration of the optical filter unit, and FIG. 2 (b) is a diagram showing the optical axis of the first phase member shown in FIG. 2 (a). Further, FIG. 2 (c) is a diagram showing an optical axis of the second phase member shown in FIG. 2 (a).

The optical filter unit 104 includes the first birefringence member 105, the first member 106, the second phase member 107, and the second birefringence member 108 in order from the side (−Z side) of the imaging optical system 101. Have. Further, the optical filter unit 104 includes a drive mechanism 109, and the drive mechanism 109 inserts and removes the first member 106 along the Y axis under the control of the control unit 103.

The optic axis of the first birefringence member 105 is a direction rotated by 45 degrees from the + Z direction to the + X direction in the ZX plane, and the optic axis of the second birefringence member 108 is in the + Z direction in the ZX plane. It is a direction rotated by 45 degrees in the direction of −X.

As shown in FIG. 2A, the first member 106 includes an infrared cut filter 110 and a first phase member 111, and the infrared cut filter 110 and the first phase member 111 are bonded to each other. There is. Then, in the first photographing mode (also referred to as a visible light photographing mode), the first member 106 is inserted into the optical path of the imaging optical system 101 by the drive mechanism 109. On the other hand, in the second photographing mode (also referred to as an infrared light photographing mode), the first member 106 is removed from the optical path of the imaging optical system 101 by the driving mechanism 109.

As shown in FIGS. 2B and 2C, the optical axes of the first phase member 111 and the second phase member 107 are both rotated 45 degrees from the + X direction to the + Y direction in the XY plane. The direction. Further, the infrared cut filter 110 is formed of a resin or the like that selectively transmits visible light and selectively absorbs infrared light.

With the above configuration, in the first imaging mode, the optical low-pass filter is formed by the first birefringent member 105, the first phase member 111, the second phase member 107, and the second birefringent member 108. .. On the other hand, in the second photographing mode, the optical low-pass filter is formed by the first birefringent member 105, the second phase member 107, and the second birefringent member 108.

By the way, the characteristics of the optical low-pass filter are determined by the phase difference added by the phase member sandwiched between the birefringent members. Therefore, the characteristics of the optical low-pass filter can be changed between the first shooting mode and the second shooting mode. As a result, as will be described later, a high quality image can be acquired regardless of the shooting mode.

<Phase difference and optical low-pass filter characteristics in the second shooting mode>
FIG. 3 is a diagram for explaining the operation of the optical low-pass filter in the second photographing mode in the optical filter unit shown in FIG. FIG. 3A is a diagram showing the action when the phase difference due to the second phase member is 180 degrees, and FIG. 3B is a diagram showing the action when the phase difference due to the second phase member is 90 degrees. It is a figure which shows. Further, FIG. 3C is a diagram showing the operation when the phase difference due to the second phase member is 0 degrees.

In FIG. 3, the materials (here, the magnitude of birefringence) and the thickness of the first birefringent member 105 and the second birefringent member 108 are assumed to be equal to each other.

Now, when the natural light L1 is incident on the first birefringent member 105, the natural light L1 is separated into two polarizing components by the birefringence of the first birefringent member 105 as follows. Light whose electric field vibrates in a direction perpendicular to the optical axis of the first birefringent member 105 (Y-axis direction) (hereinafter referred to as Y polarized light) travels straight through the first birefringent member 105. On the other hand, light whose electric field oscillates in a direction perpendicular to Y-polarized light (X-axis direction) (hereinafter referred to as X-polarized light) travels diagonally under the influence of birefringence, and from the back surface of the first birefringence member 105 It is emitted.

The separation width d1 between Y-polarized light and X-polarized light is determined by the magnitude of birefringence and the thickness of the birefringent member. That is, the first birefringent member 105 separates the natural light L1 into the Y-polarized transmitted light Lp2 and the X-polarized transmitted light Ls2 with a separation width d1.

The light emitted from the first birefringent member 105 is incident on the second phase member 107. At this time, the polarization component changes according to the phase difference due to the second phase member 107. As shown in FIG. 3A, when the phase difference is 180 degrees, the Y-polarized light Lp2 changes to X-polarized light to become Lps3, and the X-polarized light Ls2 changes to Y-polarized light to become Lsp3. It is emitted from the first phase member 107.

As shown in FIG. 3B, when the phase difference of the second phase member 107 is 90 degrees, both the Y-polarized light Lp2 and the X-polarized light Ls2 change to circularly polarized light (here, respectively). Lpr3 and Lsr3). As shown in FIG. 3C, when the phase difference of the second phase member 107 is 0 degrees, the Y-polarized light Lp2 remains Y-polarized (Lpp3), and the X-polarized light Ls2 is X. It remains polarized (Lss3).

The light emitted from the second phase member 107 propagates in different directions for each polarized light due to the birefringence of the second birefringent member 108. Similar to the first birefringent member 105, the Y-polarized light travels straight through the second birefringent member 108, and the X-polarized light travels diagonally through the second birefringent member 108 and is emitted from the back surface.

In the case of the example shown in FIG. 3A, the second birefringent member 108 further separates the light Lps3 and Lsp3 incident on the separation width d1 by the separation width d1 to obtain the Y-polarized light Lpss4 and the X-polarized light. It is emitted as light Lsp4. That is, the interval between the optical Lps3 and Lsp3 is twice that of d1.

In the case of the example shown in FIG. 3B, the second compound refraction member 108 separates the light Lpr3 into Y-polarized light Lprp4 and X-polarized light Lprs4 with a separation width d1, and Y with the light Lsr3 at a separation width d1. The polarized light Lsrp4 and the X-polarized light Lsrs4 are separated and emitted. That is, the interval between the optical Lprs4 and the Lsrp4 is twice that of d1, and the optical Lprs4 and the Lsrs4 are emitted overlapping in the middle.

In the case of the example shown in FIG. 3C, the second birefringent member 108 emits the incident light Lpp3 and Lpp3 having the separation width d1 as the overlapping light Lppp4 and Lsss4.

<Point image intensity distribution>
FIG. 4 is a diagram for explaining a point image intensity distribution of light emitted from the optical filter unit shown in FIG. Then, FIGS. 4 (a), 4 (b), and 4 (c) are point image intensities in the states shown in FIGS. 3 (a), 3 (b), and 3 (c), respectively. It is a figure which shows an example of a distribution. Further, FIG. 4D is a diagram showing an example of a point image intensity distribution when the separation widths of the first birefringent member and the second birefringence member are different.

When the phase difference due to the second phase member 107 is 180 degrees, as shown in FIG. 4A, the light Lpss4 and Lspp4 cause two peaks p2 and p3 in the point image intensity distribution.

When the phase difference due to the second phase member 107 is 90 degrees, as shown in FIG. 4B, a peak p2 due to optical Lprs4 and a peak p3 due to Lsrp4 occur in the point image intensity distribution. Further, in the point image intensity distribution, a peak p1 due to the overlappingly emitted light Lprp4 and Lsrs4 occurs. That is, three peaks occur in the point image intensity distribution.

When the phase difference due to the second phase member 107 is 0 degrees, as shown in FIG. 4C, one peak p1 is generated in the point image intensity distribution due to the overlappingly emitted light Lpppp4 and Lsss4.

In this way, the point image intensity distribution due to the transmitted light can be changed by the phase difference given by the second phase member 107.

In general, the larger the spread of the point image intensity distribution, the smaller the cutoff frequency of the optical low-pass filter, and the smaller the spread of the point image intensity distribution, the larger the cutoff frequency of the optical low-pass cut filter.

Here, the expansion of the point image intensity distribution means the expansion of the point image intensity distribution in the system in which the imaging optical system 101 and the low-pass filter are combined. That is, as shown in FIGS. 4A and 4B, when the incident light is divided into a plurality of emitted light, the entire point image formed by the plurality of dividedly emitted light is emitted. The spread of.

Therefore, as shown in FIG. 4 (a), the cutoff frequency is the smallest when the phase difference is 180 degrees, and as shown in FIG. 4 (c), the cutoff frequency is the smallest when the phase difference is 0 degrees. large. The spacing between peaks p2 and p3 shown in FIG. 4B is equal to the spacing between peaks p2 and p3 shown in FIG. 4A. On the other hand, in FIG. 4A, a peak p1 having twice the intensity of the peaks p2 and p3 exists between the peaks p2 and p3. Therefore, the cutoff frequency is larger when the phase difference is 90 degrees than when the phase difference is 180 degrees.

<About intermediate phase time>
In the above description, the cases where the phase difference is 180 degrees, 90 degrees, and 0 degrees have been described, but the characteristics of the optical low-pass filter can be changed even when other phase differences are given.

FIG. 5 is a diagram showing a point image intensity distribution when the phase difference is changed from 0 degree to 360 degrees in the optical filter unit shown in FIG.

As can be easily understood from FIG. 5, the closer the phase difference due to the second phase member 107 is to 180 degrees, the wider the point image intensity distribution spreads, and the closer the phase difference is to 0 degrees, the wider the point image intensity distribution spreads. Is small. Therefore, the closer the phase difference due to the second phase member 107 is to 180 degrees, the smaller the cutoff frequency of the optical low-pass filter, and the closer the phase difference is to 0 degrees, the larger the cutoff frequency of the optical low-pass filter. ..

<Phase difference and optical low-pass filter characteristics in the first shooting mode>
As described above, the characteristics of the optical low-pass filter in the second imaging mode (infrared light imaging mode) are determined by the phase difference given by the second phase member 107 arranged between the birefringent members. On the other hand, in the first photographing mode (visible light photographing mode), two phase members of the first phase member 111 and the second phase member 107 are arranged. Therefore, in the first photographing mode, the characteristics of the optical low-pass filter are determined by the sum of the phase difference due to the first phase member 111 and the phase difference due to the second phase member 107.

<Comparison of characteristics between the first shooting mode and the second shooting mode>
As described above, the closer the phase difference is to 180 degrees, the smaller (lower) the cutoff frequency of the optical low-pass filter is, and the closer the phase difference is to 0 degrees, the larger (higher) the cutoff frequency of the optical low-pass filter is. Here, the phase difference due to the first phase member 111 is defined as α degree, and the phase difference due to the second phase member 107 is defined as β degree. At this time, the magnitude relationship of the cutoff frequency in the first shooting mode and the second shooting mode is defined according to which of the (α + β) degree and the β degree is closer to 0 degree.

When the following equation (1) is satisfied, the cutoff frequency of the optical low-pass filter in the second shooting mode can be made larger than the cutoff frequency of the optical low-pass filter in the second shooting mode. In particular, in the equation (1), the closer the left side is to 0 and the right side is closer to 180, the difference between the cutoff frequency of the optical low-pass filter in the first shooting mode and the cutoff frequency of the optical low-pass filter in the second shooting mode. Becomes larger. For example, it is most preferable that the phase difference due to the first phase member 111 is 180 degrees and the phase difference due to the second phase member 107 is 0 degrees.
| (Α + β) (mod 360) -180 | <| β (mod 360) -180 | (1)

On the other hand, when the following equation (2) is satisfied, the cutoff frequency of the optical low-pass filter in the second shooting mode can be made smaller than the cutoff frequency of the optical low-pass filter in the first shooting mode. In particular, in the equation (2), the closer the left side is to 180 and the right side is closer to 0, the difference between the cutoff frequency of the optical low-pass filter in the first shooting mode and the cutoff frequency of the optical low-pass filter in the second shooting mode. growing. For example, it is most preferable that the phase difference due to the first phase member 111 and the phase difference due to the second phase member 107 are both 180 degrees.
| (Α + β) (mod 360) -180 |> | β (mod 360) -180 | (2)

<Comparison between the present invention and conventional examples>
Also in the image pickup apparatus (conventional example) shown in Patent Document 1 described above, an infrared cut filter is inserted in the first shooting mode (visible light shooting mode), and in the second shooting mode (infrared light shooting mode). Has removed the infrared cut filter. However, the same optical low-pass filter is used in both the first shooting mode and the second shooting mode.

On the other hand, in the image pickup apparatus 100 (invention) shown in FIG. 1, the infrared cut filter and the phase member are integrally inserted and removed. Thereby, in the present invention, the characteristics of the optical low-pass filter can be changed according to the photographing mode without adding the insertion / extraction mechanism as compared with the conventional example.

<Structure of the first member>
In the first member 106 shown in FIG. 2 described above, the infrared cut filter 110 and the first phase member 111 are bonded in order from the light incident side, but the infrared cut filter 110 and the first phase member 111 The order may be reversed. Further, the infrared cut filter 110 and the first phase member 111 do not necessarily have to be bonded to each other.

<Lasting of second phase member and birefringent member>
In FIG. 2 described above, the second phase member 107 and the second birefringent member 108 are arranged separately. On the other hand, it is preferable that the second phase member 107 and the second birefringent member 108 are laminated and integrated because the configuration becomes simple.

<Presence / absence of second phase member>
The second phase member 107 does not necessarily have to be provided. When the second phase member 107 is not provided, the birefringent member and the phase member may be designed as follows. First, in the second imaging mode, the first birefringent member and the second birefringent member are designed so that a desired cutoff frequency can be obtained. Then, the first phase member is designed so that a desired cutoff frequency can be obtained in the first photographing mode. Such a configuration is structurally preferable because a second phase member is not required.

<Positional relationship between the first member and the second phase member>
In FIG. 2, the first member 106 is located on the light incident side of the second phase member 107, but the second phase member 107 is arranged on the light incident side of the first member 106. It may be. When the second phase member 107 is located on the light incident side of the first member 106, the configuration can be obtained by laminating and integrating the second phase member 107 and the first birefringent member 105. Keep it simple.

<Optic axis of birefringent member>
The optical axis direction of the first birefringence member 105 and the optical axis direction of the second birefringence member 108 are not limited to the example shown in FIG. In this case, it is necessary that the separation direction by the first birefringent member 105 and the separation direction by the second birefringence member 108 are opposite directions. Therefore, the direction in which the optical axis of the first birefringence member 105 is projected onto a plane (XY plane) perpendicular to the light incident direction and the optical axis of the second birefringence member 108 are perpendicular to the light incident direction (XY plane). The directions projected onto the plane) must be parallel.

Further, the optical axis of the first birefringent member 105 and the optical axis 108 of the second birefringent member need to be opposite to each other with respect to a plane (XY plane) perpendicular to the light incident direction. .. For example, in the example shown in FIG. 2, the optical axis of the first birefringent member 105 is the direction rotated from the + Z direction to the + X direction, and the optical axis of the second birefringent member 108 is −X from the + Z direction. It is the direction rotated in the direction. Therefore, the direction is opposite to the XY plane.

<Optic axis of phase member>
In FIG. 2, the optical axes of the first phase member 111 and the second phase member 107 are both directions rotated by 45 degrees from the + X direction to the + Y direction in the XY plane, but are not necessarily limited to this direction. Not done. However, since it is necessary to rotate the polarization direction, both the optical axes of the first phase member 111 and the second phase member 107 need to be perpendicular to the light incident direction (Z direction). Further, both optical axes need to be in a direction not parallel to the direction in which the optical axis of the first birefringence member 105 is projected onto a plane (XY plane) perpendicular to the light incident direction.

<Material and thickness of birefringent member>
In FIG. 3, the materials (that is, the magnitude of birefringence) and the thickness used for the first birefringent member 105 and the second birefringent member 108 are assumed to be equal to each other, but the materials and thickness are different from each other. May be good. However, if the magnitudes of the birefringence of the first birefringent member 105 and the second birefringence member 108 are equal, the degree of freedom in the characteristics of the optical low-pass filter is improved, which is preferable.

Now, let d1 be the separation width by the first birefringence member 105, and d2 be the separation width by the second birefringence member 108. In this case, the light Lp2 and Ls2 that have passed through the first birefringent member 105 and are separated by the separation width d1 pass through the second birefringence member 108 and are separated by the separation width d2. Therefore, even when the phase difference due to the phase member is 0 degrees, the optical Lpppp and Lsss4 do not overlap, and as a result, the point image intensity distribution has a plurality of peaks as shown in FIG. 4D. .. Therefore, in FIG. 4D, the cutoff frequency of the optical low-pass filter is smaller than that in FIG. 4C.

As described above, when the phase difference due to the phase member is 0 degrees, the cutoff frequency of the optical low-pass filter is the largest. Therefore, if the magnitude of the birefringence of the first birefringence member 105 and the second birefringence member 108 is different, the lower limit of the cutoff frequency obtained by adjusting the phase difference is the magnitude of the birefringence. It will be higher than when they are equal. That is, the degree of freedom in the characteristics of the optical low-pass filter obtained by adjusting the phase difference is reduced.

<Anti-reflective coating>
An antireflection film may be formed on the surface of each of the above-mentioned birefringence members and the phase member. The first birefringent member 105, the second phase member 107, and the second birefringent member 108 are located in the optical path in both the first imaging mode and the second imaging mode. Therefore, the first birefringence member 105, the second phase member 107, and the second birefringence member 108 are provided with an antireflection film that functions in both visible and infrared wavelength bands. Is preferable.

On the other hand, since the first member 106 is used only in the first photographing mode, the first member 106 may be provided with an antireflection film that functions in the wavelength band of visible light.

The wider the wavelength band in which the antireflection function is required, the more complicated the configuration of the antireflection film is. The first member 106 preferably includes an antireflection film that does not function in the wavelength band of infrared light but functions in the wavelength band of visible light. When the infrared cut filter 110 and the first phase member 111 are not bonded together, the infrared cut filter 110 and the first phase member 111 do not function in the wavelength band of infrared light, and visible light. It is preferable to have an antireflection film that functions in the wavelength band of.

<Wavelength dependence of phase difference due to first phase member>
In the above description, the phase difference due to the second phase member 107 is assumed to be equal in the first photographing mode and the second photographing mode. By the way, the phase difference has wavelength dependence depending on the material constituting the phase member and the manufacturing method. In particular, when the phase member is manufactured at low cost, the wavelength dependence of the phase difference is large.

The image pickup apparatus shown in FIG. 1 is desired in both the first shooting mode and the second shooting mode as follows, even when the phase difference due to the second phase member 107 has wavelength dependence. The phase difference can be obtained.

From the characteristics of the optical low-pass filter required in the first imaging mode, the total phase difference γ in visible light required for the first phase member 111 and the second phase member 107 can be obtained. Similarly, from the characteristics of the optical low-pass filter required in the second photographing mode, the phase difference β in the infrared light required for the second phase member 107 can be obtained.

Therefore, the second phase member 107 is designed so as to provide a desired phase difference β in the wavelength band of infrared light. At this time, the phase difference due to the second phase member 107 in the wavelength band of visible light is β'(≠ β). Therefore, if the phase difference due to the first phase member 111 in the wavelength band of visible light is γ-β', a desired phase difference can be obtained in the first imaging mode and the second imaging mode.

The image pickup apparatus described in Patent Document 1 uses the same optical low-pass filter in the first photographing mode and the second photographing mode. Therefore, when the phase member in the optical low-pass filter has wavelength dependence, the characteristics of the optical low-pass filter change in the first imaging mode and the second imaging mode.

On the other hand, in the imaging apparatus shown in FIG. 1, the wavelength dependence of the second phase member 107 is compensated by the first phase member 111 used only in the first imaging mode, so that the first imaging mode and the first imaging mode A desired phase difference can be obtained in the two shooting modes.

Whether or not to apply the phase compensation by the first phase member 111 may be determined by the accuracy required for the cutoff frequency of the optical low-pass filter. The higher the cutoff frequency of the optical low-pass filter, the more it is desirable to perform phase compensation even when the difference between the phase differences β and β ′ is small. Specifically, it is desirable to perform phase compensation when the difference between the phase differences β and β ′ is 30 degrees or more.

<Vertical separation>
The image pickup apparatus shown in FIG. 1 includes a single optical filter unit 104, and the light is separated by the optical low-pass filter in only one direction (X direction in the example shown in FIG. 2). On the other hand, a plurality of optical filter units 104 may be provided to separate light in a plurality of directions. If the light is separated in a plurality of directions, the point image intensity distribution is two-dimensionally expanded, so that moire can be suppressed regardless of the texture direction of the subject.

In particular, if two optical filter units are provided and the light separation directions of the optical filter units are orthogonal to each other, the spread of the point image intensity distribution approaches a circle, which is preferable. For example, it is preferable that the directions in which the optical axis of the birefringent member is projected onto a plane (XY plane) perpendicular to the light incident direction are orthogonal to each other between the two optical filter units.

Subsequently, an example of changing the characteristics of the optical low-pass filter in the first shooting mode and the second shooting mode will be described.

<When the infrared sampling pitch is smaller than visible>
Here, a case where the sampling pitch at the time of image generation in the second shooting mode is smaller than the sampling pitch at the time of image generation in the first shooting mode will be described.

FIG. 6 is a diagram for explaining an example of sampling positions in the image pickup device shown in FIG. FIG. 6A is a diagram showing a sampling position in the first imaging mode, and FIG. 6B is a diagram showing a sampling position in the second imaging mode. In FIG. 6, the sampling position is indicated by hatching with diagonal lines.

In the first shooting mode, the pixel arrangement in the image sensor 104 is a Bayer arrangement in order to acquire the color information of the subject. In the first shooting mode, the green pixel (G pixel) is the sampling position. On the other hand, in the second shooting mode, since the color information of the subject is not required, all the pixels are the sampling positions.

When the sampling pitch is large, the Nyquist frequency becomes small, so it is better to make the cutoff frequency of the optical low-pass filter small in order to suppress moire due to the folding component. On the other hand, when the sampling pitch is small, the Nyquist frequency becomes large, so it is better to increase the cutoff frequency of the optical low-pass filter in order to obtain a high-resolution image.

Therefore, if the cutoff frequency of the optical low-pass filter in the second shooting mode is made larger than that in the second shooting mode, high quality images can be obtained in the first shooting mode and the second shooting mode. ..

In order to make the cutoff frequency of the optical low-pass filter in the second shooting mode larger than the cutoff frequency in the first shooting mode, the phase difference due to the first phase member and the phase difference due to the second phase member are required. Satisfies the above-mentioned equation (1).

<When the infrared sampling pitch is larger than visible>
Here, a case where the sampling pitch at the time of image generation in the second shooting mode is larger than the sampling pitch at the time of image generation in the first shooting mode will be described.

FIG. 7 is a diagram for explaining another example of the sampling position in the image pickup device shown in FIG. FIG. 7A is a diagram showing a sampling position in the first imaging mode, and FIG. 7B is a diagram showing a sampling position in the second imaging mode. In FIG. 7, the sampling position is indicated by hatching with diagonal lines.

<When the pixels are different>
As shown in FIG. 7, the image sensor includes pixels that receive red light (R pixels), pixels that receive green light (G pixels), pixels that receive blue light (B pixels), and infrared pixels. It has pixels (IR pixels) that receive light. The G pixels are arranged in the same pixel arrangement as the normal Bayer arrangement, and the IR pixels are arranged by replacing a part of the R pixel and the B pixel in the Bayer arrangement. Then, in the first shooting mode, the G pixel is the sampling position, and in the second shooting mode, the IR pixel is the sampling position. Therefore, the sampling pitch is larger in the second shooting mode than in the first shooting mode.

Therefore, if the cutoff frequency of the optical low-pass filter in the second shooting mode is made smaller than that in the first shooting mode, high quality images can be obtained in the first shooting mode and the second shooting mode. it can.

In order to make the cutoff frequency of the optical low-pass filter in the second shooting mode smaller than the cutoff frequency in the first shooting mode, the phase difference due to the first phase member and the phase difference due to the second phase member are required. Satisfies the above-mentioned equation (2).

The arrangement of the R pixel, the G pixel, the B pixel, and the IR pixel is not limited to the arrangement shown in FIG. 7, and the sampling pitch of the G pixel may be smaller than the sampling pitch of the IR pixel.

<When performing additive reading>
Further, the pixel arrangement of the image sensor 102 may be a Bayer arrangement, and the sampling pitches may be different in the first shooting mode and the second shooting mode. For example, since the second shooting mode for acquiring infrared light is mainly used in a low illuminance environment, the SN ratio of the image is likely to decrease as compared with the first shooting mode. Therefore, in the second shooting mode, so-called additive reading, in which a plurality of pixel signals are added and averaged and read out, is performed to improve the SN ratio.

In other words, the additive reading is not performed in the first imaging mode, and the additive reading is performed in the second imaging mode. Alternatively, additional reading is performed in both the first shooting mode and the second shooting mode, and at this time, the number of pixels to be added in the second shooting mode is larger than that in the first shooting mode.

When such reading is performed, the sampling pitch in the second shooting mode becomes larger than the sampling pitch in the first shooting mode.

<When the performance of the imaging optical system is different>
Here, a case where the imaging performance of the imaging optical system 101 differs depending on the wavelength will be described. In general, when trying to suppress the wavelength dependence in the imaging performance of the imaging optical system 101, the imaging optical system 101 becomes large and the manufacturing cost further increases. For this reason, it is difficult to suppress the wavelength dependence of imaging performance, especially when emphasis is placed on miniaturization and cost reduction.

The lower the imaging performance of the imaging optical system 101, the lower the contrast of the subject image having a large spatial frequency. Therefore, even if the cutoff frequency of the optical low-pass filter is large, moire due to the folding component is not noticeable.

On the other hand, when the imaging performance of the imaging optical system 101 is low and the cutoff frequency of the optical low-pass filter is reduced, the contrast of the subject image having a large spatial frequency is also reduced. Therefore, the quality of the subject image is deteriorated. Therefore, when the imaging performance of the imaging optical system 101 is low, it is desirable to increase the cutoff frequency of the optical low-pass filter.

For example, when the imaging performance of the imaging optical system 101 with respect to infrared light is lower than the imaging performance with respect to visible light, the cutoff frequency of the optical low-pass filter in the second imaging mode is set to be lower than that of the first imaging mode. It is also desirable to increase. For this purpose, the phase difference due to the first phase member 107 and the phase difference due to the second phase member 111 may satisfy the above-mentioned equation (1).

<When the required image quality is different>
Here, a case where the required image quality differs between the first shooting mode and the second shooting mode will be described.

In the first shooting mode for acquiring visible light, a color image is acquired. On the other hand, in the second shooting mode in which visible light and infrared light are acquired at the same time, a monochrome image is obtained. Here, it is generally known that moire due to color components is more conspicuous than moire due to luminance components. Therefore, in the first shooting mode for acquiring a color image, the suppression of moire may be emphasized, and in the second shooting mode for acquiring a monochrome image, the resolution may be emphasized. At this time, the cutoff frequency of the optical low-pass filter in the second shooting mode is made larger than that in the first shooting mode. As a result, in the second shooting mode, an image with an emphasis on resolution can be acquired, and in the first shooting mode, an image with an emphasis on suppression of moire can be acquired.

In order to make the cutoff frequency of the optical low-pass filter in the second imaging mode higher than that in the first imaging mode, the phase difference due to the first phase member 107 and the phase difference due to the second phase member 111 are described above. It suffices if the equation (1) of is satisfied.

<Infrared light only mode>
Here, in addition to the first shooting mode and the second shooting mode, an imaging device including a third shooting mode for acquiring an image only by infrared light will be described.

Infrared light, which has a longer wavelength than visible light, is less scattered by smoke and fog in the atmosphere, so that a distant subject becomes clear in an image taken only by infrared light. Therefore, the third shooting mode is used, for example, when shooting a distant subject clearly when the fog and smoke are thick.

FIG. 8 is a diagram for explaining another example of the image pickup apparatus according to the embodiment of the present invention. 8 (a) is a block diagram showing the configuration of the image pickup apparatus, and FIG. 8 (b) is a diagram showing the configuration of the optical filter unit shown in FIG. 8 (a).

In FIG. 8, the same components as those shown in FIGS. 1 and 2 are designated by the same reference numbers, and the description thereof will be omitted.

The image pickup apparatus 200 shown in FIG. 8A has an optical filter unit 204 arranged between the image pickup optical system 101 and the image pickup element 102. In the optical filter unit 204, the visible cut filter 212 is attached to the second phase member 107 to form the second member 213. Then, the first member 106 and the second member 213 are inserted and removed from the optical path by the drive mechanism 209 which is driven and controlled by the control unit 103. The visible cut filter 212 is formed from a material such as a resin that selectively absorbs visible light and selectively transmits infrared light.

In the illustrated image pickup apparatus 200, in the first photographing mode, the first member 106 is inserted into the optical path, and the second member 213 is removed from the optical path. On the other hand, in the second photographing mode, both the first member 106 and the second member 213 are removed from the optical path. Further, in the third photographing mode, the first member 106 is removed and the second member 213 is inserted into the optical path.

By performing such control, in the first photographing mode, the optical low-pass filter is formed by the first birefringent member 105, the first phase member 111, and the second birefringent member 108. On the other hand, in the second photographing mode, the optical low-pass filter is formed by the first birefringent member 105 and the second birefringent member 108. Then, in the third photographing mode, the optical low-pass filter is formed by the first birefringent member 105, the second phase member 107, the second and the birefringent member 108. That is, the characteristics of the optical low-pass filter can be changed in each of the three shooting modes of the first shooting mode, the second shooting mode, and the third shooting mode.

Subsequently, the design of each birefringent member and the phase member will be described.

In the second photographing mode, there is no phase member between the first birefringent member 105 and the second birefringent member 108. Therefore, as described in FIG. 4A described above, a point image intensity distribution having two peaks in which the distance between the peaks is twice d1 can be obtained. As described above, d1 is determined by the thickness and the magnitude of birefringence of the first birefringent member 105 and the second birefringent member 108. Therefore, the first birefringent member 105 and the second birefringent member 108 may be designed so that a desired cutoff frequency can be obtained in the second photographing mode.

In addition to the characteristics of the birefringent member, in the first imaging mode and the third imaging mode, the characteristics of the optical low-pass filter are determined by the phase difference between the first phase member 111 and the second phase member 107, respectively. To. Therefore, the first phase member 111 and the second phase member 107 may be designed so that desired cutoff frequencies can be obtained in the first imaging mode and the third imaging mode, respectively.

Since the third shooting mode is used when a distant subject is clearly shot, it is difficult to obtain an image with an emphasis on resolution. Therefore, the phase difference due to the second phase member 107 is preferably close to 180 degrees. On the other hand, in the first shooting mode and the second shooting mode, as described above, the cutoff frequency is determined according to the sampling pitch at the time of image generation, the characteristics of the imaging optical system 101, and the desired image quality. To.

For example, as described above, it is desired that the cutoff frequency of the optical low-pass filter in the second shooting mode is made higher than that in the second shooting mode. In this case, the phase difference due to the first phase member 111 may be 90 degrees, and the phase difference due to the second phase member 107 may be 180 degrees.

As a result, in the first photographing mode, the point image intensity distribution shown in FIG. 4B is obtained, and in the second photographing mode, the point image intensity distribution shown in FIG. 4A is obtained. Then, in the third photographing mode, the point image intensity distribution shown in FIG. 4C is obtained. That is, the cutoff frequency of the optical low-pass filter is the smallest in the second shooting mode, followed by the highest in the first shooting mode and the third shooting mode. As a result, a high quality image can be obtained regardless of the shooting mode.

As described above, in the embodiment of the present invention, a high quality image can be acquired even in the shooting mode in which the image is obtained by visible light and the shooting mode in which the image is obtained by infrared light.

Although the present invention has been described above based on the embodiments, the present invention is not limited to these embodiments, and various embodiments within the scope of the gist of the present invention are also included in the present invention. ..

For example, the function of the above embodiment may be used as a control method, and the image pickup apparatus may be made to execute this control method. Further, a program having the functions of the above-described embodiment may be used as a control program, and the control program may be executed by a computer included in the image pickup apparatus. The control program is recorded on a computer-readable recording medium, for example.

[Other Embodiments]
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100, 200 Imaging device 101 Imaging optical system 102 Image sensor 103 Control unit 104, 204 Optical filter unit 105, 108 Birefringence member 107, 111 Phase member 109, 209 Drive mechanism 110 Infrared cut filter 212 Visible cut filter

Claims (23)

An image pickup element in which an optical image is formed via an imaging optical system and an image signal corresponding to the optical image is output, and an optical filter unit arranged between the imaging optical system and the image pickup element are provided. It is an image sensor equipped with
The optical filter unit includes a first birefringence member, an infrared cut filter, a first phase member, and a second birefringence member arranged in order from the side of the imaging optical system.
In the first photographing mode in which the image signal is obtained by visible light, the infrared cut filter and the first phase member are inserted into the optical path of the imaging optical system, and the image signal is obtained by visible light and infrared light. An imaging apparatus comprising: a control means for controlling a driving means for extracting the infrared cut filter and the first phase member from the optical path of the imaging optical system in a second photographing mode. The imaging device according to claim 1, wherein the infrared cut filter and the first phase member are integrated. The direction in which the optical axis of the first birefringence member is projected onto a plane perpendicular to the light incident direction of the imaging optical system, and the plane perpendicular to the light incident direction of the optical axis of the second birefringence member. Is parallel to the direction projected on the
According to claim 1 or 2, the optical axis of the first birefringence member and the optical axis of the second birefringence member are opposite to each other with respect to a plane perpendicular to the light incident direction. The imaging device described. The optical axis of the first phase member is perpendicular to the light incident direction of the imaging optical system, and the optical axis of the first birefringence member is projected onto a plane perpendicular to the light incident direction. The imaging apparatus according to any one of claims 1 to 3, wherein the image pickup apparatus is not parallel to the image. The imaging according to any one of claims 1 to 4, wherein the magnitude of the birefringence of the first birefringent member and the magnitude of the birefringence of the second birefringence member are equal to each other. apparatus. The infrared cut filter and the first phase member include antireflection films that do not function in the wavelength band of infrared light but function in the wavelength band of visible light, according to claims 1 to 5. The imaging apparatus according to any one item. The optical filter unit according to claim 1 to 6, further comprising a second phase member arranged between the first birefringent member and the second birefringent member. The imaging apparatus according to any one item. A claim characterized in that the second phase member and the first birefringent member are integrated, or the second phase member and the second birefringence member are integrated. 7. The imaging apparatus according to 7. The imaging apparatus according to claim 7 or 8, wherein when the phase difference due to the first phase member is α degree and the phase difference due to the second phase member is β degree, the following equation is satisfied. ..
| (Α + β) (mod 360) -180 | <| β (mod 360) -180 | The imaging apparatus according to claim 9, wherein the phase difference due to the first phase member is 180 degrees, and the phase difference due to the second phase member is 0 degrees. The imaging device according to claim 9 or 10, wherein the sampling pitch of the image pickup device in the second shooting mode is smaller than the sampling pitch of the image pickup device in the first shooting mode. Any one of claims 9 to 11, wherein the imaging performance of the imaging optical system in the second imaging mode is lower than the imaging performance of the imaging optical system in the first imaging mode. The imaging apparatus according to the section. The invention according to any one of claims 9 to 12, wherein the resolution of the image signal obtained in the second shooting mode is higher than the resolution of the image signal obtained in the first shooting mode. Imaging device. The imaging apparatus according to claim 7 or 8, wherein when the phase difference due to the first phase member is α degree and the phase difference due to the second phase member is β degree, the following equation is satisfied.
| (Α + β) (mod 360) -180 |> | β (mod 360) -180 | The imaging apparatus according to claim 14, wherein the phase difference due to the first phase member and the phase difference due to the second phase member are both 180 degrees. The imaging device according to claim 14 or 15, wherein the sampling pitch of the image pickup device in the second shooting mode is larger than the sampling pitch of the image pickup element in the first shooting mode. The image pickup element has a pixel that acquires red light, a pixel that acquires green light, a pixel that acquires blue light, and a pixel that acquires infrared light.
The imaging apparatus according to claim 16, wherein the sampling pitch of the pixel that acquires the infrared light is larger than the sampling pitch of the pixel that acquires the green light. 14 or 16 according to claim 14, wherein in the first shooting mode, the additive reading for adding the outputs of the pixels provided in the image sensor is not performed, and in the second shooting mode, the additive reading is performed. The imaging apparatus according to. In both the first shooting mode and the second shooting mode, additive reading is performed by adding the outputs of the pixels provided in the image sensor, and the number of pixels of the additional reading in the second shooting mode is the first. The image pickup apparatus according to claim 14 or 16, wherein the number of pixels for additional reading in the photographing mode is larger than the number of pixels. The imaging apparatus according to any one of claims 7 to 19, wherein the phase difference due to the second phase member is different between the wavelength band of visible light and the wavelength band of infrared light. The optical filter unit further includes a visible cut filter arranged between the first birefringence member and the second birefringence member.
The driving means selectively inserts and removes the visible cut filter and the second phase member with respect to the optical path.
The control means controls the driving means in the first shooting mode and the second shooting mode to remove the visible cut filter and the second phase member from the optical path, and the image is emitted by infrared light. The invention according to any one of claims 7 to 20, wherein in the third photographing mode in which a signal is acquired, the driving means is controlled to insert the visible cut filter and the second phase member into the optical path. The imaging apparatus described. An image pickup element in which an optical image is formed via an imaging optical system and an image signal corresponding to the optical image is output, and an optical filter unit arranged between the imaging optical system and the image pickup element are provided. It is a control method of the image sensor provided.
The optical filter unit includes a first birefringence member, an infrared cut filter, a first phase member, and a second birefringence member arranged in order from the side of the imaging optical system.
The first step of inserting the infrared cut filter and the first phase member into the optical path of the imaging optical system in the first photographing mode in which the image signal is obtained by visible light.
A second step of removing the infrared cut filter and the first phase member from the optical path of the imaging optical system in a second imaging mode in which the image signal is obtained by visible light and infrared light.
A control method characterized by having. An image pickup element in which an optical image is formed via an imaging optical system and an image signal corresponding to the optical image is output, and an optical filter unit arranged between the imaging optical system and the image pickup element are provided. It is a control program used in the built-in image sensor.
The optical filter unit includes a first birefringence member, an infrared cut filter, a first phase member, and a second birefringence member arranged in order from the side of the imaging optical system.
In the computer provided in the imaging device,
The first step of inserting the infrared cut filter and the first phase member into the optical path of the imaging optical system in the first photographing mode in which the image signal is obtained by visible light.
A second step of removing the infrared cut filter and the first phase member from the optical path of the imaging optical system in a second imaging mode in which the image signal is obtained by visible light and infrared light.
A control program characterized by executing.