JP2012044443A - Imaging module and imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a low-noise image across a large spatial frequency region.SOLUTION: An imaging module comprises: a first imaging system; a second imaging system which is lower in response than the first imaging system in a first spatial frequency region and is higher in response than the first imaging system in a second spatial frequency region; and an image generation unit which generates an image of a subject using a spatial frequency component in the first spatial frequency region of the image of the subject taken by the first imaging system and a spatial frequency component in the second spatial frequency region of the image of the subject taken by the second imaging system.

Description

  The present invention relates to an imaging module and an imaging apparatus.

2. Description of the Related Art An imaging device having a plurality of imaging systems and including an optical filter that decreases in transmittance as the distance from the optical axis of the imaging system increases is known (see, for example, Patent Document 1). Further, a technique is known in which an image of a subject is captured twice through two optical LPFs having different spatial frequency characteristics (see, for example, Patent Document 2).
Patent Document 1 JP 2001-78214 JP Patent Document 2 JP 9-116911 A

  As an imaging device, it is desirable that an image with low noise can be provided over a wide spatial frequency region. Even if the frequency component in the spatial frequency region where the MTF is low is amplified by digital processing, noise is also amplified, and there is a problem that a low noise image cannot be provided over a wide spatial frequency region.

  In order to solve the above-mentioned problem, in the first aspect of the present invention, the imaging module is a first imaging system and the response in the first spatial frequency region is lower than that of the first imaging system, and the first spatial frequency A second imaging system having a higher response in the second spatial frequency region than the first imaging system than the first imaging system, a spatial frequency component in the first spatial frequency region of the image of the subject imaged by the first imaging system, and a second imaging An image generation unit configured to generate a subject image using a spatial frequency component in the second spatial frequency region of the subject image captured by the system.

  The first imaging system has a first optical system, and the second imaging system has an optical response in the first spatial frequency region lower than that of the first optical system, and an optical response in the second spatial frequency is the first optical system. You may have a higher second optical system.

  Each of the first optical system and the second optical system includes an optical filter that provides a light transmittance distribution on the pupil plane, and the first optical filter included in the first optical system is higher in the central area than in the peripheral area of the pupil plane. The second optical filter having light transmittance and having the second optical system may have higher light transmittance in the peripheral region than in the central region of the pupil plane.

  The first optical system may include a first lens, and the second optical system may include a second lens having an effective aperture larger than that of the first lens.

  The first optical system may include a first lens, and the second optical system may include a second lens having a larger magnification than the first lens.

  The first imaging system includes a first light receiving unit including a plurality of imaging elements that receive light from the subject through the first optical system, and the second imaging system receives light from the subject through the second optical system. The plurality of imaging elements included in the first light receiving unit may be arranged at a lower pitch than the plurality of imaging elements included in the second light receiving unit.

  The first imaging system includes a first light receiving unit including a plurality of imaging elements that receive light from the subject through the first optical system, and the second imaging system receives light from the subject through the second optical system. A reading unit that includes a second light receiving unit including a plurality of imaging elements, and the imaging module reads out from the plurality of imaging elements included in the first light receiving unit at a lower reading pitch than the plurality of imaging elements included in the second light receiving unit. May be further provided.

  The plurality of imaging elements respectively included in the first light receiving unit and the second light receiving unit may be provided on the same plane.

  The image generation unit is configured to identify an image area in which the same subject should be imaged by the first imaging system and the second imaging system based on the positions of the imaging optical axes of the first imaging system and the second imaging system. A spatial frequency component in the first spatial frequency region of the subject image captured by the first imaging system and a second spatial frequency region of the subject image captured by the second imaging system, for each image region; An image synthesis unit that generates an image of the subject by synthesizing the spatial frequency components may be included.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

1 is a diagram schematically illustrating an example of a configuration of an imaging apparatus 100. FIG. It is a figure which shows an example of the light transmittance distribution by the 1st optical filter 114 and the 2nd optical filter 124. FIG. It is a figure which shows an example of MTF of the 1st optical system 115 and the 2nd optical system 125. FIG. 3 is a diagram illustrating an example of a block configuration of an image generation unit 150. FIG. 6 is a diagram illustrating an example of filter processing in a frequency filter unit 410. FIG. It is a figure which shows an example of the image process with respect to the image imaged by each imaging system. It is a figure which shows another example of lens arrangement | positioning. It is a figure which shows an example of a structure of the light-receiving part 809 seen from the object side.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 schematically illustrates an example of the configuration of the imaging apparatus 100. The imaging device 100 according to the present embodiment aims to provide a low noise image over a relatively wide spatial frequency region. The imaging apparatus 100 includes a first imaging system 101, a second imaging system 102, a reading unit 140, an image generation unit 150, and a recording unit 180. The first imaging system 101, the second imaging system 102, the reading unit 140, and the image generation unit 150 function as an imaging module incorporated in the imaging device 100.

  The imaging optical axis of the first imaging system 101 is assumed to be parallel to the imaging optical axis of the second imaging system. The second imaging system 102 is provided with a position of the imaging optical axis different from that of the first imaging system 101 in a plane perpendicular to the imaging optical axis. In this embodiment, when the direction along the imaging optical axis is the z-axis direction, the imaging optical axis of the second imaging system 102 is provided away from the imaging optical axis of the first imaging system 101 in the x direction. Suppose that

  The second imaging system 102 has a different spatial frequency response from the first imaging system 101. Specifically, the second imaging system 102 has a lower response in the first spatial frequency region than the first imaging system 101, and a higher response in the second spatial frequency region than the first spatial frequency region. taller than. As an example of the response in the spatial frequency domain, MTF can be used as an index. Since the imaging apparatus 100 includes the second imaging system 102 in addition to the first imaging system 101, the image of the subject is obtained by combining the image captured by the first imaging system 101 and the image captured by the second imaging system 102. Can be generated. For this reason, compared with the case where the second imaging system 102 is not provided, an image with low noise can be provided over a relatively wide spatial frequency region.

  The first imaging system 101 includes a first optical system 115 and a first light receiving unit 119. The first optical system 115 includes a first lens 110, a first diaphragm 112, and a first optical filter 114. The first light receiving unit 119 includes a first color filter array 116 and a first image sensor array 118.

  The first lens 110 is an imaging lens and forms an image of subject light, which is light from the subject, on the first light receiving unit 119. The first diaphragm 112 adjusts the amount of light that passes through the first optical system 115. Light that passes through the first optical system 115 is limited to light that passes through the opening of the first diaphragm 112.

  The first optical filter 114 gives a distribution to the light transmittance on the pupil plane. The light passing through the first optical system 115 passes through the first lens 110, the first diaphragm unit 112, and the first optical filter 114 and is imaged on the first light receiving unit 119.

  The first color filter array 116 includes a plurality of color filters that respectively pass light in a predetermined wavelength range. The first color filter array 116 is formed by arranging a plurality of color filters in a matrix. For example, the first color filter array 116 includes an R light filter that selectively transmits R light belonging to the red wavelength range, a G light filter that selectively transmits G light belonging to the green wavelength range, and a blue wavelength range. B light filters that selectively transmit the B light belonging to are arranged in a matrix with a predetermined pattern.

  The first image sensor array 118 includes a plurality of image sensors that receive light from the subject through the first optical system 115. The plurality of image sensors included in the first image sensor array 118 are provided corresponding to the plurality of color filters included in the first color filter array 116. Each imaging device receives light in a specific wavelength range that has passed through the corresponding color filter, out of the subject light incident on the first color filter array 116. Each imaging element outputs an imaging signal having an intensity corresponding to the amount of received light.

  The second imaging system 102 includes a second optical system 125 and a second light receiving unit 129. The second optical system 125 includes a second lens 120, a second diaphragm unit 122, and a second optical filter 124. The second light receiving unit 129 has a second color filter array 126 and a second imaging element array 128.

  Among the optical elements included in the second imaging system 102, the second lens 120, the second diaphragm 122, the second color filter array 126, and the second imaging element array 128 are the first lens 110 and the first diaphragm, respectively. 112, the first color filter array 116, and the first image sensor array 118 have substantially the same characteristics.

  On the other hand, the second optical filter 124 gives a light transmittance distribution on the pupil plane of the second optical system 125 that is different from the light transmittance distribution that the first optical filter 114 gives to the pupil plane of the first optical system 115. Due to apodization by each of the first optical filter 114 and the second optical filter 124, the first imaging system 101 and the second imaging system 102 have different spatial frequency responses.

  An optical element included in the second imaging system will be described. The second lens 120 is an imaging lens and forms an image of subject light, which is light from the subject, on the second light receiving unit 129. The second diaphragm unit 122 adjusts the amount of light that passes through the second optical system 125. The light passing through the second optical system 125 is limited to light passing through the opening of the second diaphragm unit 122.

  The second optical filter 124 distributes the light transmittance on the pupil plane. The light passing through the second optical system 125 passes through the second lens 120, the second diaphragm unit 122, and the second optical filter 124 and is imaged on the second light receiving unit 129.

  The second color filter array 126 includes a plurality of color filters that respectively pass light in a predetermined wavelength range. The second color filter array 126 is formed by arranging a plurality of color filters in a matrix. For example, the second color filter array 126 includes an R light filter that selectively transmits R light belonging to the red wavelength range, a G light filter that selectively transmits G light belonging to the green wavelength range, and a blue wavelength range. B light filters that selectively transmit the B light belonging to are arranged in a matrix with a predetermined pattern.

  The second image sensor array 128 includes a plurality of image sensors that receive light from the subject through the second optical system 125. The plurality of image sensors included in the second image sensor array 128 are provided corresponding to the plurality of color filters included in the second color filter array 126. Each imaging device receives light in a specific wavelength range that has passed through the corresponding color filter, out of the subject light incident on the second color filter array 126. Each imaging element outputs an imaging signal having an intensity corresponding to the amount of received light.

  The difference in light transmission characteristics between the first optical filter 114 and the second optical filter 124 will be described. The first optical filter 114 included in the first optical system 115 has a higher light transmittance in the central region than in the peripheral region of the pupil plane. On the other hand, the second optical filter 124 included in the second optical system 125 has a higher light transmittance in the peripheral region than in the central region of the pupil plane.

  As described above, the first lens 110 and the second lens 120 have substantially the same characteristics, and the first aperture section 112 and the second aperture section 122 have approximately the same characteristics. For example, it is assumed that the first lens 110 and the second lens 120 have substantially the same imaging characteristics. Further, it is assumed that the first diaphragm portion 112 and the second diaphragm portion 122 have substantially the same opening size and shape. Accordingly, the difference in the spatial frequency response between the first optical system 115 and the second optical system 125 is caused by the difference in the light transmittance distribution by the first optical filter 114 and the light transmittance distribution by the second optical filter 124. It is.

  As described above, the first color filter array 116 and the second color filter array 126 have substantially the same characteristics, and the first image sensor array 118 and the second image sensor array 128 have substantially the same characteristics. . For example, it is assumed that the plurality of image sensors included in the first image sensor array 118 are arranged at substantially the same pitch as the plurality of image sensors included in the second image sensor array 128. Similarly, the plurality of color filters included in the first color filter array 116 are arranged at substantially the same pitch as the plurality of color filters included in the second color filter array 126. Also, it is assumed that the color filter array patterns that transmit light of each color are the same in the first color filter array 116 and the second color filter array 126. Therefore, the difference in the spatial frequency response between the first imaging system 101 and the second imaging system 102 is also brought about by the difference in the light transmittance distribution between the first optical filter 114 and the second optical filter 124.

  Here, the plurality of imaging elements respectively included in the first light receiving unit 119 and the second light receiving unit 129 are provided on the same plane. For example, the plurality of image sensors included in the first image sensor array 118 and the plurality of image sensors included in the second image sensor array 128 are formed on the same substrate. For example, the plurality of image sensors included in the first image sensor array 118 and the plurality of image sensors included in the second image sensor array 128 are simultaneously formed on one surface of the same substrate in the same process. In addition, the plurality of color filters included in the first color filter array 116 and the plurality of color filters included in the second color filter array 126 are also simultaneously formed in the same process on the corresponding image sensor. Thus, one light receiving unit 109 that is integrally included in the first light receiving unit 119 and the second light receiving unit 129 is manufactured. The imaging surface of the first light receiving unit 119 and the imaging surface of the second light receiving unit 129 are partial areas of the imaging surface of the light receiving unit 109. The first light receiving unit 119 having the first image sensor array 118 and the first color filter array 116 and the second light receiving unit 129 having the second image sensor array 128 and the second color filter array 126 are separated from each other. It may be manufactured and assembled.

  The plurality of imaging elements included in the first light receiving unit 119 and the plurality of imaging elements included in the second light receiving unit 129 are exposed substantially simultaneously. When each image sensor is exposed, the reading unit 140 reads an imaging signal from the plurality of image sensors included in the first light receiving unit 119 and the plurality of image sensors included in the second light receiving unit 129. The reading unit 140 may include a reading circuit formed on the same substrate as the plurality of imaging elements included in the first light receiving unit 119 and the second light receiving unit 129. One readout circuit is provided for the plurality of first imaging systems 101 and the second imaging system 102, and one readout circuit includes a plurality of imaging elements included in the first light receiving unit 119 and the second light receiving unit 129, respectively. The imaging signals may be read sequentially. Note that the readout circuit may be provided separately for each of the plurality of first imaging systems 101 and the second imaging systems 102.

  The plurality of imaging elements included in the first light receiving unit 119 and the plurality of imaging elements included in the second light receiving unit 129 may be MOS type imaging elements. When each imaging device is a MOS type imaging device formed on one surface of the same substrate in the same process, the reading unit 140 includes an imaging device formed on an imaging surface on which subject light that has passed through the first optical system 115 is incident. Partial reading can be performed from an imaging device formed on an imaging surface on which subject light that has passed through the second optical system 125 enters. The plurality of imaging elements included in the first light receiving unit 119 and the plurality of imaging elements included in the second light receiving unit 129 can be implemented by solid-state imaging elements such as a CCD type imaging element in addition to a MOS type imaging element.

  The imaging signal read by the reading unit 140 is supplied to the image generation unit 150. The imaging signals read from the plurality of imaging elements included in the first light receiving unit 119 indicate the subject image captured by the first imaging system 101. The imaging signals read from the plurality of imaging elements included in the second light receiving unit 129 indicate the subject image captured by the second imaging system 102. The image generation unit 150 includes a spatial frequency component in the first spatial frequency region of the subject image captured by the first imaging system 101 and a space in the second spatial frequency region of the subject image captured by the second imaging system 102. An image of the subject is generated using the frequency component.

  Specifically, the image generation unit 150 extracts a spatial frequency component in the first spatial frequency domain from the image of the subject imaged by the first imaging system 101. Further, the image generation unit 150 extracts a spatial frequency component in the second spatial frequency domain from the image of the subject imaged by the second imaging system 102. Then, the image generation unit 150 synthesizes the extracted spatial frequency component in the first spatial frequency domain and the spatial frequency component in the second spatial frequency domain in the spatial frequency domain.

  The image generation unit 150 may output the spatial frequency domain image signal obtained by the synthesis in the spatial frequency domain to the recording unit 180 as an image of the subject. In addition, the image generation unit 150 may convert the image signal in the spatial frequency domain into an image signal in the spatial domain and output the obtained spatial domain image signal to the recording unit 180 as an image of the subject. Good. The recording unit 180 records the image generated by the image generation unit 150.

  The recording unit 180 may record the image supplied from the image generation unit 150 in a nonvolatile memory. The non-volatile memory may be included in the recording unit 180. The non-volatile memory may be an external memory that is detachably attached to the imaging apparatus 100.

  The imaging device 100 may be an imaging device such as a mobile phone with a camera function or a digital camera. The imaging module having the first imaging system 101, the second imaging system 102, the reading unit 140, and the image generation unit 150 can be provided as a camera module for these imaging devices.

  The imaging apparatus 100 includes the first imaging system 101 having a high MTF in a relatively low spatial frequency region and the second imaging system 102 that provides an imaging signal having a high MTF in a relatively high spatial frequency region. . Then, a relatively low spatial frequency domain component of the imaging signal output by the first imaging system 101 and a relatively high spatial frequency domain component of the imaging signal output by the second imaging system 102 are used. An image signal of the subject is generated.

  If an image of a subject is generated from an imaging signal obtained by one imaging system, noise is also enhanced when digital processing is performed to enhance an imaging signal in a frequency region where the MTF is relatively low. For this reason, an image with a high SN ratio cannot be obtained, and an image with a high resolving power may not be provided. On the other hand, according to the imaging apparatus 100, the first imaging system 101 and the second imaging system 102 having a higher response in a high frequency region than the first imaging system 101 are provided, so that the degree of the enhancement processing is reduced. In some cases, the emphasis process may not be performed. For this reason, according to the imaging device 100, an image with a high SN ratio and high resolving power can be provided.

  FIG. 2 shows an example of a light transmittance distribution by the first optical filter 114 and the second optical filter 124. Here, the light transmittance distribution in the x-axis direction is shown as an example. The origin of the x axis corresponds to the position of the imaging optical axis.

  The light transmittance distribution 210 on the pupil plane by the first optical filter 114 has an upward convex distribution. For example, the light transmittance has a maximum value at the position of the imaging optical axis, and decreases monotonously toward the periphery. The light transmittance distribution 210 has a symmetrical shape with respect to the position of the imaging optical axis. The light transmittance distribution 220 on the pupil plane by the second optical filter 124 has a downward convex distribution. The light transmittance has a minimum value at the position of the imaging optical axis and monotonously increases toward the periphery. The light transmittance distribution 220 has a symmetrical shape with respect to the position of the imaging optical axis. Therefore, the second optical system 125 can have a high MTF in a relatively high spatial frequency region as compared with the first optical system 115.

  FIG. 3 shows an example of the MTFs of the first optical system 115 and the second optical system 125. Here, the dependence of MTF on the spatial frequency component in the x direction is shown. The MTF shown in this figure is an MTF for light from a specific object position.

  The first optical system 115 has an MTF 310 higher than the second optical system 125 in the first spatial frequency region where the spatial frequency is greater than 0 and lower than f1. For this reason, the first imaging system 101 can generate an imaging signal having a higher SN ratio than the second imaging system 102 in the first spatial frequency domain.

  On the other hand, the second optical system 125 has an MTF 320 higher than that of the first optical system 115 in a second spatial frequency region higher than the spatial frequency f1. For this reason, the second imaging system 102 can generate an imaging signal having a higher SN ratio than the first imaging system 101 in the second spatial frequency domain. Further, the second optical system 125 can be regarded as an optical system having higher resolution than the first optical system 115 in the second spatial frequency region. As described above, the second optical system 125 has an optical response in the first spatial frequency region lower than that of the first optical system 115 and an optical response in the second spatial frequency higher than that of the first optical system 115.

  In FIGS. 2 and 3, an example of the light transmittance distribution and the MTF is schematically shown for the sake of easy understanding. The light transmittance characteristics of the first optical filter 114 and the second optical filter 124 and the MTF characteristics of the first optical system 115 and the second optical system 125 are not limited to the illustrated characteristics. In order to make the MTF of the second optical system 125 lower in the specific spatial frequency region than the MTF of the first optical system 115 and higher in the spatial frequency region different from the specific spatial frequency region, What is necessary is just to design the light transmittance characteristic by the optical filter 124.

  By the way, even in the same optical system, the MTF varies depending on the distance to the object point. For this reason, the spatial frequency f1 at which the MTFs match between the first optical system 115 and the second optical system 125 may differ depending on the distance to the object point. When the spatial frequency f1 varies depending on the distance to the object point, it is preferable that the image generation unit 150 appropriately select the spatial frequency region to be extracted according to the distance to the object point. Even in the same optical system, the MTF may differ depending on the distance from the optical axis to the object point. The distance from the optical axis to the object point corresponds to the image area on the image side. Therefore, it is preferable that the image generation unit 150 appropriately select a spatial frequency region to be extracted according to at least one of the distance to the object point and the image region.

  FIG. 4 shows an example of a block configuration of the image generation unit 150. The image generation unit 150 includes a frequency filter unit 410, an image synthesis unit 420, and a corresponding region specifying unit 430. The frequency filter unit 410 includes a low-pass filter unit 411 and a high-pass filter unit 412.

  An imaging signal is supplied from the readout unit 140 to the frequency filter unit 410. Among these, the imaging signal output from the first imaging system 101 is supplied to the low-pass filter unit 411. The low-pass filter unit 411 extracts a frequency component in the first spatial frequency domain from the imaging signal output from the first imaging system 101 and supplies the extracted frequency component to the image synthesis unit 420. The high-pass filter unit 412 is supplied with the imaging signal output from the second imaging system 102. The high-pass filter unit 412 extracts a frequency component in the second spatial frequency domain from the imaging signal output from the second imaging system 102 and supplies the extracted frequency component to the image synthesis unit 420.

  The image synthesis unit 420 synthesizes the spatial frequency domain signal extracted by the low-pass filter unit 411 and the spatial frequency domain signal extracted by the high-pass filter unit 412 to generate an image signal corresponding to one image. To do. The image composition unit 420 outputs the generated image signal to the recording unit 180.

  Here, as described above, the first imaging system 101 and the second imaging system 102 are provided with the imaging optical axes separated from each other. For this reason, the first imaging system 101 and the second imaging system 102 capture the same subject from different viewpoints. For this reason, image displacement occurs due to so-called parallax, and the image positions do not completely coincide even with the same subject. The image shift amount of each image region is substantially determined according to the separation distance between the first imaging system 101 and the second imaging system 102 and the distance to the subject. Therefore, if the image shift amount is specified, the distance to the subject is specified for each image region based on the specified image shift amount and the separation distance between the first image pickup system 101 and the second image pickup system 102. can do.

  The corresponding area specifying unit 430 should capture the same subject with the first imaging system 101 and the second imaging system 102 based on the positions of the imaging optical axes of the first imaging system 101 and the second imaging system 102. Specify the image area. Specifically, the corresponding area specifying unit 430 specifies a corresponding area that is an area in which the same subject is imaged, from the image contents of each image indicated by the imaging signal output from the reading unit 140. More specifically, the corresponding area specifying unit 430 can specify the corresponding area by specifying the corresponding point by matching the corresponding points between the images. The corresponding area specifying unit 430 supplies information indicating the specified corresponding area to the image composition unit 420. The image composition unit 420 is imaged by the second imaging system 102 and the spatial frequency component in the first spatial frequency region of the subject image captured by the first imaging system 101 for each corresponding region identified by the corresponding region identifying unit 430. By synthesizing the spatial frequency components in the second spatial frequency region of the subject image, the subject image is generated. Accordingly, the image composition unit 420 can generate a so-called image from one viewpoint, that is, an image in which parallax is substantially eliminated.

  By the way, the difference in the position of the corresponding area or the difference in the position of the corresponding point between the images corresponds to the image shift amount. Therefore, the corresponding area specifying unit 430 is based on the difference between the positions of the specified corresponding areas or the positions of the corresponding points, and the positions of the imaging optical axes of the first imaging system 101 and the second imaging system 102, respectively. The distance to the subject can be specified for each image area. The corresponding area specifying unit 430 supplies distance information indicating the distance to the subject specified for each image area to the frequency filter unit 410. The frequency filter unit 410 appropriately selects the first spatial frequency region and the second spatial frequency region according to the distance to the subject specified for each image region. The selection of the spatial frequency domain will be described later.

  As described above, the image shift amount of each image area correlates with the separation distance between the first imaging system 101 and the second imaging system 102 and the distance to the subject, so that the distance to the subject is specified. If so, the image shift amount can be specified. The corresponding area is specified by the image shift amount. The corresponding area specifying unit 430 directly specifies the corresponding area by matching the corresponding points described above, and based on the positions of the imaging optical axes of the first imaging system 101 and the second imaging system 102 and the distance measurement information, A corresponding area may be specified. The distance measurement information may be acquired from, for example, an external distance measurement device included in the imaging apparatus 100.

  FIG. 5 shows an example of frequency filter processing in the frequency filter unit 410. The low-pass filter unit 411 performs filter processing in the frequency domain on the imaging signal output from the first imaging system 101 based on the spatial frequency f1. Specifically, the low-pass filter unit 411 converts the imaging signal output from the first imaging system 101 into a spatial frequency domain signal. For example, it is assumed that a signal in the spatial frequency domain shown in FIG. The low-pass filter unit 411 cuts the spatial frequency domain signal component higher than the spatial frequency f1 from the obtained spatial frequency domain signal, and passes the spatial frequency component of the spatial frequency domain A below the spatial frequency f1. And output to the image composition unit 420.

  The high-pass filter unit 412 performs filter processing in the frequency domain on the imaging signal output from the second imaging system 102 based on the spatial frequency f1. Specifically, the high-pass filter unit 412 converts the imaging signal output from the second imaging system 102 into a spatial frequency domain signal. For example, it is assumed that a signal in the spatial frequency domain shown in (b) of this figure is obtained from the imaging signal. The high-pass filter unit 412 cuts a signal component in the spatial frequency domain below the spatial frequency f1 from the obtained spatial frequency domain signal and passes the spatial frequency component in the spatial frequency domain B higher than the spatial frequency f1. To the image composition unit 420.

  As described above, the value of the spatial frequency f1 may vary depending on the distance to the object point or the image area. Therefore, the frequency filter unit 410 controls the frequency filter process according to the distance to the subject or the distance to the subject and the image area.

  Specifically, the frequency filter unit 410 stores in advance values of a plurality of spatial frequencies f1 in association with the distance to the subject. Based on the distance information for each image area supplied from the corresponding area specifying unit 430, the value of the spatial frequency f1 is determined for each image area. For example, for each of the plurality of image regions, the frequency filter unit 410 has the spatial frequency f1 associated with the distance closest to the distance to the subject indicated by the distance information among the plurality of spatial frequencies f1 stored in advance. Specify the value. The low-pass filter unit 411 and the high-pass filter unit 412 select a spatial frequency region A determined by the specified value of the spatial frequency f1 for each image region, and perform a filtering process on the imaging signal. As a result, the spatial frequency region A can be appropriately selected according to the distance to the object point and used for synthesis.

  Further, the frequency filter unit 410 may store a plurality of values of the spatial frequency f1 in advance in association with the distance to the subject for each of the plurality of image regions. Then, the frequency filter unit 410 determines the value of the spatial frequency f1 for each image region based on the distance information for each image region. For example, for each of the plurality of image regions, the frequency filter unit 410 has a space associated with the distance closest to the distance to the subject among the plurality of spatial frequencies f1 stored in advance for each image region. The value of the frequency f1 is specified. The low-pass filter unit 411 and the high-pass filter unit 412 select a spatial frequency region A determined by the specified value of the spatial frequency f1 for each image region, and perform a filtering process on the imaging signal. Thereby, the spatial frequency f1 can be appropriately selected according to the distance to the object point and the image region and used for the synthesis.

  FIG. 6 illustrates an example of image processing for images captured by the first imaging system 101 and the second imaging system 102. Here, it is assumed that the first imaging system 101 is located on the right side of the second imaging system 102 when the subject is viewed from the imaging apparatus 100. When a subject that is not infinite is imaged, the images of the subject are located in different image areas in the image 610 captured by the first imaging system 101 and the image 620 captured by the second imaging system 102.

  As described above, the corresponding region specifying unit 430 specifies the corresponding region 612 and the corresponding region 622 from the images 610 and 620 by corresponding point matching or the like. Since the first imaging system 101 is located on the right side of the second imaging system 102, the corresponding area 612 is located at a position shifted to the left side of the corresponding area 622. Here, for easy understanding, the subject imaged in the corresponding region 612 and the corresponding region 622 is a subject existing at the midpoint of the imaging optical axis of the first imaging system 101 and the imaging optical axis of the second imaging system 102. Suppose that

  The low-pass filter unit 411 filters the spatial frequency component of the corresponding region 612 with a low-pass filter 614 that selectively passes through the spatial frequency region A. The high-pass filter unit 412 filters the spatial frequency component of the corresponding region 622 with a high-pass filter 624 that selectively passes through the spatial frequency region B.

  The image synthesizing unit 420 obtains the image from one viewpoint by synthesizing the spatial frequency component filtered by the low-pass filter 614 and the spatial frequency component filtered by the high-pass filter 624 for each corresponding region. A power image 630 is generated. As an example, when the image synthesis unit 420 generates an image 630 whose viewpoint is an intermediate point between the imaging optical axis of the first imaging system 101 and the imaging optical axis of the second imaging system 102, the image synthesis unit 420 An image signal obtained by synthesizing the spatial frequency components of the area is generated as an image signal in an intermediate area of the corresponding area. Specifically, when generating an image 630 whose viewpoint is the midpoint between the imaging optical axis of the first imaging system 101 and the imaging optical axis of the second imaging system 102, the image composition unit 420 includes the corresponding region 612 and the corresponding region. An image signal obtained by combining the spatial frequency components of the region 622 is generated as an image signal of the central image region 632 in the image 630.

  Rather than generating an image 630 whose viewpoint is the midpoint between the imaging optical axis of the first imaging system 101 and the imaging optical axis of the second imaging system 102, it is generated from one of the first imaging system 101 and the second imaging system 102. A captured image may be generated. For example, when generating an image picked up from the second image pickup system 102, the image composition unit 420 uses an image signal obtained by combining the spatial frequency components of the corresponding region 612 and the corresponding region 622 as an image signal of the corresponding region 622. Just generate.

  FIG. 7 shows another example of the lens arrangement. From FIG. 1 to FIG. 6, the first imaging system 101 and the second imaging system 102 are respectively R light belonging to the red wavelength region, G light belonging to the green wavelength region, and B light belonging to the blue wavelength region. Suppose you want to image a subject. That is, the first imaging system 101 and the second imaging system 102 each captures a color image of three colors.

  In this lens arrangement, it is assumed that the first imaging system 101 and the second imaging system 102 are imaging systems that image a subject with G light. That is, an image for low spatial frequency and an image for high spatial frequency are acquired with G light. A third imaging system for imaging the subject with R light and B light is further provided. This figure shows an example of lens arrangement viewed from the object side. In this lens arrangement, a G light first lens 710, a G light second lens 720, and an RB light lens 730 are provided inside the lens barrel 700.

  The first lens 710 for G light replaces the first lens 110 of the first optical system 115. The second lens 720 for G light replaces the second lens 120 of the second optical system 125. Other optical elements included in the first optical system 115 and the second optical system 125 can be applied to this arrangement example. That is, the first optical system 115 and the second optical system 125 according to this arrangement example are shown in FIGS. 1 to 6 except that the imaging lens has an imaging characteristic for imaging the G light on the light receiving unit 109. The first optical system 115 and the second optical system 125 described have substantially the same optical characteristics.

  According to this lens arrangement, in addition to the first lens for G light 710 and the second lens for G light 720, an RB light lens 730, which is a lens for imaging R light and B light, is further provided. The imaging optical axis of the RB light lens 730 is provided apart from the imaging optical axes of the first G light lens 710 and the second G light lens 720 in the y direction. The RB light lens 730 functions as an imaging lens of a third imaging system that captures an image of the subject with R light and B light. Of the optical elements of the first optical system 115, the optical elements other than the first lens 110 can be further applied as optical elements of the third imaging system. The third imaging system may not have an apodization filter corresponding to the first optical filter 114.

  FIG. 8 shows an example of the configuration of the light receiving unit 809 viewed from the object side. The light receiving unit 809 corresponds to the light receiving unit 109 described with reference to FIGS. 1 to 6.

  The G light first light receiving region 819 and the G light second light receiving region 829 correspond to the first light receiving unit 119 and the second light receiving unit 129, respectively. The first color filter array 116 and the second color filter array 126 are formed of color filters that selectively transmit G light. As the image sensor that receives light that has passed through each color filter, an image sensor similar to the first image sensor array 118 and the second image sensor array 128 described with reference to FIGS. 1 to 6 is applied. The first light receiving region 819 for G light and the second light receiving region 829 for G light are only required to selectively receive G light, and two-dimensional wavelength separation is not performed within the imaging surface. The fact that wavelength separation is not performed within the imaging plane is schematically shown by a dotted line in the figure.

  The RB light receiving region 839 functions as a light receiving unit of the third imaging system. The RB light receiving region 839 is formed by arranging a color filter that selectively transmits R light and a color filter that selectively transmits B light in a predetermined pattern. The light receiving region 839 for RB light includes a plurality of imaging elements provided corresponding to the plurality of color filters. The RB light receiving region 839 is wavelength-separated into R light and B light within the imaging surface. The two-dimensional wavelength separation in the imaging plane is schematically shown by a solid line in this figure.

  According to this example, an image for a low spatial frequency and an image for a high spatial frequency can be captured with high resolution by G light. Therefore, a G image signal having a high S / N ratio over a wide spatial frequency region can be obtained. In addition, the resolution of the G image signal can be increased. Since the visual sensitivity of the human eye is highest in the green wavelength region, the image quality can be effectively enhanced by increasing the SN ratio and resolution for the G image signal.

  As described above, according to the imaging apparatus 100 described with reference to FIGS. 1 to 8, when imaging is performed with one imaging system having a specific spatial frequency response, or with a plurality of imaging systems having substantially the same spatial frequency response. Compared with the case of imaging, an image with a high SN ratio and high resolving power can be provided. In addition, the imaging signal that substantially contributes to the image signal in the high spatial frequency region is the imaging signal of the second imaging system 102. Therefore, the first imaging system 101 can allow false resolution in a high spatial frequency region. On the other hand, the imaging signal that substantially contributes to the image signal in the low spatial frequency region is the imaging signal of the first imaging system 101. Therefore, in the second imaging system 102, the MTF may be low and the SN ratio may be low in the low spatial frequency region. According to the imaging apparatus 100, since a plurality of imaging systems can share a so-called spatial frequency region, it is possible to provide an image that is properly resolved from a low range to a high range.

  In the above description, in order to make the description easy to understand, the effective spatial frequency response is made different among the plurality of imaging systems by the apodization effect by the first optical filter 114 and the second optical filter 124. The spatial frequency response can be varied depending on at least one of the effective aperture and magnification of the imaging lens in addition to the apodization effect.

  For example, by making the second lens 120 a lens having a larger effective aperture than the first lens 110, the response of the second imaging system 102 in a relatively high spatial frequency region can be enhanced compared to the first lens 110. For example, by making the opening of the second diaphragm portion 122 larger than the opening of the first diaphragm portion 112, the second lens 120 can be made a lens having a larger effective aperture than the first lens 110. In particular, the cutoff frequency of the second imaging system 102 can be extended to a high range. In addition, by making the second lens 120 a lens having a larger magnification than the first lens 110, the effective response of the second imaging system 102 in a relatively high spatial frequency region can be improved more than that of the first lens 110. it can. Thus, the spatial frequency responses of a plurality of imaging systems can be differentiated using the F value as an index.

  In the above description, in order to make the description easy to understand, the plurality of imaging elements included in the first light receiving unit 119 are arranged at the same pitch as the plurality of imaging elements included in the second light receiving unit 129. The imaging signals are read out from the plurality of imaging elements included in the first light receiving unit 119 and the plurality of imaging elements included in the second light receiving unit 129, respectively. Here, the spatial frequency response of the entire imaging system is also affected by subject light sampling characteristics by the light receiving unit. Therefore, the sampling characteristics of the light receiving unit may be varied according to the spatial frequency region that each imaging system should be in charge of.

  For example, the reading pitch of the image sensor may be different between the first imaging system 101 and the second imaging system 102. Specifically, the reading unit 140 reads from the plurality of imaging elements included in the first light receiving unit 119 at a pitch lower than the reading pitch for the plurality of imaging elements included in the second light receiving unit 129. The reading unit 140 may perform thinning readout from the plurality of imaging elements included in the first light receiving unit 119 at a higher thinning rate than the plurality of imaging elements included in the second light receiving unit 129.

  In addition, the readout unit 140 may perform binning readout with respect to a plurality of image sensors included in the first light receiving unit 119 in order to change the readout pitch. For example, the reading unit 140 may perform binning reading on the plurality of imaging elements included in the first light receiving unit 119, while not performing binning reading on the plurality of imaging elements included in the second light receiving unit 129. When the reading unit 140 performs binning reading for a plurality of image sensors included in the second light receiving unit 129, the reading unit 140 sets the number of image sensors as one unit of binning reading to the second light receiving unit than the first light receiving unit 119. You can increase at 129.

  In addition, the first imaging system 101 and the second imaging system 102 may have different pitches of the imaging element arrays. Specifically, the plurality of imaging elements included in the first light receiving unit 119 may be arranged at a lower pitch than the plurality of imaging elements included in the second light receiving unit 129.

  Since the first imaging system 101 is in charge of a relatively low first spatial frequency region, even if the sampling characteristics of the first imaging system 101 are lowered, the response in the first spatial frequency region is not significantly reduced. . Moreover, even if false resolution occurs in the second spatial frequency region by lowering the sampling characteristics, the image signal generated by the image generation unit 150 is not substantially affected. On the other hand, by reducing the sampling characteristics of the first imaging system 101, the readout speed of the imaging signal can be increased.

  The imaging device 100 can also include four or more imaging systems. The imaging apparatus 100 only needs to include at least two imaging systems having different responses in a specific spatial frequency region and other spatial frequency regions as described above.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

100 imaging device 101 first imaging system 102 second imaging system 109 light receiving unit 110 first lens 112 first diaphragm unit 114 first optical filter 115 first optical system 116 first color filter array 118 first imaging element array 119 first Light receiving unit 120 second lens 122 second diaphragm unit 124 second optical filter 125 second optical system 126 second color filter array 128 second image sensor array 129 second light receiving unit 140 reading unit 150 image generating unit 180 recording unit 210, 220 Light transmittance distribution 310, 320 MTF
430 Corresponding region specifying unit 410 Frequency filter unit 411 Low-pass filter unit 412 High-pass filter unit 420 Image composition unit 610, 620, 630 Image 612, 622 Corresponding region 614 Low-pass filter 624 High-pass filter 632 Image region 700 Lens barrel 710 G light first lens 720 G light second lens 730 RB light lens 809 light receiving portion 819 G light first light receiving region 829 G light second light receiving region 839 RB light receiving region

Claims (10)

A first imaging system;
A second imaging system having a lower response in the first spatial frequency domain than the first imaging system and a higher response in the second spatial frequency domain than the first imaging frequency system than the first imaging system;
A spatial frequency component in a first spatial frequency region of an image of a subject imaged by the first imaging system and a spatial frequency component in a second spatial frequency region of the image of the subject imaged by the second imaging system are used. And an image generation unit that generates an image of the subject. The first imaging system has a first optical system,
The second imaging system includes a second optical system having an optical response in the first spatial frequency region that is lower than that of the first optical system and an optical response in the second spatial frequency that is higher than that of the first optical system. Item 2. The imaging module according to Item 1. The first optical system and the second optical system each have an optical filter that gives a light transmittance distribution on the pupil plane,
The first optical filter of the first optical system has a higher light transmittance in the central region than in the peripheral region of the pupil plane,
The imaging module according to claim 2, wherein the second optical filter included in the second optical system has a higher light transmittance in a peripheral region than in a central region of the pupil plane. The first optical system has a first lens,
The imaging module according to claim 2, wherein the second optical system includes a second lens having an effective aperture larger than that of the first lens. The first optical system has a first lens,
The imaging module according to claim 2, wherein the second optical system includes a second lens having a larger magnification than the first lens. The first imaging system includes a first light receiving unit including a plurality of imaging elements that receive light from the subject through the first optical system,
The second imaging system includes a second light receiving unit including a plurality of imaging elements that receive light from the subject through the second optical system,
The imaging module according to claim 2, wherein the plurality of imaging elements included in the first light receiving unit are arranged at a pitch lower than that of the plurality of imaging elements included in the second light receiving unit. The first imaging system includes a first light receiving unit including a plurality of imaging elements that receive light from the subject through the first optical system,
The second imaging system includes a second light receiving unit including a plurality of imaging elements that receive light from the subject through the second optical system,
The imaging module is
6. The readout unit according to claim 2, further comprising: a readout unit that reads out from the plurality of imaging elements included in the first light receiving unit at a readout pitch lower than that of the plurality of imaging elements included in the second light receiving unit. Imaging module. The imaging module according to claim 6 or 7, wherein the plurality of imaging elements respectively included in the first light receiving unit and the second light receiving unit are provided on the same plane. The image generation unit
Corresponding region that specifies an image region in which the same subject should be imaged by the first imaging system and the second imaging system based on the positions of the imaging optical axes of the first imaging system and the second imaging system. A specific part,
For each image region, a spatial frequency component in the first spatial frequency region of the subject image captured by the first imaging system and a second spatial frequency region of the subject image captured by the second imaging system. The imaging module according to claim 1, further comprising: an image synthesis unit that generates an image of the subject by synthesizing the spatial frequency components in.   An imaging apparatus comprising the imaging module according to claim 1, wherein the imaging module images the subject.

Images