JP2015103885A - Compound-eye imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce a subject positional dislocation that may be generated when switching an angle of view, when imaging a stereoscopic subject from different view points in image formation optical systems with different angles of view.SOLUTION: A compound-eye imaging device 1 includes: a plurality of image formation optical systems which are disposed while being separated from each other in a direction orthogonal to optical axes; and imaging means 10 with which an imaging area for imaging the subject through each of the plurality of image formation optical systems is provided for each image formation optical system. The plurality of image formation optical system include: a plurality of first image formation optical systems 110a-110d each including a first angle of view; and at least one second image formation optical system 120a-120d each including a second angle of view that is wider than the first angle of view. In an optical axis direction view, an optical axis of the at least one second image formation optical system is disposed within a first area surrounded by a line connecting positions of the optical axes of the plurality of first image formation optical systems.

Description

  The present invention relates to a compound-eye imaging device having a plurality of imaging optical systems and having an imaging region for performing imaging (photoelectric conversion) for each imaging optical system.

  The compound-eye imaging device realizes miniaturization of individual lenses by dividing one imaging optical system (lens) of a normal imaging device into a plurality of parts. Patent Document 1 discloses a compound eye imaging apparatus that has two types of lenses having different angles of view (focal lengths) and images the same subject at different angles of view. In this compound-eye imaging device, a partial area of a wide image obtained by an image sensor provided for a short focus lens having a wide angle of view is obtained by an image sensor provided for a long focus lens having a narrow angle of view. Synthesize the zoomed-up image. Thereby, although the resolution of the partial area is high and the resolution of the other areas is low, an image having a wide angle of view as a whole can be obtained.

  However, in the compound eye imaging device disclosed in Patent Document 1, the short focus lens and the long focus lens are arranged apart from each other in a direction orthogonal to their optical axes. For this reason, there is a parallax between the wide image obtained by the short focus lens and the zoomed-up image obtained by the long focus lens, and the subject position is deviated in the composite image.

  Patent Document 2 discloses a compound eye imaging device capable of reducing subject position shift when images obtained by imaging from a plurality of viewpoints are combined. In this compound-eye imaging device, a plurality of second lenses having optical characteristics such as F-number and the like different from those of the first lens with the optical axis position of the first lens (or the center of gravity of the optical axes of the plurality of first lenses) as the center They are arranged on concentric circles. Then, the plurality of images obtained by imaging through the plurality of second lenses are combined while being shifted from each other so that the same subject areas included in these images overlap, and the first lens is placed on the subject area of the composite image. The subject area of the image obtained by passing the image is combined.

  Note that Patent Document 3 discloses an image generation apparatus that can generate an intermediate viewpoint image that is smooth and has no sense of incongruity when switching and displaying images obtained from a plurality of viewpoints. This image generation device acquires the position on the floor surface of the same subject included in the images from a plurality of images taken from a plurality of viewpoints, determines an intermediate point between the positions, and determines the intermediate point An intermediate viewpoint image in which a subject exists is generated.

JP 2005-303694 A JP 2012-44459 A JP 2008-217243 A

  The lens arrangement and the image composition method employed by the compound-eye imaging device disclosed in Patent Document 2 can obtain a good subject region image quality while reducing subject positional deviation for a planar subject. Is possible. However, for a three-dimensional subject, since the subject is imaged as if it were geometrically deformed from the respective viewpoints of the first and second lenses, the images obtained by the imaging are simply synthesized. It is not possible. Moreover, Patent Document 2 describes in detail the lens arrangement and the image composition method when the first and second lenses have the same angle of view, but the first and second lenses have different angles of view. The lens arrangement and image composition method are not clearly described. When the first and second lenses have different angles of view, image synthesis cannot be performed by a simple method as described in Patent Document 2.

  Furthermore, in the image generation apparatus disclosed in Patent Document 3, it is assumed that a floor surface serving as a reference for the position of the subject exists, but there may be a case where the floor surface does not exist in a general imaging environment. In many cases, an intermediate viewpoint image cannot be generated by the image generation apparatus.

  The present invention provides a compound-eye imaging device that can easily reduce subject position shift that occurs when the angle of view is switched when imaging a stereoscopic subject from different viewpoints with an imaging optical system having different angles of view. .

  A compound eye imaging device according to one aspect of the present invention includes a plurality of imaging optical systems that are spaced apart from each other in a direction orthogonal to the respective optical axes, and an object that passes through each of the plurality of imaging optical systems. An imaging region for imaging is provided with an imaging unit provided for each imaging optical system. The plurality of imaging optical systems includes a plurality of first imaging optical systems each having a first angle of view, and at least one second imaging having a second angle of view wider than the first angle of view. And an optical system. When viewed in the optical axis direction, the at least one second imaging optical system has a first area surrounded by a line connecting the optical axis positions of the plurality of first imaging optical systems. An optical axis is arranged.

  According to the present invention, since the first and second imaging optical systems having different angles of view are provided, zooming is possible without providing a zoom mechanism that moves the optical element for zooming in the optical axis direction. A small (thin) imaging device can be realized. In addition, the optical axis of the second imaging optical system is arranged in a first region surrounded by a line connecting the positions of the optical axes of the plurality of first imaging optical systems. For this reason, it becomes possible to change each viewpoint position of the image obtained through the plurality of first imaging optical systems with high accuracy on the basis of the image obtained through the second imaging optical system. For this reason, even when a three-dimensional subject is imaged, it is possible to reduce subject position deviation that occurs when the angle of view is switched.

The figure which looked at the image formation optical part of the compound eye imaging device which is Example 1 of the present invention from the optical axis direction. 1 is a block diagram illustrating a configuration of a compound eye imaging apparatus according to Embodiment 1. FIG. FIG. 3 is a diagram illustrating an image obtained by imaging with the compound-eye imaging apparatus according to the first embodiment. 3 is a flowchart illustrating a flow of image composition processing performed by the compound eye imaging apparatus according to the first embodiment. FIG. 3 is a diagram illustrating an image used for wide-angle image synthesis in the compound-eye imaging apparatus according to the first embodiment. FIG. 3 is a diagram illustrating a corresponding region extraction method used in the compound eye imaging apparatus according to the first embodiment. FIG. 3 is a diagram illustrating an image used for telephoto side image synthesis in the compound-eye imaging apparatus according to the first embodiment. 3 is a flowchart illustrating a flow of intermediate angle-of-view image generation processing performed by the compound eye imaging device according to the first embodiment. 6 is a flowchart illustrating distance information calculation processing in the compound eye imaging apparatus according to the first embodiment. The figure which looked at the image formation optical part of the compound eye imaging device which is Example 2 of the present invention from the optical axis direction. The figure which looked at the image formation optical part of the compound eye imaging device which is Example 3 of the present invention from the optical axis direction. The figure which looked at the image formation optical part of the compound-eye imaging device which is Example 4 of the present invention from the optical axis direction. The figure explaining the imaging from a different viewpoint. The figure explaining the subject position shift in an image.

  Embodiments of the present invention will be described below with reference to the drawings.

  A compound-eye imaging apparatus according to an embodiment of the present invention includes a zoom mechanism that moves a zooming optical element in the optical axis direction by including a plurality of single-focus imaging optical systems having different angles of view (focal lengths). There is no need to provide the device, and the device is made thinner while ensuring a high zoom ratio. The compound-eye imaging apparatus of the present embodiment uses an image obtained by imaging through a plurality of imaging optical systems having discretely different angles of view (hereinafter simply referred to as being obtained through the imaging optical system). Thus, it is possible to generate an image whose angle of view continuously changes. A digital zoom that can obtain a pseudo zooming effect by trimming a part of an image obtained through an imaging optical system with a certain angle of view and expanding the trimmed range to a predetermined size is also installed in general imaging devices. Has been. The compound-eye imaging apparatus of the present embodiment enables continuous zooming by interpolating between discrete field angles of a plurality of imaging optical systems by digital zoom processing. In the present embodiment, this digital zoom processing is also referred to as intermediate angle of view image generation processing.

  Further, the compound-eye imaging apparatus of the present embodiment combines a telephoto image obtained through an imaging optical system corresponding to a telephoto lens with a part of an image obtained by digital zoom. As a result, it is possible to generate a composite image having a wide field angle although the resolution of the portion of the telephoto image is high and the resolution of other portions is low. In the present embodiment, this image synthesis is referred to as different angle-of-view image synthesis processing.

  In these intermediate angle-of-view image generation processing and different angle-of-view angle image synthesis processing, an image used to synthesize an image with the desired angle of view is acquired through an imaging optical system having an angle of view close to that angle of view. For this reason, it is possible to generate a high-quality image while obtaining a high zoom ratio as compared with a case where the angle of view is simply changed only by digital zoom from an image obtained through an imaging optical system having one angle of view.

  However, the plurality of imaging optical systems provided in the compound-eye imaging apparatus of the embodiment are arranged apart from each other in the direction orthogonal to the respective optical axes, and the subject is imaged through each of these imaging optical systems. An imaging region (photoelectric conversion region) for performing the above is provided for each imaging optical system. For this reason, there is a parallax between images obtained through imaging optical systems having different angles of view. Then, due to this parallax, the position of the subject in the image greatly moves with the change of the angle of view. The cause and principle will be described with reference to FIGS.

  In FIG. 13, the imaging devices C1, C2, and C3 are spaced apart from each other by a certain distance (baseline length), and a subject A that is separated from each imaging device by a subject distance La and a subject B that is separated by a subject distance Lb. Is shown. FIG. 14 shows images obtained by imaging with the imaging devices C1, C2, and C3, respectively. As can be seen from these figures, the positions of the subject A and the subject B in the image are shifted by different amounts due to the geometric relationship according to the base line length and the subject distance. That is, the position of the subject on the image moves according to the viewpoint.

  For this reason, the compound-eye imaging apparatus of the present embodiment has the following configuration so that the angle of view can be changed while reducing the deviation of the subject position in the image using an imaging optical system having a different angle of view and viewpoint. . That is, in the present embodiment, as the plurality of imaging optical systems, a plurality of first imaging optical systems each having a first field angle and at least a second field angle wider than the first field angle. One second imaging optical system is provided. Then, when viewed in the optical axis direction, the optical axis of the second imaging optical system is disposed in the first region surrounded by a line connecting the positions of the optical axes of the plurality of first imaging optical systems. ing. According to such a configuration, an image obtained through the plurality of first imaging optical systems on the basis of the image obtained through the second imaging optical system in which the optical axis is disposed in the first region. Each viewpoint position can be changed with high accuracy. For this reason, it is possible to easily reduce the subject position shift accompanying the change in the angle of view.

Further, in the present embodiment, different angles of view of the plurality of imaging optical systems (including not only the first and second angles of view, but also the third angle of view and the fourth angle of view between them). Among these, it is desirable that any one of the angle of view W and the angle of view Wn next to the angle of view W satisfy the condition of the following expression (1).
1.1 ≦ W / Wn ≦ 3. . . (1)
If the value of W / Wn is below the lower limit of the formula (1), a large number of imaging optical systems are required to achieve a high zoom ratio of the compound-eye imaging device, and the device becomes large. It is not preferable. Further, if the value of W / Wn exceeds the upper limit value of Expression (1), it is difficult to maintain a high resolution even when a pixel-shifted composite image described later is generated, and the image quality after zooming deteriorates, which is not preferable.

  Hereinafter, specific configuration examples of this embodiment will be shown as Embodiments 1 to 4.

  FIG. 1 shows the imaging optical unit 100 of the compound-eye imaging apparatus that is Embodiment 1 of the present invention when viewed from the optical axis direction (that is, viewed from the optical axis direction). The imaging optical unit 100 includes a plurality of imaging optical systems 110a, 110b, 110c, 110d, 120a, 120b, 120c, and 120d. The plurality of image forming optical systems 110a to 110d and 120a to 120d are arranged apart from each other in a two-dimensional direction orthogonal to the respective optical axes. The optical axes of the imaging optical systems 110a to 110d and 120a to 120d extend in parallel to each other.

  The imaging optical systems 110a to 110d are first imaging optical systems having a first angle of view (focal length) θ, and hereinafter, the first imaging optical systems 110a to 110d are collectively referred to as a first. Also called an imaging optical system group. The first angle of view θ corresponds to the telephoto end angle of view which is the narrowest angle of view in the compound-eye imaging device of the present embodiment. Further, the imaging optical systems 120a to 120d have a second imaging with a second field angle (2θ which is twice the first field angle θ in this embodiment) wider than the first field angle θ. Hereinafter, these second imaging optical systems 120a to 120d are collectively referred to as a second imaging optical system group. The second field angle 2θ corresponds to the wide-angle end field angle that is the widest field angle in the compound-eye imaging apparatus of the present embodiment. In the present embodiment, the two first imaging optical systems arranged in the vertical direction and the two second imaging optical systems arranged in the vertical direction are in the horizontal direction (from left to right in FIG. 1). Alternatingly arranged.

  FIG. 2 shows the overall configuration of the compound eye imaging apparatus 1 of the present embodiment. The compound eye imaging apparatus 1 includes the imaging optical unit 100, the imaging element unit (imaging unit) 10, the A / D converter 11, the image processing unit 12, the information input unit 16, and the imaging control unit 17 described above. And an image recording medium 18, a system controller 19, and a display unit (display means) 20. Further, the compound eye imaging apparatus 1 also includes a distance information calculation unit (distance calculation unit) 21.

  The compound-eye imaging device 1 may be an imaging optical system-integrated imaging device in which the imaging optical unit 100 is integrally provided, or the imaging optical unit 100 may be attached to and detached from the imaging device main body (replacement). A possible imaging optical system exchange type imaging apparatus may be used. In this embodiment, a case where the imaging optical system is integrated will be described.

  The imaging element unit 10 includes eight imaging elements 10a to 10a that constitute imaging regions corresponding to the eight imaging optical systems 110a to 110d and 120a to 120d described above (that is, provided for each imaging optical system). 10h. Each imaging device photoelectrically converts a subject image (optical image) formed by a corresponding imaging optical system and outputs an analog imaging signal. The analog imaging signal is converted into a digital imaging signal by the A / D converter 11, and the digital imaging signal is input to the image processing unit 12. The image processing unit 12 performs various image processing such as pixel interpolation processing and color conversion processing on the digital imaging signal to generate an image. As a result, eight images can be generated by imaging (through each) of the eight imaging optical systems 110a to 110d and 120a to 120d. The image processing unit 12 also performs digital zoom processing and the like on the generated image. The image processed by the image processing unit 12 is sent to the system controller 19.

  The image processing unit 12 includes an image composition unit (image composition unit) 13, an intermediate angle of view image generation unit (image generation unit) 14, and a preview image generation unit 15.

  The image synthesizing unit 13 synthesizes four images obtained through the first imaging optical systems 110a to 110d with the pixels shifted from each other, and a first synthesized image (pixel shifted synthesized image) shown in FIG. 110 is generated. Further, the image composition unit 13 composes the four images obtained through the second imaging optical systems 120a to 120d by shifting the pixels from each other, and is wider than the first composite image 110 shown in FIG. A second composite image (pixel-shifted composite image) 120 with an imaging angle of view is generated.

  Here, in FIG. 1, the positions of the optical axes of the first imaging optical systems 110 a to 110 d are connected by a solid line (straight line) extending in the vertical direction and the horizontal direction, and a region surrounded by the solid line is represented by the first line. The area is Further, the positions of the optical axes of the second imaging optical systems 120a to 120d are connected by a broken line (straight line) extending in the vertical direction and the horizontal direction, and a region surrounded by the broken line is defined as a second region. In FIG. 1, the broken lines are slightly shifted from the positions of the optical axes of the second imaging optical systems 120 a to 120 d so that the solid lines and the broken lines are easy to see.

  As can be seen from this figure, in this embodiment, the optical axis of one second imaging optical system 120c among the plurality of second imaging optical systems 120a to 120d is arranged in the first region. . Hereinafter, the viewpoint from the second imaging optical system 120c is referred to as a reference viewpoint RP, and the second imaging optical system 120c is also referred to as a reference optical system. With this arrangement relationship, the reference optical system that becomes the reference viewpoint RP when the four images obtained through the first imaging optical systems 110a to 110d are combined to generate the first combined image. The image obtained through 120c can be used as a composite reference (reference image).

  The image synthesizing unit 13 uses corresponding point search methods such as a block matching method, which will be described later, using this reference image, and images obtained through an imaging optical system having the same angle of view or images having different angles of view. The images obtained through the image optical system are synthesized. As a result, a composite image corresponding to all the imaging angles of view can be virtually generated as an image obtained by imaging from the reference viewpoint RP. As a result, the imaging angle of view from the imaging state of the second composite image 120 through the second imaging optical system group to the imaging state of the first composite image 110 through the first imaging optical system group. The object position shift between the composite images 110 and 120 when the change (zooming) is performed can be reduced.

  The intermediate angle-of-view image generation unit 14 generates an intermediate angle-of-view image for interpolating the angle of view (intermediate angle of view) between different angles of view that the first and second imaging optical system groups have discretely. . As a method for generating an intermediate angle-of-view image, a super-resolution process that performs a resolution enhancement process using a plurality of images can be used. For super-resolution processing, ML (Maximum-Likelihood) method, MAP (Maximum A Posterior) method, POCS (Projection Onto Convex Set) method, IBP (Iterative Back Projection) method, LR (Lucy-Richardson) method, etc. are used. be able to. Further, in this embodiment, the telephoto image obtained through the first imaging optical system group corresponding to the telephoto lens is synthesized (inserted) into a partial region of the image obtained by the digital zoom. As a result, different angle-of-view image synthesis is performed in which the resolution of the partial area is high and the resolution of the other areas is low but an intermediate angle of view image can be obtained.

  The preview image generation unit 15 generates a preview image from the reference image obtained through the reference optical system 120c. When a change in the angle of view is instructed by the user, an image obtained through the reference optical system 120c serving as the reference viewpoint RP is trimmed and enlarged and displayed on the display unit 20, so that an image from the same viewpoint is always used as a preview image. Can be displayed.

  The information input unit 16 detects information input by the user selecting desired imaging conditions (aperture value, exposure time, etc.) and supplies the data to the system controller 19. Based on information from the system controller, the imaging control unit 17 moves a focus lens (not shown) included in each imaging optical system in the optical axis direction, and controls the aperture value of each imaging optical system. Furthermore, a necessary image is acquired by controlling the exposure time in the image sensor unit 10.

  The image recording medium 18 stores a plurality of still images and moving images, and stores a file header when configuring an image file.

  As described above, the display unit 20 displays a preview image, displays an image obtained by imaging, a menu screen, information on a currently selected angle of view (focal length), and the like. The display unit 20 includes a display element such as a liquid crystal panel.

  The distance information calculation unit 21 includes a reference image selection unit 22, a corresponding point extraction unit 23, and a parallax amount calculation unit 24.

  The reference image selection unit 22 selects a reference image for subject distance calculation from a plurality of images (parallax images) having parallax obtained through a plurality of imaging optical systems.

  The corresponding point extraction unit 23 extracts corresponding pixels (corresponding points) corresponding to each other between the reference image for subject distance calculation and another parallax image (reference image).

  The parallax amount calculation unit 24 calculates the parallax amounts of all the corresponding pixels extracted by the corresponding point extraction unit 23, respectively. The distance information calculation unit 21 calculates a distance to the subject in the image (subject distance information) from the calculated amount of parallax.

  Next, an image composition process for switching the imaging angle of view, which is mainly executed by the system controller 19 and the image composition unit 13 in accordance with a first image processing program as a computer program, will be described with reference to the flowchart of FIG.

  First, in step S <b> 100, when the system controller 19 receives an imaging condition and a signal instructing preparation for imaging (imaging preparation signal) from the user via the information input unit 16, the system controller 19 sets the imaging condition to the imaging controller 17. Transfer information and imaging preparation signal. The imaging control unit 17 sets the aperture values of the first and second imaging optical system groups, the exposure time (shutter speed) in the imaging element unit 10 and the like based on the input imaging conditions. In this step, it is assumed that the user inputs either the angle of view of the first imaging optical system group or the angle of view of the second imaging optical system group.

  Next, in step S <b> 101, when the system controller 19 receives a signal (imaging start signal) instructing the start of imaging from the user through the information input unit 16, the imaging controller unit 17 (imaging device) is input to the imaging controller 17. The exposure of the elements 10a to 10h) is started. The analog image signals output from the image sensors 10 a to 10 h are converted into digital image signals by the A / D converter 11 and then sent to the image processing unit 12. The image processing unit 12 generates images corresponding to the subject images respectively formed on the imaging elements 10a to 10h from the digital imaging signal. At this time, it is preferable that the image processing unit 12 performs processing for matching the luminance levels and white balance of the respective images. By this processing, it is possible to reduce adverse effects such as luminance unevenness and color unevenness in the image composition processing performed later.

  Next, in step S102, the system controller 19 determines whether or not the angle of view input by the user is the same as the angle of view (reference angle of view) of the second imaging optical system group including the reference optical system 120c. The reference field angle is preferably the widest field angle among the plurality of image forming optical system groups included in the compound-eye imaging device 1. As can be seen from FIG. 3, the image (110) with a narrow angle of view does not have information on the peripheral area of the image (120) with a wide angle of view. For this reason, it is difficult to align a plurality of images obtained through an imaging optical system group having a wide angle of view with reference to an image obtained by the imaging optical system group having a narrow angle of view. If the user input angle of view is the same as the reference angle of view, the process proceeds to S103, and if not, the process proceeds to S104.

  In step S103, the image synthesis unit 13 synthesizes the images obtained through the other second imaging optical systems 120a, 120b, and 120d with the reference image obtained through the reference optical system 120c.

  Here, a method for synthesizing images obtained through the second imaging optical systems 120a, 120b, and 120d having the same angle of view as the reference optical system 120c will be described. FIGS. 5A and 5B show images obtained through the second imaging optical systems 120c and 120d when the subjects A and B shown in FIG. 13 are imaged using the compound-eye imaging device 1, respectively. Is shown. On the image obtained from the second imaging optical systems 120c and 120d, the base length which is the distance between the optical axes between the second imaging optical systems 120c and 120d and the subject distance which is the distance to the subjects A and B The positions of the subjects A and B are different from each other by the geometrical relationship according to the above. Also, due to the different viewpoints of the two parallax images, a portion that appears on the image and a portion that is not so appear even in the same subject. For example, in the image obtained through the second imaging optical system 120c (FIG. 5A), the surface A3 of the subject A and the surface B3 of the subject B are shown, but obtained through the second imaging optical system 120d. These planes are not shown in the resulting image (FIG. 5B). On the other hand, the image obtained through the second imaging optical system 120d (FIG. 5B) does not appear in the image obtained through the second imaging optical system 120c (FIG. 5A). B side B4 is shown. In this way, the fact that the subject area shown in one image is not shown in the other image due to the difference in viewpoint is called occlusion.

  In the image composition processing in the present embodiment, an image (FIG. 5A) obtained through the reference optical system 120c is used as a reference image. Then, a subject area (corresponding point) corresponding to the subject area shown in the reference image is extracted from the image obtained through the second imaging optical system that is not the reference optical system 120c, and this is extracted as a subject on the reference image. Composite to the area.

  The corresponding point extraction method will be described with reference to FIG. The left side of FIG. 6 shows the reference image 501 shown in FIG. 5A, and the right side shows the reference image 502 as the image shown in FIG. 5B combined with the reference image. Here, image coordinates (X, Y) indicating positions in the horizontal direction and the vertical direction on the image are used. Image coordinates (X, Y) are defined with the upper left of each image shown in FIG. 6 as the origin. Further, the luminance of the image coordinates (X, Y) in the standard image 501 is F1 (X, Y), and the luminance of the image coordinates (X, Y) in the reference image 502 is F2 (X, Y).

  A pixel (shown by hatching) in the reference image 502 corresponding to a pixel (shown by hatching) at an arbitrary coordinate (X, Y) in the reference image 501 is the luminance F1 (in the reference image 502) in the reference image 501 (shown by hatching). X, Y) is obtained by searching for a pixel having the most similar luminance. However, since it is difficult to simply search for a pixel having the most similar brightness to the brightness of an arbitrary pixel, a pixel with similar brightness is searched by the block matching method using pixels near the image coordinates (X, Y). To do.

For example, a block matching process when the block size is 3 will be described. The luminance values of a total of three pixels, that is, a pixel at an arbitrary coordinate (X, Y) in the reference image 501 and two pixels before and after (X−1, Y) and (X + 1, Y), respectively,
F1 (X, Y), F1 (X-1, Y), F1 (X + 1, Y)
It becomes.

On the other hand, the luminance values of the pixels in the reference image 502 shifted by k in the X direction from the coordinates (X, Y), (X-1, Y), (X + 1, Y) are respectively
F2 (X + k, Y), F2 (X + k-1, Y), F2 (X + k + 1, Y)
It becomes.

  At this time, the similarity E with the pixel at the coordinates (X, Y) in the reference image 501 is defined by the following equation (2).

  In this formula (2), the value of similarity E is calculated by successively changing the value of k, and the smallest similarity E is given (X + k, Y), but the coordinates (X, Y) corresponds to a pixel (corresponding point). Here, the method of extracting corresponding points between images having parallax in the horizontal direction has been described, but corresponding points in the case of having parallax in the vertical direction or oblique direction can be similarly extracted.

  By synthesizing the subject area as the corresponding point obtained in this way with the reference image in units of pixels, the noise level in the reference image can be reduced, and the image quality of the output composite image can be improved. it can.

  The surface A3 of the subject A and the surface B3 of the subject B shown in FIG. 5A are obtained through the second imaging optical system 120d because there is no corresponding subject region as an occlusion region in FIG. 5B. Is not synthesized from the resulting image. Further, the surface B4 of the subject B that is shown only in FIG. 5B is not shown in the reference image, and thus is not used for composition. In this way, the occlusion region cannot be synthesized, but in most other cases, an image obtained by imaging from a viewpoint different from the reference viewpoint RP can be synthesized with the reference image, so that the entire synthesized image As a result, the noise level can be reduced.

  If the parallax is so large that the block matching method cannot be used as it is, and the shape of the subject area differs greatly between the base image and the reference image, block matching is performed after geometric conversion such as affine transformation is performed on the reference image. The synthesis process may be performed using a method. In the same manner as the image composition method described above, an image obtained through the other second imaging optical systems 120a and 120b can also be composed with the reference image.

  On the other hand, in step S104 and step S105, the system controller 19 performs different angle-of-view image composition processing, which is image composition processing when the user input view angle is different from the reference view angle.

  Here, the different angle-of-view image composition processing will be described with reference to FIG. FIGS. 7A and 7B respectively show images obtained through the first imaging optical systems 110c and 110d when the subjects A and B shown in FIG. 13 are imaged using the compound-eye imaging device 1. Is shown. FIG. 7C shows a reference image obtained through the reference optical system 120c when the subjects A and B similar to those in FIG. 13 are imaged using the compound-eye imaging device 1.

  The image obtained through the first imaging optical systems 110c and 110d has a narrower angle of view than the reference image obtained through the reference optical system 120c. For this reason, the sizes of the subjects A and B on the image are different, and these images cannot be combined as they are. Therefore, first, in step S104, the image composition unit 13 matches the reference image obtained through the reference optical system 120c with the user input angle of view (here, the angle of view of the first imaging optical system group). Trim and enlarge part of the image. In FIG. 7D, a part (center region) of the reference image shown in FIG. 7C is trimmed, and the trimmed part corresponds to the angle of view of the first imaging optical image systems 110c and 110d. An image that has been enlarged so as to be an image having an angle of view is shown. Although the resolution of the image is degraded by the trimming and enlargement processing, a new reference image (hereinafter referred to as an enlarged reference image) in which a subject having the same size as the subject on the image obtained through the first imaging optical systems 110c and 110d is captured. Can be obtained).

  In step S105, the image synthesis unit 13 synthesizes the images obtained through the first imaging optical systems 110a to 110d using the enlarged reference image obtained in step S104. The image composition method here is an image obtained through the first imaging optical systems 110a to 110d with respect to the enlarged reference image shown in FIG. It is also possible to synthesize the subject area inside in units of pixels. Further, as described above, the resolution of the reference image shown in FIG. For this reason, mutually corresponding subject regions in the images obtained through the first imaging optical systems 110a to 110d are determined using the enlarged reference image, and these positions are aligned so that images other than the enlarged reference image are obtained. You may make it synthesize | combine (it does not synthesize | combine an expansion reference | standard image).

  Here, in this embodiment, the reference optical system 120c is arranged in the first region including the optical axis of the first imaging optical system group shown in FIG. Moreover, these imaging optical systems are two-dimensionally arranged so that their optical axes are parallel to each other. As a result, as shown in FIG. 7, all of the subject areas shown in the image obtained through the reference optical system 120c arranged at the reference viewpoint RP are obtained from the plurality of images obtained through the first imaging optical system group. It appears in one of them. That is, no occlusion occurs in the subject area in the reference image.

  For example, the surfaces A1 to A3 and the surfaces B1 to B3 shown in the enlarged reference image shown in FIG. 7D include the surfaces A1, A3, B1, and B3 in the image shown in FIG. In the image shown in FIG. 7B, planes A2 and B2 are shown. For this reason, all the subject areas shown in the enlarged reference image can be obtained from either of the images shown in FIGS. Further, although the enlarged reference image is an image after trimming and enlargement processing, the subject position information from the reference viewpoint RP can be clearly determined, so that the alignment accuracy by the block matching method can be improved. Furthermore, since it is not necessary to perform viewpoint interpolation processing using distance information, it is possible to greatly reduce the processing load. Also, in the synthesis of images with different angles of view as described above, the composition is always performed based on the image obtained through the reference optical system 120c as the reference viewpoint RP, so that the subject position shift when switching the imaging angle of view is performed. Can be reduced.

  When the process proceeds from step S103 and step S105 to step S106, the system controller 19 stores the synthesized image in the recording medium 18 and ends the image synthesis process.

  Next, intermediate angle-of-view image generation processing for realizing continuous zooming performed mainly by the system controller 19, the image composition unit 13, and the intermediate angle-of-view image generation unit 14 will be described with reference to the flowchart of FIG. This intermediate angle of view image generation process is executed according to a second image processing program as a computer program.

  First, in step S <b> 200, when the imaging condition and the angle of view designated by the user are input from the information input unit 16, the system controller 19 transfers the information of the imaging condition and the angle of view to the imaging control unit 17. . Here, it is assumed that an intermediate field angle between the field angle of the first imaging optical system group and the field angle of the second imaging optical system group is input by the user.

  Next, in step S201, the imaging control unit 17 selects an imaging optical system that performs imaging according to the angle of view input by the user. Here, since the intermediate angle of view is input, all of the first imaging optical system group and the second imaging optical system group are selected as imaging optical systems used for imaging.

  Next, in step S <b> 202, the imaging control unit 17 starts exposure of the image sensor corresponding to the imaging optical system selected from the image sensor unit 10. Then, the image processing unit 12 generates an image from the digital imaging signal input from the imaging device via the A / D converter 11. As a result, an image group having the same field angle obtained through the first imaging optical system group and an image group having the same field angle obtained through the second imaging optical system group are generated.

  At this time, it is preferable that the image processing unit 12 performs processing for matching the luminance levels and white balance of the respective images. By this processing, it is possible to reduce adverse effects such as luminance unevenness and color unevenness in the image composition processing performed later.

  Next, in step S203, the image composition unit 13 performs image composition processing on the image group obtained through the first imaging optical system group to generate a first composite image, and the second result. An image synthesis process is also performed on the image group obtained through the image optical system group to generate a second synthesized image. Yeah. The image composition process is the process performed in step S103 and steps S104 and S105 in FIG. At this time, both the first and second synthesized images having different angles of view are the same viewpoint image corresponding to an image captured from the reference viewpoint RP.

  Next, in step S204, the intermediate angle-of-view image generation unit 14 trims a region corresponding to the user input angle of view from the second synthesized image that is a wide-angle synthesized image obtained through the second imaging optical system group. The area is enlarged. Further, the intermediate angle-of-view image generation unit 14 reduces the first synthesized image, which is a telephoto synthesized image obtained through the first imaging optical system group, according to the user input angle of view.

  Next, in step S250, the intermediate angle-of-view image generation unit 14 reduces the enlarged image obtained by trimming and enlarging the second synthesized image and the first synthesized image by the different angle-of-view image synthesizing process described above. An intermediate angle-of-view image is generated by combining the reduced image obtained by the processing. Then, this process ends.

  According to such an intermediate angle of view image generation process, an intermediate angle of view image corresponding to an image obtained by imaging from the reference viewpoint RP is generated. For this reason, even when the intermediate angle of view is selected by continuous zooming, an intermediate angle of view image with little subject position shift can be generated.

  Further, the preview image generation unit 15 generates a preview image from the reference image obtained through the reference optical system 120 c serving as the reference viewpoint RP and displays the preview image on the display unit 20. Therefore, even when the user instructs to change the angle of view (zooming), a preview image as an image from the same viewpoint can always be generated and displayed by trimming and displaying the reference image. Thus, since the preview image and the final view angle image at the time of changing the view angle are all generated as images obtained from the same reference viewpoint RP, the user can see an image with less discomfort due to the subject position shift. it can.

  Next, subject distance information recording processing mainly performed by the system controller 19 and the distance information calculation unit 21 will be described with reference to the flowchart of FIG. This subject distance information recording process is executed in accordance with a third image processing program as a computer program. Here, first, subject distance information recording processing using a parallax image obtained by the second imaging optical system group (second imaging optical systems 120c and 120d) that captures the widest subject space will be described.

  First, in step S300, the system controller 19 receives an imaging preparation signal input from the information input unit 16 by the user. In response to this, an image pickup signal from an image pickup element corresponding to the second imaging optical systems 120 c and 120 d in the image pickup element unit 10 is transferred to the image processing unit 12 via the A / D converter 11. The image processing unit 12 generates two parallax images obtained through the second imaging optical systems 120c and 120d from the imaging signal.

  Next, in step S301, the reference image selection unit 22 selects one of the two generated parallax images as a reference image for parallax amount calculation (that is, subject distance calculation). In this embodiment, an image obtained by the second imaging optical system 120c is selected as a reference image.

  Next, in step S302, the corresponding point extraction unit 23 detects corresponding pixels (corresponding points) between the standard image and the reference image, using the image obtained through the second imaging optical system 120d as a reference image. Corresponding pixels are, for example, pixels in which the same point of the subject A appears in two parallax images obtained by imaging the subject A. As a corresponding pixel detection method, the corresponding point extraction method described in the previous image composition process can be used.

  Next, in step S303, the parallax amount calculation unit 24 calculates the parallax amount between the extracted corresponding points. Specifically, the amount of parallax is calculated as a difference in pixel position between corresponding pixels of the standard image and the reference image.

  Next, in step S304, the distance information calculation unit 21 calculates the subject distance from the parallax amount calculated in step S303 and the focal lengths and baseline lengths of the second imaging optical systems 120c and 120d, which are known information. Is calculated. Here, the calculation of the subject distance when the second imaging optical systems 120c and 120d are used has been described. However, other imaging optical system pairs (for example, the second imaging optical system, for example) The subject distance can also be calculated using a pair of imaging optical systems 120a and 120b. When the above method is used for parallax images with different angles of view, a portion corresponding to a parallax image with a narrow angle of view is cut out from a parallax image with a wide angle of view, and corresponding pixels are extracted from the cut-out portion. It is desirable.

  As described above, according to the present embodiment, it is possible to reduce the subject position shift on the image due to zooming while reducing the thickness of the compound-eye imaging device and obtaining a high zoom ratio.

  Further, according to the present embodiment, various imaging modes can be realized. For example, in the high dynamic range mode, imaging is performed a plurality of times while changing the exposure conditions of the imaging optical systems constituting the compound eye and the imaging elements corresponding thereto. Then, by synthesizing a plurality of images obtained by a plurality of imaging operations, an image having a wide dynamic range can be acquired. In the blur addition mode, an image that emphasizes the main subject can be obtained by adding blur to the background based on the calculated subject distance information. In the background removal mode, an image in which the background other than the main subject is removed based on the subject distance calculated as described above can be obtained. Further, in the stereoscopic imaging mode, left and right parallax images are acquired by the respective imaging optical systems that constitute the compound eyes arranged in the horizontal direction and the corresponding imaging elements. Then, an image can be stored as a stereoscopic image using an image with a narrow angle of view and a part of the image with the other wide angle of view corresponding thereto.

  FIG. 10 shows the imaging optical unit 200 of the compound-eye imaging apparatus that is Embodiment 2 of the present invention when viewed from the optical axis direction. The imaging optical unit 200 is provided with first imaging optical systems 210a, 210b, 210c, and 210d, and second imaging optical systems 220a, 220b, 220c, and 220d. Further, the imaging optical unit 200 is provided with third imaging optical systems 230a, 230b, 230c, and 230d, and fourth imaging optical systems 240a, 240b, 240c, and 240d. The plurality of imaging optical systems 210a to 210d, 220a to 220d, 230a to 230d, and 240a to 240d are spaced apart from each other in a two-dimensional direction orthogonal to the respective optical axes. The optical axes of the imaging optical systems 210a to 210d, 220a to 220d, 230a to 230d, and 240a to 240d extend in parallel with each other.

  First imaging optical systems (hereinafter collectively referred to as first imaging optical system groups) 210a to 210d have a first angle of view (focal length) θ. The first angle of view θ corresponds to the telephoto end angle of view which is the narrowest angle of view in the compound-eye imaging device of the present embodiment. Also, the second imaging optical system (hereinafter collectively referred to as the first imaging optical system group) 220a to 220d has a second angle of view (in this embodiment, the first angle of view) that is wider than the first angle of view θ. 1), which is 8 times the angle of view θ of 1. The second field angle corresponds to the wide-angle end field angle that is the widest field angle in the compound-eye imaging apparatus of the present embodiment. The third image forming optical system (hereinafter collectively referred to as a third image forming optical system group) 230a to 230d has a third field angle (in this embodiment, half the second field angle 8θ). 4θ). The fourth imaging optical system (hereinafter collectively referred to as a fourth imaging optical system group) 240a to 240d has a fourth field angle (in this embodiment, 1/4 times the second field angle 8θ). 2θ).

  In the present embodiment, these first to fourth imaging optical system groups are arranged 4 × 4 in the horizontal and vertical directions. Specifically, a first imaging optical system, a second imaging optical system, a first imaging optical system, and a second imaging optical system are provided in the first column and the third column in the vertical direction. They are arranged in this order in the horizontal direction (from left to right in FIG. 10, the same applies hereinafter). In the second and fourth rows, the third imaging optical system, the fourth imaging optical system, the third imaging optical system, and the fourth imaging optical system are arranged in this order in the horizontal direction. Has been placed.

  Although not shown, the imaging element unit of the compound-eye imaging device of the present embodiment has imaging areas corresponding to the 16 imaging optical systems 210a to 210d, 220a to 220d, 230a to 230d, and 240a to 240d described above. There are 16 image sensors constituting

  In FIG. 10, the positions of the optical axes of the first imaging optical systems 210a to 210d are connected by a solid line (straight line) extending in the vertical direction and the horizontal direction, and a region surrounded by the solid line is defined as a first region. To do. The positions of the optical axes of the second imaging optical systems 220a to 220d are connected by a broken line (straight line) extending in the vertical direction and the horizontal direction, and a region surrounded by the broken line is defined as a second region. Further, the positions of the optical axes of the third imaging optical systems 230a to 230d are connected by a one-dot chain line (straight line) extending in the vertical direction and the horizontal direction, and a region surrounded by the one-dot chain line is defined as a third region. . Further, the positions of the optical axes of the fourth imaging optical systems 240a to 240d are connected by a two-dot chain line extending in the vertical direction and the horizontal direction, and a region surrounded by the two-dot chain line is defined as a fourth region. In FIG. 10, each line is shown slightly shifted from the position of the optical axis of the corresponding imaging optical system so that the solid line, the broken line, the one-dot chain line, and the two-dot chain line are easy to see.

  The first to fourth imaging optical system groups are regions in which all of the first to fourth regions overlap (the first to fourth imaging optical systems 210d, 220c, and 230b at the center of the imaging optical unit 200). , 240a (region where the optical axis is located). In the present embodiment, the second one having the widest field angle among the first to fourth imaging optical systems 210d, 220c, 230b, and 240a arranged in the overlapping region of all the first to fourth regions. The imaging optical system 220c is a reference optical system, and its arrangement position is a reference viewpoint RP. Other configurations of the compound-eye imaging apparatus of the present embodiment are the same as those of the first embodiment, and the description thereof is omitted.

  Also in the present embodiment, in the same manner as in the first embodiment, image synthesis processing and intermediate angle-of-view image generation processing are performed using an image obtained through the reference optical system 220c serving as the reference viewpoint RP as a reference image. (Omitted). Thus, when the imaging angle of view is switched from the angle of view of the second imaging optical group to the angle of view of the first, third and fourth imaging optical systems, the respective imaging optical systems It is possible to reduce the subject position shift on the composite image obtained through the group. In addition, since the optical axis of the reference optical system 220c serving as the reference viewpoint RP is disposed in an area where all the first to fourth areas overlap, the subject area shown in the reference image obtained through the reference optical system 220c is It is always included in one of the images obtained through other imaging optical systems. For this reason, it is possible to eliminate the influence of occlusion in the image composition using the reference image.

  Similarly to the first embodiment, this embodiment also enables continuous zooming without providing a zoom mechanism that moves the optical element for zooming in the optical axis direction. Further, in this embodiment, since the number of imaging optical system groups having different field angles from each other is larger than that in the first embodiment, a higher zoom ratio can be easily obtained.

  In the present embodiment, in the first to fourth angles of view that are discretely different, the case where the ratio of the angle of view W on the wide angle side to the next narrowest angle of view Wn is 2 has been described. It is only an example, and other ratio values may be used. However, it is desirable to satisfy the condition of the above-described formula (1). This is the same in other embodiments described later.

  FIG. 11 shows the imaging optical unit 300 of the compound-eye imaging apparatus that is Embodiment 3 of the present invention when viewed from the optical axis direction. The imaging optical unit 300 is provided with first imaging optical systems 310a, 310b, 310c, and 310d, and second imaging optical systems 320a, 320b, 320c, and 320d. Furthermore, the imaging optical unit 300 is provided with third imaging optical systems 330a, 330b, 330c, and 330d, and fourth imaging optical systems 340a, 340b, 340c, and 340d. The plurality of imaging optical systems 310a to 310d, 320a to 320d, 330a to 330d, and 340a to 340d are arranged apart from each other in a two-dimensional direction orthogonal to the respective optical axes. The optical axes of the imaging optical systems 310a to 310d, 320a to 320d, 330a to 330d, and 340a to 340d extend in parallel to each other.

  First imaging optical systems (hereinafter collectively referred to as first imaging optical system groups) 310a to 310d have a first angle of view (focal length) θ. The first angle of view θ corresponds to the telephoto end angle of view which is the narrowest angle of view in the compound-eye imaging device of the present embodiment. In addition, the second image forming optical system (hereinafter, also collectively referred to as a first image forming optical system group) 320a to 320d has a second angle of view (in this embodiment, the first image forming optical system group). 1), which is 8 times the angle of view θ of 1. The second field angle corresponds to the wide-angle end field angle that is the widest field angle in the compound-eye imaging apparatus of the present embodiment. Third image forming optical systems (hereinafter collectively referred to as a third image forming optical system group) 330a to 330d have a third field angle that is wider than the first field angle and narrower than the second field angle (this embodiment). In the example, it has 4θ) that is ½ times the second field angle 8θ. The fourth image forming optical system (hereinafter collectively referred to as a fourth image forming optical system group) 340a to 340d is a fourth field angle that is wider than the first field angle and narrower than the third field angle (this embodiment). In the example, it has 2θ) which is 1/4 times the second angle of view 8θ. Since the fourth imaging optical systems 340a to 340d also have an angle of view that is wider than the first field angle and narrower than the second field angle, the third imaging optical systems 330a to 330d are different from the third image forming optical systems 330a to 330d. It can also be referred to as an imaging optical system.

  In the present embodiment, these first to fourth imaging optical system groups are arranged 4 × 4 in the horizontal and vertical directions. Specifically, in the first column in the vertical direction, the first imaging optical system, the third imaging optical system, the fourth imaging optical system, and the first imaging optical system are horizontally arranged in this order. It is arranged in the direction (from left to right in FIG. 11, the same applies hereinafter). In the second column, a fourth imaging optical system, a second imaging optical system, a second imaging optical system, and a third imaging optical system are arranged in this order in the horizontal direction. . In the third column, the third imaging optical system, the second imaging optical system, the second imaging optical system, and the fourth imaging optical system are arranged in this order in the horizontal direction. . In the fourth column, the first imaging optical system, the fourth imaging optical system, the third imaging optical system, and the fourth imaging optical system are arranged in this order in the horizontal direction. .

  Although not shown, the imaging element unit of the compound-eye imaging device of the present embodiment has imaging areas corresponding to the above-described 16 imaging optical systems 310a to 310d, 320a to 320d, 330a to 330d, and 340a to 340d. There are 16 image sensors constituting

  In FIG. 11, the positions of the optical axes of the first imaging optical systems 310a to 310d are connected by a solid line (straight line) extending in the vertical direction and the horizontal direction, and a region surrounded by the solid line is defined as a first region. To do. Further, the positions of the optical axes of the second imaging optical systems 320a to 320d are connected by a broken line (straight line) extending in the vertical direction and the horizontal direction, and a region surrounded by the broken line is defined as a second region. Further, the positions of the optical axes of the third imaging optical systems 330a to 330d are connected by an alternate long and short dash line (straight line) extending in an oblique direction, and a region surrounded by the alternate long and short dash line is defined as a third region. Further, the positions of the optical axes of the fourth imaging optical systems 340a to 340d are connected by a two-dot chain line (straight line) extending in an oblique direction, and a region surrounded by the two-dot chain line is defined as a fourth region. In FIG. 11, the solid line is slightly shifted from the position of the optical axis of the first imaging optical system group so that each of the solid line, the broken line, the one-dot chain line, and the two-dot chain line is easy to see.

  The first to fourth imaging optical system groups are regions where all of the first to fourth regions overlap (the optical axes of the second imaging optical systems 320a to 320d are positioned at the center of the imaging optical unit 300). Area) to be present. In the present embodiment, among the first to fourth image forming optical system groups, the second image forming optical system group having the widest angle of view disposed in a region where all of the first to fourth regions overlap. The second imaging optical system 320c, which is one of the above, is used as a reference optical system, and its arrangement position is set as a reference viewpoint RP. Other configurations of the compound-eye imaging apparatus of the present embodiment are the same as those of the first embodiment, and the description thereof is omitted.

  Also in the present embodiment, in the same manner as in the first embodiment, image synthesis processing and intermediate angle-of-view image generation processing are performed using an image obtained through the reference optical system 320c serving as the reference viewpoint RP as a reference image (the flow of these processing is described) (Omitted). Thus, when the imaging angle of view is switched from the angle of view of the second imaging optical group to the angle of view of the first, third and fourth imaging optical systems, the respective imaging optical systems It is possible to reduce the subject position shift on the composite image obtained through the group. In addition, since the optical axis of the reference optical system 320c serving as the reference viewpoint RP is disposed in an area where the first to fourth areas all overlap, the subject area shown in the reference image obtained through the reference optical system 320c is It is always included in one of the images obtained through other imaging optical systems. For this reason, it is possible to eliminate the influence of occlusion in the image composition using the reference image.

  Similarly to the first embodiment, this embodiment also enables continuous zooming without providing a zoom mechanism that moves the optical element for zooming in the optical axis direction. Further, in this embodiment, since the number of imaging optical system groups having different field angles from each other is larger than that in the first embodiment, a higher zoom ratio can be easily obtained.

  FIG. 12 shows the imaging optical unit 400 of the compound-eye imaging apparatus that is Embodiment 4 of the present invention when viewed from the optical axis direction. The imaging optical unit 400 is provided with first imaging optical systems 410a, 410b, 410c and second imaging optical systems 420a, 420b, 420c, 420d. Further, the imaging optical unit 300 is provided with third imaging optical systems 430a, 430b, and 430c. The plurality of imaging optical systems 410a to 410c, 420a to 420d, and 430a to 430c are arranged apart from each other in a two-dimensional direction orthogonal to the respective optical axes. The optical axes of the imaging optical systems 410a to 410c, 420a to 420d, and 430a to 430c extend in parallel to each other.

  First imaging optical systems (hereinafter collectively referred to as first imaging optical system groups) 410a to 410c have a first angle of view (focal length) θ. The first angle of view θ corresponds to the telephoto end angle of view which is the narrowest angle of view in the compound-eye imaging device of the present embodiment. In addition, the second imaging optical system (hereinafter, also collectively referred to as a first imaging optical system group) 420a to 420d has a second field angle wider than the first field angle θ (in this embodiment, the first field angle). 4 [theta] which is four times the angle of view [theta] of 1. The second field angle corresponds to the wide-angle end field angle that is the widest field angle in the compound-eye imaging apparatus of the present embodiment. The third image forming optical system (hereinafter collectively referred to as a third image forming optical system group) 430a to 430c has a third field angle that is wider than the first field angle and narrower than the second field angle (this embodiment). In the example, it has 2θ) that is ½ times the second angle of view 4θ.

  In the present embodiment, these first to third imaging optical system groups are arranged so that the other imaging optical systems are arranged concentrically around the second imaging optical system 420c. Yes. Specifically, the imaging optical systems other than the second imaging optical system 420c are arranged in the counterclockwise direction in the first imaging optical system, the second imaging optical system, and the third imaging optical system. Are arranged to repeat in this order.

  Although not shown, the imaging element unit of the compound-eye imaging device according to the present embodiment configures imaging regions corresponding to the ten imaging optical systems 410a to 410c, 420a to 420d, and 430a to 430c described above. Image sensor.

  In FIG. 12, the positions of the optical axes of the first imaging optical systems 410a to 410d are connected by a solid line (straight line), and a region surrounded by the solid line is defined as a first region. Further, the positions of the optical axes of the second imaging optical systems 420a, 420b, 420d other than the second imaging optical system 420c are connected by a broken line (straight line), and the region surrounded by the broken line is defined as the second area. This is an area. Further, the positions of the optical axes of the third imaging optical systems 430a to 430c are connected by a one-dot chain line (straight line), and a region surrounded by the one-dot chain line is defined as a third region.

  The first to third imaging optical system groups are arranged so that there is a region where all of the first to third regions overlap. In the present embodiment, the second imaging optical system 420c, which is one of the second imaging optical system groups having the widest field angle among the first to third imaging optical system groups, is replaced with the first to third imaging optical system groups. All the third areas are arranged in the overlapping area. The second imaging optical system 420c is a reference optical system, and its arrangement position is a reference viewpoint RP.

  The reference optical system 420c is in a region where all of the first to third regions overlap, and is a position (or all other connections) that is the center of gravity of the position of the optical axis of all other imaging optical systems. It is preferably arranged at a position equidistant from the position of the optical axis of the image optical system. By disposing the reference optical system 420c at such a position, it is possible to prevent the parallax amount of another imaging optical system with respect to the reference optical system 420c from becoming too large. If the amount of parallax is too large, the subject position shift amount and the subject deformation amount of the image obtained for each imaging optical system group become large, and it is difficult to perform image composition. Other configurations of the compound-eye imaging apparatus of the present embodiment are the same as those of the first embodiment, and the description thereof is omitted.

  Also in the present embodiment, in the same manner as in the first embodiment, an image synthesis process and an intermediate angle-of-view image generation process are performed using an image obtained through the reference optical system 420c serving as the reference viewpoint RP as a reference image. (Omitted). Thus, when the imaging angle of view is switched from the angle of view of the second imaging optical group to the angle of view of the first, third and fourth imaging optical systems, the respective imaging optical systems It is possible to reduce the subject position shift on the composite image obtained through the group. In addition, since the optical axis of the reference optical system 420c serving as the reference viewpoint RP is arranged in an area where the first to third areas all overlap, the subject area shown in the reference image obtained through the reference optical system 420c is It is always included in one of the images obtained through other imaging optical systems. For this reason, it is possible to eliminate the influence of occlusion in the image composition using the reference image.

  Similarly to the first embodiment, this embodiment also enables continuous zooming without providing a zoom mechanism that moves the optical element for zooming in the optical axis direction. Further, in this embodiment, since the number of imaging optical system groups having different field angles from each other is larger than that in the first embodiment, a higher zoom ratio can be easily obtained.

  In each of the above-described embodiments, the case where a plurality of second imaging optical systems having the widest field angle are provided has been described. However, at least one second imaging optical system is provided, and this is used as a reference optical system. Just do it.

  Each embodiment described above is only a representative example, and various modifications and changes can be made to each embodiment in carrying out the present invention.

  It is possible to provide a compound eye imaging device that can obtain a small and good image.

DESCRIPTION OF SYMBOLS 1 Compound eye imaging device 110a-110d, 210a-210d, 310a-310d, 410a-410c 1st imaging optical system 120a-120d, 220a-220d, 320a-320d, 420a-420d 2nd imaging optical system 10 Element part

Claims (9)

A plurality of imaging optical systems disposed apart from each other in a direction perpendicular to the respective optical axes;
A compound-eye imaging device having an imaging unit for each imaging optical system having an imaging region for imaging a subject through each of the plurality of imaging optical systems,
The plurality of imaging optical systems include a plurality of first imaging optical systems each having a first field angle and at least one second field angle having a second field angle wider than the first field angle. An imaging optical system,
When viewed in the optical axis direction, the optical axis of the at least one second imaging optical system is within a first region surrounded by a line connecting the positions of the optical axes of the plurality of first imaging optical systems. A compound eye imaging device characterized in that is arranged. A plurality of the second imaging optical systems;
In the optical axis direction view, the plurality of the first region and the second region surrounded by a line connecting the optical axis positions of the plurality of second imaging optical systems overlap each other. 2. The compound eye imaging apparatus according to claim 1, wherein an optical axis of the at least one second imaging optical system of the second imaging optical system is arranged. The plurality of imaging optical systems includes a plurality of third imaging optical systems having a third field angle that is wider than the first field angle and narrower than the second field angle;
A region where the first region, the second region, and the third region surrounded by a line connecting the positions of the optical axes of the plurality of third imaging optical systems overlap each other when viewed in the optical axis direction The compound-eye imaging apparatus according to claim 2, wherein an optical axis of the at least one second imaging optical system among the plurality of second imaging optical systems is disposed therein.   The first and second imaging optical systems using, as a reference image, an image obtained by imaging through the second imaging optical system in which an optical axis is disposed in the first region, or a part thereof. 4. The compound eye imaging apparatus according to claim 1, further comprising an image synthesis unit configured to synthesize an image obtained by imaging through each of the above.   An image obtained by imaging through the second imaging optical system in which an optical axis is arranged in a region where the first region, the second region, and the third region overlap or a part thereof 4. The compound eye according to claim 3, further comprising: an image synthesizing unit that synthesizes an image obtained by imaging through each of the first, second, and third imaging optical systems with reference image as a reference image. Imaging device.   An angle of view between the angles of view of the two imaging optical systems based on two images obtained by imaging through two imaging optical systems having different angles of view among the plurality of imaging optical systems. 6. The compound-eye imaging apparatus according to claim 1, further comprising an image generation unit configured to generate an image corresponding to.   The image obtained by imaging through the second imaging optical system in which an optical axis is disposed in the first region or a part thereof is displayed as a preview image on a display unit. The compound eye imaging device according to any one of 1 to 6.   8. The distance calculation unit according to claim 1, further comprising a distance calculation unit configured to calculate a distance from the plurality of images having parallax to each other obtained by imaging through the plurality of imaging optical systems. The compound eye imaging device described in 1. Of the different angles of view of the plurality of imaging optical systems, any one of the angle of view W and the next narrowest angle of view Wn after the angle of view W are:
1.1 ≦ W / Wn ≦ 3
The compound eye imaging apparatus according to claim 1, wherein the following condition is satisfied.