WO2016051871A1

WO2016051871A1 - Multi-lens imaging device, method for manufacturing same, and information terminal device

Info

Publication number: WO2016051871A1
Application number: PCT/JP2015/066616
Authority: WO
Inventors: 慶延岸根; 達郎岩▲崎▼
Original assignee: 富士フイルム株式会社
Priority date: 2014-09-30
Filing date: 2015-06-09
Publication date: 2016-04-07

Abstract

The problem addressed by the present invention is to provide a multi-lens imaging device that mechanically absorbs individual differences in distorted shapes of images imaged by an imaging module and thereby improves the accuracy of distance recognition, a method for manufacturing the same, and an information terminal device. The problem above is resolved by a multi-lens imaging device wherein: each imaging module has an imaging lens and an imaging element for converting an optical image of a subject for which light is received via the imaging lens into an image signal, with the imaging element being arranged having alignment adjusted with respect to the imaging lens in order to correct resolving power for the optical image; and the condition of I/H being 96% < I/H × 100 < 99.8% is satisfied, where I is the height of a region in the direction running through an axial direction wherein the plurality of imaging modules are lined up, said region being formed by superimposing an image imaged by a first imaging module forming a reference out of a plurality of imaging modules on an image imaged by a second imaging module different from the first imaging module, and H is the height of the image imaged by the first imaging module in the direction running through the axial direction.

Description

Multi-eye imaging device, manufacturing method thereof, and information terminal device

The present invention relates to a multi-view imaging device, a manufacturing method thereof, and an information terminal device, and more particularly to a technique for arranging a plurality of imaging modules for estimating a distance from a subject.

In recent years, lenses for small camera modules have become more difficult to manufacture as they become thinner and have higher functions such as mounting of high-pixel lenses and bright F-number lenses. In order to maintain the desired resolution, the lens image has been increased. In order to reduce the influence of surface tilt, eccentricity adjustment, and the like, alignment adjustment is performed to adjust the position of the image sensor with respect to the position of the lens module. However, when alignment adjustment is performed, there is a drawback that the distortion shape (distortion) of the image captured by the camera changes in a rotationally asymmetric manner, and individual differences occur in the distortion shape of the captured image in accordance with the alignment adjustment amount.

The stereo camera estimates the distance to the subject from the positional deviation of the images taken by the two cameras. However, if there is an individual difference in the distortion shape of the captured images of the two cameras, an error occurs in the distance estimation because the individual position causes a shift in the subject position in the captured image.

In order to deal with such a problem, Patent Document 1 captures an initialization chart using two cameras, calculates correction parameters, and performs image processing based on the correction parameters, thereby shifting the position of the two cameras. The stereo camera which corrects the distortion accompanying this is described. Patent Document 2 describes a stereo camera that performs distortion correction with high accuracy by performing image processing using a small number of line buffers in order to correct a positional shift caused by a relative position between two or more cameras. ing.

JP 2005-351855 A JP 2013-201753 A

These stereo cameras are difficult to perform real-time processing in moving image shooting because of the computational cost of correction processing. Further, there has been a problem that the resolution of the image deteriorates due to the correction processing, and an error occurs in distance estimation.

The present invention has been made in view of such circumstances, and a multi-lens imaging device that increases the distance recognition accuracy by mechanically absorbing individual differences in the distortion shape of the captured image of the imaging module, a manufacturing method thereof, and an information terminal device The purpose is to provide.

In order to achieve the above object, one aspect of a multi-eye imaging device is a multi-eye imaging device that shoots a subject by arranging a plurality of imaging modules in a uniaxial direction, and each of the plurality of imaging modules includes an imaging lens and an imaging lens. An image sensor that converts an optical image of a subject received through the imaging lens into an image signal, and the image sensor is attached to the imaging lens by adjusting the alignment in order to correct the resolution of the optical image. An uniaxial direction in which a plurality of imaging modules in an overlapping region of an image captured by a first imaging module serving as a reference and an image captured by a second imaging module different from the first imaging module are arranged. I / H is 96, where I is the height in the orthogonal direction and H is the image height in the direction orthogonal to the uniaxial direction of the image captured by the first imaging module. <I / H × 100 <99.8% condition is satisfied.

According to this aspect, since the distortion shape of the image image | photographed with the 2nd image pick-up module and the image image | photographed with the 1st image pick-up module is in agreement, distance recognition accuracy using these images is improved. Can do.

It is preferable that I / H satisfies the condition of 98% <I / H × 100 <99.8%. Furthermore, it is more preferable that I / H satisfies the condition of 99% <I / H × 100 <99.8%. When such a condition is satisfied, the distortion shape of the image photographed by the first imaging module and the image photographed by the second imaging module more closely match, so that the distance recognition accuracy can be improved. .

A line connecting the center of the light receiving surface of the image pickup device of the first image pickup module and the center of the light receiving surface of the image pickup device of the second image pickup module passes through the X axis and passes through the center of the light receiving surface of the image pickup device of the first image pickup module. The axis on the light receiving surface of the image pickup device of the first image pickup module perpendicular to the axis is the Y axis, and the axis that passes through the center of the light receiving surface of the image pickup device of the first image pickup module and is orthogonal to the X axis and the Y axis is the Z axis. When the inclination angle θ with respect to the Y axis of the image pickup device of the second image pickup module in the YZ plane passing through the center of the light receiving surface of the image pickup device of the second image pickup module is 0.1 ° <| θ | <2 ° It is preferable to satisfy. Thereby, since the distortion shape of the image image | photographed with the 2nd imaging module and the image image | photographed with the 1st imaging module corresponds, distance recognition accuracy can be improved.

It is preferable that θ satisfies the condition of 0.1 ° <| θ | <1 °. Furthermore, it is more preferable that θ satisfies the condition of 0.1 ° <| θ | <0.5 °. When such a condition is satisfied, the distortion shape of the image photographed by the first imaging module and the image photographed by the second imaging module more closely match, so that the distance recognition accuracy can be improved. .

When the chart is arranged at a position separated from the light receiving surface of the image sensor by a predetermined distance L and the chart is photographed by the first imaging module and the second imaging module, the first photographed by the first imaging module. In the overlapping region of the second image captured by the second imaging module and the second image, a chart arranged at the center of the overlapping region is used as a central index, and a chart arranged around the overlapping region of the image is used as a peripheral index. The center distance between the center of the light receiving surface of the image sensor of the first imaging module and the center of the light receiving surface of the image sensor of the second image module is d millimeters, the focal length of the imaging lens is f millimeters, and the pixel pitch of the image sensor is s millimeters. , L is 1 meter, the distance p1 from the central index to the peripheral index in the first image and the central index to the peripheral index in the second image. The difference Delta] p (pixels) and the distance p2 of 0 <Δp <| 0.1 × (d × f) / (1000 × s) | satisfying it is preferable. Thereby, since the distortion shape of the image image | photographed with the 2nd imaging module and the image image | photographed with the 1st imaging module corresponds, distance recognition accuracy can be improved.

Further, it is preferable that Δp satisfies the condition of 0 <Δp <| 0.1 × (d × f) / (1000 × s) | in at least four peripheral indicators. Thereby, since the distortion shape of the image image | photographed with the 1st image pick-up module and the image image | photographed with the 2nd image pick-up module corresponds more, distance recognition accuracy can be improved.

It is preferable to include a parallelization processing unit that performs parallelization processing on images captured by the first imaging module and the second imaging module. Thereby, even when the optical axis direction of the imaging lens of the first imaging module and the optical axis direction of the imaging lens of the second imaging module are inclined, distance recognition can be performed appropriately. .

A distance measuring unit that measures the distance from the multi-view imaging device to the subject based on the first image captured by the first imaging module and the second image captured by the second imaging module may be provided. Accordingly, the distance to the subject can be measured based on the first image and the second image.

High-resolution image generation means for generating an image with a resolution higher than the resolution of the image sensor based on the first image captured by the first imaging module and the second image captured by the second imaging module. Is preferred. Thereby, an image having a resolution higher than that of the image sensor can be acquired.

The multi-eye imaging device preferably has two eyes. Thereby, two images can be acquired appropriately.

Further, the multi-lens imaging device is a three-lens system in which three imaging modules are arranged in a uniaxial direction, the first imaging module is a central imaging module, and the second imaging module is an imaging module at both ends. Good. By using the central imaging module as a reference, three images can be appropriately acquired.

In order to achieve the above object, one aspect of an information terminal device is a multi-eye imaging device that shoots a subject by arranging a plurality of imaging modules in a uniaxial direction. The imaging module includes an imaging lens and an imaging lens, respectively. An image sensor that converts an optical image of a subject received through the image sensor into an image signal, and the image sensor is attached to the imaging lens by adjusting the alignment in order to correct the resolution of the optical image. A direction orthogonal to a uniaxial direction in which a plurality of imaging modules in an overlapping region of an image captured by a first imaging module serving as a reference and an image captured by a second imaging module different from the first imaging module are arranged. I / H is 96% <I, where I is the height of the image taken in the first imaging module and H is the height of the image perpendicular to the uniaxial direction of the image taken by the first imaging module. And H × 100 <99.8% of satisfying multi-view imaging apparatus, and a communication means for transmitting and receiving digital data such as images.

In order to achieve the above object, one aspect of a method for manufacturing a multi-lens imaging device includes a first imaging lens and a first imaging lens that converts an optical image of a subject received through the first imaging lens into an image signal. A first imaging module including an imaging device, wherein the first imaging module in which the alignment of the first imaging device is adjusted to correct the resolution of the optical image is fixed to a support member; A second imaging module comprising: a second imaging lens; and a second imaging element that converts an optical image of a subject received through the second imaging lens into an image signal, wherein the resolution of the optical image is increased. Temporary fixing step of temporarily fixing the second imaging module in which the alignment of the second imaging element is adjusted for correction to the support member, and the fixed first imaging module and the temporarily fixed second imaging module And each An imaging step of capturing an image, a calculation step of calculating a distortion shape of an image captured by the first imaging module and a distortion shape of an image captured by the second imaging module, and the calculated distortion shape And adjusting the orientation of the second imaging module to fix the distortion shape of the image captured by the first imaging module and the distortion shape of the image captured by the second imaging module. And an adjustment step for causing the first imaging module and the second imaging module in the overlapping region of the image captured by the first imaging module and the image captured by the second imaging module to be orthogonal to the uniaxial direction. I / H is 96% <I / H × 100 <9, where I is the height in the direction and I is the height of the image orthogonal to the uniaxial direction of the image captured by the first imaging module. The condition of 9.8% is satisfied.

According to this aspect, the first imaging module is fixed and the second imaging module is temporarily fixed, and images are respectively captured by the fixed first imaging module and the temporarily fixed second imaging module. The distortion shape of each image is calculated, the calculated distortion shape is matched and the orientation of the second imaging module is adjusted and fixed, and the image captured by the first imaging module and the second imaging module are further fixed. The height in the direction orthogonal to the uniaxial direction in which the first imaging module and the second imaging module are aligned in the overlapping area with the captured image is I, and the direction orthogonal to the uniaxial direction of the image captured by the first imaging module Since the I / H satisfies the condition of 96% <I / H × 100 <99.8% when the height of the image is H, the distance recognition accuracy using these images can be increased.

According to the present invention, it is possible to increase the distance recognition accuracy by mechanically absorbing individual differences in the distortion shape of the captured image of the imaging module.

FIG. 1A is a front perspective view of the multi-eye camera 10. FIG. 1B is a rear perspective view of the multi-eye camera 10. FIG. 2 is a front perspective view of the imaging module 50. FIG. 3 is a diagram illustrating an internal configuration of the imaging module 50. FIG. 4 is a block diagram illustrating an example of the electrical configuration of the multi-view camera 10. FIG. 5 is a diagram for explaining a distortion generation mechanism of the imaging module 50. FIG. 6 is a diagram for explaining a distortion correction method of the multiview camera 10. FIG. 7 is a flowchart showing the process of the manufacturing method of the multiview camera 10. FIG. 8A is a front perspective view of the fixing member 80. FIG. 8B is a front perspective view of the fixing member 80. FIG. 8C is a front perspective view of the fixing member 80. FIG. 9A is a plan view of the fixing member 80. FIG. 9B is a plan view of the fixing member 80. FIG. 9C is a plan view of the fixing member 80. FIG. 10A is a diagram for explaining chart photographing. FIG. 10B is a diagram for explaining chart photographing. FIG. 11 is a graph showing the distortion shape of the left and right viewpoint images. FIG. 12 is a graph showing the distortion shape of the left and right viewpoint images. FIG. 13A is a diagram illustrating an optical axis tilt angle of the left viewpoint imaging unit 30L with respect to the right viewpoint imaging unit 30R. FIG. 13B is a diagram illustrating an optical axis tilt angle of the left viewpoint imaging unit 30L with respect to the right viewpoint imaging unit 30R. FIG. 14A is a diagram illustrating image overlap between a left viewpoint image and a right viewpoint image. FIG. 14B is a diagram illustrating image overlap between the left viewpoint image and the right viewpoint image. FIG. 15 is a diagram illustrating calculation of a difference between the distortion shape of the left viewpoint image and the distortion shape of the right viewpoint image. FIG. 16 is a diagram illustrating parameters of the imaging optical system of the multi-lens camera 10. FIG. 17A is a front perspective view of smartphone 100 with a multi-eye camera. FIG. 17B is a rear perspective view of smartphone 100 with a multi-eye camera. FIG. 18 is a block diagram illustrating an example of an electrical configuration of the smartphone 100 with a multi-view camera.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[Configuration of stereo camera]
The multi-lens camera according to the present embodiment is a multi-lens imaging device that shoots a subject by arranging a plurality of imaging optical systems in a uniaxial direction, and here, two eyes that shoot a stereoscopic image of the same subject viewed from two viewpoints. A camera will be described as an example.

FIG. 1A is a front perspective view of the multi-view camera 10. As shown in the figure, the multi-view camera 10 includes a housing 20, a shutter button 22, a left viewpoint imaging unit 30L, and a right viewpoint imaging unit 30R.

The housing 20 is formed in a substantially rectangular parallelepiped box shape, and the shutter button 22 is disposed on the upper surface of the housing 20 in the figure. The shutter button 22 is an input means for making preparations for shooting such as focus lock and photometry when half-pressed and for capturing an image when fully pressed.

The left-viewpoint imaging unit 30L and the right-viewpoint imaging unit 30R are arranged in the horizontal direction on the front side of the housing 20 with the left-viewpoint imaging unit 30L on the right side and the right-viewpoint imaging unit 30R on the left side. They are arranged side by side in an example of a uniaxial direction.

FIG. 1B is a rear perspective view of the multi-view camera 10. As shown in the figure, a display unit 40 and a cross key 42 are arranged on the back of the multi-view camera 10.

The display unit 40 is a lenticular lens type 3D monitor capable of stereoscopically displaying a stereoscopic image captured by the left viewpoint imaging unit 30L and the right viewpoint imaging unit 30R. The cross key 42 is an input means for the photographer to operate the multiview camera 10.

[Configuration of imaging module]
The left viewpoint imaging unit 30L and the right viewpoint imaging unit 30R of the multi-view camera 10 use the imaging modules 50 having the same shape. As shown in FIG. 2, the imaging module 50 includes an imaging lens 54 on the front surface of the housing 52. As shown in FIG. 3, the optical system of the imaging module 50 includes an imaging lens 54 and an imaging element 56.

The imaging lens 54 includes a convex lens 54-1, a convex lens 54-2, and a concave lens 54-3, and is set to a predetermined focal length f (unit: millimeter). Each lens is arranged so that a predetermined interval is provided and the respective optical axes coincide with each other. Subject light incident from a convex lens 54-1 located on the front side of the housing 52 enters the image sensor 56 via a convex lens 54-2 and a concave lens 54-3.

The image sensor 56 is composed of a two-dimensional color CCD (Charge-Coupled Device) solid-state image sensor. A large number of photodiodes are two-dimensionally arranged at a predetermined pixel pitch s (unit: millimeter) on the light receiving surface of the image sensor 56, and a color filter is arranged in a predetermined arrangement on each photodiode. ing. The optical image of the subject formed on the light receiving surface of the image sensor 56 via the imaging lens 54 is converted into signal charges corresponding to the amount of incident light by the photodiode.

[Electric configuration of stereo camera]
As shown in FIG. 4, the multi-view camera 10 includes a left viewpoint imaging unit 30L, a right viewpoint imaging unit 30R, a display unit 40, a CPU (Central Processing Unit) 61, a ROM (Read Only Memory) 62, and an operation unit 63. , Imaging signal processing unit 65, control bus 66, data bus 67, memory control unit 68, main memory 69, digital signal processing unit 70, compression / decompression processing unit 71, integration unit 72, distance estimation unit 73, external memory control unit 74 And a block such as a display control unit 76, a camera shake correction control unit 77, and an angular velocity sensor 78.

Each block operates under the control of the CPU 61. The CPU 61 controls each block by executing a control program based on the input from the operation unit 63. In addition to the control program executed by the CPU 61, the ROM 62 stores various data necessary for control. The CPU 61 controls each block of the multi-lens camera 10 by reading the control program recorded in the ROM 62 into the main memory 69 and sequentially executing it.

The main memory 69 is composed of SDRAM (Synchronous Dynamic Random Access Memory), and is used as a program execution processing area, a temporary storage area for image data, and various work areas.

The operation unit 63 includes the shutter button 22 and the cross key 42 shown in FIG. 1 and outputs a signal corresponding to each operation to the CPU 61.

The left viewpoint imaging unit 30L includes an imaging lens 54L and an imaging element 56L. Similarly, the right viewpoint imaging unit 30R includes an imaging lens 54R and an imaging element 56R. A large number of light receiving elements (not shown) are two-dimensionally arranged at a pixel pitch s (unit: millimeter) on the light receiving surfaces of the

image pickup elements

56L and 56R, and red (not shown) corresponding to each light receiving element. R, green (G), and blue (B) primary color filters are arranged in a predetermined arrangement structure. The subject light imaged on the light receiving surface is converted into an electrical signal by each light receiving element and accumulated.

The electrical signal accumulated in each light receiving element is read out to a vertical transfer path (not shown). The vertical transfer path transfers this signal to a horizontal transfer path (not shown) line by line in synchronization with a clock supplied from an image sensor driving unit (not shown). Further, the horizontal transfer path outputs the signal for one line transferred from the vertical transfer path to the imaging signal processing unit 65 in synchronization with a clock supplied from an imaging element driving unit (not shown).

Note that the output of the image signal is started when the multiview camera 10 is set to the photographing mode. That is, when the multi-lens camera 10 is set to the photographing mode, output of an image signal is started in order to display a live view image on the display unit 40. The output of the image signal for the live view image is temporarily stopped when an instruction for the main photographing is given, and is restarted when the main photographing is finished. The live view image may be displayed based on the image signal captured by the left viewpoint imaging unit 30L, or the image signal captured by the left viewpoint imaging unit 30L and the image signal captured by the right viewpoint imaging unit 30R. May be displayed in a stereoscopic manner.

Also, at the end of the main shooting, the captured image is displayed on the display unit 40 for a certain period of time (post view). The user can confirm whether or not the captured image has been properly captured by confirming the postview.

The imaging signal processing unit 65 includes a correlated double sampling circuit, a clamp processing circuit, an automatic gain control circuit, and an analog / digital converter (not shown). The correlated double sampling circuit removes noise contained in the image signal. The clamp processing circuit performs processing for removing dark current components. Further, the automatic gain control circuit amplifies the image signal from which the dark current component has been removed with a predetermined gain corresponding to a set ISO (International Standards Organization) sensitivity (imaging sensitivity). The analog image signal subjected to the required signal processing is converted into a digital image signal having a gradation width of a predetermined bit by an analog / digital converter. This image signal is so-called RAW data (raw image data), and has a gradation value indicating the density of R, G, and B for each pixel. This digital image signal is stored in the main memory 69 via the data bus 67 and the memory control unit 68.

In addition to the CPU 61 and the imaging signal processing unit 65, the control bus 66 and the data bus 67 include a memory control unit 68, a digital signal processing unit 70, a compression / decompression processing unit 71, an integration unit 72, a distance estimation unit 73, and an external memory control unit. 74, a display control unit 76, and the like are connected, and these can transmit and receive information to and from each other via a data bus 67 based on a control signal of the control bus 66.

The digital signal processing unit 70 performs predetermined signal processing on the image signals of R, G, and B colors stored in the main memory 69, and outputs an image signal (Y) composed of a luminance signal Y and color difference signals Cr and Cb. / C signal). In addition, the digital signal processing unit 70 generates an image having a resolution higher than the resolution of the

imaging elements

56L and 56R from the image signal captured by the left viewpoint imaging unit 30L and the image signal captured by the right viewpoint imaging unit 30R. You may make it function as a high-resolution image production | generation means.

In accordance with a compression command from the CPU 61, the compression / decompression processing unit 71 converts the input luminance signal Y and color difference signals Cr and Cb into an image signal (Y / C signal) in a predetermined format (for example, JPEG (Joint Photographic Experts Group). ) To generate compressed image data. Further, in accordance with a decompression command from the CPU 61, the input compressed image data is subjected to decompression processing in a predetermined format to generate non-compressed image data.

The accumulating unit 72 takes in R, G, and B image signals stored in the main memory 69 in accordance with a command from the CPU 61, and calculates an accumulated value necessary for AE control. The CPU 61 calculates a luminance value from the integrated value and obtains an exposure value from the luminance value. Further, the aperture value and the shutter speed are determined from the exposure value according to a predetermined program diagram.

The distance estimation unit 73 (an example of a distance measurement unit and an example of a parallelization processing unit) performs a parallelization process on the image captured by the left viewpoint imaging unit 30L and the image captured by the right viewpoint imaging unit 30R, and performs the parallelization process. The distance from the multiview camera 10 to the subject is estimated based on the performed image. Here, the collimating process is a process of converting both images so as to be equivalent to an image captured by a parallel optical system. The deviation amounts in the optical axis direction of the left viewpoint imaging unit 30 </ b> L and the right viewpoint imaging unit 30 </ b> R necessary for the parallelization processing are stored in advance in the ROM 62.

The external memory control unit 74 controls reading / writing of data with respect to the recording medium 75 in accordance with a command from the CPU 61. The recording medium 75 may be detachable from the main body of the multi-view camera 10 such as a memory card, or may be built in the main body of the multi-view camera 10. In the case of detachable, a card slot is provided in the main body of the multi-view camera 10, and the card slot is used by being loaded.

The display control unit 76 controls display on the display unit 40 in accordance with a command from the CPU 61. The display unit 40 is a lenticular lens type 3D monitor as described above, and has directivity for the left viewpoint image captured by the left viewpoint imaging unit 30L and the right viewpoint image captured by the right viewpoint imaging unit 30R. Can be displayed simultaneously. The display unit 40 displays a moving image (live view image) and can be used as an electronic viewfinder, and displays a photographed image before recording (postview image), a reproduced image read from the recording medium 75, and the like. Is possible.

The camera shake correction control unit 77 corrects camera shake of an image captured by moving the

image sensors

56L and 56R in a plane orthogonal to the optical axis direction in accordance with camera shake accompanying the shooting operation of the multi-lens camera 10. The angular velocity sensor 78 is detection means for detecting camera shake occurring in the multi-lens camera 10 and outputs a signal representing the angular velocity of the multi-lens camera 10 with respect to the horizontal direction and the vertical direction.

The angular velocity signals in the horizontal direction and the vertical direction detected by the angular velocity sensor 78 are input to the CPU 61 via the data bus 67. The CPU 61 calculates the horizontal and vertical movement amounts of the image sensors 56 </ b> L and 56 </ b> R from the input angular velocity signal and outputs the movement amounts to the camera shake correction control unit 77. The camera shake correction control unit 77 shifts (moves) the

image sensors

56L and 56R while keeping the direction of the light receiving surface constant by the horizontal and vertical movement amounts acquired from the CPU 61. By moving the

imaging elements

56L and 56R in this way, it is possible to correct camera shake of the left viewpoint image captured by the left viewpoint imaging unit 30L and the right viewpoint image captured by the right viewpoint imaging unit 30R.

[Distortion mechanism of imaging module]
Next, the generation mechanism of the distortion shape of the captured image of the imaging module 50 will be described with reference to FIG.

FIG. 5A is a diagram showing how the imaging module 50 captures a subject. In the ideal state where the optical axes of the convex lens 54-1, the convex lens 54-2, and the concave lens 54-3 are all coincident. The relationship between the subject light and the optical image of the subject is shown. FIG. 5B shows an example of an optical image of a subject imaged on the image sensor 56 of the imaging module 50 shown in FIG. As described above, when the optical axes of the convex lens 54-1, the convex lens 54-2, and the concave lens 54-3 are all coincident, no distorted shape is generated in the captured image.

On the other hand, FIG. 5C is a diagram showing a state of photographing in the imaging module 50 in which the lens is decentered due to a manufacturing error. Here, the optical axis of the concave lens 54-3 is the convex lens 54-1. In addition, it shows a state in which it is decentered vertically above the optical axis of the convex lens 54-2. In the case of this imaging module 50, although it is focused at the center of the imaging device 56, the focusing position is the front side of the subject with respect to the upper side in the vertical direction of the imaging device (the optical image on the lower side in the vertical direction of the subject). The front pin state and the lower side in the vertical direction of the image sensor (optical image on the upper side in the vertical direction of the subject) are in the rear pin state in which the in-focus position is a position on the back side of the subject.

(D) of FIG. 5 shows an example of an optical image of a subject formed on the image sensor 56 of the imaging module 50 shown in (c) of FIG. As shown in the figure, the subject is focused on the center in the vertical direction, but the upper and lower sides of the subject are not in focus as the distance from the center in the vertical direction increases, resulting in a blurred image.

In order to prevent a reduction in image resolution due to such eccentricity of the lens, the imaging module 50 is an optical system in which the imaging element 56 is aligned with respect to the optical axis of the imaging lens 54 and an image is formed on the imaging element 56. The resolving power of the image is corrected. FIG. 5E is a diagram illustrating a state in which the subject is photographed by the imaging module 50 that has been subjected to the alignment adjustment. As an example of alignment adjustment, the imaging module 50 shown in FIG. 5E uses a horizontal line passing through the center of the light receiving surface of the image sensor 56 as a fulcrum, and brings the upper side in the vertical direction of the image sensor 56 closer to the subject. The image sensor 56 is tilted and adjusted so that the lower side in the vertical direction is away from the subject. For details of alignment adjustment, reference can be made to, for example, Japanese Patent Application Laid-Open No. 2010-21985.

(F) of FIG. 5 shows an example of an optical image of a subject formed on the image sensor 56 of the imaging module 50 shown in (e) of FIG. As shown in the figure, by performing alignment adjustment, an image with corrected resolution can be taken. However, an image photographed by the imaging module 50 that has been adjusted in this way has a distorted shape. In the example shown in FIG. 5F, a trapezoidal distortion shape is generated.

[Distortion shape correction method]
6A is a diagram illustrating the left-viewpoint imaging unit 30L and the right-viewpoint imaging unit 30R of the multiview camera 10 and the subject that is captured by the multiview camera 10. FIG. Here, the left viewpoint imaging unit 30L and the right viewpoint imaging unit 30R are arranged so that the optical axes of the

imaging lenses

54L and 54R are parallel to each other.

When the imaging module 50 used as the left viewpoint imaging unit 30L and the imaging module 50 used as the right viewpoint imaging unit 30R are both in the ideal state shown in FIG. 5A, FIG. As shown in FIG. 5, the left viewpoint image captured by the left viewpoint imaging unit 30L and the right viewpoint image captured by the right viewpoint imaging unit 30R have the same image distortion shape. Therefore, the multiview camera 10 can estimate accurate distance information to the subject using the left and right viewpoint images.

However, for example, the imaging module 50 used as the right viewpoint imaging unit 30R is in the ideal state illustrated in FIG. 5A, and the imaging module 50 used as the left viewpoint imaging unit 30L is illustrated in FIG. In the case of the imaging module 50 in which the alignment is adjusted, the distortion shapes of the left and right viewpoint images do not match as shown in FIG. Therefore, when the multi-view camera 10 estimates the distance information to the subject using the left and right viewpoint images, a positional shift occurs in the same subject in the left and right viewpoint images, and thus an error occurs in the estimated distance information.

The theoretical limit when obtaining distance information with a twin-lens camera is
(Distance resolution) = (Distance to subject) ^ 2 ÷ (Focal distance ÷ Pixel pitch) ÷ Camera interval. Note that “(distance to subject) ^ 2” indicates the square of (distance to subject). For example, when the distance to the subject is 1000 millimeters, the focal length of the imaging lens is 3.5 millimeters, the pixel pitch of the imaging element is 1.4 micrometers, and the camera interval is 50 millimeters,
(Distance resolution) = 1000 ^ 2 / ÷ (3.5 ÷ 0.0014) ÷ 50 = 8.0 millimeters. Therefore, when the distortion shapes of the left and right viewpoint images match, the limit is that it is possible to separate whether the measured distance is 1000 millimeters or 1008 millimeters.

Further, since this 8.0 millimeter corresponds to one pixel on the image, if a displacement of one pixel occurs due to distortion, a measurement error of 8.0 millimeter occurs. For this reason, if the distance measurement accuracy is ± 10 percent when the subject distance is 1000 millimeters, the allowable distortion is 1000 millimeters × ± 10 percent ÷ 8.0 millimeters = ± 12.5 pixels.

Here, the distortion shape of the left and right viewpoint images can be matched by actually measuring the distortion shape of the left and right viewpoint images and deforming both viewpoint images or one of the viewpoint images by image processing. FIG. 6D illustrates a left and right viewpoint image in which the left viewpoint image captured by the left viewpoint imaging unit 30L is deformed to match the distortion shape. The multi-lens camera 10 can estimate accurate distance information to the subject by using the left and right viewpoint images after the image processing.

However, there are problems in that image processing is computationally expensive and the resolution of the viewpoint image is degraded.

Therefore, in the present embodiment, the left and right viewpoint images are mechanically matched in distortion by providing an angle between the optical axis of the left viewpoint imaging unit 30L and the optical axis of the right viewpoint imaging unit 30R. FIG. 6E shows the optical axis of the left viewpoint imaging unit 30L and the optical axis of the right viewpoint imaging unit 30R by fixing the right viewpoint imaging unit 30R as a reference and tilting the left viewpoint imaging unit 30L in the vertical direction. Shows a state in which an angle is given in the vertical direction. Moreover, (f) of FIG. 6 has shown the left-right viewpoint image image | photographed in the left viewpoint imaging part 30L and the right viewpoint imaging part 30R shown in (e) of FIG. By adjusting the optical axis in this way, the distortion shapes of the left and right viewpoint images can be matched as shown in FIG.

[Manufacturing method of multi-lens camera]
A method for manufacturing the multi-lens camera 10 will be described with reference to a flowchart shown in FIG.

First, the two imaging modules 50 and the fixing member 80 are prepared, and the imaging module 50 (an example of the first imaging module) used as the reference right viewpoint imaging unit 30R is fixed to the fixing member 80 (step S1, fixing). Process). It is assumed that the resolution of the optical image is corrected in each of the two imaging modules 50 by alignment adjustment. The fixing member 80 (an example of a support member) is a plate-like member for fixing the left viewpoint imaging unit 30L and the right viewpoint imaging unit 30R, and has a rectangular hole as shown in FIGS. 8A and 9A. 80R and 80L are provided. As shown in FIGS. 8B and 9B, the imaging module 50 used as the right viewpoint imaging unit 30R is bonded and fixed so as to close the hole 80R.

Subsequently, the imaging module 50 (an example of the second imaging module) used as the left viewpoint imaging unit 30L is temporarily fixed to the fixing member 80 (step S2, temporary fixing step). The imaging module 50 used as the left viewpoint imaging unit 30L is supported by a support member 82 and temporarily fixed so as to close the hole 80L as shown in FIGS. 8C and 9C. At this time, the optical axis of the imaging lens 54R (an example of the first imaging lens) of the right viewpoint imaging unit 30R and the optical axis of the imaging lens 54L (an example of the second imaging lens) of the left viewpoint imaging unit 30L are parallel to each other. Thus, the left viewpoint imaging unit 30L is temporarily fixed.

Next, a chart is photographed by the left viewpoint imaging unit 30L (an example of a temporarily fixed second imaging module) and the right viewpoint imaging unit 30R (an example of a fixed first imaging module) (step S3, Shooting process).

The chart according to the present embodiment is a dot pattern in which dots of a certain size are arranged at regular intervals, as shown in FIG. 10A. The chart is not limited to a dot pattern, and a cross pattern, a lattice pattern, a chess board pattern, or the like can be used. As shown in FIG. 10B, the chart is captured by the left viewpoint imaging unit 30L and the right viewpoint imaging unit 30R, and a left viewpoint image and a right viewpoint image (an example of an image) are acquired.

Here, a line connecting the center of the light receiving surface of the image sensor 56L (an example of the second image sensor, see FIG. 4) and the center of the light receiving surface of the image sensor 56R (an example of the first image sensor, see FIG. 4) X An axis that passes through the center of the light receiving surface of the image sensor 56R and that is perpendicular to the X axis on the light receiving surface of the image sensor 56R passes through the Y axis, passes through the center of the light receiving surface of the image sensor 56R, and is orthogonal to the X axis and Y axis Is the Z axis, the optical axis directions of the imaging lens 54L and the imaging lens 54R are parallel to each other. Therefore, there is a deviation in the X direction between the imaging range in the left viewpoint imaging unit 30L and the imaging range in the right viewpoint imaging unit 30R. , Y direction coincides.

Next, the distortion shape of the left viewpoint image and the distortion shape of the right viewpoint image are calculated from the acquired left viewpoint image and right viewpoint image (step S4, calculation step). 11A is a graph showing the distortion shape of the left viewpoint image, and FIG. 11B is a graph showing the distortion shape of the right viewpoint image. In these graphs, the horizontal axis is the distance from the center of the viewpoint image in percentage units, the vertical axis is the distortion amount in pixels (if the distortion is positive, the shift to the outside of the screen, if it is negative, the screen center direction The distortion in the entire image is plotted. Since the left viewpoint imaging unit 30L and the right viewpoint imaging unit 30R are respectively adjusted in alignment, the distortion shape of the left viewpoint image and the distortion shape of the right viewpoint image are different.

Subsequently, an optical axis tilt angle of the left viewpoint imaging unit 30L for matching the distortion shape of the right viewpoint image and the distortion shape of the left viewpoint image is calculated (step S5, calculation step). That is, the difference between the distortion shape is calculated from the distortion shape of the left viewpoint image calculated in step S4 and the distortion shape of the right viewpoint image, and left viewpoint imaging based on the right viewpoint imaging unit 30R for absorbing the calculated difference. The tilt angle of the optical axis of the part 30L is calculated. The relationship between the tilt angle of the optical axis and the amount of change in the distortion shape is stored in advance as a table.

Next, the left viewpoint imaging unit 30L is tilted by the support member 82 by the calculated tilt angle of the optical axis, and the left viewpoint imaging unit 30L is bonded and fixed to the fixing member 80 in this tilted state (step S6, adjustment process).

Finally, the fixing member 80 to which the left viewpoint imaging unit 30L and the right viewpoint imaging unit 30R are fixed is attached to the housing 20 of the multiview camera 10 (step S7).

12 (a) and 12 (b) are graphs showing the distortion shape of the left viewpoint image and the distortion shape of the right viewpoint image in the multi-view camera 10 manufactured as described above. As shown in the figure, the distortion shape of the left viewpoint image matches the distortion shape of the right viewpoint image. In this manner, the multi-view camera 10 that mechanically absorbs individual differences in the distortion shape of the left and right viewpoint images can be manufactured.

[Operation of multi-lens camera]
The operation of the multi-lens camera 10 configured as described above will be described.

FIG. 13A shows the optical axis of the imaging lens 54L of the left viewpoint imaging unit 30L and the optical axis of the imaging lens 54R of the right viewpoint imaging unit 30R. As shown in the figure, where the optical axes of the imaging lens 54R of the imaging lens 54L is assumed to face the angle theta ₁ different directions in the YZ plane.

Further, as shown in FIG. 13B, the light receiving surface of the image sensor 56L and the light receiving surface of the image sensor 56R face different directions by an angle θ ₂ in the YZ plane. This angle θ ₂ is
| θ ₂ | <| θ ₁ |
Have the relationship.

Here, the angle θ ₂ is
0.1 ° <| θ ₂ | <2 °
Meet the conditions. More preferably, 0.1 ° <| θ ₂ | <1 °
And even more preferably 0.1 ° <| θ ₂ | <0.5 °
Meet the conditions.

Further, the multi-eye camera 10, the angle theta _1, the difference in Y direction and imaging range of the imaging range and the right-view imaging unit 30R of the left-view imaging unit 30L as shown in FIG. 14A occurs.

Here, as shown in FIG. 14B, the height in the Y direction of the overlapping region of the left viewpoint image captured by the left viewpoint imaging unit 30L and the right viewpoint image captured by the right viewpoint imaging unit 30R is I, and the right viewpoint image When the height in the Y direction is H, the ratio I / H of the overlapping area to the viewpoint image is
96% <I / H × 100 <99.8%
More preferably, 98% <I / H × 100 <99.8%
More preferably 99% <I / H × 100 <99.8%
Meet the conditions.

The height of the overlapping region in the Y direction is shifted by shifting one or both of the imaging element 56L of the left viewpoint imaging unit 30L and the imaging element 56R of the right viewpoint imaging unit 30R in the Y direction by the camera shake correction control unit 77. Can be changed.

Consider the case where the multi-eye camera 10 captures the chart shown in FIG. Here, consider a case where the distance L between the multiview camera 10 and the chart is 1 meter. Here, the shooting distance is set to 1 meter (1000 millimeters) because the smartphone, which is the main application of the small camera module, often shoots at a subject distance of about 1 meter, so the distance measurement accuracy at 1 meter is the highest. This is because it is common to design / produce to be.

FIG. 15A illustrates a left viewpoint image (an example of a second image) captured by the left viewpoint imaging unit 30L, a right viewpoint image (an example of a first image) captured by the right viewpoint imaging unit 30R, It is a figure which shows the overlapping area | region which is an area | region image | photographed with both the right-and-left viewpoint images. Here, a dot arranged at the center of the overlapping area is referred to as a center index O, and dots arranged around the overlapping area are referred to as a peripheral index A, a peripheral index B, a peripheral index C, and a peripheral index D. Here, the dots arranged at the four corners (an example of at least four locations) of the overlapping area are used as the peripheral index A, the peripheral index B, the peripheral index C, and the peripheral index D.

As shown in FIG. 15B, the distances from the central index O to the peripheral index A, the peripheral index B, the peripheral index C, and the peripheral index D in the left viewpoint image are represented by P _A 1, P _B 1, P _{C, respectively.} 1 and P _D 1. Further, as shown in FIG. 15C, the distances from the central index O to the peripheral index A, the peripheral index B, the peripheral index C, and the peripheral index D in the right viewpoint image are represented by P _A 2, P _B 2, Let P _C 2 and P _D 2.

Further, as shown in FIG. 16, the focal lengths of the

imaging lenses

54L and 54R of the multi-lens camera 10 are f millimeters, the center distance between the center of the light receiving surface of the image sensor 56L and the center of the light receiving surface of the image sensor 56R is d millimeters, and The pixel pitch of the

elements

56L and 56R is s millimeters.

At this time, when the difference between the distance _{P A} 1 and the distance _{P A} 2 and [Delta] P _A, the [Delta] P _A (picture elements (pixels)),
0 <ΔP _A <| 0.1 × (d × f) / (1000 × s) |
Satisfy the condition of

Here, 0.1 indicates a distance measurement accuracy of 10%.

Similarly, the difference between the distance P _B 1 and the distance P _B 2 is ΔP _B (pixel), the difference between the distance P _C 1 and the distance P _C 2 is ΔP _C (pixel), and the distance P _D 1 and the distance P _D 2 And ΔP _D (pixel), 0 <ΔP _B <| 0.1 × (d × f) / (1000 × s) |
0 <ΔP _C <| 0.1 × (d × f) / (1000 × s) |
0 <ΔP _D <| 0.1 × (d × f) / (1000 × s) |
Satisfy the condition of

Note that it is only necessary to satisfy at least one of these conditions. In addition, the peripheral index may be set to dots other than the four corners in the overlapping area, or may be set to five or more locations.

[Configuration of information terminal device]
Next, a smartphone 100 with a multi-camera that is an information terminal device to which the multi-camera 10 is applied will be described with reference to FIGS. 17A, 17B, and 18. In addition, the same code | symbol is attached | subjected to the part which is common in FIG. 4, and the detailed description is abbreviate | omitted.

A smartphone 100 with a multi-lens camera according to the present embodiment (an example of a multi-lens camera) is a trinocular camera including three imaging optical systems, and is configured to be able to shoot a stereoscopic image when the same subject is viewed from three viewpoints. Has been. For the left viewpoint imaging unit 30L, the middle viewpoint imaging unit 30C, and the right viewpoint imaging unit 30R, the imaging module 50 (see FIG. 2) in which the alignment is adjusted is used.

FIG. 17A is a front perspective view of the smartphone 100 with a multi-view camera, and FIG. 17B is a rear perspective view of the smartphone 100 with a multi-view camera. As illustrated in FIG. 17A, the smartphone 100 with a multi-eye camera is provided with a display 104 as a display unit on the front surface of the housing 102. The display 104 is a lenticular lens type 3D monitor capable of stereoscopically displaying a stereoscopic image captured by the left viewpoint imaging unit 30L, the middle viewpoint imaging unit 30C, and the right viewpoint imaging unit 30R. Various kinds of information can also be displayed under the control of the display control unit 76.

The display 104 is integrally formed with a touch panel 106 and can accept a touch operation by a user. An operation button 108 is provided on the front surface of the housing 102.

17B, the imaging lens 54L of the left viewpoint imaging unit 30L, the imaging lens 54C of the middle

viewpoint imaging unit

30C, and 54R of the right viewpoint imaging unit 30R are exposed on the rear surface of the housing 102. The

imaging lenses

54L, 54C, and 54R have focal lengths set to 5 millimeters.

As shown in FIG. 18, the touch panel 106 and the operation button 108 output a signal corresponding to the operation to the CPU 61. The user can perform shooting by the left viewpoint imaging unit 30L, the middle viewpoint imaging unit 30C, and the right viewpoint imaging unit 30R by operating the touch panel 106 and the operation buttons 108. In the shooting mode, the display 104 functions as an electronic viewfinder that displays a live view image.

The multi-camera-equipped smartphone 100 includes a call unit 110 and a data transmission / reception unit 112. The call unit 110 performs telephone communication via a fixed telephone network, a mobile phone network, or the like. A user can make a call with a user of another electronic device using the call unit 110. The data transmitter / receiver 112 (an example of a communication unit) transmits / receives digital data via a mobile phone base station or a wireless LAN (Local Area Network). The user can send and receive various digital data through the data sending / receiving unit 112. In addition, digital data of images taken by the left viewpoint imaging unit 30L and the right viewpoint imaging unit 30R can be transmitted to another electronic device.

In the multi-camera-equipped smartphone 100 configured as described above, the left viewpoint imaging unit 30L, the middle viewpoint imaging unit 30C, and the right viewpoint imaging unit 30R are centered on the reference middle viewpoint imaging unit 30C, the left viewpoint imaging unit 30L, and The right viewpoint imaging unit 30R is arranged side by side in the uniaxial direction with both ends. Further, the distortion shape of the left middle right viewpoint image is mechanically matched with the middle viewpoint imaging unit 30C as a reference.

As a result, the overlap height of the Y direction of a region between the left viewpoint images taken by the viewpoint image and a left-view imaging unit 30L in photographed by mid-view imaging unit 30C I _L, the Y-direction of the height of the middle viewpoint images When H is set, the ratio I _L / H of the overlapping area to the viewpoint image is
96% <I _L /H×100<99.8%
More preferably, 98% <I _L /H×100<99.8%
More preferably 99% <I _L /H×100<99.8%
Meet the conditions.

Similarly, the height in the Y direction of the overlapping area between the middle viewpoint image captured by the middle viewpoint imaging unit 30C and the right viewpoint image captured by the right viewpoint imaging unit 30R is set to I _R , and the height of the middle viewpoint image in the Y direction is set. When H is set, the ratio I _R / H of the overlapping area with respect to the viewpoint image is
96% <I _R /H×100<99.8%
More preferably, 98% <I _R /H×100<99.8%
More preferably 99% <I _R /H×100<99.8%
Meet the conditions.

Further, the light-receiving surface and the light-receiving surface of the image sensor 56L of the imaging device 56C, are oriented only different angular orientation theta _L in YZ plane.

Here, the angle θ _L is
0.1 ° <| θ _L | <2 °
Meet the conditions. More preferably, 0.1 ° <| θ _L | <1 °
And even more preferably 0.1 ° <| θ _L | <0.5 °
Meet the conditions.

Similarly, the light receiving surface and the light receiving surface of the image pickup device 56R of the imaging device 56C, are oriented only different angular orientation theta _R in the YZ plane.

Here, the angle θ _R is
0.1 ° <| θ _R | <2 °
Meet the conditions. More preferably, 0.1 ° <| θ _R | <1 °
And even more preferably 0.1 ° <| θ _R | <0.5 °
Meet the conditions.

Furthermore, when the distance L between the smartphone 100 with a multi-view camera and the chart shown in FIG. 10 is taken as 1 meter, the center index O ₁ of the overlapping area between the middle viewpoint image and the left viewpoint image, and around the overlapping area placed dot peripheral indicators a, near the index B, and peripheral indicators C and peripheral indicators from D, the center index O ₁ in the middle viewpoint image peripheral indicators a, near index B, up to around the index C and the peripheral indicators D respectively The distances are P _A 1, P _B 1, P _C 1, and P _D 1, and the distances from the center index O ₁ to the peripheral index A, the peripheral index B, the peripheral index C, and the peripheral index D in the left viewpoint image are P _A 2, P _B 2, P _C 2, and P _D 2, the focal lengths of the

imaging lenses

54C and 54L are f millimeters, and the center distance between the center of the light receiving surface of the image sensor 56C and the center of the light receiving surface of the image sensor 56L. d millimeters, the pixel pitch of the

image sensors

56C and 56L is s millimeters, the difference between the distance P _A 1 and the distance P _A 2 is ΔP _A (pixel), and the difference between the distance P _B 1 and the distance P _B 2 is ΔP _B ( Pixel), the difference between the distance P _C 1 and the distance P _C 2 is ΔP _C (pixel), and the difference between the distance P _D 1 and the distance P _D 2 is ΔP _D (pixel), respectively, 0 <ΔP _A <| 0.1 × (d × f) / (1000 × s) |
0 <ΔP _B <| 0.1 × (d × f) / (1000 × s) |
0 <ΔP _C <| 0.1 × (d × f) / (1000 × s) |
0 <ΔP _D <| 0.1 × (d × f) / (1000 × s) |
Satisfy the condition of

Similarly, the center index O _{2 of} the overlapping area between the middle viewpoint image and the right viewpoint image when the distance L between the smartphone 100 with a multi-view camera and the chart shown in FIG. The arranged dots are set as the peripheral index E, the peripheral index F, the peripheral index G, and the peripheral index H, and each of the center index O ₂ to the peripheral index E, the peripheral index F, the peripheral index G, and the peripheral index H in the middle viewpoint image. The distances are P _E 1, P _F 1, P _G 1, and P _H 1, and the distances from the center index O ₁ to the peripheral index E, the peripheral index F, the peripheral index G, and the peripheral index H in the left viewpoint image are P _{_{_{E 2, P F 2, P}}} G 2, and _P H 2, the imaging lens 54C and focal length f mm 54R, the center distance between the light receiving surface center of the light receiving surface center and an image pickup device 56R of the imaging device 56C Mm, s mm pixel pitch of the

image pickup element

56C and 56R, the distance _{P E} 1 and the distance difference of [Delta] P _E between _{P E} 2 (pixels), the distance _{P F} 1 and the distance difference [Delta] P _F (pixels between _{P F} 2 ), The difference between the distance P _G 1 and the distance P _G 2 is ΔP _G (pixel), and the difference between the distance P _H 1 and the distance P _H 2 is ΔP _H (pixel), respectively, 0 <ΔP _E <| 0 .1 × (d × f) / (1000 × s) |
0 <ΔP _F <| 0.1 × (d × f) / (1000 × s) |
0 <ΔP _G <| 0.1 × (d × f) / (1000 × s) |
0 <ΔP _H <| 0.1 × (d × f) / (1000 × s) |
Satisfy the condition of

Here, the multi-lens camera-equipped smartphone 100 has been described by taking a trinocular camera including three imaging optical systems as an example. However, the multi-lens camera may have two eyes as in the multi-lens camera 10, or four or more multi-lens cameras. It may be a camera.

The technical scope of the present invention is not limited to the scope described in the above embodiment. The configurations and the like in the respective embodiments can be appropriately combined among the respective embodiments without departing from the spirit of the present invention.

DESCRIPTION OF SYMBOLS 10 ... Multi-view camera, 30L ... Left viewpoint imaging part, 30R ... Right viewpoint imaging part, 50 ... Imaging module, 54, 54L, 54R ... Imaging lens, 56, 56L, 56R ... Imaging element, 73 ... Distance estimation part, 80 ... Fixing member, 80L, 80R ... Hole, 82 ... Supporting member, 100 ... Smartphone, 104 ... Display, 106 ... Touch panel, 108 ... Operation button, O ... Center indicator, A, B, C, D ... Peripheral indicator

Claims

A multi-lens imaging device that shoots a subject by arranging a plurality of imaging modules in a uniaxial direction,
Each of the plurality of imaging modules includes an imaging lens and an imaging device that converts an optical image of a subject received through the imaging lens into an image signal, and the imaging device corrects the resolving power of the optical image. Is aligned and attached to the imaging lens,
Among the plurality of imaging modules, the plurality of imaging modules are arranged in an overlapping region between an image captured by a first imaging module serving as a reference and an image captured by a second imaging module different from the first imaging module. When the height in the direction orthogonal to the uniaxial direction is I, and the height of the image in the direction orthogonal to the uniaxial direction of the image captured by the first imaging module is H, I / H is 96% <I /H×100<99.8%
Multi-lens imaging device that satisfies the following conditions.
The I / H is 98% <I / H × 100 <99.8%
The multi-lens imaging device according to claim 1, wherein the following condition is satisfied.
The I / H is 99% <I / H × 100 <99.8%
The multi-lens imaging device according to claim 1, wherein the following condition is satisfied.
A line connecting the center of the light receiving surface of the image sensor of the first imaging module and the center of the light receiving surface of the image sensor of the second imaging module is an X axis, and light reception of the image sensor of the first imaging module An axis on the light receiving surface of the image sensor of the first imaging module that passes through the center of the surface and is perpendicular to the X axis is a Y axis, passes through the center of the light receiving surface of the image sensor of the first imaging module, and the X When the axis perpendicular to the axis and the Y axis is the Z axis,
The inclination angle θ of the second imaging module with respect to the Y axis of the imaging element in a YZ plane passing through the center of the light receiving surface of the imaging element of the second imaging module is 0.1 ° <| θ | <2 °.
The multi-lens imaging device according to any one of claims 1 to 3, which satisfies the following condition.
Θ is 0.1 ° <| θ | <1 °
The multi-lens imaging device according to claim 4, which satisfies the following condition.
Θ is 0.1 ° <| θ | <0.5 °
The multi-lens imaging device according to claim 4, which satisfies the following condition.
When the chart is arranged at a position away from the light receiving surface of the imaging element by a predetermined distance L and the chart is photographed by the first imaging module and the second imaging module, the first imaging module In the overlapping area of the first image captured by the second imaging module and the second image captured by the second imaging module, a chart disposed at the center of the overlapping area is a central index, and is disposed around the overlapping area of the image. With the chart as a peripheral index, the center distance between the center of the light receiving surface of the image sensor of the first imaging module and the center of the light receiving surface of the image sensor of the second imaging module is d millimeters, and the focal point of the imaging lens When the distance is f millimeters, the pixel pitch of the image sensor is s millimeters, and the L is 1 meter, the central index in the first image The difference Δp pixel between the distance p1 to the peripheral index and the distance p2 from the central index to the peripheral index in the second image is 0 <Δp <| 0.1 × (d × f) / (1000 × s ) ｜
The multi-lens imaging device according to any one of claims 1 to 6, which satisfies the following condition.
In at least four of the peripheral indices, the Δp pixel is 0 <Δp <| 0.1 × (d × f) / (1000 × s) |
The multi-lens imaging device according to claim 7, wherein the condition is satisfied.
9. The multi-eye imaging apparatus according to claim 1, further comprising a parallelization processing unit configured to perform parallelization processing on images captured by the first imaging module and the second imaging module, respectively. .
Distance measuring means for measuring a distance from the multi-lens imaging device to the subject based on a first image captured by the first imaging module and a second image captured by the second imaging module; The multi-eye imaging device according to any one of claims 1 to 9.
High-resolution image generation means for generating an image having a resolution higher than the resolution of the image sensor based on the first image captured by the first imaging module and the second image captured by the second imaging module. The multi-lens imaging device according to any one of claims 1 to 10, further comprising:
The multi-eye imaging device according to claim 1, wherein the multi-eye imaging device has two eyes.
The multi-lens imaging device is a three-lens system in which three imaging modules are arranged in the uniaxial direction, the first imaging module is a central imaging module, and the second imaging module is an imaging module at both ends. The multi-eye imaging device according to any one of claims 1 to 11.
The multi-lens imaging device according to any one of claims 1 to 13,
Communication means for transmitting and receiving digital data such as images;
An information terminal device comprising:
A first imaging module comprising: a first imaging lens; and a first imaging element that converts an optical image of a subject received through the first imaging lens into an image signal, wherein the optical image A fixing step of fixing the first imaging module in which the alignment of the first imaging element is adjusted to correct the resolving power to a support member;
A second imaging module comprising: a second imaging lens; and a second imaging element that converts an optical image of a subject received through the second imaging lens into an image signal, wherein the optical image A temporary fixing step of temporarily fixing the second imaging module in which the alignment of the second imaging element is adjusted to correct the resolving power to the support member;
A photographing step of photographing images with the first imaging module fixed and the second imaging module temporarily fixed;
A calculation step of calculating a distortion shape of an image captured by the first imaging module and a distortion shape of an image captured by the second imaging module;
An adjustment step of adjusting and fixing the orientation of the second imaging module based on the calculated distortion shape, the distortion shape of an image captured by the first imaging module and the second imaging module An adjustment process for matching the distortion shape of the image captured by
With
A height in a direction perpendicular to the uniaxial direction in which the first imaging module and the second imaging module are arranged in an overlapping region of an image captured by the first imaging module and an image captured by the second imaging module. I / H is 96% <I / H × 100 <99.8, where I is the height of the image taken by the first imaging module and the height of the image perpendicular to the one-axis direction. %
Of manufacturing a multi-lens imaging device that satisfies the following condition.