TWI569087B - Image pickup device and light field image pickup lens - Google Patents

Description

Camera and light field camera lens

The present invention relates to an optical device, and more particularly to an imaging device and a light field imaging lens.

When a general camera acquires information about an object in space, it usually focuses on a certain object distance by the lens to obtain a clear image of the object on the object distance. As for objects located at other object distances, if they are within the range covered by the depth of field, a clear image of the object can still be obtained. However, if it is outside the range covered by the depth of field, a blurred object image will be obtained.

In addition, for the distribution and configuration of objects in space, a general camera only obtains two-dimensional image information viewed from a certain perspective, which lacks information on the distribution of objects in the depth direction. Therefore, the two-dimensional image obtained from a general camera cannot fully understand the distribution of each object in the three-dimensional space.

In view of this, the light field camera has been developed, which can more fully obtain the distribution and configuration information of various objects in the three-dimensional space. However, current light field cameras Regardless of whether the lens array or the lens array is used, the pixel utilization rate of the system is low due to the minimization of crosstalk of each sub-image, thereby affecting the number of pixels of the output image of the final light field camera. For example, the number of output pixels of a light field camera is currently about 10% to 65% of the number of pixels used by the image sensor.

The invention provides an imaging device which can improve the effective use area of an image sensor.

The present invention provides a light field imaging lens that contributes to an effective use area of an image formed thereby.

An embodiment of the invention provides an imaging device including an imaging lens, an image sensor, and a multiple aperture optical component. The multiple aperture optical element is disposed on a light path between the imaging lens and the image sensor and includes a plurality of aperture elements arranged in an array. The ratio of the image side aperture number (f-number) of the imaging lens divided by the object side aperture number of the imaging lens falls within a range from 0.25 to 2, and the imaging device conforms to: Where L is the pitch of the aperture elements, D is the diameter of the exit pupil of the imaging lens, P is the distance from the exit pupil of the imaging lens to the image plane of the imaging lens, and a is the image plane of the imaging lens to the multiple aperture optical element The distance value of the front principal plane, and b is the distance from the back principal plane of the multiple aperture optical element to the imaging surface of the image sensor. In addition, when the image plane of the imaging lens is located on the side of the front main plane away from the image sensor, the value of a is a negative value, and when the image plane of the imaging lens is located on the side of the front main plane close to the image sensor When a is a positive value.

An embodiment of the present invention provides a light field imaging lens comprising an imaging lens and a multiple aperture optical element. The imaging lens is disposed between an object side of the light field imaging lens and an imaging surface. The multiple aperture optical element is disposed between the imaging lens and the imaging surface of the light field imaging lens, and includes a plurality of aperture elements arranged in an array. The ratio of the number of image side apertures of the imaging lens divided by the number of apertures on the object side of the imaging lens falls within a range from 0.25 to 2, and the light field imaging lens conforms to: Where L is the pitch of these aperture elements, D is the diameter of the exit pupil of the imaging lens, P is the distance from the exit pupil of the imaging lens to the image plane of the imaging lens, and a is the image plane of the imaging lens to the front of the multiple aperture optics The distance value of the principal plane, and b is the distance from the rear principal plane of the multiple aperture optical element to the imaging plane of the light field imaging lens. In addition, when the image plane of the imaging lens is located on a side of the front main plane away from the imaging plane of the light field imaging lens, the value of a is a negative value, and when the image plane of the imaging lens is located in the front main plane close to the light field imaging lens When one side of the image plane is formed, the value of a is a positive value.

The camera device and the light field camera lens according to the embodiment of the present invention are in accordance with Therefore, the sub-images formed by the multiple aperture optical elements are large and moderately partially overlap each other. In this way, the portions of the sub-images that are not partially overlapped can be more effectively used as the image information to be obtained, thereby improving the effective use area of the image and improving the effective use area of the image sensor.

The above described features and advantages of the invention will be apparent from the following description.

100, 100a, 100b, 100c, 100d‧‧‧ camera

105‧‧‧Light field camera lens

110, 110a‧‧‧ imaging lens

112‧‧‧ lens

113‧‧‧ appear

114‧‧‧ aperture diaphragm

115‧‧‧ image plane

1122‧‧‧ positive meniscus lens

1124‧‧‧negative meniscus lens

1126‧‧‧negative meniscus lens

120‧‧‧Image Sensor

122‧‧‧ imaging surface

124‧‧‧protective glass

125, 125”’ ‧ ‧ effective use area

125’, 125”‧‧‧ inscribed square

130, 130d‧‧‧Multiple aperture optics

132, 132a, 132d‧‧‧ aperture components

1322‧‧‧ front main plane

1324‧‧‧ rear main plane

134‧‧‧Transparent sheet

135, 135’ ‧ ‧ sub-image

A‧‧‧distance value

b, b’, P‧‧‧ distance

D‧‧‧diameter

L‧‧‧ pitch

M‧‧‧ magnification

R‧‧‧ Radius

r, t‧‧‧ crosstalk ratio

S‧‧‧Bianchang

W1, W2‧‧‧ length

1 is a schematic cross-sectional view of an image pickup apparatus according to an embodiment of the present invention.

2 is a schematic view showing the exit pupil of the imaging lens, the front main plane and the rear main plane of the multiple aperture optical element, the imaging surface of the image sensor, and the manner in which light is transmitted between the imaging lenses of the imaging device of FIG. 1.

FIG. 3 illustrates a specific example of the image pickup apparatus of FIG. 1.

4 is a comparison diagram of the pitch of the aperture element, the sub-image, and the effective use area of the image sensor for one sub-image in the image pickup apparatus of FIG. 1.

FIG. 5 is a comparison diagram of the pitch of the aperture element, the sub-image, and the effective use area of the image sensor for one sub-image when the sub-images are not overlapped.

6 and FIG. 7 are comparison diagrams of the aperture of the aperture element, the sub-image, the effective use area of the image sensor for one sub-image, and the degree of overlap of the two sub-images in the image pickup apparatus of FIG. .

Figure 8 is a graph of the absolute value of the aperture of the aperture component of Figure 1 versus the number of fields of view in a sub-image at different crosstalk ratios.

FIG. 9 is a cross-sectional view showing another crosstalk ratio of the image pickup apparatus of FIG. 1. FIG.

FIG. 10 is a distribution diagram of light intensity gray scales of sub-images sensed by an image sensor of the image pickup apparatus of FIG. 1. FIG.

Figure 11 is a cross-sectional view showing an image pickup apparatus according to another embodiment of the present invention.

Figure 12 is a cross-sectional view showing an image pickup apparatus according to still another embodiment of the present invention.

Figure 13 is a cross-sectional view showing an image pickup apparatus according to still another embodiment of the present invention.

1 is a cross-sectional view of an image pickup apparatus according to an embodiment of the present invention, and FIG. 2 is a front view and a rear main plane of the imaging lens of the image pickup apparatus of FIG. 1 , and a front main plane and an image sensor of the multiple aperture optical element. FIG. 3 is a schematic diagram showing the transmission mode of the imaging surface and light on these surfaces, and FIG. 3 illustrates a specific example of the image pickup apparatus of FIG. 1. Referring to FIG. 1 and FIG. 2 , the image capturing apparatus 100 of the present embodiment includes a light field imaging lens 105 and an image sensor 120 . The light field imaging lens 105 includes an imaging lens 110 and a multiple aperture optical component 130 . . The imaging lens 110 can include at least one lens 112 and an aperture stop 114. For example, as shown in FIG. 3, the imaging lens 110a may include a positive meniscus lens 1122, a negative meniscus lens 1124, and a negative meniscus lens 1126. However, the invention is not limited thereto. Further, if the imaging lens 110 does not include the aperture stop 114 formed by the light shielding member, the exit pupil and the entrance pupil of the imaging lens 110 can still be determined by the clear aperture of the lens.

The multiple aperture optical element 130 is disposed on a light path between the imaging lens 110 and the image sensor 120 and includes a plurality of aperture elements 132 arranged in an array. In the present embodiment, the multiple aperture optical element 130 is a lens array, and the aperture elements 132 are arrayed lenses. In FIG. 3, one lens in the lens array (ie, the aperture element 132a is taken as an example) is illustrated, and the aperture element 132a may be formed on a light-transmissive sheet 134. In addition, the camera device 100a further includes a cover glass 124 covering the image sensor 120 to protect the image sensor 120.

In the present embodiment, the ratio of the image side aperture number (f-number) of the imaging lens 110 divided by the object side aperture number of the imaging lens 110 falls within a range from 0.25 to 2. The definition of the number of side apertures is a value obtained by dividing the focal length of the imaging lens 110 by the diameter of the exit pupil of the imaging lens 110, and the definition of the number of apertures on the object side is the focal length of the imaging lens 110 divided by the entrance pupil of the imaging lens 110. The value obtained after the diameter.

Further, in the embodiment, the image pickup apparatus 100 conforms to:

Where L is the pitch of the aperture elements 132, D is the diameter of the exit pupil 113 of the imaging lens 110, and P is the distance from the exit pupil 113 of the imaging lens 110 to the image plane 115 of the imaging lens 110. a is the distance value of the image plane 115 of the imaging lens 110 to the front principal plane 1322 of the multiple aperture optical element 130 (in absolute value in FIG. 2), wherein the image plane 115 of the imaging lens 110 is located far from the front main plane 1322 When one side of the image sensor 120 (ie, the left side of the front main plane 1322 in the drawing), the value of a is a negative value, and when the image plane 115 of the imaging lens 110 is located near the image sensor 120 of the front main plane 1322 On one side (ie, the right side of the front main plane 1322 in the figure), the value of a is a positive value. Therefore, in this embodiment, referring to FIG. 2, the value of a is a negative value. Moreover, b is the distance from the rear major plane 1324 of the multiple aperture optical element 130 to the imaging surface 122 of the image sensor 120. That is, in the present embodiment, the front main plane 1322 is located between the image plane 115 and the rear main plane 1324 of the imaging lens, and the rear main plane 1324 is located at the front main plane 1322 and the imaging surface 122 of the image sensor 120. between. In addition, in this embodiment, the imaging surface 122 of the image sensor 120 is The imaging surface of the light field imaging lens 105.

In the present embodiment, the aperture elements 132 respectively image the exit pupils 113 of the imaging lens 110 onto the imaging surface 122 of the image sensor 120 to form a plurality of sub-images 135 (as shown in FIG. 6). The neighbor images 135 partially overlap each other.

The imaging device 100 and the light field imaging lens 105 of the present embodiment are in accordance with Thus, the sub-images 135 formed by the multiple aperture optical elements 130 are larger and moderately partially overlap each other. In this way, the portions of the sub-images 135 that are not partially overlapped can be more effectively used as the image information to be obtained, thereby improving the effective use area of the image and improving the effective use area of the image sensor 120. An example will be given below to illustrate.

4 is a comparison diagram of the pitch of the aperture element, the sub-image and the image sensor's effective use area for one sub-image in the image pickup apparatus of FIG. 1, and FIG. 5 is a diagram of the aperture element when the sub-images are not overlapped. Comparison of the effective use area of the pitch, sub-image and image sensor for one sub-image, FIG. 6 is the pitch, sub-image, and image sensor of the aperture device of the image pickup device of FIG. A comparison map of the effective use area and the degree of overlap of the two sub-images. Referring to FIG. 4, in the embodiment, the width (eg, diameter) of the sub-image 135 is greater than the pitch L of the aperture element, and the effective utilization area 125 of the image sensor 120 for one sub-image 135 is The sub-image 135 is in the range surrounded by an inscribed rectangle such as a square. It is assumed that the width of the effective utilization area 125 taken is L(1-r), where r is the crosstalk ratio. Further, if the magnification of the aperture element 132 is M, then L(1-r)≧NML, where N is the number of views of the light field camera lens 105, and different views 137 may contain image information of different viewing angles.

In this way, the relationship of M≦(1-r)/N can be obtained. Therefore, if the adjacent sub-images 135 are to be partially overlapped, M≦1/N can be satisfied.

As shown in FIG. 5, when the adjacent sub-images 135' do not overlap each other, the radius R of the sub-image 135' satisfies 0.5 L ≧ R ≧ 0. At this time, the side length S of the inscribed square 125' of the sub-image 135' is 2 × (0.707R) = 1.414R. Taking the maximum value of 0.5 L of R into the former formula, the maximum value of the side length S of the inscribed square 125' is 1.414 × 0.5 L = 0.707 L.

On the other hand, in the present embodiment, when the adjacent sub-images 135 partially overlap, assuming that the radius of the sub-image 135 is R, then , W1 and W2 are respectively two lengths as shown in FIG. 6.

In this way, the radius R of the sub-image 135 satisfies (2-1.414) L ≧ R ≧ 0.5 L, and the side length S of the inscribed square 125 ′ of the sub-image 135 is S = 2 × (0.707 R) = 1.414 R. Therefore, the sub-image The maximum value of the side length S of the inscribed square 125" of the image 135 is 2 x (1.414-1) L = 0.828L. It can be seen that the area of the inscribed square 125" of the present embodiment (ie, FIG. 6) can be larger than that of the inscribed square 125' of FIG. 5, that is, the image sensor 120 of the embodiment is for one sub- The effective utilization area 125 of the image 135 can be large.

On the other hand, when L ≧ R ≧ (2-1.414) L, the side length S = 2 (L - R) of the effective use area 125"'. Therefore, the maximum value of the side length S of the effective use area 125"' is 2 × (1.414-1) L = 0.828 L, and the minimum value is 0. The minimum value is not used because the effective utilization area 125"' is zero at this time, as shown in Fig. 7, that is, the partial overlap of the sub-image 135 is not so large that the effective utilization area 125"' is zero. Further, in the case of the above maximum value, the effective use area 125"' is also larger than the area of the inscribed square 125' of Fig. 5. From the above, it can be proved that the adjacent sub-images 135 are partially overlapped, which is effective. The effective use area 125 of the image sensor 120 for each sub-image 135 is increased, thereby increasing the effective use area of the image sensor 120 as a whole.

Figure 8 is a graph of the absolute value of the aperture of the aperture element (i.e., lens array) of Figure 1 versus the number of fields of view in a sub-image at different crosstalk ratios. It can be seen from FIG. 8 that when the error tolerance of the crosstalk ratio r is required to be small, it means that the error tolerance of the lens array in manufacturing and assembly can be large, and the number of visual fields designed at this time is also small. .

As shown in FIG. 2 , the above-mentioned crosstalk ratio r is generated because the aperture element 132 re-images the image formed by the imaging lens 110 on the image plane 115 to the imaging surface 122 of the image sensor 120. The exit pupil 113 of the imaging lens 110 is also imaged on the imaging surface 125 to form a sub-image 135 having a diameter L (1+r). According to the principle of similar triangles, the following formula (2) can be obtained:

In addition, there is another crosstalk ratio t that affects the crosstalk of the adjacent sub-images 135. As shown in FIG. 9, any point on the exit pupil 113 of the imaging lens 110 passes through the aperture element 132 in the image sensor. The image formed on the imaging surface 122 of 120 is slightly spread out, and its spread width is Lt. According to the principle of similar triangles, the following formula (3) can be obtained:

Where b' is the distance of the exit pupil 113 of the imaging lens 110 via the imaged imaging surface of the aperture element 132 to the rear principal plane 1324 of the aperture element 132.

When the crosstalk ratio r and the crosstalk ratio t are comprehensively considered, the effective use area 125 of the image sensor 120 for one sub-image 135 can be set to L(1-r-t). When r+t is greater than 0, L(1-rt)≧NL| b/a |; and when r+t<0, L(1+r+t)≧NL| b/a |, where “| |" means the absolute value of the value. After substituting r and t with the relationship of the above formula (2) and formula (3), the above formula (1) can be obtained.

The parameters of some examples of the image pickup apparatus 100 of the present embodiment are shown in the following Tables (1) and (2).

In Table 1, Fr is the ratio obtained by dividing the number of image side apertures by the number of apertures on the object side. The value of "(1) intermediate formula" in the above table means The value of the calculated result. In addition, the physical meanings of other parameters in the above table are explained above, please refer to the above, and will not be repeated here. As can be seen from Tables 1 and 2, in all instances, the total crosstalk ratio r+t is less than 0.5. Further, as can be seen from Table 1, the image pickup apparatus 100 of the present embodiment is applicable to the total length of various lenses (applicable from several millimeters to several hundreds of millimeters). In addition, as can be seen from Table 1, the lens with the Fr value away from 1 is also applicable. Further, Table 2 exemplifies various modifications of the example 5 of the total length of the lens. When the total length of the lens is long, the camera device 100 can be applied to a portable electronic device.

In an embodiment, in order to achieve a sufficient effective use area 125, the ratio of the number of image side apertures of the imaging lens divided by the number of object side apertures of the imaging lens falls within a range from 0.4 to 1.5. Moreover, in an embodiment, in order to achieve a sufficient effective utilization area 125, the camera device conforms to:

FIG. 10 is a distribution diagram of light intensity gray scales of sub-images sensed by an image sensor of the image pickup apparatus of FIG. 1. FIG. Referring to FIG. 10, when the pitch L of the aperture member 132 is 260 micrometers, in one embodiment, the normalized crosstalk ratio is 30%.

Figure 11 is a cross-sectional view showing an image pickup apparatus according to another embodiment of the present invention. Referring to FIG. 11, the image pickup apparatus 100b of the present embodiment is similar to the image pickup apparatus 100 of FIG. 1, and the difference between the two is as follows. In the image capturing apparatus 100b of the present embodiment, the imaging surface of the imaging lens 110 is on the side of the front main plane 1322 (shown in FIG. 2) of the multiple aperture optical element 130 near the image sensor 120 (that is, The right side of the front main plane 1322), at this time, the value of a is a positive value, but can still conform to the above formula (1) or (4). Both the image pickup apparatus 100 of FIG. 1 and the image pickup apparatus 100b of the present embodiment belong to the architecture of the light field camera version 2.0.

Figure 12 is a cross-sectional view showing an image pickup apparatus according to still another embodiment of the present invention. Referring to FIG. 12, the image pickup apparatus 100c of the present embodiment is similar to the image pickup apparatus 100 of FIG. 1, and the difference between the two is as follows. In the image pickup apparatus 100c of the present embodiment, The imaging surface of the lens 110 is at the position of the multiple aperture optical element 130. This architecture is the architecture of the light field camera version 1.0, but can still conform to the above formula (1) or (4).

Figure 13 is a cross-sectional view showing an image pickup apparatus according to still another embodiment of the present invention. Referring to FIG. 13, the image pickup apparatus 100d of the present embodiment is similar to the image pickup apparatus 100 of FIG. 1, and the difference between the two is as follows. In the strip device 100d of the present embodiment, the multiple aperture optical element 130d is a light shielding sheet, and the aperture elements 132d are a plurality of light-transmissive openings (such as pinholes) arranged in an array, and the front main plane of the light shielding sheet It substantially coincides with the rear main plane. In other words, for the multiple aperture optical element 130d, both the front main plane and the rear main plane fall on the plane in which the light transmission aperture is located. At this time, the image pickup apparatus 100d can still conform to the above formula (1) or (4).

In summary, the camera device and the light field camera lens according to the embodiment of the present invention are in accordance with Therefore, the sub-images formed by the multiple aperture optical elements are large and moderately partially overlap each other. In this way, the portions of the sub-images that are not partially overlapped can be more effectively used as the image information to be obtained, thereby improving the effective use area of the image and improving the effective use area of the image sensor.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention, and any one of ordinary skill in the art can make some changes and refinements without departing from the spirit and scope of the present invention. The scope of the invention is defined by the scope of the appended claims.

100‧‧‧ camera

105‧‧‧Light field camera lens

110‧‧‧ imaging lens

112‧‧‧ lens

114‧‧‧ aperture diaphragm

115‧‧‧ image plane

120‧‧‧Image Sensor

122‧‧‧ imaging surface

130‧‧‧Multiple aperture optics

132‧‧‧ aperture components

L‧‧‧ pitch

Claims (14)

An imaging device includes: an imaging lens; an image sensor; and a multiple aperture optical element disposed on an optical path between the imaging lens and the image sensor, and comprising an array a plurality of aperture elements, wherein a ratio obtained by dividing the number of image side apertures of the imaging lens by the number of apertures of the object side of the imaging lens falls within a range from 0.25 to 2, and the imaging device conforms to: Where L is the pitch of the plurality of aperture elements, D is the diameter of the exit pupil of the imaging lens, and P is the distance from the exit pupil of the imaging lens to the image plane of the imaging lens, a is the a distance value of an image plane of the imaging lens to a front principal plane of the multiple aperture optical element, and b is a distance from a rear principal plane of the multiple aperture optical element to an imaging surface of the image sensor, wherein When the image plane of the imaging lens is located on a side of the front main plane that is away from the image sensor, the value of a is a negative value, and when the image plane of the imaging lens is located in the front main plane When it is close to one side of the image sensor, the value of a is a positive value. The image pickup device of claim 1, wherein the multiple aperture optical element is a lens array, and the plurality of aperture elements are lenses arranged in an array. The image pickup device of claim 1, wherein the multiple aperture optical element is a light shielding sheet, and the plurality of aperture elements are arranged in a plurality of arrays The light transmissive opening, and the front main plane of the light shielding sheet substantially coincides with the rear main plane. The image pickup device of claim 1, wherein the plurality of aperture elements respectively image the exit pupil of the imaging lens on the imaging surface of the image sensor to form a plurality of sub-images. The plurality of sub-images partially overlap each other. The image pickup apparatus according to claim 1, wherein the front main plane is located between the image plane of the imaging lens and the rear main plane, and the rear main plane is located in the front main plane. Between the imaging surface of the image sensor. The image pickup apparatus according to claim 1, wherein a ratio obtained by dividing the number of image side apertures of the imaging lens by the number of apertures of the object side of the imaging lens is from 0.4 to 1.5. Within the scope. The image pickup device according to claim 1, wherein the image pickup device conforms to: A light field imaging lens comprising: an imaging lens disposed between an object side and an imaging surface of the light field imaging lens; and a multiple aperture optical element disposed on the imaging lens and the light field Between the imaging faces of the imaging lens, and including a plurality of aperture elements arranged in an array, wherein the ratio of the number of image side apertures of the imaging lens divided by the number of apertures of the object side of the imaging lens is falling In the range of 0.25 to 2, and the light field camera lens conforms to: Where L is the pitch of the plurality of aperture elements, D is the diameter of the exit pupil of the imaging lens, and P is the distance from the exit pupil of the imaging lens to the image plane of the imaging lens, a is the a distance value of an image plane of the imaging lens to a front principal plane of the multiple aperture optical element, and b is a distance from a rear principal plane of the multiple aperture optical element to the imaging surface of the light field imaging lens, wherein The image plane of the imaging lens is located on a side of the front main plane away from the imaging surface of the light field imaging lens, the value of a is a negative value, and when the image of the imaging lens When the plane is located on one side of the front principal plane close to the imaging plane of the light field imaging lens, the value of a is a positive value. The light field imaging lens of claim 8, wherein the multiple aperture optical element is a lens array, and the plurality of aperture elements are lenses arranged in an array. The light field imaging lens of claim 8, wherein the multiple aperture optical element is a light shielding sheet, and the plurality of aperture elements are a plurality of light-transmissive apertures arranged in an array, and the shading The front main plane of the sheet substantially coincides with the rear main plane. The light field imaging lens of claim 8, wherein the plurality of aperture elements respectively image the exit pupil of the imaging lens on the imaging surface of the light field imaging lens to form a plurality of sub-images An image, wherein the plurality of sub-images partially overlap each other. The light field imaging lens of claim 8, wherein the front main plane is located between the image plane of the imaging lens and the rear main plane, and the rear main plane is located before the front A principal plane is between the imaging surface of the light field imaging lens. The light field imaging lens of claim 8, wherein the ratio of the number of image side apertures of the imaging lens divided by the number of apertures of the object side of the imaging lens is from 0.4 to Within the scope of 1.5. The light field imaging lens of claim 8, wherein the camera device meets: