JP2018066965A - Imaging unit and resin lens assembling method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable information processing to be performed with high accuracy by easily introducing an imaging mechanism of a polarization image.SOLUTION: A resin lens is introduced as an imaging optical system of an imaging device. The resin lens has lens parts 138a, 138b, and gate parts 136a, 136b molded at positions corresponding to a gate of a die into which a resin material flows during injection molding. The resin lens is assembled to a barrel so as to be layered in optical axis direction, in such an orientation that diameters Aa, Ab passing through the gate parts 136a, 136b are orthogonal to each other.SELECTED DRAWING: Figure 6

Description

  The present invention relates to an imaging unit for acquiring information related to the state of a subject space by photographing, and a method for assembling a resin lens.

  A polarization camera that acquires polarization information of a subject by mounting a polarization filter on a sensor has been put into practical use. By using a polarization camera, it is possible to obtain information on the surface orientation of a subject to identify an object or detect the presence of a transparent object. References 1 and 2).

JP 2012-80065 A JP 2014-57231 A

  As described above, the field of introduction of the polarization camera is limited at present. On the other hand, information on real objects obtained by polarization is considered useful in more general information processing such as electronic games. When pursuing such versatility, it is desired to improve the usability while accurately obtaining the target information.

  The present invention has been made in view of these problems, and an object of the present invention is to provide a technique capable of easily introducing a polarization image photographing mechanism and performing accurate information processing.

  One embodiment of the present invention relates to an imaging unit. This imaging unit includes a resin lens for realizing imaging on an imaging surface in an imaging device, and a lens barrel in which the resin lens is assembled, and the resin lens is oriented on a plane perpendicular to the optical axis The resin lens is fixed according to a predetermined rule based on the principal axis of optical anisotropy of the resin lens.

  The “imaging unit” may be a partial unit constituting the imaging apparatus such as a lens unit, an imaging optical system, an imaging element, an image processing unit, or a unit including them.

  Yet another embodiment of the present invention relates to a method for assembling a resin lens. This method of assembling the resin lens is a method of assembling a plurality of resin lenses for realizing imaging on the imaging surface in the imaging apparatus to the lens barrel, and the inflow portion of the resin material at the time of injection molding solidifies. The resin lens is fixed so that the diameters passing through the gate portion formed in each resin lens are in two different directions.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, and the like are also effective as an aspect of the present invention.

  According to the present invention, a polarization image capturing mechanism can be easily introduced to perform accurate information processing.

It is a figure which shows notionally the structure of the imaging device in this Embodiment. It is a figure which shows the structural example of the image pick-up element with which the imaging device of this Embodiment is provided. It is a figure which shows the example which compared the normal distribution acquired from the polarization image with the raw material of a lens. It is a figure which shows the example which compared the polarization degree image with the raw material of a lens. It is a figure which shows typically the molding process of the resin lens in this Embodiment, and the state after shaping | molding. It is a figure which illustrates a mode that two lenses were assembled | attached so that a principal axis might be orthogonally crossed in this Embodiment. It is a figure which shows the example of an assembly | attachment when there are five lenses in this Embodiment. In this Embodiment, it is a figure which shows the effect by assembling | attaching a lens in the direction where a main axis is orthogonal. It is a figure which shows the error of the azimuth | direction angle of a normal, and the zenith angle calculated from the polarization image in this Embodiment. It is a figure which illustrates in-plane distribution of the normal image which generate | occur | produces when a polarization state changes. It is a figure which shows typically the path | route of the light which permeate | transmits the lens in this Embodiment. It is a figure which shows the structure of the functional block of the polarization image processing apparatus which has a function which acquires normal line information, correct | amending the in-plane distribution of a polarization state in this Embodiment.

  FIG. 1 is a diagram conceptually showing the configuration of the imaging apparatus in the present embodiment. The imaging device 12 may be realized as a single digital camera, or may be a part of a device having a function other than shooting, such as a portable terminal or a game controller. For example, it may be a part of a head mounted display that displays an image in front of the user by wearing it on the head. In this case, the imaging device 12 captures a surrounding state with a field of view corresponding to the user's line of sight, thereby displaying the captured image as it is, or rendering an image based on the field of view or the position of the real object to display VR (virtual reality ) And AR (Augmented Reality).

  The imaging device 12 includes an imaging optical system 18 including a lens unit including lenses 10a to 10e, an imaging element array 20, and an image processing unit 22. The imaging optical system 18 forms an image of the subject on the imaging surface of the imaging element array 20. The figure shows a state in which the side surfaces of the lenses 10a to 10e are viewed from a direction perpendicular to the optical axis a. As shown in the drawing, by combining a plurality of lenses 10a to 10e having various shapes having concave and convex surfaces, the incident angle of the principal ray on the imaging surface of the imaging element array 20 can be made parallel to the optical axis a and photographing is performed. The image quality can be improved and the module height can be reduced.

  Such a lens configuration has been put to practical use in general portable terminals and the like. Although the five lenses 10a to 10e are shown in the drawing, the number of lenses is not limited thereto, and the individual shapes are not particularly limited. Note that the lens unit is actually fixedly assembled to a lens barrel (not shown), and functions as the imaging optical system 18 by being attached to the housing of the imaging device 12. The image sensor array 20 includes a two-dimensional array of image sensors, converts the detected light intensity into electric charges, and outputs the charges to the image processing unit 22. The imaging element in this embodiment has a structure that detects at least the luminances of polarization components in a plurality of directions.

  The image processing unit 22 performs image processing using the two-dimensional distribution of the luminance of the light detected by the image sensor array 20 to generate a polarization image for each direction of polarization, or analyze it to analyze the subject surface. Generate information related to lines. Furthermore, the image processing unit 22 may generate a natural light (non-polarized) luminance image or a color image by averaging polarized images in a plurality of directions. The image processing unit 22 also includes a memory for storing captured image data generated as described above.

  Note that the imaging apparatus 12 may further include an operation unit by the user and a mechanism for performing a shooting operation, a shooting condition adjustment operation, and the like according to the operation content. The imaging device 12 has a mechanism for establishing communication with an external device such as a game machine or a personal computer by wire or wireless, and transmitting generated data or receiving a control signal such as a data transmission request. It's okay. However, since these mechanisms may be the same as those of a general imaging device, description thereof is omitted.

  FIG. 2 shows an example of the structure of the image sensor provided in the imaging device 12. This figure schematically shows the functional structure of the element cross section, and detailed structures such as an interlayer insulating film and wiring are omitted. The image sensor 110 includes a microlens layer 112, a polarizer layer 114, a color filter layer 116, and a light detection layer 118. The polarizer layer 114 includes a polarizer in which a plurality of linear conductor members are arranged in stripes at intervals smaller than the wavelength of incident light. When the light condensed by the microlens layer 112 enters the polarizer layer 114, the polarization component in the direction parallel to the line of the polarizer is reflected, and only the perpendicular polarization component is transmitted.

  By detecting the transmitted polarization component with the light detection layer 118, the polarization luminance is acquired. The light detection layer 118 has a semiconductor element structure such as a general CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The polarizer layer 114 includes an array of polarizers having different principal axis angles in units of charge reading in the light detection layer 118, that is, in units of pixels or larger. The right side of the figure illustrates a polarizer array 120 when the polarizer layer 114 is viewed from above.

  In the figure, the shaded lines are conductors (wires) constituting the polarizer. Each dotted rectangle represents a region of a polarizer having one principal axis angle, and the dotted line itself is not actually formed. In the illustrated example, polarizers having four main axis angles are arranged in four regions 122a, 122b, 122c, and 122d in two rows and two columns. In the figure, the polarizers on the diagonal are orthogonal in principal axis, and adjacent polarizers have a 45 ° difference. That is, polarizers with four main axis angles every 45 ° are provided.

  Each polarizer transmits a polarization component in a direction orthogonal to the direction of the wire. Thereby, in the light detection layer 118 provided below, it is possible to obtain the luminances of the polarization components in four directions every 45 ° in each of the regions corresponding to the four regions 122a, 122b, 122c, and 122d. By arranging a predetermined number of such polarizer arrays with four principal axis angles in the vertical and horizontal directions and connecting peripheral circuits for controlling the charge readout timing, the four types of polarization luminances are made into a two-dimensional distribution. An image sensor to be acquired can be realized.

  In the image pickup device 110 shown in the figure, a color filter layer 116 is provided between the polarizer layer 114 and the light detection layer 118. The color filter layer 116 includes an array of filters that respectively transmit red, green, and blue light. Thereby, the polarization luminance is obtained for each color according to the combination of the principal axis angle of the polarizer in the polarizer layer 114 positioned above and below and the color of the filter in the color filter layer 116.

  That is, the luminance of polarized light of the same color and the same direction can be obtained discretely on the image plane. The image processing unit 22 appropriately interpolates them to obtain a polarization image for each combination of direction and color. In addition, the polarization characteristics are obtained using the polarization luminance in the four directions of the same color to generate normal information, or a non-polarized color image in which the average value of the polarization luminance in the four directions is the luminance of the color at the position. Can be generated.

  An image acquisition technique using a wire grid type polarizer as illustrated is disclosed in, for example, Japanese Patent Application Laid-Open No. 2012-80065, and a technique for acquiring a color polarization image is disclosed in, for example, International Publication No. JP2008-001136. Has been. However, the element structure of the imaging device 12 in the present embodiment is not limited to that illustrated. For example, the polarizer is not limited to the wire grid type, and may be any of those that have been put into practical use, such as a linear dichroic polarizer.

Next, a method for generating normal line information of a subject using polarized luminances in a plurality of directions acquired by the element structure shown in FIG. 2 will be described. For example, Gary Atkinson and Edwin R. Hancock, “Recovery of Surface Orientation from Diffuse Polarization”, IEEE Transactions on Image Processing, June 2006, 15 (6), pp.1653-1664, JP2009-58533A. And the like. The outline will be described below. First, the brightness of the light observed through the polarizer changes as shown in the following equation with respect to the principal axis angle θ _pol of the polarizer.

Here, I _max and I _min are the maximum value and the minimum value of luminance, respectively, and φ is the polarization phase. As described above, when a polarization image is acquired with respect to the four principal axis angles θ _pol , the luminance I of the pixels at the same position satisfies Equation 1 for each principal axis angle θ _pol . Therefore, I _max , I _min , and φ can be obtained by approximating a curve passing through these coordinates (I, θ _pol ) to a cosine function using the least square method or the like. Using the thus obtained I _max and I _min , the degree of polarization ρ is obtained by the following equation.

The normal of the surface of the object can be expressed by an azimuth angle α representing the angle of the light incident surface (or exit surface in the case of diffuse reflection) and a zenith angle θ representing the angle on the surface. Further, according to the dichroic reflection model, the spectrum of the reflected light is represented by a linear sum of the specular reflection and diffuse reflection spectra. Here, specular reflection is light that is specularly reflected on the surface of the object, and diffuse reflection is light that is scattered by pigment particles that constitute the object. The azimuth angle α described above is the principal axis angle that gives the minimum luminance I _min in Equation 1 in the case of specular reflection, and the principal axis angle that gives the maximum luminance I _max in Equation 1 in the case of diffuse reflection.

The zenith angle θ has the following relationship with the degree of polarization ρ _{s in} the case of specular reflection and the degree of polarization ρ _{d in} the case of diffuse reflection.

Here, n is the refractive index of the object. The zenith angle θ is obtained by substituting the degree of polarization ρ obtained by Equation 2 into either ρ _s or ρ _d of Equation 3. Based on the azimuth angle α and the zenith angle θ thus obtained, the normal vectors (p _x , p _y , p _z ) are obtained as follows.

In this way, the normal vector of the object shown in the pixel is obtained from the relationship between the luminance I represented by each pixel of the polarization image and the principal axis angle θ _pol of the polarizer, and a normal vector distribution can be obtained for the entire image. it can. For example, in an aspect in which an object such as a game controller can be limited, the normal can be obtained with higher accuracy by adopting an appropriate model of specular reflection and diffuse reflection based on the color and material. On the other hand, since various methods for separating specular reflection and diffuse reflection have been proposed, the normal may be obtained more strictly by applying such a technique.

  In the aspect described so far, it is considered that the lenses 10a to 10e used in the imaging optical system 18 of the imaging device 12 are resin lenses. Compared with glass lenses, resin lenses can be easily mass-produced, and can reduce the weight and cost of the apparatus. Therefore, the usability of the imaging device 12 is improved, and the use range can be expected to be expanded. On the other hand, when a resin lens is used, the birefringence generated inside may have an influence that cannot be overlooked in the analysis result.

  FIG. 3 shows an example in which the normal distribution obtained from the polarization image is compared with the lens material. The image shown in the figure is a gray scale image of a normal image in which each element of the normal vector represented by Equation 4 is represented as an RGB pixel value. The same applies to the normal images shown later. Even if the image is taken under the same conditions, the normal image contains more noise components than the glass lens shown in (a), and the difference between the sphere as the subject and the background is larger. Is smaller. When a glass lens is used, the amount of noise as in (b) is comparable to the result under shooting conditions where the illuminance is about 1/10.

  FIG. 4 shows a polarization degree image in which the degree of polarization ρ obtained as an intermediate parameter when the normal image shown in FIG. 3 is acquired is represented on the image plane as a luminance value. In the case of the glass lens shown in (a), the degree of polarization ρ clearly changes between the sphere outline and the background, whereas in the case of the resin lens shown in (b), the sphere outline is unclear. As a result of calculating the normal using such a degree of polarization ρ, it is considered that the normal image is unclear as shown in FIG.

  Thus, when a resin lens is introduced for taking a polarized image, the robustness of information processing accuracy with respect to changes in photographing conditions such as illuminance may be lower than in the case of a glass lens. One reason for this is considered to be birefringence generated inside the resin lens as described above. Birefringence refers to a phenomenon in which the refractive index varies depending on the vibration direction of light passing through a substance. When birefringence occurs in the lens, the polarization state of the light incident on the image sensor array 20 is different from the original polarization state of the reflected light in the subject. As a result, it is conceivable that the shape of the subject is not correctly recognized and adversely affects subsequent information processing.

  Therefore, in this embodiment, even if a polarization image is taken using a resin lens, the influence on the result is reduced by a technique that can be easily introduced. Specifically, the direction of the lens in a plane perpendicular to the optical axis, that is, the central axis of the lens is optimized and assembled to the imaging device 12. In addition, the acquired parameter is corrected so that the component of the in-plane distribution of the polarization state caused by the lens is excluded at any stage until the normal information is calculated from the polarization image. These two methods may be combined, or one of them may be introduced independently.

  FIG. 5 schematically shows the molding process of the resin lens and the state after molding. Resin lenses are generally produced by injection molding using a mold. (A) represents the state which looked at the metal mold | die from the side surface. In the molding process of the lens, the molten resin material is injected from the nozzle of the injection molding machine as indicated by the white arrow, passes through the mold runner 130 and the gate 132, and has a cavity according to the design shape of the lens. Filled. When the resin material is cooled and hardened, it is cut at a portion indicated by an arrow C corresponding to the gate 132, and subjected to annealing treatment as appropriate, whereby a lens molded product is completed.

  (B) has shown the shape which looked at the resin lens shape | molded in that way from the upper direction. The resin lens includes a lens portion 138 and a gate portion 136. The lens unit 138 is a so-called lens that focuses incident light on the imaging surface, and has various shapes in accordance with the design in the thickness direction, like the lenses 10a to 10e shown in FIG. The gate portion 136 is a portion left after being molded and cut by the gate 132 of the mold shown in FIG. In such a molding process, the birefringence of the resin lens is caused by the flow of the resin material when injected into the cavity 134, the temperature distribution during curing, the stress during cutting, and the like. It is generated by having.

Considering these causes, it is considered that the resin lens has a principal axis of optical anisotropy in a fixed direction with respect to the gate 132 and thus the gate portion 136 in the mold. For example, as indicated by an arrow A in the lens portion 138 of (b), molecules of the resin material are oriented in the radial direction passing through the gate portion 136 that is the injection direction of the resin material, and anisotropy occurs in the refractive index. Conceivable. The refractive index n _e when the electric field of light incident on such materials is parallel to the long axis of the _molecule, and the refractive index when perpendicular to n _o, the phase difference Γ of the light generated by the differential It is expressed as

  Where λ is the wavelength of the incident light and d is the thickness of the lens. In the present embodiment, by assembling a plurality of lenses so that the main axes are orthogonal to each other and fixed in the lens barrel, the phase difference Γ generated in one lens is canceled out by another lens. To. The cancellation of the phase difference Γ is supported by the following calculation. That is, if the angle between the polarization direction of the incident light and the principal axis of the resin lens is θ, the Jones matrix J acting on the polarization state by the resin lens is given as follows.

When two lenses having the characteristics of Jones matrix J ₁ and J ₂ are arranged in an overlapping manner, Jones matrix J _m acting on the polarization state of incident light is
J _m = J ₁・ J ₂
It becomes. When the principal axes are orthogonal to each other, the angles between the polarization direction of the incident light and the principal axes of the respective lenses are θ ₁ and θ ₂ and θ ₂ = θ ₁ + π / 2. Therefore, the Jones matrix J _m is as follows: Become.

  If the difference between the phase differences Γ1 and Γ2 generated by the two lenses is (2m + 1) π (m is an integer), Equation 7 becomes a unit matrix, and outgoing light having the same polarization state as the incident light can be obtained. Become. As described above, the position of the gate portion 136 in the lens and the direction of the principal axis of the resin material are considered to have the same relationship regardless of the lens. If this is utilized, it is possible to easily assemble a plurality of lenses so that the principal axes are orthogonal with the gate portion 136 as a mark.

  FIG. 6 is a diagram illustrating a state in which two lenses are assembled so that the main axes are orthogonal to each other. In the perspective view of (a), light is incident from the upper side of the figure and emitted downward. At this time, the gate portions 136a and 136b are adjusted and assembled so that the diameter Aa passing through the gate portion 136a of the upper lens and the diameter Ab passing through the gate portion 136b of the lower lens are orthogonal to each other. That is, as shown in (b), when viewed from a direction parallel to the optical axis, the angle formed by the straight line connecting the center of the lens and the gate portions 136a and 136b is set to 90 degrees.

  In the illustrated example, the gate portions 136a and 136b are used as marks in order to make the main axes orthogonal to each other. However, if the flow direction at the time of molding can be specified, the target of the mark is not limited to the gate portion. For example, the peripheral portion of the lens may be intentionally asymmetrical or marked to be a mark. In addition, if there is a factor that determines the direction of the main axis other than the flow direction of the resin material, the same effect can be obtained if a mark is appropriately provided and the lens is assembled so that the main axis is orthogonal. The same applies when measuring the direction of the spindle precisely.

  In consideration of the molecular orientation due to the flow of the resin material as described above, the diameters Aa and Ab are highly likely to coincide with the principal axis of the optical anisotropy of the lens. Therefore, instead of canceling the phase difference with a plurality of lenses as shown in the drawing, a lens cover or a filter made of a uniaxial material may be introduced and attached so that its main axis is orthogonal to the main axis of the lens. Alternatively, in consideration of the retardation described below, a lens cover or a filter may be combined with a plurality of lenses.

  FIG. 6 shows an example in which two lenses are overlapped. However, when three or more lenses such as the five lenses shown in FIG. 1 are overlapped, the polarization state of light transmitted through them is changed to the original incidence. It is necessary to make it equal to the polarization state of light. For this reason, the direction of the lens is determined based on the retardation value R as defined below.

  The retardation value R is a parameter that determines the magnitude of the phase difference Γ represented by Equation 5, and can be measured by a general apparatus. Therefore, the retardation value is measured when the lens is assembled, and the orientation of each lens is determined based on the result. Specifically, the lenses are divided into two groups so that the total retardation values are approximately the same, for example, the difference between the total values is equal to or less than a predetermined threshold value, and the gate portions of the lenses belonging to the same group are at the same position. Like that.

  Further, as shown in FIG. 6, if the angle formed by the straight lines from the centers of the two groups of lenses toward the gate portion is 90 degrees, the phase difference Γ of the incident light is finally canceled and the two sheets The same effect as when a lens is combined can be obtained. Here, the “group” may be composed of only one lens.

  FIG. 7 is a diagram illustrating an assembly example when the number of lenses is five. In this example, the retardations of the lenses L1, L2, L3, L4, and L5 are 3, 5, 2, 7, and 9 in order from the top. Here, when the lenses L1, L3, and L4 shown in white and the lenses L2 and L5 shown in shaded form a group, the total value of retardation is 12 for the former group and 14 for the latter group. And approximately equal. Therefore, as illustrated, the positions of the gate portions 136c, 136e, and 136f are matched in the former group, and the positions of the gate portions 136d and 136g are matched in the latter group.

  If the lenses are assembled in such a direction that the angle formed by the diameter passing through the gate portion is orthogonal between the two groups, the phase difference Γ generated in each lens is canceled, and the polarization state of the light emitted from L5 is changed from L1. It can return to the same level as the incident light. FIG. 8 shows the effect of assembling the lens in such a direction that the principal axes are orthogonal. Specifically, as shown in FIG. 5, assuming that the direction of the arrow A with respect to the gate portion 136 is the main axis of the lens as shown in FIG. 5, the uniaxial film is oriented in the direction of the main axis so as to be orthogonal thereto. And a polarization image was taken (after application).

  And the normal image obtained from the said polarization | polarized-light image was compared with the result (before application) in the state without a film. Note that the illuminance was changed from 1 lux to 90 lux as a photographing condition. As shown in the figure, by applying this embodiment, it is understood that the influence of noise is reduced as a whole, and the outline of the sphere becomes clear.

  FIG. 9 shows errors in the azimuth angle and zenith angle of the normal calculated from the polarization image in the same situation as FIG. As shown in the figure, by applying this embodiment, an error can be remarkably suppressed regardless of the illuminance, particularly at the azimuth angle. From this result, it can be seen that even when a resin lens is used, the accuracy of information acquisition using a polarization image can be significantly improved by an easy method of relatively adjusting the direction of the lens.

  The aspect described so far has been a technique for reducing noise in the entire image and improving the accuracy of obtaining normals. Next, a method for correcting the in-plane distribution of the polarization state caused by the lens in the process of obtaining the normal information and acquiring the normal information more accurately will be described. FIG. 10 illustrates an in-plane distribution of a normal image generated by changing the polarization state. The images shown in (a), (b), and (c) are all normal images acquired from the actually captured polarized images, but as indicated by a circle on the right pseudo image plane, The position of the lens center is different.

  Looking at each normal image, the brightness distribution shows that the tendency of the normal distribution is different in four quadrants with the center of the lens as the origin and the horizontal axis and the vertical axis as the boundary. The boundary of the four quadrants is also shown on the right side of the normal image. As shown in the figure, when the center of the lens is changed, the boundary of the four quadrants is changed accordingly, so that it is understood that this phenomenon is caused by the lens not depending on the subject. Although illustration is omitted, there was no change in the boundary of the four quadrants even when the lens was rotated. From this, it is understood that the concentric parameters with respect to the lens center are related.

  FIG. 11 schematically shows a path of light transmitted through the lens 10e shown in FIG. As shown in FIG. 6A, the fact that the path length of light passing through the lens is different between the center (path length 150) and the periphery (for example, path length 152) of the lens causes a difference in normal distribution tendency. It is thought to be the cause. That is, when the parameter d corresponding to the light path length changes concentrically, an in-plane distribution is generated in the phase difference Γ represented by Equation 5, and the polarization state, and hence the normal characteristics, are made different on the image plane. It is thought that there is.

In order to improve such a phenomenon, it is conceivable to optimize the lens shape so that the influence of the difference in the light path length is reduced. For example, as shown in FIG. 5B, the thickness of the lens in the optical axis direction is increased as a whole. Then, as viewed from the thickness of the lens, the surface unevenness becomes relatively small, and the influence can be suppressed. Specifically, when the maximum value and the minimum value of the lens thickness are D _max and D _min , respectively, a value obtained by standardizing the difference as a ratio to the maximum value,
ΔD _r = (D _max −D _min ) / D _max
Is less than or equal to a predetermined threshold.

Here, the threshold value is determined to be an optimum value such that the normal distribution in the plane as shown in FIG. 10 is suppressed to an acceptable level and the size of the lens unit is within the allowable range. Note that the denominator of the above equation may be a reference value indicating the thickness of the lens, and may be a minimum value D _min of the thickness. In this way, the ratio (for example, (Rs−D _min ) / D _max ) of the difference in the path length of the light passing through the lens to the lens thickness is reduced, and the phase difference surface caused by the unevenness on both surfaces of the lens The influence of internal distribution can be suppressed.

  When the lens is thickened, the distance between the lenses when a plurality of lens structures are formed is widened. This can reduce the curvature of the lens surface for obtaining the same imaging state. As a result, the unevenness itself can be relaxed, which further contributes to suppression of the in-plane distribution of the phase difference. In addition, by increasing the thickness of the entire lens, an effect of suppressing the occurrence of optical anisotropy due to the injection molding process can be obtained. That is, the stress applied to the flowing resin material can be suppressed by the uneven shape of the mold, and the molecular state is made uniform throughout the entire lens. As a result, the occurrence of local birefringence can be suppressed.

  In addition to this lens shape approach, in this embodiment, as described above, correction processing is performed so as to eliminate the in-plane distribution caused by the lens at any stage until the normal image is acquired. think of. This correction process can be introduced independently of the optimization of the lens shape, and each has an effect of improving the processing accuracy of normal information acquisition. Naturally, normal information can be obtained with higher accuracy by combining both.

  FIG. 12 shows a functional block configuration of a polarization image processing apparatus having a function of acquiring normal information while correcting the in-plane distribution of the polarization state. Each functional block shown in the figure can be realized in hardware as a configuration such as an image sensor, various arithmetic circuits, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a microprocessor, and various memories. Is realized by a program read from a memory, a storage device, or a recording medium. Therefore, it is understood by those skilled in the art that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof, and is not limited to any one.

  The polarization image processing device 160 may be a part of the image processing unit 22 in the imaging device 12 shown in FIG. 1, obtains polarization image data, uses a normal information of the subject obtained from the data, or the like You may implement | achieve as an information processing apparatus which performs this information processing. The polarization image processing device 160 includes a polarization image acquisition unit 162 that acquires polarization image data, a normal information acquisition unit 164 that acquires normal information from the polarization image, an information processing unit 172 that performs information processing based on the normal information, And a data output unit 174 for outputting data to be output to an external device.

  The polarization image acquisition unit 162 acquires polarization image data in a plurality of directions. When the polarization image processing device 160 is a part of the imaging device 12, the polarization image acquisition unit 162 has the luminance output from the imaging optical system 18, the imaging device array 20, and the imaging device array 20 illustrated in FIG. 1. A mechanism for appropriately interpolating the data to generate polarized image data for each direction is included. When the polarization image processing device 160 is an information processing device such as a game machine, the polarization image acquisition unit 162 establishes communication with the imaging device 12 and acquires polarization image data. The normal line information acquisition unit 164 obtains a normal line for each pixel based on the above-described principle based on the luminance change of the corresponding pixel of the polarization image in a plurality of directions.

  Specifically, the normal information acquisition unit 164 obtains information about the normal of the subject from the polarization image, and a parameter derived from the lens with respect to any of the parameters calculated in the normal information acquisition process. A correction unit 168 that corrects the in-plane distribution is included. The calculation unit 166 acquires information related to the normal of the subject surface, such as a normal image, according to Equations 1 to 4. Actually, expression 1 may be expressed as the following expression, and the coefficients a, b, and c may be calculated to increase the efficiency of the arithmetic processing.

_Assuming that the luminances when the main axis angle θ _pol is 0 °, 45 °, 90 °, and 135 ° are I ₀ , I ₄₅ , I ₉₀ , and I ₁₃₅ , the coefficients a, b, and c in Equation 9 are as follows: Can be sought.

  Using the coefficients a, b, and c in Equation 10, the degree of polarization ρ and the polarization phase φ are obtained as follows.

  Based on the degree of polarization ρ and the polarization phase φ, normal information such as normal azimuth and zenith angles and normal vectors can be obtained for each pixel using Equation 3 and Equation 4. The normal image data can be acquired by appropriately mapping the obtained values on the image plane.

  The correction unit 168 corrects any of the parameters calculated by the calculation unit 166 so that the in-plane distribution caused by the lens becomes small. Therefore, the correction unit 168 holds the correction data 170 in an internal memory or the like. The correction data 170 is, for example, data that is determined as a two-dimensional map on an image plane by determining a correction value to be applied to a correction target parameter for each pixel or region. Here, the correction value may be a scalar, a vector, or a matrix depending on the parameter to be corrected.

  When obtaining normal information using Equations 10 and 11, intermediate parameters such as coefficients a, b, and c and the degree of polarization ρ may be corrected. Or you may correct | amend to the azimuth angle of the normal line obtained as a final result, a zenith angle, or a normal vector. That is, as long as the correction is such that the in-plane distribution caused by the lens is small, the parameters to be corrected and the specific calculation method of the correction are not limited. For example, when a uniform error is observed in the angle of the normal in each of the four quadrants having the lens center as the origin as exemplified in FIG. 10, the error of the angle is set for each quadrant.

  Then, the correction unit 168 corrects by adding the set angle error to the azimuth angle or zenith angle of the normal calculated by the calculation unit 166. Alternatively, a matrix operation is performed so that the normal vector is rotated by a set value. For example, when setting an imaging device, such a set value is obtained by setting an object having a uniform normal distribution such as a flat plate so as to cover the entire field of view and photographing a polarization image. Can be determined by comparing the normal distributions.

  According to this setting technique, characteristics unique to the lens other than the light path length, for example, the in-plane distribution due to the solidification order of the resin material and the temperature distribution during curing can be corrected. Alternatively, the in-plane distribution of the phase difference may be specified based on the optical path length, and the correction value may be determined theoretically.

  Since the correction data 170 determined in this way is basically unique to the lens, it is stored in the memory at the time of shipment of the imaging device 12 and used for correction during operation. When the polarization image processing device 160 is an information processing device such as a game machine, the correction data 170 is read from the imaging device 12 at the start of processing. Each time the correction target parameter is calculated, the correction unit 168 acquires the calculation result from the calculation unit 166, performs correction, and returns to the calculation unit 166 as necessary to continue the subsequent processing.

  The normal information acquired by the normal information acquisition unit 164 is supplied to the information processing unit 172 or the data output unit 174. When the polarization image processing device 160 is part of the imaging device 12, the data output unit 174 establishes communication with an information processing device such as a game machine and transmits normal information. When the polarization image processing device 160 is an information processing device such as a game machine, the information processing unit 172 performs information processing such as a game using normal line information, and outputs image and sound data to be output as a result. This is supplied to the data output unit 174. In this case, the data output unit 174 establishes communication with a display device or the like, and transmits data to be output.

  According to the present embodiment described above, by introducing a resin lens into an imaging device that captures a polarized image, the device can be reduced in weight and height, and an introduction barrier to a game machine, a portable terminal, or the like can be achieved. Increase the versatility by lowering. At this time, the lens is assembled by optimizing the direction around the optical axis based on the molding process of the lens. Specifically, using a part where the position of the resin material in the flow direction is fixed in the molding process, such as the gate, as a landmark, assemble multiple lenses so that the main axis of optical anisotropy due to the flow is orthogonal .

  When using three or more lenses, divide the lenses into two groups so that the total retardation value is approximately the same, align the directions for each group, and position the lenses so that the principal axes are orthogonal to each other. Assemble. By doing so, it is possible to cancel the phase difference due to birefringence generated in the lens and to make the light of the polarization state equivalent to the incident light to the imaging device enter the imaging device. As a result, it is possible to obtain the above-described effects by introducing the resin lens while maintaining the accuracy of the normal line information and, consequently, the information processing using the normal information.

  The same effect can be obtained by installing a uniaxial material filter or lens cover and mounting the lens so that the main axis is perpendicular to the gate part of the lens, making it easy without being limited by the number of lenses or retardation value. Introduction is possible. Furthermore, focusing on the fact that in-plane distribution occurs in the polarization state due to the unevenness of the lens surface, the shape of the lens is optimized. Specifically, the lens thickness is increased to such an extent that the influence of the unevenness is reduced. As a result, the in-plane retardation due to the path length difference of transmitted light can be reduced, the curvature of the surface can be reduced, and the non-uniformity of the resin material due to the stress during injection molding can be reduced. Can be suppressed.

  Further, the in-plane distribution of the polarization state is corrected at any stage of obtaining the normal information from the polarization image. By creating the data used for correction at the time of manufacturing the imaging device in a format such as a two-dimensional map, it is possible to easily correct during operation, and to achieve processing accuracy that depends on the lens design and resin molding conditions. The influence can be reduced.

  The present invention has been described based on the embodiments. Those skilled in the art will understand that the above-described embodiment is an exemplification, and that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are also within the scope of the present invention. is there.

  10a lens, 12 imaging device, 18 imaging optical system, 20 imaging device array, 22 image processing unit, 110 imaging device, 114 polarizer layer, 116 color filter layer, 118 photodetection layer, 136 gate unit, 138 lens unit, 160 Polarized image processing device, 162 Polarized image acquisition unit, 164 Normal information acquisition unit, 166 Calculation unit, 168 Correction unit, 170 Correction data, 172 Information processing unit, 174 Data output unit

Claims (8)

A resin lens for realizing imaging on the imaging surface in the imaging device;
A lens barrel assembled with the resin lens;
With
The image pickup unit, wherein the resin lens is fixed in a predetermined rule based on a principal axis of optical anisotropy of the resin lens on a plane perpendicular to the optical axis. Provided with a plurality of resin lenses arranged with the optical axis as the center,
The imaging unit according to claim 1, wherein the plurality of resin lenses are fixed so that the principal axes are in two directions orthogonal to each other. The resin lens includes a gate portion formed by solidifying an inflow portion of the resin material at the time of injection molding,
The imaging unit according to claim 2, wherein the two directions are defined by a position of the gate portion.   The difference between the total value of the retardation of the resin lens fixed in one of the two directions and the total value of the retardation of the resin lens fixed in the other direction is equal to or less than a predetermined threshold value. The image pickup unit according to claim 2, wherein the image pickup unit is an image pickup unit.   The imaging unit according to claim 1, further comprising a lens cover or a filter fixed to the resin lens according to a predetermined rule based on a principal axis of optical anisotropy.   The imaging unit according to claim 1, further comprising an imaging element that acquires polarized light luminance data by detecting polarization in a predetermined direction of light transmitted through the resin lens.   The imaging unit according to claim 6, further comprising an image processing unit that performs a process related to a polarization image using the luminance data of the polarization. A method of assembling a plurality of resin lenses for realizing imaging on an imaging surface in an imaging apparatus,
The resin lens is fixed so that the inflow portion of the resin material at the time of injection molding is solidified so that the diameters passing through the gate portion formed in each resin lens are in two directions orthogonal to each other. How to assemble resin lenses.