BR112016029727B1 - AUTOMATIC FOCUS MODULE FOR A CAMERA AND METHOD FOR CAPTURING AN IMAGE - Google Patents

Abstract

AUTOMATIC FOCUS FOR FOLDED OPTICS ARRANGE CAMERAS. The present invention relates to aspects relating to autofocus systems and techniques for an array camera having a low profile height, for example approximately 4 mm. A voice coil motor (VCM) may be positioned near a folded optics assembly on the array camera to allow vertical movement of a second light directing surface to change the focal position of the corresponding sensor. A drive member may be positioned within the coil of the VCM to provide vertical movement, and the drive member may be coupled to the second light directing surface, for example, by a lever. Correspondingly, movement of the VCM drive member can be transferred to the second light directing surface over a distance, providing autofocus capabilities without increasing the overall height of the array camera.

Description

FIELD OF TECHNIQUE

[0001] The present disclosure relates to imaging systems and methods that include a multi-camera arrangement. In particular, disclosure refers to systems and methods that enable low-profile imaging systems and mobile devices while maintaining or improving image quality.

FUNDAMENTALS

[0002] Many mobile devices, such as mobile phones and tablet-type computing devices, include cameras that can be operated by a user to capture still images and/or video. Because mobile devices are typically designed to be relatively small, it may be important to design cameras or imaging systems to be as thin as possible to maintain a low profile mobile device. Folded optics image sensor arrays (“array cameras”) enable the creation of low-profile image capture devices without shortening the focal length or reducing image resolution across the field of view of the sensor array. By redirecting light to each sensor in the array using a primary and secondary surface, and positioning the lens mounts used to focus the incoming light between the primary and secondary surfaces, the sensor array can be positioned on a smooth substrate perpendicular to the lens mounts. The longer focal length makes it possible to implement features such as optical zoom and incorporate more complicated optics that require more space than is commonly provided by the traditional mobile camera, such as adding more optical elements.

[0003] Some array cameras employ a central mirror or prism with multiple facets to divide the input light comprising the target image into multiple portions for capture by sensors in the array, wherein each facet directs a portion of the light from the image. target for a sensor in the array. Each portion of the divided light can be passed through a lens assembly and reflected off a surface positioned directly above or below a sensor, so that each sensor captures a portion of the image. Sensor fields of view can overlap to help stitch captured portions together into a complete image.

SUMMARY

[0004] The autofocus systems and techniques described herein enable multi-camera autofocus on a low profile folded optics camera. Autofocus presents a challenge with folded optics array cameras, particularly when the height of the array camera is restricted, for example, by about 4 to 5 mm. Due to light reflection off multiple surfaces toward multiple sensors and in-camera height limitations, traditional autofocus modules and techniques are not adapted to these folded-optics low-profile sensor arrays. For example, moving an autofocus lens assembly up and down over each sensor would increase the height of the system as well as change the relative positioning of the sensors' optical axes. Even the smallest voice coil motors (VCM) have too large a side length (approximately 7.5 mm) for use in driving a lens assembly back and forth between a central mirror or prism and a sensor, due to the array camera height restrictions. Reciprocating stepper piezomotors are expensive and noisy, and operate slowly and less reliably.

[0005] The problems described above, among others, are attributed to certain embodiments of folded optical array camera autofocus systems and techniques as described in this document. Array camera embodiments for use with autofocus systems and techniques may employ a central mirror or prism, e.g., with multiple surfaces or facets, to divided input light comprising the target image into multiple portions for capture by the sensors. in the arrangement, referred to herein as a “first light directing surface.” Each portion of the divided light may be passed through a lens assembly and reflected by an additional mirror, or refracted through an additional prism, directly positioned above or below a sensor, so that each sensor captures a portion of the image. The additional reflective or refractive surface is referred to herein as a “second light directing surface.” The combination of the first and second light directing surfaces, lens, and sensor are generally referred to herein as a “folded optics assembly.”

[0006] The examples and embodiments of autofocus systems and methods described herein may use a VCM to allow vertical movement of the second light directing surface to change the focal position for the corresponding sensor. For example, a VCM may be positioned next to a folded optics assembly. A drive member may be positioned within the coil of the VCM to provide vertical movement, and the drive member may be coupled to the second light directing surface, for example, through a lever or other mechanical structural member (which, in In some embodiments, it may include several structures coupled together, which herein may be collectively referred to as a “lever”). In some embodiments, the second light directing surface may be attached to a first end of the lever and a counterweight may be attached to a second end of the lever, and the drive member may be attached to an intermediate portion of the lever. Correspondingly, the movement of the VCM drive member can be transferred to the second light directing surface at a distance, providing autofocus capabilities, and this autofocus mechanism does not require increasing the overall height of the array camera. VCMs are quite agile, but require extremely low friction mechanics, and this can be achieved by the lever mechanism.

[0007] Each sensor of a multi-sensor imaging system is configured, collectively with other components of the imaging system, to capture an in-focus portion of the target imaging scene. In some circumstances, each sensor in the array may capture a portion of the image that slightly overlaps with portions captured by neighboring sensors in the array. These portions can be assembled into the target image, for example, through linear blending or other image stitching techniques.

[0008] An innovation includes an autofocus module module for a folded optical array camera, including an image sensor mounted to a substrate that defines a first plane, a first light directing surface, a second directing surface of light, wherein the first light directing surface is positioned to direct light representing a portion of a target image scene toward the second light directing surface and the second light directing surface is positioned to direct the light toward the image sensor, an actuator mounted to the substrate and including a drive member positioned for movement in a direction orthogonal to the plane of the substrate; and a lever in contact with the drive member and the second light directing surface so that movement of the drive member is transferred to the second light directing surface.

[0009] Embodiments of the autofocus module may include, for example, one or more of the following aspects. In some embodiments, the actuator includes a voice coil motor. In some embodiments, the drive member is threaded into a voice coil motor coil. Movement in the direction orthogonal to the substantially horizontal plane may be within a range of approximately 120 microns to approximately 150 microns. The autofocus module may further include a lens assembly positioned between the first light directing surface and the second light directing surface. The diameter of the lens mount may be approximately 4 mm in some embodiments. The autofocus module may further include a counterweight coupled to a first end of the lever, wherein the drive member is coupled to an intermediate portion of the lever and the second light directing surface is coupled to a second end of the lever. . The counterweight weight may be selected to at least partially balance a weight of the second light directing surface. In some embodiments, the second light directing surface comprises one of a reflective mirror and a prism. The autofocus module may further include a rod coupled to a first end of the lever using a hinge. The driving member may be coupled to an intermediate portion of the lever and the second light directing surface is coupled to a second end of the lever. In some embodiments, the rod is coupled to a structure that forms the first light directing surface, and the second light directing surface is coupled to an intermediate portion of the lever and the actuating member is coupled to a second end of the lever. In some embodiments, a first end of the lever is coupled to a structure forming the first light-directing surface, and wherein the second light-directing surface is coupled to an intermediate portion of the lever, and the drive member is coupled to a second end of the lever.

[0010] Another innovation is a system for automatically focusing a folded optical array camera including a plurality of optical mounts positioned around a structure that provide a corresponding plurality of light-bending surfaces, the system comprising, for each among the plurality of optical assemblies an image sensor mounted to a substrate defining a first plane, a second light directing surface, wherein the corresponding light bending surface of the plurality of light bending surfaces positioned to direct light represents a portion of a target imaging scene toward the second light directing surface and the second light directing surface positioned to direct light toward the image sensor, an actuator mounted to the substrate and including a drive member positioned to movement in a direction orthogonal to the plane of the substrate, and a lever contacting the driving member and the second light directing surface so that movement of the driving member is transferred to the second light directing surface. The system may further include a controller including programmable computational instructions to keep each of the plurality of optical assemblies focused at approximately the same distance, the controller being in electronic communication with the actuator of each of the plurality of optical assemblies. In some embodiments, the actuator for an optical assembly of the plurality of optical assemblies is coupled to the optical assembly and at least one adjacent optical assembly of the plurality of optical assemblies. In some embodiments, the plurality of optical assemblies comprises three optical assemblies, and wherein the lever comprises a T-shaped member including a first lever and a second lever, the first lever being coupled to the actuating member and the second lever. , the second lever being coupled at a first end to the optical assembly and at a second end to the adjacent optical assembly. In some embodiments, the first lever is coupled at a first end to a counterweight, at an intermediate portion to the drive member, and at a second end to an intermediate portion of the second lever. In some embodiments, the plurality of optical assemblies comprises four optical assemblies, and the lever is coupled at an intermediate portion to the drive member, at a first end to the optical assembly, and at a second end to the adjacent optical assembly.

[0011] Another innovation includes a method for capturing an image of a target imaging scene using a folded optics array camera comprising a plurality of folded optics assemblies mounted to a substrate defining a foreground and positioned around a structure that provide a corresponding plurality of light-bending surfaces. The method includes, for each of the plurality of folded optics assemblies, inducing movement of a driving member of an actuator in a direction orthogonal to the plane of the substrate, transferring movement of the driving member to a second driving surface. light, thereby altering a focal distance of the light representing a portion of the target image scene, wherein the corresponding light-bending surface of the plurality of light-bending surfaces is positioned to direct the light toward the second light-bending surface. light directing and the second light directing surface is positioned to direct light toward an image sensor, and capture an image that represents the portion of the light using the image sensor. In some embodiments, the movement is accomplished by a lever in contact with the drive member and the second light directing surface. The method may further include controlling the movement of the drive member for each of the plurality of folded optics assemblies. In some embodiments, the method further includes keeping each of the plurality of folded optics assemblies focused at approximately the same distance. The method may further include determining a degree of movement for a driver assembly selected from the plurality of folded optics assemblies and using the degree of movement for the selected driver assembly to determine movement for each other of the plurality of folded optics assemblies. . In some embodiments, determining movement for each other of the plurality of folded optics assemblies is based at least partially on the degree of movement for the selected driver assembly and at least partially on one or more coefficients that define a relationship between the plurality of assemblies. folded optics. In some embodiments, the method further includes performing a stitching operation on the image representing the portion of light from each of the plurality of folded optical assemblies to form the image of the target imaging scene.

[0012] Another innovation consists of a non-transitory computer-readable medium that stores instructions that, when executed, induce one or more processors to perform operations including each of a plurality of folded optics assemblies of a folded optics camera, wherein the plurality of bent optics assemblies mounted to a substrate define a first plane and positioned around a structure that provides a corresponding plurality of light bending surfaces: calculate, based at least partially on a desired focal length, the displacement of a driving member of an actuator in a direction orthogonal to the plane of the substrate, the displacement of the driving member calculated to induce a corresponding degree of displacement for a light directing surface positioned to direct light representing a portion of a scene of target image toward an image sensor, generate instructions to induce displacement of the drive member, and provide the instructions to an actuator coupled to the drive member. In some embodiments of the non-transitory computer readable medium, the operations further include determining a first degree of displacement for the drive member of a drive assembly selected from the plurality of folded optics assemblies. In some embodiments, the operations include, further, an adjusted degree of displacement for the drive member of each other assembly of the plurality of folded optics assemblies based at least partially on the first degree of displacement and at least partially on one or more coefficients defining a relationship between the plurality of folded optics mounts. In some embodiments of the non-transitory computer-readable medium, the operations further include maintaining each of the plurality of folded optical assemblies focused at approximately the same distance.

[0013] Another innovation consists of an autofocus module for a folded optical array camera, including an image sensor that defines a foreground, wherein the image sensor configured to receive light represents at least a portion of a target image scene, a lens assembly configured to focus light, wherein the lens assembly has an optical axis, a light directing surface positioned to direct light received from the lens assembly onto the image sensor , an actuator comprising a drive member positioned for movement in a direction perpendicular to the optical axis of the lens assembly, and means for transferring movement of the drive member in a direction perpendicular to the optical axis to the second light-directing surface for change an optical path length of light. The lens assembly may be positioned so that the optical axis extends substantially parallel to the foreground.

[0014] In some embodiments, the means for transferring movement of the drive member includes a lever in contact with the drive member and the second light directing surface. In some embodiments, the means for transferring motion of the drive member further includes a counterweight coupled to a first end of the lever, a middle portion of the lever coupled to the drive member, and a second end of the lever coupled to the second steering surface. of light. The means for transferring movement of the drive member may further include a tie rod coupled to a first end of the lever, an intermediate portion of the lever coupled to the drive member, and a second end of the lever coupled to the second light directing surface. In some embodiments, the autofocus module further includes a structure having an additional light directing surface positioned so that the lens assembly is between the additional light directing surface and the light directing surface. In some embodiments, the means for transferring motion of the drive member further includes a first end of the lever coupled to the drive member, a middle portion of the lever coupled to the light directing surface, and a second end of the lever coupled to the structure comprising additional light directing surface. In some embodiments, the second end of the lever is coupled to the structure having the additional light directing surface through one of a tie rod and a hinge, or both. In some embodiments, the actuator comprises a voice coil motor. Some embodiments further include a first camera of a plurality of cameras, the first camera comprising the autofocus module having the image sensor, the lens assembly, the light directing surface, the actuator, and the means to transfer the movement of the driving member. In some embodiments, each other camera of the plurality of cameras includes an autofocus module similar to the first camera. In some embodiments, the plurality of cameras is disposed around a central pyramid having a plurality of facets, each of the plurality of facets positioned to direct a portion of the light representing the target image scene toward an associated camera among the plurality of cameras.

BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Hereinafter, the disclosed aspects will be described in conjunction with the attached drawings and appendices, provided to illustrate, and not limit, the disclosed aspects, in which similar designations denote similar elements.

[0016] Figure 1A illustrates a cross-sectional side view of one embodiment of a folded optical sensor assembly.

[0017] Figure 1B illustrates a cross-sectional side view of another embodiment of a folded optical sensor assembly.

[0018] Figure 2 illustrates a block diagram of an embodiment of an image capture device.

[0019] Figure 3 illustrates a scaled comparison of one embodiment of a folded optics assembly for an array camera and a voice coil motor.

[0020] Figure 4A illustrates an embodiment of a folded optics assembly autofocus system with a secondary light bending surface that includes a mirror.

[0021] Figure 4B illustrates an embodiment of a folded optics assembly autofocus system with a secondary light bending surface that includes a prism.

[0022] Figure 4C illustrates an embodiment of a folded optics assembly autofocus system having a lever characterized as a third class of lever.

[0023] Figure 4D illustrates an embodiment of a folded optics assembly autofocus system having a lever characterized as a second class of lever.

[0024] Figures 5A-5C illustrate various embodiments of a folded optical array camera autofocus system.

[0025] Figure 6 illustrates an embodiment of a folded optics image capture process.

DETAILED DESCRIPTION I. Introduction

[0026] The implementations disclosed herein provide systems, methods and apparatus for automatic focusing on an array camera with folded optics. As previously described, a voice coil motor (VCM) can be positioned near a folded optics assembly on an array camera to allow vertical movement of the second light directing surface to change the focal position of the corresponding sensor. A drive member may be positioned within the coil of the VCM to provide vertical movement, and the drive member may be coupled to the second light directing surface, for example, by a lever. In some embodiments, the second light directing surface may be attached to a first end of the lever and a counterweight may be attached to a second end of the lever, and the drive member may be attached to an intermediate portion of the lever. Correspondingly, the movement of the VCM drive member can be transferred to the second light directing surface over a distance, providing autofocus capabilities, and this autofocus mechanism does not require an increase in the overall height of the camera. arrangement.

[0027] In some examples, a separate VCM may be provided to individually control autofocus for each folded optics assembly in the array camera. In other examples, the levers of a series of VCMs provided for the array camera may be connected, for example, by a solid or hinged mechanical linkage, so that movement of the actuating member of two or more VCMs controls autofocus. for each folded optics assembly. In further examples, at least one VCM may be coupled to the second light directing surface of each folded optics assembly in the array such that movement of the drive member of the at least one VCM controls autofocus for all of the optical assemblies. folded optics. In still other examples, a series of VCMs may be coupled to the second light directing surface of a portion of the folded optics assemblies so that movement of the drive member of each VCM controls autofocus for the corresponding portion of the VCMs. folded optics mounts.

[0028] In the following description, specific details are provided to provide a full understanding of the examples. However, the examples can be practiced without these specific details.

I. Overview of Folded Optics Array Cameras

[0029] Referring now to Figures 1A and 1B, examples of a folded optics multiple sensor assembly 100A, 100B suitable for use with the autofocus systems and techniques described herein will be described in greater detail. Figure 1A illustrates a cross-sectional side view of an example of a folded optics array 100A including image sensors 105, 125, reflective secondary light-bending surfaces 110, 135, lens mounts 115, 130, and a reflective surface. 120 that can all be mounted to a substrate 150. Figure 1B illustrates a cross-sectional side view of one embodiment of a bent optics sensor array including central prisms 141, 146 for primary light-bending surfaces 122, 124 and additional prisms that form secondary light-bending surfaces 135, 110.

[0030] Referring to Figure 1A, image sensors 105, 125 may comprise, in certain embodiments, a charge-coupled device (CCD), complementary metal oxide semiconductor (CMOS) sensor, or any other capture device. image that receives light and generates data in response to the received image. Each sensor 105, 125 may include a plurality of sensors (or sensor elements) arranged in an array. Image sensors 105, 125 may be capable of obtaining image data from still photographs and may also provide information regarding movement in a captured video stream. Sensors 105 and 125 may be an individual sensor array, or each may represent sensor arrays, e.g., a 3x1 array of sensor arrays. However, as will be understood by one skilled in the art, any suitable arrangement of sensors can be used in the disclosed implementations.

[0031] Sensors 105, 125 can be mounted on the substrate 150 as shown in Figure 1A. In some embodiments, all of the sensors may be in one plane being mounted to the smooth substrate 150. The substrate 150 may be any substantially suitable smooth material. The central reflective surface 120 and lens mounts 115, 130 may also be mounted onto the substrate 150. Various configurations are possible for mounting a sensor array or arrays, a plurality of lens mounts, and a plurality of reflective or refractive surfaces. primary and secondary.

[0032] In some embodiments, a central reflective surface 120 may be used to redirect light from a target imaging scene toward sensors 105, 125. The central reflective surface 120 may be a mirror or a plurality of mirrors, and may be smooth or shaped as needed to appropriately redirect incoming light to image sensors 105, 125. For example, in some embodiments, the central reflective surface 120 may be a mirror sized and shaped to reflect incoming light rays through from lens mounts 115, 130 to sensors 105, 125. The central reflective surface 120 can divide the light comprising the target image into multiple portions and direct each portion onto a different sensor. For example, a first side 122 of the central reflective surface 120 (also referred to as a primary light bending surface, as other embodiments may implement a prism rather than a reflective surface) may send a portion of the light corresponding to a first field of view 140 toward the left sensor 105 while a second side 124 sends a second portion of light corresponding to a second field of view 145 toward the right sensor 125. It should be appreciated that the fields of view 140, 145 of the sensors image cover at least one target image.

[0033] In some embodiments in which the receiving sensors are an arrangement of a plurality of sensors, the central reflective surface may be made from multiple refractive surfaces angled relative to each other in order to send a different portion of the receiving scene. target image towards each of the sensors. Each sensor in the array may have a substantially different field of view, and, in some embodiments, the fields of view may overlap. Certain embodiments of the central reflective surface may have complex non-planar surfaces to increase the degrees of freedom when designing the lens system. Furthermore, although the central surface is discussed as being a reflective surface, in other embodiments, the central surface may be refractive. For example, the central surface may be a prism configured with a plurality of facets, where each facet directs a portion of the light comprising the scene toward one of the sensors.

[0034] After being refracted from the central reflective surface 120, at least a portion of the input light may propagate through each of the lens assemblies 115, 130. One or more lens assemblies 115, 130 may be provided between the central reflective surface 120 and the sensors 105, 125 and the refractive surfaces 110, 135. The lens mounts 115, 130 can be used to focus the portion of the target image that is directed toward each sensor.

[0035] In some embodiments, each lens assembly may comprise one or more lenses and an actuator for moving the lens between a plurality of different lens positions throughout a housing. The actuator can be a voice coil motor (VCM), a microelectronic mechanical system (MEMS), or a shape memory alloy (SMA). The lens assembly may further comprise a lens driver for controlling the actuator.

[0036] In some embodiments, traditional autofocus techniques can be implemented by changing the focal length between lenses 115, 130 and corresponding sensors 105, 125 of each camera. In some embodiments, this can be accomplished by moving a lens barrel. Other embodiments may adjust focus by moving the central mirror up or down or by adjusting the angle of the mirror relative to the lens mount. Certain embodiments can adjust focus by moving the side mirrors over each sensor. These embodiments may allow the mount to adjust the focus of each sensor individually. Furthermore, it is possible, in some embodiments, to change the focus of the entire assembly at once, for example, by placing a lens, such as a liquid lens, over the entire assembly. In certain implementations, computational photography can be used to change the focal point of the camera array.

[0037] Multi-sided refractive surfaces, for example, refractive surfaces 110 and 135, may be provided around the central mirror 120 opposite the sensors. After passing through the lens mounts, the side refractive surfaces 110, 135 (also referred to as a secondary light bending surface, as other embodiments may implement a prism, e.g., a refractive prism, rather than a reflective surface) may reflect light (downward, as described in the orientation of Figure 1A) onto the sensors 105, 125. As described, the sensor 105 may be positioned below the reflective surface 110 and the sensor 125 may be positioned below the reflective surface 135. However, in other embodiments, the sensors may be above the side reflected surfaces, and the side refractive surfaces may be configured to reflect light upwards (see, for example, Figure 1B). Other suitable configurations of the lateral refractive surfaces and sensors are possible, in which light from each lens assembly is redirected toward the sensors. Certain embodiments may allow movement of the lateral refractive surfaces 110, 135 to change the focus or field of view of the associated sensor.

[0038] Each field of view of the sensor 140, 145 may be directed in object space by the surface of the central mirror 120 associated with that sensor. Mechanical methods can be employed to tilt the mirrors and/or move the prisms in the array so that the field of view from each camera can be directed to different locations in the object field. This can be used, for example, to implement a high dynamic range camera, increase the resolution of the camera system, or implement a plenoptic camera system. Each sensor's field of view (or each 3x1 array) can be projected into object space, and each sensor can capture a partial image comprising a portion of the target scene according to the sensor's field of view. As illustrated in Figure 1A, in some embodiments, the fields of view 140, 145 for opposing sensor arrays 105, 125 may overlap to a certain degree 150. In order to reduce the overlap 150 and form a single image, a The stitching process as described below may be used to combine images from two opposing sensor arrays 105, 125. Certain embodiments of the stitching process may employ overlay 150 to identify common features in stitching the partial images together. After stitching the overlapping images together, the stitched image can be reframed to a desired aspect ratio, for example 4:3 or 1:1, to form the final image. In some embodiments, the alignment of the optical elements pertaining to each FOV is arranged to minimize overlap 150 so that multiple images are formed into a single image with little or no image processing required to join the images.

[0039] Figure IB illustrates a cross-sectional side view of another embodiment of a folded optical array camera 100B. As shown in Figure 1B, a sensor assembly 100B includes a pair of image sensors 105, 125, each mounted to substrate 150, lens assemblies 115, 130 corresponding to image sensors 105, 125, respectively, and a surface of secondary light bending 110, 135 positioned on the coating glass 106, 126 of the image sensors 105, 125, respectively. The primary light bending surface 122 of the prism 141 directs a portion of the light from the target image scene along the optical axis 121 through the lens assembly 115, is redirected from the secondary light bending surface 110, passes through the coating glass 106, and is incident on the sensor 105. The primary light bending surface 124 of the prism 146 directs a portion of the light from the target imaging scene along the optical axis 123 through the assembly of lens 130, is redirected from the secondary light bending surface 135, passes through the coating glass 126, and is incident on the sensor 125. The folded optical array camera 100B is illustrative of a modality array camera that implements prisms rather than the refractive surfaces of the array camera 100A of Figure 1A. Each of the prisms 141, 146 is provided in an opening in the substrate 150 so that the primary light-directing surfaces 122, 124 are below the plane formed by the substrate and receive light representing the target imaging scene.

[0040] Sensors 105, 125 can be mounted on the substrate 150 as shown in Figure 1B. In some embodiments, all of the sensors may be in one plane being mounted to the smooth substrate 150. The substrate 150 may be any substantially suitable smooth material. The substrate 150 may include an opening as previously described to allow input light to pass through the substrate 150 to the primary light-bending surfaces 122, 124. Various configurations are possible for mounting a sensor array or arrays, as well as other sensor components. camera illustrated, to substrate 150.

[0041] The primary light bending surfaces 122, 124 may be prismatic surfaces as illustrated, or may be a mirror or a plurality of mirrors, and may be smooth or shaped as needed to appropriately redirect incoming light to the sensors. image 105, 125. In some embodiments, the primary light-bending surfaces 122, 124 may be formed as a central mirror pyramid or prism as illustrated in Figure 1A. The pyramid, central mirror prism, or other reflective surface can divide the light representing the target image into multiple portions and direct each portion toward a different sensor. For example, a primary light bending surface 122 may send a portion of light corresponding to a first field of view toward the left sensor 105 while the primary light bending surface 124 sends a second portion of light corresponding to a second field of view. of view toward the right sensor 125. In some embodiments in which the receiving sensors are an arrangement of a plurality of sensors, the light bending surfaces may be made from multiple refractive surfaces angled relative to each other in order to of sending a different portion of the target image scene toward each of the sensors. It must be appreciated that the cameras' fields of view cover at least the target image, and can be aligned and stitched together after capture to form a final image captured by the synthetic aperture of the array.

[0042] Each sensor in the array may have a substantially different field of view, and, in some embodiments, the fields of view may overlap.

[0043] As illustrated by Figures 1A and 1B, each array camera has a total height H. In some embodiments, the total height H may be approximately 4.5 mm or less. In other embodiments, the total height H may be approximately 4.0 mm or less. Although not illustrated, the entire camera array 100A, 100B may be provided in a housing having a corresponding internal height of approximately 4.5 mm or less or approximately 4.0 mm or less.

[0044] As used in question, the term “camera” refers to an image sensor, lens system, and a series of corresponding light-bending surfaces, e.g., the primary light-bending surface 124, assembly of lens 130, secondary light bending surface 135, and sensor 125 as illustrated in Figure 1. An array of multiple folded optics sensors, referred to as an “array” or “camera array,” may include a plurality of such cameras in various configurations. Some embodiments of array configurations are disclosed in U.S. Application Publication 2014/0111650, filed on March 15, 2013 and entitled “MULTI-CAMERA SYSTEM USING FOLDED OPTICS”, this disclosure being incorporated herein by reference. Other array camera configurations that would benefit from the autofocus systems and techniques described herein are possible.

[0045] Figure 2 depicts a high-level block diagram of a device 200 having a set of components including an image processor 220 connected to one or more cameras 215a-n. Image processor 220 is also in communication with a working memory 205, memory component 230, and device processor 250, which are in turn in communication with storage 210 and an electronic display 225.

[0046] Device 200 may be a cell phone, a digital camera, a tablet computer, a personal digital assistant, or the like. There are many portable computing devices in which a thin imaging system as described herein would provide advantages. Device 200 may also be a stationary computing device or any device in which a thin imaging system would be advantageous. A plurality of applications may be available to the user on device 200. These applications may include traditional photography and video applications, high dynamic range imaging, panoramic photo and video, or stereoscopic imaging such as 3D imaging or 3D video.

[0047] Image capture device 200 includes cameras 215a-n for capturing external images. Cameras 215a-n may comprise a sensor, lens assembly, and a primary and secondary reflective or refractive surface for redirecting a portion of a target image to each sensor, as discussed previously in connection with Figure 1. In general, N cameras 215a -n can be used, where N> 2. Therefore, the target image can be divided into N portions in which each sensor of the N cameras captures a portion of the target image according to the sensor's field of view. It will be understood that cameras 215a-n may comprise any number of cameras suitable for an implementation of the folded optics imaging device described herein. The number of sensors can be increased to achieve lower z-heights of the system, as discussed in greater detail below in relation to Figure 4, or satisfy needs for other purposes, such as having overlapping fields of view similar to those of a plenooptic camera. which may allow the ability to adjust image focus after post-processing. Other embodiments may have a field of view overlay configuration suitable for high dynamic range cameras that allows the ability to capture two simultaneous images and then stitch them together. Cameras 215a-n may be coupled to image processor 220 to communicate images to working memory 205, device processor 250, electronic display 225, and storage (memory) 210.

[0048] Image processor 220 may be configured to perform various processing operations on received image data comprising N portions of the target image in order to output a high quality stitched image, as will be described in greater detail below. The image processor 220 may be a general purpose processing unit or a processor specially designed for imaging applications. Examples of image processing operations include reframing, scaling (e.g., to a different resolution), image stitching, image format conversion, color interpolation, color processing, image filtering (e.g., spatial image filtering). ), lens artifact or defect correction, etc. The image processor 220 may, in some embodiments, comprise a plurality of processors. Certain embodiments may have a processor dedicated to each image sensor. The image processor 220 may be one or more dedicated image signal processors (ISPs) or a software implementation of a processor.

[0049] As shown, the image processor 220 is connected to a memory 230 and a working memory 205. In the illustrated embodiment, the memory 230 stores the capture control module 235, the image stitching module 240, and a operating system 245. These modules include instructions that configure the image processor 220 of the device processor 250 to perform various image processing and device management tasks. Working memory 205 may be used by image processor 220 to store a working set of processor instructions contained in memory component modules 230. Alternatively, working memory 205 may also be used by image processor 220 to store dynamic data created during operation of device 200.

[0050] As mentioned above, the image processor 220 is configured by several modules stored in the memories. The capture control module 235 may include instructions that configure the image processor 220 to adjust the focus position of cameras 215a-n. The capture control module 235 may further include instructions that control the general image capture functions of the device 200. For example, the capture control module 235 may include instructions that invoke subroutines to configure the image processor. 220 to capture raw image data of a target image scene using cameras 215a-n. The capture control module 235 may then call on the image stitching module 240 to perform a stitching technique on the N partial images captured by the cameras 215a-n and output a stitched and reframed target image to the imaging processor 220. The module capture control module 235 may also call on image stitching module 240 to perform a stitching operation on raw image data in order to output a preview image of a scene to be captured, and update the preview image. preview at certain time intervals or when the scene in the raw image data changes.

[0051] The image stitching module 240 may comprise instructions that configure the image processor 220 to perform stitching and reframing techniques on the captured image data. For example, each of the N sensors 215a-n can capture a partial image comprising a portion of the target image according to each sensor's field of view. Fields of view may share areas of overlap, as described previously and below. In order to output a single target image, the image stitching module 240 may configure the image processor 220 to combine the multiple N partial images to produce a high-resolution target image. Target imaging can occur through known image stitching techniques. Examples of image stitching can be found in U.S. Patent Application No. 11/623,050 (Docket 060170) which is incorporated herein in its entirety by reference.

[0052] For example, the image stitching module 240 may include instructions to compare the areas of overlap along the edges of the N partial images to match features in order to determine the rotation and alignment of the N partial images relative to each other. . Due to the rotation of the partial images and/or the shape of the field of view of each sensor, the combined image may form an irregular shape. Therefore, after aligning and combining N partial images, the image stitching module 240 may invoke subroutines that configure the image processor 220 to reframe the combined image to a desired shape and aspect ratio, e.g., a rectangle 4 :3 or a 1:1 square. The reframed image may be sent to device processor 250 for display on screen 225 or to save on storage 210.

[0053] Operating system module 245 configures image processor 220 to manage working memory 205 and processing resources of device 200. For example, operating system module 245 may include device triggers to manage hardware resources , such as the 215a-n cameras. Therefore, in some embodiments, the instructions contained in the image processing modules discussed above may not interact with these hardware resources directly, but instead interact through standard subroutines or APIs located in the operating system component 270. The instructions in operating system 245 can then interact directly with these hardware components. The operating system module 245 may further configure the image processor 220 to share information with the device processor 250.

[0054] Device processor 250 may be configured to control screen 225 to display the captured image, or a preview of the captured image, to a user. Screen 225 may be external to imaging device 200 or may be part of imaging device 200. Screen 225 may also be configured to provide a display that displays a preview image for use prior to capturing an image, or can be configured to display a captured image stored in memory or recently captured by the user. The screen 225 may comprise an LCD or LED screen, and may implement touch-sensitive technologies.

[0055] Device processor 250 may write data to storage module 210, for example, data representing captured images. Although the storage module 210 is graphically represented as a traditional disk device, those skilled in the art would understand that the storage module 210 can be configured as any storage media device. For example, storage module 210 may include a disk drive, such as a flexible disk drive, hard disk drive, optical disk drive, or magneto-optical disk drive, or a solid-state memory, such as a memory FLASH, RAM, ROM, and/or EEPROM. The storage module 210 may also include multiple memory units, and any of the memory units may be configured to be within the image capture device 200, or may be external to the image capture device 200. For example, The storage module 210 may include a ROM memory containing system program instructions stored within the image capture device 200. The storage module 210 may also include memory cards or high-speed memories configured to store captured images that can be removable from the camera.

[0056] Although Figure 2 depicts a device having separate components to include a processor, imaging sensor, and memory, an individual skilled in the art would recognize that these separate components can be combined in a variety of ways to achieve particular design goals. For example, in an alternative embodiment, memory components may be combined with processor components to save costs and improve performance.

[0057] Additionally, although Figure 2 illustrates two memory components, including memory component 230 comprising multiple modules and a separate memory 205 comprising working memory, an individual skilled in the art would recognize various embodiments that utilize different memory architectures. memory. For example, a design may utilize static ROM or RAM for storing processor instructions that implement modules contained in memory component 230. The processor instructions may be loaded into RAM to facilitate execution by image processor 220. For example, working memory 205 may comprise RAM memory, with instructions loaded into working memory 205 prior to execution by processor 220.

III. Autofocus Techniques Systems Overview

[0058] Figure 3 illustrates a scaled comparison of an embodiment of a portion of a folded optics assembly 300 for an array camera and a voice coil motor 310.

[0059] The folded optics assembly portion 300 includes the sensor 125, the coating glass 126, a first portion 130 of the lens system including lenses L1, L2, L3, L4, and L5, a second portion of the lens system lenses including the lens L6 positioned over the cover glass 126, where the cover glass 126 is positioned over the sensor 125. The portion of the folded optics assembly 300 as illustrated also includes the unit including the prism 136A and the block 136B ( referred to herein as an “optical element”). In other embodiments, the optical element may include only the prism 136A without the block 136B or may include a reflective surface, such as a mirror. The optical element may function as the secondary light bending surface in the bent optics system for a camera, redirecting light that has passed through the focusing portion 130 of the lens system onto the sensor 125.

[0060] Lens L6 may be a field correcting lens, in some embodiments, and may be a stationary component of the L1-L6 lens system. The secondary light bending surface 135 extends away from the lens L6, and, as illustrated, is formed as a prism 136A coupled to a support block 136B on the secondary light bending surface 135. It is possible for a mirror be placed between 136A and 136B rather than using the internal reflective characteristics of a prism to redirect the light.

[0061] As illustrated, the portion of the folded optics assembly 300 may have an overall height of approximately 4.1 mm. In other embodiments, the height may vary from approximately 4.0 mm to 4.5 mm, and, in some embodiments, 4 mm or less.

[0062] Slender cameras based on bent optics use one or more lenses plus mirrors/prisms to bend the light path and achieve a low z-height. The lens diameter for the first lens assembly 130 extends throughout the thickness of the camera and almost completely determines the overall thickness of the arrangement. This is advantageous for producing a slender camera, but makes it quite difficult to use voice coil motors (VCM) for focus and/or autofocus. This is because the smallest VCM currently available, illustrated in Figure 5 as VCM 310, has a side length of approximately 7.5 mm. Therefore, if positioned as intended (with the lens threaded into the position of cylinder 320 within opening 325 in engine 315), the z-height of the array camera would have a thickness greater than 7.5 mm.

[0063] However, VCMs consist of one of the most effective and successful actuators for focusing mobile cameras. The current alternative for focusing would be stepper piezomotors. However, these engines are more expensive, noisier, slower, and less reliable when compared to VCMs. As a result, piezomotors are rarely used in mobile cameras. Correspondingly, VCMs are desirable as actuators for the focus mechanism of an array camera implemented in a mobile device. The scale comparison of Figure 3 illustrates the increase in height that would occur if the first lens group 130 of the folded optics assembly 300 were threaded into the VCM 310, as is conventional when using VCMs for focusing purposes.

[0064] Figures 4A-4D illustrate various embodiments of a folded optics mount autofocus system that implements a VCM 160 without a significant height increase to the array camera. As illustrated, the overall height H of the folded optics camera is only reduced by a small amount due to the movement 190 of the secondary light bending surface 135. In some embodiments, the movement 190 may add approximately 120 microns to approximately 150 microns to the height of the folded optics system H. The height of the folded optics system H may be approximately 4.5 mm or less, or approximately 4.1 mm or less, in various embodiments.

[0065] Figure 4A illustrates an example of an embodiment of the autofocus assembly in a folded optics assembly 405 using a mirror 137 as the secondary light bending surface 135. The folded optics assembly 405 includes mirror 137, the assembly of lens 130, the primary light bending surface 124 incorporated in the prism 145, coating glass 126, and sensor 125. As illustrated, the sensor 125 is mounted in a rectangular slot formed in the printed circuit board 195. The autogenous protrusions 107 are part of the sensor 125 and are used to contact electrically conductive pads on the printed circuit board 195. The printed circuit board 195 is mounted on the substrate 150 and remains stationary with respect to the substrate 150. This is just one example of how the sensor 125 may be mounted to substrate 150 and makes electrical contact with a printed circuit board 195.

[0066] In the embodiment illustrated in Figure 4A, a VCM 160 is positioned adjacent to the folded optics assembly 405. Although only one folded optics assembly 405 is shown, a VCM could be placed adjacent to each folded optics assembly 405 in the array, in some modalities. The movement of VCM is perpendicular to the optical axis 123 passing through the lens mount 130 and perpendicular to a plane formed by the substrate 150, and this movement 190 may be transferred (e.g., directly transferred) to the mirror 137, which may be extremely light.

[0067] As illustrated, light representing a portion of the target image scene enters the prism 145 and follows the optical path 123 (to pass) through the lens assembly 130. The lens assembly 130 can focus the light, which, then moves to the secondary light bending surface 135 to be redirected onto the sensor 125. Movement 190 of the secondary reflective surface, a mirror 135 changes the path length of the focused light received from the lens assembly before the light is incident on sensor 125. For example, moving the secondary reflective surface 135 toward the sensor 135 shortens the length of the optical path, while moving the secondary reflective surface 135 away from the sensor 135 lengthens the length of the optical path. This movement of the secondary reflective surface 135 can shift the location of the image on the sensor. Correspondingly, the sensor 125 may be sized and positioned to receive light from the secondary reflective surface 135 along the range of motion 190 such that some portions of the sensor 125 may be reused at various positions of the secondary reflective surface 135.

[0068] Still referring to Figure 4A, the VCM 160 includes a moving part and a stationary body 160. The stationary body VCM 160, fixed to the printed circuit board 195 (as shown) coupled to the substrate 150 (or, in other embodiments, the VCM 160 may be coupled directly to the substrate 150) may include a permanent magnet and an iron housing producing a magnetic field through the air gap of the magnetic circuit. The moving part may include a coil winding (not shown) and a power cord (not shown) with one end being connected to the coil winding, as well as a drive member 165 threaded into the coil winding. When a current is supplied to the coil winding, the coil winding will move through the air gap as a result of the interaction between the current and the magnetic field, moving the drive member 165 up and down. In some embodiments, and in the embodiment illustrated in Figure 4A, the drive member 165 contacts a lever 180, and the second light-bending surface 135 may be coupled to one end of the lever 180, for example, by a coupling. 184 (e.g., glue or other adhesive or mechanical coupling). An optional counterweight 182 can be coupled to the other end of the lever 180 that can balance the movement 190 of the mirror. The counterweight may be, for example, a 0.02 g counterweight.

[0069] This mechanism is very light and by positioning the VCM 160 vertically within the array camera, the approximately 4.0 mm height of the VCM 160 body does not increase the overall height of the array. The driving member 165 may project slightly from the stationary body of the VCM 160 and would move, for example, up to approximately 120 microns to approximately 150 microns to effectively change focus considering the relatively long focal length for the optics assembly. doubled (note that the present example is at a focal length of 6 mm instead of the typical 4 mm). The lever 180 may be a very slender and elongated piece of material suitable for transferring movement from the drive member 165 to the mirror 137. As such, the drive member 165, the movement 190, and the lever 180 do not add significantly to the height general camera arrangement. The movement of the VCM 160 is agile and very fast, and requires little friction to operate properly. This can be achieved by this revealed lever mechanism, and additionally by the extremely low weight mirror.

[0070] The illustrated autofocus assembly may use a standard camera VCM, as is typically provided directly from a mass manufacturer, with an additional drive member 165, lever 180, counterweight 182, and mirror. Using a standard VCM allows for easy fabrication of the autofocus mount.

[0071] Figure 4B illustrates an example of an embodiment of the autofocus assembly in a folded optics assembly 405 using the optical element 136C (illustrated including the prism 136A and the support block 136B) to form the second optical redirection surface. light 135 which is a prism. Some embodiments may include a reflective coating between the prism 136A and the support block 136B. As illustrated, the VCM 160 may be placed adjacent to the folded optics assembly, and the drive member 165 of the VCM 160 is coupled to the optical element 136C by the lever 180 in the coupling 184, which may be a layer of adhesive, in some embodiments. Due to the greater weight of the optical element 136C relative to the mirror 137 discussed previously in relation to Figure 6A, the counterweight 182 must be suitably large to balance the optical element 136C. The greater weight of the optical element 136C and the counterweight 182 can beneficially reduce possible vibrations from the VCM 160 during movement 190.

[0072] Figure 4C illustrates an example of an embodiment of the autofocus mount on a folded optics mount with an additional riser 188 for leverage. The embodiment illustrated in Figure 4C includes an optical element 136C including a prism 136A and a support block 136B to form the second light redirecting surface 135. In various embodiments, the second light redirecting surface may be a prism or a mirror. . For example, as illustrated in Figure 4A, the riser 188 may be implemented in a folded optics assembly 405 with a mirror 137 as the second light redirecting surface 135. A first end of the lever 180 is coupled to the riser 188, a portion middle of the lever 180 contacts the drive member 165 of the VCM 160, and a second end of the lever 180 is coupled to the optical element (coupling 184). The tie rod 188 provides additional leverage and stability to the movement of the lever 180. In this embodiment, the lever 180 can be characterized, for example, as a third class of lever. Correspondingly, a counterweight is not necessary. The lever 180 may be a thin spring-like piece of metal, in some embodiments, capable of flexing due to movement of the drive member 165 of the VCM 160. The tie rod 188 may be coupled to the lever at a hinge 186, which may be a pivot, flexible coupling, weld, or bent piece of metal, in various forms. This design may increase the range of movement 190 required for a camera with a long focal length, in some embodiments, and/or may prevent a “seesaw” movement (tilting back and forth of the lever 180) around the VCM.

[0073] Figure 4D illustrates another embodiment of the autofocus assembly that implements leverage in autofocus-related movements. A first end of the lever 180 is coupled to the drive member 165 of the VCM 160, an intermediate portion of the lever is coupled in coupling 184 to the optical element (illustrated as prism 136A and block 136B, but in other embodiments it could be the mirror 137), and a second end of the lever 180 is coupled to a lower portion of the central prism 145. As illustrated, the rod 188 and the hinge 186 are positioned between the second end of the lever 180 and the prism 145. In this embodiment, the lever 180 can be characterized, for example, as a second class of lever. In some embodiments, the lever 180 may be glued or otherwise coupled directly to the lower portion of the central prism 145. Figure 4D illustrates another way to provide leverage and stability to the autofocus assembly so that no counterweight is required. This design may decrease the range of movement 190 required for a camera with a short focal length, in some embodiments, and/or may avoid a “seesaw” movement around the VCM.

[0074] In any of Figures 4A to 4D, instead of prism 145, a reflective surface can be implemented. Furthermore, the particular lens configurations are representative of various examples, but are not intended to limit the assembly of folded optics. Furthermore, in some embodiments, the first light directing surface (prism 145) may be omitted and light may enter the assembly directly through the lens assembly 130. In other embodiments, the VCM may be coupled to the first directing surface. light (prism 145) to move that surface and the secondary light directing surface can be omitted with the sensor repositioned to receive light directly from the lens assembly 130. The VCM implementations described in Figures 4A to 4D can be beneficial to autofocus on any thin folded optics camera in which a VCM cannot be positioned with the lens mount threaded through the inside of the VCM as is conventional due to height limitations in the system providing a range of moving focus values a light-bending surface across a small range of the overall camera height.

[0075] Figures 5A-5C illustrate various embodiments of a folded optics array camera autofocus system 500A, 500B, 500C including a series of VCMs 515 and a series of folded optics mounts 510.

[0076] Figure 5A illustrates an embodiment of an autofocus array camera system 500A for an array camera having three folded optics assemblies 510. System 500A includes an object prism (or pyramid) 505, and three individual cameras 510, each associated with a VCM focusing motor mechanism 515. The folded optics assemblies 510 may include any of the configurations described herein for lens assemblies, secondary light bending surfaces, and image sensors, and are positioned around a first light-bending surface 505, which may be a prism or mirrored pyramid configured to divide light representing the target imaging scene into multiple portions and direct each portion through a corresponding folded optics assembly 510 Autofocus for each folded optics assembly 510 is controlled by a corresponding VCM 515. Each VCM may be coupled to the corresponding folded optics assembly 510 via a lever 520 and couplings 515, as well as a counterweight 530. Either of these. Lever, counterweight, and/or tie combinations discussed previously may be implemented instead of the illustrated embodiment. This design can be implemented with different camera arrangements including varying numbers of folded optics mounts 510 and correspondingly varying primary light bending surfaces 505, although the illustrated example is a three-camera housing.

[0077] The movement of each VCM 515 can be carefully controlled, for example by a controller with programmable computational instructions (not illustrated), to keep all folded optics assemblies 510 focused at the same distance. Electronic communication may be provided between the controller and some or all of the VCMs 515. The controller may compute the movement required for each folded optics assembly so that all of the folded optics mounts 510 are focused at the same distance at the same time. In one example, the second light directing surface of each of the folded optics assemblies 510 may be moved by the same amount. However, due to manufacturing variation, in some embodiments, the movement of the second light directing surface of each of the folded optics assemblies 510 may be calibrated to represent the variation, e.g., based on a coefficient that defines a relationship between the folded optics mounts 510. In some embodiments, a folded optics mount 510 may be selected as a “trigger” that will be used to decide where to focus the other mounts.

[0078] Figure 5B illustrates a compact drawing of a 500B three-camera array autofocus system. Each VCM 515 is mechanically connected to a counterweight 530 by a first lever 535 and to two adjacent folded optics mounts 510 by a second lever 540 that extends between the adjacent folded optics mounts 510. The first lever 535 and the second lever 540 they may be separate levers coupled together, or they may form a solid T-shaped piece in some embodiments. The motion of each VCM 515 is correspondingly distributed between the two adjacent folded optics assemblies, and the motion of the second light directing surface of each folded optics assembly is affected by the two neighboring VCMs. Although a triangular connection between three VCMs of three folded optics assemblies is shown, other embodiments may have other numbers of VCMs and folded optics assemblies and/or other configurations of connections between the folded optics assemblies 510, e.g., a circular connection , hexagonal or rectangular, to name a few examples.

[0079] An example of autofocus control of the system of Figure 5B may provide mirror displacement when the VCM movement is known, or may provide the necessary VCM movement for a known required mirror displacement. The displacements of all individual motors can be described by a 3D vector, X = (x, y, z). The displacements of all individual second refractive surfaces can be described by a 3D vector, V = (u, v, w).

[0080] The surface and motor displacements are related by V = AX, where A is the transform matrix:

[0081] Correspondingly, by multiplying the motor displacement vector X by the transformation matrix A, the corresponding movement of the second refractive surfaces can be provided. In typical autofocus techniques, the second reflective surface displacements required for focusing are known, and corresponding motor movements are required. The corresponding motor motions can be found using X = A-1 V, where the inverse transform A-1 is:

[0082] The resulting values of x, y, and z indicate the motor movement necessary for the known second reflective surface displacement. Such values of x, y, z can be provided to a motor control system to produce the required autofocus movement. As a result, the motion of all second refractive surfaces can be exactly known, and the motion of the captured images follows directly. This movement can be represented in the stitching techniques used to generate the final image.

[0083] Although specific vectors and matrices are discussed above, these are provided as an example for the three specific VCMs, three folded optics assemblies, and triangular connection of the embodiment of Figure 5B. The sizes and values of the vectors and matrices may change based on the number of, and links between, the VCMs and the folded optics assemblies in other array camera autofocus embodiments. Such embodiments preferably have a 1:1 correspondence between the movement of motors and the movement of second refractive surfaces so that the displacement can be accurately computed.

[0084] Figure 5C illustrates a compact drawing of a 500C four-camera array autofocus system. The arrangement includes four VCMs 515 and four folded optics assemblies 510 positioned around primary light-bending surfaces of a central mirror pyramid or prism 506. Each VCM 515 is mechanically linked, using a lever 520, to two optical assemblies. adjacent folded optics 510. As illustrated, no counterweight is provided due to the direct mechanical connection between the VCMs and the folded optics assemblies. In some embodiments, additional connections may be provided diagonally across the array (e.g., from the upper right VCM to the lower left VCM and/or from the upper left VCM to the lower right VCM). Such a design makes efficient use of space, occupies substantially the same rectangular area as the array camera without the autofocus system.

[0085] In the array camera autofocus system examples in this document, a VCM motor is shown and described as actuating the lever to move the second light directing surface to change the focus position. However, in other embodiments, the VCM could be replaced by a stepper piezomotor or a microelectromechanical systems (MEMS) motor.

IV. Sample image capture process overview

[0086] Figure 6 illustrates one embodiment of a folded optics image capture process 900. Process 900 begins at block 905, in which a plurality of imaging sensor assemblies are provided. This can include any of the sensor array configurations discussed above in relation to the previous images. Sensor assemblies may include, as discussed above in connection with Figure 3, a sensor, lens system, and a reflective or refractive surface positioned to redirect light from the lens system onto the sensor.

[0087] Process 900 then moves to block 910, wherein at least one light bending surface is mounted proximate to the plurality of image sensors. For example, this step may comprise assembling a central mirror pyramid or prism between two rows of sensor arrays, wherein the central mirror pyramid or prism comprises a surface or facet associated with each sensor in the arrays.

[0088] Process 900 then passes to block 915, in which light comprising a target image of a scene is redirected using at least one light bending surface to the imaging sensors. For example, portion of the light may be redirected away from each of a plurality of surfaces to each of a plurality of sensors. This may further comprise passing light through a lens assembly associated with each sensor, and may also include redirecting light using a second surface on a sensor.

[0089] Block 915 may further comprise focusing light using the lens mount and/or through movement of any of the light-bending surfaces, for example, by providing at least one VCM configured to move a light-bending surface. along a geometric axis substantially parallel to the height of the arrangement, as previously described. Correspondingly, block 915 may incorporate the VCM control techniques discussed above to synchronize focusing provided by multiple VCMs for multiple folded optics assemblies.

[0090] Process 900 may then proceed to block 920, wherein the sensors capture a plurality of images of the target image scene. For example, each sensor may capture an image of a portion of the scene corresponding to that sensor's field of view. Taken together, the fields of view of the plurality of sensors cover at least the target image in object space.

[0091] Process 900 may then proceed to block 925 in which an image stitching method is performed to generate a single image from the plurality of images. In some embodiments, the image stitching module 240 of Figure 2 can perform stitching. This may include known image stitching techniques. Furthermore, any overlapping areas in the fields of view can generate overlap in the plurality of images, this can be used to align the images in the stitching process. For example, block 925 may further include identifying common features in the overlapping area of adjacent images and using the common features to align the images.

[0092] Then, process 900 passes to block 930 in which the stitched image is reframed to a specified aspect ratio, for example, 4:3 or 1:1. Finally, the process ends after storing the reframed image in block 935. For example, the image may be stored in storage 210 of Figure 2, or it may be stored in working memory 205 of Figure 2 for display as a preview image. visualization of the target scene.

V. Systems Implementation and Terminology

[0093] The implementations disclosed herein provide systems, methods and apparatus for automatically focusing multiple sensor array cameras. One skilled in the art will recognize that these embodiments can be implemented in hardware, software, firmware, or any combination thereof.

[0094] In some embodiments, the circuits, processes and systems discussed above may be used in a wireless communication device. Wireless communication device can be a type of electronic device used to communicate wirelessly with other electronic devices. Examples of wireless communication devices include cell phones, smart phones, Personal Digital Assistants (PDAs), digital book readers (e-readers), gaming systems, music players, netbooks, wireless modems, wireless computers, devices tablet, etc.

[0095] The wireless communication device may include one or more image sensors, two or more image signal processors, a memory that includes instructions or modules for carrying out the processes discussed above. The device may also have data, instructions and/or processor loading data from memory, one or more communication interfaces, one or more input devices, one or more output devices such as a display device and a source. /power interface. The wireless communication device may additionally include a transmitter and a receiver. The transmitter and receiver can jointly be called transceiver. The transceiver can be coupled to one or more antennas to transmit and/or receive wireless signals.

[0096] The wireless communication device may wirelessly connect to another electronic device (e.g., base station). A wireless communication device may alternatively be called a mobile device, mobile station, subscriber station, user equipment (UE), remote station, access terminal, mobile terminal, terminal, user terminal, subscriber unit, etc. Examples of wireless communication devices include laptop-type or desktop-type computers, cell phones, smart phones, wireless modems, e-readers, tablet-type devices, gaming systems, etc. Wireless communication devices may operate in accordance with one or more industry standards such as the Third Generation Partnership Project (3GPP). Thus, the general term “wireless communication device” may include wireless communication devices described with variant nomenclatures in accordance with industry standards (e.g., access terminal, user equipment (UE), remote terminal, etc.) .

[0097] The functions described herein may be stored as one or more instructions on a processor-readable or computer-readable medium. The term “computer-readable medium” refers to any available medium that can be accessed by a computer or processor. By way of example, and not limitation, such computer-readable media may comprise RAM, ROM, EEPROM, flash memory, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other media that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Magnetic disc and optical disc, as used herein, include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray® disc, where magnetic discs typically reproduce data from magnetic mode, while optical discs reproduce data optically with lasers. It should be noted that computer-readable media may be tangible and non-transitory. The term “computer program product” refers to a computing device or processor in combination with code or instructions (e.g., a “program”) that can be executed, processed, or computed by the computing device or processor. As used herein, the term “code” may refer to software, instructions, code or data that is/are executable by a computing device or processor.

[0098] The methods disclosed herein comprise one or more steps or actions to achieve the described method. The steps and/or actions of the method can be interchanged without departing from the scope of the claims. In other words, unless a specific order of steps and actions is required for the proper operation of the method being described, the order and/or use of the specific steps and/or actions may be modified without departing from the scope of the claims.

[0099] It should be noted that the terms “couple”, “that couples”, “coupled” or other variations of the word couple as used herein may indicate an indirect connection or a direct connection. For example, if a first component is “coupled” to a second component, the first component may be directly connected to the second component or directly connected to the second component. As used herein, the term “plurality” denotes two or more. For example, a plurality of components indicates two or more components.

[0100] The term “determine” encompasses a wide variety of actions, and therefore “determine” may include calculating, computing, processing, deriving, investigating, searching (e.g., searching a table, database, or other structure data), check and similar. Also, “determine” may include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), and the like. Likewise, “determine” can include resolve, select, choose, establish, and the like.

[0101] The phrase “based on” does not mean “based only on”, unless otherwise expressly specified. In other words, the phrase “based on” describes both “based only on” and “based at least on.”

[0102] In the aforementioned description, specific details are given to provide a complete understanding of the examples. However, it should be understood by a person of ordinary skill in the art that the examples can be practiced without these specific details. For example, electrical components/devices can be shown in block diagrams so as not to obscure the examples with unnecessary details. In other examples, such components, other structures, and sets of procedures may be shown in detail to further explain the examples.

[0103] Headings are included herein for reference and to help locate various sections. These titles are not intended to limit the scope of the concepts described in connection therewith. Such concepts may have applicability throughout the entire descriptive report.

[0104] It should also be noted that the examples can be described as a process, which is depicted as a flowchart, a flow diagram, a finite state diagram, a structure diagram, or a block diagram. Although a flowchart may describe operations as a sequential process, many of the operations may be performed in parallel or concurrently, and the process may be repeated. Additionally, the order of operations can be rearranged. A process is terminated when its operations are completed. A process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a software function, its termination corresponds to a return of the function to the calling function or to the main function.

[0105] The foregoing description of the disclosed implementations is provided to enable any individual skilled in the art to produce or use the present invention. Various modifications to such implementations will be readily apparent to those skilled in the art and the generic principles defined herein may be applied to other implementations without departing from the spirit or scope of the invention. Therefore, the present invention is not intended to be limited to the implementations shown in the present document, however, it must be in accordance with the broader scope, consistent with the principles and innovative features disclosed in the present invention.

Claims (14)

1. An autofocus module for a camera, comprising: an image sensor (125) mounted to a substrate (150) defining a foreground; a first reflective surface (124); a second reflective surface (135), the first light directing surface being positioned to direct light representing a portion of a scene that is incident on the first reflective surface toward the second reflective surface and the second reflective surface is positioned to direct light toward the image sensor; an actuator (160) mounted to the substrate and including a drive member (165) positioned for movement in a direction orthogonal to the plane of the substrate; characterized by the fact that a lever (180) contacts the drive member and the second reflective surface so that the movement of the drive member is transferred to the second reflective surface. 2. The autofocus module of claim 1, wherein the actuator comprises a voice coil motor. 3. Autofocus module according to claim 1, characterized by the fact that the movement in the direction orthogonal to the plane of the substrate is within a range of approximately 120 microns to approximately 150 microns. 4. The autofocus module of claim 1, further comprising a lens mount positioned between the first reflective surface and the second reflective surface. 5. Autofocus module according to claim 1, characterized by the fact that it further comprises a counterweight coupled to a first end of the lever, wherein the drive member is coupled to an intermediate portion of the lever and the second reflective surface is coupled to a second end of the lever, wherein a counterweight weight is preferably selected to at least partially balance a weight of the second reflective surface, and wherein the second reflective surface comprises one of a reflective mirror and a prism 6. Autofocus module according to claim 1, characterized in that it further comprises a rod coupled to a first end of the lever using a joint. 7. The autofocus module of claim 6, wherein the drive member is coupled to an intermediate portion of the lever and the second reflective surface is coupled to a second end of the lever, or wherein the rod is coupled to a structure forming the first reflective surface, and wherein the second reflective surface is coupled to an intermediate portion of the lever and the actuating member is coupled to a second end of the lever. 8. The autofocus module of claim 1, wherein a first end of the lever is coupled to a structure forming the first reflective surface, and wherein the second reflective surface is coupled to an intermediate portion of the lever and the actuating member is coupled to a second end of the lever. 9. Method for capturing an image of a target imaging scene with the use of a camera comprising a plurality of folded optical assemblies mounted to a substrate, the substrate defining a foreground, wherein the plurality of folded optical assemblies folded optical assemblies are positioned around a structure that provides a corresponding plurality of first reflective surfaces, the plurality of folded optical assemblies each comprising an image sensor and a second reflective surface, the method characterized by the fact that it comprises: for each of the plurality of bent optics assemblies: inducing movement of a driving member of an actuator in a direction orthogonal to the plane of the substrate; transfer movement of the drive member to the second reflective surface, through a lever in contact with the drive member and the second reflective surface, the corresponding first reflective surface of the plurality of reflective surfaces positioned to direct light toward the second surface reflective and the second reflective surface is positioned to direct light toward the image sensor; and capturing an image representing light using the image sensor. 10. The method of claim 9, further comprising maintaining each of the plurality of folded optic assemblies focused at approximately the same distance. 11. The method of claim 10, further comprising determining a quantity of movement for a driver assembly selected from the plurality of folded optics assemblies and using the quantity of movement for the selected driver assembly to determine the movement for each of the other plurality of folded optic assemblies. 12. The method of claim 11, wherein determining motion for each of the other plurality of folded optics assemblies is based at least partially on the amount of motion for the selected driver assembly and at least partially into one or more coefficients defining a relationship between the plurality of folded optical assemblies. 13. The method of claim 9, further comprising performing a stitching operation on the image representing the portion of light from each of the plurality of folded optical assemblies to form the image of the scene. 14. Computer-readable memory characterized by the fact that it has instructions stored therein that, when executed, cause one or more processors to carry out the method, as defined in any one of claims 9 to 13.