WO2020107997A1 - 成像装置和电子设备 - Google Patents

成像装置和电子设备 Download PDF

Info

Publication number
WO2020107997A1
WO2020107997A1 PCT/CN2019/104514 CN2019104514W WO2020107997A1 WO 2020107997 A1 WO2020107997 A1 WO 2020107997A1 CN 2019104514 W CN2019104514 W CN 2019104514W WO 2020107997 A1 WO2020107997 A1 WO 2020107997A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
meta
images
block
matching
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/104514
Other languages
English (en)
French (fr)
Inventor
徐锐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Oppo Mobile Telecommunications Corp Ltd
Original Assignee
Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Oppo Mobile Telecommunications Corp Ltd filed Critical Guangdong Oppo Mobile Telecommunications Corp Ltd
Publication of WO2020107997A1 publication Critical patent/WO2020107997A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H04N23/54—Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/02—Constructional features of telephone sets
    • H04M1/0202—Portable telephone sets, e.g. cordless phones, mobile phones or bar type handsets
    • H04M1/026—Details of the structure or mounting of specific components
    • H04M1/0264—Details of the structure or mounting of specific components for a camera module assembly
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/57—Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices

Definitions

  • the present application relates to the field of imaging technology, in particular to an imaging device and electronic equipment.
  • Embodiments of the present application provide an imaging apparatus and electronic equipment.
  • An imaging device includes a microlens array structure and a plurality of photosensitive pixels.
  • the microlens array structure covers a plurality of photosensitive pixel sets, and each of the photosensitive pixel sets includes a plurality of the photosensitive pixels, each of which A plurality of photosensitive pixels in the collection of photosensitive pixels receive light incident through the microlens array structure to output multiple meta images;
  • the imaging device further includes a processor, the processor is configured to fuse multiple meta images Obtain a merged image; calculate depth information of the scene based on at least two of the meta images; perform predetermined processing on the merged image according to the depth information.
  • the electronic device includes a housing and an imaging device.
  • the imaging device is mounted on the housing.
  • the imaging device includes a microlens array structure and a plurality of photosensitive pixels, the microlens array structure covering a plurality of photosensitive pixel sets, each of the photosensitive pixel sets includes a plurality of the photosensitive pixels, each of the photosensitive pixel sets
  • a plurality of photosensitive pixels receive light incident through the microlens array structure to output multiple meta images;
  • the imaging device further includes a processor, the processor is configured to fuse multiple meta images to obtain a merged image; Calculate the depth information of the scene for at least two of the meta images; perform predetermined processing on the merged image according to the depth information.
  • the electronic device includes a housing, an imaging device, and a processor.
  • the imaging device includes a microlens array structure and a plurality of photosensitive pixels, and the microlens array structure covers a plurality of photosensitive pixel sets, each of the photosensitive pixel sets includes a plurality of the photosensitive pixels, and each of the photosensitive pixel sets A plurality of photosensitive pixels in receives light incident through the microlens array structure to output multiple meta images.
  • the processor is configured to fuse multiple meta images to obtain a merged image, calculate scene depth information based on at least two meta images, and perform predetermined processing on the merged image according to the depth information.
  • FIG. 1 and FIG. 2 are schematic perspective structural views of electronic devices according to certain embodiments of the present application.
  • FIG. 3 is a schematic diagram of a partial stereoscopic structure of an imaging device according to some embodiments of the present application.
  • FIG. 4 is a schematic partial cross-sectional view of the imaging device in FIG. 3 taken along line IV-IV.
  • FIG. 5 is a schematic partial cross-sectional view of an imaging device according to some embodiments of the present application.
  • FIG. 6 is a schematic diagram of a scene of meta-image stitching in some embodiments of the present application.
  • FIG. 7 is a schematic diagram of a scenario of depth information calculation in some embodiments of the present application.
  • FIG. 8 is a three-dimensional assembly schematic diagram of a microlens array structure according to some embodiments of the present application.
  • FIG. 9 is a schematic cross-sectional view of the microlens array structure in FIG. 8 taken along line IX-IX.
  • FIG. 10 is an exploded schematic diagram of a microlens array structure according to some embodiments of the present application.
  • FIG. 11 is a three-dimensional assembly schematic diagram of a micro lens array structure according to some embodiments of the present application.
  • FIG. 12 is a schematic cross-sectional view of the microlens array structure in FIG. 11 along line XII-XII.
  • FIG. 13 is a three-dimensional assembly schematic diagram of a microlens array structure according to some embodiments of the present application.
  • FIG. 14 is a schematic cross-sectional view of the microlens array structure in FIG. 13 taken along line XIV-XIV.
  • FIG. 15 is a schematic cross-sectional view of the microlens array structure of some embodiments of the present application taken along a position corresponding to the line IX-IX in FIG. 8.
  • 16 is a three-dimensional assembly schematic diagram of a micro lens array structure according to some embodiments of the present application.
  • FIG. 17 is a three-dimensional assembly schematic diagram of a microlens array structure according to some embodiments of the present application.
  • FIG. 18 is a schematic block diagram of an electronic device according to some embodiments of the present application.
  • the first feature is “on” or “under” the second feature may be that the first and second features are in direct contact, or the first and second features are indirectly through an intermediary contact.
  • the first feature is “above”, “above” and “above” the second feature may be that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature.
  • the first feature is "below”, “below”, and “below” the second feature may be that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is less horizontal than the second feature.
  • the imaging device 100 includes a microlens array structure 10 and a plurality of photosensitive pixels 20.
  • the microlens array structure 10 covers a plurality of photosensitive pixel sets, and each photosensitive pixel set includes a plurality of photosensitive pixels 20.
  • the plurality of photosensitive pixels 20 in each photosensitive pixel set receive light incident through the microlens array structure 10 to output multiple Zhang Yuan image.
  • the imaging apparatus 100 further includes a processor 50, which is used to: fuse multiple meta images to obtain a merged image; calculate depth information of the scene based on at least two meta images; and perform predetermined processing on the merged image according to the depth information.
  • the processor 50 is further used to: select two meta images, one meta image as a reference meta image, and another meta image as a meta image to be matched; divide the reference meta image For multiple block images, and select one block image from the multiple block images as the reference block image; find the matching block image matching the reference block image in the meta image to be matched to form a matching image pair; perform the above division steps cyclically And the above searching steps to traverse multiple block images in the reference meta image to obtain multiple matching image pairs; fuse the reference block image and matching block image in each matching image pair to obtain a fusion sub-image, and stitch multiple fusion sub-images To obtain the stitching sub-image; use the stitching sub-image as the new reference meta-image, select one meta-image from the remaining multiple meta-images as the new meta-image to be matched, and execute the above steps to divide the reference meta-image into multiple The steps from the block image to the step of obtaining the stitched sub-image are fused to obtain a merged image
  • the processor 50 is further configured to divide multiple meta images into a reference image set and an image set to be matched, the reference image set includes multiple meta images, and the image set to be matched Including multiple meta images; selecting a meta image from the reference image set as the reference meta image, and selecting a meta image from the image set to be matched as the meta image to be matched; dividing the reference meta image into multiple block images, and Select one block image from multiple block images as the reference block image; find the matching block image matching the reference block image in the meta image to be matched to form a matching image pair; according to the reference block image and matching block in the matching image pair Parallax calculation of image depth information; cycle through the division step, search step and calculation step to traverse multiple block images in the reference meta image to obtain multiple depth information; cycle through the above selection of a meta image from the reference image set as The step from the reference meta image to the step of acquiring multiple depth information to traverse multiple meta images in the reference image set to obtain multiple depth information.
  • the processor 50 is further configured to: determine the foreground area and background area of the merged image according to depth information; and perform blur processing on the background area according to depth information.
  • the processor 50 is further configured to: determine a region to be focused on the merged image according to user input; and perform blur processing on the region of the merged image except the region to be focused according to depth information.
  • a partition wall 40 is provided between two adjacent sets of photosensitive pixels.
  • the microlens array structure 10 includes a plurality of microlenses 101, and each microlens 101 covers a set of photosensitive pixels.
  • the imaging device 100 further includes a substrate 30 on which a plurality of photosensitive pixels 20 are formed, and the surface of the substrate 30 that contacts the photosensitive pixels 20 has a curved structure.
  • At least two micro-transmissions 101 have different radii of curvature.
  • the microlens array structure 10 includes a first lens array 12 and a second lens array 14.
  • the first lens array includes a plurality of first bosses 126 arranged in a first direction.
  • the second lens array 14 includes a plurality of second bosses 146, which are arranged in the second direction.
  • the first boss 126 and the second boss 146 overlap in the third direction to form the microlens array structure 10.
  • the first direction intersects the second direction, and the third direction is perpendicular to the first direction and the second direction.
  • the first lens array 12 includes opposite first flat surfaces 122 and first convex surfaces 124, and a plurality of first bosses 126 form a first convex surface 124.
  • the second lens array 14 includes opposite second flat surfaces 142 and second convex surfaces 144, and a plurality of second bosses 146 form second convex surfaces 144. Both the first flat surface 142 and the second flat surface 144 are curved structures.
  • the electronic device 1000 includes a housing 400 and an imaging device 100.
  • the imaging device 100 includes a microlens array structure 10 and a plurality of photosensitive pixels 20.
  • the microlens array structure 10 covers a plurality of photosensitive pixel sets, and each photosensitive pixel set includes a plurality of photosensitive pixels 20.
  • the plurality of photosensitive pixels 20 in each photosensitive pixel set receive light incident through the microlens array structure 10 to output multiple Zhang Yuan image.
  • the imaging apparatus 100 further includes a processor 50, which is used to: fuse multiple meta images to obtain a merged image; calculate depth information of the scene based on at least two meta images; and perform predetermined processing on the merged image according to the depth information.
  • the electronic device 1000 includes a housing 400, an imaging device 100, and a processor 300.
  • the imaging device 100 includes a microlens array structure 10 and a plurality of photosensitive pixels 20.
  • the microlens array structure 10 covers a plurality of photosensitive pixel sets, and each photosensitive pixel set includes a plurality of photosensitive pixels 20.
  • the plurality of photosensitive pixels 20 in each photosensitive pixel set receive light incident through the microlens array structure 10 to output multiple Zhang Yuan image.
  • the processor 300 is used to: fuse multiple meta images to obtain a merged image; calculate the depth information of the scene according to at least two meta images; and perform predetermined processing on the merged image according to the depth information.
  • the electronic device 1000 includes a housing 400, an imaging device 100, and a processor 300.
  • the imaging device 100 is provided on the housing 400.
  • the electronic device 1000 may be a mobile phone, a tablet computer, a game machine, a smart watch, a smart bracelet, a head-mounted display device, a drone, and the like.
  • the embodiments of the present application are described by taking the electronic device 1000 as a mobile phone as an example. It can be understood that the specific form of the electronic device 1000 is not limited to a mobile phone.
  • the housing 400 can serve as a mounting carrier for functional elements of the electronic device 1000.
  • the housing 400 can provide protection against dust, fall, and water for functional elements.
  • the functional element may be a display screen 200, a proximity sensor, a receiver, and so on.
  • the housing 400 includes a main body 401 and a movable bracket 402.
  • the movable bracket 402 can move relative to the main body 401 under the driving of a driving device, for example, the movable bracket 402 can slide relative to the main body 401, To slide into the main body 401 (as shown in FIG. 2) or slide out of the main body 401 (as shown in FIG. 1).
  • Part of the functional elements may be installed on the main body 401, and another part of the functional elements (such as the imaging device 100, the receiver, the proximity sensor, etc.) may be installed on the movable bracket 402.
  • the movement of the movable bracket 402 can drive the other part of the functional element to retract into or extend from the main body 401.
  • FIGS. 1 and 2 are only examples of a specific form of the housing 400 and cannot be understood as a limitation to the housing 400 of the present application.
  • a collection window may be opened on the casing 400, and the imaging device 100 is installed in alignment with the collection window to receive light incident through the collection window.
  • the imaging device 100 is mounted on the movable bracket 402.
  • the user can trigger the movable bracket 402 to slide out from the main body 401 to drive the imaging device 100 to extend from the main body 401; when the user does not need to use the imaging device 100, the movable bracket 402 can be triggered to slide Enter the main body 401 to drive the imaging device 100 to retract into the main body 401.
  • the imaging device 100 includes a substrate 30, a plurality of photosensitive pixels 20, a microlens array structure 10, and a partition wall 40 disposed in this order.
  • the substrate 30 includes a top surface 31 and a bottom surface 32 opposite to each other, and a plurality of photosensitive pixels 20 are formed on the top surface 31 of the substrate 30.
  • the microlens array structure 10 is installed on the side of the light receiving surface of the photosensitive pixel 20.
  • the plurality of photosensitive pixels 20 can receive light incident after passing through the microlens array structure 10.
  • the top surface 31 and the bottom surface 32 of the substrate 30 are both planar structures, and a plurality of photosensitive pixels 20 are distributed on the top surface 31 of the planar structure, arranged in a planar manner, and a micro lens array structure 10 is arranged on the side of the light-receiving surface of the photosensitive pixel 20, and is also arranged in a plane.
  • the thickness of the substrate 30 is small, which can further reduce the height of the imaging device 100, which is beneficial to the integration of the imaging device 100 into an electronic device 1000 that has high requirements for thickness, such as mobile phones and tablet computers Waiting.
  • FIG. 4 the top surface 31 and the bottom surface 32 of the substrate 30 are both planar structures, and a plurality of photosensitive pixels 20 are distributed on the top surface 31 of the planar structure, arranged in a planar manner, and a micro lens array structure 10 is arranged on the side of the light-receiving surface of the photosensitive pixel 20, and is also arranged in a plane.
  • the bottom surface 32 of the substrate 30 is a planar structure, and the top surface 31 is a curved structure.
  • a plurality of photosensitive pixels 20 are distributed on the top surface 31 of the curved structure and arranged in a curved shape.
  • the lens array structure 10 is disposed on the side of the light-receiving surface of the photosensitive pixel 20, and is also arranged in a curved shape.
  • the arrangement of the plurality of photosensitive pixels 20 and the microlens array structure 10 in a curved shape can increase the angle of view of the imaging device 100, and the imaging device 100 can capture more objects in the scene.
  • each photosensitive pixel set includes a plurality of photosensitive pixels 20.
  • each photosensitive pixel set may include 70 ⁇ 70 photosensitive pixels 20.
  • the number of 70 ⁇ 70 is only an example.
  • each set of photosensitive pixels may further include 60 ⁇ 60, 80 ⁇ 80, 100 ⁇ 100 photosensitive pixels 20, etc., which is not limited herein.
  • the greater the number of photosensitive pixels 20 in each photosensitive pixel set the higher the resolution of the corresponding formed meta-image.
  • the plurality of photosensitive pixel sets may be arranged in a horizontal direction, a vertical direction, a "Tian" shape, or the like.
  • the microlens array structure 10 covers multiple sets of photosensitive pixels.
  • the plurality of photosensitive pixels 20 in each photosensitive pixel set can receive light incident after passing through the microlens array structure 10 to output a meta image corresponding to the photosensitive pixel set. In this way, multiple sets of photosensitive pixels can output multiple meta images.
  • the microlens array structure 10 includes a plurality of microlenses 101, and each microlens 101 covers a set of photosensitive pixels.
  • the plurality of microlenses 101 are structures independent of each other, and each microlens 101 is installed on the side close to the light-receiving surface of the plurality of photosensitive pixels 20 in the corresponding photosensitive pixel set.
  • the light in the scene passes through the microlens 101 and is incident on a plurality of photosensitive pixels 20 in the corresponding photosensitive pixel set, so that the plurality of photosensitive pixels 20 receive the light and correspondingly output multiple electrical signals.
  • the multiple electrical signals output by each photosensitive pixel 20 form a meta image.
  • the multiple microlenses 101 may have the same radius of curvature. In this case, the multiple microlenses 101 have the same focal length.
  • the imaging device 100 is imaging, the distances of the plurality of photosensitive pixels 20 in the plurality of photosensitive pixel sets to objects that can be clearly imaged in the scene are consistent. That is to say, assuming that the number of microlenses 101 is N, respectively L 1 , L 2 , L 3 ... L N , the corresponding number of photosensitive pixel sets is also N, which are the photosensitive pixel sets S 1 , S 2. S 3 ...S N.
  • the photosensitive pixel set S 1 can clearly image objects in the scene at a distance D from the imaging device 100 (D can be a specific value or a range of values), then the photosensitive pixel set S 2 , S 3 ... S N can also be Objects at a distance D from the imaging device 100 in the scene are clearly imaged.
  • the microlens 101 with a small radius of curvature can be selected so that the focal length of the microlens 101 is small to obtain a large depth of field, so that most objects in the scene can be clearly imaged.
  • the plurality of microlenses 101 may have different radii of curvature, where at least two microlenses 101 have different radii of curvature.
  • microlens L 1 to micro lens L N-1 have the same radius of curvature, and the radius of curvature of micro lens L N is different from the radius of curvature of micro lens L 1 ; or, micro lens L 1 to micro lens L N -2 has the same radius of curvature, the radius of curvature of micro lens L N-1 is different from the radius of curvature of micro lens L 1 , the radius of curvature of micro lens L N is the same as the radius of curvature of micro lens L 1 and micro lens L N-1 Different; or, the radius of curvature of any two microlenses 101 are different, and so on.
  • the microlens 101 located at the central position of the substrate 30 has a small radius of curvature, and the microlens 101 located at the peripheral position of the substrate 30 has a large radius of curvature.
  • the focal length of the microlens 101 located at the center position of the substrate 30 is small, and objects close to the imaging device 100 can be clearly imaged, and the focal length of the microlens 101 located at the peripheral position is large, which can match the imaging device. 100 distant objects are clearly imaged.
  • the subject when the imaging device 100 is imaging, the subject is generally located at the center of the field of view of the imaging device 100 and is closer to the imaging device 100, while the background object is generally located at the peripheral position of the field of view and is farther away from the imaging device 100
  • the microlens 101 with a smaller focal length at the center of the substrate 30 can clearly image the main body 401, and the microlens 101 with a larger focal length at the peripheral position of the substrate 30 can clearly image a background object.
  • a partition 40 is provided adjacent to the photosensitive pixels 20 in any two photosensitive pixel sets.
  • the micro lens L 1 covers the photosensitive pixel set S 1
  • the micro lens L 2 covers the photosensitive pixel set S 2 , the photosensitive pixel 20 of the photosensitive pixel set S 1 and the photosensitive pixel 20 of the photosensitive pixel set S 2
  • the adjacent wall is provided with a partition wall 40.
  • the partition wall 40 can prevent the light passing through the microlens L 1 from entering the photosensitive pixel set S 2 , while preventing the light passing through the micro lens L 2 from entering the photosensitive pixel set S 1 , and thus, can avoid passing through the micro lens L
  • the problem of generating a glare-like image in the meta image output by the photosensitive pixel set S 2 can also prevent the light passing through the microlens L 2 from entering the photosensitive pixel set S 1.
  • the meta-image output from the photosensitive pixel set S 1 produces a glare-like image.
  • the surface of the partition wall 40 may be coated with a coating capable of absorbing visible light, so as to avoid the light of high-brightness objects (such as light sources, sun, high-reflectivity objects, etc.) located outside the field of view of the photosensitive pixel set S 1
  • high-brightness objects such as light sources, sun, high-reflectivity objects, etc.
  • the partition wall 40 When it enters the position close to the partition wall 40 through the microlens L 1 , it is reflected by the partition wall 40, thereby causing a glare image in the meta image output by the photosensitive pixel set S 1 , and can also avoid being located in the photosensitive pixel set S 2
  • the light of the bright object outside the field is reflected by the partition wall 40 when it enters the position close to the partition wall 40 through the microlens L 2 , thereby causing a glare image problem in the meta image output by the photosensitive pixel set S 2 .
  • the electronic device 1000 further includes a processor 300.
  • the processor 300 is installed in the housing 400.
  • the processor 300 is electrically connected to the imaging device 100.
  • the processor 300 may be used to control the exposure of a plurality of photosensitive pixels 20 to receive light passing through the microlens array structure 10, and receive electrical signals output by each photosensitive pixel 20 to form multiple A set of meta-images corresponding to one set of photosensitive pixels.
  • the processor 300 can also be used to fuse multiple meta images to obtain a merged image, calculate depth information of the scene based on at least two meta images, and perform predetermined processing on the merged image according to the depth information.
  • the processor 300 when the processor 300 is used to fuse multiple meta images to obtain a merged image, the processor 300 actually performs the following operations: Select two meta images, one meta image as a reference meta image, and another meta image as The meta image to be matched; divide the reference meta image into multiple block images, and select one block image from the multiple block images as the reference block image; find the matching block image matching the reference block image in the meta image to be matched to Form matching image pairs; cyclically perform the above division steps and the above-mentioned searching steps to traverse multiple block images in the reference meta image to obtain multiple matching image pairs; fuse the reference block image and matching block image in each matching image pair to obtain fusion Sub-images, and stitch multiple fusion sub-images to obtain stitched sub-images; use the stitched sub-images as new reference meta-images, select one meta-image from the remaining multiple meta-images as a new meta-image to be matched, and loop Perform the above steps of dividing the reference meta image into a plurality of block images to the step
  • the processor 300 first selects two meta images from the N meta images: such as meta image P1
  • the meta image P2 is combined with the meta image P1 as the reference meta image and the meta image P2 as the meta image to be matched.
  • the processor 300 divides the reference meta image P1 into a plurality of block images, such as 9 block images: block image P1-00, block image P1-01, block image P1-02, block image P1-10, block image P1 -11, block image P1-12, block image P1-20, block image P1-21, block image P1-22.
  • the processor 300 selects one block image from the nine block images as the reference block image, for example, selects the block image P1-00 as the reference block image. After determining the reference block image P1-00, the processor 300 looks for a matching block image matching the reference block image P1-00 in the meta image P2 to be matched.
  • the processor 300 looks for the regions P2 to 00 corresponding to the positions of the reference block image P1-00 in the meta image P2 to be matched, and performs a correlation calculation between the reference block image P1-00 and the regions P2 to 00 to determine Whether the area P2 ⁇ 00 is a matching block image matching the reference block image P1-00, if the correlation is greater than a predetermined correlation value, it is determined that the area P2 ⁇ 00 is a matching block image matching the reference block image P1-00, and the area Mark P2 ⁇ 00 for subsequent image fusion; on the contrary, in the meta image P2 to be matched, starting from the area P2 ⁇ 00, move to the reference block image P1 in the x direction and/or y direction according to a predetermined moving step -00 rectangular frame of the same size, each time the rectangular frame is moved, the area framed by the rectangular frame needs to be correlated with the reference block image P1-00 to determine whether the area framed by the rectangular frame is the same as the reference block For the matching block image matching the
  • the reference block image P1-00 cannot find a matching one in the meta image P2 to be matched
  • the image matching pair is also output at this time, but the image matching pair contains only the reference block image P1-00.
  • the processor 300 After searching for the matching block image of the reference block image P1-00, the processor 300 replaces the reference block image P1-00 with the block image P1-01, and finds the matching block matching the reference block image P1-01 in the manner described above image. As shown in FIG. 6, the matching block image that matches the reference block image P1-01 is the block image P12-01. At this time, the image matching pair is output, and the image matching pair includes the reference block image P1-01 and the matching block image P12-01. Subsequently, the processor 300 continues to replace the reference block image, and continues to perform the above-mentioned searching step. This loops back and forth until the matching block images of all the block images in the reference meta image P1 are determined, and multiple pairs of image matching pairs corresponding to the number of block images are output.
  • the processor 300 fuses the block images in the image matching pair including two block images. Specifically, the reference block image and the matching block image in each pair of image matching pairs are fused to obtain a fused sub-image. For an image matching pair of a block image, the reference block image in the image matching pair is the fused sub-image.
  • the processor 300 stitches multiple fused sub-images to obtain an initial stitched sub-image. Further, the processor 300 needs to cut out the unmatched regions of the meta-image P2 to be matched except for the regions that have matched with the reference meta-image P1, and join the initial spliced sub-image with this part of the unmatched regions, thereby The final spliced sub-image Pm is obtained, wherein the number of pixels of the spliced sub-image is greater than the number of pixels of the reference meta-image P1 and the meta-image to be matched P2.
  • the unmatched area is an image that cannot be matched with this part of the reference meta image P1, indicating that the image of the unmatched area is not available in the reference meta image P1. This is due to the set of photosensitive pixels outputting the reference meta image P1 and When the stitched sub-image Pm is formed by outputting the field of view difference between the photosensitive pixel sets of the meta image P2 to be matched, the image of the unmatched area is also stitched into the stitched sub-image Pm to ensure the integrity of the picture taken by the imaging device 100 Sex.
  • the processor 300 uses the stitching sub-image Pm as a new reference meta-image, and continues to select a meta-image from the remaining meta-images P3, P4, P5...PN as a new meta-image to be matched .
  • the processor 300 divides the reference meta image Pm into a plurality of block images according to the above-mentioned fusion splicing method of the reference meta image P1 and the meta image P2 to be matched, and looks for the reference meta image Pm The multiple block images of each match the matching block images, and then perform the above fusion and stitching process to fuse and merge the reference meta image Pm and the meta image P3 to be matched into a new splicing sub-image Pm. Subsequently, the processor 300 uses the new spliced sub-image Pm as a new reference meta-image, and continues to select a meta-image from the remaining meta-images P4, P5, P6...PN as a new meta-image to be matched. This cycle repeats until all the meta images are fused and stitched together, and finally a merged image is obtained, which has a higher resolution.
  • the processor 300 when it calculates the depth information of the scene based on at least two meta images, it specifically performs the following operations: dividing multiple meta images into a reference image set and a set of images to be matched, the reference image set includes multiple meta images Images, the set of images to be matched includes multiple meta images; one meta image is selected from the reference image set as the reference meta image, and one meta image is selected from the image set to be matched as the meta image to be matched; the reference meta image is divided For multiple block images, and select one block image from multiple block images as the reference block image; find matching block images matching the reference block image in the meta image to be matched to form matching image pairs; center the matching image according to the matching images The disparity of the reference block image and the matching block image is calculated as the depth information; the division step, the searching step and the calculation step are cyclically executed to traverse the plurality of block images in the reference meta image to obtain a plurality of depth information; The step of selecting a meta image as a reference meta image in the image set to the
  • the processor 300 divides the 16 meta images into two sets: a reference image set and an image to be matched set.
  • the reference image set includes a meta image P1, a meta image P2, a meta image P5, a meta image P6, a meta image P9, a meta image P10, a meta image P13, and a meta image P14;
  • the image set to be matched includes a meta image P3 and a meta image P4 , Meta image P7, meta image P8, meta image P11, meta image P12, meta image P15, meta image P16.
  • the processor 300 selects a meta image from the reference image set as the reference meta image, for example, selects the meta image P1 as the reference meta image, and selects a meta image from the image set to be matched as the meta image to be matched, if selected
  • the meta image P3 serves as a meta image to be matched.
  • the processor 300 divides the reference meta image P1 into a plurality of block images, such as 9 block images: block image P1-00, block image P1-01, block image P1-02, block image P1-10, block image P1 -11, block image P1-12, block image P1-20, block image P1-21, block image P1-22.
  • the processor 300 selects one block image from the nine block images as the reference block image, for example, selects the block image P1-00 as the reference block image. After determining the reference block image P1-00, the processor 300 looks for a matching block image matching the reference block image P1-00 in the meta image P3 to be matched.
  • the processor 300 looks for the regions P3 to 00 corresponding to the position of the reference block image P1-00 in the meta image P3 to be matched, and performs a correlation calculation between the reference block image P1-00 and the regions P3 to 00 to determine the region Whether P3 ⁇ 00 is a matching block image matching the reference block image P1-00, if the correlation is greater than a predetermined correlation value, it is determined that the area P3 ⁇ 00 is a matching block image matching the reference block image P1-00, and the area P3 ⁇ 00 to mark for subsequent calculation of depth information; otherwise, in the meta image P3 to be matched, starting from the area P3 ⁇ 00, move to the reference block image P1 in the x direction and/or y direction according to a predetermined moving step -00 rectangular frame of the same size, each time the rectangular frame is moved, the area framed by the rectangular frame needs to be correlated with the reference block image P1-00 to determine whether the area framed by the rectangular frame is the same as the reference block For the matching block image matching
  • the reference block image P1-00 cannot find a matching one in the meta image P3 to be matched
  • the image matching pair is also output at this time, but the image matching pair contains only the reference block image P1-00.
  • the processor 300 After searching for the matching block image of the reference block image P1-00, the processor 300 replaces the reference block image P1-00 with the block image P1-01, and finds the matching block matching the reference block image P1-01 in the manner described above image. As shown in FIG. 7, the matching block image that matches the reference block image P1-01 is the block image P13-01. At this time, the image matching pair is output, and the image matching pair includes the reference block image P1-01 and the matching block image P13-01. Subsequently, the processor 300 continues to replace the reference block image, and continues to perform the above-mentioned searching step. This loops back and forth until the matching block images of all the block images in the reference meta image P1 are determined, and multiple pairs of image matching pairs corresponding to the number of block images are output.
  • the processor 300 filters out the image matching pairs containing two meta images, and performs parallax calculation on the reference block image and the matching block image in each pair of image matching pairs to obtain at least one depth information d.
  • the processor 300 is based on the coordinate position of the reference block image in the reference meta image P1, the coordinate position of the matching block image in the meta image to be matched P3, the set of photosensitive pixels outputting the reference meta image P1 and the output of the meta image to be matched P3
  • the positional relationship between the set of photosensitive pixels is used for parallax calculation to obtain at least one depth information d. In this way, a plurality of depth information d can be obtained by performing parallax calculation on the reference block image and the matching block image in the multiple image matching pairs.
  • the processor 300 selects a meta image from the remaining meta images of the reference image set as the new reference meta image, for example, selects the meta image P2 as the new reference meta image, and selects the remaining meta images from the image set to be matched Select a meta image as the new meta image to be matched, for example, select meta image P4 as the new meta image to be matched.
  • the processor 300 processes the reference meta-image P2 and the meta-image to be matched P4 according to the calculation method of the depth information d described above to obtain a plurality of depth information d.
  • the processor 300 selects a meta image from the remaining meta images of the reference image set as the new reference meta image, for example, selects the meta image P5 as the new reference meta image, and selects the remaining meta images from the image set to be matched Select a meta image as the new meta image to be matched, for example, select meta image P7 as the new meta image to be matched. This is repeated until the processor 300 finishes calculating the depth information d of the reference meta image P14 and the meta image P16 to be matched. In this way, a plurality of depth information d can be obtained, and fusion of the plurality of depth information d can obtain a depth image of the scene, where the depth information d indicates the distance between each object in the scene and the imaging device 100.
  • the processor 300 when the processor 300 is used to perform predetermined processing on the merged image according to the depth information, the processor 300 specifically performs the following operations: determining the foreground area and the background area of the merged image according to the depth information; and making the background area virtual according to the depth information ⁇ Treatment.
  • the merged image and the depth image have a certain mapping relationship, and each pixel in the merged image can find the corresponding depth information in the depth image.
  • the processor 300 may segment the merged image into the foreground area and the background area according to the depth information. Specifically, in an example, the processor 300 may directly divide the merged image into the foreground area and the background area according to a preset depth, that is, merge pixels whose depth information is greater than the preset depth into the background area, and reduce the depth information to less than Or pixels equal to the preset depth are merged into the foreground area. Subsequently, the processor 300 does not process the foreground area, or appropriately sharpens the foreground area.
  • the processor 300 performs blur processing on the background area.
  • all pixels in the background area may have the same degree of blur.
  • the processor 300 may further divide the background area, and divide the background area into multiple sub-areas from near to far, along the direction from near to far. The degree of blurring increases in sequence, where multiple pixels in each sub-region have the same degree of blurring. In this way, blurring the background area to different degrees can improve the quality of the final combined image.
  • the processor 300 when the processor 300 is used to perform predetermined processing on the merged image according to the depth information, the processor 300 specifically performs the following operations: determining a region to be focused on the merged image according to user input; and removing the merged image according to the depth information Blur the area outside the focus area.
  • the user input may include: the user clicks on the touchable display screen 200 to preview the merged image corresponding to a certain position of the display screen 200, and the processor 300 uses the position point as the center point to expand outwards by a predetermined size and shape Area to get the area to be focused.
  • the processor 300 records multiple positions of the display screen 200 clicked when the user used the imaging device 100 multiple times before, and takes the position with the most clicks as the default position. When the user does not click the display screen 200, the processor 300 takes the default The position is the center point, and an area of a predetermined size and shape is expanded outward to obtain the area to be focused.
  • the merged image and the depth image have a certain mapping relationship, and each pixel in the merged image can find the corresponding depth information in the depth image.
  • the processor 300 may not process the focus area, or perform appropriate sharpening processing on the focus area.
  • the processor 300 performs a blurring process on the area other than the area to be focused (ie, the non-focused area). Specifically, the processor 300 can perform the same blurring process on all pixels in the non-focus area.
  • the processor 300 may further divide the unfocused area into multiple sub-areas from near to far based on the depth information, and the degree of blurring of the sub-areas increases in the near and far directions, where each sub-area Multiple pixels in have the same degree of blur. In this way, blurring the out-of-focus area to different degrees can improve the quality of the final combined image.
  • the imaging device 100 is provided with a microlens array structure 10 covering a plurality of photosensitive pixel sets, and each photosensitive pixel set can output a meta image, and the meta image is processed After the processor 300 is fused, a high-resolution merged image can be obtained.
  • the existing array camera requires the use of multiple traditional cameras, making the overall size of the array camera larger, which is not conducive to integration in electronic devices with high thickness requirements, and the cost of the array camera is also Higher.
  • the imaging device 100 according to the embodiment of the present application can capture a combined image with a higher resolution without providing a plurality of conventional cameras.
  • the overall size of the imaging device 100 is small, which is beneficial for integration in an electronic device 1000 that requires a high thickness on.
  • the cost of the imaging device 100 is also low, and the manufacturing cost of the electronic device 1000 can be further reduced.
  • the microlens array structure 10 includes a first lens array 12 and a second lens array 14.
  • the first lens array 12 includes a plurality of first bosses 126 arranged in a first direction (such as the X-axis direction in FIG. 8). Each first boss 126 has a semi-cylindrical shape (or other suitable shape) to achieve an optical focusing effect.
  • the first lens array 12 includes opposite first flat surfaces 122 and first convex surfaces 124 (similar to wavy surfaces).
  • a plurality of first bosses 126 jointly form a first convex surface 124.
  • each first boss 126 includes a first sub-convex surface, and multiple first sub-convex surfaces of the plurality of first bosses 126 are connected to form a first convex surface 124.
  • the plurality of first bosses 126 together form the first flat surface 122.
  • each first boss 126 includes a first sub-flat surface, and a plurality of first sub-flat surfaces of the plurality of first bosses 126 are connected to form a first flat surface 122.
  • the second lens array 14 includes a plurality of second bosses 146 arranged in a second direction (as shown in the Y-axis direction in FIG. 8). Each second boss 146 has a semi-cylindrical shape (or other suitable shape) to achieve an optical focusing effect.
  • the second lens array 14 includes opposite second flat surfaces 142 and second convex surfaces 144 (similar to wavy surfaces). A plurality of second bosses 146 jointly form a second convex surface 144.
  • each second boss 146 includes a second sub-convex surface, and a plurality of second sub-convex surfaces of the plurality of second bosses 146 are connected to form a second convex surface 144.
  • the second flat surfaces 142 are common to the plurality of second bosses 146.
  • each second boss 146 includes a second sub-flat surface, and a plurality of second sub-flat surfaces of the plurality of second bosses 146 are connected to form a second flat surface 142.
  • the plurality of first bosses 126 and the plurality of second bosses 146 overlap in a third direction (such as the Z-axis direction in FIG. 8) to form a microlens array.
  • the first direction intersects the second direction, and the third direction is perpendicular to the first direction and the second direction.
  • the number of the first boss 126 and the second boss 146 are both 20, that is, 20 first bosses 126 are arranged along the X-axis direction to form 12, 20 first lens arrays of 1*20
  • the second bosses 146 are arranged along the Y-axis direction to form a 20*1 second lens array 14, and 20 first bosses 126 and 20 second bosses 146 overlap in the Z-axis direction to form a 20*20 microlens array .
  • the first direction intersects the second direction
  • the third direction is perpendicular to the first direction and the second direction.
  • intersection of the first direction and the second direction means that the first direction and the second direction are neither coincident nor parallel to each other, and their projections in the three-dimensional space intersect, specifically forming 30 degrees, 45 degrees, 60 degrees, 75 degrees,
  • An equal angle of 90 degrees, for example, the angle in FIG. 8 is 90 degrees.
  • the first lens array 12 including a plurality of first bosses 126 and the first lens array 12 including a plurality of second bosses 146 may be formed separately first.
  • Two lens array 14, and then the first lens array 12 and the second lens array 14 are combined so that the plurality of first bosses 126 and the plurality of second bosses 146 overlap in the third direction to form a microlens array, a microlens
  • the array includes a plurality of microlenses 101, as shown by dotted lines in FIG.
  • each microlens 101 is formed by overlapping a first boss 126 in the Y direction with a second boss 146 in the X direction, Each microlens 101 is composed of a portion of the first boss 126 overlapping the second boss 146 and a portion of the second boss 146 overlapping the first boss 126.
  • a plurality of first bosses 126 arranged in a first direction can be formed on the entire lens structure of a cube (cuboid or cuboid) by nanoimprinting technology as the first lens array 12, and the plurality of first bosses 126 are connected in sequence without gaps between each other, so that a microlens array can be formed more closely arranged, so that the imaging device 100 can collect more image information.
  • the second lens array 14 can be formed in the same manner, and the description is not repeated here.
  • the first lens array 12 and the second lens array 14 are combined so that the plurality of first bosses 126 and the plurality of second bosses 146 overlap in the third direction to form a microlens array.
  • the first lens array 12 may be placed On the second lens array 14; or the second lens array 14 is placed on the first lens array 12, and a plurality of first bosses 126 and a plurality of second bosses 146 are staggered (e.g., crisscross arrangement) , And conflict with each other in the third direction.
  • the materials used to form the first lens array 12 and the second lens array 14 may be glass, plastic, or other materials.
  • Each microlens 101 in the microlens array covers a plurality of photosensitive pixels 20 in a set of photosensitive pixels, for example, each microlens 101 covers 70*70 photosensitive pixels 20.
  • the microlens array is aligned with the photosensitive pixel array formed by the plurality of photosensitive pixels 20. Specifically, the optical axis of each microlens 101 may coincide with the normal of the photosensitive pixel array formed by the plurality of photosensitive pixels 20.
  • microlens array structure 10 of the embodiment of the present application a plurality of first bosses 126 and a plurality of second bosses 146 overlap to form a microlens array, the manufacturing process is simple, and the manufactured microlens array has good uniformity (ie Neatly arranged, consistent structure), lower cost. Because there is no photolithography heating and reflow method and mold injection or grinding processing, there is no problem that the size of the microlens cannot be enlarged, or when the size of the microlens is made larger, the diopter is limited and the consistency is poor Problem, or very difficult to process precision molds.
  • the plurality of first bosses 126 are integrally formed, and the plurality of second bosses 146 are integrally formed, the plurality of first bosses 126 do not need to be fixed together by means of glue or the like, nor do the plurality of second bosses 146 Fix it together with glue or other means.
  • the first lens array 12 and the second lens array 14 are assembled together, there will be no mutual offset between the plurality of first bosses 126 and no mutual offset between the multiple second bosses 146 , Higher assembly stability.
  • the first boss 126 and the second boss 146 can also be fixed without using glue (such as optical glue), etc., only The first boss 126 and the second boss 146 are in conflict, and the first lens array 12 and the second lens array 14 are respectively fixed by the lens barrel or other components of the imaging device 100.
  • the installation method of the entire imaging device 100 is relatively simple. When one of the lens arrays, such as the first lens array 12 or the second lens array 14, is damaged, only the first lens array 12 or the second lens array 14 can be removed for replacement.
  • each microlens 101 in the microlens array is the same, and the imaging device 100 can be used as a fixed focus lens (not involving focusing).
  • the subject object is within the effective focal length range of the imaging device 100, it can be clearly imaged, which is more suitable As the front lens of the electronic device 100.
  • the first flat surface 122 and the second flat surface 142 are combined.
  • the bonding area of the first flat surface 122 and the second flat surface 142 can be completely bonded together, the bonding is tight, the stability is higher, and no moisture or impurities will enter the first lens array 12 and the second Between the lens arrays 14, it is beneficial to ensure the service life of the microlens array structure 10 and the good imaging quality of the imaging device 100.
  • the first convex surface 124 is combined with the second flat surface 142.
  • the microlens array structure 10 is assembled with other structures of the imaging device 100 (such as a lens barrel, photosensitive pixels 20, etc.), the first flat surface 122 can be well mounted on other structures.
  • the first flat surface 122 is combined with the second convex surface 144 (combined with the first convex surface 124 and the first (The combination of the two flat surfaces 142 is similar).
  • the microlens array structure 10 is assembled with other structures of the imaging device head 100 (such as lens barrels, filters, etc.), the second flat surface 142 can be well mounted on other structures.
  • the first convex surface 124 and the second convex surface 144 are combined.
  • the microlens array structure 10 is assembled with other structures of the imaging device 100 (such as a lens barrel, photosensitive pixels 20, filters, etc.)
  • the first flat surface 122 and the second flat surface 142 can be well mounted on other Structurally, and since the first convex surface 124 and the second convex surface 144 are not exposed to the outside (the side combined with other structures), the first convex surface 124 and the second convex surface 144 will not be affected by The wear of other structures affects the optical focusing effect.
  • the first flat surface 122 is combined with the second flat surface 142
  • the first flat surface 122 is combined with the second convex surface 144
  • the first convex surface 124 is combined with the second flat surface 142
  • the first protrusion The combination of the surface 124 and the second convex surface 144 can achieve different focal lengths and angles of view, respectively.
  • the first flat surface 122 and the second flat surface 142 may be both planar structures.
  • the photosensitive pixel array formed by the plurality of photosensitive pixels 20 is also planar
  • the structure facilitates the alignment of the microlens array and the photosensitive pixel array, ensures good imaging quality of the imaging device 100, and facilitates the assembly of the imaging device 100.
  • the planar structure means that the line connecting any two points on the plane falls on this plane.
  • the first flat surface 122 and the second flat surface 142 may be curved structures.
  • the photosensitive pixel array formed by the plurality of photosensitive pixels 20 is also a curved structure, so as to achieve micro
  • the alignment of the lens array and the photosensitive pixel array ensures a good imaging quality of the imaging device 100, and the imaging device 100 has a larger field angle, can collect more light, and has a smaller volume compared to the plane combination.
  • the imaging device 100 further includes a filter (such as an infrared filter for passing infrared light or an infrared cut filter for filtering infrared light, etc.), the filter may also have a curved structure To better filter the light.
  • the degree of curvature of the first flat surface 122 is the same as the degree of curvature of the second flat surface 142.
  • the degree of curvature of the first flat surface 122 is the same as the degree of curvature of the second convex surface 144.
  • the degree of curvature of the first convex surface 124 is the same as the degree of curvature of the second flat surface 142 (as shown in FIG. 15).
  • the degree of curvature of the first convex surface 124 is the same as that of the second convex surface 144. It should be pointed out that the degree of curvature of the convex surface can be regarded as the degree of curvature of the circumscribed arc of the convex surface, or that each boss has a point with the farthest distance from the corresponding flat surface.
  • the degree of curvature is the degree of curvature of an arc formed by a plurality of points of a plurality of bosses.
  • the first flat surface 122, the first convex surface 124, the second flat surface 142, and the second convex surface 144 may be any of aspherical surfaces, spherical surfaces, Fresnel surfaces, or binary optical surfaces .
  • the first flat surface 122, the first convex surface 124, the second flat surface 142, and the second convex surface 144 are all aspherical surfaces; or, the first flat surface 122 and the first convex surface 124 are aspherical surfaces,
  • the second flat surface 142 is a spherical surface, and the second convex surface 144 is a Fresnel surface; or, the first flat surface 122 is an aspheric surface, the first convex surface 124 is a spherical surface, and the second flat surface 142 is a Fresnel surface ,
  • the second convex surface 144 is a binary optical surface and the like.
  • the lens is lighter, thinner, and flatter, and it can still maintain excellent impact resistance; when the surface type is spherical, The manufacturing process of the microlens array structure 10 is relatively simple; when the Fresnel surface is used for the face shape, the imaging device 100 has brighter images and uniform brightness, and the problem of darkening and blurring of the corners is less likely to occur; when the binary shape is used for the face shape Light weight, low cost, and can achieve new functions such as tiny, array, integration, which are difficult to complete with traditional optics.
  • the plurality of first bosses 126 and the plurality of second The microlens array formed by the boss 146 is a square microlens array.
  • the width of each first boss 126 is greater than the width of each second boss 146 or the width of each second boss 146 is greater than the width of the first boss
  • the microlens array formed by the plurality of first bosses 126 and the plurality of second bosses 146 is a rectangular microlens array. Please refer to FIG.
  • the microlens array formed by the plurality of first bosses 126 and the plurality of second bosses 146 is a parallelogram Lens array.
  • the microlens array of this embodiment may be a square microlens array, a rectangular microlens array, or a parallelogram microlens array, so as to be suitable for the imaging device 100 with various shapes, structures, or functions.
  • the present application also provides a method for manufacturing the microlens array structure 10.
  • the manufacturing method of the microlens array structure 10 of the embodiment of the present application includes:
  • a first lens array 12 is formed.
  • the first lens array 12 includes a plurality of first bosses 126, and the plurality of first bosses 126 are arranged along the first direction;
  • the second lens array 14 includes a plurality of second bosses 146, and the plurality of second bosses 146 are arranged along the second direction;
  • first lens array 12 and the second lens array 14 Combining the first lens array 12 and the second lens array 14 so that the plurality of first bosses 126 and the plurality of second bosses 146 overlap in a third direction to form a microlens array, the first direction and the second direction intersect,
  • the third direction is perpendicular to the first direction and the second direction.
  • microlens array structure 10 are all applicable to the manufacturing method of the microlens array structure 10 of this embodiment, and will not be described here.
  • the electronic device 1000 includes the imaging device 100 and the housing 400 described in any of the above embodiments.
  • the imaging device 100 is mounted on the housing 400.
  • the imaging device 100 further includes a processor 50.
  • the processor 50 is provided in the imaging device 100.
  • the processor 50 may be used to control the exposure of the plurality of photosensitive pixels 20 to receive the light passing through the microlens array structure 10 and receive the electrical signal output from each photosensitive pixel 20 to form a one-to-one corresponding meta image of the plurality of photosensitive pixels.
  • the processor 50 may also be used to fuse multiple meta images to obtain a merged image, calculate depth information of the scene based on at least two meta images, and perform predetermined processing on the merged image according to the depth information.
  • the process of the processor 50 performing the above steps is consistent with the process of the processor 300 performing the above steps, and no further description will be given here.
  • first and second are used for description purposes only, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.
  • the features defined with “first” and “second” may include at least one of the features either explicitly or implicitly.
  • the meaning of “plurality” is at least two, such as two, three, etc., unless otherwise specifically limited.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Studio Devices (AREA)

Abstract

一种成像装置(100)及电子设备(1000)。成像装置(100)包括微透镜阵列结构(10)和多个感光像素(20)，微透镜阵列结构(10)覆盖多个感光像素集合。每个感光像素集合包括多个感光像素(20)，每个感光像素集合中的多个感光像素(20)接收穿过微透镜阵列结构(10)入射的光线以输出多张元图像。成像装置(100)还包括处理器(50)，处理器(50)用于：融合多张元图像得到合并图像；根据至少两张元图像计算场景的深度信息；根据深度信息对合并图像做预定处理。

Description

成像装置和电子设备
优先权信息
本申请请求2018年11月26日向中国国家知识产权局提交的、专利申请号为201811417730.6的专利申请的优先权和权益，并且通过参照将其全文并入此处。
技术领域
本申请涉及成像技术领域，特别涉及一种成像装置和电子设备。
背景技术
现有的阵列相机一般是使用多个传统摄像头在空间上以一定规则排布，成像时多个传统摄像头均拍摄一张图像，最终通过对多张图像的拼接和融合实现高分辨率图像的获取。
发明内容
本申请的实施例提供了一种成像装置和电子设备。
本申请实施方式的成像装置包括微透镜阵列结构和多个感光像素，所述微透镜阵列结构覆盖多个感光像素集合，每个所述感光像素集合包括多个所述感光像素，每个所述感光像素集合中的多个感光像素接收穿过所述微透镜阵列结构入射的光线以输出多张元图像；所述成像装置还包括处理器，所述处理器用于：融合多张所述元图像得到合并图像；根据至少两张所述元图像计算场景的深度信息；根据所述深度信息对所述合并图像做预定处理。
本申请实施方式的电子设备包括壳体和成像装置。所述成像装置安装在所述壳体上。成像装置包括微透镜阵列结构和多个感光像素，所述微透镜阵列结构覆盖多个感光像素集合，每个所述感光像素集合包括多个所述感光像素，每个所述感光像素集合中的多个感光像素接收穿过所述微透镜阵列结构入射的光线以输出多张元图像；所述成像装置还包括处理器，所述处理器用于：融合多张所述元图像得到合并图像；根据至少两张所述元图像计算场景的深度信息；根据所述深度信息对所述合并图像做预定处理。
本申请实施方式的电子设备包括壳体、成像装置和处理器。所述成像装置包括微透镜阵列结构和多个感光像素，所述微透镜阵列结构覆盖多个感光像素集合，每个所述感光像素集合包括多个所述感光像素，每个所述感光像素集合中的多个感光像素接收穿过所述微透镜阵列结构入射的光线以输出多张元图像。所述处理器用于：融合多张所述元图像得到合并图像、根据至少两张所述元图像计算场景的深度信息、及根据所述深度信息对所述合并图像做预定处理。
本申请的附加方面和优点将在下面的描述中部分给出，部分将从下面的描述中变得明显，或通过本申请的实践了解到。
附图说明
本申请上述的和/或附加的方面和优点从下面结合附图对实施例的描述中将变得明显和容易理解，其中：
图1和图2是本申请某些实施方式的电子设备的立体结构示意图。
图3是本申请某些实施方式的成像装置的部分立体结构示意图。
图4是图3中的成像装置沿IV-IV线的部分截面示意图。
图5是本申请某些实施方式的成像装置的部分截面示意图。
图6是本申请某些实施方式的元图像拼接的场景示意图。
图7是本申请某些实施方式的深度信息计算的场景示意图。
图8是本申请某些实施方式的微透镜阵列结构的立体组装示意图。
图9是图8中的微透镜阵列结构沿IX-IX线的截面示意图。
图10是本申请某些实施方式的微透镜阵列结构的分解示意图。
图11是本申请某些实施方式的微透镜阵列结构的立体组装示意图。
图12是图11中的微透镜阵列结构沿XII-XII线的截面示意图。
图13是本申请某些实施方式的微透镜阵列结构的立体组装示意图。
图14是图13中的微透镜阵列结构沿XIV-XIV线的截面示意图。
图15是本申请某些实施方式的微透镜阵列结构沿与图8中IX-IX线对应位置截得的截面示意图。
图16是本申请某些实施方式的微透镜阵列结构的立体组装示意图。
图17是本申请某些实施方式的微透镜阵列结构的立体组装示意图。
图18是本申请某些实施方式的电子设备的模块示意图。
具体实施方式
下面详细描述本申请的实施例，所述实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的，旨在用于解释本申请，而不能理解为对本申请的限制。
在本申请中，除非另有明确的规定和限定，第一特征在第二特征“上”或“下”可以是第一和第二特征直接接触，或第一和第二特征通过中间媒介间接接触。而且，第一特征在第二特征“之上”、“上方”和“上面”可是第一特征在第二特征正上方或斜上方，或仅仅表示第一特征水平高度高于第二特征。第一特征在第二特征“之下”、“下方”和“下面”可以是第一特征在第二特征正下方或斜下方，或仅仅表示第一特征水平高度小于第二特征。
请参阅图3和图18，本申请提供一种成像装置100。成像装置100包括微透镜阵列结构10和多个感光像素20。微透镜阵列结构10覆盖多个感光像素集合，每个感光像素集合包括多个感光像素20，每个感光像素集合中的多个感光像素20接收穿过微透镜阵列结构10入射的光线以输出多张元图像。成像装置100还包括处理器50，处理器50用于：融合多张元图像得到合并图像；根据至少两张元图像计算场景的深度信息；根据深度信息对合并图像做预定处理。
请参阅图18，在某些实施方式中，处理器50还用于：选取两张元图像，一张元图像作为基准元图像，另一张元图像作为待匹配元图像；将基准元图像划分为多张块图像，并从多张块图像中选取一张块图像作为基准块图像；在待匹配元图像中寻找与基准块图像匹配的匹配块图像以形成匹配图像对；循环执行上述划分步骤及上述寻找步骤以遍历基准元图像中的多张块图像，得到多个匹配图像对；融合每个匹配图像对中的基准块图像和匹配块图像得到融合子图像，并拼接多张融合子图像以获取拼接子图像；将拼接子图像作为新的基准元图像，从剩余的多张元图像中选取一张元图像作为新的待匹配元图像，并循环执行上述将基准元图像划分为多张块图像的步骤至获取拼接子图像的步骤以融合得到合并图像，其中，循环执行将基准元图像划分为多张块图像的步骤时是以前一次的拼接子图像作为基准元图像。
请参阅图18，在某些实施方式中，处理器50还用于：将多张元图像划分为基准图像集合和待匹配图像集合，基准图像集合中包括多张元图像，待匹配图像集合中包括多张元图像；从基准图像集合中选取一张元图像作为基准元图像，从待匹配图像集合中选取一张元图像作为待匹配元图像；将基准元图像划分为多张块图像，并从多张块图像中选取一张块图像作为基准块图像；在待匹配元图像中寻找与基准块图像匹配的匹配块图像以形成匹配图像对；根据匹配图像对中的基准块图像和匹配块图像的视差计算深度信息；循环执行划分步骤、寻找步骤及计算步骤以遍历基准元图像中的多张块图像，以获取多个深度信息；循环执行上述从基准图像集合中选取一张元图像作为基准元图像的步骤至获取多个深度信息的步骤以遍历基准图像集合中的多张元图像，得到多个深度信息。
请参阅图18，在某些实施方式中，处理器50还用于：根据深度信息确定合并图像的前景区域和背景区域；根据深度信息对背景区域做虚化处理。
请参阅图18，在某些实施方式中，处理器50还用于：根据用户输入确定合并图像的待对焦区域；根据深度信息对合并图像的除待对焦区域外的区域做虚化处理。
请参阅图3和图4，在某些实施方式中，相邻两个感光像素集合之间设置有间隔壁40。
请参阅图3至图5，在某些实施方式中，微透镜阵列结构10包括多个微透镜101，每个微透镜101覆盖一个感光像素集合。成像装置100还包括衬底30，多个感光像素20形成在衬底30上，衬底30的与感光像素20接触的面为曲面结构。
请参阅图5，在某些实施方式中，至少两个微透101具有不同的曲率半径。
请参阅图8和图9，在某些实施方式中，微透镜阵列结构10包括第一透镜阵列12和第二透镜阵列14。第一透镜阵列包括多个第一凸台126，多个第一凸台126沿第一方向排列。第二透镜阵列14包括多 个第二凸台146，多个第二凸台146沿第二方向排列。第一凸台126与第二凸台146在第三方向上交叠形成微透镜阵列结构10。第一方向与第二方向相交，第三方向垂直于第一方向和第二方向。
请参阅图8和图9，在某些实施方式中，第一透镜阵列12包括相背的第一平坦面122和第一凸起面124，多个第一凸台126形成第一凸起面124。第二透镜阵列14包括相背的第二平坦面142和第二凸起面144，多个第二凸台146形成第二凸起面144。第一平坦面142和第二平坦面144均为曲面结构。
请参阅图3和图18，本申请提供一种电子设备1000。电子设备1000包括壳体400和成像装置100。成像装置100包括微透镜阵列结构10和多个感光像素20。微透镜阵列结构10覆盖多个感光像素集合，每个感光像素集合包括多个感光像素20，每个感光像素集合中的多个感光像素20接收穿过微透镜阵列结构10入射的光线以输出多张元图像。成像装置100还包括处理器50，处理器50用于：融合多张元图像得到合并图像；根据至少两张元图像计算场景的深度信息；根据深度信息对合并图像做预定处理。
请参阅图1和图3，本申请提供一种电子设备1000。电子设备1000包括壳体400、成像装置100和处理器300。成像装置100包括微透镜阵列结构10和多个感光像素20。微透镜阵列结构10覆盖多个感光像素集合，每个感光像素集合包括多个感光像素20，每个感光像素集合中的多个感光像素20接收穿过微透镜阵列结构10入射的光线以输出多张元图像。处理器300用于：融合多张元图像得到合并图像；根据至少两张元图像计算场景的深度信息；根据深度信息对合并图像做预定处理。
请参阅图1，本申请提供一种电子设备1000。电子设备1000包括壳体400、成像装置100和处理器300。成像装置100设置在壳体400上。其中，电子设备1000可以是手机、平板电脑、游戏机、智能手表、智能手环、头显设备、无人机等。本申请实施方式以电子设备1000为手机为例进行说明，可以理解，电子设备1000的具体形式不限于手机。
壳体400可以作为电子设备1000的功能元件的安装载体。壳体400可以为功能元件提供防尘、防摔、防水等保护。功能元件可以是显示屏200、接近传感器、受话器等。在本申请的实施例中，壳体400包括主体401及可动支架402，可动支架402在驱动装置的驱动下可以相对于主体401运动，例如，可动支架402可以相对于主体401滑动，以滑入主体401(如图2所示)或从主体401滑出(如图1所示)。部分功能元件(例如显示屏200)可以安装在主体401上，另一部分功能元件(如成像装置100、受话器、接近传感器等)可以安装在可动支架402上。可动支架402运动可带动该另一部分功能元件缩回主体401内或从主体401伸出。当然，图1和图2仅仅是对壳体400的一种具体形式的举例，不能理解为对本申请的壳体400的限制。
成像装置100安装在壳体400上时，壳体400上可以开设有采集窗口，成像装置100与采集窗口对准安装以接收经采集窗口入射的光线。在申请的具体实施例中，成像装置100安装在可动支架402上。用户在需要使用成像装置100时，可以触发可动支架402从主体401中滑出以带动成像装置100从主体401中伸出；在用户不需要使用成像装置100时，可以触发可动支架402滑入主体401以带动成像装置100缩回主体401中。
请参阅图3，成像装置100包括依次设置的衬底30、多个感光像素20、微透镜阵列结构10及间隔壁40。衬底30包括相背的顶面31和底面32，多个感光像素20形成在衬底30的顶面31上。微透镜阵列结构10安装在感光像素20的收光面所在一侧。多个感光像素20可以接收穿过微透镜阵列结构10后入射的光线。
在一个例子中，如图4所示，衬底30的顶面31和底面32均为平面结构，多个感光像素20分布在平面结构的顶面31上，呈平面型排列，微透镜阵列结构10设置在感光像素20的收光面所在一侧，也呈平面型排列。顶面31为平面结构时，衬底30的厚度较小，进一步地可以减小成像装置100的高度，有利于成像装置100集成到对厚度有较高要求的电子设备1000，如手机、平板电脑等中。在另一个例子中，如图5所示，衬底30的底面32为平面结构，顶面31为曲面结构，多个感光像素20分布在曲面结构的顶面31上，呈曲面型排列，微透镜阵列结构10设置在感光像素20的收光面所在一侧，也呈曲面型排列。多个感光像素20和微透镜阵列结构10呈曲面型排列可以增大成像装置100的视场角，成像装置100可以拍摄到场景中的更多物体。
形成在衬底30的顶面31上的所有感光像素20划分为多个感光像素集合，每个感光像素集合中包括多个感光像素20。在一个例子中，每个感光像素集合可以包括70×70个感光像素20。当然，70×70的数量仅为示例，在其他例子中，每个感光像素集合中还可以包括60×60、80×80、100×100个感光像素 20等等，在此不做限制。每个感光像素集合中的感光像素20的数量越多，对应形成的元图像的分辨率越高。多个感光像素集合可以呈横向排列、纵向排列、“田”字形排列等。
微透镜阵列结构10覆盖多个感光像素集合。每个感光像素集合中的多个感光像素20可以接收穿过微透镜阵列结构10后入射的光线以输出对应该感光像素集合的一张元图像。如此，多个感光像素集合可以输出多张元图像。
在一个例子中，微透镜阵列结构10包括多个微透镜101，每个微透镜101覆盖一个感光像素集合。具体地，多个微透镜101为彼此独立的结构，每个微透镜101安装在靠近对应的感光像素集合中的多个感光像素20的收光面的一侧。场景中的光线穿过微透镜101后入射到对应的感光像素集合中的多个感光像素20上，以使多个感光像素20接收光线并对应输出多个电信号，同一感光像素集合中的多个感光像素20输出的多个电信号形成一张元图像。
其中，如图4所示，多个微透镜101可以具有相同的曲率半径，此时，多个微透镜101具有相同的焦距。成像装置100成像时，多个感光像素集合中的多个感光像素20对于场景中能够清晰成像的物体的距离是一致的。也即是说，假设微透镜101个数的数量为N，分别为L 1、L 2、L 3…L N，对应的感光像素集合的数量也为N，分别为感光像素集合S 1、S 2、S 3…S N。若感光像素集合S 1能够对场景中的与成像装置100距离D(D可以是一个具体数值或者是一个数值范围)的物体清晰成像，则感光像素集合S 2、S 3…S N也能够对场景中的与成像装置100距离D的物体清晰成像。实际使用中，可以选择曲率半径较小的微透镜101以使得微透镜101的焦距较小，以获得较大的景深，从而可以对场景中的大多数物体清晰成像。
或者，如图5所示，多个微透镜101可以具有不同的曲率半径，其中，至少两个微透镜101具有不同的曲率半径。具体地，例如，微透镜L 1至微透镜L N-1具有相同的曲率半径，微透镜L N的曲率半径与微透镜L 1的曲率半径不同；或者，微透镜L 1至微透镜L N-2具有相同的曲率半径，微透镜L N-1的曲率半径与微透镜L 1的曲率半径不同，微透镜L N的曲率半径与微透镜L 1及微透镜L N-1的曲率半径均不同；或者，任意两个微透镜101的曲率半径均不同等等。实际使用中，位于衬底30的中心位置处的微透镜101具有较小的曲率半径，位于衬底30的周缘位置处的微透镜101具有较大的曲率半径。如此，位于衬底30中心位置处的微透镜101的焦距较小，可以对与成像装置100距离较近的物体清晰成像，位于周缘位置处的微透镜101的焦距较大，可以对与成像装置100距离较远的物体清晰成像。可以理解，成像装置100成像时，主体一般位于成像装置100的视场中心，且与成像装置100的距离较近，而背景物体一般位于视场的周缘位置，且与成像装置100的距离较远，位于衬底30中心位置处的焦距较小的微透镜101可以对主体401清晰成像，位于衬底30周缘位置处的焦距较大的微透镜101可以对背景物体清晰成像，如此，可以提升每张元图像的清晰度。
任意两个感光像素集合中的感光像素20的邻接处均设置有间隔壁40。例如，如图4所示，微透镜L 1覆盖感光像素集合S 1，微透镜L 2覆盖感光像素集合S 2，感光像素集合S 1的感光像素20与感光像素集合S 2的感光像素20的邻接处设置有间隔壁40。间隔壁40可以防止穿过微透镜L 1的光线入射到感光像素集合S 2中，同时防止穿过微透镜L 2的光线入射到感光像素集合S 1中，如此，可以避免穿过微透镜L 1的光线入射到感光像素集合S 2后使得感光像素集合S 2输出的元图像中产生类似炫光像的图像的问题，也可以避免穿过微透镜L 2的光线入射到感光像素集合S 1后使得感光像素集合S 1输出的元图像中产生类似炫光像的图像的问题。进一步地，可以在间隔壁40的表面涂覆有能够吸收可见光的涂层，以此避免位于感光像素集合S 1视场外的高亮物体(如光源、太阳、高反射率物体等)的光线穿过微透镜L 1入射到接近间隔壁40的位置时被间隔壁40反射，从而导致感光像素集合S 1输出的元图像中产生炫光像的问题，也可避免位于感光像素集合S 2视场外的高亮物体的光线穿过微透镜L 2入射到接近间隔壁40的位置时被间隔壁40反射，从而导致感光像素集合S 2输出的元图像中产生炫光像的问题。
请参阅图1和图3，电子设备1000还包括处理器300。处理器300安装在壳体400内。处理器300与成像装置100电连接，处理器300可以用于控制多个感光像素20曝光以接收穿过微透镜阵列结构10的光线，并接收每个感光像素20输出的电信号以形成与多个感光像素集合一一对应的元图像。处理器300还可以用于融合多张元图像得到合并图像、根据至少两张元图像计算场景的深度信息、以及根据深度信息对合并图像做预定处理。
在一个例子中，处理器300用于融合多张元图像得到合并图像时，处理器300实际上执行以下操作： 选取两张元图像，一张元图像作为基准元图像，另一张元图像作为待匹配元图像；将基准元图像划分为多张块图像，并从多张块图像中选取一张块图像作为基准块图像；在待匹配元图像中寻找与基准块图像匹配的匹配块图像以形成匹配图像对；循环执行上述划分步骤及上述寻找步骤以遍历基准元图像中的多张块图像，得到多个匹配图像对；融合每个匹配图像对中的基准块图像和匹配块图像得到融合子图像，并拼接多张融合子图像以获取拼接子图像；将拼接子图像作为新的基准元图像，从剩余的多张元图像中选取一张元图像作为新的待匹配元图像，并循环执行上述将基准元图像划分为多张块图像的步骤至获取拼接子图像的步骤以融合得到合并图像。其中，循环执行将基准元图像划分为多张块图像的步骤时是以前一次的拼接子图像作为基准元图像。
具体地，如图6所示，假设有N张元图像，分别为元图像P1、P2、P3…PN，则处理器300首先从N张元图像中选出两张元图像：如元图像P1和元图像P2，并将元图像P1作为基准元图像，元图像P2作为待匹配元图像。随后，处理器300将基准元图像P1划分为多张块图像，如9张块图像：块图像P1-00、块图像P1-01、块图像P1-02、块图像P1-10、块图像P1-11、块图像P1-12、块图像P1-20、块图像P1-21、块图像P1-22。随后，处理器300从9张块图像中选取一张块图像作为基准块图像，例如，选取块图像P1-00作为基准块图像。在确定基准块图像P1-00之后，处理器300在待匹配元图像P2中寻找与基准块图像P1-00匹配的匹配块图像。具体地，处理器300在待匹配元图像P2中寻找与基准块图像P1-00的位置对应的区域P2～00，并对基准块图像P1-00与区域P2～00做相关性计算，以判断区域P2～00是否为与基准块图像P1-00匹配的匹配块图像，如果相关性大于预定相关值，则确定区域P2～00为与基准块图像P1-00匹配的匹配块图像，并对区域P2～00做标记以便后续做图像融合；反之，则在待匹配元图像P2中，以区域P2～00为起始点，向x方向和/或y方向按照预定移动步距移动与基准块图像P1-00的大小相同的矩形框，每移动一次矩形框，矩形框所框出来的区域均需要与基准块图像P1-00做相关性计算，以判断矩形框所框出来的区域是否为与基准块图像P1-00匹配的匹配块图像，如果矩形框所框出来的区域为与基准块图像P1-00匹配的匹配块图像，则对矩形框框出来的区域做标记，否则，矩形框继续移动，直至遍历整张元图像P2为止。如图6所示，由于输出基准元图像P1的感光像素集合与输出待匹配元图像P2的感光像素集合的视场差异，基准块图像P1-00在待匹配元图像P2中无法找到与其匹配的匹配块图像，此时同样输出图像匹配对，但图像匹配对中仅包含基准块图像P1-00。
在寻找完基准块图像P1-00的匹配块图像后，处理器300将基准块图像P1-00更换为块图像P1-01，并按照上述的方式寻找与基准块图像P1-01匹配的匹配块图像。如图6所示，与基准块图像P1-01匹配的匹配块图像为块图像P12-01，此时输出图像匹配对，图像匹配对包括基准块图像P1-01和匹配块图像P12-01。随后，处理器300继续更换基准块图像，并继续执行上述寻找步骤。如此循环往复，直至确定出基准元图像P1中的所有块图像的匹配块图像，并输出与块图像的数量一致的多对图像匹配对。如图6所示，元图像P1中每张块图像对应的图像匹配对分别为：“P1-00”、“P1-01＝P12-01”、“P1-02＝P12-02”、“P1-10”、“P1-11＝P12-11”、“P1-12＝P12-12”、“P1-20”、“P1-21＝P12-21”、“P1-22＝P12-22”。随后，处理器300对包括两张块图像的图像匹配对中的块图像做融合，具体地，对每对图像匹配对中的基准块图像和匹配块图像进行融合得到融合子图像，对于仅包括一张块图像的图像匹配对而言，该图像匹配对中的基准块图像即为融合子图像。如此，即可得到与多对图像匹配对一一对应的多张融合子图像。随后，处理器300将多张融合子图像进行拼接即可得到初始的拼接子图像。进一步地，处理器300需要截取出待匹配元图像P2中除去已与基准元图像P1相匹配的区域之外的未匹配区域，并将初始的拼接子图像与这部分未匹配区域做拼接，从而得到最终的拼接子图像Pm，其中，拼接子图像的像素个数大于基准元图像P1及待匹配元图像P2的像素个数。可以理解，未匹配区域是在基准元图像P1中无法找到与这部分区域匹配的图像，说明未匹配区域的图像是基准元图像P1没有的，这是由于输出基准元图像P1的感光像素集合和输出待匹配元图像P2的感光像素集合之间的视场差异导致的，形成拼接子图像Pm时，将未匹配区域的图像也拼接到拼接子图像Pm中可以保证成像装置100拍摄的画面的完整性。
在形成拼接子图像Pm后，处理器300将拼接子图像Pm作为新的基准元图像，并从剩余的元图像P3、P4、P5…PN中继续挑选一张元图像作为新的待匹配元图像。随后，处理器300按照上述的基准元图像P1和待匹配元图像P2的融合拼接方式，将基准元图像Pm划分为多张块图像，并在待匹配元图像P3中寻找与基准元图像Pm中的多张块图像分别匹配的匹配块图像，再执行上述的融合和拼接过程以将基准元图像Pm和待匹配元图像P3融合拼接成一张新的拼接子图像Pm。随后，处理器300再将新的拼 接子图像Pm作为新的基准元图像，并从剩余的元图像P4、P5、P6…PN中继续挑选一张元图像作为新的待匹配元图像。如此循环往复，直至将所有元图像均融合拼接完毕为止，最终得到一张合并图像，合并图像具有较高的分辨率。
在一个例子中，处理器300根据至少两张元图像计算场景的深度信息时，具体执行以下操作：将多张元图像划分为基准图像集合和待匹配图像集合，基准图像集合中包括多张元图像，待匹配图像集合中包括多张元图像；从基准图像集合中选取一张元图像作为基准元图像，从待匹配图像集合中选取一张元图像作为待匹配元图像；将基准元图像划分为多张块图像，并从多张块图像中选取一张块图像作为基准块图像；在待匹配元图像中寻找与基准块图像匹配的匹配块图像以形成匹配图像对；根据匹配图像对中的基准块图像和匹配块图像的视差计算所述深度信息；循环执行划分步骤、寻找步骤及计算步骤以遍历基准元图像中的多张块图像，以获取多个深度信息；循环执行上述从基准图像集合中选取一张元图像作为基准元图像的步骤至获取多个深度信息的步骤以遍历基准图像集合中的多张元图像，得到多个深度信息。
具体地，如图7所示，假设有16张元图像，分别为元图像P1、P2、P3…P16，则处理器300将16张元图像划分为两个集合：基准图像集合和待匹配图像集合。其中，基准图像集合包括元图像P1、元图像P2、元图像P5、元图像P6、元图像P9、元图像P10、元图像P13、元图像P14；待匹配图像集合包括元图像P3、元图像P4、元图像P7、元图像P8、元图像P11、元图像P12、元图像P15、元图像P16。随后，处理器300从基准图像集合中选取一张元图像作为基准元图像，如选择元图像P1作为基准元图像，并从待匹配图像集合中选取一张元图像作为待匹配元图像，如选择元图像P3作为待匹配元图像。随后，处理器300将基准元图像P1划分为多张块图像，如9张块图像：块图像P1-00、块图像P1-01、块图像P1-02、块图像P1-10、块图像P1-11、块图像P1-12、块图像P1-20、块图像P1-21、块图像P1-22。随后，处理器300从9张块图像中选取一张块图像作为基准块图像，例如，选取块图像P1-00作为基准块图像。在确定基准块图像P1-00之后，处理器300在待匹配元图像P3中寻找与基准块图像P1-00匹配的匹配块图像。具体地，处理器300在待匹配元图像P3中寻找与基准块图像P1-00位置对应的区域P3～00，并对基准块图像P1-00与区域P3～00做相关性计算，以判断区域P3～00是否为与基准块图像P1-00匹配的匹配块图像，如果相关性大于预定相关值，则确定区域P3～00为与基准块图像P1-00匹配的匹配块图像，并对区域P3～00做标记以便后续做深度信息计算；反之，则在待匹配元图像P3中，以区域P3～00为起始点，向x方向和/或y方向按照预定移动步距移动与基准块图像P1-00的大小相同的矩形框，每移动一次矩形框，矩形框所框出来的区域均需要与基准块图像P1-00做相关性计算，以判断矩形框所框出来的区域是否为与基准块图像P1-00匹配的匹配块图像，如果矩形框所框出来的区域为与基准块图像P1-00匹配的匹配块图像，则对矩形框框出来的区域做标记，否则，矩形框继续移动，直至遍历整张元图像P3为止。如图7所示，由于输出基准元图像P1的感光像素集合与输出待匹配元图像P3的感光像素集合的视场差异，基准块图像P1-00在待匹配元图像P3中无法找到与其匹配的匹配块图像，此时同样输出图像匹配对，但图像匹配对中仅包含基准块图像P1-00。
在寻找完基准块图像P1-00的匹配块图像后，处理器300将基准块图像P1-00更换为块图像P1-01，并按照上述的方式寻找与基准块图像P1-01匹配的匹配块图像。如图7所示，与基准块图像P1-01匹配的匹配块图像为块图像P13-01，此时输出图像匹配对，图像匹配对包括基准块图像P1-01和匹配块图像P13-01。随后，处理器300继续更换基准块图像，并继续执行上述寻找步骤。如此循环往复，直至确定出基准元图像P1中的所有块图像的匹配块图像，并输出与块图像的数量一致的多对图像匹配对。如图7所示，元图像P1中每张块图像对应的图像匹配对分别为：“P1-00”、“P1-01＝P13-01”、“P1-02＝P13-02”、“P1-10”、“P1-11＝P13-11”、“P1-12＝P13-12”、“P1-20”、“P1-21＝P13-21”、“P1-22＝P13-22”。随后，处理器300将包含两张元图像的图像匹配对筛选出来，并对每一对图像匹配对中的基准块图像与匹配块图像做视差计算以得到至少一个深度信息d。具体地，处理器300基于基准块图像在基准元图像P1中的坐标位置、匹配块图像在待匹配元图像P3中的坐标位置、输出基准元图像P1的感光像素集合与输出待匹配元图像P3的感光像素集合之间的位置关系来做视差计算以得到至少一个深度信息d。如此，对多对图像匹配对中的基准块图像与匹配块图像做视差计算即可得到多个深度信息d。
随后，处理器300从基准图像集合的剩余的元图像中选取一张元图像作为新的基准元图像，如选取元图像P2作为新的基准元图像，并从待匹配图像集合的剩余的元图像中选取一张元图像作为新的待匹 配元图像，如选取元图像P4作为新的待匹配元图像。随后，处理器300按照上述的深度信息d的计算方式对基准元图像P2与待匹配元图像P4进行处理，以得到多个深度信息d。随后，处理器300从基准图像集合的剩余的元图像中选取一张元图像作为新的基准元图像，如选取元图像P5作为新的基准元图像，并从待匹配图像集合的剩余的元图像中选取一张元图像作为新的待匹配元图像，如选取元图像P7作为新的待匹配元图像。如此循环往复，直至处理器300执行完对基准元图像P14与待匹配元图像P16的深度信息d计算为止。如此，即可得到多个深度信息d，对多个深度信息d进行融合，即可得到场景的深度图像，其中，深度信息d指示场景中的各个物体与成像装置100之间的距离。
在一个例子中，处理器300用于根据深度信息对合并图像做预定处理时，处理器300具体执行以下操作：根据深度信息确定合并图像的前景区域和背景区域；根据深度信息对背景区域做虚化处理。
合并图像与深度图像具有一定的映射关系，合并图像中的每个像素可以在深度图像中找到对应的深度信息。在获取到场景的深度信息后，处理器300可以根据深度信息对合并图像做前景区域和背景区域的分割。具体地，在一个例子中，处理器300可以直接根据一个预设深度对合并图像做前景区域和背景区域的分割，即，将深度信息大于预设深度的像素归并为背景区域，将深度信息小于或等于预设深度的像素归并为前景区域。随后，处理器300对前景区域不做处理，或者对前景区域做适当的锐化处理。同时，处理器300对背景区域做虚化处理。处理器300虚化处理背景区域时，背景区域的所有像素可以具有相同的虚化程度。或者，处理器300虚化处理背景区域时，处理器300也可进一步对背景区域进行划分，将背景区域划分为由近及远的多个子区域，沿由近及远的方向，对子区域的虚化程度依次增大，其中，每个子区域中的多个像素具有相同的虚化程度。如此，对背景区域做不同程度虚化，可以提升最终输出的合并图像的质量。
在另一个例子中，处理器300用于根据深度信息对合并图像做预定处理时，处理器300具体执行以下操作：根据用户输入确定合并图像的待对焦区域；根据深度信息对合并图像的除待对焦区域外的区域做虚化处理。其中，用户输入可以包括：用户在可触摸的显示屏200上点击预览的合并图像对应于显示屏200的某个位置，处理器300以该位置点为中心点，向外扩大预定大小和形状的区域以得到待对焦区域。或者，处理器300记录用户之前多次使用成像装置100时点击的显示屏200的多个位置，并将点击次数最多的位置作为默认位置，在用户未点击显示屏200时，处理器300以默认位置为中心点，向外扩大预定大小和形状的区域以得到待对焦区域。
合并图像与深度图像具有一定的映射关系，合并图像中的每个像素可以在深度图像中找到对应的深度信息。在获取到场景的深度信息以及确定合并图像的待对焦区域后，处理器300可以对待对焦区域不做处理，或者对待对焦区域做适当的锐化处理。同时，处理器300对除待对焦区域外的区域(即，非对焦区域)做虚化处理。具体地，处理器300可以对非对焦区域中的所有像素做同一虚化程度的虚化处理。或者，处理器300还可以根据深度信息将非对焦区域进一步划分为由近及远的多个子区域，沿由近及远的方向，对子区域的虚化程度依次增大，其中，每个子区域中的多个像素具有相同的虚化程度。如此，对非对焦区域做不同程度虚化，可以提升最终输出的合并图像的质量。
综上，本申请实施方式的成像装置100和电子设备1000中，成像装置100设置了覆盖多个感光像素集合的微透镜阵列结构10，每个感光像素集合均可以输出元图像，元图像经处理器300融合后可以得到高分辨率的合并图像。与现有的阵列相机相比，现有的阵列相机需要使用多个传统摄像头，使得阵列相机的整体尺寸较大，不利于集成在对厚度要求较高的电子设备上，并且阵列相机的成本也较高。而本申请实施方式的成像装置100无需设置多个传统的摄像头即可拍摄到分辨率较高的合并图像，成像装置100的整体尺寸较小，有利于集成在对厚度要求较高的电子设备1000上。并且，成像装置100的成本也较低，进一步地可以减小电子设备1000的制造成本。
请参阅图8和图9，微透镜阵列结构10包括第一透镜阵列12和第二透镜阵列14。
第一透镜阵列12包括多个第一凸台126，多个第一凸台126沿第一方向(如图8中的X轴方向)排列。每个第一凸台126均呈半圆柱形(或其他合适的形状)，以实现光学聚焦效果。第一透镜阵列12包括相背的第一平坦面122和第一凸起面124(类似波浪面)。多个第一凸台126共同形成第一凸起面124。具体地，每个第一凸台126包括第一子凸起面，多个第一凸台126的多个第一子凸起面相连形成第一凸起面124。多个第一凸台126共同形成第一平坦面122。具体地，每个第一凸台126包括第一子平坦面，多个第一凸台126的多个第一子平坦面相连形成第一平坦面122。
第二透镜阵列14包括多个第二凸台146，多个第二凸台146沿第二方向(如图8中的Y轴方向)排列。每个第二凸台146均呈半圆柱形(或其他合适的形状)，以实现光学聚焦效果。第二透镜阵列14包括相背的第二平坦面142和第二凸起面144(类似波浪面)。多个第二凸台146共同形成第二凸起面144。具体地，每个第二凸台146包括第二子凸起面，多个第二凸台146的多个第二子凸起面相连形成第二凸起面144。多个第二凸台146共同第二平坦面142。具体地，每个第二凸台146包括第二子平坦面，多个第二凸台146的多个第二子平坦面相连形成第二平坦面142。
多个第一凸台126与多个第二凸台146在第三方向(如图8中的Z轴方向)上交叠形成微透镜阵列。第一方向与第二方向相交，第三方向垂直于第一方向和第二方向。以图8为例，第一凸台126和第二凸台146的数量均为20个，即20个第一凸台126沿X轴方向排列构成1*20的第一透镜阵列12，20个第二凸台146沿Y轴方向排列构成20*1的第二透镜阵列14，20个第一凸台126与20个第二凸台146在Z轴方向交叠形成20*20的微透镜阵列。本实施方式中，第一方向与第二方向相交，第三方向垂直于第一方向和第二方向。第一方向与第二方向相交指的是第一方向与第二方向既不重合也不相互平行，它们在立体空间的投影相交，具体地可形成30度、45度、60度、75度、90度等夹角，例如图8中该夹角为90度。
请结合图10，在制造本申请实施方式的微透镜阵列结构10时，可以先分别形成包括有多个第一凸台126的第一透镜阵列12及包括有多个第二凸台146的第二透镜阵列14，然后再将第一透镜阵列12与第二透镜阵列14组合以使多个第一凸台126与多个第二凸台146在第三方向上交叠形成微透镜阵列，微透镜阵列包括多个微透镜101，如图9中虚线所示部分，每个微透镜101由长度方向为Y方向的第一凸台126与长度方向为X方向的第二凸台146交叠形成，每个微透镜101由该第一凸台126上与该第二凸台146交叠的部分及该第二凸台146上与该第一凸台126交叠的部分组成。具体地，可以通过纳米压印技术在立方体(长方体或正方体)的整块透镜结构上形成沿第一方向排列的多个第一凸台126以作为第一透镜阵列12，多个第一凸台126依次衔接，彼此之间没有空隙，以能够形成更紧密排布的微透镜阵列，从而成像装置100能够采集更多的图像信息。第二透镜阵列14可以通过相同的方式形成，在此不重复说明。将第一透镜阵列12与第二透镜阵列14组合以使多个第一凸台126与多个第二凸台146在第三方向上交叠形成微透镜阵列可以是将第一透镜阵列12置于第二透镜阵列14上；或者将第二透镜阵列14置于第一透镜阵列12上，并使得多个第一凸台126与多个第二凸台146交错排布(例如纵横交错排布)，且在第三方向上相互抵触或接触配合。本实施方式中，形成第一透镜阵列12和第二透镜阵列14所采用的材料均可以是玻璃、塑料或其他。
微透镜阵列中的每个微透镜101均覆盖一个感光像素集合中的多个感光像素20，例如，每个微透镜101覆盖70*70个感光像素20。微透镜阵列与多个感光像素20形成的感光像素阵列对准，具体地，每个微透镜101的光轴可与多个感光像素20形成的感光像素阵列的法线重合。
本申请实施方式的微透镜阵列结构10中，多个第一凸台126与多个第二凸台146交叠形成微透镜阵列，制造工艺简单，制作出来的微透镜阵列均匀性较好(即整齐排列，结构一致)，成本较低。由于没有采用光刻加热回流的方式和模具注塑或者研磨加工的方式制作，因此不存在微透镜尺寸不能做大的问题，或是微透镜尺寸做得比较大的时候屈光度有限、一致性性较差的问题，或是精密的模具加工难度非常大的问题。
另外，由于多个第一凸台126一体成型，多个第二凸台146一体成型，多个第一凸台126不需要通过胶水等方式固定在一起，多个第二凸台146也不需要通过胶水等方式固定在一起。当将第一透镜阵列12与第二透镜阵列14组装在一起时，多个第一凸台126之间不会发生相互偏移，多个第二凸台146之间也不会发生相互偏移，组装稳定性较高。
再有，当将第一透镜阵列12与第二透镜阵列14组装在一起时，第一凸台126与第二凸台146之间也可以不用胶水(如光学胶)等方式进行固定，只需要将第一凸台126与第二凸台146抵触，通过成像装置100的镜筒或其他元件分别将第一透镜阵列12与第二透镜阵列14固定，整个成像装置100的安装方式较为简单。当其中一个透镜阵列例如第一透镜阵列12或是第二透镜阵列14损坏时，还可以仅将第一透镜阵列12或第二透镜阵列14拆卸下来进行更换。
最后，微透镜阵列中每个微透镜101的焦距相同，成像装置100可作为定焦镜头(不涉及对焦)，当被摄物目标处于成像装置100的有效焦距范围内，能够清晰成像，较适合作为电子设备100的前置镜 头。
请参阅图11和图12，在一个实施例中，当第一透镜阵列12与第二透镜阵列14组合时，第一平坦面122与第二平坦面142结合。此时，第一平坦面122与第二平坦面142的结合区域能够完全的贴合在一起，结合紧密，稳定性更高，也不会有水分或杂质进入到第一透镜阵列12与第二透镜阵列14之间，有利于保证微透镜阵列结构10的使用寿命和成像装置100良好的成像品质。
请参阅图8和图9，在一个实施例中，当第一透镜阵列12与第二透镜阵列14组合时，第一凸起面124与第二平坦面142结合。此时，微透镜阵列结构10与成像装置100的其他结构(例如镜筒、感光像素20等)组装时，第一平坦面122能够很好的安装于其他结构上。
在一个实施例中，当第一透镜阵列12与第二透镜阵列14组合时，第一平坦面122与第二凸起面144结合(与图8和图9中第一凸起面124与第二平坦面142结合类似)。此时，微透镜阵列结构10与成像装置头100的其他结构(例如镜筒、滤光片等)组装时，第二平坦面142能够很好的安装于其他结构上。
请参阅图13和图14，在一个实施例中，当第一透镜阵列12与第二透镜阵列14组合时，第一凸起面124与第二凸起面144结合。此时，微透镜阵列结构10与成像装置100的其他结构(例如镜筒、感光像素20、滤光片等)组装时，第一平坦面122和第二平坦面142能够很好的安装于其他结构上，且由于第一凸起面124与第二凸起面144由于没有暴露在外侧(与其他结构结合的一侧)，第一凸起面124与第二凸起面144不会由于受到其他结构的磨损而影响光学聚焦效果。
本实施方式中，第一平坦面122与第二平坦面142结合、第一平坦面122与第二凸起面144结合、第一凸起面124与第二平坦面142结合、第一凸起面124与第二凸起面144结合，能够分别实现不同的焦距和视场角。
请参阅图8、图11和图13，上述各实施例中，第一平坦面122和第二平坦面142可以均为平面结构，此时，多个感光像素20形成的感光像素阵列也为平面结构，以便于实现微透镜阵列与感光像素阵列的对准，保证成像装置100良好的成像品质，且有利于成像装置100的组装。可以理解，平面结构是指面上任意两点的连线整个落在此面上。
请参阅图15，上述各实施例中，第一平坦面122和第二平坦面142可以均为曲面结构，此时，多个感光像素20形成的感光像素阵列也为曲面结构，以便于实现微透镜阵列与感光像素阵列的对准，保证成像装置100良好的成像品质，且成像装置100的视场角更大，能够收集更多的光线，相较于平面结合而言体积也更小。此时，若成像装置100还包括滤光片(如用于让红外光通过的红外滤光片或用于滤除红外光的红外截止滤光片等)，则滤光片也可为曲面结构，以更好的过滤光线。
进一步地，当第一平坦面122与第二平坦面142结合时，第一平坦面122的弯曲程度与第二平坦面142的弯曲程度相同。当第一平坦面122与第二凸起面144结合时，第一平坦面122的弯曲程度与第二凸起面144的弯曲程度相同。当第一凸起面124与第二平坦面142结合时，第一凸起面124的弯曲程度与第二平坦面142的弯曲程度相同(如图15所示)。当第一凸起面124与第二凸起面144结合时，第一凸起面124的弯曲程度与第二凸起面144的弯曲程度相同。需要指出的是，凸起面的弯曲程度可以看作是凸起面外接圆弧的弯曲程度，或是每个凸台与对应的平坦面之间具有一距离最远的点，凸起面的弯曲程度为多个凸台的多个点形成的弧线的弯曲程度。
第一平坦面122、第一凸起面124、第二平坦面142和第二凸起面144的面型可以均为非球面、球面、菲涅尔面或二元光学面中的任意一种。例如，第一平坦面122、第一凸起面124、第二平坦面142和第二凸起面144均为非球面；或者，第一平坦面122和第一凸起面124为非球面，第二平坦面142为球面，第二凸起面144为菲涅尔面；或者，第一平坦面122为非球面，第一凸起面124为球面，第二平坦面142为菲涅尔面，第二凸起面144为二元光学面等。
当面型选用非球面时，有利于修正成像装置100的像差，解决视界歪曲等问题，同时透镜更轻、更薄、更平，且仍然能保持优异的抗冲击性能；当面型选用球面时，微透镜阵列结构10制造工艺较为简单；当面型选用菲涅尔面时，成像装置100成像较为明亮、亮度均匀，不易出现边角变暗、模糊的问题；当面型选用二元光学面时，透镜重量轻、成本低，并能实现传统光学难以完成的微小、阵列、集成等新功能。
请再次参阅图8，当第一方向与第二方向垂直，且每个第一凸台126的宽度等于每个第二凸台146的宽度时，多个第一凸台126与多个第二凸台146形成的微透镜阵列为正方形的微透镜阵列。请参阅图 16，当第一方向与第二方向垂直，且每个第一凸台126的宽度大于每个第二凸台146的宽度或者每个第二凸台146的宽度大于第一凸台126的宽度(如图16所示)时，多个第一凸台126与多个第二凸台146形成的微透镜阵列为长方形微透镜阵列。请参阅图17，当第一方向与第二方向相交且第一方向不与第二方向垂直时，多个第一凸台126与多个第二凸台146形成的微透镜阵列为平行四边形微透镜阵列。本实施方式的微透镜阵列可以为正方形微透镜阵列、长方形微透镜阵列或平行四边形微透镜阵列，以适用于多种不同形状、结构或功能的成像装置100。
本申请还提供一种微透镜阵列结构10的制造方法。本申请实施方式的微透镜阵列结构10的制造方法包括：
形成第一透镜阵列12，第一透镜阵列12包括多个第一凸台126，多个第一凸台126沿第一方向排列；
形成第二透镜阵列14，第二透镜阵列14包括多个第二凸台146，多个第二凸台146沿第二方向排列；和
将第一透镜阵列12与第二透镜阵列14组合以使多个第一凸台126与多个第二凸台146在第三方向上交叠形成微透镜阵列，第一方向与第二方向相交，第三方向垂直于第一方向和第二方向。
可以理解，前述对微透镜阵列结构10的解释说明均适用于本实施方式的微透镜阵列结构10的制造方法，在此不再展开说明。
请参阅图18，本申请还提供一种电子设备1000。电子设备1000包括上述任意一项实施方式所述的成像装置100和壳体400。成像装置100安装在壳体400上。其中，成像装置100还包括处理器50。处理器50设置在成像装置100内。处理器50可用于控制多个感光像素20曝光以接收穿过微透镜阵列结构10的光线，并接收每个感光像素20输出的电信号以形成与多个感光像素集合一一对应的元图像。处理器50还可以用于融合多张元图像得到合并图像、根据至少两张元图像计算场景的深度信息、以及根据深度信息对合并图像做预定处理。处理器50执行上述步骤时与处理器300执行上述步骤的过程一致，在此不再展开说明。
在本说明书的描述中，参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本申请的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不必须针对的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任一个或多个实施例或示例中以合适的方式结合。此外，在不相互矛盾的情况下，本领域的技术人员可以将本说明书中描述的不同实施例或示例以及不同实施例或示例的特征进行结合和组合。
此外，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括至少一个该特征。在本申请的描述中，“多个”的含义是至少两个，例如两个，三个等，除非另有明确具体的限定。
尽管上面已经示出和描述了本申请的实施例，可以理解的是，上述实施例是示例性的，不能理解为对本申请的限制，本领域的普通技术人员在本申请的范围内可以对上述实施例进行变化、修改、替换和变型。

Claims (20)

  1. 一种成像装置，其特征在于，所述成像装置包括微透镜阵列结构和多个感光像素，所述微透镜阵列结构覆盖多个感光像素集合，每个所述感光像素集合包括多个所述感光像素，每个所述感光像素集合中的多个感光像素接收穿过所述微透镜阵列结构入射的光线以输出多张元图像；所述成像装置还包括处理器，所述处理器用于：
    融合多张所述元图像得到合并图像；
    根据至少两张所述元图像计算场景的深度信息；和
    根据所述深度信息对所述合并图像做预定处理。
  2. 根据权利要求1所述的成像装置，其特征在于，所述处理器还用于：
    选取两张所述元图像，一张所述元图像作为基准元图像，另一张所述元图像作为待匹配元图像；
    将所述基准元图像划分为多张块图像，并从多张所述块图像中选取一张所述块图像作为基准块图像；
    在所述待匹配元图像中寻找与所述基准块图像匹配的匹配块图像以形成匹配图像对；
    循环执行上述划分步骤及上述寻找步骤以遍历所述基准元图像中的多张所述块图像，得到多个所述匹配图像对；
    融合每个匹配图像对中的所述基准块图像和所述匹配块图像得到融合子图像，并拼接多张所述融合子图像以获取拼接子图像；和
    将所述拼接子图像作为新的所述基准元图像，从剩余的多张所述元图像中选取一张所述元图像作为新的所述待匹配元图像，并循环执行上述将所述基准元图像划分为多张块图像的步骤至获取拼接子图像的步骤以融合得到所述合并图像，其中，循环执行将所述基准元图像划分为多张块图像的步骤时是以前一次的所述拼接子图像作为所述基准元图像。
  3. 根据权利要求1所述的成像装置，其特征在于，所述处理器还用于：
    将多张所述元图像划分为基准图像集合和待匹配图像集合，所述基准图像集合中包括多张所述元图像，所述待匹配图像集合中包括多张所述元图像；
    从所述基准图像集合中选取一张所述元图像作为基准元图像，从所述待匹配图像集合中选取一张所述元图像作为待匹配元图像；
    将所述基准元图像划分为多张块图像，并从多张所述块图像中选取一张所述块图像作为基准块图像；
    在所述待匹配元图像中寻找与所述基准块图像匹配的匹配块图像以形成匹配图像对；
    根据所述匹配图像对中的所述基准块图像和所述匹配块图像的视差计算所述深度信息；
    循环执行所述划分步骤、所述寻找步骤及所述计算步骤以遍历所述基准元图像中的多张所述块图像，以获取多个所述深度信息；和
    循环执行上述从所述基准图像集合中选取一张所述元图像作为基准元图像的步骤至获取多个所述深度信息的步骤以遍历所述基准图像集合中的多张所述元图像，得到多个所述深度信息。
  4. 根据权利要求1所述的成像装置，其特征在于，所述处理器还用于：
    根据所述深度信息确定所述合并图像的前景区域和背景区域；
    根据所述深度信息对所述背景区域做虚化处理。
  5. 根据权利要求1所述的成像装置，其特征在于，所述处理器还用于：
    根据用户输入确定所述合并图像的待对焦区域；
    根据所述深度信息对所述合并图像的除所述待对焦区域外的区域做虚化处理。
  6. 根据权利要求1至5任意一项所述的成像装置，其特征在于，相邻两个所述感光像素集合之间设置有间隔壁。
  7. 根据权利要求1至5任意一项所述的成像装置，其特征在于，所述微透镜阵列结构包括多个微透镜，每个所述微透镜覆盖一个所述感光像素集合；所述成像装置还包括衬底，多个所述感光像素形成在所述衬底上，所述衬底的与所述感光像素接触的面为曲面结构。
  8. 根据权利要求7所述的成像装置，其特征在于，至少两个所述微透镜具有不同的曲率半径。
  9. 根据权利要求1至5任意一项所述的成像装置，其特征在于，所述微透镜阵列结构包括第一透镜阵列和第二透镜阵列，所述第一透镜阵列包括多个第一凸台，多个所述第一凸台沿第一方向排列，所述第二透镜阵列包括多个第二凸台，多个所述第二凸台沿第二方向排列；所述第一凸台与所述第二凸台在 第三方向上交叠形成所述微透镜阵列结构，所述第一方向与所述第二方向相交，所述第三方向垂直于所述第一方向和所述第二方向。
  10. 根据权利要求9所述的成像装置，其特征在于，所述第一透镜阵列包括相背的第一平坦面和第一凸起面，多个所述第一凸台形成所述第一凸起面，所述第二透镜阵列包括相背的第二平坦面和第二凸起面，多个所述第二凸台形成所述第二凸起面；所述第一平坦面和所述第二平坦面均为曲面结构。
  11. 一种电子设备，其特征在于，包括：
    壳体；和
    成像装置，所述成像装置安装在所述壳体上，成像装置包括微透镜阵列结构和多个感光像素，所述微透镜阵列结构覆盖多个感光像素集合，每个所述感光像素集合包括多个所述感光像素，每个所述感光像素集合中的多个感光像素接收穿过所述微透镜阵列结构入射的光线以输出多张元图像；所述成像装置还包括处理器，所述处理器用于：
    融合多张所述元图像得到合并图像；
    根据至少两张所述元图像计算场景的深度信息；和
    根据所述深度信息对所述合并图像做预定处理。
  12. 根据权利要求11所述的电子设备，其特征在于，所述处理器还用于：
    选取两张所述元图像，一张所述元图像作为基准元图像，另一张所述元图像作为待匹配元图像；
    将所述基准元图像划分为多张块图像，并从多张所述块图像中选取一张所述块图像作为基准块图像；
    在所述待匹配元图像中寻找与所述基准块图像匹配的匹配块图像以形成匹配图像对；
    循环执行上述划分步骤及上述寻找步骤以遍历所述基准元图像中的多张所述块图像，得到多个所述匹配图像对；
    融合每个匹配图像对中的所述基准块图像和所述匹配块图像得到融合子图像，并拼接多张所述融合子图像以获取拼接子图像；和
    将所述拼接子图像作为新的所述基准元图像，从剩余的多张所述元图像中选取一张所述元图像作为新的所述待匹配元图像，并循环执行上述将所述基准元图像划分为多张块图像的步骤至获取拼接子图像的步骤以融合得到所述合并图像，其中，循环执行将所述基准元图像划分为多张块图像的步骤时是以前一次的所述拼接子图像作为所述基准元图像。
  13. 根据权利要求11所述的电子设备，其特征在于，所述处理器还用于：
    将多张所述元图像划分为基准图像集合和待匹配图像集合，所述基准图像集合中包括多张所述元图像，所述待匹配图像集合中包括多张所述元图像；
    从所述基准图像集合中选取一张所述元图像作为基准元图像，从所述待匹配图像集合中选取一张所述元图像作为待匹配元图像；
    将所述基准元图像划分为多张块图像，并从多张所述块图像中选取一张所述块图像作为基准块图像；
    在所述待匹配元图像中寻找与所述基准块图像匹配的匹配块图像以形成匹配图像对；
    根据所述匹配图像对中的所述基准块图像和所述匹配块图像的视差计算所述深度信息；
    循环执行所述划分步骤、所述寻找步骤及所述计算步骤以遍历所述基准元图像中的多张所述块图像，以获取多个所述深度信息；和
    循环执行上述从所述基准图像集合中选取一张所述元图像作为基准元图像的步骤至获取多个所述深度信息的步骤以遍历所述基准图像集合中的多张所述元图像，得到多个所述深度信息。
  14. 根据权利要求11所述的电子设备，其特征在于，所述处理器还用于：
    根据所述深度信息确定所述合并图像的前景区域和背景区域；
    根据所述深度信息对所述背景区域做虚化处理。
  15. 根据权利要求11所述的电子设备，其特征在于，所述处理器还用于：
    根据用户输入确定所述合并图像的待对焦区域；
    根据所述深度信息对所述合并图像的除所述待对焦区域外的区域做虚化处理。
  16. 一种电子设备，其特征在于，包括：
    壳体；
    成像装置，所述成像装置包括微透镜阵列结构和多个感光像素，所述微透镜阵列结构覆盖多个感光 像素集合，每个所述感光像素集合包括多个所述感光像素，每个所述感光像素集合中的多个感光像素接收穿过所述微透镜阵列结构入射的光线以输出多张元图像；和
    处理器，所述处理器用于：融合多张所述元图像得到合并图像、根据至少两张所述元图像计算场景的深度信息、及根据所述深度信息对所述合并图像做预定处理。
  17. 根据权利要求16所述的电子设备，其特征在于，所述处理器还用于：
    选取两张所述元图像，一张所述元图像作为基准元图像，另一张所述元图像作为待匹配元图像；
    将所述基准元图像划分为多张块图像，并从多张所述块图像中选取一张所述块图像作为基准块图像；
    在所述待匹配元图像中寻找与所述基准块图像匹配的匹配块图像以形成匹配图像对；
    循环执行上述划分步骤及上述寻找步骤以遍历所述基准元图像中的多张所述块图像，得到多个所述匹配图像对；
    融合每个匹配图像对中的所述基准块图像和所述匹配块图像得到融合子图像，并拼接多张所述融合子图像以获取拼接子图像；和
    将所述拼接子图像作为新的所述基准元图像，从剩余的多张所述元图像中选取一张所述元图像作为新的所述待匹配元图像，并循环执行上述将所述基准元图像划分为多张块图像的步骤至获取拼接子图像的步骤以融合得到所述合并图像，其中，循环执行将所述基准元图像划分为多张块图像的步骤时是以前一次的所述拼接子图像作为所述基准元图像。
  18. 根据权利要求16所述的电子设备，其特征在于，所述处理器还用于：
    将多张所述元图像划分为基准图像集合和待匹配图像集合，所述基准图像集合中包括多张所述元图像，所述待匹配图像集合中包括多张所述元图像；
    从所述基准图像集合中选取一张所述元图像作为基准元图像，从所述待匹配图像集合中选取一张所述元图像作为待匹配元图像；
    将所述基准元图像划分为多张块图像，并从多张所述块图像中选取一张所述块图像作为基准块图像；
    在所述待匹配元图像中寻找与所述基准块图像匹配的匹配块图像以形成匹配图像对；
    根据所述匹配图像对中的所述基准块图像和所述匹配块图像的视差计算所述深度信息；
    循环执行所述划分步骤、所述寻找步骤及所述计算步骤以遍历所述基准元图像中的多张所述块图像，以获取多个所述深度信息；和
    循环执行上述从所述基准图像集合中选取一张所述元图像作为基准元图像的步骤至获取多个所述深度信息的步骤以遍历所述基准图像集合中的多张所述元图像，得到多个所述深度信息。
  19. 根据权利要求16所述的电子设备，其特征在于，所述处理器还用于：
    根据所述深度信息确定所述合并图像的前景区域和背景区域；
    根据所述深度信息对所述背景区域做虚化处理。
  20. 根据权利要求16所述的电子设备，其特征在于，所述处理器还用于：
    根据用户输入确定所述合并图像的待对焦区域；
    根据所述深度信息对所述合并图像的除所述待对焦区域外的区域做虚化处理。
PCT/CN2019/104514 2018-11-26 2019-09-05 成像装置和电子设备 Ceased WO2020107997A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811417730.6 2018-11-26
CN201811417730.6A CN109348114A (zh) 2018-11-26 2018-11-26 成像装置和电子设备

Publications (1)

Publication Number Publication Date
WO2020107997A1 true WO2020107997A1 (zh) 2020-06-04

Family

ID=65318059

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/104514 Ceased WO2020107997A1 (zh) 2018-11-26 2019-09-05 成像装置和电子设备

Country Status (2)

Country Link
CN (1) CN109348114A (zh)
WO (1) WO2020107997A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116630220A (zh) * 2023-07-25 2023-08-22 江苏美克医学技术有限公司 一种荧光图像景深融合成像方法、装置及存储介质

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109445002B (zh) * 2018-11-26 2021-03-23 Oppo广东移动通信有限公司 微透镜阵列结构及其制造方法、复眼镜头、电子装置
CN109348114A (zh) * 2018-11-26 2019-02-15 Oppo广东移动通信有限公司 成像装置和电子设备
CN113924517B (zh) * 2019-06-06 2024-03-22 应用材料公司 生成复合影像的成像系统及方法
CN110177162A (zh) * 2019-06-27 2019-08-27 联想(北京)有限公司 显示屏及电子设备
CN110581956A (zh) * 2019-08-26 2019-12-17 Oppo广东移动通信有限公司 图像处理方法、装置、存储介质及电子设备
CN110505384B (zh) * 2019-08-29 2021-05-14 Oppo广东移动通信有限公司 成像系统、终端和图像获取方法
CN111641762B (zh) * 2020-05-28 2021-11-02 维沃移动通信有限公司 摄像模组及电子设备
CN114827320A (zh) * 2021-01-29 2022-07-29 北京小米移动软件有限公司 摄像装置及移动终端
CN113114906A (zh) * 2021-04-20 2021-07-13 维沃移动通信有限公司 像素结构、图像传感器、摄像组件和电子设备
CN113411474B (zh) * 2021-06-07 2023-03-21 Oppo广东移动通信有限公司 成像装置及电子设备
CN113691699B (zh) * 2021-08-02 2023-06-20 维沃移动通信有限公司 成像芯片组件、摄像模组及其对焦方法和电子设备
CN114125238A (zh) * 2021-11-30 2022-03-01 维沃移动通信有限公司 图像传感器、摄像模组及电子设备
CN115278127A (zh) * 2022-07-25 2022-11-01 Oppo广东移动通信有限公司 图像传感器、摄像头和电子装置
CN117218967A (zh) * 2023-09-13 2023-12-12 合肥联宝信息技术有限公司 一种具有摄像功能的自发光型显示屏及电子设备

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6057538A (en) * 1996-09-07 2000-05-02 U.S. Philips Corporation Image sensor in which each lens element is associated with a plurality of pixels
CN1285997A (zh) * 1997-11-14 2001-02-28 莱达·E·坦根 光电照相机和用于在光电照相机中格式化图像的方法
US20080173791A1 (en) * 2007-01-19 2008-07-24 Palum Russell J Image sensor with three sets of microlenses
CN102075679A (zh) * 2010-11-18 2011-05-25 无锡中星微电子有限公司 一种图像采集方法和装置
TW201502583A (zh) * 2013-07-02 2015-01-16 Chunghwa Picture Tubes Ltd 立體顯示裝置
CN104318568A (zh) * 2014-10-24 2015-01-28 武汉华目信息技术有限责任公司 一种图像配准的方法和系统
CN105096283A (zh) * 2014-04-29 2015-11-25 华为技术有限公司 全景图像的获取方法和装置
CN106899781A (zh) * 2017-03-06 2017-06-27 宇龙计算机通信科技(深圳)有限公司 一种图像处理方法及电子设备
CN109348114A (zh) * 2018-11-26 2019-02-15 Oppo广东移动通信有限公司 成像装置和电子设备
CN109445002A (zh) * 2018-11-26 2019-03-08 Oppo广东移动通信有限公司 微透镜阵列结构及其制造方法、复眼镜头、电子装置

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1692560A4 (en) * 2003-11-18 2009-12-30 Merlin Technologies Inc VARIABLE OPTICAL ARRAYS AND VARIABLE MANUFACTURING METHOD
JP5671842B2 (ja) * 2010-06-03 2015-02-18 株式会社ニコン 画像処理装置および撮像装置
KR101640404B1 (ko) * 2010-09-20 2016-07-18 엘지전자 주식회사 휴대 단말기 및 그 동작 제어방법
JP5699609B2 (ja) * 2011-01-06 2015-04-15 ソニー株式会社 画像処理装置および画像処理方法
KR20140094395A (ko) * 2013-01-22 2014-07-30 삼성전자주식회사 복수 개의 마이크로렌즈를 사용하여 촬영하는 촬영 장치 및 그 촬영 방법
CN103338325A (zh) * 2013-06-14 2013-10-02 杭州普维光电技术有限公司 基于全景摄像机的车底盘图像采集方法
CN103702032B (zh) * 2013-12-31 2017-04-12 华为技术有限公司 图像处理方法、装置和终端设备
EP3029931A1 (en) * 2014-12-04 2016-06-08 Thomson Licensing Image sensor unit and imaging apparatus
CN106331492B (zh) * 2016-08-29 2019-04-16 Oppo广东移动通信有限公司 一种图像处理方法及终端
CN106952222A (zh) * 2017-03-17 2017-07-14 成都通甲优博科技有限责任公司 一种交互式图像虚化方法及装置
CN108230384B (zh) * 2017-11-28 2021-08-24 深圳市商汤科技有限公司 图像深度计算方法、装置、存储介质和电子设备
CN108632506A (zh) * 2018-03-21 2018-10-09 中国科学院上海微系统与信息技术研究所 一种微透镜阵列成像系统

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6057538A (en) * 1996-09-07 2000-05-02 U.S. Philips Corporation Image sensor in which each lens element is associated with a plurality of pixels
CN1285997A (zh) * 1997-11-14 2001-02-28 莱达·E·坦根 光电照相机和用于在光电照相机中格式化图像的方法
US20080173791A1 (en) * 2007-01-19 2008-07-24 Palum Russell J Image sensor with three sets of microlenses
CN102075679A (zh) * 2010-11-18 2011-05-25 无锡中星微电子有限公司 一种图像采集方法和装置
TW201502583A (zh) * 2013-07-02 2015-01-16 Chunghwa Picture Tubes Ltd 立體顯示裝置
CN105096283A (zh) * 2014-04-29 2015-11-25 华为技术有限公司 全景图像的获取方法和装置
CN104318568A (zh) * 2014-10-24 2015-01-28 武汉华目信息技术有限责任公司 一种图像配准的方法和系统
CN106899781A (zh) * 2017-03-06 2017-06-27 宇龙计算机通信科技(深圳)有限公司 一种图像处理方法及电子设备
CN109348114A (zh) * 2018-11-26 2019-02-15 Oppo广东移动通信有限公司 成像装置和电子设备
CN109445002A (zh) * 2018-11-26 2019-03-08 Oppo广东移动通信有限公司 微透镜阵列结构及其制造方法、复眼镜头、电子装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116630220A (zh) * 2023-07-25 2023-08-22 江苏美克医学技术有限公司 一种荧光图像景深融合成像方法、装置及存储介质
CN116630220B (zh) * 2023-07-25 2023-11-21 江苏美克医学技术有限公司 一种荧光图像景深融合成像方法、装置及存储介质

Also Published As

Publication number Publication date
CN109348114A (zh) 2019-02-15

Similar Documents

Publication Publication Date Title
WO2020107997A1 (zh) 成像装置和电子设备
US10044919B2 (en) Structures and methods for capturing images by a portable electronic device
CN106462050B (zh) 用于半球形和球形成像的宽视场阵列相机
CN217404597U (zh) 成像镜头模块与电子装置
JP2018201214A (ja) フォールデッドオプティクスを用いたマルチカメラシステム
WO2020107984A1 (zh) 微透镜阵列结构及其制造方法、复眼镜头、电子装置
CN106847092B (zh) 一种显示面板、显示装置及其获取图像的方法
KR20170021788A (ko) 굴절 프리즘들을 이용하는 폴딩되는 광학 어레이 카메라
US12253795B2 (en) Lens apparatus and image pickup apparatus
JP2018529256A (ja) 低ｆナンバープレノプティックカメラのための２表面マイクロレンズアレイを有する方法および装置
CN110187463A (zh) 自动聚焦驱动组件、镜头及电子设备
CN107991838A (zh) 自适应三维立体成像系统
CN107948470B (zh) 摄像头模块和移动设备
CN103188443B (zh) 图像生成装置、数字照相机及方法
CN112004011B (zh) 一种图像采集的方法、装置及光路转变元件
Zhou et al. Focal sweep camera for space-time refocusing
CN113874772A (zh) 一种光学系统、电子设备以及显示装置
CN208724047U (zh) 镜头模组、摄像头模组及移动终端
CN208434044U (zh) 滤光组件、相机模组、图像撷取装置及电子装置
US20210203817A1 (en) Camera module array and assembly method therefor
KR102214199B1 (ko) 이동 통신 단말기
US11314150B2 (en) Phase detection autofocus (PDAF) optical system
CN110928113B (zh) 一种可变空间分辨率的光场采集装置
US20200012069A1 (en) Structures and Methods for Capturing Images by a Portable Electronic Device with a Linear Movement Switching Mechanism
CN102789056A (zh) 一种将普通摄影机变成立体电视摄影机的立体镜头

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19889255

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19889255

Country of ref document: EP

Kind code of ref document: A1