WO2019179462A1

WO2019179462A1 - Microlens array imaging system

Info

Publication number: WO2019179462A1
Application number: PCT/CN2019/078872
Authority: WO
Inventors: 张晓林; 徐越
Original assignee: 中国科学院上海微系统与信息技术研究所
Priority date: 2018-03-21
Filing date: 2019-03-20
Publication date: 2019-09-26
Also published as: CN108632506A

Abstract

The present invention relates to a microlens array imaging system, comprising: a microlens array comprising a plurality of microlenses arranged to be adjacent to each other; an opaque light-shielding fabricated structure provided between each pair of adjacent microlenses and forming a grid structure having a planar or curved surface with the plurality of microlenses; a photosensitive pixel array comprising a plurality of photosensitive elements disposed below the microlenses, respectively; and an imaging result processor receiving an electrical signal output by the photosensitive pixel array and converting the electrical signal into image information, so as to perform decoding and processing on the image information. The invention overcomes low-resolution issues of individual microlenses, and improves the overall resolution of a camera having a light field-capturing function, thereby ensuring independent imaging of each microlens, preventing interference between imaging results, and ensuring data accuracy.

Description

Microlens array imaging system

Technical field

The present invention relates to an imaging system, and more particularly to a microlens array imaging system.

Background technique

Most of the cameras currently on the market are single main lenses, which transform the three-dimensional real world into two-dimensional images, resulting in a large amount of information loss. In order to restore three-dimensional information from two-dimensional image information, it is often difficult to perform three-dimensional reconstruction. Therefore, a camera capable of collecting a light field is urgently needed to solve this problem.

Currently, cameras with the function of collecting light field are mainly camera arrays and light field cameras. Among them, the problem of camera arrays is that the volume is large, and the relative position adjustment between cameras requires a lot of manpower and time; while the light field camera is It is a one-piece structure that uses a microlens structure to record the angle information of the light. However, the image data of the light field camera is large, the image processing time is long, the real-time performance is poor, and the main lens cannot be removed, resulting in a larger overall lens size. Large, not suitable for small aircraft such as flapping machines.

Summary of the invention

In order to solve the above problems in the prior art, the present invention aims to provide a microlens array imaging system to reduce the volume and data processing calculation amount of a camera having a function of collecting a light field, and to improve real-time performance and expand an application surface.

A microlens array imaging system according to the present invention includes:

a microlens array comprising a plurality of microlenses arranged adjacent to each other;

An opaque light-shielding structure, which is disposed between every two adjacent microlenses and forms a planar or curved lattice structure with a plurality of the microlenses;

a photosensitive pixel array comprising a plurality of photosensitive elements disposed under each of the microlenses;

And an imaging result processor that receives an electrical signal output by the photosensitive pixel array and converts the electrical signal into image information to decode and process the image information.

In the above microlens array imaging system, each of the microlenses has a hexagonal shape, and the lattice structure is a honeycomb-like lattice structure.

In the above microlens array imaging system, each of the photosensitive elements is disposed on a focal plane of its corresponding microlens or adjacent to a focal plane of its corresponding microlens.

In the microlens array imaging system described above, different filter films are attached to the upper surface and/or the lower surface of each of the microlenses, and/or the surface of each of the photosensitive elements.

In the above microlens array imaging system, the imaging result processor includes: a data preliminary processing module and a three-dimensional reconstruction module, a target recognition module, and a motion detection module respectively connected to the data preliminary processing module.

Since the above technical solution is adopted, the main structure of the present invention includes a microlens array, a photosensitive pixel array, and an imaging result processor, which have the following advantages and positive effects compared with the prior art: the present invention adopts a microlens array Overcoming the problem of low resolution of each microlens, improving the overall resolution of the camera with the function of collecting light field by processing the data between different microlenses; using the light-shielding processing structure in the microlens array It ensures independent imaging of each microlens, avoids mutual interference between imaging results, and ensures the correctness of the data. By using the imaging result processor, three-dimensional reconstruction, target recognition, motion detection and other functions can be realized. In addition, the advantages of the invention include: small volume, small amount of data processing calculation, high real-time performance, and wide application range.

DRAWINGS

1 is a schematic structural view of a microlens array imaging system of the present invention;

2 is a schematic structural view of a microlens array and a light-shielding structure in a microlens array imaging system of the present invention;

3 is a schematic structural view of an imaging result processor in a microlens array imaging system of the present invention;

Figure 4 is a schematic illustration of microlens decoding in the present invention.

detailed description

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

As shown in FIG. 1-3, the present invention, that is, a microlens array imaging system, includes: a microlens array 1, an opaque light-shielding structure 2, a photosensitive pixel array 3, and an imaging result processor 4, specifically:

The microlens array 1 includes a plurality of microlenses 10 arranged adjacent to each other, each microlens 10 having a hexagonal shape;

The light-shielding structure 2 is disposed between each two adjacent microlenses 10, that is, each of the microlenses 10 has a light-shielding structure 2, thereby forming a planar or curved honeycomb shape together with the plurality of microlenses 10. Lattice structure

The photosensitive pixel array 3 is configured to receive the light transmitted by the microlens array 1 and form an electrical signal to be uniformly output to the imaging result processor 4. The photosensitive pixel array 3 includes: a plurality of photosensitive elements disposed under each of the microlenses 10 (for example a CMOS element), each photosensitive element being disposed on a focal plane of its corresponding microlens 10 or near a focal plane;

The imaging result processor 4 is configured to convert the electrical signals output by the photosensitive pixel array 3 into image information, and perform subsequent decoding, processing, and the like on the image information. The imaging result processor 4 includes: a data preliminary processing module 41 and respectively The data preliminary processing module 41 is connected to the three-dimensional reconstruction module 42, the target recognition module 43 and the motion detection module 44;

Therein, a different filter film (not shown) is attached to the upper surface and/or the lower surface of each microlens 10, and/or the surface of each photosensitive element, so that the spectra of different frequency bands can be responded to.

In the present invention, the above-described arrangement of the respective microlenses 10 in the microlens array 1 can effectively utilize the space, and by increasing the number of the microlenses 10, the imaging effect can be improved.

In the present invention, the microlenses 10 can be optically isolated by the light-shielding processing structure 2, thereby avoiding mutual interference of light between the different microlenses 10, and ensuring that the photosensitive elements under the respective microlenses 10 are not affected by adjacent microlenses. 10 The transmitted light influence, thereby also dividing the photosensitive pixel array 3 into a plurality of portions, each portion corresponding to one microlens 10, thereby making it an independent imaging structure, thereby ensuring the validity of the imaging result data. In this embodiment, the light-shielding structure 2 may be a opaque micro-isolation wall structure, or may refract or open the distance between the micro-lenses to prevent other adjacent micro-lights from being affected by other micro-lenses. Photosensitive element under the lens.

In the present invention, different filter films can be arranged on the respective microlenses 10 or the respective photosensitive elements as needed, just like a color camera, the pixels in the red, green and blue are staggered, thereby realizing the partition block response of the camera. Its functions are refined, improving camera sensitivity and adapting to different scenes. In addition, more efficient information can be obtained by jointly processing the information generated by the photosensitive elements of different spectral responses for the same position in space.

In the present invention, since the image formed by the microlens array 1 is different from the conventional camera imaging result, each microlens 10 sparsely samples the space to perform light collection at different angles to the environment (as shown in FIG. 1). ), and different photosensitive elements have different frequency bands for the spectrum of the response, and therefore, the electrical signals outputted to the photosensitive element array 3 cannot be directly observed and processed, but the imaging results between the different microlenses 10 need to be combined, that is, After the electrical signal is preprocessed by the imaging result processor 4, subsequent operations can be performed, for example, three-dimensional reconstruction, target recognition, motion detection, and the like are performed according to requirements. Specifically:

First, the electrical signal outputted by the data preliminary processing module 41 to the photosensitive pixel array 3 is digital-to-digital converted into a digital signal (of course, the electrical signal can also be converted into a pulse signal or an analog signal for subsequent operations in the bionic neuron. Processing), the signals transmitted from all the photosensitive elements become a digital image array, that is, on the one hand, the characteristics of the microlens array are utilized, that is, each microlens performs sparse sampling on the space to perform light collection at different angles to the environment. Actively calculating the spatial position of the target point, and on the other hand, combining all the image information into different perspective images, and transforming into a general image (as can be seen from FIG. 1, each photosensitive lens 10 has a different position in the corresponding space of the photosensitive element, The pixels collected at the same position of the photosensitive element can be taken out and assembled into a whole picture), and processed to adjust the brightness, white balance, noise, etc. of the image, which is equivalent to the ISP (Image Signal Processing) in the general camera. Image signal processing module;

Then, the preprocessed image information is separately transmitted to the three-dimensional reconstruction module 42, the target recognition module 43, and the motion detection module 44 for subsequent image processing to meet user requirements; for example, using the three-dimensional reconstruction module 42 to calculate the image according to the image information. Three-dimensional data (stereoscopic information) of the object and the environment, and outputting three-dimensional data based on a fixed-eye coordinate system (orthogonal coordinate or polar coordinate); using the target recognition module 43 to detect the properties of the measured object, such as face detection, face Identification, object detection (name of a specific object such as a table, a chair, a cup, etc.), object recognition (who's cup, what kind of cup, where it is produced, etc.); using the motion detection module 44 to detect the relative speed of the body and the environment, The speed of movement of the object, the speed of movement of the plurality of objects in the field of view, and the like.

FIG. 4 shows the imaging of the spatial point by the microlens array 1, wherein one target point in the space is respectively imaged by a plurality of different microlenses, and this property can be utilized to perform the restoration of the light intensity and the calculation of the target distance depth. To provide a basis for subsequent processing modules.

In summary, the present invention collects environmental information through a honeycomb-like microlens array, and uses a light-shielding processing device to optically isolate the microlenses to avoid interference and improve the validity of the imaging data, and at the same time, through the light-shielding processing structure. The photosensitive pixel array under the microlens array is divided into different regions, and different filtering films are used to make different positions in each region correspond to different angles of light, thereby utilizing the relationship between the microlens positions by the imaging result processor. The depth information of the object can be obtained; in addition, since the invention adopts a form of multiple microlens stitching, the overall resolution of the camera with the function of collecting the light field is effectively improved, and the final imaging effect is improved.

The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention, and various modifications may be made to the above-described embodiments of the present invention. The singular and equivalent variations and modifications of the claims and the description of the present invention are intended to fall within the scope of the appended claims. What has not been described in detail in the present invention are all conventional technical contents.

Claims

A microlens array imaging system, characterized in that the system comprises:

a microlens array comprising a plurality of microlenses arranged adjacent to each other;

An opaque light-shielding structure, which is disposed between every two adjacent microlenses and forms a planar or curved lattice structure with a plurality of the microlenses;

a photosensitive pixel array comprising a plurality of photosensitive elements disposed under each of the microlenses;

And an imaging result processor that receives an electrical signal output by the photosensitive pixel array and converts the electrical signal into image information to decode and process the image information.
The microlens array imaging system according to claim 1, wherein each of the microlenses has a hexagonal shape, and the lattice structure is a honeycomb-like lattice structure.
The microlens array imaging system according to claim 1, wherein each of said photosensitive elements is disposed on a focal plane of its corresponding microlens or adjacent to a focal plane of its corresponding microlens.
The microlens array imaging system according to claim 1, wherein a different filter is applied to an upper surface and/or a lower surface of each of the microlenses, and/or a surface of each of the photosensitive elements film.
The microlens array imaging system according to claim 1, wherein the system further comprises the imaging result processor comprising: a data preliminary processing module and a three-dimensional reconstruction module respectively connected to the data preliminary processing module, and target recognition Module and motion detection module.