CN110505379B - High-resolution optical field imaging method - Google Patents
High-resolution optical field imaging method Download PDFInfo
- Publication number
- CN110505379B CN110505379B CN201910737424.9A CN201910737424A CN110505379B CN 110505379 B CN110505379 B CN 110505379B CN 201910737424 A CN201910737424 A CN 201910737424A CN 110505379 B CN110505379 B CN 110505379B
- Authority
- CN
- China
- Prior art keywords
- light field
- sensor
- imaging
- light
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/54—Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/95—Computational photography systems, e.g. light-field imaging systems
- H04N23/951—Computational photography systems, e.g. light-field imaging systems by using two or more images to influence resolution, frame rate or aspect ratio
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/04—Synchronising
Abstract
The invention discloses a high-resolution light field imaging system and a high-resolution light field imaging method. The light field camera based on the micro lens array usually brings certain spatial resolution loss for acquiring the direction information of light rays, and the core idea of the invention is to utilize the high resolution characteristic of two-dimensional imaging to make up the spatial information lost for acquiring the direction information in light field imaging. The method divides scene radiation into two parts through a light splitting light path, simultaneously acquires a traditional imaging image and a light field imaging image by synchronous triggering, and finally obtains a high-resolution light field image by registering and fusing the two images through pre-calibrated sensor position information. The method can improve the spatial resolution of the traditional light field imaging and has the advantages of simple system and high integration level.
Description
Technical Field
The invention relates to a light field imaging system and a method, in particular to a high-resolution light field imaging system and a method, which are a system and a method for simultaneously acquiring and fusing a high-resolution two-dimensional image and a light field image in an equivalent coaxial light path.
Background
In the field of optical imaging, a light field camera serving as brand-new computing imaging equipment has outstanding depth of field expansion and passive distance detection capability. Among many light field imaging devices, a light field camera based on a microlens array has attracted attention because of its advantages such as high integration and portability. However, the optical field camera sacrifices the spatial resolution to acquire the angular information of the light, which greatly limits the practical application of the optical field camera. How to acquire a light field image with high spatial resolution is always a research focus and a hotspot in the field, and currently, main solutions include: reducing the aperture of the micro-lens array, and using a focused light field camera structure to reduce the directional sampling number. However, the reduction of the aperture means that the diffraction effect is enhanced, and the requirements on the process are increased sharply; the reduction in the number of directional samples means a reduction in depth resolution and increases the complexity of the processing algorithm. Conventional imaging techniques have been able to achieve very high spatial resolution, but they only record the cumulative intensity of the light during the acquisition and are unable to record the directional information of the light. If a high-resolution two-dimensional image is fused with a light field image, a high-spatial-resolution image can be acquired while a high directional sampling rate is maintained.
Fusing two-dimensional images and light-field images faces two problems: how to collect and process. Directly binding a traditional camera and a light field camera for shooting is a feasible scheme, but the following problems exist:
1. the field of view coincidence is small: because the optical centers of the cameras cannot be superposed, the fields of view of the cameras and the optical centers of the cameras are translated, so that the non-superposed regions cannot be fused, and in order to determine the non-superposed regions, separate image registration is required;
2. non-parallel optical axis brought by installation error: the non-parallel optical axes can cause affine transformation between the two images, thereby bringing about huge additional calculation;
3. the imaging primary mirror parameters are difficult to be consistent: the primary mirror of the camera is independently adjusted, so that the parameters cannot be guaranteed to be consistent, the image sizes are inconsistent, and extra view field misalignment is caused;
4. the system has large volume and high complexity: many elements in two cameras function coincided, and the redundancy is high, the volume is also comparatively huge after the combination as an organic whole.
Therefore, new structures and methods need to be proposed to meet the requirement of two-dimensional image and light field image fusion.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: in the field of light field imaging, it is a future development trend to fuse a two-dimensional optical image and a light field image and simultaneously acquire a light field image with high angular resolution and high spatial resolution, but the fusion of simple independent images has a great problem. Aiming at the defects, the invention provides a brand new imaging system and method, the system fully considers the redundancy problem of imaging elements, and adds prior constraint to registration fusion by adopting a light-splitting light path structure, so that the simplicity and the reliability of the system are improved, and the coincidence of a view field and an optical axis and the like are ensured to the maximum extent.
The scheme adopted by the invention for solving the technical problems is as follows: a high-resolution light field imaging system is characterized in that a light splitting light path is adopted to enable two-dimensional imaging and light field imaging to share the same primary mirror and the same optical axis, instantaneous two-dimensional images and light field images are simultaneously acquired by synchronous triggering, and registration fusion is carried out based on calibrated sensor information. The system and the method can acquire the light field image with high spatial resolution and high angular resolution, and the method is characterized by comprising the following parts:
the light field imaging assembly consists of an imaging main lens and a light field sensor and is used for collecting light field images, wherein the light field sensor consists of a micro lens array and a two-dimensional image sensor;
the two-dimensional high-resolution imaging assembly consists of an imaging main lens and a two-dimensional image sensor and is used for acquiring a two-dimensional image with high spatial resolution;
the imaging coupling component mainly comprises a 45-degree spectroscope and a sensor placing part, so that the two sensors are positioned at the same equivalent image distance, and are ensured to be perpendicular to the optical axis of the main lens, and the two-dimensional image and the light field image only have perspective transformation;
the synchronous triggering assembly is essentially a signal generator and is used for triggering the two-dimensional sensor and the light field sensor to synchronously acquire, so that the imaging capability of a dynamic scene is improved.
Furthermore, the system is a whole, the field of view of the light field sensor is consistent with that of the two-dimensional image sensor, and the sensor resources are not wasted while the field of view of the light field sensor is coincident with that of the two-dimensional image sensor.
Further, the system uses the light field sensor as a microlens array based light field sensor, which is composed of a microlens array and an image sensor placed in parallel at the focal length of the next microlens.
Further, the main lens and the light field sensor of the system maintain strict F number matching.
Furthermore, the two sensors are externally connected to the same trigger, so that the two sensors can acquire scene images at the same moment.
The principle of the invention is as follows: the spatial information lost for acquiring direction information in the light field imaging process is compensated by utilizing the high-resolution characteristic of two-dimensional imaging, the two sensors share the same primary mirror by using a light splitting optical path to ensure that a view field and an optical axis are overlapped, and finally, images are simultaneously acquired by a synchronous trigger.
The invention has the following advantages: the invention realizes the acquisition of the light field image and the two-dimensional image simultaneously by utilizing the coaxial light splitting optical path, ensures the high consistency of two imaging conditions such as parameters of a view field, an optical axis, a vertical axis magnification and the like by sharing the main lens, and can improve the utilization rate of information in the fusion process. The invention has the advantages of simple system, high integration level and the like, and provides a simplified design idea for improving the spatial resolution of the micro-lens array light field camera.
Drawings
FIG. 1 is a block diagram of a high resolution light field imaging system according to the present invention;
fig. 2 is a schematic diagram of the working process of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more clearly apparent, the present invention is further described in detail with reference to the schematic drawings of the working process of the present invention.
The invention relates to a high-resolution light field imaging system, which comprises:
the light field imaging assembly consists of an imaging main lens and a light field sensor and is used for collecting light field images, wherein the light field sensor consists of a micro lens array and a two-dimensional image sensor;
the two-dimensional high-resolution imaging assembly consists of an imaging main lens and a two-dimensional image sensor and is used for acquiring a two-dimensional image with high spatial resolution;
the imaging coupling component mainly comprises a 45-degree spectroscope and a sensor placing part, so that the two sensors are positioned at the same equivalent image distance, and are ensured to be perpendicular to the optical axis of the main lens, and the two-dimensional image and the light field image only have perspective transformation;
the synchronous triggering assembly is essentially a signal generator and is used for triggering the two-dimensional sensor and the light field sensor to synchronously acquire, so that the imaging capability of a dynamic scene is improved.
The system is an integral whole, the field of view of the light field sensor is consistent with that of the two-dimensional image sensor, and the sensor resources are not wasted while the field of view of the light field sensor is coincident with that of the two-dimensional image sensor.
The system uses a light field sensor which is a light field sensor based on a micro lens array and consists of the micro lens array and an image sensor which is arranged in parallel at the focal length of the next micro lens.
The main lens and the light field sensor of the system maintain strict F number matching.
The two sensors are externally connected to the same trigger, and scene images of the two sensors at the same moment can be acquired.
As shown in fig. 2, the high-resolution light field imaging system provided by the present invention comprises an imaging main lens a, a reflection-transmittance ratio of 1: 1, a spectroscope B, a two-dimensional image sensor C and a light field sensor D. The light field sensor D is composed of a conventional two-dimensional sensor and a microlens array placed in front of it at the focal length of a microlens. Firstly, calculating an image distance b by using a Gaussian imaging formula according to a preset working distance a and a main lens focal length f:
and then installing the main lens A and the light field sensor D according to the calculated distance. If the size of the light-sensitive area of the light field sensor is W, the size of the spectroscope should be larger than that of the light-sensitive area
And the beam splitter is mounted at a 45 degree angle to the optical axis. Finally, a two-dimensional image sensor is arranged perpendicular to the optical axis of the light path after light splitting, and the equivalent distance between the two-dimensional image sensor and the main lens is consistent with the distance between the light field sensor and the main lens, namely c is satisfied1+c2＝b。
Taking an imaging process of an actual object O as an example, a specific working flow of the present invention is as follows:
(1) and (3) calibrating the relative positions of the two sensors by using a checkerboard to obtain a transformation matrix R from the light field central subaperture diagram to the two-dimensional image.
(2) When an actual scene is imaged, light rays radiated by an object O firstly enter a main lens A and then reach a spectroscope B, and are divided into two parts.
(3) After the parameters such as the imaging angle, the field of view and the like are adjusted, the synchronous trigger E is utilized to respectively trigger the two-dimensional sensor and the light field sensor to simultaneously image, and two images are obtained: o ', O'.
(4) And calculating to obtain the light intensity distribution function of different angles of each spatial position through the light field image.
(5) And transforming the central sub-aperture map of the light field image to a two-dimensional image coordinate by using the transformation matrix R, and interpolating each space position of the two-dimensional image by using the light ray angle information in the corresponding light field image, so that the high spatial resolution can be reserved and the light ray angle information can be acquired.
The invention has not been described in detail and is part of the common general knowledge of a person skilled in the art.
Claims (1)
1. A high resolution light field imaging method, characterized by: the high-resolution light field imaging system comprises a light field imaging component, a light field sensor and a control module, wherein the light field imaging component consists of an imaging main lens and a light field sensor and is used for collecting light field images, and the light field sensor consists of a micro lens array and a two-dimensional image sensor; the two-dimensional high-resolution imaging assembly consists of an imaging main lens and a two-dimensional image sensor and is used for acquiring a two-dimensional image with high spatial resolution; the imaging coupling component mainly comprises a 45-degree spectroscope and a sensor placing part, so that the two sensors are positioned at the same equivalent image distance, and are ensured to be perpendicular to the optical axis of the main lens, and the two-dimensional image and the light field image only have perspective transformation; the synchronous trigger component is essentially a signal generator and is used for triggering the two-dimensional sensor and the light field sensor to synchronously acquire so as to improve the imaging capability of a dynamic scene; the system is an integral body, the field of view of the light field sensor is consistent with that of the two-dimensional image sensor, so that the field of view of the light field sensor is coincident with that of the two-dimensional image sensor, and sensor resources are not wasted; the system uses the light field sensor as a light field sensor based on a micro-lens array, which consists of a micro-lens array and a block arranged in parallel behind the micro-lens arrayAn image sensor at the focal length of the microlens; the main lens and the light field sensor of the system keep strict F number matching; the two sensors are externally connected to the same trigger, so that the two sensors can acquire scene images at the same moment, and the high-resolution light field imaging system is characterized in that an imaging main lens A and a reflectance-transmittance ratio are 1: 1, wherein the light field sensor D consists of a conventional two-dimensional sensor and a microlens array placed in front of the conventional two-dimensional sensor at a microlens focal length, and firstly, according to a predetermined working distance a and a main lens focal length f, an image distance B is calculated by using a gaussian imaging formula:
then, the main lens A and the light field sensor D are installed according to the calculated distance, and the size of the light sensitive area of the light field sensor is set to be W, then the size of the spectroscope is larger than that of the spectroscope
And the spectroscope is arranged at an included angle of 45 degrees with the optical axis, finally, the two-dimensional image sensor is arranged perpendicular to the optical axis of the light path after light splitting, and the equivalent distance from the two-dimensional image sensor to the main lens is consistent with the distance between the light field sensor and the main lens, namely, the c requirement is met1+c2B, the method simultaneously acquires a traditional imaging image and a light field imaging image through a light splitting optical path, and fuses the two images based on a fusion registration algorithm to acquire a high-resolution light field image, and specifically comprises the following steps:
step (1), calibrating the relative positions of two sensors by using a checkerboard to obtain a transformation matrix R from a light field central subaperture diagram to a two-dimensional image;
step (2), when an actual scene is imaged, light rays radiated by an object O firstly enter a main lens A and then reach a spectroscope B, and are divided into two parts;
and (3) after adjusting parameters such as an imaging angle, a field of view and the like, respectively triggering the two-dimensional sensor and the light field sensor to simultaneously image by using the synchronous trigger E to obtain two images: o', O ";
step (4), calculating through the light field image to obtain light intensity distribution functions of different angles of each space position;
and (5) transforming the central sub-aperture map of the light field image to a two-dimensional image coordinate by using a transformation matrix R, and interpolating each space position of the two-dimensional image by using light angle information in the corresponding light field image, so that the high spatial resolution can be reserved and the light angle information can be acquired.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910737424.9A CN110505379B (en) | 2019-08-09 | 2019-08-09 | High-resolution optical field imaging method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910737424.9A CN110505379B (en) | 2019-08-09 | 2019-08-09 | High-resolution optical field imaging method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110505379A CN110505379A (en) | 2019-11-26 |
| CN110505379B true CN110505379B (en) | 2021-06-01 |
Family
ID=68587144
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910737424.9A Active CN110505379B (en) | 2019-08-09 | 2019-08-09 | High-resolution optical field imaging method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110505379B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110906883B (en) * | 2019-12-02 | 2021-09-07 | 中国科学院光电技术研究所 | High-resolution three-dimensional detection method integrating multi-view vision and synthetic aperture imaging |
| CN110928113B (en) * | 2019-12-03 | 2021-10-08 | 西北工业大学 | Light field acquisition device with variable spatial resolution |
| CN113225492B (en) * | 2021-02-05 | 2023-07-28 | 深圳臻像科技有限公司 | Light field unit image generation method, system and storage medium |
| CN113538300B (en) * | 2021-06-22 | 2023-07-14 | 武汉工程大学 | High-definition light field imaging method based on liquid crystal D-type optical element |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103471715A (en) * | 2013-09-02 | 2013-12-25 | 北京航空航天大学 | Common optical path combined optical field spectral imaging method and device |
| CN106921824A (en) * | 2017-05-03 | 2017-07-04 | 丁志宇 | Circulating type mixes light field imaging device and method |
| CN108200312A (en) * | 2017-12-12 | 2018-06-22 | 中北大学 | A kind of light-field camera |
| CN108540701A (en) * | 2018-05-25 | 2018-09-14 | 北京理工大学深圳研究院 | One kind being based on the hybrid light-field camera device of microlens array |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5464891B2 (en) * | 2009-04-13 | 2014-04-09 | キヤノン株式会社 | Optical image acquisition apparatus provided with adaptive optical system, and control method thereof |
| US10057498B1 (en) * | 2013-03-15 | 2018-08-21 | Cognex Corporation | Light field vision system camera and methods for using the same |
| EP2835613B1 (en) * | 2013-08-08 | 2019-01-16 | Hexagon Technology Center GmbH | Geodesic measuring device with a microlens array |
| CN104698467B (en) * | 2015-03-26 | 2017-06-23 | 中国人民解放军济南军区72465部队 | Multiple target pulse laser distance measuring device and method on a kind of different sight lines |
-
2019
- 2019-08-09 CN CN201910737424.9A patent/CN110505379B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103471715A (en) * | 2013-09-02 | 2013-12-25 | 北京航空航天大学 | Common optical path combined optical field spectral imaging method and device |
| CN106921824A (en) * | 2017-05-03 | 2017-07-04 | 丁志宇 | Circulating type mixes light field imaging device and method |
| CN108200312A (en) * | 2017-12-12 | 2018-06-22 | 中北大学 | A kind of light-field camera |
| CN108540701A (en) * | 2018-05-25 | 2018-09-14 | 北京理工大学深圳研究院 | One kind being based on the hybrid light-field camera device of microlens array |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110505379A (en) | 2019-11-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110505379B (en) | High-resolution optical field imaging method | |
| CN103471715B (en) | A kind of light path combined type light field spectrum imaging method and device altogether | |
| US9048153B2 (en) | Three-dimensional image sensor | |
| EP3098638B1 (en) | Adaptive autofocusing system | |
| US6744569B2 (en) | Method and apparatus for omnidirectional three dimensional imaging | |
| US6304285B1 (en) | Method and apparatus for omnidirectional imaging | |
| CN109859272B (en) | Automatic focusing binocular camera calibration method and device | |
| CN107343130A (en) | High dynamic imaging module based on DMD dynamic light splitting | |
| US8908054B1 (en) | Optics apparatus for hands-free focus | |
| CN109325981B (en) | Geometric parameter calibration method for micro-lens array type optical field camera based on focusing image points | |
| CN106210489B (en) | A kind of image acquiring device based on minisize pick-up head matrix | |
| CN103091258B (en) | A kind of multi-spectral imager based on liquid zoom technology | |
| JP2006330735A (en) | Apparatus for providing panoramic stereo image with single camera | |
| EP3513550B1 (en) | Flat digital image sensor | |
| AU2020413529B2 (en) | Method and system for calibrating light field camera without white images | |
| US20070273894A1 (en) | Method and apparatus for remote spatial calibration and imaging | |
| CN108924408B (en) | Depth imaging method and system | |
| CN108881717B (en) | Depth imaging method and system | |
| CN108833909B (en) | Dual-channel focal plane registration adjusting device and method | |
| CN111263134A (en) | Positionable panoramic three-dimensional imaging system and positioning method | |
| CN105430298A (en) | Method for simultaneously exposing and synthesizing HDR image via stereo camera system | |
| WO2019048904A1 (en) | Combined stereoscopic and phase detection depth mapping in a dual aperture camera | |
| CN108805921A (en) | Image-taking system and method | |
| CN111127379B (en) | Rendering method of light field camera 2.0 and electronic equipment | |
| CN108924407B (en) | Depth imaging method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |