CN115514950A

CN115514950A - Microlens light field collecting method, system, electronic device and computer readable medium

Info

Publication number: CN115514950A
Application number: CN202210875254.2A
Authority: CN
Inventors: 许春涛; 初宇飞; 李泉; 常成洋
Original assignee: Inspur Communication Information System Co Ltd
Current assignee: Inspur Communication Information System Co Ltd
Priority date: 2022-07-25
Filing date: 2022-07-25
Publication date: 2022-12-23

Abstract

The invention discloses a micro-lens light field acquisition method, a micro-lens light field acquisition system, an electronic device and a computer readable medium, belongs to the technical field of light field imaging, and aims to solve the technical problem of how to perform three-dimensional reconstruction on a target object. The method comprises the following steps: creating a target in 3Ds Max software; calling a camera photographing function of 3DsMax, and sampling and imaging a target object through a micro lens array with the number of camera analog arrays of N x N; calling a camera to capture an image of a target object based on a camera photographing function of 3DsMax, wherein each movement of the camera corresponds to a single micro lens in the micro lens array, and obtaining N × N unit images with the same size; splicing the N x N unit images into one image to obtain a spliced image; in the 3Ds Max software, for each coordinate position, an image synthesis sub-aperture image corresponding to the target position is extracted from each unit image, one sub-aperture image corresponding to each coordinate position.

Description

Microlens light field collecting method, system, electronic device and computer readable medium

Technical Field

The invention relates to the technical field of light field imaging, in particular to a method and a system for acquiring a microlens light field, an electronic device and a computer readable medium.

Background

With the development of society and the popularization of a large number of digital products, the multidimensional visual information acquisition technology is concerned by people in the fields of medical images, satellite mapping, mobile phone cameras and the like. However, conventional imaging suffers from a number of drawbacks. First, the conventional imaging technology can only obtain information on one object plane, which is the projection information of the spatial three-dimensional light on the two-dimensional plane, thereby losing optical information on other dimensions. Secondly, the traditional imaging can only detect two-dimensional intensity information on a single object plane, and in order to obtain clear images of different depths, the clear images on the object plane can only be obtained in a mechanical focusing mode, so that the real-time performance of information acquisition is influenced.

How to carry out three-dimensional reconstruction on a target object is a technical problem to be solved.

Disclosure of Invention

The technical task of the invention is to provide a micro-lens light field acquisition method, a micro-lens light field acquisition system, an electronic device and a computer readable medium, aiming at the defects, so as to solve the technical problem of how to carry out three-dimensional reconstruction on a target object.

The invention relates to a micro-lens light field acquisition method, which simulates a micro-lens array to sample an image of a target object in 3Ds Max, and comprises the following steps:

creating a target in 3Ds Max software;

calling a camera photographing function of 3DsMax, setting camera parameters, setting a photographing track of the camera to be a rectangle, and sampling and imaging a target object through a micro-lens array with the camera analog array number of N x N;

calling a camera to capture an image of a target object based on a camera photographing function of 3DsMax, wherein each movement of the camera corresponds to a single micro lens in the micro lens array, and obtaining N × N unit images with the same size;

splicing the N-by-N unit images into one image to obtain a spliced image;

in the 3Ds Max software, the position coordinates between the unit images are consistent for the N × N unit images, and for each coordinate position, an image synthesis sub-aperture image corresponding to the target position is extracted from each unit image, and each coordinate position corresponds to one sub-aperture image.

Preferably, the camera parameters include focal length, field of view, aperture and distance range;

the camera photographing track comprises a starting point coordinate, an end point coordinate and a moving step length, wherein the moving step length is the central distance between every two adjacent micro lenses in each row in the micro lens array.

Preferably, the camera photographing function based on 3DsMax calls the camera to capture the image of the object and sets the ratio of the image to 1:1, image pixel size is N x N.

In a second aspect, the present invention provides a microlens light field collecting system for simulating and collecting light field images by the microlens light field collecting method according to any one of the first aspect, the system including:

an object building module to create an object in 3Ds Max software; (ii) a

The micro lens array configuration module is used for calling a camera photographing function of 3DsMax, setting camera parameters and setting a camera photographing track to be a rectangle so as to sample and image a target object through the micro lens array with the number of the camera analog array being N x N;

the acquisition module is used for calling a camera to capture an image of a target object based on a camera photographing function of 3DsMax, and each movement of the camera corresponds to a single micro lens in the micro lens array to obtain N × N unit images with the same size;

the splicing module is used for splicing the N x N unit images into one image to obtain a spliced image;

and the sub-aperture image construction module is used for extracting an image synthesis sub-aperture image corresponding to the target position from each unit image for each coordinate position, and each coordinate position corresponds to one sub-aperture image.

Preferably, the acquisition module is used for calling a camera to capture an image of the object based on a camera photographing function of 3DsMax, and setting the ratio of the image to be 1:1, the image pixel size is N.

In a third aspect, the electronic device of the present invention includes: at least one memory and at least one processor;

the at least one memory to store a machine readable program;

the at least one processor, configured to invoke the machine readable program, to perform the method of any of claims 1 to 3.

In a fourth aspect, the present invention provides a computer readable medium having stored thereon computer instructions, which, when executed by a processor, cause the processor to perform the method of any of the first aspects.

The microlens light field collecting method, the microlens light field collecting system, the electronic device and the computer readable medium have the following advantages:

1. on the basis of a light field imaging theory, 3Ds Max software is heuristically used for combining a micro lens array with a traditional imaging system to design a light field imaging system, and light field images are simulated and collected, so that crosstalk-free, high-resolution and rapid acquisition of the light field images are realized, and a high-efficiency simulation means is provided for the research of a light field imaging technology;

2. and processing the collected light field image, extracting an image synthesis sub-aperture image corresponding to the target position from each unit image for each coordinate position, realizing reconstruction of the light field image, and obtaining the distance from the three-dimensional object to the camera according to parallax information in the follow-up process according to slight differences as can be seen by comparing the sub-aperture images, wherein the subspace images at different view angles are slightly different.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

The invention is further described below with reference to the accompanying drawings.

FIG. 1 is a block flow diagram of a microlens light field collection method according to embodiment 1;

fig. 2 is a schematic diagram of optical field acquisition in 3Ds Max software in the microlens optical field acquisition method according to embodiment 1;

FIG. 3 is a modeling diagram of the target object at different viewing angles in the 3Ds Max software in the microlens light field collection method according to embodiment 1;

FIG. 4 is a schematic diagram of a photographing sequence in the 3Ds Max software in the microlens light field collecting method according to embodiment 1;

FIG. 5 is a partial image array collected in 3Ds Max software in a microlens light field collection method according to embodiment 1;

fig. 6 is a combined light field pattern and a partial enlarged view in the microlens light field collecting method according to embodiment 1;

FIG. 7 is a schematic diagram of the synthesis of a neutron aperture image in the microlens light field acquisition method according to example 1;

fig. 8 shows the image reconstruction results of different viewing angles in the microlens light field collecting method according to embodiment 1.

Detailed Description

The present invention is further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and can implement the present invention, but the embodiments are not intended to limit the present invention, and the embodiments and technical features of the embodiments can be combined with each other without conflict.

The embodiment of the invention provides a micro-lens light field acquisition method, a micro-lens light field acquisition system, an electronic device and a computer readable medium, which are used for solving the technical problem of how to carry out three-dimensional reconstruction on a target object.

Example 1:

s100, creating a target object in 3Ds Max software;

s200, calling a camera photographing function of 3DsMax, setting camera parameters, setting a camera photographing track to be a rectangle, and sampling and imaging a target object through a micro-lens array with the camera analog array number of N x N;

s300, calling a camera to capture an image of a target object based on a camera photographing function of 3DsMax, wherein each movement of the camera corresponds to a single micro lens in the micro lens array, and obtaining N × N unit images with the same size;

s400, splicing the N unit images into one image to obtain a spliced light field image;

and S500, in 3Ds Max software, position coordinates among the unit images are consistent for the N × N unit images, for each coordinate position, an image synthesis sub-aperture image corresponding to the target position is extracted from each unit image, and each coordinate position corresponds to one sub-aperture image.

The method of the embodiment establishes a three-dimensional target model by using 3Ds Max, realizes non-crosstalk, high-resolution and quick acquisition of the microlens light field image by establishing a virtual microlens array, and provides a high-efficiency simulation means for simulation research of the light field imaging technology. And processing the collected images to realize the reconstruction of images with different visual angles.

Based on the light field imaging technology of the microlens array, partial information of a target object is recorded from different directions by each unit lens in the microlens array, and parallax information at any point in the target object space is distributed in the whole image sensor plane and is stored and recorded.

The light field imaging system based on the micro lens array records image information of different sampling positions through the main lens through the micro lens array, and can be equivalently regarded as that a micro camera array is arranged on the plane of an optical system. The 3Ds Max software is software for three-dimensional animation scene production, and camera parameters and tracks are set in the 3Ds Max software to simulate the photographing function of the micro-lens array. FIG. 2 is a schematic diagram of the light field acquisition in 3Ds Max software.

In step S100, based on the microlens light field collection principle of 3Ds Max, a three-dimensional model is created in 3Ds Max, and the model simulates the light field imaging collection process.

The set is a specific case, in the model, three stereoscopic chinese character texts ('permit', 'spring', 'billow') are created as three-dimensional objects for acquisition, wherein coordinates of 'permit' in the model are (25mm, 10mm, 43mm), 'spring' in the model are (33mm, 25mm, 28mm), and coordinates of 'billow' in the model are (40 mm, -2mm, 15mm). The placement positions of the three Chinese character texts in the 3Ds Max software are shown in FIG. 3, and FIG. 3 (a) and FIG. 3 (b) are different view angle diagrams in the 3Ds Max software respectively.

Based on the virtual microlens array established in the step S200 in 3DsMax, camera parameters are set, and a camera photographing track is set to be rectangular, the camera photographing track includes a start coordinate, an end coordinate, and a moving step, and the moving step is a central distance between two adjacent microlenses in each row in the microlens array.

As a specific case, a "free camera" is placed in front of the three texts, simulating the sampled imaging of the main lens, i.e. the corresponding directional information under the micro-lens. The photographing track of the free camera is set to be rectangular, wherein the starting point is (0 mm, -12.5mm, 0mm), the end point is (0 mm, -12.5mm, 48mm), the moving step is 240um, and the number of the analog arrays is 200 x 200 of the microlens arrays. The photographing sequence of the camera is shown in fig. 3. The basic parameters of the camera are set as follows: focus 228.6mm, field of view 10 degrees, and distance range 45mm. In the simulation process, the parameters of the camera parameters corresponding to a single microlens are: the focal length is 228.6mm, and the caliber is 240um.

In step S300, using the camera photographing function of the 3Ds Max software, an image of a three-dimensional object is captured with a camera, and the scale of the image and the pixel size of the image are set to 1:1, image pixel size 200 x 200. Each movement of the camera corresponds to a single microlens in the microlens array, each row in fig. 4 is an image formed by adjacent sub-lenses, and the images formed by each microlens are slightly different due to the different viewing angles of the adjacent lenses to the same position of the object.

The N × N unit images obtained as described above are all single, and image processing is not convenient, so that it is necessary to integrate these images into one image through step S400 to obtain a spliced light field image.

As a specific case, an algorithm is written in Matlab, and 200 × 200 images are processed to obtain a spliced image, as shown in fig. 5.

Step S500 is to further process the spliced light field images, and extract images with the same position coordinates of each unit image in the spliced light field images to synthesize a sub-aperture image, where a schematic diagram of the sub-aperture image synthesis is shown in fig. 6.

As a specific case, the spliced images obtained by processing in Matlab are further processed, the same point of each image in the spliced images is extracted to synthesize a sub-aperture image, and a schematic diagram of sub-aperture image synthesis is shown in fig. 6.

According to the synthesis process of the sub-aperture images, a program is written in Matlab, and the light field images obtained in 3Ds Max software are processed to obtain sub-aperture images with different view angles, as shown in FIG. 7.

It can be observed from fig. 8 (a), (b), (c) and (d) that the pixel coordinates are different at the same position of the kanji 'let', i.e. the reconstructed images have different viewing angles for different positions of the same object.

Example 2:

the invention discloses a micro-lens light field acquisition system, which comprises a target object construction module, a micro-lens array configuration module, an acquisition module, a splicing module and a sub-aperture image construction module, wherein the system simulates and acquires a light field image to reconstruct an image by the method disclosed in embodiment 1.

The system establishes a three-dimensional target model by using 3Ds Max, realizes non-crosstalk, high-resolution and quick acquisition of the microlens light field image by establishing a virtual microlens array, and provides a high-efficiency simulation means for simulation research of the light field imaging technology. And processing the collected images to realize the reconstruction of images with different visual angles.

The object building module is used to create objects in the 3Ds Max software. In the execution process, the object construction module creates a three-dimensional model in the 3Ds Max based on the micro-lens light field acquisition principle of the 3Ds Max, and the model simulates the light field imaging acquisition process.

The micro-lens array configuration module is used for calling a camera photographing function of 3DsMax, setting camera parameters and setting a camera photographing track into a rectangle, and sampling and imaging a target object through the micro-lens array with the camera analog array number of N x N.

The camera parameters comprise focal length, visual field, caliber and long-distance range; the camera photographing track comprises a starting point coordinate, an end point coordinate and a moving step length, wherein the moving step length is the central distance between every two adjacent micro lenses in each row in the micro lens array.

In the specific implementation process, a free camera is arranged in front of the model to simulate the sampling imaging of the main lens, namely the corresponding direction information under the micro lens. And setting a photographing track of the free camera to be a rectangle, and setting a starting point coordinate, an end point coordinate and a moving step length, namely the micro lens array of the analog array number N x N. And setting basic parameters of the camera as follows: focal length, field of view, distance range. In the simulation process, the parameters of the camera parameters corresponding to a single microlens are: focal length, aperture (corresponding to the moving step length).

The acquisition module is used for calling a camera to capture an image of a target object based on a camera photographing function of 3DsMax, and each movement of the camera corresponds to a single micro lens in the micro lens array to obtain N x N unit images with the same size.

The acquisition module is used for calling a camera to capture an image of a target object based on a camera photographing function of 3DsMax, and setting the proportion of the image as 1:1, the image pixel size is N.

And the splicing module is used for splicing the N x N unit images into one image to obtain a spliced light field image. In a specific implementation process, an algorithm can be written in Matlab, and the N × N images are processed to obtain a spliced image.

In the 3Ds Max software, for the N × N unit images, the position coordinates between the unit images are consistent, and for each coordinate position, the sub-aperture image construction module extracts an image synthesis sub-aperture image corresponding to the target position from each unit image, and each coordinate position corresponds to one sub-aperture image. In specific implementation, a spliced image obtained through processing in Matlab can be further processed, and a sub-aperture image is synthesized at the same point of each image in the spliced image.

Example 3:

the invention provides an electronic device, comprising: at least one memory and at least one processor; the at least one memory to store a machine readable program; the at least one processor is used for calling the machine readable program and executing the method disclosed in the embodiment 1.

Example 4:

the invention is a computer readable medium having stored thereon computer instructions which, when executed by a processor, cause the processor to perform a method as disclosed in any of the embodiments of the invention. Specifically, a system or an apparatus equipped with a storage medium on which software program codes that realize the functions of any of the embodiments described above are stored may be provided, and a computer (or a CPU or MPU) of the system or the apparatus is caused to read out and execute the program codes stored in the storage medium.

In this case, the program code itself read from the storage medium can realize the functions of any of the above-described embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

Examples of the storage medium for supplying the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (e.g., CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD + RW), a magnetic tape, a nonvolatile memory card, and a ROM. Alternatively, the program code may be downloaded from a server computer via a communications network.

Further, it should be clear that the functions of any one of the above-described embodiments may be implemented not only by executing the program code read out by the computer, but also by causing an operating system or the like operating on the computer to perform a part or all of the actual operations based on instructions of the program code.

Further, it is to be understood that the program code read out from the storage medium is written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion unit connected to the computer, and then a CPU or the like mounted on the expansion board or the expansion unit is caused to perform part or all of the actual operations based on instructions of the program code, thereby realizing the functions of any of the embodiments described above.

It should be noted that not all steps and modules in the above flows and system structure diagrams are necessary, and some steps or modules may be omitted according to actual needs. The execution order of the steps is not fixed and can be adjusted as required. The system structure described in the above embodiments may be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by a plurality of physical entities, or some components in a plurality of independent devices may be implemented together.

In the above embodiments, the hardware unit may be implemented mechanically or electrically. For example, a hardware element may comprise permanently dedicated circuitry or logic (such as a dedicated processor, FPGA or ASIC) to perform the corresponding operations. The hardware elements may also comprise programmable logic or circuitry, such as a general purpose processor or other programmable processor, that may be temporarily configured by software to perform the corresponding operations. The specific implementation (mechanical, or dedicated permanent, or temporarily set) may be determined based on cost and time considerations.

While the invention has been particularly shown and described with reference to the preferred embodiments and drawings, it is not intended to be limited to the specific embodiments disclosed, and it will be understood by those skilled in the art that various other combinations of code approval means and various embodiments described above may be made, and such other embodiments are within the scope of the present invention.

Claims

1. A microlens light field collection method is characterized in that a microlens array is simulated in 3Ds Max to sample an image of a target object, and the method comprises the following steps:

creating a target in 3Ds Max software;

splicing the N unit images into one image to obtain a spliced light field image;

2. A microlens light field collection method as in claim 1, wherein said camera parameters include focal length, field of view, aperture and distance range;

3. The microlens light field collecting method as claimed in claim 1, wherein the camera is invoked to capture an image of the object based on the camera photographing function of 3DsMax, and the ratio of the image is set to 1:1, image pixel size is N x N.

4. A microlens light field acquisition system characterized by simulating acquisition of a light field image by the microlens light field acquisition method as claimed in any one of claims 1 to 3, the system comprising:

an object build module to create an object in 3Ds Max software;

the splicing module is used for splicing the N x N unit images into one image to obtain a spliced light field image;

and a sub-aperture image construction module, in 3Ds Max software, for the N × N unit images, the position coordinates between the unit images are consistent, for each coordinate position, the sub-aperture image construction module extracts an image synthesis sub-aperture image corresponding to the target position from each unit image, and each coordinate position corresponds to one sub-aperture image.

5. A microlens light field acquisition system as in claim 4, wherein said camera parameters include focal length, field of view, aperture and distance range;

6. The microlens light field acquisition system according to claim 4, wherein the acquisition module is configured to invoke a camera to capture an image of a target based on a camera photographing function of 3DsMax, and set a ratio of the image to 1:1, image pixel size is N x N.

7. An electronic device, comprising: at least one memory and at least one processor;

the at least one memory to store a machine readable program;

8. A computer readable medium having computer instructions stored thereon, which when executed by a processor, cause the processor to perform the method of any of claims 1 to 3.