CN115222657A - Lens aberration prediction method, device, electronic device and storage medium - Google Patents

Abstract

The present application relates to the field of image data processing technologies, and in particular, to a method and an apparatus for predicting an aberration of a lens, an electronic device, and a storage medium, where the method includes: acquiring pictures of a plurality of visual angles, wherein the pictures of the plurality of visual angles are acquired by the same imaging system and comprise different angle domain information; the method comprises the steps of inputting pictures of a plurality of visual angles into a preset neural network, extracting at least one visual angle characteristic of each visual angle, and predicting aberration brought by an optical lens or a shooting environment based on the at least one visual angle characteristic. Therefore, the technical problems of complex simulation propagation process, high calculation complexity and poor robustness caused by the physical optics in the related technology are solved.

Description

Lens aberration prediction method, device, electronic device and storage medium

technical field

The present application relates to the technical field of image data processing, and in particular, to a lens aberration prediction method, device, electronic device and storage medium.

Background technique

2D imaging sensors have revolutionized almost all fields, including industrial inspection, mobile devices, autonomous driving, surveillance, medical diagnostics, biology, and astronomy. Benefiting from the rapid development of the semiconductor industry, the pixel size of digital sensors has rapidly increased in the past decade. Growing, however, the practical performance of most imaging systems has reached the bottleneck of optics rather than electronics, and imperfect lenses or environmental disturbances can cause optical aberrations that affect the accuracy of imaging.

AO (Adaptive optics) achieves active aberration correction through deformable mirror arrays or spatial light modulators. However, current AO systems are usually complex, bulky, and expensive, and cannot be applied to lightweight systems or portable devices.

The proposal of scanning light field imaging provides a solution for aberration estimation on lightweight systems or portable devices. The related technology is based on the DAO (Digital Adaptive optics, digital AO) algorithm of wave optics, which can achieve a large SBP ( space-bandwidth product) for robust, versatile and high-performance 3D imaging. Specifically, the related technology can physically simulate the forward and reverse processes of light propagation, set the initial aberration value, calculate the lens point spread function, and then calculate the high-resolution image through the deconvolution process, and then pass the image and point The spread function calculation should compare the captured image with the existing sensor signal, and finally correct the aberration, and iterate continuously until convergence.

However, the related technology is based on physical optics, the forward-backward propagation process is complex, the computer simulation time is long, the convergence is uncertain, the iterative methods are different for different processes, the iterative process is different, and the robustness is poor; the deconvolution process is time-consuming and energy-consuming , the iteration makes this process unavoidable and difficult to parallelize; based on simple neural networks, such as 3D CNN, the network is overburdened in the whole process, the performance of the network prediction point spread function or aberration is unstable, or the accuracy is low , resulting in a poor reconstruction effect, which needs to be improved urgently.

SUMMARY OF THE INVENTION

This application is made based on the inventor's knowledge and discovery of the following issues:

For a gigapixel sensor, the effective pixel count of a normal imaging system is usually limited to the megapixel level, mainly due to optical aberrations caused by imperfect lenses or environmental disturbances, which cause the light emitted from a point to spread on a large area on a 2D sensor. At the same time, projecting a 3D scene to a 2D plane also leads to the loss of various degrees of freedom of the LF (Light Field), such as depth and local coherence. Therefore, obtaining high-density depth maps using integrated sensors has always been a challenge.

To solve the above problems, optical engineering experts use multiple precision engineered lenses for sequential aberration correction to achieve the perfect imaging system. However, while the difficulty of optical design and fabrication increases exponentially with SBP, high-performance incoherent imaging systems with effective SBP are usually very expensive and bulky due to the diffraction limit, which limits the number of effective pixels, such as large apertures Telescopes and Mirrors. To alleviate this problem, when given sufficient machining accuracy on a large scale, optimized lens surfaces can be fabricated by metalens and freeform optics.

In addition, image deblurring algorithms designed for 2D sensors can improve image contrast through accurate estimation of PSF (point spread function), and PSF engineering with coded aperture preserves more information by reducing zeros in the frequency domain. However, it is difficult for these methods to recover high-frequency information lost by low MTF (Modulation Transfer Function), and especially for spatially inhomogeneous aberrations, specific data priors and accurate PSF estimation are usually required. Meanwhile, these methods above are still very sensitive to dynamic environmental aberrations with small DOF (depth of field).

AO achieves active aberration correction through deformable mirror arrays or spatial light modulators. It does so by directing light emitted from a point to the same location on the 2D sensor at different angles. The aberration-affected wavefront can be guided by Star and wavefront sensor measurements can also be measured by iterative updates based on specific evaluation metrics. AO has achieved great success in both astronomy and microscopy, and has made great scientific discoveries. However, the spatially inhomogeneous aberration introduced by the 3D heterogeneity of the refractive index results in a very small effective FOV (Field of View) of the current AO method, especially for ground-based telescopes. The diameter is limited to around 40 arcseconds, making it unsuitable for large observation telescopes, and current AO systems are often complex, bulky, and expensive to apply to lightweight systems or portable devices.

The present application provides an aberration prediction method, device, electronic device and storage medium brought about by an optical lens or a shooting environment, so as to solve the problem of complicated propagation process, time-consuming and energy-consuming, and poor robustness in the related art based on physical optics. , and the reconstruction effect cannot meet the expected technical problems.

An embodiment of the first aspect of the present application provides a method for predicting aberrations of a lens, including the following steps: acquiring pictures of multiple viewing angles, wherein the pictures of the multiple viewing angles are obtained by the same imaging system and include different angle domain information; The pictures of the multiple perspectives are input into a preset neural network, at least one perspective feature of each perspective is extracted, and the aberration brought by the optical lens or the shooting environment is predicted based on the at least one perspective feature.

Optionally, in an embodiment of the present application, the method further includes: reconstructing a high-resolution picture without aberration effects according to the pictures of the multiple viewing angles and the actual point spread function.

Optionally, in an embodiment of the present application, the predicting the aberration brought by the optical lens or the shooting environment based on the at least one viewing angle feature includes: using the aberration and a preset Zernike polynomial coefficient to perform a Fitting of Nick polynomials to generate aberrations at different resolutions.

Optionally, in an embodiment of the present application, the obtaining the pictures of the multiple viewing angles includes: scanning data of the light field multi-view camera to obtain multiple data pictures; rearranging the multiple data pictures , to obtain pictures of the multiple viewing angles.

Optionally, in an embodiment of the present application, the preset neural network is a ResNet 3D neural network.

An embodiment of the second aspect of the present application provides an apparatus for predicting aberrations of a lens, including: an acquisition module configured to acquire pictures of multiple viewing angles, wherein the pictures of the multiple viewing angles are obtained by the same imaging system and include different angles Domain information; a prediction module, which inputs the pictures of the multiple perspectives into a preset neural network, extracts at least one perspective feature of each perspective, and predicts the aberration caused by the optical lens or the shooting environment based on the at least one perspective feature.

Optionally, in an embodiment of the present application, a reconstruction module is further included, configured to reconstruct a high-resolution picture without aberration effects according to the pictures of the multiple viewing angles and the actual point spread function.

Optionally, in an embodiment of the present application, the prediction module includes: a fitting unit, configured to perform Zernike polynomial fitting by using the aberration and preset Zernike polynomial coefficients to generate different resolutions. aberrations.

Optionally, in an embodiment of the present application, the acquisition module includes: a scanning unit, configured to scan the data of the light field multi-view camera to obtain multiple data pictures; and a processing unit, configured to scan the multiple data images The pictures are rearranged to obtain pictures of the multiple viewing angles.

Optionally, in an embodiment of the present application, the preset residual neural network is a ResNet 3D neural network.

An embodiment of a third aspect of the present application provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the program to achieve The method for predicting aberration of a lens as described in the above embodiments.

Embodiments of the fourth aspect of the present application provide a computer-readable storage medium, where the computer-readable storage medium stores computer instructions, where the computer instructions are used to cause the computer to perform the lens aberration prediction according to the above embodiments method.

In the embodiment of the present application, the residual neural network can be used to obtain the predicted lens aberration of the multi-view image, and the aberration with higher resolution can be obtained through aberration fitting, so as to ensure the accuracy of the disparity estimation, and achieve high speed without iteration and memory burden. Smaller image reconstruction with system robustness. As a result, the technical problems in the related art based on physical optics, which lead to complex simulation propagation process, high computational complexity, poor robustness, and unsatisfactory reconstruction effects, are solved.

Additional aspects and advantages of the present application will be set forth, in part, in the following description, and in part will be apparent from the following description, or learned by practice of the present application.

Description of drawings

The above and/or additional aspects and advantages of the present application will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:

1 is a flowchart of a method for predicting aberrations of a lens according to an embodiment of the present application;

2 is a flowchart of a method for predicting aberrations of a lens according to an embodiment of the present application;

3 is a schematic structural diagram of an apparatus for predicting aberrations of a lens according to an embodiment of the present application;

FIG. 4 is a schematic structural diagram of an electronic device provided according to an embodiment of the present application.

Detailed ways

The following describes in detail the embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, and are intended to be used to explain the present application, but should not be construed as a limitation to the present application.

The method, apparatus, electronic device, and storage medium for predicting lens aberration according to the embodiments of the present application are described below with reference to the accompanying drawings. Aiming at the technical problems that the related art mentioned in the above-mentioned background technology center is based on physical optics, resulting in complicated simulation propagation process, high computational complexity, poor robustness, and the reconstruction effect cannot meet expectations, the present application provides a lens Disparity prediction method, in this method, the residual neural network can be used to obtain the predicted lens aberration of multi-view images, while ensuring the accuracy of disparity estimation, it can achieve high-speed image reconstruction without iteration and less memory burden, with system robustness Awesome. As a result, the technical problems in the related art that are based on physical optics, resulting in complicated propagation process, time-consuming and energy-consuming, poor robustness, and the reconstruction effect not meeting expectations are solved.

Specifically, FIG. 1 is a schematic flowchart of a method for predicting aberrations of a lens according to an embodiment of the present application.

As shown in Figure 1, the aberration prediction method for this lens includes the following steps:

In step S101, pictures of multiple viewing angles are obtained, wherein the pictures of multiple viewing angles are obtained by the same imaging system and include information of different angle domains.

It can be understood that optical aberrations caused by lens or environmental interference are the main factors that lead to low effective pixels of imaging devices. For example, under the same light source, projecting a 3D scene to a 2D plane will result in the loss of degrees of freedom of the LF, For example, depth and local coherence, for this purpose, the present application can achieve high-precision image reconstruction by acquiring pictures from multiple viewing angles and performing aberration estimation.

Optionally, in an embodiment of the present application, acquiring pictures of multiple viewing angles includes: scanning data of a light field multi-view camera to obtain multiple data pictures; rearranging the multiple data pictures to obtain multiple viewing angle pictures. picture.

In the actual execution process, the embodiment of the present application can scan the data of the light field multi-view camera to obtain pictures of multiple viewing angles. pictures, and then the multi-view images are spliced into a two-dimensional image or stacked into a three-dimensional stack image. In this embodiment of the present application, a multi-view and high-resolution frequency domain image of the scanning light field can be collected by using a camera, which is convenient for subsequent processing of the correlation between the multi-view images. to predict camera aberrations and achieve high-quality image reconstruction.

In step S102, the pictures of multiple viewing angles are input into a preset neural network, at least one viewing angle feature of each viewing angle is extracted, and the aberration caused by the optical lens or the shooting environment is predicted based on the at least one viewing angle feature.

It is understandable that, based on the existing neural network, as the depth of the network increases, there will be problems such as gradient explosion and gradient disappearance, which becomes more and more difficult to train, and the network is overloaded in the whole process, and the network predicts the point spread function or The performance of aberration is unstable, which is not conducive to ensuring the quality of image reconstruction, and the residual network can effectively solve this problem through skip connections. It can obtain activation from a certain network layer, and then quickly feedback it to another layer, or even The deeper layer of the neural network provides a low-energy, efficient, and stable feature extraction process, and ensures the quality of image reconstruction.

Therefore, in this embodiment of the present application, multi-view images obtained by rearranging pictures from multiple viewpoints can be stitched into a two-dimensional image or stacked into a three-dimensional stack image and then input to a preset residual neural network, so as to extract at least one image from each viewpoint. View angle features, and then predict the corresponding aberrations of the camera.

Optionally, in an embodiment of the present application, predicting aberrations brought by an optical lens or a shooting environment based on at least one viewing angle feature includes: using aberrations and preset Zernike polynomial coefficients to perform Zernike polynomial fitting, Generate aberrations at different resolutions.

Optionally, in an embodiment of the present application, the preset residual neural network is a ResNet 3D neural network.

In the actual execution process, the preset residual neural network in the embodiment of the present application may be a ResNet 3D neural network, such as ResNet34, and the ResNet 3D neural network is used to predict the plane gradient of the disparity. Generally speaking, the residual neural network is more It is suitable for processing two-dimensional images. Since 3D models are usually represented by grid data, there is a lack of regular structure and hierarchical representation. The ResNet 3D neural network can realize three-dimensional image processing and effectively process the scanning light field between multi-view images. correlation, which facilitates the reconstruction of subsequent high-quality images.

Further, in this embodiment of the present application, image reconstruction can be implemented according to the predicted disparity.

Optionally, in an embodiment of the present application, the method further includes: reconstructing an original picture of the camera according to pictures of multiple viewing angles and an actual point spread function.

In the actual execution process, the embodiment of the present application may calculate the actual point spread function based on the aberration picture predicted by the ideal point spread function, and use the actual point spread function to perform deconvolution to obtain the picture.

The working principle of the lens aberration prediction method according to the embodiment of the present application will be described in detail below with reference to FIG. 2 with a specific embodiment.

As shown in FIG. 2, this embodiment of the present application may include the following steps:

Step S201: Scan the light field camera sensor to collect data.

Step S202 : rearranging the multi-view images, and splicing the multi-view images into a two-dimensional image or stacking them into a three-dimensional stack image. It is understandable that optical aberrations caused by lens or environmental disturbances are the main factors leading to low effective pixels of imaging devices. Under the same light source, projecting a 3D scene onto a 2D plane will result in the loss of degrees of freedom of the LF, such as Depth and local coherence, for this reason, the present application can perform aberration estimation by acquiring pictures from multiple perspectives, rearranging the multi-view pictures, and splicing the multi-view images into a two-dimensional image or stacking them into a three-dimensional stack image, so as to achieve high accuracy. Accurate image reconstruction.

Step S203: Predict the gradient of the disparity plane through ResNet3D. In this embodiment of the present application, pictures from multiple perspectives can be input to a residual neural network, such as ResNet 3D, to extract features of each perspective and predict disparity images.

Step S204: Calculate the actual point spread function. In the embodiment of the present application, an aberration picture can be obtained based on an ideal point spread function prediction, and an actual point spread function can be calculated, and a picture can be obtained by performing deconvolution using the actual point spread function.

Step S205: Calculate the actual picture. In this embodiment of the present application, an actual point spread function and a multi-view image through deconvolution can be used to calculate and generate an actual image, so as to achieve high-precision image reconstruction.

According to the lens aberration prediction method proposed in the embodiment of the present application, the residual neural network can be used to obtain the predicted lens aberration of the multi-view image, and while ensuring the accuracy of the disparity estimation, high-speed image reconstruction without iteration and less memory burden can be realized. , with system robustness. As a result, the technical problems in the related art based on physical optics, which lead to complex quasi-propagation process, high computational complexity, poor robustness, and unsatisfactory reconstruction effect, are solved.

Next, the apparatus for predicting the aberration of a lens according to the embodiments of the present application will be described with reference to the accompanying drawings.

FIG. 3 is a schematic block diagram of an apparatus for predicting an aberration of a lens according to an embodiment of the present application.

As shown in FIG. 3 , the lens aberration prediction apparatus 10 includes an acquisition module 100 and a prediction module 200 .

Specifically, the obtaining module 100 is configured to obtain pictures of multiple viewing angles, wherein the pictures of multiple viewing angles are obtained by the same imaging system and include different angle domain information.

The prediction module 200 is used for inputting pictures of multiple viewing angles into a preset neural network, extracting at least one viewing angle feature of each viewing angle, and predicting the aberration brought by the optical lens or the shooting environment based on the at least one viewing angle feature.

Optionally, in an embodiment of the present application, the lens aberration prediction apparatus 10 further includes: a reconstruction module.

Among them, the reconstruction module is used to reconstruct high-resolution pictures without aberration effects according to pictures from multiple viewing angles and the actual point spread function.

Optionally, in an embodiment of the present application, the prediction module 200 includes: a fitting unit.

The fitting unit is used for fitting the Zernike polynomial by using the aberrations and the preset Zernike polynomial coefficients to generate aberrations with different resolutions.

Optionally, in an embodiment of the present application, the acquiring module 100 includes: a scanning unit and a processing unit.

Wherein, the scanning unit is used to scan the data of the light field multi-view camera to obtain a plurality of data pictures.

The processing unit is used for rearranging multiple data pictures to obtain pictures of multiple viewing angles.

Optionally, in an embodiment of the present application, the preset residual neural network is a ResNet 3D neural network.

It should be noted that, the foregoing explanations of the embodiment of the lens aberration prediction method are also applicable to the lens aberration prediction device of this embodiment, and are not repeated here.

According to the lens aberration prediction device proposed in the embodiment of the present application, the residual neural network can be used to obtain the predicted lens aberration of the multi-view image, and the image reconstruction with high speed, no iteration and less memory burden can be realized while ensuring the accuracy of the disparity estimation. , with system robustness. As a result, the technical problems in the related art based on physical optics, which lead to complex simulation propagation process, high computational complexity, poor robustness, and unsatisfactory reconstruction effects, are solved.

FIG. 4 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. The electronic device may include:

Memory 401 , processor 402 , and computer programs stored on memory 401 and executable on processor 402 .

When the processor 402 executes the program, the method for predicting the lens aberration provided in the above embodiments is implemented.

Further, the electronic device also includes:

The communication interface 403 is used for communication between the memory 401 and the processor 402 .

The memory 401 is used to store computer programs that can be executed on the processor 402 .

The memory 401 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

If the memory 401, the processor 402 and the communication interface 403 are independently implemented, the communication interface 403, the memory 401 and the processor 402 can be connected to each other through a bus and complete communication with each other. The bus may be an Industry Standard Architecture (referred to as ISA) bus, a Peripheral Component (referred to as PCI) bus, or an Extended Industry Standard Architecture (referred to as EISA) bus or the like. The bus can be divided into address bus, data bus, control bus and so on. For ease of presentation, only one thick line is used in FIG. 4, but it does not mean that there is only one bus or one type of bus.

Optionally, in terms of specific implementation, if the memory 401, the processor 402 and the communication interface 403 are integrated on one chip, the memory 401, the processor 402 and the communication interface 403 can communicate with each other through an internal interface.

The processor 402 may be a central processing unit (Central Processing Unit, CPU for short), or a specific integrated circuit (Application Specific Integrated Circuit, ASIC for short), or is configured to implement one or more of the embodiments of the present application integrated circuit.

This embodiment also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the above method for predicting aberration of a lens.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or N of the embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.

In addition, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In the description of the present application, "N" means at least two, such as two, three, etc., unless otherwise expressly and specifically defined.

Any process or method description in the flowcharts or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or N executable instructions for implementing custom logical functions or steps of the process, And the scope of the preferred embodiments of the present application includes alternative implementations in which the functions may be performed out of the order shown or discussed, including performing the functions in a substantially simultaneous manner or in the reverse order depending upon the functions involved, which should be Those skilled in the art to which the embodiments of the present application pertain will be understood.

The logic and/or steps represented in flowcharts or otherwise described herein, for example, may be considered an ordered listing of executable instructions for implementing the logical functions, may be embodied in any computer-readable medium, For use with, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions from an instruction execution system, apparatus, or apparatus) or equipment. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or apparatus. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections (electronic devices) with one or N wires, portable computer disk cartridges (magnetic devices), random access memory (RAM), Read Only Memory (ROM), Erasable Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, followed by editing, interpretation, or other suitable medium as necessary process to obtain the program electronically and then store it in computer memory.

It should be understood that various parts of this application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the N steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented by any one of the following techniques known in the art, or a combination thereof: discrete with logic gates for implementing logic functions on data signals Logic circuits, application specific integrated circuits with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.

Those skilled in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the program can be stored in a computer-readable storage medium. When executed, one or a combination of the steps of the method embodiment is included.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, and the like. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limitations to the present application. Embodiments are subject to variations, modifications, substitutions and variations.

Claims (11)

1. A method for predicting aberration of a lens, comprising the steps of:

acquiring pictures of a plurality of visual angles, wherein the pictures of the plurality of visual angles are acquired by the same imaging system and comprise different angle domain information;

and inputting the pictures of the multiple view angles into a preset neural network, extracting at least one view angle characteristic of each view angle, and predicting aberration caused by the optical lens or the shooting environment based on the at least one view angle characteristic.

2. The method of claim 1, further comprising:

and reconstructing a high-resolution picture without aberration influence according to the pictures of the plurality of visual angles and the actual point spread function.

3. The method of claim 1, wherein predicting aberrations introduced by an optical lens or a shooting environment based on the at least one view angle feature comprises:

and fitting the Zernike polynomials by using the aberration and the preset Zernike polynomial coefficients to generate the aberrations with different resolutions.

4. The method of claim 1, wherein the obtaining the pictures of the plurality of views comprises:

scanning data of the light field multi-view camera to obtain a plurality of data pictures;

and rearranging the data pictures to obtain the pictures of the visual angles.

5. The method of any one of claims 1-4, wherein the pre-defined neural network is a ResNet3D neural network.

6. An aberration prediction apparatus for a lens, comprising:

the system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring pictures of multiple visual angles, the pictures of the multiple visual angles are acquired by the same imaging system and comprise different angle domain information;

and the prediction module is used for inputting the pictures of the plurality of visual angles into a preset neural network, extracting at least one visual angle characteristic of each visual angle and predicting aberration brought by the optical lens or the shooting environment based on the at least one visual angle characteristic.

7. The apparatus of claim 6, further comprising:

and the reconstruction module is used for reconstructing a high-resolution picture without aberration influence according to the pictures of the plurality of visual angles and the actual point spread function.

8. The apparatus of claim 6, wherein the obtaining module comprises:

the scanning unit is used for scanning data of the light field multi-view camera to obtain a plurality of data pictures;

and the processing unit is used for rearranging the plurality of data pictures to obtain the pictures of the plurality of visual angles.

9. The apparatus according to any one of claims 6-8, wherein the pre-set neural network is a ResNet3D neural network.

10. An electronic device, comprising: a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the program to implement the aberration prediction method of the lens according to any one of claims 1 to 5.

11. A computer-readable storage medium, on which a computer program is stored, the program being executable by a processor for implementing aberrations introduced by an optical lens or a shooting environment according to any one of claims 1 to 5.

Images