CN115708123A - Image display method, image display device, electronic device, and storage medium - Google Patents
Image display method, image display device, electronic device, and storage medium Download PDFInfo
- Publication number
- CN115708123A CN115708123A CN202110950880.9A CN202110950880A CN115708123A CN 115708123 A CN115708123 A CN 115708123A CN 202110950880 A CN202110950880 A CN 202110950880A CN 115708123 A CN115708123 A CN 115708123A
- Authority
- CN
- China
- Prior art keywords
- lens model
- phase
- light wave
- trained
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 238000006243 chemical reaction Methods 0.000 claims description 46
- 238000012545 processing Methods 0.000 claims description 44
- 230000009466 transformation Effects 0.000 claims description 37
- 238000012549 training Methods 0.000 claims description 23
- 238000013016 damping Methods 0.000 claims description 15
- 238000005315 distribution function Methods 0.000 claims description 14
- 238000004364 calculation method Methods 0.000 claims description 9
- 230000003287 optical effect Effects 0.000 description 37
- 238000010586 diagram Methods 0.000 description 10
- 238000005516 engineering process Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 7
- 238000005286 illumination Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 238000003384 imaging method Methods 0.000 description 3
- 230000001276 controlling effect Effects 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 238000009795 derivation Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3102—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM] using two-dimensional electronic spatial light modulators
- H04N9/312—Driving therefor
- H04N9/3126—Driving therefor for spatial light modulators in series
-
- G—PHYSICS
- G02—OPTICS
- G02F—OPTICAL DEVICES OR ARRANGEMENTS FOR THE CONTROL OF LIGHT BY MODIFICATION OF THE OPTICAL PROPERTIES OF THE MEDIA OF THE ELEMENTS INVOLVED THEREIN; NON-LINEAR OPTICS; FREQUENCY-CHANGING OF LIGHT; OPTICAL LOGIC ELEMENTS; OPTICAL ANALOGUE/DIGITAL CONVERTERS
- G02F1/00—Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulating; Non-linear optics
- G02F1/01—Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulating; Non-linear optics for the control of the intensity, phase, polarisation or colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/50—Lighting effects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3179—Video signal processing therefor
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Nonlinear Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Graphics (AREA)
- Theoretical Computer Science (AREA)
- Optics & Photonics (AREA)
- Control Of Indicators Other Than Cathode Ray Tubes (AREA)
Abstract
The application discloses an image display method, an image display device, electronic equipment and a storage medium, which relate to the technical field of display and comprise the following steps: the method comprises the steps of obtaining a trained lens model, and loading the phase of a target image onto the trained lens model, wherein the trained lens model is a linearized model, when a first light wave is input into the trained lens model, obtaining a second light wave output by the trained lens model through phase modulation, wherein the second light wave carries the phase information of the target image, inputting the second light wave into an intensity spatial light modulator, obtaining the second light wave output by the intensity spatial light modulator after intensity modulation, enabling the intensity-modulated second light wave to carry the phase information and the intensity information of the target image, and displaying the target image on a display plane through the intensity-modulated second light wave.
Description
Technical Field
The present disclosure relates to the field of display technologies, and in particular, to an image display method and apparatus, an electronic device, and a storage medium.
Background
High Dynamic Range Imaging (HDR) is a technology generally applied to computer graphics or cinematography technology and used for expanding a luminance exposure Range, and a luminance Range of an image processed by the HDR technology is larger than a luminance Dynamic Range of an ordinary digital image, that is, the HDR technology makes a bright area in the image brighter and a dark area darker, so as to increase a brightness difference of the image.
In the existing HDR technology, a laser generates collimated laser, the collimated laser is irradiated on a galvanometer of a micro electro mechanical Systems (MEMS for short), the display area is scanned by the deflection of the galvanometer to a light beam, and the modulation of a display image is controlled by controlling the laser to a switch of the laser, but this method needs the laser and the MEMS, which is high in cost, and the brightness, resolution, and bit depth of the displayed image are not good.
Disclosure of Invention
In view of the above problems, the present application provides an image display method, an image display apparatus, an electronic device, and a storage medium, which can solve the above problems.
In a first aspect, an embodiment of the present application provides an image display method, where the method includes: acquiring a trained lens model, and loading the phase of a target image onto the trained lens model, wherein the trained lens model is a linearized model; when the first light wave is input into the trained lens model, obtaining a second light wave output after the trained lens model is subjected to phase modulation, wherein the second light wave carries phase information of the target image; and inputting the second light wave into an intensity spatial light modulator to obtain the second light wave output after intensity modulation of the intensity spatial light modulator, so that the intensity-modulated second light wave carries phase information and intensity information of the target image and displays the target image on a display plane.
In a second aspect, an embodiment of the present application provides an image display apparatus, including: the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a trained lens model and loading the phase of a target image onto the trained lens model, and the trained lens model is a linearized model; the output module is used for obtaining a second light wave output after the trained lens model is subjected to phase modulation when the first light wave is input into the trained lens model, wherein the second light wave carries phase information of the target image; and the display module is used for inputting the second light wave into the intensity spatial light modulator to obtain the second light wave output after the intensity of the intensity spatial light modulator is modulated, so that the intensity-modulated second light wave carries the phase information and the intensity information of the target image and displays the target image on a display plane.
In a third aspect, an embodiment of the present application provides an electronic device, including: one or more processors; a memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications configured to perform the above-described method.
In a fourth aspect, the present application provides a computer-readable storage medium, in which a program code is stored, and the program code can be called by a processor to execute the above method.
The image display method, the image display device, the electronic device and the storage medium provided by the application obtain a trained lens model, and load the phase of a target image onto the trained lens model, wherein the trained lens model is a linearized model, when a first light wave is input into the trained lens model, a second light wave output by the trained lens model through phase modulation is obtained, wherein the second light wave carries phase information of the target image, the second light wave is input into an intensity spatial light modulator, the second light wave output by the intensity spatial light modulator after intensity modulation is obtained, so that the second light wave after intensity modulation carries the phase information and intensity information of the target image, and the second light wave after intensity modulation displays the target image on a display plane.
These and other aspects of the present application will be more readily apparent from the following description of the embodiments.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic diagram of a modulator of an electronic device according to an embodiment of the application;
FIG. 2 is a flow chart illustrating an image display method according to an embodiment of the present application;
FIG. 3 illustrates a light turning schematic provided by one embodiment of the present application;
FIG. 4a shows a second light wave shading diagram;
FIG. 4b shows a schematic diagram of a display image shading;
FIG. 5 is a flow chart illustrating an image display method according to another embodiment of the present application;
FIG. 6 is a flow chart illustrating step S220 of the image display method of FIG. 5 of the present application;
fig. 7 is a block diagram illustrating an image display apparatus according to an embodiment of the present application;
fig. 8 is a block diagram of an electronic device for executing an image display method according to an embodiment of the present application;
fig. 9 illustrates a storage unit for storing or carrying a program code implementing an image display method according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, and not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.
Hereinafter, terms that may be referred to in the embodiments of the present application will be described.
High Dynamic Range Imaging (HDR) is generally applied to computer graphics or cinematography, and is a technology for expanding a luminance exposure Range, and a luminance Range of an image processed by the HDR technology is larger than that of an ordinary digital image, that is, the HDR technology makes a bright area in the image brighter and a dark area darker, and increases a difference between brightness and darkness of the image.
In one conventional HDR method, collimated laser light is generated by a laser, and is irradiated onto a galvanometer of a Micro Electro Mechanical Systems (MEMS) that vibrates at a high speed, where the deflection of a light beam by the galvanometer scans a display area, and the modulation of a display image is controlled by controlling the laser to a switch of the laser. However, the inventor finds that the method needs a laser and a MEMS, is high in cost, and has poor brightness, resolution and bit depth of a displayed image.
In another conventional HDR method, a specific light steering device is used to dynamically modulate illumination light, and energy of the illumination light is dynamically allocated according to display content, so that a high-brightness region in the display content can obtain higher illumination brightness to realize high-brightness display, and meanwhile, the illumination brightness of a dark part in a picture can be reduced to obtain a purer dark field display expression, thereby realizing an HDR display effect. However, the inventor finds that the method cannot adapt to HDR display of different patterns, and meanwhile, the calculation method is complex, the calculation speed is slow, and real-time HDR display is difficult to realize.
In order to solve the above technical problems, the inventors have found and proposed an image display method, an apparatus, an electronic device and a storage medium through long-term research, wherein a trained lens model is obtained, and a phase of a target image is loaded onto the trained lens model, wherein the trained lens model is a linearized model, when a first light wave is input into the trained lens model, a second light wave output by phase modulation by the trained lens model is obtained, wherein the second light wave carries phase information of the target image, the second light wave is input into an intensity spatial light modulator, and the second light wave output by intensity modulation by the intensity spatial light modulator is obtained, so that the intensity-modulated second light wave carries the phase information and intensity information of the target image, and the intensity-modulated second light wave displays the target image on a display plane. The specific image display method is specifically described in the following embodiments.
To facilitate understanding of the image display method, the present application shows a modulator schematic diagram of an electronic device, and referring to fig. 1, the modulator includes a phase spatial light modulator 230 and an intensity spatial light modulator 240, the phase spatial light modulator 230 and the intensity spatial light modulator 240 are spaced apart by a focal length f and are disposed on the same plane in parallel, wherein the phase spatial light modulator 230 may be a linearized lens model.
The first light wave is incident into the phase spatial light modulator 230, and outputs the second light wave after being subjected to light diversion by the phase spatial light modulator 230. It is understood that the phase spatial light modulator 230 performs phase conversion on the first light wave, the second light wave is incident into the intensity spatial light modulator 240 to perform light intensity modulation, and the intensity spatial light modulator 240 outputs the intensity-modulated second light wave to be projected onto a display plane, so as to display an image on the display plane.
Fig. 2 is a flowchart illustrating an image display method according to an embodiment of the present application, and in a specific embodiment, the image display method is applied to the image display apparatus 100 shown in fig. 7 and the electronic device 200 configured with the image display apparatus 100 shown in fig. 8. In this embodiment, a specific process of this embodiment will be described by taking an example of applying the image display method to the electronic device 200, where the electronic device may be, but is not limited to, a projector, a television, a computer, a smart phone, a tablet computer, an intelligent wearable device, a lighting system, and the like. As will be described in detail with respect to the flow shown in fig. 1, the image display method may specifically include the following steps:
step S110, acquiring a trained lens model, and loading the phase of a target image onto the trained lens model, wherein the trained lens model is a linearized model.
And training a lens model in advance by combining the phase distribution function, storing the trained lens model, and obtaining the trained lens model from a storage position, wherein the storage position can be a local storage position of the electronic device or a storage position of a server which is in communication connection with the electronic device.
The trained lens model is a linearized model and has the advantages of high calculation speed, high stability and the like.
Step S120, when the first optical wave is input into the trained lens model, obtaining a second optical wave output after the trained lens model is phase-modulated, wherein the second optical wave carries phase information of the target image.
The first Light wave is uniform illumination Light, the first Light wave may be directly generated by a Light source, and the Light source may be a lamp, a Light Emitting Diode (LED) or a laser array with different wavelengths in an electronic device.
The first light wave is incident into the trained lens model, for example, it can be understood that, taking a free-form surface model commonly used in simulation experiments as an example, the trained lens model may be equivalent to a free-form surface, and when the first light wave is incident into the trained lens model, it may be equivalent to be incident onto the free-form surface from a normal direction of the free-form surface.
The trained lens model phase-modulates the first light wave, i.e. the trained lens model is used to redistribute the input light field, i.e. to steer the light wave input therein to obtain the output light wave.
The first light wave is input into the trained lens model, the trained lens model performs phase modulation on the first light wave to obtain a second light wave, it can be understood that the first light wave is modulated by the trained lens model to generate a steering action to obtain the second light wave, the phase of the light wave is changed after the steering treatment, namely the phase of the first light wave is different from that of the second light wave, the light steering of the trained lens model provides lossless light intensity distribution modulation, the second light wave after the modulation has higher highest brightness and lower lowest brightness, the dynamic range of the first light wave is expanded, high dynamic range display is finally realized, and the phase distribution of the second light wave obtained after the phase modulation approximately conforms to a target image. Since the trained lens model is substantially a linearized lens model, the trained lens model can rapidly and stably output the second light wave.
In one embodiment, the trained lens model simulates a free form lens, which may be, but is not limited to, a phase space intensity spatial light modulator, a variable mirror, a Liquid Crystal On Silicon (LCoS), or the like. The trained lens model simulates a free-form surface, and the first light wave is subjected to steering processing through the free-form surface, wherein the free-form surface is a non-rotationally symmetric special-shaped curved surface.
Illustratively, as shown in FIG. 3, the trained lens model simulates a free-form surface A, a free-form surfaceA is in the coordinate
The coordinates of the light field plane B (which may be the plane corresponding to the intensity spatial light modulator) are
The free-form surface A and the optical field plane B are arranged in parallel, and a certain fixed interval is arranged between the free-form surface A and the optical field plane B, when the first optical wave L1 is incident from the position a1 on the free-form surface A, the free-form surface A performs phase conversion processing on the first optical wave L1, and the free-form surface A outputs the second optical wave L2 from the position a 1. After the phase conversion, the phase of the first optical wave L1 is different from the phase of the second optical wave L2, and the phase difference between the first optical wave L1 and the second optical wave L2 is θ.
With continued reference to fig. 3, if the second light wave L2 is not deflected, the second light wave L2 travels along a straight line from the position a1 on the free-form surface a to the position b2 on the display plane. The free-form surface A has a steering processing function, so that the first light wave L1 is turned from the position a1 to be changed into a second light wave L2, and the second light wave L2 is projected to the target position B1 of the light field plane B, so that more light waves are incident to the target position B1, and the light intensity of the target position B1 as a bright area is enhanced. Since the light wave that should be focused at the b2 position is focused at the target position b1, the light wave at the b2 position is less, and the light intensity at the b2 position is reduced as a dark area.
It should be noted that, alternatively, in order to ensure the display effect of the final target image, a certain fixed interval may be determined according to the wavelength of the input light wave, specifically, an interval with the best display effect corresponding to the wavelength is obtained through experiments, multiple experiments are performed to obtain a fixed interval corresponding to each wavelength of the multiple wavelengths, a correspondence between the multiple wavelengths and the multiple fixed intervals is established to obtain the wavelength of the first light wave, and based on the correspondence, the fixed interval corresponding to the wavelength of the first light wave is determined, for example, when the wavelength of the first light wave is 550nm (length unit, nanometer), the fixed interval may be 50mm (length unit, millimeter). The fixed pitch is not limited to 50mm described above, and may be 40mm, 60mm, or the like depending on the wavelength. For example, the resolution of the picture sample is 512 × 512, and assuming that a trained lens model consistent with the resolution is adopted, the size of a single pixel of the trained lens model is 8um, the wavelength of the first light wave is set to be green light of 532nm, and the fixed distance between the modulated light field plane and the phase plane is 25cm.
It should be noted that fig. 3 only shows one first optical wave L1, and in practical applications, the number of the first optical waves is multiple. And simultaneously inputting the light field formed by a plurality of first light waves into the trained lens model to obtain a second light wave output by the trained lens model, wherein if the second light wave irradiates on a display plane, a certain light and shade distribution is presented in advance (as shown in fig. 4 a). The second optical wave is not limited to being incident from the position a1 of the free-form surface a shown in fig. 3, and may be incident at other positions on the free-form surface.
Step S130, inputting the second light wave into an intensity spatial light modulator, and obtaining the second light wave output after intensity modulation by the intensity spatial light modulator, so that the intensity-modulated second light wave carries phase information and intensity information of the target image, and the target image is displayed on a display plane.
The intensity spatial light modulator is loaded with the intensity information of the target image, the second light wave is input into the intensity spatial light modulator, and the intensity spatial light modulator is loaded with the intensity information of the target image, so that the light wave output by the intensity spatial light modulator carries the phase information of the target image. The intensity-modulated second light wave is projected on a display plane, the intensity-modulated second light wave carries phase information and intensity information of the target image, a display image shown in fig. 1 is generated, the light intensity of the display image is shown in fig. 4b, and more detailed intensity regulation and control are realized through the intensity spatial light modulator, so that the maximum brightness peak value of the finally-imaged display image is improved by 10 times.
The image display method provided by this embodiment obtains a trained lens model, and loads a phase of a target image onto the trained lens model, where the trained lens model is a linearized model, and when a first optical wave is input into the trained lens model, a second optical wave output by the trained lens model through phase modulation is obtained, where the second optical wave carries phase information of the target image, and the second optical wave is input into an intensity spatial light modulator, so as to obtain the second optical wave output by the intensity spatial light modulator after intensity modulation, so that the second optical wave after intensity modulation carries the phase information and intensity information of the target image, and the second optical wave after intensity modulation displays the target image on a display plane.
Fig. 5 is a schematic flowchart illustrating an image display method according to another embodiment of the present application, where a trained lens model needs to be trained before the trained lens model is used, and referring to fig. 5, the image display method may specifically include the following steps:
step S210, a target phase conversion formula is obtained, wherein the target phase conversion formula is obtained by performing linearization processing on an initial phase conversion formula.
The gradient of the phase fluctuation of the free-form surface determines the deflection angle of the light beam, and the second derivative of the phase fluctuation determines the curvature of the free-form surface and determines the regulation degree of the phase plane on the convergence and the divergence of the light beam. The curvature of the free-form surface can be simulated by the phase distribution function, and therefore, a target phase transformation formula can be constructed by the phase distribution function, which is specifically as follows:
first, an initial phase conversion formula is constructed. With reference to FIG. 3, the free-form surface A is located at the coordinates of
The coordinates of the light field plane B are
The free-form surface A and the light field plane B are arranged in parallel, the free-form surface A and the light field plane B are separated by a focal distance f, and the coordinate of the free-form surface A is
And coordinates of the light field plane B
The corresponding relation between the two is as follows:
wherein,
is the initial phase distribution function.
The coordinate point of each coordinate has a corresponding relation with the light intensity,
corresponding light intensity
Corresponding light intensity
Substituting the light intensity into formula (1) to obtain the formula:
wherein,
is the output light intensity of the initial phase conversion formula.
Then, performing linearization processing on the initial phase conversion formula to obtain the target phase conversion formula, which specifically includes:
carrying out reciprocal processing on the initial phase conversion formula to obtain a conversion formula after reciprocal calculation;
and carrying out root opening processing on the conversion formula after the reciprocal is solved to obtain the target phase conversion formula. As a way, when the formula obtained after performing reciprocal processing and root-opening processing on the initial phase transformation formula can accurately obtain a reasonable output light wave from the input light wave, the formula is taken as a target transformation formula.
As another mode, after performing reciprocal processing and root-opening processing on the initial phase transformation formula, the initial target conversion formula (3) is obtained as follows:
and S220, training a target phase transformation formula through input light waves and a target image to obtain a trained lens model.
The input light wave is input into a target phase conversion formula, the target phase conversion formula outputs light waves, the output light waves are optimally projected to obtain a target image, and the input light waves and the target image are used for training the target phase conversion formula so as to establish the corresponding relation between the input light waves and the target image.
In one embodiment, step S220 includes the following sub-steps:
and a substep S221 of inputting the input light wave into the target phase transformation formula to obtain a predicted image output by the target phase transformation formula.
Inputting the input light wave into a target phase transformation formula, an initial phase distribution function
So thatThe phase of the input light wave is deformed, so that a curve-shaped phase distribution function locally deflects the input light, the target phase transformation formula outputs the output light wave after phase deflection, and the output light wave projects a predicted image.
When the phase of the predicted image output by the target phase transformation formula is consistent with the phase of the target image, the target phase transformation formula establishes the corresponding relation between the input light wave and the target image, and the training of the target phase transformation formula is completed to obtain the trained lens model.
When the phase of the predicted image output by the target phase conversion formula does not coincide with the phase of the target image, the following sub-step S222 is performed.
And a substep S222, updating the target phase transformation formula according to the predicted image and the target image, and obtaining the trained lens model.
In one embodiment, an intensity difference between the intensity of the preset image and the intensity of the target image is calculated, and the target phase transformation formula is iteratively trained according to the intensity difference until the phase difference satisfies an iterative training condition, so as to obtain a trained lens model, wherein the iterative training condition may be that the intensity difference is smaller than the preset intensity difference. Because the target phase transformation formula is subjected to linearization processing, the target phase transformation formula can be rapidly output, the iteration speed is increased, and the model training speed is increased.
In another embodiment, a phase difference between the predicted image and the target image is calculated, and the target phase transformation formula is iteratively trained according to the phase difference until the phase difference satisfies an iterative training condition, so as to obtain a trained lens model, wherein the iterative training condition may be that the phase difference is smaller than a preset phase difference. The target phase transformation formula is subjected to linearization processing, so that the target phase transformation formula can be rapidly output, the iteration speed is increased, and the model training speed is increased.
As a way, a phase difference between the prediction image and the target image is obtained; updating the target phase transformation formula according to the phase difference to obtain the trained phase transformation formulaIn particular, the initial phase is distributed as a function of the phase difference
Updating to a phase distribution function p (x) so as to update a target phase conversion formula, performing iterative training on the formula until the phase difference meets a training condition, and ending the training to obtain the target phase conversion formula:
wherein i 0 Is the first light wave i 1 For the second optical wave, f focal length, p (x) is the phase distribution function.
The phase distribution function p (x) can reflect the focal length of the lens corresponding to the free-form surface, determines the convergence and divergence degree of the position to the light beam, and determines the regulated and controlled brightness distribution.
It should be noted that the iterative training is to iteratively train the target phase transformation formula, and actually to iteratively train the phase distribution function p (x), so as to train the phase distribution function that minimizes the error between the intensity of the optical wave output by the model and the target intensity.
It should be further noted that, for different display requirements, the light beams input into the trained lens models are converged and diverged to different degrees, and the corresponding different trained lens models, that is, for different convergence and divergence requirements, the different models have different phase distribution functions.
It should be noted that the above model may be trained by using multiple sets of input light waves and target images corresponding to the input light waves, and the accuracy of the trained lens model is higher when the number of sets is larger.
Optionally, in order to improve the performance of the trained lens model, the trained lens model may be processed as follows: acquiring a convolution operator; processing the trained lens model by the convolution operator and a fast fourier transform. In addition, quick and efficient calculation can be realized through a near-end operator, the stability of the calculation of the trained lens model can be enhanced through a damping factor and a damping term, the calculation result of the model can be quickly converged through the damping factor and the damping term, the brightness distribution output after the model is modulated is enabled to be consistent with the target brightness distribution of a target image, and small distortion exists, and specifically, the near-end operator and the damping factor are obtained; processing the trained lens model by the near-end operator and the damping factor. Wherein the computation speed of the trained lens model is increased by adding a near-end operator. The stability of the trained lens model is improved by adding the damping term and the damping factor, so that the trained lens model can be suitable for various images. Alternatively, the images may include types Jpg, png, tif, and so forth.
When the difference is required to be described, for different types of pictures, the selected convolution operator, the near-end operator and the damping factor may be different, so that the brightness distribution output by the trained lens model is close to the target brightness distribution of the target image, and the final display effect of the target image is improved.
Alternatively, the trained lens model may be stored locally on the electronic device, or in a server communicatively coupled to the electronic device, and may be recalled directly from a storage location when using the trained lens model.
In one embodiment, the trained lens model may be stored locally on the electronic device after pre-training is complete. When the trained lens model is used, the trained lens model can be called quickly, when the first light wave is input, the trained lens model can output the second light wave quickly, and the model is stored locally in the electronic equipment, so that the problem that the speed of outputting the second light wave by the trained lens model is reduced due to the influence of network factors can be effectively avoided, the response speed is high, and the user experience is further improved.
In another embodiment, the trained lens model may be stored in a server communicatively coupled to the electronic device after pre-training is completed. When the trained lens model is used, the electronic equipment sends information (including light intensity, phase and the like) of the first light wave to the trained lens model of the server to indicate the trained lens model to output information of the second light wave, acquires the information of the second light wave sent by the server, and generates the second light wave according to the information of the second light wave, so that the trained lens model is stored in the server, the occupation of the storage space of the electronic equipment is reduced, and the influence on the normal operation of the electronic equipment is reduced.
Step S230, acquiring a trained lens model, and loading the phase of the target image onto the trained lens model, wherein the trained lens model is a linearized model.
Step S240, when the first optical wave is input into the trained lens model, obtaining a second optical wave output after the trained lens model is phase-modulated, where the second optical wave carries phase information of the target image.
Step S250, inputting the second light wave into an intensity spatial light modulator, and obtaining the second light wave output after intensity modulation by the intensity spatial light modulator, so that the intensity-modulated second light wave carries phase information and intensity information of the target image and displays the target image on a display plane.
For detailed description of steps S230 to S250, please refer to steps S110 to S130, which are not described herein again.
In summary, the present application provides an image display method, an image display apparatus, an electronic device, and a storage medium, in which a trained lens model is obtained through iterative training of the lens model, and the trained lens model is a linear model, which can stably and rapidly output light waves, thereby improving the speed and quality of final imaging.
To implement the above method embodiments, the present embodiment provides an image display apparatus, fig. 7 shows a block diagram of the image display apparatus according to an embodiment of the present application, and referring to fig. 7, the image display apparatus 100 includes: an acquisition module 110, an output module 120, and a display module 130.
An obtaining module 110, configured to obtain a trained lens model, and load a phase of a target image onto the trained lens model, where the trained lens model is a linearized model;
an output module 120, configured to obtain a second light wave output after the trained lens model is phase-modulated when the first light wave is input into the trained lens model, where the second light wave carries phase information of the target image;
and the display module 130 is configured to input the second optical wave into the intensity spatial light modulator, obtain the second optical wave output after intensity modulation by the intensity spatial light modulator, so that the intensity-modulated second optical wave carries phase information and intensity information of the target image and displays the target image on a display plane.
Optionally, the image display apparatus 100 further includes: a building block and a linearization block.
The construction module is used for constructing an initial phase conversion formula;
and the linearization module is used for carrying out linearization processing on the initial phase conversion formula to obtain the target phase conversion formula.
Optionally, the linearization module comprises: a derivation processing submodule and a root number processing submodule.
The derivation processing submodule is used for carrying out reciprocal processing on the initial phase conversion formula to obtain a conversion formula after reciprocal calculation;
and the root number processing submodule is used for carrying out root number processing on the conversion formula after the reciprocal is solved to obtain the target phase conversion formula.
Optionally, the target phase conversion formula is:
wherein i 0 Is the first light wave i 1 For the second light wave, f is the distance between the trained lens model and the display plane, and p (x) is a phase distribution function.
Optionally, the image display device 100 further comprises: the device comprises a target phase conversion formula module and a training module.
The target phase conversion formula module is used for obtaining a target phase conversion formula, wherein the target phase conversion formula is obtained by performing linearization processing on an initial phase conversion formula;
and the training module is used for training a target phase transformation formula through input light waves and a target image to obtain a trained lens model.
Optionally, the training module comprises: a prediction module and an update module.
The prediction module is used for inputting the input light wave into the target phase transformation formula to obtain a predicted image output by the target phase transformation formula;
and the updating module is used for updating the target phase transformation formula according to the predicted image and the target image to obtain the trained lens model.
Optionally, the update module includes: the device comprises a phase difference acquisition module and a formula updating module.
The phase difference acquisition module is used for acquiring the phase difference between the predicted image and the target image;
and the formula updating module is used for updating the target phase conversion formula according to the phase difference to obtain the trained lens model.
Optionally, the image display apparatus 100 further includes: the convolution operator acquisition module and the first model processing module.
The convolution operator acquisition module is used for acquiring convolution operators;
and the first model processing module is used for processing the trained lens model through the convolution operator and the fast Fourier transform.
Optionally, the image display apparatus 100 further includes: the system comprises a near-end operator and damping factor acquisition sub-module and a second model processing module.
The near-end operator and damping factor acquisition sub-module is used for acquiring the near-end operator and the damping factor;
and the second model processing module is used for processing the trained lens model through the near-end operator and the damping factor.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses and modules may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, the coupling between the modules may be electrical, mechanical or other type of coupling.
In addition, functional modules in the embodiments of the present application may be integrated into one processing module, or each of the modules may exist alone physically, or two or more modules are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode.
Fig. 8 is a block diagram of an electronic device for executing an image display method according to an embodiment of the present application, and please refer to fig. 8, which shows a block diagram of an electronic device 200 according to an embodiment of the present application. The electronic device 200 may be a smart phone, a tablet computer, an electronic book, or other electronic devices capable of running an application program. The electronic device 200 in the present application may include one or more of the following components: a processor 210, a memory 220, and one or more applications, wherein the one or more applications may be stored in the memory 220 and configured to be executed by the one or more processors 210, the one or more programs configured to perform a method as described in the aforementioned method embodiments.
The Memory 220 may include a Random Access Memory (RAM) or a Read-Only Memory (Read-Only Memory). The memory 220 may be used to store instructions, programs, code, sets of codes, or sets of instructions. The memory 220 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing various method embodiments described below, and the like. The storage data area may also store data created by the electronic device 200 in use (such as historical profiles) and the like.
Fig. 9 shows a storage unit for storing or carrying program codes for implementing the image display method according to the embodiment of the present application, please refer to fig. 10, which shows a block diagram of a computer-readable storage medium provided by the embodiment of the present application. The computer-readable medium 300 has stored therein a program code that can be called by a processor to execute the method described in the above-described method embodiments.
The computer-readable storage medium 300 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer-readable storage medium 300 includes a non-volatile computer-readable storage medium. The computer readable storage medium 300 has storage space for program code 310 for performing any of the method steps of the method described above. The program code can be read from and written to one or more computer program products. Program code 310 may be compressed, for example, in a suitable form.
In summary, the present application provides an image display method, an image display apparatus, an electronic device, and a storage medium, which acquire a trained lens model and load a phase of a target image onto the trained lens model, wherein the trained lens model is a linearized model, and when a first optical wave is input into the trained lens model, a second optical wave output by the trained lens model through phase modulation is obtained, wherein the second optical wave carries phase information of the target image, and the second optical wave is input into an intensity spatial light modulator to obtain the second optical wave output by the intensity spatial light modulator after intensity modulation, so that the second optical wave after intensity modulation carries the phase information and intensity information of the target image, and the second optical wave after intensity modulation displays the target image on a display plane.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; the above description is only an example of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, and improvement made within the spirit and scope of the present application are included in the protection scope of the present application.
Claims (12)
1. An image display method, characterized in that the method comprises:
acquiring a trained lens model, and loading the phase of a target image onto the trained lens model, wherein the trained lens model is a linearized model;
when the first light wave is input into the trained lens model, obtaining a second light wave output after the trained lens model is subjected to phase modulation, wherein the second light wave carries phase information of the target image;
and inputting the second light wave into an intensity spatial light modulator to obtain the second light wave output after intensity modulation of the intensity spatial light modulator, so that the intensity-modulated second light wave carries phase information and intensity information of the target image and displays the target image on a display plane.
2. The method of claim 1, wherein the trained lens model is trained from a target phase transformation formula, and wherein the method further comprises, prior to obtaining the trained lens model:
constructing an initial phase conversion formula;
and carrying out linearization processing on the initial phase conversion formula to obtain the target phase conversion formula.
3. The method of claim 2, wherein the linearizing the initial phase transformation equation to obtain the target phase transformation equation comprises:
carrying out reciprocal processing on the initial phase conversion formula to obtain a conversion formula after reciprocal calculation;
and carrying out root opening processing on the conversion formula after the reciprocal is solved to obtain the target phase conversion formula.
4. The method of claim 2, wherein the target phase transformation formula is:
wherein i 0 Is the first light wave i 1 For the second light wave, f is the distance between the trained lens model and the display plane, and p (x) is a phase distribution function.
5. The method of claim 1, wherein prior to acquiring the trained lens model and loading the phase of the target image onto the trained lens model, the method further comprises:
acquiring a target phase conversion formula, wherein the target phase conversion formula is obtained by performing linearization processing on an initial phase conversion formula;
and training a target phase transformation formula by inputting the light waves and the target image to obtain a trained lens model.
6. The method of claim 5, wherein the training a target phase transformation equation by the input light waves and the target image to obtain a trained lens model comprises:
inputting the input light wave into the target phase transformation formula to obtain a predicted image output by the target phase transformation formula;
and updating the target phase transformation formula according to the predicted image and the target image to obtain the trained lens model.
7. The method of claim 6, wherein said updating the target phase inversion formula based on the predicted image and the target image to obtain the trained lens model comprises:
obtaining a phase difference between the predicted image and the target image;
and updating the target phase conversion formula according to the phase difference to obtain the trained lens model.
8. The method of claim 7, wherein after updating the target phase transformation equation based on the phase difference to obtain the trained lens model, the method further comprises:
acquiring a convolution operator;
processing the trained lens model by the convolution operator and a fast Fourier transform.
9. The method of claim 7, wherein after updating the target phase transformation equation based on the phase difference to obtain the trained lens model, the method further comprises:
acquiring a near-end operator and a damping factor;
processing the trained lens model by the near-end operator and the damping factor.
10. An image display apparatus, characterized in that the apparatus comprises:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a trained lens model and loading the phase of a target image onto the trained lens model, and the trained lens model is a linearized model;
the output module is used for obtaining a second light wave output after the trained lens model is subjected to phase modulation when the first light wave is input into the trained lens model, wherein the second light wave carries the phase information of the target image;
and the display module is used for inputting the second light wave into the intensity spatial light modulator to obtain the second light wave output after the intensity of the intensity spatial light modulator is modulated, so that the intensity-modulated second light wave carries the phase information and the intensity information of the target image and displays the target image on a display plane.
11. An electronic device, comprising:
one or more processors;
a memory;
one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications configured to perform the method of any of claims 1-9.
12. A computer-readable storage medium, having stored thereon program code that can be invoked by a processor to perform the method according to any one of claims 1 to 9.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110950880.9A CN115708123A (en) | 2021-08-18 | 2021-08-18 | Image display method, image display device, electronic device, and storage medium |
| PCT/CN2022/111972 WO2023020371A1 (en) | 2021-08-18 | 2022-08-12 | Image display method and apparatus, electronic device, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110950880.9A CN115708123A (en) | 2021-08-18 | 2021-08-18 | Image display method, image display device, electronic device, and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115708123A true CN115708123A (en) | 2023-02-21 |
Family
ID=85212506
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110950880.9A Pending CN115708123A (en) | 2021-08-18 | 2021-08-18 | Image display method, image display device, electronic device, and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN115708123A (en) |
| WO (1) | WO2023020371A1 (en) |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008216575A (en) * | 2007-03-02 | 2008-09-18 | Sony Corp | Image display method |
| JP5125245B2 (en) * | 2007-06-15 | 2013-01-23 | 独立行政法人情報通信研究機構 | Optical waveform shaping device |
| CA2890560C (en) * | 2013-10-20 | 2021-06-01 | Mtt Innovation Incorporated | Light field projectors and methods |
| US11443705B2 (en) * | 2018-11-13 | 2022-09-13 | Sony Group Corporation | Image display device for displaying moving images |
| KR102658688B1 (en) * | 2019-11-04 | 2024-04-17 | 엘지전자 주식회사 | Method and apparatus for enhancing image illumination intensity |
| CN111579097B (en) * | 2020-05-21 | 2021-05-04 | 浙江大学 | High-precision optical scattering compensation method based on neural network |
| CN112116616B (en) * | 2020-08-05 | 2022-06-07 | 西安交通大学 | Phase information extraction method based on convolutional neural network, storage medium and equipment |
-
2021
- 2021-08-18 CN CN202110950880.9A patent/CN115708123A/en active Pending
-
2022
- 2022-08-12 WO PCT/CN2022/111972 patent/WO2023020371A1/en active Application Filing
Also Published As
| Publication number | Publication date |
|---|---|
| WO2023020371A1 (en) | 2023-02-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11877102B2 (en) | Efficient, dynamic, high contrast lensing with applications to imaging, illumination and projection | |
| US10942428B2 (en) | Numerical approaches for free-form lensing: area parameterization free-form lensing | |
| CN109685869B (en) | Virtual model rendering method and device, storage medium and electronic equipment | |
| CN108471525B (en) | Control method and device for projector and projector for implementing method | |
| US20190025583A1 (en) | Near Eye Dynamic Holography | |
| JP2017505555A5 (en) | ||
| US20190004476A1 (en) | Dynamic Holography Printing Device | |
| KR102250189B1 (en) | Apparatus and method for holographic generation | |
| US10495955B2 (en) | Method and apparatus for controlling projector | |
| JP7196838B2 (en) | PHASE MODULATION DATA GENERATOR, LIGHTING DEVICE, AND PROJECTOR | |
| CN115708123A (en) | Image display method, image display device, electronic device, and storage medium | |
| CN113168019B (en) | Diffractive optical element for large-field imaging | |
| CN112819929A (en) | Water surface rendering method and device, electronic equipment and storage medium | |
| CN116309073B (en) | Low-contrast stripe SIM reconstruction method and system based on deep learning | |
| Gamo-Aranda et al. | Neural network-based system for pattern recognition through a fiber optic bundle | |
| CN118762117A (en) | Light rendering method, device, medium and program product for virtual model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |