WO2023020371A1 - 图像显示方法、装置、电子设备及存储介质 - Google Patents

图像显示方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2023020371A1
WO2023020371A1 PCT/CN2022/111972 CN2022111972W WO2023020371A1 WO 2023020371 A1 WO2023020371 A1 WO 2023020371A1 CN 2022111972 W CN2022111972 W CN 2022111972W WO 2023020371 A1 WO2023020371 A1 WO 2023020371A1
Authority
WO
WIPO (PCT)
Prior art keywords
light wave
lens model
phase
target
trained
Prior art date
Application number
PCT/CN2022/111972
Other languages
English (en)
French (fr)
Inventor
赵鹏
余新
弓殷强
李屹
Original Assignee
深圳光峰科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳光峰科技股份有限公司 filed Critical 深圳光峰科技股份有限公司
Publication of WO2023020371A1 publication Critical patent/WO2023020371A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3102—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM] using two-dimensional electronic spatial light modulators
    • H04N9/312—Driving therefor
    • H04N9/3126—Driving therefor for spatial light modulators in series
    • G—PHYSICS
    • G02—OPTICS
    • G02F—OPTICAL DEVICES OR ARRANGEMENTS FOR THE CONTROL OF LIGHT BY MODIFICATION OF THE OPTICAL PROPERTIES OF THE MEDIA OF THE ELEMENTS INVOLVED THEREIN; NON-LINEAR OPTICS; FREQUENCY-CHANGING OF LIGHT; OPTICAL LOGIC ELEMENTS; OPTICAL ANALOGUE/DIGITAL CONVERTERS
    • G02F1/00—Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulating; Non-linear optics
    • G02F1/01—Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulating; Non-linear optics for the control of the intensity, phase, polarisation or colour 
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/50—Lighting effects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3179—Video signal processing therefor

Definitions

  • the present application relates to the field of display technology, and more specifically, to an image display method, device, electronic equipment, and storage medium.
  • High Dynamic Range Imaging (HDR for short), usually used in computer graphics or film photography technology, is a technology used to expand the brightness exposure range.
  • the image processed by HDR technology has a brighter range than ordinary digital
  • the brightness dynamic range of the image is larger, that is, the HDR technology makes the bright areas in the image brighter and the dark areas darker, increasing the difference between light and dark in the image.
  • the collimated laser is generated by the laser, and the collimated laser is irradiated on the galvanometer of the high-speed vibrating Microelectromechanical Systems (MEMS), and the deflection of the beam by the galvanometer will scan the display area.
  • MEMS Microelectromechanical Systems
  • the modulation of the displayed image is controlled by controlling the switch of the laser, but this method requires lasers and MEMS, the cost is high, and the brightness, resolution, and bit depth of the displayed image are not good.
  • the present application proposes an image display method, device, electronic equipment and storage medium, which can solve the above problems.
  • an embodiment of the present application provides an image display method, the method comprising: acquiring a trained lens model, and loading the phase of a target image onto the trained lens model, wherein the trained The trained lens model is a linearized model; when the first light wave is input into the trained lens model, the second light wave output after the phase modulation of the trained lens model is obtained, wherein the second light wave carries the Phase information of the target image; input the second light wave into the intensity spatial light modulator to obtain the second light wave output by the intensity spatial light modulator after intensity modulation, so that the intensity modulated second light wave carries the phase information and intensity information of the target image and display the target image on the display plane.
  • an embodiment of the present application provides an image display device, the device comprising: an acquisition module, configured to acquire a trained lens model, and load the phase of the target image onto the trained lens model, Wherein, the trained lens model is a linearized model; the output module is used to obtain the second light wave output after the phase modulation of the trained lens model when the first light wave is input into the trained lens model , wherein the second light wave carries phase information of the target image; a display module is configured to input the second light wave into the intensity spatial light modulator, and obtain the second light wave output by the intensity spatial light modulator after intensity modulation light waves, so that the intensity-modulated second light waves carry the phase information and intensity information of the target image and display the target image on the display plane.
  • the embodiment of the present application provides an electronic device, including: one or more processors; memory; one or more application programs, wherein the one or more application programs are stored in the memory and Configured to be executed by the one or more processors, the one or more application programs are configured to perform the above method.
  • the embodiment of the present application provides a computer-readable storage medium, where a program code is stored in the computer-readable storage medium, and the program code can be invoked by a processor to execute the above method.
  • the image display method, device, electronic device, and storage medium provided by the present application obtain a trained lens model, and load the phase of the target image onto the trained lens model, wherein the trained lens model is a linearized model,
  • the trained lens model is a linearized model
  • the trained lens model can output the second light wave stably and quickly, and then pass the intensity spatial light modulator to the second light wave The intensity of the light wave is modulated, and the output intensity-modulated second light wave is projected on the display plane to quickly
  • FIG. 1 shows a schematic diagram of a modulator of an electronic device provided by an embodiment of the present application
  • FIG. 2 shows a schematic flowchart of an image display method provided by an embodiment of the present application
  • Figure 3 shows a schematic diagram of light steering provided by an embodiment of the present application
  • Fig. 4a shows a schematic diagram of light and shade distribution of the second light wave
  • Figure 4b shows a schematic diagram of the light and shade distribution of the displayed image
  • FIG. 5 shows a schematic flowchart of an image display method provided by another embodiment of the present application.
  • FIG. 6 shows a schematic flowchart of step S220 of the image display method shown in FIG. 5 of the present application
  • FIG. 7 shows a block diagram of an image display device provided by an embodiment of the present application.
  • FIG. 8 is a block diagram of an electronic device for performing an image display method according to an embodiment of the present application.
  • FIG. 9 shows a storage unit for storing or carrying program codes for realizing the image display method according to the embodiment of the present application according to the embodiment of the present application.
  • High Dynamic Range Imaging (HDR for short), usually used in computer graphics or film photography technology, is a technology used to expand the brightness exposure range.
  • the image processed by HDR technology has a brighter range than ordinary digital
  • the brightness range of the image is larger, that is, the HDR technology makes the bright areas in the image brighter and the dark areas darker, increasing the difference between light and dark in the image.
  • a collimated laser can be generated by a laser, and the collimated laser is irradiated on the galvanometer of a high-speed vibrating microelectromechanical system (Microelectromechanical Systems, referred to as MEMS), and the deflection of the beam by the galvanometer will be The display area is scanned, and the modulation of the displayed image is controlled by controlling the laser switch on and off the laser.
  • MEMS microelectromechanical Systems
  • a specific light steering device is used to dynamically modulate the illumination light, and dynamically distribute the energy of the illumination light according to the display content, so that the highlighted area in the display content can obtain higher illumination Brightness, to achieve high-brightness display, and at the same time, it will also reduce the brightness of the dark part of the screen to obtain a purer dark field display performance, and then realize the HDR display effect.
  • this method cannot adapt to HDR display of different patterns, and at the same time, the calculation method is complicated and the calculation speed is slow, making it difficult to realize real-time HDR display.
  • the inventor discovered and proposed an image display method, device, electronic equipment and storage medium after long-term research, which acquires a trained lens model and loads the phase of the target image into the trained lens model , wherein the trained lens model is a linearized model, when the first light wave is input into the trained lens model, the second light wave output by the trained lens model for phase modulation is obtained, wherein the second light wave carries the target image Phase information, inputting the second light wave into the intensity spatial light modulator to obtain the second light wave output by the intensity spatial light modulator after intensity modulation, so that the intensity modulated second light wave carries the phase information and intensity information of the target image, and the intensity The modulated second light wave displays the target image on the display plane.
  • the trained lens model is a linear model
  • the trained lens model when the first light wave is input into the trained lens model, the trained lens model can output stably and quickly
  • the intensity of the second light wave is modulated by the intensity spatial light modulator, and the output intensity modulated second light wave is projected on the display plane to quickly display the target image, reducing the cost of image display, and can display brightness, resolution, High-quality images with good bit depth.
  • the specific image display method is described in detail in the following embodiments.
  • the modulator includes a phase spatial light modulator 230 and an intensity spatial light modulator 240, the phase spatial light modulator 230 and The intensity spatial light modulators 240 are separated by a focal length f, and they are arranged parallel to each other on the same plane, wherein the phase spatial light modulator 230 may be a linearized lens model.
  • the first light wave is incident into the phase spatial light modulator 230, and after the phase spatial light modulator 230 performs light steering, the second light wave is output.
  • the phase spatial light modulator 230 performs phase conversion on the first light wave
  • the second light wave enters the intensity spatial light modulator 240 to modulate the light intensity
  • the intensity spatial light modulator 240 outputs the intensity-modulated second light wave and projects it onto the display plane to display an image on the display plane.
  • Fig. 2 shows a schematic flow chart of an image display method provided by an embodiment of the present application.
  • the image display method is applied to the image display device 100 as shown in Fig. 7 and configured with The electronic device 200 of the image display device 100 .
  • This embodiment will take the image display method applied to electronic equipment 200 as an example to illustrate the specific process of this embodiment.
  • the electronic equipment can be, but not limited to, a projector, a TV, a computer, a smart phone, a tablet computer, and a smart wearable device. , lighting system, etc.
  • the flow process shown in Figure 1 will be described in detail below, and the image display method may specifically include the following steps:
  • Step S110 acquiring a trained lens model, and loading the phase of the target image onto the trained lens model, wherein the trained lens model is a linearized model.
  • the lens model is pre-trained in combination with the phase distribution function, and the trained lens model is stored, and the trained lens model is obtained from the storage location.
  • the storage location can be a local storage location of the electronic device, or can be a The storage location of the server for the communication connection.
  • the above-mentioned trained lens model is a linear model, which has the advantages of fast calculation speed and high stability.
  • Step S120 when the first light wave is input into the trained lens model, obtain the second light wave output by the trained lens model after phase modulation, wherein the second light wave carries the phase information of the target image .
  • the first light wave is uniform illumination light, and the first light wave can be directly generated by a light source, and the light source can be a lamp in an electronic device, a light emitting diode (Light Emitting Diode, LED for short) or a laser array with different wavelengths.
  • a light source can be a lamp in an electronic device, a light emitting diode (Light Emitting Diode, LED for short) or a laser array with different wavelengths.
  • the first light wave is incident on the trained lens model.
  • the trained lens model can be equivalent to a free-form surface, and the first light wave is incident on the already-trained lens model.
  • the lens model it can be equivalent to incident on the free-form surface from the normal direction of the free-form surface.
  • the trained lens model performs phase modulation on the first light wave, that is, the trained lens model is used to redistribute the input light field, that is, it is used to divert the input light wave to obtain the output light wave.
  • the trained lens model performs phase modulation on the first light wave to obtain the second light wave.
  • the first light wave is modulated by the trained lens model to produce a steering effect
  • the second light wave the phase of the light wave has changed after the steering process, that is, the phase of the first light wave is different from the phase of the second light wave
  • the light steering of the trained lens model provides a lossless light intensity distribution modulation, after the modulation
  • the second light wave has higher maximum brightness and lower minimum brightness, which expands the dynamic range of the first light wave itself, and finally realizes high dynamic range display, and the phase distribution of the second light wave obtained after phase modulation is roughly the same as The target image matches. Since the trained lens model is essentially a linearized lens model, the trained lens model can output the second light wave quickly and stably.
  • the trained lens model simulates a free-form surface lens.
  • the free-form surface lens can be, but not limited to, a phase space intensity spatial light modulator, a variable reflector, a single crystal silicon reflective Liquid crystal (Liquid Crystal On Silicon; LCoS for short), etc.
  • the trained lens model simulates a free-form surface through which the first light wave is diverted, wherein the free-form surface is a non-rotationally symmetrical special-shaped surface.
  • the trained lens model simulates a free-form surface A
  • the coordinates of the free-form surface A are
  • the coordinates of the light field plane B (which may be the plane corresponding to the intensity spatial light modulator) are
  • the free-form surface A and the light field plane B are arranged in parallel, and there is a fixed distance between the free-form surface A and the light field plane B.
  • the second light wave L2 propagates from the position a1 on the free-form surface A to the position b2 on the display plane along a straight line.
  • the free-form surface A has the function of steering processing, so the first light wave L1 is turned from the position a1 to become the second light wave L2, and the second light wave L2 is projected to the target position b1 of the light field plane B, so that the incident on the target position b1 There are more light waves, and the light intensity of the target position b1 is enhanced as a bright area. Since the light waves that should be gathered at the b2 position are gathered at the target position b1, the light waves at the b2 position are less, and the light intensity of the b2 position is weakened as a dark area.
  • a certain fixed distance can be determined according to the wavelength of the input light wave. Specifically, the distance corresponding to the wavelength with the best display effect is obtained through experiments, and the distance is repeated several times.
  • the fixed pitch is not limited to the above-mentioned 50 mm, and the fixed pitch can also be 40 mm, 60 mm, etc. according to different wavelengths.
  • the resolution of the image sample is 512*512, assuming that a trained lens model consistent with its resolution is used, the size of a single pixel of the trained lens model is 8um, and the wavelength of the first light wave is set to 532nm
  • the fixed distance between the modulated light field plane and the phase plane is 25cm.
  • FIG. 3 only shows one first light wave L1, and in actual applications, there are multiple first light waves. Input the light field formed by multiple first light waves into the trained lens model at the same time, and obtain the second light wave output by the trained lens model. If the second light wave is irradiated on the display plane, it will present a certain light and dark distribution in advance (as shown in Figure 4a shown). Moreover, the second light wave is not limited to incident from the position a1 of the free-form surface A as shown in FIG. 3 , and may also be incident at other positions on the free-form surface.
  • Step S130 inputting the second light wave into the intensity spatial light modulator, and obtaining the intensity-modulated second light wave output by the intensity spatial light modulator, so that the intensity-modulated second light wave carries the phase of the target image information and intensity information, and display the target image on the display plane.
  • the intensity spatial light modulator is loaded with the intensity information of the target image, and the second light wave is input into the intensity spatial light modulator, and the intensity modulated second light wave output by the intensity spatial light modulator, because the intensity spatial light modulator is loaded with the target
  • the intensity information of the image so the light wave output by the intensity spatial light modulator carries the phase information of the target image.
  • the intensity-modulated second light wave is projected on the display plane, and the intensity-modulated second light wave carries the phase information and intensity information of the target image to generate a display image as shown in Figure 1, and the light intensity of the display image is shown in Figure 1
  • the intensity spatial light modulator achieves more detailed intensity regulation, which increases the peak brightness of the final imaged display image by up to 10 times.
  • the image display method provided in this embodiment acquires a trained lens model, and loads the phase of the target image onto the trained lens model, wherein the trained lens model is a linearized model, when the first light wave input has been trained
  • the second light wave output by the phase modulation of the trained lens model is obtained, wherein the second light wave carries the phase information of the target image, and the second light wave is input into the intensity spatial light modulator to obtain the intensity spatial light modulator intensity
  • the modulated output second light wave makes the intensity-modulated second light wave carry the phase information and intensity information of the target image, and the intensity-modulated second light wave displays the target image on the display plane, since the trained lens model is Linearized model, therefore, when the first light wave is input into the trained lens model, the trained lens model can output the second light wave stably and quickly, and then the intensity of the second light wave is modulated by the intensity spatial light modulator, and the output intensity
  • the modulated second light wave is projected on the display plane to quickly display the target image, which reduces the cost of image display
  • Fig. 5 shows a schematic flow chart of an image display method provided by another embodiment of the present application. Before using the trained lens model, the lens model needs to be trained. Please refer to Fig. 5.
  • the image display method may specifically include the following step:
  • Step S210 obtaining a target phase conversion formula, wherein the target phase conversion formula is obtained by linearizing the initial phase conversion formula.
  • the gradient of the phase fluctuation of the free-form surface determines the deflection angle of the beam, and the second-order derivative of the phase fluctuation determines the curvature of the free-form surface, which determines the control degree of the phase plane on the convergence and divergence of the beam.
  • the curvature of the free-form surface can be simulated by the phase distribution function, therefore, the target phase transformation formula can be constructed by the phase distribution function, as follows:
  • the coordinates of the free-form surface A are The coordinates of the light field plane B are The free-form surface A and the light field plane B are set in parallel, and the distance between the free-form surface A and the light field plane B is the focal length f, and the coordinates of the free-form surface A are and the coordinates of the light field plane B The corresponding relationship between them is:
  • the initial phase conversion formula is linearized to obtain the target phase conversion formula, which specifically includes:
  • the initial target conversion formula (3) is obtained after reciprocal processing and square root processing of the initial phase conversion formula, as follows:
  • Step S220 training the target phase transformation formula by inputting the light wave and the target image to obtain a trained lens model.
  • the input light wave enters the target phase conversion formula, and the target phase conversion formula outputs light waves.
  • the ideal situation of the output light wave is to project the target image, and the target phase conversion formula is trained with the input light wave and the target image to establish the relationship between the input light wave and the target image. Correspondence.
  • step S220 includes the following sub-steps:
  • Sub-step S221 Input the input light wave into the target phase conversion formula, and obtain the predicted image output by the target phase conversion formula.
  • the phase of the input light wave is deformed, thereby causing the curved phase distribution function to locally deflect the input light.
  • the target phase conversion formula outputs the output light wave after the phase deflection, and the output light wave projects a predicted image.
  • the target phase conversion formula When the phase of the predicted image output by the target phase conversion formula is consistent with the phase of the target image, the target phase conversion formula has established the corresponding relationship between the input light wave and the target image, the training of the target phase conversion formula is completed, and the trained lens is obtained Model.
  • Sub-step S222 updating the target phase conversion formula according to the predicted image and the target image to obtain the trained lens model.
  • the intensity difference between the intensity of the preset image and the intensity of the target image is calculated, and the target phase conversion formula is iteratively trained according to the intensity difference until the phase difference satisfies the iterative training condition, and the trained lens model is obtained , wherein the iterative training condition may be that the intensity difference is smaller than the preset intensity difference. Since the target phase conversion formula has been linearized, the target phase conversion formula can be output quickly, which speeds up the iteration speed and speeds up the model training.
  • the phase difference between the predicted image and the target image is calculated, and the target phase conversion formula is iteratively trained according to the phase difference until the phase difference satisfies the iterative training condition, and a trained lens model is obtained, wherein , the iteration training condition may be that the phase difference is smaller than a preset phase difference. Since the target phase conversion formula has been linearized, the target phase conversion formula can be output quickly, which speeds up the iteration speed and speeds up the model training.
  • i 0 is the first light wave
  • i 1 is the second light wave
  • f is the focal length
  • p(x) is a phase distribution function
  • the phase distribution function p(x) can reflect the focal length of the lens corresponding to the free-form surface, which determines the degree of convergence and divergence of the light beam at this point, and determines the luminance distribution after adjustment.
  • the iterative training is the iterative training of the target phase conversion formula.
  • iterative training is performed on the phase distribution function p(x), and the phase that minimizes the error between the intensity of the light wave output by the model and the target intensity is obtained. Distribution function.
  • the light beams input to the trained lens model are converged and diverged to different degrees, corresponding to different trained lens models, that is, for different convergence and divergence requirements, different models have Different phase distribution functions.
  • the above model can be trained by using multiple sets of input light waves and target images corresponding to the input light waves.
  • the accuracy of the trained lens model is higher.
  • the trained lens model can be processed as follows: obtain a convolution operator; perform the trained lens model through the convolution operator and fast Fourier transform to process.
  • the near-end operator can be used to achieve fast and efficient calculations, and the stability of the trained lens model calculation can also be enhanced through the damping factor and damping term.
  • the damping factor and damping term can make the calculation results of the model converge quickly. And make the output luminance distribution of the model modulation consistent with the target luminance distribution of the target image, there is less distortion, specifically, obtain the near-end operator and damping factor; through the near-end operator and the damping factor to The trained lens model is processed.
  • the trained lens model can be applied to various types of images.
  • the image may include Jpg, png, tif and other types.
  • the selected convolution operator, near-end operator and damping factor can be different, so that the brightness distribution output by the trained lens model is close to the target brightness distribution of the target image, so as to improve The final display effect of the target image.
  • the trained lens model can be stored locally on the electronic device, or stored in a server communicatively connected to the electronic device, and can be directly called from the storage location when using the trained lens model.
  • the trained lens model may be stored locally in the electronic device after the pre-training is completed.
  • the trained lens model can be quickly called, when the first light wave is input, the trained lens model can quickly output the second light wave, and, since the model is stored locally in the electronic device, it can also It effectively avoids the reduction of the second optical wave output speed of the trained lens model due to the influence of network factors, and the response speed is fast, which further improves the user experience.
  • the trained lens model may be stored in a server communicatively connected with the electronic device after the pre-training is completed.
  • the electronic device sends the information of the first light wave (including light intensity and phase, etc.) to the trained lens model of the server to instruct the trained lens model to output the information of the second light wave, And obtain the information of the second light wave sent by the server, and generate the second light wave according to the information of the second light wave, so that by storing the trained lens model in the server, the occupation of the storage space of the electronic device is reduced, and the normal operation of the electronic device is reduced. impact on operation.
  • Step S230 acquiring a trained lens model, and loading the phase of the target image onto the trained lens model, wherein the trained lens model is a linearized model.
  • Step S240 when the first light wave is input into the trained lens model, obtain the second light wave output by the trained lens model after phase modulation, wherein the second light wave carries the phase information of the target image .
  • Step S250 input the second light wave into the intensity spatial light modulator, and obtain the second light wave output by the intensity spatial light modulator after intensity modulation, so that the intensity modulated second light wave carries the phase of the target image information and intensity information and display the target image on the display plane.
  • step S230-step S250 please refer to step S110-step S130, which will not be repeated here.
  • the present application provides an image display method, device, electronic equipment and storage medium.
  • a trained lens model is obtained through iterative training of the lens model.
  • the trained lens model is a linear model that can be stably and quickly output Lightwave, improving the speed and quality of final imaging.
  • FIG. 7 shows a block diagram of an image display device provided by an embodiment of the present application. Please refer to FIG. 7.
  • the image display device 100 includes: an acquisition module 110 , an output module 120 and a display module 130 .
  • An acquisition module 110 configured to acquire a trained lens model, and load the phase of the target image onto the trained lens model, wherein the trained lens model is a linearized model;
  • An output module 120 configured to obtain a second light wave output by the trained lens model after phase modulation when the first light wave is input into the trained lens model, wherein the second light wave carries the target image phase information;
  • the display module 130 is configured to input the second light wave into the intensity spatial light modulator, and obtain the second light wave output by the intensity spatial light modulator after intensity modulation, so that the intensity modulated second light wave carries the target Phase information and intensity information of the image and display the target image on the display plane.
  • the image display device 100 further includes: a construction module and a linearization module.
  • a linearization module configured to linearize the initial phase conversion formula to obtain the target phase conversion formula.
  • the linearization module includes: a derivation processing submodule and a root square processing submodule.
  • the derivation processing sub-module is used to perform reciprocal processing on the initial phase conversion formula to obtain the reciprocal conversion formula
  • the square root processing sub-module is configured to perform square root processing on the conversion formula after calculating the reciprocal, to obtain the target phase conversion formula.
  • the target phase conversion formula is:
  • i 0 is the first light wave
  • i 1 is the second light wave
  • f is the distance between the trained lens model and the display plane
  • p(x) is a phase distribution function
  • the image display device 100 further includes: a target phase conversion formula module and a training module.
  • a target phase conversion formula module configured to obtain a target phase conversion formula, wherein the target phase conversion formula is obtained by linearizing an initial phase conversion formula
  • the training module is used to train the target phase conversion formula by inputting the light wave and the target image to obtain the trained lens model.
  • the training module includes: a prediction module and an update module.
  • a prediction module configured to input the input light wave into the target phase conversion formula, and obtain a predicted image output by the target phase conversion formula
  • An update module configured to update the target phase transformation formula according to the predicted image and the target image, to obtain the trained lens model.
  • the update module includes: a phase difference acquisition module and a formula update module.
  • phase difference acquisition module configured to obtain a phase difference between the predicted image and the target image
  • a formula updating module configured to update the target phase conversion formula according to the phase difference to obtain the trained lens model.
  • the image display device 100 further includes: a convolution operator acquisition module and a first model processing module.
  • a convolution operator acquisition module configured to acquire a convolution operator
  • the first model processing module is configured to process the trained lens model through the convolution operator and fast Fourier transform.
  • the image display device 100 further includes: a proximal operator and damping factor acquisition submodule, and a second model processing module.
  • the near-end operator and damping factor acquisition sub-module is used to obtain the near-end operator and damping factor
  • the second model processing module is used to process the trained lens model through the proximal operator and the damping factor.
  • the coupling between the modules may be electrical, mechanical or other forms of coupling.
  • each functional module in each embodiment of the present application may be integrated into one processing module, each module may exist separately physically, or two or more modules may be integrated into one module.
  • the above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules.
  • FIG. 8 is a block diagram of an electronic device for performing an image display method according to an embodiment of the present application.
  • the electronic device 200 may be an electronic device capable of running application programs, such as a smart phone, a tablet computer, and an e-book.
  • the electronic device 200 in this application may include one or more of the following components: a processor 210, a memory 220, and one or more application programs, wherein one or more application programs may be stored in the memory 220 and configured to be executed by a or a plurality of processors 210, and one or more programs are configured to execute the methods described in the foregoing method embodiments.
  • the processor 210 may include one or more processing cores.
  • the processor 210 uses various interfaces and lines to connect various parts in the entire electronic device 200, and executes or executes by running or executing instructions, programs, code sets or instruction sets stored in the memory 220, and calling data stored in the memory 220.
  • the processor 210 may adopt at least one of Digital Signal Processing (Digital Signal Processing, DSP), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and Programmable Logic Array (Programmable Logic Array, PLA). implemented in the form of hardware.
  • DSP Digital Signal Processing
  • FPGA Field-Programmable Gate Array
  • PLA Programmable Logic Array
  • the processor 210 may integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU), a modem, and the like.
  • CPU Central Processing Unit
  • GPU Graphics Processing Unit
  • the CPU mainly handles the operating system, user interface and application programs, etc.
  • the GPU is used for rendering and drawing of components to be displayed
  • the modem is used for wireless communication. It can be understood that, the above-mentioned modem may not be integrated into the processor 210, but may be realized by a communication chip alone.
  • the memory 220 may include random access memory (Random Access Memory, RAM), and may also include read-only memory (Read-Only Memory).
  • the memory 220 may be used to store instructions, programs, codes, sets of codes, or sets of instructions.
  • the memory 220 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system and instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.) , instructions for implementing the following method embodiments, and the like.
  • the data storage area can also store data (such as historical configuration files) created by the electronic device 200 during use.
  • Fig. 9 shows a storage unit used to store or carry program codes for realizing the image display method according to the embodiment of the present application according to the embodiment of the present application.
  • Program codes are stored in the computer-readable medium 300, and the program codes can be invoked by a processor to execute the methods described in the foregoing method embodiments.
  • the computer readable storage medium 300 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), EPROM, hard disk, or ROM.
  • the computer-readable storage medium 300 includes a non-transitory computer-readable storage medium (non-transitory computer-readable storage medium).
  • the computer-readable storage medium 300 has a storage space for program code 310 for executing any method steps in the above methods. These program codes can be read from or written into one or more computer program products.
  • Program code 310 may, for example, be compressed in a suitable form.
  • the present application provides an image display method, device, electronic equipment, and storage medium to obtain a trained lens model, and load the phase of the target image onto the trained lens model, wherein the trained lens
  • the model is a linearized model.
  • the intensity spatial light modulator obtains the second light wave output by the intensity spatial light modulator after intensity modulation, so that the intensity modulated second light wave carries the phase information and intensity information of the target image, and the intensity modulated second light wave is displayed on the display
  • the target image is displayed on the plane.
  • the trained lens model is a linear model
  • the trained lens model can output the second light wave stably and quickly, and then pass through the intensity space
  • the light modulator performs intensity modulation on the second light wave, and the output intensity-modulated second light wave is projected on the display plane to quickly display the target image, reduce the cost of image display, and can display high-quality images with excellent brightness, resolution, and bit depth .

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Nonlinear Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Graphics (AREA)
  • Theoretical Computer Science (AREA)
  • Optics & Photonics (AREA)
  • Control Of Indicators Other Than Cathode Ray Tubes (AREA)

Abstract

本申请公开了一种图像显示方法、装置、电子设备及存储介质，涉及显示技术领域，包括：获取已训练的透镜模型，并将目标图像的相位加载到已训练的透镜模型上，其中，已训练的透镜模型为线性化模型，当第一光波输入已训练的透镜模型时，获得已训练的透镜模型进行相位调制输出的第二光波，其中，第二光波携带目标图像的相位信息，将第二光波输入强度空间光调制器，获得强度空间光调制器强度调制后输出的第二光波，使得强度调制后的第二光波携带有目标图像的相位信息和强度信息，并且强度调制后的第二光波在显示平面上显示目标图像。

Description

图像显示方法、装置、电子设备及存储介质 技术领域
本申请涉及显示技术领域，更具体地，涉及一种图像显示方法、装置、电子设备及存储介质。
背景技术
高动态范围成像(High Dynamic Range Imaging，简称HDR)，通常应用在计算机图形学或电影摄影技术中，是用来扩大亮度曝光范围的技术，经过HDR技术处理后的图像，其亮度范围比普通数位图像的亮度动态范围更大，即HDR技术使得图像中的亮区更亮，暗区更暗，增大图像的亮暗差别。
在现有的HDR技术中，通过激光器产生准直激光，使用准直激光照射在高速振动的微机电系统(Microelectromechanical Systems，简称MEMS)的振镜上，振镜对光束的偏转会扫描出显示区域，通过对激光器的开关的控制激光来控制显示图像的调制，但这种方式需要激光器和MEMS，成本较高，并且显示的图像的亮度、分辨率、位深均不佳。
发明内容
鉴于上述问题，本申请提出了一种图像显示方法、装置、电子设备及存储介质，能够解决上述问题。
第一方面，本申请实施例提供了一种图像显示方法，所述方法包括：获取已训练的透镜模型，并将目标图像的相位加载到所述已训练的透镜模型上，其中，所述已训练的透镜模型为线性化模型；当第一光波输入所述已训练的透镜模型时，获得所述已训练的透镜模型进行相位调制后输出的 第二光波，其中，所述第二光波携带所述目标图像的相位信息；将所述第二光波输入强度空间光调制器，获得所述强度空间光调制器强度调制后输出的第二光波，以使强度调制后的第二光波携带有所述目标图像的相位信息和强度信息并在显示平面显示目标图像。
第二方面，本申请实施例提供了一种图像显示装置，所述装置包括：获取模块，用于获取已训练的透镜模型，并将目标图像的相位加载到所述已训练的透镜模型上，其中，所述已训练的透镜模型为线性化模型；输出模块，用于当第一光波输入所述已训练的透镜模型时，获得所述已训练的透镜模型进行相位调制后输出的第二光波，其中，所述第二光波携带所述目标图像的相位信息；显示模块，用于将所述第二光波输入强度空间光调制器，获得所述强度空间光调制器强度调制后输出的第二光波，以使强度调制后的第二光波携带有所述目标图像的相位信息和强度信息并在显示平面显示目标图像。
第三方面，本申请实施例提供了一种电子设备，包括：一个或多个处理器；存储器；一个或多个应用程序，其中所述一个或多个应用程序被存储在所述存储器中并被配置为由所述一个或多个处理器执行，所述一个或多个应用程序配置用于执行上述方法。
第四方面，本申请实施例提供了一种计算机可读存储介质，所述计算机可读存储介质中存储有程序代码，所述程序代码可被处理器调用执行上述方法。
本申请提供的图像显示方法、装置、电子设备及存储介质，获取已训练的透镜模型，并将目标图像的相位加载到已训练的透镜模型上，其中，已训练的透镜模型为线性化模型，当第一光波输入已训练的透镜模型时，获得已训练的透镜模型进行相位调制输出的第二光波，其中，第二光波携带目标图像的相位信息，将第二光波输入强度空间光调制器，获得强度空间光调制器强度调制后输出的第二光波，使得强度调制后的第二光波携带有目标图像的相位信息和强度信息，并且强度调制后的第二光波在显示平面上显示目标图像，由于已训练的透镜模型为线性化模型，因此，当第一光波输入已训练的透镜模型时，已训练的透镜模型可以稳定、快速地输出 第二光波，再通过强度空间光调制器对第二光波进行强度调制，输出强度调制后的第二光波投射在显示平面上快速显示目标图像，降低图像显示成本，且可以显示亮度、分辨率、位深俱佳的高质量图像。
本申请的这些方面或其他方面在以下实施例的描述中会更加简明易懂。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1示出了本申请一个实施例提供的电子设备的调制器示意图；
图2示出了本申请一个实施例提供的图像显示方法的流程示意图；
图3示出了本申请一个实施例提供的光转向示意图；
图4a示出了第二光波明暗分布示意图；
图4b示出了显示图像明暗分布示意图；
图5示出了本申请另一个实施例提供的图像显示方法的流程示意图；
图6示出了本申请的图5所示的图像显示方法的步骤S220的一种流程示意图；
图7示出了本申请一实施例提供的图像显示装置的框图；
图8是本申请实施例的用于执行根据本申请实施例的图像显示方法的电子设备的框图；
图9示出了本申请实施例的用于保存或者携带实现根据本申请实施例的图像显示方法的程序代码的存储单元。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造 性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
以下，对本申请实施例中可能涉及的术语进行介绍。
高动态范围成像(High Dynamic Range Imaging，简称HDR)，通常应用在计算机图形学或电影摄影技术中，是用来扩大亮度曝光范围的技术，经过HDR技术处理后的图像，其亮度范围比普通数位图像的亮度范围更大，即HDR技术使得图像中的亮区更亮，暗区更暗，增大图像的亮暗差别。
在现有的HDR的一种方式中，可以通过激光器产生准直激光，使用准直激光照射在高速振动的微机电系统(Microelectromechanical Systems，简称MEMS)的振镜上，振镜对光束的偏转会扫描出显示区域，通过对激光器的开关的控制激光来控制显示图像的调制。但是，发明人发现，这种方式需要激光器和MEMS，成本较高，并且显示的图像的亮度、分辨率、位深均不佳。
在现有HDR的另一种方式中，利用特定的光转向器件对照明光进行动态调制，并根据显示内容动态的分配照明光的能量，可以使显示内容中的高亮区域获得更高的照明亮度，实现高亮度显示，同时也会使画面中的暗部照明亮度下降，得到更纯净的暗场显示表现，进而实现HDR显示效果。但是，发明人发现，该方式无法适配不同图案的HDR显示，同时计算方式复杂，计算速度较慢，难以实现实时HDR显示。
为了解决上述技术问题，发明人经过长期的研究发现并提出了一种图像显示方法、装置、电子设备及存储介质，获取已训练的透镜模型，并将目标图像的相位加载到已训练的透镜模型上，其中，已训练的透镜模型为线性化模型，当第一光波输入已训练的透镜模型时，获得已训练的透镜模型进行相位调制输出的第二光波，其中，第二光波携带目标图像的相位信息，将第二光波输入强度空间光调制器，获得强度空间光调制器强度调制后输出 的第二光波，使得强度调制后的第二光波携带有目标图像的相位信息和强度信息，并且强度调制后的第二光波在显示平面上显示目标图像，由于已训练的透镜模型为线性化模型，因此，当第一光波输入已训练的透镜模型时，已训练的透镜模型可以稳定、快速地输出第二光波，再通过强度空间光调制器对第二光波进行强度调制，输出强度调制后的第二光波投射在显示平面上快速显示目标图像，降低图像显示成本，且可以显示亮度、分辨率、位深俱佳的高质量图像。其中，具体的图像显示方法在后续的实施例中具体说明。
为便于理解图像显示方法，本申请示出了一种电子设备的调制器示意图，请参阅图1，调制器包括相位空间光调制器230和强度空间光调制器240，相位空间光调制器230和强度空间光调制器240之间间隔焦距f，且两者彼此平行的设置在同一平面上，其中，相位空间光调制器230可以为线性化的透镜模型。
第一光波入射到相位空间光调制器230中，经过相位空间光调制器230进行光转向后，输出第二光波，可以理解的是，相位空间光调制器230对第一光波进行相位的转换，第二光波入射到强度空间光调制器240中进行光强度的调制，强度空间光调制器240输出强度调制后的第二光波投射到显示平面上，在显示平面上显示图像。
图2示出了本申请一个实施例提供的图像显示方法的流程示意图，在具体的实施例中，所述图像显示方法应用于如图7所示的图像显示装置100以及配置有如图8所示的图像显示装置100的电子设备200。本实施例将以图像显示方法应用于电子设备200为例说明本实施例的具体流程，所述电子设备可以为，但不限于投影仪、电视机、电脑、智能手机、平板电脑、智能穿戴设备、照明系统等。下面将针对图1所示的流程进行详细的阐述，所述图像显示方法具体可 以包括如下步骤：
步骤S110、获取已训练的透镜模型，并将目标图像的相位加载到所述已训练的透镜模型上，其中，所述已训练的透镜模型为线性化模型。
结合相位分布函数预先训练透镜模型，并且将已训练的透镜模型进行存储，从存储位置获取已训练的透镜模型，可选地，存储位置可以为电子设备本地的存储位置，也可以为与电子设备通信连接的服务器的存储位置。
其中，上述已训练的透镜模型为线性化模型，具有计算速度快、稳定性高等优点。
步骤S120、当第一光波输入所述已训练的透镜模型时，获得所述已训练的透镜模型进行相位调制后输出的第二光波，其中，所述第二光波携带所述目标图像的相位信息。
其中，第一光波为均匀照明光，第一光波可以为光源直接产生，光源可以为电子设备中的灯、发光二极管(Light Emitting Diode，简称LED)或者是具有不同波长的激光器阵列。
第一光波入射到已训练的透镜模型中，例如，可以理解，以仿真实验中常用的自由曲面模型为例，已训练的透镜模型可以等效为一种自由曲面，而第一光波入射到已训练的透镜模型时，即可等效为从自由曲面的法线方向入射到自由曲面上。
已训练的透镜模型对第一光波进行相位调制，即已训练的透镜模型用于将输入光场进行重新分配，即用于将输入其中的光波进行转向处理获得输出光波。
将第一光波输入已训练的透镜模型中，已训练的透镜模型对第一光波进行相位调制获得第二光波，可以理解的是，第一光波经过已训练的透镜模型被调 制，产生转向作用，获得第二光波，经过转向处理光波的相位已经发生变化，即第一光波的相位和第二光波的相位不同，已训练的透镜模型的光转向提供了无损的光强度分布调制，经过该调制后的第二光波具有更高的最高亮度和更低的最低亮度，拓展了第一光波自身的动态范围，进而最终实现高动态范围显示，并且经过相位调制后获得的第二光波的相位分布大致与目标图像相符。由于已训练的透镜模型实质上是已经线性化的透镜模型，因此，已训练的透镜模型可快速、稳定地输出第二光波。
在一种实施方式中，已训练的透镜模型模拟的是自由曲面透镜，可选地，自由曲面透镜可以为，但不限于相位空间强度空间光调制器、可变性反射镜、单晶硅反射式液晶(Liquid Crystal On Silicon；简称LCoS)等。已训练的透镜模型模拟出自由曲面，通过自由曲面对第一光波进行转向处理，其中，自由曲面是非旋转对称的异形曲面。
示例性的，如图3所示，已训练的透镜模型模拟出自由曲面A，自由曲面A所在的坐标为
Figure PCTCN2022111972-appb-000001
光场平面B(可以为强度空间光调制器对应的平面)的坐标为
Figure PCTCN2022111972-appb-000002
自由曲面A和光场平面B平行设置，并且自由曲面A和光场平面B之间间隔的某一固定间距，当第一光波L1从自由曲面A上的a1位置入射，经过自由曲面A对第一光波L1进行相位转换处理，自由曲面A从a1位置输出第二光波L2。经过相位转换后，第一光波L1的相位和第二光波L2的相位不同，第一光波L1和第二光波L2之间的相位差为θ。
请继续参阅图3，如果第二光波L2未经转向，则第二光波L2从自由曲面A上的a1位置沿直线传播到显示平面上的b2位置。而自由曲面A具有转向处理功能，故将第一光波L1从a1位置转向后变成第二光波L2，第二光波L2再投射到光场平面B的目标位置b1，从而对入射到目标位置b1的光波较多，目标位置b1 作为亮区其光强增强。由于本应该聚集在b2位置的光波聚集到目标位置b1，因此，b2位置的光波较少，b2位置作为暗区其光强减弱。
需要说明的是，可选地，为了保证最终的目标图像的显示效果，某一固定间距可以根据输入光波的波长决定，具体为，通过实验获得波长对应的显示效果最佳的间距，进行多次实验，获得多个波长中每个波长对应的固定间距，并建立多个波长和多个固定间距之间的对应关系，获取第一光波的波长，基于上述对应关系，确定第一光波的波长对应的固定间距，例如，当第一光波的波长为550nm(长度单位，纳米)时，固定间距可以为50mm(长度单位，毫米)。需要说明的是，固定间距不限于上述的50mm，还可以根据波长不同，固定间距可以为40mm、60mm等。例如，图片样例的分辨率为512*512，假定采用了与其分辨率一致的已训练的透镜模型，已训练的透镜模型的单个像素的尺寸为8um，第一光波的波长设定为532nm的绿光，调制后的光场平面与相位平面的固定距离为25cm。
还需要说明的是，图3仅示出了一条第一光波L1，在实际的应用中，第一光波的数量为多条。将多条第一光波形成的光场同时输入已训练的透镜模型，获得已训练的透镜模型输出的第二光波，第二光波如果照射在显示平面上预先呈现出一定的明暗分布(如图4a所示)。并且，第二光波不限于图3所示的从自由曲面A的a1位置入射，还可以在自由曲面上其他位置入射。
步骤S130、将所述第二光波输入强度空间光调制器，获得所述强度空间光调制器强度调制后输出的第二光波，以使强度调制后的第二光波携带有所述目标图像的相位信息和强度信息，并在显示平面显示目标图像。
强度空间光调制器被加载了目标图像的强度信息，将第二光波输入强度空间光调制器，强度空间光调制器输出的强度调制后的第二光波，由于强度空间 光调制器被加载了目标图像的强度信息，因此强度空间光调制器输出的光波中携带有目标图像的相位信息。强度调制后的第二光波投射在显示平面上，强度调制后的第二光波携带有所述目标图像的相位信息和强度信息，生成如图1所示的显示图像，显示图像的光强如图4b所示，通过强度空间光调制器实现了更细致的强度调控，使得最终成像的显示图像的亮度峰值最高提升10倍。
本实施例提供的图像显示方法，获取已训练的透镜模型，并将目标图像的相位加载到已训练的透镜模型上，其中，已训练的透镜模型为线性化模型，当第一光波输入已训练的透镜模型时，获得已训练的透镜模型进行相位调制输出的第二光波，其中，第二光波携带目标图像的相位信息，将第二光波输入强度空间光调制器，获得强度空间光调制器强度调制后输出的第二光波，使得强度调制后的第二光波携带有目标图像的相位信息和强度信息，并且强度调制后的第二光波在显示平面上显示目标图像，由于已训练的透镜模型为线性化模型，因此，当第一光波输入已训练的透镜模型时，已训练的透镜模型可以稳定、快速地输出第二光波，再通过强度空间光调制器对第二光波进行强度调制，输出强度调制后的第二光波投射在显示平面上快速显示目标图像，降低图像显示成本，且可以显示亮度、分辨率、位深俱佳的高质量图像。
图5示出了本申请另一个实施例提供的图像显示方法的流程示意图，在使用已训练的透镜模型之前，需要对透镜模型进行训练，请参阅图5，所述图像显示方法具体可以包括如下步骤：
步骤S210、获取目标相位转化公式，其中，所述目标相位转换公式由初始相位转换公式进行线性化处理获得。
自由曲面的相位起伏的梯度决定了光束的偏转角度，而相位起伏的二阶导 数则确定了自由曲面的曲率，决定了相位平面对光束的汇聚和发散的调控程度。可以通过相位分布函数来模拟自由曲面的曲率，因此，可以通过相位分布函数构建目标相位转化公式，具体如下：
首先，构建初始相位转换公式。结合图3，自由曲面A所在的坐标为
Figure PCTCN2022111972-appb-000003
光场平面B的坐标为
Figure PCTCN2022111972-appb-000004
自由曲面A和光场平面B平行设置，并且自由曲面A和光场平面B之间间隔焦距f，自由曲面A所在的坐标
Figure PCTCN2022111972-appb-000005
和光场平面B的坐标
Figure PCTCN2022111972-appb-000006
之间的对应关系为：
Figure PCTCN2022111972-appb-000007
其中，
Figure PCTCN2022111972-appb-000008
为初始相位分布函数。
每个坐标的坐标点与光强存在对应关系，
Figure PCTCN2022111972-appb-000009
对应光强
Figure PCTCN2022111972-appb-000010
对应的光强
Figure PCTCN2022111972-appb-000011
将光强代入公式(1)中，获得公式：
Figure PCTCN2022111972-appb-000012
其中，
Figure PCTCN2022111972-appb-000013
为初始相位转换公式的输出光强。
然后，对所述初始相位转换公式进行线性化处理，获得所述目标相位转换公式，具体包括：
对所述初始相位转换公式进行倒数处理，获得求倒数后的转换公式；
对所述求倒数后的转换公式进行开根号处理，获得所述目标相位转换公式。作为一种方式，当对初始相位转化公式进行倒数处理和开根号处理后获得的公式能够准确根据输入光波获得合理的输出光波时，以该公式作为目标转换公式。
作为另一种方式，对初始相位转化公式进行倒数处理和开根号处理后，获得初始目标转换公式(3)，如下所示：
Figure PCTCN2022111972-appb-000014
步骤S220、通过输入光波和目标图像训练目标相位转化公式，获得已训练的透镜模型。
输入光波输入目标相位转化公式，目标相位转化公式输出光波，输出光波最理想的情况是投射出得到目标图像，用输入光波和目标图像训练目标相位转化公式，以建立输入光波和目标图像之间的对应关系。
在一种实施方式中，步骤S220包括如下子步骤：
子步骤S221、将所述输入光波输入所述目标相位转化公式，获得所述目标相位转化公式输出的预测图像。
将输入光波输入目标相位转化公式中，初始相位分布函数
Figure PCTCN2022111972-appb-000015
使得输入光波的相位变形，从而导致曲线形的相位分布函数对输入光线进行局部偏转，目标相位转化公式输出经过相位偏转后的输出光波，输出光波投射出预测图像。
当目标相位转化公式输出的预测图像的相位和目标图像的相位一致时，则目标相位转化公式已经建立了输入光波和目标图像之间的对应关系，目标相位转化公式训练完成，获得已训练的透镜模型。
当目标相位转化公式输出的预测图像的相位和目标图像的相位不一致时，则执行下述子步骤S222。
子步骤S222、根据所述预测图像和所述目标图像更新所述目标相位转化公式，获得所述已训练的透镜模型。
在一种实施方式中，计算预设图像的强度和目标图像的强度之间的强度差，根据强度差对目标相位转化公式进行迭代训练，直至相位差满足迭代训练 条件，获得已训练的透镜模型，其中，迭代训练条件可以是强度差小于预设强度差。由于目标相位转化公式经过线性化处理，因此，目标相位转化公式可以快速输出，加快迭代速度，使得模型训练速度加快。
在另一种实施方式中，计算预测图像和目标图像两者之间的相位差，根据相位差对目标相位转化公式进行迭代训练，直至相位差满足迭代训练条件，获得已训练的透镜模型，其中，迭代训练条件可以是相位差小于预设相位差。由于目标相位转化公式经过线性化处理，因此，目标相位转化公式可以快速输出，加快迭代速度，使得模型训练速度加快。
作为一种方式，获得所述预测图像和所述目标图像之间的相位差；根据所述相位差更新所述目标相位转化公式，获得所述已训练的透镜模型，具体地，根据相位差将初始相位分布函数
Figure PCTCN2022111972-appb-000016
更新至相位分布函数p(x)，从而更新目标相位转换公式，对该公式进行迭代训练，直至所述相位差满足训练条件时，结束训练，获得的所述目标相位转换公式为：
Figure PCTCN2022111972-appb-000017
其中，i 0为所述第一光波，i 1为所述第二光波，f焦距，p(x)为相位分布函数。
其中，相位分布函数p(x)可以反映自由曲面对应的透镜焦距，决定了该处对光束的汇聚和发散的程度，决定了经过调控后的光亮度分布。
需要说明的是，迭代训练是对目标相位转化公式进行迭代训练，实际上是对相位分布函数p(x)进行迭代训练，训练出使得模型输出的光波的强度与目标强度之间误差最小的相位分布函数。
还需要说明的是，针对不同显示需求，对输入已训练的透镜模型的光束 进行不同程度的汇聚和发散，对应的不同的已训练的透镜模型，即对于不同汇聚和发散要求，不同的模型具有不同的相位分布函数。
还需要说明的是，可以采用多组输入光波和输入光波对应的目标图像对上述模型进行训练，当使用的组数越多时，训练得到的透镜模型精确度越高。
可选地，为了提高已训练的透镜模型的性能，可以对已训练的透镜模型进行如下处理：获取卷积算子；通过所述卷积算子和快速傅立叶变换对所述已训练的透镜模型进行处理。此外，还可以通过近端算子实现快速、高效的计算，还可以通过阻尼因子和阻尼项来增强已训练的透镜模型计算的稳定性，阻尼因子和阻尼项可以使模型的计算结果快速收敛，且使得模型调制后输出的亮度分布与目标图像的目标亮度分布较为一致，存在较小的失真，具体的，获取近端算子和阻尼因子；通过所述近端算子和所述阻尼因子对所述已训练的透镜模型进行处理。其中，通过增加近端算子，提高已训练的透镜模型的计算速度。通过增加阻尼项和阻尼因子以提升已训练的透镜模型的稳定性，使得已训练的透镜模型可以适用于各类图像。可选地，图像可以包括Jpg、png、tif等类型。
需要说明的时，对于不同类型的图片，选取的卷积算子、近端算子和阻尼因子可以不同，以使已训练的透镜模型输出的亮度分布与目标图像的目标亮度分布接近，以提升目标图像最终的显示效果。
可选地，已训练的透镜模型可以存储于电子设备本地，或者存储于与电子设备通信连接的服务器中，在使用已训练的透镜模型时，可以直接从存储位置调用。
在一种实施方式中，已训练的透镜模型可以预先训练完成后存储于电子设备本地。在使用已训练的透镜模型时，可以快速调用该已训练的透镜模型，输入第一光波时，已训练的透镜模型可以快速输出第二光波，并且，由于该模型 存储在电子设备本地，也可以有效避免由于网络因素的影响降低了已训练的透镜模型输出第二光波的速度，响应速度快，进一步提升用户体验。
在另一种实施方式中，已训练的透镜模型可以预先训练完成后存储在与电子设备通信连接的服务器。在使用已训练的透镜模型时，电子设备将第一光波的信息(包括光强和相位等)发送至服务器的已训练的透镜模型，以指示该已训练的透镜模型输出第二光波的信息，并获取服务器发送的第二光波的信息，根据第二光波信息生成第二光波，从而通过将已训练的透镜模型存储在服务器的方式，减少对电子设备的存储空间的占用，降低对电子设备正常运行的影响。
步骤S230、获取已训练的透镜模型，并将目标图像的相位加载到所述已训练的透镜模型上，其中，所述已训练的透镜模型为线性化模型。
步骤S240、当第一光波输入所述已训练的透镜模型时，获得所述已训练的透镜模型进行相位调制后输出的第二光波，其中，所述第二光波携带所述目标图像的相位信息。
步骤S250、将所述第二光波输入强度空间光调制器，获得所述强度空间光调制器强度调制后输出的第二光波，以使强度调制后的第二光波携带有所述目标图像的相位信息和强度信息并在显示平面显示目标图像。
其中，步骤S230-步骤S250的具体描述请参阅步骤S110-步骤S130，在此不再赘述。
综上所述，本申请提供一种图像显示方法、装置、电子设备及存储介质，通过对透镜模型的迭代训练获得已训练的透镜模型，已训练的透镜模型为线性模型可以稳定、快速地输出光波，提高最终成像的速度和质量。
为实现上述方法类实施例，本实施例提供一种图像显示装置，图7示出了本申请一实施例提供的图像显示装置的框图，请参阅图7，图像显示装置100 包括：获取模块110、输出模块120和显示模块130。
获取模块110，用于获取已训练的透镜模型，并将目标图像的相位加载到所述已训练的透镜模型上，其中，所述已训练的透镜模型为线性化模型；
输出模块120，用于当第一光波输入所述已训练的透镜模型时，获得所述已训练的透镜模型进行相位调制后输出的第二光波，其中，所述第二光波携带所述目标图像的相位信息；
显示模块130，用于将所述第二光波输入强度空间光调制器，获得所述强度空间光调制器强度调制后输出的第二光波，以使强度调制后的第二光波携带有所述目标图像的相位信息和强度信息并在显示平面显示目标图像。
可选地，图像显示装置100还包括：构建模块和线性化模块。
构建模块，用于构建初始相位转换公式；
线性化模块，用于对所述初始相位转换公式进行线性化处理，获得所述目标相位转换公式。
可选地，线性化模块包括：求导处理子模块和开根号处理子模块。
求导处理子模块，用于对所述初始相位转换公式进行倒数处理，获得求倒数后的转换公式；
开根号处理子模块，用于对所述求倒数后的转换公式进行开根号处理，获得所述目标相位转换公式。
可选地，所述目标相位转换公式为：
Figure PCTCN2022111972-appb-000018
其中，i 0为所述第一光波，i 1为所述第二光波，f为所述已训练的透镜模型和所述显示平面之间的距离，p(x)为相位分布函数。
可选地，图像显示装置100还包括：目标相位转化公式模块和训练模块。
目标相位转化公式模块，用于获取目标相位转化公式，其中，所述目标相位转换公式由初始相位转换公式进行线性化处理获得；
训练模块，用于通过输入光波和目标图像训练目标相位转化公式，获得已训练的透镜模型。
可选地，训练模块包括：预测模块和更新模块。
预测模块，用于将所述输入光波输入所述目标相位转化公式，获得所述目标相位转化公式输出的预测图像；
更新模块，用于根据所述预测图像和所述目标图像更新所述目标相位转化公式，获得所述已训练的透镜模型。
可选地，更新模块包括：相位差获取模块和公式更新模块。
相位差获取模块，用于获得所述预测图像和所述目标图像之间的相位差；
公式更新模块，用于根据所述相位差更新所述目标相位转化公式，获得所述已训练的透镜模型。
可选地，图像显示装置100还包括：卷积算子获取模块以及第一模型处理模块。
卷积算子获取模块，用于获取卷积算子；
第一模型处理模块，用于通过所述卷积算子和快速傅立叶变换对所述已训练的透镜模型进行处理。
可选地，图像显示装置100还包括：近端算子和阻尼因子获取子模块、以及第二模型处理模块。
近端算子和阻尼因子获取子模块，用于获取近端算子和阻尼因子；
第二模型处理模块，用于通过所述近端算子和所述阻尼因子对所述已训练 的透镜模型进行处理。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述装置和模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，模块相互之间的耦合可以是电性，机械或其它形式的耦合。
另外，在本申请各个实施例中的各功能模块可以集成在一个处理模块中，也可以是各个模块单独物理存在，也可以两个或两个以上模块集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。
图8是本申请实施例的用于执行根据本申请实施例的图像显示方法的电子设备的框图，请参阅图8，其示出了本申请实施例提供的一种电子设备200的结构框图。该电子设备200可以是智能手机、平板电脑、电子书等能够运行应用程序的电子设备。本申请中的电子设备200可以包括一个或多个如下部件：处理器210、存储器220以及一个或多个应用程序，其中一个或多个应用程序可以被存储在存储器220中并被配置为由一个或多个处理器210执行，一个或多个程序配置用于执行如前述方法实施例所描述的方法。
其中，处理器210可以包括一个或者多个处理核。处理器210利用各种接口和线路连接整个电子设备200内的各个部分，通过运行或执行存储在存储器220内的指令、程序、代码集或指令集，以及调用存储在存储器220内的数据，执行电子设备200的各种功能和处理数据。可选地，处理器210可以采用数字信号处理(Digital Signal Processing，DSP)、现场可编程门阵列(Field－Programmable Gate Array，FPGA)、可编程逻辑阵列(Programmable Logic Array，PLA)中 的至少一种硬件形式来实现。处理器210可集成中央处理器(Central Processing Unit，CPU)、图形处理器(Graphics Processing Unit，GPU)和调制解调器等中的一种或几种的组合。其中，CPU主要处理操作系统、用户界面和应用程序等；GPU用于负责待显示组件的渲染和绘制；调制解调器用于处理无线通信。可以理解的是，上述调制解调器也可以不集成到处理器210中，单独通过一块通信芯片进行实现。
存储器220可以包括随机存储器(Random Access Memory，RAM)，也可以包括只读存储器(Read-Only Memory)。存储器220可用于存储指令、程序、代码、代码集或指令集。存储器220可包括存储程序区和存储数据区，其中，存储程序区可存储用于实现操作系统的指令、用于实现至少一个功能的指令(比如触控功能、声音播放功能、图像播放功能等)、用于实现下述各个方法实施例的指令等。存储数据区还可以存储电子设备200在使用中所创建的数据(比如历史配置文件)等。
图9示出了本申请实施例的用于保存或者携带实现根据本申请实施例的图像显示方法的程序代码的存储单元，请参阅10，其示出了本申请实施例提供的一种计算机可读存储介质的结构框图。该计算机可读介质300中存储有程序代码，所述程序代码可被处理器调用执行上述方法实施例中所描述的方法。
计算机可读存储介质300可以是诸如闪存、EEPROM(电可擦除可编程只读存储器)、EPROM、硬盘或者ROM之类的电子存储器。可选地，计算机可读存储介质300包括非易失性计算机可读介质(non-transitory computer-readable storage medium)。计算机可读存储介质300具有执行上述方法中的任何方法步骤的程序代码310的存储空间。这些程序代码可以从一个或者多个计算机程序产品中读出或者写入到这一个或者多个计算机程序产品中。程序代码310可以 例如以适当形式进行压缩。
综上所述，本申请提供一种图像显示方法、装置、电子设备及存储介质，获取已训练的透镜模型，并将目标图像的相位加载到已训练的透镜模型上，其中，已训练的透镜模型为线性化模型，当第一光波输入已训练的透镜模型时，获得已训练的透镜模型进行相位调制输出的第二光波，其中，第二光波携带目标图像的相位信息，将第二光波输入强度空间光调制器，获得强度空间光调制器强度调制后输出的第二光波，使得强度调制后的第二光波携带有目标图像的相位信息和强度信息，并且强度调制后的第二光波在显示平面上显示目标图像，由于已训练的透镜模型为线性化模型，因此，当第一光波输入已训练的透镜模型时，已训练的透镜模型可以稳定、快速地输出第二光波，再通过强度空间光调制器对第二光波进行强度调制，输出强度调制后的第二光波投射在显示平面上快速显示目标图像，降低图像显示成本，且可以显示亮度、分辨率、位深俱佳的高质量图像。
最后应说明的是：以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；以上所述，仅为本申请的实施例而已，并非用于限定本申请的保护范围。凡在本申请的精神和范围之内所作的任何修改、等同替换和改进等，均包含在本申请的保护范围之内。

Claims (12)

  1. 一种图像显示方法，其特征在于，所述方法包括：
    获取已训练的透镜模型，并将目标图像的相位加载到所述已训练的透镜模型上，其中，所述已训练的透镜模型为线性化模型；
    当第一光波输入所述已训练的透镜模型时，获得所述已训练的透镜模型进行相位调制后输出的第二光波，其中，所述第二光波携带所述目标图像的相位信息；
    将所述第二光波输入强度空间光调制器，获得所述强度空间光调制器强度调制后输出的第二光波，以使强度调制后的第二光波携带有所述目标图像的相位信息和强度信息并在显示平面显示目标图像。
  2. 根据权利要求1所述的方法，其特征在于，所述已训练的透镜模型由目标相位转换公式训练获得，所述获取已训练的透镜模型之前，所述方法还包括：
    构建初始相位转换公式；
    对所述初始相位转换公式进行线性化处理，获得所述目标相位转换公式。
  3. 根据权利要求2所述的方法，其特征在于，所述对所述初始相位转换公式进行线性化处理，获得所述目标相位转换公式，包括：
    对所述初始相位转换公式进行倒数处理，获得求倒数后的转换公式；
    对所述求倒数后的转换公式进行开根号处理，获得所述目标相位转换公式。
  4. 根据权利要求2所述的方法，其特征在于，所述目标相位转换公式为：
    Figure PCTCN2022111972-appb-100001
    其中，i 0为所述第一光波，i 1为所述第二光波，f为所述已训练的透镜模型和所述显示平面之间的距离，p(x)为相位分布函数。
  5. 根据权利要求1所述的方法，其特征在于，所述获取已训练的透镜模型，并将目标图像的相位加载到所述已训练的透镜模型上之前，所述方 法还包括：
    获取目标相位转化公式，其中，所述目标相位转换公式由初始相位转换公式进行线性化处理获得；
    通过输入光波和目标图像训练目标相位转化公式，获得已训练的透镜模型。
  6. 根据权利要求5所述的方法，其特征在于，所述通过输入光波和目标图像训练目标相位转化公式，获得已训练的透镜模型，包括：
    将所述输入光波输入所述目标相位转化公式，获得所述目标相位转化公式输出的预测图像；
    根据所述预测图像和所述目标图像更新所述目标相位转化公式，获得所述已训练的透镜模型。
  7. 根据权利要求6所述的方法，其特征在于，所述根据所述预测图像和所述目标图像更新所述目标相位转化公式，获得所述已训练的透镜模型，包括：
    获得所述预测图像和所述目标图像之间的相位差；
    根据所述相位差更新所述目标相位转化公式，获得所述已训练的透镜模型。
  8. 根据权利要求7所述的方法，其特征在于，所述根据所述相位差更新所述目标相位转化公式，获得所述已训练的透镜模型之后，所述方法还包括：
    获取卷积算子；
    通过所述卷积算子和快速傅立叶变换对所述已训练的透镜模型进行处理。
  9. 根据权利要求7所述的方法，其特征在于，所述根据所述相位差更新所述目标相位转化公式，获得所述已训练的透镜模型之后，所述方法还包括：
    获取近端算子和阻尼因子；
    通过所述近端算子和所述阻尼因子对所述已训练的透镜模型进行处理。
  10. 一种图像显示装置，其特征在于，所述装置包括：
    获取模块，用于获取已训练的透镜模型，并将目标图像的相位加载到所述已训练的透镜模型上，其中，所述已训练的透镜模型为线性化模型；
    输出模块，用于当第一光波输入所述已训练的透镜模型时，获得所述已训练的透镜模型进行相位调制后输出的第二光波，其中，所述第二光波携带所述目标图像的相位信息；
    显示模块，用于将所述第二光波输入强度空间光调制器，获得所述强度空间光调制器强度调制后输出的第二光波，以使强度调制后的第二光波携带有所述目标图像的相位信息和强度信息并在显示平面显示目标图像。
  11. 一种电子设备，其特征在于，包括：
    一个或多个处理器；
    存储器；
    一个或多个应用程序，其中所述一个或多个应用程序被存储在所述存储器中并被配置为由所述一个或多个处理器执行，所述一个或多个应用程序配置用于执行如权利要求1-9任一项所述的方法。
  12. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质中存储有程序代码，所述程序代码可被处理器调用执行如权利要求1-9任一项所述的方法。
PCT/CN2022/111972 2021-08-18 2022-08-12 图像显示方法、装置、电子设备及存储介质 WO2023020371A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110950880.9A CN115708123A (zh) 2021-08-18 2021-08-18 图像显示方法、装置、电子设备及存储介质
CN202110950880.9 2021-08-18

Publications (1)

Publication Number Publication Date
WO2023020371A1 true WO2023020371A1 (zh) 2023-02-23

Family

ID=85212506

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/111972 WO2023020371A1 (zh) 2021-08-18 2022-08-12 图像显示方法、装置、电子设备及存储介质

Country Status (2)

Country Link
CN (1) CN115708123A (zh)
WO (1) WO2023020371A1 (zh)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080252977A1 (en) * 2007-03-02 2008-10-16 Sony Corporation Image display method
CN101981490A (zh) * 2007-06-15 2011-02-23 独立行政法人情报通信研究机构 光波形整形装置
US20160295178A1 (en) * 2013-10-20 2016-10-06 Mtt Innovation Incorporated Light field projectors and methods
CN111579097A (zh) * 2020-05-21 2020-08-25 浙江大学 基于神经网络的高精度光学散射补偿方法
CN112116616A (zh) * 2020-08-05 2020-12-22 西安交通大学 基于卷积神经网络的相位信息提取方法、存储介质及设备
US20210133938A1 (en) * 2019-11-04 2021-05-06 Lg Electronics Inc. Method and apparatus for enhancing illumination intensity of image
CN113039484A (zh) * 2018-11-13 2021-06-25 索尼集团公司 图像显示装置

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080252977A1 (en) * 2007-03-02 2008-10-16 Sony Corporation Image display method
CN101981490A (zh) * 2007-06-15 2011-02-23 独立行政法人情报通信研究机构 光波形整形装置
US20160295178A1 (en) * 2013-10-20 2016-10-06 Mtt Innovation Incorporated Light field projectors and methods
CN113039484A (zh) * 2018-11-13 2021-06-25 索尼集团公司 图像显示装置
US20210133938A1 (en) * 2019-11-04 2021-05-06 Lg Electronics Inc. Method and apparatus for enhancing illumination intensity of image
CN111579097A (zh) * 2020-05-21 2020-08-25 浙江大学 基于神经网络的高精度光学散射补偿方法
CN112116616A (zh) * 2020-08-05 2020-12-22 西安交通大学 基于卷积神经网络的相位信息提取方法、存储介质及设备

Also Published As

Publication number Publication date
CN115708123A (zh) 2023-02-21

Similar Documents

Publication Publication Date Title
JP7104098B2 (ja) 結像、照明、および投影を用途とする効率的、動的、高コントラストなレンジング方法及び装置
US11480921B2 (en) Method for generating a holographic image, processor, holographic image display device, and apparatus
US9936175B2 (en) Numerical approaches for free-form lensing: area parameterization free-form lensing
JP2014178393A (ja) 画像投影システム及び画像投影方法
US11694302B2 (en) Dynamic uniformity correction
KR102250189B1 (ko) 홀로그래피 재생 장치 및 재생 방법
JP2015097350A (ja) 画像処理装置およびマルチプロジェクションシステム
US10495955B2 (en) Method and apparatus for controlling projector
US9992464B1 (en) Method and system for screen correction
WO2023020371A1 (zh) 图像显示方法、装置、电子设备及存储介质
JP7196838B2 (ja) 位相変調データ生成装置、照明装置、およびプロジェクタ
US11681363B2 (en) Waveguide correction map compression
CN107796594A (zh) 一种基于空间光调制器的大气湍流模拟器
US11994689B2 (en) Diffractive optical elements for large-field image display
US20220244527A1 (en) Micro-electro-mechanical scanning mirror control method, control apparatus, and system
US20140375692A1 (en) Image processing device and image processing method
CN116258803A (zh) 体积云生成方法、装置、计算机可读存储介质及电子设备
TW202213970A (zh) 信號處理裝置、信號處理方法、程式及照明裝置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22857683

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE