WO2020192503A1 - 一种物体深度信息的确定方法、电子设备和电路系统 - Google Patents

一种物体深度信息的确定方法、电子设备和电路系统 Download PDF

Info

Publication number
WO2020192503A1
WO2020192503A1 PCT/CN2020/079806 CN2020079806W WO2020192503A1 WO 2020192503 A1 WO2020192503 A1 WO 2020192503A1 CN 2020079806 W CN2020079806 W CN 2020079806W WO 2020192503 A1 WO2020192503 A1 WO 2020192503A1
Authority
WO
WIPO (PCT)
Prior art keywords
grating
voltage
light beam
camera
period
Prior art date
Application number
PCT/CN2020/079806
Other languages
English (en)
French (fr)
Inventor
王恒
叶海水
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2020192503A1 publication Critical patent/WO2020192503A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/207—Image signal generators using stereoscopic image cameras using a single 2D image sensor
    • H04N13/218—Image signal generators using stereoscopic image cameras using a single 2D image sensor using spatial multiplexing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/271—Image signal generators wherein the generated image signals comprise depth maps or disparity maps
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/275—Image signal generators from 3D object models, e.g. computer-generated stereoscopic image signals

Definitions

  • This application relates to the field of terminal technology, and in particular to a method for determining object depth information, electronic equipment and circuit systems.
  • the captured images can be three-dimensional images.
  • three-dimensional images can reflect more information about the shooting object. It is also more in line with people's perception of the real world.
  • Three-dimensional images are usually obtained by constructing three-dimensional models.
  • commonly used three-dimensional model construction methods include: stereo vision method, structured light method, and time of flight (ToF) method.
  • the electronic device is a camera
  • the light source set on the camera emits a light beam to the object to be photographed
  • the light beam illuminates the object to be photographed
  • the reflected light is captured by the camera
  • the light beam is emitted by the camera
  • the time difference between and the received reflected light beam is used to calculate the distance between each area of the surface of the object and the camera to generate the depth information of the object, and then construct a three-dimensional model of the object according to the depth information.
  • a current solution is that an electronic device such as a camera can scan the area of the photographed object through a beam of light to obtain distance information between different areas on the surface of the photographed object and the camera.
  • the light source on the electronic device needs to move, so use
  • the camera of this scheme is equipped with a scanning device.
  • the position of the scanning device moves to drive the position of the light source to move.
  • the light source can irradiate the light beam on different areas of the photographed object during the movement to obtain the photograph The distance information of the different areas of the object and the camera.
  • the camera has problems with poor stability and reliability.
  • the camera needs to be a scanning device. Provides a large enough space for movement, so the camera still has the problem of large volume and low internal space utilization.
  • the embodiments of the present application provide a method, electronic equipment and circuit system for determining object depth information, which are used to solve the problems of poor camera stability and reliability and low space utilization caused by the movement of the light source driven by the scanning device in the prior art. problem.
  • an embodiment of the present application provides a method for determining object depth information, which is applied to an electronic device.
  • the electronic device may be an electronic device such as a mobile phone or a smart camera.
  • the electronic device may include a light source, a grating device, and at least one camera.
  • the method includes: controlling the grating structure of the grating device to a first structure, the light beam generated by the light source generates a first diffracted beam on the grating device, the diffraction angle of the first diffracted beam is a first angle, and The first diffracted light beam illuminates the first area on the object to be photographed; wherein the first reflected light beam reflected by the first area is captured by the camera; the grating structure of the grating device is adjusted to change to the second structure, so that The light beam generates a second diffracted light beam on the grating device, the diffraction angle of the second diffracted light beam is a second angle, and the second diffracted light beam irradiates a second area on the object to be photographed; wherein, The second reflected light beam reflected by the second area is captured by the camera; and the depth information of the object to be photographed is determined according to the first reflected light beam and the second reflected light beam.
  • the electronic device controls the structure of the grating device to change during the process of collecting the depth information of the photographed object, so that the light beam can be emitted from different diffraction angles to scan the photographed object, but the position of the light source is different. Need to move, so there is no vibration, which can improve the reliability of electronic equipment and ensure the imaging quality. At the same time, there is no need to reserve moving space. Therefore, the volume of the depth camera can be reduced, which is more conducive to system integration, and overcomes the existing technology. The inherent defects of mechanical scanning devices.
  • the grating device may be a liquid crystal on silicon (liquid crystal on silicon, LCoS) spatial light modulator;
  • the LCoS spatial light modulator includes a liquid crystal layer, a first electrode layer, and a second electrode layer , The liquid crystal layer is located between the first electrode layer and the second electrode layer.
  • controlling the grating structure of the grating device to be the first structure may include: applying a first voltage between the first electrode layer and the second electrode layer, so that the liquid crystal layer has the first structure;
  • the first structure includes: the refractive index of the liquid crystal layer changes periodically in a first direction with a first period as a period; wherein the refractive index in each period presents an N-level step in the first direction
  • the formula increases or decreases, the first direction is parallel to the plane where the liquid crystal layer is, and N is an integer greater than or equal to 2
  • adjusting the grating structure of the grating device to the second structure may include: A second voltage is applied between the layer and the second electrode layer, so that the liquid crystal layer has a second structure; the second structure includes: the refractive index of the liquid crystal layer is in a second period in the first direction The period changes periodically; wherein, the refractive index in each period increases or decreases in Q steps in the first direction, Q is an integer greater than or equal to 2, and Q ⁇ N.
  • the grating device can be realized based on the LCoS spatial light modulator.
  • the gap between the first electrode layer and the second electrode layer By controlling the voltage between the first electrode layer and the second electrode layer in the LCoS spatial light modulator, the gap between the first electrode layer and the second electrode layer
  • the structure of the liquid crystal layer is changed, so that the light beam projected on the liquid crystal layer is emitted from different diffraction angles to realize the scanning of the object.
  • the whole process does not need to move the position of the light source, so it can improve the reliability of electronic equipment and ensure imaging Quality, reduce the volume of the depth camera, and improve the signal-to-noise ratio of the reflected light signal.
  • controlling the grating structure of the grating device to be the first structure may include: determining the first corresponding to the first angle according to the first correspondence between the diffraction angle and the phase modulation amount. Phase modulation amount; wherein the first phase adjustment amount changes periodically in the first direction with the first period as a period, and the phase modulation amount in each period is in the first direction N-level stepwise increase or decrease; according to the second correspondence between the phase modulation amount and the voltage, determine the first voltage corresponding to the first phase modulation amount; wherein, the first voltage is in the first side In the upward direction, the first period is used as the period to change periodically, and the voltage in each period increases or decreases in N steps in the first direction; in the first electrode layer and the second electrode The first voltage is applied between the layers, so that the refractive index of the liquid crystal layer periodically changes in the first direction with the first period as a period.
  • the first phase modulation amount corresponding to the first angle may be determined based on the first correspondence between the diffraction angle and the phase modulation amount, and then the first phase modulation amount corresponding to the first angle may be determined according to the second correspondence relationship between the phase modulation amount and the voltage.
  • the first voltage corresponding to the phase modulation amount is applied between the first electrode layer and the second electrode layer in the LCoS spatial light modulator, so that the diffraction angle of the beam is the first angle, and irradiates the object to be photographed The first area. It realizes the precise control of the diffraction angle of the beam according to the demand, and improves the accuracy and reliability of the object depth information collection.
  • adjusting the grating structure of the grating device to the second structure may include: determining the second angle corresponding to the second angle according to the first correspondence between the diffraction angle and the phase modulation amount.
  • Two-phase modulation amount; wherein the second phase modulation amount periodically changes in the first direction with the second period as a period, and the phase modulation amount in each period is in the first direction In a Q-level stepwise increase or decrease; a second voltage corresponding to the second phase modulation amount is determined according to the second corresponding relationship between the phase modulation amount and the voltage; wherein, the second voltage is in the first
  • the direction changes periodically with the second period as the period, and the voltage in each period increases or decreases in Q steps in the first direction; in the first electrode layer and the second electrode layer
  • the second voltage is applied between the electrode layers, so that the refractive index of the liquid crystal layer periodically changes in the first direction with the second period as a period.
  • the second phase modulation amount corresponding to the second angle may be determined based on the first correspondence between the diffraction angle and the phase modulation amount, and then the second phase modulation amount corresponding to the second angle may be determined according to the second correspondence relationship between the phase modulation amount and the voltage.
  • the second voltage corresponding to the phase modulation amount is applied between the first electrode layer and the second electrode layer in the LCoS spatial light modulator, so that the diffraction angle of the beam is the second angle, and it is irradiated on the object to be photographed The second area.
  • the diffraction angle of the beam can be accurately controlled according to the requirements, and the accuracy and reliability of the object depth information collection is improved.
  • the grating device may also be an acousto-optic deflector; the acousto-optic deflector includes a driving power supply, an acousto-optic medium, and a piezoelectric transducer, and correspondingly, the grating of the grating device is controlled.
  • the structure is the first structure, and may include: controlling the driving power supply to input a third voltage to the piezoelectric transducer, so that the piezoelectric transducer generates ultrasonic waves of the first frequency, and the ultrasonic waves of the first frequency are transmitted After entering the acousto-optic medium, the acousto-optic medium forms a first structure; adjusting the grating structure of the grating device to change to the second structure may include: controlling the driving voltage to input the second structure to the piezoelectric transducer The four voltages cause the piezoelectric transducer to generate ultrasonic waves of the second frequency. After the ultrasonic waves of the second frequency are transmitted into the acousto-optic medium, the acousto-optic medium forms a second structure.
  • This embodiment can also implement a grating device based on the acousto-optic deflector.
  • a grating device By controlling the voltage input from the driving power supply in the acousto-optic deflector to the piezoelectric transducer, the structure of the acousto-optic medium can be changed, and the projected acoustic The light beam of the optical medium is emitted from different diffraction angles to realize the scanning of the photographed object.
  • the whole process does not need to move the position of the light source, so it can improve the reliability of electronic equipment, ensure the imaging quality, and reduce the volume of the depth camera. Improve the signal-to-noise ratio of the reflected light signal.
  • the third corresponding relationship between the diffraction angle and the frequency of the ultrasonic wave may be used to determine the The first frequency of the ultrasonic wave corresponding to an angle; and the third voltage corresponding to the first frequency is determined according to the fourth correspondence between the frequency of the ultrasonic wave and the voltage of the driving power source.
  • the first frequency of the ultrasonic wave corresponding to the first angle may be determined based on the third correspondence between the diffraction angle and the frequency of the ultrasonic wave, and then the fourth correspondence between the frequency of the ultrasonic wave and the voltage of the driving power source may be determined.
  • the third voltage corresponding to the first frequency is input to the piezoelectric transducer through the driving power supply, so that the diffraction angle of the light beam is the first angle and irradiates the first area on the object to be photographed. It realizes the precise control of the diffraction angle of the beam according to the demand, and improves the accuracy and reliability of the object depth information collection.
  • the third corresponding relationship between the diffraction angle and the frequency of the ultrasonic wave may be used to determine the The second frequency of the ultrasonic wave corresponding to the two angles; and the fourth voltage corresponding to the second frequency is determined according to the fourth corresponding relationship between the frequency of the ultrasonic wave and the voltage of the driving power source.
  • the second frequency of the ultrasonic wave corresponding to the second angle may be determined based on the third correspondence between the diffraction angle and the frequency of the ultrasonic wave, and then the fourth correspondence between the frequency of the ultrasonic wave and the voltage of the driving power source may be determined.
  • the fourth voltage corresponding to the second frequency is input to the piezoelectric transducer through the driving power supply, so that the diffraction angle of the light beam is the second angle, and it irradiates the second area on the object to be photographed.
  • the diffraction angle of the beam can be accurately controlled according to the requirements, and the accuracy and reliability of the object depth information collection is improved.
  • an embodiment of the present application further provides an electronic device, the electronic device includes: at least one processor, a light source, a grating device, and at least one camera; the light source is used to generate a light beam and project the light beam To the grating device; the at least one processor for controlling the grating structure of the grating device to be the first structure; wherein, when the grating device is the first structure, the light beam is in the grating device The first diffracted beam is generated on the upper surface, the diffraction angle of the first diffracted beam is the first angle, and the first diffracted beam irradiates the first area on the object to be photographed; the at least one camera is used to capture the first The first reflected light beam reflected by a region; the at least one processor is also used to adjust the grating structure of the grating device to change to a second structure; wherein, when the grating device is in the second structure, the light beam is A second diffracted light beam is generated on the grating
  • the grating device is an LCoS spatial light modulator; the LCoS spatial light modulator includes a liquid crystal layer; the grating structure of the grating device is a first structure, including: the refraction of the liquid crystal layer The refractive index changes periodically in the first direction with the first period as the period; wherein, the refractive index in each period increases or decreases in N steps in the first direction, and the first direction is parallel to all The plane where the liquid crystal layer is located, N is an integer greater than or equal to 2; the grating structure of the grating device is a second structure, including: the refractive index of the liquid crystal layer is periodic in the first direction with a second period as a period sexual change; wherein the refractive index in each period increases or decreases in Q steps in the first direction, where Q is an integer greater than or equal to 2, and Q ⁇ N.
  • the grating device is an acousto-optic deflector;
  • the acousto-optic deflector includes a driving power supply, an acousto-optic medium, and a piezoelectric transducer;
  • the driving power supply is used to: The transducer inputs a third voltage;
  • the piezoelectric transducer is used to: generate ultrasonic waves of the first frequency under the driving of the third voltage; after the ultrasonic waves of the first frequency are transmitted into the acousto-optic medium,
  • the acousto-optic medium forms a first structure;
  • the driving power supply is also used to control the driving power supply to input a fourth voltage to the piezoelectric transducer;
  • the piezoelectric transducer is also used to: Under the driving of four voltages, ultrasonic waves of the second frequency are generated; after the ultrasonic waves of the second frequency are transmitted into the acousto-optic medium, the acousto-optic
  • the implementation of this application provides a circuit system, which may be one or more chips, such as a system-on-a-chip (SoC).
  • SoC system-on-a-chip
  • the circuit system is used to generate a first control signal, the first control signal is used to control the grating structure of the grating device to the first structure; the circuit system is also used to generate a second control signal, the second control signal is used To control the grating structure of the grating device to change to the second structure.
  • SoC system-on-a-chip
  • an embodiment of the present application also provides an electronic device.
  • the electronic device includes a light source, a grating device, at least one camera, at least one processor, and a memory; the memory is used to store one or more computer programs; when the one or more computer programs stored in the memory are used by the at least one When the processor is executed, the electronic device can implement the foregoing first aspect and any possible design technical solution of the first aspect.
  • an embodiment of the present application also provides an electronic device.
  • the electronic device includes modules/units that execute the foregoing first aspect or any one of the possible design methods of the first aspect; these modules/units can be implemented through hardware Realization can also be realized by hardware executing corresponding software.
  • the embodiments of the present application also provide a computer-readable storage medium.
  • the computer-readable storage medium includes a computer program.
  • the computer program runs on an electronic device, the electronic device executes the first On the one hand and any possible design technical solutions of the first aspect.
  • the embodiments of the present application also provide a program product, including instructions, which when the program product runs on an electronic device, cause the electronic device to execute the first aspect of the embodiments of the present application and any one of the first aspects thereof. Possible technical solutions designed.
  • FIG. 1 is a schematic diagram of an application scenario provided by an embodiment of the application
  • Figure 2 is a schematic diagram of another application scenario provided by an embodiment of the application.
  • Figure 3 is a schematic diagram of another application scenario provided by an embodiment of the application.
  • FIG. 4 is a schematic structural diagram of a mobile phone 100 according to an embodiment of the application.
  • FIG. 5A is a schematic structural diagram of a depth camera 200 provided by an embodiment of this application.
  • 5B is a schematic structural diagram of a depth camera 200 provided by an embodiment of the application.
  • 6A, 6B, 6C, and 6D are schematic diagrams of a liquid crystal panel in an LCoS spatial light modulator provided by an embodiment of the application;
  • 6E is a schematic diagram of a possible correspondence between the voltage applied by the liquid crystal layer and the phase modulation amount of the light by the liquid crystal layer in an embodiment of the application;
  • FIG. 6F provides a phase modulation amount distribution diagram according to an embodiment of this application.
  • FIG. 6G provides another phase modulation amount distribution diagram for an embodiment of this application.
  • FIG. 7 is a schematic diagram of another liquid crystal panel in the LCoS spatial light modulator provided by an embodiment of the application.
  • FIG. 8 is another phase modulation amount distribution diagram provided by an embodiment of this application.
  • FIG. 9 is another phase modulation amount distribution diagram provided by an embodiment of this application.
  • FIG. 10 is a flowchart of a possible method for collecting depth information in an embodiment of this application.
  • FIG. 11 is a schematic diagram of a process of a possible depth camera 200 collecting depth information in an embodiment of this application;
  • Figure 12(a) is a schematic diagram of a linear beam in an embodiment of the application.
  • Figure 12(b) is a schematic diagram of a point beam in an embodiment of the application.
  • FIG. 13 is a schematic diagram of a possible two-dimensional scan in an embodiment of this application.
  • FIG. 14 is a schematic diagram of the structure of an acousto-optic deflector in an embodiment of the application.
  • FIG. 16 is a schematic diagram of another possible depth camera 200 collecting depth information in an embodiment of the application.
  • FIG. 17 is a schematic diagram of another possible two-dimensional scanning in an embodiment of this application.
  • the grating involved in the embodiment of the present application is also called a diffraction grating.
  • the grating can be classified into an amplitude modulation grating and a phase modulation grating according to the modulation effect of the grating on the incident light.
  • the phase modulation grating means that the refractive index of different regions on the grating can be changed, and different regions can respectively modulate the phase of the incident light, resulting in the superimposition of the outgoing light with different phases to produce diffracted beams.
  • phase modulation gratings such as liquid crystal on silicon (LCoS).
  • the acousto-optic effect involved in the embodiments of the present application refers to that when ultrasonic waves pass through the medium, the local compression and elongation of the medium will cause elastic strain.
  • the strain periodically changes with time and space, causing the medium to appear dense and dense, as A grating. Diffraction phenomenon occurs when light passes through a medium disturbed by ultrasonic waves, which makes the propagation direction of light deflect. This phenomenon is called acousto-optic effect.
  • acousto-optical deflectors which can be specifically Raman-Ness acousto-optic deflectors, or Bragg acousto-optic deflectors, etc.
  • AOD acousto-optical deflectors
  • the grating device involved in the embodiment of the present application may be any one of the above-mentioned gratings or a combined grating of multiple gratings, which is not limited in the embodiment of the present application.
  • the grating devices are mainly LCoS and AOD as examples, so the specific structure of LCoS and AOD will be introduced later.
  • the light source on the camera emits light beam 1 to the object to be photographed (the face is taken as an example in Figure 1).
  • light beam 1 is projected on area A of the face, area A reflects light beam 1, and reflected light beam 1 is captured by the camera lens .
  • the camera determines the emission time of the light beam 1 and the time that the reflected light beam 1 is captured by the camera lens, and the distance between the area A on the face and the camera is determined by the time difference between the two.
  • the camera can change the emission direction of the emitted light beam.
  • the light source on the camera emits light beam 2 to area B on the face, area B reflects light beam 2, and reflected light beam 2 is captured by the camera lens.
  • the camera can determine the distance between the area B on the face and the camera.
  • a grating device (not shown in FIG. 1) is provided in the camera, and the light beam emitted by the light source irradiates the face through the grating device. Since the grating structure of the grating device may change, The direction of the beam can be changed to scan multiple times on the face to obtain different areas of the face and the distance information of the camera. Therefore, the position of the light source does not need to be changed, so there is no need to set up a scanning device, which avoids the movement of the scanning device.
  • the camera has problems such as poor stability and reliability and low space utilization.
  • the distances from the different areas of the human face to the camera can be obtained, and then the depth information of the human face can be obtained to construct a 3D image of the human face.
  • FIG. 2 is a schematic diagram of another application scenario provided by an embodiment of this application.
  • the mobile phone is integrated with a light source and a camera, and a grating device (not shown in Figure 2) is also provided inside.
  • the light source on the mobile phone emits beam 1 to the object being photographed (the face is taken as an example in Figure 2). Assuming that the beam 1 is projected on the area A of the face, the area A reflects the beam 1, and the reflected beam 1 is captured by the camera of the mobile phone .
  • the mobile phone determines the emission time of the light beam 1 and the time when the reflected light beam 1 is captured by the camera, and the distance between the face area A and the mobile phone is determined by the time difference between the two.
  • the mobile phone can change the emission direction of the emitted light beam. For example, please continue to see Figure 2.
  • the light source on the mobile phone emits light beam 2 to area B on the face, area B reflects light beam 2, and reflected light beam 2 is captured by the phone camera.
  • the mobile phone can determine the distance between the area B on the face and the camera phone.
  • a grating device (not shown in Figure 2) is provided in the mobile phone, and the light beam emitted by the light source irradiates the face through the grating device.
  • the direction of the beam can be changed to scan the face multiple times to obtain the distance information between different areas of the face and the mobile phone. Therefore, the position of the light source does not need to be changed, so there is no need to set up a scanning device, which avoids the movement of the scanning device.
  • Mobile phone stability and reliability are poor, and space utilization is low.
  • the distance from the different areas of the face to the camera phone can be obtained, and then the depth information of the entire face can be obtained.
  • the three-dimensional modeling of the face can obtain the three-dimensional feature information of the face.
  • the mobile phone can realize the functions of face recognition in scenes such as face punching in, unlocking the mobile phone, unlocking applications, and face payment, or integrating the 3D modeling function of the face in the mobile phone In the mobile phone app, or integrated in WeChat video, facetime, Twitter, Moments and other scenes that need to be shot, the shooting function of 3D images can be realized in these scenes.
  • FIG. 3 is a schematic diagram of another application scenario provided by an embodiment of this application.
  • the car is provided with a light source and a camera, and a grating device (not shown in Figure 3) is also provided inside.
  • the light source and the camera can be installed at the rear of the car as shown in FIG. 3, but also at the head, body, etc., which is not specifically limited in the embodiment of the present application.
  • the light source on the car emits light beam 1 to the obstacle. Assuming that light beam 1 is projected on area A on the obstacle, area A reflects light beam 1, and reflected light beam 1 is captured by the camera on the car.
  • the car determines the emission time of the light beam 1 and the time when the reflected light beam 1 is captured by the camera on the car, and the distance between the area A on the obstacle and the car is determined by the time difference between the two.
  • the car can change the emission direction of the emitted light beam. For example, please continue to see Figure 3.
  • the light source on the car emits light beam 2 to area B on the obstacle, area B reflects light beam 2, and reflected light beam 2 is captured by the car’s camera ,
  • the car can determine the distance between the area B on the obstacle and the car.
  • a grating device (not shown in FIG. 3) is provided in a car, and the light beam emitted by the light source irradiates the obstacle through the grating device. Since the grating structure of the grating device may change, Furthermore, the exit direction of the light beam can be changed, and the obstacle can be scanned multiple times to obtain the distance information between different areas on the obstacle surface and the car. Therefore, the position of the light source does not need to be changed, so there is no need to set up a scanning device, which avoids the scanning device The problems of poor stability and reliability, large size and low space utilization caused by movement.
  • the distance from the different areas of the obstacle to the car can be calculated, and then the depth information of the entire obstacle can be obtained.
  • 3D modeling of obstacles based on the depth information of the obstacles can obtain information such as the shape, volume, and size of the obstacles. Furthermore, the shape, volume, and size of the obstacles can be reminded to the vehicle owner through images, text or voice. Size and other information.
  • a display screen can be set in the car to generate a 3D image of the obstacle based on the depth information of the obstacle, and display the three-dimensional image of the obstacle on the display screen, thereby providing a better user experience for the car owner.
  • the three-dimensional information of the above-mentioned obstacles can be provided to the automatic driving system of the car to help the automatic driving system control the car to automatically avoid the obstacle.
  • the electronic device in the embodiment of the present application can be a camera, a mobile phone, a vehicle-mounted system, etc.
  • the electronic device in the embodiment of the present application can also be other devices, such as a tablet Computers, Virtual Reality (VR) glasses, wearable devices (such as smart watches), etc.
  • Exemplary embodiments of portable terminals include but are not limited to carrying Or portable terminals with other operating systems.
  • the above-mentioned electronic device may not be a portable terminal, but a desktop computer capable of implementing an image shooting function.
  • FIG. 4 shows a schematic structural diagram of the mobile phone 100.
  • the mobile phone 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, Mobile communication module 151, wireless communication module 152, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone interface 170D, sensor module 180, buttons 190, motor 191, indicator 192, camera 193, display screen 194, and user An identification module (subscriber identification module, SIM) card interface 195, a light source 196, a grating device 197, etc.
  • SIM subscriber identification module
  • the sensor module 180 may include pressure sensor 180A, gyroscope sensor 180B, air pressure sensor 180C, magnetic sensor 180D, acceleration sensor 180E, distance sensor 180F, proximity light sensor 180G, fingerprint sensor 180H, temperature sensor 180J, touch sensor 180K, ambient light Sensor 180L, bone conduction sensor 180M, etc.
  • the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the mobile phone 100.
  • the mobile phone 100 may include more or fewer components than shown, or combine certain components, or split certain components, or arrange different components.
  • the illustrated components can be implemented in hardware, software, or a combination of software and hardware.
  • the processor 110 may include one or more processing units.
  • the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), and an image signal.
  • AP application processor
  • GPU graphics processing unit
  • ISP image signal processor
  • controller memory
  • video codec digital signal processor
  • DSP digital signal processor
  • baseband processor baseband processor
  • NPU neural-network processing unit
  • the controller may be the nerve center and command center of the mobile phone 100.
  • the controller can generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching and executing instructions.
  • a memory may also be provided in the processor 110 to store instructions and data.
  • the memory in the processor 110 is a cache memory.
  • the memory can store instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. Repeated accesses are avoided, the waiting time of the processor 110 is reduced, and the efficiency of the system is improved.
  • the mobile phone 100 implements a display function through a GPU, a display screen 194, and an application processor.
  • the GPU is a microprocessor for image processing, connected to the display 194 and the application processor.
  • the GPU is used to perform mathematical and geometric calculations for graphics rendering.
  • the processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.
  • the display screen 194 is used to display images, videos, etc.
  • the display screen 194 includes a display panel.
  • the display panel can adopt liquid crystal display (LCD), organic light-emitting diode (OLED), active-matrix organic light-emitting diode or active-matrix organic light-emitting diode (active-matrix organic light-emitting diode).
  • LCD liquid crystal display
  • OLED organic light-emitting diode
  • active-matrix organic light-emitting diode active-matrix organic light-emitting diode
  • AMOLED flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diode (QLED), etc.
  • the mobile phone 100 may include one or N display screens 194, and N is a positive integer greater than one.
  • the internal memory 121 may be used to store computer executable program code, where the executable program code includes instructions.
  • the processor 110 executes various functional applications and data processing of the mobile phone 100 by running instructions stored in the internal memory 121.
  • the internal memory 121 may include a storage program area and a storage data area.
  • the storage program area can store an operating system, at least one application program (such as a sound playback function, an image playback function, etc.) required by at least one function.
  • the data storage area can store data (such as audio data, phone book, etc.) created during the use of the mobile phone 100.
  • the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), etc.
  • UFS universal flash storage
  • the proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode.
  • the light emitting diode may be an infrared light emitting diode.
  • the mobile phone 100 emits infrared light to the outside through the light emitting diode.
  • the mobile phone 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the mobile phone 100. When insufficient reflected light is detected, the mobile phone 100 can determine that there is no object near the mobile phone 100.
  • the mobile phone 100 may use the proximity light sensor 180G to detect that the user holds the mobile phone 100 close to the ear to talk, so as to automatically turn off the screen to save power.
  • the proximity light sensor 180G can also be used in leather case mode, and the pocket mode will automatically unlock and lock the screen.
  • the ambient light sensor 180L is used to sense the brightness of the ambient light.
  • the mobile phone 100 can adaptively adjust the brightness of the display 194 according to the perceived brightness of the ambient light.
  • the ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures.
  • the ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the mobile phone 100 is in the pocket to prevent accidental touch.
  • the fingerprint sensor 180H is used to collect fingerprints.
  • the mobile phone 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photographs, fingerprint answering calls, and so on.
  • the temperature sensor 180J is used to detect temperature.
  • the mobile phone 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy.
  • Touch sensor 180K also called “touch panel”.
  • the touch sensor 180K may be disposed on the display screen 194, and the touch screen is composed of the touch sensor 180K and the display screen 194, which is also called a “touch screen”.
  • the touch sensor 180K is used to detect touch operations acting on or near it.
  • the touch sensor can pass the detected touch operation to the application processor to determine the type of touch event.
  • the visual output related to the touch operation can be provided through the display screen 194.
  • the touch sensor 180K may also be disposed on the surface of the mobile phone 100, which is different from the position of the display screen 194.
  • the camera 193 is used to capture still images or videos.
  • the camera 193 may include photosensitive elements such as a lens group and an image sensor, where the lens group includes a plurality of lenses (convex lens or concave lens) for collecting light signals reflected by the photographed object and transmitting the collected light signals to the image sensor .
  • the image sensor generates an image of the photographed object based on the light signal.
  • the light source 196 can be used to send a light beam to the grating device 197, and the light beam is emitted from the mobile phone via the grating device 197 and projected onto the surface of the object to be photographed.
  • the grating structure of the grating device 197 may change, which in turn causes the angle of the light beam irradiated on the grating device to be emitted from the grating device to change, which in turn causes the position of the light beam projected on the surface of the object to be photographed to change.
  • the light source 196 may be a laser emitter, an infrared light emitter, a visible light emitter, or the like. If the light source 196 is a laser emitter, the emitted light beam is laser; if the light source 196 is an infrared emitter, the emitted light beam is infrared light; if the light source 196 is a visible light emitter, the emitted light beam is visible light. Of course, the light source 196 may also be a light source emitting structured light, such as a dot matrix projector.
  • the camera 193 may include 1-N cameras.
  • the camera needs to capture the light beam emitted by the light source on the mobile phone to obtain the depth information. Therefore, if the light source is an infrared light emitter, the corresponding camera can be an infrared camera.
  • the mobile phone 100 includes a camera, that is, the camera used for taking pictures and videos and the camera used for collecting depth information used by the camera application are the same camera. If the mobile phone 100 includes multiple cameras, the camera used for taking pictures and videos and the camera used for collecting depth information used by the camera application may be different cameras.
  • the camera used by the camera application is a visible light camera, and the camera used to collect the depth information of the photographed object is an infrared camera.
  • one camera is used to capture images such as a visible light camera, and the other camera is used to capture depth such as an infrared camera.
  • the main interface includes images of various applications, such as the image of the camera application.
  • the user clicks the image of the camera application on the touch screen, and the touch sensor 180K detects the user's click operation and sends the click operation to The processor 110, the processor 110 determines that the user clicks the camera application according to the position of the click operation, the processor 110 starts the camera application, turns on the camera 193 (the activation sequence of the visible light camera and the infrared camera is not limited), and the display screen 194 displays the camera The interface of the application, such as the viewfinder interface.
  • the visible light camera After starting the visible light camera, the visible light camera collects the visible light reflected by the photographed object, generates a 2D image of the photographed object based on the captured visible light information, and sends it to the processor 110;
  • the processor 110 activates the light source 196 to send the first infrared beam to the grating device 197.
  • the first infrared beam is projected to the first area on the surface of the object to be photographed via the grating device 197, and then the first area reflects the first infrared beam.
  • the reflected first infrared beam is captured by the infrared camera;
  • the processor 110 determines the first distance between the first area of the surface of the photographed object and the mobile phone based on the time difference between the first infrared beam emitted from the light source 196 and received by the infrared camera;
  • the processor 110 controls the grating structure in the grating device to change; after the grating structure changes, the light source 196 sends the grating device 197 The second infrared light beam, the second infrared light beam is projected to the second area of the surface of the photographed object via the grating device 197; after that, the second area reflects the second infrared light beam, and the reflected second infrared light beam is captured by the infrared camera; the processor is based on The time difference between the second infrared light beam emitted from the light source 196 and received by the infrared camera determines the distance between the second area of the surface of the photographed object and the mobile phone;
  • the infrared beams emitted by the light source 196 are projected to different areas on the surface of the object to be photographed, so as to achieve the purpose of scanning all areas on the surface of the object to be photographed, thereby obtaining the sum of the various areas on the surface of the object being photographed.
  • the distance of the mobile phone produces the depth information of the object being photographed.
  • the processor 110 combines the depth information of the photographed object to perform three-dimensional modeling of the photographed object to generate a three-dimensional model of the photographed object; combines the three-dimensional model of the photographed object and the 2D image to generate a three-dimensional image of the photographed object, And the three-dimensional image is displayed on the display screen 194.
  • the processor 110 and the grating device 197 may be directly connected, and the processor 110 outputs a control signal to the grating device 197 to control the structural change of the grating in the grating device 197.
  • the processor 110 outputs a first control signal to control the grating of the grating device 197 to adopt the first structure, or the processor 110 outputs a second control signal to control the grating of the grating device 197 to adopt the second structure.
  • the processor 110 and the grating device 197 may also be indirectly connected through other devices, and the control signal output by the processor 110 is converted by other devices and then input to the grating device 197.
  • the processor 110 is connected through a digital-to-analog conversion chip, a driving chip, and a grating device 197, the processor 110 outputs a control signal to the digital-to-analog conversion chip, and the digital-to-analog conversion chip performs digital-to-analog conversion on the output control signal of the processor 110 and outputs a modulation signal.
  • the driving chip the grating device 197 is driven by the driving chip to have the first structure or the second structure.
  • the mobile phone 100 can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. For example, music playback, recording, etc.
  • the mobile phone 100 can receive the key 190 input, and generate key signal input related to the user settings and function control of the mobile phone 100.
  • the mobile phone 100 can use the motor 191 to generate a vibration notification (such as an incoming call vibration notification).
  • the indicator 192 in the mobile phone 100 can be an indicator light, which can be used to indicate the charging status, power change, and can also be used to indicate messages, missed calls, notifications, and so on.
  • the SIM card interface 195 in the mobile phone 100 is used to connect to the SIM card.
  • the SIM card can be connected to and separated from the mobile phone 100 by inserting into the SIM card interface 195 or pulling out from the SIM card interface 195.
  • FIG. 5A shows a schematic diagram of the structure of the depth camera 200.
  • the depth camera 200 may include a processor 210, a light source 221, a grating device 222, a camera 220, a memory 230, and the like.
  • the light source 221 may be used to send a light beam to the grating device 222, and the light beam is emitted from the depth camera 200 through the grating device 222 and projected to the surface of the object to be photographed.
  • the grating device 222 may be arranged in front of the light source 221, for example, on the optical path of the light beam emitted by the light source 221.
  • the camera 220 cannot be arranged on the light path of the light emitted by the light source 221 to ensure that the light beam emitted by the light source 221 can reach the grating device 222 and be emitted from the grating device 222.
  • the grating structure of the grating device 222 may change, which in turn causes the angle at which the light beam irradiated on the grating device 222 emerges from the grating device 222 to change, which in turn causes the position of the light beam projected on the surface of the object to be photographed to change.
  • the processor 210 the light source 221, the grating device 222, the camera 220, the memory 230, etc.
  • the processor 210 please refer to the specific implementations of the processor, light source, grating device, camera, and memory in the mobile phone respectively, which will not be repeated here. .
  • the camera 220 may include 1-N cameras. If the depth camera 200 includes one camera, the camera used for taking pictures and video recording and the camera used for collecting depth information are the same camera. If the depth camera 200 includes multiple cameras, the camera used for taking pictures and videos and the camera used for collecting depth information may be different cameras.
  • the depth camera 200 has only one camera, and the light emitted by the light source 221 is visible light, and the camera is a visible light camera as an example.
  • the camera 220 collects the visible light reflected by the photographed object, generates a 2D image of the photographed object based on the captured visible light information, and sends it to the processor 210;
  • the processor 210 activates the light source 221 to send the first light beam to the grating device 222, and the first light beam is projected to the first area of the surface of the object to be photographed via the grating device 222. After that, the first area reflects the first light beam.
  • the first light beam is captured by the camera 220; the processor 210 determines the first distance between the first area of the surface of the photographed object and the depth camera 200 based on the time difference between the first light beam emitted from the light source 221 and received by the camera 220;
  • the processor 210 controls the grating structure in the grating device to change; after the grating structure changes, the light source 221 sends to the grating device 222 The second light beam, the second light beam is projected to the second area of the surface of the photographed object via the grating device 222; after that, the second area reflects the second light beam, and the reflected second light beam is captured by the camera 220; the processor is based on the second light beam from The time difference between when the light source 221 is emitted and received by the camera 220 determines the distance between the second area of the surface of the object being photographed and the depth camera 200;
  • the light beam emitted by the light source 221 is projected to different areas on the surface of the object to be photographed, so as to achieve the purpose of scanning all areas on the surface of the object to be photographed, and to obtain various areas and depths of the surface of the object.
  • the distance of the camera 200 generates depth information of the object being photographed.
  • the processor 210 combines the depth information of the photographed object to perform three-dimensional modeling of the photographed object to generate a three-dimensional model of the photographed object; combine the three-dimensional model of the photographed object and the 2D image to generate a three-dimensional image of the photographed object.
  • the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the depth camera 200.
  • the depth camera 200 may include more or fewer components than shown, or combine certain components, or split certain components, or arrange different components.
  • the illustrated components can be implemented by hardware, software, or a combination of software and hardware.
  • Example 1 The grating device 222 in the depth camera 200 is an LCoS spatial light modulator.
  • the depth camera 200 includes a processor 210, a camera 220, a memory 230, a light source 221, and an LcoS spatial light modulator 222.
  • the structure of the LcoS spatial light modulator 222 is described below.
  • the LCoS spatial light modulator 222 includes an electrode layer and a liquid crystal layer.
  • the electrode layer includes a positive electrode layer and a negative electrode layer disposed oppositely.
  • the liquid crystal layer is formed by a large number of liquid crystal molecules, and the liquid crystal layer is disposed on the positive and negative electrode layers. between.
  • the relative position of the positive and negative electrode layers can be in addition to the positive electrode layer on the upper layer of the liquid crystal layer and the negative electrode layer on the lower layer of the liquid crystal layer as shown in FIG. 6A.
  • the relative position of the positive and negative electrode layers can be It is exchanged, that is, the positive electrode layer is on the lower layer of the liquid crystal layer and the negative electrode layer is on the upper layer of the liquid crystal layer, which is not specifically limited in the embodiment of the present application.
  • the deflection angle of the liquid crystal molecules may be the angle between the liquid crystal molecules and the horizontal plane; different deflection angles of the liquid crystal molecules will result in different refractive indices of the liquid crystal molecules. Therefore, in FIG. 6B, the refractive indexes of the liquid crystal molecules in the two dashed frames are different, and the refractive indexes of the liquid crystal molecules in the two dashed frames and the liquid crystal molecules outside the dashed frames are also different.
  • FIGS. 6A and 6B illustrate the structure of the LCoS spatial light modulator 222, and the principle of diffraction of the light beam in the LCoS spatial light modulator 222 is described below.
  • the liquid crystal layer is divided into continuous M regions (the size of the M regions may be the same or different), as shown in FIG. 6C;
  • phase modulation amount is the phase difference between the emitted light and the incident light.
  • the phase of the incident beam is the same, but due to the phase modulation amount Different, so the phases of the outgoing beams passing through different sub-regions are adjusted differently, that is, the phases of the outgoing beams of different sub-regions are different), the light emitted by different sub-regions are superimposed on each other, so that area 1 can produce a beam of diffracted light .
  • the process of generating diffracted light in the area 1 on the liquid crystal layer is described.
  • the other areas of the M areas are also similar to the process, and will not be repeated.
  • the amount of phase modulation of the light by the M regions varies periodically with space (one region is a period), and the diffracted light generated by the M regions are superimposed to form a diffracted beam, which is emitted from the LCoS spatial light modulator 222.
  • the voltage is different, the deflection angle of the liquid crystal molecules is different, so the refractive index is different, so the phase modulation amount of the light irradiated on different sub-regions is different.
  • the incident light irradiated to each sub-region The phases are the same, but because the phase modulation amounts of different sub-regions are different, the phases of the emitted light from different sub-regions are different. It can be seen that the voltage of each sub-region is related to the phase modulation amount.
  • the phase modulation amount of the liquid crystal layer to the light in order to cause the light beam to be diffracted in the LCoS spatial light modulator 222, it is necessary to make the phase modulation amount of the liquid crystal layer to the light vary with the space. It changes periodically (the phase of the incident light in each area of the liquid crystal layer is the same), that is, the refractive index of the liquid crystal layer needs to be changed periodically with the space, that is, the arrangement structure of the liquid crystal molecules in the liquid crystal layer needs to change with the space It changes periodically, that is, it is necessary to make the voltage applied to the liquid crystal layer change periodically with space.
  • FIG. 6E is a schematic diagram of a possible correspondence between the voltage applied by the liquid crystal layer and the phase modulation amount in the LCoS spatial light modulator 222.
  • the corresponding relationship may be determined by the experimenter according to experiments, and stored in the camera 200. Controlling the voltages of different sub-regions according to the corresponding relationship shown in FIG. 6E will make the phases of the outgoing beams of different sub-regions exhibit a certain law.
  • FIG. 6E is only an example, and is not a limitation on the corresponding relationship between the voltage and the phase modulation. Those skilled in the art can set the relationship between the voltage and the phase modulation according to the actual situation. Not limited.
  • the voltage of sub-region 1 is v1 in Figure 6E, then the phase modulation amount of sub-region 1 is ⁇ /4; the voltage of sub-region 2 is V2 in Figure 6E, then The phase modulation amount of area 2 is ⁇ /2.
  • the phase modulation amount of the sub-regions 1 to n in the region 1 can be controlled to show a stepped (or stepped) distribution law.
  • FIG. 6F the phase of the outgoing beam of each sub-region (each step) is shown, where the height of a single step (step) is 2 ⁇ /n.
  • the voltage applied to the sub-regions 1 to n also has a stepped (or stepped) distribution law.
  • the refractive index values of the liquid crystal molecules in the sub-regions 1 to n also have a stepped (or stepped) distribution law.
  • FIG. 6F shows the distribution law of the phase modulation amount corresponding to area 1 in the LCoS spatial light modulator 222, and the other areas in the M areas are all in a similar manner to area 1.
  • the phase of the emitted light from all sub-regions in each region of the entire liquid crystal layer can present a distribution law similar to that shown in FIG. 6F, as shown in FIG. 6G, the phase modulation amount corresponding to region 1, region 2, and region M
  • the distribution law is consistent.
  • the LCoS spatial light modulator 222 may be a reflective diffraction grating as shown in FIG. 6D, or a transmissive diffraction grating. As shown in FIG. 7, the embodiment of the present application No specific restrictions. In the following description of this article, the LCoS spatial light modulator 222 is a reflective diffraction grating as an example for detailed description.
  • the emitted light rays of the M*N sub-regions on the LCoS spatial light modulator 222 are superimposed to form a diffracted light beam that is emitted from the LCoS spatial light modulator 222.
  • the principle of diffraction of the LCoS spatial light modulator 222 is introduced.
  • the structure of the LCoS spatial light modulator 222 may be changed, resulting in the LCoS spatial light before and after the structure change.
  • the diffracted light beams generated by the modulator 222 have different exit directions. In this case, the diffracted light beams generated by the LCoS spatial light modulator 222 can irradiate different areas on the photographed object.
  • the following describes the process of the structural change of the LCoS spatial light modulator 222, which causes the exit angle of the diffracted beam (hereinafter referred to as the diffraction angle) to change.
  • the beam diffraction angle ⁇ should satisfy:
  • the diffraction angle (the exit angle of the outgoing beam) ⁇ is related to N*d.
  • the control parameters N and/or d the liquid crystal layer in the LCoS spatial light modulator 222 can be changed.
  • the structure can further control the change of the exit angle ⁇ of the exit beam.
  • the corresponding relationship between ⁇ and N*d may be stored in the camera 200, and the corresponding relationship may be determined by the experimenter based on experiments.
  • the depth camera 200 may store the phase modulation amount distribution map corresponding to each group of N and ⁇ , and the depth camera 200 controls the structure of the liquid crystal layer in the LCoS spatial light modulator 222 based on the phase modulation amount distribution map.
  • the exit angle of the diffracted beam is ⁇ corresponding to the phase modulation amount distribution map.
  • the depth camera 200 may store the corresponding relationship between the voltage and the phase modulation amount as shown in FIG. 6E.
  • the corresponding relationship can be represented by the phase modulation amount distribution diagrams shown in FIGS. 8 and 9; if the depth camera 200 needs to control the exit angle of the diffracted beam of the LCoS spatial light modulator 222 to be 2.775°, Then the mobile phone 100 can determine the values of M and N (equivalent to determining the number of regions and the number of subregions in each region) according to the phase modulation amount distribution map shown in FIG. 8, and then determine the subregions in each region. The phase modulation amount of the region is then determined according to the corresponding relationship shown in FIG. 6E to determine the voltage of each subregion, and then voltage is applied to each subregion.
  • the process of acquiring depth information by the depth camera 200 is described below. Refer to Figure 10, the process includes the following steps:
  • the light source 221 on the camera 200 emits a first light beam to the LCoS spatial light modulator 222.
  • the processor 210 applies a first voltage to the electrode layer on the LCoS spatial light modulator 222 so that the structure of the liquid crystal layer assumes the first structure; the first light beam is diffracted on the LCoS spatial light modulator 222 to generate a first diffracted light beam, Emitted along the first direction, the first diffracted light beam is projected to the first area on the surface of the photographed object.
  • the first light beam A1 emitted by the light source 221 in the camera 200 is irradiated on the LCoS spatial light modulator 222, and the processor 210 according to the phase modulation amount distribution diagram shown in FIG. Determine the phase modulation amount corresponding to each area on the liquid crystal layer; and then determine the voltage that needs to be applied to each area according to the corresponding relationship between the pre-saved phase modulation amount and the voltage that needs to be applied (such as the corresponding relationship shown in Figure 6E) The corresponding voltage is applied to the area, so the LCoS spatial light modulator 222 has the first structure.
  • the angle is projected from the liquid crystal layer and projected onto the first area of the face.
  • the first diffracted light beam is reflected on the first area on the photographed object, and the reflected first reflected light beam is received by the camera 220.
  • the diffracted beam A2 is reflected on the first area of the photographed object, and the reflected beam A2' is received by the camera 220 at time t1'; the processor 210 determines the first area on the photographed object according to t1 and t1'. The distance of the area from the camera 200.
  • the processor 210 determines the time when the first light beam is emitted from the light source 221 and the time when the first light beam is received by the camera 220, and calculates the first distance from the first area to the camera 200 according to the time difference between the two.
  • the processor 210 applies a second voltage to the electrode layer on the LCoS spatial light modulator 222 to make the structure of the liquid crystal layer present the second structure; the second light beam is diffracted on the LCoS spatial light modulator 222 to generate a second diffracted light beam , Emitted along the second direction, and the second diffracted light beam is projected to the second area of the surface of the object to be photographed; wherein the first direction and the second direction are different.
  • the second beam B1 emitted by the light source 221 in the camera 200 irradiates the LCoS spatial light modulator 222, and the processor 210 determines the liquid crystal layer according to the phase modulation amount distribution diagram shown in FIG. The phase modulation amount corresponding to each area on the above; and then according to the corresponding relationship between the pre-saved phase modulation amount and the magnitude of the voltage that needs to be applied (as shown in Figure 6E), determine the voltage that needs to be applied to each area, and apply it to each area The corresponding voltage is applied, so the LCoS spatial light modulator 222 has the second structure.
  • the second diffracted beam is reflected on the second area, and the reflected second reflected beam is received by the camera 220.
  • the diffracted beam B2 is reflected in the second area of the photographed object, and the reflected beam B2' is received by the camera 220 at time t2'; the processor 210 can determine the second area to the camera according to t2' and t2. 200 distance.
  • the processor 210 determines the time when the second light beam is emitted from the light source 221 and the time received by the camera 220, and calculates the second distance from the second area to the camera 200 according to the time difference between the two.
  • the above process only describes that the voltage applied to the LCoS spatial light modulator 222 by the processor 210 is changed from the first voltage to the second voltage.
  • the processor 210 can continue to change the voltage applied to the LCoS spatial light modulator 222.
  • the voltage is applied to make the light beam emerge in multiple different directions, and then project to different areas on the surface of the object to be photographed, complete the scanning of the surface of the object, and then obtain the depth information of the object.
  • the processor 210 executes the above steps S1002-step S1007 multiple times to change the voltage applied to the electrode layer of the LCoS spatial light modulator 222 multiple times, so that the light beam can be emitted in multiple different directions, and then projected onto the surface of the object being photographed. In different areas, complete the scanning of the surface of the photographed object, and then obtain the depth information of the photographed object.
  • the two phase modulation amount distribution diagrams of Figure 8 and Figure 9 are taken as examples.
  • the camera 200 can store the corresponding diffraction angle (such as 0 degrees to 90 degrees).
  • Phase modulation amount distribution map the camera 200 can start from the phase modulation amount distribution map corresponding to the smallest diffraction angle, that is, first control the LCoS to adjust the structure according to the phase modulation amount distribution map corresponding to the smallest diffraction angle, so that the diffraction angle is the smallest diffraction angle ; Then, the phase modulation amount distribution can be adjusted so that the diffraction angle gradually increases until the diffraction angle reaches the maximum value.
  • the camera 200 may also start from the phase modulation amount distribution map corresponding to the largest diffraction angle, and then gradually reach the phase modulation amount distribution map corresponding to the smallest diffraction angle, and the embodiment of the present application does not limit this sequence.
  • the camera 200 may also first determine a rough angle range of the diffraction angle, and then only control the LCoS with the phase modulation amount distribution map corresponding to the diffraction angle within the angle range, which is not limited in the embodiment of the present application.
  • the light beam emitted by the light source 221 may be a linear light beam, a point light beam or a surface light source, which is not specifically limited.
  • the depth camera 200 only needs to control the light beam to move in one direction (ie, one-dimensional scanning) to complete the scanning of the surface of the photographed object; the light beam emitted by the light source may be a point
  • the depth camera 200 needs to control the beam to move in two directions (ie, two-dimensional scanning) to complete the scanning of the surface of the photographed object.
  • Figure 12(a) is a schematic diagram of a linear beam.
  • only one liquid crystal panel needs to be installed in the LCoS spatial light modulator 222 in the depth camera 200 to complete the beam movement in the y direction. Can complete the scanning of the surface of the object being photographed.
  • Fig. 12(b) is a schematic diagram of a point beam.
  • the LCoS spatial light modulator 222 in the depth camera 200 can realize two-dimensional scanning by arranging two mutually orthogonal liquid crystal panels.
  • the liquid crystal panel 1 is used to control the movement of the light beam in the x direction
  • the liquid crystal panel 2 is used to control the movement of the light beam in the y direction.
  • the embodiment of the present application sets the LCoS spatial light modulator in the depth camera, and adjusts the voltage applied by the LCoS spatial light modulator on the liquid crystal layer, so that the structure of the liquid crystal layer in the LCoS spatial light modulator is changed. , So that the diffraction angle of the diffracted beam is different, so the diffracted beam can be projected to different areas of the object to be photographed, and finally the scanning of the object is realized by the light beam.
  • the position of the light source does not need to move, that is, the position of the light source does not need to be moved by the scanning device, so there is no vibration and can improve reliability. Therefore, it can reduce the volume of the depth camera, which is more conducive to system integration, and overcomes the inherent defects of the mechanical scanning device in the prior art.
  • Example 2 The grating device 222 in the depth camera 200 is an acousto-optic deflector.
  • the structure of the acousto-optic polarizer is described below.
  • the acousto-optic deflector 222 includes a driving power source 222a, an acousto-optic medium 222b, and a piezoelectric transducer 222c.
  • the driving power supply 222a is used to drive the piezoelectric transducer 222c to generate ultrasonic waves.
  • the ultrasonic waves After the ultrasonic waves are transmitted into the acousto-optic medium 222b, it can cause the local compression and elongation of the acousto-optic medium to generate elastic strain.
  • the strain is periodic with time and space. Change, make the medium appear sparse and dense. When a light beam passes through a medium disturbed by ultrasonic waves, diffraction occurs, that is, the acousto-optic effect.
  • ⁇ B is the Bragg angle
  • ⁇ is the wavelength of incident light
  • n is the refractive index of the medium
  • ⁇ S is the wavelength of the sound wave in the medium
  • ⁇ i and ⁇ d are the incident angle and exit angle of the light, respectively. Since the Bragg angle is generally small, sin ⁇ B ⁇ B.
  • ⁇ S is the speed of sound in the acousto-optic medium 222b.
  • the angle ⁇ between the diffracted light and the incident light is The deflection angle of is equal to 2 times the Bragg angle:
  • the deflection angle ⁇ of the beam can be changed to achieve the purpose of controlling the direction of beam propagation.
  • the depth camera 200 can store the first corresponding relationship between the frequency f S of the ultrasonic wave and the diffraction angle ⁇ of the diffracted beam, and store The second correspondence between the frequency f S of the ultrasonic wave and the driving voltage of the driving power supply 222a. Therefore, after the camera 200 determines the diffraction angle ⁇ , it can determine the frequency of the ultrasonic wave corresponding to the diffraction angle ⁇ according to the diffraction angle ⁇ and the first corresponding relationship, and then determine the driving power source according to the frequency of the ultrasonic wave and the second corresponding relationship. 222a drive voltage. In this case, the camera 200 can control the diffraction angle ⁇ of the diffraction angle by controlling the drive voltage of the drive power supply 222a.
  • the camera 200 may start from the smallest diffraction angle to the largest diffraction angle, that is, first control the acousto-optic modulator to control the driving power supply 222a according to the voltage corresponding to the frequency corresponding to the smallest diffraction angle, so that the diffraction angle is the smallest diffraction angle. Then, the drive power 222a can be controlled according to the voltage corresponding to the frequency corresponding to the larger diffraction angle, so that the diffraction angle is increased until the maximum diffraction angle is reached.
  • the camera 200 may also start from the maximum diffraction angle and gradually reach the minimum diffraction angle, and the embodiment of the present application does not limit this order.
  • the process of acquiring depth information by the depth camera 200 is described below. See Figure 15.
  • the process includes the following steps:
  • the light source 221 on the camera 200 emits a first light beam to the acousto-optic deflector 222.
  • the processor 210 controls the driving power supply 222a to input the first voltage to the piezoelectric transducer 222c, and drives the piezoelectric transducer 222c to generate ultrasonic waves of the first frequency; after the ultrasonic waves of this frequency are transmitted into the acousto-optic medium 222b, the acousto-optic medium A first sparse and dense structure is formed; the first light beam is diffracted on the first sparse and dense structure to generate a first diffracted light beam, which is emitted along a first direction, and the first diffracted light beam is projected to a first area on the surface of the object to be photographed.
  • the first light beam A3 emitted by the light source 221 in the camera 200 irradiates the acousto-optic medium 222 b of the acousto-optic deflector 222.
  • the processor 210 controls the driving power supply 222a to input a voltage of V1 to the piezoelectric transducer 222c, so that the piezoelectric transducer 222c generates ultrasonic waves with a frequency of f S.
  • the ultrasonic waves enter the acousto-optic medium 222b the acousto-optic medium 222b is created.
  • the local compression and elongation produces elastic deformation, and the first dense structure appears.
  • the first light beam A3 is diffracted through the acousto-optic medium 222b with such a dense and dense structure, and the diffracted light beam A4 is emitted from the liquid crystal layer along a diffraction angle of ⁇ 3 and is projected onto the first area on the human face.
  • the processor 210 determines the time when the first light beam is emitted from the light source 221 and the time when the camera 220 receives the first reflected light beam, and calculates the first distance from the first area to the camera 200 according to the time difference between the two.
  • the first diffracted beam A4 is reflected in the first area, and the reflected beam A4' is received by the camera 220 at time t3'.
  • the processor 210 determines the distance from the first area to the camera 200 according to t3' and t3.
  • the processor 210 controls the driving power supply 222a to input the second voltage to the piezoelectric transducer 222c, and drives the piezoelectric transducer 222c to generate ultrasonic waves of the second frequency; after the ultrasonic waves of this frequency are transmitted to the acousto-optic medium 222b, the acousto-optic medium A second sparse and dense structure is formed; the second beam is diffracted on the second sparse and dense structure to generate a second diffracted beam, which is emitted along the second direction, and the second diffracted beam is projected onto the second area on the surface of the object to be photographed; The second direction is different from the first direction.
  • the second beam B3 emitted by the light source 221 in the camera 200 enters the acousto-optic medium 222b of the acousto-optic deflector 222.
  • the processor 210 controls the driving power supply 222a to the piezoelectric transducer 222c inputs a voltage of V2 so that the piezoelectric transducer 222c generates ultrasonic waves with a frequency of f S + ⁇ f S.
  • the degree of elastic deformation of the acousto-optic medium is changed.
  • the densification conditions of the sparseness and densities change to form a second densification structure.
  • the second light beam B3 is diffracted through the acousto-optic medium of the second densification structure, and the second diffracted light beam B4 emerges from the liquid crystal layer along the diffraction angle of ⁇ 4. Projected onto the second area of the face.
  • the second diffracted beam is reflected on the second area, and the reflected second reflected beam is received by the camera 220.
  • the processor 210 determines the time when the second light beam is emitted from the light source 221 and the time when the camera 220 receives the second reflected light beam, and calculates the second distance from the second area to the camera 200 according to the time difference between the two.
  • the second diffracted beam B4 is reflected in the second area, and the reflected beam B4' is received by the camera 220 at time t4', and the processor 210 of the camera 200 determines the second area to reach the second area according to t4' and t4. The distance of the camera 200.
  • the above process only describes that the voltage applied by the processor 210 to the acousto-optic deflector 222 is changed from the first voltage to the second voltage.
  • the processor 210 can continue to change the voltage applied to the acousto-optic deflector 222. Voltage, so that the light beam is emitted in multiple different directions, and then projected to different areas on the surface of the object to be photographed, complete the scanning of the surface of the object to be photographed, and then obtain the depth information of the object.
  • the light beam emitted by the light source 221 may be a linear beam or a point beam, which is not specifically limited in the embodiment of the present application.
  • the depth camera 200 when the light beam emitted by the light source is a linear beam, the depth camera 200 only needs to control the beam to move in one direction (that is, one-dimensional scanning) to complete the scanning of the surface of the object to be photographed. At this time, only an acousto-optic deflection is required. ⁇ 222 can be.
  • the depth camera 200 needs to control the beam to move in two directions (that is, two-dimensional scanning) to complete the scanning of the object.
  • two orthogonal acousto-optic deflection The two-dimensional scanning can be realized by the cascade connection of the devices 222.
  • the acousto-optic deflector 1 is used to control the movement of the light beam in the x direction
  • the acousto-optic deflector 2 is used to control the movement of the light beam in the y direction.
  • the acousto-optic medium 222b exhibits a different density structure, so that the light beam passes through the acousto-optic medium 222b.
  • the degree of diffraction causes the light beam to be projected from different diffraction angles to different areas on the surface of the object to be photographed, and finally to realize the scanning of the surface of the object to be photographed by the light beam.
  • the position of the light source does not need to move, that is, the position of the light source does not need to be moved by the scanning device, so there is no vibration and can improve Reliability and imaging quality are guaranteed, and there is no need to reserve moving space. Therefore, the volume of the depth camera can be reduced, which is more conducive to system integration, and overcomes the inherent defects of the mechanical scanning device in the prior art.
  • example 1 and example 2 are respectively introduced by taking the grating device as an LCoS and an acousto-optic deflector as examples.
  • other gratings can also be used, as long as the grating structure can be adjusted to make the illuminated grating
  • the diffraction angles of the beams above can be different.
  • the embodiments of the present application also provide a circuit system.
  • the circuit system may be one or more chips, such as a system on a chip.
  • the circuit system may be a component of the mobile phone 100 shown in FIG. 4, the depth camera 200 shown in FIG. 5A, or the depth camera 200 shown in FIG. 5A.
  • the circuit system is used to generate a first control signal, the first control signal is used to control the grating structure of the grating device to the first structure; the circuit system is also used to generate a second control signal, the second control signal is used To control the grating structure of the grating device to change to the second structure.
  • the term “when” can be interpreted as meaning “if" or “after” or “in response to determining" or “in response to detecting".
  • the phrase “when determining" or “if detected (statement or event)” can be interpreted as meaning “if determined" or “in response to determining" or “when detected (Condition or event stated)” or “in response to detection of (condition or event stated)”.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website, computer, server, or data center.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center integrated with one or more available media.
  • the usable medium may be a magnetic medium, (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state hard disk).

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)

Abstract

本申请提供一种物体深度信息的确定方法、电子设备和电路系统，其中方法包括：控制光栅器件的光栅结构为第一结构，光源产生的光束在光栅器件上产生第一衍射光束，第一衍射光束的衍射角为第一角度，第一衍射光束照射到待拍摄物体上的第一区域；其中，第一区域反射的第一反射光束被摄像头捕捉；调整光栅器件的光栅结构改变为第二结构，使得光束在光栅器件上产生第二衍射光束，第二衍射光束的衍射角为第二角度，第二衍射光束照射到待拍摄物体上的第二区域；其中，第二区域反射的第二反射光束被摄像头捕捉；根据第一反射光束和第二反射光束，确定待拍摄物体的深度信息。

Description

一种物体深度信息的确定方法、电子设备和电路系统
本申请要求在2019年03月26日提交中国国家知识产权局、申请号为201910234518.4、申请名称为“一种深度信息采集方法和装置”的中国专利申请的优先权，在2019年6月26日提交中国专利局、申请号为201910561182.2，申请名称为“一种物体深度信息的确定方法、电子设备和电路系统”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及终端技术领域，尤其涉及一种物体深度信息的确定方法、电子设备和电路系统。
背景技术
随着终端技术的进步，电子设备比如手机或相机等，拍摄功能越来越强大，而且拍摄的图像可以是三维图像，与二维图像相比，三维图像可以反映拍摄物体的更多的信息，也更符合人们对真实世界的认知。
三维图像通常是通过构建三维模型得到的，目前常用的三维模型构建方法包括：立体视觉法、结构光法以及飞行时间(time of flight，ToF)法等。以飞行时间为例，且以电子设备是相机为例，相机上设置的光源向被拍摄物体发射光束，该光束照射到拍摄物体上，经过拍摄物体反射，反射光被相机捕捉，通过相机发射光束和接收的反射光束之间的时间差来计算被拍摄物体表面各个区域与相机之间的距离，以产生被拍摄物体的深度信息，进而根据深度信息构建被拍摄物体的三维模型。
目前的一种方案为，电子设备比如相机可以通过光束对拍摄物体的区域进行扫描以获取拍摄物体表面不同区域和相机的距离信息，在这种方案中电子设备上的光源需要移动位置，因此使用这种方案的相机内设置有扫描装置，该扫描装置的位置发生移动以带动光源的位置发生移动，这样的话，光源在移动的过程中可以将光束照射到拍摄物体的不同区域上，以获取拍摄物体的不同区域和相机的距离信息。
但是，扫描装置在发生移动时，会发生振动，甚至还会触碰到相机内的其它器件，导致对它器件产生干扰，因此相机存在稳定性和可靠性差的问题；其次，相机需要为扫描装置提供足够大空间移动，因此相机还存在体积大，内部空间利用率较低的问题。
发明内容
本申请实施例提供一种物体深度信息的确定方法、电子设备和电路系统，用于解决现有技术中通过扫描装置带动光源移动所导致的相机稳定性和可靠性差、空间利用率较低等的问题。
第一方面，本申请实施例提供一种物体深度信息的确定方法，应用于电子设备。该电子设备可以是手机、智能相机等电子设备。该电子设备可以包括光源、光栅器件以及至少一个摄像头。该方法包括：控制所述光栅器件的光栅结构为第一结构，所述光源产生的光 束在所述光栅器件上产生第一衍射光束，所述第一衍射光束的衍射角为第一角度，所述第一衍射光束照射到待拍摄物体上的第一区域；其中，所述第一区域反射的第一反射光束被所述摄像头捕捉；调整所述光栅器件的光栅结构改变为第二结构，使得所述光束在所述光栅器件上产生第二衍射光束，所述第二衍射光束的衍射角为第二角度，所述第二衍射光束照射到所述待拍摄物体上的第二区域；其中，所述第二区域反射的第二反射光束被所述摄像头捕捉；根据所述第一反射光束和所述第二反射光束，确定所述待拍摄物体的深度信息。
本申请实施例中，电子设备在采集被拍摄物体的深度信息的过程中，通过控制光栅器件的结构发生变化，可以实现光束从不同衍射角度射出，对被拍摄物体进行扫描，而光源的位置不需要移动，因此不会产生振动，可以提高电子设备的可靠性，保证成像质量，同时也不需要预留移动空间，因此能够减小深度相机的体积，更利于系统集成，克服了现有技术中机械性的扫描装置固有的缺陷。并且，本申请实施例基于采集被拍摄物体的深度信息的方案中，可以在保证被拍摄物体的不同区域都能被扫描到的同时，还可以保证接收到的反射光束具有较大的能量，进而获得信噪比较高的反射光信号，能够避免由于信噪比不佳，导致最终构建的三维模型精确性下降的问题。
在一种可能的设计中，所述光栅器件可以是硅基液晶(liquid crystal on silicon，LCoS)空间光调制器；所述LCoS空间光调制器包括液晶层、第一电极层、第二电极层，所述液晶层位于所述第一电极层和所述第二电极层之间。相应的，控制所述光栅器件的光栅结构为第一结构，可以包括：在所述第一电极层和所述第二电极层之间施加第一电压，使得所述液晶层呈第一结构；所述第一结构包括：所述液晶层的折射率在第一方向上以第一周期为周期呈周期性变化；其中，每个周期内的折射率在所述第一方向上呈N级阶梯式递增或递减，所述第一方向平行于所述液晶层所在平面，N为大于等于2的整数；调整所述光栅器件的光栅结构改变为第二结构，可以包括：在所述第一电极层和所述第二电极层之间施加第二电压，使得所述液晶层呈第二结构；所述第二结构包括：所述液晶层的折射率在所述第一方向上以第二周期为周期呈周期性变化；其中，每个周期内的折射率在所述第一方向上呈Q级阶梯式递增或递减，Q为大于等于2的整数，Q≠N。
本实施方式可以基于LCoS空间光调制器实现光栅器件，通过控制LCoS空间光调制器中第一电极层、第二电极层之间的电压，即可使得第一电极层、第二电极层之间的液晶层的结构发生变化，进而使得投射到液晶层的光束从不同衍射角度射出，实现对被拍摄物体进行扫描，整个过程不需要移动光源的位置，所以能够提高电子设备的可靠性，保证成像质量，减小深度相机的体积，同时还可以提高反射光信号的信噪比。
在一种可能的设计中，控制所述光栅器件的光栅结构为第一结构，可以包括：根据衍射角和相位调制量之间的第一对应关系，确定与所述第一角度对应的第一相位调制量；其中，所述第一相位调整量在所述第一方向上以所述第一周期为周期呈周期性变化，且每个周期内的相位调制量在所述第一方向上呈N级阶梯式递增或递减；根据相位调制量和电压之间的第二对应关系，确定与所述第一相位调制量对应的第一电压；其中，所述第一电压在所述第一方向上以所述第一周期为周期呈周期性变化，且每个周期内的电压在所述第一方向上呈N级阶梯式递增或递减；在所述第一电极层和所述第二电极层之间施加所述第一电压，使得所述液晶层的折射率在所述第一方向上以所述第一周期为周期呈周期性变化。
本实施方式可以基于衍射角和相位调制量之间的第一对应关系确定与第一角度对应的 第一相位调制量，然后再根据相位调制量和电压之间的第二对应关系确定与第一相位调制量对应的第一电压，通过在LCoS空间光调制器中的第一电极层和第二电极层之间施加第一电压，使得光束的衍射角为第一角度，照射到待拍摄物体上的第一区域。实现了根据需求精准的控制光束的衍射角度，提高了物体深度信息采集的精准性和可靠性。
在一种可能的设计中，调整所述光栅器件的光栅结构改变为第二结构，可以包括：根据衍射角和相位调制量之间的第一对应关系，确定与所述第二角度对应的第二相位调制量；其中，所述第二相位调制量在所述第一方向上以所述第二周期为周期呈周期性变化，且每个周期内的相位调制量在所述第一方向上呈Q级阶梯式递增或递减；根据相位调制量和电压之间的第二对应关系，确定与所述第二相位调制量对应的第二电压；其中，所述第二电压在所述第一方向上以所述第二周期为周期呈周期性变化，且每个周期内的电压在所述第一方向上呈Q级阶梯式递增或递减；在所述第一电极层和所述第二电极层之间施加所述第二电压，使得所述液晶层的折射率在所述第一方向上以所述第二周期为周期呈周期性变化。
本实施方式可以基于衍射角和相位调制量之间的第一对应关系确定与第二角度对应的第二相位调制量，然后再根据相位调制量和电压之间的第二对应关系确定与第二相位调制量对应的第二电压，通过在LCoS空间光调制器中的第一电极层和第二电极层之间施加第二电压，使得光束的衍射角为第二角度，照射到待拍摄物体上的第二区域。实现了根据需求精准的控制光束的衍射角度发生改变，提高了物体深度信息采集的精准性和可靠性。
在一种可能的设计中，所述光栅器件还可以是声光偏转器；所述声光偏转器包括驱动电源、声光介质以及压电换能器，相应的，控制所述光栅器件的光栅结构为第一结构，可以包括：控制所述驱动电源向所述压电换能器输入第三电压，使得所述压电换能器产生第一频率的超声波，所述第一频率的超声波传入所述声光介质后，所述声光介质形成第一结构；调整所述光栅器件的光栅结构改变为第二结构，可以包括：控制所述驱动电压向所述压电换能器输入第四电压，使得所述压电换能器产生第二频率的超声波，所述第二频率的超声波传入所述声光介质后，所述声光介质形成第二结构。
本实施方式还可以基于声光偏转器实现光栅器件，通过控制声光偏转器中的驱动电源向压电换能器输入的电压，即可使得声光介质的结构发生变化，进而使得投射到声光介质的光束从不同衍射角度射出，实现对被拍摄物体进行扫描，整个过程不需要移动光源的位置，所以能够提高电子设备的可靠性，保证成像质量，减小深度相机的体积，同时还可以提高反射光信号的信噪比。
在一种可能的设计中，在控制所述驱动电源向所述压电换能器输入第三电压之前，还可以根据衍射角和超声波的频率之间的第三对应关系，确定与所述第一角度对应的超声波的第一频率；根据超声波的频率和驱动电源的电压之间的第四对应关系，确定与所述第一频率对应的第三电压。
本实施方式可以基于衍射角和超声波的频率之间的第三对应关系确定与第一角度对应的超声波的第一频率，然后再根据超声波的频率和驱动电源的电压之间的第四对应关系确定与第一频率对应的第三电压，通过驱动电源向压电换能器输入第三电压，即可使得光束的衍射角为第一角度，照射到待拍摄物体上的第一区域。实现了根据需求精准的控制光束的衍射角度，提高了物体深度信息采集的精准性和可靠性。
在一种可能的设计中，在控制所述驱动电压向所述压电换能器输入第四电压之前，还 可以根据衍射角和超声波的频率之间的第三对应关系，确定与所述第二角度对应的超声波的第二频率；根据超声波的频率和驱动电源的电压之间的第四对应关系，确定与所述第二频率对应的第四电压。
本实施方式可以基于衍射角和超声波的频率之间的第三对应关系确定与第二角度对应的超声波的第二频率，然后再根据超声波的频率和驱动电源的电压之间的第四对应关系确定与第二频率对应的第四电压，通过驱动电源向压电换能器输入第四电压，即可使得光束的衍射角为第二角度，照射到待拍摄物体上的第二区域。实现了根据需求精准的控制光束的衍射角度发生改变，提高了物体深度信息采集的精准性和可靠性。
第二方面，本申请实施例还提供一种电子设备，所述电子设备包括：至少一个处理器、光源、光栅器件以及至少一个摄像头；所述光源，用于产生光束，并将所述光束投射到所述光栅器件上；所述至少一个处理器，用于控制所述光栅器件的光栅结构为第一结构；其中，当所述光栅器件为第一结构时，所述光束在所述光栅器件上产生第一衍射光束，所述第一衍射光束的衍射角为第一角度，所述第一衍射光束照射到待拍摄物体上的第一区域；所述至少一个摄像头，用于捕捉所述第一区域反射的第一反射光束；所述至少一个处理器，还用于调整所述光栅器件的光栅结构改变为第二结构；其中，所述光栅器件为第二结构时，所述光束在所述光栅器件上产生第二衍射光束，所述第二衍射光束的衍射角为第二角度，所述第二衍射光束照射到所述待拍摄物体上的第二区域；所述至少一个摄像头，还用于捕捉所述第二区域反射的第二反射光束；所述至少一个处理器，还用于根据所述第一反射光束和所述第二反射光束，确定所述待拍摄物体的深度信息。
在一种可能的设计中，所述光栅器件为LCoS空间光调制器；所述LCoS空间光调制器包括液晶层；所述光栅器件的光栅结构为第一结构，包括：所述液晶层的折射率在第一方向上以第一周期为周期呈周期性变化；其中，每个周期内的折射率在所述第一方向上呈N级阶梯式递增或递减，所述第一方向平行于所述液晶层所在平面，N为大于等于2的整数；所述光栅器件的光栅结构为第二结构，包括：所述液晶层的折射率在所述第一方向上以第二周期为周期呈周期性变化；其中，每个周期内的折射率在所述第一方向上呈Q级阶梯式递增或递减，Q为大于等于2的整数，Q≠N。
在一种可能的设计中，所述光栅器件为声光偏转器；所述声光偏转器包括驱动电源、声光介质以及压电换能器；所述驱动电源用于：向所述压电换能器输入第三电压；所述压电换能器用于：在所述第三电压的驱动下产生第一频率的超声波；所述第一频率的超声波传入所述声光介质后，所述声光介质形成第一结构；所述驱动电源还用于：控制所述驱动电源向所述压电换能器输入第四电压；所述压电换能器还用于：在所述第四电压的驱动下产生第二频率的超声波；所述第二频率的超声波传入所述声光介质后，所述声光介质形成第二结构。
第三方面，本申请实施提供还一种电路系统，该电路系统可以是一个或多个芯片，比如，片上系统(system-on-a-chip，SoC)。该电路系统用于生成第一控制信号，所述第一控制信号用于控制光栅器件的光栅结构为第一结构；所述电路系统还用于生成第二控制信号，所述第二控制信号用于控制所述光栅器件的光栅结构改变为第二结构。
第四方面，本申请实施例还提供一种电子设备。该电子设备包括光源、光栅器件、至少一个摄像头，至少一个处理器和存储器；所述存储器用于存储一个或多个计算机程序； 当所述存储器存储的一个或多个计算机程序被所述至少一个处理器执行时，使得所述电子设备能够实现上述第一方面及其第一方面任一可能设计的技术方案。
第五方面，本申请实施例还提供一种电子设备，所述电子设备包括执行上述第一方面或者第一方面的任意一种可能的设计的方法的模块/单元；这些模块/单元可以通过硬件实现，也可以通过硬件执行相应的软件实现。
第六方面，本申请实施例还提供一种计算机可读存储介质，所述计算机可读存储介质包括计算机程序，当计算机程序在电子设备上运行时，使得所述电子设备执行本申请实施例第一方面及其第一方面任一可能设计的技术方案。
第七方面，本申请实施例还提供一种程序产品，包括指令，当所述程序产品在电子设备上运行时，使得所述电子设备执行本申请实施例第一方面及其第一方面任一可能设计的技术方案。
附图说明
为了更清楚地说明本申请实施例的技术方案，下面将对本申请实施例中所需要使用的附图作简单地介绍，显而易见地，下面所介绍的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本申请实施例提供的一种应用场景的示意图；
图2为本申请实施例提供的另一种应用场景的示意图；
图3为本申请实施例提供的另一种应用场景的示意图；
图4为本申请实施例提供的一种手机100的结构示意图；
图5A为本申请实施例提供的一种深度相机200的结构示意图；
图5B为本申请实施例提供的一种深度相机200的结构示意图；
图6A、图6B、图6C、图6D为本申请实施例提供的LCoS空间光调制器中的液晶面板的示意图；
图6E为本申请实施例中液晶层施加的电压和液晶层对光的相位调制量的一种可能的对应关系的示意图；
图6F为本申请实施例提供一种相位调制量分布图；
图6G为本申请实施例提供另一种相位调制量分布图；
图7为本申请实施例提供的LCoS空间光调制器中的另一种液晶面板的示意图；
图8为本申请实施例提供另一种相位调制量分布图；
图9为本申请实施例提供另一种相位调制量分布图；
图10为本申请实施例中一种可能的采集深度信息的方法流程图；
图11为本申请实施例中一种可能的深度相机200采集深度信息的过程示意图；
图12(a)为本申请实施例中线状光束的示意图；
图12(b)为本申请实施例中点状光束的示意图；
图13为本申请实施例中一种可能的二维扫描的示意图；
图14为本申请实施例中声光偏转器的结构示意图；
图15为本申请实施例中另一种可能的采集深度信息的方法流程图；
图16为本申请实施例中另一种可能的深度相机200采集深度信息的过程示意图；
图17为本申请实施例中另一种可能的二维扫描的示意图。
具体实施方式
为使本申请实施例的目的、技术方案和优点更加清楚，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述。
以下，对本申请中的部分用语进行解释说明，以便于本领域技术人员理解。
本申请实施例涉及的光栅，也称衍射光栅，光栅可以有多种，比如按照光栅对入射光的调制作用可以将光栅分类为振幅调制光栅和相位调制光栅。其中，相位调制光栅是指光栅上不同区域的折射率可以变化，不同区域能对入射光的相位进行分别调制，导致相位不同的出射光叠加，产生衍射光束。相位调制光栅可以有多种，比如硅基液晶(liquid crystal on silicon，LCoS)。
本申请实施例涉及的声光效应，是指超声波通过介质时会造成介质的局部压缩和伸长而产生弹性应变，该应变随时间和空间作周期性变化，使介质出现疏密相间的现象，如同一个光栅。当光通过受到超声波扰动的介质时就会发生衍射现象，使得光的传播方向发生偏转，这种现象称之为声光效应。采用声光效应原理的光栅可以有多种，比如声光偏转器(acousto optical deflectors，AOD)，具体可以是拉曼-奈斯声光偏转器，或者布拉格声光偏转器等，本申请实施例不限定。
需要说明的是，本申请实施例涉及的光栅器件，可以是上述任一种光栅，或者多种光栅的组合光栅，本申请实施例不作限定。下文中，主要以光栅器件是LCoS和AOD为例，所以LCoS和AOD的具体结构将在后文介绍。
本申请实施例涉及的多个，是指大于或等于两个。
需要说明的是，本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中字符“/”，如无特殊说明，一般表示前后关联对象是一种“或”的关系。且在本申请实施例的描述中，“第一”、“第二”等词汇，仅用于区分描述的目的，而不能理解为指示或暗示相对重要性，也不能理解为指示或暗示顺序。
下面介绍本申请实施例提供的几种可能的应用场景。
应用场景1：
照相机上的光源向被拍摄物体(图1中以人脸为例)上发射光束1，假设光束1投射在人脸上的区域A，区域A反射光束1，反射光束1被照相机的镜头捕捉到。照相机确定光束1的发射时间和反射光束1被照相机镜头捕捉到的时间，通过二者之间的时间差确定人脸上区域A与照相机的距离。
之后，照相机可以改变发射光束的发射方向，比如，请继续参见图1所示，照相机上光源向人脸上的区域B发射光束2，区域B反射光束2，反射光束2被照相机镜头捕捉到，照相机可以确定人脸上区域B与照相机的距离。
需要说明的是，在本申请实施例中，照相机中设置光栅器件(图1中未示出)，光源发射的光束经过光栅器件照射到人脸上，由于光栅器件的光栅结构可以发生变化，进而可以改变光束的出射方向，实现在人脸上扫描多次，以获取人脸不同区域和照相机的距离信息，所以，光源的位置无需改变，因此不再需要设置扫描装置，避免了扫描装置移动导致照相 机稳定性和可靠性差、空间利用率低等问题。
进一步的，通过不断调整光栅器件的光栅结构，向人脸上的不同区域投射光束，就可以得到人脸不同区域到照相机的距离，进而得到人脸的深度信息，进而构建人脸的3D图像。
应用场景2：
请参见图2所示，为本申请实施例提供的另一种应用场景的示意图。如图2所示，手机上集成有光源和摄像头，内部还设置有光栅器件(图2中未示出)。
手机上的光源向被拍摄物体(图2中以人脸为例)上发射光束1，假设光束1投射在人脸上的区域A，区域A反射光束1，反射光束1被手机的摄像头捕捉到。手机确定光束1的发射时间和反射光束1被摄像头捕捉到的时间，通过二者之间的时间差确定人脸上区域A与手机的距离。
之后，手机可以改变发射光束的发射方向，比如，请继续参见图2所示，手机上光源向人脸上的区域B发射光束2，区域B反射光束2，反射光束2被手机摄像头捕捉到，手机可以确定人脸上区域B与照手机的距离。
需要说明的是，在本申请实施例中，手机中设置光栅器件(图2中未示出)，光源发射的光束经过光栅器件照射到人脸上，由于光栅器件的光栅结构可以发生变化，进而可以改变光束的出射方向，实现在人脸上扫描多次，以获取人脸不同区域和手机的距离信息，所以，光源的位置无需改变，因此不再需要设置扫描装置，避免了扫描装置移动导致手机稳定性和可靠性差、空间利用率低等问题。
进一步的，通过不断调整光栅器件的光栅结构，向人脸上的不同区域投射光束，就可以得到人脸不同区域到照手机的距离，进而获得整个人脸的深度信息，基于该深度信息对人脸进行三维建模，就可以得到人脸的三维特征信息。
进一步的，基于人脸的三维特征信息，手机可实现人脸打卡、解锁手机、解锁应用、人脸支付等场景中的人脸识别的功能，或者是将人脸的三维建模功能集成在手机的手机app中，或者集成在微信视频、facetime，推特、朋友圈等需要拍摄的任何场景中，可在这些场景中实现三维图像的拍摄功能。
应用场景3：
请参见图3所示，为本申请实施例提供的另一种应用场景的示意图。如图3所示，汽车上设置有光源和摄像头，内部还设置有光栅器件(图3中未示出)。其中，在具体实施时，光源和摄像头除了可以如图3所示的设置在汽车尾部外，还可以设置在汽车头部、车身等位置，本申请实施例不做具体限制。
汽车上的光源向障碍物上发射光束1，假设光束1投射在障碍物上的区域A，区域A反射光束1，反射光束1被汽车上的摄像头捕捉到。汽车确定光束1的发射时间和反射光束1被汽车上的摄像头捕捉到的时间，通过二者之间的时间差确定障碍物上区域A与汽车的距离。
之后，汽车可以改变发射光束的发射方向，比如，请继续参见图3所示，汽车上光源向障碍物上的区域B发射光束2，区域B反射光束2，反射光束2被汽车的摄像头捕捉到，汽车可以确定障碍物上区域B与汽车的距离。
需要说明的是，在本申请实施例中，在汽车中设置光栅器件(图3中未示出)，光源发射的光束经过光栅器件照射到障碍物上，由于光栅器件的光栅结构可以发生变化，进而可 以改变光束的出射方向，实现在障碍物上扫描多次，以获取障碍物表面不同区域和汽车的距离信息，所以，光源的位置无需改变，因此不再需要设置扫描装置，避免了扫描装置移动所带来的稳定性和可靠性差、体积大及空间利用率较低的问题。
进一步的，通过不断调整光栅器件的光栅结构，向障碍物上的不同区域投射光束，就可以计算得到障碍物不同区域到照汽车的距离，进而获得整个障碍物的深度信息。
进一步的，基于障碍物的深度信息对障碍物进行三维建模可获得障碍物的形状、体积、尺寸等信息，进而还可以通过图像、文字或语音等方式向车主提示障碍物的形状、体积、尺寸等信息。
进一步的，还可以在车内设置显示屏，基于障碍物的深度信息生成障碍物的3D图像，并在显示屏上显示障碍物的三维图像，进而为车主提供更好的用户体验。
进一步地，上述障碍物的三维信息可以提供给汽车的自动驾驶系统，以帮助自动驾驶系统控制汽车自动规避障碍物。
以上列举了3中可能的应用场景，即本申请实施例中的电子设备可以是相机、手机、车载系统等，在实际应用中，本申请实施例中的电子设备还可以是其它设备，诸如平板电脑、虚拟现实(Virtual Reality，VR)眼镜、可穿戴设备(如智能手表)等。便携式终端的示例性实施例包括但不限于搭载
Figure PCTCN2020079806-appb-000001
或者其它操作系统的便携式终端。还应当理解的是，在本申请其他一些实施例中，上述电子设备也可以不是便携式终端，而是能够实现图像拍摄功能的台式计算机。
(1)以电子设备是手机为例，图4示出了手机100的结构示意图。
手机100可以包括处理器110，外部存储器接口120，内部存储器121，通用串行总线(universal serial bus，USB)接口130，充电管理模块140，电源管理模块141，电池142，天线1，天线2，移动通信模块151，无线通信模块152，音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，传感器模块180，按键190，马达191，指示器192，摄像头193，显示屏194，以及用户标识模块(subscriber identification module，SIM)卡接口195、光源196以及光栅器件197等。其中传感器模块180可以包括压力传感器180A，陀螺仪传感器180B，气压传感器180C，磁传感器180D，加速度传感器180E，距离传感器180F，接近光传感器180G，指纹传感器180H，温度传感器180J，触摸传感器180K，环境光传感器180L，骨传导传感器180M等。
可以理解的是，本申请实施例示意的结构并不构成对手机100的具体限定。在本申请另一些实施例中，手机100可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。
其中，处理器110可以包括一个或多个处理单元，例如：处理器110可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，存储器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。
其中，控制器可以是手机100的神经中枢和指挥中心。控制器可以根据指令操作码和 时序信号，产生操作控制信号，完成取指令和执行指令的控制。
处理器110中还可以设置存储器，用于存储指令和数据。在一些实施例中，处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据，可从所述存储器中直接调用。避免了重复存取，减少了处理器110的等待时间，因而提高了系统的效率。
手机100通过GPU，显示屏194，以及应用处理器等实现显示功能。GPU为图像处理的微处理器，连接显示屏194和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器110可包括一个或多个GPU，其执行程序指令以生成或改变显示信息。
显示屏194用于显示图像，视频等。显示屏194包括显示面板。显示面板可以采用液晶显示屏(liquid crystal display，LCD)，有机发光二极管(organic light-emitting diode，OLED)，有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的，AMOLED)，柔性发光二极管(flex light-emitting diode，FLED)，Miniled，MicroLed，Micro-oLed，量子点发光二极管(quantum dot light emitting diodes，QLED)等。在一些实施例中，手机100可以包括1个或N个显示屏194，N为大于1的正整数。
内部存储器121可以用于存储计算机可执行程序代码，所述可执行程序代码包括指令。处理器110通过运行存储在内部存储器121的指令，从而执行手机100的各种功能应用以及数据处理。内部存储器121可以包括存储程序区和存储数据区。其中，存储程序区可存储操作系统，至少一个功能所需的应用程序(比如声音播放功能，图像播放功能等)等。存储数据区可存储手机100使用过程中所创建的数据(比如音频数据，电话本等)等。此外，内部存储器121可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件，闪存器件，通用闪存存储器(universal flash storage，UFS)等。
其中，接近光传感器180G可以包括例如发光二极管(LED)和光检测器，例如光电二极管。发光二极管可以是红外发光二极管。手机100通过发光二极管向外发射红外光。手机100使用光电二极管检测来自附近物体的红外反射光。当检测到充分的反射光时，可以确定手机100附近有物体。当检测到不充分的反射光时，手机100可以确定手机100附近没有物体。手机100可以利用接近光传感器180G检测用户手持手机100贴近耳朵通话，以便自动熄灭屏幕达到省电的目的。接近光传感器180G也可用于皮套模式，口袋模式自动解锁与锁屏。
环境光传感器180L用于感知环境光亮度。手机100可以根据感知的环境光亮度自适应调节显示屏194亮度。环境光传感器180L也可用于拍照时自动调节白平衡。环境光传感器180L还可以与接近光传感器180G配合，检测手机100是否在口袋里，以防误触。
指纹传感器180H用于采集指纹。手机100可以利用采集的指纹特性实现指纹解锁，访问应用锁，指纹拍照，指纹接听来电等。
温度传感器180J用于检测温度。在一些实施例中，手机100利用温度传感器180J检测的温度，执行温度处理策略。
触摸传感器180K，也称“触控面板”。触摸传感器180K可以设置于显示屏194，由触摸传感器180K与显示屏194组成触摸屏，也称“触控屏”。触摸传感器180K用于检测作用于其上或附近的触摸操作。触摸传感器可以将检测到的触摸操作传递给应用处理器，以确定触摸事件类型。可以通过显示屏194提供与触摸操作相关的视觉输出。在另一些实施例中， 触摸传感器180K也可以设置于手机100的表面，与显示屏194所处的位置不同。
摄像头193用于捕获静态图像或视频。通常，摄像头193可以包括感光元件比如镜头组和图像传感器，其中，镜头组包括多个透镜(凸透镜或凹透镜)，用于采集被拍摄物体反射的光信号，并将采集的光信号传递给图像传感器。图像传感器根据所述光信号生成被拍摄物体的图像。其中，光源196可以用于向光栅器件197发送光束，光束经由光栅器件197从手机射出，投射至被拍摄物体的表面。光栅器件197的光栅结构可以发生变化，进而导致照射到光栅器件的光束从光栅器件射出的角度发生变化，进而导致光束投射在被拍摄物体的表面的位置发生变化。
应理解，光源196发出的光束具有良好的单色性。光源196可以是激光发射器、红外光发射器、可见光发射器等。如果光源196是激光发射器，则发出的光束是激光，如果光源196是红外发射器，则发出的光束是红外光，如果光源196是可见光发射器，则发出的光束是可见光。当然，光源196还可以是发射结构光的光源，比如点阵投射器。
在本申请实施例中，摄像头193可以包括1-N个摄像头。如前述内容可知，为了获取拍摄物体的深度信息，摄像头需要捕捉手机上光源发射出的光束以得到深度信息，所以，如果光源是红外光发射器，那么对应的，摄像头可以是红外摄像头。
因此，如果手机100包括一个摄像头，即相机应用所使用的用于拍照和录像的摄像头和用于采集深度信息的摄像头是同一摄像头。如果手机100包括多个摄像头，相机应用所使用的用于拍照和录像的摄像头和用于采集深度信息的摄像头可以是不同的摄像头。比如，相机应用所使用的摄像头是可见光摄像头，用于采集被拍摄物体深度信息的摄像头是红外摄像头。
以两个摄像头为例，且以一个摄像头用于拍摄图像比如可见光摄像头，另一个摄像头用于采集深度比如红外摄像头为例。
假设显示屏194上显示主界面，主界面中包括各个应用程序的图像，比如相机应用的图像，用户在触摸屏上点击相机应用的图像，触摸传感器180K检测到用户的点击操作，将点击操作发送给处理器110，处理器110根据点击操作的位置，确定用户点击相机应用，处理器110启动相机应用，打开摄像头193(可见光摄像头以及红外摄像头，二者的启动顺序不限定)，显示屏194显示相机应用的界面，例如取景界面。
在启动可见光摄像头之后，可见光摄像头采集被拍摄物体反射的可见光，并基于捕捉到的可见光信息生成被拍摄物体的2D图像，并发送给处理器110；
在启动红外摄像头之后，处理器110启动光源196向光栅器件197发送第一红外光束，第一红外光束经由光栅器件197投射至被拍摄物体的表面的第一区域，之后；第一区域反射第一红外光束，反射的第一红外光束被红外摄像头捕捉；处理器110基于第一红外光束从光源196发出到被红外摄像头接收的时间差确定被拍摄物体的表面的第一区域和手机的第一距离；
在第一红外光束从光栅器件射出之后，或者在第一红外光束被手机的红外摄像头接收之后，处理器110控制光栅器件内光栅结构发生变化；在光栅结构变化之后，光源196向光栅器件197发送第二红外光束，第二红外光束经由光栅器件197投射至被拍摄物体的表面的第二区域；之后，第二区域反射第二红外光束，反射的第二红外光束被红外摄像头捕捉；处理器基于第二红外光束从光源196发出到被红外摄像头接收的时间差确定被拍摄物 体的表面的第二区域和手机的距离；
通过多次这样的光栅结构调整过程，使得光源196发射的红外光束投射到被拍摄物体表面的不同区域，实现对被拍摄物体表面的全部区域进行扫描的目的，进而获得被拍摄物体表面各个区域和手机的距离，产生被拍摄物体的深度信息。
最后，处理器110结合被拍摄物体的深度信息对被拍摄物体进行三维建模，生成被拍摄物体的三维模型；将被拍摄物体的三维模型和2D图像结合处理，生成被拍摄物体的三维图像，并将该三维图像在显示屏194上进行显示。
应理解，在本申请实施例中，处理器110和光栅器件197可以是直接相连，处理器110输出控制信号至光栅器件197，控制光栅器件197中的光栅的结构变化。例如，处理器110输出第一控制信号控制光栅器件197的光栅呈第一结构，或处理器110输出第二控制信号控制光栅器件197的光栅呈第二结构。
处理器110和光栅器件197也可以是通过其它器件间接相连，处理器110输出的控制信号经过其它器件转换后再输入到光栅器件197。例如，处理器110通过数模转换芯片、驱动芯片和光栅器件197连接，处理器110输出控制信号至数模转换芯片，数模转换芯片对处理器110输出控制信号进行数模转换后输出调制信号到驱动芯片，由驱动芯片驱动光栅器件197呈第一结构或第二结构。
另外，手机100可以通过音频模块170，扬声器170A，受话器170B，麦克风170C，耳机接口170D，以及应用处理器等实现音频功能。例如音乐播放，录音等。手机100可以接收按键190输入，产生与手机100的用户设置以及功能控制有关的键信号输入。手机100可以利用马达191产生振动提示(比如来电振动提示)。手机100中的指示器192可以是指示灯，可以用于指示充电状态，电量变化，也可以用于指示消息，未接来电，通知等。手机100中的SIM卡接口195用于连接SIM卡。SIM卡可以通过插入SIM卡接口195，或从SIM卡接口195拔出，实现和手机100的接触和分离。
(2)以电子设备是深度相机为例，图5A示出了深度相机200的结构示意图。
深度相机200可以包括处理器210、光源221、光栅器件222、摄像头220以及存储器230等。
其中，光源221可以用于向光栅器件222发送光束，光束经由光栅器件222从深度相机200射出，投射至被拍摄物体的表面。光栅器件222可以设置在光源221的前方，比如设置在光源221发出的光束的光路上。摄像头220不能设置在光源221发出的光线的光路上，以保证光源221发出的光束能够到达光栅器件222并从光栅器件222射出。光栅器件222的光栅结构可以发生变化，进而导致照射到光栅器件222的光束从光栅器件222射出的角度发生变化，进而导致光束投射在被拍摄物体的表面的位置发生变化。
对于处理器210、光源221、光栅器件222、摄像头220以及存储器230等的具体实现方式可以分别参考上述手机中的处理器、光源、光栅器件、摄像头以及存储器的具体实现方式，此处不再赘述。
例如，摄像头220可以包括1-N个摄像头。如果深度相机200包括一个摄像头，则用于拍照和录像的摄像头和用于采集深度信息的摄像头是同一摄像头。如果深度相机200包括多个摄像头，则用于拍照和录像的摄像头和用于采集深度信息的摄像头可以是不同的摄像头。
以深度相机200只有一个摄像头为例，且以光源221发出的光为可见光，摄像头为可见光摄像头为例。
摄像头220启动之后，摄像头220采集被拍摄物体反射的可见光，并基于捕捉到的可见光信息生成被拍摄物体的2D图像，并发送给处理器210；
与此同时，处理器210启动光源221向光栅器件222发送第一光束，第一光束经由光栅器件222投射至被拍摄物体的表面的第一区域，之后；第一区域反射第一光束，反射的第一光束被摄像头220捕捉；处理器210基于第一光束从光源221发出到被摄像头220接收的时间差确定被拍摄物体的表面的第一区域和深度相机200的第一距离；
在第一光束从光栅器件射出之后，或者在第一光束被深度相机200的摄像头220接收之后，处理器210控制光栅器件内光栅结构发生变化；在光栅结构变化之后，光源221向光栅器件222发送第二光束，第二光束经由光栅器件222投射至被拍摄物体的表面的第二区域；之后，第二区域反射第二光束，反射的第二光束被摄像头220捕捉；处理器基于第二光束从光源221发出到被摄像头220接收的时间差确定被拍摄物体的表面的第二区域和深度相机200的距离；
通过多次这样的光栅结构调整过程，使得光源221发射的光束投射到被拍摄物体表面的不同区域，实现对被拍摄物体表面的全部区域进行扫描的目的，进而获得被拍摄物体表面各个区域和深度相机200的距离，产生被拍摄物体的深度信息。
最后，处理器210结合被拍摄物体的深度信息对被拍摄物体进行三维建模，生成被拍摄物体的三维模型；将被拍摄物体的三维模型和2D图像结合处理，生成被拍摄物体的三维图像。
可以理解的是，本申请实施例示意的结构并不构成对深度相机200的具体限定。在本申请另一些实施例中，深度相机200可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件，或软件和硬件的组合实现。
下面，以图5A所示的深度相机200为例，介绍本申请实施例提供的采集深度信息的两种示例。
示例1：深度相机200中的光栅器件222是LCoS空间光调制器。
请参见图5B，深度相机200包括：处理器210、摄像头220、存储器230、光源221以及LcoS空间光调制器222。
下面介绍LcoS空间光调制器222的结构。
请参见图6A，LCoS空间光调制器222包括电极层和液晶层，其中电极层包括相对设置的正电极层和负电极层，液晶层由大量液晶分子形成，液晶层设置在正、负电极层之间。在具体实施过程中，正、负电极层的相对位置除了可以是如图6A所示的正电极层在液晶层上层、负电极层在液晶层的下层外，正、负电极层的相对位置可以被调换，即正电极层在液晶层下层、负电极层在液晶层的上层，本申请实施例对此不做具体限制。
当在电极层施加有电压时，正、负电极层之间形成电场，液晶分子在电场的作用下发生偏转。例如图6B所示，用虚线框中示出的液晶分子区域施加有电压，虚线框外的液晶分 子区域未施加有电压，加载有电压的电极层区域对应的液晶分子(即虚线框中的液晶分子)在电场的作用下发生了偏转。
不同的电场会使液晶分子的倾斜角度不同。如图6B所示，假设电压V1小于电压V2，则施加电压为V1的区域的电场强度E1小于施加电压为V2的区域的电场强度E2，则施加电压为V1的区域中的液晶分子的偏转角度小于施加电压为V2的区域中的液晶分子的偏转角度。其中，液晶分子的偏转角度可以是液晶分子与水平面的夹角；液晶分子的偏转角度不同会导致液晶分子的折射率不同。因此，图6B中，两个虚线框中的液晶分子的折射率不同，而且，两个虚线框中的液晶分子和虚线框外的液晶分子的折射率也不同。
应理解，图6A和图6B介绍了LCoS空间光调制器222的结构，下面介绍光束在LCoS空间光调制器222发生衍射的原理。
(1)液晶层被划分为连续的M个区域(M个区域的大小可以相同或不同)，如图6C所示；
针对M个区域中的任一区域，以区域1为例，请参见图6D，进一步将区域1划分为n个大小相同的子区域。
(2)区域1中的不同子区域上加载不同大小的电压，使得不同子区域中的液晶分子发生不同程度的偏转，进而使得不同子区域的折射率不同，如图6D中的子区域1、2、3，分别加载V1、V2、V3大小的电压，使得区域1、2、3的折射率不同。
当光束投射到区域1后，由于区域1中的不同子区域的折射率不同，导致区域1中不同子区域的出射光束之间会产生相位差(或光程差)(因为不同子区域的折射率不同，所以不同子区域的相位调制量不同，相位调制量即出射光和入射光的相位差值，所以对于区域1上的每个子区域来说，入射光束的相位相同，但是由于相位调制量不同，所以经过不同子区域的出射光束的相位被调整的不同，即不同子区域的出射光束的相位不同)，不同子区域发射出的光线相互叠加，这样的话，区域1可以产生一束衍射光线。
上述过程(2)中，描述了液晶层上的区域1产生衍射光线的过程，对于M个区域中的其它区域，也是类似的过程，不多赘述。M个区域对光的相位调制量随空间呈周期性变化(一个区域为一个周期)，M个区域产生的衍射光线叠加在一起，形成衍射光束，从LCoS空间光调制器222射出。
由上面的描述可知，电压不同，液晶分子的偏转角度不同，所以折射率不同，所以对照射到不同子区域上光线的相位调制量不同，对于不同的子区域，照射到各个子区域的入射光线的相位相同，但是由于不同子区域的相位调制量不同，所以不同子区域的出射光线的相位不同。由此可知，每个子区域的电压和相位调制量相关，所以在本申请一些实施例中，为了使得光束在LCoS空间光调制器222中发生衍射，需要使得液晶层对光的相位调制量随空间呈周期性变化(液晶层各区域的入射光的相位都相同)，也即需要使得液晶层的折射率随空间呈周期性变化，也即需要使得液晶层中的液晶分子的排布结构随空间呈周期性变化，也即需要使得液晶层上施加的电压随空间呈周期性变化。
示例性的，图6E为LCoS空间光调制器222中液晶层施加的电压和相位调制量的一种可能的对应关系的示意图。该对应关系可以是实验人员根据试验确定的，并存储在相机200中。按照图6E所示的对应关系控制不同子区域的电压，会使得不同子区域的出射光束的相位呈现一定的规律。应理解，图6E仅是一种举例，并不是对电压和相位调制量之间的对应 关系的限定，本领域技术人员可以根据实际情况设置电压和相位调制量之间的关系，本申请实施例不作限定。
举例来说，结合图6D和图6E，子区域1的电压为图6E中的v1，那么子区域1的相位调制量是π/4；子区域2的电压为图6E中的V2，那么子区域2的相位调制量为π/2。
应理解，在一些实施例中，通过图6E所示的电压和相位调制量的对应关系，可以控制区域1中的子区域1～n的相位调制量呈现台阶式(或阶梯式)的分布规律，如图6F，示出每个子区域(每个阶梯)的出射光束的相位，其中单个台阶(阶梯)的高度为2π/n。相应的，子区域1～n上施加的电压也呈台阶式(或阶梯式)的分布规律。相应的，子区域1～n中液晶分子的折射率的值也呈台阶式(或阶梯式)的分布规律。
图6F示出LCoS空间光调制器222中的区域1对应的相位调制量的分布规律，对于M个区域中的其它区域，均以和区域1类似的方式。这样，整个液晶层中的每个区域内的所有子区域的出射光线的相位都可以呈现类似图6F所示的分布规律，如图6G，区域1、区域2、区域M对应的相位调制量的分布规律一致。
需要说明的是，在具体实施时，LCoS空间光调制器222可以是如图6D所示的反射式的衍射光栅外，还可以是透射式的衍射光栅，如图7所示，本申请实施例不做具体限制。在本文接下来的描述中，主要以LCoS空间光调制器222是反射式的衍射光栅为例进行详细说明。
应理解，LCoS空间光调制器222上的M*N个子区域的出射光线叠加，形成衍射光束从LCoS空间光调制器222射出。
需要说明的是，在上述过程中，介绍了LCoS空间光调制器222发生衍射的原理，在本申请实施例中，LCoS空间光调制器222的结构可以发生变化，导致结构变化前后的LCoS空间光调制器222产生的衍射光束的出射方向不同，这样的话，LCoS空间光调制器222产生的衍射光束可以照射到拍摄物体上的不同区域。
下面，介绍LCoS空间光调制器222结构变化，进而导致衍射光束的出射角度(下文简称衍射角度)发生变化的过程。
假设LCoS空间光调制器222中液晶层的总长度(或者入射光覆盖的区域的总长度)为L，设单个区域内(即一个周期)内划分N个子区域，每个子区域的长度为d，根据反射式闪耀光栅的理论，光束衍射角θ应满足：
Figure PCTCN2020079806-appb-000002
当θ值较小时(比如0～10°)，
Figure PCTCN2020079806-appb-000003
由此可知，衍射角度(出射光束的出射角度)θ和N*d相关，在λ不变的情况下，通过控制参数N和/或d改变，可以改变LCoS空间光调制器222中液晶层的结构，进而可控制出射光束的出射角度θ的改变。
在本申请实施例中，相机200中可以存储θ和N*d的对应关系，该对应关系可以是实验人员可以根据实验确定的。
假设入射光束覆盖的区域总长度为L不变，一个子区域的长度d不变(比如为预设数量个像素的宽度)，在已知λ和N的情况下，当N＝4时，由M＝L/Nd可知M的值比如M＝8，根据θ和N*d的关系可知光束的衍射角度为θ＝2.775°；当N＝16时，由M＝L/Nd可知M的值比如M＝2，根据θ和N*d的关系可知光束的衍射角度为θ＝0.694°。
在一些实施例中，深度相机200中可以存储每一组N和θ对应的相位调制量分布图，深度相机200基于该相位调制量分布图控制LCoS空间光调制器222中液晶层的结构，那么衍射光束的出射角度就是与该相位调制量分布图对应的θ。例如，参见图8、图9所示，图8为N＝4、θ＝2.775°时的相位调制量分布图，图9为N＝16、θ＝0.694°时的相位调制量分布图。
在本申请实施例中，深度相机200中可以存储比如图6E所示电压和相位调制量的对应关系。在一些可能的设计中，该对应关系可以通过图8和图9所示的相位调制量分布图来表示；若深度相机200要控制LCoS空间光调制器222的衍射光束的出射角度是2.775°，那么手机100可以根据图8所示的相位调制量分布图确定M、N的取值(相当于确定区域的数量，以及每个区域内的子区域的数量)，然后确定每个区域中的子区域的相位调制量，然后根据图6E所示的对应关系，确定每个子区域的电压，然后在各个子区域上施加电压。
下面介绍深度相机200采集深度信息的过程。请参见图10，该过程包括如下步骤：
S1001、相机200上的光源221向LCoS空间光调制器222发射第一光束。
S1002、处理器210在LCoS空间光调制器222上的电极层施加第一电压使得液晶层的结构呈第一结构；第一光束在LCoS空间光调制器222上发生衍射，产生第一衍射光束，沿第一方向射出，第一衍射光束投射到被拍摄物体表面的第一区域。
举例来说，请参照图11，在t1时刻，相机200中的光源221发出的第一光束A1照射到LCoS空间光调制器222上，处理器210根据如图8所示的相位调制量分布图确定液晶层上各个区域对应的相位调制量；然后按照预先保存的相位调制量和需要施加的电压的对应关系(比如图6E所示的对应关系)，确定各个区域需要施加的电压，并向各个区域施加相应的电压，所以LCoS空间光调制器222呈第一结构，第一光束A1在第一结构的LCoS空间光调制器222上发生衍射现象，形成的衍射光束A2以θ＝2.775°的衍射角度从液晶层射出，投射到人脸上的第一区域上。
S1003、第一衍射光束在被拍摄物体上的第一区域上发生反射，反射回的第一反射光束被摄像头220接收。
继续参见图11，衍射光束A2在被拍摄物体的第一区域反射，反射回的光束A2’在t1’时刻被摄像头220接收；处理器210根据t1和t1’，确定被拍摄物体上的第一区域距离相机200的距离。
S1004、处理器210确定第一光束从光源221发出的时间和被摄像头220接收到的时间，根据二者之间的时间差计算第一区域到相机200的第一距离。
S1005、处理器210在LCoS空间光调制器222上的电极层施加第二电压使得液晶层被的结构呈现第二结构；第二光束在LCoS空间光调制器222上发生衍射，产生第二衍射光束，沿第二方向射出，第二衍射光束投射到被拍摄物体表面的第二区域；其中，第一方向和第二方向不同。
请继续参照图11，在t2时刻，相机200中的光源221发出的第二光束B1照射到LCoS空间光调制器222上，处理器210根据如图9所示的相位调制量分布图确定液晶层上各个区域对应的相位调制量；然后按照预先保存的相位调制量和需要施加的电压的大小的对应关系(如图6E所示的对应关系)，确定各个区域需要施加的电压，并向各个区域施加对应的电压，所以LCoS空间光调制器222呈第二结构。第二光束B1在第二种结构的LCoS空 间光调制器222上发生衍射现象，形成衍射光束B2并沿着θ＝0.694°衍射角度从液晶层射出，投射到人脸上的第二区域上。
S1006、第二衍射光束在第二区域上发生反射，反射回的第二反射光束被摄像头220接收。
继续参见图11，衍射光束B2在被拍摄物体的第二区域发生反射，反射回的光束B2’在t2’时刻被摄像头220接收；处理器210根据t2’和t2，可以确定第二区域到相机200的距离。
S1007、处理器210确定第二光束从光源221发出的时间和被摄像头220接收到的时间，根据二者之间的时间差计算第二区域到相机200的第二距离。
可选的，上述过程只描述处理器210在LCoS空间光调制器222上施加的电压由第一电压改变为第二电压，实际上，处理器210还可以继续改变施加在LCoS空间光调制器222上的电压，以使得光束沿多个不同的方向出射，进而投射到被拍摄物体表面的不同区域，完成对被拍摄物体表面的扫描，进而获得被拍摄物体的深度信息。
处理器210通过多次执行上述步骤S1002-步骤S1007，多次改变施加在LCoS空间光调制器222电极层上的电压，可以使得光束沿多个不同的方向出射，进而投射到被拍摄物体表面的不同区域，完成对被拍摄物体表面的扫描，进而获得被拍摄物体的深度信息。
需要说明的是，上面的实施例中以图8和图9两个相位调制量分布图为例，在实际应用中，相机200中可以存储每个衍射角(比如0度到90度)对应的相位调制量分布图，相机200可以先从最小的衍射角对应的相位调制量分布图开始，即先控制LCoS按照最小的衍射角对应的相位调制量分布图调整结构，使得衍射角度为最小衍射角；然后，可以调整相位调制量分布，使得衍射角逐渐增大，直到衍射角达到最大值。当然，相机200也可以从最大衍射角对应的相位调制量分布图开始，然后逐渐到最小的衍射角对应的相位调制量分布图，本申请实施例对此顺序不作限定。
当然，在实际应用中，相机200还可以先确定一个衍射角度的大致角度范围，然后只以该角度范围内的衍射角度对应的相位调制量分布图控制LCoS，本申请实施例不作限定。
可选的，在本申请实施例中，光源221发出的光束可以是线状光束，也可以是点状光束或者面状光源，对此不做具体限制。在光源221发出的光束可以是线状光束时，深度相机200只需要控制光束在一个方向上移动(即一维扫描)即可完成对被拍摄物体表面的扫描；在光源发出的光束可以是点状光束时，深度相机200需要控制光束在两个方向上移动(即二维扫描)才能完成对被拍摄物体表面的扫描。
举例来说，图12(a)为线状光束的示意图，在这种情况下，深度相机200中LCoS空间光调制器222中只需要设置一个液晶面板，即可完成光束在y方向上移动即可完成对被拍摄物体表面的扫描。
图12(b)为点状光束的示意图，在这种情况下，深度相机200中LCoS空间光调制器222可以通过设置两个相互正交的液晶面板的方式实现二维扫描。比如图13所示，液晶面板1用于控制光束在x方向上移动，液晶面板2用于控制光束在y方向上移动。
通过以上描述可知，本申请实施例通过在深度相机中设置LCoS空间光调制器，通过调节LCoS空间光调制器施加在液晶层上的电压，使得LCoS空间光调制器中的液晶层的结构发生变化，进而使得衍射光束的衍射角度不同，所以衍射光束可以投射到被拍摄物体的不 同区域，最终实现光束对被拍摄物体的扫描。可见，本申请实施例提供的深度相机在采集被拍摄物体的深度信息的过程中，光源的位置不需要移动，即不需要扫描装置带动光源的位置发生移动，因此不会产生振动，可以提高可靠性，保证成像质量，同时也不需要预留移动空间，因此能够减小深度相机的体积，更利于系统集成，克服了现有技术中机械性的扫描装置固有的缺陷。
并且，本申请实施例基于采集被拍摄物体的深度信息的方案中，可以在保证被拍摄物体的不同区域都能被扫描到的同时，还可以保证接收到的反射光束具有较大的能量，进而获得信噪比较高的反射光信号，能够避免由于信噪比不佳，导致最终构建的三维模型精确性下降的问题。
示例2：深度相机200中的光栅器件222是声光偏转器。
下面介绍声光偏振器的结构。
请参见图14，声光偏转器222包括驱动电源222a、声光介质222b和压电换能器222c。其中，驱动电源222a用于驱动压电换能器222c产生超声波，超声波传入声光介质222b后，可以造成声光介质的局部压缩和伸长而产生弹性应变，该应变随时间和空间作周期性变化，使介质出现疏密相间的现象。当光束通过受到超声波扰动的介质时就会发生衍射现象，即声光效应。
下文以布拉格声光偏转器为例进行详细说明。
根据布拉格衍射的基本理论，衍射应满足布拉格条件：
Figure PCTCN2020079806-appb-000004
θ i＝θ d＝θ B
其中，θ B为布拉格角，λ为入射光波长，n为介质折射率，λ S为声波在介质中的波长，θ i与θ d分别为光的入射角与出射角。由于布拉格角一般较小，所以有sinθ B≈θ B。其中，θ B＝λ/(2nλ S)＝(λ/2nν S)f S，ν S为声光介质222b中的声速，由布拉格原理可知，衍射光与入射光间的夹角θ，即光束的偏转角等于布拉格角的2倍：
Figure PCTCN2020079806-appb-000005
由此可见，只要改变超声波的频率f S，就可以改变光束的偏转角θ，达到控制光束传播方向的目的。
通过以上描述可知，超声波的频率f S和衍射光束的衍射角度θ相关，在一些实施例中，深度相机200可以存储超声波的频率f S和衍射光束的衍射角度θ的第一对应关系，并存储超声波的频率f S和驱动电源222a的驱动电压的第二对应关系。因此，相机200确定衍射角度θ之后，可以根据该衍射角度θ和第一对应关系，确定与所述衍射角度θ对应的超声波的频率，然后根据该超声波的频率和第二对应关系，确定驱动电源222a的驱动电压。这样的话，相机200通过控制驱动电源222a的驱动电压，就可以实现控制衍射角的衍射角度θ的目的。
示例性的，相机200可以从最小的衍射角度开始直到最大的衍射角度，即，先控制声光调制器按照最小的衍射角对应的频率对应的电压来控制驱动电源222a，使得衍射角度为最小衍射角；然后，可以按照较大的衍射角对应的频率对应的电压来控制驱动电源222a， 使得衍射角度增大，直到达到衍射角最大。当然，相机200也可以从最大衍射角开始，逐渐到最小的衍射角，本申请实施例对此顺序不作限定。
下面介绍深度相机200采集深度信息的过程。请参见图15，该过程包括如下步骤：
S1501、相机200上的光源221向声光偏转器222发射第一光束。
S1502、处理器210控制驱动电源222a向压电换能器222c输入第一电压，驱动压电换能器222c产生第一频率的超声波；该频率的超声波传入声光介质222b后，声光介质形成第一疏密结构；第一光束在第一疏密结构上发生衍射，产生第一衍射光束，沿第一方向射出，第一衍射光束投射到被拍摄物体表面的第一区域。
举例来说，参见图16，在t3时刻，相机200中的光源221发出的第一光束A3照射到声光偏转器222的声光介质222b。处理器210控制驱动电源222a向压电换能器222c输入V1大小的电压，以使压电换能器222c产生频率为f S的超声波，超声波传入声光介质222b后，造成声光介质222b的局部压缩和伸长而产生弹性形变，出现第一种疏密结构。第一光束A3经过这种疏密结构的声光介质222b发生衍射，形成衍射光束A4沿θ3的衍射角度从液晶层射出，投射到人脸上的第一区域。
S1503、第一衍射光束在第一区域上发生反射，反射回的第一反射光束被摄像头220接收。
S1504、处理器210确定第一光束从光源221发出的时间和摄像头220接收到第一反射光束的时间，根据二者之间的时间差计算第一区域到相机200的第一距离。
继续参见图16所示，第一衍射光束A4在第一区域发生反射，反射回的光束A4’在t3’时刻被摄像头220接收。处理器210根据t3’和t3，确定第一区域到相机200的距离。
S1505、处理器210控制驱动电源222a向压电换能器222c输入第二电压，驱动压电换能器222c产生第二频率的超声波；该频率的超声波传入声光介质222b后，声光介质形成第二疏密结构；第二光束在第二疏密结构上发生衍射，产生第二衍射光束，沿第二方向射出，第二衍射光束投射到被拍摄物体表面的第二区域；其中，第二方向和第一方向不同。
请继续参照图16，在t4时刻，相机200中的光源221发出的第二束光束B3进入声光偏转器222的声光介质222b，此时处理器210控制驱动电源222a向压电换能器222c输入V2大小的电压，以使压电换能器222c产生频率为f S+△f S的超声波，超声波传入声光介质222b后，造成声光介质的弹性形变程度发生改变，声光介质的疏密状况发生变化，形成第二种疏密结构，第二光束B3经过该第二种疏密结构的声光介质发生衍射，形成第二衍射光束B4沿θ4的衍射角度从液晶层射出，投射到人脸上的第二区域上。
S1506、第二衍射光束在第二区域上发生反射，反射回的第二反射光束被摄像头220接收。
S1507、处理器210确定第二光束从光源221发出的时间和摄像头220接收第二反射光束的时间，根据二者之间的时间差计算第二区域到相机200的第二距离。
请继续参见图16，第二衍射光束B4在第二区域发生反射，反射回的光束B4’在t4’时刻被摄像头220接收，相机200的处理器210根据t4’和t4，确定第二区域到相机200的距离。
可选的，上述过程只描述处理器210在声光偏转器222上施加的电压由第一电压改变为第二电压，实际上，处理器210还可以继续改变施加在声光偏转器222上的电压，以使 得光束沿多个不同的方向出射，进而投射到被拍摄物体表面的不同区域，完成对被拍摄物体表面的扫描，进而获得被拍摄物体的深度信息。
与LCoS空间光调制器类似，在本实施例中，光源221发出的光束可以是线状光束，也可以是点状光束，本申请实施例不做具体限制。
其中，在光源发出的光束是线状光束时，深度相机200只需要控制光束在一个方向上移动(即一维扫描)即可完成对被拍摄物体表面的扫描，此时只需要一个声光偏转器222即可。
在光源发出的光束是点状光束时，深度相机200需要控制光束在两个方向上移动(即二维扫描)才能完成对被拍摄物体的扫描，此时通过两个相互正交的声光偏转器222级联的方式即可实现二维扫描。比如图17所示，声光偏转器1用于控制光束在x方向上移动，声光偏转器2用于控制光束在y方向上移动。
通过以上描述可知，本申请实施例通过在深度相机中设置声光偏转器222，通过调节超声波的频率，使得声光介质222b呈现不同的疏密结构，使得光束在通过声光介质222b时发生不同程度的衍射，进而使得光束从不同的衍射角度投射到被拍摄物体表面的不同区域，最终实现光束对被拍摄物体表面的扫描。可见，本申请实施例提供的深度相机在对被拍摄物体进行光束扫描的过程中，光源的位置不需要移动，即不需要通过扫描装置带动光源的位置发生移动，因此不会产生振动，可以提高可靠性，保证成像质量，同时也不需要预留移动空间，因此能够减小深度相机的体积，更利于系统集成，克服了现有技术中机械性的扫描装置固有的缺陷。
并且，本申请实施例基于采集被拍摄物体的深度信息的方案中，可以在保证被拍摄物体的不同区域都能被扫描到的同时，还可以保证接收到的反射光束具有较大的能量，进而获得信噪比较高的反射光信号，能够避免由于信噪比不佳，导致最终构建的三维模型精确性下降的问题。
应理解，上述的示例1和示例2分别以光栅器件是LCoS和声光偏转器为例进行介绍的，在实际应用中，还可以采用其他光栅，只要能够实现通过调整光栅结构，使得照射的光栅上的光束的衍射角度不同即可。
基于同一技术构思，本申请实施例还提供一种电路系统，电路系统可以是一个或多个芯片，比如可以是片上系统。在一些实施例中，电路系统可以是图4所示的手机100、图5A所示的深度相机200或图5A所示的深度相机200中的一个部件。该电路系统用于生成第一控制信号，所述第一控制信号用于控制光栅器件的光栅结构为第一结构；所述电路系统还用于生成第二控制信号，所述第二控制信号用于控制所述光栅器件的光栅结构改变为第二结构。
本申请的各个实施方式可以任意进行组合，以实现不同的技术效果。
以上所述，以上实施例仅用以对本申请的技术方案进行了详细介绍，但以上实施例的说明只是用于帮助理解本申请实施例的方法，不应理解为对本申请实施例的限制。本技术领域的技术人员可轻易想到的变化或替换，都应涵盖在本申请实施例的保护范围之内。
上述实施例中所用，根据上下文，术语“当…时”可以被解释为意思是“如果…”或“在…后”或“响应于确定…”或“响应于检测到…”。类似地，根据上下文，短语“在确定…时”或“如果检测到(所陈述的条件或事件)”可以被解释为意思是“如果确定…”或 “响应于确定…”或“在检测到(所陈述的条件或事件)时”或“响应于检测到(所陈述的条件或事件)”。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线)或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如DVD)、或者半导体介质(例如固态硬盘)等。
为了解释的目的，前面的描述是通过参考具体实施例来进行描述的。然而，上面的示例性的讨论并非意图是详尽的，也并非意图要将本申请限制到所公开的精确形式。根据以上教导内容，很多修改形式和变型形式都是可能的。选择和描述实施例是为了充分阐明本申请的原理及其实际应用，以由此使得本领域的其他技术人员能够充分利用具有适合于所构想的特定用途的各种修改的本申请以及各种实施例。

Claims (14)

  1. 一种物体深度信息的确定方法，其特征在于，应用于电子设备，所述电子设备包括光源、光栅器件以及至少一个摄像头，所述方法包括：
    控制所述光栅器件的光栅结构为第一结构，所述光源产生的光束在所述光栅器件上产生第一衍射光束，所述第一衍射光束的衍射角为第一角度，所述第一衍射光束照射到待拍摄物体上的第一区域；其中，所述第一区域反射的第一反射光束被所述摄像头捕捉；
    调整所述光栅器件的光栅结构改变为第二结构，使得所述光束在所述光栅器件上产生第二衍射光束，所述第二衍射光束的衍射角为第二角度，所述第二衍射光束照射到所述待拍摄物体上的第二区域；其中，所述第二区域反射的第二反射光束被所述摄像头捕捉；
    根据所述第一反射光束和所述第二反射光束，确定所述待拍摄物体的深度信息。
  2. 如权利要求1所述的方法，其特征在于，所述光栅器件为硅基液晶LCoS空间光调制器；所述LCoS空间光调制器包括液晶层、第一电极层、第二电极层，所述液晶层位于所述第一电极层和所述第二电极层之间；
    控制所述光栅器件的光栅结构为第一结构，包括：
    在所述第一电极层和所述第二电极层之间施加第一电压，使得所述液晶层呈第一结构；所述第一结构包括：所述液晶层的折射率在第一方向上以第一周期为周期呈周期性变化；其中，每个周期内的折射率在所述第一方向上呈N级阶梯式递增或递减，所述第一方向平行于所述液晶层所在平面，N为大于等于2的整数；
    调整所述光栅器件的光栅结构改变为第二结构，包括：
    在所述第一电极层和所述第二电极层之间施加第二电压，使得所述液晶层呈第二结构；所述第二结构包括：所述液晶层的折射率在所述第一方向上以第二周期为周期呈周期性变化；其中，每个周期内的折射率在所述第一方向上呈Q级阶梯式递增或递减，Q为大于等于2的整数，Q≠N。
  3. 如权利要求2所述的方法，其特征在于，控制所述光栅器件的光栅结构为第一结构，包括：
    根据衍射角和相位调制量之间的第一对应关系，确定与所述第一角度对应的第一相位调制量；其中，所述第一相位调整量在所述第一方向上以所述第一周期为周期呈周期性变化，且每个周期内的相位调制量在所述第一方向上呈N级阶梯式递增或递减；
    根据相位调制量和电压之间的第二对应关系，确定与所述第一相位调制量对应的第一电压；其中，所述第一电压在所述第一方向上以所述第一周期为周期呈周期性变化，且每个周期内的电压在所述第一方向上呈N级阶梯式递增或递减；
    在所述第一电极层和所述第二电极层之间施加所述第一电压，使得所述液晶层的折射率在所述第一方向上以所述第一周期为周期呈周期性变化。
  4. 如权利要求3所述的方法，其特征在于，调整所述光栅器件的光栅结构改变为第二结构，包括：
    根据衍射角和相位调制量之间的第一对应关系，确定与所述第二角度对应的第二相位调制量；其中，所述第二相位调制量在所述第一方向上以所述第二周期为周期呈周期性变化，且每个周期内的相位调制量在所述第一方向上呈Q级阶梯式递增或递减；
    根据相位调制量和电压之间的第二对应关系，确定与所述第二相位调制量对应的第二电压；其中，所述第二电压在所述第一方向上以所述第二周期为周期呈周期性变化，且每个周期内的电压在所述第一方向上呈Q级阶梯式递增或递减；
    在所述第一电极层和所述第二电极层之间施加所述第二电压，使得所述液晶层的折射率在所述第一方向上以所述第二周期为周期呈周期性变化。
  5. 如权利要求1所述的方法，其特征在于，所述光栅器件为声光偏转器；所述声光偏转器包括驱动电源、声光介质以及压电换能器；
    控制所述光栅器件的光栅结构为第一结构，包括：
    控制所述驱动电源向所述压电换能器输入第三电压，使得所述压电换能器产生第一频率的超声波，所述第一频率的超声波传入所述声光介质后，所述声光介质形成第一结构；
    调整所述光栅器件的光栅结构改变为第二结构，包括：
    控制所述驱动电压向所述压电换能器输入第四电压，使得所述压电换能器产生第二频率的超声波，所述第二频率的超声波传入所述声光介质后，所述声光介质形成第二结构。
  6. 如权利要求5所述的方法，其特征在于，在控制所述驱动电源向所述压电换能器输入第三电压之前，所述方法还包括：
    根据衍射角和超声波的频率之间的第三对应关系，确定与所述第一角度对应的超声波的第一频率；
    根据超声波的频率和驱动电源的电压之间的第四对应关系，确定与所述第一频率对应的第三电压。
  7. 如权利要求6所述的方法，其特征在于，在控制所述驱动电压向所述压电换能器输入第四电压之前，所述方法还包括：
    根据衍射角和超声波的频率之间的第三对应关系，确定与所述第二角度对应的超声波的第二频率；
    根据超声波的频率和驱动电源的电压之间的第四对应关系，确定与所述第二频率对应的第四电压。
  8. 一种电子设备，其特征在于，所述电子设备包括：至少一个处理器、光源、光栅器件以及至少一个摄像头；
    所述光源，用于产生光束，并将所述光束投射到所述光栅器件上；
    所述至少一个处理器，用于控制所述光栅器件的光栅结构为第一结构；
    其中，当所述光栅器件为第一结构时，所述光束在所述光栅器件上产生第一衍射光束，所述第一衍射光束的衍射角为第一角度，所述第一衍射光束照射到待拍摄物体上的第一区域；
    所述至少一个摄像头，用于捕捉所述第一区域反射的第一反射光束；
    所述至少一个处理器，还用于调整所述光栅器件的光栅结构改变为第二结构；
    其中，所述光栅器件为第二结构时，所述光束在所述光栅器件上产生第二衍射光束，所述第二衍射光束的衍射角为第二角度，所述第二衍射光束照射到所述待拍摄物体上的第二区域；
    所述至少一个摄像头，还用于捕捉所述第二区域反射的第二反射光束；
    所述至少一个处理器，还用于根据所述第一反射光束和所述第二反射光束，确定所述 待拍摄物体的深度信息。
  9. 如权利要求8所述的电子设备，其特征在于，所述光栅器件为LCoS空间光调制器；所述LCoS空间光调制器包括液晶层；
    所述光栅器件的光栅结构为第一结构，包括：所述液晶层的折射率在第一方向上以第一周期为周期呈周期性变化；其中，每个周期内的折射率在所述第一方向上呈N级阶梯式递增或递减，所述第一方向平行于所述液晶层所在平面，N为大于等于2的整数；
    所述光栅器件的光栅结构为第二结构，包括：所述液晶层的折射率在所述第一方向上以第二周期为周期呈周期性变化；其中，每个周期内的折射率在所述第一方向上呈Q级阶梯式递增或递减，Q为大于等于2的整数，Q≠N。
  10. 如权利要求8所述的电子设备，其特征在于，所述光栅器件为声光偏转器；所述声光偏转器包括驱动电源、声光介质以及压电换能器；
    所述驱动电源用于：向所述压电换能器输入第三电压；
    所述压电换能器用于：在所述第三电压的驱动下产生第一频率的超声波；所述第一频率的超声波传入所述声光介质后，所述声光介质形成第一结构；
    所述驱动电源还用于：控制所述驱动电源向所述压电换能器输入第四电压；
    所述压电换能器还用于：在所述第四电压的驱动下产生第二频率的超声波；所述第二频率的超声波传入所述声光介质后，所述声光介质形成第二结构。
  11. 一种电路系统，其特征在于，
    所述电路系统用于生成第一控制信号，所述第一控制信号用于控制光栅器件的光栅结构为第一结构；
    所述电路系统还用于生成第二控制信号，所述第二控制信号用于控制所述光栅器件的光栅结构改变为第二结构。
  12. 一种电子设备，其特征在于，包括光源、光栅器件、至少一个摄像头，至少一个处理器和存储器；
    所述存储器用于存储一个或多个计算机程序；当所述存储器存储的一个或多个计算机程序被所述至少一个处理器执行时，使得所述电子设备能够实现如权利要求1-7任一所述的方法。
  13. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质包括计算机程序，当计算机程序在电子设备上运行时，使得所述电子设备执行如权利要求1-7任一所述的方法。
  14. 一种程序产品，其特征在于，包括指令，当所述指令在计算机上运行时，使得所述计算机执行如权利要求1-7任一项所述的方法。
PCT/CN2020/079806 2019-03-26 2020-03-18 一种物体深度信息的确定方法、电子设备和电路系统 WO2020192503A1 (zh)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN201910234518.4 2019-03-26
CN201910234518 2019-03-26
CN201910561182.2A CN110430415A (zh) 2019-03-26 2019-06-26 一种物体深度信息的确定方法、电子设备和电路系统
CN201910561182.2 2019-06-26

Publications (1)

Publication Number Publication Date
WO2020192503A1 true WO2020192503A1 (zh) 2020-10-01

Family

ID=68409588

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/079806 WO2020192503A1 (zh) 2019-03-26 2020-03-18 一种物体深度信息的确定方法、电子设备和电路系统

Country Status (2)

Country Link
CN (1) CN110430415A (zh)
WO (1) WO2020192503A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110430415A (zh) * 2019-03-26 2019-11-08 华为技术有限公司 一种物体深度信息的确定方法、电子设备和电路系统
CN114402253A (zh) * 2019-11-21 2022-04-26 华为技术有限公司 使用单个调制层进行衍射深度神经网络（d2nn）处理
CN113075641A (zh) * 2020-01-03 2021-07-06 华为技术有限公司 一种tof深度传感模组和图像生成方法

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103246072A (zh) * 2013-05-15 2013-08-14 福州大学 一种2d/3d转换的动态光栅
CN104597572A (zh) * 2015-01-16 2015-05-06 华中科技大学 一种基于硅基液晶的波长选择开关
CN107424186A (zh) * 2016-05-19 2017-12-01 纬创资通股份有限公司 深度信息测量方法及装置
CN108540791A (zh) * 2018-04-25 2018-09-14 京东方科技集团股份有限公司 一种双视显示方法和装置
CN109327653A (zh) * 2018-10-31 2019-02-12 Oppo广东移动通信有限公司 图像获取方法、图像获取装置、结构光组件及电子装置
EP3445049A1 (en) * 2017-08-14 2019-02-20 Facebook Technologies, LLC Camera assembly with programmable diffractive optical element for depth sensing
CN109425994A (zh) * 2017-08-31 2019-03-05 脸谱科技有限责任公司 深度相机组件、头戴式显示器和用于深度感测的方法
CN110430415A (zh) * 2019-03-26 2019-11-08 华为技术有限公司 一种物体深度信息的确定方法、电子设备和电路系统

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5907428A (en) * 1996-09-06 1999-05-25 Fuji Photo Film Co., Ltd. Acousto-optic element light deflector light beam scanning apparatus and image recording apparatus
CA2468391A1 (en) * 2001-12-07 2003-06-12 Discovision Associates A solid state microoptoelectromechanical system (moens) for reading a photonics diffractive memory
CN207937745U (zh) * 2018-02-08 2018-10-02 宁波舜宇光电信息有限公司 结构光投射装置、深度相机

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103246072A (zh) * 2013-05-15 2013-08-14 福州大学 一种2d/3d转换的动态光栅
CN104597572A (zh) * 2015-01-16 2015-05-06 华中科技大学 一种基于硅基液晶的波长选择开关
CN107424186A (zh) * 2016-05-19 2017-12-01 纬创资通股份有限公司 深度信息测量方法及装置
EP3445049A1 (en) * 2017-08-14 2019-02-20 Facebook Technologies, LLC Camera assembly with programmable diffractive optical element for depth sensing
CN109425994A (zh) * 2017-08-31 2019-03-05 脸谱科技有限责任公司 深度相机组件、头戴式显示器和用于深度感测的方法
CN108540791A (zh) * 2018-04-25 2018-09-14 京东方科技集团股份有限公司 一种双视显示方法和装置
CN109327653A (zh) * 2018-10-31 2019-02-12 Oppo广东移动通信有限公司 图像获取方法、图像获取装置、结构光组件及电子装置
CN110430415A (zh) * 2019-03-26 2019-11-08 华为技术有限公司 一种物体深度信息的确定方法、电子设备和电路系统

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ZHANG, ZHONGHUA ET AL.: "Passage", PRINCIPLES AND TECHNOLOGY OF OPTOELECTRONICS: 3, 31 October 2009 (2009-10-31), pages 224 - 226, XP009523654, ISBN: 978-7-81124-893-7 *

Also Published As

Publication number Publication date
CN110430415A (zh) 2019-11-08

Similar Documents

Publication Publication Date Title
WO2020192503A1 (zh) 一种物体深度信息的确定方法、电子设备和电路系统
US10897607B2 (en) Mobile terminal and method for controlling the same
KR101961666B1 (ko) 이동 단말기 및 그 제어방법
KR101892013B1 (ko) 이동 단말기
KR20160149068A (ko) 이동단말기 및 그 제어방법
KR20170005649A (ko) 3차원 깊이 카메라 모듈 및 이를 구비하는 이동 단말기
KR20180104970A (ko) 단말기 및 그 제어 방법
EP3936908A1 (en) Phase modulator and phase modulator array including the same
CN114079726B (zh) 一种拍摄方法及设备
US20220329726A1 (en) Wearable electronic device including cameras
CN115484383B (zh) 拍摄方法及相关装置
KR20170026002A (ko) 3차원 깊이 카메라 모듈 및 이를 구비하는 이동 단말기
CN115407308A (zh) 一种3d传感模组、3d传感方法及电子设备
CN117836698A (zh) 用于控制ar图像亮度的方法及装置
CN111711750B (zh) 基于人工智能的图像处理方法、装置、设备及介质
KR20160005862A (ko) 이동 단말기 및 그 제어방법
CN115580690A (zh) 图像处理的方法和电子设备
KR20220005283A (ko) 이미지 개선을 위한 전자장치 및 그 전자장치의 카메라 운용 방법
KR20220007004A (ko) 위상 변조기 및 이를 포함하는 위상 변조기 어레이
US11979515B2 (en) Mobile terminal and control method thereof
KR102067599B1 (ko) 이동 단말기 및 그 제어방법
CN116055871B (zh) 视频处理方法及其相关设备
US11726310B2 (en) Meta optical device and electronic apparatus including the same
US20230251362A1 (en) Method of removing interference and electronic device performing the method
KR101981124B1 (ko) 이동 단말기

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20778437

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20778437

Country of ref document: EP

Kind code of ref document: A1