WO2020088063A1

WO2020088063A1 - Electronic device

Info

Publication number: WO2020088063A1
Application number: PCT/CN2019/102735
Authority: WO
Inventors: 张学勇
Original assignee: Oppo广东移动通信有限公司
Priority date: 2018-10-31
Filing date: 2019-08-27
Publication date: 2020-05-07
Also published as: CN109067951A

Abstract

An electronic device (1000), comprising a display screen (10) and a structured light assembly (20), wherein the display screen (10) comprises a display area (11) having a front face (12) and a back face (13) facing opposite directions, and the front face (12) is used for displaying an image; and the structured light assembly (20) comprises a structured light projector (21). The structured light projector (21) is arranged on the side of the back face (13) of the display screen (10), and the structured light projector (21) is used for emitting structured light passing through the display area (11).

Description

Electronic device

Priority information

This application requests the priority and rights of the patent application with the patent application number 201811287335.0 filed with the State Intellectual Property Office of China on October 31, 2018, and the full text of which is hereby incorporated by reference.

Technical field

This application relates to the field of consumer electronics technology, and more specifically, to an electronic device.

Background technique

The mobile terminal can be equipped with a depth camera and a display screen. The depth camera can be used to obtain the depth information of the object. The display screen can be used to display text, patterns and other content. It is usually necessary to open a window on the display screen, for example, to form a bangs screen to make the display The display area of the screen is staggered from the position of the depth camera.

Summary of the invention

Embodiments of the present application provide an electronic device.

The electronic device according to the embodiment of the present application includes a display screen and a structured light component. The display screen includes a display area formed with opposite front and back surfaces, the front surface being used to display images. The structured light assembly includes a structured light projector that is disposed on a side of the display screen where the back side is located, and the structured light projector is used to emit structured light that passes through the display area.

In the electronic device of the embodiment of the present application, since the structured light projector is provided on the side where the back of the display screen is located, and the structured light emitted by the structured light projector passes through the display area and enters the external environment, there is no The opening of the structured light projector is aligned, and the screen of the electronic device is relatively high.

Additional aspects and advantages of the embodiments of the present application will be partially given in the following description, and some will become apparent from the following description, or be learned through practice of the embodiments of the present application.

BRIEF DESCRIPTION

The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

FIG. 1 is a schematic structural diagram of an electronic device according to some embodiments of the present application.

FIG. 2 is a partial structural diagram of an electronic device according to some embodiments of the present application.

FIG. 3 is a schematic cross-sectional view of the electronic device of some embodiments of the present application taken along line A-A shown in FIG. 2.

4 is a schematic diagram of a structured light projector according to some embodiments of the present application.

FIG. 5 is a schematic cross-sectional view of an electronic device according to certain embodiments of the present application taken along the line A-A shown in FIG. 2.

6 and 7 are schematic diagrams of partial structures of electronic devices according to certain embodiments of the present application.

FIG. 8 is a schematic cross-sectional view of an electronic device according to some embodiments of the present application along a position corresponding to line A-A shown in FIG. 2.

9 and 10 are schematic diagrams of partial structures of electronic devices according to some embodiments of the present application.

11 to 15 are schematic cross-sectional views of the electronic device of certain embodiments of the present application along the position corresponding to line A-A shown in FIG. 2.

16 is a schematic flowchart of an image acquisition method according to some embodiments of the present application.

17 is a schematic block diagram of an image acquisition device according to some embodiments of the present application.

18 and 19 are schematic flowcharts of image acquisition methods in some embodiments of the present application.

FIG. 20 is a schematic diagram of an image acquisition method according to some embodiments of the present application.

21 is a schematic flowchart of an image acquisition method according to some embodiments of the present application.

22 is a schematic diagram of an image acquisition method according to some embodiments of the present application.

23 to 27 are schematic flowcharts of image acquisition methods in some embodiments of the present application.

FIG. 28 is a schematic diagram of an image acquisition method according to some embodiments of the present application.

FIG. 29 is a schematic flowchart of an image acquisition method according to some embodiments of the present application.

FIG. 30 is a schematic diagram of an image acquisition method according to some embodiments of the present application.

31 to 34 are schematic flowcharts of an image acquisition method in some embodiments of the present application.

FIG. 35 is a schematic diagram of an optical path of structured light emitted by a structured light projector in some embodiments of the present application.

36 to 41 are schematic flowcharts of an image acquisition method according to some embodiments of the present application.

detailed description

The embodiments of the present application will be further described below with reference to the drawings. The same or similar reference numerals in the drawings indicate the same or similar elements or the elements having the same or similar functions throughout.

In addition, the embodiments of the present application described below with reference to the drawings are exemplary, and are only used to explain the embodiments of the present application, and cannot be construed as limiting the present application.

In this application, unless clearly specified and defined otherwise, the first feature is “on” or “under” the second feature may be that the first and second features are in direct contact, or the first and second features are indirectly through an intermediary contact. Moreover, the first feature is “above”, “above” and “above” the second feature may be that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. The first feature is "below", "below", and "below" the second feature may be that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is less horizontal than the second feature.

The electronic device 1000 according to the embodiment of the present application includes the display screen 10 and the structured light assembly 20. The display screen 10 includes a display area 11 formed with a front face 12 and a back face 13 opposite to each other. The front face 12 is used to display images. The structured light assembly 20 includes a structured light projector 21 disposed on the side of the back surface 13 of the display screen 10, and the structured light projector 21 is used to emit light through The structured light of the display area 11 is described.

In some embodiments, the structured light assembly 20 further includes a structured light camera 22, the structured light camera 22 is disposed on the side where the back surface 13 of the display screen 10 is located, and the structured light camera 22 is used In order to receive the modulated structured light passing through the display area 11.

In some embodiments, the display screen 10 is formed with a through slot 14 penetrating the front surface 12 and the back surface 13, and the structured light assembly 20 further includes a structured light camera 22, which is disposed on On the side where the back surface 13 of the display screen 10 is located, the structured light camera 22 is used to receive the modulated structured light passing through the through slot 14.

In some embodiments, the through slot 14 includes a notch 141 formed on the edge of the display screen 10; and / or the through slot 14 includes a through hole 142 spaced from the edge of the display screen 10.

In some embodiments, the edge of the display screen 10 includes any one or more of an upper edge, a lower edge, a left edge, and a right edge.

In some embodiments, the structured light assembly 20 further includes a floodlight 23 that is aligned with the structured light camera 22 and the same through slot 14.

In some embodiments, the electronic device 1000 further includes a cover plate 40 that is disposed on a side of the display screen 10 where the front surface 12 is located. An infrared transmission layer 50 is provided on the area corresponding to the groove 14.

In some embodiments, the region of the display screen 10 corresponding to the structured light projector 21 is formed with an infrared antireflection film 60; and / or the display screen 10 corresponds to the structured light projector 21 The area where the infrared transmission layer 50 is formed.

In some embodiments, the electronic device 1000 further includes a cover plate 40, the cover plate 40 is disposed on the side of the front surface 12 of the display screen 10, the cover plate 40 and the structured light are projected An infrared antireflection film 60 is formed in a region corresponding to the device 21.

In some embodiments, the display screen 10 is formed with a through slot 14 penetrating through the front face 12 and the back face 13, the electronic device 1000 further includes a visible light camera 70, the visible light camera 70 and the through slot 14 is aligned, and a visible light anti-reflection film 80 and / or an infrared cut-off film 90 are formed in the area of the cover plate 40 corresponding to the through slot 14.

In some embodiments, the structured light assembly 20 further includes a floodlight 23, the floodlight 23 is disposed on the side where the back surface 13 of the display screen 10 is located, and the floodlight 23 is used for To emit supplementary light passing through the display area 11.

In some embodiments, the display screen 10 is formed with a through slot 14 penetrating through the front surface 12 and the back surface 13, and the structured light assembly 20 further includes a floodlight 23, and the floodlight 23 is disposed at On the side where the back surface 13 of the display screen 10 is located, the floodlight 23 is used to emit supplementary light passing through the through slot 14.

In some embodiments, the display area 11 includes a first sub-display area 111 and a second sub-display area 112. The structured light emitted by the structured light projector 21 passes through the first sub-display area 111. The pixel density of the first sub-display area 111 is smaller than the pixel density of the second sub-display area 112.

In some embodiments, the first sub-display area 111 is used to display the status icon of the electronic device 1000.

In some embodiments, the first sub-display area 111 is located near the edge of the display area 11, and the second sub-display area 112 is located in the middle of the display area 11.

In some embodiments, the display area 11 includes a first sub-display area 111 and a second sub-display area 112. The structured light emitted by the structured light projector 21 passes through the first sub-display area 111. The first sub-display area 111 and the second sub-display area 112 can be independently controlled and displayed in different display states.

In some embodiments, the different display states include one or more of turning on or off, displaying at different brightness, and displaying at different refresh rates.

In some embodiments, when the structured light projector 21 emits structured light, the first sub-display area 111 goes out; or when the structured light projector 21 emits structured light, the first light is turned down. The display brightness of the sub-display area 111; or when the structured light projector 21 emits structured light, adjust the refresh frequency of the first sub-display area 111 so that the turn-on time of the first sub-display area 111 and the The opening time of the structured light projector 21 is staggered.

In some embodiments, when the structured light projector 21 is not activated, both the first sub-display area 111 and the second sub-display area 112 are turned on and displayed at the same refresh frequency.

In some embodiments, the display screen 10 is a liquid crystal display screen, or a Micro LED display screen, or an OLED display screen.

Please refer to FIGS. 1 and 2. The electronic device 1000 according to the embodiment of the present application includes a display screen 10 and a structured light component 20. The electronic device 1000 may further include a housing 30, which may be used to install functional devices such as the display screen 10, the structured light assembly 20, and the functional device may also be a main board, a dual camera module, a receiver, and so on. The specific form of the electronic device 1000 may be a mobile phone, a tablet computer, a smart watch, a head-mounted display device, etc. This application uses the electronic device 1000 as a mobile phone for illustration. It can be understood that the specific form of the electronic device 1000 is not limited to a mobile phone, and is not limited herein.

The display screen 10 may be installed on the housing 30. Specifically, the display screen 10 may be installed on one surface of the housing 30 or on both opposite surfaces of the housing 30 at the same time. In the example shown in FIG. 1, the display screen 10 is installed in front of the housing 30, and the display screen 10 can cover 85% or more of the area of the front, for example, 85%, 86%, 87%, 88% , 89%, 90%, 91%, 92%, 93%, 95% and even 100%. The display screen 10 can be used to display images, and the images can be text, images, videos, icons, and other information. The specific type of the display screen 10 may be a liquid crystal display screen, an OLED display screen, a Micro LED display screen, or the like. The display screen 10 includes a display area 11, which can be used to display images. To adapt to the needs of different types of electronic devices 1000 and different users, the shape of the display area 11 may be circular, elliptical, racetrack, rounded rectangle, rectangular, etc.

Referring to FIG. 3, the display area 11 is formed with a front surface 12 and a back surface 13 that are opposite to each other. The front surface 12 can be used to display an image. receive. Pixels are formed in the display area 11. In one example, the pixels can emit light to display corresponding colors. In another example, the pixels display corresponding colors under the influence of the backlight. There is usually a microscopic gap between pixels, and light rays will diffract when passing through the microscopic gap.

In some examples, the display screen 10 may further include a non-display area, and the non-display area may be formed on the periphery of the display area 11. The non-display area may not be used for display, and the non-display area may be used for combining with the housing 30 or for wiring. For example, the non-display area may be combined with the housing 30 by viscose without affecting the display function of the display area 11. The display screen 10 may also be a touch display screen integrated with a touch control function. After the user obtains the image information displayed on the display screen 10, the user may perform touch control on the display screen 10 to achieve a predetermined interactive operation.

The structured light component 20 can use the structured light to obtain the depth information of the target object for three-dimensional modeling, three-dimensional image generation, distance measurement, and the like. The structured light assembly 20 may be installed in the housing 30 of the electronic device 1000, specifically after being installed on the bracket, and then the support and the structured light assembly 20 are installed in the housing 30 together. The structured light assembly 20 may include a structured light projector 21, a structured light camera 22 and a floodlight 23.

Please refer to FIG. 1 to FIG. 4, the structured light projector 21 is disposed on the side where the back 13 of the display screen 10 is located, or the structured light projector 21 is disposed below the display area 11, and the structured light projector 21 is used for transmission Structured light passing through the display area 11. Specifically, the structured light projector 21 may include a light source 211, a collimating element 212, and a diffractive optical element 213. The light (such as infrared laser light) emitted by the light source 211 is first collimated by the collimating element 212, and then diffracted by the diffractive optical element 213 It is emitted from the structured light projector 21 and then passes through the display area 11 to project to the outside world. Both the microscopic gap of the display area 11 and the diffractive structure on the diffractive optical element 213 have a diffractive effect on the light emitted by the light source 211.

The structured light that passes through the display area 11 and enters the outside world may contain both a pattern diffracted by the diffractive optical element 213 (the pattern includes a plurality of spots diffracted by the diffractive optical element 213), and a microscopic gap diffracted by the display screen 10 The formed pattern (the pattern includes a plurality of spots diffracted by the diffractive optical element 213 and diffracted by the display screen 10), so that the speckle pattern after passing through the display area 11 has a high irrelevance, which is beneficial to subsequent Speckle patterns are processed. In one example, the transmittance of the display area 11 can reach 60% or more, so that the structured light emitted by the structured light projector 21 passes through the display area 11 with less loss.

The structured light camera 22 may be an infrared camera. The structured light is emitted to the target object. After being modulated by the target object, it can be acquired by the structured light camera 22. The structured light camera 22 receives the modulated structured light to obtain a speckle image. The speckle image is After processing, the depth data of the target object is obtained. The structured light camera 22 can also be arranged on the side of the back side 13 of the display screen 10, that is, under the display screen 10, which can be specifically arranged on the same bracket as the structured light projector 21, or the structured light camera 22 is directly installed on壳 30 上。 Housing 30. At this time, the light incident surface of the structured light camera 22 can be aligned with the display area 11, and the structured light modulated by the target object passes through the display area 11 and then received by the structured light camera 22, specifically, the structured light modulated by the target object It can be diffracted by the microscopic gap of the display screen 10 and then received by the structured light camera 22.

The floodlight 23 can be used to emit supplementary light outward, and the supplementary light can be used to supplement the light intensity in the environment when the ambient light is weak. In one example, the supplementary light may be infrared light. After the supplementary light is emitted onto the target object and reflected by the target object, it can be acquired by the structured light camera 22 to obtain a two-dimensional image of the target object, and the two-dimensional image information can be used for identity recognition. The floodlight 23 may also be disposed on the side where the back 13 of the display screen 10 is located, that is, under the display screen 10, and may be specifically disposed on the same bracket as the structured light projector 21 and the structured light camera 22. At this time, the supplementary light emitted by the floodlight 23 passes through the microscopic gap of the display area 11 and enters the external environment. The reflected supplementary light can again pass through the microscopic gap to be received by the structured light camera 22.

In summary, since the structured light projector 21 is disposed on the side where the back 13 of the display screen 10 is located, and the structured light emitted by the structured light projector 21 passes through the display area 11 and enters the external environment, there is no need In the opening where the structured light projector 21 is aligned, the screen area of the electronic device 1000 is relatively high.

Please refer to FIG. 5. In some embodiments, the display screen 10 is formed with a through slot 14, and the through slot 14 does not have a display function. The through groove 14 penetrates the front surface 12 and the rear surface 13. The structured light camera 22 is arranged on the side where the back surface 13 of the display screen 10 is located, and the structured light camera 22 is used to receive the modulated structured light passing through the through slot 14.

At this time, the light incident surface of the structured light camera 22 can be aligned with the through slot 14, and the structured light modulated by the target object passes through the through slot 14 and then received by the structured light camera 22. In this embodiment, since the modulated structured light does not need to pass through the microscopic gap of the display area 11 and will not be diffracted again by the microscopic gap, the speckle image acquired by the structured light camera 22 is the speckle image after modulation of the target object To reduce the difficulty of subsequent calculation of depth images based on speckle images.

Specifically, in the example shown in FIG. 6, the through slot 14 includes a notch 141 formed on the edge of the display screen 10, or in other words, the through slot 14 intersects the edge of the display screen 10. The notch 141 may be specifically formed on any one or more edges such as the upper edge, the lower edge, the left edge, and the right edge of the display screen 10. The shape of the notch 141 may be any shape such as a triangle, a semicircle, a rectangle, a track shape, etc., which is not limited herein.

In the example shown in FIG. 7, the through slot 14 includes a through hole 142 spaced from the edge of the display screen 10, or the through slot 14 is opened within a range enclosed by the edge of the display screen 10. The through hole 142 may specifically be close to any one or more edges such as the upper edge, the lower edge, the left edge, and the right edge of the display screen 10. The shape of the through hole 142 may be any shape such as a triangle, a circle, a rectangle, and a racetrack, which is not limited herein.

In some examples, the through slot 14 may also include the aforementioned notch 141 and the through hole 142 at the same time. The number of the notch 141 and the through hole 142 may be equal or unequal.

Referring to FIG. 8, in some embodiments, the floodlight 23 is disposed on the side where the back 13 of the display screen 10 is located, and the floodlight 23 is used to emit supplementary light passing through the through slot 14.

At this time, the supplementary light is directly emitted to the outside after passing through the through slot 14, and the supplementary light will not be weakened in the process of passing through the display area 11 to ensure that the target object receives a large amount of supplementary light.

Similar to the structured light camera 22, as shown in FIG. 9, the through slot 14 includes a notch 141 formed on the edge of the display screen 10, or the through slot 14 intersects the edge of the display screen 10. The notch 141 may be specifically formed on any one or more edges such as the upper edge, the lower edge, the left edge, and the right edge of the display screen 10. The shape of the notch 141 may be any shape such as a triangle, a semicircle, a rectangle, a track shape, etc., which is not limited herein.

Alternatively, as shown in FIG. 10, the through slot 14 includes a through hole 142 spaced from the edge of the display screen 10, or the through slot 14 is opened within a range enclosed by the edge of the display screen 10. The through hole 142 may specifically be close to any one or more edges such as the upper edge, the lower edge, the left edge, and the right edge of the display screen 10. The shape of the through hole 142 may be any shape such as a triangle, a circle, a rectangle, a racetrack, etc., and is not limited herein.

In addition, in the examples shown in FIGS. 8 to 10, the floodlight 23 and the structured light camera 22 may correspond to the same through slot 14. In the example shown in FIG. 11, the floodlight 23 and the structured light camera 22 can correspond to different through slots 14.

Please refer to FIG. 3, FIG. 5, FIG. 8 and FIG. 11. In some embodiments, the electronic device 1000 further includes a cover 40. The cover 40 is disposed on the side where the front surface 12 of the display screen 10 is located. When the display screen 10 is provided with a through slot 14, an infrared transmission layer 50 is provided on the area of the cover plate 40 corresponding to the through slot 14.

The cover plate 40 may be made of a material with good light transmission performance, such as glass or sapphire. The infrared transmission layer 50 may be an infrared transmission ink or an infrared transmission film. The infrared transmission layer 50 has a high transmittance for infrared light (for example, light with a wavelength of 940 nanometers), for example, the transmittance can reach 85% Or more, and the transmittance of light other than infrared light is low or makes light other than infrared light completely impermeable. Therefore, it is difficult for the user to see the structured light camera 22 or the floodlight 23 aligned with the through slot 14 through the cover plate 40, and the appearance of the electronic device 1000 is more beautiful.

Please refer to FIG. 1 again. In some embodiments, the display area 11 includes a first sub-display area 111 and a second sub-display area 112. The structured light emitted by the structured light projector 21 passes through the first sub-display area 111, and the pixel density of the first sub-display area 111 is smaller than the pixel density of the second sub-display area 112.

The pixel density of the first sub-display area 111 is less than the pixel density of the second sub-display area 112, that is, the micro-gap of the first sub-display area 111 is greater than the micro-gap of the second sub-display area 112. The blocking effect of light is small, and the transmittance of light passing through the first sub-display area 111 is high. Therefore, the structured light emitted by the structured light projector 21 has a higher transmittance through the first sub-display area 111.

In one example, the first sub-display area 111 may be used to display the status icon of the electronic device 1000, for example, to display the battery level, network connection status, system time, etc. of the electronic device 1000. The first sub-display area 111 may be located near the edge of the display area 11, and the second sub-display area 112 may be located in the middle of the display area 11.

Please refer to FIG. 1 again. In some embodiments, the display area 11 includes a first sub-display area 111 and a second sub-display area 112. The structured light emitted by the structured light projector 21 passes through the first sub-display area 111. The one sub-display area 111 and the second sub-display area 112 can be independently controlled and displayed in different display states. At this time, the pixel density of the first sub-display area 111 and the pixel density of the second sub-display area 112 may be equal, or the pixel density of the first sub-display area 111 is less than the pixel density of the second sub-display area 112.

Among them, different display states may be on or off, display with different brightness, display with different refresh frequency, and so on. The display states of the first sub-display area 111 and the second sub-display area 112 can be independently controlled. The user can control the normal display of the second sub-display area 112 according to actual needs, and the first sub-display area 111 cooperates with the structured light projector 21 use. For example, when the structured light projector 21 emits structured light, the first sub-display area 111 may be turned off, or the display brightness of the first sub-display area 111 may be lowered, or the refresh frequency of the first sub-display area 111 may be adjusted so that the first sub-display area 111 The turn-on time of the display area 111 is staggered from the turn-on time of the structured light projector 21, so as to reduce the influence of the first sub-display area 111 on the structured light projector 21 projecting the speckle pattern onto the scene. When the structured light projector 21 is not activated, both the first sub-display area 111 and the second sub-display area 112 may be turned on and displayed at the same refresh rate.

Please refer to FIG. 12. In some embodiments, the electronic device 1000 further includes a cover plate 40. The cover plate 40 is disposed on the side of the front surface 12 of the display screen 10. The area of the cover plate 40 corresponding to the structured light projector 21 The infrared antireflection film 60 is formed.

The infrared anti-reflection film 60 can increase the transmittance of infrared light. When the structured light projector 21 projects an infrared laser, the infrared anti-reflection film 60 can increase the transmittance of the infrared laser through the cover plate 40 to reduce the infrared laser penetration The loss of the cover 40 reduces the power consumption of the electronic device 1000. Specifically, the infrared antireflection film 60 may be plated on the upper surface, or the lower surface of the cover plate 40, or on both the upper surface and the lower surface.

Of course, the area on the cover plate 40 corresponding to the structured light camera 22 may also be formed with an infrared antireflection film 60 to reduce the loss of external infrared light passing through the cover plate 40 before reaching the structured light camera 22. The area corresponding to the floodlight 23 on the cover plate 40 may also be formed with an infrared antireflection film 60 to reduce the loss of supplementary light emitted by the floodlight 23 when passing through the cover plate 40. At this time, the area on the cover 40 that does not correspond to the structured light projector 21, the structured light camera 22, and the floodlight 23 may be formed with a visible light antireflection film 80 to improve the visible light emitted by the display screen 10 when passing through the cover 40 Transmittance.

Referring to FIG. 13, in some embodiments, an infrared anti-reflection film 60 is formed in the area of the display screen 10 corresponding to the structured light projector 21.

The infrared antireflection film 60 can increase the transmittance of infrared light. When the structured light projector 21 projects an infrared laser, the infrared antireflection film 60 can increase the transmittance of the infrared laser through the display screen 10 to reduce the infrared laser penetration The loss of the display screen 10 further reduces the power consumption of the electronic device 1000. Specifically, the infrared antireflection film 60 may be formed on the front surface 12 or the back surface 13 of the display area 11, or the front surface 12 or the back surface 13 of the display area 11 may be formed at the same time. In one example, the infrared antireflection film 60 may also be formed inside the display screen 10, for example, when the display screen 10 is a liquid crystal display, the infrared antireflection film 60 may be formed on the polarizer in the display screen 10, or formed on The electrode plate of the display screen 10 and the like.

Of course, when the through-slot 14 is not provided at the position corresponding to the display screen 10 and the structured light camera 22, the area of the display screen 10 and the structured light camera 22 may also be formed with an infrared antireflection film 60. When the through slot 14 is not provided at the position corresponding to the display screen 10 and the floodlight 23, the area corresponding to the display screen 10 and the floodlight 23 may also form an infrared antireflection film 60.

Referring to FIG. 14, in some embodiments, an infrared transmission layer 50 is formed in the area of the display screen 10 corresponding to the structured light projector 21. As described above, the infrared transmission layer 50 has a high transmittance for infrared light, and a low transmittance for light other than infrared light (such as visible light) or makes light other than infrared light (such as visible light) completely impermeable. However, it is difficult for the user to see the structured light projector 21.

At the same time, when the through-slot 14 is not provided at the position corresponding to the display screen 10 and the structured light camera 22, the area corresponding to the display screen 10 and the structured light camera 22 may also form an infrared transmission layer 50 to reduce infrared rays passing through the display screen 10 The effect of light other than light on the structured light camera 22. When the through slot 14 is not provided at the position corresponding to the display screen 10 and the floodlight 23, the infrared transmission layer 50 may also be formed in the area corresponding to the display screen 10 and the floodlight 23.

Please refer to FIG. 15. In some embodiments, the display screen 10 is formed with a through slot 14 penetrating the front surface 12 and the back surface 13. The electronic device 1000 further includes a visible light camera 70 that is aligned with the through slot 14. A visible light antireflection film 80 and / or an infrared cutoff film 90 are formed in the region of the cover plate 40 corresponding to the through groove 14.

The visible light camera 70 can be used to receive visible light passing through the cover plate 40 and the through slot 14 to obtain images. Forming a visible light antireflection film 80 on the area of the cover plate 40 corresponding to the through groove 14 can increase the transmittance of visible light when passing through the cover plate 40, so as to improve the imaging quality of the visible light camera 70. The formation of the infrared cut-off film 90 on the area of the cover plate 40 corresponding to the through slot 14 can reduce the transmittance of infrared light when passing through the cover plate 40, or completely prevent infrared light from entering the visible light camera 70 to reduce infrared light to the visible light camera 70 The impact of imaging.

Please refer to FIGS. 1 and 16. The present application also provides an image acquisition method, which can be used in the structured light assembly 20 described in any one of the above embodiments. The structured light assembly 20 is provided on the electronic device 1000. The structured light assembly 20 includes a structured light projector 21 and a structured light camera 22. The structured light projector 21 is disposed on the side of the back surface 13 of the display screen 10. The structured light projector 21 is used to emit structured light passing through the display area 11. Image acquisition methods include:

00: Control the structured light projector 21 to emit structured light toward the display area 11 of the display screen 10;

01: Control the structured light camera 22 to capture speckle images generated by structured light; and

02: Obtain a depth image based on the measurement spots in the speckle image and the reference spots in the reference image.

Please refer to FIGS. 1 and 17. The image acquisition method according to the embodiment of the present application may be implemented by the image acquiring device 400 according to the embodiment of the present application. The image acquisition device 400 includes a control module 401 and a calculation module 402. Step 00 and step 01 can be implemented by the control module 401. Step 02 may be implemented by the calculation module 402. That is to say, the control module 401 can be used to control the structured light projector 21 to emit structured light toward the display area 11 of the display screen 10, and to control the structured light camera 22 to capture speckle images generated by the structured light. The calculation module 402 may be used to obtain a depth image based on the measured spots in the speckle image and the reference spots in the reference image.

Please refer to FIG. 1 again. The image acquisition method according to the embodiment of the present application may be applied to the structured light assembly 20 described in any one of the above embodiments. The structured light assembly 20 further includes a processor 200, and steps 00, 01, and 02 can be implemented by the processor 200. That is to say, the processor 200 can be used to control the structured light projector 21 to emit structured light toward the display area 11 of the display screen 10, control the structured light camera 22 to capture the speckle image generated by the structured light, and according to the speckle image Obtain the depth image by measuring the spot in the reference spot and the reference spot in the reference image. The processor 200 of the structured light module 20 and the processor of the electronic device 1000 may be two independent processors; or, the processor 200 of the structured light module 20 and the processor of the electronic device 1000 may be the same processor. In a specific embodiment of the present application, the processor 200 of the structured light assembly 20 and the processor of the electronic device 1000 are the same processor 200.

Specifically, after the structured light projector 21 is turned on, it can project structured light into the scene, and the structured light projected into the scene will form a speckle pattern with multiple spots. Due to the different distances between multiple target objects in the scene and the structured light projector 21, the speckle pattern projected onto the target object will be modulated due to the difference in the height of the target object surface, and cause multiple of the speckle patterns The spots are shifted to different degrees. After the shifted spots are collected by the structured light camera 22, a speckle image including a plurality of measurement spots can be formed. After the processor 200 acquires the speckle image, it can calculate the depth data of multiple pixels according to the offset of the measured speckle in the speckle image relative to the reference speckle in the reference image, and multiple pixels with depth data can be formed A depth image. Among them, the reference image is obtained by calibration in advance.

The image acquisition method and the electronic device 1000 of the embodiment of the present application place the structured light projector 21 on the side of the back 13 of the display screen 10, that is, the structured light projector 21 is provided under the display screen 10, and the display screen 10 does not need to be opened and structured The through slot 14 aligned with the light projector 21 has a relatively high screen occupation of the electronic device 1000, and does not affect the acquisition of depth images.

Please refer to FIGS. 1, 5, 8 and 18. In some embodiments, the structured light projector 21 and the structured light camera 22 are arranged together on the side of the back 13 of the display screen 10, and the display screen 10 is provided with When the through slot 14 aligned with the light incident surface of the structured light camera 22 is received, the structured light camera 22 receives the modulated structured light passing through the through slot 14. At this time, controlling the structured light camera 22 to capture the speckle image generated by the structured light in step 01 includes:

011: Control the structured light camera 22 to receive the structured light directly diffracted by the display area 11 and reflected by the target object when it is emitted to obtain a speckle image. The speckle image includes a plurality of measurement spots, and the plurality of measurement spots including the laser light is only The first measuring spot diffracted by the diffractive optical element 213 (shown in FIG. 4) and reflected by the target object and the laser beam pass through the diffractive optical element 213 for a first diffraction and then diffracted again by the display screen 10 and reflected by the target object to form a second measuring spot ; Specifically, the first measurement spot is that after the laser beam is diffracted by the diffractive optical element 213 and passes through the display screen 10, it is not diffracted by the display screen 10, that is, it is directly projected onto the target object without encountering a microscopic gap, and is modulated and reflected by the target object The second measurement spot is formed by the laser light diffracted by the diffractive optical element 213 and then diffracted by the display screen 10 after passing through the display screen 10, that is, it is projected onto the target object after encountering a microscopic gap, and is modulated and reflected by the target object;

Step 02: Obtaining a depth image based on the measured spots in the speckle image and the reference spots in the reference image includes:

021: Acquire a depth image according to the first measurement spot and the second measurement spot in the speckle image and the reference spot in the reference image.

Please refer to FIG. 17 again. In some embodiments, step 011 may be implemented by the control module 401. Step 021 can be implemented by the calculation module 402.

Please refer to FIG. 1 again. In some embodiments, both step 011 and step 021 may be implemented by the processor 200. That is to say, the processor 200 can also be used to control the structured light camera 22 to receive the structured light that is diffracted by the display area 11 and reflected directly by the target object when being emitted to obtain a speckle image, and the speckle image includes multiple measurement spots The multiple measurement spots include the first measurement spot formed by laser light diffracted by the diffractive optical element 213 and reflected by the target object, and the first measurement spot formed by laser light diffracted by the diffractive optical element 213 and then diffracted by the display screen 10 second time and reflected by the target object Two measuring spots. The processor 200 may also be used to obtain a depth image according to the first measurement spot and the second measurement spot in the speckle image and the reference spot in the reference image.

Specifically, referring to FIG. 4, the structured light projector 21 generally includes a light source 211, a collimating element 212 and a diffractive optical element 213. Among them, the light source 211 is used to emit laser light; the collimating element 212 is used to collimate the laser light emitted by the light source 211; the diffractive optical element 213 is used to diffract the laser light collimated by the collimating element 212 to project structured light into the scene The structured light in the scene forms a speckle pattern, and the speckle pattern includes a plurality of spots, which are formed by the diffraction of the laser light only through the diffractive optical element 213.

LCD screens, OLED screens, Micro LED screens, and other types of display screens 10 usually have a fixed pixel arrangement structure on the display area 11, and microscopic gaps are formed between adjacent pixels. When a single-point laser passes through these microscopic gaps Will be diffracted to produce a series of spots. When the pixel arrangement structure in the display area 11 is different, the arrangement of the spots of the speckle pattern formed after the single-point laser passes through the display area 11 is also different. The structured light emitted by the structured light projector 21 is usually an infrared laser. In this way, when the structured light projector 21 is placed on the side of the back 13 of the display screen 10, that is, below the display screen 10, the infrared laser emitted by the structured light projector 21 passes through the display area 11 will also be The gap is diffracted to produce a speckle pattern with multiple spots. Thereby, the plurality of spots in the speckle pattern projected by the structured light projector 21 in the space simultaneously include the first spot formed by the laser light diffracted only by the diffractive optical element 213 and the laser light is diffracted once by the diffractive optical element 213 and then by the display screen 10 The second spot formed by the second diffraction.

When the structured light camera 22 is imaging, the structured light camera 22 receives the structured light reflected by the target object in the scene to form a speckle image. In the embodiment of the present application, since the display screen 10 is provided with a through slot 14, the light incident surface of the structured light camera 22 is aligned with the through slot 14, the through slot 14 does not have a micro gap, and is diffracted once by the diffractive optical element 213 The laser light reflected by the second-order diffraction of the display screen 10 and modulated by the target object will not be diffracted when passing through the slot 14. The structured light camera 22 receives the structured light diffracted by the display area 11 and reflected directly by the target object. The multiple measurement spots in the formed speckle image simultaneously include the first measurement spot formed by laser light diffracted only by the diffractive optical element 213 and reflected by the target object and the laser light is diffracted once by the diffractive optical element 213 and then diffracted by the display screen 10 twice The second measurement spot formed by the reflection of the target object.

After the speckle image is captured by the structured light camera 22, the processor 200 may directly calculate the depth image according to the first measurement spot and the second measurement spot in the speckle image and the reference spot in the reference image. The calculation method of the depth image may include the following two.

Referring to FIG. 19, in a calculation method, step 021 includes:

0211: Calculate the offset of all measurement spots relative to all reference spots; and

0212: Calculate depth data according to the offset to obtain a depth image.

Correspondingly, the image acquisition method also includes:

031: When calibrating the reference image, the structured light camera 22 is controlled to receive the structured light that is diffracted by the display area 11 and reflected directly by the calibrated object when it exits to obtain a reference image, and the reference image includes multiple reference spots.

Please refer to FIG. 17 again, both step 0211 and step 0212 can be implemented by the calculation module 402. Step 031 can be implemented by the control module 401.

Please refer to FIG. 1 again. Step 0211, step 0212, and step 031 may be implemented by the processor 200. That is to say, the processor 200 can also be used to calculate the offsets of all measurement spots relative to all reference spots and calculate depth data according to the offsets to obtain depth images. The processor 200 can also be used to control the structured light camera 22 to receive the structured light directly diffracted by the display area 11 and reflected by the calibrated object when it is calibrated to obtain the reference image. The reference image includes multiple reference spots.

Specifically, referring to FIG. 20, in the process of calibrating the reference image, the structured light projector 21 and the structured light camera 22 are both provided on the side of the back 13 of the display screen 10, and the display screen 10 is provided with a structured light camera 22 The structured light camera 22 can receive the modulated structured light passing through the through-slot 14 through the through-slot 14 where the light incident surface is aligned. In this way, in the calibration scene and the actual use scene, the installation positions of the structured light projector 21 and the structured light camera 22 relative to the display screen 10 are the same. In the calibration scenario, the processor 200 controls the structured light projector 21 to emit structured light. The structured light passes through the display area 11 and is projected onto a calibration object at a predetermined distance from the structured light assembly 20, such as the calibration plate, and is reflected back by the calibration plate The structured light passing through the through slot 14 is received by the structured light camera 22. At this time, the structured light camera 22 receives the structured light emitted by the structured light projector 21, diffracted by the display screen 10 and reflected by the calibration plate, and then directly incident through the through slot 14, the formed reference image includes multiple reference spots . Wherein, the reference spot simultaneously includes a first reference spot corresponding to the first measurement spot and a second reference spot corresponding to the second measurement spot. The first reference spot is formed when the laser light is diffracted by the diffractive optical element 213 when passing through the diffractive optical element 213, and is not diffracted by the display screen 10 after passing through the display screen 10, and is modulated and reflected by the calibration plate; the second reference spot is formed by the laser light The diffractive optical element 213 is diffracted once by the diffractive optical element 213, and when passing through the display screen 10, it is diffracted again by the display screen 10, and is modulated and reflected by the calibration plate. Although the speckle image includes both the first measurement spot and the second measurement spot, and the reference image includes both the first reference spot and the second reference spot, in this calculation method, the processor 200 does not The first measurement spot and the second measurement spot are distinguished, and the first reference spot and the second reference spot in the reference image are not distinguished, but the depth image is directly based on all the measurement spots and all the reference spots. Calculation. Specifically, the processor 200 first calculates the offsets of all measurement spots relative to all reference spots, and then calculates multiple depth data based on the multiple offsets, thereby obtaining a depth image.

Referring to FIG. 21, in another calculation method, step 021 includes:

0213: Calculate the offset of the first measurement spot relative to the first reference spot, and the offset of the second measurement spot relative to the second reference spot; and

0214: Calculate depth data according to the offset to obtain a depth image.

At this time, the image acquisition method also includes:

032: When calibrating the reference image, the structured light camera 22 is controlled to receive the structured light directly reflected by the calibrated object after being emitted from the structured light projector 21 and directly incident to obtain a first reference image, and the first reference image includes multiple reference spots The multiple reference spots include a first reference spot formed by multiple laser beams diffracted by the diffractive optical element 213 only and reflected by the calibration object;

033: When calibrating the reference image, the structured light camera 22 is controlled to receive the structured light diffracted by the display area 11 and reflected directly by the calibrated object when being emitted to obtain a second reference image, which includes a plurality of reference spots, The multiple reference spots include a first reference spot formed by laser light diffracted only by the diffractive optical element 213 and reflected by the calibrated object, and a second reference spot formed by the laser light diffracted once by the diffractive optical element 213 and then diffracted by the display screen 10 and reflected by the calibrated object Reference spot

041: Compare the first reference image with the second reference image to obtain a second reference spot;

051: Calculate the ratio between the average value of the brightness of the multiple second reference spots and the average value of the brightness of the multiple first reference spots as the preset ratio, and calculate the average value of the brightness of the multiple first reference spots and As the preset brightness;

061: Calculate the actual ratio between each measured spot and the preset brightness; and

071: The measurement spot whose actual ratio is greater than the preset ratio is classified as the first measurement spot, and the measurement spot whose actual ratio is less than the first preset ratio is classified as the second measurement spot.

Please refer to FIG. 17 again. Step 0213, step 0214, step 041, step 051, step 061, and step 071 can all be implemented by the calculation module 402. Both step 032 and step 033 can be implemented by the control module 401.

Please refer to FIG. 1 again. Step 0213, step 0214, step 032, step 033, step 041, step 051, step 061, and step 071 may all be implemented by the processor 200. That is to say, the processor 200 can also be used to control the structured light camera 22 to receive the structured light directly reflected by the calibrated object and directly incident after being emitted from the structured light projector 21 when the reference image is calibrated to obtain the first reference image, When the reference image is calibrated, the structured light camera 22 is controlled to receive the structured light that is diffracted by the display area 11 and reflected directly by the calibrated object when being emitted to obtain a second reference image, compare the first reference image with the second reference image to obtain the second Two reference spots, calculating the ratio between the average value of the brightness of the plurality of second reference spots and the average value of the brightness of the plurality of first reference spots as a preset ratio, and calculating the average of the brightness of the plurality of first reference spots Value as the preset brightness. The processor 200 can also be used to calculate the actual ratio between each measurement spot and the preset brightness, classify the measurement spots whose actual ratio is greater than the preset ratio as the first measurement spots, and classify the measurement spots whose actual ratio is less than the preset ratio It is classified as the second measurement spot. The processor 200 may be further used to calculate the offset of the first measurement spot relative to the first reference spot, and the offset of the second measurement spot relative to the second reference spot, and calculate depth data according to the offset to obtain a depth image .

In this calculation mode, the processor 200 needs to calibrate the first reference image and the second reference image. Specifically, the processor 200 first controls the structured light projector 21 to emit structured light to the calibration plate in a scene without the screen 10 blocking, and then controls the structured light camera 22 to receive the structured light directly reflected by the calibration plate to obtain the first A reference image, where the multiple reference spots included in the first reference image are the first reference spots, the first reference spots are laser beams diffracted by the diffractive optical element 213 when passing through the diffractive optical element 213, and directly exit the calibration plate It is formed by direct reflection after being modulated and reflected by the calibration plate. Subsequently, the processor 200 calibrates the second reference image according to the first calculation method, that is, the reference image calibration method in step 031. At this time, the second reference image includes both the first reference spot corresponding to the first measurement spot and the second reference spot corresponding to the second measurement spot. Among them, in the calibration scene of the first reference image and the second reference image, the relative positions between the calibration plate, the structured light projector 21 and the structured light camera 22 remain unchanged, and the structured light projector 21 and the structured light camera 21 The relative position between them also remains unchanged. Subsequently, the processor 200 marks the coordinates of the first reference spot in the first reference image, and filters out the first reference spots in the second reference image according to the coordinates of the first reference spot, and the remaining reference spots in the second reference image This is the second reference spot. In this way, the processor 200 can distinguish the first reference spot and the second reference spot among all reference spots of the second reference image.

As the depth data is calculated later, the measurement spots in the speckle image also need to be distinguished. Specifically, the first measurement spot and the second measurement spot can be distinguished by brightness. It can be understood that the first measurement spot is formed by the first-order diffraction of laser light through the diffractive optical element 213, and the second measurement spot is formed by the first-order diffraction of the laser light through the diffractive optical element 213 and the second-order diffraction of the display screen 10 to form the second measurement The number of times the laser spot is diffracted is greater than the number of times the laser spot forming the first measurement spot is diffracted. Therefore, the energy loss of the laser spot forming the first measurement spot is small, and the energy loss of the laser spot forming the second measurement spot is large. The brightness of the second measurement spot will be lower than the brightness of the first measurement spot. As such, it is feasible to distinguish the first measurement spot from the second measurement spot based on the brightness. Then, after the reference image calibration is completed, it is necessary to further calibrate the preset brightness and the preset ratio for distinguishing the first measurement spot from the second measurement spot. Specifically, after the processor 200 distinguishes the first reference spot and the second reference spot, the processor 200 calculates the average value of the brightness of the plurality of first reference spots in the second reference image, and calculates the second reference image The average value of the brightness of the plurality of second reference spots. Subsequently, the processor 200 takes the average value of the brightness of the plurality of first reference spots as the preset brightness, and takes the ratio between the average value of the brightness of the plurality of second reference spots and the average value of the brightness of the plurality of first reference spots As a preset ratio.

In the subsequent depth data calculation, the processor 200 first calculates the brightness of each measurement spot. Subsequently, the processor 200 calculates the actual ratio between each measurement spot and the preset brightness, and classifies the measurement spots whose actual ratio is greater than or equal to the preset ratio as the first measurement spot, and measures the actual ratio less than the preset ratio The spots are classified as second measurement spots, thereby distinguishing the first measurement spots from the second measurement spots. For example, as shown in FIG. 22, assuming that the preset ratio is 0.8, the speckle image captured by the structured light camera 22 in actual use includes measurement spot A and measurement spot B. Among them, if the ratio between the brightness of the measurement spot A and the preset brightness is less than 0.8, the measurement spot A is classified into the second measurement spot. At this time, the measurement spot A is diffracted by the laser through the diffraction optical element 213 and then displayed The measurement spot formed by the second diffraction of the screen 10 and reflected by the target object; the ratio between the brightness of the measurement spot B and the preset brightness is greater than or equal to 0.8, then the measurement spot B is classified into the first measurement spot, which is explained at this time The measurement spot B is a measurement spot formed by laser light diffracted once by the diffractive optical element 213 and reflected by the target object. Among them, the preset ratio of 0.8 is only an example.

After the processor 200 distinguishes the first measurement spot and the second measurement spot, since the first reference spot and the second reference spot in the second reference image have also been distinguished, the processor 200 can use the speckle image and the second measurement spot. Two reference images calculate depth data. Specifically, the processor 200 first calculates the offset of the first measurement spot relative to the first reference spot, and the offset of the second measurement spot relative to the second reference spot. Subsequently, the processor 200 calculates multiple depth data based on multiple offsets, and the multiple depth data can constitute a depth image.

Compared with the first calculation method, the second calculation method distinguishes between the first measurement spot and the second measurement spot, and distinguishes between the first reference spot and the second reference spot, which can be based on the more accurate first measurement spot The corresponding relationship with the first reference spot and the corresponding relationship between the second measurement spot and the second reference spot are calculated to obtain a more accurate offset, further obtain more accurate depth data, and improve the accuracy of the acquired depth image.

In some embodiments, the preset brightness and the preset ratio are determined by the ambient brightness of the scene and the luminous power of the structured light projector 21. It can be understood that there is an infrared light component in the ambient light, and this part of the infrared light component may be superimposed on the measurement spot to increase the brightness of the measurement spot; the luminous power of the structured light projector 21 is closely related to the brightness of the measurement spot. When it is larger, the brightness of the measurement spot is correspondingly higher; when the luminous power is smaller, the brightness of the measurement spot is correspondingly lower. Therefore, different ambient brightness and luminous power should have different preset brightness and preset ratio. The preset brightness and preset ratio under different ambient brightness and different luminous power can also be calibrated according to the calibration process of step 032 and step 033. During the calibration process, the ambient brightness of the calibration scene and the luminous power of the structured light projector 21 are changed to obtain a preset brightness and a preset ratio corresponding to the ambient brightness and luminous power, wherein the light emission of the structured light projector 21 is changed The power can be realized by changing the driving current of the light source 211. The corresponding relationship among the ambient brightness, the luminous power, the preset brightness and the preset ratio can be stored in the memory 300 (shown in FIG. 1) in the form of a mapping table. In the subsequent calculation of the depth image in the second calculation method, the processor 200 first obtains the ambient brightness and luminous power of the scene, and looks up the preset brightness and preset ratio corresponding to the current ambient brightness and luminous power in the mapping table, Then, the first measurement spot and the second measurement spot are distinguished based on the searched preset brightness and preset ratio. In this way, the accuracy of the distinction between the first measurement spot and the second measurement spot can be improved.

In some embodiments, the diffractive optical element 213 can be used to compensate the structured light diffracted by the display screen 10 in addition to diffracting the laser light emitted by the light source 211 of the structured light projector 21 to increase the number of measurement spots or reference spots. The uniformity of the brightness makes the uniformity of the brightness of the multiple spots in the speckle pattern projected into the scene better, which is beneficial to improve the accuracy of acquiring the depth image. Specifically, the convex or concave structures in the diffractive optical element 213 may be arranged densely in the middle and sparse on both sides, then the diffraction effect of the middle part of the diffractive optical element 213 is stronger than that of the edge part. In this way, the laser light incident on the middle portion of the diffractive optical element 213 can be diffracted with more light beams, and the laser light incident on the edge portion of the diffractive optical element 213 can be diffracted with less light beams, thereby making the speckle projected into the scene The brightness of the pattern has high uniformity.

In the prior art, the staggered setting of the display screen and the depth camera results in a relatively low screen occupation of the mobile terminal. In summary, in the image acquisition method of the embodiment of the present application, both the structured light projector 21 and the structured light camera 22 are located on the display screen 10 When the structured light camera 22 receives the modulated structured light passing through the through slot 14, the processor 200 can directly calculate the depth image based on the first measurement spot and the second measurement spot, compared to By using only the first measurement spot to calculate the depth image, the diffraction effect of the display screen 10 increases the number of measurement spots and the randomness of the arrangement of the measurement spots, which is beneficial to improve the accuracy of acquiring the depth image. Further, the image acquisition method according to the embodiment of the present application can appropriately simplify the complexity of the structure of the diffraction grating in the diffractive optical element 213, and in turn, the number of measurement spots and the randomness of the arrangement are increased by the diffraction effect of the display screen 10, While ensuring the accuracy of acquiring the depth image, the manufacturing process of the structured light projector 21 can be simplified.

Please refer to FIG. 1, FIG. 5, FIG. 8 and FIG. 23. In some embodiments, the structured light projector 21 and the structured light camera 22 are arranged together on the side of the back 13 of the display screen 10, and the display screen 10 is opened When there is a through slot 14 aligned with the light incident surface of the structured light camera 22, the structured light camera 22 receives the modulated structured light passing through the through slot 14. At this time, step 01 includes:

011: Control the structured light camera 22 to receive the structured light directly diffracted by the display area 11 and reflected by the target object when it is emitted to obtain a speckle image. The speckle image includes a plurality of measurement spots, and the plurality of measurement spots including the laser light is only The first measurement spot diffracted by the diffractive optical element 213 and reflected by the target object and the second measurement spot formed by the laser beam diffracted once by the diffractive optical element 213 and secondarily diffracted by the display screen 10 and reflected by the target object; specifically, the first The measurement spot is formed after the laser beam is diffracted by the diffractive optical element 213 and is not diffracted by the display screen 10 when passing through the display screen 10, that is, it is directly projected onto the target object without encountering a micro gap, and is modulated and reflected by the target object; second The measurement spot is formed by the laser beam diffracted by the diffractive optical element 213 and then diffracted by the display screen 10 after passing through the display screen 10, that is, it is projected onto the target object after encountering a microscopic gap, and is modulated and reflected by the target object;

Step 02 includes:

022: Filter out the second measurement spot in the speckle image to obtain the first measurement spot;

023: Acquire a depth image according to the first measurement spot and the reference spot in the reference image.

Please refer to FIG. 17 again. Step 011 may be implemented by the control module 401. Both step 022 and step 023 can be implemented by the calculation module 402.

Please refer to FIG. 1 again. Step 011, step 022, and step 023 may all be implemented by the processor 200. That is to say, the processor 200 can also be used to control the structured light camera 22 to receive the structured light diffracted by the display area 11 and reflected directly by the target object when it is emitted to obtain a speckle image and filter out the second of the speckle images The speckle is measured to obtain a first measurement spot, and a depth image is obtained according to the first measurement spot and the reference spot in the reference image.

Specifically, when the structured light projector 21 and the structured light camera 22 are provided together on the side of the back surface 13 of the display screen 10, and the display screen 10 is provided with a through slot 14 aligned with the light incident surface of the structured light camera 22, the structure The optical camera 22 captures a speckle image containing the first measurement spot and the second measurement spot. In the calculation of the subsequent depth image, the processor 200 may filter out the second measurement spots in the speckle image, and only calculate the depth image based on the remaining first measurement spots with the reference spots in the reference image. At this time, the reference spot in the reference image should include only the first reference spot formed by a plurality of laser beams diffracted only by the diffractive optical element 213 and reflected by the calibration object. Therefore, by filtering out the second measurement speckle in the speckle image, the influence of the display screen 10 on the structured light can be eliminated, so that the depth image acquired by the electronic device 1000 can be ensured when the screen footprint of the electronic device 1000 is relatively high. The accuracy is also higher.

That is to say, please refer to Figure 24, the image acquisition method also includes:

Step 023 includes:

0231: Calculate the offset of the first measurement spot relative to the first reference spot; and

0232: Calculate depth data according to the offset to obtain a depth image.

Please refer to FIG. 17 again. Step 032 may be implemented by the control module 401. Both Step 0231 and Step 0232 can be implemented by the calculation module 402.

Please refer to FIG. 1 again. Step 032, step 0231, and step 0232 may all be implemented by the processor 200. That is to say, the processor 200 can also be used to control the structured light camera 22 to receive the structured light directly reflected by the calibrated object after being emitted from the structured light projector 21 and directly incident when the reference image is calibrated to obtain the first reference image and calculate the first A measurement spot is offset from the first reference spot, and depth data is calculated according to the offset to obtain a depth image.

Specifically, after the processor 200 filters out the second measurement spots, only the first measurement spots remain in the speckle image, then the speckle image should be the first reference image containing only the first reference spots corresponding to the first measurement spots To calculate the depth image. The calibration process of the first reference image is the same as the calibration process of placing the structured light projector 21 in a scene that is not covered by the display screen 10 in step 032, and will not be repeated here. The multiple reference spots in the first reference image captured by structured light are the first reference spots formed by laser light diffracted only by the diffractive optical element 213 and reflected by the calibration object. In this way, the processor 200 can calculate the offset of the first measurement spot relative to the first reference spot, and then calculate multiple depth data based on the multiple offsets to obtain a depth image.

The processor 200 may filter out the second measurement spot by brightness. That is to say, referring to FIG. 25, in some embodiments, the image acquisition method further includes:

032: When calibrating the reference image, the structured light camera 22 is controlled to receive the structured light directly reflected by the calibrated object and directly incident after exiting from the structured light projector 21 to obtain a first reference image, and the first reference image includes multiple Reference spots, the multiple reference spots include a first reference spot formed by a plurality of laser beams diffracted only by the diffractive optical element 213 and reflected by the calibration object;

033: When calibrating the reference image, the structured light camera 22 is controlled to receive the structured light diffracted by the display area 11 and reflected directly by the calibrated object when being emitted to obtain a second reference image, which includes a plurality of reference spots, The multiple reference spots include a first reference spot formed by laser light diffracted by the diffractive optical element 213 and reflected by the calibration object and a laser beam diffracted once by the diffractive optical element 213 and then diffracted by the display screen 10 twice and reflected by the calibration object Two reference spots;

041: Compare the first reference image with the second reference image to obtain a second reference spot; and

051: Calculate the ratio between the average value of the brightness of the plurality of second reference spots and the average value of the brightness of the plurality of first reference spots as the preset ratio, and calculate the average value of the brightness of the plurality of first reference spots as Preset brightness

Step 022 includes:

0221: Calculate the actual ratio between each measured spot and the preset brightness;

0222: classify the measurement spot whose actual ratio is greater than the preset ratio as the first measurement spot, and classify the measurement spot whose actual ratio is less than the preset ratio as the second measurement spot; and

0223: Filter out the second measurement spot from all the measurement spots to obtain the first measurement spot.

Please refer to FIG. 17 again. Step 032 and step 033 can be implemented by the control module 401. Step 041, step 051, step 0221, step 0222, and step 0223 can all be implemented by the calculation module 401.

Please refer to FIG. 1 again. Step 032, step 033, step 041, step 051, step 0221, step 0222 and step 0223 can all be implemented by the processor 200. That is to say, the processor 200 can also be used to control the structured light camera 22 to receive the structured light directly reflected by the calibrated object after being emitted from the structured light projector 21 and directly incident when the reference image is calibrated to obtain the first reference image, and When the reference image is calibrated, the structured light camera 22 is controlled to receive the structured light directly diffracted by the display area 11 and reflected by the calibration object when it is emitted to obtain a second reference image. The processor 200 may also be used to compare the first reference image and the second reference image to obtain the second reference spot, and calculate the average value of the brightness of the plurality of second reference spots and the average value of the brightness of the plurality of first reference spots The ratio is used as the preset ratio, and the average value of the brightness of the plurality of first reference spots is calculated and used as the preset brightness. The processor 200 can also be used to calculate the actual ratio between each measurement spot and the preset brightness, classify the measurement spots whose actual ratio is greater than the preset ratio as the first measurement spots, and classify the measurement spots whose actual ratio is less than the preset ratio It is classified as a second measurement spot, and the second measurement spot is filtered out from all the measurement spots to obtain the first measurement spot.

Among them, the process of calibrating the first reference image described in step 032 is the same as the calibration process of calibrating the structured light projector 21 in the scene that is not blocked by the display screen 10 in step 032, and the second calibration described in step 033 The process of the reference image and the aforementioned step 031 place the structured light projector 21 and the structured light camera 22 on the side of the back side 13 of the display screen 10, and the light incident surface of the structured light camera 22 is aligned with the through slot 14 of the display screen 10 The calibration process for calibration in the same scenario is the same and will not be repeated here.

After obtaining the first reference image and the second reference image, the processor 200 may use the same method as the foregoing step 041, that is, determine the first in the second reference image according to the coordinates of the first reference spot in the first reference image The reference spot, and the remaining reference spot is the second reference spot, thereby distinguishing the first reference spot and the second reference spot. Subsequently, the processor 200 can calculate the preset brightness and the preset ratio based on the distinguished first reference spot and second reference spot calibration in the same manner as the foregoing step 051.

Similarly, in the calculation of the subsequent depth image, the processor 200 may adopt the same manner as the aforementioned step 061 and the aforementioned step 071, that is, distinguish the first measurement spot from the second based on the calibrated preset ratio and preset brightness Measure the spots, then filter out the second measurement spots, leaving only the first measurement spots, then calculate the offset of the first measurement spots relative to the first reference spots, and finally calculate the depth data based on the offsets, thereby obtaining the depth image .

In some embodiments, the preset brightness and the preset ratio are also determined by the ambient brightness of the scene and the luminous power of the structured light projector 21. In this way, the accuracy of filtering out the second measurement spot can be improved.

In some embodiments, the diffractive optical element 213 can be used to compensate the structured light diffracted by the display screen 10 in addition to diffracting the laser light emitted by the light source 211 of the structured light projector 21 to increase the number of measurement spots or reference spots. The uniformity of the brightness makes the uniformity of the brightness of the multiple spots in the speckle pattern projected into the scene better, which is beneficial to improve the accuracy of acquiring the depth image.

In summary, in the image acquisition method of the embodiment of the present application, when both the structured light projector 21 and the structured light are located under the display screen 10, and the structured light camera 22 receives the modulated structured light passing through the through slot 14, it is filtered first The second measurement spot calculates the depth image based only on the remaining first measurement spot, which reduces the amount of data processing by the processor 200 and helps speed up the process of acquiring the depth image.

Please refer to FIG. 1, FIG. 3 and FIG. 26. In some embodiments, the structured light projector 21 and the structured light camera 22 are both disposed on the side of the back surface 13 of the display screen 10, and the display screen 10 has no through slot 14 The structured light camera 22 receives modulated structured light that passes through the display area 11 twice. At this time, step 01 includes:

012: Control the structured light camera 22 to receive the structured light diffracted by the display area 11 and reflected by the target object when it is emitted and then diffracted by the display area 11 to obtain a speckle image. The speckle image includes multiple measurement spots, multiple The measurement spot includes a laser beam that is diffracted only once by the diffractive optical element 213 (shown in FIG. 4) and reflected by the target object. The laser beam passes through the diffractive optical element 213 and is diffracted once by the display screen 10 and diffracted again by the target object The second measurement spot formed by reflection and the third measurement spot formed by the laser beam diffracted once by the diffractive optical element 213 and then diffracted again by the display screen 10 and reflected by the target object and then diffracted by the display screen 10 three times; specifically, the A measurement spot is formed after the laser beam is diffracted by the diffractive optical element 213 and is not diffracted by the display screen 10 when passing through the display screen 10, that is, it is directly projected onto the target object without encountering a micro gap, and is modulated and reflected by the target object; The second measurement spot is that the laser beam is diffracted by the display screen 10 after being diffracted by the diffractive optical element 213, and then diffracted by the display screen 10 After being projected onto the target object, it is not diffracted by the display screen 10 when it is modulated and reflected by the target object and passes through the display screen 10 again; the third measurement spot is that the laser light is diffracted by the diffractive optical element 213 after passing through the display screen 10 and then diffracted by the display screen 10 , That is, it is projected onto the target object after encountering the micro gap, and is modulated and reflected by the target object again after passing through the display screen 10 and diffracted by the micro gap in the display screen 10 again;

Step 02 includes:

024: Calculate the depth image based on the first measurement spot, the second measurement spot, and the third measurement spot in the speckle image and the reference spot in the reference image.

Please refer to FIG. 17 again. Step 012 may be implemented by the control module 401. Step 024 can be implemented by the calculation module 402.

Please refer to FIG. 1 again. Both step 012 and step 024 may be implemented by the processor 200. That is to say, the processor 200 can also be used to control the structured light camera 22 to receive the structured light diffracted by the display area 11 when it is emitted and reflected by the target object and then diffracted by the display area 11 after being incident to obtain a speckle image. The processor 200 may also be used to calculate a depth image based on the first measurement spot, the second measurement spot, and the third measurement spot in the speckle image and the reference spot in the reference image.

Specifically, referring to FIG. 4, the light source 211 of the structured light projector 21 emits laser light to be diffracted by the diffractive optical element 213 to form structured light and project it into the scene to form a speckle pattern. The speckle pattern includes a plurality of spots, which are formed by the diffraction of laser light only through the diffractive optical element 213.

When the structured light camera 22 is imaging, the structured light camera 22 receives the structured light reflected by the target object in the scene to form a speckle image. In the embodiment of the present application, since the display screen 10 does not have a through slot 14, the first diffraction by the diffractive optical element 213 and the second diffraction by the display screen 10, and the laser light reflected back after being modulated by the target object will pass through the display screen 10 again Diffracted by the display area 11 in the display screen 10, the structured light camera 22 receives the diffraction by the display area 11 when passing through the display area 11 after being emitted, and is diffracted by the display area 11 again after passing through the display area 11 after being reflected by the target object Structured light, the multiple measurement spots in the formed speckle image also include the first measurement spot formed by laser light diffracted only by the diffractive optical element 213 and reflected by the target object. The second measurement spot formed by the second diffraction and reflected by the target object, and the third measurement formed by the laser once diffracted by the diffractive optical element 213 and then diffracted again by the display screen 10 and reflected by the target object by the third diffraction of the display screen 10 spot.

After the structured light camera 22 captures the speckle image, the processor 200 can directly calculate the depth image from the first measurement spot, the second measurement spot, the third measurement spot, and the reference image in the speckle image. At this time, the multiple reference spots in the reference image need to include a first reference spot, a second reference spot, and a third reference spot. The calculation method of the depth image may include the following two.

Referring to FIG. 27, in a calculation method, step 024 includes:

0241: Calculate the offset of all measurement spots relative to all reference spots; and

0242: Calculate depth data according to the offset to obtain a depth image.

Correspondingly, the image acquisition method also includes:

034: When the reference image is calibrated, the structured light camera 22 is controlled to receive the structured light diffracted by the display area 11 when it is emitted and diffracted by the display area 22 after being reflected by the calibrated object to obtain the reference image. spot.

Please refer to FIG. 17 again. Step 0241 and step 0242 can be implemented by the calculation module 402. Step 034 can be implemented by the control module 401.

Please refer to FIG. 1 again. Step 0241, step 0242, and step 034 may be implemented by the processor 200. That is to say, the processor 200 can also be used to calculate the offsets of all measurement spots relative to all reference spots, and calculate depth data according to the offsets to obtain depth images. The processor 200 can also be used to control the structured light camera 22 to receive the structured light diffracted by the display area 11 when diffracted by the display area 11 and reflected by the calibrated object and then diffracted by the display area 22 when it is incident to obtain the reference image. Includes multiple reference spots.

Specifically, referring to FIG. 28, in the process of calibrating the reference image, the structured light projector 21 and the structured light camera 22 are both disposed on the side of the back surface 13 of the display screen 10, and the display screen 10 is not provided with a through slot 14. In this way, in the calibration scene and the actual use scene, the installation positions of the structured light projector 21 and the structured light camera 22 relative to the display screen 10 are the same. In the calibration scenario, the processor 200 controls the structured light projector 21 to emit structured light. The structured light passes through the display area 11 and is projected to the calibration plate at a predetermined distance from the structured light assembly 20. The structured light reflected back by the calibration plate again After passing through the display area 11, it is received by the structured light camera 22. At this time, the structured light camera 22 receives the structured light emitted by the structured light projector 21, diffracted by the display screen 10, reflected by the calibration plate, and then diffracted through the display screen 10. The formed reference image includes many Reference spots. Wherein, the reference spots simultaneously include a first reference spot corresponding to the first measurement spot, a second reference spot corresponding to the second measurement spot, and a third reference spot corresponding to the third measurement spot. The first reference spot is that the laser light is diffracted by the diffractive optical element 213 once when passing through the diffractive optical element 213, and is not diffracted by the display screen 10 after passing through the display screen 10, and is reflected by the calibration plate and still passes through the display screen 10 when it passes through the display screen 10 again. 10 formed by diffraction. The second reference spot is that the laser light is diffracted by the diffractive optical element 213 once when passing through the diffractive optical element 213, and is secondarily diffracted by the display screen 10 after passing through the display screen 10, and is reflected by the calibration plate, but not by the display screen after passing through the display screen 10 again 10 formed by diffraction. The third reference spot is that the laser light is first diffracted by the diffractive optical element 213 when passing through the diffractive optical element 213, and is secondarily diffracted by the display screen 10 after passing through the display screen 10, and reflected by the calibration plate, and then by the display screen 10 after passing through the display screen 10 again Formed by the third diffraction.

Although the speckle image includes the first measurement spot, the second measurement spot, and the third measurement spot, the reference image includes the first reference spot, the second reference spot, and the third reference spot at the same time. However, in this calculation method, the processor 200 does not distinguish between the first measurement spot, the second measurement spot, and the third measurement spot in the speckle image, nor does it distinguish the first reference spot, the second measurement spot in the reference image. The second reference spot and the third reference spot are distinguished, but the depth image is calculated directly based on all the measurement spots and all the reference spots. Specifically, the processor 200 first calculates the offsets of all measurement spots relative to all reference spots, and then calculates multiple depth data based on the multiple offsets, thereby obtaining a depth image.

Referring to FIG. 29, in another calculation method, step 024 includes:

0243: Calculate the offset of the first measurement spot relative to the first reference spot, the offset of the second measurement spot relative to the second reference spot, and the offset of the third measurement spot relative to the third reference spot; and

0244: Calculate depth data according to the offset to obtain a depth image.

At this time, the image acquisition method also includes:

035: When calibrating the reference image, the structured light camera 22 is controlled to receive the structured light directly reflected by the calibrated object and directly incident after exiting from the structured light projector 21 to obtain a first reference image, and the first reference image includes multiple reference spots The multiple reference spots include a first reference spot formed by multiple laser beams diffracted by the diffractive optical element 213 only and reflected by the calibration object;

036: When the reference image is calibrated, the structured light camera 22 is controlled to receive the structured light that is diffracted by the display area 11 and reflected directly by the calibrated object when being emitted to obtain a second reference image, and the second reference image includes multiple reference spots, The multiple reference spots include a first reference spot formed by laser light diffracted only by the diffractive optical element 213 and reflected by the calibrated object, and a second reference spot formed by the laser light diffracted once by the diffractive optical element 213 and then diffracted by the display screen 10 twice and reflected by the calibrated object Reference spot

037: When the reference image is calibrated, the structured light camera 22 is controlled to receive the structured light diffracted by the display area 11 and diffracted by the display area 11 when it is diffracted by the display area 11 and reflected by the calibration object to obtain the third reference image. The third reference image includes multiple reference spots. The multiple reference spots include a first reference spot formed by laser light diffracted only by the diffractive optical element 213 and reflected by the calibration object. The laser light is diffracted by the diffractive optical element 213 once and then diffracted by the display screen 10 The second reference spot diffracted and reflected by the calibration object, and the laser beam is diffracted once by the diffractive optical element 213 and then diffracted again by the display panel 10 and reflected by the calibration object. spot;

042: Compare the first reference image with the second reference image to obtain the second reference spot, and compare the third reference image with the second reference image to obtain the third reference spot;

052: Calculate the ratio between the average value of the brightness of the multiple second reference spots and the average value of the brightness of the multiple first reference spots as the first preset ratio, and calculate the average value of the brightness of the multiple third reference spots and The ratio between the average values of the brightness of the plurality of first reference spots is used as the second preset ratio, and the average value of the brightness of the plurality of first reference spots is calculated and used as the preset brightness;

062: Calculate the actual ratio between each measured spot and the preset brightness; and

072: Classify the measurement spot whose actual ratio is greater than the first preset ratio as the first measurement spot, classify the measurement spot whose actual ratio is less than the first preset ratio and greater than the second preset ratio as the second measurement spot, and classify The measurement spots whose actual ratio is less than the second preset ratio are classified as the third measurement spots.

Please refer to FIG. 17 again. Step 0243, step 0244, step 042, step 052, step 062, and step 072 may all be implemented by the calculation module 402. Step 035, step 036, and step 037 can all be implemented by the control module 401.

Please refer to FIG. 1 again. Step 0243, step 0244, step 035, step 036, step 037, step 042, step 052, step 062, and step 072 may all be implemented by the processor 200. That is to say, the processor 200 can also be used to control the structured light camera 22 to receive the structured light that is directly reflected by the calibrated object and directly incident after being emitted from the structured light projector 21 when the reference image is calibrated to obtain the first reference image. When the reference image is referenced, the structured light camera 22 is controlled to receive the structured light that is diffracted by the display area 11 and reflected directly by the calibrated object when being emitted to obtain a second reference image, and when the reference image is calibrated, the structured light camera 22 is controlled to receive and display the emitted light After the area 11 is diffracted and reflected by the calibration object, the structured light is diffracted by the display area 11 again when incident through the display area 11 to obtain a third reference image. The processor 200 may also be used to compare the first reference image and the second reference image to obtain a second reference spot, and to compare the third reference image and the second reference image to obtain a third reference spot, and calculate a plurality of second reference spots The ratio between the average value of the brightness of the plurality of first reference spots and the average value of the brightness of the plurality of first reference spots as the first preset ratio, calculating the average value of the brightness of the plurality of third reference spots and the brightness of the plurality of first reference spots The ratio between the average values is used as the second preset ratio, and the average value of the brightness of the plurality of first reference spots is calculated and used as the preset brightness. The processor 200 can also be used to calculate the actual ratio between each measured spot and the preset brightness, classify the measured spots whose actual ratio is greater than the first preset ratio as the first measured spot, and classify the actual ratio as being smaller than the first preset ratio The measurement spots larger than the second preset ratio are classified as second measurement spots, and the measurement spots whose actual ratio is smaller than the second preset ratio are classified as third measurement spots.

In this calculation mode, the processor 200 needs to calibrate the first reference image, the second reference image, and the third reference image.

Specifically, the processor 200 first controls the structured light projector 21 to emit structured light to the calibration plate in a scene without the screen 10 blocking, and then controls the structured light camera 22 to receive the structured light directly reflected by the calibration plate to obtain the first One reference image. Among them, the multiple reference spots included in the first reference image are the first reference spots. The first reference spots are laser beams diffracted by the diffractive optical element 213 when passing through the diffractive optical element 213, and are directly emitted to the calibration plate and modulated by the calibration plate It is formed by direct incidence after reflection.

Subsequently, both the structured light projector 21 and the structured light camera 22 are provided on the side of the back surface 13 of the display screen 10, and the display screen 10 is provided with a through slot 14 aligned with the light incident surface of the structured light camera 22, structured light When the camera 22 can receive the modulated structured light passing through the through slot 14, the processor 200 controls the structured light projector 21 to emit structured light. The structured light passes through the display area 11 and is projected to a predetermined distance from the structured light assembly 20 At the distance of the calibration plate, the structured light reflected by the calibration plate passes through the through slot 14 and is received by the structured light camera 22 to obtain a second reference image. Wherein, the multiple reference spots in the second reference image include both the first reference spot and the second reference spot. The first reference spot is formed when the laser light is diffracted by the diffractive optical element 213 when passing through the diffractive optical element 213, and is not diffracted by the display screen 10 after passing through the display screen 10, and is modulated and reflected by the calibration plate; the second reference spot is formed by the laser light The diffractive optical element 213 is diffracted once by the diffractive optical element 213, and when passing through the display screen 10, it is diffracted again by the display screen 10, and is modulated and reflected by the calibration plate.

Subsequently, the processor 200 calibrates the third reference image according to the first calculation method, that is, the reference image calibration method described in step 034. At this time, the third reference image includes a first reference spot corresponding to the first measurement spot, a second reference spot corresponding to the second measurement spot, and a third reference spot corresponding to the third measurement spot.

Among them, in the calibration scene of the first reference image, the second reference image, and the third reference image, the relative position between the calibration plate, the structured light projector 21 and the structured light camera 22 remains unchanged, and the structured light projector The relative position between 21 and structured light camera 21 also remains unchanged.

Subsequently, the processor 200 first marks the first coordinates of the first reference spot in the first reference image, and then filters out the first reference spots in the second reference image according to the coordinates of the first reference spot, and the remaining in the second reference image The reference spot is the second reference spot. The processor 200 marks the second coordinates of the second reference spot in the second reference image. The processor 200 then filters out the first reference spot and the second reference spot in the third reference image according to the first coordinate and the second coordinate in the second reference image, and the remaining reference spot in the third reference image is the third Reference spots. In this way, the processor 200 can distinguish the first reference spot, the second reference spot, and the third reference spot among all reference spots of the third reference image.

As the depth data is calculated later, the measurement spots in the speckle image also need to be distinguished. Specifically, the first measurement spot, the second measurement spot, and the third measurement spot can be distinguished by brightness. It can be understood that the first measurement spot is formed by the first order diffraction of the laser light through the diffractive optical element 213, the second measurement spot is formed by the first order diffraction of the laser light through the diffractive optical element 213 and the second order diffraction of the display screen 10, and the third measurement spot It is formed by the first diffraction of the diffractive optical element 213 and the second and third diffraction of the display screen 10. The laser beam forming the second measurement spot is diffracted more times than the laser beam forming the first measurement spot is diffracted to form the third measurement The number of times the laser spot is diffracted is greater than the number of times the laser spot forming the second measurement spot is diffracted. Therefore, the energy loss of the laser spot forming the first measurement spot is the smallest, and the energy loss of the laser spot forming the third measurement spot is the largest. The brightness of the second measurement spot will be lower than the brightness of the first measurement spot, and the brightness of the third measurement spot will be lower than the brightness of the second measurement spot. As such, it is feasible to distinguish the first measurement spot, the second measurement spot, and the spot measurement spot based on the brightness. Then, after the reference image calibration is completed, it is necessary to further calibrate the preset brightness and the preset ratio for distinguishing the first measurement spot, the second measurement spot, and the third measurement spot. Specifically, after the processor 200 distinguishes the first reference spot, the second reference spot, and the third reference spot, the processor 200 calculates the average value of the brightness of the plurality of first reference spots in the third reference image, and calculates The average value of the brightness of the plurality of second reference spots in the third reference image and the average value of the brightness of the plurality of third reference spots in the third reference image are obtained. Subsequently, the processor 200 takes the average value of the brightness of the plurality of first reference spots as the preset brightness, and takes the ratio between the average value of the brightness of the plurality of second reference spots and the average value of the brightness of the plurality of first reference spots As the first preset ratio, a ratio between the average value of the brightness of the plurality of third reference spots and the average value of the brightness of the plurality of first reference spots is taken as the second preset ratio.

In the subsequent depth data calculation, the processor 200 first calculates the brightness of each measurement spot. Subsequently, the processor 200 calculates the actual ratio between each measurement spot and the preset brightness, and classifies the measurement spot whose actual ratio is greater than or equal to the first preset ratio as the first measurement spot, and classifies the actual ratio as being less than the first preset The measurement spots whose ratio is greater than or equal to the second preset ratio are classified as the second measurement spots, and the measurement spots whose actual ratio is less than the second preset ratio are classified as the third measurement spots, thereby distinguishing the first measurement spots, The second measurement spot and the third measurement. For example, as shown in FIG. 30, assuming that the preset ratio is 0.8, the speckle image captured by the structured light camera 22 in actual use includes measurement spot A, measurement spot B, and measurement spot C. Among them, if the ratio between the brightness of the measurement spot A and the preset brightness is less than 0.8 and greater than or equal to 0.6, then the measurement spot A is classified as the second measurement spot, which means that the measurement spot A is the laser light passing through the diffractive optical element 213 Measurement spot formed by first diffraction and second order diffraction by the display screen 10 and reflected by the target object; if the ratio between the brightness of the measurement spot B and the preset brightness is greater than or equal to 0.8, the measurement spot B is classified as the first In the measurement spot, it is stated that the measurement spot B is a measurement spot formed by laser light diffracted once by the diffractive optical element 213 and reflected by the target object; if the ratio between the brightness of the measurement spot C and the preset brightness is less than 0.6, the measurement spot will be measured C is classified into the third measurement spot. At this time, the measurement spot C is a measurement that the laser beam is diffracted once by the diffractive optical element 213 and then diffracted again by the display screen 10 and reflected by the calibration object. spot. Among them, the preset ratios 0.8 and 0.6 are only examples.

After the processor 200 distinguishes the first measurement spot, the second measurement spot, and the third measurement spot, since the first reference spot, the second reference spot, and the third reference spot in the third reference image have also been distinguished, then process The device 200 can use the speckle image and the third reference image to calculate the depth data. Specifically, the processor 200 first calculates the offset of the first measurement spot relative to the first reference spot, the offset of the second measurement spot relative to the second reference spot, and the offset of the third measurement spot relative to the third reference spot Displacement. Subsequently, the processor 200 calculates multiple depth data based on multiple offsets, and the multiple depth data can constitute a depth image.

Compared with the first calculation method, the second calculation method distinguishes the first measurement spot, the second measurement spot, and the third measurement spot, and distinguishes the first reference spot, the second reference spot, and the third reference spot, It can be calculated based on the more accurate correspondence between the first measurement spot and the first reference spot, the correspondence between the second measurement spot and the second reference spot, and the correspondence between the third measurement spot and the third reference spot The offset of the data can further obtain more accurate depth data and improve the accuracy of the acquired depth image.

In some embodiments, the preset brightness, the first preset ratio and the second preset ratio are determined by the ambient brightness of the scene and the luminous power of the structured light projector 21. In this way, the accuracy of distinguishing the first measurement spot, the second measurement spot, and the third measurement spot can be improved.

In summary, in the image acquisition method of the embodiment of the present application, both the structured light projector 21 and the structured light camera 22 are located on the side of the back 13 of the display screen 10, and the structured light camera 22 receives the image twice through the display area 11 When the structured light is modulated, the processor 200 can directly calculate the depth image based on the first measurement spot, the second measurement spot, and the third measurement spot. Compared with the method of calculating the depth image using only the first measurement spot, the display screen 10 The diffractive effect increases the number of measurement spots and the randomness of the arrangement of measurement spots, which is beneficial to improve the accuracy of acquiring depth images. Further, the image acquisition method according to the embodiment of the present application can appropriately simplify the complexity of the structure of the diffraction grating in the diffractive optical element 213, and in turn, the number of measurement spots and the randomness of the arrangement are increased by the diffraction effect of the display screen 10, While ensuring the accuracy of acquiring the depth image, the manufacturing process of the structured light projector 21 can be simplified.

Please refer to FIG. 1, FIG. 3 and FIG. 31. In some embodiments, the structured light projector 21 and the structured light camera 22 are both disposed on the side of the back side 13 of the display screen 10, and the display screen 10 does not have a through slot 14 The structured light camera 22 receives modulated structured light that passes through the display area 11 twice. At this time, step 01 includes:

012: Control the structured light camera 22 to receive the structured light diffracted by the display area 11 and reflected by the target object when it is emitted and then diffracted by the display area 11 to obtain a speckle image. The speckle image includes multiple measurement spots, multiple The measurement spot includes a first measurement spot where the laser light is diffracted only once by the diffractive optical element 213 and reflected by the target object, and a second measurement is formed when the laser light is diffracted once by the diffractive optical element 213 and then diffracted by the display screen 10 twice and reflected by the target object The speckle and the laser are diffracted once by the diffractive optical element 213 and then diffracted again by the display screen 10 and reflected by the target object. The third measurement spot is formed by the third diffraction of the display screen 10; specifically, the first measurement spot is the laser The diffractive optical element 213 is not diffracted by the display screen 10 after passing through the display screen 10, that is, it is directly projected onto the target object without encountering a micro gap, and is formed after being modulated and reflected by the target object; the second measurement spot is the laser light passing through The diffractive optical element 213 is diffracted by the display screen 10 after passing through the display screen 10 after being diffracted, that is, projected after encountering a micro gap The target object, which is modulated and reflected by the target object, is not diffracted by the display screen 10 when it passes through the display screen 10 again; the third measurement spot is that the laser light is diffracted by the diffractive optical element 213 after passing through the display screen 10 and then diffracted by the display screen 10. It is projected to the target object after reaching the microscopic gap, and is modulated and reflected by the target object, passes through the display screen 10 again and is diffracted by the microscopic gap in the display screen 10 again;

Step 02 includes:

025: filter out the second measurement spot and the third measurement spot in the speckle image to obtain the first measurement spot; and

026: Acquire a depth image according to the first measurement spot and the reference spot in the reference image.

Please refer to FIG. 17 again. Step 012 may be implemented by the control module 401. Both step 025 and step 026 can be implemented by the calculation module 402.

Please refer to FIG. 1 again. Step 012, step 025, and step 026 may be implemented by the processor 200. That is to say, the processor 200 can also be used to control the structured light camera 22 to receive the structured light diffracted by the display area 11 when diffracted by the display area 11 after being emitted and reflected by the target object and then diffracted by the display area 11 to obtain a speckle image and filter out scattered light The second measurement spot and the third measurement spot in the spot image to obtain the first measurement spot, and the depth image is obtained according to the first measurement spot and the reference spot in the reference image.

Specifically, when the structured light projector 21 and the structured light camera 22 are arranged on the side of the back side 13 of the display screen 10, and the display screen 10 is not provided with a through slot 14, the structured light camera 22 captures the first measurement Speckle images of spots, second measurement spots and third measurement spots. In the calculation of the subsequent depth image, the processor 200 may filter out the second measurement spot and the third measurement spot in the speckle image, and only use the remaining first measurement spot to do the depth image with the reference spot in the reference image Calculation. At this time, the reference spot in the reference image should include only the first reference spot formed by a plurality of laser beams diffracted only by the diffractive optical element 213 and reflected by the calibration object. Therefore, by filtering out the second measurement spot and the third measurement spot in the speckle image, the influence of the display screen 10 on the structured light can be eliminated, thereby ensuring that the electronic device 1000 has a relatively high screen footprint, the electronic device 1000 The accuracy of the acquired depth image is also high.

That is to say, please refer to Figure 32, the image acquisition method also includes:

Step 026 includes:

0261: Calculate the offset of the first measurement spot relative to the first reference spot; and

0262: Calculate depth data according to the offset to obtain a depth image.

Please refer to FIG. 17 again. Step 035 may be implemented by the control module 401. Both step 0261 and step 0262 can be implemented by the calculation module 402.

Please refer to FIG. 1 again. Step 035, step 0261, and step 0262 can all be implemented by the processor 200. That is to say, the processor 200 can also be used to control the structured light camera 22 to receive the structured light directly reflected by the calibrated object after being emitted from the structured light projector 21 and directly incident when the reference image is calibrated to obtain the first reference image and calculate the first A measurement spot offset relative to the first reference spot, and calculating depth data according to the offset to obtain a depth image.

Specifically, after the processor 200 filters out the second measurement spot and the third measurement spot, only the first measurement spot remains in the speckle image, then the speckle image should now contain only the first reference spot corresponding to the first measurement spot The first reference image is used to calculate the depth image. The calibration process of the first reference image is the same as the calibration process of placing the structured light projector 21 in a scene that is not covered by the display screen 10 in step 035, which will not be repeated here. The multiple reference spots in the first reference image captured by structured light are the first reference spots formed by laser light diffracted only by the diffractive optical element 213 and reflected by the calibration object. In this way, the processor 200 can calculate the offset of the first measurement spot relative to the first reference spot, and then calculate multiple depth data based on the multiple offsets to obtain a depth image.

The processor 200 may filter out the second measurement spot and the third measurement spot by brightness. That is to say, referring to FIG. 33, in some embodiments, the image acquisition method further includes:

036: When the reference image is calibrated, the structured light camera 22 is controlled to receive the structured light that is diffracted by the display area 11 and reflected directly by the calibrated object when being emitted to obtain a second reference image, and the second reference image includes multiple reference spots, The multiple reference spots include a first reference spot formed by laser light diffracted by the diffractive optical element 213 and reflected by the calibration object and a laser beam diffracted once by the diffractive optical element 213 and then diffracted by the display screen 10 twice and reflected by the calibration object Two reference spots;

042: Compare the first reference image and the second reference image to obtain the second reference spot, and compare the third reference image and the second reference image to obtain the third reference spot ;

052: Calculate the ratio between the average value of the brightness of the plurality of second reference spots and the average value of the brightness of the plurality of first reference spots as the first preset ratio, and calculate the plurality of third The ratio between the average value of the brightness of the reference spots and the average value of the brightness of the plurality of first reference spots is used as the second preset ratio, and the average value of the brightness of the plurality of first reference spots is calculated and As the preset brightness;

Step 025 includes:

0251: Calculate the actual ratio between each measured spot and the preset brightness;

0252: The measurement spots whose actual ratio is greater than the first preset ratio are classified as the first measurement spots, and the measurement spots whose actual ratio is less than the first preset ratio and greater than the second preset ratio are classified as the second measurement spots, and Measurement spots whose actual ratio is less than the second preset ratio are classified as third measurement spots; and

0253: Filter the second measurement spot and the third measurement spot from all the measurement spots to obtain the first measurement spot.

Please refer to FIG. 17 again. Step 035, step 036, and step 037 may all be implemented by the control module 401. Step 042, step 052, step 0251, step 0252, and step 0253 can all be implemented by the calculation module 401.

Please refer to FIG. 1 again. Step 035, step 036, step 037, step 042, step 052, step 0251, step 0252, and step 0253 can all be implemented by the processor 200. That is to say, the processor 200 can also be used to control the structured light camera 22 to receive the structured light that is directly reflected by the calibrated object and directly incident after being emitted from the structured light projector 21 when the reference image is calibrated to obtain the first reference image. When the reference image is referenced, the structured light camera 22 is controlled to receive the structured light that is diffracted by the display area 11 and reflected directly by the calibrated object when being emitted to obtain a second reference image, and when the reference image is calibrated, the structured light camera 22 is controlled to receive and display the emitted light After the area 11 is diffracted and reflected by the calibration object, the structured light is diffracted by the display area 11 again when incident through the display area 11 to obtain a third reference image. The processor 200 may be further used to compare the first reference image with the second reference image to obtain the second reference spot, and compare the third reference image with the second reference image to obtain the first Three reference spots, calculating a ratio between the average value of the brightness of the plurality of second reference spots and the average value of the brightness of the plurality of first reference spots as the first preset ratio, calculating a plurality of The ratio between the average value of the brightness of the third reference spots and the average value of the brightness of the plurality of first reference spots is used as the second preset ratio, and the average of the brightness of the plurality of first reference spots is calculated Value as the preset brightness. The processor 200 can also be used to calculate the actual ratio between each measured spot and the preset brightness, classify the measured spots whose actual ratio is greater than the first preset ratio as the first measured spot, and classify the actual ratio as being smaller than the first preset ratio The measurement spots that are greater than the second preset ratio are classified as second measurement spots, the measurement spots whose actual ratio is less than the second preset ratio are classified as third measurement spots, and the second measurement spots are filtered out of all measurement spots And the third measurement spot to obtain the first measurement spot.

Among them, the process of calibrating the first reference image described in step 035 is the same as the calibration process in step 035 where the structured light projector 21 is placed in a scene that is not blocked by the display screen 10, and the second process described in step 036 The process of the reference image and the foregoing step 036 place the structured light projector 21 and the structured light camera 22 on the side of the back side 13 of the display screen 10, and the structured light camera 22 enters the light surface and aligns with the through slot 14 of the display screen 10 The calibration process for the calibration in the same scenario is the same. The process of calibrating the third reference image described in step 037 is the same as that in the previous step 037. The structured light projector 21 and the structured light camera 22 are placed on the back side 13 of the display screen 10 In addition, the calibration process for the calibration in the scenario where the display screen 10 does not have the through slot 14 is consistent, and will not be repeated here.

After obtaining the first reference image, the second reference image, and the third reference image, the processor 200 can determine the first coordinate of the first reference spot in the first reference image in the same manner as the foregoing step 042 The first reference spot in the second reference image, the remaining reference spot in the second reference image is the second reference spot, and the second coordinates of the second reference spot are marked to distinguish the first reference spot in the second reference image And the second reference spot. Subsequently, the processor 200 determines the first reference spot and the second reference spot in the third reference image according to the first coordinate and the second coordinate, and the remaining reference spot in the third reference image is the third The reference spots, so that the first reference spots, the second reference spots, and the third reference spots in the third reference image can be distinguished. Then, the processor 200 can calibrate to obtain the preset brightness, the first preset ratio, and the second preset based on the distinguished first reference spot, second reference spot, and third reference spot in the same manner as the foregoing step 052 ratio.

Similarly, in the calculation of the subsequent depth image, the processor 200 may adopt the same method as the foregoing step 062 and the foregoing step 072, that is, distinguish based on the calibrated first preset ratio, second preset ratio, and preset brightness The first measurement spot, the second measurement spot and the third measurement spot are filtered out, and then the second measurement spot and the third measurement spot are filtered out, leaving only the first measurement spot, and then calculating the first measurement spot relative to the first reference spot The offset, and finally the depth data is calculated based on the offset to obtain a depth image.

In some embodiments, the preset brightness, the first preset ratio and the second preset ratio are also determined by the ambient brightness of the scene and the luminous power of the structured light projector 21. In this way, the accuracy of filtering the second measurement spot and the third measurement spot can be improved.

In summary, in the image acquisition method of the embodiment of the present application, when both the structured light projector 21 and the structured light are located under the display screen 10, and the display screen 10 does not have a through slot 14, the second measurement spot and the third measurement spot are filtered out first Calculating the depth image only based on the remaining first measurement spots reduces the amount of data processing by the processor 200, which is beneficial to speed up the process of acquiring the depth image.

Please refer to FIGS. 1, 34 and 35. In some embodiments, when the structured light projector 21 is disposed on the side of the back surface 13 of the display screen 10, the electronic device 1000 further includes a compensation optical element 500. The compensation optical element 500 is provided between the diffractive optical element and the display screen 10. The structured light emitted by the structured light projector 21 sequentially passes through the compensation optical element 500 and the display screen 10 and exits into the scene. The compensation optical element 500 is used to cancel the diffraction effect of the display screen 10. At this time, the structured light camera 22 may be disposed on the side of the back surface 13 of the display screen 10, and the display screen 10 may not have a through slot 14, correspondingly, the structured light camera 22 receives the compensation optical element 500 and the display area 11 sequentially , The display area 11, the modulated structured light of the compensation optical element 500; or, the structured light camera 22 may be provided on the side of the back 13 of the display screen 10, and the display screen 10 is provided with a through slot 14, the structured light camera 22 The light incident surface is aligned with the through slot 14, and correspondingly, the structured light camera 22 receives the modulated structured light that sequentially passes through the compensation optical element 500, the display area 11, and the through slot 14.

Step 01 includes:

013: Control the structured light camera 22 to receive the structured light that passes through the compensation optical element 500 and the display area 11 of the display screen 10 and is reflected by the target object in order to obtain a speckle image when the light is emitted. The compensation optical element 500 is used to offset the display screen 10 Diffraction, the speckle image includes multiple measurement spots, and the multiple measurement spots include measurement spots formed by laser light diffracted only by the diffractive optical element 213 and reflected by the target object;

Referring to FIG. 17, step 013 may be implemented by the control module 401.

Please refer to FIG. 1 again. Step 013 may be implemented by the processor 200. That is to say, the processor 200 can also be used to control the structured light camera 22 to receive the structured light that passes through the compensation optical element 500 and the display area 11 of the display screen 10 and is reflected by the target object in order to obtain a speckle image.

In the image acquisition method according to the embodiment of the present application, a compensation optical element 500 is provided between the structured light projector 21 and the display screen 10 to offset the diffraction effect of the display screen 10. Wherein, the compensation optical element 500 and the display screen 10 may be arranged at a certain distance (as shown in FIG. 35); or, the compensation optical element 500 and the back surface 13 of the display screen 10 may be provided in close contact (not shown). In this way, the compensation optical element 500 and the portion of the display screen 10 opposite to the compensation optical element 500 can form a plane mirror, and the number of spots will not be changed when the structured light passes through the plane mirror. Therefore, the speckle formed by the structured light emitted into the scene The speckles in the pattern may be regarded as including only the laser light diffracted by the diffractive optical element 213, and the measurement spot may be regarded as the laser only diffracted by the diffractive optical element 213 and reflected by the target object.

Specifically, please refer to FIG. 1, FIG. 5, FIG. 8 and FIG. 36. In some embodiments, when the structured light camera 22 is disposed on the side of the back surface 13 of the display screen 10, and the display screen 10 is provided with the structure When the light incident surface of the optical camera 22 is aligned with the through slot 14, step 013 includes:

0131: Control the structured light camera 22 to directly receive the structured light that directly passes through the compensation optical element 500 and the display area 11 and is reflected by the target object when being emitted to obtain a speckle image;

The image acquisition method also includes:

038: When the reference image is calibrated, the structured light camera 22 is controlled to receive the structured light that directly passes through the compensation optical element 500 and the display area 11 and is reflected by the calibrated object when being emitted to obtain a reference image. The reference image includes multiple reference spots, The multiple reference spots include reference spots formed by laser light diffracted only by the diffractive optical element 213 and reflected by the calibration object;

Step 02 includes:

0271: Calculate the offset of the measurement spot relative to the reference spot; and

0272: Calculate depth data according to the offset to obtain a depth image.

Please refer to FIG. 17 again. Step 0131 and step 038 can be implemented by the control module 401. Both step 0271 and step 0272 can be implemented by the calculation module 402.

Please refer to FIG. 1 again. Step 0131, step 038, step 0271, and step 0272 can all be implemented by the processor 200. That is to say, the processor 200 can also be used to control the structured light camera 22 to receive the structured light that directly passes through the compensation optical element 500 and the display area 11 and is reflected by the target object in order to obtain the speckle image and the calibration reference image Time-controlled structured light camera 22 receives structured light that directly passes through the compensation optical element 500 and the display area 11 and is reflected by the calibration object at the time of emission to obtain a reference image, calculate the offset of the measurement spot relative to the reference spot, and according to the offset Calculate the depth data to get the depth image.

Wherein, the area of the compensation optical element 500 should be slightly greater than or equal to the divergent area formed by the structured light emitted by the structured light projector 21, so that all the structured light emitted by the structured light projector 21 can pass through the compensation optical element 500, which can be offset The diffraction effect of the display screen 10 is lost. In addition, the compensation optical element 500 cannot block the light incident surface of the structured light camera 22, that is, the compensation optical element 500 cannot overlap the through slot 14. It can be understood that the through slot 14 does not have a diffractive effect, and the structured light reflected by the target object will not be diffracted when passing through the through slot 14, therefore, it is not necessary to provide a compensation optical element 500 at the position of the through slot 14 to offset the display area 11 diffraction effect. Conversely, if the compensation optical element 500 is provided at the position of the through slot 14, the structured light passing through the compensation optical element 500 will be diffracted by the compensation optical element 500, resulting in the speckle image received by the structured light camera 22 including the laser beam After one diffraction, the diffractive optical element 213 passes through the compensation optical element 500 and the plane mirror formed by the portion of the display screen 10 opposite to the compensation optical element 500 and is diffracted by the compensation optical element 500 to form a measurement spot.

When the structured light camera 22 is disposed on the side of the back surface 13 of the display screen 10, and the display screen 10 is provided with a through slot 14 aligned with the light incident surface of the structured light camera 22, the light source 211 of the structured light projector 21 emits The laser light will pass through the compensation optical element 500 and the display area 11 in turn. The structured light camera 22 receives the light through the plane mirror composed of the compensation optical element 500 and the display screen 10, is modulated by the target object, and then reflects through the through slot 14 Structured light. Since the compensation optical element 500 counteracts the diffraction effect of the display area 11, the speckle image captured by the structured light camera 22 includes only measurement spots formed by laser light diffracted only once by the diffractive optical element 213 and reflected by the target object, without occurrence of laser light The measurement spot formed by the diffractive optical element 213 is diffracted once and diffracted again by the display screen 10 and reflected by the target object.

Correspondingly, the reference spot in the reference image should also only include the reference spot formed by laser light diffracted only once by the diffractive optical element 213 and reflected by the calibration object, then the calibration scene should be: placing the structured light projector 21 and structured light camera 22 On the side where the back surface 13 of the display screen 10 provided with the compensation optical element 500 is located, the light incident surface of the structured light camera 22 is aligned with the through slot 14 of the display screen 10. In this way, in the calibration scene and the actual use scene, the installation positions of the structured light projector 21 and the structured light camera 22 relative to the display screen 10 are the same. The processor 200 controls the structured light projector 21 to emit structured light. The structured light sequentially passes through the compensation optical element 500 and the display screen 10 and then is projected to the calibration plate at a predetermined distance from the structured light assembly 20, and the structured light reflected back by the calibration plate The structured light camera 22 receives the through slot 14. At this time, the structured light camera 22 receives the laser light which is diffracted by the diffractive optical element 211 once after being emitted by the light source 211 and reflected by the calibration plate, and then directly incident through the through slot 14. The multiple reference spots included in the formed reference image are The laser beam is diffracted only once by the diffractive optical element 213 and reflected by the calibration object to form a reference spot.

The processor 200 does not need to filter out the measurement spots formed by the laser through two diffractions when calculating the depth image, and can directly calculate the depth image based on the measurement spots formed by the only laser through only one diffraction and the reference spots in the reference image. Specifically, the processor 200 calculates the offset between the measurement spot and the reference spot, and then calculates the depth data according to the offset, thereby obtaining a depth image.

Similarly, please refer to FIG. 3 and FIG. 37. In some embodiments, when the structured light camera 22 is disposed on the back side of the display screen 10, and the display screen 10 is not provided with a through slot 14, step 013 includes:

0132: Control the structured light camera 22 to sequentially pass through the compensation optical element 500 and the display area 11 when it is emitted and reflected by the target object, and then to pass through the structured light of the display area 11 and the compensation optical element 500 in order to obtain a speckle image;

The image acquisition method also includes:

039: When the reference image is calibrated, the structured light camera 22 is controlled to pass through the compensation optical element 500 and the display area 11 sequentially when receiving the emission and is reflected by the calibrated object, and then passes through the structured light of the display area 11 and the compensation optical element 500 in order to obtain a reference Image, the reference image includes multiple reference spots, and the multiple reference spots include reference spots formed by laser light diffracted only by the diffractive optical element 213 and reflected by the calibration object;

Step 02 includes:

0272: Calculate depth data according to the offset to obtain a depth image.

Please refer to FIG. 17 again. Both step 0132 and step 039 may be implemented by the control module 401. Both step 0271 and step 0272 can be implemented by the calculation module 402.

Please refer to FIG. 1 again. Step 0132, step 039, step 0271, and step 0272 can all be implemented by the processor 200. That is to say, the processor 200 can also be used to control the structured light camera 22 to sequentially pass through the compensation optical element 500 and the display area 11 when it is emitted and reflected by the target object, and then to pass through the structured light of the display area 11 and the compensation optical element 500 in sequence when it is incident To obtain a speckle image, control the structured light camera 22 to sequentially pass through the compensation optical element 500 and the display area 11 when receiving the emitted light, and to reflect the structured light passing through the display area 11 and the compensation optical element 500 when it is incident after being reflected by the calibration object To obtain a reference image, calculate an offset of the measurement spot relative to the reference spot, and calculate depth data according to the offset to obtain a depth image.

The compensation optical element 500 should completely cover the structured light projector 21 and the structured light camera 22 at the same time. In this way, on the one hand, the structured light emitted by the structured light projector 21 can all pass through the compensation optical element 500, which can cancel the diffraction effect of the display screen 10; on the other hand, the structured light reflected by the target object can also all pass through the compensation optic The element 500 counteracts the diffraction effect of the display screen 10, so that the speckle image captured by the structured light camera 22 includes only measurement spots formed by laser light diffracted only once by the diffractive optical element 213 and reflected by the target object.

Specifically, when the structured light camera 22 is disposed on the side where the back 13 of the display screen 10 is located, and the display screen 10 is not provided with a through slot 14, the laser light emitted by the light source 211 of the structured light projector 21 will sequentially pass through the compensation optical element 500 and the display area 11, the structured light camera 22 receives the light through the plane mirror composed of the compensation optical element 500 and the display screen 10, is reflected by the target object, and then passes through the plane mirror composed of the display screen 10 and the compensation optical element 500 Structured light incident. Since the compensation optical element 500 counteracts the diffraction effect of the display area 11, the speckle image captured by the structured light camera 22 includes only measurement spots formed by laser light diffracted only once by the diffractive optical element 213 and reflected by the target object, without occurrence of laser light The diffractive optical element 213 is diffracted once and then diffracted by the display screen 10 twice and reflected by the target object to form a measurement spot, nor will the laser light diffracted by the diffractive optical element 213 once by the display screen 10 and diffracted again by the target object The measurement spot formed by the third-order diffraction of the display screen 10 again.

Correspondingly, the reference spot in the reference image should also only include the reference spot formed by laser light diffracted only once by the diffractive optical element 213 and reflected by the calibration object, then the calibration scene should be: placing the structured light projector 21 and structured light camera 22 On the side where the back surface 13 of the display screen 10 provided with the compensation optical element 500 is located, the display screen 10 is not provided with a through slot 14. In this way, in the calibration scene and the actual use scene, the installation positions of the structured light projector 21 and the structured light camera 22 relative to the display screen 10 are the same. The processor 200 controls the structured light projector 21 to emit structured light. The structured light sequentially passes through the compensation optical element 500 and the display screen 10 and then is projected to the calibration plate at a predetermined distance from the structured light assembly 20, and the structured light reflected back by the calibration plate After passing through the display screen 10 and the compensation optical element 500 in sequence, it is received by the structured light camera 22, and the multiple reference spots included in the formed reference image are the reference spots formed by the laser light diffracted only once by the diffractive optical element 213 and reflected by the calibration object .

When the processor 200 calculates the depth image, it is not necessary to filter out the measurement spots formed by the laser through multiple diffractions, and the depth image can be directly calculated based on the measurement spots formed by the laser through only one diffraction and the reference spots in the reference image. Specifically, the processor 200 calculates the offset between the measurement spot and the reference spot, and then calculates the depth data according to the offset, thereby obtaining a depth image.

In summary, the image acquisition method according to the embodiment of the present application compensates for the diffraction effect of the display screen 10 by providing the compensation optical element 500. In this way, the speckle image captured by the structured light camera 22 includes only the laser light diffracted by the diffractive optical element 213 once. The measurement spot reflected by the target object without the measurement spot formed by the laser diffracted by the diffractive optical element 213 once and then diffracted by the display screen 10 second time and reflected by the target object, and the laser spot diffracted by the diffractive optical element 213 once The measurement spot formed by the second diffraction of the display screen 10 and reflected by the target object and then by the third diffraction of the display screen 10, the processor 200 does not need to perform the filtering operation, and can be directly based on all measurement spots and reference images in the speckle image All reference blobs in are used to calculate depth images, which simplifies the calculation process of depth images and speeds up the acquisition of depth images.

Please refer to FIG. 1, FIG. 4 and FIG. 38. In some embodiments, the diffractive optical element 213 in the structured light projector 21 is replaced with an optical element 214. The optical element 214 is used to compensate the brightness of the structured light diffracted by the display screen 10. Of uniformity. Step 01 includes:

014: Control the structured light camera 22 to receive the structured light diffracted by the display area 11 of the display screen 10 and reflected by the target object when it is emitted to obtain a speckle image, and the optical element 214 in the structured light projector 21 is used to compensate the display screen 10 The uniformity of the brightness of the diffracted structured light, and the speckle image includes multiple measurement spots.

Please refer to FIG. 17 again. Step 014 may be implemented by the control module 401.

Please refer to FIG. 1 again. Step 014 may also be implemented by the processor 200. That is to say, the processor 200 can also be used to control the structured light camera 22 to receive the structured light diffracted by the display area 11 of the display screen 10 and reflected by the target object when it exits to obtain a speckle image. 21 in the structured light projector The optical element 214 is used to compensate the uniformity of the brightness of the structured light diffracted by the display screen 10.

Specifically, a micro gap is formed between adjacent pixels in the display area 11 of the display screen 10, and the structured light emitted by the structured light projector 21 passes through the display area 11 and is diffracted by the display area 11 to form multiple spots. However, the brightness distribution of the spots diffracted by the display area 11 is not uniform.

The image acquisition method of the embodiment of the present application forms a plurality of spots by means of the diffraction effect of the display screen 10, and replaces the diffractive optical element 213 in the structured light projector 21 with one that can compensate for the uniformity of the brightness of the structured light diffracted by the display screen 10. The optical element 214, that is, the laser light emitted by the light source 211 of the structured light projector 21 passes through the optical element 214 and the display screen 10 in sequence, and there are multiple spots in the speckle pattern projected onto the scene, and the brightness of the spots is relatively uniform, where, The spots are diffracted by the display screen 10, and the uniformity of the brightness of the spots is compensated by the optical element 214.

In this way, the measurement spots in the speckle image captured by the structured light camera 22 are directly formed by the diffraction effect of the display screen 10, and the processor 200 can calculate the depth image based on these measurement spots. The optical element 214 compensates for the uniformity of the brightness of the structured light diffracted by the display screen 10, which is beneficial to improve the accuracy of the acquired depth image.

Please refer to FIG. 1, FIG. 4, FIG. 5, FIG. 8 and FIG. 39. In some embodiments, when the structured light projector 21 and the structured light camera 22 are both disposed on the back side of the display screen 10, and the display screen 10 is provided with a through slot 14, and when the light incident surface of the structured light camera 22 is aligned with the through slot 14, step 014 includes:

0141: Control the structured light camera 22 to receive the structured light directly diffracted by the display area 11 and reflected by the target object when being emitted to obtain a speckle image, and the optical element 214 is used to compensate the uniformity of the brightness of the structured light diffracted by the display screen 10 , The speckle image includes a plurality of measurement spots, and the plurality of measurement spots includes a first measurement spot formed by laser light diffused through the optical element 214 and then diffracted by the display screen 10 once and reflected by the target object;

The image acquisition method also includes:

091: When the reference image is calibrated, the structured light camera 22 is controlled to receive the structured light diffracted by the display area 11 and reflected directly by the calibrated object when it is emitted to obtain the reference image, and the optical element 214 is used to compensate the structured light diffracted by the display screen 10 The uniformity of the brightness, the reference image includes a plurality of reference spots, and the plurality of reference spots includes a first reference spot formed by laser light diffused through the optical element 214 and then diffracted by the display screen 10 once and reflected by the calibration object;

Step 02 includes:

0281: Calculate the offset of the first measurement spot relative to the first reference spot; and

0282: Calculate depth data according to the offset to obtain a depth image.

Please refer to FIG. 17 again, both step 0141 and step 091 can be implemented by the control module 401. Both step 0281 and step 0282 can be implemented by the calculation module 402.

Please refer to FIG. 1 again. Step 0141, step 091, step 0281, and step 0282 can all be implemented by the processor 200. That is to say, the processor 200 can also be used to control the structured light camera 22 to receive the structured light diffracted by the display area 11 and reflected directly by the target object when it is emitted to obtain a speckle image, and the optical element 214 is used to compensate the display screen 10 The uniformity of the brightness of the diffracted structured light. The processor 200 can also be used to control the structured light camera 22 to receive the structured light that is diffracted by the display area 11 and reflected directly by the calibration object to obtain the reference image when the reference image is calibrated, and the optical element 214 is used to compensate the display screen 10 The uniformity of the brightness of the diffracted structured light. The processor 200 may also be used to calculate an offset of the first measurement spot relative to the first reference spot, and calculate depth data according to the offset to obtain a depth image.

Specifically, when the structured light camera 22 is disposed on the side of the back surface 13 of the display screen 10, and the display screen 10 is provided with a through slot 14 aligned with the light incident surface of the structured light camera 22, the structured light projector 21 The laser light emitted by the light source 211 sequentially passes through the optical element 214 and the display area 11 of the display screen 10 to form structured light and exit into the scene. The structured light is reflected by the target object and then enters the through slot 14 to be received by the structured light camera 22. Since the optical element 214 is only used to compensate the uniformity of the brightness of the structured light diffracted by the display screen 10 without increasing the number of measurement spots, and the through slot 14 does not have a microscopic gap, it will not diffract the structured light reflected back, so The speckle image captured by the structured light camera 22 includes only the first measurement spot formed by the laser light diffused through the optical element 214 and then diffracted by the display screen 10 once and reflected by the target object.

Correspondingly, the reference spot in the reference image should also include only the first reference spot formed by the laser diffused through the optical element 214 and then diffracted by the display screen 10 once and reflected by the calibration object, then the calibration scene should be the one with the optical element 214 The structured light projector 21 and the structured light camera 22 are placed on the side of the back surface 13 of the display screen 10, and the light incident surface of the structured light camera 22 is aligned with the through slot 14 of the display screen 10. In this way, in the calibration scene and the actual use scene, the installation positions of the structured light projector 21 and the structured light camera 22 relative to the display screen 10 are the same. The processor 200 controls the light source 211 of the structured light projector 21 to emit laser light, and the laser light passes through the optical element 214 and the display screen 10 in turn to form structured light and is projected onto the calibration plate at a predetermined distance from the structured light assembly 20, which is reflected back by the calibration plate The structured light passing through the through slot 14 is received by the structured light camera 22. At this time, the structured light camera 22 receives the laser light emitted by the light source 211, diffused by the diffractive optical element 213, diffracted by the display screen 10 once and reflected by the calibration plate, and then directly incident through the through slot 14, the reference image formed includes The multiple reference spots are the first reference spots formed by the laser light diffusing through the optical element 214 and then diffracted by the display screen 10 once and reflected by the calibration object.

When calculating the depth image, the processor 200 directly calculates the depth image based on the first measurement spot and the first reference spot in the reference image. Specifically, the processor 200 calculates the offset between the first measurement spot and the first reference spot, and then calculates depth data according to the offset, thereby obtaining a depth image.

Similarly, referring to FIGS. 3 and 40, in some embodiments, when the structured light camera 22 is disposed on the side of the back of the display screen 10 and the display screen 10 is not provided with a through slot 14, step 014 includes:

0142: Control the structured light camera 22 to receive the structured light diffracted by the display area 11 when being emitted and reflected by the target object and then diffracted by the display area 11 when incident to obtain a speckle image, and the optical element 214 is used to compensate the structure of the diffraction of the display screen 10 The uniformity of the brightness of the light, the speckle image includes multiple measurement spots, the multiple measurement spots include the first measurement spot formed by the laser light diffused through the optical element 214 and then diffracted by the display screen 10 once and reflected by the target object, and the laser light passes through the optical The element 214 diffuses a second measurement spot formed by the second diffraction of the display screen 10 after being diffracted once by the display screen 10 and reflected by the target object.

Please refer to FIG. 17 again. Step 0142 can be implemented by the control module 401.

Please refer to FIG. 1 again. Step 0142 can also be implemented by the processor 200. That is to say, the processor 200 can also be used to control the structured light camera 22 to receive the structured light diffracted by the display area 11 when diffracted by the display area 11 after being emitted and reflected by the target object and then diffracted by the display area 11 to obtain a speckle image. The optical element 214 It is used to compensate the uniformity of the brightness of the structured light diffracted by the display screen 10.

Specifically, when the structured light camera 22 is disposed on the side where the back 13 of the display screen 10 is located, and the display screen 10 is not provided with a through slot 14, the laser light emitted by the light source 211 of the structured light projector 21 passes through the optical element 214 and After the display area 11 of the display screen 10, structured light is formed and emitted into the scene. The structured light is reflected by the target object and then enters through the display screen 10 to be received by the structured light camera 22. Since the optical element 214 is only used to compensate the uniformity of the brightness of the structured light diffracted by the display screen 10 without increasing the number of measurement spots, and the display screen 10 has a microscopic gap to diffract the structured light reflected back, therefore, the structure The speckle image captured by the optical camera 22 includes a plurality of laser spots diffused by the optical element 214 and then diffracted once by the display screen 10 and reflected by the target object, and the first measurement spot formed by the laser beam diffused by the optical element 214 and then by the display screen 10 The second measurement spot formed by the second diffraction of the display screen 10 after being diffracted once and reflected by the target object.

Please refer to FIG. 40 again. In a calculation method, step 02 includes:

0283: Calculate the offset of all measurement spots relative to all reference spots; and

0284: Calculate depth data according to the offset to obtain a depth image.

Correspondingly, the image acquisition method also includes:

092: When the reference image is calibrated, the structured light camera 22 is controlled to receive the structured light diffracted by the display area 11 when diffracted by the display area 11 after being emitted and reflected by the calibrated object and then diffracted by the display area 11 to obtain the reference image. The uniformity of the brightness of the structured light diffracted by the screen 10, the reference image includes a plurality of reference spots, the plurality of reference spots including the first reference spots formed after the laser light diffuses through the optical element 214 and is diffracted by the display screen 10 once and reflected by the calibration object The second laser beam is diffused by the optical element 214 and then diffracted by the display screen 10 once and reflected by the calibration object, and then is second diffracted by the display screen 10 second diffraction.

Please refer to FIG. 17 again. Step 0283 and step 0284 can be implemented by the calculation module 402. Step 092 can be implemented by the control module 401.

Please refer to FIG. 1 again. Step 0283, step 0284, and step 092 may also be implemented by the processor 200. That is to say, the processor 200 can also be used to calculate the offsets of all measurement spots relative to all reference spots, and calculate depth data according to the offsets to obtain depth images. The processor 200 can also be used to control the structured light camera 22 to receive the structured light diffracted by the display area 11 when diffracted by the display area 11 and reflected by the calibrated object and then diffracted by the display area 11 when the reference image is calibrated to obtain the reference image. It is used to compensate the uniformity of the brightness of the structured light diffracted by the display screen 10.

Specifically, in the process of calibrating the reference image, the structured light projector 21 provided with the optical element 214 and the structured light camera 22 are placed on the side of the back surface 13 of the display screen 10, wherein the display screen 10 does not have a through slot 14 . In this way, in the calibration scene and the actual use scene, the installation positions of the structured light projector 21 and the structured light camera 22 relative to the display screen 10 are the same. The processor 200 controls the light source 211 of the structured light projector 21 to emit laser light, and the laser light passes through the optical element 214 and the display screen 10 in turn to form structured light and is projected onto the calibration plate at a predetermined distance from the structured light assembly 20, which is reflected back by the calibration plate After passing through the display screen 10, the structured light is received by the structured light camera 22. At this time, the reference image captured by the structured light camera 22 includes multiple first reference spots and multiple second reference spots at the same time. Among them, the first reference spot is the laser light diffused through the optical element 214 and then diffracted by the display screen 10 and reflected by the calibrated object. The second reference spot is the laser light diffused through the optical element 214 and then diffracted by the display screen 10 and the calibrated object After reflection, it is formed by the second order diffraction of the display screen 10 again. Although the speckle image includes both the first measurement spot and the second measurement spot, the reference image includes both the first reference spot and the second reference spot. However, in this calculation method, the processor 200 does not distinguish between the first measurement spot and the second measurement spot in the speckle image, nor does it distinguish between the first reference spot and the second reference spot in the reference image Instead, the depth image is calculated directly based on all the measurement spots and reference spots. Specifically, the processor 200 first calculates the offsets of all measurement spots relative to all reference spots, and then calculates multiple depth data based on the multiple offsets, thereby obtaining a depth image.

Referring to FIG. 41, in another calculation method, step 02 includes:

0285: Calculate the offset of the first measurement spot relative to the first reference spot, and the offset of the second measurement spot relative to the second reference spot; and

0286: Calculate depth data according to the offset to obtain a depth image.

Correspondingly, the image acquisition method also includes:

092: When the reference image is calibrated, the structured light camera 22 is controlled to receive the structured light diffracted by the display area 11 when diffracted by the display area 11 after being emitted and reflected by the calibrated object and then diffracted by the display area 11 to obtain the reference image, and the optical element 214 is used to compensate the display The uniformity of the brightness of the structured light diffracted by the screen 10, the reference image includes a plurality of reference spots, the plurality of reference spots including the first reference spots formed after the laser light diffuses through the optical element 214 and is diffracted by the display screen 10 once and reflected by the calibration object A second reference spot formed by the laser beam diffusing through the optical element 214 and then diffracted by the display screen 10 once and reflected by the calibration object and then again diffracted by the display screen 10;

043: Compare the first reference image with the second reference image to obtain a second reference spot;

053: Calculate the ratio between the average of the brightness of the multiple second reference spots and the average of the brightness of the multiple first reference spots as the preset ratio, and calculate the average of the brightness of the multiple first reference spots and As the preset brightness;

063: Calculate the actual ratio between each measured spot and the preset brightness; and

073: Classify the measurement spot whose actual ratio is greater than the preset ratio as the first measurement spot, and classify the measurement spot whose actual ratio is less than the preset ratio as the second measurement spot.

Please refer to FIG. 17 again. Step 0285, step 0286, step 43, step 053, step 063, and step 073 can all be implemented by the calculation module 402. Both step 091 and step 092 can be implemented by the control module 401.

Please refer to FIG. 1 again. Step 0285, step 0286, step 091, step 092, and step 43, step 053, step 063, and step 073 can all be implemented by the processor 200. That is to say, the processor 200 can also be used to calculate the offset of the first measurement spot relative to the first reference spot, and the offset of the second measurement spot relative to the second reference spot, and calculate the depth according to the offset Data to obtain depth images. The processor 200 can also be used to control the structured light camera 22 to receive the structured light directly diffracted by the display area 11 and reflected by the calibration object when the reference image is calibrated to obtain the reference image, and to control the structured light when the reference image is calibrated The camera 22 receives the structured light diffracted by the display area 11 when it is emitted and reflected by the calibration object and then diffracted by the display area 11 when incident to obtain a reference image, wherein the optical element 214 is used to compensate the brightness of the structured light diffracted by the display screen 10 Uniformity. The processor 200 may also be used to compare the first reference image and the second reference image to obtain the second reference spot, and calculate the average value of the brightness of the plurality of second reference spots and the average value of the brightness of the plurality of first reference spots The ratio is used as the preset ratio, and the average value of the brightness of the plurality of first reference spots is calculated and used as the preset brightness. The processor 200 can also be used to calculate the actual ratio between each measurement spot and the preset brightness, classify the measurement spots whose actual ratio is greater than the preset ratio as the first measurement spots, and classify the measurement spots whose actual ratio is less than the preset ratio It is classified as the second measurement spot.

In this calculation mode, the processor 200 needs to calibrate the first reference image and the second reference image. The calibration process of the first reference image and the aforementioned step 091 place the structured light projector 21 provided with the optical element 214 and the structured light camera 22 on the side of the back side 13 of the display screen 10, and the structured light camera 22 enters the light The calibration process in the scene where the surface is aligned with the through slot 14 of the display screen 10 is consistent. The calibration process of the second reference image is the same as the step 092 where the structured light projector 21 provided with the optical element 214 and the structured light camera 22 are placed The calibration process is the same on the side where the back surface 13 of the display screen 10 is located, and the display screen 10 does not have the through slot 14, and will not be repeated here.

After the processor 200 calibrates the first reference image and the second reference image, the processor 200 marks the coordinates of the first reference spot in the first reference image, and filters out the second reference image according to the coordinates of the first reference spot One reference spot, and the remaining reference spots in the second reference image are the second reference spots. In this way, the processor can distinguish the first reference spot and the second reference spot from all the reference spots in the second reference image.

As the depth data is calculated later, the measurement spots in the speckle image also need to be distinguished. Specifically, the first measurement spot and the second measurement spot can be distinguished by brightness. It can be understood that the first measurement spot is formed by the laser diffused through the optical element 214 and then diffracted by the display screen 10 once, and the second measurement spot is formed by the diffused optical element 214 and then formed by the first and second diffraction of the display screen 10, forming the first The second measurement spot laser is diffracted more times than the first measurement spot laser is diffracted. Therefore, the energy loss of the laser forming the first measurement spot is smaller, and the energy loss of the laser forming the second measurement spot is larger The brightness of the second measurement spot will be lower than the brightness of the first measurement spot. In this way, the first measurement spot and the second measurement spot can be distinguished based on the brightness. Then, after the reference image calibration is completed, it is necessary to further calibrate the brightness for distinguishing the first measurement spot from the second measurement spot. That is to say, after the processor 200 distinguishes the first reference spot and the second reference spot, the processor 200 needs to calculate the average value of the brightness of the plurality of first reference spots in the second reference image, and calculate the first The average value of the brightness of multiple second reference spots in the two reference images. Subsequently, the processor 200 takes the average value of the brightness of the plurality of first reference spots as the preset brightness, and takes the ratio between the average value of the brightness of the plurality of second reference spots and the average value of the brightness of the plurality of first reference spots As a preset ratio.

In the subsequent depth data calculation, the processor 200 first calculates the brightness of each measurement spot. Subsequently, the processor 200 calculates the actual ratio between each measurement spot and the preset brightness, and classifies the measurement spots whose actual ratio is greater than or equal to the preset ratio as the first measurement spot, and measures the actual ratio less than the preset ratio The spots are classified as second measurement spots, thereby distinguishing the first measurement spots from the second measurement spots.

In some embodiments, the preset brightness and the preset ratio are determined by the ambient brightness of the scene and the luminous power of the structured light projector 21. In this way, the accuracy of the distinction between the first measurement spot and the second measurement spot can be improved.

In summary, in the image acquisition method of the embodiment of the present application, the measurement spots in the speckle image captured by the structured light camera 22 are directly formed by the diffraction effect of the display screen 10, and the processor 200 can calculate the depth image based on these measurement spots . The optical element 214 compensates the brightness uniformity of the structured light diffracted by the display screen 10, which is beneficial to improve the accuracy of acquiring the depth image.

In the description of this specification, the description referring to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means specific features described in conjunction with the embodiment or examples , Structure, material or characteristic is included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. In addition, without contradicting each other, those skilled in the art may combine and combine different embodiments or examples and features of the different embodiments or examples described in this specification.

In addition, the terms “first” and “second” are used for description purposes only, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined with "first" and "second" may include at least one of the features either explicitly or implicitly. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise specifically limited.

Although the embodiments of the present application have been shown and described above, it can be understood that the above-mentioned embodiments are exemplary and cannot be construed as limitations to the present application. Those of ordinary skill in the art can The embodiments are changed, modified, replaced, and modified.

Claims

An electronic device, characterized in that it includes:

A display screen, the display screen including a display area formed with opposite front and back surfaces, the front surface being used to display images; and

A structured light assembly, the structured light assembly includes a structured light projector, the structured light projector is disposed on a side of the display screen where the back side is located, and the structured light projector is used to emit through the display area Structured light.
The electronic device according to claim 1, wherein the structured light component further comprises a structured light camera, the structured light camera is disposed on a side of the display screen where the back surface is located, and the structured light camera It is used to receive the modulated structured light passing through the display area.
The electronic device according to claim 1, wherein the display screen is formed with a through slot penetrating the front surface and the back surface, and the structured light assembly further includes a structured light camera, the structured light camera is disposed at On the side where the back surface of the display screen is located, the structured light camera is used to receive modulated structured light passing through the through slot.
The electronic device according to claim 3, wherein the through slot includes a notch formed on an edge of the display screen; and / or

The through slot includes a through hole spaced from an edge of the display screen.
The electronic device according to claim 4, wherein the edge of the display screen includes any one or more of an upper edge, a lower edge, a left edge, and a right edge.
The electronic device according to claim 3, wherein the structured light assembly further comprises a floodlight, and the floodlight and the structured light camera are aligned with the same through slot.
The electronic device according to claim 3, characterized in that the electronic device further comprises a cover plate, the cover plate is disposed on a side of the display screen where the front side is located, and the cover plate and the An infrared transmission layer is provided on the area corresponding to the through slot.
The electronic device according to claim 1, wherein an infrared anti-reflection coating is formed in an area of the display screen corresponding to the structured light projector; and / or

An infrared transmission layer is formed in an area of the display screen corresponding to the structured light projector.
The electronic device according to claim 1, characterized in that the electronic device further comprises a cover plate, the cover plate is disposed on a side of the front side of the display screen, and the structure of the cover plate and the light An infrared anti-reflection coating is formed in the area corresponding to the projector.
The electronic device according to claim 9, wherein the display screen is formed with a through-groove penetrating the front surface and the back surface, the electronic device further includes a visible light camera, the visible light camera and the through groove Alignment is set, and the region corresponding to the through groove on the cover plate is formed with a visible light antireflection film and / or an infrared cutoff film.
The electronic device according to any one of claims 1 to 9, wherein the structured light assembly further comprises a floodlight, and the floodlight is provided on a side of the display screen where the back surface is located, The floodlight is used to emit supplementary light passing through the display area.
The electronic device according to claim 1 or 2, wherein the display screen is formed with a through-groove penetrating through the front surface and the back surface, and the structured light assembly further includes a floodlight, the floodlight Set on the side where the back side of the display screen is located, the floodlight is used to emit supplementary light passing through the through slot.
The electronic device according to any one of claims 1 to 9, wherein the display area includes a first sub-display area and a second sub-display area, and the structured light emitted by the structured light projector passes through the In the first sub-display area, the pixel density of the first sub-display area is less than the pixel density of the second sub-display area.
The electronic device according to claim 13, wherein the first sub-display area is used to display a status icon of the electronic device.
The electronic device of claim 13, wherein the first sub-display area is located near the edge of the display area, and the second sub-display area is located in the middle of the display area.
The electronic device according to any one of claims 1 to 9, wherein the display area includes a first sub-display area and a second sub-display area, and the structured light emitted by the structured light projector passes through the The first sub-display area, the first sub-display area and the second sub-display area can be independently controlled and displayed in different display states.
The electronic device according to claim 16, wherein the different display states include one or more of turning on or off, displaying with different brightness, and displaying with different refresh frequencies.
The electronic device according to claim 16, wherein when the structured light projector emits structured light, the first sub-display area is extinguished; or

When the structured light projector emits structured light, the display brightness of the first sub-display area is reduced; or

When the structured light projector emits structured light, the refresh frequency of the first sub-display area is adjusted so that the turn-on time of the first sub-display area is staggered from the turn-on time of the structured light projector.
The electronic device according to claim 16, wherein when the structured light projector is not activated, both the first sub-display area and the second sub-display area are turned on and displayed at the same refresh frequency.
The electronic device according to any one of claims 1 to 9, wherein the display screen is a liquid crystal display screen, or a Micro LED display screen, or an OLED display screen.