CN109618085B

CN109618085B - Electronic equipment and mobile platform

Info

Publication number: CN109618085B
Application number: CN201910007534.XA
Authority: CN
Inventors: 张学勇
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2019-01-04
Filing date: 2019-01-04
Publication date: 2021-05-14
Anticipated expiration: 2039-01-04
Also published as: CN109618085A

Abstract

The application discloses an electronic device and a mobile platform. The electronic device includes a body and a plurality of time-of-flight components disposed at a plurality of different orientations of the body. Each time-of-flight component comprises an optical transmitter and an optical receiver, the field angle of each optical receiver is any value from 180 degrees to 200 degrees, and the field angle of each optical transmitter is larger than or equal to that of each optical receiver. The light emitter is used for emitting laser pulses to the outside of the body, and the light receiver is used for receiving the laser pulses emitted by the corresponding light emitter reflected by the shot target. The plurality of light emitters emit laser pulses in a time-sharing manner, the plurality of light receivers expose in a time-sharing manner to acquire a panoramic depth image, and when the light receiver in any one time-of-flight component exposes, the light emitters in other time-of-flight components are all closed. The electronic equipment can obtain comprehensive depth information at one time by emitting laser pulses in different directions in a time-sharing mode through the light emitters and exposing the light receivers in a time-sharing mode.

Description

Electronic equipment and mobile platform

Technical Field

The present application relates to the field of image acquisition technologies, and more particularly, to an electronic device and a mobile platform.

Background

In order to diversify the functions of the electronic device, a depth image acquiring device may be provided on the electronic device to acquire a depth image of a subject. However, the current depth image acquiring device can acquire only a depth image in one direction or one angle range, and the acquired depth information is less.

Disclosure of Invention

The embodiment of the application provides electronic equipment and a mobile platform.

The electronic equipment comprises a body and a plurality of time-of-flight components arranged on the body, wherein the plurality of time-of-flight components are respectively positioned at a plurality of different orientations of the body, each time-of-flight component comprises a light emitter and a light receiver, the field angle of each light receiver is any value from 180 degrees to 200 degrees, the field angle of each light emitter is larger than or equal to that of each light receiver, the light emitters are used for emitting laser pulses to the outside of the body, and the light receivers are used for receiving the laser pulses emitted by the corresponding light emitters reflected by a shot target; the light emitters in the plurality of time-of-flight components emit the laser pulses in a time-sharing manner, the light receivers in the plurality of time-of-flight components receive the laser pulses in a time-sharing manner to acquire a panoramic depth image, and the light emitters in the other time-of-flight components are all turned off when the light receivers in any one of the time-of-flight components are exposed.

The mobile platform comprises a body and a plurality of time-of-flight components arranged on the body, wherein the plurality of time-of-flight components are respectively positioned at a plurality of different orientations of the body, each time-of-flight component comprises a light emitter and a light receiver, the field angle of each light receiver is any value from 180 degrees to 200 degrees, the field angle of each light emitter is larger than or equal to that of each light receiver, the light emitters are used for emitting laser pulses to the outside of the body, and the light receivers are used for receiving the laser pulses emitted by the corresponding light emitters reflected by a shot target; the light emitters in the plurality of time-of-flight components emit the laser pulses in a time-sharing manner, the light receivers in the plurality of time-of-flight components receive the laser pulses in a time-sharing manner to acquire a panoramic depth image, and the light emitters in the other time-of-flight components are all turned off when the light receivers in any one of the time-of-flight components are exposed.

In the electronic equipment and the mobile platform of the embodiment of the application, the plurality of light emitters in the plurality of different directions of the body emit laser pulses in a time-sharing manner, and the plurality of light receivers expose in a time-sharing manner to acquire a panoramic depth image, so that more comprehensive depth information can be acquired at one time.

Additional aspects and advantages of embodiments of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of embodiments of the present application.

Drawings

The above and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

FIG. 1 is a schematic structural diagram of an electronic device according to some embodiments of the present application;

FIG. 2 is a block diagram of an electronic device according to some embodiments of the present application;

FIG. 3 is a schematic diagram of the time at which a plurality of light emitters time-share emit laser pulses and the time at which a plurality of light receivers time-share expose according to some embodiments of the present application;

FIGS. 4(a) and 4(b) are schematic diagrams of the timing of time-shared emission of laser pulses by multiple light emitters and the timing of time-shared exposure by multiple light receivers of certain embodiments of the present application;

FIGS. 5(a) and 5(b) are schematic diagrams of the timing of time-shared emission of laser pulses by multiple light emitters and the timing of time-shared exposure by multiple light receivers of certain embodiments of the present application;

6(a) -6 (c) are schematic diagrams of the time of time-shared emission of laser pulses by multiple light emitters and the time of time-shared exposure by multiple light receivers according to some embodiments of the present application;

FIG. 7 is a block diagram of an electronic device according to some embodiments of the present application;

FIG. 8 is a schematic diagram of an application scenario of an electronic device according to some embodiments of the present application;

FIG. 9 is a schematic diagram of a coordinate system for initial depth image stitching according to some embodiments of the present application;

fig. 10 to 14 are schematic views of application scenarios of an electronic device according to some embodiments of the present application;

fig. 15-18 are schematic structural views of a mobile platform according to some embodiments of the present disclosure.

Detailed Description

Embodiments of the present application will be further described below with reference to the accompanying drawings. The same or similar reference numbers in the drawings identify the same or similar elements or elements having the same or similar functionality throughout. The embodiments of the present application described below in conjunction with the drawings are exemplary only and should not be construed as limiting the present application.

Referring to fig. 1 and 2 together, an electronic device 100 according to an embodiment of the present disclosure includes a body 10, a time-of-flight assembly 20, a camera assembly 30, a microprocessor 40, and an application processor 50.

The body 10 includes a plurality of different orientations. For example, in fig. 1, the body 10 can have four different orientations, in the clockwise direction: the device comprises a first direction, a second direction, a third direction and a fourth direction, wherein the first direction is opposite to the third direction, and the second direction is opposite to the fourth direction. The first direction is a direction corresponding to the upper side of the body 10, the second direction is a direction corresponding to the right side of the body 10, the third direction is a direction corresponding to the lower side of the body 10, and the fourth direction is a direction corresponding to the left side of the body 10.

The time of flight assembly 20 is disposed on the body 10. The number of time of flight assemblies 20 may be plural, with a plurality of time of flight assemblies 20 being located in a plurality of different orientations of the body 10. In particular, the number of time-of-flight components 20 may be two, respectively time-of-

flight components

20a and 20 b. The time of flight assembly 20a is disposed in a first orientation and the time of flight assembly 20b is disposed in a third orientation. Of course, the number of time-of-flight assemblies 20 may also be four (or any other number greater than two), and two additional time-of-flight assemblies 20 may be provided in the second and fourth orientations, respectively. In the embodiment of the present application, the number of the time-of-flight components 20 is two for illustration, and it can be understood that two time-of-flight components 20 can achieve obtaining of the panoramic depth image (the panoramic depth image means that the field angle of the panoramic depth image is greater than or equal to 180 degrees, for example, the field angle of the panoramic depth image may be 180 degrees, 240 degrees, 360 degrees, 480 degrees, 720 degrees, and the like), which is beneficial to saving the manufacturing cost of the electronic device 100, and reducing the volume and power consumption of the electronic device 100. The electronic device 100 of the present embodiment may be a portable electronic device such as a mobile phone, a tablet computer, and a notebook computer, which is provided with a plurality of time-of-flight components 20, and in this case, the main body 10 may be a mobile phone body, a tablet computer body, a notebook computer body, and the like. For the electronic device 100 with a higher thickness requirement, for example, a mobile phone, because the thickness of the body of the mobile phone is required to be thinner, the time-of-flight components 20 cannot be installed on the side of the body, so the arrangement of using two time-of-flight components 20 to obtain the panoramic depth image can solve the above problem, and at this time, the two time-of-flight components 20 can be installed on the front and the back of the body of the mobile phone, respectively. In addition, the manner in which the two time-of-flight components 20 can acquire the panoramic depth image is also beneficial to reducing the amount of computation of the panoramic depth image. Compared with the method of acquiring the panoramic depth image by setting the structured light assembly, the method has the advantages that the accuracy of measuring the depth of a long-distance shot object by the flight time assembly 20 is high, the method is suitable for measuring the long-distance panoramic depth, the data amount required to be processed when the depth is calculated by the flight time assembly 20 is small, the time required for acquiring the multi-frame panoramic depth image is short, and the method is suitable for application scenes with high requirements on the frame rate of the panoramic depth image.

Each time of flight assembly 20 includes an optical transmitter 22 and an optical receiver 24. The light emitter 22 is used for emitting laser pulses to the outside of the body 10, and the light receiver 24 is used for receiving the laser pulses emitted by the corresponding light emitter 22 reflected by the object to be shot. Specifically, time-of-flight assembly 20a includes an optical transmitter 22a and an optical receiver 24a, and time-of-flight assembly 20b includes an optical transmitter 22b and an optical receiver 24 b. The light emitter 22a and the light emitter 22b are respectively configured to emit laser pulses to a first direction and a third direction outside the body 10, and the light receiver 24a and the light receiver 24b are respectively configured to receive the laser pulses emitted by the light emitter 22a reflected by the object to be photographed in the first direction and the laser pulses emitted by the light emitter 22b reflected by the object to be photographed in the third direction, so as to cover different areas outside the body 10.

The angle of view of each optical receiver 24 is any of 180 to 200 degrees, and the angle of view of each optical transmitter 22 is greater than or equal to the angle of view of the corresponding optical receiver 24. Wherein, the angle of view of the optical transmitter 22 being larger than the angle of view of the corresponding optical receiver 24 means that the angle of view of the optical transmitter 22 is slightly larger than the angle of view of the corresponding optical receiver 24. For example, if the angle of view of the optical receiver 24 is 180 degrees, the corresponding optical transmitter 22 may have an angle of view of 181 degrees, 182.5 degrees, 185 degrees, 187 degrees, 188 degrees, 190 degrees, etc.; the viewing angle of the optical receiver 24 is 200 degrees, and the corresponding optical transmitter 22 may have 200.5 degrees, 201 degrees, 203 degrees, 204 degrees, 207 degrees, 208.6 degrees, 209 degrees, 210 degrees, etc. In the following description, the angle of view of the optical receiver 24 is taken as an example, and the angle of view of the optical transmitter 22 may be greater than or equal to the corresponding angle of view of the optical receiver 24, and the description will not be repeated here.

In one embodiment, the field angles of both

optical receivers

24a and 24b are 180 degrees. When the field angle of the optical receiver 24 is small, the lens distortion is small, and the quality of the obtained initial depth image is good, so that the quality of the obtained panoramic depth image is good, and more accurate depth information can be obtained.

In one embodiment, the sum of the field angles of

optical receivers

24a and 24b is equal to 360 degrees. Specifically, the field angles of the

optical receivers

24a and 24b may both be 180 degrees, and the field angles of the two optical receivers 24 do not overlap with each other, so as to achieve acquisition of a 360-degree or approximately 360-degree panoramic depth image.

In one embodiment, the sum of the field angles of

optical receivers

24a and 24b is greater than 360 degrees, and the field angles of the two optical receivers 24 overlap each other. Specifically, the angles of view of the

optical receivers

24a and 24b may each be 200 degrees, and the angles of view between the two optical receivers 24 overlap each other. When the panoramic depth image is obtained, the edge overlapping parts of the two initial depth images can be identified, and then the two initial depth images are spliced into the 360-degree panoramic depth image. Since the angles of view between the two light receivers 24 overlap, it can be ensured that the acquired panoramic depth image covers 360 degrees of depth information outside the body 10.

Of course, the specific value of the field angle of each optical receiver 24 is not limited to the above example, and those skilled in the art can set the field angle of the optical receiver 24 to any value between 180 degrees and 200 degrees as required, for example: the angle of view of the optical receiver 24 is 180 degrees, 181 degrees, 185 degrees, 187 degrees, 190 degrees, 195 degrees, 196.5 degrees, 198 degrees, 199 degrees, 200 degrees, or any value therebetween, and preferably the angle of view of the optical transmitter 22 is also any value between 180 degrees and 200 degrees, as long as the angle of view of the optical transmitter 22 is greater than or equal to the angle of view of the corresponding optical receiver 24, which is not limited herein.

In general, when the angles of view of the

optical emitters

22a and 22b overlap, if the

optical emitters

22a and 22b emit laser pulses at the same time, the laser pulses emitted by the

optical emitters

22a and 22b are likely to interfere with each other. Thus, to improve the accuracy of the acquired depth information, the light emitters 22 in the two time-of-flight assemblies 20 may time-share the emitted laser pulses, and correspondingly, the light receivers 24 in the two time-of-flight assemblies 20 are also time-shared exposed to receive the emitted laser pulses from the corresponding light emitters 22 to acquire the panoramic depth image. Wherein the optical transmitter 22 in the other time-of-flight assembly 20 is turned off while the optical receiver 24 in any one of the time-of-flight assemblies 20 is exposed. In this way, the optical receiver 24 can only receive the laser pulses emitted by the corresponding optical transmitter 22, and does not collect the laser pulses emitted by the remaining optical transmitters 22, so that the above-mentioned interference problem can be avoided, and the accuracy of the received laser pulses can be ensured.

Specifically, referring to fig. 3 and 4, in one embodiment, the two light emitters 22 of the two time-of-flight components 20 emit laser pulses sequentially and continuously, and the exposure time of the light receiver 24 of each time-of-flight component 20 is within the time range of the light emitter 22 emitting the laser pulses. The light emitter 22a emits the laser pulse in time-sharing with the light emitter 22b, and the light emitter 22b immediately starts emitting the laser pulse from the timing at which the light emitter 22a stops emitting the laser pulse, and the light emitter 22a immediately starts emitting the laser pulse from the timing at which the light emitter 22b stops emitting the laser pulse. The time at which light emitter 22a and light emitter 22b emit laser pulses together constitute an alternating period T. At this time, the exposure modes of the light receiver 24a and the light receiver 24b may include the following two types:

(1) the light receiver 24a and the light receiver 24b are sequentially and continuously exposed. Specifically, the exposure times of the two photoreceivers 24 respectively coincide with the times at which the corresponding phototransmitters 22 emit laser pulses. As shown in fig. 3, the

light receivers

24a and 24b are alternately exposed in sequence. The exposure start time of the photoreceiver 24a coincides with the start time of the laser pulse emitted by the phototransmitter 22a of the current alternation period T, the exposure cutoff time of the photoreceiver 24a coincides with the cutoff time of the laser pulse emitted by the phototransmitter 22a of the current alternation period T, the exposure start time of the photoreceiver 24b coincides with the start time of the laser pulse emitted by the phototransmitter 22b of the current alternation period T, and the exposure cutoff time of the photoreceiver 24b coincides with the cutoff time of the laser pulse emitted by the phototransmitter 22b of the current alternation period T. At this time, the optical receiver 24a can only receive the laser pulse emitted by the optical transmitter 22a, and cannot collect the laser pulse emitted by the optical transmitter 22 b; the optical receiver 24b can only receive the laser pulses emitted by the optical transmitter 22b, but not the laser pulses emitted by the optical transmitter 22 a. In the control mode of sequentially and continuously exposing the light receiver 24a and the light receiver 24b, the light receiver 24a and the light emitter 22a are synchronously controlled, the light receiver 24b and the light emitter 22b are synchronously controlled, and the control logic is simpler.

(2) As shown in fig. 4, the light receiver 24a and the light receiver 24b are exposed sequentially and at predetermined intervals. Wherein the exposure time of at least one of the light receivers 24 is less than the time that the corresponding light emitter 22 emits a laser pulse. Specifically, as shown in fig. 4(a), in one example, the light receiver 24a and the light receiver 24b are alternately exposed in sequence. The exposure time of the optical receiver 24a is less than the time of the optical transmitter 22a emitting the laser pulse, the exposure time of the optical receiver 24b is equal to the time of the optical transmitter 22b emitting the laser pulse, the exposure start time of the optical receiver 24a is greater than the start time of the optical transmitter 22a emitting the laser pulse of the current alternation period T, the exposure cut-off time is less than the cut-off time of the optical transmitter 22a emitting the laser pulse of the current alternation period T, and the optical receiver 24bThe exposure start time and the exposure stop time coincide with the start time and the stop time of the laser pulse emitted by the light emitter 22b of the current alternation period T, respectively, and the exposure stop time of the light receiver 24a and the exposure start time of the light receiver 24b of the current alternation period T are separated by a predetermined time delta T₁The predetermined time Δ T is set between the exposure off time of the light receiver 24b and the exposure start time of the light receiver 24a of the next alternation period T₂，Δt₁And Δ t₂May be equal or different. The optical receiver 24a can only receive the laser pulses emitted by the optical transmitter 22a, and the optical receiver 24b can only receive the laser pulses emitted by the optical transmitter 22 b. As shown in fig. 4(b), in another example, the light receiver 24a and the light receiver 24b are alternately exposed in sequence. The exposure time of the optical receiver 24a is less than the time of the optical transmitter 22a emitting the laser pulse, the exposure time of the optical receiver 24b is also less than the time of the optical transmitter 22b emitting the laser pulse, the exposure start time of the optical receiver 24a is greater than the start time of the optical transmitter 22a emitting the laser pulse of the current alternation period T, the exposure cut-off time is less than the cut-off time of the optical transmitter 22a emitting the laser pulse of the current alternation period T, the exposure start time of the optical receiver 24b is greater than the start time of the optical transmitter 22b emitting the laser pulse of the current alternation period T, the exposure cut-off time is less than the cut-off time of the optical transmitter 22b emitting the laser pulse of the current alternation period T, and the interval between the exposure cut-off time of the optical receiver 24a and the exposure₁The predetermined time Δ T is set between the exposure off time of the light receiver 24b and the exposure start time of the light receiver 24a of the next alternation period T₂，Δt₁And Δ t₂May be equal or different. The optical receiver 24a can only receive the laser pulses emitted by the optical transmitter 22a, and the optical receiver 24b can only receive the laser pulses emitted by the optical transmitter 22 b. In the control mode of the

light receivers

24a and 24b sequentially and alternately exposing at the predetermined time, the exposure time of at least one light receiver 24 is shorter than the time of the corresponding light emitter 22 emitting the laser pulse, which is beneficial to reducing the power consumption of the electronic device 100.

In the control mode in which the two light emitters 22 in the two time-of-flight assemblies 20 sequentially and continuously emit laser pulses, the frame rate at which the time-of-flight assemblies 20 acquire the initial depth images is higher, and the method is suitable for scenes with higher requirements on the frame rate at which the initial depth images are acquired.

Referring to fig. 5 and 6, in another embodiment, the two light emitters 22 of the two time-of-flight assemblies 20 sequentially emit laser pulses at predetermined time intervals, that is, the light emitter 22a and the light emitter 22b alternately emit laser pulses, and the off-time of the light emitter 22a emitting laser pulses is separated from the start-time of the light emitter 22b emitting laser pulses in the current alternating period T by the predetermined time Δ T₃The predetermined time Δ T is set between the cut-off time at which the light emitter 22b emits the laser pulse and the start time at which the light emitter 22a emits the laser pulse in the next alternation period T₄，Δt₃Can be compared with Δ t₄Equal or unequal, wherein the time at which the

light emitters

22a and 22b emit the laser pulses, and the predetermined time Δ t₃And a predetermined time Δ t₄Together constituting an alternating period T. At this time, the exposure modes of the light receiver 24a and the light receiver 24b may include the following two types:

(1) the light receiver 24a and the light receiver 24b are sequentially and continuously exposed. Specifically, as shown in fig. 5(a), in one example, the exposure start time of the photoreceiver 24a coincides with the start time of the laser pulse emitted by the phototransmitter 22a of the current alternating period T, the exposure off-time coincides with the off-time of the laser pulse emitted by the phototransmitter 22a of the current alternating period T, and the exposure start time of the photoreceiver 24b coincides with the off-time of the laser pulse emitted by the phototransmitter 22a of the current alternating period T, and the exposure off-time coincides with the start time of the laser pulse emitted by the phototransmitter 22a of the next alternating period T. The optical receiver 24a can only receive the laser pulses emitted by the optical transmitter 22a, and the optical receiver 24b can only receive the laser pulses emitted by the optical transmitter 22 b. As shown in fig. 5(b), in another example, the exposure start time of the photoreceiver 24a coincides with the start time of the laser pulse emitted by the phototransmitter 22a of the current alternation period T, the exposure off time coincides with the start time of the laser pulse emitted by the phototransmitter 22b of the current alternation period T, the exposure start time of the photoreceiver 24b coincides with the start time of the laser pulse emitted by the phototransmitter 22b of the current alternation period T, and the exposure off time coincides with the start time of the laser pulse emitted by the phototransmitter 22a of the next alternation period T. The optical receiver 24a can only receive the laser pulses emitted by the optical transmitter 22a, and the optical receiver 24b can only receive the laser pulses emitted by the optical transmitter 22 b.

(2) The light receiver 24a and the light receiver 24b are exposed sequentially and at predetermined intervals. Specifically, as shown in fig. 6(a), in one example, the exposure start time and the exposure off time of the photoreceiver 24a coincide with the start time and the off time, respectively, at which the phototransmitter 22a of the current alternating period T emits the laser pulse, and the exposure start time and the exposure off time of the photoreceiver 24b coincide with the start time and the off time, respectively, at which the phototransmitter 22b of the current alternating period T emits the laser pulse, and the exposure off time of the photoreceiver 24a is spaced from the exposure start time of the photoreceiver 24b of the current alternating period T by the predetermined time Δ T₃The exposure off time of the light receiver 24b is separated from the exposure start time of the light receiver 24a of the next alternation period T by a predetermined time Δ T₄. The optical receiver 24a can only receive the laser pulses emitted by the optical transmitter 22a, and the optical receiver 24b can only receive the laser pulses emitted by the optical transmitter 22 b. As shown in fig. 6(b), in another example, the exposure start time and the exposure off time of the photoreceiver 24a coincide with the start time and the off time of the laser pulse emitted from the phototransmitter 22a of the current alternation period T, respectively, and the exposure start time of the photoreceiver 24b is greater than the off time of the laser pulse emitted from the phototransmitter 22a of the current alternation period T, and the exposure off time is less than the start time of the laser pulse emitted from the phototransmitter 22a of the next alternation period T. The exposure off time of the light receiver 24a is separated from the exposure start time of the light receiver 24b of the current alternation period T by a predetermined time Δ T₅The exposure off time of the light receiver 24b is separated from the exposure start time of the light receiver 24a of the next alternation period T by a predetermined time Δ T₆，Δt₅And Δ t₆May be equal or different. The optical receiver 24a can only receive the laser pulses emitted by the optical transmitter 22a, and the optical receiver 24b can only receive the laser pulses emitted by the optical transmitter 22 b. As shown in fig. 6(c), in yet another example, the exposure start time of the light receiver 24a is greater than the cut-off time of the light emitter 22b emitting the laser pulse of the previous alternating period T, the exposure cut-off time is less than the start time of the light emitter 22b emitting the laser pulse of the current alternating period T, and the exposure start time of the light receiver 24b is greater than the exposure cut-off time of the light receiver 24a of the current alternating period T, and the exposure cut-off time of the light receiver 24b is less than the exposure start time of the light receiver 24a of the next alternating period T. The exposure off time of the light receiver 24a is separated from the exposure start time of the light receiver 24b of the current alternation period T by a predetermined time Δ T₅The exposure off time of the light receiver 24b is separated from the exposure start time of the light receiver 24a of the next alternation period T by a predetermined time Δ T₆，Δt₅And Δ t₆May be equal or different. The optical receiver 24a can only receive the laser pulses emitted by the optical transmitter 22a, and the optical receiver 24b can only receive the laser pulses emitted by the optical transmitter 22 b.

In the control mode that the two light emitters 22 in the two time-of-flight components 20 sequentially continue to emit laser pulses at predetermined intervals, the frame rate of the initial depth image acquired by the time-of-flight components 20 is low, which is suitable for a scene with low requirement on the frame rate of the initial depth image acquired, and is beneficial to reducing the power consumption of the electronic device 100.

Referring to fig. 1 and 2, a camera assembly 30 is disposed on the body 10. The number of camera assemblies 30 may be multiple, one time-of-flight assembly 20 for each camera assembly 30. For example, when the number of time-of-flight components 20 is two, the number of camera assemblies 30 is also two, and the two camera assemblies 30 are respectively disposed in the first orientation and the third orientation.

A plurality of camera head assemblies 30 are each connected to an application processor 50. Each camera assembly 30 is used to capture a scene image of a subject and output to the application processor 50. In the present embodiment, the two camera assemblies 30 are respectively used for capturing the scene image of the subject in the first orientation and the scene image of the subject in the third orientation and outputting the captured images to the application processor 50. It will be appreciated that the field angle of each camera assembly 30 is the same or approximately the same as the optical receiver 24 of the corresponding time-of-flight assembly 20 to enable a better match of each scene image with the corresponding initial depth image.

The camera assembly 30 may be a visible light camera 32 or an infrared light camera 34. When camera assembly 30 is a visible light camera 32, the scene image is a visible light image; when camera assembly 30 is an infrared camera 34, the scene image is an infrared light image.

Referring to fig. 2 and 7, the microprocessor 40 may be a processing chip.

In one embodiment, as shown in FIG. 2, the number of microprocessors 40 may be multiple, with one time-of-flight component 20 for each microprocessor 40. For example, in the present embodiment, the number of time-of-flight components 20 is two, and the number of microprocessors 40 is also two. Each microprocessor 40 is connected to both the optical transmitter 22 and the optical receiver 24 in the corresponding time of flight assembly 20. Each microprocessor 40 can drive the corresponding light emitter 22 to emit laser pulses through the driving circuit, and the two light emitters 22 can emit laser pulses in a time-sharing manner through the control of the two microprocessors 40. Each microprocessor 40 is also used for providing clock information of the received laser pulse to the corresponding light receiver 24 to expose the light receiver 24, and realizing time-sharing exposure of the two light receivers 24 through the control of the two microprocessors 40. Each microprocessor 40 is also configured to derive an initial depth image from the laser pulses emitted by the corresponding light emitter 22 and the laser pulses received by the light receiver 24. For example, the two microprocessors 40 obtain an initial depth image P1 according to the laser pulses emitted by the light emitter 22a and received by the light receiver 24a, and an initial depth image P2 according to the laser pulses received by the laser pulse light receiver 24b emitted by the light emitter 22b, respectively (as shown in the upper part of fig. 8). Each microprocessor 40 may also perform algorithm processing such as tiling, distortion correction, self-calibration, etc. on the initial depth image to improve the quality of the initial depth image.

In another embodiment, as shown in FIG. 7, the number of microprocessors 40 may also be one. At this time, the microprocessor 40 is simultaneously connected to the light emitter 22a, the light receiver 24a, the light emitter 22b, and the light receiver 24 b. One microprocessor 40 can control two different driving circuits in a time-sharing manner to respectively drive the corresponding light emitters 22 to emit laser pulses, and can also provide clock information for receiving the laser pulses to the two light receivers 24 in a time-sharing manner to enable the light receivers 24 to expose in a time-sharing manner, and obtain an initial depth image P1 according to the laser pulses emitted by the light emitter 22a and the laser pulses received by the light receiver 24a, and obtain an initial depth image P2 according to the laser pulses received by the laser pulse light receiver 24b emitted by the light emitter 22b in sequence (as shown in the upper part of fig. 8). Two microprocessors 40 have faster processing speed and less latency than one microprocessor 40. However, one microprocessor 100 is advantageous in reducing the size of the electronic device 100 and in reducing the manufacturing cost of the electronic device 100, compared to two microprocessors 40 in one microprocessor 40.

When there are two microprocessors 40, both microprocessors 40 are connected to the application processor 50 to transmit the initial depth image to the application processor 50. Where there is one microprocessor 40, one microprocessor 40 is connected to the application processor 50 to transmit the initial depth image to the application processor 50. In one embodiment, the microprocessor 40 may be connected to the application Processor 50 through a Mobile Industry Processor Interface (MIPI), and specifically, the microprocessor 40 is connected to a Trusted Execution Environment (TEE) of the application Processor 50 through the Mobile Industry Processor Interface, so as to directly transmit data (initial depth image) in the microprocessor 40 to the TEE, so as to improve the security of information in the electronic device 100. Here, both the code and the memory area in the trusted Execution Environment are controlled by the access control unit and cannot be accessed by a program in the untrusted Execution Environment (REE), and both the trusted Execution Environment and the untrusted Execution Environment may be formed in the application processor 50.

The application processor 50 may function as a system of the electronic device 100. The application processor 50 may reset the microprocessor 40, wake the microprocessor 40, debug the microprocessor 40, and so on. The application processor 50 may also be connected to a plurality of electronic components of the electronic device 100 and control the plurality of electronic components to operate according to a predetermined mode, for example, the application processor 50 is connected to the visible light camera 32 and the infrared light camera 34 to control the visible light camera 32 and the infrared light camera 34 to capture a visible light image and an infrared light image and process the visible light image and the infrared light image; when the electronic apparatus 100 includes a display screen, the application processor 50 may control the display screen to display a predetermined screen; the application processor 50 may also control an antenna of the electronic device 100 to transmit or receive predetermined data or the like.

Referring to fig. 8, the application processor 50 may be configured to synthesize two initial depth images obtained by the two microprocessors 40 into one frame of panoramic depth image according to the field angle of the optical receiver 24; alternatively, the application processor 50 may be configured to combine two initial depth images sequentially acquired by one microprocessor 40 into one frame of the panoramic depth image according to the field angle of the optical receiver 24.

Specifically, referring to fig. 1 and 9, a rectangular coordinate system XOY is established with the center of the body 10 as a center O, the transverse axis as an X axis, and the longitudinal axis as a Y axis, in the rectangular coordinate system XOY, the field of view of the light receiver 24a is located between 350 degrees and 190 degrees (counterclockwise rotation, the same applies), and the field of view of the light receiver 24b is located between 170 degrees and 10 degrees, and then the application processor 50 stitches the initial depth image P1 and the initial depth image P2 into a 360-degree panoramic depth image P12 of one frame according to the field angles of the two light receivers 24, so as to use the depth information.

In the initial depth image obtained by the microprocessor 40 according to the laser pulse emitted by the light emitter 22 and the laser pulse received by the light receiver 24, the depth information of each pixel is the distance between the subject in the corresponding direction and the light receiver 24 in the direction. That is, the depth information of each pixel in the initial depth image P1 is the distance between the subject in the first orientation and the light receiver 24 a; the depth information of each pixel in the initial depth image P2 is the distance between the subject in the third orientation and the light receiver 24 b. In the process of splicing a plurality of initial depth images of a plurality of azimuths into a 360-degree panoramic depth image of one frame, firstly, depth information of each pixel in each initial depth image is converted into unified depth information, and the unified depth information represents the distance between each shot target of each azimuth and a certain reference position. After the depth information is converted into the unified depth information, the application processor 40 is convenient to perform the splicing of the initial depth image according to the unified depth information.

Specifically, one reference coordinate system is selected, and the reference coordinate system may be an image coordinate system of the light receiver 24 in a certain direction as the reference coordinate system, or another coordinate system may be selected as the reference coordinate system. Take FIG. 9 as an example, take x_o-y_o-z_oThe coordinate system is a reference coordinate system. Coordinate system x shown in fig. 9_a-y_a-z_aIs the image coordinate system of the light receiver 24a, coordinate system x_b-y_b-z_bIs the image coordinate system of the light receiver 24 b. The application processor 50 is based on a coordinate system x_a-y_a-z_aWith reference coordinate system x_o-y_o-z_oThe rotation matrix and the translation matrix between convert the depth information of each pixel in the initial depth image P1 into unified depth information according to the coordinate system x_b-y_b-z_bWith reference coordinate system x_o-y_o-z_oThe rotation matrix and the translation matrix in between convert the depth information of each pixel in the initial depth image P2 into unified depth information.

After the depth information conversion is completed, a plurality of initial depth images are positioned in a uniform reference coordinate system, and one pixel point of each initial depth image corresponds to one coordinate (x)_o,y_o,z_o) Then the stitching of the initial depth images can be done by coordinate matching. For example, a certain pixel point P in the initial depth image P1_aHas the coordinates of (x)_o1,y_o1,z_o1) In the initial depth image P2, a certain pixel point P_bAlso has the coordinate of (x)_o1,y_o1,z_o1) Due to the fact thatP_aAnd P_bIf the coordinate values are the same in the current reference coordinate system, the pixel point P is indicated_aAnd pixel point P_bWhen the initial depth image P1 and the initial depth image P2 are spliced at the same point, a pixel point P is_aNeeds and pixel point P_bAnd (4) overlapping. In this way, the application processor 50 can perform stitching of multiple initial depth images according to the matching relationship of the coordinates, and obtain a 360-degree panoramic depth image.

It should be noted that, performing the stitching of the initial depth image based on the matching relationship of the coordinates requires that the resolution of the initial depth image needs to be greater than a preset resolution. It can be appreciated that if the resolution of the initial depth image is low, the coordinate (x) is_o,y_o,z_o) Will also be relatively low, in which case matching directly from the coordinates may occur P_aPoint sum P_bThe points do not actually coincide but differ by an offset, and the value of the offset exceeds the error limit. If the resolution of the image is high, the coordinate (x)_o,y_o,z_o) Will be relatively high, in which case the matching is done directly from the coordinates, even if P is_aPoint sum P_bThe points are not actually overlapped and differ by an offset, but the value of the offset is smaller than an error limit value, namely, the offset is within an error allowable range, and the splicing of the initial depth image cannot be greatly influenced.

It is understood that the subsequent embodiments may adopt the above-mentioned manner to splice or synthesize the two initial depth images, and are not described one by one.

The application processor 50 may also synthesize the two initial depth images and the two corresponding visible light images into a three-dimensional scene image for display for the user to view. For example, if the two visible light images are the visible light image V1 and the visible light image V2, the application processor 50 synthesizes the initial depth image P1 and the visible light image V1, synthesizes the initial depth image P2 and the visible light image V2, and then splices the two synthesized images to obtain a 360-degree three-dimensional scene image of one frame. Or, the application processor 50 firstly splices the initial depth image P1 and the initial depth image P2 to obtain a frame of 360-degree panoramic depth image, and splices the visible light image V1 and the visible light image V2 to obtain a frame of 360-degree panoramic visible light image; and then the panoramic depth image and the panoramic visible light image are synthesized into a 360-degree three-dimensional scene image.

Referring to fig. 10, in an embodiment, the application processor 50 is configured to identify the object to be shot according to two initial depth images respectively acquired by two microprocessors 40 and two scene images acquired by two camera assemblies 30, or according to two initial depth images sequentially acquired by one microprocessor 40 and two scene images acquired by two camera assemblies 30.

Specifically, when the scene image is an infrared light image, the two infrared light images may be the infrared light image I1 and the infrared light image I2, respectively. The application processor 50 identifies a photographic subject in a first orientation from the initial depth image P1 and the infrared light image I1, and a photographic subject in a third orientation from the initial depth image P2 and the infrared light image I2, respectively. When the scene image is a visible light image, the two visible light images are the visible light image V1 and the visible light image V2, respectively. The application processor 50 identifies a photographic subject in a first orientation from the initial depth image P1 and the visible light image V1, and a photographic subject in a third orientation from the initial depth image P2 and the visible light image V2, respectively.

When the photographic subject is identified as face recognition, the application processor 50 performs face recognition with higher accuracy using the infrared light image as the scene image. The process of face recognition by the application processor 50 from the initial depth image and the infrared light image may be as follows:

firstly, face detection is carried out according to the infrared light image to determine a target face area. Because the infrared light image comprises the detail information of the scene, after the infrared light image is acquired, the human face detection can be carried out according to the infrared light image, so that whether the infrared light image contains the human face or not can be detected. And if the infrared light image contains the human face, extracting a target human face area where the human face is located in the infrared light image.

Then, the living body detection processing is performed on the target face region according to the initial depth image. Each initial depth image corresponds to the infrared light image, and the initial depth image comprises the depth information of the corresponding infrared light image, so that the depth information corresponding to the target face area can be obtained according to the initial depth image. Further, since the living body face is stereoscopic and the face displayed, for example, on a picture, a screen, or the like, is planar, it is possible to determine whether the target face region is stereoscopic or planar according to the acquired depth information of the target face region, thereby performing living body detection on the target face region.

And if the living body detection is successful, acquiring target face attribute parameters corresponding to the target face area, and performing face matching processing on the target face area in the infrared light image according to the target face attribute parameters to obtain a face matching result. The target face attribute parameters refer to parameters capable of representing attributes of a target face, and the target face can be identified and matched according to the target face attribute parameters. The target face attribute parameters include, but are not limited to, face deflection angles, face brightness parameters, facial features parameters, skin quality parameters, geometric feature parameters, and the like. The electronic apparatus 100 may previously store the face attribute parameters for matching. After the target face attribute parameters are acquired, the target face attribute parameters can be compared with the face attribute parameters stored in advance. And if the target face attribute parameters are matched with the pre-stored face attribute parameters, the face recognition is passed.

It should be noted that the specific process of the application processor 50 performing face recognition according to the initial depth image and the infrared light image is not limited to this, for example, the application processor 50 may also assist in detecting a face contour according to the initial depth image to improve face recognition accuracy, and the like. The process of the application processor 50 performing face recognition based on the initial depth image and the visible light image is similar to the process of the application processor 50 performing face recognition based on the initial depth image and the infrared light image, and will not be further described herein.

Referring to fig. 10 and 11, the application processor 50 is further configured to combine the two initial depth images acquired by the two microprocessors 40 into one merged depth image according to the field angle of the optical receiver 24 when the identification of the object fails according to the two initial depth images and the two scene images, combine the two scene images acquired by the two camera assemblies 30 into one merged scene image, and identify the object according to the merged depth image and the merged scene image; alternatively, the application processor 50 is further configured to combine two initial depth images sequentially acquired by one microprocessor 40 into one frame of combined depth image according to the field angle of the optical receiver 24, combine two scene images acquired by two camera assemblies 30 into one frame of combined scene image, and identify the target according to the combined depth image and the combined scene image, when the target identification fails according to the two initial depth images and the two scene images.

Specifically, in the embodiment shown in fig. 10 and 11, since the field angle of the light receiver 24 of each time-of-flight component 20 is limited, and there may be a case where half of the human face is located in the initial depth image P1 and the other half is located in the initial depth image P2, the application processor 50 synthesizes the initial depth image P1 and the initial depth image P2 into one frame of merged depth image P12, and correspondingly synthesizes the infrared light image I1 and the infrared light image I2 (or the visible light image V1 and the visible light image V2) into one frame of merged scene image I12 (or V12), so as to re-identify the object to be photographed from the merged depth image P12 and the merged scene image I12 (or V12).

Referring to fig. 12 and 13, in an embodiment, the application processor 50 is configured to determine a distance variation between the subject and the electronic device 100 according to a plurality of initial depth images.

Specifically, each light receiver 24 may receive laser light pulses multiple times. For example, at a first time t₁The optical receiver 24a receives the laser pulse at a second time t₂The optical receiver 24b receives the laser pulse (first time t)₁And a second time t₂In the same alternating period T), at this time, two microprocessors 40 correspondingly obtain an initial depth image P11 and an initial depth image P21, or one microprocessor 40 sequentially obtains an initial depth image P11 and an initial depth image P21; at a third time t₃The optical receiver 24a receives the laser pulse at a fourth time t₄The optical receiver 24b receives the laser pulse (third time t)₃And a fourth time t₄In the same alternation period T), two microprocessors 40 obtain the initial depth image P12 and the initial depth image P22 correspondingly, or one microprocessor 40 obtains the initial depth image P12 and the initial depth image P22 sequentially. Then, the application processor 50 determines a distance change between the subject in the first orientation and the electronic device 100 from the initial depth image P11 and the initial depth image P12, and determines a distance change between the subject in the third orientation and the electronic device 100 from the initial depth image P21 and the initial depth image P22, respectively.

It is understood that, since the depth information of the subject is included in the initial depth image, the application processor 50 may determine a distance change between the subject corresponding to the orientation and the electronic apparatus 100 from a depth information change at a plurality of consecutive times.

Referring to fig. 14, the application processor 50 is further configured to combine two initial depth images acquired by two microprocessors 40 into one merged depth image according to the field angle of the optical receiver 24 when determining that the distance change fails according to the multiple initial depth images, where the application processor 50 continuously performs a combining step to obtain multiple frames of continuous merged depth images, and determines the distance change according to the multiple frames of merged depth images; alternatively, the application processor 50 is further configured to combine two initial depth images sequentially acquired by one microprocessor 40 into one merged depth image according to the field angle of the optical receiver 24 when determining that the distance change fails according to the multiple initial depth images, and the application processor 50 continuously performs the combining step to obtain multiple frames of continuous merged depth images and determines the distance change according to the multiple frames of merged depth images.

In particular, in the embodiment shown in fig. 14, since the optical receiver 24 of each time-of-flight component 20 has a limited field of view, there may be a situation where half of the human face is located in the initial depth image P11 and the other half is located in the initial depth image P21, and the application processor 50 will determine the first time t₁P11 and a second time t₂Is synthesized into a frame of merged depth image P121, and is correspondingly combined with the merged depth image P21Third time t₃P12 and a fourth time t₄The initial depth image P22 is synthesized into one frame of the merged depth image P122, and then the distance change is newly judged from the two frames of the merged depth images P121 and P122 after merging.

Referring to fig. 14, when it is determined that the distance change is a distance decrease according to the plurality of initial depth images or when it is determined that the distance change is a distance decrease according to the multi-frame merged depth image, the application processor 50 increases a frame rate of the initial depth images collected from the plurality of initial depth images transmitted from the microprocessor 40 for determining the distance change. Specifically, when the number of the microprocessors 40 is multiple, the application processor 50 may increase the frame rate of the initial depth images collected from the multiple initial depth images transmitted by at least one of the microprocessors 40 to determine the distance change; when the number of the microprocessors 40 is one, the application processor 50 increases the frame rate of the initial depth images collected from the initial depth images transmitted by the microprocessors 40 to determine the distance change.

It can be understood that when the distance between the subject and the electronic device 100 decreases, the electronic device 100 cannot predict whether the distance decrease is dangerous, and therefore, the application processor 50 may increase the frame rate of the initial depth image collected from the plurality of initial depth images transmitted by the microprocessor 40 to determine the distance change, so as to focus on the distance change more closely. Specifically, when determining that the distance corresponding to a certain orientation decreases, the application processor 50 may increase the frame rate of the initial depth image acquired from the plurality of initial depth images transmitted by the microprocessor 40 for determining the distance change in the orientation.

For example, at a first time t₁A microprocessor 40 obtains an initial depth image P11 at a second time t₂The other microprocessor 40 obtains an initial depth image P21 (or one microprocessor 40 at a first time t₁Obtaining an initial depth image P11 and at a second time t₂Obtaining an initial depth image P21); at a third time t₃A microprocessor 40 obtains an initial depth image P12 at a fourth time t₄Another microprocessor 40 obtains an initial depth image P22 (or a microprocessor 40 at a third time t)₃Obtaining an initial depth image P12 and at a fourth time t₄Obtaining an initial depth image P22); at a fifth time t₅A microprocessor 40 obtains an initial depth image P13 at a sixth time t₆The other microprocessor 40 obtains an initial depth image P23 (or one microprocessor 40 at a fifth time t₅Obtaining an initial depth image P13 and at a sixth time t₆Obtaining an initial depth image P23); at a seventh instant t₇A microprocessor 40 obtains an initial depth image P14 at an eighth time t₈The other microprocessor 40 obtains an initial depth image P24 (or one microprocessor 40 at a seventh instant t)₇Obtaining an initial depth image P14 and at an eighth time t₈The initial depth image P24 is obtained). Wherein the first time t₁And a second time t₂Within the same alternating period T, at a third time T₃And a fourth time t₄Within the same alternating period T, at a fifth time T₅And a sixth time t₆Within the same alternating period T, at a seventh time T₇And an eighth time t₈In the same alternating period T.

Under normal circumstances, the application processor 50 selects an initial depth image P11 and an initial depth image P14 to judge the distance change between the subject at the first orientation and the electronic device 100; the initial depth image P21 and the initial depth image P24 are selected to judge the distance change between the subject in the third direction and the electronic device 100. The frame rate of the application processor 50 for acquiring the initial depth image in each direction is one frame acquired every two frames, that is, one frame is selected every three frames.

When the distance corresponding to the first direction is determined to decrease according to the initial depth image P11 and the initial depth image P14, the application processor 50 selects the initial depth image P11 and the initial depth image P13 to determine the distance between the subject in the first direction and the electronic device 100. The frame rate at which the application processor 50 acquires the initial depth image of the first orientation is changed to acquire one frame every other frame, i.e., one frame is selected every two frames. While the frame rates of other orientations remain the same, i.e. the application processor 50 still selects the initial depth image P21 and the initial depth image P24 to determine the distance change.

Of course, the application processor 50 may increase the frame rate of the initial depth image collected from the plurality of initial depth images of each azimuth transmitted from the microprocessor 40 to determine the distance change when determining that the distance corresponding to any azimuth decreases. Namely: when the distance between the subject in the first position and the electronic device 100 is determined to be decreased according to the initial depth image P11 and the initial depth image P14, the application processor 50 selects the initial depth image P11 and the initial depth image P13 to determine the distance change between the subject in the first position and the electronic device 100, and selects the initial depth image P21 and the initial depth image P23 to determine the distance change between the subject in the third position and the electronic device 100.

The application processor 50 may also determine the change in distance as the distance decreases, in conjunction with the visible light image or the infrared light image. Specifically, the application processor 50 identifies the photographic subject from the visible light image or the infrared light image, and then determines the distance change from the initial depth image at a plurality of times, thereby controlling the electronic apparatus 100 to perform different operations with respect to different photographic subjects and different distances. Alternatively, the microprocessor 40 controls the frequency of the laser pulses emitted by the corresponding light emitter 22 and the exposure of the light receiver 24 to be increased, etc., as the distance decreases.

It should be noted that the electronic device 100 of the present embodiment may also be used as an external terminal, and may be fixedly mounted or detachably mounted on a portable electronic device such as a mobile phone, a tablet computer, a notebook computer, etc., or may be fixedly mounted on a movable object such as a vehicle body (as shown in fig. 13 and 14), an unmanned aerial vehicle body, a robot body, or a ship body. When the electronic device 100 is used specifically, a frame of panoramic depth image is synthesized according to the plurality of initial depth images as described above, and the panoramic depth image may be used for three-dimensional modeling, instant positioning and mapping (SLAM), and augmented reality display. When the electronic device 100 recognizes a subject as described above, the method may be applied to face recognition unlocking and payment of a portable electronic device, or applied to obstacle avoidance of a robot, a vehicle, an unmanned aerial vehicle, a ship, or the like. When the electronic apparatus 100 determines that the distance between the subject and the electronic apparatus 100 changes as described above, the present invention can be applied to automatic travel, object tracking, and the like of robots, vehicles, unmanned planes, ships, and the like.

Referring to fig. 2 and fig. 15, the present embodiment further provides a mobile platform 300. The mobile platform 300 includes a body 10 and a plurality of time-of-flight assemblies 20 disposed on the body 10. The plurality of time of flight assemblies 20 are respectively located at a plurality of different orientations of the body 10. Each time of flight assembly 20 includes an optical transmitter 22 and an optical receiver 24. The angle of view of each optical receiver 24 is any of 180 to 200 degrees, and the angle of view of each optical transmitter 22 is greater than or equal to the angle of view of the optical receiver 24. The light emitter 22 is used for emitting laser pulses to the outside of the body 10, and the light receiver 24 is used for receiving the laser pulses emitted by the corresponding light emitter 22 reflected by the object to be shot. The optical transmitters 22 in the plurality of time-of-flight components 20 time-share laser pulses and the optical receivers 24 in the plurality of time-of-flight components 20 time-share exposures to acquire panoramic depth images.

Specifically, the body 10 may be a vehicle body, an unmanned aerial vehicle fuselage, a robot body, or a ship body.

Referring to fig. 15, when the body 10 is a vehicle body, the number of the plurality of time-of-flight assemblies 20 is two, and the two time-of-flight assemblies 20 are respectively installed at two sides of the vehicle body, for example, at a front end and a rear end of the vehicle body, or at a right side and a left side of the vehicle body. The vehicle body can drive the two flight time assemblies 20 to move on the road, and a 360-degree panoramic depth image on a traveling route is constructed to be used as a reference map and the like; or acquiring a plurality of initial depth images in different directions to identify the photographed target, and determining the distance change between the photographed target and the mobile platform 300, so as to control the vehicle body to accelerate, decelerate, stop, detour, and the like, thereby implementing unmanned obstacle avoidance. In this way, different operations are performed according to different photographic subjects when the distance decreases, and the vehicle can be made more intelligent.

Referring to fig. 16, when the main body 10 is an unmanned aerial vehicle body, the number of the plurality of time of flight assemblies 20 is two, and the two time of flight assemblies 20 are respectively installed on two opposite sides of the unmanned aerial vehicle body, such as the front side and the rear side or the left side and the right side, or on two opposite sides of a cradle head carried on the unmanned aerial vehicle body. The unmanned aerial vehicle fuselage can drive a plurality of flight time subassemblies 20 and fly in the air to take photo by plane, patrol and examine etc. unmanned aerial vehicle can return the panorama depth image who obtains and give ground control end, also can directly carry out SLAM. A plurality of time of flight components 20 can realize that unmanned aerial vehicle accelerates, decelerates, stops, keeps away barrier, object tracking.

Referring to fig. 17, when the main body 10 is a robot main body, such as a sweeping robot, the number of the plurality of time-of-flight assemblies 20 is two, and the two time-of-flight assemblies 20 are respectively installed on two opposite sides of the robot main body. The robot body can drive the plurality of flight time assemblies 20 to move at home, and initial depth images in a plurality of different directions are acquired so as to identify a shot target and judge the distance change between the shot target and the mobile platform 300, so that the robot body is controlled to move, and the robot is enabled to clear away garbage, avoid obstacles and the like.

Referring to fig. 18, when the body 10 is a ship body, the number of the plurality of time-of-flight assemblies 20 is two, and the two time-of-flight assemblies 20 are respectively installed at two opposite sides of the ship body. The ship body can drive the flight time assembly 20 to move, and initial depth images in a plurality of different directions are acquired, so that a shot target is accurately identified in a severe environment (for example, a foggy environment), the distance change between the shot target and the mobile platform 300 is judged, and the safety of marine navigation is improved.

The mobile platform 300 according to the embodiment of the present application is a platform capable of moving independently, and the plurality of time-of-flight components 20 are mounted on the body 10 of the mobile platform 300 to obtain a panoramic depth image. However, the electronic device 100 of the embodiment of the present application is generally not independently movable, and the electronic device 100 may be further mounted on a movable apparatus such as the mobile platform 300, thereby assisting the apparatus in acquiring the panoramic depth image.

It should be noted that the above explanations of the body 10, the time-of-flight assembly 20, the camera assembly 30, the microprocessor 40, and the application processor 50 of the electronic device 100 are also applicable to the mobile platform 300 according to the embodiment of the present application, and the descriptions thereof are not repeated here.

Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations of the above embodiments may be made by those of ordinary skill in the art within the scope of the present application, which is defined by the claims and their equivalents.

Claims

1. An electronic device, characterized in that the electronic device comprises:

a body;

the time-of-flight components are respectively positioned at a plurality of different orientations of the body, each time-of-flight component comprises a light emitter and a light receiver, the field angle of each light receiver is any value from 180 degrees to 200 degrees, the field angle of each light emitter is larger than or equal to that of each light receiver, the light emitters are used for emitting laser pulses to the outside of the body, and the light receivers are used for receiving the laser pulses emitted by the corresponding light emitters reflected by a shot target;

the light emitters in the plurality of time-of-flight components emit the laser pulses in a time-shared manner, the light receivers in the plurality of time-of-flight components receive the laser pulses in a time-shared manner to obtain a plurality of initial depth images, and the light emitters in the other time-of-flight components are all turned off when the light receivers in any one of the time-of-flight components are exposed; and

the application processor is used for converting the depth information of each pixel in each initial depth image into unified depth information under a reference coordinate system according to a rotation matrix and a translation matrix between each image coordinate system and the reference coordinate system, wherein any pixel point in each initial depth image corresponds to a coordinate value, and the application processor is used for splicing the converted initial depth images according to the unified depth information and through coordinate matching to obtain a panoramic depth image; when a plurality of initial depth images are spliced, if the pixel points with the same coordinate value exist and the resolution of the initial depth images corresponding to the pixel points is larger than the preset resolution, the pixel points with the same coordinate value are overlapped.

2. The electronic device of claim 1, wherein the time-of-flight component comprises two, and two of the phototransmitters in the two time-of-flight components sequentially and uninterruptedly transmit the laser pulses, and the exposure time of the photoreceiver in each time-of-flight component is within the time range of the phototransmitter in that time-of-flight component that transmits the laser pulses.

3. The electronic device of claim 1, wherein the time-of-flight components comprise two, the two optical transmitters of the two time-of-flight components sequentially transmit the laser pulses at predetermined intervals, and the two optical receivers of the two time-of-flight components sequentially transmit the laser pulses at predetermined intervals.

4. The electronic device of claim 1, wherein the time-of-flight components comprise two, the two optical transmitters of the two time-of-flight components sequentially transmit the laser pulses at a predetermined time interval, and the two optical receivers of the two time-of-flight components sequentially and uninterruptedly expose.

5. The electronic device according to claim 1, wherein the time-of-flight components comprise two, the electronic device further comprising two microprocessors, each microprocessor corresponding to one of the time-of-flight components, both microprocessors being connected to the application processor, each microprocessor being configured to process the laser pulses received by the optical receiver of the corresponding time-of-flight component to obtain an initial depth image and transmit the initial depth image to the application processor; the application processor is used for synthesizing the two initial depth images acquired by the two microprocessors into one frame of the panoramic depth image according to the field angle of the optical receiver.

6. The electronic device according to claim 1, wherein the time-of-flight components comprise two, the electronic device further comprising a microprocessor, the microprocessor being connected to the application processor, the microprocessor being configured to sequentially process the laser pulses received by the two light receivers of the two time-of-flight components to obtain two initial depth images and transmit the two initial depth images to the application processor; and the application processor is used for synthesizing the two initial depth images acquired by the microprocessor into one frame of the panoramic depth image according to the field angle of the optical receiver.

7. The electronic device according to claim 1, wherein the time-of-flight components comprise two, the electronic device further comprising two microprocessors, each microprocessor corresponding to one of the time-of-flight components, both microprocessors being connected to the application processor, each microprocessor being configured to process the laser pulses received by the optical receiver of the corresponding time-of-flight component to obtain an initial depth image and transmit the initial depth image to the application processor;

the electronic equipment further comprises two camera assemblies arranged on the body, each camera assembly corresponds to one time-of-flight assembly, the two camera assemblies are both connected with the application processor, and each camera assembly is used for collecting a scene image of the shot target and outputting the scene image to the application processor;

the application processor is used for identifying the shot target according to the two initial depth images acquired by the two microprocessors and the two scene images acquired by the two camera assemblies.

8. The electronic device according to claim 7, wherein the application processor is further configured to, when the recognition of the target object from the two initial depth images and the two scene images fails, combine the two initial depth images acquired by the two microprocessors into one merged depth image according to a field angle of the optical receiver, combine the two scene images acquired by the two camera assemblies into one merged scene image, and recognize the target object according to the merged depth image and the merged scene image.

9. The electronic device according to claim 1, wherein the time-of-flight components comprise two, the electronic device further comprising a microprocessor, the microprocessor being connected to the application processor, the microprocessor being configured to sequentially process the laser pulses received by the two light receivers of the two time-of-flight components to obtain two initial depth images and transmit the two initial depth images to the application processor;

the application processor is used for identifying the shot target according to the two initial depth images acquired by the microprocessor and the two scene images acquired by the two camera assemblies.

10. The electronic device according to claim 9, wherein the application processor is further configured to, when the recognition of the target object from the two initial depth images and the two scene images fails, combine the two initial depth images acquired by the microprocessor into one merged depth image according to a field angle of the optical receiver, combine the two scene images acquired by the two camera assemblies into one merged scene image, and recognize the target object according to the merged depth image and the merged scene image.

11. The electronic device according to claim 1, wherein the time-of-flight components comprise two, the electronic device further comprising two microprocessors, each microprocessor corresponding to one of the time-of-flight components, both microprocessors being connected to the application processor, each microprocessor being configured to process the laser pulses received by the optical receiver of the corresponding time-of-flight component multiple times to obtain an initial depth image and transmit the initial depth image to the application processor; the application processor is used for judging the distance change between the shot target and the electronic equipment according to the plurality of initial depth images.

12. The electronic device according to claim 11, wherein the application processor is further configured to combine two initial depth images acquired by two microprocessors into one merged depth image according to a field angle of the optical receiver when determining that the distance change fails according to the multiple initial depth images, and the application processor continuously performs the combining step to obtain multiple frames of continuous merged depth images and determines the distance change according to the multiple frames of merged depth images.

13. The electronic device according to claim 1, wherein the time-of-flight components include two, the electronic device further comprising a microprocessor, the microprocessor being connected to the application processor, the microprocessor being configured to sequentially process the laser pulses received by the optical receiver of the corresponding time-of-flight component for a plurality of times to obtain an initial depth image and transmit the initial depth image to the application processor; the application processor is used for judging the distance change between the shot target and the electronic equipment according to the plurality of initial depth images.

14. The electronic device according to claim 13, wherein the application processor is further configured to synthesize two initial depth images acquired by the microprocessor into one merged depth image according to a field angle of the optical receiver when determining that the distance change fails according to a plurality of initial depth images, and the application processor continuously performs the synthesizing step to obtain a plurality of frames of continuous merged depth images and determines the distance change according to the plurality of frames of merged depth images.

15. The electronic device according to any one of claims 11 to 14, wherein the application processor is further configured to increase a frame rate of the initial depth image acquired from the plurality of initial depth images transmitted by the microprocessor to determine the distance change when the distance change is determined to be a distance decrease.

16. A mobile platform, comprising:

a body;

17. The mobile platform of claim 16, wherein the body is a vehicle body, an unmanned aerial vehicle fuselage, a robot body, or a ship body.