CN109391709B

CN109391709B - Electronic device, control method thereof, control device, and computer-readable storage medium

Info

Publication number: CN109391709B
Application number: CN201811048630.0A
Authority: CN
Inventors: 张学勇
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2018-09-10
Filing date: 2018-09-10
Publication date: 2020-11-06
Anticipated expiration: 2038-09-10
Also published as: CN109391709A; WO2020052278A1

Abstract

The invention discloses a control method of an electronic device. The electronic device comprises a time-of-flight module and a visible light camera. The control method comprises the following steps: controlling a visible light camera to collect a current visible light image of the face; performing face authentication according to the current visible light image; the flight time control module is used for collecting a current depth image of the face; performing depth authentication according to the current depth image; and controlling the electronic device to execute a predetermined operation when both the face authentication and the depth authentication pass. In addition, the invention also discloses a control device of the electronic device, the electronic device and a computer readable storage medium. According to the control method of the electronic device, the control device of the electronic device, the electronic device and the computer readable storage medium, the electronic device is controlled to execute the preset operation when the face authentication and the deep authentication pass, and the reliability of identity authentication is high.

Description

Electronic device, control method thereof, control device, and computer-readable storage medium

Technical Field

The present invention relates to the field of consumer electronics, and more particularly, to a method for controlling an electronic device, a control device for an electronic device, and a computer-readable storage medium.

Background

With the rapid development of electronic technology, electronic devices such as smart phones and tablet computers have become more and more popular. The electronic device usually verifies whether the user has the related usage right according to the two-dimensional image of the face input by the user, and the reliability of the identity authentication mode is low.

Disclosure of Invention

The embodiment of the invention provides a control method of an electronic device, a control device of the electronic device, the electronic device and a computer readable storage medium.

The embodiment of the invention provides a control method of an electronic device, wherein the electronic device comprises a flight time module and a visible light camera, and the control method comprises the following steps: controlling the visible light camera to collect a current visible light image of the face; performing face authentication according to the current visible light image; controlling the flight time module to acquire a current depth image of the face; performing depth authentication according to the current depth image; and controlling the electronic device to execute a predetermined operation when the face authentication and the depth authentication both pass.

The embodiment of the invention provides a control device of an electronic device, wherein the electronic device comprises a flight time module and a visible light camera, and the control device comprises a first control module, a first authentication module, a second control module, a second authentication module and a third control module; the first control module is used for controlling the visible light camera to collect the current visible light image of the face; the first authentication module is used for carrying out face authentication according to the current visible light image; the second control module is used for controlling the flight time module to collect the current depth image of the face; the second authentication module is used for performing depth authentication according to the current depth image; and the third control module is used for controlling the electronic device to execute preset operation when the human face authentication and the deep authentication both pass.

Embodiments of the present invention provide an electronic device comprising a time-of-flight module, a visible light camera, one or more processors, a memory, and one or more programs stored in the memory and configured to be executed by the one or more processors, the programs comprising instructions for performing the control method of the electronic device of the above embodiments.

Embodiments of the present invention provide a computer-readable storage medium including a computer program for use in conjunction with an electronic apparatus, the computer program being executable by a processor to perform the method of controlling the electronic apparatus according to the above embodiments.

According to the control method of the electronic device, the control device of the electronic device, the electronic device and the computer readable storage medium, the electronic device is controlled to execute the preset operation when the face authentication and the deep authentication pass, and the reliability of identity authentication is high.

Additional aspects and advantages of embodiments of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of embodiments of the invention.

Drawings

The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

FIG. 1 is a schematic flow chart of a method for controlling an electronic device according to some embodiments of the present invention;

FIG. 2 is a block diagram of a control device of an electronic device according to some embodiments of the present invention;

FIG. 3 is a schematic structural diagram of an electronic device according to some embodiments of the invention;

FIG. 4 is a flow chart illustrating a method for controlling an electronic device according to some embodiments of the invention;

FIG. 5 is a block diagram of a control device of an electronic device according to some embodiments of the present invention;

FIG. 6 is a schematic diagram of a scenario of a control method of an electronic device according to some embodiments of the invention;

FIG. 7 is a flow chart illustrating a method for controlling an electronic device according to some embodiments of the invention;

FIG. 8 is a block diagram of a control device of an electronic device according to some embodiments of the present invention;

FIG. 9 is a flow chart illustrating a method for controlling an electronic device according to some embodiments of the invention;

FIG. 10 is a block schematic diagram of a control device of an electronic device according to some embodiments of the invention;

FIG. 11 is a schematic diagram of a scenario of a control method of an electronic device according to some embodiments of the invention;

FIG. 12 is a block diagram of an electronic device according to some embodiments of the invention;

FIG. 13 is a schematic diagram of a connection state of an electronic device and a computer-readable storage medium according to some embodiments of the invention;

FIG. 14 is a schematic perspective view of a time of flight module according to some embodiments of the present invention;

FIG. 15 is a schematic top view of a time of flight module according to certain embodiments of the invention;

FIG. 16 is a schematic bottom view of a time of flight module according to some embodiments of the invention;

FIG. 17 is a schematic side view of a time of flight module according to certain embodiments of the invention;

FIG. 18 is a schematic cross-sectional view of the time of flight module of FIG. 15 taken along line XVIII;

FIG. 19 is an enlarged schematic view of the XIX portion of the time of flight module shown in FIG. 18;

fig. 20 is a schematic front view of a time-of-flight module according to some embodiments of the invention when the flexible circuit board is not bent.

Detailed Description

The following further describes embodiments of the present invention with reference to the drawings. The same or similar reference numbers in the drawings identify the same or similar elements or elements having the same or similar functionality throughout.

In addition, the embodiments of the present invention described below with reference to the accompanying drawings are exemplary only for the purpose of explaining the embodiments of the present invention, and are not to be construed as limiting the present invention.

Referring to fig. 1 and fig. 3, a control method of an electronic device 100 is provided according to an embodiment of the invention. The electronic device 100 includes a time-of-flight module 30 and a visible light camera 40. The control method comprises the following steps:

011: controlling the visible light camera 40 to collect the current visible light image of the face;

012: performing face authentication according to the current visible light image;

013: the time-of-flight control module 30 collects the current depth image of the face;

014: performing depth authentication according to the current depth image; and

015: when both the face authentication and the depth authentication pass, the electronic device 100 is controlled to perform a predetermined operation.

Referring to fig. 2 and 3, a control device 10 of an electronic device 100 is provided according to an embodiment of the invention. The electronic device 100 includes a time-of-flight module 30 and a visible light camera 40. The control device 10 includes a first control module 11, a first authentication module 12, a second control module 13, a second authentication module 14, and a third control module 15. The control method according to the embodiment of the present invention can be realized by the control device 10 according to the embodiment of the present invention. For example, a first control module 11 may be used to perform the method in 011, a first authentication module 12 may be used to perform the method in 012, a second control module 13 may be used to perform the method in 013, a second authentication module 14 may be used to perform the method in 014, and a third control module 15 may be used to perform the method in 015.

That is, the first control module 11 may be configured to control the visible light camera 40 to capture a current visible light image of the human face. The first authentication module 12 may be configured to perform face authentication according to the current visible light image. The second control module 13 may be configured to control the time-of-flight module 30 to acquire a current depth image of the face. The second authentication module 14 may be used for depth authentication from the current depth image. The third control module 15 may be configured to control the electronic apparatus 100 to perform a predetermined operation when both the face authentication and the deep authentication pass.

According to the control method and the control device 10 of the electronic device 100, the electronic device 100 is controlled to execute the predetermined operation when the face authentication and the deep authentication both pass, and the reliability of the identity authentication is high.

Referring to fig. 3, the electronic device 100 may be a mobile phone, a tablet computer, a smart watch, a smart band, a smart wearable device, and the like, and it is understood that the electronic device 100 is not limited to the mobile phone.

The electronic device 100 may include a housing 101 and a stand 102. The time of flight module 30 and the visible light camera 40 are disposed on the support 102. The time of flight module 30, the visible light camera 40, and the cradle 102 are all housed within the housing 101 and are able to extend from the housing 101. Specifically, when the visible light camera 40 is used for acquiring a visible light image of a human face or the time-of-flight module 30 is used for acquiring a depth image of the human face, the bracket 102 drives the time-of-flight module 30 and the visible light camera 40 to move towards the outside of the housing 101 so as to extend out of the housing 101, so as to acquire the visible light image or the depth image.

In the embodiment of the present invention, the visible light camera 40 collects the current visible light image of the face and the time-of-flight module 30 collects the current depth image of the face, and the sequence between the two images may be arbitrary. For example, the visible light camera 40 collects the current visible light image of the face first, and the time-of-flight module 30 collects the current depth image of the face; or, the time-of-flight module 30 collects the current depth image of the face first, and the visible light camera 40 collects the current visible light image of the face; or, the visible light camera 40 collects the current visible light image of the face while the visible light camera 40 collects the current visible light image of the face, so as to reduce the time for completing the face authentication and the depth authentication.

In some embodiments, the step of controlling the time-of-flight module 30 to acquire the current depth image of the face (i.e., 013) is performed when the face authentication is passed. That is, the second control module 13 may be configured to control the time-of-flight module 30 to acquire the current depth image of the face when the face authentication passes.

Specifically, the electronic device 100 first acquires the current visible light image of the human face through the visible light camera 40, and then performs human face authentication according to the current visible light image. When the face authentication fails, the electronic device 100 directly judges that the identity authentication fails without acquiring the current depth image of the face through the flight time module 30; when the face authentication passes, the electronic device 100 collects the current depth image of the face through the flight time module 30, and performs the depth authentication according to the current depth image. Since the time-of-flight module 30 does not need to acquire the current depth image of the face when the face authentication fails, the energy consumption of the electronic device 100 is saved.

In some embodiments, the step of controlling the visible light camera 40 to capture the current visible light image of the human face (i.e., 011) is performed when the depth authentication is passed. That is, the first control module 11 may be configured to control the visible light camera 40 to capture a current visible light image of the human face when the depth authentication passes.

Specifically, the electronic device 100 first acquires the current depth image of the face through the time-of-flight module 30, and then performs depth authentication according to the current depth image. When the deep authentication fails, the electronic device 100 directly judges that the identity authentication fails without acquiring the current visible light image of the face through the visible light camera 40; when the depth authentication is passed, the electronic device 100 collects the current visible light image of the face through the visible light camera 40, and performs face authentication according to the current visible light image. Since the visible light camera 40 does not need to collect the current visible light image of the human face when the deep authentication fails, the energy consumption of the electronic device 100 is saved.

In some embodiments, the predetermined operation includes at least one of unlocking, illuminating a display screen 103 (shown in FIG. 3) of the electronic device 100, electronically paying, and opening a predetermined application of the electronic device 100. That is to say, the user can directly obtain the corresponding operation authority in the electronic device 100 by performing the face authentication and the deep authentication, and does not need to manually control the electronic device 100 to execute the predetermined operation, which is beneficial to improving the user experience and has higher security.

Referring to fig. 4, in some embodiments, the step of controlling the visible light camera 40 to acquire the current visible light image of the face (i.e. 011) and the step of controlling the time-of-flight module 30 to acquire the current depth image of the face (i.e. 013) are performed during the unlocking process of the electronic device 100, and the control method further includes:

016: controlling a visible light camera 40 to collect a reference visible light image of the face in an encryption process;

the step of performing face authentication from the current visible light image (i.e., 012) includes:

0122: judging a first similarity between the current visible light image and the reference visible light image;

the control method further comprises the following steps:

017: controlling a flight time module 30 to collect a reference depth image of the face in an encryption process;

the step of performing depth authentication according to the current depth image (i.e., 014) includes:

0142: judging a second similarity between the current depth image and the reference depth image;

when the face authentication and the depth authentication both pass, the step (i.e. 015) of controlling the electronic device 100 to perform the predetermined operation includes:

0152: and when the first similarity is greater than a first threshold and the second similarity is greater than a second threshold, controlling the electronic device 100 to unlock.

Referring to fig. 5, in some embodiments, the first control module 11 is configured to control the visible light camera 40 to capture a current visible light image of the human face when the electronic device 100 is in the unlocking process. The second control module 13 is configured to control the time-of-flight module 30 to acquire a current depth image of the face when the electronic device 100 is in an unlocking process. The control device 10 further comprises a fourth control module 16 and a fifth control module 17. The first authentication module 12 includes a first authentication unit 122, the second authentication module 14 includes a second authentication unit 142, and the third authentication module 15 includes a first control unit 152. The fourth control module 16 may be used to perform the method of 016, the fifth control module 17 may be used to perform the method of 017, the first authentication unit 122 may be used to perform the method of 0122, the second authentication unit 142 may be used to perform the method of 0142, and the first control unit 152 may be used to perform the method of 0152.

That is, the fourth control module 16 may be configured to control the visible light camera 40 to capture a reference visible light image of the human face during the encryption process. The first authentication unit 122 may be configured to determine a first similarity between the current visible light image and the reference visible light image. The fifth control module 17 may be configured to control the time-of-flight module 30 to acquire the reference depth image of the face during the encryption process. The second authentication unit 142 may be configured to determine a second similarity between the current depth image and the reference depth image. The first control unit 152 may be configured to control the electronic device 100 to unlock when the first similarity is greater than a first threshold and the second similarity is greater than a second threshold.

In the embodiment of the present invention, the reference visible light image and the reference depth image of the face are recorded in advance in the encryption process of the electronic device 100, so as to be used as a comparison reference for performing face authentication and depth authentication in the unlocking process of the electronic device 100.

The reference depth image serving as the comparison reference may include reference depth images of a face at a plurality of different angles, and the plurality of reference depth images may be acquired after the head of the user is deflected to different angles. For example, the user performs a yaw motion such as a left yaw and a right yaw, respectively, under the guidance of the content displayed on the display screen 103. During the deflection process, the time-of-flight module 30 collects a plurality of reference depth images of the face. Finally, the reference depth image comprises a reference depth image corresponding to the right deviation of the face, a reference depth image corresponding to the front face of the face and a reference depth image corresponding to the left deviation of the face.

Taking fig. 6 as an example, the reference depth images include a reference depth image 1 corresponding to a right offset of a human face, a reference depth image 2 corresponding to a front face of the human face, and a reference depth image 3 corresponding to a left offset of the human face.

In one embodiment, when determining the second similarity between the current depth image and the reference depth image, the electronic device 100 sequentially or concurrently determines a similarity 1 between the current depth image and the reference depth image 1, a similarity 2 between the current depth image and the reference depth image 2, and a similarity 3 between the current depth image and the reference depth image 3, and selects a maximum value among the similarities 1, 2, and 3 as the second similarity between the current depth image and the reference depth image.

In another embodiment, when determining the second similarity between the current depth image and the reference depth image, the electronic device 100 first detects a deflection direction and a deflection angle of the face relative to the electronic device 100 according to the current depth image, and when the face deviates 90 degrees to the right relative to the electronic device 100, the electronic device 100 selects the reference depth image 1 as the reference depth image to be compared with the current depth image, that is, the electronic device 100 uses the similarity 1 between the current depth image and the reference depth image as the second similarity between the current depth image and the reference depth image. When the face is not deflected relative to the electronic device 100 (i.e., the current depth image is the face front image), the electronic device 100 selects the reference depth image 2 as the reference depth image to be compared with the current depth image, that is, the electronic device 100 uses the similarity 2 between the current depth image and the reference depth image 2 as the second similarity between the current depth image and the reference depth image. When the face deviates 90 degrees to the left with respect to the electronic device 100, the electronic device 100 selects the reference depth image 3 as the reference depth image to be compared with the current depth image, that is, the electronic device 100 uses the similarity 3 between the current depth image and the reference depth image as the second similarity between the current depth image and the reference depth image.

Similarly, the reference visible light image serving as the comparison reference may also include reference visible light images of a human face at a plurality of different angles, the plurality of reference visible light images are obtained in a manner similar to that of the plurality of reference depth images, and the manner of determining the first similarity is also similar to that of determining the second similarity, which is not described in detail herein. In the embodiment of the invention, when the user needs to perform face authentication and depth authentication, the user can perform face authentication and depth authentication by adopting any deflection angle without strictly aligning the visible light camera 40 and the flight time module 30 according to a certain angle, which is beneficial to improving user experience.

After determining the first similarity, the electronic device 100 performs face authentication according to the first similarity. After determining the second similarity, the electronic device 100 performs identity authentication according to the second similarity. Taking the first threshold as 80% and the second threshold as 70% as an example, when the first similarity is greater than 70% and the second similarity is greater than 70%, both the face authentication and the depth authentication pass, and the electronic device 100 performs a predetermined operation. It should be noted that the first threshold and the second threshold are percentage of similarity, and the first threshold and the second threshold may be the same value or different values.

Referring again to fig. 4, in some embodiments, the control method further includes:

018: when the first similarity is greater than the third threshold and the second similarity is less than the fourth threshold, locking the electronic device 100; wherein the third threshold is greater than the first threshold, and the fourth threshold is less than the second threshold.

Referring again to fig. 5, in some embodiments, the control device 10 further includes a locking module 18. The locking module 18 may be used to perform the method in 018.

That is, the locking module 18 may be configured to lock the electronic device 100 when the first similarity is greater than the third threshold and the second similarity is less than the fourth threshold. Wherein the third threshold is greater than the first threshold, and the fourth threshold is less than the second threshold.

Specifically, the first threshold is 80% and the second threshold is 70%, and in this case, the third threshold may be 95% and the fourth threshold may be 30%. When the first similarity is 99% and the second similarity is 20%, the first similarity is greater than a third threshold, which indicates that the similarity between the current visible light image of the face and the reference visible light image is high, and the second similarity is much smaller than a fourth threshold, which indicates that the similarity between the current depth image of the face and the reference depth image is low, it is likely that the non-owner performs identity authentication by using a photo or other imitations, and at this time, the electronic device 100 enters a locked state to protect the security of information in the electronic device 100.

It should be noted that the setting of the first threshold, the second threshold, the third threshold, and the fourth threshold is not limited to the above example, and the user may set the first threshold, the second threshold, the third threshold, and the fourth threshold according to the needs of the user, or the first threshold, the second threshold, the third threshold, and the fourth threshold may be already set before the electronic device 100 leaves the factory.

Referring to fig. 7, in some embodiments, the control method further includes:

019: when the deep authentication is passed and the face authentication is not passed, acquiring the color temperature and the brightness of ambient light;

020: carrying out color correction on the current visible light image according to the color temperature and the brightness; and

021: and carrying out face authentication again according to the current visible light image after color correction.

Referring to fig. 8, in some embodiments, the control device 10 further includes an obtaining module 19, a correcting module 20, and a third authenticating module 21. The acquisition module 19 may be used to perform the method in 019, the correction module 20 may be used to perform the method 020, and the third authentication module 21 may be used to perform the method 021.

That is, the obtaining module 19 may be configured to obtain the color temperature and brightness of the ambient light when the deep authentication passes and the face authentication fails. The correction module 20 can be used to perform color correction on the current visible light image according to the color temperature and brightness. The third authentication module 21 may be configured to perform face authentication again according to the color-corrected current visible light image.

It can be understood that when the deep authentication is passed and the face authentication is not passed, indicating that the object to be measured is not a photo, the electronic device 100 obtains the color temperature and the brightness of the ambient light, performs color correction on the current visible light image according to the color temperature and the brightness to improve the image quality, and performs the face authentication again according to the current visible light image after the color correction to avoid an error in the face authentication result caused by the influence of the color temperature and the brightness of the ambient light, thereby ensuring the reliability of the face authentication.

Of course, in other embodiments, the electronic device 100 may also perform noise reduction processing on the current visible light image before the face authentication, so as to further improve the image quality.

Referring to fig. 9, in some embodiments, the step of controlling the visible light camera 40 to capture the current visible light image of the human face (i.e. 011) includes:

0112: controlling the visible light camera 40 to collect a plurality of frames of current visible light images;

the control method further comprises the following steps:

022: identifying a plurality of target characteristic points in each frame of current visible light image;

023: judging whether the plurality of target characteristic points are in a motion state or not according to the relative position change among the plurality of target characteristic points in the multi-frame current visible light image; and

when a plurality of target feature points are in motion, the method proceeds to a step of performing face authentication according to the current visible light image (i.e. 012), or proceeds to a step of controlling the time-of-flight module 30 to acquire the current depth image of the face (i.e. 013).

Referring to fig. 10, in some embodiments, the first control module 11 further includes a second control unit 112. The control device 10 further comprises an identification module 22 and a judgment module 23. The second control unit 112 may be used to perform the method of 0112, the identification module 22 may be used to perform the method of 022, and the determination module 23 may be used to perform the method of 023.

That is, the second control unit 112 may be configured to control the visible light camera 40 to capture a plurality of frames of the current visible light image. The identification module 22 may be configured to identify a plurality of target feature points in each of the current visible light images. The judging module 23 may be configured to judge whether the plurality of target feature points are in a motion state according to a relative position change between the plurality of target feature points in the plurality of frames of current visible light images. The first authentication module 12 is configured to perform face authentication according to a current visible light image when the plurality of target feature points are in a motion state; alternatively, the second control module 13 may be configured to control the time-of-flight module 30 to acquire the current depth image of the face when the plurality of target feature points are in a motion state.

Referring to fig. 11, the target feature points may be feature points in target organs (e.g., eyes, nose, mouth) of the human face. The embodiment of the invention takes the target characteristic points as the upper lip characteristic point A and the lower lip characteristic point B of the mouth as an example. The multi-frame current visible light image is at least two frames of current visible light images. In the example shown in fig. 11, the mouth of the first frame of the current visible light image is in a closed state, and the target feature point a and the target feature point B have a certain distance therebetween. The mouth of the current visible light image of the second frame is in an open state, and the target characteristic point A is overlapped with the target characteristic point B. The electronic device 100 determines that the target feature point a and the target feature point B are in a motion state according to the relative position change of the target feature point a and the target feature point B in the first frame of the current visible light image and the second frame of the current visible light image, which can indicate that the object to be measured is a living body. At this moment, the electronic device 100 performs face authentication according to the current visible light image or controls the flight time module 30 to collect the current depth image of the face to perform depth authentication, so that not only can the energy consumption of the electronic device 100 be saved, but also the situation that a non-owner adopts a photo or other imitations to perform identity authentication can be avoided, and the reliability of the identity authentication is high.

Referring to fig. 12, an electronic device 100 according to an embodiment of the invention includes a time-of-flight module 30, a visible light camera 40, one or more processors 50, a memory 60, and one or more programs. Wherein one or more programs are stored in the memory 60 and configured to be executed by the one or more processors 50, the programs including instructions for performing the control method of the electronic device 100 of any of the above embodiments. For example, the program includes instructions for executing the following control method of the electronic apparatus 100: 011: controlling the visible light camera 40 to collect the current visible light image of the face; 012: performing face authentication according to the current visible light image; 013: the time-of-flight control module 30 collects the current depth image of the face; 014: performing depth authentication according to the current depth image; and 015: when both the face authentication and the depth authentication pass, the electronic device 100 is controlled to perform a predetermined operation.

Referring to fig. 13, a computer readable storage medium 200 according to an embodiment of the present invention includes a computer program for use in conjunction with the electronic device 100. The computer program can be executed by the processor 50 to implement the control method of the electronic device 100 according to any of the above embodiments. For example, the computer program may be executed by the processor 50 to perform the following control method of the electronic apparatus 100: 011: controlling the visible light camera 40 to collect the current visible light image of the face; 012: performing face authentication according to the current visible light image; 013: the time-of-flight control module 30 collects the current depth image of the face; 014: performing depth authentication according to the current depth image; and 015: when both the face authentication and the depth authentication pass, the electronic device 100 is controlled to perform a predetermined operation.

Illustratively, the time-of-flight module 30 of an embodiment of the present invention may have the following structure.

Referring to fig. 14 to 17, the time-of-flight module 30 includes a first substrate assembly 31, a pad 32, a light emitter 33 and a light receiver 34. The first substrate assembly 31 includes a first substrate 311 and a flexible circuit board 312 connected to each other. The pads 32 are disposed on the first substrate 311. The optical transmitter 33 is used for emitting optical signals outwards, and the optical transmitter 33 is arranged on the cushion block 32. The flexible circuit board 312 is bent, and one end of the flexible circuit board 312 is connected to the first substrate 311, and the other end is connected to the light emitter 33. The optical receiver 34 is disposed on the first substrate 311, the optical receiver 34 is used for receiving the reflected optical signal emitted by the optical transmitter 33, the optical receiver 34 includes a housing 341 and an optical element 342 disposed on the housing 341, and the housing 341 and the spacer 32 are integrally connected.

In the electronic device 100 according to the embodiment of the present invention, since the optical transmitter 33 is disposed on the pad 32, the height of the optical transmitter 33 can be increased by the pad 32, and further, the height of the emitting surface of the optical transmitter 33 is increased, so that the optical signal emitted by the optical transmitter 33 is not easily shielded by the optical receiver 34, and the optical signal can be completely irradiated onto the object to be measured.

Specifically, the first substrate assembly 31 includes a first substrate 311 and a flexible circuit board 312. The first substrate 311 may be a printed circuit board or a flexible circuit board, and a control circuit of the time-of-flight module 30 and the like may be laid on the first substrate 311. One end of the flexible circuit board 312 may be connected to the first substrate 311, and the flexible circuit board 312 may be bent at a certain angle, so that the relative positions of the devices connected to the two ends of the flexible circuit board 312 may be selected more.

Referring to fig. 14 and 18, the pads 32 are disposed on the first substrate 311. In one example, the pad 32 is in contact with the first substrate 311 and is carried on the first substrate 311, and specifically, the pad 32 may be bonded to the first substrate 311 by gluing or the like. The material of the pad 32 may be metal, plastic, etc. In the embodiment of the present invention, a surface of the pad block 32 combined with the first substrate 311 may be a plane, and a surface of the pad block 32 opposite to the combined surface may also be a plane, so that the light emitter 33 has better stability when disposed on the pad block 32.

The optical transmitter 33 is used for emitting an optical signal, specifically, the optical signal may be infrared light, and the optical signal may be a dot matrix light spot emitted toward the object to be measured, and the optical signal is emitted from the optical transmitter 33 at a certain divergence angle. The light emitter 33 is disposed on the pad 32, in the embodiment of the invention, the light emitter 33 is disposed on a side of the pad 32 opposite to the first substrate 311, or the pad 32 separates the first substrate 311 and the light emitter 33, so that a height difference is formed between the light emitter 33 and the first substrate 311. The light emitter 33 is further connected to a flexible circuit board 312, the flexible circuit board 312 is bent, one end of the flexible circuit board 312 is connected to the first substrate 311, and the other end of the flexible circuit board 312 is connected to the light emitter 33, so as to transmit a control signal of the light emitter 33 from the first substrate 311 to the light emitter 33, or transmit a feedback signal of the light emitter 33 (for example, time information, frequency information of an emitted light signal of the light emitter 33, temperature information of the light emitter 33, and the like) to the first substrate 311.

Referring to fig. 14, 15 and 17, the optical receiver 34 is used for receiving the reflected optical signal emitted by the optical transmitter 33. The light receiver 34 is disposed on the first substrate 311, and the contact surfaces of the light receiver 34 and the first substrate 311 are disposed substantially flush with the contact surfaces of the pads 32 and the first substrate 311 (i.e., the mounting start points of the two are on the same plane). Specifically, the light receiver 34 includes a housing 341 and an optical element 342. The housing 341 is disposed on the first substrate 311, the optical element 342 is disposed on the housing 341, the housing 341 may be a lens holder and a lens barrel of the optical receiver 34, and the optical element 342 may be a lens or the like disposed in the housing 341. Further, the optical receiver 34 may further include a photosensitive chip (not shown), and the optical signal reflected by the object to be measured is irradiated into the photosensitive chip after being acted by the optical element 342, and the photosensitive chip responds to the optical signal. The time-of-flight module 30 calculates a time difference between the light signal emitted from the light emitter 33 and the light signal reflected by the object to be measured received by the light sensing chip, and further obtains depth information of the object to be measured, where the depth information may be used for distance measurement, depth image generation, three-dimensional modeling, or the like. In the embodiment of the present invention, the housing 341 is integrally connected to the cushion block 32. Specifically, the housing 341 and the pad 32 may be integrally formed, for example, the housing 341 and the pad 32 are made of the same material and integrally formed by injection molding, cutting, or the like; or the housing 341 and the cushion block 32 are made of different materials and are integrally formed by two-color injection molding or the like. The housing 341 and the pad 32 may also be formed separately, and the two form a matching structure, and when the time-of-flight module 30 is assembled, the housing 341 and the pad 32 may be connected into a whole and then disposed on the first substrate 311 together; alternatively, one of the case 341 and the pad 32 may be disposed on the first substrate 311, and the other may be disposed on the first substrate 311 and integrally connected.

In the electronic device 100 according to the embodiment of the present invention, since the optical transmitter 33 is disposed on the pad 32, the height of the optical transmitter 33 can be increased by the pad 32, and further, the height of the emitting surface of the optical transmitter 33 is increased, so that the optical signal emitted by the optical transmitter 33 is not easily shielded by the optical receiver 34, and the optical signal can be completely irradiated onto the object to be measured. The exit surface of the optical transmitter 33 may be flush with the entrance surface of the optical receiver 34, or the exit surface of the optical transmitter 33 may be slightly lower than the entrance surface of the optical receiver 34, or the exit surface of the optical transmitter 33 may be slightly higher than the entrance surface of the optical receiver 34.

Referring to fig. 16 and 18, in some embodiments, the first substrate assembly 31 further includes a reinforcing plate 313, and the reinforcing plate 313 is bonded to a side of the first substrate 311 opposite to the pad 32. The reinforcing plate 313 may cover one side surface of the first substrate 311, and the reinforcing plate 313 may be used to increase the strength of the first substrate 311 and prevent the first substrate 311 from deforming. In addition, the reinforcing plate 313 may be made of a conductive material, such as a metal or an alloy, and when the time of flight module 30 is mounted on the electronic device 100, the reinforcing plate 313 and the chassis 101 may be electrically connected, so that the reinforcing plate 313 is grounded, and the interference of static electricity of external components on the time of flight module 30 is effectively reduced.

Referring to fig. 18 to 20, in some embodiments, the pad 32 includes a protrusion 325 extending from a side edge 3111 of the first substrate 311, and the flexible circuit board 312 is bent around the protrusion 325. Specifically, a portion of the pad 32 is directly carried on the first substrate 311, and another portion is not in direct contact with the first substrate 311 and protrudes relative to a side edge 3111 of the first substrate 311 to form a protrusion 325. The flexible circuit board 312 may be connected to the side edge 3111, the flexible circuit board 312 is bent around the protrusion 325, or the flexible circuit board 312 is bent to make the protrusion 325 be located in a space surrounded by the bending of the flexible circuit board 312, when the flexible circuit board 312 is subjected to an external force, the flexible circuit board 312 will not collapse inwards to cause an excessive bending degree, which may damage the flexible circuit board 312.

Further, as shown in fig. 19, in some embodiments, the outer side surface 3251 of the protruding portion 325 is a smooth curved surface (e.g., an outer side surface of a cylinder), that is, the outer side surface 3251 of the protruding portion 325 does not form a curvature abrupt change, and even if the flexible circuit board 312 is bent along with the outer side surface 3251 of the protruding portion 325, the bending degree of the flexible circuit board 312 is not too large, thereby further ensuring the integrity of the flexible circuit board 312.

Referring to fig. 3 to 16, in some embodiments, the time-of-flight module 30 further includes a connector 36, and the connector 36 is connected to the first substrate 311. The connector 36 is used to connect the first board assembly 31 and an external device. The connector 36 and the flexible circuit board 312 are respectively connected to opposite ends of the first substrate 311. The connector 36 may be a connecting socket or a connecting head, and when the time-of-flight module 30 is installed in the casing 101, the connector 36 may be connected to a motherboard of the electronic device 100, so that the time-of-flight module 30 is electrically connected to the motherboard. The connectors 36 and the flexible circuit board 312 are respectively connected to opposite ends of the first substrate 311, for example, the connectors may be respectively connected to the left and right ends of the first substrate 311, or respectively connected to the front and rear ends of the first substrate 311.

Referring to fig. 15 and 16, in some embodiments, the optical transmitter 33 and the optical receiver 34 are arranged along a straight line L, and the connector 36 and the flexible circuit board 312 are respectively located on two opposite sides of the straight line L. It will be appreciated that the time-of-flight module 30 may already be relatively large in size in the direction of the line L due to the arrangement of the optical transmitter 33 and the optical receiver 34. The connectors 36 and the flexible circuit board 312 are respectively disposed on two opposite sides of the straight line L, so that the size of the time-of-flight module 30 along the direction of the straight line L is not increased, and the time-of-flight module 30 is conveniently mounted on the casing 101 of the electronic device 100.

Referring to fig. 18 and 19, in some embodiments, a receiving cavity 323 is formed at a side of the pad 32 combined with the first substrate 311. The time-of-flight module 30 further includes an electronic component 35 disposed on the first substrate 311, and the electronic component 35 is accommodated in the accommodating cavity 323. The electronic component 35 may be a capacitor, an inductor, a transistor, a resistor, or the like, and the electronic component 35 may be electrically connected to a control circuit laid on the first substrate 311 and used for driving or controlling the operation of the optical transmitter 33 or the optical receiver 34. The electronic component 35 is accommodated in the accommodating cavity 323, so that the space in the pad 32 is reasonably utilized, the electronic component 35 is arranged without increasing the width of the first substrate 311, and the overall size of the time-of-flight module 30 is reduced. The number of the receiving cavities 323 may be one or more, a plurality of the receiving cavities 323 may be spaced apart from each other, and when the spacer 32 is mounted, the receiving cavities 323 may be aligned with the positions of the electronic components 35 and the spacer 32 may be disposed on the first substrate 311.

Referring to fig. 18 and 20, in some embodiments, the cushion block 32 is provided with a bypass through hole 324 communicated with the at least one receiving cavity 323, and the at least one electronic component 35 extends into the bypass through hole 324. It is understood that when the electronic component 35 needs to be accommodated in the accommodating chamber 323, the height of the electronic component 35 is required to be not higher than the height of the accommodating chamber 323. For the electronic component 35 having a height higher than the accommodating cavity 323, a bypass through hole 324 corresponding to the accommodating cavity 323 may be formed, and the electronic component 35 may partially extend into the bypass through hole 324, so as to arrange the electronic component 35 without increasing the height of the cushion block 32.

Referring to FIG. 18, in some embodiments, the light emitter 33 includes a second substrate assembly 331, a light source assembly 332, and a housing 333. The second substrate assembly 331 is disposed on the pad block 32, and the second substrate assembly 331 is connected to the flexible circuit board 312. The light source assembly 332 is disposed on the second substrate assembly 331, and the light source assembly 332 serves to emit a light signal. The housing 333 is disposed on the second substrate assembly 331, and the housing 333 forms an accommodating space 3331 for accommodating the light source assembly 332. The flexible circuit board 312 may be removably attached to the second substrate assembly 331. The light source assembly 332 is electrically connected to the second substrate assembly 331. The casing 333 may be bowl-shaped, and an opening of the casing 333 is covered on the second substrate assembly 331 downward to accommodate the light source assembly 332 in the accommodating space 3331. In the embodiment of the present invention, the housing 333 is provided with a light exit 3332 corresponding to the light source module 332, and the optical signal emitted from the light source module 332 passes through the light exit 3332 and then is emitted out, and the optical signal may directly pass through the light exit 3332, or pass through the light exit 3332 after changing the optical path through other optical devices.

With continued reference to fig. 18, in some embodiments, the second substrate assembly 331 includes a second substrate 3311 and a stiffener 3312. The second substrate 3311 is connected to the flexible circuit board 312. The light source assembly 332 and the reinforcement member 3312 are disposed on opposite sides of the second substrate 3311. The second substrate 3311 may be a printed wiring board, a flexible wiring board or the like, and the second substrate 3311 may have a control wiring laid thereon. The reinforcement 3312 may be fixedly connected to the second substrate 3311 by gluing, riveting, or the like, and the reinforcement 3312 may increase the overall strength of the second substrate assembly 331. When the light emitter 33 is disposed on the pad 32, the stiffener 3312 may directly contact the pad 32, the second substrate 3311 may not be exposed to the outside and does not need to directly contact the pad 32, and the second substrate 3311 is not easily contaminated by dust and the like.

In the embodiment shown in fig. 18, the stiffener 3312 is formed separately from the block 32. When assembling the time-of-flight module 30, the pads 32 may be first mounted on the first substrate 311, and at this time, the two ends of the flexible circuit board 312 are respectively connected to the first substrate 311 and the second substrate 3311, and the flexible circuit board 312 may not be bent first (as shown in fig. 20). The flexible circuit board 312 is then bent such that the stiffener 3312 is disposed on the pad 32.

Of course, in other embodiments, the reinforcement 3312 and the pad 32 may be integrally formed, for example, by injection molding, and the pad 32 and the light emitter 33 may be mounted on the first substrate 311 together when the time-of-flight module 30 is assembled.

Referring to fig. 20, in some embodiments, a first positioning member 3313 is formed on the reinforcement member 3312. The cushion block 32 includes a body 321 and a second positioning member 322, and the second positioning member 322 is formed on the body 321. When the second substrate assembly 331 is disposed on the pad 32, the first positioning member 3313 is engaged with the second positioning member 322. Specifically, the first positioning member 3313 and the second positioning member 322 cooperate to effectively limit the relative movement between the second substrate assembly 331 and the pad block 32. The specific types of the first positioning member 3313 and the second positioning member 322 can be selected according to the requirements, for example, the first positioning member 3313 is a positioning hole formed on the reinforcement member 3312, and the second positioning member 322 is a positioning post which extends into the positioning hole to match the first positioning member 3313 and the second positioning member 322; or the first positioning member 3313 is a positioning post formed on the reinforcement member 3312, the second positioning member 322 is a positioning hole, and the positioning post extends into the positioning hole to make the first positioning member 3313 and the second positioning member 322 cooperate with each other; or the number of the first positioning parts 3313 and the second positioning parts 322 is plural, part of the first positioning parts 3313 are positioning holes, part of the second positioning parts 322 are positioning columns, part of the first positioning parts 3313 are positioning columns, and part of the second positioning parts 322 are positioning holes, and the positioning columns extend into the positioning holes to enable the first positioning parts 3313 and the second positioning parts 322 to be matched with each other.

Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.

Claims

1. A control method of an electronic device, wherein the electronic device comprises a time-of-flight module and a visible light camera, the control method comprising:

controlling the visible light camera to collect a current visible light image of the face;

performing face authentication according to the current visible light image;

controlling the flight time module to acquire a current depth image of the face;

performing depth authentication according to the current depth image; and

when the face authentication and the depth authentication both pass, controlling the electronic device to execute a preset operation;

the step of controlling the visible light camera to acquire the current visible light image of the face and the step of controlling the flight time module to acquire the current depth image of the face are performed when the electronic device is unlocked, and the control method further includes:

controlling the visible light camera to collect a reference visible light image of the face in an encryption process;

the step of performing face authentication according to the current visible light image comprises:

judging a first similarity between the current visible light image and the reference visible light image;

the control method further comprises the following steps:

controlling the flight time module to collect a reference depth image of a face in an encryption process, wherein the reference depth image comprises a plurality of reference depth images of the face at different angles, and the plurality of different angles of the face comprise a front face, a left side face and a right side face;

the step of performing depth authentication according to the current depth image includes:

judging the similarity between the current depth image and the plurality of reference depth images and selecting the maximum value as a second similarity;

the step of controlling the electronic device to execute a predetermined operation when the face authentication and the depth authentication both pass includes:

and when the first similarity is larger than a first threshold and the second similarity is larger than a second threshold, controlling the electronic device to unlock.

2. The control method according to claim 1, wherein the step of controlling the time-of-flight module to acquire the current depth image of the face is performed when the face authentication is passed; or

And the step of controlling the visible light camera to collect the current visible light image of the face is carried out when the depth authentication is passed.

3. The control method of claim 1, wherein the predetermined operation comprises at least one of unlocking, illuminating a display screen of the electronic device, electronically paying, and opening a predetermined application of the electronic device.

4. The control method according to claim 1, characterized by further comprising:

locking the electronic device when the first similarity is greater than a third threshold and the second similarity is less than a fourth threshold;

wherein the third threshold is greater than the first threshold, and the fourth threshold is less than the second threshold.

5. The control method according to claim 1, characterized by further comprising:

when the deep authentication passes and the face authentication does not pass, acquiring the color temperature and the brightness of ambient light;

carrying out color correction on the current visible light image according to the color temperature and the brightness; and

and performing the face authentication again according to the current visible light image after the color correction.

6. The control method according to claim 1, wherein the step of controlling the visible light camera to capture the current visible light image of the human face comprises:

controlling the visible light camera to collect a plurality of frames of the current visible light images;

the control method further comprises the following steps:

identifying a plurality of target feature points in each frame of the current visible light image;

judging whether the target characteristic points are in a motion state or not according to the relative position change among the target characteristic points in the current visible light images of multiple frames; and

and when the target feature points are in a motion state, performing the step of carrying out face authentication according to the current visible light image, or controlling the flight time module to acquire the current depth image of the face.

7. A control device of an electronic device, the electronic device comprising a time-of-flight module and a visible light camera, the control device comprising:

the fourth control module is used for controlling the visible light camera to collect a reference visible light image of a human face in an encryption process;

the first control module is used for controlling the visible light camera to collect the current visible light image of the face when the electronic device is in an unlocking process;

the first authentication module is used for performing face authentication according to the current visible light image and comprises a first authentication unit which is used for judging a first similarity between the current visible light image and the reference visible light image;

the fifth control module is used for controlling the flight time module to acquire a reference depth image of a face in an encryption process, wherein the reference depth image comprises reference depth images of the face at a plurality of different angles, and the plurality of different angles of the face comprise a front face, a left side face and a right side face;

the second control module is used for controlling the flight time module to acquire the current depth image of the face when the electronic device is in an unlocking process;

the second authentication module is used for performing depth authentication according to the current depth image and comprises a second authentication unit, and the second authentication unit is used for judging the similarity between the current depth image and the plurality of reference depth images and selecting the maximum value as a second similarity;

the third control module is used for controlling the electronic device to execute a preset operation when the face authentication and the depth authentication both pass, and the third control module comprises a first control unit which is used for controlling the electronic device to unlock when the first similarity is larger than a first threshold value and the second similarity is larger than a second threshold value.

8. An electronic device comprising a time-of-flight module, a visible light camera, one or more processors, memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs comprising instructions for performing the control method of the electronic device of any of claims 1-6.

9. A computer-readable storage medium, comprising a computer program for use in conjunction with an electronic apparatus, the computer program being executable by a processor to perform the method of controlling the electronic apparatus of any one of claims 1 to 6.