CN112948792A

CN112948792A - Unlocking method and device

Info

Publication number: CN112948792A
Application number: CN201911173301.3A
Authority: CN
Inventors: 吴义镇; 勾军委; 张德志; 张柏雄
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2019-11-26
Filing date: 2019-11-26
Publication date: 2021-06-11

Abstract

The embodiment of the application relates to an unlocking method, which comprises the following steps: collecting environmental information of a current environment; identifying whether a current environment is a safe environment; when the current environment is identified to be a safe environment, determining a face matching threshold value, and collecting face information and/or voice information; identifying gray face information and determining face matching degree; when the face matching degree is greater than or equal to the face matching threshold, unlocking the terminal equipment; and/or identifying voice information, and unlocking the terminal equipment when the voice information is determined to be the pre-stored user voice. And unlocking or voice unlocking by recognizing the current environment as a safe environment and performing face recognition under the safe environment. The unlocking method and the unlocking device have the advantages that the quick identification and unlocking can be realized through computer vision based on artificial intelligence in a safe environment, the calculation complexity is reduced, and the unlocking speed is increased.

Description

Unlocking method and device

Technical Field

The application relates to the field of artificial intelligence, in particular to an unlocking method and device in a safe environment.

Background

With the progress of the times, the unlocking mode of the terminal equipment is changed. At present, the face recognition technology is rapidly developed, and the unlocking mode through the face is gradually popularized. The face recognition technology is a biological recognition technology for carrying out identity recognition based on face feature information of people, and is characterized in that a camera or a camera is adopted to collect images or video streams containing faces, the faces are automatically detected and tracked in the images, and then the faces of the detected faces are recognized. At present, most terminal devices support unlocking in a face recognition mode. But the face unlocking technology supported by different manufacturers is different. Because the human face is easy to be counterfeited, the human face is prevented from being counterfeited by capturing more human face features and improving the calculation complexity in the prior art, so that the counterfeiting difficulty is improved.

For example, in some existing face unlocking technical solutions, the feature of a face, such as a prior art face recognition schematic diagram shown in fig. 1, is mainly focused on to ensure the accuracy and success rate of face recognition by capturing higher definition face features and increasing the computational complexity. Some feature-based face detection techniques, for example, perform face detection by using features such as color, contour, texture, structure, or histogram. Or for example, some face detection technologies based on template matching need to extract a face template from a database, then adopt a certain template matching strategy to match the captured face image with the picture extracted from the template library, and determine the face size and position information according to the correlation and the size of the matched template. Or for example, some face detection technologies based on statistics, a large number of face positive and negative sample libraries formed by collecting images of 'faces' and 'non-faces' are collected, and the system is intensively trained by adopting a statistical method, so that the detection and classification of the faces and the non-faces are realized.

Because the existing scheme only focuses on extracting human face features when unlocking is performed, the input data is a high-definition large image of about 10M, and the unlocking time delay is long and is about 1.2 seconds.

Disclosure of Invention

The embodiment of the application provides an unlocking method and device. The terminal equipment continuously collects and acquires the environmental information in a low-power consumption mode and determines whether the current environment is a safe environment. Under the safe environment, the identification and the unlocking of the terminal equipment are completed in a low-power-consumption mode, so that the calculation complexity is reduced, and the quick identification and the unlocking are realized.

In a first aspect, an unlocking method is provided, and the method includes: collecting environmental information of a current environment; identifying environment information and determining whether the current environment is a safe environment; when the current environment is identified to be a safe environment, determining a face matching threshold; collecting gray face information; identifying gray face information and determining face matching degree; and when the face matching degree is greater than or equal to the face matching threshold, unlocking the terminal equipment. According to the method and the device, the safe environment is identified, the gray level image face identification is carried out under the safe environment, the rapid identification and unlocking are carried out through computer vision based on artificial intelligence under the safe environment, the calculation complexity is reduced, and the unlocking speed is increased.

In one possible embodiment, the environmental information includes: one or more of grayscale ambient image information, ambient sound information, speed information, and location information. The application can identify the current environment by acquiring various different environment information, so that the identification result is more accurate.

In one possible embodiment, identifying the environment information and determining whether the current environment is a secure environment comprises: identifying gray level environment image information and determining the safety level of the current environment; or identifying environmental sound information and determining the safety level of the current environment; or identifying speed information and determining the safety level of the current environment; or identifying the position information and determining the safety level of the current environment. According to the method and the device, the safety level of the current safety environment can be respectively determined according to different environment information, so that the identification result is more accurate.

In one possible embodiment, determining the face matching threshold value according to the security context comprises: and determining a face matching threshold according to the security level. According to the method and the device, different face matching thresholds can be matched according to different security environments, so that unlocking is faster under the situation that the security level is higher.

In one possible implementation, the security level is inversely related to the face matching threshold. According to the method and the device, the safety level is inversely related to the face matching threshold, so that the higher the safety environment level is, the lower the face matching threshold is, and the safer environment face recognition unlocking is easier.

In one possible embodiment, after identifying the current environment as the secure environment, the method further comprises: collecting user voice information; identifying the user voice information, and determining whether the user voice information is pre-stored user voice; when the face matching degree is greater than or equal to the face matching threshold, unlocking the terminal equipment comprises the following steps: and when the face matching degree is greater than or equal to the face matching threshold value and the voice information is the pre-stored user voice, unlocking the terminal equipment. This application still combines pronunciation to carry out the unblock, after discerning the environment for the safety ring border, discerns user's pronunciation and unblock promptly, can combine more modes to carry out the unblock under having guaranteed the safety ring border, has greatly promoted user experience when increasing the security.

In one possible embodiment, the method further comprises: and when the current environment is identified to be a non-safe environment, performing conventional unlocking judgment on the terminal equipment. According to the method and the device, after the current environment is identified to be not a safe environment, conventional unlocking can be performed, and various unlocking modes of the user in different environments are guaranteed.

In a second aspect, an unlocking method is provided, and the method includes: collecting environmental information of a current environment; identifying environment information and determining whether the current environment is a safe environment; when the current environment is identified as a safe environment, collecting user voice information; identifying the user voice information, and determining whether the user voice information is pre-stored user voice; and when the user voice information is pre-stored user voice, unlocking the terminal equipment. This application is through discerning the security environment to carry out speech recognition under the security environment, realized under the security environment, based on artificial intelligence and carry out quick discernment and unblock through pronunciation, reduced the calculation complexity and promoted unblock speed simultaneously.

In one possible embodiment, after identifying the current environment as the secure environment, the method further comprises: determining a face matching threshold; collecting gray face information; identifying gray face information and determining face matching degree; when the voice information is pre-stored user voice, the unlocking operation of the terminal equipment comprises the following steps: and when the voice information is pre-stored user voice and the face matching degree is greater than or equal to the face matching threshold value, unlocking the terminal equipment. This application still combines face identification to unblock, after discerning the environment for the security environment, carries out the unblock through face identification, has guaranteed that can have under the security environment to combine more modes to unblock, has greatly promoted user experience when increasing the security.

In one possible embodiment, determining the unlock match threshold comprises, based on the security context: and determining an unlocking matching threshold according to the security level of the current environment. According to the method and the device, different face matching thresholds can be matched according to different security environments, so that unlocking is faster under the situation that the security level is higher.

In a third aspect, an unlocking device is provided, the device comprising: the acquisition module is used for acquiring the environmental information of the current environment; the identification module is used for identifying the environment information and determining whether the current environment is a safe environment; the determining module is used for determining a face matching threshold when the current environment is identified as a safe environment; the acquisition module is also used for acquiring gray face information; the identification module is also used for identifying the gray face information and determining the face matching degree; and the unlocking module is used for unlocking the terminal equipment when the face matching degree is greater than or equal to the face matching threshold. According to the method and the device, the safe environment is identified, the gray level image face identification is carried out under the safe environment, the rapid identification and unlocking are carried out through computer vision based on artificial intelligence under the safe environment, the calculation complexity is reduced, and the unlocking speed is increased.

In one possible embodiment, the identification module is further configured to: identifying gray level environment image information and determining the safety level of the current environment; or identifying environmental sound information and determining the safety level of the current environment; or identifying speed information and determining the safety level of the current environment; or identifying the position information and determining the safety level of the current environment. According to the method and the device, the safety level of the current safety environment can be respectively determined according to different environment information, so that the identification result is more accurate.

In one possible embodiment, the determining module is further configured to: and determining an unlocking matching threshold according to the safety level. According to the method and the device, different face matching thresholds can be matched according to different security environments, so that unlocking is faster under the situation that the security level is higher.

In one possible embodiment, the collecting module is further configured to collect user voice information; the recognition module is also used for recognizing the user voice information and determining whether the user voice information is prestored user sound; the unlocking module is also used for unlocking the terminal equipment when the face matching degree is greater than or equal to the face matching threshold value and the voice information is pre-stored user voice. This application still combines pronunciation to carry out the unblock, after discerning the environment for the safety ring border, discerns user's pronunciation and unblock promptly, can combine more modes to carry out the unblock under having guaranteed the safety ring border, has greatly promoted user experience when increasing the security.

In one possible embodiment, the unlocking module is further configured to: and when the current environment is identified to be a non-safe environment, performing conventional unlocking judgment on the terminal equipment. According to the method and the device, after the current environment is identified to be not a safe environment, conventional unlocking can be performed, and various unlocking modes of the user in different environments are guaranteed.

In a fourth aspect, there is provided an unlocking device, comprising: the acquisition module is used for acquiring the environmental information of the current environment; the identification module is used for identifying the environment information and determining whether the current environment is a safe environment; the acquisition module is also used for acquiring the voice information of the user when the current environment is identified as the safe environment; the recognition module is also used for recognizing the user voice information and determining whether the user voice information is prestored user sound; and the unlocking module is used for unlocking the terminal equipment when the user voice information is pre-stored user voice. This application is through discerning the security environment to carry out speech recognition under the security environment, realized under the security environment, based on artificial intelligence and carry out quick discernment and unblock through pronunciation, reduced the calculation complexity and promoted unblock speed simultaneously.

In one possible embodiment, the apparatus further comprises: the determining module is used for determining a face matching threshold when the current environment is identified as a safe environment; the acquisition module is also used for acquiring gray face information; the identification module is also used for identifying the gray face information and determining the face matching degree; the unlocking module is also used for unlocking the terminal equipment when the voice information is pre-stored user voice and the face matching degree is greater than or equal to the face matching threshold value. The application further combines the face recognition to unlock, after the environment is recognized to be the safe environment, the unlocking is carried out through the face recognition, so that the unlocking can be carried out in more modes under the safe environment, and the user experience is greatly improved while the safety is improved.

In one possible embodiment, the determining module is further configured to: and determining an unlocking matching threshold according to the security level of the current environment. According to the method and the device, different face matching thresholds can be matched according to different security environments, so that unlocking is faster under the situation that the security level is higher.

In a fifth aspect, a computer-readable storage medium is provided, having instructions stored thereon, wherein the instructions, when executed on a terminal, cause the terminal to perform the method of the first aspect.

In a sixth aspect, a computer-readable storage medium is provided, having instructions stored thereon, which, when executed on a terminal, cause the terminal to perform the method of the second aspect.

In a seventh aspect, a terminal device is provided, wherein the terminal device includes a processor, the processor is coupled with a memory, and the processor reads and executes instructions in the memory, so that the terminal device implements the method of the first aspect.

In an eighth aspect, a terminal device is provided, wherein the terminal device includes a processor, the processor is coupled with a memory, and the processor reads and executes instructions in the memory, so that the terminal device implements the method of the second aspect.

The application discloses an unlocking method and device, wherein whether a current environment is a safe environment or not is identified, and a low-pixel picture is collected to perform face identification matching and unlocking in the safe environment; or under the safe environment, determining whether the user is the user himself or not by matching the voiceprint, and unlocking. The unlocking method and the unlocking device have the advantages that the quick identification and unlocking can be realized through computer vision based on artificial intelligence in a safe environment, the calculation complexity is reduced, and the unlocking speed is increased.

Drawings

FIG. 1 is a schematic diagram of a prior art face recognition;

fig. 2 is a schematic view of an application scenario of an unlocking method according to an embodiment of the present application;

fig. 3 is a flowchart of an unlocking method according to an embodiment of the present application;

fig. 4 is a flowchart of another unlocking method provided in the embodiment of the present application;

fig. 5 is a flowchart of another unlocking method provided in the embodiment of the present application;

FIG. 6 is a block diagram of an unlocking system according to an embodiment of the present disclosure;

fig. 7 is a schematic view of an unlocking device according to an embodiment of the present disclosure;

FIG. 8 is a schematic view of another unlocking device provided in the embodiments of the present application;

fig. 9 is a schematic diagram of a terminal device according to an embodiment of the present application;

fig. 10 is a schematic diagram of another terminal device provided in the embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.

The method and the device are applied to the scene of safe environment unlocking. In one example, the unlocking scenario of the security environment may be a vehicle-mounted environment, an office environment, a home environment, or the like. As shown in fig. 2, an application scenario diagram in a vehicle-mounted environment is shown, after a user enters the vehicle-mounted environment, a terminal device waiting for unlocking may acquire current image information through a camera by using computer vision, and determine whether the current environment is a safe environment. If the terminal equipment determines that the current environment is a safe environment, face recognition can be carried out by collecting gray images. Certainly, if the terminal device determines that the current environment is the safe environment, the voiceprint of the user can be collected for judgment, so that the terminal device is unlocked under the condition that the voiceprint is judged to be correct. It should be understood by those skilled in the art that fig. 2 is merely an illustration of one security environment, and the present application may be applied to any other security environment, and the present application is not limited herein. According to the method, the unlocking device and the unlocking method, under the safe environment, the unlocking device can quickly identify and unlock based on artificial intelligence and through computer vision, so that the calculation complexity is reduced, and the unlocking speed is increased.

It should be noted by those skilled in the art that the terminal device referred to in the present application may be any smart device having a camera function and/or a sound collecting function, such as a mobile phone, a wearable device, a tablet computer, and a vehicle-mounted terminal.

The technical solutions in the embodiments of the present application will be described in detail below with reference to the drawings in the embodiments of the present application.

As shown in fig. 3, fig. 3 is a flowchart of an unlocking method according to an embodiment of the present application.

Fig. 3 shows a flowchart of an unlocking method, which may include the following steps:

s301, collecting environmental information of the current environment.

In one embodiment, the terminal device collects environment information of a current environment through a camera. The environment information may be any information capable of describing the current environment, and for example, the environment information may include grayscale environment image information, environment sound information, speed information, position information, and the like.

In one example, the camera of the terminal device may be in a normally open state, and at this time, the camera of the terminal device collects current environmental information. In another example, a camera of the terminal device may be turned on when the terminal device needs to be unlocked, and the terminal device may still collect the environment information through the camera. In another example, for the case that the camera is turned on when the camera needs to be unlocked, the face information may be collected at the same time, and the terminal device may only focus on the environment information and ignore other information. In another example, a microphone of the terminal device may also collect ambient sound information, a speed sensor of the terminal device may also collect current speed information, and a position sensor of the terminal device may also locate the current position. It is to be understood that the present application also encompasses that corresponding information may be collected by any other sensor for describing the current environment.

S302, identifying the environment information and determining whether the current environment is a safe environment.

In one embodiment, the terminal device identifies current environment information and determines whether the current environment is a secure environment.

In one example, the terminal device identifies the environment information acquired by the camera through a pre-trained safe environment identification model, and identifies whether the current environment is a safe environment. It should be noted by those skilled in the art that the secure environment may be any environment such as an in-vehicle environment, an office environment, a residential home environment, etc., and the application is not limited thereto.

In one example, the secure environment recognition model is a neural network model that is pre-trained through machine learning. The training process may be to collect data through hardware, for example, the hardware may include a camera, a microphone, a speed sensor, a position sensor, a universal serial bus interface, a wireless interface, a bluetooth interface, and the like. And then the data collected by the hardware is transmitted to the service corresponding to the service layer. For example, data collected by a camera is transmitted to a camera service, and data collected by hardware such as a speed sensor, a position sensor, a universal serial bus interface, a wireless interface, a bluetooth interface and the like is transmitted to an environment sensing service. Meanwhile, the camera service located in the service layer also transmits the output data to the environment perception service. And then the environment perception service transmits the input data to a classifier of a decision layer for making a safety environment decision, and trains a safety environment recognition model. In another example, since the training process of the security environment recognition model is performed by a microphone, a speed sensor, a position sensor, a usb interface, a wireless interface, a bluetooth interface, etc., when the terminal device is connected to a car or a wireless network located in a home via the usb interface, the wireless interface, the bluetooth interface, etc., the environment recognition model can also recognize that the current device is located in the security environment. In another example, the environment recognition model can also collect the sound of opening and closing a vehicle door or other environment sounds with typical characteristics, and indirectly judge whether the terminal equipment is in a safe environment.

It can be understood by those skilled in the art that the manner of determining the security environment provided in the above embodiments can be used as an auxiliary determination, and the main determination of the security environment is based on the determination still performed according to the collected gray-scale environment image information.

S303, when the current environment is identified to be a safe environment, determining a human face matching threshold value.

In one embodiment, when the terminal device identifies that the current environment is any environment such as a vehicle-mounted environment, an office environment, a family living environment and the like according to a safety environment identification model trained in advance, and determines that the current environment is a safety environment, the current face matching threshold is determined.

In one example, different security environments may have different levels of security, so the terminal device may set different security levels. For example, the security of the public environment is very low, and the terminal device can set the security level of the public environment to level 1; the security of the office environment is relatively low, and the terminal equipment can set the security level of the public environment to be level 2; the safety of the vehicle-mounted environment is higher, and the terminal equipment can set the safety level of the vehicle-mounted environment to be 3 level; the safety of the domestic living environment is very high, and the terminal equipment can set the safety level of the domestic living environment to be 4 level. The setting of the safety level can be set from low to high in sequence, namely, the level 1 is the lowest safety level, and then the higher the number is, the higher the safety level is, the safer the environment is represented; the setting of the security level can also be set from high to low, that is, level 1 is the highest security level, and then the lower the security level is, the more unsafe the environment is, the application is not limited herein.

In another example, the terminal device may set different face matching thresholds for different security levels. For example, for a home dwelling environment with the highest security level, such as level 4, a very low face matching threshold, such as 60% or 0.6, may be set. Since the terminal device detects that the current security environment level is high, it can be considered that the unlocking is very safe in the current environment. At this time, compared with the threshold value for unlocking in a conventional manner, for example, the threshold value may be 98% or 0.98, the terminal device may appropriately reduce the face matching threshold value, so that the terminal device may unlock more quickly. For another example, for a vehicle-mounted environment, if the terminal device determines that the current security level is level 3, which is relatively secure, the terminal device may set a lower face matching threshold, for example, 70% or 0.7. It can be understood that the higher the security level of the environment, the lower the face matching threshold set by the terminal device may be, and vice versa.

And S304, collecting gray-scale face information.

In one embodiment, after the terminal device identifies that the current environment is a safe environment, the terminal device may further employ a camera to acquire grayscale face information.

In one example, the terminal device identifies that the current environment is any environment such as a vehicle-mounted environment, an office environment, a home environment and the like according to a safety environment identification model trained in advance, and determines that the current environment is a safety environment. And then, the terminal equipment can acquire a gray face image by adopting low power through a camera and obtain gray face information. In one example, the low resolution of a grayscale face image may be 480 x 480. In another example, the camera in S301 may also acquire the gray-scale face information at the same time when acquiring the environment information, and at this time, the gray-scale face information does not need to be acquired again.

According to the method and the device, the calculated amount required by unlocking is reduced by acquiring the face image with lower data volume, such as the gray face information, in the safe environment, and the low-power-consumption quick unlocking is realized

S305, identifying the gray face information and determining the face matching degree.

In one embodiment, the terminal device identifies the collected gray-scale face information and determines the face matching degree.

In one example, the terminal device identifies the collected gray-scale face information, matches the face information according to the face information pre-stored in the terminal device by the user, and determines the matching degree between the currently collected gray-scale face information and the pre-stored face information, that is, the face matching degree. In one example, the face matching degree may be a percentage count mode, or a decimal technology mode, for example, the matching degree is 70%, or the matching degree is 0.7.

And S306, when the face matching degree is larger than or equal to the face matching threshold, unlocking the terminal equipment.

In one embodiment, the terminal device compares the determined face matching degree with a face matching threshold, and when the face matching degree is greater than or equal to the face matching threshold, the terminal device determines that the face of the user is currently recognized and performs an unlocking operation.

In one example, the terminal device compares the determined face matching degree with a face matching threshold, for example, the current face matching degree is 80% or 0.8, and the current face matching threshold is 70% or 0.7. The terminal device determines that 80% or 0.8 is greater than 70% or 0.7, and the terminal device determines that the face of the user is currently recognized, and performs an unlocking operation. In another example, if the current face matching degree is 80% or 0.8, and the current face matching threshold is 90% or 0.9. And the terminal equipment determines that the face of the user is not recognized currently, and then the terminal equipment executes conventional unlocking operation.

The application discloses an unlocking method and device, wherein whether the current environment is a safe environment or not is identified, a low-pixel picture is collected in the safe environment to carry out face identification and matching, and unlocking operation is executed after the face of a user is identified. The unlocking method and the unlocking device have the advantages that the quick identification and unlocking can be realized through computer vision based on artificial intelligence in a safe environment, the calculation complexity is reduced, and the unlocking speed is increased.

As shown in fig. 4, fig. 4 is a flowchart of another unlocking method provided in the embodiment of the present application.

Fig. 4 shows a flow chart of another unlocking method, which may include the following steps:

s401, collecting the environmental information of the current environment.

In one example, the camera of the terminal device may be in a normally open state, and at this time, the camera of the terminal device collects current environmental information. In another example, a camera of the terminal device may be turned on when the terminal device needs to be unlocked, and the terminal device may still collect the environment information through the camera. In another example, for the case that the camera is turned on when unlocking is required, other information except the environment may be collected at the same time, and the terminal device may only focus on the environment information and ignore the other information. In another example, a microphone of the terminal device may also collect ambient sound information, a speed sensor of the terminal device may also collect current speed information, and a position sensor of the terminal device may also locate the current position. It is to be understood that the present application also encompasses that corresponding information may be collected by any other sensor for describing the current environment.

S402, identifying the environment information and determining whether the current environment is a safe environment.

It is understood that S402 is the same as S302 in the manner of identifying the environment, and for convenience of description, the details are not repeated herein.

It can be understood by those skilled in the art that the manner of determining the security environment given in the above embodiments can be used as an auxiliary determination, and the main determination of the security environment is subject to the determination according to the acquired environment image.

And S403, when the current environment is identified as the safe environment, collecting the voice information of the user.

In one embodiment, when the terminal device recognizes that the current environment is the safe environment according to the pre-trained safe environment recognition model, the terminal device may collect the user voice information through the microphone.

In one example, the terminal device identifies that the current environment is any environment such as a vehicle-mounted environment, an office environment, a home environment and the like according to a safety environment identification model trained in advance, and determines that the current environment is a safety environment. And then, the terminal equipment collects the voice information of the current user during speaking through a microphone.

S404, identifying the user voice information, and determining whether the user voice information is a pre-stored user voice.

In one embodiment, the terminal device identifies the collected user voice information and determines whether the currently collected user voice information is a pre-stored user voice.

In one example, the terminal device may store the user's voice in advance and use this as a reference. When the unlocking operation is needed, the terminal equipment acquires the current user voice information and compares the current user voice information with the pre-stored user voice information to determine whether the currently acquired user voice information is the pre-stored user voice. In one example, it may be determined whether the collected user voice information is a pre-stored user voice by detecting a voiceprint. Wherein, the voiceprint not only has the characteristics of specificity, but also has the characteristic of relative stability. When a person becomes adult, the voice can be kept relatively stable and unchanged for a long time. Experiments prove that whether a speaker intentionally imitates the voice and tone of other people or speaks with whisper and whisper, even if the imitation is vivid, the voiceprint of the speaker is always the same. Therefore, the method and the device can acquire the voiceprint information from the voice information by acquiring the user voice information of the current user, and then compare the voiceprint information with the user voiceprint which is stored in advance so as to determine whether the unlocking right exists currently.

S405, when the user voice information is pre-stored user voice, unlocking the terminal equipment.

In one embodiment, when the terminal device determines that the user voice information is the pre-stored user voice, the terminal device is determined to be unlocked.

In one example, the terminal device determines whether the current user voice information is a pre-stored user voice, that is, the terminal device determines whether the current user voiceprint information is a pre-stored user voiceprint, and after the terminal device determines that the current user voiceprint information is a pre-stored user voiceprint, the terminal device determines that the pre-stored user voice is currently recognized, and then the terminal device performs an unlocking operation. In another example, if the terminal device determines that the current user voiceprint information is not a pre-stored user voiceprint, the terminal device determines that the pre-stored user voice is not currently recognized, and then the terminal device performs a normal unlocking operation.

The application discloses an unlocking method and device, which are used for identifying whether the current environment is a safe environment or not, determining whether the current environment is the user himself or not by matching voiceprints in the safe environment, and executing unlocking operation after the voice of the user is identified. The unlocking method and the unlocking device have the advantages that the quick identification and unlocking can be realized through computer vision based on artificial intelligence in a safe environment, the calculation complexity is reduced, and the unlocking speed is increased.

As shown in fig. 5, fig. 5 is a flowchart of another unlocking method provided in the embodiment of the present application.

Fig. 5 is a flowchart of another unlocking method, which is implemented by combining the methods in fig. 3 and fig. 4 to perform face recognition and voice recognition to unlock the terminal device, and the method may include the following steps:

s501, collecting environmental information of the current environment.

In one example, the camera of the terminal device may be in a normally open state, and at this time, the camera of the terminal device collects current environmental information. In another example, a camera of the terminal device may be turned on when the terminal device needs to be unlocked, and the terminal device may still collect the environment information through the camera. In another example, for the case that the camera is turned on when the terminal needs to be unlocked, other information except the environment may be collected at the same time, and the terminal device may focus on only the environment information and ignore the other information. In another example, a microphone of the terminal device may also collect ambient sound information, a speed sensor of the terminal device may also collect current speed information, and a position sensor of the terminal device may also locate the current position. It is to be understood that the present application also encompasses that corresponding information may be collected by any other sensor for describing the current environment.

S502, identifying the environment information and determining whether the current environment is a safe environment.

In another example, if the current environment is determined to be the secure environment, S503 and S506 are executed simultaneously.

And S503, determining a face matching threshold value.

In one embodiment, according to a safety environment recognition model trained in advance, the current environment is recognized to be any environment such as a vehicle-mounted environment, an office environment, a family living environment and the like, and when the current environment is determined to be the safety environment, the current face matching threshold value is determined.

In another example, the terminal device may set different face matching thresholds for different security levels. For example, for a home dwelling environment with the highest security level, such as level 4, a very low face matching threshold, such as 60% or 0.6, may be set. Since the terminal device detects that the current security environment level is high, it can be considered that the unlocking is very safe in the current environment. At this time, compared with the threshold value for unlocking in a conventional manner, for example, the threshold value may be 98% or 0.98, the terminal device may appropriately reduce the face matching threshold value, so that the terminal device may unlock more quickly. For different security levels, according to the security level. For another example, for a vehicle-mounted environment, if the terminal device determines that the current security level is level 3, which is relatively secure, the terminal device may set a lower face matching threshold, for example, 70% or 0.7. It can be understood that the higher the security level of the environment, the lower the face matching threshold set by the terminal device may be, and vice versa.

And S504, collecting gray-level face information.

In one embodiment, the terminal device may employ a camera to collect the grayscale face information.

In one example, the terminal device collects a gray-scale face image by using low power through a camera, and obtains gray-scale face information. In one example, the low resolution of a grayscale face image may be 480 x 480. In another example, the camera in S301 may also acquire the gray-scale face information at the same time when acquiring the environment information, and at this time, the gray-scale face information does not need to be acquired again.

And S505, identifying the gray face information and determining the face matching degree.

It should be noted by those skilled in the art that after S502, S506 may be performed simultaneously with S503.

S506, collecting voice information of the user.

And S507, identifying the voice information of the user.

In one embodiment, the terminal device identifies the collected user voice information.

In one example, the terminal device identifies voiceprint information in the collected user voice information. Wherein, the voiceprint not only has the characteristics of specificity, but also has the characteristic of relative stability. When a person becomes adult, the voice can be kept relatively stable and unchanged for a long time. Experiments prove that whether a speaker intentionally imitates the voice and tone of other people or speaks with whisper and whisper, even if the imitation is vivid, the voiceprint of the speaker is always the same. Therefore, the method and the device can acquire the voiceprint information from the voice information by acquiring the user voice information of the current user, and then compare the voiceprint information with the user voiceprint which is stored in advance so as to determine whether the unlocking right exists currently.

After S505 and S507, S508 may also be performed.

S508, whether the face matching degree is larger than or equal to the face matching threshold value or not and whether the user voice information is pre-stored user voice or not are determined.

In one embodiment, the terminal device compares the determined face matching degree with a face matching threshold, and when the face matching degree is greater than or equal to the face matching threshold, the terminal device determines that the face of the user is currently recognized; and the terminal device also needs to determine whether the user voice information is a pre-stored user voice.

In one example, the terminal device compares the determined face matching degree with a face matching threshold, for example, the current face matching degree is 80% or 0.8, and the current face matching threshold is 70% or 0.7. The terminal device determines that 80% or 0.8 is greater than 70% or 0.7 and the terminal device determines that the face of the user is currently recognized. In another example, if the current face matching degree is 80% or 0.8, and the current face matching threshold is 90% or 0.9. The terminal device determines that the face of the user is not currently recognized. And the terminal equipment also determines whether the currently collected user voice information is the pre-stored user voice. In another example, the terminal device may store the user's voice in advance and use it as a reference. The terminal equipment acquires the current user voice information and compares the current user voice information with the pre-stored user voice information to determine whether the currently acquired user voice information is the pre-stored user voice. In one example, whether the collected voice is the pre-stored voice of the user can be determined by detecting the voiceprint information in the collected voice information of the user. In one example, the terminal device determines whether the current user voice information is a pre-stored user voice, that is, the terminal device determines whether the current user voiceprint information is a pre-stored user voiceprint, and after the terminal device determines that the current user voiceprint information is a pre-stored user voiceprint, the terminal device determines that the pre-stored user voice is currently recognized. In another example, if the terminal device determines that the current user voiceprint information is not a pre-stored user voiceprint, the terminal device determines that the pre-stored user voice is not currently recognized.

When both the condition that the terminal device recognizes the face and the condition that the terminal device recognizes the pre-stored user voice are satisfied, S509 is performed. In another example, when either one of the two conditions that the terminal device recognizes the human face and the terminal device recognizes the pre-stored user voice is not satisfied, the terminal device is normally unlocked.

According to the method and the device, the unlocking is carried out by combining the face recognition and the voice recognition, after the environment is recognized to be a safe environment, the face recognition and the voice recognition can be combined at the same time, and the unlocking is carried out after the requirements of the face recognition and the voice recognition are met at the same time. The unlocking can be performed in more modes under the condition of ensuring the safety environment, and the user experience is greatly improved while the safety is improved.

And S509, unlocking the terminal equipment.

In one embodiment, when the terminal device determines that the face matching degree is greater than or equal to the face matching threshold and the user voice information is the pre-stored user voice, the terminal device determines that the face of the user is currently recognized and the voice of the user is recognized at the same time, and at this time, it determines to unlock the terminal device.

In one example, when the terminal device determines that the face matching degree is greater than or equal to a face matching threshold, a face is recognized; and the terminal equipment determines that the user voice information is the pre-stored user voice, namely after recognizing the pre-stored user voice. The terminal device determines that the unlocking operation can be performed.

The application discloses an unlocking method and device, wherein whether the current environment is a safe environment or not is identified, a low-pixel picture is collected to perform face recognition matching under the safe environment, whether the user is determined by matching a voiceprint, and when the face of the user is determined to be recognized and the voice of the user is recognized, the unlocking operation is executed. The unlocking method and the unlocking device have the advantages that the quick identification and unlocking can be realized through computer vision based on artificial intelligence in a safe environment, the calculation complexity is reduced, and the unlocking speed is increased.

As shown in fig. 6, fig. 6 is a schematic diagram of an unlocking system framework provided in an embodiment of the present application.

Fig. 6 shows a schematic diagram of an unlocking system framework, which includes an application layer 601, a system architecture layer 602, and a hardware layer 603.

The system architecture first collects external information such as environment information, gray-level face information, and user voice information through one or more hardware layers (hardware) 603. One or more of the hardware may include a camera, a microphone, a speed sensor, a position sensor, a universal serial bus interface, a wireless interface, a bluetooth interface, etc. The hardware layer 603 then transmits the collected data to a system architecture layer (framework)602 for analysis and identification of the data. Wherein the system architecture layer 602 is configured to identify whether the current environment is a safe environment, and whether face information is detected and/or whether a pre-stored user voice is identified. Finally, the system architecture layer 602 transmits the identified result to the application layer (application)601, so that the unlocking application at the application layer 601 performs an unlocking operation according to the result of the system architecture layer 602.

As shown in fig. 7, fig. 7 is a schematic view of an unlocking device according to an embodiment of the present application.

Fig. 7 shows an unlocking device 700, the device 700 comprising: an acquisition module 701, configured to acquire environment information of a current environment; an identifying module 702, configured to identify environment information and determine whether a current environment is a secure environment; a determining module 703, configured to determine a face matching threshold when the current environment is identified as a safe environment; the acquisition module 701 is further configured to acquire gray-level face information; the recognition module 702 is further configured to recognize gray-level face information and determine a face matching degree; and the unlocking module 704 is used for unlocking the terminal equipment when the face matching degree is greater than or equal to the face matching threshold.

In one possible embodiment, the environmental information includes: one or more of grayscale ambient image information, ambient sound information, speed information, and location information.

In one possible implementation, the identification module 702 is further configured to: identifying gray level environment image information and determining the safety level of the current environment; or identifying environmental sound information and determining the safety level of the current environment; or identifying speed information and determining the safety level of the current environment; or identifying the position information and determining the safety level of the current environment.

In one possible implementation, the determining module 703 is further configured to: and determining an unlocking matching threshold according to the safety level.

In one possible implementation, the security level is inversely related to the face matching threshold.

In a possible embodiment, the collecting module 701 is further configured to collect voice information of a user; the recognition module 702 is further configured to recognize the user voice information, and determine whether the user voice information is a pre-stored user sound; the unlocking module 704 is further configured to unlock the terminal device when the face matching degree is greater than or equal to the face matching threshold and the voice information is a pre-stored user voice.

In one possible implementation, the unlocking module 704 is further configured to: and when the current environment is identified to be a non-safe environment, performing conventional unlocking judgment on the terminal equipment.

As shown in fig. 8, fig. 8 is a schematic view of another unlocking device provided in the embodiment of the present application.

Fig. 8 shows an unlocking device 800, the device 800 comprising: an acquisition module 801, configured to acquire environment information of a current environment; an identifying module 802, configured to identify environment information and determine whether a current environment is a secure environment; the acquisition module 801 is further configured to acquire user voice information when the current environment is identified as a safe environment; the recognition module 802 is further configured to recognize the user voice information, and determine whether the user voice information is a pre-stored user sound; and the unlocking module 804 is configured to unlock the terminal device when the user voice information is a pre-stored user voice.

In one possible embodiment, the identification module 802 is further configured to: identifying gray level environment image information and determining the safety level of the current environment; or identifying environmental sound information and determining the safety level of the current environment; or identifying speed information and determining the safety level of the current environment; or identifying the position information and determining the safety level of the current environment.

In one possible embodiment, the apparatus 800 further comprises: a determining module 803, configured to determine a face matching threshold when the current environment is identified as a safe environment; the acquisition module 801 is further configured to acquire gray-level face information; the recognition module 802 is further configured to recognize gray-level face information and determine a face matching degree; the unlocking module 804 is further configured to unlock the terminal device when the voice information is a pre-stored user voice and the face matching degree is greater than or equal to the face matching threshold.

In one possible implementation, the determining module 803 is further configured to: and determining an unlocking matching threshold according to the security level of the current environment.

In one possible implementation, the unlocking module 804 is further configured to: and when the current environment is identified to be a non-safe environment, performing conventional unlocking judgment on the terminal equipment.

Fig. 9 is a schematic diagram of a terminal device according to an embodiment of the present application.

Fig. 9 provides a terminal device 900, which device 900 may include a processor 901, memory 902, a communication interface 903, and a bus 904. The processor 901, the memory 902, the communication interface 903 and the communication connection may be established through a bus 904 in the terminal device. The communication interface 903 is used for transmitting and receiving external information.

Processor 901 may be a Central Processing Unit (CPU).

Memory 902 may include volatile memory (volatile memory), such as random-access memory (RAM); the memory 902 may also include a non-volatile memory (e.g., a read-only memory (ROM), a flash memory, a Hard Disk Drive (HDD) or a Solid State Drive (SSD); the memory 902 may also comprise a combination of the above-described types of memory.

The embodiments of fig. 3 and 5 described above provide a method for acquiring diversity data, which is executed by the processor 901. The document data and/or calculated data in this application will be stored in the memory 902. In addition, the memory 902 will be used for storing program instructions and the like executed by the processor for implementing one method for acquiring diversity data described in the embodiments of fig. 3 and 5.

Fig. 10 provides a terminal device 1000, and the device 1000 may include a processor 1001, a memory 1002, a communication interface 1003, and a bus 1004. The processor 1001, the memory 1002, and the communication interface 1003 in the terminal apparatus can establish communication connection through the bus 1004. The communication interface 1003 is used for transmitting and receiving external information.

The processor 1001 may be a Central Processing Unit (CPU).

The memory 1002 may include volatile memory (volatile memory), such as random-access memory (RAM); the memory 1002 may also include a non-volatile memory (e.g., a read-only memory (ROM)), such as a flash memory, a Hard Disk Drive (HDD), or a Solid State Drive (SSD); the memory 1002 may also comprise a combination of the above-described types of memory.

The embodiments of fig. 4 and 5 described above provide a method for acquiring diversity data, which is executed by the processor 1001. The file data and/or calculated data in this application will be stored in the memory 1002. In addition, the memory 1002 will be used for storing program instructions and the like executed by the processor to implement a method for acquiring diversity data according to the embodiments in fig. 4 and 5.

It will be further appreciated by those of ordinary skill in the art that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

It will be understood by those skilled in the art that all or part of the steps in the method for implementing the above embodiments may be implemented by a program, and the program may be stored in a computer-readable storage medium, where the storage medium is a non-transitory medium, such as a random access memory, a read only memory, a flash memory, a hard disk, a solid state disk, a magnetic tape (magnetic tape), a floppy disk (floppy disk), an optical disk (optical disk), and any combination thereof.

The above description is only for the preferred embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An unlocking method, characterized in that the method comprises:

collecting environmental information of a current environment;

identifying the environment information, and determining whether the current environment is a safe environment;

when the current environment is identified to be a safe environment, determining a face matching threshold;

collecting gray face information;

identifying the gray-scale face information to determine the face matching degree;

and when the face matching degree is greater than or equal to the face matching threshold, unlocking the terminal equipment.

2. The method of claim 1, wherein the environmental information comprises: one or more of grayscale ambient image information, ambient sound information, speed information, and location information.

3. The method of claim 2, wherein the identifying the environment information, determining whether the current environment is a secure environment comprises:

identifying the gray level environment image information, and determining the safety level of the current environment; or

Identifying the environmental sound information, and determining the safety level of the current environment; or

Identifying the speed information and determining the safety level of the current environment; or

And identifying the position information and determining the safety level of the current environment.

4. The method of claim 3, wherein determining a face matching threshold based on the secure environment comprises:

and determining a face matching threshold according to the safety level.

5. The method of claim 4, wherein the security level is inversely related to the face matching threshold.

6. The method of claim 1, wherein after the identifying the current environment as a secure environment, the method further comprises:

collecting user voice information;

identifying the user voice information, and determining whether the user voice information is a pre-stored user sound;

when the face matching degree is greater than or equal to the face matching threshold, unlocking the terminal equipment comprises the following steps:

and when the face matching degree is greater than or equal to the face matching threshold value and the voice information is pre-stored user voice, unlocking the terminal equipment.

7. The method of any of claims 1-6, further comprising:

and when the current environment is identified to be a non-safe environment, performing conventional unlocking judgment on the terminal equipment.

8. An unlocking method, characterized in that the method comprises:

collecting environmental information of a current environment;

when the current environment is identified to be a safe environment, collecting user voice information;

and when the user voice information is pre-stored user voice, unlocking the terminal equipment.

9. The method of claim 8, wherein the context information comprises: one or more of grayscale ambient image information, ambient sound information, speed information, and location information.

10. The method of claim 9, wherein the identifying the environment information, determining whether the current environment is a secure environment comprises:

11. The method of claim 8, wherein after the identifying the current environment as a secure environment, the method further comprises:

determining a face matching threshold;

collecting gray face information;

when the voice information is pre-stored user voice, the unlocking operation of the terminal equipment comprises the following steps:

and when the voice information is pre-stored user voice and the face matching degree is greater than or equal to the face matching threshold, unlocking the terminal equipment.

12. The method of claim 11, wherein the determining an unlock match threshold from the secure environment comprises:

and determining an unlocking matching threshold according to the security level of the current environment.

13. The method of claim 12, wherein the security level is inversely related to the face matching threshold.

14. The method of any of claims 8-13, further comprising:

15. An unlocking device, characterized in that the device comprises:

the acquisition module is used for acquiring the environmental information of the current environment;

the identification module is used for identifying the environment information and determining whether the current environment is a safe environment;

the determining module is used for determining a face matching threshold when the current environment is identified as a safe environment;

the acquisition module is also used for acquiring gray face information;

the identification module is also used for identifying the gray-scale face information and determining the face matching degree;

and the unlocking module is used for unlocking the terminal equipment when the face matching degree is greater than or equal to the face matching threshold.

16. The apparatus of claim 15, wherein the environment information comprises: one or more of grayscale ambient image information, ambient sound information, speed information, and location information.

17. The apparatus of claim 16, wherein the identification module is further configured to:

18. The apparatus of claim 17, wherein the determination module is further configured to:

and determining an unlocking matching threshold according to the safety level.

19. The apparatus of claim 18, wherein the security level is inversely related to the face matching threshold.

20. The apparatus of claim 15,

the acquisition module is also used for acquiring the voice information of the user;

the recognition module is further used for recognizing the user voice information and determining whether the user voice information is a pre-stored user voice;

the unlocking module is further used for unlocking the terminal equipment when the face matching degree is larger than or equal to the face matching threshold and the voice information is pre-stored user voice.

21. The apparatus of any of claims 15-20, wherein the unlocking module is further configured to:

22. An unlocking device, characterized in that the device comprises:

the acquisition module is also used for acquiring user voice information when the current environment is identified as a safe environment;

and the unlocking module is used for unlocking the terminal equipment when the user voice information is pre-stored user voice.

23. The apparatus of claim 22, wherein the environment information comprises: one or more of grayscale ambient image information, ambient sound information, speed information, and location information.

24. The apparatus of claim 23, wherein the identification module is further configured to:

25. The apparatus of claim 22, wherein the apparatus further comprises:

the acquisition module is also used for acquiring gray face information;

the unlocking module is further used for unlocking the terminal equipment when the voice information is pre-stored user voice and the face matching degree is larger than or equal to the face matching threshold value.

26. The apparatus of claim 25, wherein the determination module is further configured to:

27. The apparatus of claim 26, wherein the security level is inversely related to the face matching threshold.

28. The apparatus of any of claims 22-27, wherein the unlocking module is further configured to:

29. A computer-readable storage medium having instructions stored thereon, which, when run on a terminal, cause the terminal to perform the method of any one of claims 1-7.

30. A computer-readable storage medium having instructions stored thereon, which, when run on a terminal, cause the terminal to perform the method of any one of claims 8-14.

31. A terminal device, characterized in that the terminal device comprises a processor coupled with a memory, the processor reading and executing instructions in the memory, causing the terminal device to implement the method of any of claims 1-7.

32. A terminal device, characterized in that the terminal device comprises a processor coupled with a memory, the processor reading and executing instructions in the memory, causing the terminal device to implement the method of any of claims 8-14.