WO2019196074A1

WO2019196074A1 - Electronic device and facial recognition method therefor

Info

Publication number: WO2019196074A1
Application number: PCT/CN2018/082893
Authority: WO
Inventors: 田浦延
Original assignee: 深圳阜时科技有限公司
Priority date: 2018-04-12
Filing date: 2018-04-12
Publication date: 2019-10-17
Also published as: CN108513706A

Abstract

Disclosed are an electronic device and a facial recognition method therefor, the facial recognition method comprising: S1, when the electronic device is required to perform facial recognition, determining whether a current ambient light intensity falls within a preset light intensity range; S2, when the current ambient light intensity falls outside the preset light intensity range, controlling the electronic device to perform front side light supplementation; S3, controlling an image sensor on the front side of the electronic device to collect a facial 3D image of an object in front of the electronic device; S4, matching facial features within the collected facial 3D image and depth information of the facial features with corresponding facial features in a preset facial 3D image template, so as to determine whether the identity of the object in front is legitimate. The electronic device operates the described facial recognition method.

Description

Electronic device and face recognition method thereof

Technical field

The application relates to an electronic device and a face recognition method thereof.

Background technique

With the trend of full-screen development of smart phones, the proportion of screens on the front of mobile phones is getting higher and higher. Therefore, other components such as buttons and fingerprint recognition modules cannot be set because the space is limited. Since the fingerprint recognition module cannot be set, in order to realize the identification, the industry has proposed to use the front camera on the front of the mobile phone to obtain facial information and perform facial recognition.

However, the recognition effect of the existing facial recognition technology needs to be improved.

Application content

The embodiments of the present application aim to at least solve one of the technical problems existing in the prior art. To this end, embodiments of the present application need to provide an electronic device and a face recognition method thereof.

The present application provides a facial recognition method for an electronic device, including the following steps:

S1, when the electronic device needs to perform facial recognition, determine whether the current ambient light intensity is within a preset light intensity range;

S2, when the current ambient light intensity is outside the preset light intensity range, controlling the electronic device to perform front fill light;

S3, controlling an image sensor on the front side of the electronic device to collect a 3D image of the face of the object in front of the electronic device;

S4: Match the facial features in the acquired facial 3D image with the facial features in the preset facial 3D image template to determine whether the identity of the front object is legal.

In the embodiment of the present application, when facial recognition is required, if the environment surrounding the electronic device is in the case of backlight or low light, the electronic device is controlled to perform front fill light, so that the electronic device can collect a clear facial image and improve facial recognition. The effect is to enhance the user experience.

In some embodiments, the step S1 further comprises: performing the step S3 when the current ambient light intensity is within a preset light intensity range.

In some embodiments, the fill light in step S2 is performed simultaneously with the acquisition of the face 3D image in step S3.

In some embodiments, in the step S2: when the current ambient light intensity is outside the preset light intensity range, the fill light device on the front side of the control electronic device is turned on to perform front fill light.

In some embodiments, one or more of the brightness, color temperature, and color of the fill light device are further adjusted when the fill light device on the front of the control electronics is turned on.

In some embodiments, the fill light device on the front side of the electronic device includes a fill light source disposed at the top end of the electronic device.

In some embodiments, the preset light intensity range includes an upper limit value and a lower limit value; in the step S2: when the current ambient light intensity is less than a lower limit value of the preset light intensity range And controlling the fill light source to be turned on to perform front fill light.

In some embodiments, the lower limit is a light intensity threshold for the object to be in a low light environment.

In some embodiments, the preset light intensity range includes an upper limit value and a lower limit value; in the step S2: when the current ambient light intensity is greater than an upper limit value of the preset light intensity range And controlling the fill light source to be turned on to perform front fill light.

In some embodiments, the upper limit is a light intensity threshold at which the object is in a backlighting environment.

In some embodiments, the preset light intensity range includes an upper limit value and a lower limit value; in the step S2: when the current ambient light intensity is less than a lower limit value of the preset light intensity range Adjust the display of the electronic device to the maximum brightness for front fill.

In some embodiments, in the step S2, if the sum of the current ambient light intensity and the maximum brightness of the display screen is still less than the lower limit value of the preset light intensity range, the fill light source of the front end of the electronic device is controlled. Turning on to enable the fill light source to perform front fill light together with the display screen.

In some embodiments, the electronic device further includes an image sensor located on the front side of the electronic device; the step S3 includes: controlling the image sensor of the electronic device to be turned on to collect a 2D image of the face of the object in front of the electronic device; The face 2D image is converted to obtain a face 3D image of the object in front of the electronic device.

In some embodiments, the electronic device further includes two image sensors located on the front side of the electronic device; the step S3 includes: controlling two image sensors of the electronic device to be turned on, respectively acquiring a 2D image of the face of the object in front of the electronic device; The facial 2D image acquired by the two image sensors is subjected to a synthesis process to obtain a face 3D image of the object in front of the electronic device.

In some embodiments, the step S1 further includes: determining a current working state of the electronic device; determining whether face recognition is required according to the current working state of the electronic device.

In some embodiments, the current working state of the electronic device includes a state of the screen, a bright screen unlocked state, and a bright screen unlocked state; and determining, according to the current working state of the electronic device, whether facial recognition is required:

Determining whether there is a screen wake-up operation when the electronic device is in the interest screen state; determining that face recognition is required when determining that there is a screen wake-up operation;

Determining that facial recognition is required when the electronic device is in a bright, ununlocked state;

When the electronic device is in the unlocked state of the bright screen, it is determined whether there is an application-initiated authentication request currently, and if there is an authentication request, it is determined that facial recognition is required.

In some embodiments, the screen wake-up operation includes: picking up the electronic device, touching the display of the electronic device, approaching the front preset range of the electronic device, and pressing one or more of the electronic device function buttons.

In some embodiments, before performing step S4, the method further includes the step of extracting the facial features in the facial 3D image acquired in step S3.

The embodiment of the present application provides an electronic device, including an image sensor, a fill light device, an ambient light sensor, and a processor. The processor is configured to: determine whether the current ambient light intensity is located when the electronic device needs to perform face recognition. The light intensity range is set; when the current ambient light intensity is outside the preset light intensity range, the light-filling device of the control electronic device performs front fill light; and the image sensor is controlled to collect a 3D image of the face of the object in front of the electronic device; The facial features in the acquired facial 3D image are matched with the facial features in the preset facial 3D image template to determine whether the identity of the front object is legal.

In the embodiment of the present application, when facial recognition is required, if the environment around the electronic device is dark, the electronic device is controlled to fill light, so that the electronic device can collect a clear facial image, improve the facial recognition effect, and improve the user experience.

In some embodiments, the processor is further configured to directly control the image sensor to acquire a facial image of an object in front of the electronic device when the current ambient light intensity is within a preset light intensity range.

In some embodiments, the image sensor is located at a front end of the electronic device.

In some embodiments, the fill light device includes a fill light source located at a top end of the electronic device.

In some embodiments, the fill light source is a flexible light.

In some embodiments, the processor is further configured to: when the current ambient light intensity is outside a preset light intensity range, control the fill light source to be turned on for front fill light.

In some embodiments, the preset light intensity range includes an upper limit value and a lower limit value; when the current ambient light intensity is less than a lower limit value of the preset light intensity range, the processor control station The fill light source is turned on for front fill light.

In some embodiments, the preset light intensity range includes an upper limit value and a lower limit value; when the current ambient light intensity is greater than an upper limit value of the preset light intensity range, the processor control station The fill light source is turned on for front fill light.

In some embodiments, one or more of the brightness, color temperature, and color of the fill light source are further adjusted when the fill light source is turned on.

In some embodiments, when the light-filling device includes a display screen located on the front side of the electronic device, and the display screen is further used to display a screen; the preset light intensity range includes an upper limit value and a lower limit value; The processor is further configured to: when the current ambient light intensity is less than a lower limit value of the preset light intensity range, adjust the display screen to a maximum brightness to perform front fill light.

In some embodiments, when the light-filling device further includes a fill light source located at the top of the front surface of the electronic device, the processor is further configured to: if the sum of the current ambient light intensity and the maximum brightness of the display screen is still less than the pre- The lower limit value of the light intensity range is set to control the fill light source to be turned on.

In some embodiments, the processor controls the fill light device to operate simultaneously with the image sensing module when the current ambient light intensity is outside of a predetermined light intensity range.

In some embodiments, the image sensor is used to collect a 2D image of a face of an object in front of the electronic device, and convert the acquired 2D image of the face to obtain a face 3D of the object in front of the electronic device. image.

In some embodiments, the image sensor is two, and the two image sensors are used to separately collect a facial 2D image of an object in front of the electronic device, and synthesize the facial 2D image acquired by the two image sensors to obtain A 3D image of the face of the object in front of the electronic device.

In some embodiments, the processor is further configured to: determine a current working state of the electronic device, and determine whether face recognition is required according to a current working state of the electronic device.

In some embodiments, the current working state of the electronic device includes a screen state, a bright screen unlocked state, and a bright screen unlocked state; the processor is further configured to:

When the electronic device is in the state of the interest screen, it is determined whether there is a screen wake-up operation; when it is determined that there is a screen wake-up operation, it is determined that face recognition is required;

In some embodiments, the screen wake-up operation includes: picking up the electronic device, touching the display screen of the electronic device, approaching the front surface of the electronic device, and pressing one or more of the electronic device function keys.

The additional aspects and advantages of the embodiments of the present application will be set forth in part in the description which follows.

DRAWINGS

The above and/or additional aspects and advantages of the embodiments of the present application will become apparent and readily understood from

1 is a schematic flow chart of a face recognition method according to a first embodiment of the present application;

2 is a schematic flow chart of a face recognition method according to a second embodiment of the present application;

FIG. 3 is a schematic diagram of functional modules of a mobile terminal according to an embodiment of the present application.

detailed description

The embodiments of the present application are described in detail below, and the examples of the embodiments are illustrated in the drawings, wherein the same or similar reference numerals are used to refer to the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are intended to be illustrative, and are not to be construed as limiting.

In the description of the present application, it is to be understood that the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, features defining "first" or "second" may include one or more of the described features either explicitly or implicitly. In the description of the present application, the meaning of "a plurality" is two or more unless specifically and specifically defined otherwise. "Contact" or "touch" includes direct or indirect contact.

In the description of the present application, it should be noted that the terms "installation", "connected", and "connected" are to be understood broadly, and may be fixed or detachable, for example, unless otherwise specifically defined and defined. Connected, or integrally connected; may be mechanically connected, or may be electrically connected or may communicate with each other; may be directly connected or indirectly connected through an intermediate medium, may be internal communication of two elements or interaction of two elements relationship. For those skilled in the art, the specific meanings of the above terms in the present application can be understood on a case-by-case basis.

The following disclosure provides many different embodiments or examples for implementing the different structures of the present application. In order to simplify the disclosure of the present application, the components and settings of the specific examples are described below. Of course, they are merely examples and are not intended to limit the application. In addition, the present application may repeat reference numerals and/or reference numerals in the various examples, which are for the purpose of simplicity and clarity, and do not indicate the relationship between the various embodiments and/or settings discussed. Moreover, the present application provides examples of various specific processes and materials, but one of ordinary skill in the art will recognize the use of other processes and/or the use of other materials.

Further, the described features, structures may be combined in one or more embodiments in any suitable manner. In the following description, numerous specific details are set forth However, those skilled in the art should appreciate that the technical solution of the present application can be practiced without one or more of the specific details, or other structures, components, and the like. In other instances, well-known structures or operations are not shown or described in detail to avoid obscuring the application.

The facial recognition method refers to collecting facial information of an object through a camera, and matching the collected facial information with the registered facial template. If the matching is successful, the identity of the object is determined to be legal, and if the matching fails, the identity of the object is determined to be illegal.

The objects herein include, for example, human bodies or other living organisms. The face information includes, for example, 2D image information or 3D image information. When facial recognition is performed based on 2D image information, it can be realized by using an existing camera, and it is not necessary to additionally provide an imaging device. However, the camera can capture a clear facial image under normal lighting conditions, and in the case of backlight or low light, a clear facial image cannot be acquired, thereby affecting the facial recognition effect, for example, because the acquired facial image is blurred. Long recognition process, even face recognition is not possible. In this way, the user experience is also poor. Backlighting here refers to the situation when the object is facing away from the light source, and weak light refers to the situation where the surrounding environment is dark.

In this regard, the embodiment of the present application proposes a face recognition method capable of realizing face recognition regardless of any illumination environment, and improves the face recognition effect. 1 is a schematic flow chart of a face recognition method of an electronic device according to a first embodiment of the present application. The facial recognition method of the electronic device includes the following steps:

S1, when the electronic device needs to perform facial recognition, determine whether the current ambient light intensity is within a preset light intensity range; when the current ambient light intensity is outside the preset light intensity range, perform step S2, when the current ambient light When the intensity is within the preset light intensity range, step S3 is performed;

The facial recognition method operates on an electronic device. The electronic device is, for example, but not limited to, a suitable type of electronic product such as a consumer electronic product, a home-based electronic product, a vehicle-mounted electronic product, or a financial terminal product. Among them, consumer electronic products such as mobile phones, tablets, notebook computers, desktop monitors, computer integrated machines. Home-based electronic products such as smart door locks, TVs, refrigerators, wearable devices, etc. Vehicle-mounted electronic products such as car navigation systems, car DVDs, etc. Financial terminal products such as ATM machines, terminals for self-service business, etc.

The front surface of the electronic device is provided with a corresponding sensing device, such as an ambient light sensor, for collecting the light intensity of the surrounding environment of the electronic device when the electronic device needs to perform facial recognition, so as to determine whether the image is in a backlighting environment or a low light environment when the image is captured. Specifically, when the object in front of the electronic device faces away from the light source, the image sensor on the front side of the electronic device faces the light source, so if the light of the light source is strong, the light intensity collected by the image sensor is very strong. When the surrounding environment of the electronic device is dark, the light intensity collected by the image sensor is weak. Of course, the illumination intensity of the surrounding environment of the electronic device can also be collected by other sensing devices, such as an image sensor disposed on the front of the electronic device.

In the embodiment of the present application, a critical range, that is, a preset light intensity range, including an upper limit value and a lower limit value, wherein the upper limit value is a critical value in the case of backlighting, and the lower limit value is in the case of low light The critical value. If the ambient light intensity is within the preset light intensity range, that is, the current ambient light intensity is greater than or equal to the lower limit value and less than or equal to the upper limit value, it indicates that the electronic device can collect a clear facial image in the current environment. To realize facial recognition; if the ambient light intensity is outside the preset light intensity range, that is, the current ambient light intensity is greater than the upper limit value, or less than the lower limit value, it means that the electronic device cannot collect a clear face in the current environment. The image caused the face recognition to fail.

When the current ambient light intensity is collected, and the current ambient light intensity is greater than the upper limit value of the preset light intensity range, or is less than the lower limit value of the preset light intensity range, the control electronic device performs front fill light, thereby The light intensity in the environment around the electronic device is greater than a preset threshold to achieve facial recognition.

The image sensor on the front of the electronic device is, for example, a CCD or a CMOS image sensor. The front camera of the front surface of the existing electronic device can be used to collect the 2D image of the face of the front object, and then the acquired 2D image can be converted to obtain the front. A 3D image of the object.

S4: Match the collected facial 3D image with a preset facial 3D image template to determine whether the identity of the front object is legal.

Before the electronic device is used, the user needs to register the face 3D information, form a face template, and store it for matching of the 3D image at the time of face recognition. Image matching, for example, adopts facial feature extraction, first extracts corresponding facial features and depth information of facial features, such as eyes, nose, eyebrows, lips, jaws, etc. from the 3D image, and then extracts the extracted facial features and faces. The depth information of the feature is compared with the corresponding facial feature in the 3D image template. If the comparison is consistent, the matching is successful, and the identity of the object in front is confirmed; if the comparison is inconsistent, the matching fails, and the identity of the object in front is invalid.

In the embodiment of the present application, when facial recognition is required, if the environment surrounding the electronic device is in the case of backlight or low light, the electronic device is controlled to fill light, so that the electronic device can collect a clear facial image and improve the facial recognition effect. To enhance the user experience.

In some embodiments, the above steps S2 and S3 are performed simultaneously, that is, while the control electronic device performs front fill light, the image sensor on the front side of the electronic device is controlled to collect the facial image. It should be noted that, while speaking here, it does not require strict consistency in time, but also has a sequence of time. It's just that the time difference is short, basically negligible, giving people the feeling that the two are at the same time.

In some embodiments, step S2 further includes: controlling the fill light device on the front side of the electronic device to turn on when the current ambient light intensity is outside the preset light intensity range.

In some embodiments, the light-filling device on the front surface of the electronic device may be a display screen of the electronic device or a fill light source located at the top of the electronic device, and the fill light source is an additional component that is dedicated to fill light. , for example, a soft light. When the light is supplemented by the display of the electronic device, the existing structure can be used to achieve the light filling, which saves the cost. When the fill light is used to fill the light, although the cost is increased, it does not affect the normal operation of other components (such as the display screen), and can also achieve a better fill light effect.

Further, further, one or several parameters of the brightness, color temperature or color of the light-filling device are also adjusted while controlling the light-filling device on the front side of the electronic device. The fill light device comprises, for example, a plurality of LEDs or other light-emitting elements. Moreover, the brightness, color and color temperature of the light-emitting element can be controlled as needed. In this embodiment, by controlling one or several parameters of the brightness, color, and color temperature of the LED, the light signal emitted by the light-filling device is softened, so that when the facial image is collected, the user does not feel too glaring and improves. user experience. It can be understood that the specific values of the foregoing parameters are not limited, and can be flexibly selected according to actual use conditions.

In some embodiments, in step S2: when the current ambient light intensity is outside the preset light intensity range, the display screen of the electronic device is adjusted to the maximum brightness to perform front fill light.

If the electronic device fills the front side of the electronic device through the display screen, when the current ambient light intensity is less than or equal to the preset threshold, the display screen is gradually increased to the maximum brightness to perform front fill light. The brightness of the display can be adjusted one by one to avoid discomfort caused by sudden increase in brightness.

Further, in step S2, if the sum of the current ambient light intensity and the maximum brightness of the display screen is still less than the lower limit value of the preset light intensity range, the fill light source of the front end of the control electronic device is turned on, so that the The fill light source is complemented with the display screen.

Since the brightness of the display itself is limited, if the brightness of the display cannot meet the fill light requirement, the fill light source at the top of the front side of the control electronic device is turned on, so that the fill light source and the display screen together fill the light. Therefore, in this embodiment, the display screen is first adjusted to the maximum brightness, and the current ambient light intensity is collected. If the current ambient light intensity is still less than the lower limit value of the preset light intensity range, it means that the fill light of the display cannot be compensated. Light requires that the fill light source at the top of the front of the control electronics is turned on. If the current ambient light intensity is greater than or equal to the lower limit of the preset light intensity range, it means that the brightness of the display can meet the fill light requirement, so the fill light can only be performed by the brightness of the display screen.

Of course, in order to make the control process simple, it is also possible to adjust the display screen to the maximum brightness and control the fill light source to turn on when determining that the current ambient light intensity is less than the lower limit value of the preset light intensity range, so that the display screen Front fill light with the fill light source. It's so simple and convenient.

Further, the electronic device further includes an image sensor located on the front side of the electronic device; the step S3 includes: controlling the image sensor of the electronic device to be turned on to acquire a 2D image of the face of the object in front of the electronic device; and 2D images of the collected face The conversion is performed to obtain a 3D image of the face of the object in front of the electronic device.

The front of the electronic device includes a front panel, and the top of the front panel is provided with an opening. The image sensor includes, for example, a camera located within the aperture and an image processing circuit electrically coupled to the camera. By controlling the camera on the front of the electronic device to turn on, the optical signal reflected by the object in front of the electronic device is sensed, the sensed optical signal is converted into a corresponding electrical signal, and the converted electrical signal is transmitted to the image processing circuit. When receiving the electrical signal, the image processing circuit processes the electrical signal to obtain a 2D image of the face of the object in front of the electronic device, and converts the 2D image to obtain a 3D image of the face of the object in front of the electronic device. There are many methods for converting a 2D image into a 3D image, for example, acquiring the two-dimensional coordinates of each pixel of the 2D image, and acquiring the gray value corresponding to each pixel, and using the obtained gray value as the three-dimensional coordinates. The Z coordinate, the obtained two-dimensional coordinates of each pixel point is taken as the XY coordinate in the three-dimensional coordinates, the three-dimensional space coordinates corresponding to each pixel point are obtained, and then the respective pixel points of the three-dimensional space coordinates are colored to obtain the face of the object in front of the electronic device. 3D image. Of course, other methods of 2D to 3D images can be used.

Further, the electronic device further includes two image sensors located on the front side of the electronic device; the step S3 includes: controlling two image sensors of the electronic device to be turned on, respectively acquiring a 2D image of the face of the object in front of the electronic device; and the two image sensors The acquired facial 2D image is subjected to a synthesis process to obtain a face 3D image of the object in front of the electronic device.

The front surface of the electronic device includes a front panel, and the top of the front panel is provided with two openings for respectively receiving the camera of the image sensor. The two cameras are provided with corresponding image processing circuits or share the same image processing circuit. Two sets of image information are obtained by controlling the two cameras on the front of the electronic device to be turned on. The image processing circuit receives the two sets of image information, and synthesizes the two sets of image information using a human left eye and right eye imaging principle to obtain a face 3D image of the object in front of the electronic device.

In some embodiments, the image sensors are all RGB image sensors, and may of course be infrared sensors. If the image sensor is an RGB image sensor, the electronic device can not only realize photographing but also realize identification. In addition, the existing hardware structure can be used to realize the image acquisition of the object in front of the electronic device, and the corresponding processing algorithm can be used to realize the generation of the 3D image, thereby performing identity recognition. Therefore, the electronic device of the embodiment of the present application realizes various functions, and is also easy to implement without increasing cost.

Further, referring to FIG. 2, FIG. 2 is a schematic flowchart of a face recognition method according to a second embodiment of the present application. In the facial recognition method of the electronic device, before the step S1, the method further includes:

Step S5, determining a current working state of the electronic device;

The current working state is divided according to the display state of the display screen, for example, including the state of the information screen, the unlit state of the bright screen, and the unlocked state of the bright screen. The state of the information screen means that the display screen is not lit and is in an extinguished state. The unlit state of the bright screen means that the display is lit, but the unlocked interface is displayed on the display. In this state, the electronic device is not unlocked and cannot enter the running page of the electronic device. The unlocked state of the bright screen means that the display is lit and the corresponding interface is displayed according to the specific operation. In this state, the electronic device has been unlocked, and the user can use the electronic device normally.

Step S6: Determine whether face recognition is required according to the current working state of the electronic device.

Specifically, when the electronic device is in the state of the interest screen, the user may use the electronic device at any time, so in order to quickly respond to the user's operation, some sensors on the electronic device are still in operation to monitor the electronic device. Use case to determine if there is a screen wake-up operation. For example, touch sensors, gravity sensors, distance sensors, and the like. The screen wake-up operation includes picking up one or more of an electronic device, a display screen of the touch electronic device, a preset range of the object near the front of the device, and a function button for pressing the electronic device. It can be understood that the screen wake-up operation of the electronic device can be flexibly set according to the needs of the user.

Taking the touch sensor as an example, in the embodiment of the present application, when the electronic device is in the interest screen state, the touch sensor of the control electronic device performs touch detection, and touch action on the display screen (eg, normal touch operation, pressing operation, sliding operation) Test and respond. When the electronic device is in the interest screen state, the touch sensor will perform touch detection at the first scanning frequency (lower power consumption state). When it is detected that the object touches the display screen, it is determined that face recognition is required. When the electronic device is in the unscreened state of the bright screen or the unlocked state of the bright screen, the touch sensor performs touch detection at the second scanning frequency (higher power consumption state or normal power consumption state), thereby reducing the state of the electronic device Power consumption. Wherein the first scanning frequency is less than the second scanning frequency. Of course, the touch sensor can also be touch-detected in a normal power consumption state.

When the electronic device is in the unlit state of the bright screen, the user may use the electronic device at any time in this case, so it is determined that face recognition is required, and the process proceeds to step S1.

When the electronic device is in the bright-screen unlocked state, it is further determined whether there is currently an authentication request initiated by the application to the operating system, for example, the payment application initiates an authentication request for mobile payment. If there is an authentication request, it is determined that face recognition is required, and the process proceeds to step S1, otherwise the electronic device performs a normal operation.

In some embodiments, after step S4 above, the method further comprises: after the face recognition is successful, controlling the electronic device to perform a corresponding operation. For example, controlling electronic device unlocking, performing payment transaction operations, and the like.

Correspondingly, an embodiment of the present application proposes an electronic device for performing the method of the above embodiment. The electronic device is exemplified by a mobile terminal. Referring to FIG. 3, FIG. 3 is a schematic diagram of functional modules of a mobile terminal according to an embodiment of the present application. The mobile terminal 100 includes an image sensor 101, a fill light device 102, an ambient light sensor 103, a processor 104, and a memory 105. The ambient light sensor 103 is disposed on the front side of the electronic device for collecting the light intensity of the environment surrounding the electronic device. The image sensor 101 is disposed on the front side of the electronic device for collecting a facial image of an object in front of the electronic device. The light-filling device 102 is configured to fill light in the case where the surrounding environment of the electronic device is backlight or low light to facilitate image capturing of the image sensor 101. The processor 104 is configured to determine whether the current ambient light intensity is within a preset light intensity range when the electronic device needs to perform facial recognition; and control the electronic device to perform the front when the current ambient light intensity is outside the preset light intensity range. Filling the light; controlling the image sensor on the front side of the electronic device, acquiring a 3D image of the face of the object in front of the electronic device; matching the acquired 3D image of the face with the preset face 3D image template to determine whether the identity of the object in front is legal.

It should be noted that, in some embodiments, one or more component structures may be combined or omitted, for example, the processor 104 and the memory 105 are integrated into a control chip or the like. Additionally, the mobile terminal 100 can include other components (eg, communication circuitry, power supplies, buses, microphones, cameras, etc.) that are not combined or included in the components shown in FIG. Moreover, for the sake of brevity, only some of the components of the mobile terminal are shown in FIG.

Specifically, the image sensor 101 is disposed on the front side of the electronic device, such as a CCD or CMOS image sensor. The image sensor 101 does not require additional settings. For example, when the front camera is disposed on the front side of the existing electronic device, the front camera can be used to collect the face image. Of course, it is also possible to set an image sensor corresponding to the shooting parameters according to the shooting requirements, and to collect the facial image for facial recognition.

The processor 104 is for including any processing circuitry that is provided to control the operation and performance of the mobile terminal 100. The processor 104 is used to run an operating system, an app, a media playback application, or any other application software, and is used to handle interactions with a user, and the like. The processor 104 is, for example, a control IC in which the processing circuits are integrated, or a processor cluster including distributed processing circuits, for example, a central processing unit CPU for centrally controlling various components of the electronic device, and for an electronic device. Image processing GPU in graphics processing. Of course, other dedicated processors, such as a coprocessor for monitoring the detection results of the sensors of the electronic device, a baseband processor for electronic device communication, a face recognition processor for electronic device for face recognition, etc., may also be provided. Wait.

Memory 105, for example, includes one or more computer storage media including hard disks, floppy disks, Flash, ROM, RAM, and any other suitable type of storage component or any combination thereof. Memory 105 is used to store any program code files available to processor 104, such as operating systems, application software, and functional modules. The memory 105 is also used to store processing data processed by the processor 104 and processing results such as application data, user operation information, user setting information, multimedia data, and the like. It will be appreciated that the memory 105 can be provided separately or integrated with the processor 104.

Further, the computer storage medium in the memory 105 stores a plurality of program code files for the processor 104 to call to execute the unlock control method described in the foregoing embodiment, thereby implementing unlock control in the state of the electronic device.

The front surface of the electronic device includes a front panel, and the top of the front panel is provided with two openings for respectively receiving the camera of the image sensor. The two cameras are provided with corresponding image processing circuits or share the same image processing circuit. Two sets of image information are obtained by controlling the two cameras on the front of the electronic device to be turned on. The image processing circuit receives the two sets of image information, and synthesizes the two sets of image information by using the left eye and right eye imaging principles of the person to obtain a face 3D image of the object in front of the electronic device.

In some embodiments, the image sensors are all RGB image sensors, and may of course be infrared sensors. If the image sensor is an RGB image sensor, the electronic device can not only realize photographing but also realize identification. In addition, the existing hardware structure can be used to realize the image acquisition of the object in front of the electronic device, and the corresponding processing algorithm can be used to realize the generation of the 3D image, thereby performing identity recognition. Therefore, the electronic device of the embodiment of the present application achieves various functions, and is also easy to implement without increasing cost.

In some embodiments, the light-filling device 102 is disposed on the front surface of the electronic device, for example, a display screen 106 including an electronic device, and a fill light source 107 located at the top of the front surface of the electronic device. The fill light source 107 is additionally provided and dedicated to supplement Light components, such as soft lights. The display screen 106 is also used to display a picture. When performing face recognition, one of the display screen 106 and the fill light source 107 is selected for supplemental light according to actual needs, or a combination of the two is used to fill light. For example, when the fill light source 107 is used to fill light, the processor 104 is configured to: when the current ambient light intensity is outside the preset light intensity range, control the fill light source to be turned on to perform front fill light.

When the light is supplemented by the display 106 of the electronic device, the light can be achieved by using the existing structure, and the cost is saved. When the fill light is applied by the fill light source 107, although the cost is increased, it does not affect the normal operation of other components (such as a display screen), and a better fill light effect can be achieved.

The fill light device 102 includes, for example, a plurality of LEDs or other light emitting elements. Further, the light-filling parameters such as brightness, color, and color temperature of the light-emitting element can be controlled as needed. In this embodiment, by controlling one or several fill light parameters of the brightness, color and color temperature of the LED, the light signal emitted by the light-filling device is softened, so that the user does not feel too glaring when performing face image acquisition. , improved user experience. For example, the processor 104 is configured to further adjust one or more parameters of the brightness, color temperature and color of the fill light source when the fill light source is turned on.

In some embodiments, the processor 104 is further configured to: when the current ambient light intensity is less than a lower limit value of the preset light intensity range, adjust the display screen to the maximum brightness to perform front fill light.

If the electronic device fills the front side of the electronic device through the display screen, when the current ambient light intensity is less than the lower limit value of the preset light intensity range, the display screen is gradually increased to the maximum brightness to perform front fill light. The brightness of the display can be adjusted one by one to avoid discomfort caused by sudden increase in brightness.

Further, the processor 104 is further configured to: if the sum of the current ambient light intensity and the maximum brightness of the display 106 is still less than a lower limit of the preset light intensity range, control the fill light source of the front end of the electronic device to be turned on, The fill light source 107 is complemented with the display screen 106 to complement the front side.

Since the illumination capability of the display screen itself is limited, if the brightness of the display screen 106 cannot meet the fill light requirement, the fill light source 107 of the front end of the control electronic device is turned on, so that the fill light source 107 and the display screen 106 together perform front fill light. Therefore, in this embodiment, the display screen 106 is first adjusted to the maximum brightness, and the current ambient light intensity is collected. If the current ambient light intensity is still less than the lower limit value of the preset light intensity range, it indicates that the fill light of the display screen 106 cannot be The fill light requirement is reached, at which time the fill light source 107 on the front side of the control electronics is turned on. If the current ambient light intensity is greater than or equal to the lower limit value of the preset light intensity range, it means that the brightness of the display screen 106 can meet the fill light requirement, and therefore only the brightness of the display screen 106 can be used to fill the light.

Further, the processor 104 is further configured to: determine a current working state of the electronic device, and determine whether face recognition is required according to a current working state of the electronic device.

The current working state is divided according to the display state of the display screen, for example, including the state of the information screen, the unlit state of the bright screen, and the unlocked state of the bright screen. The state of the information screen means that the display screen is not lit and is in an extinguished state. The unlit state of the bright screen means that the display is lit, but the unlocked interface is displayed on the display. In this state, the electronic device is not unlocked and cannot enter the main page of the electronic device. The unlocked state of the bright screen means that the display is lit and the corresponding interface is displayed according to the specific operation. In this state, the electronic device has been unlocked, and the user can use the electronic device normally.

Specifically, when the electronic device is in the state of the interest screen, the user may use the electronic device at any time, so in order to quickly respond to the user's operation, some sensors on the electronic device are still in operation to monitor the electronic device. Use case to determine if there is a screen wake-up operation. For example, touch sensors, gravity sensors, distance sensors, and the like. The screen wake-up operation includes one or more of picking up the electronic device, touching the display of the electronic device, the object is close to the front preset range of the device, and pressing the function button of the electronic device. It can be understood that the screen wake-up operation of the electronic device can be flexibly set according to the needs of the user.

Taking the touch sensor 108 provided on the electronic device as an example, in the embodiment of the present application, when the electronic device is in the interest screen state, the touch sensor 108 of the control electronic device performs touch detection, and touch action on the display screen 106 (for example, a normal touch) Operation, pressing operation, sliding operation) to detect and respond. When the electronic device is in the interest screen state, the touch sensor 108 will perform touch detection at the first scanning frequency (lower power consumption state). When it is detected that the object touches the display screen, it is determined that face recognition is required. When the electronic device is in the bright un unlocked state or the bright-screen unlocked state, the touch sensor 108 performs touch detection at the second scanning frequency (higher power consumption state or normal power consumption state), thereby reducing the electronic device state of the screen. Power consumption. Wherein the first scanning frequency is less than the second scanning frequency. Of course, the touch sensor 108 can also be touch-detected in a normal power consumption state at all times.

When the electronic device is in a bright, ununlocked state, the user may use the electronic device at any time in this case, thus determining that face recognition is required.

When the electronic device is in the bright-screen unlocked state, it is further determined whether there is currently an authentication request initiated by the application to the operating system, for example, the payment application initiates an authentication request for mobile payment. If there is an authentication request, it is determined that face recognition is required, otherwise the electronic device performs normal operations.

In the description of the present specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiment", "example", "specific example", or "some examples", etc. The specific features, structures, materials, or characteristics described in the embodiments or examples are included in at least one embodiment or example of the application. In the present specification, the schematic representation of the above terms does not necessarily mean the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples.

While the embodiments of the present application have been shown and described above, it is understood that the foregoing embodiments are illustrative and are not to be construed as limiting the scope of the present application. The embodiments are subject to changes, modifications, substitutions and variations.

Claims

A facial recognition method for an electronic device includes the following steps:

S1, when the electronic device needs to perform facial recognition, determine whether the current ambient light intensity is within a preset light intensity range;

S2, when the current ambient light intensity is outside the preset light intensity range, controlling the electronic device to perform front fill light;

S3, when the current ambient light intensity is within a preset light intensity range, controlling an image sensor on the front side of the electronic device to collect a 3D image of the face of the object in front of the electronic device;

S4. Match the facial features in the acquired facial 3D image and the depth information of the facial features with the corresponding facial features in the preset facial 3D image template to determine whether the identity of the front object is legal.
The method for recognizing a face of an electronic device according to claim 1, wherein the step S1 further comprises: performing step S3 when the current ambient light intensity is within a preset light intensity range.
The face recognition method according to claim 1, wherein the fill light in the step S2 is performed simultaneously with the collection of the face 3D image in the step S3.
The method for recognizing a face of an electronic device according to claim 1, wherein in the step S2, when the current ambient light intensity is outside the preset light intensity range, the light-filling device on the front side of the electronic device is controlled to be turned on. To make a positive fill light.
The facial recognition method of an electronic device according to claim 4, wherein one or more of brightness, color temperature and color of the light-filling device are further adjusted when the light-filling device on the front side of the control electronic device is turned on.
The face recognition method for an electronic device according to claim 4, wherein the light filling device on the front side of the electronic device comprises a fill light source disposed at a top end of the front surface of the electronic device.
The facial recognition method of an electronic device according to claim 6, wherein the predetermined light intensity range includes an upper limit value and a lower limit value; and in the step S2: when the current ambient light intensity is less than the When the lower limit value of the preset light intensity range is used, the fill light source is controlled to be turned on to perform front fill light.
The facial recognition method of an electronic device according to claim 7, wherein the lower limit value is a light intensity threshold value of the object in a low light environment.
The facial recognition method of an electronic device according to claim 6, wherein the predetermined light intensity range includes an upper limit value and a lower limit value; and in the step S2: when the current ambient light intensity is greater than the When the upper limit value of the preset light intensity range is set, the fill light source is controlled to be turned on to perform front fill light.
A face recognition method for an electronic device according to claim 9, wherein said upper limit value is a light intensity threshold value of the object in a backlight environment.
The facial recognition method of an electronic device according to claim 1, wherein the predetermined light intensity range includes an upper limit value and a lower limit value; and in the step S2: when the current ambient light intensity is less than the When the lower limit of the preset light intensity range is set, adjust the display of the electronic device to the maximum brightness for front fill.
The facial recognition method of an electronic device according to claim 11, wherein in the step S2, if the sum of the current ambient light intensity and the maximum brightness of the display screen is still less than a lower limit value of the preset light intensity range And controlling the fill light source on the front side of the electronic device to be turned on, so that the fill light source and the display screen perform front fill light together.
The facial recognition method of an electronic device according to claim 1, wherein the electronic device further comprises an image sensor located on the front side of the electronic device; and the step S3 comprises: controlling the image sensor of the electronic device to be turned on to collect the electronic device. A 2D image of the face of the object in front of the device; converting the acquired 2D image of the face to obtain a 3D image of the face of the object in front of the electronic device.
The method for recognizing a face of an electronic device according to claim 1, wherein the electronic device further comprises two image sensors located on the front side of the electronic device; and the step S3 comprises: controlling two image sensors of the electronic device to be turned on, respectively acquiring a 2D image of the face of the object in front of the electronic device; synthesizing the 2D image of the face acquired by the two image sensors to obtain a 3D image of the face of the object in front of the electronic device.
The method for recognizing a face of an electronic device according to any one of claims 1 to 14, wherein the step S1 further comprises: determining a current working state of the electronic device; determining whether the electronic device is in accordance with a current working state of the electronic device Face recognition is required.
The method for recognizing a face of an electronic device according to claim 11, wherein the current working state of the electronic device comprises a state of a screen, a state of a bright screen, an unlocked state, and a state in which a bright screen has been unlocked; The working status determines whether facial recognition is required, including:

Determining whether there is a screen wake-up operation when the electronic device is in the interest screen state; determining that face recognition is required when determining that there is a screen wake-up operation;

Determining that facial recognition is required when the electronic device is in a bright, ununlocked state;

When the electronic device is in the unlocked state of the bright screen, it is determined whether there is an application-initiated authentication request currently, and if there is an authentication request, it is determined that facial recognition is required.
The facial recognition method according to claim 12, wherein the screen wake-up operation comprises: picking up an electronic device, touching a display screen of the electronic device, approaching a front preset range of the electronic device, and pressing a function button of the electronic device One or several.
The face recognition method according to claim 1, wherein before performing step S4, the method further comprises the step of extracting the facial features and the depth information of the facial features in the face 3D image acquired in step S3.
An electronic device includes an image sensor, a fill light device, an ambient light sensor, and a processor; the processor is configured to: determine whether the current ambient light intensity is within a preset light intensity range when the electronic device needs to perform face recognition When the current ambient light intensity is outside the preset light intensity range, the light-filling device of the control electronic device performs front fill light; the image sensor is controlled to collect a face 3D image of the object in front of the electronic device; and the collected face is collected; The facial features in the 3D image and the depth information of the facial features are matched with corresponding facial features in the preset facial 3D image template to determine whether the identity of the front object is legal.
The electronic device according to claim 19, wherein the processor is further configured to: directly control the image sensor to collect a face of an object in front of the electronic device when the current ambient light intensity is within a preset light intensity range; image.
The electronic device of claim 19 wherein said image sensor is located at a front end of said electronic device.
The electronic device of claim 19 wherein said fill light means comprises a fill light source located at a top end of the electronic device.
The electronic device of claim 22 wherein said fill light source is a flexible light.
The electronic device according to claim 22, wherein the processor is further configured to: when the current ambient light intensity is outside a preset light intensity range, control the fill light source to be turned on for positive compensation. Light.
The electronic device according to claim 22, wherein the predetermined light intensity range includes an upper limit value and a lower limit value; when the current ambient light intensity is less than a lower limit value of the preset light intensity range The processor controls the fill light source to be turned on to perform front fill light.
The electronic device according to claim 25, wherein said lower limit value is a light intensity threshold value of the object in a low light environment.
The electronic device according to claim 22, wherein the predetermined light intensity range includes an upper limit value and a lower limit value; when the current ambient light intensity is greater than an upper limit value of the preset light intensity range The processor controls the fill light source to be turned on to perform front fill light.
The electronic device according to claim 27, wherein said upper limit value is a light intensity threshold value of the object in a backlight environment.
The electronic device according to claim 24, wherein one or more of brightness, color temperature and color of the fill light source are further adjusted when the fill light source is turned on.
The electronic device according to claim 19, wherein said light filling means comprises a display screen located on the front side of the electronic device, and said display screen is further used for displaying a picture; said preset light intensity range includes an upper limit The value and the lower limit value; the processor is further configured to: when the current ambient light intensity is less than a lower limit value of the preset light intensity range, adjust the display screen to a maximum brightness to perform front fill light.
The electronic device according to claim 30, wherein said light filling device further comprises a fill light source located at a top end of the electronic device, said processor further configured to: if the current ambient light intensity and the maximum display screen If the sum of the brightness is still less than the lower limit of the preset light intensity range, the fill light source is controlled to be turned on.
The electronic device of claim 19, wherein the processor controls the fill light device to operate simultaneously with the image sensing module when the current ambient light intensity is outside of a predetermined light intensity range.
The electronic device according to claim 19, wherein the image sensor is used for acquiring a 2D image of a face of an object in front of the electronic device, and converting the acquired 2D image of the face to obtain A 3D image of the face of the object in front of the electronic device.
The electronic device according to claim 19, wherein the image sensors are two, and the two image sensors are configured to separately collect facial 2D images of objects in front of the electronic device and collect the faces of the two image sensors. The 2D image is subjected to a synthesis process to obtain a face 3D image of the object in front of the electronic device.
The electronic device of claim 19 wherein said processor is operative to control said fill light device and said image sensor to operate simultaneously when a current ambient light intensity is outside of a predetermined range of light intensity.
The electronic device according to any one of claims 19 to 35, wherein the processor is further configured to: determine a current working state of the electronic device, and determine whether face recognition is required according to a current working state of the electronic device. .
The electronic device according to claim 36, wherein the current working state of the electronic device comprises a state of the screen, a bright screen unlocked state, and a bright screen unlocked state; and the processor is further configured to:

When the electronic device is in the state of the interest screen, it is determined whether there is a screen wake-up operation; when it is determined that there is a screen wake-up operation, it is determined that face recognition is required;

Determining that facial recognition is required when the electronic device is in a bright, ununlocked state;

When the electronic device is in the unlocked state of the bright screen, it is determined whether there is an application-initiated authentication request currently, and if there is an authentication request, it is determined that facial recognition is required.
The electronic device according to claim 37, wherein the screen wake-up operation comprises: picking up an electronic device, touching a display screen of the electronic device, approaching an object to a front preset range of the electronic device, and pressing one of the electronic device function buttons Kind or several.