CN111222446A - Face recognition method, face recognition device and mobile terminal - Google Patents
Face recognition method, face recognition device and mobile terminal Download PDFInfo
- Publication number
- CN111222446A CN111222446A CN201911421186.7A CN201911421186A CN111222446A CN 111222446 A CN111222446 A CN 111222446A CN 201911421186 A CN201911421186 A CN 201911421186A CN 111222446 A CN111222446 A CN 111222446A
- Authority
- CN
- China
- Prior art keywords
- face
- face image
- image
- pixel
- frames
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4007—Interpolation-based scaling, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
Abstract
The application is applicable to the technical field of face recognition, and provides a face recognition method, a face recognition device, a mobile terminal and a computer readable storage medium, comprising the following steps: acquiring continuous N frames of initial images aiming at a target face; acquiring a face image of the target face in each frame of initial image in the N frames of initial images; performing sub-pixel interpolation on the N frames of face images to obtain a first face image; inputting the first face image into a preset neural network to obtain a second face image; and recognizing the face in the second face image. The face recognition precision can be improved through the method and the device.
Description
Technical Field
The present application belongs to the field of face recognition technology, and in particular, relates to a face recognition method, a face recognition apparatus, a mobile terminal, and a computer-readable storage medium.
Background
The face recognition is a biological recognition technology for performing identity recognition based on face feature information of a person, and an image containing a face is acquired by using a camera device and detected in the image, so that the face of the detected face is recognized, which can also be called as face recognition and face recognition. However, when the face is small in the image or the imaging quality is poor, the face recognition accuracy may be reduced.
Disclosure of Invention
The application provides a face recognition method, a face recognition device, a mobile terminal and a computer readable storage medium, so as to improve face recognition accuracy.
In a first aspect, an embodiment of the present application provides a face recognition method, where the face recognition method includes:
acquiring continuous N frames of initial images aiming at a target face, wherein N is an integer greater than 1;
acquiring a face image of the target face in each frame of initial image in the N frames of initial images, wherein the face image of the target face in each frame of initial image refers to an image of an area where the target face is located in each frame of initial image, the N frames of initial images correspond to the N frames of face images, and the N frames of face images have the same size;
performing sub-pixel interpolation on the N frames of face images to obtain a first face image, wherein the size of the first face image is larger than that of each frame of face image, and the resolution of the first face image is larger than that of each frame of face image;
inputting the first face image into a preset neural network to obtain a second face image, wherein the size of the second face image is larger than that of the first face image, and the resolution of the second face image is larger than that of the first face image;
and recognizing the face in the second face image.
In a second aspect, an embodiment of the present application provides a face recognition apparatus, where the face recognition apparatus includes:
the system comprises an initial image acquisition module, a face recognition module and a face recognition module, wherein the initial image acquisition module is used for acquiring continuous N frames of initial images aiming at a target face, and N is an integer greater than 1;
a face image obtaining module, configured to obtain a face image of the target face in each frame of initial image in the N frames of initial images, where the face image of the target face in each frame of initial image refers to an image of an area where the target face is located in each frame of initial image, the N frames of initial images correspond to the N frames of face images, and the sizes of the N frames of face images are the same;
the sub-pixel interpolation module is used for performing sub-pixel interpolation on the N frames of face images to obtain a first face image, wherein the size of the first face image is larger than that of each frame of face image, and the resolution of the first face image is larger than that of each frame of face image;
the face image input module is used for inputting the first face image into a preset neural network to obtain a second face image, wherein the size of the second face image is larger than that of the first face image, and the resolution of the second face image is larger than that of the first face image;
and the face recognition module is used for recognizing the face in the second face image.
In a third aspect, an embodiment of the present application provides a mobile terminal, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the face recognition method according to the first aspect when executing the computer program.
In a fourth aspect, the present application provides a computer-readable storage medium, where a computer program is stored, and the computer program, when executed by a processor, implements the steps of the face recognition method according to the first aspect.
In a fifth aspect, an embodiment of the present application provides a computer program product, which, when running on a mobile terminal, causes the mobile terminal to execute the steps of the face recognition method according to the first aspect.
It is thus clear from top to bottom, this application acquires the continuous N frame initial image to the target face earlier, acquire the face image of target face in every frame initial image again, and carry out sub-pixel interpolation to N frame face image, obtain the first face image that size and resolution ratio all enlarge, input first face image to predetermineeing neural network again, carry out image enhancement to first face image, the size and the resolution ratio of further enlarging first face image, this application combines together through many frame super-partials and deep learning promptly, can effectively promote the quantity of true detail in the face image, and then improve the face identification precision.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
Fig. 1 is a schematic flow chart illustrating an implementation of a face recognition method according to an embodiment of the present application;
FIG. 2 is an exemplary graph of sub-pixel interpolation;
fig. 3 is a schematic flow chart illustrating an implementation of a face recognition method according to a second embodiment of the present application;
FIG. 4 is a diagram illustrating an example of a process of processing a face image;
fig. 5 is a schematic structural diagram of a face recognition apparatus according to a third embodiment of the present application;
fig. 6 is a schematic structural diagram of a mobile terminal according to a fourth embodiment of the present application;
fig. 7 is a schematic structural diagram of a mobile terminal according to a fifth embodiment of the present application.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to" determining "or" in response to detecting ". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".
Furthermore, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing or implying relative importance.
Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless expressly specified otherwise.
In order to explain the technical solution described in the present application, the following description will be given by way of specific examples.
Referring to fig. 1, which is a schematic view of an implementation flow of a face recognition method provided in an embodiment of the present application, where the face recognition method is applied to a mobile terminal, as shown in the figure, the face recognition method may include the following steps:
step S101, acquiring continuous N frames of initial images aiming at a target face.
Wherein N is an integer greater than 1.
Optionally, the value range of N is [3,16], and when the value range of N is [3,16], the operation amount increase degree and the face recognition accuracy improvement degree of the mobile terminal in the face recognition process are relatively economical.
In the embodiment of the present application, the target face may refer to a face to be recognized. When a target face is photographed by a camera device of the mobile terminal, N continuous frames of images (namely N initial images) captured by the camera device can be acquired.
Step S102, acquiring the face image of the target face in each frame of initial image in the N frames of initial images.
The face image of the target face in each initial image refers to an image of an area where the target face is located in each initial image, the N initial images correspond to N face images, the N face images have the same size, for example, the size of each of the N face images is W × H, W is the width of the face image, and H is the height of the face image.
In the embodiment of the present application, acquiring the face image of the target face in each frame of the initial image may refer to cutting out the face image of the target face from each frame of the initial image. Specifically, a central point of a target face in each frame of initial image is obtained, the central point is used as a central point of a face image, and the face image with a preset size is cut out, wherein the face image comprises the whole target face.
And step S103, performing sub-pixel interpolation on the N frames of face images to obtain a first face image.
The size of the first facial image is larger than that of each frame of facial image, and the resolution of the first facial image is larger than that of each frame of facial image. Optionally, the user may set the size and resolution of the first face image according to actual needs.
In the embodiment of the application, the sub-pixel interpolation is carried out on the N frames of face images for realizing multi-frame super-resolution, so that the real detail quantity of the face images can be increased, the detail definition of the face images is improved, the noise is reduced, and the first face image with higher resolution is obtained.
Optionally, the performing sub-pixel interpolation on the N frames of face images to obtain a first face image includes:
selecting one frame of face image from the N frames of face images as a reference face image;
carrying out bilinear interpolation on the reference face image to obtain a third face image, wherein the size of the third face image is the same as that of the first face image, and the resolution of the third face image is greater than that of the reference face image;
a1, performing bilinear interpolation on an ith frame of face image in the remaining N-1 frames of face images to obtain a fourth face image, wherein the remaining N-1 frames of face images refer to face images except the reference face image in the N frames of face images, the size of the fourth face image is the same as that of the first face image, the resolution of the fourth face image is greater than that of the ith frame of face image, and i is an integer greater than zero and less than or equal to N-1;
step a2, performing image matching on the third face image and the fourth face image to obtain a first pixel in the third face image and a sub-pixel matched with the first pixel in the fourth face image;
a3, acquiring pixel values of four pixels adjacent to the sub-pixels in the fourth face image;
a4, acquiring pixel values of the sub-pixels in the fourth face image according to pixel values of four pixels adjacent to the sub-pixels in the fourth face image;
a5, adding the pixel value of the first pixel in the third face image and the pixel value of the sub-pixel in the fourth face image for averaging, and taking the average value as the pixel value of the first pixel in the third face image;
repeatedly executing the steps a1, a2, a3, a4 and a5 until the remaining N-1 frames of face images are traversed, and determining the processed third face image as the first face image, wherein the processed third face image is the third face image of which the pixel values of the first pixels are subjected to addition and averaging processing.
In the embodiment of the application, the definition of each frame of face image in N frames of face images can be obtained first, the face image with the highest definition in the N frames of face images is used as a reference face image, bilinear interpolation is performed on the reference face image, so that the detail characteristics of the reference face image can be enhanced, the size of the reference face image is enlarged, the resolution of the reference face image is improved, and the reference face image with the enlarged size and the improved resolution is the third face image.
For any frame of face image in the rest N-1 frames of face images, bilinear interpolation can be carried out on the frame of face image, the real details of the frame of face image are enhanced, the size of the frame of face image is enlarged, the resolution of the frame of face image is improved, and the frame of face image with enlarged size and improved resolution is the fourth face image.
The third face image and the fourth face image are subjected to image matching, a matching point which represents the same content in the third face image and the fourth face image can be obtained, a first pixel in the third face image and a sub-pixel which is matched with the first pixel in the fourth face image can be obtained according to the matching point which represents the same content in the third face image and the fourth face image, and four pixels which are adjacent to the sub-pixel in the fourth face image and pixel values of the four pixels can be obtained according to the position of the sub-pixel in the fourth face image. It should be noted that the first pixel in the third face image is obtained by performing image matching on the third face image and the fourth face image, and the fourth face images corresponding to different frames of face images in the remaining N-1 frames of face images may be different, and then the first pixel in the third face image may also be different for different frames of face images, for example, after performing image matching on two different frames of face images (for example, the first frame of face image and the second frame of face image) in the remaining N-1 frames of face images, the coordinate of the first pixel in the third face image is obtained as (1, 1), after performing image matching on the third face image and the fourth face image corresponding to the first frame of face image, the coordinate of the first pixel in the third face image is obtained as (1, 2).
Optionally, the obtaining, according to pixel values of four pixels adjacent to the sub-pixel in the fourth face image, a pixel value of the sub-pixel in the fourth face image includes:
acquiring offsets of four pixels adjacent to the sub-pixels in the fourth face image and the sub-pixels;
and acquiring the pixel values of the sub-pixels in the fourth face image according to the offsets of the four pixels adjacent to the sub-pixels in the fourth face image and the sub-pixels and the pixel values of the four pixels adjacent to the sub-pixels in the fourth face image.
FIG. 2 is an exemplary graph of sub-pixel interpolation, in which OP1,1、OP1,2、OP2,1、OP2,2The pixels in the third face image; OP (optical fiber)1,1Is the first pixel in the third face image and is also the sub-pixel in the fourth face image, IP1,1、IP1,2、IP2,1、IP2,2Four pixels adjacent to the sub-pixel in the fourth face image. The pixel values of the sub-pixels in the fourth face image may be calculated using the following formula:
IP in the above formula1,1、IP1,2、IP2,1、IP2,2Are respectively pixel IP1,1Pixel value, pixel IP of1,2Pixel value, pixel IP of2,1Pixel value, pixel IP of2,2The weight used in the formula is the offset.
Optionally, the image matching the third face image and the fourth face image, and obtaining a first pixel in the third face image and a sub-pixel matched with the first pixel in the fourth face image includes:
and performing image matching on the third face image and the fourth face image to obtain a point in the fourth face image, which is matched with a pixel in the third face image, if the coordinate of the point in the fourth face image is a decimal, determining that the point is a sub-pixel of the fourth face image, and determining that a pixel in the third face image, which is matched with the point, is a first pixel in the third face image.
And step S104, inputting the first face image into a preset neural network to obtain a second face image.
The size of the second face image is larger than that of the first face image, and the resolution of the second face image is larger than that of the first face image. Optionally, the user may set the size and resolution of the second face image according to actual needs.
The preset neural network may refer to a preset neural network for improving the resolution of the first face image, including but not limited to, the rearrangement of the image from depth to spatial data block depth-to-space layer. A deep learning hyper-resolution network (namely a preset neural network) is added after multi-frame hyper-resolution, the deep learning hyper-resolution model can remove noise and blur, and the resolution is improved on the basis of ensuring the face characteristics to be unchanged.
And step S105, recognizing the face in the second face image.
In this embodiment of the application, the face in the second face image may be recognized through a preset face detection algorithm, where the preset face detection algorithm may refer to a preset algorithm for face recognition, and a specific algorithm is not limited herein.
According to the embodiment of the application, continuous N frames of initial images aiming at a target face are obtained firstly, the face image of the target face in each frame of initial image is obtained again, sub-pixel interpolation is carried out on the N frames of face images, the first face image with the amplified size and resolution is obtained, the first face image is input to the preset neural network again, image enhancement is carried out on the first face image, the size and the resolution of the first face image are further amplified, namely, the application combines multi-frame super-score with deep learning, the number of real details in the face image can be effectively increased, and the face recognition precision is further improved.
Referring to fig. 3, which is a schematic view of an implementation flow of a face recognition method provided in the second embodiment of the present application, where the face recognition method is applied to a mobile terminal, as shown in the figure, the face recognition method may include the following steps:
step S301, obtaining the light intensity of the current environment and/or the size of the target face in the picture.
The current environment may be a current photographing environment, and the light intensity of the current environment may be obtained through a light sensor in the mobile terminal. The picture may refer to a picture captured by a camera device.
Step S302, if the light intensity of the current environment is smaller than an intensity threshold and/or the size of the target face in the picture is smaller than a size threshold, acquiring N continuous frames of initial images with the same exposure degree aiming at the target face.
In the embodiment of the application, if the light intensity of the current photographing environment is smaller than the intensity threshold, the light intensity of the current photographing environment is poor, and the imaging quality is affected, so that the initial images with the same exposure of the continuous N frames of the target face can be obtained, the face image with high resolution is obtained by using the initial images with the same exposure of the N frames, and the face recognition precision is further improved; if the size of the target face in the picture captured by the camera device is smaller than the size threshold, the target face in the picture is smaller, and the face recognition precision is influenced, so that continuous N frames of initial images with the same exposure can be obtained, the N frames of initial images with the same exposure are used for obtaining a high-resolution face image, and the face recognition precision is further improved; if the light intensity of the current photographing environment is smaller than the intensity threshold and the size of the target face in the picture captured by the camera device is smaller than the size threshold, the situation that the light of the current photographing environment is poor and the target face in the picture is small, and the face recognition accuracy is affected is shown, so that N continuous frames of initial images with the same exposure can be obtained, high-resolution face images are obtained by using the N frames of initial images with the same exposure, and the face recognition accuracy is improved.
Step S303, acquiring the face image of the target face in each frame of initial image in N frames of initial images.
The step is the same as step S102, and reference may be made to the related description of step S102, which is not repeated herein.
Optionally, the obtaining the face image of the target face in each frame of initial image includes:
selecting a frame of initial image from the N frames of initial images as a reference initial image;
acquiring the target face from the reference initial image;
and acquiring the face image of the target face in each frame of initial image in the N frames of initial images according to the target face acquired in the reference initial image.
In the embodiment of the application, the definition of each frame of initial image in the N frames of initial images can be obtained, the initial image with the highest definition in the N frames of initial images is used as a reference initial image, a target face needing to be over-divided can be obtained from the reference initial image, then the N frames of initial images are cut according to the target face determined from the reference initial image, and the face image of the target face in the N frames of initial images is cut.
And step S304, performing sub-pixel interpolation on the N frames of face images to obtain a first face image.
The step is the same as step S103, and reference may be made to the related description of step S103, which is not described herein again.
Step S305, inputting the first face image into a preset neural network, and obtaining a second face image.
The step is the same as step S104, and reference may be made to the related description of step S104, which is not repeated herein.
As shown in fig. 4, which is an exemplary diagram of a face image processing process, first five continuous frames of initial images of a target face are obtained, then one frame is selected as a reference initial image, the target face is selected from the reference initial image, five frames of initial images are cut according to the target face, five frames of face images are cut, sub-pixel interpolation is performed on the five frames of face images to obtain a first face image, and then the first face image is input to a preset neural network to obtain a second face image.
And step S306, recognizing the face in the second face image.
The step is the same as step S105, and reference may be made to the related description of step S105, which is not repeated herein.
According to the embodiment of the application, when the face is small in the picture or the light of the current photographing environment is weak, the number of real details in the face image can be effectively increased through combination of multi-frame super-resolution and deep learning, and then the face recognition precision is improved.
Fig. 5 is a schematic structural diagram of a face recognition apparatus provided in the third embodiment of the present application, and for convenience of description, only the relevant portions of the third embodiment of the present application are shown.
The face recognition apparatus includes:
an initial image obtaining module 51, configured to obtain N consecutive initial images of a target face, where N is an integer greater than 1;
a face image obtaining module 52, configured to obtain a face image of the target face in each frame of initial image in the N frames of initial images, where the face image of the target face in each frame of initial image refers to an image of an area where the target face is located in each frame of initial image, the N frames of initial images correspond to the N frames of face images, and the sizes of the N frames of face images are the same;
a sub-pixel interpolation module 53, configured to perform sub-pixel interpolation on the N frames of face images to obtain a first face image, where the size of the first face image is greater than that of each frame of face image, and the resolution of the first face image is greater than that of each frame of face image;
a face image input module 54, configured to input the first face image to a preset neural network, so as to obtain a second face image, where a size of the second face image is larger than a size of the first face image, and a resolution of the second face image is larger than a resolution of the first face image;
and a face recognition module 55, configured to recognize a face in the second face image.
Optionally, the sub-pixel interpolation module 53 includes:
the face selection unit is used for selecting one frame of face image from the N frames of face images as a reference face image;
a first obtaining unit, configured to perform bilinear interpolation on the reference face image to obtain a third face image, where a size of the third face image is the same as that of the first face image, and a resolution of the third face image is greater than that of the reference face image;
a second obtaining unit, configured to perform bilinear interpolation on an ith frame of face image in remaining N-1 frames of face images to obtain a fourth face image, where the remaining N-1 frames of face images refer to face images, except for the reference face image, in the N frames of face images, a size of the fourth face image is the same as a size of the first face image, a resolution of the fourth face image is greater than a resolution of the ith frame of face image, and i is an integer greater than zero and less than or equal to N-1;
a first obtaining unit, configured to perform image matching on the third face image and the fourth face image, and obtain a first pixel in the third face image and a sub-pixel in the fourth face image, where the sub-pixel is matched with the first pixel;
a second acquisition unit configured to acquire pixel values of four pixels adjacent to the sub-pixels in the fourth face image;
a third obtaining unit, configured to obtain pixel values of the sub-pixels in the fourth face image according to pixel values of four pixels adjacent to the sub-pixels in the fourth face image;
a pixel determination unit, configured to add and average a pixel value of the first pixel in the third face image and a pixel value of the sub-pixel in the fourth face image, and use the average value as the pixel value of the first pixel in the third face image;
and the image determining unit is used for repeatedly executing the second obtaining unit, the first obtaining unit, the second obtaining unit, the third obtaining unit and the pixel determining unit until the residual N-1 frames of face images are traversed, and determining the processed third face image as the first face image, wherein the processed third face image is the third face image obtained by adding and averaging the pixel values of the first pixels.
Optionally, the third obtaining unit is specifically configured to:
acquiring offsets of four pixels adjacent to the sub-pixels in the fourth face image and the sub-pixels;
and acquiring the pixel values of the sub-pixels in the fourth face image according to the offsets of the four pixels adjacent to the sub-pixels in the fourth face image and the sub-pixels and the pixel values of the four pixels adjacent to the sub-pixels in the fourth face image.
Optionally, the first obtaining unit is specifically configured to:
performing image matching on the third face image and the fourth face image to obtain a point in the fourth face image, which is matched with a pixel in the third face image;
and if the coordinate of the point in the fourth face image is a decimal, determining that the point is a sub-pixel of the fourth face image, and a pixel matched with the point in the third face image is a first pixel in the third face image.
Optionally, the face recognition apparatus further includes:
the parameter acquisition module is used for acquiring the light intensity of the current environment and/or the size of the target face in the picture;
the initial image obtaining module 51 is specifically configured to:
and if the light intensity of the current environment is smaller than an intensity threshold value and/or the size of the target face in the picture is smaller than a size threshold value, acquiring continuous N frames of initial images aiming at the target face.
Optionally, the exposure levels of the N initial images are the same.
Optionally, the facial image acquiring module 52 includes:
the image selecting unit is used for selecting one frame of initial image from the N frames of initial images as a reference initial image;
a face obtaining unit, configured to obtain the target face from the reference initial image;
and the image acquisition unit is used for acquiring the face image of the target face in each frame of initial image in the N frames of initial images according to the target face acquired in the reference initial image.
The face recognition device provided in the embodiment of the present application can be applied to the first method embodiment and the second method embodiment, and for details, reference is made to the description of the first method embodiment and the second method embodiment, and details are not repeated here.
Fig. 6 is a schematic structural diagram of a mobile terminal according to a fourth embodiment of the present application. The mobile terminal as shown in the figure may include: one or more processors 601 (only one shown); one or more input devices 602 (only one shown), one or more output devices 603 (only one shown), and memory 604. The processor 601, the input device 602, the output device 603, and the memory 604 are connected by a bus 605. The memory 604 is used for storing instructions and the processor 601 is used for executing instructions stored by the memory 604. Wherein:
the processor 601 is configured to obtain N consecutive initial images of a target face, where N is an integer greater than 1; acquiring a face image of the target face in each frame of initial image in the N frames of initial images, wherein the face image of the target face in each frame of initial image refers to an image of an area where the target face is located in each frame of initial image, the N frames of initial images correspond to the N frames of face images, and the N frames of face images have the same size; performing sub-pixel interpolation on the N frames of face images to obtain a first face image, wherein the size of the first face image is larger than that of each frame of face image, and the resolution of the first face image is larger than that of each frame of face image; inputting the first face image into a preset neural network to obtain a second face image, wherein the size of the second face image is larger than that of the first face image, and the resolution of the second face image is larger than that of the first face image; and recognizing the face in the second face image.
Optionally, the processor 601 is specifically configured to:
selecting one frame of face image from the N frames of face images as a reference face image;
carrying out bilinear interpolation on the reference face image to obtain a third face image, wherein the size of the third face image is the same as that of the first face image, and the resolution of the third face image is greater than that of the reference face image;
a1, performing bilinear interpolation on an ith frame of face image in the remaining N-1 frames of face images to obtain a fourth face image, wherein the remaining N-1 frames of face images refer to face images except the reference face image in the N frames of face images, the size of the fourth face image is the same as that of the first face image, the resolution of the fourth face image is greater than that of the ith frame of face image, and i is an integer greater than zero and less than or equal to N-1;
step a2, performing image matching on the third face image and the fourth face image to obtain a first pixel in the third face image and a sub-pixel matched with the first pixel in the fourth face image;
a3, acquiring pixel values of four pixels adjacent to the sub-pixels in the fourth face image;
a4, acquiring pixel values of the sub-pixels in the fourth face image according to pixel values of four pixels adjacent to the sub-pixels in the fourth face image;
a5, adding the pixel value of the first pixel in the third face image and the pixel value of the sub-pixel in the fourth face image for averaging, and taking the average value as the pixel value of the first pixel in the third face image;
repeatedly executing the steps a1, a2, a3, a4 and a5 until the remaining N-1 frames of face images are traversed, and determining the processed third face image as the first face image, wherein the processed third face image is the third face image of which the pixel values of the first pixels are subjected to addition and averaging processing.
Optionally, the processor 601 is specifically configured to:
acquiring offsets of four pixels adjacent to the sub-pixels in the fourth face image and the sub-pixels;
and acquiring the pixel values of the sub-pixels in the fourth face image according to the offsets of the four pixels adjacent to the sub-pixels in the fourth face image and the sub-pixels and the pixel values of the four pixels adjacent to the sub-pixels in the fourth face image.
Optionally, the processor 601 is specifically configured to:
performing image matching on the third face image and the fourth face image to obtain a point in the fourth face image, which is matched with a pixel in the third face image;
and if the coordinate of the point in the fourth face image is a decimal, determining that the point is a sub-pixel of the fourth face image, and a pixel matched with the point in the third face image is a first pixel in the third face image.
Optionally, before acquiring N consecutive initial images of the target face, the processor 601 is further configured to:
and acquiring the light intensity of the current environment and/or the size of the target face in the picture.
Optionally, the processor 601 is specifically configured to:
and if the light intensity of the current environment is smaller than an intensity threshold value and/or the size of the target face in the picture is smaller than a size threshold value, acquiring continuous N frames of initial images aiming at the target face.
Optionally, the exposure levels of the N initial images are the same.
Optionally, the processor 601 is specifically configured to:
selecting a frame of initial image from the N frames of initial images as a reference initial image;
acquiring the target face from the reference initial image;
and acquiring the face image of the target face in each frame of initial image in the N frames of initial images according to the target face acquired in the reference initial image.
It should be understood that, in the embodiment of the present Application, the Processor 601 may be a Central Processing Unit (CPU), and the Processor may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The input device 602 may include a touch pad, a fingerprint sensor (for collecting fingerprint information of a user and direction information of the fingerprint), a microphone, a data receiving interface, and the like. The output device 603 may include a display (LCD, etc.), speakers, a data transmission interface, and the like.
The memory 604 may include both read-only memory and random access memory, and provides instructions and data to the processor 601. A portion of the memory 604 may also include non-volatile random access memory. For example, the memory 604 may also store device type information.
In a specific implementation, the processor 601, the input device 602, the output device 603, and the memory 604 described in this embodiment of the present application may execute the implementation described in the embodiment of the face recognition method provided in this embodiment of the present application, or may execute the implementation described in the face recognition apparatus described in the third embodiment of the present application, which is not described herein again.
Fig. 7 is a schematic structural diagram of a mobile terminal according to a fifth embodiment of the present application. As shown in fig. 7, the mobile terminal 7 of this embodiment includes: one or more processors 70 (only one shown), a memory 71, and a computer program 72 stored in the memory 71 and executable on the at least one processor 70. The processor 70, when executing the computer program 72, implements the steps in the various face recognition method embodiments described above. The mobile terminal 7 may be a desktop computer, a notebook, a palm computer, a cloud server, or other computing devices. The mobile terminal may include, but is not limited to, a processor 70, a memory 71. It will be appreciated by those skilled in the art that fig. 7 is only an example of a mobile terminal 7 and does not constitute a limitation of the mobile terminal 7, and that it may comprise more or less components than those shown, or some components may be combined, or different components, e.g. the mobile terminal may further comprise input output devices, network access devices, buses, etc.
The processor 70 may be a central processing unit CPU, but may also be other general purpose processors, digital signal processors DSP, application specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 71 may be an internal storage unit of the mobile terminal 7, such as a hard disk or a memory of the mobile terminal 7. The memory 71 may also be an external storage device of the mobile terminal 7, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the mobile terminal 7. Further, the memory 71 may also include both an internal storage unit and an external storage device of the mobile terminal 7. The memory 71 is used for storing the computer program and other programs and data required by the mobile terminal. The memory 71 may also be used to temporarily store data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus/mobile terminal and method may be implemented in other ways. For example, the above-described apparatus/mobile terminal embodiments are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow in the method of the embodiments described above can be realized by a computer program, which can be stored in a computer-readable storage medium and can realize the steps of the embodiments of the methods described above when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
When the computer program product runs on a mobile terminal, the steps in the method embodiments can be realized when the mobile terminal executes the computer program product.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.
Claims (10)
1. A face recognition method is characterized by comprising the following steps:
acquiring continuous N frames of initial images aiming at a target face, wherein N is an integer greater than 1;
acquiring a face image of the target face in each frame of initial image in the N frames of initial images, wherein the face image of the target face in each frame of initial image refers to an image of an area where the target face is located in each frame of initial image, the N frames of initial images correspond to the N frames of face images, and the N frames of face images have the same size;
performing sub-pixel interpolation on the N frames of face images to obtain a first face image, wherein the size of the first face image is larger than that of each frame of face image, and the resolution of the first face image is larger than that of each frame of face image;
inputting the first face image into a preset neural network to obtain a second face image, wherein the size of the second face image is larger than that of the first face image, and the resolution of the second face image is larger than that of the first face image;
and recognizing the face in the second face image.
2. The method of claim 1, wherein the performing sub-pixel interpolation on the N frames of face images to obtain a first face image comprises:
selecting one frame of face image from the N frames of face images as a reference face image;
carrying out bilinear interpolation on the reference face image to obtain a third face image, wherein the size of the third face image is the same as that of the first face image, and the resolution of the third face image is greater than that of the reference face image;
a1, performing bilinear interpolation on an ith frame of face image in the remaining N-1 frames of face images to obtain a fourth face image, wherein the remaining N-1 frames of face images refer to face images except the reference face image in the N frames of face images, the size of the fourth face image is the same as that of the first face image, the resolution of the fourth face image is greater than that of the ith frame of face image, and i is an integer greater than zero and less than or equal to N-1;
step a2, performing image matching on the third face image and the fourth face image to obtain a first pixel in the third face image and a sub-pixel matched with the first pixel in the fourth face image;
a3, acquiring pixel values of four pixels adjacent to the sub-pixels in the fourth face image;
a4, acquiring pixel values of the sub-pixels in the fourth face image according to pixel values of four pixels adjacent to the sub-pixels in the fourth face image;
a5, adding the pixel value of the first pixel in the third face image and the pixel value of the sub-pixel in the fourth face image for averaging, and taking the average value as the pixel value of the first pixel in the third face image;
repeatedly executing the steps a1, a2, a3, a4 and a5 until the remaining N-1 frames of face images are traversed, and determining the processed third face image as the first face image, wherein the processed third face image is the third face image of which the pixel values of the first pixels are subjected to addition and averaging processing.
3. The face recognition method according to claim 2, wherein the obtaining of the pixel value of the sub-pixel in the fourth face image based on the pixel values of four pixels adjacent to the sub-pixel in the fourth face image comprises:
acquiring offsets of four pixels adjacent to the sub-pixels in the fourth face image and the sub-pixels;
and acquiring the pixel values of the sub-pixels in the fourth face image according to the offsets of the four pixels adjacent to the sub-pixels in the fourth face image and the sub-pixels and the pixel values of the four pixels adjacent to the sub-pixels in the fourth face image.
4. The method of claim 2, wherein the image matching the third face image with the fourth face image to obtain a first pixel in the third face image and a sub-pixel in the fourth face image matching the first pixel comprises:
performing image matching on the third face image and the fourth face image to obtain a point in the fourth face image, which is matched with a pixel in the third face image;
and if the coordinate of the point in the fourth face image is a decimal, determining that the point is a sub-pixel of the fourth face image, and a pixel matched with the point in the third face image is a first pixel in the third face image.
5. The face recognition method of claim 1, before acquiring the N consecutive initial images for the target face, further comprising:
acquiring the light intensity of the current environment and/or the size of a target face in a picture;
correspondingly, the acquiring of the N consecutive initial images of the target face includes:
and if the light intensity of the current environment is smaller than an intensity threshold value and/or the size of the target face in the picture is smaller than a size threshold value, acquiring continuous N frames of initial images aiming at the target face.
6. The face recognition method of claim 1, wherein the exposure levels of the N initial images are the same.
7. The face recognition method according to any one of claims 1 to 6, wherein the obtaining the face image of the target face in each of the N initial images comprises:
selecting a frame of initial image from the N frames of initial images as a reference initial image;
acquiring the target face from the reference initial image;
and acquiring the face image of the target face in each frame of initial image in the N frames of initial images according to the target face acquired in the reference initial image.
8. A face recognition apparatus, characterized in that the face recognition apparatus comprises:
the system comprises an initial image acquisition module, a face recognition module and a face recognition module, wherein the initial image acquisition module is used for acquiring continuous N frames of initial images aiming at a target face, and N is an integer greater than 1;
a face image obtaining module, configured to obtain a face image of the target face in each frame of initial image in the N frames of initial images, where the face image of the target face in each frame of initial image refers to an image of an area where the target face is located in each frame of initial image, the N frames of initial images correspond to the N frames of face images, and the sizes of the N frames of face images are the same;
the sub-pixel interpolation module is used for performing sub-pixel interpolation on the N frames of face images to obtain a first face image, wherein the size of the first face image is larger than that of each frame of face image, and the resolution of the first face image is larger than that of each frame of face image;
the face image input module is used for inputting the first face image into a preset neural network to obtain a second face image, wherein the size of the second face image is larger than that of the first face image, and the resolution of the second face image is larger than that of the first face image;
and the face recognition module is used for recognizing the face in the second face image.
9. A mobile terminal comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the face recognition method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the face recognition method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911421186.7A CN111222446B (en) | 2019-12-31 | 2019-12-31 | Face recognition method, face recognition device and mobile terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911421186.7A CN111222446B (en) | 2019-12-31 | 2019-12-31 | Face recognition method, face recognition device and mobile terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111222446A true CN111222446A (en) | 2020-06-02 |
| CN111222446B CN111222446B (en) | 2023-05-16 |
Family
ID=70808321
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911421186.7A Active CN111222446B (en) | 2019-12-31 | 2019-12-31 | Face recognition method, face recognition device and mobile terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111222446B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112509144A (en) * | 2020-12-09 | 2021-03-16 | 深圳云天励飞技术股份有限公司 | Face image processing method and device, electronic equipment and storage medium |
| CN115908119A (en) * | 2023-01-05 | 2023-04-04 | 广州佰锐网络科技有限公司 | Facial image beauty processing method and system based on artificial intelligence |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102902966A (en) * | 2012-10-12 | 2013-01-30 | 大连理工大学 | Super-resolution face recognition method based on deep belief networks |
| CN107481188A (en) * | 2017-06-23 | 2017-12-15 | 珠海经济特区远宏科技有限公司 | A kind of image super-resolution reconstructing method |
| CN107895345A (en) * | 2017-11-29 | 2018-04-10 | 浙江大华技术股份有限公司 | A kind of method and apparatus for improving facial image resolution ratio |
| CN110008811A (en) * | 2019-01-21 | 2019-07-12 | 北京工业职业技术学院 | Face identification system and method |
| CN110246084A (en) * | 2019-05-16 | 2019-09-17 | 五邑大学 | A kind of super-resolution image reconstruction method and its system, device, storage medium |
-
2019
- 2019-12-31 CN CN201911421186.7A patent/CN111222446B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102902966A (en) * | 2012-10-12 | 2013-01-30 | 大连理工大学 | Super-resolution face recognition method based on deep belief networks |
| CN107481188A (en) * | 2017-06-23 | 2017-12-15 | 珠海经济特区远宏科技有限公司 | A kind of image super-resolution reconstructing method |
| CN107895345A (en) * | 2017-11-29 | 2018-04-10 | 浙江大华技术股份有限公司 | A kind of method and apparatus for improving facial image resolution ratio |
| CN110008811A (en) * | 2019-01-21 | 2019-07-12 | 北京工业职业技术学院 | Face identification system and method |
| CN110246084A (en) * | 2019-05-16 | 2019-09-17 | 五邑大学 | A kind of super-resolution image reconstruction method and its system, device, storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112509144A (en) * | 2020-12-09 | 2021-03-16 | 深圳云天励飞技术股份有限公司 | Face image processing method and device, electronic equipment and storage medium |
| CN115908119A (en) * | 2023-01-05 | 2023-04-04 | 广州佰锐网络科技有限公司 | Facial image beauty processing method and system based on artificial intelligence |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111222446B (en) | 2023-05-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110660066B (en) | Training method of network, image processing method, network, terminal equipment and medium | |
| CN109064428B (en) | Image denoising processing method, terminal device and computer readable storage medium | |
| CN109146855B (en) | Image moire detection method, terminal device and storage medium | |
| CN108921806B (en) | Image processing method, image processing device and terminal equipment | |
| CN110473242B (en) | Texture feature extraction method, texture feature extraction device and terminal equipment | |
| CN110335216B (en) | Image processing method, image processing apparatus, terminal device, and readable storage medium | |
| CN109005368B (en) | High dynamic range image generation method, mobile terminal and storage medium | |
| CN111368717B (en) | Line-of-sight determination method, line-of-sight determination device, electronic apparatus, and computer-readable storage medium | |
| CN109286758B (en) | High dynamic range image generation method, mobile terminal and storage medium | |
| CN109214996B (en) | Image processing method and device | |
| CN107786780B (en) | Video image noise reduction method and device and computer readable storage medium | |
| CN112214773B (en) | Image processing method and device based on privacy protection and electronic equipment | |
| CN108764139B (en) | Face detection method, mobile terminal and computer readable storage medium | |
| CN111131688B (en) | Image processing method and device and mobile terminal | |
| CN111290684B (en) | Image display method, image display device and terminal equipment | |
| CN111667504A (en) | Face tracking method, device and equipment | |
| CN111222446B (en) | Face recognition method, face recognition device and mobile terminal | |
| CN113592776A (en) | Image processing method and device, electronic device and storage medium | |
| CN111340722B (en) | Image processing method, processing device, terminal equipment and readable storage medium | |
| CN113570725A (en) | Three-dimensional surface reconstruction method and device based on clustering, server and storage medium | |
| CN111062279B (en) | Photo processing method and photo processing device | |
| CN114493988A (en) | Image blurring method, image blurring device and terminal equipment | |
| CN111754435A (en) | Image processing method, image processing device, terminal equipment and computer readable storage medium | |
| CN111833285A (en) | Image processing method, image processing device and terminal equipment | |
| CN110610178A (en) | Image recognition method, device, terminal and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |