CN106295566B - Facial expression recognizing method and device - Google Patents
Facial expression recognizing method and device Download PDFInfo
- Publication number
- CN106295566B CN106295566B CN201610653790.2A CN201610653790A CN106295566B CN 106295566 B CN106295566 B CN 106295566B CN 201610653790 A CN201610653790 A CN 201610653790A CN 106295566 B CN106295566 B CN 106295566B
- Authority
- CN
- China
- Prior art keywords
- key point
- expression recognition
- human face
- face region
- topography
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/174—Facial expression recognition
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The disclosure is directed to a kind of facial expression recognizing method and devices, belong to image identification technical field.The described method includes: detecting human face region from images to be recognized；Obtain the key point in human face region；Topography is extracted from human face region according to key point；Topography is identified using the Expression Recognition model for completing training, obtains Expression Recognition result.The disclosure carries out feature extraction and expression differentiation at face key point by using the Expression Recognition model for completing training, and two sseparated steps are combined into one, to reduce accumulated error, improve the accuracy of facial expression recognition.Also, due to the characteristic information for only extracting the topography at face key point, rather than extract the global characteristics information of entire human face region, can it is more accurate, efficiently extract the feature for embodying emotional state, further increase the accuracy of facial expression recognition.
Description
Technical field
This disclosure relates to image identification technical field, in particular to a kind of facial expression recognizing method and device.
Background technique
Facial expression recognition refers to the emotional state for identifying from given facial image and determining face.For example, glad, sad
Wound, surprised, frightened, detest, anger etc..Facial expression recognition is widely used to psychic science, neuroscience, work at present
The fields such as Cheng Kexue and computer science.
In the related art, facial expression recognition includes following two key steps: first, it is detected from images to be recognized
Human face region, and countenance feature is extracted from human face region, wherein HOG (Histogram of Oriented can be used
Gradient, histograms of oriented gradients), LBP (Local Binary Pattern, local binary patterns), the features such as Gabor mention
Algorithm is taken to extract countenance feature；Second, based on countenance feature carry out expression classification, obtain Expression Recognition as a result, its
In, Adaboost algorithm, SVM (Support Vector Machine, support vector machines) algorithm, random can be used in sorting algorithm
Forest algorithm etc..
Summary of the invention
The embodiment of the present disclosure provides a kind of facial expression recognizing method and device.The technical solution is as follows:
According to the first aspect of the embodiments of the present disclosure, a kind of facial expression recognizing method is provided, which comprises
Human face region is detected from images to be recognized；
Obtain the key point in the human face region；
Topography is extracted from the human face region according to the key point；
The topography is identified using the Expression Recognition model for completing training, obtains Expression Recognition result.
It is optionally, described that topography is extracted from the human face region according to the key point, comprising:
Obtain the image block around each key point；
Each described image block is overlapped or is spliced by preset order, the topography is obtained.
Optionally, the image block obtained around each key point, comprising:
For each key point, the image block of the predetermined size centered on the key point is intercepted.
Optionally, described that the topography is identified using the Expression Recognition model for completing training, obtain expression
Recognition result, comprising:
Using the characteristic information of topography described in the Expression Recognition model extraction for completing training；
Using the Expression Recognition model according to the characteristic information, the Expression Recognition result is determined.
Optionally, the key point obtained in the human face region, comprising:
The human face region is zoomed into target size；
The key point is positioned from the human face region after scaling.
Optionally, the Expression Recognition model is convolutional neural networks model.
According to the second aspect of an embodiment of the present disclosure, a kind of facial expression recognition device is provided, described device includes:
Face detection module is configured as detecting human face region from images to be recognized；
Key point obtains module, is configured as obtaining the key point in the human face region；
Image zooming-out module is configured as extracting topography from the human face region according to the key point；
Expression Recognition module is configured as knowing the topography using the Expression Recognition model for completing training
Not, Expression Recognition result is obtained.
Optionally, described image extraction module, comprising:
Image block acquisition submodule is configured as obtaining the image block around each key point；
Image block handles submodule, is configured as that each described image block is overlapped or is spliced by preset order, obtains
To the topography.
Optionally, described image block acquisition submodule is configured as intercepting with the key point each key point
Centered on predetermined size image block.
Optionally, the Expression Recognition module, comprising:
Feature extraction submodule is configured as the spy using topography described in the Expression Recognition model extraction for completing training
Reference breath；
Identify and determine submodule, be configured as using the Expression Recognition model according to the characteristic information, determine described in
Expression Recognition result.
Optionally, the key point obtains module, comprising:
Face scales submodule, is configured as the human face region zooming to target size；
Key point positioning submodule is configured as positioning the key point from the human face region after scaling.
Optionally, the Expression Recognition model is convolutional neural networks model.
According to the third aspect of an embodiment of the present disclosure, a kind of facial expression recognition device is provided, described device includes:
Processor；
For storing the memory of the executable instruction of the processor；
Wherein, the processor is configured to:
Human face region is detected from images to be recognized；
Obtain the key point in the human face region；
Topography is extracted from the human face region according to the key point；
The topography is identified using the Expression Recognition model for completing training, obtains Expression Recognition result.
The technical solution that the embodiment of the present disclosure provides can include the following benefits:
By detecting human face region from image to be identified, obtain the key point in human face region, according to key point from
Topography is extracted in human face region, and topography is identified using the Expression Recognition model for completing training, obtains table
Feelings recognition result；It solves in the related technology since feature extraction and expression differentiation are two sseparated steps, needs to adopt respectively
It is handled with two different algorithms, leads to the problem of influencing the precision of facial expression recognition there are accumulated error；It has used
Feature extraction and expression differentiation at face key point are carried out at trained Expression Recognition model, two sseparated steps are closed two
It is one, to reduce accumulated error, improves the accuracy of facial expression recognition.Also, due to only extracting at face key point
The characteristic information of topography, rather than extract the global characteristics information of entire human face region, can it is more accurate, efficiently extract
The feature for embodying emotional state, further increases the accuracy of facial expression recognition.
It should be understood that above general description and following detailed description be only it is exemplary and explanatory, not
The disclosure can be limited.
Detailed description of the invention
The drawings herein are incorporated into the specification and forms part of this specification, and shows the implementation for meeting the disclosure
Example, and together with specification for explaining the principles of this disclosure.
Fig. 1 is a kind of flow chart of facial expression recognizing method shown according to an exemplary embodiment；
Fig. 2A is a kind of flow chart of the facial expression recognizing method shown according to another exemplary embodiment；
Fig. 2 B is the key that Fig. 2A illustrated embodiment is related to the schematic diagram of point location；
Fig. 2 C is a kind of structural schematic diagram of the convolutional neural networks illustrated；
Fig. 3 is a kind of block diagram of facial expression recognition device shown according to an exemplary embodiment；
Fig. 4 is a kind of block diagram of the facial expression recognition device shown according to another exemplary embodiment；
Fig. 5 is a kind of block diagram of device shown according to an exemplary embodiment.
Specific embodiment
Example embodiments are described in detail here, and the example is illustrated in the accompanying drawings.Following description is related to
When attached drawing, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements.Following exemplary embodiment
Described in embodiment do not represent all implementations consistent with this disclosure.On the contrary, they be only with it is such as appended
The example of the consistent device and method of some aspects be described in detail in claims, the disclosure.
In the related art, feature extraction and expression differentiation are two sseparated steps, need to be respectively adopted two kinds of differences
Algorithm handled, there are accumulated errors, influence the precision of facial expression recognition.The technical side that the embodiment of the present disclosure provides
Case carries out feature extraction and expression differentiation at face key point using the Expression Recognition model for completing training, two is separated
The step of be combined into one, to reduce accumulated error, improve the accuracy of facial expression recognition.Also, due to only extracting face
The characteristic information of topography at key point, rather than extract the global characteristics information of entire human face region, can it is more accurate,
The feature for embodying emotional state is efficiently extracted, the accuracy of facial expression recognition is further increased.
The method that the embodiment of the present disclosure provides, the executing subject of each step can be the electronics with image-capable and set
It is standby, such as PC, smart phone, tablet computer, server etc..For ease of description, in following methods embodiment, with
The executing subject of each step is illustrated for electronic equipment.
Fig. 1 is a kind of flow chart of facial expression recognizing method shown according to an exemplary embodiment.This method can be with
It comprises the following steps:
In a step 101, human face region is detected from images to be recognized.
In a step 102, the key point in human face region is obtained.
Key point is also referred to as characteristic point, face key point or human face characteristic point, and expression can be embodied by referring in human face region
The face location of state, including but not limited to eyes (such as canthus, eyeball center, eye tail), nose (such as nose, the wing of nose), mouth
(such as corners of the mouth, labial angle, lip), chin, eyebrow angle face location.
In step 103, topography is extracted from human face region according to key point.
At step 104, topography is identified using the Expression Recognition model for completing training, obtains Expression Recognition
As a result.
In conclusion method provided in this embodiment, solves in the related technology since feature extraction and expression differentiation are
Two sseparated steps need to be respectively adopted two different algorithms and are handled, and cause to influence face table there are accumulated error
The problem of precision of feelings identification.The feature extraction and expression at face key point are carried out using the Expression Recognition model for completing training
Differentiate, two sseparated steps are combined into one, to reduce accumulated error, improve the accuracy of facial expression recognition.Also,
Due to only extracting the characteristic information of the topography at face key point, rather than extract the global characteristics letter of entire human face region
Breath, can it is more accurate, efficiently extract embody emotional state feature, further increase the accuracy of facial expression recognition.
Fig. 2A is a kind of flow chart of the facial expression recognizing method shown according to another exemplary embodiment.This method can
To comprise the following steps:
In step 201, human face region is detected from images to be recognized.
Electronic equipment detects human face region using relevant Face datection algorithm from images to be recognized.In the present embodiment
In, the specific type of Face datection algorithm is not construed as limiting.For example, Face datection algorithm can be LBP algorithm and Adaboost
The combination of algorithm extracts characteristics of image using LBP algorithm from images to be recognized, and using Adaboost cascade classifier according to
Characteristics of image determines human face region.Images to be recognized can be include face image.
In step 202, human face region is zoomed into target size.
Since the human face region size in different images is different, in order to ensure the accuracy of subsequent characteristics point location, will mention
The human face region taken zooms to fixed target size.In the present embodiment, the size of target size is not construed as limiting, it can root
It is preset according to actual conditions.For example, the target size is 96 × 96 (pixels).
In step 203, key point is positioned from the human face region after scaling.
Key point is also referred to as characteristic point, face key point or human face characteristic point, and expression can be embodied by referring in human face region
The face location of state, including but not limited to eyes (such as canthus, eyeball center, eye tail), nose (such as nose, the wing of nose), mouth
(such as corners of the mouth, labial angle, lip), chin, eyebrow angle face location.
Electronic equipment positions key point from the human face region after scaling using relevant face key point location algorithm.?
In the present embodiment, the specific type of face key point location algorithm is not construed as limiting.For example, face key point location algorithm can be with
It is SDM (Supervised Descent Method supervises descent method) algorithm.
In one example, as shown in Figure 2 B, it is multiple that acquisition is positioned from the human face region 21 after scaling using SDM algorithm
Key point, each key point is as shown in pore in figure.The position of the key point of required positioning and quantity can be preset, example
Such as 20.
In step 204, the image block around each key point is obtained.
Image block around key point refers to the image block of the predetermined size including the key point.In an example
In, for each key point, intercept the image block of the predetermined size centered on key point.In the present embodiment, to predetermined
The size of size is not construed as limiting, and can be preset according to the actual situation.For example, the predetermined size is 32 × 32 (pixels).
Optionally, before carrying out crucial point location, human face region can be converted to gray level image, it is fixed in gray level image
Position key point.Correspondingly, the image block around the key point of interception is also gray level image.
In step 205, each image block is overlapped or is spliced by preset order, obtain topography.
It before carrying out Expression Recognition, needs to integrate each image block, is input to expression knowledge as a whole
Other model.In a kind of possible embodiment, Expression Recognition mould is used as after each image block is overlapped by preset order
The input of type.In alternatively possible embodiment, know after each image block is spliced by preset order as expression
The input of other model.
Since the usually multiple and different key point of the key point of acquisition corresponds to different face locations, thus it is each
Image block needs are overlapped or splice according to preset order.It include that eyes, nose, mouth, eyebrow angle, chin are with key point
Example, preset order can be followed successively by eyes, nose, mouth, chin, eyebrow angle.That is, whether in the training of Expression Recognition model
Stage, or the Expression Recognition stage is being carried out using Expression Recognition model, the multiple images block intercepted from any image is equal
It is overlapped or splices according to same preset order, mutually unified with the structure of this input data for guaranteeing Expression Recognition model.
In step 206, topography is identified using the Expression Recognition model for completing training, obtains Expression Recognition
As a result.
Electronic equipment is using the characteristic information for completing trained Expression Recognition model extraction topography, using Expression Recognition
Model determines Expression Recognition result according to characteristic information.In the present embodiment, feature extraction and expression differentiate by Expression Recognition
Model is completed, and without being completed in two steps by two different algorithms, two sseparated steps is combined into one, to reduce
Accumulated error improves the accuracy of facial expression recognition.
In one example, Expression Recognition model be convolutional neural networks (Convolutional Neural Network,
CNN) model, or be depth convolutional neural networks model.Convolutional neural networks have powerful ability in feature extraction, utilize volume
Product neural network carries out feature extraction and expression differentiates, accuracy is higher.Convolutional neural networks include an input layer, at least one
A convolutional layer, at least one full articulamentum and an output layer.Wherein, the input data of input layer is and presses each image block
The topography obtained after sequence superposition or splicing；The output of output layer respectively indicates n kind expression the result is that length is the vector of n
Probability, n is integer greater than 1.Convolutional layer is used for feature extraction.Full articulamentum is for carrying out group to the feature that convolutional layer extracts
It closes and abstract, obtains being suitable for the data that output layer is classified.
In conjunction with reference Fig. 2 C, a kind of structural schematic diagram of convolutional neural networks is illustrated.The convolutional Neural net
Network includes 1 input layer, 3 convolutional layers (the first convolutional layer C1, the second convolutional layer C2 and third convolutional layer C3), 2 full connections
Layer (the first complete full articulamentum FC5 of articulamentum FC4 and second) and 1 output layer (Softmax layers).Assuming that being mentioned from human face region
20 key points are taken, the size of the image block around each key point intercepted is 32 × 32 (pixels), the input of input layer
Data are 20 × 32 × 32 superimposed image or stitching image.Wherein the number of convolution kernel is divided in 3 convolutional layers C1, C2 and C3
It Wei 36,64 and 32.The step-length of first convolutional layer C1 is 2, and the length of image and width are all contracted to after the first convolutional layer C1 calculating
The step-length of half originally, the second convolutional layer C2 and third convolutional layer C3 are 1.It should be noted that volume shown in fig. 2 C
Product neural network be only it is exemplary and explanatory, be not used to limit the disclosure.In general, the number of plies of convolutional neural networks
More, effect is better but the calculating time also can be longer, in practical applications, in combination with the requirement to accuracy of identification and efficiency, if
Count the convolutional neural networks of the appropriate number of plies.
In conclusion method provided in this embodiment, carries out face key point using the Expression Recognition model for completing training
The feature extraction at place and expression differentiate, two sseparated steps are combined into one, to reduce accumulated error, improve human face expression
The accuracy of identification.Also, due to the characteristic information for only extracting the topography at face key point, rather than extract entire face
The global characteristics information in region, can it is more accurate, efficiently extract embody emotional state feature, further increase face table
The accuracy of feelings identification.
Following is embodiment of the present disclosure, can be used for executing embodiments of the present disclosure.It is real for disclosure device
Undisclosed details in example is applied, embodiments of the present disclosure is please referred to.
Fig. 3 is a kind of block diagram of facial expression recognition device shown according to an exemplary embodiment.The device has real
The existing exemplary function of the above method, the function it is real can also to execute corresponding software by hardware realization by hardware
It is existing.The apparatus may include: face detection module 310, key point obtain module 320, image zooming-out module 330 and Expression Recognition
Module 340.
Face detection module 310 is configured as detecting human face region from images to be recognized.
Key point obtains module 320, is configured as obtaining the key point in the human face region.
Image zooming-out module 330 is configured as extracting topography from the human face region according to the key point.
Expression Recognition module 340 is configured as carrying out the topography using the Expression Recognition model for completing training
Identification, obtains Expression Recognition result.
In conclusion device provided in this embodiment, carries out face key point using the Expression Recognition model for completing training
The feature extraction at place and expression differentiate, two sseparated steps are combined into one, to reduce accumulated error, improve human face expression
The accuracy of identification.Also, due to the characteristic information for only extracting the topography at face key point, rather than extract entire face
The global characteristics information in region, can it is more accurate, efficiently extract embody emotional state feature, further increase face table
The accuracy of feelings identification.
Fig. 4 is a kind of block diagram of the facial expression recognition device shown according to another exemplary embodiment.The device has
Realize that the exemplary function of the above method, the function can also execute corresponding software by hardware realization by hardware
It realizes.The apparatus may include: face detection module 310, key point obtain module 320, image zooming-out module 330 and expression and know
Other module 340.
Face detection module 310 is configured as detecting human face region from images to be recognized.
Key point obtains module 320, is configured as obtaining the key point in the human face region.
Image zooming-out module 330 is configured as extracting topography from the human face region according to the key point.
Expression Recognition module 340 is configured as carrying out the topography using the Expression Recognition model for completing training
Identification, obtains Expression Recognition result.
In one example, described image extraction module 330, comprising: at image block acquisition submodule 330a and image block
Manage submodule 330b.
Image block acquisition submodule 330a is configured as obtaining the image block around each key point.
Image block handles submodule 330b, is configured as that each described image block is overlapped or is spelled by preset order
It connects, obtains the topography.
In one example, described image block acquisition submodule 330b is configured as each key point, interception with
The image block of predetermined size centered on the key point.
In one example, the Expression Recognition module 340, comprising: feature extraction submodule 340a and identification determine son
Module 340b.
Feature extraction submodule 340a is configured as using topography described in the Expression Recognition model extraction for completing training
Characteristic information.
It identifies and determines submodule 340b, be configured as being determined using the Expression Recognition model according to the characteristic information
The Expression Recognition result.
In one example, the key point obtains module 320, comprising: face scales submodule 320a and key point is fixed
Bit submodule 320b.
Face scales submodule 320a, is configured as the human face region zooming to target size.
Key point positioning submodule 320b is configured as positioning the key point from the human face region after scaling.
In one example, the Expression Recognition model is convolutional neural networks model.
In conclusion device provided in this embodiment, carries out face key point using the Expression Recognition model for completing training
The feature extraction at place and expression differentiate, two sseparated steps are combined into one, to reduce accumulated error, improve human face expression
The accuracy of identification.Also, due to the characteristic information for only extracting the topography at face key point, rather than extract entire face
The global characteristics information in region, can it is more accurate, efficiently extract embody emotional state feature, further increase face table
The accuracy of feelings identification.
It should be noted is that device provided by the above embodiment is when realizing its function, only with above-mentioned each function
The division progress of module, can be according to actual needs and by above-mentioned function distribution by different function for example, in practical application
Energy module is completed, i.e., the content structure of equipment is divided into different functional modules, to complete whole described above or portion
Divide function.
About the device in above-described embodiment, wherein modules execute the concrete mode of operation in related this method
Embodiment in be described in detail, no detailed explanation will be given here.
One exemplary embodiment of the disclosure additionally provides a kind of facial expression recognition device, can be realized disclosure offer
Facial expression recognizing method.The device includes: processor, and the memory of the executable instruction for storage processor.Its
In, processor is configured as:
Human face region is detected from images to be recognized；
Obtain the key point in the human face region；
Topography is extracted from the human face region according to the key point；
The topography is identified using the Expression Recognition model for completing training, obtains Expression Recognition result.
Optionally, the processor is configured to:
Obtain the image block around each key point；
Each described image block is overlapped or is spliced by preset order, the topography is obtained.
Optionally, the processor is configured to:
For each key point, the image block of the predetermined size centered on the key point is intercepted.
Optionally, the processor is configured to:
Using the characteristic information of topography described in the Expression Recognition model extraction for completing training；
Using the Expression Recognition model according to the characteristic information, the Expression Recognition result is determined.
Optionally, the processor is configured to:
The human face region is zoomed into target size；
The key point is positioned from the human face region after scaling.
Optionally, the Expression Recognition model is convolutional neural networks model.
Fig. 5 is a kind of block diagram of device 500 shown according to an exemplary embodiment.For example, device 500 can be movement
Phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, Medical Devices, body-building equipment,
Personal digital assistant etc..
Referring to Fig. 5, device 500 may include following one or more components: processing component 502, memory 504, power supply
Component 506, multimedia component 508, audio component 510, input/output (I/O) interface 512, sensor module 514, Yi Jitong
Believe component 516.
The integrated operation of the usual control device 500 of processing component 502, such as with display, telephone call, data communication, phase
Machine operation and record operate associated operation.Processing component 502 may include that one or more processors 520 refer to execute
It enables, to perform all or part of the steps of the methods described above.In addition, processing component 502 may include one or more modules, just
Interaction between processing component 502 and other assemblies.For example, processing component 502 may include multi-media module, it is more to facilitate
Interaction between media component 508 and processing component 502.
Memory 504 is configured as storing various types of data to support the operation in device 500.These data are shown
Example includes the instruction of any application or method for operating on device 500, contact data, and telephone book data disappears
Breath, picture, video etc..Memory 504 can be by any kind of volatibility or non-volatile memory device or their group
It closes and realizes, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM) is erasable to compile
Journey read-only memory (EPROM), programmable read only memory (PROM), read-only memory (ROM), magnetic memory, flash
Device, disk or CD.
Power supply module 506 provides electric power for the various assemblies of device 500.Power supply module 506 may include power management system
System, one or more power supplys and other with for device 500 generate, manage, and distribute the associated component of electric power.
Multimedia component 508 includes the screen of one output interface of offer between described device 500 and user.One
In a little embodiments, screen may include liquid crystal display (LCD) and touch panel (TP).If screen includes touch panel, screen
Curtain may be implemented as touch screen, to receive input signal from the user.Touch panel includes one or more touch sensings
Device is to sense the gesture on touch, slide, and touch panel.The touch sensor can not only sense touch or sliding action
Boundary, but also detect duration and pressure associated with the touch or slide operation.In some embodiments, more matchmakers
Body component 508 includes a front camera and/or rear camera.When device 500 is in operation mode, such as screening-mode or
When video mode, front camera and/or rear camera can receive external multi-medium data.Each front camera and
Rear camera can be a fixed optical lens system or have focusing and optical zoom capabilities.
Audio component 510 is configured as output and/or input audio signal.For example, audio component 510 includes a Mike
Wind (MIC), when device 500 is in operation mode, when such as call mode, recording mode, and voice recognition mode, microphone is matched
It is set to reception external audio signal.The received audio signal can be further stored in memory 504 or via communication set
Part 516 is sent.In some embodiments, audio component 510 further includes a loudspeaker, is used for output audio signal.
I/O interface 512 provides interface between processing component 502 and peripheral interface module, and above-mentioned peripheral interface module can
To be keyboard, click wheel, button etc..These buttons may include, but are not limited to: home button, volume button, start button and lock
Determine button.
Sensor module 514 includes one or more sensors, and the state for providing various aspects for device 500 is commented
Estimate.For example, sensor module 514 can detecte the state that opens/closes of device 500, and the relative positioning of component, for example, it is described
Component is the display and keypad of device 500, and sensor module 514 can be with 500 1 components of detection device 500 or device
Position change, the existence or non-existence that user contacts with device 500,500 orientation of device or acceleration/deceleration and device 500
Temperature change.Sensor module 514 may include proximity sensor, be configured to detect without any physical contact
Presence of nearby objects.Sensor module 514 can also include optical sensor, such as CMOS or ccd image sensor, at
As being used in application.In some embodiments, which can also include acceleration transducer, gyro sensors
Device, Magnetic Sensor, pressure sensor or temperature sensor.
Communication component 516 is configured to facilitate the communication of wired or wireless way between device 500 and other equipment.Device
500 can access the wireless network based on communication standard, such as Wi-Fi, 2G or 3G or their combination.In an exemplary reality
It applies in example, communication component 516 receives broadcast singal or the related letter of broadcast from external broadcasting management system via broadcast channel
Breath.In one exemplary embodiment, the communication component 516 further includes near-field communication (NFC) module, to promote short distance logical
Letter.For example, radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra wide band (UWB) can be based in NFC module
Technology, bluetooth (BT) technology and other technologies are realized.
In the exemplary embodiment, device 500 can be believed by one or more application specific integrated circuit (ASIC), number
Number processor (DSP), digital signal processing appts (DSPD), programmable logic device (PLD), field programmable gate array
(FPGA), controller, microcontroller, microprocessor or other electronic components are realized, for executing the above method.
In the exemplary embodiment, a kind of non-transitorycomputer readable storage medium including instruction, example are additionally provided
It such as include the memory 504 of instruction, above-metioned instruction can be executed by the processor 520 of device 500 to complete the above method.For example,
The non-transitorycomputer readable storage medium can be ROM, random access memory (RAM), CD-ROM, tape, floppy disk
With optical data storage devices etc..
A kind of non-transitorycomputer readable storage medium, when the instruction in the storage medium is by the processing of device 500
When device executes, so that device 500 is able to carry out the above method.
It should be understood that referenced herein " multiple " refer to two or more."and/or", description association
The incidence relation of object indicates may exist three kinds of relationships, for example, A and/or B, can indicate: individualism A exists simultaneously A
And B, individualism B these three situations.Character "/" typicallys represent the relationship that forward-backward correlation object is a kind of "or".
Those skilled in the art after considering the specification and implementing the invention disclosed here, will readily occur to its of the disclosure
Its embodiment.This application is intended to cover any variations, uses, or adaptations of the disclosure, these modifications, purposes or
Person's adaptive change follows the general principles of this disclosure and including the undocumented common knowledge in the art of the disclosure
Or conventional techniques.The description and examples are only to be considered as illustrative, and the true scope and spirit of the disclosure are by following
Claim is pointed out.
It should be understood that the present disclosure is not limited to the precise structures that have been described above and shown in the drawings, and
And various modifications and changes may be made without departing from the scope thereof.The scope of the present disclosure is only limited by the accompanying claims.
Claims (11)
1. a kind of facial expression recognizing method, which is characterized in that the described method includes:
Human face region is detected from images to be recognized；
Obtain the key point in the human face region；
Obtain the image block around each key point；
Each described image block is overlapped or is spliced by preset order, topography is obtained；
The topography is identified using the Expression Recognition model for completing training, obtains Expression Recognition result.
2. the method according to claim 1, wherein the image block obtained around each key point, comprising:
For each key point, the image block of the predetermined size centered on the key point is intercepted.
3. the method according to claim 1, wherein the Expression Recognition model using completion training is to described
Topography is identified, Expression Recognition result is obtained, comprising:
Using the characteristic information of topography described in the Expression Recognition model extraction for completing training；
Using the Expression Recognition model according to the characteristic information, the Expression Recognition result is determined.
4. the method according to claim 1, wherein the key point obtained in the human face region, comprising:
The human face region is zoomed into target size；
The key point is positioned from the human face region after scaling.
5. method according to any one of claims 1 to 4, which is characterized in that the Expression Recognition model is convolutional Neural
Network model.
6. a kind of facial expression recognition device, which is characterized in that described device includes:
Face detection module is configured as detecting human face region from images to be recognized；
Key point obtains module, is configured as obtaining the key point in the human face region；
Image zooming-out module is configured as obtaining the image block around each key point, and each described image block is suitable by presetting
Sequence is overlapped or splices, and obtains topography；
Expression Recognition module is configured as identifying the topography using the Expression Recognition model for completing training, be obtained
To Expression Recognition result.
7. device according to claim 6, which is characterized in that
Described image block acquisition submodule is configured as intercepting pre- centered on the key point each key point
The image block of scale cun.
8. device according to claim 6, which is characterized in that the Expression Recognition module, comprising:
Feature extraction submodule is configured as the feature letter using topography described in the Expression Recognition model extraction for completing training
Breath；
It identifies and determines submodule, be configured as determining the expression according to the characteristic information using the Expression Recognition model
Recognition result.
9. device according to claim 6, which is characterized in that the key point obtains module, comprising:
Face scales submodule, is configured as the human face region zooming to target size；
Key point positioning submodule is configured as positioning the key point from the human face region after scaling.
10. according to the described in any item devices of claim 6 to 9, which is characterized in that the Expression Recognition model is convolutional Neural
Network model.
11. a kind of facial expression recognition device, which is characterized in that described device includes:
Processor；
For storing the memory of the executable instruction of the processor；
Wherein, the processor is configured to:
Human face region is detected from images to be recognized；
Obtain the key point in the human face region；
Obtain the image block around each key point；
Each described image block is overlapped or is spliced by preset order, topography is obtained；
The topography is identified using the Expression Recognition model for completing training, obtains Expression Recognition result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610653790.2A CN106295566B (en) | 2016-08-10 | 2016-08-10 | Facial expression recognizing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610653790.2A CN106295566B (en) | 2016-08-10 | 2016-08-10 | Facial expression recognizing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106295566A CN106295566A (en) | 2017-01-04 |
| CN106295566B true CN106295566B (en) | 2019-07-09 |
Family
ID=57668257
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610653790.2A Active CN106295566B (en) | 2016-08-10 | 2016-08-10 | Facial expression recognizing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106295566B (en) |
Families Citing this family (36)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11151361B2 (en) | 2017-01-20 | 2021-10-19 | Intel Corporation | Dynamic emotion recognition in unconstrained scenarios |
| CN106778751B (en) * | 2017-02-20 | 2020-08-21 | 迈吉客科技（北京）有限公司 | Non-facial ROI (region of interest) identification method and device |
| CN107369196B (en) * | 2017-06-30 | 2021-08-24 | Oppo广东移动通信有限公司 | Expression package manufacturing method and device, storage medium and electronic equipment |
| CN108304936B (en) * | 2017-07-12 | 2021-11-16 | 腾讯科技（深圳）有限公司 | Machine learning model training method and device, and expression image classification method and device |
| CN107679448B (en) * | 2017-08-17 | 2018-09-25 | 平安科技（深圳）有限公司 | Eyeball action-analysing method, device and storage medium |
| CN107679447A (en) * | 2017-08-17 | 2018-02-09 | 平安科技（深圳）有限公司 | Facial characteristics point detecting method, device and storage medium |
| CN107633204B (en) * | 2017-08-17 | 2019-01-29 | 平安科技（深圳）有限公司 | Face occlusion detection method, apparatus and storage medium |
| CN107633207B (en) * | 2017-08-17 | 2018-10-12 | 平安科技（深圳）有限公司 | AU characteristic recognition methods, device and storage medium |
| CN107679449B (en) * | 2017-08-17 | 2018-08-03 | 平安科技（深圳）有限公司 | Lip motion method for catching, device and storage medium |
| CN107886070A (en) * | 2017-11-10 | 2018-04-06 | 北京小米移动软件有限公司 | Verification method, device and the equipment of facial image |
| CN107832746A (en) * | 2017-12-01 | 2018-03-23 | 北京小米移动软件有限公司 | Expression recognition method and device |
| CN107958230B (en) * | 2017-12-22 | 2020-06-23 | 中国科学院深圳先进技术研究院 | Facial expression recognition method and device |
| CN108073910B (en) | 2017-12-29 | 2021-05-07 | 百度在线网络技术（北京）有限公司 | Method and device for generating human face features |
| CN108197593B (en) * | 2018-01-23 | 2022-02-18 | 深圳极视角科技有限公司 | Multi-size facial expression recognition method and device based on three-point positioning method |
| CN108304793B (en) * | 2018-01-26 | 2021-01-08 | 北京世纪好未来教育科技有限公司 | Online learning analysis system and method |
| CN108304709B (en) * | 2018-01-31 | 2022-01-04 | Oppo广东移动通信有限公司 | Face unlocking method and related product |
| CN108399370A (en) * | 2018-02-02 | 2018-08-14 | 达闼科技（北京）有限公司 | The method and cloud system of Expression Recognition |
| CN108596221B (en) * | 2018-04-10 | 2020-12-01 | 江河瑞通（北京）技术有限公司 | Image recognition method and device for scale reading |
| CN108710829A (en) * | 2018-04-19 | 2018-10-26 | 北京红云智胜科技有限公司 | A method of the expression classification based on deep learning and the detection of micro- expression |
| CN110147805B (en) | 2018-07-23 | 2023-04-07 | 腾讯科技（深圳）有限公司 | Image processing method, device, terminal and storage medium |
| CN109241835A (en) * | 2018-07-27 | 2019-01-18 | 上海商汤智能科技有限公司 | Image processing method and device, electronic equipment and storage medium |
| CN109190487A (en) * | 2018-08-07 | 2019-01-11 | 平安科技（深圳）有限公司 | Face Emotion identification method, apparatus, computer equipment and storage medium |
| CN109145871B (en) * | 2018-09-14 | 2020-09-15 | 广州杰赛科技股份有限公司 | Psychological behavior recognition method, device and storage medium |
| CN109271930B (en) * | 2018-09-14 | 2020-11-13 | 广州杰赛科技股份有限公司 | Micro-expression recognition method, device and storage medium |
| WO2020097795A1 (en) * | 2018-11-13 | 2020-05-22 | 北京比特大陆科技有限公司 | Image processing method, apparatus and device, and storage medium and program product |
| CN109934173B (en) * | 2019-03-14 | 2023-11-21 | 腾讯科技（深圳）有限公司 | Expression recognition method and device and electronic equipment |
| CN109977867A (en) * | 2019-03-26 | 2019-07-05 | 厦门瑞为信息技术有限公司 | A kind of infrared biopsy method based on machine learning multiple features fusion |
| CN110020638B (en) * | 2019-04-17 | 2023-05-12 | 唐晓颖 | Facial expression recognition method, device, equipment and medium |
| CN111008589B (en) * | 2019-12-02 | 2024-04-09 | 杭州网易云音乐科技有限公司 | Face key point detection method, medium, device and computing equipment |
| CN111144348A (en) * | 2019-12-30 | 2020-05-12 | 腾讯科技（深圳）有限公司 | Image processing method, image processing device, electronic equipment and storage medium |
| CN113128309A (en) * | 2020-01-10 | 2021-07-16 | 中移(上海)信息通信科技有限公司 | Facial expression recognition method, device, equipment and medium |
| CN114120386A (en) * | 2020-08-31 | 2022-03-01 | 腾讯科技（深圳）有限公司 | Face recognition method, device, equipment and storage medium |
| CN112580527A (en) * | 2020-12-22 | 2021-03-30 | 之江实验室 | Facial expression recognition method based on convolution long-term and short-term memory network |
| CN112818838A (en) * | 2021-01-29 | 2021-05-18 | 北京嘀嘀无限科技发展有限公司 | Expression recognition method and device and electronic equipment |
| CN113807205A (en) * | 2021-08-30 | 2021-12-17 | 中科尚易健康科技(北京)有限公司 | Locally enhanced human meridian recognition method and device, equipment and storage medium |
| CN117373100B (en) * | 2023-12-08 | 2024-02-23 | 成都乐超人科技有限公司 | Face recognition method and system based on differential quantization local binary pattern |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104346607A (en) * | 2014-11-06 | 2015-02-11 | 上海电机学院 | Face recognition method based on convolutional neural network |
| CN104850825A (en) * | 2015-04-18 | 2015-08-19 | 中国计量学院 | Facial image face score calculating method based on convolutional neural network |
| CN105005774A (en) * | 2015-07-28 | 2015-10-28 | 中国科学院自动化研究所 | Face relative relation recognition method based on convolutional neural network and device thereof |
| CN105469087A (en) * | 2015-07-13 | 2016-04-06 | 百度在线网络技术（北京）有限公司 | Method for identifying clothes image, and labeling method and device of clothes image |
| CN105654049A (en) * | 2015-12-29 | 2016-06-08 | 中国科学院深圳先进技术研究院 | Facial expression recognition method and device |
| CN105825192A (en) * | 2016-03-24 | 2016-08-03 | 深圳大学 | Facial expression identification method and system |
-
2016
- 2016-08-10 CN CN201610653790.2A patent/CN106295566B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104346607A (en) * | 2014-11-06 | 2015-02-11 | 上海电机学院 | Face recognition method based on convolutional neural network |
| CN104850825A (en) * | 2015-04-18 | 2015-08-19 | 中国计量学院 | Facial image face score calculating method based on convolutional neural network |
| CN105469087A (en) * | 2015-07-13 | 2016-04-06 | 百度在线网络技术（北京）有限公司 | Method for identifying clothes image, and labeling method and device of clothes image |
| CN105005774A (en) * | 2015-07-28 | 2015-10-28 | 中国科学院自动化研究所 | Face relative relation recognition method based on convolutional neural network and device thereof |
| CN105654049A (en) * | 2015-12-29 | 2016-06-08 | 中国科学院深圳先进技术研究院 | Facial expression recognition method and device |
| CN105825192A (en) * | 2016-03-24 | 2016-08-03 | 深圳大学 | Facial expression identification method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106295566A (en) | 2017-01-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106295566B (en) | Facial expression recognizing method and device | |
| CN105426867B (en) | Recognition of face verification method and device | |
| CN106339680B (en) | Face key independent positioning method and device | |
| CN105608425B (en) | The method and device of classification storage is carried out to photo | |
| CN105631403B (en) | Face identification method and device | |
| CN105809704B (en) | Identify the method and device of image definition | |
| CN106295511B (en) | Face tracking method and device | |
| CN106295515B (en) | Determine the method and device of the human face region in image | |
| CN109190449A (en) | Age recognition methods, device, electronic equipment and storage medium | |
| CN106548145A (en) | Image-recognizing method and device | |
| US9536132B2 (en) | Facilitating image capture and image review by visually impaired users | |
| CN105528078B (en) | The method and device of controlling electronic devices | |
| CN109063580A (en) | Face identification method, device, electronic equipment and storage medium | |
| CN106980840A (en) | Shape of face matching process, device and storage medium | |
| CN107368810A (en) | Method for detecting human face and device | |
| CN107463903B (en) | Face key point positioning method and device | |
| CN107886070A (en) | Verification method, device and the equipment of facial image | |
| CN106971164A (en) | Shape of face matching process and device | |
| CN105354560A (en) | Fingerprint identification method and device | |
| CN105631406A (en) | Method and device for recognizing and processing image | |
| CN108197585A (en) | Recognition algorithms and device | |
| CN111488774A (en) | Image processing method and device for image processing | |
| CN107992811A (en) | Face identification method and device | |
| CN109034150A (en) | Image processing method and device | |
| CN107977636B (en) | Face detection method and device, terminal and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |