CN106295566B - Facial expression recognizing method and device - Google Patents

Facial expression recognizing method and device Download PDF

Info

Publication number
CN106295566B
CN106295566B CN201610653790.2A CN201610653790A CN106295566B CN 106295566 B CN106295566 B CN 106295566B CN 201610653790 A CN201610653790 A CN 201610653790A CN 106295566 B CN106295566 B CN 106295566B
Authority
CN
China
Prior art keywords
key point
expression recognition
human face
face region
topography
Prior art date
Application number
CN201610653790.2A
Other languages
Chinese (zh)
Other versions
CN106295566A (en
Inventor
杨松
张旭华
王百超
Original Assignee
北京小米移动软件有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京小米移动软件有限公司 filed Critical 北京小米移动软件有限公司
Priority to CN201610653790.2A priority Critical patent/CN106295566B/en
Publication of CN106295566A publication Critical patent/CN106295566A/en
Application granted granted Critical
Publication of CN106295566B publication Critical patent/CN106295566B/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/00221Acquiring or recognising human faces, facial parts, facial sketches, facial expressions
    • G06K9/00302Facial expression recognition

Abstract

The disclosure is directed to a kind of facial expression recognizing method and devices, belong to image identification technical field.The described method includes: detecting human face region from images to be recognized;Obtain the key point in human face region;Topography is extracted from human face region according to key point;Topography is identified using the Expression Recognition model for completing training, obtains Expression Recognition result.The disclosure carries out feature extraction and expression differentiation at face key point by using the Expression Recognition model for completing training, and two sseparated steps are combined into one, to reduce accumulated error, improve the accuracy of facial expression recognition.Also, due to the characteristic information for only extracting the topography at face key point, rather than extract the global characteristics information of entire human face region, can it is more accurate, efficiently extract the feature for embodying emotional state, further increase the accuracy of facial expression recognition.

Description

Facial expression recognizing method and device

Technical field

This disclosure relates to image identification technical field, in particular to a kind of facial expression recognizing method and device.

Background technique

Facial expression recognition refers to the emotional state for identifying from given facial image and determining face.For example, glad, sad Wound, surprised, frightened, detest, anger etc..Facial expression recognition is widely used to psychic science, neuroscience, work at present The fields such as Cheng Kexue and computer science.

In the related art, facial expression recognition includes following two key steps: first, it is detected from images to be recognized Human face region, and countenance feature is extracted from human face region, wherein HOG (Histogram of Oriented can be used Gradient, histograms of oriented gradients), LBP (Local Binary Pattern, local binary patterns), the features such as Gabor mention Algorithm is taken to extract countenance feature;Second, based on countenance feature carry out expression classification, obtain Expression Recognition as a result, its In, Adaboost algorithm, SVM (Support Vector Machine, support vector machines) algorithm, random can be used in sorting algorithm Forest algorithm etc..

Summary of the invention

The embodiment of the present disclosure provides a kind of facial expression recognizing method and device.The technical solution is as follows:

According to the first aspect of the embodiments of the present disclosure, a kind of facial expression recognizing method is provided, which comprises

Human face region is detected from images to be recognized;

Obtain the key point in the human face region;

Topography is extracted from the human face region according to the key point;

The topography is identified using the Expression Recognition model for completing training, obtains Expression Recognition result.

It is optionally, described that topography is extracted from the human face region according to the key point, comprising:

Obtain the image block around each key point;

Each described image block is overlapped or is spliced by preset order, the topography is obtained.

Optionally, the image block obtained around each key point, comprising:

For each key point, the image block of the predetermined size centered on the key point is intercepted.

Optionally, described that the topography is identified using the Expression Recognition model for completing training, obtain expression Recognition result, comprising:

Using the characteristic information of topography described in the Expression Recognition model extraction for completing training;

Using the Expression Recognition model according to the characteristic information, the Expression Recognition result is determined.

Optionally, the key point obtained in the human face region, comprising:

The human face region is zoomed into target size;

The key point is positioned from the human face region after scaling.

Optionally, the Expression Recognition model is convolutional neural networks model.

According to the second aspect of an embodiment of the present disclosure, a kind of facial expression recognition device is provided, described device includes:

Face detection module is configured as detecting human face region from images to be recognized;

Key point obtains module, is configured as obtaining the key point in the human face region;

Image zooming-out module is configured as extracting topography from the human face region according to the key point;

Expression Recognition module is configured as knowing the topography using the Expression Recognition model for completing training Not, Expression Recognition result is obtained.

Optionally, described image extraction module, comprising:

Image block acquisition submodule is configured as obtaining the image block around each key point;

Image block handles submodule, is configured as that each described image block is overlapped or is spliced by preset order, obtains To the topography.

Optionally, described image block acquisition submodule is configured as intercepting with the key point each key point Centered on predetermined size image block.

Optionally, the Expression Recognition module, comprising:

Feature extraction submodule is configured as the spy using topography described in the Expression Recognition model extraction for completing training Reference breath;

Identify and determine submodule, be configured as using the Expression Recognition model according to the characteristic information, determine described in Expression Recognition result.

Optionally, the key point obtains module, comprising:

Face scales submodule, is configured as the human face region zooming to target size;

Key point positioning submodule is configured as positioning the key point from the human face region after scaling.

Optionally, the Expression Recognition model is convolutional neural networks model.

According to the third aspect of an embodiment of the present disclosure, a kind of facial expression recognition device is provided, described device includes:

Processor;

For storing the memory of the executable instruction of the processor;

Wherein, the processor is configured to:

Human face region is detected from images to be recognized;

Obtain the key point in the human face region;

Topography is extracted from the human face region according to the key point;

The topography is identified using the Expression Recognition model for completing training, obtains Expression Recognition result.

The technical solution that the embodiment of the present disclosure provides can include the following benefits:

By detecting human face region from image to be identified, obtain the key point in human face region, according to key point from Topography is extracted in human face region, and topography is identified using the Expression Recognition model for completing training, obtains table Feelings recognition result;It solves in the related technology since feature extraction and expression differentiation are two sseparated steps, needs to adopt respectively It is handled with two different algorithms, leads to the problem of influencing the precision of facial expression recognition there are accumulated error;It has used Feature extraction and expression differentiation at face key point are carried out at trained Expression Recognition model, two sseparated steps are closed two It is one, to reduce accumulated error, improves the accuracy of facial expression recognition.Also, due to only extracting at face key point The characteristic information of topography, rather than extract the global characteristics information of entire human face region, can it is more accurate, efficiently extract The feature for embodying emotional state, further increases the accuracy of facial expression recognition.

It should be understood that above general description and following detailed description be only it is exemplary and explanatory, not The disclosure can be limited.

Detailed description of the invention

The drawings herein are incorporated into the specification and forms part of this specification, and shows the implementation for meeting the disclosure Example, and together with specification for explaining the principles of this disclosure.

Fig. 1 is a kind of flow chart of facial expression recognizing method shown according to an exemplary embodiment;

Fig. 2A is a kind of flow chart of the facial expression recognizing method shown according to another exemplary embodiment;

Fig. 2 B is the key that Fig. 2A illustrated embodiment is related to the schematic diagram of point location;

Fig. 2 C is a kind of structural schematic diagram of the convolutional neural networks illustrated;

Fig. 3 is a kind of block diagram of facial expression recognition device shown according to an exemplary embodiment;

Fig. 4 is a kind of block diagram of the facial expression recognition device shown according to another exemplary embodiment;

Fig. 5 is a kind of block diagram of device shown according to an exemplary embodiment.

Specific embodiment

Example embodiments are described in detail here, and the example is illustrated in the accompanying drawings.Following description is related to When attached drawing, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements.Following exemplary embodiment Described in embodiment do not represent all implementations consistent with this disclosure.On the contrary, they be only with it is such as appended The example of the consistent device and method of some aspects be described in detail in claims, the disclosure.

In the related art, feature extraction and expression differentiation are two sseparated steps, need to be respectively adopted two kinds of differences Algorithm handled, there are accumulated errors, influence the precision of facial expression recognition.The technical side that the embodiment of the present disclosure provides Case carries out feature extraction and expression differentiation at face key point using the Expression Recognition model for completing training, two is separated The step of be combined into one, to reduce accumulated error, improve the accuracy of facial expression recognition.Also, due to only extracting face The characteristic information of topography at key point, rather than extract the global characteristics information of entire human face region, can it is more accurate, The feature for embodying emotional state is efficiently extracted, the accuracy of facial expression recognition is further increased.

The method that the embodiment of the present disclosure provides, the executing subject of each step can be the electronics with image-capable and set It is standby, such as PC, smart phone, tablet computer, server etc..For ease of description, in following methods embodiment, with The executing subject of each step is illustrated for electronic equipment.

Fig. 1 is a kind of flow chart of facial expression recognizing method shown according to an exemplary embodiment.This method can be with It comprises the following steps:

In a step 101, human face region is detected from images to be recognized.

In a step 102, the key point in human face region is obtained.

Key point is also referred to as characteristic point, face key point or human face characteristic point, and expression can be embodied by referring in human face region The face location of state, including but not limited to eyes (such as canthus, eyeball center, eye tail), nose (such as nose, the wing of nose), mouth (such as corners of the mouth, labial angle, lip), chin, eyebrow angle face location.

In step 103, topography is extracted from human face region according to key point.

At step 104, topography is identified using the Expression Recognition model for completing training, obtains Expression Recognition As a result.

In conclusion method provided in this embodiment, solves in the related technology since feature extraction and expression differentiation are Two sseparated steps need to be respectively adopted two different algorithms and are handled, and cause to influence face table there are accumulated error The problem of precision of feelings identification.The feature extraction and expression at face key point are carried out using the Expression Recognition model for completing training Differentiate, two sseparated steps are combined into one, to reduce accumulated error, improve the accuracy of facial expression recognition.Also, Due to only extracting the characteristic information of the topography at face key point, rather than extract the global characteristics letter of entire human face region Breath, can it is more accurate, efficiently extract embody emotional state feature, further increase the accuracy of facial expression recognition.

Fig. 2A is a kind of flow chart of the facial expression recognizing method shown according to another exemplary embodiment.This method can To comprise the following steps:

In step 201, human face region is detected from images to be recognized.

Electronic equipment detects human face region using relevant Face datection algorithm from images to be recognized.In the present embodiment In, the specific type of Face datection algorithm is not construed as limiting.For example, Face datection algorithm can be LBP algorithm and Adaboost The combination of algorithm extracts characteristics of image using LBP algorithm from images to be recognized, and using Adaboost cascade classifier according to Characteristics of image determines human face region.Images to be recognized can be include face image.

In step 202, human face region is zoomed into target size.

Since the human face region size in different images is different, in order to ensure the accuracy of subsequent characteristics point location, will mention The human face region taken zooms to fixed target size.In the present embodiment, the size of target size is not construed as limiting, it can root It is preset according to actual conditions.For example, the target size is 96 × 96 (pixels).

In step 203, key point is positioned from the human face region after scaling.

Key point is also referred to as characteristic point, face key point or human face characteristic point, and expression can be embodied by referring in human face region The face location of state, including but not limited to eyes (such as canthus, eyeball center, eye tail), nose (such as nose, the wing of nose), mouth (such as corners of the mouth, labial angle, lip), chin, eyebrow angle face location.

Electronic equipment positions key point from the human face region after scaling using relevant face key point location algorithm.? In the present embodiment, the specific type of face key point location algorithm is not construed as limiting.For example, face key point location algorithm can be with It is SDM (Supervised Descent Method supervises descent method) algorithm.

In one example, as shown in Figure 2 B, it is multiple that acquisition is positioned from the human face region 21 after scaling using SDM algorithm Key point, each key point is as shown in pore in figure.The position of the key point of required positioning and quantity can be preset, example Such as 20.

In step 204, the image block around each key point is obtained.

Image block around key point refers to the image block of the predetermined size including the key point.In an example In, for each key point, intercept the image block of the predetermined size centered on key point.In the present embodiment, to predetermined The size of size is not construed as limiting, and can be preset according to the actual situation.For example, the predetermined size is 32 × 32 (pixels).

Optionally, before carrying out crucial point location, human face region can be converted to gray level image, it is fixed in gray level image Position key point.Correspondingly, the image block around the key point of interception is also gray level image.

In step 205, each image block is overlapped or is spliced by preset order, obtain topography.

It before carrying out Expression Recognition, needs to integrate each image block, is input to expression knowledge as a whole Other model.In a kind of possible embodiment, Expression Recognition mould is used as after each image block is overlapped by preset order The input of type.In alternatively possible embodiment, know after each image block is spliced by preset order as expression The input of other model.

Since the usually multiple and different key point of the key point of acquisition corresponds to different face locations, thus it is each Image block needs are overlapped or splice according to preset order.It include that eyes, nose, mouth, eyebrow angle, chin are with key point Example, preset order can be followed successively by eyes, nose, mouth, chin, eyebrow angle.That is, whether in the training of Expression Recognition model Stage, or the Expression Recognition stage is being carried out using Expression Recognition model, the multiple images block intercepted from any image is equal It is overlapped or splices according to same preset order, mutually unified with the structure of this input data for guaranteeing Expression Recognition model.

In step 206, topography is identified using the Expression Recognition model for completing training, obtains Expression Recognition As a result.

Electronic equipment is using the characteristic information for completing trained Expression Recognition model extraction topography, using Expression Recognition Model determines Expression Recognition result according to characteristic information.In the present embodiment, feature extraction and expression differentiate by Expression Recognition Model is completed, and without being completed in two steps by two different algorithms, two sseparated steps is combined into one, to reduce Accumulated error improves the accuracy of facial expression recognition.

In one example, Expression Recognition model be convolutional neural networks (Convolutional Neural Network, CNN) model, or be depth convolutional neural networks model.Convolutional neural networks have powerful ability in feature extraction, utilize volume Product neural network carries out feature extraction and expression differentiates, accuracy is higher.Convolutional neural networks include an input layer, at least one A convolutional layer, at least one full articulamentum and an output layer.Wherein, the input data of input layer is and presses each image block The topography obtained after sequence superposition or splicing;The output of output layer respectively indicates n kind expression the result is that length is the vector of n Probability, n is integer greater than 1.Convolutional layer is used for feature extraction.Full articulamentum is for carrying out group to the feature that convolutional layer extracts It closes and abstract, obtains being suitable for the data that output layer is classified.

In conjunction with reference Fig. 2 C, a kind of structural schematic diagram of convolutional neural networks is illustrated.The convolutional Neural net Network includes 1 input layer, 3 convolutional layers (the first convolutional layer C1, the second convolutional layer C2 and third convolutional layer C3), 2 full connections Layer (the first complete full articulamentum FC5 of articulamentum FC4 and second) and 1 output layer (Softmax layers).Assuming that being mentioned from human face region 20 key points are taken, the size of the image block around each key point intercepted is 32 × 32 (pixels), the input of input layer Data are 20 × 32 × 32 superimposed image or stitching image.Wherein the number of convolution kernel is divided in 3 convolutional layers C1, C2 and C3 It Wei 36,64 and 32.The step-length of first convolutional layer C1 is 2, and the length of image and width are all contracted to after the first convolutional layer C1 calculating The step-length of half originally, the second convolutional layer C2 and third convolutional layer C3 are 1.It should be noted that volume shown in fig. 2 C Product neural network be only it is exemplary and explanatory, be not used to limit the disclosure.In general, the number of plies of convolutional neural networks More, effect is better but the calculating time also can be longer, in practical applications, in combination with the requirement to accuracy of identification and efficiency, if Count the convolutional neural networks of the appropriate number of plies.

In conclusion method provided in this embodiment, carries out face key point using the Expression Recognition model for completing training The feature extraction at place and expression differentiate, two sseparated steps are combined into one, to reduce accumulated error, improve human face expression The accuracy of identification.Also, due to the characteristic information for only extracting the topography at face key point, rather than extract entire face The global characteristics information in region, can it is more accurate, efficiently extract embody emotional state feature, further increase face table The accuracy of feelings identification.

Following is embodiment of the present disclosure, can be used for executing embodiments of the present disclosure.It is real for disclosure device Undisclosed details in example is applied, embodiments of the present disclosure is please referred to.

Fig. 3 is a kind of block diagram of facial expression recognition device shown according to an exemplary embodiment.The device has real The existing exemplary function of the above method, the function it is real can also to execute corresponding software by hardware realization by hardware It is existing.The apparatus may include: face detection module 310, key point obtain module 320, image zooming-out module 330 and Expression Recognition Module 340.

Face detection module 310 is configured as detecting human face region from images to be recognized.

Key point obtains module 320, is configured as obtaining the key point in the human face region.

Image zooming-out module 330 is configured as extracting topography from the human face region according to the key point.

Expression Recognition module 340 is configured as carrying out the topography using the Expression Recognition model for completing training Identification, obtains Expression Recognition result.

In conclusion device provided in this embodiment, carries out face key point using the Expression Recognition model for completing training The feature extraction at place and expression differentiate, two sseparated steps are combined into one, to reduce accumulated error, improve human face expression The accuracy of identification.Also, due to the characteristic information for only extracting the topography at face key point, rather than extract entire face The global characteristics information in region, can it is more accurate, efficiently extract embody emotional state feature, further increase face table The accuracy of feelings identification.

Fig. 4 is a kind of block diagram of the facial expression recognition device shown according to another exemplary embodiment.The device has Realize that the exemplary function of the above method, the function can also execute corresponding software by hardware realization by hardware It realizes.The apparatus may include: face detection module 310, key point obtain module 320, image zooming-out module 330 and expression and know Other module 340.

Face detection module 310 is configured as detecting human face region from images to be recognized.

Key point obtains module 320, is configured as obtaining the key point in the human face region.

Image zooming-out module 330 is configured as extracting topography from the human face region according to the key point.

Expression Recognition module 340 is configured as carrying out the topography using the Expression Recognition model for completing training Identification, obtains Expression Recognition result.

In one example, described image extraction module 330, comprising: at image block acquisition submodule 330a and image block Manage submodule 330b.

Image block acquisition submodule 330a is configured as obtaining the image block around each key point.

Image block handles submodule 330b, is configured as that each described image block is overlapped or is spelled by preset order It connects, obtains the topography.

In one example, described image block acquisition submodule 330b is configured as each key point, interception with The image block of predetermined size centered on the key point.

In one example, the Expression Recognition module 340, comprising: feature extraction submodule 340a and identification determine son Module 340b.

Feature extraction submodule 340a is configured as using topography described in the Expression Recognition model extraction for completing training Characteristic information.

It identifies and determines submodule 340b, be configured as being determined using the Expression Recognition model according to the characteristic information The Expression Recognition result.

In one example, the key point obtains module 320, comprising: face scales submodule 320a and key point is fixed Bit submodule 320b.

Face scales submodule 320a, is configured as the human face region zooming to target size.

Key point positioning submodule 320b is configured as positioning the key point from the human face region after scaling.

In one example, the Expression Recognition model is convolutional neural networks model.

In conclusion device provided in this embodiment, carries out face key point using the Expression Recognition model for completing training The feature extraction at place and expression differentiate, two sseparated steps are combined into one, to reduce accumulated error, improve human face expression The accuracy of identification.Also, due to the characteristic information for only extracting the topography at face key point, rather than extract entire face The global characteristics information in region, can it is more accurate, efficiently extract embody emotional state feature, further increase face table The accuracy of feelings identification.

It should be noted is that device provided by the above embodiment is when realizing its function, only with above-mentioned each function The division progress of module, can be according to actual needs and by above-mentioned function distribution by different function for example, in practical application Energy module is completed, i.e., the content structure of equipment is divided into different functional modules, to complete whole described above or portion Divide function.

About the device in above-described embodiment, wherein modules execute the concrete mode of operation in related this method Embodiment in be described in detail, no detailed explanation will be given here.

One exemplary embodiment of the disclosure additionally provides a kind of facial expression recognition device, can be realized disclosure offer Facial expression recognizing method.The device includes: processor, and the memory of the executable instruction for storage processor.Its In, processor is configured as:

Human face region is detected from images to be recognized;

Obtain the key point in the human face region;

Topography is extracted from the human face region according to the key point;

The topography is identified using the Expression Recognition model for completing training, obtains Expression Recognition result.

Optionally, the processor is configured to:

Obtain the image block around each key point;

Each described image block is overlapped or is spliced by preset order, the topography is obtained.

Optionally, the processor is configured to:

For each key point, the image block of the predetermined size centered on the key point is intercepted.

Optionally, the processor is configured to:

Using the characteristic information of topography described in the Expression Recognition model extraction for completing training;

Using the Expression Recognition model according to the characteristic information, the Expression Recognition result is determined.

Optionally, the processor is configured to:

The human face region is zoomed into target size;

The key point is positioned from the human face region after scaling.

Optionally, the Expression Recognition model is convolutional neural networks model.

Fig. 5 is a kind of block diagram of device 500 shown according to an exemplary embodiment.For example, device 500 can be movement Phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, Medical Devices, body-building equipment, Personal digital assistant etc..

Referring to Fig. 5, device 500 may include following one or more components: processing component 502, memory 504, power supply Component 506, multimedia component 508, audio component 510, input/output (I/O) interface 512, sensor module 514, Yi Jitong Believe component 516.

The integrated operation of the usual control device 500 of processing component 502, such as with display, telephone call, data communication, phase Machine operation and record operate associated operation.Processing component 502 may include that one or more processors 520 refer to execute It enables, to perform all or part of the steps of the methods described above.In addition, processing component 502 may include one or more modules, just Interaction between processing component 502 and other assemblies.For example, processing component 502 may include multi-media module, it is more to facilitate Interaction between media component 508 and processing component 502.

Memory 504 is configured as storing various types of data to support the operation in device 500.These data are shown Example includes the instruction of any application or method for operating on device 500, contact data, and telephone book data disappears Breath, picture, video etc..Memory 504 can be by any kind of volatibility or non-volatile memory device or their group It closes and realizes, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM) is erasable to compile Journey read-only memory (EPROM), programmable read only memory (PROM), read-only memory (ROM), magnetic memory, flash Device, disk or CD.

Power supply module 506 provides electric power for the various assemblies of device 500.Power supply module 506 may include power management system System, one or more power supplys and other with for device 500 generate, manage, and distribute the associated component of electric power.

Multimedia component 508 includes the screen of one output interface of offer between described device 500 and user.One In a little embodiments, screen may include liquid crystal display (LCD) and touch panel (TP).If screen includes touch panel, screen Curtain may be implemented as touch screen, to receive input signal from the user.Touch panel includes one or more touch sensings Device is to sense the gesture on touch, slide, and touch panel.The touch sensor can not only sense touch or sliding action Boundary, but also detect duration and pressure associated with the touch or slide operation.In some embodiments, more matchmakers Body component 508 includes a front camera and/or rear camera.When device 500 is in operation mode, such as screening-mode or When video mode, front camera and/or rear camera can receive external multi-medium data.Each front camera and Rear camera can be a fixed optical lens system or have focusing and optical zoom capabilities.

Audio component 510 is configured as output and/or input audio signal.For example, audio component 510 includes a Mike Wind (MIC), when device 500 is in operation mode, when such as call mode, recording mode, and voice recognition mode, microphone is matched It is set to reception external audio signal.The received audio signal can be further stored in memory 504 or via communication set Part 516 is sent.In some embodiments, audio component 510 further includes a loudspeaker, is used for output audio signal.

I/O interface 512 provides interface between processing component 502 and peripheral interface module, and above-mentioned peripheral interface module can To be keyboard, click wheel, button etc..These buttons may include, but are not limited to: home button, volume button, start button and lock Determine button.

Sensor module 514 includes one or more sensors, and the state for providing various aspects for device 500 is commented Estimate.For example, sensor module 514 can detecte the state that opens/closes of device 500, and the relative positioning of component, for example, it is described Component is the display and keypad of device 500, and sensor module 514 can be with 500 1 components of detection device 500 or device Position change, the existence or non-existence that user contacts with device 500,500 orientation of device or acceleration/deceleration and device 500 Temperature change.Sensor module 514 may include proximity sensor, be configured to detect without any physical contact Presence of nearby objects.Sensor module 514 can also include optical sensor, such as CMOS or ccd image sensor, at As being used in application.In some embodiments, which can also include acceleration transducer, gyro sensors Device, Magnetic Sensor, pressure sensor or temperature sensor.

Communication component 516 is configured to facilitate the communication of wired or wireless way between device 500 and other equipment.Device 500 can access the wireless network based on communication standard, such as Wi-Fi, 2G or 3G or their combination.In an exemplary reality It applies in example, communication component 516 receives broadcast singal or the related letter of broadcast from external broadcasting management system via broadcast channel Breath.In one exemplary embodiment, the communication component 516 further includes near-field communication (NFC) module, to promote short distance logical Letter.For example, radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra wide band (UWB) can be based in NFC module Technology, bluetooth (BT) technology and other technologies are realized.

In the exemplary embodiment, device 500 can be believed by one or more application specific integrated circuit (ASIC), number Number processor (DSP), digital signal processing appts (DSPD), programmable logic device (PLD), field programmable gate array (FPGA), controller, microcontroller, microprocessor or other electronic components are realized, for executing the above method.

In the exemplary embodiment, a kind of non-transitorycomputer readable storage medium including instruction, example are additionally provided It such as include the memory 504 of instruction, above-metioned instruction can be executed by the processor 520 of device 500 to complete the above method.For example, The non-transitorycomputer readable storage medium can be ROM, random access memory (RAM), CD-ROM, tape, floppy disk With optical data storage devices etc..

A kind of non-transitorycomputer readable storage medium, when the instruction in the storage medium is by the processing of device 500 When device executes, so that device 500 is able to carry out the above method.

It should be understood that referenced herein " multiple " refer to two or more."and/or", description association The incidence relation of object indicates may exist three kinds of relationships, for example, A and/or B, can indicate: individualism A exists simultaneously A And B, individualism B these three situations.Character "/" typicallys represent the relationship that forward-backward correlation object is a kind of "or".

Those skilled in the art after considering the specification and implementing the invention disclosed here, will readily occur to its of the disclosure Its embodiment.This application is intended to cover any variations, uses, or adaptations of the disclosure, these modifications, purposes or Person's adaptive change follows the general principles of this disclosure and including the undocumented common knowledge in the art of the disclosure Or conventional techniques.The description and examples are only to be considered as illustrative, and the true scope and spirit of the disclosure are by following Claim is pointed out.

It should be understood that the present disclosure is not limited to the precise structures that have been described above and shown in the drawings, and And various modifications and changes may be made without departing from the scope thereof.The scope of the present disclosure is only limited by the accompanying claims.

Claims (11)

1. a kind of facial expression recognizing method, which is characterized in that the described method includes:
Human face region is detected from images to be recognized;
Obtain the key point in the human face region;
Obtain the image block around each key point;
Each described image block is overlapped or is spliced by preset order, topography is obtained;
The topography is identified using the Expression Recognition model for completing training, obtains Expression Recognition result.
2. the method according to claim 1, wherein the image block obtained around each key point, comprising:
For each key point, the image block of the predetermined size centered on the key point is intercepted.
3. the method according to claim 1, wherein the Expression Recognition model using completion training is to described Topography is identified, Expression Recognition result is obtained, comprising:
Using the characteristic information of topography described in the Expression Recognition model extraction for completing training;
Using the Expression Recognition model according to the characteristic information, the Expression Recognition result is determined.
4. the method according to claim 1, wherein the key point obtained in the human face region, comprising:
The human face region is zoomed into target size;
The key point is positioned from the human face region after scaling.
5. method according to any one of claims 1 to 4, which is characterized in that the Expression Recognition model is convolutional Neural Network model.
6. a kind of facial expression recognition device, which is characterized in that described device includes:
Face detection module is configured as detecting human face region from images to be recognized;
Key point obtains module, is configured as obtaining the key point in the human face region;
Image zooming-out module is configured as obtaining the image block around each key point, and each described image block is suitable by presetting Sequence is overlapped or splices, and obtains topography;
Expression Recognition module is configured as identifying the topography using the Expression Recognition model for completing training, be obtained To Expression Recognition result.
7. device according to claim 6, which is characterized in that
Described image block acquisition submodule is configured as intercepting pre- centered on the key point each key point The image block of scale cun.
8. device according to claim 6, which is characterized in that the Expression Recognition module, comprising:
Feature extraction submodule is configured as the feature letter using topography described in the Expression Recognition model extraction for completing training Breath;
It identifies and determines submodule, be configured as determining the expression according to the characteristic information using the Expression Recognition model Recognition result.
9. device according to claim 6, which is characterized in that the key point obtains module, comprising:
Face scales submodule, is configured as the human face region zooming to target size;
Key point positioning submodule is configured as positioning the key point from the human face region after scaling.
10. according to the described in any item devices of claim 6 to 9, which is characterized in that the Expression Recognition model is convolutional Neural Network model.
11. a kind of facial expression recognition device, which is characterized in that described device includes:
Processor;
For storing the memory of the executable instruction of the processor;
Wherein, the processor is configured to:
Human face region is detected from images to be recognized;
Obtain the key point in the human face region;
Obtain the image block around each key point;
Each described image block is overlapped or is spliced by preset order, topography is obtained;
The topography is identified using the Expression Recognition model for completing training, obtains Expression Recognition result.
CN201610653790.2A 2016-08-10 2016-08-10 Facial expression recognizing method and device CN106295566B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610653790.2A CN106295566B (en) 2016-08-10 2016-08-10 Facial expression recognizing method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610653790.2A CN106295566B (en) 2016-08-10 2016-08-10 Facial expression recognizing method and device

Publications (2)

Publication Number Publication Date
CN106295566A CN106295566A (en) 2017-01-04
CN106295566B true CN106295566B (en) 2019-07-09

Family

ID=57668257

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610653790.2A CN106295566B (en) 2016-08-10 2016-08-10 Facial expression recognizing method and device

Country Status (1)

Country Link
CN (1) CN106295566B (en)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190325203A1 (en) * 2017-01-20 2019-10-24 Intel Corporation Dynamic emotion recognition in unconstrained scenarios
CN106778751A (en) * 2017-02-20 2017-05-31 迈吉客科技(北京)有限公司 A kind of non-face ROI recognition methods and device
CN107633207B (en) * 2017-08-17 2018-10-12 平安科技(深圳)有限公司 AU characteristic recognition methods, device and storage medium
CN107633204B (en) * 2017-08-17 2019-01-29 平安科技(深圳)有限公司 Face occlusion detection method, apparatus and storage medium
CN107679447A (en) * 2017-08-17 2018-02-09 平安科技(深圳)有限公司 Facial characteristics point detecting method, device and storage medium
CN107679448B (en) * 2017-08-17 2018-09-25 平安科技(深圳)有限公司 Eyeball action-analysing method, device and storage medium
CN107679449B (en) 2017-08-17 2018-08-03 平安科技(深圳)有限公司 Lip motion method for catching, device and storage medium
CN107958230A (en) * 2017-12-22 2018-04-24 中国科学院深圳先进技术研究院 Facial expression recognizing method and device
CN108073910A (en) * 2017-12-29 2018-05-25 百度在线网络技术(北京)有限公司 For generating the method and apparatus of face characteristic
CN108304709A (en) * 2018-01-31 2018-07-20 广东欧珀移动通信有限公司 Face unlocking method and related product
CN108399370A (en) * 2018-02-02 2018-08-14 达闼科技(北京)有限公司 The method and cloud system of Expression Recognition
CN108596221A (en) * 2018-04-10 2018-09-28 江河瑞通(北京)技术有限公司 The image-recognizing method and equipment of rod reading

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104346607A (en) * 2014-11-06 2015-02-11 上海电机学院 Face recognition method based on convolutional neural network
CN104850825A (en) * 2015-04-18 2015-08-19 中国计量学院 Facial image face score calculating method based on convolutional neural network
CN105005774A (en) * 2015-07-28 2015-10-28 中国科学院自动化研究所 Face relative relation recognition method based on convolutional neural network and device thereof
CN105469087A (en) * 2015-07-13 2016-04-06 百度在线网络技术(北京)有限公司 Method for identifying clothes image, and labeling method and device of clothes image
CN105654049A (en) * 2015-12-29 2016-06-08 中国科学院深圳先进技术研究院 Facial expression recognition method and device
CN105825192A (en) * 2016-03-24 2016-08-03 深圳大学 Facial expression identification method and system

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104346607A (en) * 2014-11-06 2015-02-11 上海电机学院 Face recognition method based on convolutional neural network
CN104850825A (en) * 2015-04-18 2015-08-19 中国计量学院 Facial image face score calculating method based on convolutional neural network
CN105469087A (en) * 2015-07-13 2016-04-06 百度在线网络技术(北京)有限公司 Method for identifying clothes image, and labeling method and device of clothes image
CN105005774A (en) * 2015-07-28 2015-10-28 中国科学院自动化研究所 Face relative relation recognition method based on convolutional neural network and device thereof
CN105654049A (en) * 2015-12-29 2016-06-08 中国科学院深圳先进技术研究院 Facial expression recognition method and device
CN105825192A (en) * 2016-03-24 2016-08-03 深圳大学 Facial expression identification method and system

Also Published As

Publication number Publication date
CN106295566A (en) 2017-01-04

Similar Documents

Publication Publication Date Title
Shan et al. 3D human action segmentation and recognition using pose kinetic energy
CN101236600A (en) The image processing apparatus and an image processing method
RU2596580C2 (en) Method and device for image segmentation
CN104125396B (en) Image capturing method and device
WO2017181769A1 (en) Facial recognition method, apparatus and system, device, and storage medium
JP2003216955A (en) Method and device for gesture recognition, dialogue device, and recording medium with gesture recognition program recorded thereon
CN104615769B (en) Picture classification method and device
US20120304067A1 (en) Apparatus and method for controlling user interface using sound recognition
CN103546627A (en) Mobile terminal and control method thereof
CN104243819B (en) Photo acquisition methods and device
Kang et al. Development of head detection and tracking systems for visual surveillance
CN104408426B (en) Facial image glasses minimizing technology and device
CN105430262B (en) Filming control method and device
CN105426857B (en) Human face recognition model training method and device
CN105512605B (en) Face image processing process and device
WO2017045258A1 (en) Photographing prompting method, device and apparatus, and nonvolatile computer storage medium
US8879803B2 (en) Method, apparatus, and computer program product for image clustering
CN104753766B (en) Expression sending method and device
CN104461014A (en) Screen unlocking method and device
KR101185243B1 (en) Apparatus and method for registering a plurlity of face image for face recognition
CN106572299A (en) Camera switching-on method and device
CN104850828B (en) Character recognition method and device
CN106651955B (en) Method and device for positioning target object in picture
CN105354543A (en) Video processing method and apparatus
CN102054165A (en) Image processing apparatus and image processing method

Legal Events

Date Code Title Description
PB01 Publication
C06 Publication
SE01 Entry into force of request for substantive examination
C10 Entry into substantive examination
GR01 Patent grant
GR01 Patent grant