CN109815793A - Micro- expression describes method, apparatus, computer installation and readable storage medium storing program for executing - Google Patents
Micro- expression describes method, apparatus, computer installation and readable storage medium storing program for executing Download PDFInfo
- Publication number
- CN109815793A CN109815793A CN201811527706.8A CN201811527706A CN109815793A CN 109815793 A CN109815793 A CN 109815793A CN 201811527706 A CN201811527706 A CN 201811527706A CN 109815793 A CN109815793 A CN 109815793A
- Authority
- CN
- China
- Prior art keywords
- expression
- micro
- facial image
- feature vector
- fisrt feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000014509 gene expression Effects 0.000 title claims abstract description 114
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000009434 installation Methods 0.000 title claims abstract description 29
- 230000001815 facial effect Effects 0.000 claims abstract description 115
- 239000000284 extract Substances 0.000 claims abstract description 12
- 239000000203 mixture Substances 0.000 claims abstract description 7
- 239000011159 matrix material Substances 0.000 claims description 54
- 238000004590 computer program Methods 0.000 claims description 20
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 8
- 230000009466 transformation Effects 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 7
- 210000001097 facial muscle Anatomy 0.000 claims description 4
- 238000012512 characterization method Methods 0.000 abstract description 6
- 238000005457 optimization Methods 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 4
- 210000001061 forehead Anatomy 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 230000008921 facial expression Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000000295 complement effect Effects 0.000 description 2
- 210000003128 head Anatomy 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 235000013372 meat Nutrition 0.000 description 2
- 210000000056 organ Anatomy 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 210000000216 zygoma Anatomy 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The present invention provides a kind of micro- expression and describes method, apparatus, computer installation and computer readable storage medium.It includes: scanning facial image that micro- expression, which describes method, detects the fisrt feature point of the facial image, wherein fisrt feature point is not by the non-colinear characteristic point of micro- expression influence；Using the facial image of fisrt feature point composition as first frame, and the subsequent frame that will test is aligned with the first frame；Piecemeal is carried out to the facial image according to the fisrt feature point, forms the one group of sub-block not overlapped；Extract the corresponding histogram feature vector for indicating the facial image of each sub-block；And the histogram feature vector is cascaded as high dimensional feature vector, and using the high dimensional feature vector as the description form of micro- expression.Micro- expression provided by the invention, which describes method, can adapt to micro- expression space-time characterisation and computation complexity, realize the optimization of mobile computer device operating experience.
Description
Technical field
The present invention relates to field of computer technology more particularly to a kind of micro- expression describe method, apparatus, computer installation and
Computer readable storage medium.
Background technique
Micro- expression (micro-expression) identification technology is the identification technology based on micro- expression.By identifying that user holds
Very short micro- expression of continuous time is interacted with electronic equipment.Compared with macroscopical expression, the feature of micro- expression maximum is to continue
Time is short, intensity is small, is only difficult with the micro- expression of eye recognition.Therefore, at present most commonly using computer vision come
Micro- expression automatic identification is realized, however, to reach the premise accurately identified for this identification method is: effective feature description
Mode.Although current researcher proposes some feasible feature descriptors in recognition of face, macroscopical Expression Recognition, and takes
Obtained good effect.But for the space-time characterisation and computation complexity of micro- expression, directly features described above descriptor is expanded
It is infeasible into micro- Expression Recognition.
Summary of the invention
In view of the foregoing, it is necessary to propose a kind of micro- expression that can adapt to micro- expression space-time characterisation and computation complexity
Method, apparatus, computer installation and computer readable storage medium are described.
The present invention provides a kind of micro- expression and describes method, which comprises
Facial image is scanned, the fisrt feature point of the facial image is detected, wherein fisrt feature point is not by micro-
The non-colinear characteristic point of expression influence；
Using the facial image of fisrt feature point composition as first frame, and the subsequent frame that will test and the head frame pair
Together；
Piecemeal is carried out to the facial image according to the fisrt feature point, forms the one group of sub-block not overlapped；
Extract the corresponding histogram feature vector for indicating the facial image of each sub-block；And
The histogram feature vector is cascaded as high dimensional feature vector, and using the high dimensional feature vector as micro- expression
Description form.
The scanning facial image in one of the embodiments, detects the fisrt feature point of the facial image, wherein
The fisrt feature point be do not included: by the step of non-colinear characteristic point of micro- expression influence
Facial image is scanned, is detected in the facial image of first scan not by the non-colinear feature of micro- expression influence
Point；And
The non-colinear characteristic point is extracted, and using the non-colinear characteristic point as fisrt feature point.
It is described using the facial image of fisrt feature point composition as first frame in one of the embodiments, and will inspection
The step of subsequent frame measured is aligned with the first frame include:
The fisrt feature point is formed into fisrt feature matrix, and using the fisrt feature matrix as the facial image
First frame；
Obtain the corresponding second characteristic matrix of subsequent frame of the facial image；And
According to the fisrt feature matrix and the second characteristic matrix, the subsequent frame is aligned with the first frame.
It is described according to the fisrt feature matrix and the second characteristic matrix in one of the embodiments, it will be described
The step of subsequent frame is aligned with the first frame include:
The fisrt feature matrix is compared with the second characteristic matrix, obtains affine transformation matrix；And
Referring to the affine transformation matrix, the subsequent frame is aligned with the first frame.
It is described in one of the embodiments, that piecemeal is carried out to the facial image according to the fisrt feature point, it is formed
The step of one group of sub-block not overlapped includes:
Obtain the alignment matrix after being aligned with the first frame；And
According to the movement of preset facial muscle and the corresponding relationship of expression, piecemeal is carried out to the alignment matrix, obtain with
The facial image is corresponding and one group of sub-block not overlapping.
It is described in one of the embodiments, to extract the corresponding histogram for indicating the facial image of each sub-block
The step of feature vector includes:
In each sub-block, gray processing processing is carried out to the facial image；And
By gray processing, treated that the facial image is placed in tri- planes of XY, XZ, YZ；And
Extract the corresponding histogram feature vector of each sub-block.
It is described in one of the embodiments, that the histogram feature vector is cascaded as high dimensional feature vector, and by institute
The step of stating description form of the high dimensional feature vector as micro- expression, comprising:
Obtain the corresponding histogram feature vector of each sub-block；
The histogram feature vector of each sub-block is cascaded, high dimensional feature vector is obtained；And
Using the high dimensional feature vector after cascade as the description form of micro- expression.
A kind of micro- expression describes device, and described device includes:
Scan module detects the fisrt feature point of the facial image, wherein described first is special for scanning facial image
Sign point is by the non-colinear characteristic point of micro- expression influence；
Alignment module, the facial image for forming the fisrt feature point will test subsequent as first frame
Frame is aligned with the first frame；
Piecemeal module is formed and is not overlapped for carrying out piecemeal to the facial image according to the fisrt feature point
One group of sub-block；
Extraction module, for extracting the corresponding histogram feature vector for indicating the facial image of each sub-block；
And
Cascade module, for the histogram feature vector to be cascaded as high dimensional feature vector, and by the high dimensional feature
Description form of the vector as micro- expression.
A kind of computer installation, the computer installation include processor and memory, and the processor is for executing institute
Realize that micro- expression describes method when stating the computer program stored in memory.
A kind of computer readable storage medium is stored with computer program on the computer readable storage medium, described
Realize that micro- expression describes method when computer program is executed by processor.
In conclusion micro- expression of the present invention describes method by scanning facial image, the facial image is detected
Fisrt feature point, wherein fisrt feature point be not by the non-colinear characteristic point of micro- expression influence；By the fisrt feature
The facial image of point composition is as first frame, and the subsequent frame that will test is aligned with the first frame；According to the fisrt feature point
Piecemeal is carried out to the facial image, forms the one group of sub-block not overlapped；It extracts described in the corresponding expression of each sub-block
The histogram feature vector of facial image；And the histogram feature vector is cascaded as high dimensional feature vector, and by the height
Description form of the dimensional feature vector as micro- expression.To which realization can adapt to micro- expression space-time characterisation and computation complexity.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
The embodiment of invention for those of ordinary skill in the art without creative efforts, can also basis
The attached drawing of offer obtains other attached drawings.
Fig. 1 is the flow chart that micro- expression that the embodiment of the present invention one provides describes method.
Fig. 2 is that micro- expression provided by Embodiment 2 of the present invention describes functional block diagram in device preferred embodiment.
Fig. 3 is the schematic diagram for the computer installation that the embodiment of the present invention three provides.
The present invention that the following detailed description will be further explained with reference to the above drawings.
Specific embodiment
To better understand the objects, features and advantages of the present invention, with reference to the accompanying drawing and specific real
Applying mode, the present invention will be described in detail.It should be noted that in the absence of conflict, presently filed embodiment and reality
The feature applied in mode can be combined with each other.
In the following description, numerous specific details are set forth in order to facilitate a full understanding of the present invention, described embodiment
Only some embodiments of the invention, rather than whole embodiments.Based on the embodiment in the present invention, this field
Those of ordinary skill's every other embodiment obtained without making creative work, belongs to guarantor of the present invention
The range of shield.
Unless otherwise defined, all technical and scientific terms used herein and belong to technical field of the invention
The normally understood meaning of technical staff is identical.Term as used herein in the specification of the present invention is intended merely to description tool
The purpose of the embodiment of body, it is not intended that in the limitation present invention.
Description and claims of this specification and term " first " in above-mentioned attached drawing, " second " and " third " etc. are
For distinguishing different objects, not for description particular order.In addition, term " includes " and their any deformations, it is intended that
Non-exclusive include in covering.Such as the process, method, system, product or equipment for containing a series of steps or units do not have
It is defined in listed step or unit, but optionally further comprising the step of not listing or unit, or optionally further comprising
For the intrinsic other step or units of these process, methods, product or equipment.
Preferably, micro- expression of the invention describes method and applies in one or more computer installation.The calculating
Machine device is that one kind can be according to the instruction for being previously set or storing, the automatic equipment for carrying out numerical value calculating and/or information processing,
Its hardware includes but is not limited to microprocessor, specific integrated circuit (Application Specific Integrated
Circuit, ASIC), programmable gate array (Field-Programmable Gate Array, FPGA), digital processing unit
(Digital Signal Processor, DSP), embedded device etc..
The computer installation can be the calculating such as desktop PC, laptop, tablet computer, server and set
It is standby.The computer installation can carry out people by modes such as keyboard, mouse, remote controler, touch tablet or voice-operated devices with user
Machine interaction.
Embodiment one:
Fig. 1 is the step flow chart that the micro- expression of the present invention describes method preferred embodiment.The stream according to different requirements,
The sequence of step can change in journey figure, and certain steps can be omitted.
As shown in fig.1, micro- expression, which describes method, specifically includes following steps.
Step S1, facial image is scanned, the fisrt feature point of the facial image is detected, wherein fisrt feature point is
Not by the non-colinear characteristic point of micro- expression influence.
In the present embodiment, by discriminate response diagram fitting (Discriminative Response Map Fitting,
Abbreviation DRMF) method detects the fisrt feature point of the facial image.The fisrt feature point is micro- expression to the face figure
As influencing the smallest characteristic point, the characteristic point etc. as where the nose of face face, forehead.
In one embodiment, the scanning facial image detects the fisrt feature point of the facial image, wherein described
It includes: the facial image for obtaining first scan that fisrt feature point, which is not by the step of non-colinear characteristic point of micro- expression influence,；Base
It chooses in the facial image not vulnerable to 3 non-colinear characteristic points of micro- expression influence；This 3 non-colinear characteristic points are made
For the fisrt feature point；The fisrt feature point is converted into the coordinate points in three-dimensional space, and with reference to the seat of three-dimensional space
These coordinate points are integrated into fisrt feature matrix by mark system.In order to which the facial image is described.
Step S2, using the facial image of fisrt feature point composition as head frame, and the subsequent frame that will test and institute
State first frame alignment.
In the present embodiment, with 1 second for facial image described in intermittent scanning, the fisrt feature point that first scan is arrived is corresponding
Frame headed by facial image.Using the facial image scanned after the first frame as subsequent frame.
In one embodiment, the second feature point of the subsequent frame is obtained.Wherein, the second feature point is the people
Not vulnerable to the point of micro- expression influence, such as nose, forehead, ear etc. in face image.Pass through the second feature point and described first
Feature point alignment realizes that the subsequent frame is aligned with the first frame.
In one embodiment, the second feature point is placed in three bit spaces, thus by the second feature o'clock sharp
It is combined into second characteristic matrix.The second characteristic matrix is compared with the fisrt feature matrix.It is non-colinear by 3
The second characteristic matrix is scaled the alignment matrix for being aligned the fisrt feature matrix by feature point alignment, and is aligned square herein
Subsequent operation is carried out in battle array.Wherein, the quantity of the second feature point is multiple.
Step S3, piecemeal is carried out to the facial image according to the fisrt feature point, forms the one group of son not overlapped
Block.
Wherein, the facial image piecemeal is to utilize FACS (Facial Action Coding System, facial behavior
Coded system) Facial action unit is described.And according to human face characteristic point coordinate by face mark off it is independent, include
Imitate the sub-block of micro- expression information.And FACS is a kind of method for measuring face action.
In one embodiment, it is based on the corresponding facial image of the fisrt feature point, by FACS to different
Facial muscle acts and the corresponding relationship of different expressions.FACS according to the anatomic characteristic of face, by face be divided into it is several both
The mutually indepedent moving cell connected each other again.Then motion feature, its main region controlled of these moving cells are analyzed
Domain and associated expression provide photo and photo explanation.And then the facial image is divided into one group of complementary overhangs
Sub-block.
In one embodiment, the quantity of the sub-block is unlimited, as long as not overlapping.Such as it can be according to micro- expression
The facial image is divided into 4 sub-blocks: eye areas, brow region, cheekbone area, mouth area by middle face face organ
Domain, and this 4 sub-blocks do not overlap.
Step S4, the corresponding histogram feature vector for indicating the facial image of each sub-block is extracted.
In one embodiment, in each sub-block, CPTOP (Cross Patterns on three is used
Orthogonal planes, the cross-mode on three orthogonal planes) operator gray processing expression sequence X Y, XZ and YZ tri-
Histogram feature vector is extracted in a plane, and will be extracted histogram feature vector from three planes and be cascaded into a higher-dimension
Feature vector of the feature vector as the CPTOP operator.
Wherein, the CPTOP operator is that micro- facial expression image sequence is divided into tri- orthogonal planes of XY, XZ and YZ, each
Statistic histogram is generated using texture description operator in orthogonal plane.Sampling mould of the CPTOP operator in each orthogonal plane
Formula, the CPTOP operator is 8 in the sampled point number of each plane, but selection is located at difference in each sample direction
Two sampled points of radius.
In one embodiment, in each sub-block, using CPTOP operator gray processing expression sequence X Y, XZ
It is calculated with extraction histogram feature vector, the feature vector for being then cascaded into a higher-dimension in tri- planes of YZ as the CPTOP
The feature vector of son.The step includes:
It is sampled respectively on the four direction (0, π, I) of two circle shaped neighborhood regions of RXYin, RXYex, using following coding mode, I is corresponding points
Gray value；It is sampled respectively on the four direction that XZ plan radius is two circle shaped neighborhood regions of RXTin, RXTex；In YZ plane half
Diameter be two circle shaped neighborhood regions of RYTin, RYTex four direction on sample respectively；Each plane generate respectively CPXY, CPXZ and
The statistic histogram of CPYZ, then three cascades the vector to be formed compared with higher-dimension, the feature vector as the CPTOP operator.
Step S5, the histogram feature vector is cascaded as high dimensional feature vector, and the high dimensional feature vector is made
For the description form of micro- expression.
Wherein, the description form of micro- expressive features refers to carrying out micro- expression description with data.
In one embodiment, the step of histogram feature vector of each sub-block being cascaded into the feature vector of higher-dimension
It include: to be extracted in tri- planes of expression sequence X Y, XZ and YZ of gray processing using the CPTOP operator straight in each sub-block
Then square figure feature vector is cascaded into feature vector of the feature vector of a higher-dimension as the CPTOP；By above-mentioned each sub-block
Histogram feature vector be cascaded into the feature vector of higher-dimension, the description form as micro- expression.
In another embodiment, micro- expression describes method, which is characterized in that described according to the fisrt feature
Matrix and the second characteristic matrix, the step of subsequent frame is aligned with the first frame include: by the fisrt feature square
Battle array is compared with the second characteristic matrix, obtains affine transformation matrix；And referring to the affine transformation matrix, after described
Continuous frame is aligned with the first frame.
In another embodiment, described that piecemeal is carried out to the facial image according to the fisrt feature point, it is formed mutual
The step of nonoverlapping one group of sub-block includes: the alignment matrix obtained after being aligned with the first frame；And according to preset face's flesh
Meat movement and the corresponding relationship of expression, carry out piecemeal to the alignment matrix, obtain corresponding with the facial image and mutually not
One group of sub-block of overlapping.
In another embodiment, described to extract the corresponding histogram spy for indicating the facial image of each sub-block
The step of levying vector includes: to carry out gray processing processing to the facial image in each sub-block；And gray processing is handled
The facial image afterwards is placed in tri- planes of XY, XZ, YZ；And extract the corresponding histogram feature of each sub-block to
Amount.
In another embodiment, described that the histogram feature vector is cascaded as high dimensional feature vector, and will be described
The step of description form of the high dimensional feature vector as micro- expression, comprising: obtain the corresponding histogram of each sub-block
Feature vector；The histogram feature vector of each sub-block is cascaded, high dimensional feature vector is obtained；And it will be after cascade
Description form of the high dimensional feature vector as micro- expression.
In conclusion micro- expression of the present invention describes method by scanning facial image, the facial image is detected
Fisrt feature point, wherein fisrt feature point be not by the non-colinear characteristic point of micro- expression influence；By the fisrt feature
The facial image of point composition is as first frame, and the subsequent frame that will test is aligned with the first frame；According to the fisrt feature point
Piecemeal is carried out to the facial image, forms the one group of sub-block not overlapped；It extracts described in the corresponding expression of each sub-block
The histogram feature vector of facial image；And the histogram feature vector is cascaded as high dimensional feature vector, and by the height
Description form of the dimensional feature vector as micro- expression.To which realization can adapt to micro- expression space-time characterisation and computation complexity.
Embodiment two:
Fig. 2 is the functional block diagram that the micro- expression of the present invention describes device preferred embodiment.
As shown in fig.2, it may include scan module 201, alignment module 202, piecemeal module that micro- expression, which describes device 20,
203, extraction module 204 and cascade module 205.
The scan module 201 detects the fisrt feature point of the facial image, wherein described for scanning facial image
Fisrt feature point is not by the non-colinear characteristic point of micro- expression influence.
In the present embodiment, the scan module 201 is fitted (Discriminative by discriminate response diagram
Response Map Fitting, abbreviation DRMF) method detects the fisrt feature point of the facial image.The fisrt feature point
The smallest characteristic point is influenced on the facial image for micro- expression, the characteristic point etc. as where the nose of face face, forehead.
In one embodiment, the scan module 201 scans facial image, detects the fisrt feature of the facial image
Point, wherein it includes: the scan module 201 that fisrt feature point, which is not by the step of non-colinear characteristic point of micro- expression influence,
Obtain the facial image of first scan；It is chosen based on the facial image not vulnerable to 3 non-colinear features of micro- expression influence
Point；Using this 3 non-colinear characteristic points as the fisrt feature point；The fisrt feature point is converted in three-dimensional space
Coordinate points, and these coordinate points are integrated into fisrt feature matrix with reference to the coordinate system of three-dimensional space.In order to the face
Image is described.
The facial image that the alignment module 202 is used to form the fisrt feature point will test as first frame
Subsequent frame be aligned with the first frame.
In the present embodiment, the alignment module 202 was facial image described in intermittent scanning with 1 second, and first scan is arrived
Frame headed by the corresponding facial image of fisrt feature point.Using the facial image scanned after the first frame as subsequent frame.
In one embodiment, the alignment module 202 obtains the second feature point of the subsequent frame.Wherein, described
Two characteristic points are in the facial image not vulnerable to the point of micro- expression influence, such as nose, forehead, ear etc..Pass through described second
Characteristic point and the fisrt feature point alignment realize that the subsequent frame is aligned with the first frame.
In one embodiment, the second feature point is placed in three bit spaces by the alignment module 202, thus by institute
It states second feature point and is integrated into second characteristic matrix.The second characteristic matrix is compared with the fisrt feature matrix.
The second characteristic matrix is scaled to the alignment square for being aligned the fisrt feature matrix by 3 non-colinear feature point alignments
Battle array, and subsequent operation is carried out in this alignment matrix.Wherein, the quantity of the second feature point is multiple.
The piecemeal module 203 is used to carry out piecemeal to the facial image according to the fisrt feature point, is formed mutually not
One group of sub-block of overlapping.
Wherein, the facial image piecemeal is to utilize FACS (Facial Action Coding System, facial behavior
Coded system) Facial action unit is described.And according to human face characteristic point coordinate by face mark off it is independent, include
Imitate the sub-block of micro- expression information.And FACS is a kind of method for measuring face action.
In one embodiment, the piecemeal module 203 is based on the corresponding facial image of the fisrt feature point, leads to
FACS is crossed to different facial muscle movements and the corresponding relationship of different expressions.FACS is according to the anatomic characteristic of face, by people
Face is divided into moving cell that is several not only mutually indepedent but also connecting each other.Then analyze these moving cells motion feature, its
The main region and associated expression controlled, provides photo and photo explanation.And then the facial image is divided into
One group of sub-block of complementary overhangs.
In one embodiment, the quantity of the sub-block is unlimited, as long as not overlapping.Such as it can be according to micro- expression
The facial image is divided into 4 sub-blocks: eye areas, brow region, cheekbone area, mouth area by middle face face organ
Domain, and this 4 sub-blocks do not overlap.
The extraction module 204 is used to extract the corresponding histogram feature for indicating the facial image of each sub-block
Vector.
In one embodiment, the piecemeal module 203 uses CPTOP (Cross in each sub-block
Patterns on three orthogonal planes, the cross-mode on three orthogonal planes) operator gray processing table
Histogram feature vector is extracted in tri- planes of feelings sequence X Y, XZ and YZ, and will extract histogram feature from three planes
Vector is cascaded into feature vector of the feature vector of a higher-dimension as the CPTOP operator.
Wherein, the CPTOP operator is that micro- facial expression image sequence is divided into tri- orthogonal planes of XY, XZ and YZ, each
Statistic histogram is generated using texture description operator in orthogonal plane.Sampling mould of the CPTOP operator in each orthogonal plane
Formula, the CPTOP operator is 8 in the sampled point number of each plane, but selection is located at difference in each sample direction
Two sampled points of radius.
In one embodiment, the piecemeal module 203 is in each sub-block, using CPTOP operator in gray processing
Tri- planes of expression sequence X Y, XZ and YZ on extract histogram feature vector, be then cascaded into the feature vector of a higher-dimension
Feature vector as the CPTOP operator.The step includes: for any pixel point O in micro- facial expression image sequence, in XY
Plan radius is samples respectively on the four direction (0, π, I) of two circle shaped neighborhood regions of RXYin, RXYex, using following coding staff
Formula, I are the gray values of corresponding points；It is adopted respectively on the four direction that XZ plan radius is two circle shaped neighborhood regions of RXTin, RXTex
Sample；It is sampled respectively on the four direction that YZ plan radius is two circle shaped neighborhood regions of RYTin, RYTex；It is generated respectively in each plane
The statistic histogram of CPXY, CPXZ and CPYZ, then three cascades the vector to be formed compared with higher-dimension, as the CPTOP operator
Feature vector.
The cascade module 205 is used to the histogram feature vector being cascaded as high dimensional feature vector, and by the height
Description form of the dimensional feature vector as micro- expression.
Wherein, the description form of micro- expressive features refers to carrying out micro- expression description with data.
In one embodiment, the histogram feature vector of each sub-block is cascaded into higher-dimension by the cascade module 205
Feature vector the step of include: in each sub-block, using the CPTOP operator gray processing expression sequence X Y, XZ and YZ
Histogram feature vector is extracted in three planes, is then cascaded into feature of the feature vector of a higher-dimension as the CPTOP
Vector；The histogram feature vector of above-mentioned each sub-block is cascaded into the feature vector of higher-dimension, the description form as micro- expression.
In another embodiment, micro- expression describes method, which is characterized in that described according to the fisrt feature
Matrix and the second characteristic matrix, the step of subsequent frame is aligned with the first frame include: by the fisrt feature square
Battle array is compared with the second characteristic matrix, obtains affine transformation matrix；And referring to the affine transformation matrix, after described
Continuous frame is aligned with the first frame.
In another embodiment, described that piecemeal is carried out to the facial image according to the fisrt feature point, it is formed mutual
The step of nonoverlapping one group of sub-block includes: the alignment matrix obtained after being aligned with the first frame；And according to preset face's flesh
Meat movement and the corresponding relationship of expression, carry out piecemeal to the alignment matrix, obtain corresponding with the facial image and mutually not
One group of sub-block of overlapping.
In another embodiment, described to extract the corresponding histogram spy for indicating the facial image of each sub-block
The step of levying vector includes: to carry out gray processing processing to the facial image in each sub-block；And gray processing is handled
The facial image afterwards is placed in tri- planes of XY, XZ, YZ；And extract the corresponding histogram feature of each sub-block to
Amount.
In another embodiment, described that the histogram feature vector is cascaded as high dimensional feature vector, and will be described
The step of description form of the high dimensional feature vector as micro- expression, comprising: obtain the corresponding histogram of each sub-block
Feature vector；The histogram feature vector of each sub-block is cascaded, high dimensional feature vector is obtained；And it will be after cascade
Description form of the high dimensional feature vector as micro- expression.
In conclusion micro- expression of the present invention, which describes method, scans facial image, inspection by the scan module 201
The fisrt feature point of the facial image is surveyed, wherein fisrt feature point is not by the non-colinear characteristic point of micro- expression influence；
The facial image that the alignment module 202 forms the fisrt feature point is as first frame, and the subsequent frame that will test and institute
State first frame alignment；The piecemeal module 203 carries out piecemeal to the facial image according to the fisrt feature point, and formation does not weigh mutually
One group of folded sub-block；It is special that the extraction module 204 extracts the corresponding histogram for indicating the facial image of each sub-block
Levy vector；And the histogram feature vector is cascaded as high dimensional feature vector by the cascade module 205, and the higher-dimension is special
Levy description form of the vector as micro- expression.To which realization can adapt to micro- expression space-time characterisation and computation complexity.
Embodiment three
Fig. 3 is the schematic diagram of computer installation preferred embodiment of the present invention.
The computer installation 30 includes memory 31, processor 32 and is stored in the memory 31 and can be in institute
The computer program 33 run on processor 32 is stated, such as micro- expression describes program.The processor 32 executes the computer
Realize that above-mentioned micro- expression describes the step in embodiment of the method, such as step S1~S5 shown in FIG. 1 when program 33.Alternatively, institute
It states and realizes that above-mentioned micro- expression describes the function of each module in Installation practice, example when processor 32 executes the computer program 33
Such as the module 201~205 in Fig. 2.
Illustratively, the computer program 33 can be divided into one or more module/units, it is one or
Multiple module/units are stored in the memory 31, and are executed by the processor 32, to complete the present invention.Described one
A or multiple module/units can be the series of computation machine program instruction section that can complete specific function, and described instruction section is used
In implementation procedure of the description computer program 33 in the computer installation 30.For example, the computer program 33 can
With scan module 201, alignment module 202, piecemeal module 203, extraction module 204 and the cascade module being divided into Fig. 2
205.Each module concrete function is referring to embodiment two.
The computer installation 30 can be the calculating such as desktop PC, notebook, palm PC and cloud server
Equipment.It will be understood by those skilled in the art that the schematic diagram is only the example of computer installation 30, do not constitute to calculating
The restriction of machine device 30 may include perhaps combining certain components or different portions than illustrating more or fewer components
Part, such as the computer installation 30 can also include input-output equipment, network access equipment, bus etc..
Alleged processor 32 can be central processing unit (Central Processing Unit, CPU), can also be
Other general processors, digital signal processor (Digital Signal Processor, DSP), specific integrated circuit
(Application Specific Integrated Circuit, ASIC), ready-made programmable gate array (Field-
Programmable Gate Array, FPGA) either other programmable logic device, discrete gate or transistor logic,
Discrete hardware components etc..General processor can be microprocessor or the processor 32 is also possible to any conventional processing
Device etc., the processor 32 are the control centres of the computer installation 30, are entirely calculated using various interfaces and connection
The various pieces of machine device 30.
The memory 31 can be used for storing the computer program 33 and/or module/unit, and the processor 32 passes through
Operation executes the computer program and/or module/unit being stored in the memory 31, and calls and be stored in memory
Data in 31 realize the various functions of the computer installation 30.The memory 31 can mainly include storing program area and
Storage data area, wherein storing program area can application program needed for storage program area, at least one function etc.；Store number
It can be stored according to area and created data etc. are used according to computer installation 30.In addition, memory 31 may include that high speed is random
Memory is accessed, can also include nonvolatile memory, such as hard disk, memory, plug-in type hard disk, intelligent memory card (Smart
Media Card, SMC), secure digital (Secure Digital, SD) card, flash card (Flash Card), at least one disk
Memory device, flush memory device or other volatile solid-state parts.
If the integrated module/unit of the computer installation 30 is realized in the form of SFU software functional unit and as independence
Product when selling or using, can store in a computer readable storage medium.Based on this understanding, of the invention
It realizes all or part of the process in above-described embodiment method, can also instruct relevant hardware come complete by computer program
At the computer program can be stored in a computer readable storage medium, and the computer program is held by processor
When row, it can be achieved that the step of above-mentioned each embodiment of the method.Wherein, the computer program includes computer program code, institute
Stating computer program code can be source code form, object identification code form, executable file or certain intermediate forms etc..It is described
Computer-readable medium may include: any entity or device, recording medium, U that can carry the computer program code
Disk, mobile hard disk, magnetic disk, CD, computer storage, read-only memory (ROM, Read-Only Memory), arbitrary access
Memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium etc..It needs
It is bright, the content that the computer-readable medium includes can according in jurisdiction make laws and patent practice requirement into
Row increase and decrease appropriate, such as do not include electric load according to legislation and patent practice, computer-readable medium in certain jurisdictions
Wave signal and telecommunication signal.
In several embodiments provided by the present invention, it should be understood that disclosed computer installation and method, it can be with
It realizes by another way.For example, computer installation embodiment described above is only schematical, for example, described
The division of unit, only a kind of logical function partition, there may be another division manner in actual implementation.
It, can also be in addition, each functional unit in each embodiment of the present invention can integrate in same treatment unit
It is that each unit physically exists alone, can also be integrated in same unit with two or more units.Above-mentioned integrated list
Member both can take the form of hardware realization, can also realize in the form of hardware adds software function module.
It is obvious to a person skilled in the art that invention is not limited to the details of the above exemplary embodiments, Er Qie
In the case where without departing substantially from spirit or essential attributes of the invention, the present invention can be realized in other specific forms.Therefore, no matter
From the point of view of which point, the present embodiments are to be considered as illustrative and not restrictive, and the scope of the present invention is by appended power
Benefit requires rather than above description limits, it is intended that all by what is fallen within the meaning and scope of the equivalent elements of the claims
Variation is included in the present invention.Any reference signs in the claims should not be construed as limiting the involved claims.This
Outside, it is clear that one word of " comprising " does not exclude other units or steps, and odd number is not excluded for plural number.It is stated in computer installation claim
Multiple units or computer installation can also be implemented through software or hardware by the same unit or computer installation.The
One, the second equal words are used to indicate names, and are not indicated any particular order.
Finally it should be noted that the above examples are only used to illustrate the technical scheme of the present invention and are not limiting, although reference
Preferred embodiment describes the invention in detail, those skilled in the art should understand that, it can be to of the invention
Technical solution is modified or equivalent replacement, without departing from the spirit and scope of the technical solution of the present invention.
Claims (10)
1. a kind of micro- expression describes method, which is characterized in that the described method includes:
Facial image is scanned, the fisrt feature point of the facial image is detected, wherein fisrt feature point is not by micro- expression
The non-colinear characteristic point of influence；
Using the facial image of fisrt feature point composition as first frame, and the subsequent frame that will test is aligned with the first frame；
Piecemeal is carried out to the facial image according to the fisrt feature point, forms the one group of sub-block not overlapped；
Extract the corresponding histogram feature vector for indicating the facial image of each sub-block；And
The histogram feature vector is cascaded as high dimensional feature vector, and using high dimensional feature vector the retouching as micro- expression
State form.
2. micro- expression as described in claim 1 describes method, which is characterized in that the scanning facial image detects the people
The fisrt feature point of face image, wherein fisrt feature point is not wrapped by the step of non-colinear characteristic point of micro- expression influence
It includes:
Facial image is scanned, is detected in the facial image of first scan not by the non-colinear characteristic point of micro- expression influence；And
The non-colinear characteristic point is extracted, and using the non-colinear characteristic point as fisrt feature point.
3. micro- expression as described in claim 1 describes method, which is characterized in that the people for forming the fisrt feature point
Face image is as first frame, and the step of subsequent frame that will test is aligned with the first frame includes:
The fisrt feature point is formed into fisrt feature matrix, and using the fisrt feature matrix as the head of the facial image
Frame；
Obtain the corresponding second characteristic matrix of subsequent frame of the facial image；And
According to the fisrt feature matrix and the second characteristic matrix, the subsequent frame is aligned with the first frame.
4. micro- expression as claimed in claim 3 describes method, which is characterized in that described according to the fisrt feature matrix and institute
The step of stating second characteristic matrix, the subsequent frame is aligned with the first frame include:
The fisrt feature matrix is compared with the second characteristic matrix, obtains affine transformation matrix；And
Referring to the affine transformation matrix, the subsequent frame is aligned with the first frame.
5. micro- expression as described in claim 1 describes method, which is characterized in that it is described according to the fisrt feature point to described
Facial image carries out piecemeal, and the step of forming the one group of sub-block not overlapped includes:
Obtain the alignment matrix after being aligned with the first frame；And
According to the movement of preset facial muscle and the corresponding relationship of expression, piecemeal is carried out to the alignment matrix, obtain with it is described
Facial image is corresponding and one group of sub-block not overlapping.
6. micro- expression as described in claim 1 describes method, which is characterized in that described to extract the corresponding table of each sub-block
The step of showing the histogram feature vector of the facial image include:
In each sub-block, gray processing processing is carried out to the facial image；And
By gray processing, treated that the facial image is placed in tri- planes of XY, XZ, YZ；And
Extract the corresponding histogram feature vector of each sub-block.
7. micro- expression as claimed in claim 6 describes method, which is characterized in that described to cascade the histogram feature vector
For high dimensional feature vector, and using the high dimensional feature vector as the step of description form of micro- expression include:
Obtain the corresponding histogram feature vector of each sub-block；
The histogram feature vector of each sub-block is cascaded, high dimensional feature vector is obtained；And
Using the high dimensional feature vector after cascade as the description form of micro- expression.
8. a kind of micro- expression describes device, which is characterized in that described device includes:
Scan module detects the fisrt feature point of the facial image, wherein fisrt feature point for scanning facial image
For not by the non-colinear characteristic point of micro- expression influence；
Alignment module, facial image for forming the fisrt feature point as first frame, and the subsequent frame that will test with
The head frame alignment；
Piecemeal module forms one group not overlapped for carrying out piecemeal to the facial image according to the fisrt feature point
Sub-block；
Extraction module, for extracting the corresponding histogram feature vector for indicating the facial image of each sub-block；And
Cascade module, for the histogram feature vector to be cascaded as high dimensional feature vector, and by the high dimensional feature vector
Description form as micro- expression.
9. a kind of computer installation, which is characterized in that the computer installation includes processor and memory, and the processor is used
Micro- expression as claimed in any of claims 1 to 7 in one of claims is realized when executing the computer program stored in the memory
Description method.
10. a kind of computer readable storage medium, computer program, feature are stored on the computer readable storage medium
It is, micro- expression description as claimed in any of claims 1 to 7 in one of claims is realized when the computer program is executed by processor
Method.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811527706.8A CN109815793A (en) | 2018-12-13 | 2018-12-13 | Micro- expression describes method, apparatus, computer installation and readable storage medium storing program for executing |
| PCT/CN2019/092144 WO2020119058A1 (en) | 2018-12-13 | 2019-06-20 | Micro-expression description method and device, computer device and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811527706.8A CN109815793A (en) | 2018-12-13 | 2018-12-13 | Micro- expression describes method, apparatus, computer installation and readable storage medium storing program for executing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109815793A true CN109815793A (en) | 2019-05-28 |
Family
ID=66602938
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811527706.8A Pending CN109815793A (en) | 2018-12-13 | 2018-12-13 | Micro- expression describes method, apparatus, computer installation and readable storage medium storing program for executing |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN109815793A (en) |
| WO (1) | WO2020119058A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111178262A (en) * | 2019-12-30 | 2020-05-19 | 中国电子科技集团公司电子科学研究院 | Micro expression detection method and device and computer readable storage medium |
| WO2020119058A1 (en) * | 2018-12-13 | 2020-06-18 | 平安科技（深圳）有限公司 | Micro-expression description method and device, computer device and readable storage medium |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113920569B (en) * | 2021-11-06 | 2024-09-03 | 北京九州安华信息安全技术有限公司 | Micro-expression vertex positioning method and device based on characteristic difference |
| CN114220154A (en) * | 2021-12-20 | 2022-03-22 | 王越 | Micro-expression feature extraction and identification method based on deep learning |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104298981A (en) * | 2014-11-05 | 2015-01-21 | 河北工业大学 | Face microexpression recognition method |
| CN106096537A (en) * | 2016-06-06 | 2016-11-09 | 山东大学 | A kind of micro-expression automatic identifying method based on multi-scale sampling |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104732216A (en) * | 2015-03-26 | 2015-06-24 | 江苏物联网研究发展中心 | Expression recognition method based on key points and local characteristics |
| CN105913038B (en) * | 2016-04-26 | 2019-08-06 | 哈尔滨工业大学深圳研究生院 | A kind of micro- expression recognition method of dynamic based on video |
| CN106127196B (en) * | 2016-09-14 | 2020-01-14 | 河北工业大学 | Facial expression classification and identification method based on dynamic texture features |
| CN109815793A (en) * | 2018-12-13 | 2019-05-28 | 平安科技（深圳）有限公司 | Micro- expression describes method, apparatus, computer installation and readable storage medium storing program for executing |
-
2018
- 2018-12-13 CN CN201811527706.8A patent/CN109815793A/en active Pending
-
2019
- 2019-06-20 WO PCT/CN2019/092144 patent/WO2020119058A1/en active Application Filing
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104298981A (en) * | 2014-11-05 | 2015-01-21 | 河北工业大学 | Face microexpression recognition method |
| CN106096537A (en) * | 2016-06-06 | 2016-11-09 | 山东大学 | A kind of micro-expression automatic identifying method based on multi-scale sampling |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020119058A1 (en) * | 2018-12-13 | 2020-06-18 | 平安科技（深圳）有限公司 | Micro-expression description method and device, computer device and readable storage medium |
| CN111178262A (en) * | 2019-12-30 | 2020-05-19 | 中国电子科技集团公司电子科学研究院 | Micro expression detection method and device and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2020119058A1 (en) | 2020-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110020620B (en) | Face recognition method, device and equipment under large posture | |
| CN109558864B (en) | Face key point detection method, device and storage medium | |
| US9349076B1 (en) | Template-based target object detection in an image | |
| CN112241731B (en) | Attitude determination method, device, equipment and storage medium | |
| CN109815793A (en) | Micro- expression describes method, apparatus, computer installation and readable storage medium storing program for executing | |
| JP7475772B2 (en) | IMAGE GENERATION METHOD, IMAGE GENERATION DEVICE, COMPUTER DEVICE, AND COMPUTER PROGRAM | |
| CN109409994A (en) | The methods, devices and systems of analog subscriber garments worn ornaments | |
| CN109948397A (en) | A kind of face image correcting method, system and terminal device | |
| CN110379020A (en) | A kind of laser point cloud painting methods and device based on generation confrontation network | |
| CN110378203A (en) | Image processing method, device, terminal and storage medium | |
| CN113689436A (en) | Image semantic segmentation method, device, equipment and storage medium | |
| CN110288715A (en) | Virtual necklace try-in method, device, electronic equipment and storage medium | |
| CN108388889A (en) | Method and apparatus for analyzing facial image | |
| CN107944381A (en) | Face tracking method, device, terminal and storage medium | |
| CN108491812A (en) | The generation method and device of human face recognition model | |
| CN111507259B (en) | Face feature extraction method and device and electronic equipment | |
| CN111680670A (en) | Cross-mode human head detection method and device | |
| CN111159609A (en) | Attribute information modification method and related device | |
| Lv et al. | A challenge of deep‐learning‐based object detection for hair follicle dataset | |
| CN114639152A (en) | Multi-modal voice interaction method, device, equipment and medium based on face recognition | |
| CN113902671A (en) | Image steganography method and system based on random texture | |
| CN112417927A (en) | Method for establishing human body posture recognition model, human body posture recognition method and device | |
| CN113239814B (en) | Facial expression recognition method, device, equipment and medium based on optical flow reconstruction | |
| CN109376602A (en) | A kind of finger vein identification method, device and terminal device | |
| CN115147469A (en) | Registration method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |