WO2020119058A1 - 微表情描述方法、装置、计算机装置及可读存储介质 - Google Patents

微表情描述方法、装置、计算机装置及可读存储介质 Download PDF

Info

Publication number
WO2020119058A1
WO2020119058A1 PCT/CN2019/092144 CN2019092144W WO2020119058A1 WO 2020119058 A1 WO2020119058 A1 WO 2020119058A1 CN 2019092144 W CN2019092144 W CN 2019092144W WO 2020119058 A1 WO2020119058 A1 WO 2020119058A1
Authority
WO
WIPO (PCT)
Prior art keywords
feature
face image
micro
frame
blocks
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/092144
Other languages
English (en)
French (fr)
Inventor
郭玲玲
姜宜君
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Technology Shenzhen Co Ltd
Original Assignee
Ping An Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Technology Shenzhen Co Ltd filed Critical Ping An Technology Shenzhen Co Ltd
Publication of WO2020119058A1 publication Critical patent/WO2020119058A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition

Definitions

  • the present application relates to the field of computer technology, and in particular, to a micro-expression description method, device, computer device, and non-volatile readable storage medium.
  • Micro-expression recognition technology is a recognition technology based on micro-expression. Interact with electronic devices by recognizing micro-expressions with a very short duration of the user. Compared with macro-expressions, micro-expressions are characterized by short duration and low intensity. It is difficult to recognize micro-expressions by human eyes alone. Therefore, the most common use of computer vision is to realize automatic recognition of micro-expressions.
  • the premise of achieving accurate recognition for this recognition method is: an effective feature description method. Although researchers have put forward some feasible feature descriptors in face recognition and macro facial expression recognition, and have achieved good results. However, for the spatiotemporal characteristics and computational complexity of micro-expressions, it is not feasible to directly extend the above feature descriptors to micro-expression recognition.
  • An embodiment of the present application provides a micro-expression description method.
  • the method includes:
  • the histogram feature vectors are cascaded into high-dimensional feature vectors, and the high-dimensional feature vectors are used as the description form of micro-expressions.
  • An embodiment of the present application provides a micro-expression description device.
  • the device includes:
  • a scanning module configured to scan a face image and detect a first feature point of the face image, wherein the first feature point is a non-collinear feature point that is not affected by micro-expressions;
  • An alignment module configured to use the face image composed of the first feature points as the first frame, and align the detected subsequent frames with the first frame;
  • a blocking module configured to block the face image according to the first feature point to form a group of sub-blocks that do not overlap each other;
  • An extraction module configured to extract a histogram feature vector representing the face image corresponding to each of the sub-blocks
  • a cascading module is used to cascade the histogram feature vectors into high-dimensional feature vectors, and use the high-dimensional feature vectors as a description form of micro-expressions.
  • An embodiment of the present application provides a computer device.
  • the computer device includes a processor and a memory.
  • the processor is used to implement a micro-expression description method when executing computer-readable instructions stored in the memory.
  • An embodiment of the present application provides a non-volatile readable storage medium having computer-readable instructions stored on the non-volatile readable storage medium, the computer-readable instructions being executed by a processor to implement micro-expression description method.
  • the above micro-expression description method, system, computer device and non-volatile readable storage medium detect the first feature point of the face image by scanning the face image, wherein the first feature point is not subject to micro-expression Non-collinear feature points affected; the face image composed of the first feature point is taken as the first frame, and the detected subsequent frames are aligned with the first frame; the face is based on the first feature point
  • the image is divided into blocks to form a set of non-overlapping sub-blocks; the histogram feature vector representing the face image corresponding to each sub-block is extracted; and the histogram feature vector is cascaded into high-dimensional features Vector, and use the high-dimensional feature vector as the description form of the micro-expression.
  • the realization can adapt to the spatiotemporal characteristics of micro-expressions and the computational complexity.
  • FIG. 1 is a flowchart of a micro-expression description method provided in Embodiment 1 of the present application.
  • FIG. 2 is a functional block diagram of a preferred embodiment of a micro-expression description device provided in Embodiment 2 of the present application.
  • FIG. 3 is a schematic diagram of a computer device provided in Embodiment 3 of the present application.
  • the micro-expression description method of the present application is applied to one or more computer devices.
  • the computer device is a device that can automatically perform numerical calculation and/or information processing according to a preset or stored instruction, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC) , Programmable gate array (Field-Programmable Gate Array, FPGA), digital processor (Digital Signal Processor, DSP), embedded equipment, etc.
  • ASIC Application Specific Integrated Circuit
  • FPGA Field-Programmable Gate Array
  • DSP Digital Signal Processor
  • the computer device may be a computing device such as a desktop computer, a notebook computer, a tablet computer, and a server.
  • the computer device can interact with the user through a keyboard, a mouse, a remote control, a touchpad, or a voice control device.
  • FIG. 1 is a flowchart of steps of a preferred embodiment of a micro-expression description method of the present application. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted.
  • the micro-expression description method specifically includes the following steps.
  • Step S1 Scan the face image to detect the first feature point of the face image, where the first feature point is a non-collinear feature point that is not affected by the micro-expression.
  • the first feature point of the face image is detected by a discriminant response map fitting (Discriminative Response Map Fitting, DRMF for short) method.
  • the first feature point is a feature point that has minimal influence on the face image, such as a nose and a forehead of the face.
  • the facial image is scanned to detect a first feature point of the facial image, wherein the first feature point is a non-collinear feature point that is not affected by micro-expressions.
  • Step S2 Use the face image composed of the first feature points as the first frame, and align the detected subsequent frames with the first frame.
  • the face image is scanned in a cycle of 1 second, and the face image corresponding to the first feature point scanned for the first time is taken as the first frame.
  • the face image scanned after the first frame is used as a subsequent frame.
  • the second feature point of the subsequent frame is obtained.
  • the second feature point is a point in the face image that is not easily affected by micro-expressions, such as nose, forehead, ears, etc.
  • the second feature points are placed in a three-dimensional space, so that the second feature points are integrated into a second feature matrix. Comparing the second feature matrix with the first feature matrix. The second feature matrix is converted into an alignment matrix that aligns with the first feature matrix through alignment of three non-collinear feature points, and subsequent operations are performed on the alignment matrix. Wherein, the number of the second feature points is multiple.
  • Step S3 Block the face image according to the first feature point to form a group of sub-blocks that do not overlap each other.
  • the face image segmentation uses FACS (Facial Action Coding System) to describe the facial action unit. According to the coordinates of face feature points, the face is divided into independent sub-blocks containing effective micro-expression information. And FACS is a method of measuring facial movements.
  • FACS Joint Action Coding System
  • the corresponding relationship between different facial muscle actions and different expressions is performed by FACS.
  • FACS divides human face into several independent and interconnected motion units. Then analyze the motion characteristics of these motion units, the main areas they control and the expressions related to them, and give photos and photo descriptions. Furthermore, the face image is divided into a group of complementary overlapping sub-blocks.
  • the number of the sub-blocks is not limited, as long as they do not overlap each other.
  • the face image can be divided into four sub-blocks according to the facial facial organs in the micro-expression: eye area, eyebrow area, cheekbone area, and mouth area, and the four sub-blocks do not overlap each other.
  • Step S4 Extract a histogram feature vector corresponding to each face block that represents the face image.
  • a CPTOP Crosswise pattern on three orthogonal planes is used in the grayscale expression sequence XY, XZ and YZ.
  • the histogram feature vectors are extracted from each plane, and the histogram feature vectors extracted from the three planes are cascaded into a high-dimensional feature vector as the feature vector of the CPTOP operator.
  • the CPTOP operator is to divide the micro-expression image sequence into three orthogonal planes of XY, XZ and YZ, and use the texture description operator in each orthogonal plane to generate a statistical histogram.
  • the sampling mode of the CPTOP operator in each orthogonal plane, the number of sampling points of the CPTOP operator in each plane is 8, but two sampling points located at different radii are selected in each sampling direction.
  • a CPTOP operator is used to extract histogram feature vectors on three planes of gray-scale expression sequences XY, XZ, and YZ, and then cascaded into a high-dimensional feature The vector serves as the feature vector of the CPTOP operator.
  • the steps include: for any pixel point O in the micro-expression image sequence, samples are respectively taken in four directions (0, ⁇ , I) in two circular neighborhoods with radius XYin and RXYex in the XY plane, using the following encoding method , I is the gray value of the corresponding point; sampling in four directions of two circular neighborhoods with radius RXTin and RXTex in XZ plane; four directions of two circular neighborhoods with radius RYTin and RYTex in YZ plane Up-sampling separately; generate statistical histograms of CPXY, CPXZ, and CPYZ in each plane, and then cascade the three to form a higher-dimensional vector as the feature vector of the CPTOP operator.
  • Step S5 Cascade the histogram feature vectors into high-dimensional feature vectors, and use the high-dimensional feature vectors as a description form of micro-expressions.
  • micro-expression features refers to the use of data to describe micro-expressions.
  • the step of concatenating the histogram feature vectors of the sub-blocks into high-dimensional feature vectors includes: in each sub-block, using the CPTOP operator in the gray-scale expression sequence XY, XZ Extract histogram feature vectors on the three planes of YZ and cascade into a high-dimensional feature vector as the CPTOP feature vector; cascade the histogram feature vectors of the above sub-blocks into high-dimensional feature vectors as The description form of the micro expression.
  • the step of aligning the subsequent frame with the first frame according to the first feature matrix and the second feature matrix includes: aligning the first feature matrix with the first Comparing the two feature matrices to obtain an affine transformation matrix; and referring to the affine transformation matrix to align the subsequent frame with the first frame.
  • the step of dividing the face image according to the first feature point to form a set of non-overlapping sub-blocks includes: obtaining an alignment matrix aligned with the first frame And dividing the alignment matrix according to the preset correspondence between facial muscle actions and expressions to obtain a set of sub-blocks corresponding to the face image and not overlapping each other.
  • the step of extracting the histogram feature vector representing the face image corresponding to each of the sub-blocks includes: on each of the sub-blocks, graying the face image Degree processing; and placing the grayscale processed face image on three planes of XY, XZ, and YZ; and extracting the histogram feature vector corresponding to each of the sub-blocks.
  • the step of cascading the histogram feature vectors into a high-dimensional feature vector and using the high-dimensional feature vector as a description form of micro-expressions includes: obtaining each of the sub-blocks Corresponding to the histogram feature vector; cascade the histogram feature vectors of each of the sub-blocks to obtain a high-dimensional feature vector; and use the cascaded high-dimensional feature vector as a description form of micro-expressions.
  • the micro-expression description method described in this application detects the first feature point of the facial image by scanning the facial image, where the first feature point is a non-collinear feature that is not affected by the micro-expression Point; the face image composed of the first feature point as the first frame, and the detected subsequent frames are aligned with the first frame; the face image is divided into blocks according to the first feature point to form A group of sub-blocks that do not overlap each other; extract the histogram feature vector corresponding to each of the sub-blocks representing the face image; and cascade the histogram feature vectors into high-dimensional feature vectors, and merge the High-dimensional feature vectors are used as the description form of micro-expressions.
  • the realization can adapt to the spatiotemporal characteristics of micro-expressions and the computational complexity.
  • FIG. 2 is a functional block diagram of a preferred embodiment of a micro-expression description device of the present application.
  • the micro-expression description device 20 may include a scanning module 201, an alignment module 202, a segmentation module 203, an extraction module 204, and a cascading module 205.
  • the scanning module 201 is used to scan a face image to detect a first feature point of the face image, where the first feature point is a non-collinear feature point that is not affected by micro-expressions.
  • the scanning module 201 detects the first feature point of the face image through a discriminant response map fitting (Discriminative Response Map Fitting, DRMF for short) method.
  • the first feature point is a feature point that has minimal influence on the face image, such as a nose and a forehead of the face.
  • the scanning module 201 scans a face image in the following manner to detect the first feature point of the face image, where the first feature point is a non-collinear feature that is not affected by micro-expressions Points:
  • the scanning module 201 acquires the first scanned face image; based on the face image, 3 non-collinear feature points that are not easily affected by micro-expressions are selected; these 3 non-collinear feature points are used as the First feature points; convert the first feature points into coordinate points in three-dimensional space, and integrate these coordinate points into a first feature matrix with reference to the coordinate system in three-dimensional space. In order to describe the face image.
  • the alignment module 202 is configured to use the face image composed of the first feature points as the first frame, and align the detected subsequent frames with the first frame.
  • the alignment module 202 scans the face image with a period of 1 second, and takes the face image corresponding to the first feature point scanned for the first time as the first frame.
  • the face image scanned after the first frame is used as a subsequent frame.
  • the alignment module 202 obtains the second feature point of the subsequent frame.
  • the second feature point is a point in the face image that is not easily affected by micro-expressions, such as nose, forehead, ears, etc.
  • the alignment module 202 places the second feature point in a three-dimensional space, thereby integrating the second feature point into a second feature matrix. Comparing the second feature matrix with the first feature matrix. The second feature matrix is converted into an alignment matrix that aligns with the first feature matrix through alignment of three non-collinear feature points, and subsequent operations are performed on the alignment matrix. Wherein, the number of the second feature points is multiple.
  • the blocking module 203 is configured to block the face image according to the first feature point to form a group of sub-blocks that do not overlap each other.
  • the face image segmentation uses FACS (Facial Action Coding System) to describe the facial action unit. According to the coordinates of face feature points, the face is divided into independent sub-blocks containing effective micro-expression information. And FACS is a method of measuring facial movements.
  • FACS Joint Action Coding System
  • the blocking module 203 uses FACS to correspond the relationship between different facial muscle actions and different expressions.
  • FACS divides human face into several independent and interconnected motion units. Then analyze the motion characteristics of these motion units, the main areas they control and the expressions related to them, and give photos and photo descriptions. Furthermore, the face image is divided into a group of complementary overlapping sub-blocks.
  • the number of the sub-blocks is not limited, as long as they do not overlap each other.
  • the face image can be divided into four sub-blocks according to the facial facial organs in the micro-expression: eye area, eyebrow area, cheekbone area, and mouth area, and the four sub-blocks do not overlap each other.
  • the extraction module 204 is used to extract a histogram feature vector corresponding to each face block that represents the face image.
  • the segmentation module 203 uses a CPTOP (Cross Patterns on three orthogonal planes) cross-patterns on three orthogonal planes in each grayscale expression sequence Extract histogram feature vectors on three planes of XY, XZ and YZ, and cascade the histogram feature vectors extracted from the three planes into a high-dimensional feature vector as the feature vector of the CPTOP operator.
  • CPTOP Cross Patterns on three orthogonal planes
  • the CPTOP operator is to divide the micro-expression image sequence into three orthogonal planes of XY, XZ and YZ, and use the texture description operator in each orthogonal plane to generate a statistical histogram.
  • the sampling mode of the CPTOP operator in each orthogonal plane, the number of sampling points of the CPTOP operator in each plane is 8, but two sampling points located at different radii are selected in each sampling direction.
  • the partitioning module 203 extracts the histogram feature vectors on the three planes of grayscale expression sequences XY, XZ, and YZ using the CPTOP operator on each of the sub-blocks, and then cascades A high-dimensional feature vector is used as the feature vector of the CPTOP operator.
  • the specific implementation method may be: for any pixel point O in the micro-expression image sequence, samples are respectively taken in four directions (0, ⁇ , I) in two circular neighborhoods with radius XYin and RXYex in the XY plane, and the following coding is adopted Method, I is the gray value of the corresponding point; sampling in four directions of two circular neighborhoods with radius RXTin and RXTex in XZ plane; four in two circular neighborhoods with radius RYTin and RYTex in YZ plane Sampling in the direction separately; generating statistical histograms of CPXY, CPXZ and CPYZ in each plane, and then cascading the three to form a higher-dimensional vector as the feature vector of the CPTOP operator.
  • the cascading module 205 is used to cascade the histogram feature vectors into high-dimensional feature vectors, and use the high-dimensional feature vectors as a description form of micro-expressions.
  • micro-expression features refers to the use of data to describe micro-expressions.
  • the cascading module 205 may cascade the histogram feature vectors of the sub-blocks into high-dimensional feature vectors in the following manner: In each sub-block, use the CPTOP operator in the gray Extract the histogram feature vector on the three planes of the degree of expression sequence XY, XZ and YZ, and then cascade into a high-dimensional feature vector as the CPTOP feature vector; cascade the histogram feature vectors of the above sub-blocks High-dimensional feature vectors are used as the description form of micro-expressions.
  • the implementation manner of aligning the subsequent frame with the first frame according to the first feature matrix and the second feature matrix may be: aligning the first feature matrix with the The second feature matrix is compared to obtain an affine transformation matrix; and referring to the affine transformation matrix, the subsequent frame is aligned with the first frame.
  • the specific implementation method of dividing the face image according to the first feature point to form a group of non-overlapping sub-blocks may be: after obtaining alignment with the first frame Alignment matrix; and according to the preset correspondence between facial muscle actions and expressions, block the alignment matrix to obtain a set of sub-blocks corresponding to the face image and not overlapping each other.
  • the method for extracting the histogram feature vector corresponding to the face image corresponding to each of the sub-blocks may be: on each of the sub-blocks, the face image Perform grayscale processing; and place the face image after grayscale processing on three planes of XY, XZ, and YZ; and extract a histogram feature vector corresponding to each of the sub-blocks.
  • the concatenation of the histogram feature vectors into high-dimensional feature vectors, and the implementation of using the high-dimensional feature vectors as a description form of micro-expressions may be: acquiring each The histogram feature vector corresponding to the block; concatenating the histogram feature vectors of each of the sub-blocks to obtain a high-dimensional feature vector; and using the concatenated high-dimensional feature vector as a description form of micro-expressions.
  • the micro-expression description method described in this application scans the face image through the scanning module 201 to detect the first feature point of the face image, wherein the first feature point is not affected by the micro-expression Non-collinear feature points;
  • the alignment module 202 uses the face image composed of the first feature points as the first frame, and aligns the detected subsequent frames with the first frame;
  • the first feature point blocks the face image to form a group of sub-blocks that do not overlap each other;
  • the extraction module 204 extracts a histogram feature vector corresponding to the face image corresponding to each of the sub-blocks And the cascading module 205 cascades the histogram feature vectors into high-dimensional feature vectors, and uses the high-dimensional feature vectors as a description form of micro-expressions.
  • the realization can adapt to the spatiotemporal characteristics of micro-expressions and the computational complexity.
  • FIG. 3 is a schematic diagram of a preferred embodiment of the computer device of the present application.
  • the computer device 30 includes a memory 31, a processor 32, and computer-readable instructions 33 stored in the memory 31 and executable on the processor 32, such as a micro-expression description program.
  • the processor 32 executes the computer-readable instruction 33
  • the steps in the above embodiment of the micro-expression description method are implemented, for example, steps S1 to S5 shown in FIG. 1.
  • the processor 32 executes the computer-readable instruction 33
  • the functions of the modules in the above embodiment of the micro-expression description device are implemented, for example, the modules 201 to 205 in FIG. 2.
  • the computer-readable instructions 33 may be divided into one or more modules/units, the one or more modules/units are stored in the memory 31, and executed by the processor 32, To complete this application.
  • the one or more modules/units may be a series of computer-readable instruction instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer-readable instructions 33 in the computer device 30.
  • the computer-readable instructions 33 may be divided into the scanning module 201, the alignment module 202, the segmentation module 203, the extraction module 204, and the cascading module 205 in FIG. For specific functions of each module, see Embodiment 2.
  • the computer device 30 may be a computing device such as a desktop computer, a notebook, a palmtop computer and a cloud server.
  • a computing device such as a desktop computer, a notebook, a palmtop computer and a cloud server.
  • the schematic diagram is only an example of the computer device 30 and does not constitute a limitation on the computer device 30, and may include more or less components than the illustration, or a combination of certain components, or different Components, for example, the computer device 30 may also include input and output devices, network access devices, buses, and the like.
  • the so-called processor 32 may be a central processing unit (Central Processing Unit, CPU), or other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), Ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • the general-purpose processor may be a microprocessor or the processor 32 may also be any conventional processor, etc.
  • the processor 32 is the control center of the computer device 30 and connects the entire computer device 30 with various interfaces and lines The various parts.
  • the memory 31 may be used to store the computer-readable instructions 33 and/or modules/units, and the processor 32 executes or executes the computer-readable instructions and/or modules/units stored in the memory 31, and The data stored in the memory 31 is called to realize various functions of the computer device 30.
  • the memory 31 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, at least one function required application programs, etc.; the storage data area may store data created according to the use of the computer device 30 Wait.
  • the memory 31 may include a high-speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart, Media, Card, SMC), and a secure digital (SD) Card, flash memory card (Flash), at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.
  • non-volatile memory such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart, Media, Card, SMC), and a secure digital (SD) Card, flash memory card (Flash), at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.
  • the module/unit integrated in the computer device 30 may be stored in a computer-readable storage medium.
  • the present application can implement all or part of the processes in the methods of the above embodiments, and can also be completed by instructing relevant hardware through computer-readable instructions.
  • the computer-readable instructions can be stored in a non-volatile In reading the storage medium, when the computer-readable instructions are executed by the processor, the steps of the foregoing method embodiments may be implemented.
  • the computer readable instructions include computer readable instruction codes, and the computer readable instruction codes may be in source code form, object code form, executable file, or some intermediate form, etc.
  • the computer-readable medium may include: any entity or device capable of carrying the computer-readable instruction code, recording medium, U disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access), electrical carrier signals, telecommunications signals and software distribution media, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in jurisdictions. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media Does not include electrical carrier signals and telecommunications signals.
  • the functional units in the embodiments of the present application may be integrated in the same processing unit, or each unit may exist alone physically, or two or more units may be integrated in the same unit.
  • the above integrated unit can be implemented in the form of hardware, or in the form of hardware plus software function modules.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

本申请提供一种微表情描述方法、装置、计算机装置及非易失性可读存储介质。微表情描述方法包括：扫描人脸图像，检测所述人脸图像的第一特征点，其中所述第一特征点为不受微表情影响的非共线特征点；将所述第一特征点组成的人脸图像作为首帧，并将检测到的后续帧与所述首帧对齐；根据所述第一特征点对所述人脸图像进行分块，形成互不重叠的一组子块；提取每一所述子块对应的表示所述人脸图像的直方图特征向量；及将所述直方图特征向量级联为高维特征向量，并将所述高维特征向量作为微表情的描述形式。本申请提供的微表情描述方法能够适应微表情时空特性和计算复杂度，实现移动计算机装置操作体验优化。

Description

微表情描述方法、装置、计算机装置及可读存储介质
本申请要求于2018年12月13日提交中国专利局，申请号为201811527706.8发明名称为“微表情描述方法、装置、计算机装置及可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，尤其涉及一种微表情描述方法、装置、计算机装置及非易失性可读存储介质。
背景技术
微表情(micro-expression)识别技术即基于微表情的识别技术。通过识别用户持续时间非常短的微表情与电子设备进行互动。与宏观表情相比，微表情最大的特点是持续时间短、强度小，仅凭人眼识别微表情是很难的。因此，目前最常见的是利用计算机视觉来实现微表情自动识别，然而，针对这种识别方式要达到准确识别的前提是：有效的特征描述方式。虽然目前研究人员在人脸识别、宏观表情识别中提出了一些可行的特征描述符，且取得了不错的效果。但是，针对微表情的时空特性和计算复杂度，直接将上述特征描述符拓展到微表情识别中是不可行的。
发明内容
鉴于以上内容，有必要提出一种能够适应微表情时空特性和计算复杂度的微表情描述方法、装置、计算机装置及非易失性可读存储介质。
本申请一实施方式提供一种微表情描述方法，所述方法包括：
扫描人脸图像，检测所述人脸图像的第一特征点，其中所述第一特征点为不受微表情影响的非共线特征点；
将所述第一特征点组成的人脸图像作为首帧，并将检测到的后续帧与所述首帧对齐；
根据所述第一特征点对所述人脸图像进行分块，形成互不重叠的一组子块；
提取每一所述子块对应的表示所述人脸图像的直方图特征向量；及
将所述直方图特征向量级联为高维特征向量，并将所述高维特 征向量作为微表情的描述形式。
本申请一实施方式提供一种微表情描述装置，所述装置包括：
扫描模块，用于扫描人脸图像，检测所述人脸图像的第一特征点，其中所述第一特征点为不受微表情影响的非共线特征点；
对齐模块，用于将所述第一特征点组成的人脸图像作为首帧，并将检测到的后续帧与所述首帧对齐；
分块模块，用于根据所述第一特征点对所述人脸图像进行分块，形成互不重叠的一组子块；
提取模块，用于提取每一所述子块对应的表示所述人脸图像的直方图特征向量；及
级联模块，用于将所述直方图特征向量级联为高维特征向量，并将所述高维特征向量作为微表情的描述形式。
本申请一实施方式提供一种计算机装置，所述计算机装置包括处理器和存储器，所述处理器用于执行所述存储器中存储的计算机可读指令时实现微表情描述方法。
本申请一实施方式提供一种非易失性可读存储介质，所述非易失性可读存储介质上存储有计算机可读指令，所述计算机可读指令被处理器执行时实现微表情描述方法。
上述微表情描述方法、系统、计算机装置及非易失性可读存储介质，通过扫描人脸图像，检测所述人脸图像的第一特征点，其中所述第一特征点为不受微表情影响的非共线特征点；将所述第一特征点组成的人脸图像作为首帧，并将检测到的后续帧与所述首帧对齐；根据所述第一特征点对所述人脸图像进行分块，形成互不重叠的一组子块；提取每一所述子块对应的表示所述人脸图像的直方图特征向量；及将所述直方图特征向量级联为高维特征向量，并将所述高维特征向量作为微表情的描述形式。从而，实现能够适应微表情时空特性和计算复杂度。
附图说明
为了更清楚地说明本申请实施方式的技术方案，下面将对实施方式描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本申请的一些实施方式，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例一提供的微表情描述方法的流程图。
图2是本申请实施例二提供的微表情描述装置较佳实施例中的功能模块图。
图3是本申请实施例三提供的计算机装置的示意图。
具体实施方式
为了能够更清楚地理解本申请的上述目的、特征和优点，下面结合附图和具体实施方式对本申请进行详细描述。需要说明的是，在不冲突的情况下，本申请的实施方式及实施方式中的特征可以相互组合。
在下面的描述中阐述了很多具体细节以便于充分理解本申请，所描述的实施方式仅仅是本申请一部分实施方式，而不是全部的实施方式。基于本申请中的实施方式，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施方式，都属于本申请保护的范围。
除非另有定义，本文所使用的所有的技术和科学术语与属于本申请的技术领域的技术人员通常理解的含义相同。本文中在本申请的说明书中所使用的术语只是为了描述具体的实施方式的目的，不是旨在于限制本申请。
优选地，本申请的微表情描述方法应用在一个或者多个计算机装置中。所述计算机装置是一种能够按照事先设定或存储的指令，自动进行数值计算和/或信息处理的设备，其硬件包括但不限于微处理器、专用集成电路(Application Specific Integrated Circuit，ASIC)、可编程门阵列(Field－Programmable Gate Array，FPGA)、数字处理器(Digital Signal Processor，DSP)、嵌入式设备等。
所述计算机装置可以是桌上型计算机、笔记本电脑、平板电脑、服务器等计算设备。所述计算机装置可以与用户通过键盘、鼠标、遥控器、触摸板或声控设备等方式进行人机交互。
实施例一：
图1是本申请微表情描述方法较佳实施例的步骤流程图。根据不同的需求，所述流程图中步骤的顺序可以改变，某些步骤可以省略。
参阅图1所示，微表情描述方法具体包括以下步骤。
步骤S1、扫描人脸图像，检测所述人脸图像的第一特征点，其中所述第一特征点为不受微表情影响的非共线特征点。
本实施例中，通过判别式响应图拟合(Discriminative Response Map Fitting，简称DRMF)方法检测所述人脸图像的第一特征点。所述第一特征点为微表情对所述人脸图像影响最小的特征点，如人脸面部的鼻子、额头所在的特征点等。
在一个实施例中，所述扫描人脸图像，检测所述人脸图像的第一特征点，其中所述第一特征点为不受微表情影响的非共线特征点的步骤包括：获取首次扫描的人脸图像；基于所述人脸图像 选取不易受微表情影响的3个非共线的特征点；将这3个非共线的特征点作为所述第一特征点；将所述第一特征点转换为三维空间中的坐标点，并参考三维空间的坐标系将这些坐标点整合为第一特征矩阵。以便于对所述人脸图像进行描述。
步骤S2、将所述第一特征点组成的人脸图像作为首帧，并将检测到的后续帧与所述首帧对齐。
本实施例中，以1秒为周期扫描所述人脸图像，将首次扫描到的第一特征点对应的人脸图像为首帧。将所述首帧之后扫描到的人脸图像作为后续帧。
在一个实施例中，获取所述后续帧的第二特征点。其中，所述第二特征点为所述人脸图像中不易受微表情影响的点，如鼻子，额头、耳朵等。通过所述第二特征点与所述第一特征点对齐，实现所述后续帧与所述首帧对齐。
在一个实施例中，将所述第二特征点置于三位空间中，从而将所述第二特征点整合为第二特征矩阵。将所述第二特征矩阵与所述第一特征矩阵进行比对。通过3个非共线的特征点对齐将所述第二特征矩阵换算为对齐所述第一特征矩阵的对齐矩阵，并在此对齐矩阵上进行后续操作。其中，所述第二特征点的数量为多个。
步骤S3、根据所述第一特征点对所述人脸图像进行分块，形成互不重叠的一组子块。
其中，所述人脸图像分块是利用FACS(Facial Action Coding System，面部行为编码系统)对面部动作单元进行描述。并依据人脸特征点坐标将人脸划分出独立的、包含有效微表情信息的子块。且FACS是测量面部动作的一种方法。
在一个实施例中，基于所述第一特征点对应的所述人脸图像，通过FACS对不同的脸部肌肉动作与不同表情的对应关系。FACS根据人脸的解剖学特点，将人脸划分成若干既相互独立又相互联系的运动单元。然后分析这些运动单元的运动特征、其所控制的主要区域以及与之相关的表情，给出照片及照片说明。进而将所述人脸图像分为互补重叠的一组子块。
在一个实施例中，所述子块的数量不限，只要互不重叠即可。例如可以根据微表情中人脸面部器官将所述人脸图像划分为4个子块：眼睛区域、眉毛区域、颧骨区域、嘴巴区域，且这4个子块互不重叠。
步骤S4、提取每一所述子块对应的表示所述人脸图像的直方图特征向量。
在一个实施例中，在每一所述子块上，使用CPTOP(Cross  Patterns on three orthogonal planes，三个正交平面上的交叉模式)算子在灰度化的表情序列XY、XZ和YZ三个平面上提取直方图特征向量，并将从该三个平面上提取直方图特征向量级联成一个高维的特征向量作为所述CPTOP算子的特征向量。
其中，所述CPTOP算子是将微表情图像序列划分为XY、XZ和YZ三个正交平面，在各正交平面中使用纹理描述算子产生统计直方图。所述CPTOP算子在各正交平面中的采样模式，所述CPTOP算子在每个平面的采样点数目为8个，但是在每个采样方向上选择位于不同半径的两个采样点。
在一个实施例中，在每一所述子块上，使用CPTOP算子在灰度化的表情序列XY、XZ和YZ三个平面上提取直方图特征向量，然后级联成一个高维的特征向量作为所述CPTOP算子的特征向量。所述步骤包括：对于微表情图像序列中任一像素点O，在XY平面半径为RXYin、RXYex两个圆形邻域的四个方向(0，π，I)上分别采样，采用如下编码方式，I是对应点的灰度值；在XZ平面半径为RXTin、RXTex两个圆形邻域的四个方向上分别采样；在YZ平面半径为RYTin、RYTex两个圆形邻域的四个方向上分别采样；在各平面分别产生CPXY、CPXZ和CPYZ的统计直方图，然后三者级联形成较高维的向量，作为所述CPTOP算子的特征向量。
步骤S5、将所述直方图特征向量级联为高维特征向量，并将所述高维特征向量作为微表情的描述形式。
其中，微表情特征的描述形式指的是用数据进行微表情描述。
在一个实施例中，将所述各子块的直方图特征向量级联成高维的特征向量的步骤包括：在每个子块，使用所述CPTOP算子在灰度化的表情序列XY、XZ和YZ三个平面上提取直方图特征向量，然后级联成一个高维的特征向量作为所述CPTOP的特征向量；将上述各子块的直方图特征向量级联成高维的特征向量，作为微表情的描述形式。
在另一个实施例中，所述根据所述第一特征矩阵及所述第二特征矩阵，将所述后续帧与所述首帧对齐的步骤包括：将所述第一特征矩阵与所述第二特征矩阵进行比对，得到仿射变换矩阵；及参照所述仿射变换矩阵，将所述后续帧与所述首帧对齐。
在另一个实施例中，所述根据所述第一特征点对所述人脸图像进行分块，形成互不重叠的一组子块的步骤包括：获取与所述首帧对齐后的对齐矩阵；及根据预设的脸部肌肉动作与表情的对应关系，对所述对齐矩阵进行分块，得到与所述人脸图像对应的且互不重叠的一组子块。
在另一个实施例中，所述提取每一所述子块对应的表示所述 人脸图像的直方图特征向量的步骤包括：在每一所述子块上，对所述人脸图像进行灰度化处理；及将灰度化处理后的所述人脸图像置于XY、XZ、YZ三个平面上；及提取每一所述子块对应的直方图特征向量。
在另一个实施例中，所述将所述直方图特征向量级联为高维特征向量，并将所述高维特征向量作为微表情的描述形式的步骤，包括：获取每一所述子块对应的所述直方图特征向量；将每一所述子块的直方图特征向量进行级联，得到高维特征向量；及将级联后的高维特征向量作为微表情的描述形式。
综上所述，本申请所述的微表情描述方法通过扫描人脸图像，检测所述人脸图像的第一特征点，其中所述第一特征点为不受微表情影响的非共线特征点；将所述第一特征点组成的人脸图像作为首帧，并将检测到的后续帧与所述首帧对齐；根据所述第一特征点对所述人脸图像进行分块，形成互不重叠的一组子块；提取每一所述子块对应的表示所述人脸图像的直方图特征向量；及将所述直方图特征向量级联为高维特征向量，并将所述高维特征向量作为微表情的描述形式。从而，实现能够适应微表情时空特性和计算复杂度。
实施例二：
图2为本申请微表情描述装置较佳实施例的功能模块图。
参阅图2所示，微表情描述装置20可以包括扫描模块201、对齐模块202、分块模块203、提取模块204及级联模块205。
所述扫描模块201用于扫描人脸图像，检测所述人脸图像的第一特征点，其中所述第一特征点为不受微表情影响的非共线特征点。
本实施例中，所述扫描模块201通过判别式响应图拟合(Discriminative Response Map Fitting，简称DRMF)方法检测所述人脸图像的第一特征点。所述第一特征点为微表情对所述人脸图像影响最小的特征点，如人脸面部的鼻子、额头所在的特征点等。
在一个实施例中，所述扫描模块201通过以下方式实现扫描人脸图像，检测所述人脸图像的第一特征点，其中所述第一特征点为不受微表情影响的非共线特征点：所述扫描模块201获取首次扫描的人脸图像；基于所述人脸图像选取不易受微表情影响的3个非共线的特征点；将这3个非共线的特征点作为所述第一特征点；将所述第一特征点转换为三维空间中的坐标点，并参考三维空间的坐标系将这些坐标点整合为第一特征矩阵。以便于对所述人脸图像进行描述。
所述对齐模块202用于将所述第一特征点组成的人脸图像作为首帧，并将检测到的后续帧与所述首帧对齐。
本实施例中，所述对齐模块202以1秒为周期扫描所述人脸图像，将首次扫描到的第一特征点对应的人脸图像为首帧。将所述首帧之后扫描到的人脸图像作为后续帧。
在一个实施例中，所述对齐模块202获取所述后续帧的第二特征点。其中，所述第二特征点为所述人脸图像中不易受微表情影响的点，如鼻子，额头、耳朵等。通过所述第二特征点与所述第一特征点对齐，实现所述后续帧与所述首帧对齐。
在一个实施例中，所述对齐模块202将所述第二特征点置于三位空间中，从而将所述第二特征点整合为第二特征矩阵。将所述第二特征矩阵与所述第一特征矩阵进行比对。通过3个非共线的特征点对齐将所述第二特征矩阵换算为对齐所述第一特征矩阵的对齐矩阵，并在此对齐矩阵上进行后续操作。其中，所述第二特征点的数量为多个。
所述分块模块203用于根据所述第一特征点对所述人脸图像进行分块，形成互不重叠的一组子块。
其中，所述人脸图像分块是利用FACS(Facial Action Coding System，面部行为编码系统)对面部动作单元进行描述。并依据人脸特征点坐标将人脸划分出独立的、包含有效微表情信息的子块。且FACS是测量面部动作的一种方法。
在一个实施例中，所述分块模块203基于所述第一特征点对应的所述人脸图像，通过FACS对不同的脸部肌肉动作与不同表情的对应关系。FACS根据人脸的解剖学特点，将人脸划分成若干既相互独立又相互联系的运动单元。然后分析这些运动单元的运动特征、其所控制的主要区域以及与之相关的表情，给出照片及照片说明。进而将所述人脸图像分为互补重叠的一组子块。
在一个实施例中，所述子块的数量不限，只要互不重叠即可。例如可以根据微表情中人脸面部器官将所述人脸图像划分为4个子块：眼睛区域、眉毛区域、颧骨区域、嘴巴区域，且这4个子块互不重叠。
所述提取模块204用于提取每一所述子块对应的表示所述人脸图像的直方图特征向量。
在一个实施例中，所述分块模块203在每一所述子块上，使用CPTOP(Cross Patterns on three orthogonal planes，三个正交平面上的交叉模式)算子在灰度化的表情序列XY、XZ和YZ三个平面上提取直方图特征向量，并将从该三个平面上提取直方图特征向量级联成一个高维的特征向量作为所述CPTOP算子的特征向量。
其中，所述CPTOP算子是将微表情图像序列划分为XY、XZ和YZ三个正交平面，在各正交平面中使用纹理描述算子产生统计直方图。所述CPTOP算子在各正交平面中的采样模式，所述CPTOP算子在每个平面的采样点数目为8个，但是在每个采样方向上选择位于不同半径的两个采样点。
在一个实施例中，所述分块模块203在每一所述子块上，使用CPTOP算子在灰度化的表情序列XY、XZ和YZ三个平面上提取直方图特征向量，然后级联成一个高维的特征向量作为所述CPTOP算子的特征向量。具体实现方式可以是：对于微表情图像序列中任一像素点O，在XY平面半径为RXYin、RXYex两个圆形邻域的四个方向(0，π，I)上分别采样，采用如下编码方式，I是对应点的灰度值；在XZ平面半径为RXTin、RXTex两个圆形邻域的四个方向上分别采样；在YZ平面半径为RYTin、RYTex两个圆形邻域的四个方向上分别采样；在各平面分别产生CPXY、CPXZ和CPYZ的统计直方图，然后三者级联形成较高维的向量，作为所述CPTOP算子的特征向量。
所述级联模块205用于将所述直方图特征向量级联为高维特征向量，并将所述高维特征向量作为微表情的描述形式。
其中，微表情特征的描述形式指的是用数据进行微表情描述。
在一个实施例中，所述级联模块205可以通过以下方式实现将所述各子块的直方图特征向量级联成高维的特征向量：在每个子块，使用所述CPTOP算子在灰度化的表情序列XY、XZ和YZ三个平面上提取直方图特征向量，然后级联成一个高维的特征向量作为所述CPTOP的特征向量；将上述各子块的直方图特征向量级联成高维的特征向量，作为微表情的描述形式。
在另一个实施例中，所述根据所述第一特征矩阵及所述第二特征矩阵将所述后续帧与所述首帧对齐的实现方式可以是：将所述第一特征矩阵与所述第二特征矩阵进行比对，得到仿射变换矩阵；及参照所述仿射变换矩阵，将所述后续帧与所述首帧对齐。
在另一个实施例中，所述根据所述第一特征点对所述人脸图像进行分块，形成互不重叠的一组子块的具体实现方式可以是：获取与所述首帧对齐后的对齐矩阵；及根据预设的脸部肌肉动作与表情的对应关系，对所述对齐矩阵进行分块，得到与所述人脸图像对应的且互不重叠的一组子块。
在另一个实施例中，所述提取每一所述子块对应的表示所述人脸图像的直方图特征向量的实现方式可以是：在每一所述子块上，对所述人脸图像进行灰度化处理；及将灰度化处理后的所述人脸图像置于XY、XZ、YZ三个平面上；及提取每一所述子块对 应的直方图特征向量。
在另一个实施例中，所述将所述直方图特征向量级联为高维特征向量，并将所述高维特征向量作为微表情的描述形式的实现方式可以是：获取每一所述子块对应的所述直方图特征向量；将每一所述子块的直方图特征向量进行级联，得到高维特征向量；及将级联后的高维特征向量作为微表情的描述形式。
综上所述，本申请所述的微表情描述方法通过所述扫描模块201扫描人脸图像，检测所述人脸图像的第一特征点，其中所述第一特征点为不受微表情影响的非共线特征点；所述对齐模块202将所述第一特征点组成的人脸图像作为首帧，并将检测到的后续帧与所述首帧对齐；所述分块模块203根据所述第一特征点对所述人脸图像进行分块，形成互不重叠的一组子块；所述提取模块204提取每一所述子块对应的表示所述人脸图像的直方图特征向量；及所述级联模块205将所述直方图特征向量级联为高维特征向量，并将所述高维特征向量作为微表情的描述形式。从而，实现能够适应微表情时空特性和计算复杂度。
实施例三
图3为本申请计算机装置较佳实施例的示意图。
所述计算机装置30包括存储器31、处理器32以及存储在所述存储器31中并可在所述处理器32上运行的计算机可读指令33，例如微表情描述程序。所述处理器32执行所述计算机可读指令33时实现上述微表情描述方法实施例中的步骤，例如图1所示的步骤S1～S5。或者，所述处理器32执行所述计算机可读指令33时实现上述微表情描述装置实施例中各模块的功能，例如图2中的模块201～205。
示例性的，所述计算机可读指令33可以被分割成一个或多个模块/单元，所述一个或者多个模块/单元被存储在所述存储器31中，并由所述处理器32执行，以完成本申请。所述一个或多个模块/单元可以是能够完成特定功能的一系列计算机可读指令指令段，所述指令段用于描述所述计算机可读指令33在所述计算机装置30中的执行过程。例如，所述计算机可读指令33可以被分割成图2中的扫描模块201、对齐模块202、分块模块203、提取模块204及级联模块205。各模块具体功能参见实施例二。
所述计算机装置30可以是桌上型计算机、笔记本、掌上电脑及云端服务器等计算设备。本领域技术人员可以理解，所述示意图仅仅是计算机装置30的示例，并不构成对计算机装置30的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件，例如所述计算机装置30还可以包括输入输出设备、 网络接入设备、总线等。
所称处理器32可以是中央处理单元(Central Processing Unit，CPU)，还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者所述处理器32也可以是任何常规的处理器等，所述处理器32是所述计算机装置30的控制中心，利用各种接口和线路连接整个计算机装置30的各个部分。
所述存储器31可用于存储所述计算机可读指令33和/或模块/单元，所述处理器32通过运行或执行存储在所述存储器31内的计算机可读指令和/或模块/单元，以及调用存储在存储器31内的数据，实现所述计算机装置30的各种功能。所述存储器31可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序等；存储数据区可存储根据计算机装置30的使用所创建的数据等。此外，存储器31可以包括高速随机存取存储器，还可以包括非易失性存储器，例如硬盘、内存、插接式硬盘，智能存储卡(Smart Media Card,SMC)，安全数字(Secure Digital,SD)卡，闪存卡(Flash Card)、至少一个磁盘存储器件、闪存器件、或其他易失性固态存储器件。
所述计算机装置30集成的模块/单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请实现上述实施例方法中的全部或部分流程，也可以通过计算机可读指令来指令相关的硬件来完成，所述的计算机可读指令可存储于一非易失性可读存储介质中，所述计算机可读指令在被处理器执行时，可实现上述各个方法实施例的步骤。其中，所述计算机可读指令包括计算机可读指令代码，所述计算机可读指令代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。所述计算机可读介质可以包括：能够携带所述计算机可读指令代码的任何实体或装置、记录介质、U盘、移动硬盘、磁碟、光盘、计算机存储器、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、电载波信号、电信信号以及软件分发介质等。需要说明的是，所述计算机可读介质包含的内容可以根据司法管辖区内立法和专利实践的要求进行适当的增减，例如在某些司法管辖区，根据立法和专利实践，计算机可读介质不包括电载波信号和电信信号。
在本申请所提供的几个实施例中，应该理解到，所揭露的计算机装置和方法，可以通过其它的方式实现。例如，以上所描述的计算机装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
另外，在本申请各个实施例中的各功能单元可以集成在相同处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在相同单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用硬件加软件功能模块的形式实现。
对于本领域技术人员而言，显然本申请不限于上述示范性实施例的细节，而且在不背离本申请的精神或基本特征的情况下，能够以其他的具体形式实现本申请。因此，无论从哪一点来看，均应将实施例看作是示范性的，而且是非限制性的，本申请的范围由所附权利要求而不是上述说明限定，因此旨在将落在权利要求的等同要件的含义和范围内的所有变化涵括在本申请内。不应将权利要求中的任何附图标记视为限制所涉及的权利要求。此外，显然“包括”一词不排除其他单元或步骤，单数不排除复数。计算机装置权利要求中陈述的多个单元或计算机装置也可以由同一个单元或计算机装置通过软件或者硬件来实现。第一，第二等词语用来表示名称，而并不表示任何特定的顺序。
最后应说明的是，以上实施例仅用以说明本申请的技术方案而非限制，尽管参照较佳实施例对本申请进行了详细说明，本领域的普通技术人员应当理解，可以对本申请的技术方案进行修改或等同替换，而不脱离本申请技术方案的精神和范围。

Claims (20)

  1. 一种微表情描述方法，其特征在于，所述方法包括：
    扫描人脸图像，检测所述人脸图像的第一特征点，其中所述第一特征点为不受微表情影响的非共线特征点；
    将所述第一特征点组成的人脸图像作为首帧，并将检测到的后续帧与所述首帧对齐；
    根据所述第一特征点对所述人脸图像进行分块，形成互不重叠的一组子块；
    提取每一所述子块对应的表示所述人脸图像的直方图特征向量；及
    将所述直方图特征向量级联为高维特征向量，并将所述高维特征向量作为微表情的描述形式。
  2. 如权利要求1所述的微表情描述方法，其特征在于，所述扫描人脸图像，检测所述人脸图像的第一特征点，其中所述第一特征点为不受微表情影响的非共线特征点的步骤包括：
    扫描人脸图像，检测首次扫描的所述人脸图像中不受微表情影响的非共线特征点；及
    提取所述非共线特征点，并将所述非共线特征点作为第一特征点。
  3. 如权利要求1所述的微表情描述方法，其特征在于，所述将所述第一特征点组成的人脸图像作为首帧，并将检测到的后续帧与所述首帧对齐的步骤包括：
    将所述第一特征点组成第一特征矩阵，并将所述第一特征矩阵作为所述人脸图像的首帧；
    获取所述人脸图像的后续帧对应的第二特征矩阵；及
    根据所述第一特征矩阵及所述第二特征矩阵，将所述后续帧与所述首帧对齐。
  4. 如权利要求3所述的微表情描述方法，其特征在于，所述根据所述第一特征矩阵及所述第二特征矩阵，将所述后续帧与所述首帧对齐的步骤包括：
    将所述第一特征矩阵与所述第二特征矩阵进行比对，得到仿射变换矩阵；及
    参照所述仿射变换矩阵，将所述后续帧与所述首帧对齐。
  5. 如权利要求1所述的微表情描述方法，其特征在于，所述根据所述第一特征点对所述人脸图像进行分块，形成互不重叠的一组子块的步骤包括：
    获取与所述首帧对齐后的对齐矩阵；及
    根据预设的脸部肌肉动作与表情的对应关系，对所述对齐矩阵进行分块，得到与所述人脸图像对应的且互不重叠的一组子块。
  6. 如权利要求1所述的微表情描述方法，其特征在于，所述提取每一所述子块对应的表示所述人脸图像的直方图特征向量的步骤包括：
    在每一所述子块上，对所述人脸图像进行灰度化处理；及
    将灰度化处理后的所述人脸图像置于XY、XZ、YZ三个平面上；及
    提取每一所述子块对应的直方图特征向量。
  7. 如权利要求6所述的微表情描述方法，其特征在于，所述将所述直方图特征向量级联为高维特征向量，并将所述高维特征向量作为微表情的描述形式的步骤包括：
    获取每一所述子块对应的所述直方图特征向量；
    将每一所述子块的直方图特征向量进行级联，得到高维特征向量；及
    将级联后的高维特征向量作为微表情的描述形式。
  8. 一种微表情描述装置，其特征在于，所述装置包括：
    扫描模块，用于扫描人脸图像，检测所述人脸图像的第一特征点，其中所述第一特征点为不受微表情影响的非共线特征点；
    对齐模块，用于将所述第一特征点组成的人脸图像作为首帧，并将检测到的后续帧与所述首帧对齐；
    分块模块，用于根据所述第一特征点对所述人脸图像进行分块，形成互不重叠的一组子块；
    提取模块，用于提取每一所述子块对应的表示所述人脸图像的直方图特征向量；及
    级联模块，用于将所述直方图特征向量级联为高维特征向量，并将所述高维特征向量作为微表情的描述形式。
  9. 一种计算机装置，其特征在于，所述计算机装置包括处理器和存储器，所述处理器用于执行所述存储器中存储的计算机可读指令时实现以下步骤：
    扫描人脸图像，检测所述人脸图像的第一特征点，其中所述第一特征点为不受微表情影响的非共线特征点；
    将所述第一特征点组成的人脸图像作为首帧，并将检测到的后续帧与所述首帧对齐；
    根据所述第一特征点对所述人脸图像进行分块，形成互不重叠的一组子块；
    提取每一所述子块对应的表示所述人脸图像的直方图特征向量；及
    将所述直方图特征向量级联为高维特征向量，并将所述高维特征向量作为微表情的描述形式。
  10. 如权利要求9所述的计算机装置，其特征在于，所述扫描人脸图像，检测所述人脸图像的第一特征点，其中所述第一特征点为不受微表情影响的非共线特征点的步骤包括：
    扫描人脸图像，检测首次扫描的所述人脸图像中不受微表情影响的非共线特征点；及
    提取所述非共线特征点，并将所述非共线特征点作为第一特征点。
  11. 如权利要求9所述的计算机装置，其特征在于，所述将所述第一特征点组成的人脸图像作为首帧，并将检测到的后续帧与所述首帧对齐的步骤包括：
    将所述第一特征点组成第一特征矩阵，并将所述第一特征矩阵作为所述人脸图像的首帧；
    获取所述人脸图像的后续帧对应的第二特征矩阵；及
    根据所述第一特征矩阵及所述第二特征矩阵，将所述后续帧与所述首帧对齐。
  12. 如权利要求11所述的计算机装置，其特征在于，所述根据所述第一特征矩阵及所述第二特征矩阵，将所述后续帧与所述首帧对齐的步骤包括：
    将所述第一特征矩阵与所述第二特征矩阵进行比对，得到仿射变换矩阵；及
    参照所述仿射变换矩阵，将所述后续帧与所述首帧对齐。
  13. 如权利要求9所述的计算机装置，其特征在于，所述根据所述第一特征点对所述人脸图像进行分块，形成互不重叠的一组子块的步骤包括：
    获取与所述首帧对齐后的对齐矩阵；及
    根据预设的脸部肌肉动作与表情的对应关系，对所述对齐矩阵进行分块，得到与所述人脸图像对应的且互不重叠的一组子块。
  14. 如权利要求9所述的计算机装置，其特征在于，所述提取每一所述子块对应的表示所述人脸图像的直方图特征向量的步骤包括：
    在每一所述子块上，对所述人脸图像进行灰度化处理；及
    将灰度化处理后的所述人脸图像置于XY、XZ、YZ三个平面上；及
    提取每一所述子块对应的直方图特征向量。
  15. 如权利要求14所述的计算机装置，其特征在于，所述将所述直方图特征向量级联为高维特征向量，并将所述高维特征向 量作为微表情的描述形式的步骤包括：
    获取每一所述子块对应的所述直方图特征向量；
    将每一所述子块的直方图特征向量进行级联，得到高维特征向量；及
    将级联后的高维特征向量作为微表情的描述形式。
  16. 一种非易失性可读存储介质，所述非易失性可读存储介质上存储有计算机可读指令，其特征在于，所述计算机可读指令被处理器执行时实现以下步骤：
    扫描人脸图像，检测所述人脸图像的第一特征点，其中所述第一特征点为不受微表情影响的非共线特征点；
    将所述第一特征点组成的人脸图像作为首帧，并将检测到的后续帧与所述首帧对齐；
    根据所述第一特征点对所述人脸图像进行分块，形成互不重叠的一组子块；
    提取每一所述子块对应的表示所述人脸图像的直方图特征向量；及
    将所述直方图特征向量级联为高维特征向量，并将所述高维特征向量作为微表情的描述形式。
  17. 如权利要求16所述的非易失性可读存储介质，其特征在于，所述将所述第一特征点组成的人脸图像作为首帧，并将检测到的后续帧与所述首帧对齐的步骤包括：
    将所述第一特征点组成第一特征矩阵，并将所述第一特征矩阵作为所述人脸图像的首帧；
    获取所述人脸图像的后续帧对应的第二特征矩阵；及
    根据所述第一特征矩阵及所述第二特征矩阵，将所述后续帧与所述首帧对齐。
  18. 如权利要求17所述的非易失性可读存储介质，其特征在于，所述根据所述第一特征矩阵及所述第二特征矩阵，将所述后续帧与所述首帧对齐的步骤包括：
    将所述第一特征矩阵与所述第二特征矩阵进行比对，得到仿射变换矩阵；及
    参照所述仿射变换矩阵，将所述后续帧与所述首帧对齐。
  19. 如权利要求16所述的非易失性可读存储介质，其特征在于，所述提取每一所述子块对应的表示所述人脸图像的直方图特征向量的步骤包括：
    在每一所述子块上，对所述人脸图像进行灰度化处理；及
    将灰度化处理后的所述人脸图像置于XY、XZ、YZ三个平面上；及
    提取每一所述子块对应的直方图特征向量。
  20. 如权利要求19所述的非易失性可读存储介质，其特征在于，所述将所述直方图特征向量级联为高维特征向量，并将所述高维特征向量作为微表情的描述形式的步骤包括：
    获取每一所述子块对应的所述直方图特征向量；
    将每一所述子块的直方图特征向量进行级联，得到高维特征向量；及
    将级联后的高维特征向量作为微表情的描述形式。
PCT/CN2019/092144 2018-12-13 2019-06-20 微表情描述方法、装置、计算机装置及可读存储介质 Ceased WO2020119058A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201811527706.8A CN109815793A (zh) 2018-12-13 2018-12-13 微表情描述方法、装置、计算机装置及可读存储介质
CN201811527706.8 2018-12-13

Publications (1)

Publication Number Publication Date
WO2020119058A1 true WO2020119058A1 (zh) 2020-06-18

Family

ID=66602938

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/092144 Ceased WO2020119058A1 (zh) 2018-12-13 2019-06-20 微表情描述方法、装置、计算机装置及可读存储介质

Country Status (2)

Country Link
CN (1) CN109815793A (zh)
WO (1) WO2020119058A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113920569A (zh) * 2021-11-06 2022-01-11 北京九州安华信息安全技术有限公司 基于特征差异的微表情顶点定位方法及装置
CN114220154A (zh) * 2021-12-20 2022-03-22 王越 一种基于深度学习的微表情特征提取与识别方法
CN116758613A (zh) * 2023-06-15 2023-09-15 平安科技（深圳）有限公司 基于人工智能的微表情检测方法及相关设备

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109815793A (zh) * 2018-12-13 2019-05-28 平安科技（深圳）有限公司 微表情描述方法、装置、计算机装置及可读存储介质
CN111178262A (zh) * 2019-12-30 2020-05-19 中国电子科技集团公司电子科学研究院 微表情检测方法、装置及计算机可读存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104732216A (zh) * 2015-03-26 2015-06-24 江苏物联网研究发展中心 基于关键点和局部特征的表情识别方法
CN105913038A (zh) * 2016-04-26 2016-08-31 哈尔滨工业大学深圳研究生院 一种基于视频的动态微表情识别方法
CN106096537A (zh) * 2016-06-06 2016-11-09 山东大学 一种基于多尺度采样的微表情自动识别方法
CN106127196A (zh) * 2016-09-14 2016-11-16 河北工业大学 基于动态纹理特征的人脸表情的分类与识别方法
CN109815793A (zh) * 2018-12-13 2019-05-28 平安科技（深圳）有限公司 微表情描述方法、装置、计算机装置及可读存储介质

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104298981A (zh) * 2014-11-05 2015-01-21 河北工业大学 人脸微表情的识别方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104732216A (zh) * 2015-03-26 2015-06-24 江苏物联网研究发展中心 基于关键点和局部特征的表情识别方法
CN105913038A (zh) * 2016-04-26 2016-08-31 哈尔滨工业大学深圳研究生院 一种基于视频的动态微表情识别方法
CN106096537A (zh) * 2016-06-06 2016-11-09 山东大学 一种基于多尺度采样的微表情自动识别方法
CN106127196A (zh) * 2016-09-14 2016-11-16 河北工业大学 基于动态纹理特征的人脸表情的分类与识别方法
CN109815793A (zh) * 2018-12-13 2019-05-28 平安科技（深圳）有限公司 微表情描述方法、装置、计算机装置及可读存储介质

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113920569A (zh) * 2021-11-06 2022-01-11 北京九州安华信息安全技术有限公司 基于特征差异的微表情顶点定位方法及装置
CN114220154A (zh) * 2021-12-20 2022-03-22 王越 一种基于深度学习的微表情特征提取与识别方法
CN114220154B (zh) * 2021-12-20 2025-02-11 天津大学 一种基于深度学习的微表情特征提取与识别方法
CN116758613A (zh) * 2023-06-15 2023-09-15 平安科技（深圳）有限公司 基于人工智能的微表情检测方法及相关设备

Also Published As

Publication number Publication date
CN109815793A (zh) 2019-05-28

Similar Documents

Publication Publication Date Title
Hsu et al. Ratio-and-scale-aware YOLO for pedestrian detection
CN109359538B (zh) 卷积神经网络的训练方法、手势识别方法、装置及设备
WO2020119058A1 (zh) 微表情描述方法、装置、计算机装置及可读存储介质
US11069151B2 (en) Methods and devices for replacing expression, and computer readable storage media
CN110675487B (zh) 基于多角度二维人脸的三维人脸建模、识别方法及装置
CN110852310B (zh) 三维人脸识别方法、装置、终端设备及计算机可读介质
CN108765278B (zh) 一种图像处理方法、移动终端及计算机可读存储介质
CN102667810B (zh) 数字图像中的面部识别
WO2021056705A1 (zh) 基于语义分割网络的人体外表损伤检测方法及相关设备
US11354883B2 (en) Image processing method and apparatus, and electronic device
CN113642428B (zh) 人脸活体检测方法、装置、电子设备及存储介质
JP5361524B2 (ja) パターン認識システム及びパターン認識方法
CN113221771A (zh) 活体人脸识别方法、装置、设备、存储介质及程序产品
CN110991258B (zh) 一种人脸融合特征提取方法及系统
CN112949437A (zh) 一种手势识别方法、手势识别装置及智能设备
WO2019119396A1 (zh) 人脸表情识别方法及装置
CN114758145B (zh) 一种图像脱敏方法、装置、电子设备及存储介质
KR102421604B1 (ko) 이미지 처리 방법, 장치 및 전자 기기
CN113012030A (zh) 图像拼接方法、装置及设备
WO2022166258A1 (zh) 行为识别方法、装置、终端设备及计算机可读存储介质
CN110321908A (zh) 图像识别方法、终端设备及计算机可读存储介质
CN111144374B (zh) 人脸表情识别方法及装置、存储介质和电子设备
CN113435358B (zh) 用于训练模型的样本生成方法、装置、设备、程序产品
CN115359490A (zh) 文本方向识别方法、装置、设备及存储介质
CN117746502B (zh) 图像标注方法、动作识别方法、装置和电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19896006

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19896006

Country of ref document: EP

Kind code of ref document: A1