WO2017164478A1 - Procédé et appareil de reconnaissance de micro-expressions au moyen d'une analyse d'apprentissage profond d'une dynamique micro-faciale - Google Patents

Procédé et appareil de reconnaissance de micro-expressions au moyen d'une analyse d'apprentissage profond d'une dynamique micro-faciale Download PDF

Info

Publication number
WO2017164478A1
WO2017164478A1 PCT/KR2016/012772 KR2016012772W WO2017164478A1 WO 2017164478 A1 WO2017164478 A1 WO 2017164478A1 KR 2016012772 W KR2016012772 W KR 2016012772W WO 2017164478 A1 WO2017164478 A1 WO 2017164478A1
Authority
WO
WIPO (PCT)
Prior art keywords
spatial
learning
frames
fine
learning model
Prior art date
Application number
PCT/KR2016/012772
Other languages
English (en)
Korean (ko)
Inventor
노용만
김대회
Original Assignee
한국과학기술원
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from KR1020160063555A external-priority patent/KR102036955B1/ko
Application filed by 한국과학기술원 filed Critical 한국과학기술원
Publication of WO2017164478A1 publication Critical patent/WO2017164478A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Definitions

  • the present invention relates to a fine target recognition technology, and more particularly, to a method and an apparatus capable of recognizing a fine expression through deep learning analysis of fine face dynamics.
  • Facial analysis includes biometrics, security, human-computer interaction, and more recently, healthcare, smart home control, and human sensing to understand and recognize human emotions. It is widely attracting attention in a very wide field.
  • Fine face dynamic information is distributed in a few milliseconds that is difficult to visually identify.
  • This fine facial dynamic is caused by intentional or unintentional facial muscle movement and contains important information such as facial expression, facial recognition, and facial condition detection.
  • dynamics on the microscopic time scale which are invisible to the naked eye, can provide critical information that cannot be provided in the visible areas, such as intrinsic feature extraction or spontaneous facial emotions, which are useful for human identification. have.
  • Embodiments of the present invention provide a method and apparatus capable of recognizing a fine expression through deep learning analysis of fine face dynamics.
  • embodiments of the present invention provide a method and apparatus for analyzing a fine facial dynamic feature in a video including a face using deep learning and recognizing a facial expression using the deep learning.
  • a method of learning a fine facial expression extracts frames of predefined fine expressions from an input video, and generates a spatial learning model by learning spatial features of the extracted frames. ; And extracting the spatial feature of the frames of the input video using the generated spatial learning model, and generating the temporal learning model using the extracted spatial feature of the frames. Learning each of them.
  • the generating of the spatial learning model includes a classification error minimization function, a variance minimization function in the same class in the feature space, a facial expression state classification error minimization function, a variance minimization function in the facial expression state in the feature space, and preserving the continuity of the facial expression state in the feature space.
  • the spatial learning model may be generated by learning spatial features of the extracted frames using five objective functions of the function.
  • the generating of the spatial learning model may generate the spatial learning model by learning a convolutional neural network (CNN) with respect to the spatial features of the extracted frames.
  • CNN convolutional neural network
  • the learning of each of the fine expressions may learn each of the fine expressions by generating the temporal learning model using the spatial feature extracted for the frames based on a recursive neural network.
  • the recursive neural network may include at least one of a recurrent neural network (RNN), a gated recurrent unit (GRU), and a long short-term memory (LSTM).
  • RNN recurrent neural network
  • GRU gated recurrent unit
  • LSTM long short-term memory
  • a method of recognizing a fine facial expression comprising: extracting spatial features of frames of a video using a spatial learning model that learns spatial features of predefined fine expressions; Calculating a recognition value for each of the fine expressions based on a recursive neural network using the spatial features extracted for the frames and a pre-trained temporal learning model; And recognizing a fine expression in the video based on the calculated recognition value.
  • the recursive neural network may include at least one of a recurrent neural network (RNN), a gated recurrent unit (GRU), and a long short-term memory (LSTM).
  • RNN recurrent neural network
  • GRU gated recurrent unit
  • LSTM long short-term memory
  • the apparatus for fine expression learning is a space for extracting frames for predefined fine expressions from an input video, and learning a spatial feature of the extracted frames to generate a spatial learning model.
  • Learning unit And extracting the spatial feature of the frames of the input video using the generated spatial learning model, and generating the temporal learning model using the extracted spatial feature of the frames. It includes a time learning unit for learning each of them.
  • the spatial learning unit has five objective functions: a classification error minimization function, a variance minimization function in the same class in the feature space, a facial expression state classification error minimization function, a variance minimization function in the facial expression state in the feature space, and a continuity preservation function of the facial expression state in the feature space.
  • the spatial learning model may be generated by learning spatial features of the extracted frames.
  • the spatial learner may generate the spatial learning model by learning a convolutional neural network (CNN) with respect to the spatial features of the extracted frames.
  • CNN convolutional neural network
  • the temporal learning unit may learn each of the fine expressions by generating the temporal learning model using the spatial features extracted for the frames based on a recursive neural network.
  • the recursive neural network may include at least one of a recurrent neural network (RNN), a gated recurrent unit (GRU), and a long short-term memory (LSTM).
  • RNN recurrent neural network
  • GRU gated recurrent unit
  • LSTM long short-term memory
  • An apparatus for recognizing fine expressions includes an extractor which extracts spatial features of frames of a video by using a spatial learning model that learns spatial features of predefined fine expressions; A calculator configured to calculate a recognition value for each of the fine expressions based on a recursive neural network using the spatial features extracted for the frames and a pre-trained temporal learning model; And a recognition unit recognizing a fine expression in the video based on the calculated recognition value.
  • the recursive neural network may include at least one of a recurrent neural network (RNN), a gated recurrent unit (GRU), and a long short-term memory (LSTM).
  • RNN recurrent neural network
  • GRU gated recurrent unit
  • LSTM long short-term memory
  • a deep facial dynamic feature in a video including a face may be analyzed using deep learning, and facial expression may be recognized using the deep learning.
  • an efficient fine facial expression video recognition system framework may be configured.
  • the microscopic movement of a person may be analyzed and captured and widely applied in various fields such as medicine, psychology, human-computer interaction and multimedia, entertainment, human sensing, and the like.
  • FIG. 1 illustrates an exemplary diagram for describing a method of learning a micro expression through deep learning analysis of micro facial dynamics according to an exemplary embodiment of the present invention.
  • Figure 2 shows a conceptual illustration of the objective function of the facial expression state emphasis learning method in the method according to the present invention.
  • FIG. 3 is a conceptual illustration of recursive neural network based dynamic sequence analysis in a method according to the present invention.
  • FIG. 4 is a flowchart illustrating an operation of a method for recognizing a fine facial expression according to an exemplary embodiment of the present invention.
  • Figure 5 shows the configuration of the apparatus for learning a fine expression in one embodiment of the present invention.
  • FIG. 6 is a diagram illustrating a configuration of an apparatus for recognizing fine expressions in an embodiment of the present invention.
  • Embodiments of the present invention are intended to analyze the fine facial dynamics in a video including a face using deep learning and to efficiently recognize facial expressions using the same.
  • FIG. 1 illustrates an exemplary diagram for describing a method of learning a micro expression through deep learning analysis of micro facial dynamics according to an exemplary embodiment of the present invention.
  • the method for learning fine expressions is composed of an expression state emphasized learning process and a dynamic sequence analysis using recurrent neural network process. This will be described as follows.
  • the facial expression state emphasis learning process may perform the facial expression state emphasis learning process in a convolution neural network (CNN), and a five-step facial expression state that is predefined in each input video, for example, long video and short video, for example, onset Only onset-to-apex, apex, apex-to-offset, and offset images can be sampled and learned.
  • CNN convolution neural network
  • the facial expression state emphasizing learning process may generate a spatial learning model by extracting frames for a five-stage facial expression state from the input video and learning a CNN about spatial features of the extracted frames.
  • the classification error minimization function the variance minimization function in the same class in the feature space
  • the facial expression state classification error minimization function the variance minimization function in the facial expression state in the feature space
  • the continuity preservation function of the facial expression state in the feature space The spatial learning model can be generated by learning the spatial features of the extracted frames using the two objective functions, which are described in FIG. 2.
  • the recursive neural network based dynamic sequence analysis process uses the spatial learning model generated by the facial expression state emphasis learning process to extract spatial features of all frames of the input video, and extracts the extracted spatial features of all frames. By using this to generate a temporal learning model, each of the fine expressions is learned.
  • the recursive neural network based dynamic sequence analysis process generates a temporal learning model using the spatial features extracted for all frames based on the recursive neural network, thereby learning each of the fine expressions. and at least one of a recurrent neural network (GRU), a gated recurrent unit (GRU), and a long short-term memory (LSTM).
  • GRU recurrent neural network
  • GRU gated recurrent unit
  • LSTM long short-term memory
  • Dynamic sequence analysis process based on recursive neural network is to learn recursive neural network based on time axis.
  • deep learning (or learning) technology is used to learn features that enable analysis of fine facial motion changes.
  • five levels of expression states for each expression for example, onset, onset-to-apex, apex, apex-to-offset, and offset, in order to have discernment in the change of minute movements
  • each sample video is trained by sampling only images (or frames only) corresponding to the five-stage facial expression state, and based on a recurrent neural network (RNN) as a test or second stage.
  • RNN recurrent neural network
  • five objective functions may be used to increase the difference between the motion states in the feature space, and the five objective functions will be described with reference to FIG. 2.
  • FIG. 2 is a conceptual illustration of the objective function of the facial expression state emphasis learning method in the method according to the present invention.
  • five objective functions are classified in a classification space minimization (E1) in a feature space.
  • Each color illustrated in FIG. 2 may mean a kind of facial expression, and a shape may mean facial expression state, and each function will be described below.
  • the minimizing expression classification error (E1) function is a function for minimizing the classification error in classifying each fine expression, that is, the class.
  • the classification error minimization function is expressed as in Equation 1 below. Can be.
  • c is the class index
  • i is the index of the training sample
  • t c i is the true value of the sample (1 if the class of sample i is c and 0 otherwise)
  • Equation 2 The minimizing intra-class variation (E2) function in the same class in the feature space may be expressed as Equation 2 below.
  • y c, p, i means the feature vector for the sample x c, p, i , m c means the average vector of the feature vector of the learning samples of class c, d c min is different from It can mean half the distance from the nearest class among the classes.
  • Equation 3 Minimizing expression state classification error (E3) function can be expressed as in Equation 3 below.
  • p denotes the facial expression state index
  • t p i denotes the true facial expression state value of the sample (1 only when the facial expression state index of sample i is p and 0 otherwise)
  • Equation 4 The minimizing expression state variation (E4) function in the facial expression state in the feature space may be expressed as Equation 4 below.
  • m c, p may mean an average vector of feature vectors of the learning samples belonging to the facial expression state p of the facial expression class c
  • ⁇ may mean a parameter for determining a distribution range of the facial expression state.
  • the preserving expression state continuity (E5) function in the feature space allows expressions existing between two expression states among the five levels of expression state used for learning to exist between two expression states in the feature space. This is related to the second stage of dynamic sequence analysis.
  • the E5 function can make a frame exist between apex-to-offset and offset for a frame that exists between apex-to-offset and offset.
  • the continuity preservation (E5) function of the facial expression state in the feature space can be expressed as Equation 5 below.
  • the dynamic feature or spatial feature learned by the expression state emphasis learning process makes the dynamic sequence analysis easier through the recursive neural network in the second stage by increasing the difference according to the motion state in the feature space, and describes the dynamic sequence analysis. Is as follows.
  • the facial features extracted in the first step are analyzed only for the fine motion of each frame, so it is necessary to analyze the change of the fine motion over time in the entire video.
  • the second step involves recursive neural network-based face dynamic modeling and analysis.
  • Recursive neural network-based dynamic sequence analysis uses recursive neural networks to model various feature changes that appear in fine motion from a series of sequential input frames.
  • the recursive neural network in the present invention may be used a simple RNN, a gated recurrent unit (GRU), a long short-term memory (LSTM), etc.
  • the recursive neural network based dynamic sequence analysis shown in Figure 3 is an example using LTSM It is shown.
  • recursive neural network based dynamic sequence analysis uses an expression state emphasized CNN model to spatially space all frames (onset to offset) of the input video.
  • Each of the microscopic expressions can be learned by extracting a typical feature and generating a temporal learning model using a recursive neural network, for example, LTSM, using the extracted spatial features for all frames.
  • a recursive neural network for example, LTSM
  • the spatial learning model and the temporal learning model learned by the above-described process may be used to recognize the fine expression of the video to recognize the fine expression.
  • the method of fine expression learning extracts the features of every frame of a video through a spatial learning model emphasizing the facial expression state learned in the first step, and changes the time between all frames based on the recursive neural network.
  • a spatial learning model emphasizing the facial expression state learned in the first step, and changes the time between all frames based on the recursive neural network.
  • the spatial learning model and the temporal learning model generated by the fine expression learning method may be used to recognize the fine expression in the video to recognize the fine expression, which will be described with reference to FIG. 4.
  • FIG. 4 is a flowchart illustrating a method for recognizing a fine facial expression according to an exemplary embodiment of the present invention.
  • the method for recognizing a fine facial expression uses all frames of a video, for example, an onset frame or an offset frame, to recognize the fine facial expression using a previously learned spatial learning model. Extract the spatial features (S410).
  • step S410 When the spatial feature of each of the frames is extracted by step S410, the recognition value of each of the fine expressions defined in advance based on the recursive neural network using the extracted spatial feature and the pre-trained temporal learning model for all the frames is extracted. Calculate (S420).
  • the recognition value of each of the fine expressions may be a probability value of each of the fine expressions.
  • step S420 When the recognition value for each of the fine expressions is calculated in step S420, the fine expression of the corresponding video is recognized based on the calculated recognition value (S430).
  • the fine expression having the largest value among the recognition values calculated for each of the fine expressions may be recognized as the fine expression of the corresponding video.
  • the spatial learning model pre-learned by emphasizing the spatial state of all frames (onset to offset) of the video to recognize the micro-expression ( A temporal learning model and a recursive neural network pre-trained on each of the microexpressions with respect to the extracted spatial features. analysis result) is recognized as a fine expression of the video.
  • the method for recognizing the micro expression according to the present invention can easily extract the micro expression by extracting unique features useful for human identification or recognition of spontaneous facial emotion by constructing a neural network for fine facial movements which cannot be identified by the naked eye. have.
  • the methods according to the present invention can recognize a fine expression by providing a learning method considering a facial expression state and a fine expression recognition framework through deep learning analysis of fine face dynamics.
  • the present invention provides a recursive neural network-based learning method based on a time axis and provides a result of learning the spatial information of the facial expression, and recognizes the fine expression of the video using the learning model generated by the learned method. .
  • the method for recognizing a fine facial expression according to the present invention can recognize a fine facial expression by fusing facial spatial information, time information, and motion information, and can perform effective facial recognition in consideration of fine facial dynamics through the present invention.
  • FIG. 5 illustrates a configuration of a micro expression learning apparatus in an embodiment of the present invention, and illustrates a configuration of an apparatus for performing the method of FIGS. 1 to 3.
  • the micro-expression learning apparatus 500 includes a spatial learner 510 and a time learner 520.
  • the spatial learner 510 extracts frames of predefined fine expressions from the input video, and learns spatial features of the extracted frames to generate a spatial learning model.
  • the spatial learning unit 510 minimizes a classification error minimizing function, a variance minimizing function in the same class in the feature space, a facial expression state classification error minimizing function, a variance minimizing function in the facial expression state in the feature space, and preserves the continuity of the facial expression state in the feature space.
  • Spatial learning model can be generated by learning spatial features of the extracted frames using five objective functions of the function, and spatial learning by learning CNN about the spatial features of the extracted frames. You can create a model.
  • the temporal learning unit 520 extracts the spatial features of all the frames of the input video using the generated spatial learning model, and generates the temporal learning model using the spatial features extracted for all the frames. Thereby learning each of the fine expressions.
  • the temporal learning unit 520 can learn each of the fine expressions by generating a temporal learning model using spatial features extracted for all frames based on the recursive neural network, and the recursive neural network is an RNN. and at least one of a recurrent neural network (GRU), a gated recurrent unit (GRU), and a long short-term memory (LSTM).
  • GRU recurrent neural network
  • GRU gated recurrent unit
  • LSTM long short-term memory
  • the apparatus for learning fine expressions may include not only the contents described with reference to FIG. 5 but also the contents of FIGS. 1 to 3 described above.
  • FIG. 6 illustrates a configuration of an apparatus for recognizing fine expressions in an embodiment of the present invention, and illustrates a configuration of an apparatus for performing the method of FIG. 4.
  • the apparatus for recognizing micro expressions 600 includes an extractor 610, a calculator 620, and a recognizer 630.
  • the extractor 610 extracts the spatial features of all the frames of the video to recognize the fine expression using a spatial learning model that learns the spatial features of the predefined fine expressions.
  • the calculator 620 calculates a recognition value for each of the fine expressions based on a recursive neural network using a spatial feature extracted for all frames and a pre-trained temporal learning model.
  • the calculator 620 may calculate a probability value for each of the fine expressions.
  • the recognition unit 630 recognizes the fine expression in the video based on the calculated recognition value.
  • the recognition unit 630 may recognize the fine expression having the largest value among the recognition values calculated for each of the fine expressions as the fine expression of the corresponding video.
  • the apparatus for recognizing a fine facial expression may include not only the contents of FIG. 6 but also the contents of FIGS. 1 to 4.
  • the system or apparatus described above may be implemented with hardware components, software components, and / or combinations of hardware components and software components.
  • the systems, devices, and components described in the embodiments may include, for example, processors, controllers, arithmetic logic units (ALUs), digital signal processors, microcomputers, field programmable arrays (FPAs). ), A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers.
  • the processing device may execute an operating system (OS) and one or more software applications running on the operating system.
  • the processing device may also access, store, manipulate, process, and generate data in response to the execution of the software.
  • OS operating system
  • the processing device may also access, store, manipulate, process, and generate data in response to the execution of the software.
  • processing device includes a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that it may include.
  • the processing device may include a plurality of processors or one processor and one controller.
  • other processing configurations are possible, such as parallel processors.
  • the software may include a computer program, code, instructions, or a combination of one or more of the above, and configure the processing device to operate as desired, or process it independently or collectively. You can command the device.
  • Software and / or data may be any type of machine, component, physical device, virtual equipment, computer storage medium or device in order to be interpreted by or to provide instructions or data to the processing device. Or may be permanently or temporarily embodied in a signal wave to be transmitted.
  • the software may be distributed over networked computer systems so that they may be stored or executed in a distributed manner.
  • Software and data may be stored on one or more computer readable recording media.
  • the method according to the embodiments may be embodied in the form of program instructions that may be executed by various computer means and recorded on a computer readable medium.
  • the computer readable medium may include program instructions, data files, data structures, etc. alone or in combination.
  • the program instructions recorded on the media may be those specially designed and constructed for the purposes of the embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts.
  • Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks.
  • Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.
  • the hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)

Abstract

L'invention concerne un procédé et un appareil permettant de reconnaître des micro-expressions au moyen d'une analyse d'apprentissage profond d'une dynamique micro-faciale. Selon un mode de réalisation de l'invention, un procédé d'apprentissage de micro-expressions consiste à : extraire des trames pour des micro-expressions prédéfinies dans une vidéo d'entrée et apprendre des caractéristiques spatiales pour les trames extraites de façon à générer un modèle d'apprentissage spatial; et extraire des caractéristiques spatiales pour toutes les trames de la vidéo d'entrée à l'aide du modèle d'apprentissage spatial généré, puis générer un modèle d'apprentissage temporel à l'aide des caractéristiques spatiales extraites pour toutes les trames, ce qui permet d'apprendre chacune des micro-expressions.
PCT/KR2016/012772 2016-03-25 2016-11-08 Procédé et appareil de reconnaissance de micro-expressions au moyen d'une analyse d'apprentissage profond d'une dynamique micro-faciale WO2017164478A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
KR20160036064 2016-03-25
KR10-2016-0036064 2016-03-25
KR10-2016-0063555 2016-05-24
KR1020160063555A KR102036955B1 (ko) 2016-03-25 2016-05-24 미세 얼굴 다이나믹의 딥 러닝 분석을 통한 미세 표정 인식 방법 및 장치

Publications (1)

Publication Number Publication Date
WO2017164478A1 true WO2017164478A1 (fr) 2017-09-28

Family

ID=59900519

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2016/012772 WO2017164478A1 (fr) 2016-03-25 2016-11-08 Procédé et appareil de reconnaissance de micro-expressions au moyen d'une analyse d'apprentissage profond d'une dynamique micro-faciale

Country Status (1)

Country Link
WO (1) WO2017164478A1 (fr)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107742117A (zh) * 2017-11-15 2018-02-27 北京工业大学 一种基于端到端模型的人脸表情识别方法
CN107808150A (zh) * 2017-11-20 2018-03-16 珠海习悦信息技术有限公司 人体视频动作识别方法、装置、存储介质及处理器
CN108596039A (zh) * 2018-03-29 2018-09-28 南京邮电大学 一种基于3d卷积神经网络的双模态情感识别方法及系统
CN108932500A (zh) * 2018-07-09 2018-12-04 广州智能装备研究院有限公司 一种基于深度神经网络的动态手势识别方法及系统
CN109063643A (zh) * 2018-08-01 2018-12-21 中国科学院合肥物质科学研究院 一种用于脸部信息部分隐藏条件下的面部表情痛苦度识别方法
WO2019120032A1 (fr) * 2017-12-21 2019-06-27 Oppo广东移动通信有限公司 Procédé de construction de modèle, procédé de photographie, dispositif, support d'informations et terminal
CN109993061A (zh) * 2019-03-01 2019-07-09 珠海亿智电子科技有限公司 一种人脸检测与识别方法、系统以及终端设备
CN110348271A (zh) * 2018-04-04 2019-10-18 山东大学 一种基于长短时记忆网络的微表情识别方法
CN110569795A (zh) * 2018-03-13 2019-12-13 腾讯科技（深圳）有限公司 一种图像识别方法、装置以及相关设备
CN112699815A (zh) * 2020-12-30 2021-04-23 常州码库数据科技有限公司 基于时空运动增强网络的动态表情识别方法及系统
CN112801009A (zh) * 2021-02-07 2021-05-14 华南理工大学 基于双流网络的面部情感识别方法、装置、介质及设备
CN113537008A (zh) * 2021-07-02 2021-10-22 江南大学 基于自适应运动放大和卷积神经网络的微表情识别方法
CN110097004B (zh) * 2019-04-30 2022-03-29 北京字节跳动网络技术有限公司 面部表情识别方法和装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20100081874A (ko) * 2009-01-07 2010-07-15 포항공과대학교 산학협력단 사용자 맞춤형 표정 인식 방법 및 장치
JP2012008779A (ja) * 2010-06-24 2012-01-12 Nippon Telegr & Teleph Corp <Ntt> 表情学習装置、表情認識装置、表情学習方法、表情認識方法、表情学習プログラム及び表情認識プログラム
US20130300900A1 (en) * 2012-05-08 2013-11-14 Tomas Pfister Automated Recognition Algorithm For Detecting Facial Expressions
KR20160027576A (ko) * 2014-09-01 2016-03-10 유형근 얼굴인식형 인터랙티브 디지털 사이니지장치

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20100081874A (ko) * 2009-01-07 2010-07-15 포항공과대학교 산학협력단 사용자 맞춤형 표정 인식 방법 및 장치
JP2012008779A (ja) * 2010-06-24 2012-01-12 Nippon Telegr & Teleph Corp <Ntt> 表情学習装置、表情認識装置、表情学習方法、表情認識方法、表情学習プログラム及び表情認識プログラム
US20130300900A1 (en) * 2012-05-08 2013-11-14 Tomas Pfister Automated Recognition Algorithm For Detecting Facial Expressions
KR20160027576A (ko) * 2014-09-01 2016-03-10 유형근 얼굴인식형 인터랙티브 디지털 사이니지장치

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
H. KOBAYASHI ET AL.: "Dynamic recognition of basic facial expressions by discrete-time recurrent neural network", NEURAL NETWORKS, 1993. IJCNN '93-NAGOYA. PROCEEDINGS OF 1993 INTERNATIONAL JOINT CONFERENCE ON, 6 August 2002 (2002-08-06), pages 155 - 158, XP000499135 *

Cited By (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107742117A (zh) * 2017-11-15 2018-02-27 北京工业大学 一种基于端到端模型的人脸表情识别方法
CN107808150A (zh) * 2017-11-20 2018-03-16 珠海习悦信息技术有限公司 人体视频动作识别方法、装置、存储介质及处理器
WO2019120032A1 (fr) * 2017-12-21 2019-06-27 Oppo广东移动通信有限公司 Procédé de construction de modèle, procédé de photographie, dispositif, support d'informations et terminal
CN110569795B (zh) * 2018-03-13 2022-10-14 腾讯科技（深圳）有限公司 一种图像识别方法、装置以及相关设备
CN110569795A (zh) * 2018-03-13 2019-12-13 腾讯科技（深圳）有限公司 一种图像识别方法、装置以及相关设备
CN108596039B (zh) * 2018-03-29 2020-05-05 南京邮电大学 一种基于3d卷积神经网络的双模态情感识别方法及系统
CN108596039A (zh) * 2018-03-29 2018-09-28 南京邮电大学 一种基于3d卷积神经网络的双模态情感识别方法及系统
CN110348271A (zh) * 2018-04-04 2019-10-18 山东大学 一种基于长短时记忆网络的微表情识别方法
CN108932500A (zh) * 2018-07-09 2018-12-04 广州智能装备研究院有限公司 一种基于深度神经网络的动态手势识别方法及系统
CN108932500B (zh) * 2018-07-09 2019-08-06 广州智能装备研究院有限公司 一种基于深度神经网络的动态手势识别方法及系统
CN109063643A (zh) * 2018-08-01 2018-12-21 中国科学院合肥物质科学研究院 一种用于脸部信息部分隐藏条件下的面部表情痛苦度识别方法
CN109063643B (zh) * 2018-08-01 2021-09-28 中国科学院合肥物质科学研究院 一种用于脸部信息部分隐藏条件下的面部表情痛苦度识别方法
CN109993061B (zh) * 2019-03-01 2021-12-07 珠海亿智电子科技有限公司 一种人脸检测与识别方法、系统以及终端设备
CN109993061A (zh) * 2019-03-01 2019-07-09 珠海亿智电子科技有限公司 一种人脸检测与识别方法、系统以及终端设备
CN110097004B (zh) * 2019-04-30 2022-03-29 北京字节跳动网络技术有限公司 面部表情识别方法和装置
CN112699815A (zh) * 2020-12-30 2021-04-23 常州码库数据科技有限公司 基于时空运动增强网络的动态表情识别方法及系统
CN112801009A (zh) * 2021-02-07 2021-05-14 华南理工大学 基于双流网络的面部情感识别方法、装置、介质及设备
CN113537008A (zh) * 2021-07-02 2021-10-22 江南大学 基于自适应运动放大和卷积神经网络的微表情识别方法
CN113537008B (zh) * 2021-07-02 2024-03-29 江南大学 基于自适应运动放大和卷积神经网络的微表情识别方法

Similar Documents

Publication Publication Date Title
WO2017164478A1 (fr) Procédé et appareil de reconnaissance de micro-expressions au moyen d&#39;une analyse d&#39;apprentissage profond d&#39;une dynamique micro-faciale
WO2018212494A1 (fr) Procédé et dispositif d&#39;identification d&#39;objets
WO2019098414A1 (fr) Procédé et dispositif d&#39;apprentissage hiérarchique de réseau neuronal basés sur un apprentissage faiblement supervisé
WO2017213398A1 (fr) Modèle d&#39;apprentissage pour détection de région faciale saillante
WO2020071701A1 (fr) Procédé et dispositif de détection d&#39;un objet en temps réel au moyen d&#39;un modèle de réseau d&#39;apprentissage profond
Hisham et al. Arabic sign language recognition using Ada-Boosting based on a leap motion controller
WO2011096651A2 (fr) Procédé et dispositif d&#39;identification de visage
WO2020122432A1 (fr) Dispositif électronique et procédé d&#39;affichage d&#39;une image tridimensionnelle de celui-ci
WO2016108327A1 (fr) Procédé de détection de véhicule, structure de base de données pour la détection de véhicule, et procédé de construction de base de données pour détection de véhicule
WO2020246655A1 (fr) Procédé de reconnaissance de situation et dispositif permettant de le mettre en œuvre
CN112527113A (zh) 手势识别及手势识别网络的训练方法和装置、介质和设备
WO2022097927A1 (fr) Procédé de détection d&#39;événement vidéo en direct sur la base d&#39;interrogations en langage naturel, et appareil correspondant
WO2020032506A1 (fr) Système de détection de vision et procédé de détection de vision l&#39;utilisant
WO2021040287A1 (fr) Dispositif et procédé de nouvelle identification de personnes
WO2020231005A1 (fr) Dispositif de traitement d&#39;image et son procédé de fonctionnement
Silanon Thai Finger‐Spelling Recognition Using a Cascaded Classifier Based on Histogram of Orientation Gradient Features
WO2019240330A1 (fr) Système de prédiction de force basé sur des images et procédé correspondant
WO2015056893A1 (fr) Appareil de traitement d&#39;image et procédé de commande de celui-ci
WO2021235682A1 (fr) Procédé et dispositif de réalisation d&#39;une prédiction de comportement à l&#39;aide d&#39;une attention auto-focalisée explicable
Senthilkumar et al. Suspicious human activity detection in classroom examination
WO2022139327A1 (fr) Procédé et appareil de détection d&#39;énoncés non pris en charge dans la compréhension du langage naturel
WO2019225799A1 (fr) Procédé et dispositif de suppression d&#39;informations d&#39;utilisateur à l&#39;aide d&#39;un modèle génératif d&#39;apprentissage profond
Do et al. Hand detection and tracking in videos for fine-grained action recognition
Malik et al. Multiview human action recognition system based on openpose and knn classifier
Aravindan et al. A Smart Assistive System for Visually Impaired to Inform Acquaintance Using Image Processing (ML) Supported by IoT

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16895604

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 16895604

Country of ref document: EP

Kind code of ref document: A1