CN113378733A - System and device for constructing emotion diary and daily activity recognition - Google Patents
System and device for constructing emotion diary and daily activity recognition Download PDFInfo
- Publication number
- CN113378733A CN113378733A CN202110673859.9A CN202110673859A CN113378733A CN 113378733 A CN113378733 A CN 113378733A CN 202110673859 A CN202110673859 A CN 202110673859A CN 113378733 A CN113378733 A CN 113378733A
- Authority
- CN
- China
- Prior art keywords
- emotion
- recognition
- behavior
- diary
- constructing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000008451 emotion Effects 0.000 title claims abstract description 57
- 230000000694 effects Effects 0.000 title claims abstract description 22
- 230000006399 behavior Effects 0.000 claims abstract description 69
- 230000008909 emotion recognition Effects 0.000 claims abstract description 21
- 238000012544 monitoring process Methods 0.000 claims abstract description 14
- 230000002996 emotional effect Effects 0.000 claims description 7
- 238000012806 monitoring device Methods 0.000 claims description 7
- 230000002159 abnormal effect Effects 0.000 claims description 6
- 238000013527 convolutional neural network Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 5
- 230000005856 abnormality Effects 0.000 claims description 4
- 238000010276 construction Methods 0.000 claims description 4
- 238000013135 deep learning Methods 0.000 claims description 4
- 238000001514 detection method Methods 0.000 claims description 3
- 206010003805 Autism Diseases 0.000 description 4
- 208000020706 Autistic disease Diseases 0.000 description 4
- 206010000117 Abnormal behaviour Diseases 0.000 description 2
- 238000011161 development Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000003340 mental effect Effects 0.000 description 1
- 238000000034 method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Abstract
The invention relates to a system and a device for constructing emotion diaries and daily activity recognition, wherein the system comprises a capturing system, a recognition system and a display system, wherein the recognition system comprises an emotion recognition system and a daily monitoring system, and the system is used for recognizing the emotion and behavior of a special child, so that a teacher and parents can quickly know the emotion change and behavior ability of the child.
Description
Technical Field
The invention relates to a recognition system and a device, in particular to a system and a device for constructing emotion diaries and recognizing daily activities.
Background
By 2019, the development status report of the Chinese autism education and rehabilitation industry discloses that: the number of Chinese autism children exceeds 200 ten thousand, and the autism children and normal children have differences in sense, emotion, limb, behavior or speech and the like. In the market, a targeted information management system, an education system, an ability evaluation system, a teaching aid product and the like are developed for special children such as autism, but the emotion and behavior expressions of the children are not known and analyzed, so that the emotion change, the behavior ability and the like of the children can be known, the thought and the need of the children can be judged accurately, and the problem to be solved urgently is solved.
Disclosure of Invention
The invention aims to provide a system and a device for constructing an emotion diary and daily activity recognition.
In order to solve the technical problems, the invention is realized by the following technical scheme:
the invention relates to a system for constructing emotion diaries and daily activity recognition, which comprises a capturing system, a recognition system and a display system, wherein the recognition system comprises an emotion recognition system and a daily monitoring system.
Further, the capturing system is used for capturing face images, character behaviors or videos and transmitting the face images, the character behaviors or the videos to the recognition system.
Furthermore, the emotion recognition system comprises an emotion recognition module, an emotion recording module and an emotion diary construction module.
Furthermore, the daily monitoring system comprises a behavior analysis module, a behavior recording module and a monitoring and abnormity feedback module.
Further, the emotion recognition module trains an emotion detection model by using the face image trained based on the deep learning frame, and labels the face image with corresponding emotion.
Further, the CNN convolutional neural network in the emotion recognition module can output a softmax list of various emotions by recognizing the face image, and display the emotional state with the highest score.
Further, the emotion recording module continuously records the emotion state output by the emotion recognition module, transmits the emotion state to the emotion diary construction module, and constructs and outputs the emotion diary.
Further, the behavior analysis module uses a behavior recognition model pre-trained and serialized based on the behavior data set.
Further, the behavior analysis module completes deep learning using 3 dresinet for human recognition and ImageNet for image classification, resulting in spatio-temporal components for activity recognition.
Further, a behavior recognition model in the behavior analysis module recognizes the image or video character behaviors and outputs behavior names.
Furthermore, the behavior recording module continuously records the character behaviors output by the behavior analysis module and transmits the character behaviors to the monitoring and abnormality feedback module.
Further, the monitoring and abnormity feedback module compares the human behavior with the preset behavior, if the preset behavior is met, the system only records, and if the preset behavior is not met, the behavior recording module records and feeds back abnormity.
Further, the display system receives emotional states, emotional diaries, daily behaviors, abnormal feedback and the like output by the recognition system.
An apparatus for constructing emotion diary and daily activity recognition includes a capturing device, a recognition device and a display device, wherein the recognition device includes an emotion recognition device and a daily monitoring device.
Further, the capturing device is used for capturing a human face image, a behavior action image or a behavior video, and is preferably a camera or a video camera.
Further, the emotion recognition device is used for processing the images captured by the capturing device and feeding back the result.
Further, the daily monitoring device is used for processing images or videos captured by the capturing device and making feedback and judgment.
Further, the daily monitoring device comprises an abnormality feedback device, preferably an automatic encoder.
Further, the display device is used for outputting the results of emotion recording, emotion diary, character behavior, abnormal feedback, and the like, and is preferably a display screen.
The invention has the advantages that: the emotion and the behavior of the special children are recognized, and the physical and mental changes of the children are known in time.
Drawings
FIG. 1 is a system configuration of the present invention.
Fig. 2 is a flow of emotion recognition according to the present invention.
FIG. 3 is a daily monitoring process according to the present invention.
FIG. 4 is an emotion recognition module of the present invention.
FIG. 5 is a behavior analysis module of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments.
Referring to fig. 1-5, the working process of the present invention is as follows:
the invention relates to a system for constructing emotion diaries and daily activity recognition, which comprises a capturing system, a recognition system and a display system, wherein the recognition system comprises an emotion recognition system and a daily monitoring system.
The capturing system captures face images, character behaviors or videos and then transmits the face images, the character behaviors or the videos to the recognition system. And an emotion recognition system in the recognition system recognizes the face emotion and records the face emotion, and meanwhile, an emotion diary is constructed according to the continuous emotion state and is fed back to a teacher or a parent through a display system. The daily monitoring system identifies the character behaviors and videos and compares the behaviors with preset behaviors, if the behaviors are matched with the preset behaviors, the system only records the behaviors, if the behaviors are not matched with the preset behaviors, the abnormal behaviors are fed back while the behaviors are recorded, and the abnormal behaviors are fed back to parents or teachers through the display system.
An apparatus for constructing emotion diary and daily activity recognition includes a capturing device, a recognition device and a display device, wherein the recognition device includes an emotion recognition device and a daily monitoring device.
The capturing device is a camera and is used for capturing a face image, a behavior and motion image or a behavior video; the identification device is a microcomputer and is used for processing images or videos captured by the camera and making feedback and judgment; the display device is a display screen and outputs results such as emotion records, emotion diaries, character behaviors, abnormal feedback and the like, so that teachers and parents can quickly know emotion changes and behavior abilities of children.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof, and it is therefore intended that the present embodiments be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein.
Claims (10)
1. A system for constructing emotion diaries and daily activity recognition comprises a capture system, a recognition system and a display system, and is characterized in that: the capturing system is used for capturing face images, character behaviors or videos and transmitting the face images, the character behaviors or the videos to the recognition system; the recognition system comprises an emotion recognition system and a daily monitoring system.
2. A system for constructing an emotion diary and identification of daily activities as recited in claim 1, wherein: the emotion recognition system comprises an emotion recognition module, an emotion recording module and an emotion diary construction module; the emotion recognition module comprises a face image training emotion detection model trained on the basis of a deep learning framework and a CNN convolutional neural network; the emotion detection model labels corresponding emotion of the face image; the CNN convolutional neural network can output a softmax list of various emotions by recognizing the face image, and display the emotional state with the highest score.
3. A system for constructing an emotion diary and identification of daily activities as recited in claim 2, wherein: and the emotion recording module continuously records the emotion state output by the emotion recognition module, transmits the emotion state to the emotion diary construction module, and constructs and outputs the emotion diary.
4. A system for constructing an emotion diary and identification of daily activities as recited in claim 1, wherein: the daily monitoring system comprises a behavior analysis module, a behavior recording module and a monitoring and abnormity feedback module; the behavior analysis module includes a behavior recognition model pre-trained and serialized based on the behavior data set and a spatiotemporal component for activity recognition that performs deep learning based on human recognition parameters and the image classification data set, recognizes image or video character behaviors, and outputs behavior names.
5. A system for constructing an emotion diary and identification of daily activities as recited in claim 4, wherein: the behavior recording module continuously records the character behaviors output by the behavior analysis module and transmits the character behaviors to the monitoring and abnormity feedback module.
6. A system for constructing an emotion diary and identification of daily activities as recited in claim 5, wherein: the monitoring and abnormal feedback module compares the behavior of the human being with the preset behavior, if the preset behavior is met, the system only records, and if the preset behavior is not met, the behavior recording module records and feeds back the abnormality.
7. A system for constructing an emotion diary and identification of daily activities as claimed in any of claims 1 to 6, wherein: and the display system receives the information such as emotional state, emotional diary, daily behavior, abnormal feedback and the like output by the recognition system.
8. An apparatus for constructing emotion diary and daily activity recognition, comprising a capturing device, a recognition device and a display device, characterized in that: a system for constructing an emotional diary and daily activity recognition system as claimed in any of claims 1 to 7.
9. A system for constructing an emotion diary and identification of daily activities as recited in claim 8, wherein: the capturing device is used for capturing face images, behavior and motion images or behavior videos, and preferably a camera or a video camera; the recognition device comprises an emotion recognition device and a daily monitoring device.
10. A system for constructing an emotion diary and identification of daily activities as recited in claim 9, wherein: the emotion recognition device is used for processing the images captured by the capture device and feeding back results; the daily monitoring device is used for processing the images or videos captured by the capturing device and making feedback and judgment; the daily monitoring device comprises an abnormality feedback device, preferably an automatic encoder; the display device is used for outputting the results of emotion recording, emotion diary, character behavior, abnormal feedback and the like, and preferably a display screen.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110673859.9A CN113378733A (en) | 2021-06-17 | 2021-06-17 | System and device for constructing emotion diary and daily activity recognition |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110673859.9A CN113378733A (en) | 2021-06-17 | 2021-06-17 | System and device for constructing emotion diary and daily activity recognition |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113378733A true CN113378733A (en) | 2021-09-10 |
Family
ID=77577573
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110673859.9A Pending CN113378733A (en) | 2021-06-17 | 2021-06-17 | System and device for constructing emotion diary and daily activity recognition |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113378733A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117473304A (en) * | 2023-12-28 | 2024-01-30 | 天津大学 | Multi-mode image labeling method and device, electronic equipment and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106650621A (en) * | 2016-11-18 | 2017-05-10 | 广东技术师范学院 | Deep learning-based emotion recognition method and system |
| CN108216252A (en) * | 2017-12-29 | 2018-06-29 | 中车工业研究院有限公司 | A kind of subway driver vehicle carried driving behavior analysis method, car-mounted terminal and system |
| CN108764169A (en) * | 2018-05-31 | 2018-11-06 | 厦门大学 | A kind of driver's Emotion identification based on machine learning and display device and method |
| CN109431524A (en) * | 2018-12-13 | 2019-03-08 | 东莞理工学院 | The system of identification auxiliary autism children monitoring is savoured based on human body |
| CN109875579A (en) * | 2019-02-28 | 2019-06-14 | 京东方科技集团股份有限公司 | Emotional health management system and emotional health management method |
| CN110349674A (en) * | 2019-07-05 | 2019-10-18 | 昆山杜克大学 | Autism-spectrum obstacle based on improper activity observation and analysis assesses apparatus and system |
| CN111353366A (en) * | 2019-08-19 | 2020-06-30 | 深圳市鸿合创新信息技术有限责任公司 | Emotion detection method and device and electronic equipment |
| CN111814775A (en) * | 2020-09-10 | 2020-10-23 | 平安国际智慧城市科技股份有限公司 | Target object abnormal behavior identification method, device, terminal and storage medium |
| CN112233800A (en) * | 2020-11-19 | 2021-01-15 | 吾征智能技术(北京)有限公司 | Disease prediction system based on abnormal behaviors of children |
-
2021
- 2021-06-17 CN CN202110673859.9A patent/CN113378733A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106650621A (en) * | 2016-11-18 | 2017-05-10 | 广东技术师范学院 | Deep learning-based emotion recognition method and system |
| CN108216252A (en) * | 2017-12-29 | 2018-06-29 | 中车工业研究院有限公司 | A kind of subway driver vehicle carried driving behavior analysis method, car-mounted terminal and system |
| CN108764169A (en) * | 2018-05-31 | 2018-11-06 | 厦门大学 | A kind of driver's Emotion identification based on machine learning and display device and method |
| CN109431524A (en) * | 2018-12-13 | 2019-03-08 | 东莞理工学院 | The system of identification auxiliary autism children monitoring is savoured based on human body |
| CN109875579A (en) * | 2019-02-28 | 2019-06-14 | 京东方科技集团股份有限公司 | Emotional health management system and emotional health management method |
| CN110349674A (en) * | 2019-07-05 | 2019-10-18 | 昆山杜克大学 | Autism-spectrum obstacle based on improper activity observation and analysis assesses apparatus and system |
| CN111353366A (en) * | 2019-08-19 | 2020-06-30 | 深圳市鸿合创新信息技术有限责任公司 | Emotion detection method and device and electronic equipment |
| CN111814775A (en) * | 2020-09-10 | 2020-10-23 | 平安国际智慧城市科技股份有限公司 | Target object abnormal behavior identification method, device, terminal and storage medium |
| CN112233800A (en) * | 2020-11-19 | 2021-01-15 | 吾征智能技术(北京)有限公司 | Disease prediction system based on abnormal behaviors of children |
Non-Patent Citations (1)
| Title |
|---|
| 武玉伟, 北京理工大学出版社 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117473304A (en) * | 2023-12-28 | 2024-01-30 | 天津大学 | Multi-mode image labeling method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102013176B (en) | On-line study system | |
| Joo et al. | Visual persuasion: Inferring communicative intents of images | |
| Littlewort et al. | Automated measurement of children's facial expressions during problem solving tasks | |
| Yun et al. | Automatic recognition of children engagement from facial video using convolutional neural networks | |
| CN111242049A (en) | Student online class learning state evaluation method and system based on facial recognition | |
| US20150044657A1 (en) | Video-based teacher assistance | |
| CN104464406A (en) | Real-time interactive online learning platform | |
| CN109214664B (en) | Emotional behavior comprehensive analysis system based on artificial intelligence | |
| CN111986530A (en) | Interactive learning system based on learning state detection | |
| CN109685692A (en) | A kind of noninductive acquisition and analysis system of various dimensions student learning behavior | |
| Dawood et al. | Affective computational model to extract natural affective states of students with Asperger syndrome (AS) in computer-based learning environment | |
| CN113378733A (en) | System and device for constructing emotion diary and daily activity recognition | |
| Padrón-Rivera et al. | Identification of action units related to affective states in a tutoring system for mathematics | |
| Chiu et al. | A bayesian classification network-based learning status management system in an intelligent classroom | |
| CN111178263A (en) | Real-time expression analysis method and device | |
| Narayanan et al. | Attention analysis in e-learning environment using a simple web camera | |
| Ashwin et al. | Unobtrusive students' engagement analysis in computer science laboratory using deep learning techniques | |
| Magdin et al. | Are instructed emotional states suitable for classification? Demonstration of how they can significantly influence the classification result in an automated recognition system | |
| CN111768863B (en) | Infant development monitoring system and method based on artificial intelligence | |
| Horvat et al. | Emerging opportunities for education in the time of covid-19-adaptive e-learning intelligent agent based on assessment of emotion and attention | |
| Hou | Deep Learning-Based Human Emotion Detection Framework Using Facial Expressions | |
| CN113095208B (en) | Attention observation and reminding system applied to college English teaching classroom | |
| Agarwal et al. | Semi-Supervised Learning to Perceive Children's Affective States in a Tablet Tutor | |
| CN111079692B (en) | Campus behavior analysis method based on face recognition and used in K12 education stage | |
| Malhotra et al. | Student Invigilation Detection Using Deep Learning and Machine After Covid-19: A Review on Taxonomy and Future Challenges |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210910 |