WO2020166756A1

WO2020166756A1 - Device for recognizing presenter's mental stress and providing feedback

Info

Publication number: WO2020166756A1
Application number: PCT/KR2019/002886
Authority: WO
Inventors: 이경우; 문준형; 이준일; 천동미; 이문희
Original assignee: 연세대학교 산학협력단
Priority date: 2019-02-14
Filing date: 2019-03-13
Publication date: 2020-08-20
Also published as: KR20200099379A; KR102218713B1

Abstract

The present invention relates to a device for recognizing a presenter's mental stress and providing feedback, the device comprising: a biometrics measuring unit mounted in a presenter's apparatus to measure biometrics of the presenter who uses the presenter's apparatus; a stress recognizing unit for analyzing the presenter's measured biometrics and recognizing a mental stress state of the presenter on the basis of a recognition model; and a feedback providing unit for providing the presenter with the presenter's recognized mental stress state as feedback. Therefore, the present invention can collect biometrics of the presenter who does not additionally wear devices such as a smart watch and an upper body band, in a situation where the presenter is giving a presentation, and can recognize the presenter's stress state by using the collected biometrics and provide the recognized stress state as feedback, so as to improve the presenter's mental stress state.

Description

Speaker's mental stress recognition and feedback provision device

The present invention relates to a technology for recognizing a speaker's mental stress and providing feedback, and in more detail, it is possible to recognize a speaker's mental stress state using biometric information collected from a plurality of biometric information sensors and provide it as a feedback to the speaker. It relates to a speaker's mental stress recognition and feedback providing device.

Stress exists in all areas of human life, so no one can avoid it. Stress can also mean some change that humans need to adapt. When we are in a stressful situation, the sympathetic part of the autonomic nervous system is activated by the body's response to the stress, and the body's resources are mobilized to respond to emergency situations.

Mental stress is divided into acute and chronic. Repeated acute stress or chronic stress has a negative effect on people.

In the case of a person with an important presentation ahead of a large audience, the presenter will experience a great deal of stress when he comes to the podium to make a presentation and looks at people's faces. Presenting a presentation in front of many people does not cause physical harm, but the human body is automatically tense because it is considered a psychological threat. When a person perceives stress, a physiological reaction occurs.

In particular, mental stress in an important presentation situation can lead to many negative consequences by influencing the presentation immediately. For example, if you don't make a successful presentation ahead of important contracts, product launches, elections, or the completion of your academic process, it will be difficult to achieve the desired results.

[Prior technical literature]

[Patent Literature]

(Patent Document 1) Korean Patent Registration No. 10-0418434 (2004.01.31)

(Patent Document 2) Korean Patent Registration No. 10-0231473 (1999.08.30)

The present invention was devised in view of the above points, and the problem to be solved in the present invention is to recognize the mental stress state of the speaker using biometric information collected from a plurality of biometric information sensors and provide it as a feedback to the speaker. It is to provide a device for recognizing the mental stress of the presenter and providing feedback.

In addition, the presenter's mental stress recognition and feedback providing device capable of recognizing the presenter's mental stress state by measuring the presenter's motion recognition and biometric information using the presenter tool and providing feedback to the presenter are provided.

In addition, the presenter's mental stress recognition and feedback providing device capable of providing feedback to the presenter in the form of vibration and heat of the recognized mental stress state of the presenter is provided.

The present inventor's mental stress recognition and feedback providing apparatus for achieving the above object includes a biometric information measuring unit mounted on a presenter tool and measuring biometric information to a presenter using the presenter tool, and the measured A stress recognition unit that analyzes biometric information and recognizes a mental stress state of the speaker based on a recognition model, and a feedback providing unit that provides feedback of the recognized mental stress state of the speaker to the speaker. It features.

The biometric information measuring unit measures the movement of the presenter's hand using the presenter tool corresponding to a presenter or a pointer and generates movement information, and a second measuring body temperature of the presenter and generating body temperature change information It may include a measuring member, and a third measuring member that measures the skin conductivity (GSR) of the speaker and generates activation information of the sympathetic nervous system.

The stress recognition unit may recognize the mental stress state of the speaker by analyzing the presence and degree of a physiological reaction experienced when there is stress based on the measured biometric information of the speaker, and calculating a stress score based on the result.

The feedback providing unit may provide feedback to the speaker in at least one of vibration generation and heat generation based on the recognized mental stress state of the speaker.

The disclosed technology can have the following effects. However, since it does not mean that a specific embodiment should include all of the following effects or only the following effects, it should not be understood that the scope of the rights of the disclosed technology is limited thereby.

The apparatus for recognizing mental stress of the speaker and providing feedback according to an embodiment of the present invention recognizes the mental stress state of the speaker based on the biometric information collected from a plurality of biometric information sensors, and vibrates or heats the recognized stress state of the speaker. Can be provided as feedback to the presenter.

The apparatus for recognizing mental stress and providing feedback of a speaker according to an embodiment of the present invention can collect biometric information without wearing additional equipment such as a smart watch or an upper body band in a situation where the speaker announces, and uses the collected biometric information. It is possible to improve the mental stress state of the presenter by recognizing the stress state and providing feedback.

1 is a diagram illustrating a system for recognizing mental stress and providing feedback of a speaker according to an embodiment of the present invention.

FIG. 2 is a block diagram illustrating the apparatus for recognizing mental stress and providing feedback in FIG. 1.

FIG. 3 is a detailed block diagram illustrating the apparatus for recognizing mental stress and providing feedback in FIG. 2.

FIG. 4 is a flowchart illustrating a mental stress recognition and feedback providing process performed by the mental stress recognition and feedback providing apparatus of FIG. 2.

The best mode for implementation of the present invention is a biometric information measuring unit mounted on a speaker tool and measuring biometric information to a speaker using the speaker tool, analyzing the measured biometric information of the speaker, and based on a recognition model. Providing a device for recognizing mental stress and providing feedback of a speaker including a stress recognition unit for recognizing the mental stress state of the speaker, and a feedback providing unit for providing the recognized mental stress state of the speaker as a feedback to the speaker do.

Since the description of the present invention is merely an embodiment for structural or functional description, the scope of the present invention should not be construed as being limited by the embodiments described in the text. That is, since the embodiments can be variously changed and have various forms, the scope of the present invention should be understood to include equivalents capable of realizing the technical idea. In addition, since the object or effect presented in the present invention does not mean that a specific embodiment should include all of them or only those effects, the scope of the present invention should not be understood as being limited thereto.

Meanwhile, the meaning of terms described in the present application should be understood as follows.

Terms such as "first" and "second" are used to distinguish one component from other components, and the scope of rights is not limited by these terms. For example, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component.

When a component is referred to as being "connected" to another component, it should be understood that although it may be directly connected to the other component, another component may exist in the middle. On the other hand, when it is mentioned that a certain component is "directly connected" to another component, it should be understood that no other component exists in the middle. On the other hand, other expressions describing the relationship between the constituent elements, that is, "between" and "just between" or "neighboring to" and "directly neighboring to" should be interpreted as well.

Singular expressions are to be understood as including plural expressions unless the context clearly indicates otherwise, and terms such as “comprise” or “have” refer to implemented features, numbers, steps, actions, components, parts, or It is to be understood that it is intended to designate that a combination exists and does not preclude the presence or addition of one or more other features or numbers, steps, actions, components, parts, or combinations thereof.

In each step, the identification code (for example, a, b, c, etc.) is used for convenience of explanation, and the identification code does not describe the order of each step, and each step has a specific sequence clearly in context. Unless otherwise stated, it may occur differently from the stated order. That is, each of the steps may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the reverse order.

The present invention can be embodied as computer-readable codes on a computer-readable recording medium, and the computer-readable recording medium includes all types of recording devices storing data that can be read by a computer system. . Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. Further, the computer-readable recording medium is distributed over a computer system connected by a network, so that the computer-readable code can be stored and executed in a distributed manner.

All terms used herein have the same meaning as commonly understood by one of ordinary skill in the field to which the present invention belongs, unless otherwise defined. Terms defined in commonly used dictionaries should be construed as having meanings in the context of related technologies, and cannot be construed as having an ideal or excessive formal meaning unless explicitly defined in the present application.

Referring to FIG. 1, a system 100 for recognizing and providing feedback on mental stress of a speaker may include a speaker terminal 110, a device for recognizing and providing feedback on mental stress 130, and a database 150.

The presenter terminal 110 may include at least one wearable sensor and may correspond to a computing device capable of transmitting a biosignal sensed from the wearable sensor to the mental stress recognition and feedback providing device 130, and may correspond to a smartphone or a laptop computer. Alternatively, it may be implemented as a computer, not necessarily limited thereto, and may be implemented as various devices such as a tablet PC.

In one embodiment, the presenter terminal 110 may be implemented as a presenter tool such as a presenter or a pointer that the presenter holds and uses when presenting. In this case, the presenter can measure biometric information to the presenter without wearing additional equipment.

The presenter terminal 110 may be connected to the apparatus 130 for recognizing mental stress and providing feedback through a network.

The mental stress recognition and feedback providing device 130 can recognize the presenter's mental stress state at the present time based on a plurality of sensing information received from the presenter terminal 110 and provide the recognized presenter's stress state as feedback to the presenter. It can be implemented as a server that corresponds to a computer or program that can do. The apparatus 130 for recognizing mental stress and providing feedback may be wirelessly connected to the speaker terminal 110 through Bluetooth, WiFi, and the like, and may exchange data with the speaker terminal 110 through a network.

In one embodiment, the mental stress recognition and feedback providing apparatus 130 may provide the presenter's mental stress state recognized at the present time to the presenter terminal 110, and based on this, various services suitable for the presenter's mental stress. You can also provide The presenter can check his current mental stress through the presenter terminal 110 and can select or use a service automatically recommended from the mental stress recognition and feedback providing device 130 to improve stress.

In addition, the speaker recognizes mental stress through the presenter terminal 110 and evaluates information and services provided from the feedback providing device 130 to recognize or recommend information that is recognized or recommended by the mental stress recognition and feedback providing device 130. You can feed back accuracy.

The apparatus 130 for recognizing mental stress and providing feedback may be implemented including the database 150, and may be implemented independently of the database 150. When implemented independently of the database 150, the apparatus 130 for recognizing mental stress and providing feedback may be connected to the database 150 by wire or wirelessly to exchange data.

The database 150 is a storage device capable of storing various pieces of information necessary for mental stress recognition. The database 150 may store biometric information received from the speaker terminal 110, and is not necessarily limited thereto, and information collected or processed in various forms in the process of recognizing the mental stress of the speaker based on a plurality of biometric information Can save them.

The database 150 may be composed of at least one independent sub-database storing information belonging to a specific range, and may be composed of an integrated database in which at least one independent sub-database is integrated into one. When composed of at least one independent sub-database, each of the sub-databases may be wirelessly connected through Bluetooth, WiFi, or the like, and data may be exchanged with each other through a network.

When configured as an integrated database, the database 150 may include a control unit for integrating each of the sub-databases into one and managing data exchange and control flow therebetween.

2 and 3 are block diagrams and detailed block diagrams illustrating the apparatus for recognizing mental stress and providing feedback in FIG. 1.

2 and 3, the apparatus 130 for recognizing mental stress and providing feedback may include a biometric information measuring unit 210, a stress recognizing unit 230, a feedback providing unit 250, and a control unit 270. have.

The biometric information measuring unit 210 may be implemented by being included in the mental stress recognition and feedback providing device 130, and may be implemented independently of the mental stress recognition and feedback providing device 130. When implemented independently of the mental stress recognition and feedback providing device 130, the biometric information measuring unit 210 may be connected to the mental stress recognition and feedback providing device 130 by wire or wirelessly to exchange data.

In one embodiment, the biometric information measuring unit 210 may be included in the speaker terminal 110 and implemented, and may exchange data with the mental stress recognition and feedback providing device 130 through the speaker terminal 110. .

In an embodiment, the biometric information measuring unit 210 may be mounted on a speaker tool and measure biometric information on a speaker who uses the speaker tool. Here, the presenter tool may correspond to tools belonging to a range that a presenter, such as a presenter or a pointer, can hold and use at the time of presentation. By attaching the biometric information measuring unit 210 to the presenter tool, the presenter does not need to wear a separate wearable device such as a smart watch including sensors and an upper body band to collect biometric information.

In an embodiment, the biometric information measuring unit 210 may measure biometric information according to a preset measurement period and measurement time by reflecting characteristics of the biometric information. For example, in the case of a body temperature sensor that measures a speaker's body temperature, it may have a measurement period in minutes, and may have a measurement time of at least 60 seconds or more.

The biometric information measuring unit 210 may include a plurality of measuring members.

Among the plurality of measuring members, the first measuring member 211 may measure a movement of a speaker's hand using a speaker tool and generate movement information. In one embodiment, the first measuring member 211 may be implemented as an accelerometer and an angular velocity meter. The first measuring member 211 may generate motion information by measuring the frequency, intensity, and range of the vertical or horizontal or rotating movement of the speaker tool.

Among the plurality of measuring members, the second measuring member 213 may measure a speaker's body temperature and generate body temperature change information. In one embodiment, the second measuring member 213 may be implemented as a body temperature sensor. The second measuring member 213 measures the body temperature of the speaker and divides the measured total body temperature by the measurement time in minutes, thereby generating body temperature change information per minute.

Among the plurality of measuring members, the third measuring member 215 may measure skin conductivity (GSR) of the speaker and generate activation information of the sympathetic nervous system. In one embodiment, the third measuring member 215 may be implemented as a skin conductivity sensor. The third measuring member 215 may measure a skin conductivity that is increased when sweat occurs in the speaker's skin and generate activation information of the speaker's sympathetic nervous system based on the measured skin conductivity. The amplitude of skin conductivity increases when aroused or negatively sensitized, and the speed of reaction increases when it is a sensitive stimulus. The sensitivity of the response increases with sudden or sensitive stimuli. Therefore, skin conductivity can be used as a useful index for negative sensibility, that is, stress.

In one embodiment, the biometric information measurement unit 210 recognizes mental stress by pre-filtering incorrect measurement values due to malfunction of each measurement member through comparison with each preset measurement threshold value by reflecting the characteristics of the biometric information. And it is possible to increase the accuracy of the mental stress recognition performed by the filtering providing device 130.

The stress recognition unit 230 may analyze the measured biometric information of the presenter and recognize the presenter's mental stress state based on the recognition model. Here, the recognition model can be said to be a database that finds and stores useful indicators for recognizing the stress state of speakers using machine learning or artificial intelligence (AI). The stress recognition unit 230 may recognize the presenter's mental stress state by analyzing the presence and degree of a physiological reaction experienced when there is stress based on the measured biometric information of the speaker, and calculating a stress score based on the result. .

In an embodiment, the stress recognition unit 230 may include a stress analysis member 231 and a stress recognition member 233.

The stress analysis member 231 may collect measurement information of each of the plurality of

measurement members

211, 213, and 215, and classify the collected measurement information into stress measurement indicators corresponding to physiological responses experienced when there is stress. Here, the stress measurement indicators may include pulse (heart rate), blood pressure, skin conductance, body temperature, etc. from the fact that the heart beats quickly, sweats, and undergoes physical changes such as changes in body temperature due to mental stress upon presentation by the presenter. .

In one embodiment, the stress analysis bonsai 231 classifies the characteristics of the measured biometric information for each of the stress measurement indicators, and based on the characteristics of the biometric information and the individual speaker characteristics, each stress measurement index of the presenter at the present time. Individual ratings of stars can be classified.

Here, the classification of individual speaker characteristics may correspond to matching information between biometric information collected and accumulated for each speaker by reflecting personal disposition or constitution difference for each speaker and speaker motion or speaker emotion. Individual speaker characteristic classification is a stress recognition model created by learning the information collected for each speaker in consideration of the fact that the speaker's behavior may be different even though it is the same biometric information, and the speaker's feelings about the movement may also be different. May correspond to. Classification of individual speaker characteristics may be generated by learning learning data configured by inputting the speaker's biometric information and outputting the actions the speaker has acted at the time the biometric information is measured or the emotion felt by the speaker.

In one embodiment, the stress analysis member 231 determines the speaker's motion based on the collected biometric information, and determines an individual rating for each indicator of the corresponding speaker based on the predefined stress measurement indicator rating table based on the speaker's motion. I can. For example, if the body temperature measured at the current time point has risen by 2°C or more compared to the body temperature measured at the previous time point, it may be determined that it belongs to the stress level 2 in the body temperature change index. More specifically, the stress analysis member 231 may determine the speaker's motion based on the biometric information for each of the biometric information measuring units 210, and the current speaker based on the speaker's motion and the biometric information of the speaker associated with the motion. Mental stress can be rated for each indicator.

In an embodiment, the stress analysis member 231 may determine the speaker's motion by learning learning data that inputs biometric information for a specific period of time and outputs an actual speaker's motion at a time when the biometric information is measured. The stress analysis member 231 may generate a learning model in which, when biometric information is input by learning learning data composed of biometric information and actual speaker motion information for a specific period in the past, a speaker motion corresponding to the biometric information is output.

The stress analysis member 231 may generate a learning model using previously collected learning data for each speaker, and may store the learning result in the database 150. The stress analysis member 231 may update the learning model by reflecting the biometric information collected in real time and the speaker's motion inferred based on the biometric information or the speaker's feedback on the speaker's mental stress.

In an embodiment, the stress analysis member 231 may classify a grade for each stress measurement index according to the severity level of each biomarker.

The mental stress state of the speaker can be recognized by learning learning data that inputs the speaker's motion and biometric information about the speaker's motion and outputs the speaker's emotion information about the speaker's motion.

When the stress analysis member 231 learns learning data that inputs the presenter's motion and the biometric information of the respective speaker's motion for each speaker and outputs the corresponding speaker's emotion information, the speaker motion and the biometric information are input. A learning model in which speaker emotion information corresponding to a corresponding speaker motion and biometric information is output may be generated.

The stress recognition member 233 may recognize the level of stress by calculating an overall stress score and a rating when individual ratings for the stress indicators are determined by the stress analysis member 231. Since it is difficult for the stress recognition member 233 to finally determine the level of stress depending on one of the stress indicators, it is possible to increase the accuracy of stress recognition by using various stress indicators in combination.

In an embodiment, the stress recognition member 233 may calculate the total stress score and grade based on the number of stress indices belonging to each grade based on results obtained from various stress indices. In other words, the number of indicators belonging to the first grade (β ₁ ), the number of indicators belonging to the second grade (β ₂ ), and the number of indicators belonging to the third grade (β ₃ ) are obtained, and at this time, the index belonging to the first grade. Based on the number of, the number of indicators belonging to the 2nd grade and the number of indicators belonging to the 3rd grade are weighted 2 and 3, respectively, so that the number of indicators belonging to the higher grade in the situation where the number of indicators belonging to each grade is the same. Should be considered more important.

In one embodiment, the stress recognition member 233 is a weighted user by applying a weight determined based on at least one of the user's propensity and the current time point to various stress indices analyzed by the stress analysis member 231 Mental stress can be elicited. The stress recognition member 233 learns feedback information of the presenter on the perceived mental stress for each speaker or point of view, and generates weight information between the biometric information measuring units 210 for each speaker or point of time in advance, and the database 150 Can be saved on.

For example, the stress recognition member 233 has a mental stress derived from the second measuring member 213 measuring body temperature change among the plurality of measuring

members

211, 213, 215 than the mental stress derived from other biometric information measuring members. For the presenter with high accuracy, a relatively high weight is given to the mental stress derived from the second measuring member 213, which measures body temperature change, compared to other biometric information measuring members. Information can be derived.

In addition, the stress recognition member 233 is a presenter whose mental stress derived from the third measuring member 215 that measures skin conductance (sympathetic nervous system activation) in the interval between the start of the presentation and 10 minutes elapsed is relatively high accuracy. In regard to the mental stress derived from the third measuring member 215 that measures skin conductance (sympathetic nervous system activation) than other biometric information measuring members in recognizing the mental stress of the presenter between the start of the presentation and the elapsed 10 minutes, By assigning a relatively high weight to each other, it is possible to derive weighted mental stress information according to the current point in time.

In an embodiment, the stress recognition member 233 may obtain a stress score indicating a level of mental stress of the final speaker through a summation process based on the weighted number of indicators belonging to each level of the speaker.

The feedback providing unit 250 may provide feedback to the presenter on the mental stress state of the presenter recognized by the stress recognition unit 230. In one embodiment, the feedback providing unit 250 provides feedback to the speaker in the form of vibration or heat by adjusting the vibration intensity of the vibration member 251 or the heating temperature of the heating member 253 based on the speaker's stress score. can do.

In an embodiment, the feedback providing unit 250 may be mounted on a speaker tool to provide feedback to the speaker by varying the vibration intensity or heating temperature in real time according to the level of mental stress of the speaker.

At this time, the presenter can recognize the stress he is receiving according to the degree of vibration or heat generation, and can make an effort to relieve it.

The controller 270 controls the overall operation of the apparatus 130 for recognizing mental stress and providing feedback, and controls the flow of control or data between the biometric information measurement unit 210, the stress recognition unit 230, and the feedback providing unit 250. Can be managed.

4 is a flowchart illustrating a process of recognizing mental stress and providing feedback performed by the apparatus for recognizing and providing feedback on mental stress in FIG. 1.

Referring to FIG. 4, the apparatus 130 for recognizing mental stress and providing feedback may measure biometric information including a speaker's movement, body temperature change, and sympathetic nervous system activation through the biometric information measuring unit 210 (step S410). . The apparatus 130 for recognizing mental stress and providing feedback may recognize the presenter's mental stress state by analyzing the presence and degree of a physiological reaction based on the measured biometric information of the presenter and calculating a stress score (step S430). The apparatus 130 for recognizing mental stress and providing feedback may provide feedback to the presenter in the form of vibration or heat on the stress state of the presenter recognized through the stress recognizing unit 250 (step S450).

Although the above has been described with reference to preferred embodiments of the present invention, those skilled in the art will variously modify and change the present invention within the scope not departing from the spirit and scope of the present invention described in the following claims. You will understand that you can do it.

본 발명을 지원한 국가연구개발사업National R&D project that supported the present invention

Assignment identification number: 2018-11-0998

Ministry name: Ministry of Science and Technology Information and Communication

Research and management professional institution: Information and Communication Technology Promotion Center

Research Project Name: Information and Communication Broadcasting R&D Project

Research Project Title: Research and development of emotional intelligence that can infer and judge the emotions of the other person and communicate and respond accordingly

Contribution Rate: 1/1

Organizer: Korea Advanced Institute of Science and Technology

Research Period: 2018.07.01 ~ 2019.04.30

[Explanation of code]

100: Presenter's mental stress recognition and feedback provision system

110: presenter terminal

130: mental stress recognition and feedback providing device

150: database

210: biometric information measuring unit 211,213,215: measuring member

230: stress recognition unit

231: stress analysis member 233: stress recognition member

250: feedback providing unit

251: vibration member 253: heating member

270: control unit

As described above, the apparatus for recognizing mental stress and providing feedback of a speaker according to the present invention may be used in an industry that provides services related to psychological training and stress management to promote self-understanding or self-development.

Claims

A biometric information measuring unit mounted on a speaker tool and measuring biometric information to a speaker using the speaker tool;

A stress recognition unit that analyzes the measured biometric information of the speaker and recognizes a mental stress state of the speaker based on a recognition model; And

A device for recognizing and providing feedback on mental stress of a speaker, including a feedback providing unit providing feedback to the speaker on the recognized mental stress state of the speaker.
The method of claim 1, wherein the biometric information measuring unit

A first measuring member measuring movement of the presenter's hand using the presenter tool corresponding to a presenter or a pointer and generating movement information;

A second measuring member measuring body temperature of the speaker and generating body temperature change information; And

And a third measuring member that measures the skin conductivity (GSR) of the speaker and generates information on activation of the sympathetic nervous system.
The method of claim 1, wherein the stress recognition unit

Based on the measured biometric information of the presenter, the presence and extent of a physiological reaction experienced when there is stress are analyzed, and a stress score is calculated based on the result to recognize the mental stress state of the presenter. Devices for stress recognition and feedback.
The method of claim 1, wherein the feedback providing unit

A device for recognizing and providing feedback on mental stress of a speaker, characterized in that feedback is provided to the speaker in at least one of vibration generation and heat generation based on the recognized mental stress state of the speaker.