WO2022168973A1 - 学習装置および評価情報出力装置 - Google Patents
学習装置および評価情報出力装置 Download PDFInfo
- Publication number
- WO2022168973A1 WO2022168973A1 PCT/JP2022/004684 JP2022004684W WO2022168973A1 WO 2022168973 A1 WO2022168973 A1 WO 2022168973A1 JP 2022004684 W JP2022004684 W JP 2022004684W WO 2022168973 A1 WO2022168973 A1 WO 2022168973A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- nodding
- information
- nodding motion
- motion
- evaluation information
- Prior art date
Links
- 238000011156 evaluation Methods 0.000 title claims abstract description 143
- 230000033001 locomotion Effects 0.000 claims abstract description 212
- 230000008451 emotion Effects 0.000 claims description 9
- 230000001133 acceleration Effects 0.000 claims description 6
- 241001465754 Metazoa Species 0.000 claims description 3
- 238000004458 analytical method Methods 0.000 claims description 3
- 238000007405 data analysis Methods 0.000 claims description 3
- 238000012986 modification Methods 0.000 description 11
- 230000004048 modification Effects 0.000 description 11
- 230000000694 effects Effects 0.000 description 9
- 238000006243 chemical reaction Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 5
- 241000282412 Homo Species 0.000 description 3
- 230000005484 gravity Effects 0.000 description 3
- 238000000034 method Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 230000004886 head movement Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000011158 quantitative evaluation Methods 0.000 description 2
- 238000003646 Spearman's rank correlation coefficient Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000010219 correlation analysis Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06398—Performance of employee with respect to a job function
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
- G06Q10/101—Collaborative creation, e.g. joint development of products or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/174—Facial expression recognition
- G06V40/176—Dynamic expression
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B3/00—Manually or mechanically operated teaching appliances working with questions and answers
- G09B3/02—Manually or mechanically operated teaching appliances working with questions and answers of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student
Definitions
- Patent Document 1 Japanese Patent Application Laid-Open No. 2019-211962
- the learning device of the first aspect is a learning device that learns one or more human nodding motions, and includes a learning unit.
- the learning unit receives information about the nodding motion and generates a trained model that outputs nodding motion evaluation information.
- the learning unit learns the information about the nodding motion and the evaluation information of the nodding motion as a learning data set to generate a trained model.
- this learning device by learning information about nodding motions, which are reactions of attendees of meetings, etc., and evaluation information of nodding motions, as a learning data set, it is possible to perform qualitative evaluation of group activities such as meetings. You can generate a trained model that can
- An evaluation information output device is an evaluation information output device that outputs evaluation information related to nodding motions based on one or more human nodding motions, comprising a first information acquisition unit and an output unit. , provided.
- the first information acquisition unit acquires information about the nodding motion.
- the output unit has a trained model that receives information about the nodding motion and outputs nodding motion evaluation information.
- the output unit estimates and outputs evaluation information of the nodding motion from information about the nodding motion using the trained model.
- the trained model is obtained by learning information about the nodding motion and evaluation information of the nodding motion as a learning data set.
- a trained model is used to estimate evaluation information of nodding motions from information about nodding motions that are reactions of attendees of meetings, etc., thereby performing a qualitative evaluation of group activities such as meetings. be able to.
- the evaluation information output device of the third aspect is the device of the second aspect and further includes a second information acquisition unit.
- the second information acquisition unit acquires information about an object that causes a nodding motion.
- the object causing the nodding motion includes at least one of speech during a meeting, data or objects presented in a presentation, or a human or animal performance.
- the output unit outputs the information about the target that causes the nodding motion and the evaluation information of the nodding motion in association with each other.
- this evaluation information output device by linking and outputting information about an object that causes a nodding motion and evaluation information of the nodding motion, the evaluation information of the nodding motion caused by the specific target is output. can be obtained.
- the evaluation information output device of the fourth viewpoint is the device of the second or third viewpoint, and the evaluation information of the nodding motion is information related to human emotions.
- the information about human emotion includes information about human emotions, or information about whether or not a human being is satisfied with an object that causes a nodding motion.
- the nodding motion evaluation information can be obtained by analysis of human face image data or voice data analysis.
- this evaluation information output device it is possible to determine the quality of a nod by estimating the intention of a human's nodding motion from the information on the nodding motion.
- the evaluation information output device of the fifth aspect is the device of the second or third aspect, and the evaluation information of the nodding motion is an evaluation of the object that causes the nodding motion.
- the evaluation of the subject responsible for the nodding motion is the result of a manual questionnaire given to humans.
- this evaluation information output device it is possible to determine the quality of a nodding action by obtaining an evaluation of the subject that causes the nodding action in a questionnaire.
- the evaluation information output device of the sixth aspect is the device of any one of the second to fifth aspects, and the information about the nodding motion includes at least one of the number of nodding motions, the period, or the amplitude.
- this evaluation information output device it is possible to obtain information on the reactions of attendees at a conference, etc. by measuring the number of nodding movements, the period, or the amplitude.
- the evaluation information output device of the seventh aspect is the device of any one of the second aspect to the fifth aspect, wherein the information about the nodding motion is the timing of the nodding motion, the strength of the nodding motion, or the , the presence or absence of a person with the same nodding motion timing, or at least one of the number of people with the same nodding motion timing.
- This evaluation information output device measures the timing of a nodding motion, the strength of a nodding motion, the presence or absence of a person nodding at the same time among a plurality of people, or the number of people nodding at the same time. It is possible to obtain information on the reaction of
- the evaluation information output device of the eighth aspect is the device of the third aspect, and the information on the nodding motion includes the time from the speech during the meeting to the occurrence of the nodding motion.
- this evaluation information output device it is possible to determine the quality of a nodding motion by measuring the time from the time a speech is made during a meeting until the nodding motion occurs.
- the evaluation information output device of the ninth aspect is the device of any one of the second aspect to the fifth aspect, wherein the information about the nodding motion is at least one of the frequency of the nodding motion and the frequency of the nodding motion per unit time. including.
- the quality of the nodding motion can be determined by acquiring the frequency of the nodding motion or the frequency of the nodding motion per unit time.
- the evaluation information output device is the device according to any one of the second aspect to the ninth aspect, wherein the first information acquisition unit comprises a seat sensor, motion capture, acceleration sensor, depth camera, or human moving image. is used to obtain information about the nodding motion obtained by recognizing the nodding motion.
- the first information acquisition unit comprises a seat sensor, motion capture, acceleration sensor, depth camera, or human moving image.
- this evaluation information output device by using a seat sensor, motion capture, acceleration sensor, depth camera, or human video, it is possible to easily acquire information related to nodding gestures during meetings.
- FIG. 2 is a functional block diagram of the learning device 1;
- FIG. 3 is a functional block diagram of the evaluation information output device 100;
- FIG. 4 is a diagram showing an example of learning data; It is a figure for demonstrating a learning process. 4 is a flowchart of the evaluation information output device 100;
- 2 is a functional block diagram of an evaluation information output device 200;
- FIG. 2 shows an evaluation information output device 100 of this embodiment.
- the evaluation information output device 100 is implemented by a computer.
- the evaluation information output device 100 includes a first information acquisition section 101 and an output section 102 .
- the first information acquisition unit 101 acquires information 11 regarding nodding motions.
- the output unit 102 has a trained model 40 and uses the trained model 40 to estimate evaluation information 21 of nodding motions from information 11 related to nodding motions and outputs the estimated nodding motion evaluation information 21 .
- the first information acquisition unit 101 acquires information 11 regarding a nodding motion.
- the information 11 on the nodding motion is the number of nodding motions.
- the first information acquisition unit 101 acquires the number of nodding motions using a seat surface sensor (not shown).
- the number of nodding movements is measured using a seat surface sensor (pressure sensor).
- the participants of the conference are sitting in chairs on which seat sensors are installed.
- the position of the center of gravity P and the total load WT are obtained based on the magnitude of the pressure applied to the seat surface sensors installed at the four corners of the chair.
- the seat surface sensor is made to learn the change in the center of gravity P and the magnitude of the total load WT by machine learning, and detects whether or not it corresponds to the nodding motion.
- the seat sensor detects the timing of a person's nodding from the change in the weight of the center of gravity.
- the seat sensor divides the time during the meeting into a plurality of frames and calculates the amount of nodding, which is the number of nodding movements. Whether or not each participant in the conference is nodding is determined on a frame-by-frame basis. For participant k, the nodding amount akt at a certain time t is calculated by the following equation 1, where Sk is the total number of nodding frames of participant k . Formula 1:
- Equation 2 For example, if there are six participants, the amount of nodding for that frame is calculated by adding up the frames assigned to each participant for all six participants.
- the nodding amount A t at a certain time t is calculated by the following equation 2. Equation 2:
- Equation 1 by weighting the reciprocal of the total number of nodding frames for each participant, when calculating the amount of nodding in Equation 2, the nodding of people who often nod has little effect, and the nods of people who do not often nod. The nod is making the impact bigger.
- the output unit 102 has a trained model 40 and uses the trained model 40 to estimate and output nodding motion evaluation information 21 from the nodding motion related information 11 .
- a processor such as a CPU or GPU can be used for the output unit 102 .
- the output unit 102 reads a program stored in a storage device (not shown) and performs predetermined image processing and arithmetic processing according to the program. Furthermore, the output unit 102 can write the calculation result to the storage device and read the information stored in the storage device according to the program.
- a storage device can be used as a database.
- the trained model 40 used by the evaluation information output device 100 will be described.
- the trained model 40 uses an estimation model that has been trained in advance using the learning apparatus 1 with a training data set that includes information about the nodding motion and evaluation information of the nodding motion.
- the learning device 1 of this embodiment is shown in FIG.
- the learning device 1 is implemented by a computer.
- the learning device 1 includes a learning section 2 .
- the learning unit 2 learns the information 10 regarding the nodding motion and the nodding motion evaluation information 20 as the learning data set 30 .
- the information 10 on the nodding motion is the number of nodding motions (amount of nodding).
- the nodding motion evaluation information 20 is an evaluation of the object that causes the nodding motion.
- the object that causes the nodding motion is the speech during the meeting. Speech during the meeting shall relate to the ideas generated during the meeting.
- information 10 related to the nodding motion is obtained using a seat surface sensor.
- the nodding motion evaluation information 20 is obtained by a manual questionnaire.
- the trained model 40 is obtained by learning the information 10 on the nodding motion and the nodding motion evaluation information 20 using the learning data set 30 .
- FIG. 1 An example of learning data is shown in FIG.
- the number of nodding movements (amount of nodding) of the participants of the conference when an idea is generated during the conference is used as the information on the nodding movements.
- the average value of the results of the questionnaire regarding the ideas generated during the meeting is used as evaluation information for the nodding motion.
- a questionnaire is given to the participants of the conference, and the ideas are evaluated on a 5-point scale.
- the nodding amount A1 for the idea X1 generated at time t1 during the meeting is input, and learning is performed using a teacher data set whose output is the average value "2" of the questionnaire results.
- learning is performed using a teacher data set in which the amount of nodding A 2 for idea X 2 generated at time t 2 during the meeting is input, and the average value of the questionnaire results "3" is output.
- learning is performed using a teacher data set in which the amount of nodding A 3 for idea X 3 generated at time t 3 during the meeting is input, and the average value of the questionnaire results "4" is output.
- learning is performed using a teacher data set in which the amount of nodding An for an idea Xn generated at time tn during the meeting is input, and the average value "2" of the questionnaire results is output.
- Fig. 4 shows an example of a scatter diagram created with the vertical axis representing the total amount of nodding, which is the sum of all participants' nodding frames, and the horizontal axis representing the average value of the questionnaire results.
- 13 groups of 6 participants are participating in the conference.
- the total nodding amount for each group is simply calculated by summing all the nodding frames of the participants for each group.
- FIG. 1 A flowchart of the evaluation information output device 100 is shown in FIG. In this embodiment, a case where the evaluation information output device 100 is used in a conference will be described.
- the meeting is started in step S1.
- Conference attendees are filmed on camera. Also, conference attendees are sitting in chairs with pressure sensors installed.
- the first information acquisition unit 101 acquires the number of nodding motions (amount of nodding) of the participants in the conference (step S2).
- the output unit 102 estimates the average value of the questionnaire results from the number of nodding actions of the participants in the meeting (step S3).
- the output unit 102 outputs the average value of the questionnaire results estimated in step S3 to a display (not shown) (step S4).
- the display shows the estimated mean value of the questionnaire results.
- the learning device 1 is a learning device 1 that learns nodding motions of one or more people, and includes a learning unit 2 .
- the learning unit 2 generates a trained model 40 that receives information about the nodding motion and outputs nodding motion evaluation information.
- the learning unit 2 learns the nodding motion information 10 and nodding motion evaluation information 21 as a learning data set 40 to generate a trained model 30 .
- this learning device 1 by learning information 10 on nodding motions, which are reactions of attendees of a meeting, etc., and evaluation information 20 of the nodding motions as a learning data set 30, qualitative evaluation of group activities such as meetings is performed. can generate a trained model 40 that can perform
- the evaluation information output device 100 is an evaluation information output device 100 that outputs evaluation information related to nodding motions based on one or more human nodding motions, and includes a first information acquisition unit 101 and , and an output unit 102 .
- the first information acquisition unit 101 acquires information 11 regarding nodding motions.
- the output unit 102 has a trained model 40 that receives information 11 on a nodding motion and outputs nodding motion evaluation information 21 .
- the output unit 102 uses the trained model 40 to estimate the nodding motion evaluation information 21 from the nodding motion related information 11 and outputs the estimated nodding motion evaluation information 21 .
- the trained model 40 is obtained by learning the information 10 on the nodding motion and the nodding motion evaluation information 20 as the learning data set 30 .
- This evaluation information output device 100 uses a trained model 30 to estimate the evaluation information of the nodding motion from the information 11 on the nodding motion, which is the reaction of the attendees of the meeting or the like. can be evaluated.
- the output unit 102 of the evaluation information output device 100 can present the intellectual productivity score of the meeting as nodding motion evaluation information based on the number of nodding motions along the elapsed time of the meeting.
- the evaluation information output device 100 evaluates the intellectual productivity of the conference based on the nodding motions of the participants and their timing.
- the synchrony of the participants may be evaluated based on the timing and length of nodding of the participants. By evaluating the nod after the speech, the degree of contribution of the speech in the meeting can be evaluated.
- the nodding motion evaluation information 21 is an evaluation of the object that causes the nodding motion.
- the evaluation of the subject responsible for the nodding motion is the result of a manual questionnaire given to humans.
- the quality of the nodding motion can be determined by obtaining an evaluation of the subject that causes the nodding motion in a questionnaire.
- the information 11 related to the nodding motion includes the number of nodding motions.
- This evaluation information output device 100 measures the number of nodding motions and calculates the amount of nodding, thereby obtaining information on the reactions of attendees at meetings and the like.
- the first information acquisition unit 101 acquires the information 11 regarding the nodding motion obtained by recognizing the nodding motion using the seat surface sensor.
- this evaluation information output device 100 it is possible to easily measure the number 11 of nodding gestures during a meeting by using a seat sensor.
- FIG. 6 shows the evaluation information output device 200 of Modification 1A.
- the evaluation information output device 200 is implemented by a computer.
- the evaluation information output device 200 includes a first information acquisition section 201 , a second information acquisition section 203 and an output section 202 .
- the first information acquisition unit 201 acquires the information 11 regarding the nodding motion.
- the second information acquisition unit 203 acquires the information 50 regarding the object that causes the nodding motion.
- the information 50 related to the object that causes the nodding motion is the utterance during the meeting.
- the second information acquisition unit 203 shoots a moving image using a camera (not shown), and acquires ideas, which are utterances during the meeting, from the moving image.
- the output unit 202 outputs, as output data 60, data in which the information 50 regarding the object that causes the nodding motion and the evaluation information 21 of the nodding motion are linked. Also, the output unit 202 may output evaluation data in which the minutes information of the meeting developed in chronological order and the intellectual productivity score, which is the evaluation information 21 of the nodding motion, are combined.
- the information 50 related to the target causing the nodding motion and the evaluation information 21 of the nodding motion are output in association with each other. evaluation information can be obtained.
- the information 11 regarding the nodding motion is the number of nodding motions, but it may be the period or the amplitude of the nodding motion.
- the first information acquisition unit 101 may acquire information related to nodding, including the timing of the nodding motion. By acquiring the timing of the nodding motion, it is possible to grasp the number of participants who nodded at the meeting. The greater the number of people who nodded, the higher the possibility that an idea was created before the nodding motion. Nodding is more common when ideas are generated than when there is no dialogue.
- the information 11 related to the nodding motion includes the timing of the nodding motion such as whether the participant nods immediately after speaking during the meeting or nods after a delay, the strength of the nodding motion, the presence or absence of people nodding at the same time, and the number of such people. good.
- the characteristics of a nod by using the time from the idea to the occurrence of the nodding motion, the length of the nodding motion (frequency), the number of nodding motions (frequency per unit time), etc. . Since the time from coming up with an idea to nodding is captured in a video, it is digitized by extracting the time difference from the utterance timing. Also, the frequency of the nodding motion is digitized by extracting the frequency contained in the image.
- the number of nodding motions which is the information 11 related to nodding motions, is the total amount of nodding for each group of participants in the conference, but the present invention is not limited to this.
- the number of nodding motions performed by each of the participants in the conference may be used as the information 11 regarding the nodding motions.
- the evaluation information 21 of the nodding motion is the result of a manual questionnaire given to humans, and a questionnaire was conducted as to whether the ideas generated during the meeting were impressive. Although the case has been described, it is not limited to this. A questionnaire may be conducted as to whether the ideas generated during the meeting are original, convincing, or socially acceptable.
- the object causing the nodding motion has been described as being an idea, which is an utterance during a meeting, but the present invention is not limited to this.
- the object that causes the nodding motion may include at least one of data or objects presented in the presentation, or a human or animal performance.
- the evaluation information 21 of the nodding motion is the evaluation regarding the object causing the nodding motion, but it may be information regarding human emotions.
- Information about human emotions includes information about whether or not a person is satisfied with an object that causes human emotions or a nodding motion.
- the nodding motion evaluation information can be obtained by analysis of human face image data or voice data analysis.
- the quality of the nodding can be determined by estimating the intention of the human nodding motion from the information 11 regarding the nodding motion.
- Motion capture can estimate the movement of the head using a motion capture device such as OptiTrack.
- a motion capture device such as OptiTrack.
- JIN Meme etc. are mentioned as an acceleration sensor.
- An acceleration sensor can be used to capture head movements.
- a depth camera or kinect sensor can also be used to measure the number of nodding movements.
- nodding motions can be detected from head movements based on video of participants during the conference.
- learning device 2 learning units 100 and 200 evaluation information output devices 101 and 201 first information acquisition unit 203 second information acquisition units 102 and 202 output units 10 and 11 nodding motion information 20 and 21 nodding motion evaluation information 30 for learning Data set 40 Trained model 50 Information about the object causing the nodding motion
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Entrepreneurship & Innovation (AREA)
- Strategic Management (AREA)
- Educational Administration (AREA)
- Economics (AREA)
- Software Systems (AREA)
- Development Economics (AREA)
- Data Mining & Analysis (AREA)
- Tourism & Hospitality (AREA)
- Marketing (AREA)
- General Business, Economics & Management (AREA)
- Quality & Reliability (AREA)
- Operations Research (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Game Theory and Decision Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Educational Technology (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Description
本実施形態の評価情報出力装置100を図2に示す。評価情報出力装置100は、コンピュータにより実現されるものである。評価情報出力装置100は、第1情報取得部101と、出力部102と、を備える。第1情報取得部101は、うなずき動作に関する情報11を取得する。出力部102は、学習済みモデル40を有し、学習済みモデル40を用いてうなずき動作に関する情報11からうなずき動作の評価情報21を推定して出力する。
(2-1)第1情報取得部
第1情報取得部101は、うなずき動作に関する情報11を取得する。うなずき動作に関する情報11は、うなずき動作の回数である。本実施形態では、第1情報取得部101は、座面センサ(図示せず)を使ってうなずき動作の回数を取得する。
式1:
出力部102は、学習済みモデル40を有し、学習済みモデル40を用いてうなずき動作に関する情報11からうなずき動作の評価情報21を推定して出力する。出力部102には、CPU又はGPUといったプロセッサを使用できる。出力部102は、記憶装置(図示せず)に記憶されているプログラムを読み出し、このプログラムに従って所定の画像処理や演算処理を行う。さらに、出力部102は、プログラムに従って、演算結果を記憶装置に書き込んだり、記憶装置に記憶されている情報を読み出したりすることができる。記憶装置は、データベースとして用いることができる。
評価情報出力装置100が使用する学習済みモデル40について説明する。学習済みモデル40は、学習装置1を使ってあらかじめうなずき動作に関する情報と、うなずき動作の評価情報を含む学習用データセットによって学習させた推定モデルを用いる。
評価情報出力装置100のフローチャートを図5に示す。本実施形態では、評価情報出力装置100を会議で使用する場合について説明する。
(5-1)
本実施形態に係る学習装置1では、1又は複数の人間のうなずき動作を学習する学習装置1であって、学習部2を備える。学習部2は、うなずき動作に関する情報を入力とし、うなずき動作の評価情報を出力する、学習済みモデル40、を生成する。学習部2は、うなずき動作に関する情報10と、うなずき動作の評価情報21とを学習用データセット40として学習して、学習済みモデル30を生成する。
本実施形態に係る評価情報出力装置100では、1又は複数の人間のうなずき動作に基づいて、前記うなずき動作に関する評価情報を出力する、評価情報出力装置100であって、第1情報取得部101と、出力部102と、を備える。第1情報取得部101は、うなずき動作に関する情報11を取得する。出力部102は、うなずき動作に関する情報11を入力とし、うなずき動作の評価情報21を出力する、学習済みモデル40を有する。出力部102は、学習済みモデル40を用いてうなずき動作に関する情報11からうなずき動作の評価情報21を推定して出力する。学習済みモデル40は、うなずき動作に関する情報10と、うなずき動作の評価情報20とを学習用データセット30として学習したものである。
本実施形態に係る評価情報出力装置100では、うなずき動作の評価情報21は、うなずき動作の起因となる対象に対する評価である。うなずき動作の起因となる対象に対する評価は、人間に対して行われる、手動のアンケートの結果である。
本実施形態に係る評価情報出力装置100では、うなずき動作に関する情報11は、うなずき動作の回数を含む。
本実施形態に係る評価情報出力装置100では、第1情報取得部101は、座面センサを用いて、うなずき動作を認識することで得られたうなずき動作に関する情報11を取得する。
(6-1)変形例1A
本実施形態に係る評価情報出力装置100では、第1情報取得部101と、出力部102とを備える場合について説明したが、第2情報取得部をさらに備えるようにしてもよい。
本実施形態に係る評価情報出力装置100では、うなずき動作に関する情報11がうなずき動作の回数である場合について説明したが、うなずき動作の周期、又は振幅でもよい。
本実施形態に係る評価情報出力装置100では、うなずき動作に関する情報11であるうなずき動作の回数が会議の参加者のグループ毎の総うなずき量である場合について説明したが、これに限るものではない。会議の参加者の各個人のうなずき動作の回数を、うなずき動作に関する情報11としてもよい。
本実施形態に係る評価情報出力装置100では、うなずき動作の評価情報21が人間に対して行われる手動のアンケートの結果であり、会議中に発生したアイデアが印象的であるかについてアンケートを行った場合について説明したがこれに限るものではない。会議中に発生したアイデアが、独創的である、納得できる、または社会的に受け入れられるかについて、アンケートを行うようにしてもよい。
本実施形態に係る評価情報出力装置100では、うなずき動作の起因となる対象は、会議中の発言であるアイデアである場合に説明したがこれに限るものではない。うなずき動作の起因となる対象は、プレゼンテーションで提示されたデータもしくは物、又は人もしくは動物のパフォーマンスの少なくとも1つを含むようにしてもよい。
本実施形態に係る評価情報出力装置100では、うなずき動作の評価情報21がうなずき動作の起因となる対象に関する評価である場合について説明したが、人間の感情に関する情報でもよい。人間の感情に関する情報は、人間の喜怒哀楽、またはうなずき動作の起因となる対象に対して納得しているか否かに関する情報を含む。うなずき動作の評価情報は、人間の顔面の画像データの解析又は音声データの解析によって取得できる。
本実施形態に係る評価情報出力装置100では、第1情報取得部101が座面センサからうなずき動作に関する情報を取得する場合について説明したが、これに限るものではない。モーションキャプチャ、加速度センサ、デプスカメラ、又は人間の動画を用いて、うなずき動作を認識することで得られたうなずき動作に関する情報を取得するようにしてもよい。
以上、本開示の実施形態を説明したが、特許請求の範囲に記載された本開示の趣旨及び範囲から逸脱することなく、形態や詳細の多様な変更が可能なことが理解されるであろう。
2 学習部
100、200 評価情報出力装置
101、201 第1情報取得部
203 第2情報取得部
102、202 出力部
10、11 うなずき動作に関する情報
20、21 うなずき動作の評価情報
30 学習用データセット
40 学習済みモデル
50 うなずき動作の起因となる対象に関する情報
Claims (10)
- 1又は複数の人間のうなずき動作を学習する学習装置であって、
前記うなずき動作に関する情報(10)を入力とし、前記うなずき動作の評価情報(20)を出力する、学習済みモデル(40)、を生成する、学習部(2)、
を備え、
前記学習部は、前記うなずき動作に関する情報と、前記うなずき動作の評価情報とを学習用データセット(30)として学習して、前記学習済みモデルを生成する、
学習装置(1)。 - 1又は複数の人間のうなずき動作に基づいて、前記うなずき動作に関する評価情報を出力する、評価情報出力装置であって、
前記うなずき動作に関する情報(11)を取得する、第1情報取得部(101、201)と、
前記うなずき動作に関する情報を入力とし、前記うなずき動作の評価情報を出力する、学習済みモデルを有し、前記学習済みモデルを用いて前記うなずき動作に関する情報から前記うなずき動作の評価情報(21)を推定して出力する、出力部(102、202)と、
を備え、
前記学習済みモデルは、前記うなずき動作に関する情報と、前記うなずき動作の評価情報とを学習用データセットとして学習したものである、
評価情報出力装置(100、200)。 - 前記うなずき動作の起因となる対象に関する情報(50)を取得する、第2情報取得部(203)、
をさらに備え、
前記うなずき動作の起因となる対象は、会議中の発言、プレゼンテーションで提示されたデータもしくは物、又は人もしくは動物のパフォーマンスの少なくとも1つを含み、
前記出力部は、前記うなずき動作の起因となる対象に関する情報と、前記うなずき動作の評価情報とを紐づけて出力する、
請求項2に記載の評価情報出力装置。 - 前記うなずき動作の評価情報は、前記人間の感情に関する情報であり、
前記人間の感情に関する情報は、前記人間の喜怒哀楽に関する情報、または前記うなずき動作の起因となる対象に対して前記人間が納得しているか否かに関する情報を含み、
前記うなずき動作の評価情報は、前記人間の顔面の画像データの解析又は音声データの解析によって取得できる、
請求項2又は3に記載の評価情報出力装置。 - 前記うなずき動作の評価情報は、前記うなずき動作の起因となる対象に対する評価であり、
前記うなずき動作の起因となる対象に対する評価は、前記人間に対して行われる、手動のアンケートの結果である、
請求項2又は3に記載の評価情報出力装置。 - 前記うなずき動作に関する情報は、前記うなずき動作の回数、周期、又は振幅の少なくとも1つを含む、
請求項2から5のいずれかに記載の評価情報出力装置。 - 前記うなずき動作に関する情報は、前記うなずき動作のタイミング、前記うなずき動作の強さ、又は、前記複数の人間のうち、前記うなずき動作のタイミングが同一である、前記人間の有無、若しくは前記うなずき動作のタイミングが同一である人数の少なくとも1つを含む、
請求項2から5のいずれかに記載の評価情報出力装置。 - 前記うなずき動作に関する情報は、前記会議中の発言後から前記うなずき動作が発生するまでの時間を含む、
請求項3に記載の評価情報出力装置。 - 前記うなずき動作に関する情報は、前記うなずき動作の周波数、又は単位時間あたりの前記うなずき動作の頻度の少なくとも1つを含む、
請求項2から5のいずれかに記載の評価情報出力装置。 - 前記第1情報取得部は、座面センサ、モーションキャプチャ、加速度センサ、デプスカメラ、又は前記人間の動画を用いて、前記うなずき動作を認識することで得られた前記うなずき動作に関する情報を取得する、
請求項2から9のいずれかに記載の評価情報出力装置。
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US18/275,182 US20240112083A1 (en) | 2021-02-05 | 2022-02-07 | Learning apparatus and evaluation information output apparatus |
EP22749844.1A EP4290429A1 (en) | 2021-02-05 | 2022-02-07 | Learning device and evaluation information output device |
CN202280013277.0A CN116888606A (zh) | 2021-02-05 | 2022-02-07 | 学习装置以及评价信息输出装置 |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2021-017261 | 2021-02-05 | ||
JP2021017261 | 2021-02-05 | ||
JP2022001614A JP7260826B2 (ja) | 2021-02-05 | 2022-01-07 | 学習装置および評価情報出力装置 |
JP2022-001614 | 2022-01-07 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022168973A1 true WO2022168973A1 (ja) | 2022-08-11 |
Family
ID=82741589
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2022/004684 WO2022168973A1 (ja) | 2021-02-05 | 2022-02-07 | 学習装置および評価情報出力装置 |
Country Status (3)
Country | Link |
---|---|
US (1) | US20240112083A1 (ja) |
EP (1) | EP4290429A1 (ja) |
WO (1) | WO2022168973A1 (ja) |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2019211962A (ja) | 2018-06-04 | 2019-12-12 | 株式会社東芝 | 会議生産性可視化システム、会議生産性可視化装置、可視化方法およびプログラム |
-
2022
- 2022-02-07 EP EP22749844.1A patent/EP4290429A1/en active Pending
- 2022-02-07 US US18/275,182 patent/US20240112083A1/en active Pending
- 2022-02-07 WO PCT/JP2022/004684 patent/WO2022168973A1/ja active Application Filing
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2019211962A (ja) | 2018-06-04 | 2019-12-12 | 株式会社東芝 | 会議生産性可視化システム、会議生産性可視化装置、可視化方法およびプログラム |
Non-Patent Citations (3)
Title |
---|
INOUE KOJI, DIVESH LALA, YOSHII KAZUYOSHI, TAKANASHI KATSUYA, KAWAHARA TATSUYA: "Engagement Recognition from Listener’s Behaviors in Spoken Dialogue Using a Latent Character Model", TRANSACTIONS OF THE JAPANESE SOCIETY FOR ARTIFICIAL INTELLIGENCE, 社団法人 人工知能学会, vol. 33, no. 1, 5 January 2018 (2018-01-05), pages 1 - 12, XP055956787, ISSN: 1346-0714, DOI: 10.1527/tjsai.DSH-F * |
KENTO NISHIMURAT, YUICHI ITOHT, KEN FUJIWARA, KAZUYUKI FUJITA, YUKO MATSU, KEN HIKONO, TAKAO ONOYE: "A Study on the Relationship between Communication and Nodding in an Idea Generation Task by Sense Chair", 2017 INTERNATIONAL SYMPOSIUM ON NONLINEAR THEORY AND ITS APPLICATIONS, NOLTA2017, CANCUN, MEXICO, DECEMBER 4-7, 2017, IEICE, JP, vol. 120, no. 136, 14 January 2021 (2021-01-14), JP , pages 65 - 70, XP009538694 * |
YAMAMOTO GOH, SAKANE YUTAKA, TAKEBAYASHI YOICHI: "Nod detection and Retrieval of meaningful communications with a Multimodal headset", IPSJ SIG TECHNICAL REPORT, no. 94, 26 June 2003 (2003-06-26), pages 13 - 19, XP055956797, ISSN: 2188-8736 * |
Also Published As
Publication number | Publication date |
---|---|
US20240112083A1 (en) | 2024-04-04 |
EP4290429A1 (en) | 2023-12-13 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Bailenson et al. | The effect of behavioral realism and form realism of real-time avatar faces on verbal disclosure, nonverbal disclosure, emotion recognition, and copresence in dyadic interaction | |
US20180078184A1 (en) | Dual-task performing ability evaluation method and dual-task performing ability evaluation system | |
US9848170B2 (en) | System and method for improving communication productivity | |
JP2019055064A (ja) | 診断支援情報提供装置及び診断支援情報提供方法 | |
WO2022025200A1 (ja) | 反応解析システムおよび反応解析装置 | |
JP6715410B2 (ja) | 評価方法、評価装置、評価プログラム、および、評価システム | |
JP2021071773A (ja) | 動作評価装置、動作評価方法、動作評価システム | |
JP2013008114A (ja) | 会議品質評価装置及び会議品質評価方法 | |
Samad et al. | A pilot study to identify autism related traits in spontaneous facial actions using computer vision | |
WO2019146405A1 (ja) | 表情解析技術を用いた商品に対するモニタの反応を評価するための情報処理装置、情報処理システム、プログラム | |
KR20190140767A (ko) | 몰입도 측정장치 및 방법 | |
WO2022168973A1 (ja) | 学習装置および評価情報出力装置 | |
JP7260826B2 (ja) | 学習装置および評価情報出力装置 | |
Palazzi et al. | Spotting prejudice with nonverbal behaviours | |
CN116888606A (zh) | 学习装置以及评价信息输出装置 | |
CN114998440B (zh) | 基于多模态的测评方法、装置、介质及设备 | |
Hakura et al. | An exploration toward emotion estimation from facial expressions for systems with quasi-personality | |
JP2020107216A (ja) | 情報処理装置、その制御方法およびプログラム | |
WO2023084715A1 (ja) | 情報処理装置、情報処理方法、及びプログラム | |
JP7014761B2 (ja) | 認知機能推定方法、コンピュータプログラム及び認知機能推定装置 | |
JP2019105972A (ja) | 情報処理装置及びプログラム | |
JP2019146897A (ja) | 顔の肌の追従性の推定方法、推定装置及び推定プログラム、並びに、皮下組織の粘弾性の推定方法、推定装置及び推定プログラム | |
Fang et al. | Estimation of cohesion with feature categorization on small scale groups | |
WO2022064622A1 (ja) | 感情解析システム | |
Artiran et al. | Analysis of Gaze, Head Orientation and Joint Attention in Autism with Triadic VR Interviews |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22749844 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 18275182 Country of ref document: US |
|
WWE | Wipo information: entry into national phase |
Ref document number: 202280013277.0 Country of ref document: CN |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 2022749844 Country of ref document: EP Effective date: 20230905 |