JP7066115B2 - Public speaking support device and program - Google Patents

Description

The present invention relates to a public speaking support device and a program.

In learning public speaking techniques, it has been pointed out that it is important to practice while experiencing the situation as if it were a practice. Therefore, even if the public speaking technique is taught in lectures and exercises, it is difficult to learn by self-practice. On the other hand, practicing public speaking in a situation that is just like practice was difficult due to factors such as securing audience personnel, securing a place, and difficulty in self-evaluation.

A technique is known in which face information regarding the orientation of a face is calculated from an image taken by a camera, and an index indicating the degree of eye contact is calculated and displayed based on the calculated face information (for example, Patent Document 1). See).
In addition, based on the evaluation of the line of sight detected by the line-of-sight direction detection device, the evaluation value is lowered if the user is always looking at the screen of the personal computer, and the evaluation value is raised if the viewer is looking at the audience. The technique is known (see, for example, Patent Document 2).
Further, there is known a technique in which when it is detected that the presenter is not facing the audience for a certain period of time or more, the behavior is judged to be unfavorable and the presenter is notified that the behavior is unfavorable (for example). , Patent Document 3).

Japanese Unexamined Patent Publication No. 2008-139762 Japanese Unexamined Patent Publication No. 2007-219161 Japanese Unexamined Patent Publication No. 2012-255866

However, in the above-mentioned conventional technique, there is a problem that the trainee cannot learn which direction to turn next in the practice of public speaking.

In addition, in the practice of public speaking, for example, when reproducing the situation as it is practiced by using a device using virtual reality (VR: Virtual Reality) or augmented reality (AR: Augmented Reality) technology, these devices and equipment Is required to have sufficient performance to withstand the processing load of VR video, and the cost is high.

The present invention has been made in view of the above points, and provides a public speaking support device and a program capable of learning which direction the trainee should turn next in the practice of public speaking.

Further, in the practice of public speaking, a public speaking support device capable of reducing the burden on the device when reproducing the situation as if it were practiced will be provided. Here, the situation just like practice is a situation in which the trainee feels the gaze of the audience and feels a sense of tension.

(1) The present invention has been made to solve the above-mentioned problems, and one aspect of the present invention is a detection unit that detects the orientation of the head of the trainee and a head detected by the detection unit. A field image that is an image as a virtual reality in a direction corresponding to the orientation of the part , or an augmented reality image that is an image that is displayed superimposed on the field of view of the trainee or an image of the field of view as augmented reality in the direction. A display unit that displays the image in a direction opposite to the direction of the head, and a position on the field image that indicates the direction of the head detected by the detection unit when the display unit displays the field image. By displaying information indicating a position on the field image different from the above on the display unit, or when the display unit displays the augmented reality image, the orientation of the head detected by the detection unit is determined. The head detected by the detection unit by displaying information indicating the position on the image of the field of view or the field of view different from the position on the image of the field of view or the field of view as the augmented reality image on the display unit. It is a public speaking support device including a motion index presentation unit that presents a head orientation different from the orientation of the unit.

(2) Further, in one aspect of the present invention, in the above public speaking support device, the orientation of the head detected by the detection unit is analyzed, and the field image showing the orientation of the head from the analysis result , the said. The public, which is information in which the head movement analysis unit that generates head movement information, which is information indicating the position of the visual field or the position on the image of the visual field, and the movement of the head and the evaluation of the movement of the head are associated with each other. The movement of the head is evaluated based on the storage unit in which the speaking evaluation information is stored, the head movement information generated by the head motion analysis unit, and the public speaking evaluation information stored in the storage unit. From the evaluation result, the field image different from the position indicated by the head motion information, the performance evaluation unit that generates the head motion instruction information which is the information indicating the position on the visual field or the image of the visual field, and the performance evaluation unit are further provided. The motion index presenting unit is detected by the detection unit by displaying on the display unit the field image indicated by the head motion instruction information and information indicating the field or the position of the field on the image. Presents a head orientation that is different from the head orientation.

(3) Further, in one aspect of the present invention, in the public speaking support device, the motion index presenting unit positions the position indicated by the head motion information and the position indicated by the head motion instruction information with each other. It is displayed on the display unit in a distinguishable manner.

(4) Further, one aspect of the present invention further includes a performance evaluation presentation unit in the public speaking support device, and the performance evaluation presentation unit is based on the head motion information and the public speaking evaluation information. The performance evaluation information, which is information indicating the evaluation of the public speaking performance of the trainee, is generated, and the generated performance evaluation information is displayed on the display unit to present the performance evaluation.

(5) Further, one aspect of the present invention is based on the results of analyzing the voice recording unit that records the voice of the trainee and the voice recorded by the voice recording unit in the public speaking support device. A voice analysis unit that generates speech speed information, which is information indicating the speaking speed of the trainee, is further provided, and the public speaking evaluation information stored in the storage unit includes the speaking speed information and evaluation of public speaking. The performance evaluation presentation unit further includes the information associated with the above, generates performance evaluation information based on the speech speed information and the public speaking evaluation information, and displays the generated performance evaluation information on the display unit. By letting it present, the evaluation of the performance is presented.

(6) Further, in one aspect of the present invention, in the public speaking support device, the voice analysis unit generates the speech speed information in units of syllables.

(7) Further, in one aspect of the present invention, in the public speaking support device, the voice analysis unit analyzes the voice recorded by the voice recording unit, and the voice volume of the trainee is shown from the analysis result. The public speaking evaluation information that generates voice volume information, which is information, and is stored in the storage unit further includes information in which the voice volume information is associated with the public speaking evaluation, and the performance evaluation presentation unit is described. Performance evaluation information is generated based on the voice volume information and the public speaking evaluation information, and the generated performance evaluation information is displayed on the display unit to present the performance evaluation.

(8) Further, in one aspect of the present invention, in the above public speaking support device, a heartbeat information generation unit that generates heartbeat information indicating the heartbeat value of the trainee and a heartbeat generated by the heartbeat information generation unit. The public speaking evaluation information stored in the storage unit is further provided with a heart rate variability analysis unit that analyzes the information and generates heart rate variability information that is information indicating the heart rate variability of the trainee from the analysis result. Further, the performance evaluation presenting unit generates performance evaluation information based on the heart rate variability information and the public speaking evaluation information. The performance evaluation is presented by displaying the generated performance evaluation information on the display unit.
(9) Further, one aspect of the present invention is a state in which the degree of tension of the trainee to which the field image displayed on the display unit or the augmented reality image is presented is analyzed in the public speaking support device. The public speaking evaluation information further provided with an analysis unit and stored in the storage unit further includes information in which the tension level of the trainee is associated with the evaluation of the tension level of the trainee, and the performance. The evaluation presentation unit evaluates the tension level of the trainee based on the tension level of the trainee analyzed by the state analysis unit and the public speaking evaluation information stored in the storage unit. The performance evaluation is presented by generating information and displaying the generated tension evaluation information on the display unit.

( 10 ) Further, in one aspect of the present invention, in the public speaking support device, the display unit is a transmissive display unit.

( 11 ) Further, in one aspect of the present invention, the field image or the augmented reality image includes an image of the eyes of an audience.

( 12 ) Further, one aspect of the present invention is a detection step for detecting the orientation of the head of the trainee and a virtual reality of the direction corresponding to the orientation of the head detected by the detection step on the computer. The field image, which is an image , or the augmented reality image, which is an image superimposed on the image of the trainee's field of view or the field of view as augmented reality in the direction, is oriented so as to face the direction of the head. The display step to be displayed on the display unit, and when the display unit displays the field image, the field image showing a direction different from the position on the field image indicating the direction of the head detected by the detection step. The field of view or the field of view indicating the orientation of the head detected by the detection step by displaying information indicating the upper position on the display unit, or when the display unit displays the augmented reality image. By displaying information indicating the field of view or the position of the field of view on the image as the augmented reality image on the display unit, the orientation of the head is different from the direction of the head detected by the detection step. It is a program for executing the motion index presentation step for presenting the direction of the head.

According to the present invention, in the practice of public speaking, the trainee can learn which direction to turn next.

It is a figure which shows an example of the overview of the public speaking support apparatus of 1st Embodiment of this invention. It is a side view which shows an example of the overview of the head of the trainee wearing the public speaking support device of this embodiment. It is a figure which shows an example of the field of view image of the public speaking support apparatus of this embodiment. It is a figure which shows an example of the functional structure of the public speaking support apparatus of this embodiment. It is a figure which shows an example of the head motion model of this embodiment. It is a figure which shows an example of the processing of the public speaking support apparatus of this embodiment. It is a figure which shows an example of the setting screen of the public speaking support device of this embodiment. It is a figure which shows an example of the heart rate measurement screen of the public speaking support device of this embodiment. It is a figure which shows an example of the evaluation presentation screen of the public speaking support apparatus of this embodiment. It is a figure which shows an example of the form of the public speaking support device of this embodiment. It is a figure which shows an example of the form of the public speaking support device of this embodiment. It is a figure which shows an example of the overview of the public speaking support apparatus of the 2nd Embodiment of this invention. It is a figure which shows an example of the functional structure of the public speaking support apparatus of this embodiment. It is a figure which shows an example of the overview of the public speaking support apparatus of 3rd Embodiment of this invention. It is a figure which shows an example of the functional structure of the public speaking support apparatus of this embodiment. It is a figure which shows an example of the field of view image of the public speaking support apparatus of this embodiment. It is a figure which shows an example of the field of view image of the public speaking support apparatus of this embodiment. It is a figure which shows an example of the overview of the public speaking support apparatus of this embodiment.

(First Embodiment)
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In this embodiment, the case of speech in English will be described as an example of public speaking.
FIG. 1 is a diagram showing an example of an overview of the public speaking support device D1 of the present embodiment.
The public speaking support device D1 (hereinafter referred to as the support device D1) takes the form of a head-mounted display (HMD: Head Mounted Display) and is attached to the head of the trainee T1 (hereinafter referred to as the trainee T1) for speech. And used.
The support device D1 includes a sensor (not shown) that detects the orientation of the head of the trainee T1. The support device D1 displays the visual field image in the direction corresponding to the direction of the head of the trainee T1 detected by the sensor on the display unit (not shown) with the direction facing the direction of the head. ..

FIG. 2 is a side view showing an example of an overview of the head of the trainee T1 wearing the support device D1 of the present embodiment.
The coordinate system fixed at the position where the trainee T1 is located is a three-dimensional Cartesian coordinate system X, Y, Z. The coordinate system fixed to the head of the trainee T1 is defined as a three-dimensional Cartesian coordinate system x, y, z. Here, the x-axis of the three-dimensional Cartesian coordinate system x, y, z is the direction of the front surface of the head of the trainee T1, and the z-axis is the vertical upward direction.
The orientation of the head of the trainee T1 HD1 is the positive orientation of the x-axis of the three-dimensional Cartesian coordinate system x, y, z. Head orientation The orientation DD1 relative to HD1 is the negative orientation of the x-axis of the three-dimensional Cartesian coordinate system x, y, z.

FIG. 3 is a diagram showing an example of the field image I1 of the support device D1 of the present embodiment.
The support device D1 displays an image of the scenery of the virtual presentation venue and a moving image of a virtual audience (referred to as a virtual audience) on the visual field image I1. The support device D1 displays an image of the landscape of the presentation venue 360 degrees at an angle around the Z axis in the three-dimensional Cartesian coordinate systems X, Y, and Z according to the orientation HD1 of the head.

The support device D1 moves the virtual audience according to the evaluation of the speech of the trainee T1 evaluated by the own device. The support device D1 displays a virtual audience that reacts negatively when the trainee T1 has a low speech rating. On the other hand, if the trainee T1's speech is highly evaluated, the support device D1 displays a virtual audience that responds positively.
The support device D1 may display a virtual audience that reacts negatively in the first half of the speech, and may display a virtual audience that reacts positively in the latter half of the speech. Alternatively, the assistive device D1 may display a virtual audience that always responds to either positive or negative during the speech.
Here, the negative reaction is, for example, a cheek stick, a doze, a yawn, a frowning face, a face down, etc., and the positive reaction is, for example, a nod, a smile, etc. The trainee T1 can feel psychological feedback from the reaction of the virtual audience.

The support device D1 displays the virtual audience by using computer graphics (CG: Computer Graphics) or a live-action moving image, depending on the setting. The display F3 displays the above settings for the virtual audience and the time of the speech.

The support device D1 detects the eye contact movement (movement of the face and neck) of the trainee T1 as the head orientation HD1. The support device D1 analyzes the detected head orientation HD1 and generates head movement instruction information which is information indicating the position on the visual field image I1 that the trainee T1 should see next.
Basic research on public speaking has shown that it is excellent public speaking to change the place to face (the audience who makes eye contact) at regular intervals (about 5 seconds) and see the entire audience evenly. There is. Head movement instruction information is generated based on the results of this study.

The support device D1 displays the position indicated by the generated head movement instruction information on the visual field image I1. Here, the support device D1 is a translucent panel PN1, PN2, PN3, PN4, PN5, PN6, in which the positions to be seen next by the trainee T1 are displayed side by side in a grid pattern of 2 rows and 5 columns. Select from PN7, PN8, PN9, and PN10. Hereinafter, the panels PN1, PN2, PN3, PN4, PN5, PN6, PN7, PN8, PN9, and PN10 may be collectively referred to as a panel PN. The support device D1 identifies the specific panel from other panels by increasing the brightness of the frame of the specific panel (panel PN3 as an example in FIG. 3 and the following description) selected from the panel PN. to enable. As a result, the support device D1 informs the trainee T1 of the position to be seen next.
Therefore, by using the support device D1, the trainee can learn which direction to turn next in the practice of public speaking.

The support device D1 includes a voice recording device (not shown) and acquires the voice of the trainee T1. The support device D1 analyzes the acquired voice and displays the speaking speed evaluation information, which is information indicating the evaluation of the speaking speed of the trainee T1, as the display F2.
The support device D1 includes a biosensor (not shown) and acquires information indicating the heart rate value of the trainee T1. The support device D1 analyzes the acquired information indicating the heart rate value of the trainee T1, and displays the tension evaluation information, which is the information indicating the evaluation of the tension of the trainee T1, as the display F1.

When the speech is completed, the support device D1 presents the evaluation score of the speech of the trainee T1 based on the movement of the trainee T1's eye contact, the speaking speed, and the heartbeat.

The support device D1 displays the image S1 of the speech material on the podium on the visual field image I1. Here, the image S1 of the speech material is an image of the script or an image of the material of the presentation, which is selected and set in advance by the trainee T1. The trainee T1 can be trained while looking at the image S1 of the speech material in the visual field image I1. The image S1 displayed on the visual field image I1 may be appropriately selected and set by the trainee T1 or the like.
The support device D1 can turn off the display of the image S1 on the visual field image I1 based on the operation by the trainee T1 or the like. When the display is turned off, the support device D1 may display an image of the podium instead of the image S1 on the field image I1. The trainee T1 can practice with a speech or improvisation stored in advance without looking at an image such as a script on the visual field image I1.
Hereinafter, the functional configuration of the support device D1 that realizes that the trainee can learn which direction to turn next in the speech practice will be described with reference to FIG.

[Functional configuration of support device D1]
The support device D1 includes a head-mounted display 1, a mobile terminal 2, and a biosensor 3. The head-mounted display 1 and the mobile terminal 2 may or may not be integrally configured.
The head-mounted display 1 includes a voice recording device 10, a motion sensor 11, an image generation device 12, a display device 13, and a voice output device 14.

The voice recording device 10 acquires the voice of the trainee T1 and records the acquired voice as voice information. The voice recording device 10 supplies the recorded voice information to the mobile terminal 2.
The motion sensor 11 detects the orientation HD1 of the head of the trainee T1. The motion sensor 11 supplies the detected head orientation HD1 to the mobile terminal 2 and the image generation device 12.

The image generation device 12 generates a visual field image I1 in a direction corresponding to the direction HD1 of the head detected by the motion sensor 11 and supplies the visual field image I1 to the display device 13.
The image generation device 12 increases the brightness of the frame of a specific panel (panel PN3) in the panel PN according to the output from the operation index presenting unit 26.
The image generation device 12 acquires the evaluation information of public speaking described later from the performance evaluation presentation unit 25, and generates the display F2 and the display F3 corresponding to the speech speed evaluation information and the tension evaluation information, respectively. The image generation device 12 generates a virtual audience that reacts positively or negatively according to the acquired evaluation information.

The display device 13 displays the visual field image I1 acquired from the image generation device 12 in a direction opposite to the direction HD1 of the head.
The display device 13 displays the position to be seen next by the trainee T1 generated by the mobile terminal 2 on the visual field image I1.
The display device 13 displays the speech speed evaluation information acquired from the mobile terminal 2 on the visual field image I1 as the display F2. When the tension evaluation information generated by the mobile terminal 2 is input, the display device 13 displays the information as the display F1 on the visual field image I1.
At the end of the speech, the display device 13 presents the evaluation score of the speech of the trainee T1 calculated by the mobile terminal 2.
The display device 13 is, for example, a liquid crystal display or an organic electroluminescence (EL) display.

The voice output device 14 outputs voice based on the voice message input from the mobile terminal 2. The audio output device 14 is, for example, a speaker.

The mobile terminal 2 includes a voice analysis unit 20, a head motion analysis unit 21, a heart rate variability analysis unit 22, a performance evaluation unit 23, a speech performance index database 24, a performance evaluation presentation unit 25, an operation index presentation unit 26, and a voice message generation unit. 27 is included.

The voice analysis unit 20 acquires the voice recorded by the voice recording device 10. The voice analysis unit 20 analyzes the acquired voice and generates speech speed information which is information indicating the speed at which the trainee T1 speaks from the analysis result. The voice analysis unit 20 supplies the generated speech speed information to the performance evaluation unit 23.

The voice analysis unit 20 calculates the speed at which the trainee T1 speaks at predetermined time intervals when the speech speed information is generated. Here, the predetermined time interval is, for example, 5 seconds. The voice analysis unit 20 calculates the speed at which the trainee T1 speaks as the number of syllables per unit time by converting the number of words spoken by the trainee T1 at a predetermined time into the number of syllables. That is, the voice analysis unit 20 generates speech speed information in units of syllables.
Since the voice analysis unit 20 calculates the speed at which the trainee T1 speaks in units of syllables, the trainee T1 is based on the common standard of the number of syllables even if the word length differs due to the difference in language. Can calculate the speaking speed. Therefore, the voice analysis unit 20 can support multiple languages.

The head motion analysis unit 21 acquires the head orientation HD1 detected by the motion sensor 11. The head motion analysis unit 21 analyzes the acquired head orientation HD1 and generates head motion information based on the analysis result. Here, the head motion information is information indicating the position of the visual field image I1 indicating the orientation HD1 of the head. The head motion information includes the time when the motion sensor 11 detects the head orientation HD1. The head motion analysis unit 21 supplies the generated head motion information to the performance evaluation unit 23.

The heart rate variability analysis unit 22 acquires the heart rate information of the trainee T1 generated by the biological sensor 3. The heart rate variability analysis unit 22 analyzes the acquired heart rate information, and based on the analysis result, generates heart rate variability information which is information indicating the heart rate variability of the trainee T1. The heart rate variability analysis unit 22 supplies the generated heart rate variability information to the performance evaluation unit 23.

The heart rate variability information indicates whether or not the heart rate has increased more than a predetermined number of times with respect to the normal heart rate of the trainee T1 measured in advance. The predetermined number of times is, for example, five times. The heart rate variability analysis unit 22 measures the normal heart rate value of the trainee T1 while the heart rate measurement screen I3, which will be described later, is displayed on the display device 13.

The performance evaluation unit 23 acquires head movement information from the head movement analysis unit 21. The performance evaluation unit 23 reads the public speaking evaluation information from the speech performance index database 24.
Here, as an example of the public speaking evaluation information, the information in which the movement of the head of the trainee T1 is associated with the evaluation of the movement of the head, the speech speed information, and the evaluation of the public speaking correspond. There is information associated with the attached information, heart rate variability information, and public speaking evaluation.

The public speaking evaluation information is an index calculated based on the accumulation of basic research on public speaking described above. The public speaking evaluation information is higher with respect to the head motion information as the time in which the position of the visual field image I1 indicated by the head motion information coincides with the position of the visual field image I1 indicated by the head motion model is longer. Assign a rating.
Here, the head movement model is information representing the ideal head movement for excellent speech based on the above-mentioned basic research results on public speaking as the position of the visual field image I1 at each time. ..

The performance evaluation unit 23 evaluates the movement of the head of the trainee T1 based on the acquired head movement information and the read public speaking evaluation information. From the evaluation result, the performance evaluation unit 23 determines the movement of the head, which is highly evaluated for the movement of the head. The performance evaluation unit 23 generates head movement instruction information from the determination result. The performance evaluation unit 23 supplies the generated head motion instruction information to the motion index presentation unit 26. The head motion instruction information indicates a position on the visual field image I1 that is different from the position indicated by the head motion information.
In addition, when there are a plurality of head movements in which the evaluation of the head movement is high, the performance evaluation unit 23 randomly selects one from the head movements in which the evaluation of the plurality of head movements is high. You can do it.

At the end of the speech, the performance evaluation unit 23 generates eye contact evaluation information from the evaluation results. Here, the eye contact evaluation information is information indicating the evaluation of the eye contact operation of the trainee T1. The performance evaluation unit 23 supplies the generated eye contact evaluation information to the performance evaluation presentation unit 25.

The performance evaluation unit 23 acquires speech speed information from the voice analysis unit 20. The performance evaluation unit 23 evaluates the speaking speed of the trainee T1 based on the acquired speaking speed information and the public speaking evaluation information acquired from the speech performance index database 24. The performance evaluation unit 23 generates speech speed evaluation information from the evaluation results. The performance evaluation unit 23 supplies the generated speech speed evaluation information to the performance evaluation presentation unit 25.

From the results of the basic research on public speaking mentioned above, when a non-native English speaker speaks in English to an audience including non-native English speakers, an example of an ideal speaking speed is 140 to 150 words per second. It is said to be around (wpm). The public speaking rating information is based on the ideal speaking speed converted to syllables per second (sps). In the public speaking evaluation information, for example, speaking speed and evaluation are associated with "slow" from 0 sps to 0.7 sps, "just right" from 2.71 sps to 5.99 sps, and "fast" from 6 sps or more. ing.

The performance evaluation unit 23 acquires heart rate variability information from the heart rate variability analysis unit 22. The performance evaluation unit 23 evaluates the degree of tension of the trainee T1 based on the acquired heart rate variability information and the public speaking evaluation information acquired from the speech performance index database 24. The performance evaluation unit 23 generates tension evaluation information from the evaluation results. The performance evaluation unit 23 supplies the generated tension evaluation information to the performance evaluation presentation unit 25.

According to the public speaking evaluation information, among the number of times the heart rate of the trainee T1 was measured, the number of times the heart rate increased more than a predetermined number of times than the normal heart rate of the trainee T1 measured in advance, and the number of times. It is associated with the evaluation of the degree of tension.

The speech performance index database 24 stores public speaking evaluation information. The speech performance index database 24 may be, for example, a storage such as a flash memory or a hard disk, or may be a ROM (Read Only Memory). The speech performance index database 24 may store public speaking evaluation information input from other devices such as an external server device via, for example, a wired or wireless network.

The performance evaluation presentation unit 25 acquires eye contact evaluation information, speech speed evaluation information, and tension evaluation information from the performance evaluation unit 23.
During the speech, the performance evaluation presentation unit 25 presents the acquired speech speed evaluation information and the acquired heart rate variability information by displaying them on the display device 13 via the image generation device 12.
At the end of the speech, the performance evaluation presentation unit 25 calculates the evaluation score of the speech of the trainee T1. The performance evaluation presentation unit 25 presents the calculated evaluation score by displaying it on the display device 13.

In the present embodiment, the performance evaluation unit 23 handles the case where the eye contact evaluation information, the speech speed evaluation information, and the tension degree evaluation information are generated, and the performance evaluation presentation unit 25 uses the eye contact evaluation information. , Speaking speed evaluation information and tension evaluation information may be generated.
In that case, the performance evaluation presentation unit 25 generates eye contact evaluation information, which is information indicating the public speaking evaluation of the trainee T1, based on the head movement information and the public speaking evaluation information, and the generated eye contact. The evaluation information is presented by displaying it on the display device 13.
The performance evaluation presentation unit 25 generates speech speed evaluation information based on the speech speed information and the public speaking evaluation information, and presents the generated speech speed evaluation information by displaying it on the display device 13.

The motion index presentation unit 26 acquires head motion instruction information from the performance evaluation unit 23. The motion index presentation unit 26 increases the brightness of the frame of the panel PN3 corresponding to the position on the field image I1 indicating the direction indicated by the acquired head motion instruction information via the image generation device 12, and the display device 13 To display. That is, the motion index presenting unit 26 presents the direction indicated by the acquired head motion instruction information by displaying the position on the visual field image I1 corresponding to the direction indicated by the acquired head motion instruction information on the display device 13. do.
The motion index presenting unit 26 can distinguish between the position indicated by the head motion information and the position indicated by the head motion instruction information by increasing the brightness of the frame of the panel PN3 corresponding to the position on the visual field image I1. It is displayed on the display device 13 in various manners.

The motion index presentation unit 26 may acquire head motion information from the head motion analysis unit 21. The motion index presentation unit 26 is a panel corresponding to a position where the acquired head motion information is different from the position on the visual field image I1 indicated by the acquired head motion information when the head orientation HD1 does not change for a predetermined time. May be randomly selected from the panel PN. In that case, the operation index presenting unit 26 increases the brightness of the frame of the selected panel and causes the display device 13 to display it.
That is, the motion index presenting unit 26 causes the display device 13 to display a position on the field image I1 that is different from the position on the field image I1 indicating the orientation HD1 of the head detected by the motion sensor 11. A head orientation HD1 different from the head orientation HD1 detected by 11 is presented.
This prevents the trainee T1 from continuing to face the same location in public speaking practice.

Further, the motion index presentation unit 26 has a plurality of acquired head motion information on the visual field image I1 indicated by the acquired head motion information when the head orientation HD1 changes for a predetermined time and a predetermined number of times or more. A panel corresponding to a position different from the position may be randomly selected from the panel PN. As a result, the trainee T1 can prevent the trainee T1 from changing his / her facing place more frequently than necessary in the practice of public speaking.

The voice message generation unit 27 acquires an evaluation score from the performance evaluation unit 23. The voice message generation unit 27 generates a voice message according to the acquired evaluation score. The voice message generation unit 27 causes the voice output device 14 to output the generated voice message at the end of the speech.

The biological sensor 3 measures the heartbeat of the trainee T1 at predetermined time intervals, generates heartbeat information indicating the heartbeat value, and supplies the heartbeat information to the mobile terminal 2. Here, the predetermined time is, for example, 10 seconds. The biosensor 3 is, for example, a wearable terminal provided with a pulse sensor.

Here, an example of a head motion model will be described with reference to FIG.
FIG. 5 is a diagram showing an example of a head movement model of the present embodiment.
The squares of 2 rows and 5 columns correspond to the translucent panel PN in the field image I1. As an example, it is assumed that the position of the visual field image I1 showing the orientation HD1 of the head at the start of the speech is the panel PN1.
The panel PN that the trainee T1 should see next changes at intervals of about 5 seconds. Panel PN3 is a panel that the trainee T1 should see while the elapsed time from the start of the speech is 5 to 10 seconds. During this time, the operation index presentation unit 26 raises the brightness of the frame of the panel PN3. Hereinafter, the panel to be viewed by the trainee T1 changes in the order of panel PN10, panel PN8, panel PN6, panel PN3, and panel PN5.

The method of changing the brightness of the panel frame that the trainee T1 should see will be described. At the beginning of the speech, the trainee T1 is looking at the panel PN1. After a predetermined time (for example, about 1 to 2 seconds) has elapsed, the operation index presenting unit 26 starts to increase the brightness of the frame of the panel PN3 to be viewed next. However, the operation index presentation unit 26 continuously increases the brightness of the frame of the panel PN3 to be viewed next. This is because in the head motion model, the head orientation HD1 does not suddenly change from the panel PN1 currently being viewed to the panel PN3 to be viewed next, but from the leftmost panel PN1 through the panel PN2. Corresponds to the gradual change to the position of the front panel PN3.

The operation index presenting unit 26 starts to reduce the brightness of the frame of the panel PN3 when a predetermined time (for example, about 5 seconds) has elapsed after starting to increase the brightness of the frame of the panel PN3. However, the operation index presenting unit 26 so that the brightness of the frame of the panel PN3 returns to the original brightness when a predetermined time (for example, about 10 seconds) elapses after starting to increase the brightness of the frame of the panel PN3. , The brightness of the frame of the panel PN3 is continuously lowered. In the operation index presentation unit 26, when a predetermined time (for example, about 1 to 2 seconds) has elapsed since the operation index presentation unit 26 started to reduce the brightness of the frame of the panel PN3, the panel PN10 is the next panel to be viewed. Start increasing the brightness of the frame.

The change in the luminance of the frame of the panel PN3 will be described with reference to the graph showing the time variation of the luminance. This graph represents luminance as a continuous function K with respect to time on a coordinate plane consisting of a horizontal axis representing time and a vertical axis representing luminance. The corresponding time at the start of the speech is 0 seconds.
In the interval from time 0 seconds to time t1 (for example, about 1 second to 2 seconds), the function K is a constant function that takes the original brightness value of the panel PN3. In the interval from time t1 to time t2 (for example, about 3 seconds), the function K is a downwardly convex monotonic increase function. In the interval from time t2 to time t3 (for example, about 5 seconds), the function K is a monotonically increasing function that is convex upward. In the interval from time t3 to time t4 (for example, about 6 seconds to 7 seconds), the function K is a monotonically decreasing function that is convex upward. In the interval from time t4 to time t5 (for example, about 10 seconds), the function K is an upwardly convex monotonic decrease function. In the time interval from the time t5 to the next change in luminance, the function K is a constant function that takes the original luminance value of the panel PN3.
Here, at time t4, the brightness of the frame of the panel PN10 to be seen next to the panel PN3 begins to change. The method of changing the brightness of the frame of the panel PN10 is represented by a function in which the above function K is translocated in the positive direction of the horizontal axis for time t4 in the same manner as the method of changing the brightness of the frame of the panel PN3.

In the present embodiment, as an example, the case where the brightness of the frame of the panel to be seen next is increased has been described, but the method in which the support device D1 indicates the direction indicated by the head movement instruction information is not limited to this. The support device D1 may increase the brightness of the entire panel to be viewed next, for example. Further, the support device D1 may display the panel using an arrangement other than the squares of 2 rows and 5 columns, or may change the area of the panel according to the arrangement used. For example, the support device D1 may display a panel having a quarter area of the panel PN in a grid pattern of 4 rows and 10 columns, and present a region corresponding to the next direction to be viewed with higher accuracy. good.
The support device D1 is, for example, the outline of a region corresponding to the next direction to be viewed (for example, a rectangular region, a circular region, or a region showing a part of a virtual audience) without displaying the panel, or a region of the region. You may increase the overall brightness. The support device D1 may change, for example, the color (hue, lightness, saturation) of the region corresponding to the direction to be viewed next. For example, the support device D1 may indicate the direction to be viewed next by using an image such as an arrow without displaying the panel.
The method in which the support device D1 indicates the direction indicated by the head movement instruction information is not limited to these. The support device D1 displays, for example, the image to be marked in the visual field image I1 and intermittently moves the image to be marked at a speed and timing determined based on the motion index, so that the direction to be viewed next is to be seen. May be shown. Here, the mark image is, for example, an image having a color (for example, red) that can be easily visually recognized in the visual field image I1 and a predetermined shape (for example, a round shape). The support device D1 may blink the image to be a mark, or may change the color of the image to be a mark at a predetermined cycle. The trainee T1 can turn to the next direction to be seen while following the image of the mark that is moved and displayed by moving the head.

[Flow of speech practice using support device D1]
Next, the flow of speech practice using the support device D1 will be described with reference to FIG.
FIG. 6 is a diagram showing an example of processing of the support device D1 of the present embodiment.

(Step S100) When the power is turned on, the support device D1 causes the display device 13 to display the setting screen I2 for setting the speech practice. The trainee T1 makes various settings for speech practice while looking at the setting screen I2.
Here, the setting screen I2 will be described with reference to FIG. 7.

FIG. 7 is a diagram showing an example of the setting screen I2 of the support device D1 of the present embodiment.
The trainee T1 makes various settings for speech practice by selecting each area of display P1, display P2, and display P3. The display P1 is a display for selecting a practice mode or a production mode. In the practice mode, as described in the visual field image I1 of FIG. 3, during the speech, the panel showing the virtual audience and the motion index, the evaluation information such as the speaking speed, and the information such as the practice mode are displayed.

In the production mode, the support device D1 displays the visual field image I1 including the virtual audience and the virtual presentation venue during the speech, but the evaluation information (display F1) and the speech speed evaluation information (display F1) as shown in FIG. Display F2), panel PN is not displayed. In the production mode, the trainee T1 can practice while looking at the visual field image I1 that is closer to the actual situation than in the practice mode.

The display P2 is a display for selecting whether to display the virtual audience by CG or by a live-action moving image. The display P3 is a display for selecting a speech time. In this embodiment, as an example, a case where the speech time is selected from 2 minutes and 5 minutes is shown.
When the start button P4 is selected, the support device D1 ends the display of the setting screen I2.
Returning to FIG. 6, the flow of speech practice will be continued. However, the case where the practice mode is selected will be described below.

(Step S101) The support device D1 displays the heart rate measurement screen I3.
Here, the heart rate measurement screen I3 will be described with reference to FIG.
FIG. 8 is a diagram showing an example of the heart rate measurement screen I3 of the support device D1 of the present embodiment.
The trainee T1 reads the text displayed on the heart rate measurement screen I3 aloud for 30 seconds. The biological sensor 3 measures the heart rate of the trainee T1. The biological sensor 3 supplies the information indicating the measured heartbeat value to the mobile terminal 2 as the information indicating the normal heartbeat value of the trainee T1.
Returning to FIG. 6, the flow of speech practice will be continued.

(Step S102) The trainee T1 starts a speech.
(Step S103) The motion sensor 11 detects the head orientation HD1 of the trainee T1 and supplies the detected head orientation HD1 to the mobile terminal 2.
(Step S104) The display device 13 displays the field image I1 input from the image generation device 12 in a direction facing the head direction HD1.

(Step S105) The performance evaluation unit 23 generates head movement instruction information, speech speed evaluation information, and tension evaluation information.
(Step S106) The motion index presentation unit 26 acquires head motion instruction information from the performance evaluation unit 23. The motion index presenting unit 26 presents the panel, which is a position on the visual field image I1 indicating the direction indicated by the acquired head motion instruction information, by displaying the panel on the display device 13.
The performance evaluation presentation unit 25 acquires the speech speed evaluation information and the tension evaluation information from the performance evaluation unit 23. During the speech, the performance evaluation presentation unit 25 presents the acquired speech speed evaluation information and the acquired heart rate variability information by displaying them on the display device 13.

(Step S107) The mobile terminal 2 determines whether or not the speech has ended. When the speech time set in step S100 has elapsed, the mobile terminal 2 determines that the speech has ended. If the speech time set in step S100 has not elapsed, the mobile terminal 2 determines that the speech has not ended.
When it is determined that the speech is completed (step S107; YES), the mobile terminal 2 executes the process of step S108. On the other hand, when it is determined that the speech is not completed (step S107; NO), the mobile terminal 2 repeats the process of step S103.

(Step S108) The performance evaluation presentation unit 25 calculates the evaluation score of the speech of the trainee T1.
For example, the performance evaluation presentation unit 25 uses only the speech speed evaluation information corresponding to a predetermined time (for example, 30 seconds, 1 minute, 3 minutes, etc.) at the beginning of the speech to calculate the evaluation score for the speech speed. May be good. This is because the trainee T1 can adjust the speaking speed by checking the display F2 displayed in the visual field image I1 during the speech, and the speaking speed is evaluated over the entire speech. This is because when the evaluation score is calculated using the information, it may not be possible to correctly evaluate the current speech speed of the trainee T1.
The performance evaluation presentation unit 25 presents the calculated evaluation score by displaying it on the display device 13. The performance evaluation presentation unit 25 presents comments for each evaluation score item according to the evaluation score.
The voice message generation unit 27 causes the voice output device 14 to output the voice message generated according to the evaluation score, and ends a series of processing.
The support device D1 is a voice output device 14 that imitates the environmental sound of the presentation venue from the start of the speech (S102) to the end of the speech (S107; YES), that is, during the public speaking training. May be output to. Here, the voice imitating the environmental sound is noise in the presentation hall, reaction sound (applause, voice) of the audience, and the like. The trainee T1 can practice in a more natural state in terms of experience, while listening to the sound imitating the environmental sound, as compared with the silent state in which the sound imitating the environmental sound is not output.
Here, the evaluation presentation screen I4 will be described with reference to FIG.

FIG. 9 is a diagram showing an example of the evaluation presentation screen I4 of the support device D1 of the present embodiment.
On the evaluation presentation screen I4, comments are displayed for each item of tension, speaking speed, and eye contact. Each comment is output as a voice message. The display P5 is a graph showing the evaluation score for each item.

[summary]
According to the support device D1 according to the above-described embodiment, the position different from the position seen by the trainee T1 can be displayed as the position to be seen next by the trainee T1. Therefore, according to the support device D1, the trainee can learn which direction to turn next in the practice of public speaking.

According to the support device D1 according to the above-described embodiment, the position where the trainee T1 should see next is a position different from the position where the trainee T1 is looking, and the position where the evaluation of public speaking is high is displayed. can. Therefore, according to the support device D1, in the practice of public speaking, the trainee can learn in which direction the public speaking is highly evaluated.

Further, according to the support device D1 according to the above-described embodiment, the trainee T1 can distinguish the position to be seen next from the position not to be seen next. Therefore, according to the support device D1, the trainee can visually grasp which direction to turn next in the speech practice.

Further, according to the support device D1 according to the above-described embodiment, the trainee T1 can know the evaluation of his / her own public speaking based on the public speaking of the trainee T1. Therefore, according to the support device D1, the trainee T1 can learn while confirming the evaluation of his / her public speaking.

Further, according to the support device D1 according to the above-described embodiment, the trainee T1 can know the evaluation of the speaking speed in his / her public speaking. Therefore, according to the support device D1, the trainee T1 can learn an appropriate speaking speed in the practice of public speaking.

Further, according to the support device D1 according to the above-described embodiment, the speaking speed of the trainee T1 can be evaluated in units of syllables. Therefore, according to the support device D1, in the practice of public speaking, the trainee T1 can learn an appropriate speaking speed in various languages.

Further, according to the support device D1 according to the above-described embodiment, the trainee T1 can know the evaluation of the voice volume in his / her public speaking. Therefore, according to the support device D1, the trainee T1 can learn an appropriate voice volume in the practice of public speaking.

Further, according to the support device D1 according to the above-described embodiment, the trainee T1 can know the evaluation of the degree of tension in his / her public speaking. Therefore, according to the support device D1, the trainee T1 can learn an appropriate degree of tension in the practice of public speaking.

The HMD of the support device D1 of the present embodiment may be an HMD specialized for displaying a moving image of virtual reality (VR), or may be configured by combining a smartphone with dedicated goggles, for example. ..
In the present embodiment, as an example, the case where the support device D1 takes the form of an HMD has been described, but the support device D1 may take the form of a display installed in a lecture hall, for example. Further, the support device D1 may take a form in which the trainee T1 holds and uses a terminal provided with a display device, such as a tablet terminal.

Further, in one embodiment of the present invention, the support device D1 may evaluate the voice volume of the trainee T1, and in this case, the support device D1 is information indicating the evaluation of the voice volume of the trainee T1. The voice volume evaluation information is displayed in the field image I1 as, for example, an image (not shown) showing the volume level. Specifically, the voice analysis unit 20 analyzes the acquired voice and generates voice volume information which is information indicating the voice volume of the trainee T1 from the analysis result. The voice analysis unit 20 supplies the generated voice volume information to the performance evaluation unit 23. The performance evaluation unit 23 acquires voice volume information from the voice analysis unit 20. The performance evaluation unit 23 evaluates the voice volume of the trainee T1 based on the acquired voice volume information, and generates voice volume evaluation information from the evaluation result. The performance evaluation unit 23 supplies the generated voice volume evaluation information to the performance evaluation presentation unit 25. The performance evaluation presentation unit 25 presents the voice volume evaluation information by displaying it on the display device 13.

FIG. 10 is a diagram showing an example of the form of the public speaking support device of the present embodiment.
The support device D2 is a display hung on the ceiling of the lecture hall. The orientation of the head of the trainee T2 HD2 is the positive orientation of the x-axis of the three-dimensional Cartesian coordinate system x, y, z. Head orientation The orientation DD2 relative to HD2 is the negative orientation of the x-axis of the three-dimensional Cartesian coordinate system x, y, z. The support device D2 is installed in a direction orthogonal to the X-axis of the three-dimensional Cartesian coordinate system X, Y, Z in a negative direction of the X-axis.

FIG. 11 is a diagram showing an example of the form of the public speaking support device of the present embodiment.
The support device D3 is a tablet terminal, which is held and used by the trainee T3. The orientation of the head of the trainee T3 HD3 is the positive orientation of the x-axis of the three-dimensional Cartesian coordinate system x, y, z. Head orientation The orientation DD3 relative to HD3 is the negative orientation of the x-axis of the three-dimensional Cartesian coordinate system x, y, z.

The support device D1 may display the image S1 perpendicularly to the x-axis on, for example, the field image I1. In that case, the support device D1 displays the image S1, for example, behind the venue on the field image I1.
In the present embodiment, the case of measuring the heart rate of the trainee T1 has been described, but instead of the heart rate, for example, the pulse of the trainee T1 may be measured.

(Second embodiment)
Hereinafter, the second embodiment of the present invention will be described in detail with reference to the drawings.
In the first embodiment, an example in which the trainee T1 wears the support device D1 and practices public speaking has been described. In the second embodiment, an example is in which the speaker wears a support device to practice public speaking, and the support device presents to the speaker the place (audience to make eye contact) to which the support device faces during public speaking. Will be explained.

FIG. 12 is a diagram showing an example of an overview of the support device D1a of the present embodiment.
The support device D1a is a spectacle-type display device, and is worn and used by the speaker T1a. The display device 13a is a transmissive monitor. The support device D1a causes the display device 13a to superimpose the position on the visual field image that the speaker T1a should face during public speaking on the visual field image.

FIG. 13 is a diagram showing an example of the functional configuration of the support device D1a of the present embodiment.
The head-mounted display 1 and the glasses 1a are different between the configuration of the support device D1a of the present embodiment (FIG. 13) and the configuration of the support device D1 according to the first embodiment (FIG. 4). The presence / absence of the image generation device 12 and the display device 13a are different between the head-mounted display 1 and the glasses 1a. Since the other configurations are the same as those of the support device D1 according to the first embodiment, the description thereof will be omitted, and in the second embodiment, the parts different from the first embodiment will be mainly described.

The support device D1a includes glasses 1a, a mobile terminal 2, and a biosensor 3. The glasses 1a and the mobile terminal 2 may or may not be integrally configured.
The spectacles 1a includes a voice recording device 10, a motion sensor 11, a display device 13a, and a voice output device 14.

The display device 13a is a transmissive monitor. The speaker T1a can visually recognize the scenery in the direction corresponding to the direction HD1a of his / her head in the state where the support device D1a is attached.
The support device D1a uses the panel PN presented by the motion index presentation unit 26, the speech speed evaluation information presented by the performance evaluation presentation unit 25, and the heart rate variability information in a direction corresponding to the head orientation HD1a of the speaker T1a. It is displayed overlaid on the landscape of.

The support device D1a may display the virtual audience by superimposing the virtual audience on the landscape in the direction corresponding to the orientation HD1a of the head of the speaker T1a by using the AR technique.

According to the support device D1a according to the above-described embodiment, the speaker T1a can present the position to be seen next during the practice of public speaking. Therefore, according to the support device D1, the trainee can know which direction to turn next during the practice of public speaking.

In the above embodiment, the case where the evaluation of the degree of tension is performed by measuring the heart rate of the trainee T1 has been described, but the method of evaluating the degree of tension is not limited to this. The evaluation of the degree of tension may be performed by measuring, for example, the number of blinks of the trainee T1, the amount of hand sweat, the blood pressure, the electrical activity of the skin, the amount of saliva, the change in complexion, and the like. In these cases, the biosensor is, for example, a wearable terminal equipped with various sensors.

Further, in the above embodiment, the speaking speed is used for the evaluation of public speaking, but the speaking speed may also be used for the evaluation of the degree of tension. The faster the speaking speed, the higher the tension is evaluated. The speaking speed may be measured using, for example, a smartphone. When the speaking speed is measured using a smartphone, there is an advantage that a camera is not required.
The evaluation of public speaking may include not only the evaluation of speech speed but also the evaluation of pronunciation. Here, the pronunciation evaluation is an evaluation of the pronunciation of a foreign language such as English. When the evaluation of pronunciation is included in the evaluation of public speaking, the voice analysis unit 20 generates pronunciation information showing the result of comparing the pronunciation of the trainee T1 with the ideal pronunciation. The performance evaluation unit 23 evaluates the pronunciation of the trainee T1 based on the pronunciation information generated by the voice analysis unit 20.

Further, in the above embodiment, the case where the evaluation score of the speech of the trainee T1 is presented on the evaluation presentation screen I4 has been described, but this evaluation score may be stored as practice record data. Further, the practice record data may include the voice of the trainee T1 during the public speaking training recorded by the voice recording device 10 as voice data.
For example, when the trainee T1 is a student or a student, the practice record data can be referred to by the teacher or trainer of the trainee T1 and used when instructing the trainee T1. The practice record data may be made available only to the teacher or trainer of the trainee T1. For example, the practice record data may be transferred to a database or the like that can be accessed only by the teacher or trainer of the trainee T1. Alternatively, the practice record data may be encrypted so that only the teacher or trainer of the trainee T1 can refer to it.

In addition, the portion used by the trainee T1 during the public speaking training may be immediately notified. The part used by the trainee T1 during the public speaking training is, for example, the part where the speech script and the words spoken by the trainee T1 do not match in terms of timing or audio content. For example, the voice analysis unit 20 determines whether or not the speech script in the image S1 of the speech material displayed on the podium on the visual field image I1 and the words uttered by the trainee T1 are synchronized. .. The criterion for determining whether or not the speech script and the words uttered by the trainee T1 are synchronized may be set based on the time lag and the voice lag.
When the voice analysis unit 20 determines that the speech script in the image S1 and the words uttered by the trainee T1 are not synchronized, the voice analysis unit 20 outputs the determination result to the image generation device 12. The image generation device 12 generates a visual field image I1 including an image and a message indicating that the trainee T1 has been used, based on the determination result output by the voice analysis unit 20. The trainee T1 can recognize the part used by the trainee T1 by seeing an image or a message indicating that the trainee T1 has been used during the speech training.

Also, depending on the preset speech script, the ideal speech time for speaking the content of this speech script may be calculated. The ideal speech time may be calculated page by page or sentence by sentence in the speech script. The ideal speech time calculated for each page or sentence of the speech script may be used to determine the location used by the trainee T1. The calculated ideal speech time may be displayed on the field image I1.

Further, in the above embodiment, the image of the landscape of the virtual presentation venue displayed on the visual field image I1 is an image of the landscape of the lecture room. The image of the landscape of the virtual presentation venue may be selected from a plurality of types. The image of the landscape of the virtual presentation venue of a plurality of types is, for example, an image of the landscape of the academic society, an image of the landscape of the wedding venue, and the like.

Further, in the above embodiment, the design of the virtual audience displayed on the visual field image I1 is the design of the audience, but the design of the virtual audience may be changed. The design of the virtual audience displayed on the field image I1 may be changed to, for example, a design of a foreign audience, an older audience, an animal, a character, or the like. It is desirable to practice public speaking repeatedly, but it is often boring after repeated practice. By making it possible to change the design of the virtual audience displayed on the field image I1, the trainee T1 can be less likely to get tired of practicing public speaking.

In the above embodiment, the case where the virtual audience is displayed on the visual field image I1 has been described, but instead of the virtual audience, a part of the body of the audience, for example, an image of the eyes of the audience may be displayed. Here, the image of the eyes of the audience may be, for example, an image of the eyes of an illustration, an image of a CG eye close to a live-action image, or an image of a deformed (deformed) eye. When displaying the image of the eyes of the audience on the field image I1, the surroundings of the image of the eyes of the audience should be so that the image of the eyes of the audience is not buried in the image of the landscape of the virtual presentation venue and difficult to recognize. , For example, may be applied using a translucent skin color.

It is useful to use an image of the eyes of the audience instead of the virtual audience on the visual field image I1 when one wants to practice public speaking in a situation that is just like practicing. Here, the situation just like practice is a situation in which the trainee feels the gaze of the audience and feels a sense of tension.
Compared to the case where only the image of the landscape of the virtual presentation venue is displayed on the field image I1, the trainee is more likely to use the image of the eyes of the audience instead of the virtual audience on the field image I1. Experimental results have been obtained showing that the person feels nervous. In this experiment, the subjective tension of the trainee obtained by the interview is evaluated by a five-step scale method. For the subjective tension of the trainee when only the image of the scenery of the virtual presentation venue is displayed on the visual field image I1, the image of the eyes of the audience is displayed on the visual field image I1 instead of the virtual audience. It has been verified by the Friedman test that the subjective tension of the trainee when used is significantly increased.

The position of the image of the eyes of the audience may be determined based on the position of the desk or chair in the image of the landscape of the virtual presentation venue and the shape of the presentation venue. The position of the image of the eyes of the audience may be randomly arranged regardless of the position of the desk or chair in the image of the landscape of the virtual presentation venue and the shape of the presentation venue.

The image of the eyes of the audience may be a blinking motion as a moving image. The image of the eyes of the audience may blink. The number of blinking movements and blinks of the audience's eye image may be variable by setting. Further, the number of blinking movements and blinking of the image of the eyes of the audience may be changed according to the degree of interest of the audience. This setting may be a setting for the number of blinking movements or blinks of the image of the eyes of the audience per hour, or may be a setting for the number of times in one public speaking practice. This setting may be made for each image of the eyes of multiple audiences.

The image of the eyes of the audience may, as a moving image, change the position of the black eyes within the contour of the eyes according to the evaluation of the speech of the trainee T1. The position of the black eye in the contour of the eye may be such that the trainee T1 is not looking at the position of the black eye, for example, if the speech evaluation of the trainee T1 is low, the audience is less interested. To prevent the position of the black eye in the contour of the eye from looking toward the trainee T1, for example, to move the position of the black eye in the contour of the eye to a position other than the center in the contour of the eye.

Further, as an image of the eyes of the audience, a figure in which three points are arranged at the vertices of an inverted triangle may be displayed. A figure in which three points are arranged at the vertices of an inverted triangle allows the trainee T1 to practice public speaking with a greater sense of tension than a figure in which three points are arranged at the vertices of a triangle. For figures in which three points are placed at the vertices of an inverted triangle, the points placed at the two vertices above the inverted triangle are displayed using black, and the points placed at one vertex below the inverted triangle are displayed. It can be displayed using red. The points placed at the two vertices above the inverted triangle are recognized by the trainee T1 as eyes, and the points placed at the one vertex below the inverted triangle are recognized as the mouth by the trainee T1. ..

When the image of the eyes of the audience is used instead of the virtual audience on the field image I1, the amount of data of the field image I1 can be reduced as compared with the case where the virtual audience is used. Further, when the image of the eyes of the audience is used instead of the virtual audience on the visual field image I1, the processing load of the head-mounted display 1 can be reduced as compared with the case where the virtual audience is used. In addition, compared to the case of increasing the variation of the virtual audience, it is possible to increase the variation such as the type of eyes of the audience, the number of blinks, and the number of blinks without rapidly increasing the amount of data. Therefore, various reactions of the audience can be easily displayed to the non-trainer T1.

In the second embodiment, the support device D1a can superimpose the virtual audience on the landscape in the direction corresponding to the orientation HD1a of the head of the speaker T1a by using the technique of AR. In the second embodiment, in the support device D1a, when the virtual audience is made into a live-action video in order to realize a situation just like a practice, the moving image of the virtual audience is required to have abundant movement patterns. Therefore, when the image of the eyes of the audience is used instead of the virtual audience, the amount of data of the visual field image I1 can be reduced. Further, when the image of the eyes of the audience is used instead of the virtual audience, the processing load of the support device D1a can be reduced.
In the support device D1a of the second embodiment, when the audience is insufficient in the practice of public speaking, the insufficient audience may be supplemented by the image of the eyes of the audience. In the support device D1a of the second embodiment, even when the audience is insufficient in the practice of public speaking, the practice of public speaking can be performed in a situation as if it were practiced.

(Third embodiment)
Hereinafter, a third embodiment of the present invention will be described in detail with reference to the drawings.
In the first embodiment and the second embodiment, the case where the trainee or the lecturer wears and uses the support device has been described, but in the third embodiment, the support device is used as the terminal device. An example will be described.

FIG. 14 is a diagram showing an example of an overview of the support device D1b of the present embodiment. The training device D1b is a public speaking support device for the trainee to train public speaking while looking at a visual field image. The support device D1b is used as a terminal device 2b. The terminal device 2b is provided with a camera 11b. The terminal device 2b is, for example, a support device for practicing a presentation in a video conference.
On the display device 13b of the terminal device 2b, a field image I5 including an image of the eyes of the audience is displayed together with an image of the scenery of the virtual presentation venue. Here, the display device 13b is a display. Using the terminal device 2b, the trainee can practice the presentation while feeling the line of sight of the audience and feeling the tension.
The biosensor 3 attached to the trainee measures the heartbeat of the trainee. The biological sensor 3 supplies information indicating the measured heart rate value to the terminal device 2b. The biosensor 3 supplies information indicating a heart rate value to the terminal device 2b using, for example, short-range wireless communication. The terminal device 2b evaluates the degree of tension of the trainee based on the information indicating the heart rate value supplied by the biological sensor 3. The display device 13b displays the degree of tension evaluated by the terminal device 2b on the visual field image I5.

FIG. 15 is a diagram showing an example of the functional configuration of the support device D1b of the present embodiment.
The presence or absence of the head-mounted display 1 differs between the configuration of the support device D1b of the present embodiment (FIG. 15) and the configuration of the training device D1 according to the first embodiment (FIG. 4). The voice recording device 10b, the camera 11b, the display device 13b, and the voice output device 14b are not provided in the head-mounted display 1. Further, when the mobile terminal 2 (FIG. 4) and the terminal device 2b (FIG. 15) are compared, the state analysis unit 22b, the performance evaluation unit 23b, the speech performance index database 24b, the performance evaluation presentation unit 25b, and the image It is different from the generation unit 28b. Since the other configurations are the same as those of the support device D1 according to the first embodiment, the description thereof will be omitted, and in the third embodiment, the parts different from the first embodiment will be mainly described.
The terminal device 2b includes a voice analysis unit 20, a state analysis unit 22b, a performance evaluation unit 23b, a speech performance index database 24b, a performance evaluation presentation unit 25b, a voice message generation unit 27, and an image generation unit 28b. Be prepared.

The state analysis unit 22b analyzes the state of the trainee presented with the visual field image I5 displayed on the display device 13b. The trainee's condition is, for example, the degree of tension of the trainee presented with the visual field image I5 including the image of the eyes of the audience.
The state analysis unit 22b acquires the heartbeat information of the trainee generated by the biological sensor 3. The state analysis unit 22b analyzes the acquired heartbeat information as the degree of tension of the trainee. Further, the state analysis unit 22b acquires a photographed image of the trainee photographed by the camera 11b. The state analysis unit 22b analyzes the number of blinks of the trainee from the acquired captured image as the degree of tension of the trainee.
The state analysis unit 22b generates state information indicating the state of the trainee from the analyzed state of the trainee. The state analysis unit 22b generates heart rate variability information and blink frequency information as state information. The state analysis unit 22b supplies the generated state information to the performance evaluation unit 23b.

The performance evaluation unit 23b evaluates the performance of public speaking by the trainee based on the state of the trainee analyzed by the state analysis unit 22b and the public speaking evaluation information stored in the speech performance index database 24b. Here, in the present embodiment, the public speaking evaluation information is information in which the state of the trainee and the evaluation of the state of the trainee are associated with each other. The performance of public speaking by the trainee is, for example, the degree of tension of the trainee. The performance evaluation unit 23b generates tension evaluation information indicating the tension of the trainee. In addition, the performance evaluation unit 23b generates speech speed evaluation information.

Public speaking evaluation information is stored in the speech performance index database 24b.
The performance evaluation presentation unit 25b acquires tension evaluation information and speech speed evaluation information from the performance evaluation unit 23b. The performance evaluation presentation unit 25b presents the acquired tension evaluation information and the acquired speech speed evaluation information by displaying them on the display device 13b through the image generation unit 28b.

The image generation unit 28b generates a field image I5 including an image of the eyes of the audience. The image generation unit 28b supplies the generated visual field image I5 to the display device 13b.
The display device 13b displays the field image I5 acquired from the image generation unit 28b. That is, the display device 13b displays the visual field image I5 including the image of the eyes of the audience.
The voice recording device 10b is, for example, a microphone provided in the terminal device 2b. The audio output device 14b is, for example, a speaker.

FIG. 16 is a diagram showing an example of the field image I5 of the support device D1b of the present embodiment. In the field image I5, an eye image E1 to an eye image E8, which are images of the eyes of the audience, an image F1b showing tension evaluation information, and a speech speed evaluation information are superimposed on the image of the landscape of the virtual presentation venue. The image F2b shown is displayed. The eye image E1 to the eye image E8 are CG eye images close to a live-action image. In the eye image E1 to the eye image E8, the periphery of the eye image is painted using a translucent skin color. Of the eye images E1 to E8, the images other than the eye image E5 are images in a state where the eyes are open. Of the eye images E1 to the eye images E8, the eye image E5 is an image in which the eyes are closed.

The trainee can practice the presentation while feeling the eyes of the audience and feeling the tension only by displaying the eye images E1 to the eye images E8. The load on the support device D1b is reduced when the eye images E1 to E8 are displayed as compared with the case where the virtual audience is displayed. According to the support device D1b, in the practice of public speaking, it is possible to reduce the burden on the device when reproducing the situation as it is practiced.

FIG. 17 is a diagram showing an example of the field image I6 of the support device D1b of the present embodiment. In the field image I6, instead of the eye images E1 to E8, which are CG eye images close to the actual image in the field image I5, the eye images A1 to A8, which are the images of the eyes in the illustration, are displayed.
The trainee can practice the presentation while feeling the eyes of the audience and feeling the tension only by displaying the eye images A1 to A8, which are the images of the eyes in the illustration.

[summary]
According to the support device D1b according to the above-described embodiment, it is possible to reduce the burden on the device when reproducing the situation as if it were practiced in the practice of public speaking.
The support device D1b may be used in the practice of presentation in a video conference. When the support device D1b is used in the practice of presentation in a video conference, the image of the eyes of the audience is not displayed in the field image, and the image showing the tension evaluation information, the image showing the speech speed evaluation information, and the trainee T1c The next position to look at is displayed.

FIG. 18 is a diagram showing an example of an overview of the training device D1c of the present embodiment. The training device D1c is an HMD as an example. The trainee T1c can practice the presentation while feeling the line of sight of the audience and feeling nervous.

Although the embodiment of the present invention has been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment and may be appropriately modified without departing from the spirit of the present invention. can. The configurations described in each of the above-described embodiments may be combined.

It should be noted that each part included in each device in the above embodiment may be realized by dedicated hardware, or may be realized by a memory and a microprocessor.

Each part of each device is composed of a memory and a CPU (Central Processing Unit), and the function is realized by loading and executing a program for realizing the function of each part of each device in the memory. May be.

Further, the control unit can record a program for realizing the function of each part of each device on a computer-readable recording medium, read the program recorded on the recording medium into the computer system, and execute the program. Processing by each provided part may be performed.
The term "computer system" as used herein includes hardware such as an OS and peripheral devices.

Further, the "computer system" includes the homepage providing environment (or display environment) if the WWW system is used.
Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, and a storage device such as a hard disk built in a computer system. Further, a "computer-readable recording medium" is a communication line for transmitting a program via a network such as the Internet or a communication line such as a telephone line, and dynamically holds the program for a short period of time. In that case, it also includes those that hold the program for a certain period of time, such as the volatile memory inside the computer system that is the server or client. Further, the above-mentioned program may be for realizing a part of the above-mentioned functions, and may be further realized for realizing the above-mentioned functions in combination with a program already recorded in the computer system.

1 ... head mount display, 2 ... mobile terminal, 3 ... biosensor, 10, 10b ... voice recording device, 11 ... motion sensor, 11b ... camera, 12 ... image generator, 13, 13b ... display device, 14, 14b ... Voice output device, 20 ... Voice analysis unit, 21 ... Head motion analysis unit, 22 ... Heart rate variability analysis unit, 22b ... State analysis unit, 23, 23b ... Performance evaluation unit, 24, 24b ... Speech performance index database, 25, 25b ... Performance evaluation presentation unit, 26 ... Motion index presentation unit, 27 ... Voice message generation unit, 28b ... Image generation unit, D1, D2, D3, D1a, D1b ... Support device, T1, T2, T3, T1c ... Trained Person, T1a ... Speaker, HD1, HD2, HD3, HD1a ... Head orientation, DD1, DD2, DD3, DD1a ... Direction facing the head orientation, E1, E2, E3, E4, E5, E6, E7 , E8, A1, A2, A3, A4, A5, A6, A7, A8 ... Eye image

Claims (12)

A detector that detects the orientation of the trainee's head,
It is displayed superimposed on the field image which is an image as a virtual reality in the direction corresponding to the direction of the head detected by the detection unit, or the field of view of the trainee or the image of the field of view as augmented reality in the direction. A display unit that displays an augmented reality image that is an image that faces the direction of the head.
When the display unit displays the field of view image, the display unit displays information indicating a position on the field of view image different from the position on the field of view image indicating the direction of the head detected by the detection unit. Or, when the display unit displays the augmented reality image, the field of view or the field of view different from the position on the image of the field of view indicating the direction of the head detected by the detection unit or the said. An motion index presenting unit that presents a head orientation different from the head orientation detected by the detection unit by displaying information indicating the position of the visual field on the image on the display unit as the augmented reality image . ,
A public speaking support device equipped with. The orientation of the head detected by the detection unit is analyzed, and the head motion information, which is information indicating the visual field image showing the orientation of the head, the visual field, or the position on the image of the visual field, is generated from the analysis result. Head motion analysis unit and
A storage unit that stores public speaking evaluation information, which is information in which the movement of the head and the evaluation of the movement of the head are associated with each other.
The head movement is evaluated based on the head movement information generated by the head movement analysis unit and the public speaking evaluation information stored in the storage unit, and the head movement information is obtained from the evaluation result. A performance evaluation unit that generates head movement instruction information that is information indicating the field image , the field, or a position on the image of the field that is different from the indicated position.
Further prepare
The motion index presenting unit displays the head field image indicated by the head motion instruction information, information indicating the visual field or the position of the visual field on the image on the display unit, and the head detected by the detection unit. The public speaking support device according to claim 1, which presents the orientation of the head different from the orientation of the part. The public speaking support according to claim 2, wherein the motion index presenting unit displays the position indicated by the head motion information and the position indicated by the head motion instruction information on the display unit in a manner distinguishable from each other. Device. Further equipped with a performance evaluation presentation section ,
The performance evaluation presentation unit generates performance evaluation information, which is information indicating the evaluation of the public speaking performance of the trainee, based on the head movement information and the public speaking evaluation information, and generates the performance evaluation. The public speaking support device according to claim 2 or 3, wherein the information is displayed on the display unit to present the evaluation of the performance . A voice recording unit that records the voice of the trainee, and
A voice analysis unit that analyzes the voice recorded by the voice recording unit and generates speech speed information that is information indicating the speaking speed of the trainee from the analysis result.
Further prepare
The public speaking evaluation information stored in the storage unit further includes information in which the speech speed information and the public speaking evaluation are associated with each other.
The performance evaluation presentation unit generates performance evaluation information based on the speech speed information and the public speaking evaluation information, and presents the performance evaluation by displaying the generated performance evaluation information on the display unit. The public speaking support device according to claim 4. The public speaking support device according to claim 5, wherein the voice analysis unit generates the speech speed information in units of syllables. The voice analysis unit analyzes the voice recorded by the voice recording unit, and generates voice volume information which is information indicating the voice volume of the trainee from the analysis result.
The public speaking evaluation information stored in the storage unit further includes information in which the voice volume information and the public speaking evaluation are associated with each other.
The performance evaluation presentation unit generates performance evaluation information based on the voice volume information and the public speaking evaluation information, and displays the generated performance evaluation information on the display unit to present the performance evaluation. Item 5 or the public speaking support device according to claim 6. A heartbeat information generation unit that generates heartbeat information indicating the heartbeat value of the trainee, and
A heart rate variability analysis unit that analyzes the heart rate information generated by the heart rate information generation unit and generates heart rate variability information that is information indicating the heart rate variability of the trainee from the analysis results.
Further prepare
The public speaking evaluation information stored in the storage unit further includes information in which the heart rate variability information is associated with the public speaking evaluation.
The performance evaluation presentation unit generates performance evaluation information based on the heart rate variability information and the public speaking evaluation information, and presents the performance evaluation by displaying the generated performance evaluation information on the display unit. The public speaking support device according to any one of claims 4 to 7. Further, a state analysis unit for analyzing the tension of the trainee to which the visual field image or the augmented reality image displayed on the display unit is presented is provided.
The public speaking evaluation information stored in the storage unit further includes information in which the tension level of the trainee is associated with the evaluation of the tension level of the trainee.
The performance evaluation presentation unit indicates the tension level of the trainee based on the tension level of the trainee analyzed by the state analysis unit and the public speaking evaluation information stored in the storage unit. The performance evaluation is presented by generating the degree evaluation information and displaying the generated tension evaluation information on the display unit.
The public speaking support device according to any one of claims 4 to 8. The public speaking support device according to any one of claims 1 to 9 , wherein the display unit is a transparent display unit. The public speaking support device according to any one of claims 1 to 10 , wherein the field image or the augmented reality image includes an image of the eyes of an audience. On the computer
A detection step that detects the orientation of the trainee's head,
It is displayed superimposed on the field image which is an image as a virtual reality in the direction corresponding to the direction of the head detected by the detection step, or the field of view of the trainee or the image of the field of view as augmented reality in the direction. A display step of displaying an augmented reality image, which is an image, on the display unit in a direction opposite to the direction of the head.
When the display unit displays the field image, the display indicates information indicating a position on the field image indicating a direction different from the position on the field image indicating the direction of the head detected by the detection step. The above, which is different from the field of view indicating the orientation of the head detected by the detection step or the position on the image of the field of view when the display unit displays the augmented reality image. By displaying information indicating the field of view or the position of the field of view on the image on the display unit as the augmented reality image, an operation index that presents the direction of the head different from the direction of the head detected by the detection step. Presentation steps and
A program to execute.

Images