JP2022040894A - Lesson course system and lesson course method - Google Patents

Abstract

To provide a lesson course system and a lesson course method by which a participant can take a lesson to learn how to move his or her body without caring about other participants.SOLUTION: A lesson course system includes: a lecturer side system for tracking a motion of a lecturer's body, generating lecturer motion information on the basis of a tracking result, generating a lecturer avatar image in which a lecturer avatar representing the lecturer moves on the basis of the lecturer motion information, and transmitting the generated lecturer avatar image; and a participant side system for tracking a motion of a participant's body, generating participant motion information on the basis of a tracking result, generating a participant avatar image in which a participant avatar representing the participant moves on the basis of the participant motion information, receiving the lecturer avatar image transmitted from the lecturer side system, and displaying the generated participant avatar image and the received lecturer avatar image in correspondence with each other.SELECTED DRAWING: Figure 1

Description

The present invention relates to a lesson training system and a lesson training method.

There is known a dance device that allows you to experience as if you and another person are dancing together by projecting your own dance image and the dance image of another person at a remote location onto a single display unit (for example). , Patent Document 1).

Japanese Unexamined Patent Publication No. 2016-174631

The technique described in Patent Document 1 is not suitable for taking lessons such as dance without worrying about the eyes of other students, for example, because each person can see each other's movements.

The present invention has been made in view of the above, and is a lesson training system that allows a student to take a lesson for learning how to move the body without worrying about the eyes of other students. And the purpose is to provide lesson training methods.

The lesson training system according to the present invention tracks the movement of the instructor's body, generates instructor motion information based on the tracking result, and generates an instructor avatar image in which the instructor avatar indicating the instructor moves based on the instructor motion information. Then, the instructor side system that sends the generated instructor avatar image, the instructor's body movement is tracked, the instructor motion information is generated based on the tracking result, and the instructor avatar indicating the instructor is the instructor. Along with generating a student avatar image that moves based on the person motion information, the instructor avatar image transmitted from the instructor side system is received, and the generated student avatar image corresponds to the received instructor avatar image. It is equipped with a student-side system that allows and displays.

The lesson training method according to the present invention tracks the movement of the instructor's body, generates instructor motion information based on the tracking result, and generates an instructor avatar image in which the instructor avatar indicating the instructor moves based on the instructor motion information. Then, the generated instructor avatar image is transmitted, the movement of the student's body is tracked, the student motion information is generated based on the tracking result, and the student avatar indicating the student is used as the student motion information. Along with generating a student avatar image that moves based on the above, the transmitted instructor avatar image is received, and the generated student avatar image and the received instructor avatar image are displayed in association with each other.

According to the present invention, a student can take a lesson for learning how to move the body without worrying about the eyes of other students.

FIG. 1 is a functional block diagram showing an example of a lesson training system according to the present embodiment. FIG. 2 is a diagram showing one aspect when the instructor operates. FIG. 3 is a diagram showing an example of the instructor avatar video. FIG. 4 is a diagram showing one aspect when the student operates. FIG. 5 is a diagram showing an example of a student avatar video. FIG. 6 is a diagram showing an example of a VR image displayed on the display device of the instructor side system. FIG. 7 is a diagram showing an example of a VR image displayed on the display device of the student side system. FIG. 8 is a diagram showing an example of a VR image displayed on the display device of the student side system. FIG. 9 is a flowchart showing an example of the lesson training method according to the present embodiment.

Hereinafter, embodiments of the lesson training system and the lesson training method according to the present invention will be described with reference to the drawings. The present invention is not limited to this embodiment. In addition, the components in the following embodiments include those that can be easily replaced by those skilled in the art, or those that are substantially the same.

FIG. 1 is a functional block diagram showing an example of the lesson training system 100 according to the present embodiment. As shown in FIG. 1, the lesson training system 100 includes an instructor-side system 10, a student-side system 40, and a server 30. The lesson training system 100 is a system for taking lessons for learning how to move the body, such as dance, yoga, and gymnastics. The lesson training system 100 can provide training to a plurality of students by, for example, having a configuration in which a plurality of student-side systems 40 are provided.

The instructor-side system 10 includes a tracking device 11, a control device 12, a display device 13, and an audio output device 14.

The tracking device 11 detects the movement of the instructor's body. The tracking device 11 has a detection unit 11a and a vibration generation unit 11b (see FIG. 2). The detection unit 11a is attached to each part of the body of the instructor and detects the position, speed, and acceleration of each part. In the present embodiment, the detection unit 11a is attached to, for example, the head, shoulders, chest, elbows, hands, hips, knees, feet, and the like. The mounting location of the detection unit 11a is not limited to the above, and may be mounted on another location. The detection unit 11a transmits the detection result to the control device 12. The detection unit 11a may have other configurations such as a camera as long as it can detect the position, speed, and acceleration of each part of the body of the instructor. The vibration generation unit 11b is provided integrally with the detection unit 11a and vibrates under the control of the control device 12.

The control device 12 controls each part of the instructor side system 10. The control device 12 has a processing device such as a CPU (Central Processing Unit) and a storage device such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The control device 12 includes a motion generation unit 16, an avatar generation unit 17, a VR image generation unit 18, a display control unit 19, a communication unit 20, and a storage unit 21.

The motion generation unit 16 acquires the detection result of the detection unit 11a. The motion generation unit 16 generates instructor motion information indicating the movement of each part of the instructor's body based on the position, speed, and acceleration information of each part of the instructor's body acquired as a detection result.

The avatar generation unit 17 generates an instructor avatar image in which the instructor avatar moves based on the instructor motion information. The instructor avatar is stored in the storage unit 21 or the like in advance. The instructor avatar can be displayed three-dimensionally in a virtual space, for example.

The VR video generation unit 18 acquires information on the VR (Virtual Reality) space received from the server 30 by the communication unit 20 described later. The VR space is a three-dimensional image showing a lesson room such as a dance. Based on the acquired information in the VR space, the VR image generation unit 18 generates a VR image in which the instructor avatar image or the student avatar image described later is combined in the VR space.

The display control unit 19 controls the display operation in the display device 13. The display control unit 19 causes the display device 13 to display the VR image generated by the VR image generation unit 18, for example.

The communication unit 20 transmits / receives information to / from the server 30. The communication unit 20 receives, for example, information on the video in the VR space, information on the student avatar video described later, information on the student motion, and the like from the server 30. Further, the communication unit 20 transmits each information such as the information of the instructor avatar video generated by the control device 12 and the instructor motion information to the server 30.

The storage unit 21 stores various types of information. The storage unit 21 has storage such as a hard disk drive or a solid state drive. An external storage medium such as a removable disk may be used as the storage unit 21. The storage unit 21 stores a preset instructor avatar. The storage unit 21 stores information such as programs and data for each unit of the control device 12 to perform processing. The storage unit 21 stores the video information in the VR space received from the server 30.

The display device 13 displays various information such as images under the control of the display control unit 19. The display device 13 can display, for example, the VR image generated by the VR image generation unit 18. Examples of the display device 13 include, but are not limited to, a head-mounted display and the like, and a flat panel display and the like may be used.

The voice output device 14 outputs voice based on the control of the control device 12. As the audio output device 14, for example, a speaker or the like is used.

The student-side system 40 includes a tracking device 41, a control device 42, a display device 43, and an audio output device 44.

The tracking device 41 detects the movement of the student's body. The tracking device 41 has a detection unit 41a and a vibration generation unit 41b (see FIG. 4). The detection unit 41a is attached to each part of the student's body and detects the position, speed, and acceleration of each part. In the present embodiment, the detection unit 41a is attached to, for example, a portion similar to the detection unit 11a, that is, the head, shoulders, chest, elbows, hands, hips, knees, feet, and the like. The mounting location of the detection unit 41a is not limited to the above, and may be mounted on another location. The detection unit 41a transmits the detection result to the control device 42. The detection unit 41a may have other configurations such as a camera as long as it can detect the position, speed, and acceleration of each part of the student's body. The vibration generation unit 41b is provided integrally with the detection unit 41a and vibrates under the control of the control device 42.

The control device 42 controls each part of the student side system 40. The control device 42 has a processing device such as a CPU and a storage device such as RAM or ROM. The control device 42 includes a motion generation unit 46, an avatar generation unit 47, a semi-transparent avatar generation unit 48, a VR image generation unit 49, a display control unit 50, a deviation information output unit 51, and a communication unit 52. It has a storage unit 53.

The motion generation unit 46 acquires the detection result of the detection unit 41a. The motion generation unit 46 generates student motion information indicating the movement of each part of the student's body based on the position, speed, and acceleration information of each part of the student's body acquired as a detection result.

The avatar generation unit 47 generates a student avatar image in which the student avatar moves based on the student motion information. The avatar generation unit 47 stores the generated student avatar image in the storage unit 53. The student avatar is stored in the storage unit 53 or the like in advance. The student avatar can be displayed three-dimensionally in, for example, a virtual space.

The semi-transparent avatar generation unit 48 generates a semi-transparent avatar image in which the generated student avatar image is in a state of increasing transparency. The semi-transparent avatar generation unit 48 stores the semi-transparent avatar image in the storage unit 53 separately from the student avatar image.

The VR video generation unit 49 acquires information on the VR space received by the communication unit 52, which will be described later. The VR image generation unit 49 generates a VR image in which the instructor avatar image and the student avatar image or the translucent avatar image are combined in the VR space based on the acquired information of the VR space. The instructor avatar video is received by the communication unit 52, which will be described later. When the instructor avatar image and the student avatar image are combined, the VR image generation unit 49 can generate a VR image so that the semi-transparent avatar image is superimposed on the instructor avatar image. The VR image generation unit 49 can superimpose, for example, a translucent avatar and an instructor avatar with respect to their respective waists. Further, when the instructor avatar image and the translucent avatar image are combined, the VR image generation unit 49 generates a VR image so that the instructor avatar image and the student avatar image are displayed side by side in the VR space. be able to. Further, the VR video generation unit 49 prevents the student avatar video generated by the other student side system 40 from being combined with the VR video. As a result, it is possible to prevent the student avatar video of the other student S from being displayed on the display device 43. Note that the student avatar video generated by the other student-side system 40 may not be received from, for example, the server 30.

The display control unit 50 controls the display operation in the display device 43. The display control unit 50 causes the display device 43 to display, for example, the VR image generated by the VR image generation unit 49.

The deviation information output unit 51 detects the deviation between the movement of the instructor and the movement of the student, and outputs the detection result. The deviation information output unit 51 acquires the instructor motion information received by the communication unit 52. The deviation information output unit 51 calculates the deviation between the movement of the instructor T and the movement of the student S based on the acquired instructor motion information and the student motion information generated by the motion generation unit 46. The deviation information output unit 51 can calculate the deviation for each location detected by the detection unit 41a, for example. In this case, the deviation information output unit 51 determines the height between the instructor T and the student S based on the distance between the sensors 11a mounted on the instructor T and the distance between the sensors 41a mounted on the student S. Find the ratio of. Then, the deviation information detection unit 51 uses the obtained result to match the scale of each value included in the instructor motion information with each value included in the student motion information. The deviation information output unit 51 calculates the deviation between the movement of the instructor T and the movement of the student S in a state where the scales match. When the size of the deviation is larger than a predetermined value, the deviation information output unit 51 can control the vibration generation unit 41b provided integrally with the detection unit 41a at the corresponding portion to vibrate. This makes it possible to easily inform the student of which part of the body the movement should be corrected. Further, even if the size of the deviation is smaller than the predetermined value, the deviation information output unit 51 is predetermined in the order of the largest deviation among all the corresponding parts where the deviation occurs or the corresponding parts where the deviation occurs. The vibration generation unit 41b provided integrally with the detection unit 41a at the corresponding portion of the number may be controlled to vibrate. Further, the deviation information output unit 51 may display the presence or absence of deviation on the display device 43 via the display control unit 50, or may output it as voice by the voice output device 44 described later.

The communication unit 52 transmits / receives information to / from the server 30. The communication unit 52 receives information such as VR space video information, lecturer avatar video information, and lecturer motion information from the server 30. Further, the communication unit 52 transmits each information such as the student avatar video information, the semi-transparent avatar video information, and the student motion information generated by the control device 42 to the server 30.

The storage unit 53 stores various information. The storage unit 53 has storage such as a hard disk drive or a solid state drive. An external storage medium such as a removable disk may be used as the storage unit 53. The storage unit 53 stores a preset student avatar. The storage unit 53 stores information such as programs and data for each unit of the control device 42 to perform processing. The storage unit 53 stores the video information in the VR space received from the server 30.

The display device 43 displays various information such as images under the control of the display control unit 50. The display device 43 can display, for example, a VR spatial image. Examples of the display device 43 include, but are not limited to, a head-mounted display and the like, and a flat panel display and the like may be used.

The voice output device 44 outputs voice based on the control of the control device 42. As the audio output device 44, for example, a speaker or the like is used.

The server 30 has a communication unit 31, a processing unit 32, and a storage unit 33. The communication unit 31 transmits / receives information to / from the communication unit 20 of the instructor-side system 10 and the communication unit 52 of the student-side system 40, which will be described later. The processing unit 32 manages each lesson course. When necessary information is input by an input unit (not shown) or the like, the processing unit 32 sets, for example, the VR space for each lesson course, the position of the instructor avatar image and the student avatar image to be arranged in the VR space, and the like. It is possible to set the maximum number of students who can take the course.

The processing unit 32 controls the transmission / reception of information in the communication unit 31. Under the control of the processing unit 32, various information such as information about the VR space, instructor motion information, instructor avatar video information, student motion information, and student avatar video information is transmitted and received via the communication unit 31. The storage unit 33 stores various types of information.

The storage unit 33 stores various information transmitted / received by the communication unit 31. In the lesson training system 100, the storage unit 33 stores, for example, the instructor T's avatar for the instructor and the student S's student avatar in the lesson class, and the processing unit 32 stores the instructor's avatar video and the student's avatar. An image, a translucent avatar image, and each VR image may be generated. In this case, the avatar generation unit 17 and the VR image generation unit 18 of the instructor side system 10 and the avatar generation unit 47, the translucent avatar generation unit 48, and the VR image generation unit 49 of the student side system 40 may not be provided. .. As a result, the amount of communication between the instructor-side system 10 and the student-side system 40 and the server 30 can be suppressed.

Next, a lesson training method using the above lesson training system 100 will be described. 2 to 8 are diagrams showing an example of a lesson training method. FIG. 2 is a diagram showing one aspect when the instructor T operates. As shown in FIG. 2, the instructor T performs a sample operation with the detection unit 11a and the vibration generation unit 11b attached to each part of the body. Each detection unit 11a detects the position, speed, and acceleration of each part of the body according to the operation of the instructor T, and transmits the detection result to the control device 12. In the control device 12, the motion generation unit 16 acquires the detection result of the detection unit 11a, and generates instructor motion information indicating the movement of each part of the instructor's body based on the acquired detection result. The motion generation unit 16 stores the generated instructor motion information in the storage unit 21. The communication unit 20 transmits the generated instructor motion information to the server 30.

After the instructor motion information is generated, the avatar generation unit 17 generates an instructor avatar image based on the instructor motion information. FIG. 3 is a diagram showing an example of the instructor avatar video. As shown in FIG. 3, the avatar generation unit 17 generates an instructor avatar image in which the instructor avatar AT performs an operation corresponding to the instructor's operation. The avatar generation unit 17 stores the generated information of the instructor avatar image in the storage unit 21. The communication unit 20 transmits the generated information of the instructor avatar video to the server 30.

The server 30 receives the instructor motion information and the instructor avatar video information transmitted from the instructor side system 10 by the communication unit 31. The server 30 stores each received information in the storage unit 33.

In the student side system 40, the communication unit 52 accesses the server 30 by a predetermined operation of the student S, and receives the information of the image of the VR space, the instructor motion information, and the information of the instructor avatar image. The VR image generation unit 49 generates a VR image based on the received VR space image information and the instructor avatar image information. The display control unit 50 causes the display device 43 to display the generated VR image.

As shown in FIG. 4, the students S1 and S2 who use different student-side systems 40 at different places, while watching the VR image displayed on the display device 43, generate the detection unit 41a and vibration in each part of the body. With the part 41b attached, it is possible to perform an operation according to the movement of the instructor avatar AT. Each detection unit 41a detects the position, speed, and acceleration of each part of the body according to the movement of the student S1, and transmits the detection result to the control device 42. In the control device 42, the motion generation unit 46 acquires the detection result of the detection unit 41a, and generates the student motion information indicating the movement of each part of the body of the students S1 and S2 based on the acquired detection result. The motion generation unit 46 stores the generated student motion information in the storage unit 53. The communication unit 52 transmits the generated student motion information to the server 30.

After the student motion information is generated, the avatar generation unit 47 generates the student avatar image based on the student motion information. FIG. 5 is a diagram showing an example of a student avatar video. As shown in FIG. 5, the avatar generation unit 47 generates a student avatar image in which the student avatars AS1 and AS2 perform an operation corresponding to the operation of the instructor T (lecturer avatar AT). Further, the semi-transparent avatar generation unit 48 generates a semi-transparent avatar image based on the generated student avatar image. The avatar generation unit 47 stores the generated student avatar video and semi-transparent avatar video information in the storage unit 53. The communication unit 52 transmits the generated student avatar video and translucent avatar video information to the server 30.

The server 30 receives the student motion information, the student avatar video, and the semi-transparent avatar video information transmitted from the student side system 40 by the communication unit 31. The server 30 stores each received information in the storage unit 33.

FIG. 6 is a diagram showing an example of a VR image displayed on the display device 13 of the instructor side system 10. In the instructor side system 10, the control device 12 receives the student motion information, the student avatar video, and the translucent avatar video information stored in the storage unit 33 of the server 30 via the communication unit 20. The VR image generation unit 18 generates a VR image based on each received information. The display control unit 19 displays the generated VR image on the display device 13. As shown in FIG. 6, when there are a plurality of student S, the display control unit 19 can display the student avatar video for the plurality of student avatar AS on the display device 13. For example, the student avatars AS1 and AS2 of the students S1 and S2 can be displayed on the display device 13 in a state of being arranged in the VR space Va. As a result, the operation of the plurality of students S (AS1, AS2) can be confirmed collectively from the instructor T side.

7 and 8 are diagrams showing an example of a VR image displayed on the display device 43 of the student side system 40. In the student side system 40, the VR image generation unit 18 generates a VR image so that the instructor avatar image received from the server 30 and the generated student avatar image are arranged correspondingly in the VR space. Can be done.

As shown in FIG. 7, for example, the VR image generation unit 49 can generate a VR image in which the instructor avatar AT and the translucent avatar AH are superimposed in the VR space Vb. The display control unit 19 displays the generated VR image on the display device 13. By comparing the instructor avatar AT and the translucent avatar AH displayed in a state of being superimposed on the display device 43, the student S can more easily grasp the difference between the operation of the student S and the operation of the instructor T. In this case, the VR image generation unit 18 makes the student S clearly grasp the deviation of the upper body and the deviation of the lower body by, for example, matching the position of the waist of the instructor avatar AT with the position of the waist of the translucent avatar AH. be able to.

Further, the VR image generation unit 49 can generate a VR image in which the instructor avatar AT and the student avatar AS1 are arranged side by side in the VR space Vb, as shown in FIG. 8, for example. The display control unit 19 displays the generated VR image on the display device 13. The student S can easily grasp the difference between the operation of the student S and the operation of the instructor T by comparing the instructor avatar AT displayed side by side on the display device 43 with the student avatar AS1.

The deviation information output unit 51 acquires the instructor motion information received by the communication unit 52, and based on the acquired instructor motion information and the student motion information generated by the motion generation unit 46, the movement of the instructor and the student. Calculate the deviation from the movement of. The deviation information output unit 51 may control the deviation information output unit 51 to vibrate the vibration generating unit 41b provided integrally with the detection unit 41a at the corresponding location when the deviation size exceeds a predetermined value. can.

For example, as a result of the deviation calculation, the deviation information output unit 51 has a deviation smaller than a predetermined value for the positions of the right shoulder, left shoulder, right elbow, left elbow, and left foot of the student avatar AS1 with respect to the instructor avatar AT. However, it is assumed that the positions of the head, the right arm, the left arm, the right knee, the right foot, and the left knee are determined to be larger than the predetermined values. In this case, the deviation information output unit 51 is the detection unit 41a attached to the body of the student S, which is attached to the head, right arm, left arm, right knee, right foot, and left knee, which are determined to have a large deviation. The vibration generating portion 41b integrated with the 41a is vibrated. The vibration of the vibration generating portion 41b enables the student S to easily and quickly grasp the portion where the deviation is large with respect to the movement of the body of the instructor T, which can be linked to the motion correction. The deviation information output unit 51 may vibrate the vibration generating unit 41b only for a predetermined number of parts in descending order of deviation. In this case, the student S can more easily grasp the part to be corrected with priority on the body movement of the instructor T.

FIG. 9 is a flowchart showing an example of the lesson training method according to the present embodiment. FIG. 9 shows the instructor-side step, which is the operation of the instructor-side system 10, and the student-side step, which is the operation of the student-side system 40, in correspondence with each other. As shown in FIG. 9, in the instructor side system 10, the tracking device 11 is positioned by the detection unit 11a and the vibration generating unit 11b attached to each part of the body of the instructor T according to the movement of the instructor T. , Velocity and acceleration are detected, and the detection result is transmitted to the control device 12 (step S110). Further, in the student-side system, the tracking device 41 uses the detection unit 41a and the vibration generation unit 41b attached to each part of the body of the student S to determine the position and speed of each part of the body according to the movement of the student S. The acceleration is detected and the detection result is transmitted to the control device 42 (step S210).

In the instructor side system 10, the motion generation unit 16 acquires the detection result of the detection unit 11a and generates instructor motion information based on the acquired detection result (step S120). Further, in the student side system 40, the motion generation unit 46 acquires the detection result of the detection unit 41a and generates the student motion information based on the acquired detection result (step S220).

In the instructor side system 10, the avatar generation unit 17 generates an instructor avatar image based on the instructor motion information (step S130). Further, in the student side system 40, the avatar generation unit 47 generates a student avatar image based on the student motion information (step S230).

In the instructor side system 10, the communication unit 20 transmits / receives information to / from the server 30 (step S140). In step S140, the communication unit 20 transmits the generated instructor motion information and the instructor avatar video to the server 30. Further, in step S140, the communication unit 20 receives the student motion information and the student avatar video, which will be described later, from the server 30. Further, in the student side system 40, the communication unit 52 transmits each generated information to the server 30 (step S240). In step S240, the communication unit 52 transmits the generated student motion information and the student avatar video to the server 30. Further, in step S240, the communication unit 52 receives the instructor motion information and the instructor avatar video from the server 30.

In the instructor side system 10, the VR image generation unit 18 generates a VR image based on each received information. The display control unit 19 displays the generated VR image on the display device 13 (step S150). By looking at the display device 13, the instructor T can easily grasp whether or not the movement of the student S is appropriate. Further, in the student side system 40, the VR video generation unit 49 generates a VR video based on each received information. The display control unit 50 displays the generated VR image on the display device 43 (step S250). By looking at the display device 43, the student S can easily grasp the movement of the instructor T as a sample.

Further, in the student side system, the deviation information output unit 51 acquires the instructor motion information received by the communication unit 52, and is based on the acquired instructor motion information and the student motion information generated by the motion generation unit 46. Then, the difference between the movement of the instructor and the movement of the student is calculated (step S260). The deviation information output unit 51 determines whether or not the deviation information output unit 51 is larger than a predetermined value (step S270). When the deviation information output unit 51 determines that the deviation size is larger than a predetermined value (Yes in step S270), the deviation information output unit 51 controls to vibrate the vibration generating unit 41b provided integrally with the detection unit 41a at the corresponding location. (Step S280). When the deviation information output unit 51 determines that the size of the deviation is not larger than the predetermined value (No in step S270), the deviation information output unit 51 skips step S280.

In the instructor side system 10, the control device 12 determines whether or not all the operations have been completed after displaying the VR image (step S160). When the control device 12 determines that all the operations have not been completed (No in step S160), the control device 12 causes the operations after step S110 to be repeated. Further, when it is determined that all the operations are completed (Yes in step S160), the control device 12 ends the operations. Further, in the student-side system 40, the control device 42 determines whether or not all the operations are completed after displaying the VR image or generating the vibration (step S260). When the control device 42 determines that all the operations have not been completed (No in step S260), the control device 42 causes the operations after step S210 to be repeated. Further, when it is determined that all the operations are completed (Yes in step S260), the control device 42 ends the operations.

As described above, the lesson training system 100 according to the present embodiment tracks the body movement of the instructor T, generates instructor motion information based on the tracking result, and the instructor avatar indicating the instructor T is based on the instructor motion information. The instructor side system 10 that generates the instructor avatar image that moves and sends the generated instructor avatar image, tracks the body movement of the student S, generates the student motion information based on the tracking result, and the student S The student avatar showing the above generates a student avatar image that moves based on the student motion information, and also receives the instructor avatar image transmitted from the instructor side system 10, and the generated instructor avatar image and the received instructor avatar. It is equipped with a student-side system 40 that displays images in correspondence with each other.

Further, in the lesson class method according to the present embodiment, the movement of the body of the instructor T is tracked, the instructor motion information is generated based on the tracking result, and the instructor avatar indicating the instructor T moves based on the instructor motion information. A student avatar image that generates a video, tracks the body movement of the student S, generates a student motion information based on the tracking result, and the student avatar indicating the student S moves based on the student motion information. Is generated, and the generated student avatar image and the instructor avatar image generated in the instructor side step are displayed in association with each other.

According to this configuration, the instructor avatar image transmitted from the instructor side system 10 and the student avatar image generated by the student side system 40 are displayed on the display device 43 of the student system 40, and the other student S. Since the video about is not displayed, the student S can take the lesson class without worrying about the eyes of other students S. Further, since the instructor avatar image and the student avatar image are displayed in correspondence with each other, the student S can easily grasp the operation of the sample instructor T in correspondence with the student's own operation. As a result, the student S can intuitively understand the appropriate operation.

In the lesson training system 100 according to the present embodiment, the student side system 40 superimposes the student avatar image on the instructor avatar image and displays it. As a result, the student S can more reliably grasp the matching portion or the non-matching portion between the movement of the instructor T and the movement of the student S himself / herself.

In the lesson training system 100 according to the present embodiment, the instructor side system 10 transmits the instructor motion information, and the student side system 40 receives the instructor motion information transmitted from the instructor side system 10 and receives the instructor motion. Based on the information and the generated student motion information, the deviation between the movement of the instructor T and the movement of the student S is detected, and the detection result is output. As a result, the deviation between the movement of the instructor T and the movement of the student S can be surely grasped by the student S.

In the lesson training system 100 according to the present embodiment, the student-side system 40 vibrates a predetermined number of parts in the detected deviation in which the magnitude of the deviation exceeds a predetermined value or in descending order of the deviation. As a result, the student S can more easily grasp the part to be corrected with priority on the body movement of the instructor T.

The technical scope of the present invention is not limited to the above-described embodiment, and changes can be made as appropriate without departing from the spirit of the present invention. For example, the lesson training system 100 according to the above embodiment has been described by exemplifying a configuration in which the server 30 is separately provided for the instructor side system 10 and the student side system 40, but the present invention is not limited to this. For example, the server 30 may not be provided, and the function of the server 30 may be installed in at least one of the instructor side system 10 and the student side system 40.

Further, in the above embodiment, the mode in which the instructor avatar and the student avatar are displayed three-dimensionally in the virtual space has been described as an example, but the present invention is not limited to this. The instructor avatar and the student avatar may be, for example, a mode in which a state viewed from a predetermined direction is displayed two-dimensionally on a virtual plane.

AH ... Semi-transparent avatar, S, S1, S2 ... Student, T ... Instructor, AS, AS1, AS2 ... Student avatar, Va, Vb ... VR space, F ... Bottom of VR space, 10 ... Instructor side system, 11 , 41 ... Tracking device, 11a, 41a ... Detection unit, 11b, 41b ... Vibration generation unit, 12,42 ... Control device, 13,43 ... Display device, 14,44 ... Audio output device, 16,46 ... Motion generation unit , 17,47 ... Avatar generation unit, 18,49 ... VR image generation unit, 19,50 ... Display control unit, 20,31,52 ... Communication unit, 21,33,53 ... Storage unit, 30 ... Server, 32 ... VR information processing department, 40 ... student side system, 48 ... semi-transparent avatar generation department, 51 ... deviation information output unit, 100 ... lesson training system

Claims (5)

The movement of the instructor's body is tracked, the instructor motion information is generated based on the tracking result, the instructor avatar indicating the instructor generates the instructor avatar image that moves based on the instructor motion information, and the generated instructor avatar image is used. The instructor side system to send and
The movement of the student's body is tracked, the student motion information is generated based on the tracking result, and the student avatar indicating the student generates a student avatar image that moves based on the student motion information. A lesson training system including a student side system that receives the teacher avatar video transmitted from the teacher side system and displays the generated student avatar video in correspondence with the received teacher avatar video. The lesson training system according to claim 1, wherein the student-side system superimposes and displays the student avatar image on the instructor avatar image. The instructor-side system transmits the instructor motion information,
The student-side system receives the instructor motion information transmitted from the instructor-side system, and based on the received instructor motion information and the generated student motion information, the instructor's movement and the instructor's attendance. The lesson training system according to claim 1 or 2, wherein the deviation from the movement of the person is detected and the detection result is output. The lesson training system according to claim 3, wherein the student-side system vibrates a predetermined number of parts of the detected deviations in the order of the deviations exceeding a predetermined value or the deviations in descending order. The movement of the instructor's body is tracked, the instructor motion information is generated based on the tracking result, the instructor avatar indicating the instructor generates the instructor avatar image that moves based on the instructor motion information, and the generated instructor avatar image is used. Send and
The movement of the student's body is tracked, the student motion information is generated based on the tracking result, and the student avatar indicating the student generates a student avatar image in which the student avatar moves based on the student motion information. A lesson class method in which the transmitted instructor avatar image is received, and the generated student avatar image and the received instructor avatar image are displayed in association with each other.

Images