JP2009018127A - Learning support device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a learning support device allowing a user to effectively learn an exercise. <P>SOLUTION: This learning support device 10 is provided with: a target movement information storage part 131 storing object information 131a indicating a target movement of a learning object site in a target exercise of a user; a target movement creation part 135 calculating the user's field of view based on the position of the user's head and the posture and, when the user performs the target exercise based on the calculated user's field of view, an image display position of an image display part 113 and object information 131a, creating drawing information, or image data for displaying an image of the target movement in the user's field of view; and the image display part 113 for displaying the movement of the learning object site in the target exercise based on the drawing information. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a learning support apparatus and a learning support method that support learning of exercise.

  Traditionally, in order to improve learning in exercise, it has been widely practiced to receive coaching guidance, see the exercises of advanced players, and watch videos of their exercise using videos and mirrors. Yes. However, beginners who begin to exercise in the future and beginners who do not easily improve often cannot imagine the target exercise well from words and movements, and in such cases it is difficult to improve.

  Thus, for example, Patent Literature 1 and Patent Literature 2 describe a technique that enables learning that is more intuitive and bodily sensation by presenting the trajectory of a part such as a hand or a foot in the exercise of an advanced person.

  In the technique disclosed in Patent Document 1, in a teaching material for learning how to carry a brush in calligraphy, the strength of the writing pressure and the position of the center of gravity in carrying a brush by an advanced user are displayed as a graphic along the movement path of the brush. The user can practice calligraphy while imagining the movement of a brush by an advanced user by tracing the graphic displayed on the teaching material according to the meaning of each graphic.

In the technique described in Patent Document 2, in a device for learning how to carry a brush in drawing, a reference line is displayed, and a deviation from the writing line written by the user is measured by tracing the reference line. Determine sex. A user repeats an effort so that the judgment result of skillfulness improves, and it becomes easy to imagine a senior's exercise.
JP 2004-170736 A JP 2000-75783 A

  However, in the techniques described in Patent Document 1 and Patent Document 2, for example, when the position and posture of the user with respect to the displayed graphic and reference line are different from those of the advanced user, the user's exercise is the exercise of the advanced user. It can be far away. Then, it is good to learn to obtain the result of one pattern of exercise, but it is not necessarily suitable for advanced exercise learning, and conversely, it is highly likely that it will be difficult to achieve the original purpose of exercise improvement. Become.

  The present invention has been made in view of such a point, and an object thereof is to provide a learning support device and a learning support method that enable effective learning of exercise.

  A learning support device according to the present invention is a learning support device for learning a predetermined motion, and a target motion acquisition unit that acquires target motion information indicating a target motion that is a motion of a part to be learned in the predetermined motion; And a drawing unit that draws the target motion in conformity with the field of view of the user who learns the predetermined motion based on the target motion information acquired by the target motion acquisition unit.

  The learning support method of the present invention is a learning support method for learning a predetermined motion, and a target motion acquisition step of acquiring target motion information indicating a target motion that is a motion of a learning target part in the predetermined motion; A drawing step of drawing an image of the target motion in conformity with a field of view of the user who learns the predetermined motion based on the target motion information acquired in the target motion acquisition step.

  According to the present invention, since the image of the target motion is drawn in conformity with the user's field of view, the user can trace the motion of the specific part in the predetermined motion at a more correct position and posture. As a result, it is possible to learn exercise more effectively than when learning exercise indirectly from the result of exercise.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(Embodiment 1)
FIG. 1 is a diagram illustrating a scene in which the learning support apparatus according to Embodiment 1 of the present invention is used. The first embodiment is an example in which the present invention is applied to a learning support apparatus that supports learning of golf club swing.

  As shown in FIG. 1, the learning support device 10 includes a head mounted display (HMD) 110, a triaxial acceleration sensor 120, and a control terminal 130. The control terminal 130 is placed near the user 150 who practices swinging a golf club. The head mounted display 110 is attached to the head of the user 150. The triaxial acceleration sensor 120 is detachably attached to an exercise device or a body part (hereinafter referred to as “learning target”) that the user 150 focuses on when learning the target exercise of the user 150. Here, an example is shown in which a club head 160 of a golf club is selected as a learning target, and the three-axis acceleration sensor 120 is attached to the club head 160.

  The head mounted display 110 is an optical see-through display device, and is based on information sent from the control terminal 130 to an image of a real field of view, such as the golf ball 170 and the club head 160 in front of the user 150. A computer-generated image is optically superimposed and displayed.

  Specifically, the head mounted display 110 performs drawing on the drawing surface fixed to the mount display 110 based on drawing information described later sent from the control terminal 130. In addition, the head mounted display 110 measures the viewpoint that is the starting point of the line of sight of the user 150 and the position and posture of the user 150 head, and transmits them to the control terminal 130. Here, the viewpoint is the position of the pupil of the user 150, and the posture refers to the direction and inclination of the object.

  The triaxial acceleration sensor 120 measures its own acceleration as an analog value, converts the measurement data into acceleration data that is a digital value, and transmits the acceleration data to the control terminal 130.

  The control terminal 130 is a predetermined physical quantity such as a target trajectory of the club head 160 (hereinafter referred to as a “target trajectory”) and a head surface direction at each position of the target trajectory in the target swing of the user 150 (the target trajectory and the target trajectory). The operation indicated by the predetermined physical quantity and the information indicating this operation are hereinafter collectively referred to as “target operation”). Based on the information from the head mounted display 110, the control terminal 130 creates drawing information, which is image data for drawing with the acquired target motion adapted to the field of view of the user 150. Send.

  Furthermore, the control terminal 130 determines the actual trajectory of the club head 160 (hereinafter referred to as “real trajectory”) and the orientation of the head surface at each position of the real trajectory based on the information sent from the three-axis acceleration sensor 120. A predetermined physical quantity (the actual trajectory and the operation indicated by the predetermined physical quantity and information indicating this operation is hereinafter collectively referred to as “actual operation”) is acquired. Then, the control terminal 130 compares the actual motion with the target motion, creates drawing information for rendering the comparison result in conformity with the field of view of the user 150, and transmits the drawing information to the head mounted display 110.

  By using the learning support device 10 as described above, the target motion is drawn at a position suitable for the field of view of the user 150. Thereby, in swing learning, the user 150 can perform swing practice while visually confirming the target motion of the club head 160, and can check the actual motion of the club head 160 by comparing with the target motion. . By repeatedly practicing using such a learning support device 10, the user 150 can bring his / her swing exercise closer to the target swing exercise.

  Next, a detailed configuration of the learning support apparatus 10 will be described.

  FIG. 2 is a block diagram illustrating a detailed configuration of the learning support apparatus 10.

  In the learning support apparatus 10 illustrated in FIG. 2, the head mounted display 110 includes a visual field information measurement unit 111, a position / orientation information measurement unit 112, and an image display unit 113. The triaxial acceleration sensor 120 has a motion information measurement unit 121. The control terminal 130 includes a target motion information storage unit 131, a learning target acquisition unit 132, a field of view information acquisition unit 133, a position and orientation information acquisition unit 134, a target motion creation unit 135, a target motion drawing unit 136, a motion information acquisition unit 137, and A passage determination unit 138 is included.

  The target motion information storage unit 131 stores a plurality of object information 131a-1, 131a-2,..., 131a-N (N is a natural number) corresponding to different parts in advance. The object information 131a is basic information for calculating a target motion to be learned. Specifically, the object information 131a defines an object that is a symbol for visually indicating the predetermined physical quantity described above at one or a plurality of positions constituting the target trajectory to be learned.

  FIG. 3 is a diagram showing an example of the contents of the object information 131a corresponding to the club head (hereinafter referred to as “club” as appropriate).

  As shown in FIG. 3, the object information 131a includes a part name 201, an object number 202, an immediately preceding object number 203, an immediately following object number 204, a point importance degree 205, a position coordinate 206, a size 207, an acceleration 208, and others. It consists of information 209. These pieces of information are described for each position constituting the target trajectory of the part.

  The part name 201 describes which part the other information is related to. The object number 202 describes an object number as position identification information. In the immediately preceding object number 203, the number of another object corresponding to the immediately preceding position in the target trajectory is described. The immediately following object number 204 describes the number of another object corresponding to the position immediately after the part in the target trajectory. The point importance degree 205 describes the importance degree as an object point. In the position coordinates 206, position coordinates (x, y, z) representing a position in a coordinate system (hereinafter referred to as “reference coordinate system”) fixed to the control terminal 130 are described. In the size 207, information specifying the size of the object is described. The acceleration 208 describes the acceleration of the part at the corresponding position. The other information 209 describes a predetermined physical quantity other than the acceleration related to the target motion and visual information when drawing the object.

  As the predetermined physical quantity related to the target exercise, for example, acceleration, posture, speed, rotation speed, rotation acceleration, and force applied by the user 150 to the learning target may be described. Further, as the visual information when drawing the object, attributes such as the shape, resolution, color, transparency, luminance, density, and gradation of the object are described.

  The learning target acquisition unit 132 of FIG. 2 gives the user 150 a plurality of parts such as a club head 160 and hands corresponding to the object information 131a-1, 131a-2,. The learning target information that is presented and indicates the selection result is output to the target action creating unit 135. Specifically, the learning target acquisition unit 132 displays a control panel for receiving various operations including selection of the learning target on a liquid crystal touch panel (not shown), and receives the learning target selection of the user 150. .

  FIG. 4 is a diagram illustrating an example of the appearance of the control panel displayed by the learning target acquisition unit 132.

  The control panel 210 includes buttons for accepting operations for starting and ending processing for learning exercises, switching display menus, and switching sound volumes. FIG. 4 shows an appearance of the control panel 210 when a menu for selecting a learning target is displayed.

  The control panel 210 may further display a screen for performing various other settings such as a menu for selecting a learning level. Here, the learning level is a level that is classified according to the speed of tracing the target action when, for example, guiding the actual action by displaying the objects sequentially in the order to be traced.

  Note that instead of the selection operation on the control panel 210, the user 150 may be photographed with a digital camera, and the selection of the learning target or the learning level may be accepted based on the gesture of the user 150. In this case, the learning target acquisition unit 132 sets the association between each option and a specific gesture in advance, analyzes the image of the user 150 captured by the digital camera, and sets the preset gesture and Perform matching.

  For example, an arm set gesture can be set in association with the start of menu selection, a pointing gesture can be set in selection of a target region, and a second arm set gesture can be set in correspondence with selection end. In this case, the user 150 performs the gesture such as performing the arm assembly for a few seconds and then indicating the part to be learned with the finger for a few seconds and then performing the arm assembly again, so that the user 150 remains in the practice position away from the learning support device 10. It is possible to select a learning target and a learning level.

  2 includes a viewpoint detection device (not shown) including a light source, a mirror, and a small digital camera, and detects the viewpoint of the user 150 for each unit time using the viewpoint detection device. To do. Specifically, the visual field information measurement unit 111 uses a corneal reflection method in which eye movement is detected by applying light from a light source to the cornea, reflecting the reflected image with a mirror, and capturing it with a small digital camera. The visual field information measuring unit 111 acquires information representing the detected visual line direction in a coordinate system (hereinafter referred to as “HMD coordinate system”) fixed to the head mounted display 110 main body as visual field information. Then, the visibility information measuring unit 111 transmits the visibility information to the control terminal 130 as time-series data for each unit time by wireless communication.

  The visual field information acquisition unit 133 receives the visual field information transmitted from the visual field information measurement unit 111 and outputs the visual field information to the target motion creation unit 135.

  The position / orientation information measuring unit 112 includes a magnetic sensor (not shown) fixed to the head mounted display 110 main body. The position and orientation of the magnetic sensor are acquired by the position / orientation information acquisition unit 134.

  The position / orientation information acquisition unit 134 acquires the position and orientation of the magnetic sensor provided in the position / orientation information measurement unit 112, and uses the information representing the position and orientation of the magnetic sensor in the reference coordinate system as the position / orientation information as the target operation. The data is output to the creation unit 135.

  Since the magnetic sensor is fixed to the head mounted display 110 main body, the position and orientation of the head mounted display 110 in the reference coordinate system and the position of the drawing surface provided on the head mounted display 110 are obtained from the position and orientation information. Is possible. Further, since the HMD coordinate system 242 is also fixed to the main body of the head mounted display 110, it is possible to acquire the line-of-sight direction of the user 150 in the reference coordinate system from the view information and the position and orientation information. The relationship between the coordinate systems will be described later with reference to another drawing.

  The target motion creation unit 135 acquires object information 131a corresponding to the part indicated by the learning target information from the target motion information storage unit 131. The target action creating unit 135 creates drawing information for drawing the target action from the image of the object on the drawing surface of the head mounted display 110 from the view information and the position / orientation information according to the description content of the acquired object information 131a. Then, the target action creating unit 135 outputs the created drawing information to the target action drawing unit 136.

  In addition, the target action creation unit 135 outputs the acquired object information 131a to the passage determination unit 138. Then, the target motion creation unit 135 creates new drawing information based on the passage determination information output from the passage determination unit 138 and outputs the new drawing information to the target motion drawing unit 136. The passage determination information is information indicating a comparison result between the actual motion to be learned and the target motion, and includes a passage determination result indicating whether or not the actual trajectory to be learned can trace the target trajectory to be learned.

  The target action drawing unit 136 transmits the drawing information created by the target action creating unit 135 to the image display unit 113 of the head mounted display 110 by wireless communication.

  The image display unit 113 has a drawing surface (not shown), and draws an object on the drawing surface based on drawing information sent from the control terminal 130. The user 150 can practice swinging while simultaneously viewing the image of the object drawn on the drawing surface and the golf ball 170 and the club head 160 in real field of view.

  FIG. 5 is a diagram illustrating an example of a drawing surface of the head mounted display 110.

  As shown in FIG. 5, an object image 232 that is a two-dimensional image or a three-dimensional image is displayed on the drawing surface 230 so as to be superimposed on the real-view image 231 based on the drawing information sent from the control terminal 130. To do. Here, the default symbol of the object is a sphere, and the drawing surface 230 when the object image 232 corresponding to the object numbers A1 to A3 shown in FIG. Is illustrated. Note that, here, an object number is added to the inside of the object image 232, but it is not always necessary to add an object number to the object image 232.

  Note that the method of drawing an object is not limited to the above contents. For example, a video-type see-through display that displays an image of a fusion of real-world video and computer-generated video on a display, or a retinal display that directly draws computer-generated video on the retina using a low-power laser, or computer-generated A projection type display device that directly projects the information may be used.

  The motion information measuring unit 121 acquires acceleration data representing the acceleration in the triaxial direction measured by the triaxial acceleration sensor 120 in the reference coordinate system as motion information for each unit time. Then, the exercise information measuring unit 121 transmits the exercise information to the control terminal 130 as time series data for each unit time by wireless communication.

  The motion information acquisition unit 137 receives the motion information (acceleration data) sent from the motion information measurement unit 121 of the three-axis acceleration sensor 120 and outputs it to the passage determination unit 138.

  The passage determination unit 138 acquires the actual operation of the learning target (here, the club head 160) from the exercise information. The passage determination unit 138 compares and analyzes the acquired actual motion and the target motion to be learned defined in the object information 131a, and creates passage determination information including the above-described passage determination result. Then, the passage determination unit 138 outputs the created passage determination information to the target action creation unit 135. As described above, the target motion creation unit 135 creates drawing information for feeding back the passage determination result to the user 150 based on the passage determination information.

  Note that the method of acquiring the actual operation to be learned is not limited to the above content. For example, a pressure sensor may be attached to the learning target to obtain acceleration data, or the movement of the marker attached to the learning target may be photographed with a camera and the photographed image may be analyzed.

  The control terminal 130 and the head mounted display 110 include, for example, a central processing unit (CPU), a storage medium such as a ROM (read only memory) that stores a control program, a working memory such as a random access memory (RAM), and a communication circuit Each is provided. In this case, the functions of the above-described units of the control terminal 130 and the head mounted display 110 are realized by the CPU executing the control program.

  Here, the principle of calculating the relationship between the coordinate systems and the drawing position of the object in the learning support apparatus 10 will be described.

  FIG. 6 is a diagram showing the principle of calculating the relationship between the coordinate systems and the drawing position of the object.

In FIG. 6, a reference coordinate system 241 (X _h , Y _h , Z _h ) is a coordinate system with the position and orientation information acquisition unit 134 fixed to the position and orientation of the control terminal 130 as the origin. The position / orientation information acquisition unit 134 of the control terminal 130 measures the position and orientation of the magnetic sensor fixed to the head mounted display 110 using the reference coordinate system 241.

The HMD coordinate system 242 (X _e , Y _e , Z _e ) is a coordinate system that is fixed to the viewpoint detection device 112 and has a predetermined position on the drawing surface 230 of the head mounted display 110 as an origin. The viewpoint detection device 112 fixed to the head mounted display 110 detects the position of the viewpoint 243 of the user 150 using the HMD coordinate system 242.

  For simplicity of explanation, it is assumed here that the coordinate system on the drawing surface 230 coincides with the HMD coordinate system 242.

  The position and orientation of the magnetic sensor 111a in the HMD coordinate system 242 are measured in advance. Alternatively, the magnetic sensor 111a and the viewpoint detection device 112 are fixed so that the position and posture of the magnetic sensor 111a in the HMD coordinate system 242 become a predetermined position and posture. Thereby, position coordinates and vectors can be converted between the HMD coordinate system 242 and the reference coordinate system 241.

  Further, the position and orientation of the drawing surface 230 in the HMD coordinate system 242 are measured in advance. Alternatively, the viewpoint detection device 112 is fixed so that the position and orientation of the drawing surface 230 in the HMD coordinate system 242 become a predetermined position and orientation. Thereby, position coordinates and vectors can be converted between the HMD coordinate system 242 and the coordinate system on the drawing surface 230.

  That is, the position on the viewpoint 243 and the drawing surface 230 of the user 150 can also be expressed by the reference coordinate system 241.

  Further, the position coordinates of the object 250 defined in the object information 131a are values of the reference coordinate system 241, and the intersection of the straight line 251 connecting the viewpoint 243 and the object 250 with the drawing surface 230 in the reference coordinate system 241. It can be easily obtained. This intersection is the position of the image of the object 250 on the drawing surface 230 when the user 150 views the object 250 through the drawing surface 230.

  As described above, in the learning support device 10, geometric matching between the reference coordinate system 241 and other coordinate systems is achieved.

  The target motion creation unit 135 uses the relationship between the coordinate systems described above to determine the object image on the drawing surface 230 based on the viewpoint 243 of the user 150, the position of the drawing surface 230, and the three-dimensional position of the object 250. The position and orientation to be drawn 232 are obtained. Thereby, for example, the learning support apparatus 10 uses the three-dimensional target trajectory 252 of the object 250 defined by the reference coordinate system 241 in the object information 131a as the two-dimensional target trajectory of the object image 232 on the drawing surface 230. It can be converted into H.253 and displayed.

  However, since the area of the drawing surface 230 is finite, the object image 232b that should be drawn corresponding to the object 250b located outside the range of the drawing surface 230 as viewed from the viewpoint 243 is not actually drawn. become.

  In addition, the learning support device 10 stores in advance the standing position and orientation of the basic posture that the user 150 should take in the exercise for realizing the target motion defined by the object information 131a, that is, the target exercise. The relative positions of the control terminal 130 and the golf ball 170 are set so that the standing position and orientation of the user 150 taking the basic posture matches the stored standing position and orientation. By such setting and the transformation of the coordinate system, it is possible to draw the target motion in the field of view of the user 150 when the user 150 performs the targeted exercise in conformity with the field of view of the user 150.

Incidentally, standing position of the user 150 to the initial position of the golf ball 170, since it is determined in relation to the direction in which hit the ball, golf ball 170 initial position reference coordinate system _261, based on the (X w _{of, Y} w, Z _w ) may be set. Specifically, for example, the initial position of the golf ball 170 is the origin, the vector from the control terminal 130 toward the golf ball 170 is the X axis, the vertical direction is the Z axis, and the cross product direction of the X axis and the Z axis is the Y axis. A coordinate system to be used is a reference coordinate system 261 (X _w , Y _w , Z _w ). In this case, the control terminal 130 may be installed such that the initial position of the golf ball 170 is located on the ground or floor 262 at a predetermined distance and direction with respect to the control terminal 130. Alternatively, a reference coordinate system 261 with the initial position of the golf ball 170 as the origin is attached by attaching a marker to the golf ball 170 and specifying the position and direction of the marker from the captured image of the digital camera provided in the control terminal 130. May be set.

  Further, the reference coordinate system 260 may be set based on the position and posture of the user 150 such as the body. In this case, for example, a magnetic sensor is attached to a predetermined position on the body of the user 150, and the position and orientation of the magnetic sensor are detected by the control terminal 130. Thereby, it is possible to draw a target motion in conformity with the field of view of the user 150 even for movement or movement involving a large movement.

  Next, an example of the principle of passage determination by the passage determination unit 138 will be described.

  FIG. 7 is a diagram illustrating an example of the principle of passage determination by the passage determination unit 138.

  The passage determination unit 138 double-integrates the acceleration data from the three-axis acceleration sensor 120 and obtains a relative change from the initial position of the three-dimensional position of the learning target (in this case, the club head 160), thereby obtaining the actual operation of the learning target. Is calculated. Specifically, the passage determination unit 138 acquires, for example, a state in which the head surface of the club head 160 is along the golf ball as the initial position and initial posture of the learning target. Then, the passage determination unit 138 sequentially obtains the time series of the motion results by repeating the process of obtaining the motion after a minute time from the acceleration at a certain time of the learning target by the Euler method or the like.

  It is difficult to determine which point 271 of the actual trajectory 270 to be learned should be compared with which object 250. Therefore, in the present embodiment, a position 271 where the distance from a certain object 250 in the actual trajectory 270 to be learned is the minimum is set as a position to be compared with the object 250. When the distance between the learning target position 271 and the position of the object 250 (hereinafter referred to as “deviation amount”) is smaller than a predetermined threshold value, it is determined that the distance has succeeded, and the deviation amount is greater than or equal to the predetermined threshold value. In the case, it is determined that it has failed.

  Here, success means that the real trajectory 270 to be learned has passed the object 250 located in the target trajectory, and failure means that the real trajectory 270 to be learned has passed the object 250 located in the target trajectory. That didn't happen.

  The passage determination unit 138 calculates a learning target position 271 for each minute time. It is assumed that the position (x, y, z) of the learning target at a certain time is displaced to the position (x + Δx, y + Δy, z + Δz) after a short time. The passage determination unit 138 uses the position (x, y, z) before displacement when the displacement amount d ′ after displacement becomes larger than the displacement amount d before displacement as the position where the distance from the object 250 is minimized. Is determined.

Specifically, the passage determining unit 138 first sequentially determines whether or not the following expression (1) is satisfied with respect to the position (x _a , y _a , z _a ) of the object 250.

  When there are a large number of objects 250, the processing load increases if the above calculation is performed on all the objects 250. Therefore, the passage determination unit 138 performs passage determination one by one in accordance with the order of the objects defined in the object information 131a. For example, as shown in FIG. 7, it is assumed that an object 250-A1 corresponding to the object number A1 and an object 250-A2 corresponding to the object number A2 which is the next object number exist. In this case, the passage determination unit 138 first performs determination using the expression (1) only for the object 250-A1, and after the expression (1) is satisfied, the expression (1) for the next object 250-A2 is established. What is necessary is just to perform the determination using.

  The learning target is assumed to have passed positions 271-1 to 271-3 shown in FIG. 7 in this order. In the case where the determination of Expression (1) is executed for the object 250-A2, for example, when the position 271-1 shown in FIG. 7 is set as (x, y, z), the learning target is the object 250-A2. Since moving in the approaching direction, equation (1) does not hold. On the other hand, when the positions 271-2 and 271-3 shown in FIG. 7 are set as (x, y, z) and (x + Δx, y + Δy, z + Δz), the learning object moves in a direction away from the object 250 -A 2. Therefore, Formula (1) is materialized.

  When the passage determination unit 138 determines that the expression (1) is established, the minimum deviation amount of the learning target actual trajectory 270 with respect to the position of the object 250 from the learning target position (x, y, z) at that time. d is calculated using, for example, the following equation (2).

  Then, the passage determination unit 138 compares the calculated deviation amount d with a predetermined threshold value, and determines whether or not it is successful.

  Further, when the passage determination unit 138 determines that the expression (1) is established, the posture of the learning target at that time (here, the normal vector of the head surface of the club head 160) is defined in the object information 131a. Compare with the posture of the object.

  Specifically, for example, the passage determination unit 138 calculates an angle (hereinafter referred to as “shift angle”) formed by a vector indicating the learning target posture and a vector defined by the object information 131a. Then, the passage determination unit 138 determines that the posture is correct when the absolute value of the deviation angle is smaller than a predetermined threshold, and the posture is not correct when the absolute value of the deviation angle is equal to or larger than the predetermined threshold. Is determined.

  Further, when the passage determination unit 138 determines that the expression (1) is satisfied, the passage determination unit 138 compares the acceleration of the learning target at that time with the acceleration of the object 250 defined in the object information 131a.

  Specifically, for example, the passage determination unit 138 calculates the difference between the acceleration to be learned and the acceleration defined by the object information 131a. If the absolute value of the calculation result is smaller than a predetermined threshold, the acceleration is appropriate. If the absolute value of the calculation result is equal to or greater than a predetermined threshold, it is determined that the acceleration is not appropriate.

  Further, based on the acceleration when passing through the position of a certain object 250, the ratio of acceleration when passing through the position of each other object 250 is calculated, and it is determined whether or not the acceleration is appropriate based on the magnitude of the ratio. May be. In this case, it is possible to perform an operation of tracing the target motion by the slow motion while capturing the relative acceleration change.

  Note that the physical quantity to be compared between the actual operation and the target operation is not limited to the above. However, the physical quantity to be compared needs to be defined in advance in the object information 131a as one element constituting the target action.

  Next, the operation of the learning support apparatus 10 having the configuration shown in FIG. 2 will be described in detail.

  FIG. 8 is a flowchart showing an operation flow of the learning support apparatus 10.

  First, in step S <b> 1100, the learning target acquisition unit 132 acquires learning target information indicating a learning target desired by the user, and outputs the acquired learning target information to the target action creation unit 135. Here, since “club” is selected, learning target information for the club head 160 as a learning target is output.

  In step S1200, the target motion creation unit 135 acquires object information 131a corresponding to the part indicated by the learning target information from the target motion information storage unit 131 as target motion information. Here, since the club head 160 is a learning target, the object information 131a in which “club” is set in the part name 201 shown in FIG. 3 is acquired. In addition, the target action creation unit 135 outputs the acquired object information 131a to the passage determination unit 138.

  In step S1300, the target motion creation unit 135 has acquired the position coordinates 206 described in the acquired object information 131a by the position and orientation information and view information acquisition unit 133 acquired by the position and orientation information acquisition unit 134. The view information is converted into the HMD coordinate system 242. Then, the target motion creating unit 135 performs drawing target calculation for selecting the object 250 to be rendered, and then creates image data (hereinafter simply referred to as “target motion”) for visualizing and expressing the target motion. Then, it is output to the target motion drawing unit 136 as drawing information.

  For example, at the start of learning, the drawing target is the object 250 with the object numbers A1, A2, and A3, and the position coordinates represented by the HMD coordinate system 242 in the position coordinates P1, P2, and P3 shown in FIG. It is assumed that P-2 and P-3. In this case, the target action creating unit 135 draws the object image 232 of the object 250 with the object number A1 at the position coordinate P-1 with the size S1, and the object image 232 of the object 250 with the object number A2 is drawn at the position P−. The drawing information for drawing the object image 232 of the object 250 with the object number A3 at the size S3 is created at the position P-3 with the size S3.

  The details of the above-described drawing object calculation are as follows. First, the target action creating unit 135 determines the order of the objects 250 to be traced by the user from the object number 202 of the object information 131a, the immediately preceding object number 203, and the immediately following object number 204. Then, the target action creating unit 135 determines whether or not the intersection of the straight line connecting the viewpoint 243 of the user 150 and the position of each object 250 and the drawing surface 230 of the image display unit 113 belongs to the inside of the drawing surface 230. Judgment is made by calculation according to the tracing order. Then, the target action creating unit 135 temporarily stops the calculation when an object whose intersection does not belong to the drawing surface 230 appears, and selects the object 250 whose intersection belongs to the drawing surface 230 as a drawing target. Then, the target action drawing unit 136 is requested to draw the drawing target.

  In step S1400, the target action drawing unit 136 draws the target action on the drawing surface 230 of the head mounted display 110 as shown in FIG. 5 based on the drawing instruction of the target action creating unit 135. Specifically, the target action drawing unit 136 transmits the drawing information input from the target action creating unit 135 to the image display unit 113 of the head mounted display 110 by wireless communication.

  In this manner, the object image 232 is displayed at a corresponding position on the drawing surface 230 along the target trajectory to be learned by the processing of steps S1100 to S1400 in FIG. While viewing the object image 232, the user 150 exercises to make the actual learning target (here, the club head 160) follow the target motion.

  FIG. 9 is a diagram illustrating a state in which the learning target is moved so as to trace the object image 232. Here, a case where the learning target is a hand and the object 250 is symbolized by a combination of a sphere, a disk, and other three-dimensional figures is illustrated.

  As shown in FIG. 9, object images 232-1 to 232-3 are displayed along the target trajectory of the hand that is the learning target. The user 150 exercises so that the hand of his / her hand 281 traces the object images 232-1 to 232-3. For example, the plane of the disc indicates the orientation of the palm, and the triangular pyramid or cylinder indicates acceleration or deceleration and its size. By notifying the user 150 of the meanings of these symbols in advance, the user 150 exercises while confirming not only the target trajectory but also physical quantities related to the target exercise such as the orientation of the palm and the force. be able to.

  FIG. 10 is a diagram illustrating a specific example of the object 250.

  As shown in FIG. 10A, the object 250 is a symbol composed of, for example, a face surface 282 and a subsequent object direction instruction unit 283. The face surface 282 is a plane having a predetermined size, and expresses the position and posture of the head surface. The subsequent object direction indicating unit 283 is a cone having the face surface 282 as a base, the magnitude of acceleration of the head surface is determined by the height of the cone, and the moving direction of the head surface is determined by the direction from the center of the face surface 282 to the vertex. Express. When a certain object 250-1 and the next object 250-2 exist, as shown in FIG. 10A, the subsequent object direction instruction unit 283 of the object 250-1 determines the position of the next object 250-2. Indicates the direction.

  Further, as shown in FIG. 10A, a similar cone 284 in which the cone of the subsequent object direction indicating unit 283 is reduced is illustrated, and the thickness 285 of the portion sandwiched between the subsequent object direction indicating unit 283 and the cone 284 is illustrated. Thus, the force applied to the club head 160 may be expressed.

  By using such an object 250, not only the position of the head surface but also the adjustment of the posture and the force to be applied can be sensed to the user 150.

  In the golf swing, there is no scene of moving the part from the front of the user 150 to the back, so the symbol shown in FIG. However, when there is an action of moving the hand from the front to the back like a martial arts punch, it may be difficult to grasp the shape of the subsequent object direction instruction unit 283.

  Therefore, in such a case, as shown in FIG. 10B, a hole 286 may be provided in the subsequent object direction instructing section 283 by cutting out the apex portion.

  By using such an object 250, the direction and acceleration of the next object 250 can be sensibly determined by the position and size of the hole 286 regardless of the posture of the object 250 with respect to the user 150. I can tell you.

  In this way, various symbols can be used as objects depending on the learning scene and the content to be learned. In addition, by using various attributes such as size, resolution, color, transparency, brightness, density, gradation, shape, etc., it is possible to express more types of physical quantities and to express the physical quantities in detail. It becomes.

  In step S1500 of FIG. 8, the exercise information acquisition unit 137 starts acquiring exercise information of the user who has started the swing operation (tracing operation). In addition, the passage determination unit 138 starts computation for passage determination. Specifically, the exercise information acquisition unit 137 receives acceleration data sent from the exercise information measurement unit 121 every unit time and outputs the acceleration data to the passage determination unit 138. The passage determination unit 138 sequentially measures the actual operation to be learned from the acceleration data. Then, the passage determining unit 138 selects one of the objects 250 to be drawn from the objects 250 defined in the object information 131a according to the order, and the second and subsequent actual motions are measured. The deviation d of the actual trajectory 270 to be learned with respect to the selected object 250 is obtained by the procedure described in FIG.

  In step S1600, the passage determination unit 138 determines the passage of the target trajectory to be learned by comparing the obtained deviation amount d with a predetermined threshold using the method described in FIG. Specifically, the passage determination unit 138 determines whether or not the learning target has passed successfully, whether or not the learning target posture is correct, and whether or not the learning target acceleration is appropriate. Here, it is determined whether or not the club head 160 has passed through the correct position, whether or not the orientation of the head surface at the time of passing is correct, and whether or not the force applied to the club head 160 is appropriate.

  In step S1700, the passage determination unit 138 outputs each determination result as determination result information to the target motion creation unit 135, and instructs the target motion creation unit 135 to accumulate the determination result.

  In step S 1800, the target action creating unit 135 requests the target action drawing unit 136 to change the attribute of the selected object image 232 according to the content of the determination result information, and causes the object image 232 to be displayed on the drawing surface 230. Redraw. Specifically, for example, when the learning target succeeds in passing, the target action creating unit 135 replaces the drawing information that has already been passed to the target action drawing unit 136 with the drawing information in which the attribute of the corresponding object 250 is changed, An image based on the new drawing information is drawn on the drawing surface 230.

  The target motion creating unit 135 emphasizes the object image 232 to be traced next to the user by highlighting it by attaching a marker, changing the size, or blinking in accordance with the current position of the learning target. You may make it notify. Thereby, the movement of the learning target can be led, and the user can easily grasp the target trajectory intuitively.

  Further, as the user moves, the position and posture of the user's head change, and the actual field of view that can be seen through the drawing surface 230 of the head mounted display 110 also changes. Therefore, in consideration of this, the above-described drawing object calculation may be performed again in step S1800, and the drawing object may be selected again.

  In step S1900, the target action creating unit 135 determines whether the processing has been completed for all the objects 250 to be drawn. If the unprocessed object 250 remains (S1900: NO), the target action creating unit 135 returns to step S1500, reselects the next drawing target object 250, and repeats the subsequent processing. . The processing in steps S1500 to S1900 may be executed at predetermined time intervals, or at a timing when the position of the viewpoint 243 of the user 150 or the positional displacement of the head mounted display 110 becomes larger than a predetermined threshold. You may do it.

  FIG. 11 is a diagram illustrating an example of how the symbol of the object image 232 changes due to redrawing of the target action. Here, only the object images 232-1 to 232-3 and the club head 160 among the images that are in the field of view of the user 150 are illustrated.

  Here, the target action creating unit 135 executes processing for replacing the object image 232 of the object 250 that has passed successfully with an explosion image that disappears after a certain period of time as the attribute change of the object image 232 in step S1800. And

  As shown in FIG. 11, when the passage of the object 250 indicated by the object image 232-1 is successful at time t1, an explosion image 290 is drawn instead of the object image 232-1 at time t2. If the object 250 indicated by the object image 232-2 is successfully passed at time t3, an explosion image 290 is drawn instead of the object image 232-2 at time t4.

  On the other hand, when the passage of the object 250 indicated by the object image 232-3 fails at time t5, the explosion image 290 is not drawn and the object image 232-3 remains as it is. As a result, the user 150 can intuitively confirm in real time while exercising whether or not the learning target has passed through the object image 232 arranged on the target trajectory.

  Assuming that the club head 160 passes the position corresponding to the object image 232-1 at time t1, the object image 232-1 is reached before time t2 when it reaches the position corresponding to the next object image 232-2. It is desirable to notify the user 150 of the success or failure of the passage.

  In other words, when the vicinity of the position corresponding to the object images 232-1 and 232-2 in which the club head 160 moves back and forth at the times t1 and t2, the position of the club head 160 that is the basis of redrawing is measured. When the time until redrawing is completed (the time required for the processing of steps S1500 to S1800 in FIG. 8) is Δt, it is desirable that the following expression (3) is satisfied.

  From this, the target action creating unit 135 may select the drawing target so as to satisfy the expression (3). Specifically, the target motion creating unit 135 thins out the object 250 defined in the object information 131a based on the moving speed of the club head 160 or based on the speed level selected by the user 150, for example, immediately in advance. Priority is given to the object that can be notified of the result of the above.

  FIG. 12 is a diagram illustrating an example of a state after the symbol of the object image 232 is changed by redrawing the target action.

  Here, it is assumed that the learning target acquisition unit 132 handles an object having a high point importance 205 as a point object and an object having a low point importance 205 as a general object. A point object is an object that changes its shape based on a passage determination result indicating whether or not it has been traced correctly. A general object is an object whose shape is constant regardless of the determination result.

  In addition, as a change in the attribute of the object image 232 in step S1800, the learning target acquisition unit 132 executes a process of increasing the size of the object image 232 when the object 250 that has failed to pass is a point object. To do.

  As shown in FIG. 3, the point importance 205 of the objects 250 having the object numbers A1 and A4 is high, and the point importance 205 of the object 250 having the object numbers A2 and A3 is low.

  Therefore, when the passage fails in all of the objects 250 corresponding to the object numbers A1 to A4, as in the target action 291-1 shown in FIG. 12A, the point object, that is, the object whose point importance degree 205 is large. Only the object image 232 corresponding to the numbers A1 and A4 is changed to a large size. As described above, by mixing the point object and the general object, the user 150 can intuitively recognize the point to be conscious of.

  Further, for example, a dotted line object image 232 may be arranged as an image of the object 250 that connects the objects 250 adjacent in order as in the target action 291-2 illustrated in FIG. The object 250 connecting the objects 250 can also be handled as a kind of general object.

  It should be noted that in the point importance 205 of the object information 131a, the point importance of more stages such as large, medium, and small is described, the importance “large” is the point object, the importance “medium” is the general object, and the importance “small”. May be a general object that is displayed by a dotted line.

  Further, in order to cause the user 150 to perform a smoother movement, more general objects may be arranged between the point objects. Further, in accordance with the calculation capability of the target motion creation unit 135 and the rendering capability of the target motion drawing unit 136, the general object may be omitted in order to reduce the processing load on these device units.

  Also, it goes without saying that the display method of the object image 232 shown in FIGS. 12A and 12B may be applied to the drawing of the initial target motion, not after the redrawing.

  If the target motion creation unit 135 repeats the processing of steps S1500 to S1900 in FIG. 8 and completes the processing for all the objects 250 to be drawn (S1900: YES), the process proceeds to step S2000.

  In step S2000, the learning support apparatus 10 displays the target action and the actual action in a superimposed manner, and ends the series of processes. Specifically, the passage determination unit 138 outputs the displacement from the initial position of the learning target with the passage of time to the target motion creating unit 135 as the actual motion of the learning target. The target action creating unit 135 creates drawing information for displaying the target action superimposed on the actual action to be learned, and outputs the drawing information to the target action drawing unit 136.

  FIG. 13 is a diagram illustrating a state in which the target motion is displayed superimposed on the actual motion to be learned.

  As shown in FIG. 13, on the drawing surface 230 of the head mounted display 110, an object image 232 indicating the comparison result between the actual motion and the target motion is displayed so as to overlap the actual trajectory 292 to be learned. Here, an example in which gradation is applied to the object image 232 so that the color becomes darker in the shift direction is illustrated. By displaying such a comparison result, the user 150 immediately recognizes the deviation of the actual motion with respect to the target motion in the overall flow of motion, and efficiently improves the overall flow of own motion. can do.

  As described above, according to the present embodiment, the learning support device 10 gives a predetermined physical quantity, such as a posture or acceleration of a learning target, related to a user's target motion to the user. Present at the corresponding position on the orbit. Also, such target motion is presented by drawing at a position that does not overlap the target trajectory in three dimensions. Thereby, for example, the movement of the learning target in a target exercise such as exercise of an advanced user can be traced, and learning can be performed while experiencing the target exercise. Therefore, it is possible to prompt the user to learn the target exercise reliably.

  In other words, as compared to the case where the user himself / herself spatially imagines the target motion in his / her head and learns the exercise by imitating the appearance from text and videos, the improvement of the exercise in a short time is achieved. Can do. In particular, for a user who has little knowledge in the field to be learned and a beginner or beginner who is not good at image creation, the target action can be recognized easily and accurately, so that the learning efficiency can be significantly improved.

  In addition, the trajectory and predetermined physical quantity of the learning target in the actual motion of the user are measured, and compared with the trajectory and predetermined physical quantity of the learning target in the target motion, the comparison result is fed back. Thereby, it is possible to obtain effective information for the user to bring his / her exercise closer to the target exercise, and the learning efficiency can be further improved.

  In addition, since it is possible to experience the trajectory of the learning target in other people's movements, it is possible to provide learning support rich in entertainment.

  It should be noted that various variations other than the above contents can be applied to the display method of the target action. What method should be applied may be determined according to, for example, the calculation capability of the target motion creation unit 135 and the rendering capability of the target motion drawing unit 136. Alternatively, a plurality of methods may be selected by the target action creating unit 135, and the method to be applied may be switched according to the learning level of the user 150 and the content to be checked.

  In addition, object information is created and stored from the measured trajectory of the user's movement and predetermined physical quantities at each position, and the contents of this object information are used as basic information of the target movement. Also good. As a result, the user can provide his / her exercise as a target exercise to another person, or can acquire any other person's exercise as a target exercise, and can also provide learning support rich in communicability. it can. In other words, it is possible to make a wide range of contributions to the user beyond simple exercise learning support.

(Embodiment 2)
In the second embodiment, a case will be described in which the attribute of an object is changed in accordance with the result of passage determination as a learning target and the user's exercise proficiency.

  FIG. 14 is a block diagram showing a detailed configuration of the learning support apparatus according to the second embodiment of the present invention, and corresponds to FIG. 2 of the first embodiment. The same parts as those in FIG.

  As illustrated in FIG. 14, the learning support device 30 includes a control terminal 330 instead of the control terminal 130 in FIG. 2.

  The control terminal 330 acquires the target motion to be learned and displays the target motion at a corresponding position on the head mounted display 110. In addition, the control terminal 330 measures the actual motion to be learned, estimates the proficiency level of the user 150 from the comparison result between the target motion and the actual motion, and changes the display of the target motion according to the proficiency level.

  The control terminal 330 includes a target action creation unit 335 instead of the target action creation unit 135 of FIG. 2, and further includes a proficiency level estimation unit 339.

  The target action creating unit 335 creates an object drawing plan, which is a creation plan for creating drawing information, based on the description content of the acquired object information 131a, and outputs the created drawing information to the target action drawing unit 136. . The object drawing plan has the same configuration as the object information 131a shown in FIG. 3, but the position coordinates of the object 250 are defined by the coordinate information of the HMD coordinate system 242.

  The target action creating unit 335 outputs the passage determination information output from the passage determination unit 138 to the proficiency level estimation unit 339, and changes the object drawing plan in accordance with an instruction from the proficiency level estimation unit 339. Then, the target action creation unit 335 creates new drawing information in which the display of the target action is changed based on the changed object drawing plan.

  FIG. 15 is a diagram illustrating an example of the configuration of the passage determination information input to the proficiency level estimation unit 339.

  As shown in FIG. 15, the passage determination information 410 has a hierarchical structure. The passage determination information 410 includes one or more part unit results 411. Each part unit result 411 includes a part name 412 and one or a plurality of object unit results 413. Each object unit result 413 includes an object number 414, a failure number 415, and one or more deviation explanations 416. Each deviation description 416 includes an item name 417, a deviation content 418, and a deviation difference 419.

  The part unit result 411 indicates a comparison result between the target action and the actual action for each part.

  The part name 412 corresponds to the part name 201 in the object information 131a shown in FIG. 3 of the first embodiment, and indicates a learning target. The object unit result 413 indicates a comparison result between the target action and the actual action for each object. The object number 414 corresponds to the object number 202 in the object information 131a, and indicates the object number of an object in which some deviation has occurred between the target action and the actual action. The number of failures 415 indicates the number of items having a difference between the target action and the actual action. The deviation explanation 416 indicates the contents of deviation for each item, and is described continuously by the number described in the number of failures 415.

  In the deviation explanation 416, the item name 417 indicates an item to be checked, which is an item having a deviation between the target action and the actual action. The deviation content 418 indicates the specific content of the deviation. The deviation difference 419 indicates the magnitude of the deviation. The unit of values described in the deviation content 418 and the deviation difference 419 is determined in advance for each type of the item name 417.

  The proficiency level estimation unit 339 in FIG. 14 accumulates the determination result indicated by the input passage determination information 410, estimates the proficiency level of the user 150 from the accumulated determination result, and creates a target action according to the proficiency level The unit 335 is instructed to change the display of the object image. Specifically, the proficiency level estimation unit 339 holds in advance correction policy information that defines a policy for correcting the display of the target action according to the proficiency level.

  FIG. 16 is a diagram illustrating an example of the content of the modified policy information.

  As shown in FIG. 16, the modified policy information 420 includes determination result contents 421 and change request contents 422. In the determination result contents 421, contents that can be described in the deviation explanation 416 of the passage determination information 410 are described. The change request content 422 is a display of the target motion to be requested to the target motion creation unit 335 when the content described in the deviation description 416 of the passage determination information 410 is applied to the content described in the determination result content 421. The contents of the change are described.

  For example, a change request content 422 of “color: darker inner color and larger size: larger inward” is described with respect to the determination result content 421 of “deviation content: inside”. This is because when the deviation content 418 of the passage determination information 410 describes that the actual trajectory is shifted to the inside of the target trajectory, the inner color is darkened and the size of the object 250 is increased. Indicates that a change that expands inward should be given. Thereby, the user 150 can intuitively grasp the direction in which the actual trajectory should be corrected from the gradation of the object.

  In addition, a change request content 421 “modify the trajectory to the actual trajectory side” is described with respect to the determination result content 421 “deviation content: inside or outside and deviation difference: greater than a predetermined threshold”. This is because in the passage determination information 410, the deviation content 418 describes that the actual activation has deviated inside or outside the target method trajectory, and the value described in the deviation difference 419 is greater than a predetermined threshold value. Is larger, it indicates that the object 250 should be changed to move the target trajectory toward the actual trajectory. Thereby, the user 150 can perform learning using a target trajectory that can be more easily reached, and can learn the final target motion step by step.

  The proficiency level estimation unit 339 illustrated in FIG. 14 executes a proficiency level estimation process that estimates the proficiency level of the user 150 and appropriately requests display change of the object image 232 every time measurement is completed for a series of actual operations. .

  In the proficiency level estimation process, when the proficiency level estimation unit 339 estimates that the proficiency level is low, the proficiency level estimation unit 339 creates change request information describing the specific content of the display change of the object image 232 according to the modified policy information 420. The created change request information is output to the target action creating unit 335.

  The modification policy information 420 can describe various change request contents such as an increase in the number of objects 250 to be rendered and a change in the attributes of the object 250 such as changes in luminance and gradation.

  FIG. 17 is a diagram illustrating an example of a configuration of change request information created by the proficiency level estimation unit 339.

  As shown in FIG. 17, the change request information 430 has a hierarchical structure. The change request information 430 includes one or a plurality of part unit change requests 431. Each part unit change request 431 includes a part name 432 and one or object unit change request 433. Each object unit change request 433 includes an object number 434, a change request number 435, and one or more change descriptions 436. Each change description 436 includes an item name 437 and change contents 438.

  Each part unit change request information 431 indicates a change request content of a target action for each part.

  The part name 432 corresponds to the part name 201 in the object information 131a shown in FIG. 3 of the first embodiment, and indicates a change target. The object unit change request 433 indicates the change contents for each object. The object number 434 corresponds to the object number 202 in the object information 131a, and indicates the object number of the object to be changed. The number of change requests 435 indicates the number of items for which a change is requested. The change description 436 indicates the content of requesting a change for each item, and is described continuously by the number described in the change request number 435.

  In the change description 436, an item name 437 indicates an item for which a change is requested. The change content 438 indicates the specific content of the change. The unit of the value described in the change content 438 is determined in advance for each type of the item name 437.

  Hereinafter, the proficiency level estimation process executed by the proficiency level estimation unit 339 will be described in detail.

  FIG. 18 is a flowchart showing a flow of proficiency level estimation processing executed by the proficiency level estimation unit 339. The proficiency level estimation unit 339 executes the proficiency level estimation process described below, for example, immediately before step S2000 in FIG. 8 of the first embodiment.

  First, in step S1910, the proficiency level estimation unit 339 inputs the passage determination information 410 illustrated in FIG. 15 from the passage determination unit 138 via the target action creation unit 335.

  FIG. 19 is a diagram showing a specific example of the contents of the passage determination information 410, and corresponds to FIG.

  As shown in FIG. 19, the passage determination information 410a is associated with a part name 412a “club”, an object number 414a “Am”, a failure number 415a “2”, and an item name 417a-1 “position”. The deviation contents 418a-1 and "a", the deviation difference 419a-1 "a", and the "slow" deviation contents 418a-2 and "b" associated with the item name 417a-2 "acceleration" A shift difference 419a-2 is described. This indicates that the club head 160 is a learning target, and in the object number Am, there was a failure that the position was shifted inward by a and a failure that the acceleration was slow by b.

  Then, in step S1920 shown in FIG. 18, the proficiency level estimation unit 339 accumulates the determination result indicated by the input passage determination information 410 within the range of the maximum usable memory capacity. Alternatively, the proficiency level estimation unit 339 accumulates determination results corresponding to a predetermined number of passage determination information 410 with priority given to new information.

  In step S1930, the proficiency level estimation unit 339 calculates the probability that the learning target actual motion has passed the learning target goal motion (hereinafter referred to as “passing probability”) from the accumulated determination result.

  Here, the passage of the target motion is not only the success of the learning target, but also the target motion for each element that constitutes the target motion, such as that the posture of the learning target is correct and the acceleration of the learning target is appropriate. Including a match for.

  The passing probability is an index for determining whether the user has traced correctly. As the pass probability, the ratio of the number of passes to the number of judgment results for the past several times may be adopted, and not only the previous trial result but also the randomness such as the frequency and tendency of failure in the past several times The analysis results may be adopted.

  In step S1940, the proficiency level estimation unit 339 determines whether or not the calculated passing probability is greater than or equal to a predetermined threshold value for each object 250. Specifically, for example, if the proficiency level estimation unit 339 sets a threshold value as a condition that the determination result for the past 10 times has failed 5 times or more and has failed 3 times or more continuously, for each object 250 Then, the maximum value of the number of failures and the number of consecutive failures is calculated as a passage probability from the determination results for the past 10 times, and the calculated result is compared with a threshold value. The proficiency level estimation unit 339 determines that the object 150 is a point that the user 150 is not good at for the object 250 having a passage probability equal to or higher than a predetermined threshold (S1940: YES), and proceeds to step S1950. On the other hand, the proficiency level estimation unit 339 determines that the object 250 whose pass probability is less than the predetermined threshold (S1940: NO) is not a point that the user 150 is not good at, and proceeds to step S1950. First, the process directly returns to the process of step S2000 in FIG.

  In step S1950, the proficiency level estimation unit 339 requests the target motion creation unit 335 to change the object drawing plan, and returns to the process of step S2000 in FIG. Specifically, the proficiency level estimation unit 339 refers to the correction policy information 420 illustrated in FIG. 16 and acquires the correction content to be applied to the object 250 determined to be a point that the user 150 is not good at. Then, change request information 430 shown in FIG. 17 is created from the correction content acquired for each object 250, and the created change request information 430 is output to the target action creation unit 335.

  FIG. 20 is a diagram showing a specific example of the contents of the change request information 430 created from the passage determination information 410a shown in FIG.

  It is assumed that the deviation difference a described in the passage determination information 410a is small. In this case, in the modified policy information 420 shown in FIG. 16, the determination result content 421 that matches the content of the passage determination information 410a is “deviation content: inside”, and the corresponding change request content 422 is “color: inside”. The color is dark and the size is “inward”.

  Accordingly, as shown in FIG. 20, the change request information 430a includes a part name 432a “club”, an object number 434a “Am”, a change request number 435a “2”, and an item name 437a-1 “color”. A change content 438a-1 “Inner color is darkened” associated with “” and a change content 438a-2 “Enlarge inside” associated with the item name 437a-2 “size” are described. This requests that the object 250 with the object number Am of the club head 160 be given a change that darkens the inner color and enlarges the size inward.

  The target action creation unit 335 that has received the change request information 430 updates the contents of the object drawing plan according to the contents. For example, when the passage determination information 410a shown in FIG. 19 is received, the target action creation unit 335 darkens the inner color of the object 250 with the object number Am of the club head 160 in the current object drawing plan. , Give the change to expand the size inward. Then, drawing information is created based on the updated object drawing plan, and the created drawing information is output to the target action drawing unit 136.

  As a result of the proficiency level estimation process described above, in step S2000 of FIG. 8, the target action whose attribute has changed is displayed according to the proficiency level of the user 150 and the content of the deviation of the actual action from the target action.

  Thus, according to the present embodiment, the user's proficiency level is estimated from the accumulated trial results of the user, and the display of the target action is changed according to the proficiency level. Thereby, learning of the point which a user is not good at can be reinforced.

  It is easy for users to get tired of practicing by repeating the same things or not doing things. Therefore, the repetition with the change in consideration of the proficiency level can increase the user's willingness to practice and is effective in improving the exercise. In addition, the user can clearly distinguish and recognize the weak part and the other part only by looking at the drawn target action. As a result, not only can the user have high awareness of the problem, but also the third party who gives advice to the user can easily grasp the characteristics of the user's operation. Furthermore, for example, in the practice after a long time has passed since the previous practice, the technology acquired in the previous practice is an ambiguous memory for the user, but by using the learning support device 30 of the present embodiment. Since the unskilled point can be recognized more accurately, the reproducibility of the technology acquired by the user can be improved.

(Embodiment 3)
In the third embodiment, a case will be described in which the motions of a plurality of parts are displayed together with the relevance of the plurality of parts.

  In many cases, the target motion is linked in a balanced manner. In other words, if the positional relationship between the parts, the timing to start moving, or the like is out of order, it becomes difficult to realize the target operation. Therefore, in the learning support apparatus according to the present embodiment, not only the operation of the learning object but also the positional relationship between the plurality of learning objects and the timing of starting movement (hereinafter simply referred to as “relevance”) are displayed.

  FIG. 21 is a block diagram showing a detailed configuration of the learning support apparatus according to the third embodiment of the present invention, and corresponds to FIG. 2 of the first embodiment. The same parts as those in FIG.

  As illustrated in FIG. 21, the learning support device 50 includes a control terminal 530 instead of the control terminal 130 illustrated in FIG. 2. The control terminal 530 includes a target action creating unit 535 including a relevance determining unit 539 and a pass determining unit 538 instead of the target action creating unit 135 and the passage determining unit 138 shown in FIG. In addition, the target motion information storage unit 131 in the control terminal 530 according to the present embodiment describes object related information that describes which object defined in each object information 131a is related to which timing for a plurality of timings in the target motion. 131b is further stored.

  FIG. 22 is a diagram illustrating an example of the contents of the object related information 131b.

  As shown in FIG. 22, the object related information 131b is associated with each of the part names 441 “hand, foot, club, waist,...” “T1, t2, t3,. An object number is described for each of the plurality of timings 442. The part name 441 corresponds to the part name 201 of the object information 131a shown in FIG. The number of timings 442 corresponds to the number of objects defined in the object information 131a. The described object and number correspond to the object number 202 of the object information 131a.

  A plurality of object numbers described corresponding to the same timing 442 indicates that the operations of the parts indicated by the corresponding objects 250 are linked. For example, as shown in FIG. 22, in correspondence with the timing “t1”, an object number “A1” of a part name 441 “club” and an object number “C1” of a part name 441 “hand” Is described. This indicates that the movement of the club head 160 indicated by the object with the object number A1 and the movement of the hand indicated by the object with the object number C1 are linked movements realized at the same timing in the target movement.

  Such a set of objects linked at the same timing or a set of object numbers of linked objects is hereinafter referred to as a “linked object set”. Further, for example, when each object of the object numbers A1, B2, and C1 is a linked object set, it is appropriately expressed as (A1, B1, C1).

  The control terminal 530 in FIG. 21 acquires the target motion to be learned, and displays the target motion at a corresponding position on the head mounted display 110. In addition, when a plurality of parts are selected as learning targets, the control terminal 530 displays the operation of each learning target and their relevance.

  The target action creating unit 535 creates drawing information in accordance with the description content of the acquired object information 131a, and outputs the created drawing information to the target action drawing unit 136. In addition, when a plurality of parts are selected as learning targets, the target action creation unit 535 uses the relevance determination unit 539 to determine the relevance of the plurality of learning targets. Specifically, the relevance determination unit 539 executes relevance determination processing that determines relevance of a plurality of learning targets based on the object related information 131b of the target motion information storage unit 131 and outputs a linked object set. To do. Then, the target action creation unit 535 creates drawing information to which a new object (hereinafter referred to as “relevance object”) indicating the determined relevance is added.

  Further, when a plurality of parts are selected as learning targets, the target action creating unit 535 outputs a plurality of corresponding object information 131a and a linked object set to the passage determining unit 538.

  The passage determination unit 538 compares and analyzes the actual motion acquired via the motion information acquisition unit 137 and the target motion of the learning target defined in the object information 131a, creates passage determination information, and the generated passage determination Information is output to the target action creating unit 135. In addition, when the passage determination unit 538 receives a plurality of object information 131a and a linked object set from the target motion creation unit 135, the passage determination unit 538 considers the relationship between the plurality of learning target motions, Perform comparative analysis.

  Hereinafter, the relevance determination process executed by the relevance determination unit 539 will be described in detail.

  FIG. 23 is a flowchart showing the flow of relevance determination processing executed by the relevance determination unit 539. For example, in step S1100 of FIG. 8 of the first embodiment, when two parts “hand” and “club” are selected as learning targets in the control panel 210 shown in FIG. Two object information 131a to be acquired is acquired. As described above, when a plurality of pieces of object information 131a are obtained by the target action creation unit 535, the relevance determination unit 539 executes relevance determination processing described below immediately after step S1200 in FIG. 8, for example. To do.

  First, in step S1210, the relevance determination unit 539 selects one from the plurality of object information 131a, and extracts all the object numbers 202 from the selected object information 131a.

  In step S1220, the relevancy determination unit 539 refers to the object related information 131b and extracts object numbers of other parts associated with the same timing 442 for each object number 202 extracted in step S1210. To do. Then, the relevancy determination unit 539 sets the object number 202 and the object numbers of other parts extracted corresponding to the object number 202 to the linked object set.

  In step S1230, the relevance determination unit 539 outputs the interlocking object set set in step S1220 to the target action creation unit 535, and returns to the process of step S1300 in FIG.

  In step S1300 in FIG. 8, the target action creation unit 535 outputs the plurality of object information 131a acquired in step S1100 in FIG. 8 and the interlocked object set output from the relationship determination unit 539 to the passage determination unit 538. To do. The processing of the passage determination unit 538 will be described later.

  In addition, the target action creating unit 535 performs a drawing target calculation for each of a plurality of objects constituting the linked object set. In addition, among the linked object sets output from the relevance determination unit 539, the object for which all of the objects are to be drawn is determined as the drawing target for the relevance object.

  For example, at a certain timing, the object numbers of the objects 250 belonging to the drawing surface 230 are object numbers A2 and A3 for the part name 441 “club”, and the object numbers C1 and C2 for the part name 441 “hand”. , C3. In this case, the linked object sets (A2, C2) and (A3, C3) are all set as drawing targets, but the linked object sets (A1, C1) are not set as drawing targets. Therefore, the target action creating unit 535 determines the linked object sets (A2, C2) and (A3, C3) as the drawing target of the related object.

  Then, the target action creating unit 535 creates drawing information for drawing the object determined as the drawing target and the related object, and outputs the drawing information to the target action drawing unit 136.

  FIG. 24 is a diagram illustrating an example of the contents of drawing target data, which is a data portion indicating a drawing target, of the drawing information generated by the target action generation unit 535. Here, the case where the line which connects an object is used as a related object is illustrated.

  As shown in FIG. 24, the drawing target data 450 describes the object number, position coordinates, size, and color of the drawing target object. Information described about the object corresponds to the description content of the original object information 131a. In addition, the drawing target data 450 describes information related to related objects.

  Here, a case where the object numbers of the drawing target object 250 are A2, A3, C1, C2, and C3 is illustrated. In this case, as described above, the association object sets (A2, C2) and (A3, C3) are targets for drawing the related objects. Therefore, the drawing target data 450 describes that a line connecting the linked object sets (A2, C2) and (A3, C3) is drawn as the related object.

  Note that the target action creation unit 535 creates drawing target data 450 in which symbols differ for each type of object, such as assigning different colors for each learning target, instead of following the description content of the object information 131a as it is. Also good.

  FIG. 25 is a diagram illustrating an example of a state of the drawing surface 230 when drawing is performed based on the drawing target data 450 illustrated in FIG. 24.

  As shown in FIG. 25, the object image 232 corresponding to the object numbers A2, A3, C1, C2, and C3 is drawn on the drawing surface 230. On the drawing surface 230, lines 460-2 and 460-3 connecting the linked object sets (A2, C2) and (A3, C3) are drawn as images of related objects.

  By practicing while looking at the drawing surface 230 displaying such an image of the related object, the user can learn the operation of each part while confirming the relation between the plurality of parts.

  Note that an image of a related object may be drawn even for a linked object set in which only some objects 250 exist in the drawing surface 230. In this case, after setting the linked object set for all objects, the objects to be rendered are selected and only the part located inside the drawing surface is extracted from the set linked object image. To be drawn.

  FIG. 26 is a diagram illustrating an example of a state of the drawing surface 230 when an image of a related object is drawn even for a linked object set in which only some objects 250 exist in the drawing surface 230. This corresponds to FIG.

  As shown in FIG. 26, not only the lines 460-2 and 460-3 but also the line 460-1 that connects the linked object set (A 1, C 1) where one object 250 is located outside the drawing surface 230 is also outside. The drawing is performed in a direction toward the direction in which the positioned object 250 exists.

  Next, operations after the passage determination will be described.

  The passage determination unit 538 performs the same passage determination as that in each embodiment 1 in step S1600 of FIG. In addition, when the passage determination unit 538 receives a plurality of pieces of object information 131a and a linked object set from the target action creating unit 135, the passage determining unit 538 sets the part that has passed the position of the object 250 earliest for each linked object set. Then, time difference information in which the time difference from the part that passed through is recorded is created. Then, the passage determining unit 538 outputs the created time difference information to the target action creating unit 635.

  Upon receiving the time difference information, the target motion creation unit 635 additionally draws an arrow with each object pair that has the earliest passing object 250 and the other object 250 as the end point for each linked object set. Create Then, the target action creating unit 635 replaces the drawing information already passed to the target action drawing unit 136 with the drawing information with the added arrow, and causes the drawing surface 230 to draw an image based on the new drawing information.

  The target action creating unit 635 may create drawing information that expresses the magnitude of the time difference with the thickness of the arrow. As a result, the comparison result between the actual motion and the target motion including the relationship between the motions of the plurality of learning targets is displayed on the drawing surface 230.

  FIG. 27 is a diagram illustrating an example of the state of the drawing surface 230 when the comparison result between the actual motion and the target motion including the relationship among the plurality of learning target motions is drawn, and corresponds to FIG. It is.

  Here, the case where the actual trajectory passage times for object numbers A2 and A3 are delayed and the delay of object number A2 is large is illustrated. In this case, as shown in FIG. 27, a thick arrow 461-2 is drawn from the object image 232 with the object number C2 toward the object image 232 with the object number A2, and the object with the object number A3 from the object image 232 with the object number C3. A thin arrow 461-3 is drawn toward the image 232. This indicates that “the hand has moved before the club and the difference has been reduced”. The user 150 intuitively sees the comparison result including the relationship between the motions of the plurality of learning objects, and intuitively finds the difference between the balance of interlocking of the plurality of parts in the target motion and the balance of interlocking of the plurality of parts in the actual motion. Can be understood.

  As described above, according to the present embodiment, since the relevance of a plurality of parts in the target motion is displayed, it is possible to promote an intuitive understanding of the deviation between the parts during the exercise of the user, and the balance between the plurality of parts. Effective learning can be realized. Therefore, beginners and beginners can efficiently learn how to use the entire body. Moreover, the expert can also identify the cause when the improvement of the technology is stagnant due to the imbalance of the interlocking of the plurality of parts, and can further improve the technology.

  In addition, at the time of passage determination, a learning target is set as a reference for comparison between the actual action and the target action, and for a learning target other than the learning target set as the reference, which point should be compared with which object This determination may be made with reference to the learning target set as the reference.

  Specifically, for example, the passage determination unit sets a priority in advance for the learning target, and determines the highest priority among the learning targets corresponding to the plurality of received object information. Set to. Then, the passage determination unit performs a comparative analysis similar to that of the first embodiment for the learning target object information set as the reference, while the other object information is the actual learning target set as the reference. The actual motion and the target motion may be compared between each measurement time of the motion and each timing corresponding to the object that is the comparison target for each actual motion.

  Of course, the interlocking object set may be composed of three or more objects. In addition, the object itself is formed in a line or bar shape, a plurality of points are set for one object, the relationship between a plurality of parts is expressed by a single object, and each point is compared with the actual operation. You may do it.

(Embodiment 4)
In the fourth embodiment, a case where the amount of information drawn on the drawing surface is optimized according to the situation will be described.

  For example, for the user 150 whose speed of tracing the target action has increased, it may feel that the time until the object image 232 or the comparison result is displayed is delayed. In such a case, by reducing the amount of information drawn on the drawing surface 230, the processing load can be reduced, and the time until the object image 232 and the comparison result are displayed can be shortened. Therefore, the learning support apparatus according to the present embodiment optimizes the amount of information to be drawn on the drawing surface by excluding a part of the object 250 defined in the object information 131a from the drawing target.

  FIG. 28 is a block diagram showing a detailed configuration of the learning support apparatus according to the fourth embodiment of the present invention, and corresponds to FIG. 21 of the third embodiment. The same parts as those in FIG. 21 are denoted by the same reference numerals, and description thereof will be omitted.

  As illustrated in FIG. 28, the learning support device 60 includes a control terminal 630 instead of the control terminal 530 in FIG. 21. The control terminal 630 includes a target motion creation unit 635 and a passage determination unit 638 having a relevance determination unit 539 instead of the target motion creation unit 535 and the passage determination unit 538 in FIG. A control unit 639 is included.

  The target action creating unit 635 creates an object placement plan that defines what object is placed at which position in accordance with the description content of the acquired object information 131a, and sends the created object placement plan to the information amount control unit 639. Output. When the target action creating unit 635 receives drawing permission from the information amount control unit 639, the target action creating unit 635 creates drawing information as it is according to the created object arrangement plan, and sends the created drawing information to the target action drawing unit 136. Output. On the other hand, when receiving an object arrangement plan correction command for instructing correction of the object arrangement plan from the information amount control unit 639, the target action creation unit 635 corrects the contents of the object arrangement plan and corrects the object arrangement plan after correction. Create drawing information based on. In addition, the target action creation unit 635 outputs the received object placement plan correction command to the passage determination unit 638.

  FIG. 29 is a diagram illustrating an example of the contents of the object arrangement plan.

  As shown in FIG. 29, the object arrangement plan 470 includes a part name 471, an object number 472, a point importance 473, a position coordinate 474, a size 475, other information 476, and a timing 477. The part name 471, the object number 472, the point importance 473, the position coordinates 474, the size 475, and the other information 476 are respectively the part name 201, the object number 202, and the point importance 205 of the object information 131a shown in FIG. , Position coordinates 206, size 207, and other information 209, respectively. However, the coordinate information of the HMD coordinate system 242 is described in the position coordinate 474 of the object arrangement plan 470. The timing 477 corresponds to the timing 442 of the object related information 131b shown in FIG.

  Here, an example is shown in which three parts of “hand”, “foot”, and “club” are selected as learning objects, and drawing object selection at a certain timing is completed. For convenience of explanation, information that is not subject to an object arrangement plan correction instruction to be described later is underlined.

  The information amount control unit 639 stores in advance a control instruction that defines a policy for reducing the information amount. When the information amount control unit 639 receives the object placement plan from the target action creation unit 635, the information amount control unit 639 executes an information amount optimization process for appropriately creating an object placement plan correction command and optimizing the information amount.

  Specifically, the information amount control unit 639 calculates a value indicating the information amount of information presented to the user (hereinafter simply referred to as “information amount”). Then, the information amount control unit 639 outputs a drawing permission to the target action creating unit 635 when the calculated information amount is equal to or less than a predetermined threshold. On the other hand, when the calculated information amount exceeds a predetermined threshold, the information amount control unit 639 creates an object arrangement plan correction instruction according to the control instruction, and sends the created object arrangement plan correction instruction to the target action creation unit 635. Output. Details of the information amount optimization processing will be described later.

  FIG. 30 is a diagram illustrating an example of the content of a control instruction stored in the information amount control unit 639.

  As shown in FIG. 30, the control instruction 480 describes a modification policy to be applied to the object arrangement plan 470 in order to reduce the amount of information. Here, for example, “1: Delete general object / thinning out when there are multiple point objects” is described. This indicates a policy that, in principle, general objects are deleted, and if there are a plurality of general objects between point objects in the order indicated by the object number 472, some of the plurality of general objects are deleted. Further, “2: Delete peripheral object” is described. This indicates a policy of deleting an object located in the periphery when the drawing surface 230 is divided into the vicinity of the center and the periphery thereof.

  The passage determination unit 638 shown in FIG. 28 creates a passage determination information by comparing and analyzing the actual motion acquired via the motion information acquisition unit 137 and the target motion of the learning target defined in the object information 131a. The passed determination information is output to the target action creating unit 135. Further, when the passage determination unit 638 and the plurality of object information 131a are received from the target motion creation unit 135, the association determination unit 538 is used to acquire a linked object set, and the relationship between the plurality of learning target motions is obtained. Compare and analyze the actual action and the target action. When the object placement plan correction command is received from the target motion creation unit 635, the objects 250 that are the target of the comparison analysis between the actual motion and the target motion are reduced according to the contents of the object placement plan correction command.

  Hereinafter, the information amount optimization process executed by the information amount control unit 639 will be described in detail.

  FIG. 31 is a flowchart showing a flow of information amount optimization processing executed by the information amount control unit 639. For example, when the information amount control unit 639 receives the object arrangement plan 470 from the target action creation unit 635 immediately after step S1300 in FIG. 8, the information amount control unit 639 executes an information amount optimization process described below.

  First, in step S1310, the information amount control unit 639 calculates an information amount when drawing information is generated as it is from the received object arrangement plan 470. However, when an object arrangement plan command has already been created by the processing described later, the information amount control unit 639 calculates an information amount when drawing information is created according to the created object arrangement plan correction command.

  Various definitions and calculation methods can be adopted as the definition method and calculation method of the information amount. For example, the amount of information may be the total number of object images 232 in the field of view of the user 150, the area ratio of the object image 232 to the drawing surface 230 that can be considered to substantially match the field of view, or the density at which the object images 232 are accumulated. That's fine. Further, considering the human characteristic that there is a tendency to gaze at an image arranged near the center of the field of view, the area ratio and density are calculated by performing higher weighting near the center of the drawing surface 230, and the calculation result May be adopted as the amount of information.

  Here, it is assumed that the area of the object image 232 for each importance is common for all learning targets, the score indicating the display area of the general object 250 is “1”, and the score indicating the display area of the point object 250 is “3”. It is assumed that the score indicating the display area of the related object is “2”. In addition, the weighting coefficient when it is located in a predetermined range near the center on the drawing surface 230 is “3”, and the weighting coefficient when it is located in another range is “1”. The information amount control unit 639 calculates the amount of information by modeling the state of the drawing surface 230 based on the setting and weighting of points for such an object.

  Specifically, when the number of points of the kth object is Pk and the weighting coefficient corresponding to the position is Wk, the number of objects including related objects specified in the object arrangement plan 470 is m. The amount control unit 639 calculates the information amount Q using, for example, the following equation (4).

  In step S1320, the information amount control unit 639 determines whether the information amount calculated in step S1330 exceeds a predetermined threshold. It is desirable that a plurality of predetermined threshold values are prepared according to the level of the user 150. In this case, the information amount control unit 639 may receive a level input via the control panel 210 and use a threshold corresponding to the input level. If the calculated information amount exceeds the predetermined threshold (S1320: YES), the information amount control unit 639 proceeds to step S1330.

  In step S1330, the information amount control unit 639 creates an object arrangement plan modification instruction according to the control instruction 480 shown in FIG. 30, and if the object arrangement plan modification instruction has already been created, the object to be deleted is created. Add information that specifies. Then, the information amount control unit 639 returns to step S1310 and calculates the information amount for the object arrangement plan correction instruction. The processes in steps S1310 to S1330 are repeated until the information amount becomes equal to or less than a predetermined threshold value. As a result, the object arrangement plan correction instruction finally has the content of controlling the information amount to be equal to or less than a predetermined threshold.

  When the calculated information amount is equal to or smaller than the predetermined threshold value, or when the information amount is equal to or smaller than the predetermined threshold value in the original object arrangement plan 470 (S1320: NO), the process proceeds to step S1340.

  In step S1340, when the object arrangement plan correction instruction is generated, the information amount control unit 639 outputs the object arrangement plan correction instruction to the target action generation unit 635, and when the object arrangement plan correction instruction is not generated. The drawing permission is output to the target action creating unit 635. And it returns to the process of step S1400 of FIG.

  FIG. 32 is a diagram showing an example of the contents of the object arrangement plan correction instruction.

  As shown in FIG. 32, the object arrangement plan modification instruction 471 has the same configuration as the object arrangement plan 470 shown in FIG.

  The target action creation unit 635 reflects the contents described in the object placement plan modification instruction 471 in the object placement plan 470, and creates drawing information based on the reflected object placement plan 470. Thereby, the amount of information drawn on the drawing surface 230 is reduced.

  FIG. 33 is a diagram illustrating an example of a state of the drawing surface 230 when drawing is performed based on an object arrangement plan 470.

  As shown in FIG. 33, here, an image 232a of a point object indicated by a large ellipse, an image 232b of a general object indicated by a small circle, and a line 460 indicated by a straight line connecting the object images 232 as related objects Is drawn. In addition, seven point object images 232a, four general object images 232b, and five lines 460 are displayed in the central area 472 that performs high weighting, and one point is displayed in the periphery of the central area 472. An object image 232a is displayed.

  The amount of information calculated from the object arrangement plan 470 that is the source of the drawing surface 230 is calculated as 3 × (3 × 7 + 1 × 4 + 2 × 5) + 1 × (3 × 1) = 108. When the threshold value used in step S1320 of FIG. 31 is 80, for example, the calculated information amount exceeds the threshold value, the information amount control unit 639 creates an object arrangement plan correction instruction, and the information amount on the drawing surface 230 is reduced. Is done.

  FIG. 34 is a diagram illustrating an example of the state of the drawing surface 230 when the object arrangement plan 470 that is the basis of the drawing surface 230 illustrated in FIG. 33 is modified by an instruction from the information amount control unit 639.

  Compared to the drawing surface 230 shown in FIG. 33, in the drawing surface 230 shown in FIG. 34, an image 232 a of one point object around the central area 472, an image 232 b of three general objects, and two lines 460 are included. It has been reduced and the screen is cleaner.

  The amount of information calculated from the object arrangement plan 470 that is the source of the drawing surface 230 is calculated as 3 × (3 × 6 + 1 × 1 + 2 × 3) = 75. That is, it can be seen that a screen that is easier to view is displayed when the calculated amount of information is equal to or less than the threshold.

  Thus, according to the present embodiment, the information amount control unit 639 modifies the object arrangement plan according to the control instruction 480 and reduces the amount of information drawn on the drawing surface. Thereby, the processing load can be reduced, and the time until the object image 232 and the comparison result are displayed can be shortened. Further, since the amount of information is reduced by reducing information with low importance, depending on the level of the user, necessary information can be easily captured.

  In addition, when a solid figure is used as an object or when an object is subjected to dynamic changes such as rotation, expansion, and reduction for the purpose of emphasis, higher visual effects such as flicker prevention can be obtained. . For example, if learning is performed for parts of more detailed units such as fingers and joints, the number of objects displayed may increase and information may be excessive. It is possible to achieve both learning unit subdivision.

  When learning multiple parts at the same time, or when each object has a meaning such as showing a change in physical quantity, not just a passing point, the reaction time until the user searches for and recognizes the target to be traced Tend to be longer. It is undesirable for motor learning that the reaction time is too long and the smoothness of the movement is disturbed. In general, it is known that the faster the eye movement, the lower the visual acuity and the narrower the visual field. In particular, when the head is frequently moved in order to check a plurality of parts, the eye movement also becomes high speed, and accurate recognition of displayed information becomes difficult. In this respect, according to the present embodiment, it is possible to limit the amount of information that affects the reaction time and the accuracy of information recognition, and to provide higher-quality motor learning support.

(Embodiment 5)
In the fifth embodiment, a case will be described in which a target motion portion located in a user's blind spot area, such as an area outside the drawing surface, is rearranged in the user's field of view.

  Since the field of view of the user 150 and the size of the drawing surface 230 are limited, even if the user 150 bends, warps, or twists during exercise, the user 150 cannot view the area. Exists. Hereinafter, such a region is defined as a “blind spot”. Here, the blind spot includes a region that cannot be seen without sacrificing the form of motion and a region that does not enter the field of view in the desired line-of-sight direction (field of view) in the ideal form. Specifically, for example, a club head 160 at the top position in golf or a kicking foot in soccer corresponds to an exercise operation located in a blind spot.

  It is desirable to draw information on the target motion part (hereinafter referred to as “dead zone motion”) located in such a blind spot on the drawing surface 230 in order to sensuously learn the flow and direction of the motion. However, such a blind spot movement is drawn at a position different from the position where the target movement of the learning target should be originally displayed. If drawing is performed as it is, the user 150 may be confused as to whether or not it is the original position. Invite.

  Therefore, the learning support device 70 according to the present embodiment displays the blind spot motion in a pseudo manner in the user's field of view, and also displays information indicating that the image is related to the blind spot motion in the blind spot motion image. .

  FIG. 35 is a block diagram showing a detailed configuration of the learning support apparatus according to the fifth embodiment of the present invention, and corresponds to FIG. 2 of the first embodiment. The same parts as those in FIG.

  As shown in FIG. 35, the learning support apparatus 70 has a control terminal 730 instead of the control terminal 130 of FIG. The control terminal 730 includes a target motion creation unit 735 that stores a blind spot drawing instruction 739 in advance, instead of the target motion creation unit 135 of FIG. In addition, the target motion information storage unit 131 in the control terminal 730 of this embodiment stores object information 131c including blind spot information indicating which part of the target motion is a blind spot motion.

  FIG. 36 is a diagram illustrating an example of the contents of the object information 131c.

  As shown in FIG. 36, the object information 131c has the same configuration as the object information 131a shown in FIG. 3 of the first embodiment, and further, blind spot information indicating whether or not it corresponds to each position related to the blind spot. 481. In the blind spot information 481, information such as “a blind spot start point”, “a blind spot start point”, “a blind spot point”, and “a blind spot end point” are described in association with the object number 202.

  Here, the point immediately before the start of the blind spot is the position of the object 250 immediately before the target action enters the blind spot. The blind spot start point is the position of the first object 250 that enters the blind spot. The blind spot is the position of the object 250 in the blind spot. The blind spot end point is the position of the first object coming out of the blind spot. The target motion part from the blind spot start point to the blind spot end point, that is, the trajectory of the learning target in the blind spot motion is referred to as a “dead spot trajectory”.

  Here, as shown in FIG. 36, the object 250 with the object number A3 is the object 250 immediately before the target motion enters the blind spot, and the object 250 with the object number A4 is the first object 250 that enters the field of view. Further, the objects 250 having the object numbers A5 to A7 are the objects 250 in the blind spot, and the object 250 having the object number A8 is the first object 250 exiting from the blind spot.

  The target action creating unit 735 in FIG. 35 creates an object drawing plan having the same configuration as the object arrangement plan 470 shown in FIG. 29 of the fourth embodiment, for example, according to the description content of the object information 131c. This is output to the target motion drawing unit 136. At this time, the target motion creation unit 735 draws the blind spot motion on the drawing surface 230 based on the blind spot information 481 of the object information 131c in accordance with the blind spot rendering instruction 739, and various information indicating that it is a blind spot motion image. Create drawing information to draw. Here, it is assumed that the target action creating unit 735 creates drawing information so that each object 250 is displayed in order according to the order defined in the object information 131c. In addition, the target motion creation unit 735 sequentially erases the object image 232 of the object 250 that has passed the actual trajectory from the drawing surface 230 based on the passage determination information input from the passage determination unit 138.

  FIG. 37 is a diagram showing an example of the contents of the blind spot drawing instruction sheet 739.

  In the blind spot drawing instruction 739, a condition 482 relating to drawing of blind spot movement and drawing of various information indicating that the image is a blind spot action and an instruction content 483 relating to drawing are defined.

  Here, predetermined blind spot start information and blind spot end information are used as information indicating that the image is a blind spot motion image. The blind spot start information is information indicating in which area the blind spot start point and the blind spot object image 232 are to be drawn. The blind spot end information is information indicating in which area the blind spot end point is drawn.

  Here, the blind spot drawing instruction 739 includes, for example, “from the timing at which the real trajectory passes through the object two points before the blind spot start point to the timing at which the real trajectory passes through the object two points after the blind spot start point”. In response to the condition 482, an instruction content 483 of “draw blind spot start information at a predetermined position” is described. Further, “predetermined from the timing at which the real trajectory passes through the object immediately before the blind spot start point to the timing at which the real trajectory passes through the object one point before the blind spot end point”, “predetermined The instruction content 483 of “drawing a blind spot motion and an arrow indicating the order of tracing the blind spot motion” is described at the position “. Further, in response to the condition 482 “from the timing at which the real trajectory passes through the object three points before the blind spot end point to the timing at which the real trajectory passes through the object at the blind spot end point”, An instruction content 483 “draw end information” is described. In addition, for example, “the actual trajectory within the predetermined distance from the position of the object immediately after the dead angle end point from the timing when the actual trajectory entered within the predetermined distance from the immediately preceding start of the blind spot. Corresponding to the condition “until the timing when“ is entered ”, an instruction content“ draw a blind spot motion at a predetermined position ”may be described.

  When the blind spot motion is displayed as a plurality of object images 232, the target motion creation unit 735 associates the relative positions of the plurality of object images 232 with the relative positions in the original target motion. Further, the target motion creation unit 735 sets the predetermined position for drawing the blind spot start information as the vicinity of the object image 232 at the blind spot start point, and sets the predetermined position for drawing the blind spot end information as the vicinity of the object image 232 at the blind spot end point. To do.

  FIG. 38 is a diagram showing an example of the state of the drawing surface 230 when drawing is performed based on the blind spot drawing instruction sheet 739 shown in FIG. 37 from the object information 131c shown in FIG.

  FIG. 38A shows a state of the drawing surface 230 at the timing when the actual trajectory passes through the object 250 with the object number A1. In this case, the blind spot drawing instruction 739 satisfies the condition 482 “from the timing when the real trajectory passes through the object two points before the blind spot start point to the timing when the real trajectory passes two objects after the blind spot start point”. Applicable. Therefore, as shown in FIG. 38A, the blind spot start information 484 is drawn in the vicinity of the blind spot start point. Here, as the blind spot start information 484, an arrow and a character string “dead spot start” are drawn.

  FIG. 38B shows a state of the drawing surface 230 at the timing when the actual trajectory passes through the object 250 with the object number A2. In this case, the condition 482 “from the timing when the real trajectory has passed through the object immediately before the blind spot start point to the timing when the real trajectory has passed through the object immediately before the blind spot end point” of the blind spot drawing instruction 739. It corresponds to. Therefore, as shown in FIG. 38B, the blind spot start information 484, the object image 232 of the blind spot motion, and the arrow 485 indicating the order of tracing the blind spot motion are drawn.

  In the blind spot operation, since the object image 232 is arranged at a position different from the original position in the target operation, it is difficult to imagine the original position. Therefore, displaying the arrow 485 described above can help the user 150 imagine the original trajectory.

  FIG. 38C shows the state of the drawing surface 230 at the timing when the actual trajectory passes through the object 250 with the object number A6. In this case, the condition 482 in the blind spot drawing instruction 739 is “from the timing when the real trajectory passes through the object three objects before the blind spot end point to the timing when the real trajectory passes through the object at the blind spot end point”. Therefore, the blind spot end information 486 is drawn in the vicinity of the blind spot end point as shown in FIG. Here, as the blind spot end information 486, an arrow and a character string “dead spot end” are drawn.

  FIG. 38D shows a state of the drawing surface 230 at the timing when the actual trajectory passes through the object 250 with the object number A7. Also in this case, the condition 482 in the blind spot drawing instruction 739 is “from the timing when the real trajectory passes through the object three points before the blind spot end point to the timing when the real trajectory passes through the object at the blind spot end point”. The blind spot end information 486 is still drawn.

  As described above, according to the present embodiment, the image of the blind spot motion is displayed on the drawing surface together with the information indicating that the image is related to the blind spot motion. Accordingly, the movement of the trajectory to be learned and the predetermined physical quantity in the blind spot motion can be presented to the user separately from the original trajectory position. Therefore, the user can learn the target action without making an effort to move the body more than necessary and bring the target action into view. As a result, it is possible to more accurately perform training for the invisible portion that conventionally had to rely on intuition.

  In each of the embodiments, a case has been described in which target operations and comparison results are displayed in a pseudo manner using a head-mounted display, but the present invention is not limited to this. 3D (dimension) holograms and other 3D image technologies, as well as 3D image drawing technology that draws 3D images by continuously generating bright spots that have been converted into plasma by laser light in the space, are used to display target actions and comparison results. Also good. When such a technique is used, there are fewer restrictions on the drawing target, and it is possible to display more parts of the target motion.

  In each of the embodiments, the case where the present invention is applied to learning of golf club swing has been described. However, the present invention is applied to learning of various types of movement that can define or measure the movement of a part of a body or an exercise equipment. Of course you can. For example, the present invention can be applied not only to various sports such as tennis, martial arts, and baseball but also to learning of exercises in various fields such as musical instrument performance and rehabilitation. Furthermore, the present invention may be applied not to learning purposes but to entertainment purposes such as games for tracing other people and their own exercises.

  The learning support device and the learning support method according to the present invention are useful for effective learning of exercise in various fields such as various sports, musical instrument performances, and rehabilitation.

Explanatory drawing which shows the mode of the scene where the learning assistance apparatus which concerns on Embodiment 1 of this invention is used. FIG. 1 is a block diagram showing a detailed configuration of a learning support apparatus according to Embodiment 1 Explanatory drawing which shows an example of the content of the object information in Embodiment 1 FIG. 3 is a plan view showing an example of the appearance of a control panel in the first embodiment FIG. 3 is a plan view illustrating an example of a drawing surface according to the first embodiment. Explanatory drawing which shows the principle of calculation of the relationship of each coordinate system and drawing position of an object in Embodiment 1 Explanatory drawing which shows an example of the principle of the passage determination in Embodiment 1. The flowchart which shows the flow of operation | movement of the learning assistance apparatus which concerns on Embodiment 1. FIG. Explanatory drawing which shows a mode that the learning object in Embodiment 1 is operated. Explanatory drawing which shows the specific example of the object in Embodiment 1. Explanatory drawing which shows an example of a mode that the symbol of the object image in Embodiment 1 changes Explanatory drawing which shows an example of the mode after the symbol of the object image in Embodiment 1 changed Explanatory drawing which shows the state which overlapped and displayed the target operation | movement on the real operation | movement of learning object in Embodiment 1 The block diagram which shows the detailed structure of the learning assistance apparatus which concerns on Embodiment 2 of this invention. Explanatory drawing which shows an example of a structure of the passage determination information in Embodiment 2. Explanatory drawing which shows an example of the content of the correction policy information in Embodiment 2 Explanatory drawing which shows an example of a structure of the change request information in Embodiment 2. The flowchart which shows the flow of the proficiency level estimation process in Embodiment 2 Explanatory drawing which shows the specific example of the content of the passage determination information in Embodiment 2. Explanatory drawing which shows the specific example of the content of the change request information produced from the passage determination information shown in FIG. 19 in Embodiment 2. The block diagram which shows the detailed structure of the learning assistance apparatus which concerns on Embodiment 3 of this invention. Explanatory drawing which shows an example of the content of the object relevant information in Embodiment 3 The flowchart which shows the flow of the relevance determination process in Embodiment 3. Explanatory drawing which shows an example of the content of the drawing object data in Embodiment 3 The top view which shows an example of the mode of the drawing surface when drawing is performed based on the drawing object data shown in FIG. 23 in Embodiment 3. FIG. 9 is a plan view illustrating an example of a drawing surface when only some objects exist in the drawing surface in the third embodiment The top view which shows an example of the mode of the drawing surface when the relationship of the operation | movement of the some learning object in Embodiment 3 is drawn The block diagram which shows the detailed structure of the learning assistance apparatus which concerns on Embodiment 4 of this invention. Explanatory drawing which shows an example of the content of the object arrangement plan in Embodiment 4 Explanatory drawing which shows an example of the content of the control instruction in Embodiment 4 The flowchart which shows the flow of the information amount optimization process in Embodiment 4. Explanatory drawing which shows an example of the content of the object arrangement plan correction command in Embodiment 4 The top view which shows an example of the state of the drawing surface when drawing is performed based on the object arrangement plan in Embodiment 4 The top view which shows an example of the mode of the drawing surface when correction is performed with respect to the object arrangement plan in Embodiment 4 The block diagram which shows the detailed structure of the learning assistance apparatus which concerns on Embodiment 5 of this invention. Explanatory drawing which shows an example of the content of the object information in Embodiment 5 Explanatory drawing which shows an example of the content of the blind spot drawing instruction in Embodiment 5 The top view which shows an example of the mode of the drawing surface when drawing is performed based on the blind spot drawing instruction in Embodiment 5

Explanation of symbols

10, 30, 50, 60, 70 Learning support device 110 Head mounted display 111 Visibility information measurement unit 112 Position and orientation information measurement unit 113 Image display unit 120 Three-axis acceleration sensor 121 Motion information measurement unit 130, 330, 530, 630, 730 Control terminal 131 Target motion information storage unit 132 Learning target acquisition unit 133 Visibility information acquisition unit 134 Position and orientation information acquisition unit 135, 335, 535, 635, 735 Target motion creation unit 136 Target motion drawing unit 137 Motion information acquisition unit 138, 538 , 638 Passage determination unit 210 Control panel 339 Proficiency level estimation unit 539 Relevance determination unit 639 Information amount control unit

Claims (15)

A learning support device for learning a predetermined exercise,
A target motion acquisition unit for acquiring target motion information indicating a target motion that is a motion of a learning target part in the predetermined motion;
Based on the target motion information acquired by the target motion acquisition unit, a rendering unit that renders the target motion in conformity with the field of view of the user who learns the predetermined motion;
A learning support apparatus. The drawing unit
The target motion in the user's field of view when the user performs the predetermined motion is drawn at a position that does not overlap with the trajectory of the learning target region when the user performs the predetermined motion.
The learning support apparatus according to claim 1. The operation of the learning target part is performed with respect to the target trajectory of the learning target part and the posture, speed, acceleration, rotational speed, rotational acceleration, and the learning target part of the learning target part at a position constituting the target trajectory. A predetermined physical quantity including at least one of the force to be applied, and
The learning support apparatus according to claim 2. The drawing unit arranges and draws an object visually indicating the predetermined physical quantity at a corresponding position;
The learning support apparatus according to claim 3. When the predetermined physical quantity includes a direction component, the object has a surface whose normal is the direction or a protrusion protruding toward the direction.
The learning support apparatus according to claim 4. A measurement unit that measures an actual motion that is the motion of the learning target part in the actual motion of the user, and generates information indicating the actual motion;
A comparison unit that compares the target motion information acquired by the target motion acquisition unit with information indicating the actual motion generated by the measurement unit;
The drawing unit
The information indicating the comparison result by the comparison unit is drawn in conformity with the field of view of the user.
The learning support apparatus according to claim 1. The drawing unit
An object visually indicating the predetermined physical quantity is arranged and drawn at a corresponding position, and according to the comparison result by the comparison unit, the size, resolution, color, transparency, brightness, density, gradation, image of the object, And changing an attribute including at least one of presence / absence of drawing,
The learning support apparatus according to claim 6. The comparison unit includes:
A comparison between the target motion information and information indicating the actual motion is performed for each object,
The drawing unit
Draw information indicating the comparison result for each object before the learning target part reaches the position of the next object,
The learning support apparatus according to claim 7. The target motion information further includes importance for each object,
The drawing unit
Changing an attribute including at least one of the size, resolution, color, transparency, luminance, density, gradation, and presence / absence of drawing of the object according to the importance,
The learning support apparatus according to claim 4. A proficiency level determination unit for determining the proficiency level of the user for the predetermined exercise from the comparison result by the comparison unit;
The drawing unit
Changing an attribute including at least one of the size, resolution, color, transparency, luminance, density, gradation, and presence / absence of drawing of the image of the object according to a determination result by the proficiency level determination unit;
The learning support apparatus according to claim 6. The operation of the learning target part includes a trajectory of the first part and a position of the second part when the first part is located at a position constituting the trajectory of the first part,
The drawing unit
Draw and associate the position constituting the trajectory of the first part and the position of the second part when the first part is located at the position;
The learning support apparatus according to claim 1. An information amount control unit that calculates the amount of information drawn in the user's field of view and reduces the objects to be drawn when the calculated amount of information exceeds a predetermined threshold;
The learning support device according to claim 4 or 7, further comprising: A blind spot trajectory discriminating unit for discriminating a blind spot trajectory located in the blind spot of the user's field of view among the target trajectories;
The drawing unit
Notifying that the target motion in the blind spot trajectory is an image of the target motion in the blind spot of the user's field of view, and drawing in the user's field of view,
The learning support apparatus according to claim 3. The drawing unit
An image display unit that displays an image based on input image data;
A position and orientation acquisition unit for acquiring the position and orientation of the user's head;
A viewpoint acquisition unit that acquires the position of the user's viewpoint;
Based on the position and orientation of the user's head acquired by the position / orientation acquisition unit and the position of the user's viewpoint acquired by the viewpoint acquisition unit, the user's field of view is calculated, and the calculated user's field of view And image data for displaying an image of the target motion in the field of view of the user when the user performs the predetermined exercise based on the image display position of the image display unit and the generated image A target motion generation unit for outputting data to the drawing unit;
The learning support apparatus according to claim 1, further comprising: A learning support method for learning a predetermined exercise,
A target motion acquisition step of acquiring target motion information indicating a target motion that is a motion of a learning target part in the predetermined motion;
Based on the target motion information acquired in the target motion acquisition step, a rendering step of rendering an image of the target motion in conformity with the field of view of the user who learns the predetermined motion;
A learning support method.

Images