JP6679951B2 - Teaching Assist System and Teaching Assist Program - Google Patents

Description

  The present invention relates to a teaching assistance system and a teaching assistance program.

  With the recent advances in digital image capturing technology, it is possible to easily capture a user's enjoyment of sports as a digital moving image. Furthermore, it has been attempted not only to record the state of the user's sports as a video but also to diagnose the user's form in this sports by digital image recognition technology.

  For example, Patent Document 1 describes an invention for accurately diagnosing golf swing using a computer. The solution of the abstract of Patent Document 1 is "at least two places spaced apart in the longitudinal direction of the club shaft 11a, and the head / neck / right shoulder / left shoulder / right elbow / left elbow / left wrist of the golfer 15".・ Set reference points M1 to M17 that are color-specified in at least one location selected from the right wrist, right hip, left hip, right knee, left knee, right ankle, left ankle, right ankle, left ankle, and the golfer 15 The moving images at the time of swing are photographed by the cameras 13-1 and 13-2, the swing moving images are converted into a plurality of still images and stored in the computer 14, and the reference points M1 to M17 of the plurality of still images are stored. From the coordinate data, checkpoints in a plurality of swing postures set from the address to the impact are digitized, and a swing diagnosis is made by comparing with a pre-input ideal value. ”

JP, 2004-313479, A

  According to the invention described in Patent Document 1, it is possible to mechanically diagnose the foam and output the diagnostic result of the swing evaluation item. However, for a student (user) diagnosed by the present invention, it is not always clear how to approach the ideal form based on the diagnostic results of these evaluation items. Therefore, coaching is necessary.

  Coaches who are proficient in this sport can teach students how to approach the ideal form. However, the coach needs to practice one-on-one with the students to provide this oral instruction, which limits time and space. Further, in a sports center in which a plurality of coaches are enrolled, the coaching content may differ or may conflict with each coach. In other words, it was necessary to educate coaches in advance so that any coach could give instruction without overlooking important instruction points.

  Therefore, it is an object of the present invention to enable a coach to assist a coach assist system and a coach assist program without overlooking important coach points.

In order to achieve the above-mentioned object, the teaching assist system according to the present invention is based on a reproduction unit that reproduces a moving image of a user's form flow , and a reference operation timing determined in advance corresponding to the type of the form flow. And a guidance assisting means for extracting the operation timing to be the recommended guidance point from the moving image and causing the reproducing means to perform a pause or a throw at the extracted operation timing, the reproducing means comprising: While playing back and displaying the moving image in the display area, displaying a plurality of partial images cut out from the moving image at different timings side by side in time series in the second display area, and displaying the partial images among the partial images. for partial image corresponding to the operation timing of the instruction recommended points, displays are denoted by the identification mark, it And it features.
Further, the teaching assist program according to the present invention causes the computer to reproduce the moving image of the user's form flow on the basis of a reference operation timing that is predetermined corresponding to the reproducing unit and the type of the form flow. The operation timing which is the recommended instruction point is extracted from the above, and the reproduction means is caused to function as the instruction assisting means for executing the pause or the throw at the extracted operation timing. The moving image is reproduced and displayed in the second display area, and a plurality of partial images cut out from the moving image at different timings are displayed side by side in chronological order in the second display area, and the teaching recommended points among the plurality of partial images are displayed. and for the corresponding partial image to the operation timing of displays are denoted by the identification mark, And wherein the door.

  According to the present invention, any coach can be taught without overlooking important teaching points.

It is a block diagram which shows the outline of the guidance assistance system in this embodiment. It is a block diagram which shows the structure of a server apparatus. It is a figure which shows the ideal form example stored in the form database of a server apparatus. It is a block diagram which shows the structure of a tablet terminal of a coach and a student. FIG. 8 is a sequence diagram showing a shooting operation, a guidance assist operation, and a guidance video browsing operation. It is a flow chart which shows the normal playback processing of a photography animation. It is a flow chart which shows guidance assistance processing. It is a figure which shows a guidance assistance screen. It is a figure which shows the guidance assistance screen in an address. It is a figure which shows the guidance assistance screen just before the top. It is a figure which shows the guidance assistance screen in a top. It is a figure which shows the guidance assistance screen in impact. It is a figure which shows the example of a recording time chart. It is a figure which shows the example of a recording time chart when leaving a pause for a predetermined period of time.

Hereinafter, modes for carrying out the present invention will be described in detail with reference to the drawings.
FIG. 1 is a configuration diagram showing an outline of the teaching assistance system S in the present embodiment.
The teaching assistance system S includes, for example, a camera 4 installed in a sports school, tablet terminals 2 and 3, and a server device 1 that can communicate with the camera 4 and the tablet terminals 2 and 3 via a network N. It The server device 1 is a computer resource provided on a so-called cloud and will be described in detail with reference to FIG. 2 described later.

The tablet terminal 2 is a terminal operated by a coach at a sports school. The tablet terminal 3 is a terminal operated by a student at home or the like. These tablet terminals 2 and 3 will be described in detail later with reference to FIGS. 4 (a) and 4 (b).
When a student practices various sports such as golf, tennis, and swimming at a sports school, the camera 4 shoots a series of motion pictures of the student who receives instruction from a coach. The camera 4 is a fixed-type image pickup device installed in a practice area of a sports school, and transmits a moving image captured by the server device 1 via the network N in a stream.

  When the coach teaches the student, he or she selects a guidance point in a series of operations and reproduces the guidance content at the guidance point while playing the moving image shot on the tablet terminal 2. The guidance content input via the tablet terminal 2 is stored in the server device 1 on the cloud in synchronization with the reproduction timing at the guidance point.

  After coaching with the tablet terminal 2 by the coach, the student will self-learn the teaching contents afterwards at home or the like. At the time of self-learning, the teaching assistance system S causes the student's tablet terminal 3 to reproduce a moving image. At the timing selected as the instruction point in the series of operations in the video, the server device 1 reads out the instruction content by the coach stored in synchronization with the timing, and the instruction content is read by the student's tablet. It is reproduced on the terminal 3. This allows the students to self-learn the contents of the coach's instruction at the sports school, using the instruction points as keys.

FIG. 2 is a block diagram showing the configuration of the server device 1.
The server device 1 includes a CPU (Central Processing Unit) 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, a communication unit 14, and an HDD (Hard Disk Drive) 15.
The RAM 12 is a volatile memory and is used by the CPU 11 as a temporary storage area. The ROM 13 is a non-volatile memory, and stores, for example, a BIOS (Basic i / o system). The communication unit 14 communicates with the camera 4 and the tablet terminals 2 and 3 via the network N (see FIG. 1).

  In the HDD 15, the form information generation program 151, the captured moving image 152 taken by the camera 4, the form database 153, the form information 154, the guidance assist server program 155, the guidance information 156, the guidance moving picture encoding program 157, and the student reproduce at the time of self-study. An instruction video 158 and a video distribution program 159 are stored.

When the CPU 11 executes a camera control program (not shown), the moving image captured by the camera 4 is stored in the HDD 15 as a captured moving image 152.
The form information generation program 151 is executed by the CPU 11 to detect the image area of the student from each frame of the captured moving image 152. The image of the student detected here is stored in the HDD 15 as form information 154.
The form database 153 is a database that stores ideal form examples of various forms. An ideal form example stored in the form database 153 will be described in detail later with reference to FIG.

The guidance assist server program 155 and the moving image distribution program 159 are executed by the CPU 11 to display the guidance assist screen (see FIG. 8) on the tablet terminal 2. The CPU 11 that executes the moving image distribution program 159 and the tablet terminal 2 cooperate with each other to reproduce the captured moving image 152 on the guidance assist screen. Further, the CPU 11 which executes the instruction assisting server program 155 extracts the operation timing serving as the instruction point from the captured moving image 152, and transitions the captured moving image 152 to the pause or slow at the extracted operation timing. Further, the information input by the coach via the tablet terminal 2 is stored in the HDD 15 as guidance information 156.
The instruction video encoding program 157 is a program that is executed by the CPU 11 and encodes the instruction video 158 from the captured video 152 and the instruction information 156. The instruction video 158 encoded here is distributed to the student's tablet terminal 3 by the CPU 11 that executes the video distribution program 159.

FIG. 3 is a diagram showing an ideal form example stored in the form database 153 of the server device 1. This example form illustrates seven teaching points for golf swing.
These seven teaching points are named Address, Halfway Back, Top, Halfway Down, Impact, Follow and Finish. The coach can accurately teach the form to the students by focusing on the seven teaching points of the swing.
For each teaching point, a point to be noted is recorded in advance, as indicated by, for example, a ring. By describing this point, different coaches can be taught at the same level.

FIG. 4A is a block diagram showing the configuration of the tablet terminal 2 of the coach.
The tablet terminal 2 includes a CPU 21, a RAM 22, a ROM 23, a communication unit 24, a touch panel display 25, a microphone 26, a flash memory 27, and a speaker 28. The flash memory 27 is a non-volatile temporary storage device, and stores a teaching assistance program 271.

  The RAM 22 is a volatile memory and is used by the CPU 21 as a temporary storage area. The ROM 23 is a non-volatile memory. The communication unit 24 communicates with the server device 1 and the like via the communication network N (see FIG. 1). The touch panel display 25 is formed by stacking a touch panel on a liquid crystal panel, for example, and displays information and inputs operation information. The microphone 26 records voice and the speaker 28 outputs voice. The microphone 26 records the content of instruction by the voice of the coach.

  The guidance assist program 271 is executed by the CPU 21 and cooperates with the server device 1 to display a guidance assistance screen (see FIG. 8) on the tablet terminal 2. Accordingly, the coach can browse the captured moving image 152 (see FIG. 2), select an arbitrary playback position of the moving image as the instruction point, and input the instruction content related to the selected instruction point. The guidance content input by the coach is stored as guidance information 156 (see FIG. 2) by the server device 1.

FIG. 4B is a block diagram showing the configuration of the student's tablet terminal 3.
Like the tablet terminal 2, the tablet terminal 3 includes a CPU 31, a RAM 32, a ROM 33, a communication unit 34, a touch panel display 35, a microphone 36, a flash memory 37, and a speaker 38. The flash memory 37 is a non-volatile temporary storage device, and stores an instruction browser program 371. The guidance browser program 371 is executed by the CPU 31 and cooperates with the server device 1 to display the guidance video 158 on the tablet terminal 3 on a guidance browser screen (not shown). As a result, the student can browse the instruction content at the reproduction position selected by the coach as the instruction point of the moving image.

FIG. 5 is a sequence diagram showing a shooting operation, a guidance assisting operation, and a browsing video 158 browsing operation, which will be described with reference to FIG. 2 as appropriate.
Sequences Q10 to Q13 show shooting operations when a student practices various sports such as golf, tennis, and swimming at a sports school.
The camera 4 shoots a moving image of the shooting content including the front and rear of the swing form of the student as it is (sequence Q10), and transmits the captured moving image as a stream to the server device 1 (sequence Q11). Server device 1 stores the received moving image as captured moving image 152 (sequence Q12). Note that a plurality of swing forms may be captured in the captured moving image 152.

Sequences Q20 to Q26 show the operations performed when the coach gives instructions to the students. The coach activates the guidance assist program 271 on the tablet terminal 2 (sequence Q20), receives the distribution of the captured moving image 152 from the server device 1, and normally plays back the captured content including before and after the swing on the tablet terminal 2 ( Sequence Q21). Here, the normal reproduction is to reproduce each frame on the same time axis as that at the time of shooting. This gives the coach a complete view of the student's form.
After that, the server device 1 extracts, from the captured moving image 152, the form portion, which is a specific operation related to the swing itself, by image recognition. The server device 1 makes a frame-by-frame thumbnail of the form portion and extracts seven timings that are preliminarily teaching points in the form portion. The form portion extracted by the server device 1 is selectably displayed on the tablet terminal 2 of the coach.
When the coach selects the form portion on the tablet terminal 2, the tablet terminal 2 transmits the information on the selected form to the server device 1 (sequence Q22). When the play button 62 (see FIG. 8) is subsequently tapped, the server device 1 plays back the form portion in a very slow motion (sequence Q23), and pauses and notifies the tablet terminal 2 at the timing extracted as the instruction point. And do.
The coach selects a guidance point in the student form via the tablet terminal 2 and inputs guidance information at the guidance point. The input guidance information is transmitted from the tablet terminal 2 to the server device 1 (sequence Q24). When the coach finishes the instruction, the coach instructs the end of the instruction recording. The instruction to end the instructional recording is transmitted from the tablet terminal 2 to the server device 1.
The server device 1 stores the guidance information 156 in synchronization with the reproduction timing at the guidance point (sequence Q25) and generates the guidance moving image 158 (sequence Q26).

Sequences Q30 and Q31 show the operation when the student performs self-learning of the teaching content afterwards at the home or the like after the coach teaches the tablet terminal 2.
The student activates the guidance browser program 371 on the tablet terminal 3 (sequence Q30). The tablet terminal 3 receives and plays back the instruction video 158 from the server device 1 in a stream (sequence Q31). In this instruction video 158, the instruction content is reproduced on the tablet terminal 3 at the timing selected as an instruction point by the coach. This allows the students to self-learn the contents of the coach's instruction at the sports school, using the instruction points as keys.
Note that the instruction video 158 may be encoded and distributed in real time when the student's tablet terminal 3 requests the instruction video 158 without pre-encoding the instruction video 158. Further, the captured moving image 152 and the instruction information 156 may be transmitted to the student tablet terminal 3 without encoding the instruction moving image 158, and the instruction video may be reproduced on the student tablet terminal 3 without limitation.

FIG. 6 is a flowchart showing a normal reproduction process of the captured moving image 152. In the description of FIG. 6, FIGS. 2 and 5 will be referred to as appropriate.
The normal reproduction process of the captured moving image 152 is a process when the guidance assist program 271 is activated on the tablet terminal 2 and an instruction to reproduce the captured moving image 152 is given. This normal reproduction process is executed by the CPU 11 of the server device 1 in the sequence Q21 shown in FIG.

The CPU 11 of the server device 1 normally reproduces the captured moving image 152 on the tablet terminal 2 (step S20).
After the normal reproduction, the CPU 11 of the server device 1 extracts the form part, which is a specific operation related to the swing itself, from the captured moving image 152 by image recognition (step S21). The server device 1 makes a frame-by-frame thumbnail of the form part (step S22), and extracts seven timings which are preliminarily set as instruction points in the form part (step S23). As a result, the form portion extracted by the server device 1 is selectably displayed on the form selection screen (not shown) of the tablet terminal 2 of the coach.

FIG. 7 is a flowchart showing the teaching assistance process. In the description of FIG. 7, FIG. 4A and FIG. 8 will be referred to as appropriate.
This guidance assist processing is executed after the form portion is selected on the form selection screen (not shown) of the tablet terminal 2 of the coach and the play button 62 (see FIG. 8) is tapped.
If the frame being reproduced is not the instruction point (step S30 → No), the CPU 11 of the server device 1 reproduces this frame in slow motion very slowly (step S31). If the pause button 63 on the tablet terminal 2 is not tapped (step S32 → No), the CPU 11 advances the current frame by one (step S33) until the selected form ends (step S34 →). Yes), these processes are repeated (step S34 → No).
If the frame being reproduced is the instruction point (step S30 → Yes), the CPU 11 of the server device 1 causes the tablet terminal 2 to perform a pause (step S35) and a guidance notification (step S36) at this timing, and the instruction information. The input is waited for (step S37).
At this time, the coach determines whether or not to give the instruction at that timing, and if the instruction is necessary, taps the chalk board 52 or the chalk pallet 53 to leave the instruction information. The CPU 11 of the server device 1 reflects the guidance information (step S39) if any button other than the pause button 63 is tapped (step S38 → No), and if the pause button 63 is tapped (step S38 → Yes). ), The process proceeds to step S33 to process the next frame. This allows coaches to overlook coaching points.

FIG. 8 is a diagram showing the guidance assist screen 5.
The instruction assist screen 5 is displayed on the touch panel display 25 by the CPU 21 of the tablet terminal 2 of the coach executing the instruction assist program 271. On the teaching assistance screen 5, a series of student thumbnails 51 from the address to the finish are reduced and displayed at the upper part, and seven previously recognized teaching points are displayed in a distinguishable frame. In this way, since only the student form part is cut out and displayed as thumbnails, it is possible to arrange a larger number of student thumbnails 51 in the teaching assistance screen 5 as compared to displaying the frames in thumbnail form.
Above the student thumbnails 51, red triangular tags 511 and 512 indicating teaching points are displayed. The tags 511 and 512 indicating the teaching points are input, for example, by the coach tapping the tag setting button 71.
Below the student thumbnail 51, a moving image area 54 in which the captured moving image 152 (see FIG. 2) is reproduced is arranged. On the left side of the moving image area 54, a chalk board 52 for writing chalk and a chalk palette 53 for designating the color of chalk are arranged. The coach can input character information and graphic information as instruction contents using the chalk board 52. On the right side of the moving image area 54, a preview control 6 for controlling the reproduction position and a tag control 7 for controlling the tag are arranged.

  The preview control 6 includes an instruction recording button 61 for instructing the start / end of instruction recording, a play button 62, a pause button 63, a frame return button 64, a frame advance button 65, a fast reverse button 66, and a fast forward button 67. To be done. The tag control 7 is configured to include a tag setting button 71 for setting tags and tag moving buttons 72, 73 for moving between set tags.

  The coach checks the moving image centering on the points that are identified and displayed, and if there is a point to be instructed, the coach points are input using the tag setting button 71. The coach can also move between a plurality of teaching points by tapping the tag move buttons 72, 73. The contents of the instruction such as the voice of the coach and the writing of chalk at the time of the instruction and the gesture and gesture are stored in the server device 1 in association with the tags 511 and 512.

  Note that the red triangular tags 511 and 512 are also displayed on the student's tablet terminal 3. By selecting these, the teaching contents before and after that can be reproduced.

An example in which the coach teaches based on the captured moving image 152 of the student will be described with reference to FIGS. 9 to 12 below.
FIG. 9 is a diagram showing the guidance assist screen 5 at the address. The guidance assist screen 5 is displayed on the tablet terminal 2 after the coach operates the tablet terminal 2 to select a form in the sequence Q22 shown in FIG.
In the moving image area 54 of the teaching assistance screen 5, the frame of the student at the address is displayed. An annular ring 81 is displayed on the left hand part of this student. The ring 81 is information recorded in the form database 153 shown in FIG. The coach can easily teach the problem of the form of the student by reviewing the angle / direction of the left hand shown by the ring 81.

Among the plurality of student thumbnails 51, the student thumbnail 51 related to this frame is enlarged and displayed. Since the form associated with this address is a predetermined point, a rectangular frame indicating that point is displayed superimposed on the student thumbnail 51. In addition to the address, seven teaching points: halfway back, top, halfway down, impact, follow, and finish are shown in a rectangular frame. Furthermore, “a guidance point“ address ”has been detected” is displayed below the moving image area 54.
When the coach taps the instruction recording button 61, the tablet terminal 2 starts instruction recording and voice input, and when the coach taps the reproduction button 62, the instruction assist screen 5 shown in FIG.

FIG. 10 is a diagram showing the guidance assist screen 5 immediately before the top.
In the moving image area 54 of the teaching assistance screen 5, the frame of the student immediately before the top is displayed. An annular ring 82 is displayed on the left hand portion of this student. The ring 82 is displayed based on the information recorded in the form database 153 shown in FIG. The coach can easily teach the problem of the form of the student by reviewing the angle and direction of the left hand shown by the ring 82.
In the plurality of student thumbnails 51, the student thumbnails 51 related to this frame are enlarged and displayed.

FIG. 11 is a diagram showing the guidance assist screen 5 at the top.
In the moving image area 54 of the teaching assistance screen 5, the frame of the student at the top is displayed. An annular ring 83 is displayed on the right knee of this student. The ring 83 is information recorded in the form database 153 shown in FIG. The coach can easily teach the problem of the form of the student by reviewing the angle and direction of the right knee shown by the ring 83.

Above the moving image area 54, the student thumbnail 51 related to this frame among the plurality of student thumbnails 51 is enlarged and displayed. Further, since the form on the top is a predetermined point, a rectangular frame indicating that point is displayed so as to be superimposed on the student thumbnail 51. Furthermore, “a guidance point“ top ”has been detected” is displayed below the moving image area 54. Since this frame is a teaching point, video playback is paused and the current frame is repeatedly displayed.
For example, the coach taps the tag setting button 71 to set a tag and gives voice guidance about the angle of the knee and the like. This guidance voice is recorded by the microphone 26. After that, the coach taps the pause button 63 to restart the moving image reproduction.

FIG. 12 is a diagram showing the guidance assist screen 5 at impact.
In the moving image area 54 of the teaching assistance screen 5, the student's frame at impact is displayed.
Among the plurality of student thumbnails 51, the student thumbnail 51 related to this frame is enlarged and displayed. Further, since the form in this impact is a predetermined point, a rectangular frame indicating that point is displayed so as to be superimposed on the student thumbnail 51. Furthermore, “a guidance point“ impact ”has been detected” is displayed below the moving image area 54. Since this frame is a teaching point, video playback is paused and the current frame is repeatedly displayed.

  Here, the coach taps the chalk board 52 and draws an arrow 82 at a portion to be swung. The coach further taps the tag setting button 71 to set a tag and gives voice guidance on this swing. This voice is recorded by the microphone 26. After that, the coach taps the tag move button 72 to transit to the frame immediately before the tag is set.

FIG. 13 is a diagram showing an example of a recording time chart.
The horizontal axis of this recording time chart represents the time in the captured moving image 152, and the vertical axis represents the time in the instructional moving image 158. The position of the guidance video 158 with respect to the captured video 152 is indicated by a thick solid line, and the timing of the guidance voice is indicated by a solid arrow beside it. This recording time chart shows a state after the coach operates the tablet terminal 2 to select a form in the sequence Q22 shown in FIG.

At the time t0 of the instruction video 158 and at the 0.0th second of the video image 152, the coach taps the instruction recording button 61, and then taps the play button 62 at time t1. As a result, the server device 1 and the tablet terminal 2 perform slow reproduction of the captured moving image 152. Hereinafter, when simply described as “time”, it means the time of the instruction video 158. The guidance assist screen 5 in this case is shown in FIG.
At time t2 and the halfway back near 0.2 seconds of the captured video 152, the tablet terminal 2 pauses the slow playback of the captured video 152. The coach determines that there is no particular instruction at this instruction point, and taps the reproduction button 62 at time t3 to cause the captured moving image 152 to be slowly reproduced. The tag assigned at time t3 is displayed on the teaching assistance screen 5 as a tag 511.
At time t4 and at the top near 0.5 seconds of the captured video 152, the tablet terminal 2 pauses the playback of the captured video 152. The coach tags at time t5, draws a guidance point at time t6, and utters a guidance voice “This area is in the top position ...” from time t5 to t7. The guidance assistance screen 5 in this case is shown in FIG. The coach further taps the play button 62 at time t7 to slowly play back the captured moving image 152 and advance to the next instruction point.

At time t8 and half way down near 0.8 seconds of the captured video 152, the tablet terminal 2 pauses the slow playback of the captured video 152. The coach determines that there is no particular instruction at this instruction point, and taps the reproduction button 62 at time t9 to slowly reproduce the captured moving image 152.
At time t10 and at an impact near 1.1 seconds of the captured moving image 152, the tablet terminal 2 temporarily stops the slow playback of the captured moving image 152. The coach tags at time t11, draws a guidance point at time t12, and utters a guidance voice "from impact to follow, with such a swing ..." From time t11 to t13. The guidance assist screen 5 in this case is shown in FIG. The tag assigned at time t7 is displayed on the teaching assistance screen 5 as a tag 512. The coach further taps the tag move button 72 at time t13 to move to the tag before the current frame.

Then, the coach utters a guidance voice saying "If you look at the top form again ..." from time t13 to t14, and then taps the tag move button 73 at time t14 to move to the tag after the current frame. . After that, the coach taps the play button 62 at time t15 to restart the slow playback of the captured moving image 152.
At time t15 and half way down near 1.4 seconds of the captured moving image 152, the tablet terminal 2 pauses the slow playback of the captured moving image 152. The coach determines that there is no particular instruction at this instruction point, and taps the reproduction button 62 at time t17 to slowly reproduce the captured moving image 152.

  At time t18 and at the finish in the vicinity of 1.6 seconds of the captured video 152, the tablet terminal 2 pauses the slow playback of the captured video 152. The coach determines that there is no particular instruction at this instruction point, and taps the instruction recording button 61 at time t19 to stop the instruction recording. As a result, the contents taught by the coach are saved as the guidance information 156. The coaching information 156 allows the coach to teach a student form.

FIG. 14 is a diagram showing an example of a recording time chart when the temporary stop is left for a predetermined time.
The time chart from time t20 to time t26 is similar to that from time t0 to time t6 shown in FIG. After drawing the instruction point at time t26, the coach leaves the tablet terminal 2 without operating it.
When a predetermined time elapses at time t27, the tablet terminal 2 advances the reproduction position of the captured moving image 152 to half way down near 0.8 seconds which is the next instruction point. The coach decides that this instruction point will not be instructed and leaves it. At time t28, when a predetermined time has passed after being left unattended, the tablet terminal 2 advances the reproduction position of the captured moving image 152 to an impact around 1.1 seconds, which is the next instruction point. This allows the coach to motivate him to operate the tablet terminal 2 within a predetermined time. Further, the size of the instruction video 158 can be suppressed to reduce the load on the server device 1.

  The time chart from time t29 to time t37 is similar to that from time t11 to time t19 shown in FIG.

(Modification)
The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention. For example, the following (a) to (f) are available.
(A) In the above embodiment, the form related to golf swing has been described. However, it is not limited to this, and it is a form when scratching water with a crawl for swimming, a form related to the inclination of the shoulder line when sprinting, a form when kicking a ball with a free kick of soccer, a table tennis It may be any sports form such as a form for catching a ball with a racket, an edge form in a skiing competition, a volleyball receive form, or a baseball batting form. Further, it is not limited to the form of sports, as long as it is a specific action that has multiple teaching points in a specific action pattern that serves as a model, in addition to sports, it is possible to perform various ceremonies such as specific actions of dance and the angle of bowing in Shinto May be a specific motion, rehabilitation walking training motion, or the like.
(B) In the above embodiment, the tablet terminal 2 was temporarily stopped at each teaching point. However, the present invention is not limited to this, and the tablet terminal 2 may transition to further slow playback near each teaching point, and may transition to the original slow playback when further away from each teaching point.
(C) In the above-mentioned embodiment, one camera 4 photographs from the front of the student. However, the present invention is not limited to this, and a plurality of cameras may be used to take images at a plurality of angles such as the oblique back direction, front direction, and upward direction of the student. When coaching, show multiple playback video views side-by-side with front-view playback video views. The coaches are arranged so that the time axes of the playback video views are the same, which makes it easier for the coach to teach while clarifying the teaching points from each angle.

(D) The content of instruction from the coach in the above embodiment was recording of voice or recording on a chalk board. However, the present invention is not limited to this, and the coach's appearance when the coach teaches while playing a video on the tablet terminal 2 may be photographed as a guidance image. Further, a demonstration form by the coach's gestures and hand gestures at each instruction point, or an instruction video of a student who is directly instructed by the coach may be recorded in synchronization with the instruction point. Furthermore, the coach may directly touch the students to take a picture, and a still image or a moving image may be taken as the instruction image. When the student plays the moving image, a still image or a moving image of the moving image at each teaching point is displayed in synchronization with the playing of the moving image of the student's form. As a result, the coach can instruct the student without operating the tablet terminal 2.

(E) The coach demonstration form may be further developed to reproduce a professional moving image. For example, a professional ideal form close to a student's form is extracted by comparison with the form database 153 stored in the cloud server device 1 and stored in association with the moving image of the student. Then, when the coach gives the instruction by playing back the photographed images of the students, the professional moving image is also played back, and the instruction given by the coach is given at the instruction points automatically extracted or the tag positions arbitrarily selected. Record the contents by voice or chalkboard. By doing so, when the student plays back the moving image for learning, the image of the student and the professional image are displayed in contrast with each other, and the contents of instruction by the coach at that time are automatically played back. Therefore, the students can perform self-learning efficiently.
(F) The recording is not limited to recording by the server device 1 provided in the cloud, but may be recording by a computer provided in a gym or sports club, for example.

Hereinafter, the inventions described in the claims attached to the application of this application will be additionally described. The claim numbers listed in the appendices are as set forth in the claims initially attached to the application for this application.
[Appendix]
<Claim 1>
Playback means for playing a video of the user's form,
Instruction assisting means for extracting an operation timing serving as an instruction point from the video, and causing the reproducing means to perform a pause or a throw at the extracted operation timing,
An instruction assist system characterized by comprising:
<Claim 2>
The instruction assisting means extracts an operation timing serving as an instruction point from the moving image, when the moving image is reproduced by the reproducing means,
The teaching assistance system according to claim 1, wherein:
<Claim 3>
If the operation timing that is the instruction point is extracted from the moving image, the instruction assisting unit causes the reproducing unit to perform slow reproduction of the moving image,
The teaching assistance system according to claim 2, wherein:
<Claim 4>
The instruction assisting means causes the reproducing means to slowly reproduce the moving image again when there is no user input for a predetermined time at the operation timing.
The teaching assistance system according to claim 3, wherein:
<Claim 5>
The teaching assist means notifies the user that it is the operation timing,
The teaching assistance system according to claim 1, wherein:
<Claim 6>
The teaching assist means records the teaching content input by the user at the operation timing in association with the operation timing.
The teaching assistance system according to any one of claims 1 to 5, wherein
<Claim 7>
The instruction assisting means, when there is no user input for a predetermined time at the operation timing, causes the reproducing means to perform a pause or a throw of the operation timing next to the operation timing.
The teaching assistance system according to any one of claims 1 to 6, wherein:
<Claim 8>
Playback means for playing a video of the user's form,
Instruction assisting means for extracting an operation timing to be an instruction point from the video and executing a pause or a throw at the extracted operation timing,
Teaching assistance program to make the computer function as.

S Guidance Assist System 1 Server Device 151 Form Information Generation Program 152 Shooting Video 153 Form Database 154 Form Information 155 Guidance Assist Server Program 156 Guidance Information 157 Guidance Video Encoding Program 158 Guidance Video 159 Video Distribution Program 2 Tablet Terminal 271 Guidance Assist Program 3 Tablet Terminal 371 Guidance Browser Program 4 Camera 5 Guidance Assist Screen 51 Student Thumbnail 511, 512 Tag 52 Chalk Board 53 Chalk Palette 54 Video Area 6 Preview Control 61 Guidance Record Button 62 Play Button 63 Pause Button 7 Tag Controls 81-83 Circle ( Guidance content)
84 arrow (contents of instruction)
91-93 Guidance

Claims (7)

Playback means for playing a video of the user's form flow ,
Based on the reference operation timing that is determined in advance corresponding to the type of form flow, the operation timing that is the recommended teaching point is extracted from the video , and at the extracted operation timing, the playback means pauses or Instruction assisting means to execute throw,
Equipped with
The reproducing means is
The moving image is reproduced and displayed in the first display area, and a plurality of partial images cut out from the moving image at different timings are arranged and displayed in chronological order in the second display area, and the plurality of partial images are displayed. Of these, the partial image corresponding to the operation timing which is the recommended instruction point is displayed with an identification mark,
A teaching assistance system characterized by this. The teaching assist means extracts an operation timing serving as a teaching point from the moving image when the moving image is being reproduced by the reproducing means.
The teaching assistance system according to claim 1, wherein: The guidance assisting means sets the video playback by the playback means to slow playback during extraction of the operation timing that is the guidance recommended point from the video.
The teaching assistance system according to claim 2, wherein: The teaching assist means causes the reproducing means to slowly reproduce the moving image again when there is no user input for a predetermined time during the pause.
The teaching assistance system according to any one of claims 1 to 3, wherein: If the moving image reaches the operation timing while the moving image is being reproduced by the reproducing unit , the teaching assist unit notifies the user that the operation timing has come.
The teaching assistance system according to any one of claims 1 to 4, wherein: The teaching assist means records the teaching content input by the user at the operation timing in association with the operation timing.
The teaching assistance system according to any one of claims 1 to 5, wherein Computer,
Playback means for playing back a video of the user's form flow,
Based on the reference operation timing that is predetermined corresponding to the type of form flow, the operation timing that is the recommended teaching point is extracted from the video, and the reproduction means temporarily stops or at the extracted operation timing. Teaching assistance means to execute throw,
Function as
The reproducing means is
The moving image is reproduced and displayed in the first display area, and a plurality of partial images cut out from the moving image at different timings are arranged and displayed in chronological order in the second display area, and the plurality of partial images are displayed. Of these, the partial image corresponding to the operation timing which is the recommended instruction point is displayed with an identification mark,
A teaching assistance program characterized by this.

Images