JP5966971B2 - Information processing apparatus and program - Google Patents

Description

  The present invention relates to a technical field of outputting exercise content for supporting exercise performed by a user while listening to music.

  For example, Patent Document 1 discloses a technique for outputting auxiliary sound such as narration when outputting music and a motion video representing a motion performed in accordance with the music.

JP 2010-178772 A

  When outputting a support voice such as a narration for supporting a user who performs exercise, the display timing of the motion video may be different from the output timing of the support voice. At this time, if the support voice output timing is corrected in order to eliminate the timing shift, the rhythm of the user's movement may be disturbed. For example, it is assumed that a support voice that counts “1, 2, 3, 4” in accordance with the beat is output. Here, in order to eliminate the timing shift, for example, it is assumed that the output timing of the audio portion “3” is advanced. Then, the time interval between the output timing of the audio part “2” and the output timing of the audio part “3” becomes shorter than the time interval of beats corresponding to the tempo. Therefore, the user cannot perform an exercise operation in accordance with the rhythm.

  The present invention has been made in view of the above points, and is an information processing apparatus capable of correcting the output timing of sound so as not to disturb the rhythm of outputting sound that supports a user who performs exercise. And providing a program.

  In order to solve the above-mentioned problem, the invention according to claim 1 stores musical score information indicating a musical score of music, operational information indicating an exercise operation, and audio information indicating a voice supporting a user who performs the exercise operation. Based on the storage means, the specified tempo and the synchronization information generated in accordance with the musical score information stored in the storage means, and the motion information stored in the storage means. Based on the first control means for displaying video information indicating video on the display means, the synchronization signal, and the score information stored in the storage means, the output means outputs music information indicating the music. 2 control means, third control means for outputting the audio information stored in the storage means by the output means based on the synchronization signal, and the video information is displayed on the display means. When it is determined by the first determination means that there is a deviation exceeding a predetermined time between the timing and the timing at which the audio information is output by the output means; Second determination means for determining whether the output part of the audio information output by the output means is an audio part output in time with a beat, and the output part is converted to a beat by the second determination means. Correction means for correcting the output timing of the audio information when it is determined that the audio portion is not output together.

The invention according to claim 2 further includes third determination means for determining whether or not the sound information stored in the storage means includes a sound portion output in time with a beat, and the correction means includes If it is determined by the third determining means that the audio information does not include an audio portion that is output in time with the beat, the output timing of the audio information is not corrected, and the audio information is added to the beat by the third determining means. Only when it is determined that an audio portion that is output together is included, the second determining means outputs the audio portion that is output by the output means in the audio information in accordance with the beat. It is characterized by determining whether it is.

  The invention according to claim 3 is a timing at which the video information is displayed on the display unit when the second determination unit determines that the output part is an audio part that is output in time with a beat. And a fourth determination unit that determines whether there is a deviation exceeding a second predetermined time longer than the predetermined time with respect to a timing at which the audio information is output by the output unit. The output timing of the audio information is corrected when the determination unit determines that there is the deviation.

According to a fourth aspect of the present invention, the output portion determined to be an audio portion output in time with a beat by the second determination means is output during the first beat of any measure of the score. A fifth determining means for determining whether the output portion is a sound portion to be played, wherein the correcting means is the sound portion output by the fifth determining means during the first beat of the measure. When determined, the timing at which the video information is displayed on the display means and the timing at which the audio information is output by the output means, regardless of whether or not there is a deviation exceeding the second predetermined time. , characterized in that it does not correct the output timing of the audio information.

  According to a fifth aspect of the present invention, the storage means stores the audio information indicating the volume along the time series of the supported voice, and the time information indicated by the audio information stored in the storage means Among the output positions specified by the first specifying means for specifying a temporal output position at which the volume is maximized and the output position specified by the first specifying means, the volume is maximized at a time interval corresponding to the designated tempo. Second storage means for specifying the plurality of output positions, and second storage means for storing range information indicating temporal ranges corresponding to the plurality of output positions specified by the second specification means, Acquisition means for acquiring a temporal output position of the output portion output by the output means, wherein the second determination means is configured such that the output position acquired by the acquisition means is the second output position. Record If the range indicated by the range information stored in means, and judging the said output portion is a voice section which is output in accordance with the beat.

  According to a sixth aspect of the present invention, the storage unit stores the audio information indicating the volume along the time series of the supported audio, and the correction unit is an audio part output in time with a beat Among the different audio parts, when an audio part whose volume is less than a predetermined value is output, the output timing of the audio information is corrected preferentially.

  According to the seventh aspect of the present invention, the storage means stores the specified tempo, the musical score information indicating the musical score of the music, the operational information indicating the exercise operation, and the audio information indicating the voice supporting the user performing the exercise operation. A first control step of causing the display means to display video information indicating a video on which the exercise motion is performed based on the synchronization signal generated according to the musical score information and the motion information stored in the storage means A second control step for outputting music information indicating the music by an output means based on the synchronization signal and the musical score information stored in the storage means, and the storage means based on the synchronization signal. A third control step for outputting the audio information stored in the output means by the output means; a timing at which the video information is displayed on the display means; and the output means. A first determination step for determining whether or not there is a deviation exceeding a predetermined time from a timing at which the audio information is output; and when the first determination step determines that there is a deviation, the output of the audio information A second determination step for determining whether the output portion output by the means is an audio portion output in time with a beat, and an audio portion in which the output portion is output in time with a beat in the second determination step When it is determined that it is not, the computer is caused to execute a correction step of correcting the output timing of the audio information.

  According to the first or seventh aspect of the invention, when a voice part different from the voice part to be output in accordance with the beat is output, the output timing of the voice information is corrected. Therefore, it is possible to correct the output timing of the sound so that the rhythm of outputting the sound that supports the user who performs the exercise is not disturbed.

  According to the second aspect of the present invention, it is possible to prevent unnecessary correction from being performed when there is no problem even if the output timing of the audio information is different from the output timing of the video information. .

  According to the third aspect of the present invention, when a sound part different from the sound part to be output in accordance with the beat is output, correction is performed preferentially, while the sound part should be output in accordance with the beat. Correction may be performed even when an audio portion is being output. Therefore, the timing shift can be quickly resolved.

  According to the fourth aspect of the present invention, among the audio portions that are output in time with the beat, the audio portion that is to be output during the first beat of a measure that is particularly important for the user to take a rhythm is output. Correction is not performed. Therefore, it is possible to quickly eliminate the timing shift while preventing the rhythm from which the voice supporting the user performing the exercise is output from being disturbed.

  According to the fifth aspect of the present invention, it is possible to appropriately determine whether the sound portion is to be output in time with the beat.

  According to the sixth aspect of the present invention, it is possible to preferentially perform correction when the sound to be supported is in a silent state or a state according to silence. Therefore, it is possible to prevent the sound heard by the user from becoming unnatural due to the correction.

It is a figure showing an example of outline composition of exercise content generation system 1 of this embodiment. (A) is a figure which shows an example of the correction method of a support voice, (B) is a figure which shows the structural example of a permission level table. (A) is a graph which shows an example of the waveform of the audio signal of support voice, (B) is a graph which shows an example of the waveform of the signal which takes the absolute value of the audio signal of support voice, (C) It is a graph which shows an example of the waveform of the signal which differentiated the signal shown to (B). 10 is a flowchart illustrating a processing example of permission level table generation processing of a CPU 51 of the output terminal 5. (A) is a flowchart which shows the process example of the exercise | movement content reproduction | regeneration processing of CPU51 of the output terminal 5, (B) is a flowchart which shows the process example of the synchronous process of CPU51 of the output terminal 5. FIG. 10 is a flowchart illustrating a processing example of support voice reproduction processing of a CPU 51 of the output terminal 5.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In addition, embodiment described below is embodiment at the time of applying this invention to the exercise content production | generation system which produces | generates the exercise content for supporting exercise. The exercise content includes video and audio for supporting exercise.

[1. Configuration of Exercise Content Generation System 1]
First, the configuration of the exercise content generation system 1 according to the present embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating a schematic configuration example of an exercise content generation system 1 according to the present embodiment. As shown in FIG. 1, the exercise content generation system 1 includes a distribution server 2 and one or more output terminals 5. The distribution server 2 and the output terminal 5 can be connected via the network 10. The network 10 includes, for example, the Internet. A database 3 is connected to the distribution server 2. Information relating to exercise and information relating to music are registered in the database 3. The distribution server 2 distributes information registered in the database 3 to the output terminal 5 periodically or in response to a request from the output terminal 5.

  The output terminal 5 is a terminal device installed in the facility 4, for example. The output terminal 5 is an example of an information processing apparatus of the present invention. The output terminal 5 is used by a user 41 of the facility 4. The facility 4 may be a sports facility, for example. The user 41 receives an exercise lesson at the sports facility. The exercise lesson is a lesson composed of a plurality of exercise movements. The output terminal 5 in this case may be a personal computer, for example.

  The output terminal 5 can be connected to the monitor 57. The monitor 57 may be a display device including a plurality of speakers 64 and a display 67. In this case, the output terminal 5 can be connected to the display 67. The output terminal 5 can be connected to a speaker 64. When the output terminal 5 outputs an audio signal to the monitor 57, music or the like is output from the speaker 64. When the output terminal 5 outputs a video signal to the monitor 57, an exercise video or the like is displayed on the display 67. The motion video is a moving image that shows a figure 83 performing a motion motion. The figure 83 is a virtual object in the form of, for example, a person, an animal, a virtual creature, or a robot. The figure 83 is arranged in a three-dimensional virtual space. The output terminal 5 outputs a signal so that the music output from the speaker 64 and the movement of the figure 83 displayed on the display 67 are synchronized. Outputting music and motion video is an example of outputting motion content. The user 41 can exercise while watching the figure 83 displayed on the display 67 while listening to the music output from the speaker 64. The operator 42 can operate the output terminal 5 using the remote controller 66 or the like. The user 41 and the operator 42 may be the same person. When the facility 4 is a sports facility, the operator 42 may be an instructor, for example.

[2. Synchronization of motion video and support audio]
The exercise content output by the monitor 57 when an exercise lesson is performed includes music, exercise video, and support audio. The support voice is a voice for supporting the user 41 who exercises. For example, the support voice may include a narration that explains the movement and a voice that instructs the movement. In addition, the support voice may include, for example, a voice that is output in accordance with the beat of the music. The sound output according to the beat of the music is, for example, a sound whose volume is maximized at a time interval that is a natural number multiple of the length of one beat corresponding to the tempo of the music. For example, when the tempo is 120, the time interval that is a natural number multiple of the length of one beat is 0.5 seconds, 1 second, 1.5 seconds, or the like. Examples of the sound that is output in accordance with the beat of the music include a sound such as “Ichinisan” and a sound such as a clapping time.

  In order to reproduce the exercise content, the output terminal 5 stores a music sequencer, a 3D engine, and a support audio reproduction program. The music sequencer is a program for playing back music based on MIDI data. MIDI data is data in MIDI (Musical Instrument Digital Interface) format. MIDI data is data indicating the musical score of music. MIDI data is an example of musical score information in the present invention. The music sequencer may include a synthesizer as software, for example. The 3D engine is a program for generating a motion image obtained by projecting a figure 83 that moves in a three-dimensional virtual space onto a two-dimensional plane. The support audio reproduction program is a program for reproducing support audio based on the support audio data. The support audio data is data indicating an audio signal of support audio. Specifically, the support voice data is data indicating a time series of sample values of the volume of the support voice at predetermined time intervals, for example. The data format of the support audio data may be, for example, WAV (RIFF waveform Audio Format), AAC (Advanced Audio Coding), MP3 (MPEG Audio Layer-3), or the like. The support voice data is an example of voice information in the present invention.

  When reproducing the exercise content, the output terminal 5 needs to reproduce the exercise video, the music, and the support audio in synchronization. The output terminal 5 generates an event according to the designated tempo and MIDI data by executing the music sequencer. The event is a signal for performing synchronization. An event is an example of a synchronization signal in the present invention. The tempo is specified in MIDI data, for example, or specified in advance. In the MIDI data, a delta time and an event are described. The delta time indicates the time from when a certain event is output until the next event is output. Events include normal MIDI events, 3D control extended events, audio playback extended events, and the like. A normal MIDI event is an event for controlling the reproduction of music. Normally, MIDI events are output to the synthesizer.

  The extended event for 3D control is an event extended to control processing by the 3D engine. The extended event for 3D control includes, for example, an event for synchronizing a motion image with music. This event is called a video synchronization event. The video synchronization event is output to the 3D engine at a time interval corresponding to a specified tempo, for example. In place of the video synchronization event, a MIDI clock may be output to the 3D engine. The MIDI clock is an event output at a time interval corresponding to the tempo. By executing the 3D engine, the output terminal 5 generates a frame image constituting the motion video at a predetermined frame rate and causes the display 67 to display the frame image. The output terminal 5 synchronizes the display timing of the motion video with the output timing of the music based on the video synchronization event. Details of the synchronization method will be described later. Due to this synchronization, for example, the display of some frame images constituting the motion video may be omitted, or the playback of the motion video may be temporarily stopped. However, it is difficult for the user 41 to perceive that the frame image is omitted or the reproduction of the motion video is temporarily stopped. The output terminal 5 can synchronize the output timing of the music and the display timing of the motion video at any time.

  The extended event for audio reproduction is an event for reproducing support audio. The extended event for audio playback includes, for example, an event for specifying support audio data used for playback, an event for starting playback of support audio using the specified support audio data, and the like. The extended event for audio reproduction is output to the support audio reproduction program. The output terminal 5 executes the support sound reproduction program, and starts outputting the support sound at the timing when the sound reproduction extended event is output.

  By the audio reproduction extended event, the output terminal 5 can synchronize the timing of starting the output of the support audio with the motion video. However, after the output of the support audio is started, the output timing of the support audio and the display timing of the motion video may be shifted. For example, when the processing load on the output terminal 5 is high, the output timing of the support voice may be delayed. Further, for example, the output timing of the support voice may be advanced. Therefore, after the output of the support voice is started, it is conceivable to correct the output timing of the support voice and synchronize the output timing of the support voice and the display timing of the motion video. For example, when the output timing of the support voice is late, the output terminal 5 does not output a part of the support voice. Thereby, the output timing of the support voice can be advanced. For example, when the output timing of the support voice is early, the output terminal 5 temporarily stops the output of the support voice and then restarts the support voice output. As a result, the output timing of the support voice can be delayed. However, if such a correction is performed while a support voice that is timed in accordance with the beat is output, the rhythm in which the support voice is output is disturbed. For example, it is assumed that the correction is performed between “d” and “sun” in the support voice that counts as “first day”. Then, the time interval from the output of “d” to the output of “sun” deviates from the beat time interval corresponding to the designated tempo. Then, the user 41 cannot perform an exercise operation according to the rhythm.

  Therefore, the output terminal 5 outputs the support audio indicated by the support audio data from the speaker 64 when the display timing of the motion video and the display timing of the support audio have a time difference exceeding a predetermined threshold value. It is determined whether the audio part that is present is an audio part that is output in time with the beat. The output terminal 5 corrects the output timing of the support voice when it is determined that the output voice part is not the voice part output in time with the beat.

  A specific example will be described below. FIG. 2A illustrates an example of a support voice correction method. For example, it is assumed that a narration is output after a sound that counts as “supporting sound” is output several times as a support sound. FIGS. 2A and 2A show the progress of music over time. 2A and 2A show an example in which one measure is composed of four beats. The motion video display timing can be synchronized with the music output timing as needed. For this reason, FIGS. 2A and 2A also show the progress of the motion video. 2A and 2B show changes in the voice level of the support voice over time. The sound level indicates the volume. Each voice part included in the support voice corresponds to the playback position of the support voice. The reproduction position indicates a temporal position where the corresponding audio part should be output. This temporal position is the time that has elapsed since the reproduction of the music was started. Times T0 to T5, T11 to T15, and T25 to T29 shown in FIGS. 2A and 2B are playback positions. The time interval of each time corresponds to the length of the beat. For example, the time T1 corresponds to “1”. Therefore, “1” should be output when the time T1 has elapsed from the start of playback of the support voice.

  From time T1 to time T4, “very good” is output. Here, the output timing of the support voice matches the display timing of the motion video. After that, at the times T11 and T12, “Sanshi” should be output. However, from this point in time, it is assumed that the output timing of the support sound is delayed by a threshold value or more than the display timing of the motion video. Therefore, in practice, “Sun” is output later than the timing at which the time T11 has elapsed from the start of music playback. That is, the elapsed time from the start of music playback and the playback position of the support voice are shifted. Here, the output terminal 5 determines whether the currently output audio part is an audio part output in time with a beat. “Sanshi” is an audio part that is output in time with the beat. Therefore, the output terminal 5 does not correct the support voice. Thereafter, narration is output at times T13 to T27. Narration is not an audio part that is output in time with the beat. Therefore, the output terminal 5 corrects the support voice. Specifically, as shown in FIGS. 2 (A) and 2 (4), the playback position of the support voice is accelerated by not outputting a part of the voice corresponding to the range of the time T13 to T14.

  The output terminal 5 may correct the output timing of the support voice even if the currently output audio portion is an audio portion output in time with the beat. However, the output terminal 5 preferentially corrects when the currently output audio part is not the audio part output in time with the beat, rather than when the audio part is output in time with the beat. . Specifically, the threshold value for determining the difference between the display timing of the motion video and the display timing of the support voice is changed. That is, the threshold value for the sound part output in time with the beat is set larger than the threshold value for the sound part different from the sound part output in time with the beat. The smaller the threshold, the higher the probability that correction will be performed.

  The output terminal 5 supports the support voice when the currently outputted voice part is a voice part that is output in time with the beat and is output during the first beat of the measure. The output timing is not corrected. The reason is that the audio part output during the first beat of the measure is an especially important audio part for the user to take a rhythm. For example, in FIGS. 2A and 2B, the output terminal 5 does not perform correction when the playback position of the support voice is between times T1 and T2.

  The output terminal 5 gives the highest priority when the currently output audio part is not an audio part output in time with the beat and the currently output audio part is silent. Make corrections. For example, in FIGS. 2 (A) and 2 (2), a silent portion is output between times T25 and T26. At this time, the output terminal 5 performs correction with the highest priority. Specifically, the threshold for determining the difference between the display timing of the motion video and the display timing of the support audio is minimized. Thereby, it is possible to prevent the support voice heard by the user 41 from being unnaturally heard by the correction. Note that the output terminal 5 also considers a voice part having a voice level equal to or lower than a predetermined value to be silent.

  The output terminal 5 does not have to correct the output timing of the support voice when the support voice indicated by one support voice data does not include a voice portion that is output in time with the beat. In this case, even if the output timing of the support voice is shifted from the display timing of the motion video, the rhythm of the user 41 exercising is not disturbed.

  Below, the process for determining whether it is the audio | voice part output according to a beat is demonstrated. The output terminal 5 generates information used for determination in advance. Specifically, the output terminal 5 generates a permission level table. The permission level table is table information in which the range of support audio playback positions and permission levels are registered in association with each other. The permission level table is an example of range information in the present invention. The permission level indicates the degree to which the support voice is preferentially corrected. The higher the permission level, the higher the priority. Any one of levels 0 to 4 is set as the permission level. Level 0 is an audio part that is output in time with a beat, and is set corresponding to the range of the playback position of the audio part that is output during the first beat of a measure. At the playback position corresponding to level 0, the support voice is not corrected. Level 1 is an audio part that is output in time with the beat, and is set corresponding to the range of the reproduction position of the audio part that is output during a beat different from the first beat of the measure. Level 2 is set corresponding to the range of the reproduction position of the sound part that is not output in time with the beat. Level 3 is set corresponding to the range of the reproduction position of the silent part that is not output in time with the beat. For levels 1 to 3, thresholds th1 to th3 for determining a difference between the display timing of the motion video and the output timing of the support sound are set. These threshold values satisfy th1> th2> th3. It is assumed that the reproduction position of the audio part currently output by the speaker 64 is in a range corresponding to level 0 or 1 in the permission level table. In this case, the output terminal 5 determines that the audio part is an audio part that is output in time with the beat. On the other hand, it is assumed that the reproduction position of the audio part currently output from the speaker 64 is in a range corresponding to level 2 or 3 in the permission level table. In this case, the output terminal 5 determines that the audio part is not an audio part output in time with the beat.

  FIG. 2B is a diagram illustrating a configuration example of the permission level table. In FIG. 2B, level 3 is set for 0 seconds, and level 0 is set for 10 seconds. In this case, the sound part whose playback position is between 0 seconds and 10 seconds indicates that there is no sound. Level 1 is set for 10.250 seconds. In this case, the audio part whose playback position is between 10 seconds and 10.250 is an audio part that is output in time with the beat, and is an audio part that is output during the first beat of the measure. Indicates.

  The output terminal 5 generates a permission level table based on the support voice data. FIG. 3A is a graph showing an example of a waveform of an audio signal of support voice. The output terminal 5 generates an audio signal of support audio from the support audio data. And the output terminal 5 produces | generates the signal which takes the absolute value of the audio | voice level which the audio | voice signal of support audio | voice shows. At this time, the output terminal 5 changes the audio level below the predetermined value to the audio level 0. That is, the sound having a sound level less than the predetermined value is silenced.

  FIG. 3B is a graph showing an example of a waveform of a signal that takes the absolute value of the voice signal of the support voice. The output terminal 5 specifies the reproduction position where the sound level becomes maximum from the signal shown in FIG. Specifically, the output terminal 5 generates a signal obtained by differentiating the signal shown in FIG. FIG. 3C is a graph illustrating an example of a waveform of a signal obtained by differentiating the signal illustrated in FIG. The output terminal 5 extracts a zero cross point from the signal shown in FIG. The zero cross point is a reproduction position where the signal shown in FIG. A zero cross point appears both when the sound level shown in FIG. 3B is maximum and minimum. Therefore, the output terminal 5 extracts the zero cross point when the sound level becomes maximum. For example, in FIG. 3C, times P1 to P5 and the like are extracted as zero cross points.

  The output terminal 5 calculates a time interval between zero cross points in which the order of appearance in the signal shown in FIG. For example, the time interval between time P1 and time P2, the time interval between time P2 and time P3, the time interval between time P3 and time P4, the time interval between time P4 and time P5, the time between time P5 and time P6 The interval etc. are calculated. Assume that the time interval between two zero cross points is a time interval according to the length of the beat. In this case, the output terminal 5 determines that the audio part corresponding to the range surrounded by the two zero cross points is an audio part output in accordance with the beat. For example, if the tempo is 120, the length of one beat is 0.5 seconds. In this case, when the time interval between the zero cross points is a natural multiple of 0.5 seconds, the audio part corresponding to the range surrounded by the zero cross points is an audio part output in time with the beat. In FIG. 3C, the time interval between the time P1 and the time P2, the time interval between the time P2 and the time P3, and the time interval between the time P3 and the time P4 are assumed to be one beat long. Therefore, the audio part corresponding to the range from time P1 to time P4 is an audio part that is output in time with the beat. On the other hand, the time interval between time P4 and time P5 and the time interval between time P5 and time P6 are not time intervals according to the length of the beat. Therefore, the audio part corresponding to the range from the time P4 to the time P6 is not an audio part output in time with the beat.

  When the output terminal 5 specifies the audio part output in time with the beat, the output terminal 5 determines whether the playback position of the audio part is the playback position of the first beat of the measure. For example, the playback position of the first beat of a measure can be calculated based on the tempo and the time signature set in the MIDI data. When the playback position of the audio part output in time with the beat is the playback position of the first beat of the measure, the output terminal 5 registers the playback position and level 0 in association with each other in the permission level table. When the playback position of the audio portion output in time with the beat is not the playback position of the first beat of the measure, the output terminal 5 registers the playback position and level 1 in association with each other in the permission level table. For example, when the time P1 is the reproduction position of the first beat of the measure, the output terminal 5 registers the time P1 and the level 0 in association with each other. In this case, the times P2 to P4 are not the playback position of the first beat of the measure. Accordingly, the output terminal 5 registers each of the times P2 to P4 in association with the level 1.

  When the output terminal 5 specifies a voice part that is different from the voice part output in time with the beat, the output terminal 5 determines whether or not the voice part is sounded. When the audio portion is sounded, the output terminal 5 associates the reproduction position of the audio portion with level 2 and registers them in the permission level table. When the audio part is silent, the output terminal 5 associates the reproduction position of the audio part with level 3 and registers them in the permission level table. In FIG. 3 (C), there is no sound between time P4 and P5 and between time Ta and Tb. In this case, the output terminal 5 registers time P4 and level 2 in association with each other. The output terminal 5 registers time Ta and level 3 in association with each other. The output terminal 5 registers time Tb and level 2 in association with each other.

[3. Configuration of each device]
Next, the configuration of each device included in the athletic content generation system will be described with reference to FIG.

[3-1. Configuration of distribution server 2]
As shown in FIG. 1, the distribution server 2 includes a CPU 21, a ROM 22, a RAM 23, a bus 24, an I / O interface 25, a display control unit 26, a disk drive 28, a network communication unit 30, and an HDD (hard disk drive) 37. . The CPU 21 is connected to the ROM 22, the RAM 23, the bus 24, and the I / O interface 25 via the bus 24. The CPU 21 controls each unit of the distribution server 2 by executing a program stored in the ROM 22 or the HDD 37. Connected to the I / O interface 25 are a database 3, a display control unit 26, a disk drive 28, a network communication unit 30, a keyboard 31, a mouse 32, and an HDD 37. The CPU 21 accesses the database 3 via the I / O interface 25. The display control unit 26 outputs a video signal to the monitor 27 based on the control of the CPU 21. The disk drive 28 writes data to and reads data from the recording medium 29. The network communication unit 30 performs control for the distribution server 2 to connect to the network 10. The HDD 37 stores an OS, various control programs, and the like.

  In the database 3, data such as motion data, MIDI data, and support voice data are registered. Motion data is registered for each movement. The motion data is data that defines the motion motion of the figure 83 in the three-dimensional virtual space. The motion data is data for displaying the motion motion of the figure 83 on the display 67. The motion data includes the coordinates of each part of the body of the figure 83 according to the progress of the motion movement. Motion data is an example of exercise information in the present invention. The motion data is registered in association with the motion information. The motion information is identification information indicating an exercise motion. Each MIDI data is created in accordance with a predetermined exercise operation. The reason is that when the music and the motion video are output, the support voice corresponding to the motion motion indicated by the motion video is output. It is necessary to describe an extended event for audio reproduction in the MIDI data so that a support audio corresponding to the exercise operation is output. When the MIDI data is created, for example, the tempo is also determined. For example, the tempo corresponding to the speed at which the exercise operation is performed is determined. The tempo is described in MIDI data, for example. MIDI data is registered in association with motion information.

[3-2. Configuration of output terminal 5]
As shown in FIG. 1, the output terminal 5 includes a CPU 51, a ROM 52, a RAM 53, a bus 54, an I / O interface 55, a display control unit 56, a disk drive 58, a network communication unit 60, an audio output unit 63, and a signal receiving unit. 65 and HDD 7. The CPU 51 is connected to the ROM 52, the RAM 53, the bus 54, and the I / O interface 55 via the bus 54. The CPU 51 has a clock function and a timer function. The CPU 51 controls each unit of the output terminal 5 by executing a program stored in the ROM 52 or the HDD 7. Connected to the I / O interface 55 are an HDD 7, a display control unit 56, an audio output unit 63, a disk drive 58, a network communication unit 60, a keyboard 61, a mouse 62, and a signal receiving unit 65. The display control unit 56 outputs a video signal to the monitor 57 based on the control of the CPU 51. The audio output unit 63 outputs an audio signal to the monitor 57 based on the control of the CPU 51. The disk drive 58 writes and reads data to and from the recording medium 59. The signal receiving unit 65 receives a signal output from the remote controller 66. The remote controller 66 is for the operator 42 to operate the output terminal 5.

  The HDD 7 is an example of a storage unit and a second storage unit in the present invention. The HDD 7 stores motion data, MIDI data, support audio data, and the like distributed from the distribution server 2. The HDD 7 stores lesson information, permission level tables, threshold values th1 to th3, and the like. The lesson information is lesson information that defines the composition of an exercise lesson. For example, the output terminal 5 generates lesson information based on the operation of the operator 42. The exercise lesson includes, for example, a plurality of exercise operations. In addition, for example, music is assigned to each exercise motion constituting the exercise lesson. The music allocated to the exercise motion is a music output from the speaker 64 when the motion image of the exercise motion to which the music is allocated is displayed on the display 67. The lesson information includes, for example, a plurality of motion information. Each of the plurality of pieces of motion information indicates an exercise operation constituting the exercise lesson. The plurality of pieces of motion information are arranged in the order of execution of the exercise operation. As described above, MIDI data is associated with motion information. The music indicated by the MIDI data is assigned to the exercise motion indicated by the motion information. The permission level table is stored in association with the lesson ID. The lesson ID indicates an exercise lesson in which the permission level table is used.

  The HDD 7 further stores various programs such as an OS, an exercise support program, a music sequencer, a 3D engine, a support voice reproduction program, and an audio device driver. The exercise support program is a program for supporting the exercise of the user 41. The exercise support program causes the CPU 51 as a computer to execute at least a first control step, a second control step, a third control step, a first determination step, a second determination step, and a correction step.

  The CPU 51 outputs various events based on the MIDI data by executing the music sequencer. Specifically, the CPU 51 sequentially reads, for example, a set of delta time and event from the MIDI data. The CPU 51 sleeps for a time corresponding to the delta time and the specified tempo, and then outputs an event. When the normal MIDI event is output, the CPU 51 controls the output of the audio signal by the audio output unit 63 based on the normal MIDI event. As a result, the CPU 51 causes the audio output unit 63 to output a music audio signal to the speaker 64 and causes the speaker 64 to output the music.

  The CPU 51 performs a motion video reproduction process based on the motion data by executing the 3D engine. Specifically, the CPU 51 generates frame images constituting the motion video at a time interval corresponding to a predetermined frame rate and causes the display 67 to display the frame images. The process of generating a frame image includes projective transformation, clipping, hidden surface removal, shading, texture mapping, and the like. The CPU 51 sequentially outputs the frame images to the display control unit 56, so that an exercise image in which the figure 83 performs an exercise operation is displayed on the display 67. The CPU 51 manages the 3D management playback time by storing it in the RAM 53. The 3D management playback time indicates the time that has elapsed since the start of playback of the motion video. The playback start time of the motion video matches, for example, the playback start time of the motion content. The CPU 51 generates a frame image corresponding to the current 3D management playback time. For example, assume that the frame rate is 30 and the current 3D management playback time is 2 seconds. At this time, the CPU 51 generates a frame image of the 60th frame from the top of the frame images constituting the motion video. On the other hand, the normal MIDI event output by the music sequencer is output at a timing corresponding to the delta time described in the MIDI data. Here, there are cases where the output timing of the music and the display timing of the motion video are shifted. Therefore, the CPU 51 synchronizes the display timing of the motion video with the output timing of the music, for example, by the method described below. The CPU 51 counts the number of times that the video synchronization event is output by the music sequencer from the start of the reproduction of the motion video. For example, assume that the frame rate is 30 and the time interval at which the video synchronization event is output is 0.5 seconds. When the current 3D management playback time is 1.9 seconds, the frame image of the 57th frame from the top is displayed. However, assume that the fourth video synchronization event is output at this time. In this case, 2 seconds have elapsed since the start of music playback. Therefore, the output terminal 5 changes the 3D management playback time to 2 seconds. That is, the CPU 51 matches the 3D management playback time with the time elapsed from the start of music playback. Then, the CPU 51 displays the 60th frame image corresponding to the changed 3D management playback time. For example, the CPU 51 matches the 3D management playback time with the time elapsed from the start of music playback only when the difference between the 3D management playback time and the time elapsed since the start of music playback is a predetermined value or more. May be.

  The CPU 51 reproduces the support sound by executing the support sound reproduction program. The CPU 51 starts playback of the support voice based on the extended event for voice playback. Specifically, the CPU 51 sequentially reads data from the support voice data and sequentially transmits it to the audio device driver. The audio device driver is software that provides an interface to the audio output unit 63. Further, the CPU 51 corrects the output timing of the support audio when the display timing of the motion video and the output timing of the support audio are equal to or greater than the threshold. Details of the correction process will be described later.

  By executing the audio device driver, the CPU 51 causes the audio output unit 63 to output an audio signal of the support audio based on the data transmitted from the support audio reproduction program. The CPU 51 manages the time elapsed since the start of the playback of the support voice using one support voice data as the driver management playback position by storing it in the RAM 53. For example, the CPU 51 updates the driver management reproduction position as the output of the audio signal proceeds. When the processing load on the CPU 51 is high, the output of the audio signal by the audio device driver may be delayed. For this reason, the output timing of the support audio is shifted from the display timing of the motion video. In this case, the progress of the driver management playback position is delayed.

  Various programs may be downloaded from the server such as the distribution server 2 via the network 10, for example. Various programs may be recorded on the recording medium 59 and read via the disk drive 58. Note that the music sequencer and the 3D engine may be dedicated hardware instead of a program. The output terminal 5 may include a 3D engine or a music sequencer as hardware.

[4. Operation of Exercise Content Generation System 1]
Next, the operation of the exercise content generation system 1 will be described with reference to FIGS. FIG. 4 is a flowchart illustrating a processing example of the permission level table generation processing of the CPU 51 of the output terminal 5. The CPU 51 generates lesson information based on the operation of the operator 42. The CPU 51 acquires motion data and MIDI data corresponding to each of the plurality of motion information included in the lesson information from the HDD 7. CPU51 couple | bonds the acquired some motion data in the order according to the execution order of exercise | movement operation | movement. As a result, the CPU 51 generates motion data for an exercise lesson corresponding to the lesson information. In addition, the CPU 51 combines the acquired plurality of MIDI data in the order according to the execution order of the exercise operation. As a result, the CPU 51 generates MIDI data for exercise lessons corresponding to the lesson information. The CPU 51 stores the generated motion data and MIDI data in the HDD 7 in association with the lesson ID of the generated lesson information. For example, the CPU 51 may start the permission level table generation process at this time.

  As shown in FIG. 4, the CPU 51 sequentially reads data such as delta time and event from the generated MIDI data, thereby searching for an audio reproduction extended event from the MIDI data. Next, the CPU 51 determines whether or not an audio reproduction extended event has been read (step S1). At this time, if the CPU 51 determines that the audio reproduction extended event has been read (step S1: YES), the CPU 51 proceeds to step S2.

  In step S <b> 2, the CPU 51 acquires support audio data corresponding to the read audio reproduction extended event from the HDD 7. And CPU51 produces | generates the waveform data SG1 which shows the audio | voice signal of a support audio | voice from the acquired support audio | voice data. The waveform data SG1 includes, for example, audio level sample values at predetermined time intervals. Next, the CPU 51 takes the absolute value of each sample value of the waveform data SG1. Thereby, CPU51 produces | generates the waveform data SG2 of the signal which shows the absolute value of an audio | voice signal (step S3). Next, the CPU 51 changes sample values less than a predetermined threshold among sample values included in the waveform data SG2 to zero. Thereby, CPU51 produces | generates the waveform data SG3 which shows the signal which silences the audio | voice whose audio | voice level is less than predetermined value (step S4). The threshold value may be set, for example, between −20 db and −25 db. Next, the CPU 51 removes a signal component having a frequency equal to or higher than a predetermined cutoff frequency from the signal indicated by the waveform data SG3 using a low-pass filter. Thereby, CPU51 produces | generates the waveform data SG4 which shows the signal from which noise was removed (step S5). The low-pass filter may be hardware or software. The cutoff frequency may be set between 100 and 400 Hz, for example. The waveform data SG4 indicates the waveform of the signal in FIG. Next, the CPU 51 calculates the difference between the sample values included in the waveform data SG4. Thereby, CPU51 produces | generates the waveform data SG5 which shows the signal which differentiated the signal which waveform data SG4 shows (step S6).

  Next, the CPU 51 extracts a time P indicating a reproduction position that becomes a zero cross point based on the waveform data SG4. Time P is the time that has elapsed since the reproduction of the music was started. The CPU 51 assigns a number to each time P in the order in which the zero cross points appear in the waveform data SG4. The CPU 51 records the extracted time P in the RAM 53 (step S7). Next, the CPU 51 acquires the tempo from the MIDI data. Next, the CPU 51 calculates the length of one beat corresponding to the tempo. Next, the CPU 51 calculates the time interval between each time P and the time P that appears next. For example, in the example of FIG. 3C, the time interval between the time P1 and the time P2 is calculated. CPU51 records the time P whose time interval is a natural number multiple of the length of 1 beat as time TP (step S8). The CPU 51 may record the time TP even if there is a slight difference between the time interval between the times P and the natural number multiple of the length of one beat. Next, the CPU 51 acquires a time signature from the MIDI data. Based on the time signature, the CPU 51 records the time corresponding to the first beat of the bar in the recorded time TP as time TP0 in the RAM 53 (step S9). For example, assume that the time signature is 4/4 time and the tempo is 120. In this case, the range where the reproduction position is 0 to 0.5 seconds, the range of 2 to 2.5 seconds, and the like are the time ranges corresponding to the first beat of the measure. Next, the CPU 51 records a time other than the time TP0 in the recorded time TP as the time TP1 in the RAM 53 (step S10). Next, the CPU 51 records a time other than the time TP in the time P as the time TP2 in the RAM 53 (step S11). Next, the CPU 51 specifies a time range in which the sound level is 0 among the time P recorded as the time TP2 and the next time P appearing as a silent section. Then, the CPU 51 records the reproduction position where the silent period starts in the RAM 53 as the time TP3 (step S12). Further, the CPU 51 records the reproduction position at which the silent section ends in the RAM 53 as T2. For example, in the example of FIG. 3C, in the range of time P4 and P5, time Ta is a reproduction position where the silent period starts, and time Tb is a reproduction position where the silent period ends.

  Next, the CPU 51 performs registration with respect to the permission level table (step S13). Specifically, the CPU 51 registers the time TP0 and the level 0 in the permission level table in association with each other. Further, the CPU 51 associates the time TP1 with the level 1 and registers them in the permission level table. Further, the CPU 51 registers the time TP2 and the level 2 in the permission level table in association with each other. Further, the CPU 51 registers the time TP3 and the level 3 in the permission level table in association with each other. Next, the CPU 51 proceeds to step S1.

  In step S1, if the CPU 51 detects the end of the MIDI data and determines that the audio reproduction extended event has not been read (step S1: NO), the CPU 51 ends the permission level table generation process. The CPU 51 stores the generated permission level table in the HDD 7 in association with the lesson ID of the generated lesson information.

  FIG. 5A is a flowchart illustrating a processing example of the exercise content reproduction process of the CPU 51 of the output terminal 5. For example, the instructor selects the exercise lesson to be executed by operating the output terminal 5. The CPU 51 acquires MIDI data, motion data, and a permission level table corresponding to the lesson ID of the selected exercise lesson from the HDD 7. The CPU 51 starts the exercise content reproduction process based on the acquired data. When starting the exercise content playback process, the CPU 51 acquires the current time from the clock function of the CPU 51. Then, the CPU 51 stores the acquired time in the RAM 53 as the exercise content reproduction start time. Further, the CPU 51 acquires the tempo from the MIDI data. In the exercise content playback process, the CPU 51 sequentially reads a set of delta time and event from the beginning of the MIDI data. And CPU51 performs the process according to the read event.

  Specifically, as shown in FIG. 5A, the CPU 51 executes a process of reading one set of delta time and event from the MIDI data (step S21). Next, the CPU 51 determines whether or not the end of the MIDI data is detected in the process of step S21 (step S22). At this time, if the CPU 51 determines that the end of the MIDI data is not detected (step S22: NO), the CPU 51 sleeps for a time corresponding to the read delta time and the specified tempo. Then, the CPU 51 proceeds to step S23.

  In step S23, the CPU 51 determines whether or not the read event is a normal MIDI event. At this time, if the CPU 51 determines that the event is a normal MIDI event (step S23: YES), the CPU 51 proceeds to step S24. On the other hand, when the CPU 51 determines that the event is not a normal MIDI event (step S23: NO), the CPU 51 proceeds to step S25. In step S24, the CPU 51 outputs the read normal MIDI event to the software synthesizer. Next, the CPU 51 proceeds to step S21.

  In step S25, the CPU 51 determines whether or not the read event is a 3D control extended event. At this time, if the CPU 51 determines that the event is an extended event for 3D control (step S25: YES), the CPU 51 proceeds to step S26. On the other hand, if the CPU 51 determines that the event is not an extended event for 3D control (step S25: NO), the CPU 51 proceeds to step S27. If the event is neither a normal MIDI event nor a 3D control extended event, the event is an audio playback extended event. In step S <b> 26, the CPU 51 outputs the read 3D control extended event to the 3D engine. Next, the CPU 51 proceeds to step S21.

  In step S27, the CPU 51 outputs the read audio reproduction extended event to the support audio reproduction program. Next, the CPU 51 proceeds to step S21. In step S22, if the CPU 51 determines that the end of the MIDI data has been detected (step S22: YES), it ends the exercise content playback process.

  FIG. 6 is a flowchart showing a processing example of support voice reproduction processing of the CPU 51 of the output terminal 5. In the exercise content playback process, an audio playback extended event for starting playback of the support audio is output. Then, the output terminal 5 starts support voice reproduction processing. The CPU 51 initializes the position for reading data from the support audio data corresponding to the extended event for audio reproduction. The data read position corresponds to the address where each data is stored in the support voice data. The CPU 51 sets the data read position to the head of the support audio data.

  Next, the CPU 51 determines whether or not at least one of level 0 and level 1 is registered in the permission level table (step S31). At this time, if the CPU 51 determines that at least one of level 0 and level 1 is registered (step S31: YES), the CPU 51 proceeds to step S32. On the other hand, if the CPU 51 determines that neither level 0 nor level 1 is registered (step S31: NO), the process proceeds to step S33.

  In step S32, the CPU 51 sets the correction flag to TRUE. Next, the CPU 51 proceeds to step S34. In step S33, the CPU 51 sets the correction flag to FALSE. Next, the CPU 51 proceeds to step S34. The correction flag is information indicating whether or not the support voice is corrected. When the correction flag is TRUE, the CPU 51 corrects the support voice. When the correction flag is FALSE, the CPU 51 does not correct the support voice.

  In step S <b> 34, the CPU 51 sets the time elapsed from the start of playback of the exercise content as the support audio playback start time tb. For example, the CPU 51 acquires the current time from the clock function. Then, the CPU 51 calculates time tb by subtracting the start of exercise content reproduction from the current time. Next, the CPU 51 acquires the 3D management playback time t1 from the 3D engine (step S35). Next, the CPU 51 acquires a time t2 indicating a driver management reproduction position from the audio device driver (step S36). Next, the CPU 51 acquires a permission level corresponding to the time tb + t2 from the permission level table (step S37). The time tb + t2 is the time that has elapsed since the start of the reproduction of the exercise content. Next, the CPU 51 calculates the difference diff by subtracting the time tb + t2 from the time t1 (step S38). The difference diff indicates a difference between the display timing of the motion video and the output timing of the support sound. Next, the CPU 51 calculates the absolute value ad of the difference diff (step S39). Next, the CPU 51 determines whether or not the correction flag is TRUE (step S40). At this time, if the CPU 51 determines that the correction flag is TRUE (step S40: YES), the CPU 51 proceeds to step S41. On the other hand, if the CPU 51 determines that the correction flag is not TRUE (step S40: NO), the CPU 51 proceeds to step S48.

  In step S41, the CPU 51 determines whether or not the absolute value ad is larger than the threshold value th3. At this time, when the CPU 51 determines that the absolute value ad is equal to or less than the threshold th3 (step S41: NO), the CPU 51 proceeds to step S48. On the other hand, when the CPU 51 determines that the absolute value ad is larger than the threshold th3 (step S41: YES), the CPU 51 proceeds to step S42. In step S42, the CPU 51 determines whether or not the acquired permission level is zero. At this time, if the CPU 51 determines that the permission level is 0 (step S42: YES), the CPU 51 proceeds to step S48. On the other hand, when the CPU 51 determines that the permission level is not 0 (step S42: NO), the CPU 51 proceeds to step S43. In step S43, the CPU 51 determines whether or not the acquired permission level is 1. At this time, if the CPU 51 determines that the permission level is not 1 (step S43: NO), the process proceeds to step S45. On the other hand, when the CPU 51 determines that the permission level is 1 (step S43: YES), the CPU 51 proceeds to step S44. In step S44, the CPU 51 determines whether or not the absolute value ad is larger than the threshold value th1. At this time, if the CPU 51 determines that the absolute value ad is equal to or less than the threshold th1 (step S44: NO), the CPU 51 proceeds to step S48. On the other hand, when the CPU 51 determines that the absolute value ad is larger than the threshold th1 (step S44: YES), the CPU 51 proceeds to step S47.

  In step S45, the CPU 51 determines whether or not the acquired permission level is 2. At this time, if the CPU 51 determines that the permission level is not 2 (step S45: NO), the process proceeds to step S47. On the other hand, when the CPU 51 determines that the permission level is 2 (step S45: YES), the CPU 51 proceeds to step S46. In step S46, the CPU 51 determines whether or not the absolute value ad is larger than the threshold value th2. At this time, when the CPU 51 determines that the absolute value ad is equal to or less than the threshold th2 (step S46: NO), the CPU 51 proceeds to step S48. On the other hand, when the CPU 51 determines that the absolute value ad is larger than the threshold th2 (step S46: YES), the CPU 51 proceeds to step S47.

  In step S47, the CPU 51 executes a synchronization process. FIG. 5B is a flowchart illustrating a processing example of the synchronization processing of the CPU 51 of the output terminal 5. As shown in FIG. 5B, the CPU 51 determines whether or not the difference diff is larger than 0 (step S61). At this time, if the CPU 51 determines that the difference diff is greater than 0 (step S61: YES), the CPU 51 proceeds to step S62. On the other hand, when the CPU 51 determines that the difference diff is 0 or less (step S61: NO), the CPU 51 proceeds to step S63. In step S62, the CPU 51 moves the read position of the data from the support voice data toward the end of the support voice data by the number of bytes corresponding to the time indicated by the absolute value ad. In step S63, the CPU 51 sleeps for the time indicated by the absolute value ad. When finishing the process of step S62 or S63, the CPU 51 ends the synchronization process and proceeds to step S48.

  As shown in FIG. 6, in step S <b> 48, the CPU 51 reads data corresponding to the data size of the buffer secured in the RAM 53 from the support audio data, and stores it in the buffer. At this time, the CPU 51 reads data corresponding to the data size of the buffer from the current data reading position. Next, the CPU 51 determines whether or not data can be read from the support voice data (step S49). At this time, if the CPU 51 determines that the data could be read (step S49: YES), the CPU 51 proceeds to step S51. In step S51, the CPU 51 moves the data read position by the data size of the buffer toward the end of the support audio data. Then, the CPU 51 transmits the data stored in the buffer to the audio device driver. Next, the CPU 51 proceeds to step S35. On the other hand, if the CPU 51 determines that the data could not be read (step S49: NO), the support voice reproduction process is terminated. In this case, the data read position has reached the end of the support voice data.

  As described above, according to the present embodiment, the CPU 51 determines whether there is a deviation exceeding the threshold value between the display timing of the motion video and the output timing of the support audio. When the CPU 51 determines that there is a deviation exceeding the threshold, the CPU 51 determines whether the audio part output from the speaker 64 in the support audio is an audio part output in accordance with the beat. The CPU 51 corrects the output timing of the support voice when it is determined that the voice part is not output in time with the beat. Therefore, it is possible to correct the output timing of the support voice so that the rhythm at which the support voice is output is not disturbed.

1 Exercise content generation system 2 Distribution server 3 Database 5 Output terminal 7 HDD
51 CPU
52 ROM
53 RAM
56 Display Control Unit 60 Network Communication Unit 63 Audio Output Unit 57 Monitor 64 Speaker 67 Display

Claims (7)

Storage means for storing musical score information indicating a musical score of music, operational information indicating an exercise motion, and audio information indicating a voice that supports a user performing the exercise motion;
Video information indicating a video on which the exercise operation is performed based on a synchronization signal generated according to a specified tempo and the musical score information stored in the storage means, and the operation information stored in the storage means First control means for displaying on the display means;
Second control means for causing the output means to output music information indicating the music based on the synchronization signal and the score information stored in the storage means;
Third control means for causing the output means to output the audio information stored in the storage means based on the synchronization signal;
First determination means for determining whether there is a deviation exceeding a predetermined time between the timing at which the video information is displayed on the display means and the timing at which the audio information is output by the output means;
When it is determined by the first determination means that there is the deviation, a second determination is made as to whether or not the output portion of the audio information output by the output means is an audio portion output in time with a beat. A determination means;
Correction means for correcting the output timing of the audio information when the second determination means determines that the output portion is not an audio portion output in time with a beat;
An information processing apparatus comprising: And further comprising third determination means for determining whether the sound information stored in the storage means includes a sound portion output in time with a beat,
The correction means does not correct the output timing of the audio information when the third determination means determines that the audio information does not include an audio portion that is output in time with a beat,
Only when it is determined by the third determination means that the audio information includes an audio portion that is output in time with the beat, the second determination means outputs the output of the audio information that is output by the output means. The information processing apparatus according to claim 1, wherein the information processing unit determines whether the part is an audio part output in time with a beat. When the second determining means determines that the output part is an audio part that is output in time with a beat, the timing at which the video information is displayed on the display means, and the audio information is output by the output means And a fourth determination means for determining whether there is a deviation exceeding a second predetermined time longer than the predetermined time.
3. The information processing apparatus according to claim 1, wherein the correction unit corrects the output timing of the audio information when the fourth determination unit determines that there is a shift. 4. It is determined whether the output portion determined to be an audio portion output in time with a beat by the second determination means is an audio portion output during the first beat of any measure of the score. And a fifth determination means for
The correcting means determines when the video information is displayed on the display means when the fifth determining means determines that the output portion is an audio portion output during the first beat of the measure. the on and when audio information is output by the output unit, regardless of whether there is a deviation exceeding the second predetermined time, according to claim 3, characterized in that does not correct the output timing of the audio information The information processing apparatus described in 1. The storage means stores the audio information indicating a volume along a time series of the supported audio,
First specifying means for specifying a temporal output position at which the volume is maximized in the time series indicated by the audio information stored in the storage means;
Second specifying means for specifying a plurality of the output positions at which the volume is maximized at a time interval corresponding to the specified tempo among the output positions specified by the first specifying means;
Second storage means for storing range information indicating temporal ranges corresponding to the plurality of output positions specified by the second specifying means;
Obtaining means for obtaining a temporal output position of the output portion being output by the output means;
Further comprising
The second determination means is a sound in which the output portion is output in time with a beat when the output position acquired by the acquisition means is within a range indicated by the range information stored in the second storage means. The information processing apparatus according to claim 1, wherein the information processing apparatus is determined to be a part. The storage means stores the audio information indicating a volume along a time series of the supported audio,
The correction means preferentially corrects the output timing of the audio information when an audio portion whose volume is less than a predetermined value is output from audio portions different from the audio portion output in time with the beat. The information processing apparatus according to claim 1, wherein the information processing apparatus is an information processing apparatus. It is generated according to the specified tempo and the musical score information stored in the storage means for storing the musical score information indicating the musical score of the music, the operational information indicating the motion and the voice information indicating the voice supporting the user performing the motion. A first control step for causing the display means to display video information indicating a video on which the exercise motion is performed based on the synchronization signal and the motion information stored in the storage means;
A second control step of causing the output means to output music information indicating the music based on the synchronization signal and the score information stored in the storage means;
A third control step for causing the output means to output the audio information stored in the storage means based on the synchronization signal;
A first determination step of determining whether there is a deviation exceeding a predetermined time between a timing at which the video information is displayed on the display means and a timing at which the audio information is output by the output means;
When it is determined in the first determination step that there is a deviation, a second determination is made as to whether or not the output part output by the output means in the audio information is an audio part output in time with a beat. A determination step;
A correction step of correcting the output timing of the audio information when the second determination step determines that the output portion is not an audio portion output in time with a beat;
A program that causes a computer to execute.