WO2024190759A1 - 情報処理方法、情報処理システムおよびプログラム - Google Patents

情報処理方法、情報処理システムおよびプログラム Download PDF

Info

Publication number
WO2024190759A1
WO2024190759A1 PCT/JP2024/009447 JP2024009447W WO2024190759A1 WO 2024190759 A1 WO2024190759 A1 WO 2024190759A1 JP 2024009447 W JP2024009447 W JP 2024009447W WO 2024190759 A1 WO2024190759 A1 WO 2024190759A1
Authority
WO
WIPO (PCT)
Prior art keywords
performance
user
playing
character
action
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2024/009447
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
陽 前澤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yamaha Corp
Original Assignee
Yamaha Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yamaha Corp filed Critical Yamaha Corp
Priority to JP2025506856A priority Critical patent/JPWO2024190759A1/ja
Priority to CN202480014672.XA priority patent/CN120752695A/zh
Publication of WO2024190759A1 publication Critical patent/WO2024190759A1/ja
Priority to US19/327,922 priority patent/US20260050410A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—Three-dimensional [3D] animation
    • G06T13/40—Three-dimensional [3D] animation of characters, e.g. humans, animals or virtual beings
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/80—Two-dimensional [2D] animation, e.g. using sprites
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10G—REPRESENTATION OF MUSIC; RECORDING MUSIC IN NOTATION FORM; ACCESSORIES FOR MUSIC OR MUSICAL INSTRUMENTS NOT OTHERWISE PROVIDED FOR, e.g. SUPPORTS
    • G10G1/00—Means for the representation of music
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H1/00—Details of electrophonic musical instruments
    • G10H1/36—Accompaniment arrangements

Definitions

  • This disclosure relates to technology that assists users in playing musical instruments.
  • Patent Document 1 discloses a performance agent that automatically plays a piece of music in accordance with the user's performance.
  • Patent Document 1 the performance of a user is supported by the automatic performance of a performance agent.
  • the development of further technology that intuitively or interactively supports the performance of users is expected. Taking the above circumstances into consideration, one aspect of the present disclosure aims to intuitively or interactively support the performance of users.
  • an information processing method analyzes the state of a performance by a user in real space, displays a performance character that performs performance actions in a virtual space on a display device, and controls the guiding actions of the performance character according to the state of the performance by the user.
  • An information processing system includes a performance analysis unit that analyzes the state of a performance by a user in real space, and a display control unit that displays a performance character performing in a virtual space on a display device and controls the behavior of the performance character according to the state of the performance by the user.
  • a program causes a computer system to function as a performance analysis unit that analyzes the state of a performance by a user in real space, and a display control unit that displays a performance character performing in a virtual space on a display device and controls the behavior of the performance character according to the state of the performance by the user.
  • FIG. 1 is a block diagram of a performance system according to a first embodiment.
  • FIG. 2 is a block diagram illustrating an example of a functional configuration of an information processing system.
  • FIG. FIG. 2 is a schematic diagram of a playing character.
  • FIG. 2 is a block diagram of a display control unit.
  • FIG. 13 is a flowchart of an operation control process.
  • FIG. 11 is a block diagram of a performance system according to a second embodiment.
  • FIG. 1 is a block diagram illustrating the configuration of a performance system 100A according to the first embodiment.
  • the performance system 100A of the first embodiment is a computer system that supports a user U in playing a piece of music (hereinafter referred to as a "target piece of music").
  • the performance system 100A includes a performance device 10 and an information processing system 20.
  • the performance device 10 is connected to the information processing system 20 by wire or wirelessly.
  • the information processing system 20 may be mounted on the performance device 10.
  • the performance device 10 may also be interpreted as an element constituting the information processing system 20.
  • the performance device 10 is an input device that accepts the performance of a target piece of music by the user U.
  • the performance device 10 is, for example, a MIDI (Musical Instrument Digital Interface) controller equipped with a keyboard 11.
  • the keyboard 11 is composed of a number of keys 12 that correspond to different pitches.
  • the user U plays the target piece of music by operating each key 12 in sequence.
  • the target musical piece in the first embodiment includes a melody part and an accompaniment part.
  • the melody part is a performance part that is made up of the melody sounds of the target musical piece
  • the accompaniment part is a performance part that is made up of the accompaniment sounds of the target musical piece.
  • the user U plays the melody part using the performance device 10. Note that the melody part is an example of the "first part,” and the accompaniment part is an example of the "second part.”
  • the performance device 10 outputs performance data P representing a performance by the user U to the information processing system 20.
  • the performance data P is, for example, event data conforming to the MIDI standard. Specifically, the performance data P specifies the pitch corresponding to the key 12 operated by the user U. With each operation by the user U, the performance data P is transmitted sequentially from the performance device 10.
  • the information processing system 20 is a computer system that analyzes and assists the user U in playing a target piece of music.
  • the information processing system 20 is realized by an information device such as a smartphone, a tablet terminal, or a personal computer.
  • the information processing system 20 comprises a control device 21, a storage device 22, a display device 23, an operation device 24, a sound source device 25, and a sound emission device 26.
  • the information processing system 20 may be realized as a single device, or may be realized as multiple devices configured separately from each other.
  • the control device 21 is a single or multiple processors that control each element of the information processing system 20.
  • the control device 21 is configured with one or more types of processors, such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an SPU (Sound Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or an ASIC (Application Specific Integrated Circuit).
  • processors such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an SPU (Sound Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or an ASIC (Application Specific Integrated Circuit).
  • the storage device 22 is a single or multiple memories that store the programs executed by the control device 21 and various data used by the control device 21.
  • a well-known recording medium such as a semiconductor recording medium or a magnetic recording medium, or a combination of multiple types of recording media, is used as the storage device 22.
  • a portable recording medium that is detachable from the information processing system 20, or a recording medium that the control device 21 can access via a communication network (e.g., cloud storage) may be used as the storage device 22.
  • the storage device 22 stores music data M.
  • Music data M specifies the time sequence of notes (i.e., musical score) that make up the target music piece.
  • music data M is composed of melody data M1 that specifies each note in the melody part, and accompaniment data M2 that specifies each note in the accompaniment part.
  • Melody data M1 and accompaniment data M2 are time-series data in which note data N that specifies notes and instructs performance, and time data that specifies the time point when each note data N is read out are arranged.
  • Music data M is, for example, a file in a format that conforms to the MIDI standard (SMF: Standard MIDI File).
  • the display device 23 displays images under the control of the control device 21.
  • various display panels such as a liquid crystal display panel or an organic EL (Electroluminescence) panel are used as the display device 23.
  • the operation device 24 is an instruction input device that accepts instructions from the user U.
  • an operator operated by the user U or a touch panel integrated with the display device 23 is used as the operation device 24.
  • the display device 23 or the operation device 24, which are separate from the information processing system 20 may be connected to the information processing system 20 by wire or wirelessly.
  • the sound source device 25 is sequentially supplied with the performance data P generated by the performance device 10 and the note data N of the accompaniment data M2.
  • the sound source device 25 generates an audio signal V representing the waveform of the musical tones specified by the performance data P and the note data N.
  • an audio signal V is generated that represents a mixture of the musical tones of the melody part played by the user U and the musical tones of the accompaniment part of the target piece of music.
  • the function of the sound source device 25 may be realized by the control device 21 executing a program.
  • the sound emitting device 26 reproduces the musical sound represented by the audio signal V.
  • a speaker or a headphone is used as the sound emitting device 26.
  • a D/A converter that converts the audio signal V from digital to analog, and an amplifier that amplifies the audio signal V are not shown in the figure.
  • the sound emitting device 26, which is separate from the information processing system 20, may be connected to the information processing system 20 by wire or wirelessly.
  • FIG. 2 is a block diagram illustrating an example of the functional configuration of the information processing system 20.
  • the control device 21 executes a program stored in the storage device 22 to realize multiple functions (performance control unit 31, display control unit 32, performance analysis unit 33) for analyzing and supporting the performance by the user U.
  • the performance control unit 31 is a sequencer that sequentially outputs the note data N of the accompaniment data M2 of the musical piece data M to the sound source device 25. That is, the performance control unit 31 instructs the sound source device 25 to generate each musical tone of the accompaniment part specified by the accompaniment data M2.
  • the performance data P generated by the performance device 10 is sequentially supplied to the sound source device 25. Therefore, the musical tones of the melody part played by the user U and the musical tones of the accompaniment part of the target musical piece are reproduced in parallel by the sound emission device 26.
  • the performance control unit 31 variably controls the time at which the note data N is output to the sound source device 25 so that the playback of the musical tones of the accompaniment part follows the performance by the user U.
  • the time at which the user U plays each note of the melody part of the target piece of music changes dynamically due to the musical expression intended by the user U or a lack of performance technique on the part of the user U. Therefore, the time at which the performance control unit 31 outputs each note data N of the accompaniment data M2 to the sound source device 25 is also variable.
  • the performance control unit 31 estimates the time at which the user U is currently playing in the target song (hereinafter referred to as the "performance time") by analyzing the time series of the performance data P. For example, the performance control unit 31 estimates the performance time by comparing the melody data M1 of the song data M with the time series of the performance data P. The estimation of the performance time is performed sequentially in parallel with the performance of the melody part by the user U. For example, a known acoustic analysis technique (score alignment) described in JP 2015-79183 A or the like may be arbitrarily adopted for estimating the performance time.
  • the performance control unit 31 outputs each note data N to the sound source device 25 so that the generation of the accompaniment part by the sound source device 25 is synchronized with the progress of the performance time. Therefore, the reproduction of the musical tones of the melody part played by the user U and the reproduction of the musical tones of the accompaniment part of the target song are synchronized with each other.
  • the display control unit 32 in FIG. 2 displays an image (hereinafter referred to as a "guide image G") that supports the user U in playing the target song on the display device 23.
  • FIG. 3 is a schematic diagram of the guide image G.
  • the guide image G is an animated video that includes a virtual musical instrument 50 and a musical character 60.
  • the virtual musical instrument 50 and the performing character 60 are virtual objects placed in a virtual space.
  • the virtual musical instrument 50 is a virtual keyboard instrument.
  • the performing character 60 is a human-like object (agent) with an appearance that mimics a performer playing the virtual musical instrument 50 in the virtual space.
  • the performing character 60 in the first embodiment is a virtual performer who plays the accompaniment part of the target piece of music in the virtual space. While the performing character 60 is located in the virtual space, the user U is located in the real space.
  • FIG. 4 is an explanatory diagram of a musical performance character 60.
  • the musical performance character 60 is a three-dimensional object including both arms, a chest, and a head. Eyes 61 (right and left eyes) are provided on the head of the musical performance character 60. Therefore, a line of sight L is defined for the musical performance character 60.
  • the skeleton of the playing character 60 is composed of multiple control points 63 and multiple connecting parts 64.
  • Each control point 63 corresponds to a joint of the playing character 60, and is movable within the virtual space.
  • the connecting parts 64 are line segments that connect each control point 63 to another. The movement of the playing character 60 is controlled by changing the position of each control point 63 and the angle of the connecting parts 64 relative to each control point 63.
  • observation point Q is set in the virtual space.
  • Observation point Q is the point from which the virtual space is observed.
  • a virtual camera is installed at observation point Q in the virtual space.
  • the virtual camera is a virtual imaging device that images the virtual space.
  • the guide image G is a video image of the virtual space captured by the virtual camera.
  • the display control unit 32 generates image data of the guide image G by known image processing such as 3D rendering, and outputs the image data to the display device 23. Therefore, the guide image G, which shows the performance character 60 playing the virtual musical instrument 50 as viewed from the observation point Q, is displayed on the display device 23.
  • the observation point Q corresponds to the viewpoint of the user U in the virtual space.
  • FIG. 5 is a block diagram illustrating the functional configuration of the display control unit 32.
  • the display control unit 32 in the first embodiment includes a first operation control unit 321 and a second operation control unit 322.
  • the first movement control unit 321 controls the movement of the playing character 60 playing the virtual musical instrument 50 (hereinafter referred to as "playing movement"). Specifically, the first movement control unit 321 links the playing movement of the playing character 60 to the playback of the accompaniment part. Specifically, the playing character 60 sways its body in rhythm with the accompaniment part, and when musical tones corresponding to each note in the accompaniment part are played, the playing character 60 executes the action of pressing a key.
  • the playback of the musical tones of the accompaniment part follows the performance by the user U. Therefore, the first operation control unit 321 controls the performance action of the playing character 60 playing the accompaniment part so as to follow the performance of the melody part by the user U.
  • the user U can perceive the sensation that the playing character 60 is playing the accompaniment part of the target song.
  • the playback of the musical tones of the melody part played by the user U and the playback of the musical tones of the accompaniment part are mutually synchronized. Therefore, according to the first embodiment, it is possible for the user U to perceive the sensation that he or she is playing the target song in a cooperative ensemble or duet with the playing character 60.
  • the first movement control unit 321 controls each control point 63 and each linking part 64 of the playing character 60 according to the musical note data N output by the performance control unit 31, thereby linking the playing movement of the playing character 60 to the playback of the accompaniment part. Specifically, the first movement control unit 321 generates control data according to the musical note data N output by the performance control unit 31. The control data specifies the position of each control point 63 and the angle of the linking part 64 relative to each control point 63.
  • a trained model is used to generate the control data according to the note data N.
  • the trained model is a statistical prediction model that has learned the relationship between the note data N and the control data through machine learning.
  • the first movement control unit 321 generates the control data by processing the note data N sequentially output by the performance control unit 31 using the trained model, and controls the performance movement of the playing character 60 according to the control data.
  • the trained model is composed of a deep neural network such as a convolutional neural network or a recurrent neural network.
  • a known technology described in JP 2019-139294 A is used to generate the control data according to the note data N.
  • the second movement control unit 322 in FIG. 5 controls the movement of the playing character 60 to guide the performance by the user U (hereinafter referred to as "guiding movement").
  • the guiding movement is, for example, the movement of the body of the playing character 60 or a change in the facial expression of the playing character 60.
  • the guiding movement is a movement related to the performance of the melody part by the user U. In other words, the guiding movement is a movement that is not directly related to the performance (performance movement) of the accompaniment part of the playing character 60 itself.
  • the guiding movement can also be expressed as a movement in which there is less change in the control points 63 or the connecting parts 64 compared to the performance movement, for example.
  • the performance analysis unit 33 in FIG. 2 analyzes the state of the performance by the user U (hereinafter referred to as the "performance state"). Specifically, the performance analysis unit 33 analyzes the performance state by processing the performance data P output by the performance device 10. For example, the performance analysis unit 33 analyzes the performance state by comparing the melody data M1 of the target piece of music with the time series of the performance data P.
  • the second movement control unit 322 controls the guiding action of the playing character 60 according to the playing state of the user U.
  • the second movement control unit 322 causes the playing character 60 to execute a specific guiding action when a predetermined condition (hereinafter referred to as an "action condition") is met for the playing state.
  • the playing action is an action that is controlled according to the accompaniment part
  • the guiding action is an action that is controlled according to the playing state of the user U. Specific examples of playing states and guiding actions are listed below.
  • the operating condition in the operation example 1a is that the performance of the melody part by the user U stops for a predetermined time. That is, the performance analysis unit 33 judges whether the performance by the user U stops for a predetermined time. Specifically, the performance analysis unit 33 judges that the performance by the user U has stopped when the performance data P corresponding to a specific note is not output from the performance device 10 even after a predetermined time has passed since the melody data M1 instructs the sounding of the specific note.
  • the second movement control unit 322 causes the playing character 60 to perform a guidance action of moving the line of sight L toward the observation point Q in the virtual space.
  • the guidance action of the movement example 1a is an action of changing the line of sight L of the playing character 60 from state A, in which the line of sight L is directed toward the virtual musical instrument 50, to state B, in which the line of sight L is directed toward the observation point Q, as illustrated in FIG. 6.
  • State B is a state in which the playing character 60 directs its head and line of sight L toward the virtual user U in the virtual space.
  • the second movement control unit 322 realizes the above guidance action by controlling the direction of the head of the playing character 60.
  • the guiding action in operation example 1a is an action in which the performing character 60 gives the user U a wink to urge him or her to continue playing.
  • the user U visually recognizes the guiding action in operation example 1a, the user U can intuitively or interactively understand that he or she should continue playing the melody part.
  • Example 1b The operating condition in the operation example 1b is that the performance of the melody part by the user U is completed. Specifically, the performance analysis unit 33 determines whether or not the last note specified by the melody data M1 has been played by the user U.
  • the second movement control unit 322 causes the performing character 60 to perform a guiding movement of moving the line of sight L toward the observation point Q in the virtual space, similar to the movement example 1a.
  • the guiding movement of the movement example 1b is a movement in which the performing character 60 notifies the user U that the ensemble has ended.
  • the user U who visually recognizes the guiding movement of the movement example 1b, can intuitively or interactively understand that the performance of the melody part has ended.
  • the guiding action of moving the line of sight L of the playing character 60 toward the observation point Q is recognized by the user U as the playing character 60 giving a wink to the user U. Therefore, the user U who visually recognizes the guiding action can intuitively or interactively grasp that there is something he or she should pay attention to regarding his or her own performance.
  • the second movement control unit 322 causes the playing character 60 to perform a guiding action to signal the start of the performance of the melody part by the user U.
  • the second movement control unit 322 identifies the time of the start of the performance by referring to the melody data M1. Then, the second movement control unit 322 causes the playing character 60 to perform a guiding action of rocking the head up and down at the time when the performance of the melody part starts at the beginning of the target music piece, for example.
  • the second movement control unit 322 causes the playing character 60 to perform a guiding action of rocking the head up and down at the time when the performance of the melody part resumes from a rest of a duration exceeding a predetermined value in the middle of the target music piece. Therefore, the user U can intuitively or interactively grasp the time when the performance of the melody part should be started.
  • the operating condition in the operation example 1d is that the performance tempo of the melody part performed by the user U fluctuates unstably.
  • the performance analysis unit 33 periodically identifies the performance tempo of the user U and determines whether the performance tempo fluctuates over time. For example, the performance analysis unit 33 determines that the performance tempo fluctuates unstably when the variance of the performance tempo exceeds a predetermined value, and determines that the performance tempo is stable when the variance of the performance tempo is below the predetermined value.
  • the second movement control unit 322 causes the performing character 60 to perform a conducting movement as a guiding movement.
  • the conducting movement is a guiding movement in which one arm (or both arms) of the performing character 60 swings at a period corresponding to the standard performance tempo specified by the music data M. Therefore, the user U who visually recognizes the guiding movement can intuitively or interactively grasp that his or her own performance tempo is not stable and the standard performance tempo of the target music piece.
  • the operating condition in the operation example 1e is that the user U is hesitating to play the melody part.
  • the situation in which the user U is hesitating to play is, for example, a situation in which the user U is unsure of how to play or a situation in which the user U cannot accurately grasp which part of the target music piece should be played.
  • the performance analysis unit 33 determines that the user U is hesitating to play when (1) the playing tempo of the user U is slower than the standard playing tempo, (2) the time of playing the note is delayed with respect to the time of the note specified by the melody data M1, or (3) the performance by the user U stops for a predetermined period of time.
  • the second movement control unit 322 causes the playing character 60 to execute a guiding movement to encourage the user U to play.
  • the second movement control unit 322 causes the playing character 60 to swing its head or arms at a cycle corresponding to the standard playing tempo specified by the music data M.
  • FIG. 7 is a flowchart of the process (hereinafter referred to as the "motion control process") executed by the control device 21 to control the playing character 60.
  • the motion control process is started in response to an instruction from the user U via the operation device 24.
  • the control device 21 (performance control unit 31) outputs the note data N of the accompaniment data M2 to the sound source device 25 (S1).
  • the sound source device 25 generates an audio signal V representing the musical tones specified by the note data N and the musical tones specified by the performance data P. Therefore, the musical tones of the melody part played by the user U and the musical tones of the accompaniment part of the target piece of music are reproduced in parallel from the sound emission device 26.
  • the control device 21 display control unit 32
  • the control device 21 (performance analysis unit 33) analyzes the performance state of the user U (S3) and determines whether or not the operating conditions for the performance state are met (S4). If the operating conditions are met (S4: YES), the control device 21 (display control unit 32) causes the playing character 60 to perform a guidance action (S5). On the other hand, if the operating conditions are not met (S4: NO), the playing character 60 does not perform a guidance action.
  • the control device 21 determines whether it is approaching the time when the performance of the melody part should start (S6). Specifically, the control device 21 determines whether it has reached a point a predetermined time before the first note of the target piece of music or the note after a long rest. If it is approaching the time when the performance of the melody part should start (S6: YES), the control device 21 (display control unit 32) causes the playing character 60 to perform a guide action that signals the start of performance of the melody part (S7).
  • the control device 21 determines whether a predetermined end condition has been met (S8).
  • the end condition is, for example, that a predetermined time has elapsed since the end of the performance of the target song, or that an instruction to end the performance has been given by operating the operating device 24. If the end condition is not met (S8: NO), the control device 21 moves the process to step S1. In other words, the processes exemplified above (S1 to S7) are repeated until the end condition is met. If the end condition is met (S8: YES), the control device 21 ends the operation control process.
  • the movements of the playing character 60 performing in the virtual space are controlled according to the playing state of the user U in the real space. Therefore, the user U can smoothly progress with the performance while checking the movements of the playing character 60. In other words, it is possible to intuitively or interactively support the performance by the user U.
  • the playing character 60 executes a guidance action. Therefore, the user U can intuitively or interactively understand that the action conditions for the performance by the user U have been met.
  • Second embodiment A second embodiment of the present disclosure will be described. Note that, for elements having the same functions as those in the first embodiment in each of the following exemplary aspects, the same reference numerals as those in the first embodiment will be used, and detailed descriptions of each will be omitted as appropriate.
  • FIG. 8 is a block diagram of a performance system 100B in the second embodiment.
  • the performance system 100B in the second embodiment includes a display unit 70 in addition to the performance device 10 and information processing system 20 similar to those in the first embodiment.
  • the display unit 70 is a video device (HMD: Head Mounted Display) that is worn on the head of the user U.
  • HMD Head Mounted Display
  • a goggle-type or eyeglass-type HMD is used as the display unit 70.
  • the display unit 70 includes a detection device 71 and a display device 72.
  • the detection device 71 is a sensor that outputs a detection signal D according to the direction of the display unit 70.
  • the detection device 71 is composed of sensors such as a gyro sensor that detects angular velocity or an acceleration sensor that detects acceleration.
  • the display unit 70 is worn on the head of the user U, and therefore the detection signal D generated by the detection device 71 can also be expressed as a signal that indicates the direction of the user U's head.
  • the display device 72 displays images under the control of the information processing system 20.
  • various display panels such as a liquid crystal display panel or an organic EL panel are used as the display device 72.
  • the display device 72 is a non-transmissive display panel that does not transmit incoming light from the real space, and is placed in front of both eyes of the user U.
  • the display control unit 32 in the second embodiment displays a guide image G on the display device 72 of the display unit 70.
  • the guide image G in the second embodiment is a stereoscopic video composed of an image for the right eye and an image for the left eye. Therefore, the user U perceives a three-dimensional effect in the virtual musical instrument 50 and the playing character 60. As described above, since the guide image G is displayed on the display device 72 in the second embodiment, the display device 23 may be omitted.
  • the display control unit 32 controls the position and direction of the virtual camera (observation point Q) in the virtual space in response to the detection signal D supplied from the detection device 71. Therefore, the virtual line of sight of the user U in the virtual space (hereinafter referred to as the "observation line of sight") is controlled in response to the direction of the user U's head detected by the detection device 71.
  • the observation line of sight corresponds to the optical axis of the virtual camera.
  • the display control unit 32 of the second embodiment controls the movement of the playing character 60 displayed on the display device 72 in the same manner as in the first embodiment.
  • the display control unit 32 controls the guiding movement of the playing character 60 in the same manner as in the above-mentioned operation examples 1a to 1e. Therefore, the same effect as in the first embodiment is achieved in the second embodiment.
  • the display control unit 32 (second action control unit 322) of the second embodiment also causes the playing character 60 to perform an action (hereinafter referred to as a "reaction action") that reacts to the behavior of the user U. Specifically, the display control unit 32 controls the reaction action of the playing character 60 according to the observation line of sight of the user U. Specific examples of reaction actions are given below.
  • the second movement control unit 322 causes the playing character 60 to execute a reaction movement of moving the line of sight L toward the observation point Q in the virtual space.
  • the state in which the observation line of sight is directed toward the head of the playing character 60 means that the user U is staring at the playing character 60. Therefore, the reaction movement in the movement example 2a corresponds to the movement of the playing character 60 staring back at the user U in the virtual space, or the movement of the playing character 60 looking back at the user U.
  • the performance control unit 31 may control the speed at which the accompaniment part progresses (i.e., the performance tempo) according to the period of oscillation of the observation line of sight. For example, the performance control unit 31 increases the performance tempo the shorter the period of oscillation of the observation line of sight. Specifically, the performance control unit 31 controls the performance tempo of the accompaniment part so that the beat of the accompaniment part arrives at the end point of the oscillation of the observation line of sight. According to operation example 2b, it is possible to give the user U the sensation that the playing character 60 is playing in sync with the oscillation of the user U's head.
  • the content of the guidance action performed by the playing character 60 is not limited to the above examples.
  • the combination of the action condition that triggers the guidance action and the guidance action performed by the playing character 60 is arbitrary and is not limited to the examples in each of the above-mentioned forms.
  • the guidance actions exemplified below may be performed by the playing character 60.
  • the second operation control unit 322 causes the playing character 60 to perform a guidance operation that provides the user U with advice regarding playing.
  • the second movement control unit 322 causes the playing character 60 to execute a guidance action of informing the user U of the pitch to be played. Specifically, the second movement control unit 322 causes the playing character 60 to execute a guidance action of speaking while directing the line of sight L to the observation point Q. In parallel with the guidance action by the playing character 60, the second movement control unit 322 identifies the pitch to be played next by the user U from the melody data M1, and causes the sound emitting device 26 to reproduce a sound informing the user of the pitch. For example, a sound such as "Next, play 'C'" is reproduced from the sound emitting device 26. Therefore, the user U can be given the sensation that the playing character 60 is presenting the pitch to be played next by speaking. In other words, the user U can be provided with a unique customer experience of being instructed on playing by the playing character 60.
  • the second movement control unit 322 causes the playing character 60 to execute a guidance action to encourage the performance by the user U. Specifically, the second movement control unit 322 causes the playing character 60 to execute a guidance action of speaking while directing the line of sight L toward the observation point Q. In parallel with the guidance action, the second movement control unit 322 causes the sound emitting device 26 to reproduce a voice such as "Let's have more fun playing.” In other words, a unique customer experience in which the playing character 60 gives the user U motivation to continue playing can be provided.
  • the second movement control unit 322 causes the playing character 60 to execute a guidance action to notify the user of the performance mistake. Specifically, the second movement control unit 322 causes the playing character 60 to execute a guidance action of speaking while directing the line of sight L toward the observation point Q. In parallel with the guidance action, the second movement control unit 322 causes the sound emitting device 26 to reproduce a sound to notify the user U of the performance mistake. For example, a sound such as "The note you just played, 'C', was actually a 'D'" is reproduced from the sound emitting device 26. In other words, the user U can be provided with a unique customer experience of being instructed on performance by the playing character 60.
  • the second movement control unit 322 causes the playing character 60 to perform a guidance action to notify the user of the evaluation of the previous performance. Specifically, the second movement control unit 322 causes the playing character 60 to perform a guidance action of speaking while directing the line of sight L toward the observation point Q. In parallel with the guidance action by the playing character 60, the second movement control unit 322 evaluates the previous performance by the user U and causes the sound emitting device 26 to reproduce a sound notifying the user U of the evaluation result. For example, a sound such as "Your performance today was 50 points! is reproduced from the sound emitting device 26.
  • the second movement control unit 322 may also reproduce a sound that suggests to the user U a piece of music to be played next from the sound emitting device 26. For example, a piece of music similar in melody to the target piece of music played by the user U immediately before is proposed to the user U. As described above, the user U can be provided with a unique customer experience of enjoying playing a piece of music together with the playing character 60.
  • the second action control unit 322 causes the playing character 60 to execute a reaction action that reacts to the user U.
  • the reaction action is an action that changes the facial expression of the playing character 60 to an expression that corresponds to various emotions, such as "happiness” or "sadness.”
  • the reaction action is an example of a "guidance action.”
  • the second movement control unit 322 causes the playing character 60 to execute a reaction movement according to the length of time that the user U was able to play the melody part without making any performance errors (hereinafter referred to as "playing duration"). For example, the second movement control unit 322 makes the playing character 60's facial expression closer to "enjoyment” as the playing duration increases. According to the above aspect, it is possible to provide the user U with a unique customer experience in which the playing character 60 motivates the user U to reduce playing errors.
  • the second movement control unit 322 causes the playing character 60 to execute a reaction movement according to the number of playing mistakes made by the user U. For example, the second movement control unit 322 makes the facial expression of the playing character 60 closer to "sadness” or “anger” as the number of playing mistakes increases. According to the above aspect, as in aspect B1, it is possible to provide the user U with a unique customer experience in which the playing character 60 motivates the user U to reduce playing mistakes.
  • the performance analysis unit 33 may detect the performance position in the target song as the performance state of the user U, and the display control unit 32 may cause the playing character 60 to perform a guidance action, with the operating condition being that the performance position is within a specific section of the target song (hereinafter referred to as the "specific section").
  • the display control unit 32 may cause the playing character 60 to perform a guidance action when the user U is playing a specific section of the target song.
  • the contents of the guidance action are arbitrary.
  • the playing character 60 performs any of the guidance actions exemplified above while playing within the specific section.
  • the specific section is, for example, any one of a number of structural sections obtained by dividing the target song on the time axis according to musical meaning.
  • the structural sections are, for example, sections such as the intro, verse, bridge, chorus, and outro.
  • the specific section may also be set according to past performances by the user U. For example, a section in which the user U made a mistake in a past performance, or a section that the user U was able to play in the past may be set as the specific section.
  • a section specified by the user U by operating the operation device 24 may also be set as the specific section.
  • the playing character 60 performs a guiding action when the user U is playing a specific section of the target song, so that the user U's attention can be drawn to the specific section of the target song.
  • the user U can be provided with a unique customer experience in which the playing character 60's guiding action allows the user U to understand that a specific section of the target song is being played.
  • the information processing system 20 includes the sound source device 25 and the sound output device 26, but the sound source device 25 and the sound output device 26 may be mounted on the performance device 10.
  • an electronic musical instrument including the sound source device 25 and the sound output device 26 may be used as the performance device 10.
  • the performance state of the user U is analyzed from the time series of the performance data P generated by the performance device 10, but the configuration or method by which the performance analysis unit 33 analyzes the performance state is arbitrary and is not limited to the examples in each of the above-mentioned embodiments.
  • the performance analysis unit 33 analyzes the performance state using the acoustic signal generated by the sound pickup device.
  • the performance analysis unit 33 analyzes the performance state using the image signal generated by the imaging device.
  • the performance state may be analyzed using various sensors that detect the movements of the user U.
  • the display control unit 32 may display multiple playing characters 60 corresponding to different performance parts of the target song on the display device 23. For example, each playing character 60 plays a different type of instrument.
  • the display control unit 32 controls the movement of each of the multiple playing characters 60. For example, the display control unit 32 controls the direction of the line of sight L of each playing character 60.
  • the display control unit 32 controls the line of sight L of each of the multiple playing characters 60 in a direction toward an observation point Q in the virtual space. That is, a state in which the line of sight L of the multiple playing characters 60 is directed toward a virtual user U in the virtual space is realized.
  • the direction of the line of sight L of the multiple playing characters 60 is not limited to the direction of the observation point Q.
  • the direction of the line of sight L of each playing character 60 (e.g., the playing character 60 to which the line of sight L should be directed) may be set in advance for each playing character 60 according to a predetermined rule.
  • the line of sight L of the other playing characters 60 is directed toward a playing character 60 corresponding to a solo playing part among multiple playing parts of the target song.
  • the display control unit 32 may also control the operation of each of the multiple playing characters 60 depending on, for example, whether an operation condition related to the performance state is met. For example, when the performance by the user U stops for a predetermined period of time, the display control unit 32 controls the line of sight L of the multiple playing characters 60 to the direction of the observation point Q. In other words, when the user U hesitates to play, a situation is reproduced in which the multiple playing characters 60 all gaze at the user U.
  • the display control unit 32 (second operation control unit 322) sets the direction of the line of sight L of the playing character 60 to the direction of a reference point set in the virtual space.
  • the display control unit 32 also controls the direction of the line of sight L of the playing character 60 to the direction of one of a number of observation points Q in the virtual space. In other words, a state in which the playing character 60 is directing its head and line of sight L toward one of a number of virtual users U in the virtual space is reproduced.
  • the display control unit 32 controls the playing character 60 so that the line of sight L of the playing character 60 is directed to the observation point Q of a user U among the multiple users U whose playing state satisfies a predetermined operating condition.
  • the operating condition is, for example, that the playing state is a solo performance. That is, the display control unit 32 directs the line of sight L of the playing character 60, for example, to the observation point Q of a user U corresponding to a solo playing part among the multiple playing parts. That is, the playing character 60 performs an operation of directing its head and line of sight L to a user U among the multiple users U who is in charge of a specific playing part.
  • the display control unit 32 may also control the movement of the playing character 60 depending on the playing skill of each user U.
  • the performance analysis unit 33 calculates an evaluation score for each user U by comparing the music data M of the target music piece with the performance data P of each user U.
  • the evaluation score is an index related to the playing skill of the user U.
  • the display control unit 32 directs the line of sight L of the playing character 60 toward the observation point Q corresponding to the user U with the minimum or maximum evaluation score among the multiple users U.
  • the method of selecting the observation point Q to which the line of sight L of the playing character 60 is directed among the multiple observation points Q is not limited to the above examples and may be arbitrary.
  • multiple playing characters 60 corresponding to different playing parts may be displayed on the display device 23.
  • the virtual instrument 50 played by the playing character 60 is a keyboard instrument, but the type of virtual instrument 50 is not limited to a keyboard instrument.
  • any type of instrument such as a string instrument, a wind instrument, or a percussion instrument, may be displayed as the virtual instrument 50 played by the playing character 60. Note that the display of the virtual instrument 50 may be omitted.
  • the display unit 70 may be equipped with an imaging device that captures an image of the area in front of the user U's head in real space (i.e., in the direction of the line of sight).
  • the display control unit 32 displays the virtual musical instrument 50 and the playing character 60 on the display device 72 against a background of an image of real space captured by the imaging device.
  • the display control unit 32 displays the virtual musical instrument 50 and the playing character 60 using augmented reality (AR) or mixed reality (MR).
  • AR augmented reality
  • MR mixed reality
  • a transmissive display panel through which incoming light from real space passes may be used as the display device 72 of the display unit 70.
  • the virtual musical instrument 50 and the playing character 60 may be displayed against the background of an optical image formed by incoming light from real space.
  • the information processing system 20 may be realized by a server device that communicates with the performance device 10 via a communication network.
  • the information processing system 20 generates a guide image G according to the performance data P supplied from the performance device 10, and transmits image data of the guide image G to an information device of the user U.
  • the information device is, for example, a device such as a smartphone or a personal computer, and displays the guide image G according to image data received from the information processing system 20.
  • the display device 23 of the display unit 70 worn on the head of the user U may display the guide image G according to the image data received from the information processing system 20.
  • the functions of the information processing system 20 exemplified above are realized by the cooperation of one or more processors constituting the control device 21 and the program stored in the storage device 22.
  • the program according to the present disclosure can be provided in a form stored in a computer-readable recording medium and installed in a computer.
  • the recording medium is, for example, a non-transitory recording medium, and a good example is an optical recording medium (optical disk) such as a CD-ROM, but also includes any known type of recording medium such as a semiconductor recording medium or a magnetic recording medium.
  • a non-transitory recording medium includes any recording medium except a transient, propagating signal, and does not exclude volatile recording media.
  • the storage medium that stores the program in the distribution device corresponds to the non-transitory recording medium described above.
  • An information processing method analyzes the state of a performance by a user in real space, displays on a display device a playing character that performs performance actions in virtual space, and controls the guiding actions of the playing character according to the state of the performance by the user.
  • the guiding actions of the playing character that performs performance actions in virtual space are controlled according to the state of the performance by the user in real space. Therefore, the user can smoothly progress with the performance while visually checking the guiding actions of the playing character. In other words, it is possible to support the user's performance intuitively or interactively.
  • the “performance state” refers to the state of the user playing the musical instrument. For example, the state of whether the user is playing smoothly, the number of performance errors, the state of the user progressing/stopping the performance, etc. are analyzed as the “performance state.”
  • Virtual space is an imaginary space that is realized through various types of information processing, such as image processing.
  • real space is a space that actually exists in the real world, and is a concept that contrasts with virtual space.
  • a “performing character” is a virtual object (agent) that plays a musical instrument within a virtual space.
  • a specific example of a performing character is a virtual living being such as a human, but inanimate objects such as robots can also be included in the "performing character".
  • a “performing character” is a virtual user who plays a specific part in parallel with a different part being played by a user (i.e., playing a duet), but a virtual user who plays a piece of music independently of the performance by the user is also included in the "performing character”.
  • the "guiding action" of the performing character is an action for guiding the user's performance.
  • the guiding action is a movement of the performing character's body (e.g. posture) or a change in facial expression.
  • the performing character's “facial expression” is the expression (appearance) of a state of emotion, feeling, will, thought, etc. "Body movement” and “change in facial expression” are not necessarily clearly distinguishable concepts. In other words, it is conceivable that the "action” of the performing character may correspond to both "body movement” and "change in facial expression.”
  • Control according to the state of performance means that the presence or absence, or content, of the performing character's guiding action is linked to the state of the user's performance.
  • the performing character executes a guiding action when a certain condition is met regarding the state of the user's performance.
  • the process of temporarily changing the performing character's actions when a specific condition is met regarding the state of the performance is also included in "control according to the state of the performance.”
  • the guidance action is controlled by having the playing character execute the guidance action when a predetermined condition is met regarding the state of the performance by the user.
  • the playing character executes the guidance action when a predetermined condition is met regarding the performance by the user. Therefore, the user can intuitively or interactively grasp that a predetermined condition has been met regarding their own performance.
  • Specific conditions are various conditions related to the state of the performance.
  • examples of specified conditions include the user stopping playing, the music ending, the performance proceeding smoothly, and there being many performance errors.
  • the predetermined condition includes the performance being stopped for a predetermined period of time.
  • the line of sight of the playing character moves toward the observation point. Therefore, the user can intuitively or interactively grasp that the performance should continue.
  • the predetermined condition includes the end of the performance.
  • the playing character performs a guidance action. Therefore, the user can intuitively or interactively grasp the end of the performance.
  • the guiding action includes an action of the performing character moving his/her gaze toward an observation point in the virtual space.
  • the performing character moves his/her gaze toward the observation point.
  • the performing character gives the user a wink.
  • the user can intuitively or interactively grasp that there is something about the performance that he/she should pay attention to.
  • the "observation point” is the point in the virtual space where the viewpoint is set. In other words, the image of the virtual space observed from the observation point is displayed on the display device. Specifically, the “observation point” is the position in the virtual space where the virtual camera is set.
  • the "gaze" of a musical character when the musical character has one or more eyes, is a straight line extending in the direction of the eye.
  • a viewer e.g., a user
  • the movement of the musical character's gaze is an example of a process for controlling the facial expression of the musical character.
  • the guiding action includes an action of signaling the user to start playing the music.
  • the start of playing the music is signaled by the playing character. Therefore, the user can intuitively or interactively grasp the time when performance should start.
  • starting performance includes not only starting performance at the beginning of a piece of music, but also resuming performance from a long rest in the middle of a piece of music.
  • the performance action is an action in which the performing character plays a second part of the musical piece other than the first part, so as to follow the performance of the first part of the musical piece by the user.
  • the performing character plays the second part so as to follow the performance of the first part by the user. Therefore, it is possible for the user to perceive the sensation of playing an ensemble or duet with the performing character.
  • control of the guiding action includes having the playing character execute the guiding action when the user is playing a specific section of a piece of music.
  • the playing character executes a specific action when a specific section of a piece of music is being played, so that the user's attention can be drawn to the specific section of the piece of music.
  • a “specific section” is, for example, any one of a number of structural sections that divide a song on a time axis according to musical meaning.
  • Structural sections are, for example, sections such as the intro, verse, bridge, chorus, and outro.
  • a “specific section” may also be set according to a user's past performance. For example, a section in which a user made a mistake in a past performance, or a section that the user was able to play in the past, may be set as a "specific section.”
  • control of the guiding action further includes controlling the guiding action of the playing character in response to the line of sight of the user in the virtual space.
  • aspect 10 when the state in which the user's line of sight is directed toward the playing character in the virtual space continues for a predetermined period of time, the playing character is made to execute the guiding action of moving the line of sight of the playing character toward the user in the virtual space.
  • the musical performance character includes a head, and in controlling the guidance action, when the user's line of sight is shaken, the musical performance character is made to execute the action of shaking the head as the guidance action.
  • the performance by the user is a performance of a first part of a piece of music, and further, playback of a second part of the music other than the first part is controlled, and in controlling the playback, the progress speed of the playback of the second part is controlled according to the period of the user's gaze oscillation.
  • An information processing system includes a performance analysis unit that analyzes the state of a performance by a user in real space, and a display control unit that displays a performance character performing in a virtual space on a display device and controls the behavior of the performance character according to the state of the performance by the user.
  • a program according to one aspect (aspect 14) of the present disclosure causes a computer system to function as a performance analysis unit that analyzes the state of a performance by a user in real space, and a display control unit that displays a performance character performing in a virtual space on a display device and controls the behavior of the performance character according to the state of the performance by the user.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Electrophonic Musical Instruments (AREA)
PCT/JP2024/009447 2023-03-13 2024-03-11 情報処理方法、情報処理システムおよびプログラム Ceased WO2024190759A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2025506856A JPWO2024190759A1 (https=) 2023-03-13 2024-03-11
CN202480014672.XA CN120752695A (zh) 2023-03-13 2024-03-11 信息处理方法、信息处理系统及程序
US19/327,922 US20260050410A1 (en) 2023-03-13 2025-09-12 Information processing method, information processing system, and non-transitory computer-readable storage medium storing information processing program

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2023-038550 2023-03-13
JP2023038550 2023-03-13

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US19/327,922 Continuation US20260050410A1 (en) 2023-03-13 2025-09-12 Information processing method, information processing system, and non-transitory computer-readable storage medium storing information processing program

Publications (1)

Publication Number Publication Date
WO2024190759A1 true WO2024190759A1 (ja) 2024-09-19

Family

ID=92755201

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2024/009447 Ceased WO2024190759A1 (ja) 2023-03-13 2024-03-11 情報処理方法、情報処理システムおよびプログラム

Country Status (4)

Country Link
US (1) US20260050410A1 (https=)
JP (1) JPWO2024190759A1 (https=)
CN (1) CN120752695A (https=)
WO (1) WO2024190759A1 (https=)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0830807A (ja) * 1994-07-18 1996-02-02 Fuji Television:Kk 演奏連動型動画生成装置、音声連動型動画生成装置及びこれを利用したカラオケ装置
JP2019056871A (ja) * 2017-09-22 2019-04-11 ヤマハ株式会社 再生制御方法および再生制御装置
JP2020148921A (ja) * 2019-03-14 2020-09-17 株式会社タカラトミー 演奏装置
JP2021043258A (ja) * 2019-09-06 2021-03-18 ヤマハ株式会社 制御システム、及び制御方法
JP2022102913A (ja) * 2020-12-25 2022-07-07 株式会社コナミデジタルエンタテインメント ゲームシステム、それに用いるコンピュータプログラム、及び制御方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0830807A (ja) * 1994-07-18 1996-02-02 Fuji Television:Kk 演奏連動型動画生成装置、音声連動型動画生成装置及びこれを利用したカラオケ装置
JP2019056871A (ja) * 2017-09-22 2019-04-11 ヤマハ株式会社 再生制御方法および再生制御装置
JP2020148921A (ja) * 2019-03-14 2020-09-17 株式会社タカラトミー 演奏装置
JP2021043258A (ja) * 2019-09-06 2021-03-18 ヤマハ株式会社 制御システム、及び制御方法
JP2022102913A (ja) * 2020-12-25 2022-07-07 株式会社コナミデジタルエンタテインメント ゲームシステム、それに用いるコンピュータプログラム、及び制御方法

Also Published As

Publication number Publication date
US20260050410A1 (en) 2026-02-19
JPWO2024190759A1 (https=) 2024-09-19
CN120752695A (zh) 2025-10-03

Similar Documents

Publication Publication Date Title
US8696456B2 (en) Music-based video game with user physical performance
JP6137935B2 (ja) 身体動作評価装置、カラオケシステム、及びプログラム
US8858330B2 (en) Music video game with virtual drums
TWI470473B (zh) 電子娛樂系統中之相關姿勢反饋
CN111052223B (zh) 播放控制方法、播放控制装置及记录介质
Weinberg et al. Robotic musicianship: embodied artificial creativity and mechatronic musical expression
JP6515057B2 (ja) シミュレーションシステム、シミュレーション装置及びプログラム
JP4457983B2 (ja) 演奏操作援助装置及びプログラム
JP5377878B2 (ja) プログラム、情報記憶媒体、及びゲームシステム
WO2009007512A1 (en) A gesture-controlled music synthesis system
EP1229513A2 (en) Audio signal outputting method and BGM generation method
JP2001215963A (ja) 音楽演奏装置、音楽演奏ゲーム装置及び記録媒体
JP6070652B2 (ja) リファレンス表示装置およびプログラム
Zlatintsi et al. A web-based real-time kinect application for gestural interaction with virtual musical instruments
JP7810053B2 (ja) 情報処理システム、情報処理方法およびプログラム
Cosentino et al. Human–robot musical interaction
WO2024190759A1 (ja) 情報処理方法、情報処理システムおよびプログラム
JP3978506B2 (ja) 楽音生成方法
WO2025029338A1 (en) Modifying gameplay experiences
JP5750234B2 (ja) 音出力装置、音出力プログラム
JP2006259471A (ja) 歌唱練習システムおよび歌唱練習システム用プログラム
JP4648707B2 (ja) プログラム、情報記憶媒体及びゲームシステム
JP2004287144A (ja) 音楽再生と動画表示の制御装置およびそのプログラム
JP2024175437A (ja) 情報処理方法および情報処理装置
EP4693274A1 (en) Information processing device, information processing method, and program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24770864

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202480014672.X

Country of ref document: CN

ENP Entry into the national phase

Ref document number: 2025506856

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 2025506856

Country of ref document: JP

WWP Wipo information: published in national office

Ref document number: 202480014672.X

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 24770864

Country of ref document: EP

Kind code of ref document: A1