JP6862667B2 - Musical score display control device and program - Google Patents

Description

The present invention relates to a score display control device and a program for displaying a score of a musical score, and more particularly to a display that assists the practice when practicing the performance of a musical score using the score as a model.

Conventionally, some electronic musical instruments and the like can display a musical score image on a display based on the musical score data of a musical piece to be played. One of the important things in practicing playing a piece of music is to memorize it. It is necessary to practice repeatedly to memorize.

In the technique of electronically displaying a musical score on a display such as an electronic musical instrument, there is a technique of displaying the musical score so as to encourage a secret score. For example, Patent Document 1 describes that notes after the position of the performance are deleted from the score according to the progress of the performance (practice) of the music. In Patent Document 1, which note is to be erased is determined according to the difficulty of playing, for example, a phrase having a large number of notes or fingering is difficult.

Further, Patent Document 2 describes that when electronically displaying a score on a display of an electronic musical instrument or the like, various comments, instructions, etc. corresponding to the time position on the score are added in association with the time position. There is.

The difficulty of a piece of music can vary for individual users. Also, even for a single user, the parts of the song that are difficult to play, the parts that should be emphasized for practice, and the points to be noted when playing will change depending on the proficiency level of the song. ..

Japanese Unexamined Patent Publication No. 2008-003329 JP 2015-026069

The present invention has been made in view of the above points, and an object of the present invention is to provide a musical score display control device capable of efficiently and appropriately assisting a performance practice according to a user's proficiency in music performance. To do.

The present invention is based on a display unit that displays a score that serves as a model for a user's performance of a certain musical piece, an input unit that inputs sound data corresponding to the performance, and the score and the input sound data. A scoring unit for scoring the performance and a storage unit for accumulating the scoring results as a history of performances performed by the user in the past for the same music, and accumulating the history for a plurality of users. and stuff, based on the history of the storage unit stored the scoring result, and means for statistically determining the learning level for at least a portion of the musical composition, wherein the learning level to be determined, A display control unit that controls to change the display mode of the score for at least a part of the music according to the proficiency of the individual user and the proficiency of a user group composed of a plurality of users. The musical score display control device includes a device that controls how to change the display mode of the musical score according to the proficiency level of the user group.

According to the musical score display control device configured as described above, the scoring result of the performance based on the sound data generated by playing musical instruments, as history relates playing the user has performed in the past for the same said music, accumulate, and more This history is accumulated for the user of the music, and the proficiency level for at least a part of the music is statistically determined based on the history of the accumulated scoring result, and the proficiency level determined here is an individual user. It includes proficiency and user groups proficiency comprising a plurality users, the change of the musical score display mode for at least a portion of the music according to the individual user's proficiency level, and depending on the skill level of the user group It controls how to change the display mode of the musical score . Therefore, the proficiency level of individual users and the proficiency level of user groups consisting of a plurality of users can be appropriately determined based on the accumulated history, and the musical score display mode is changed according to the determined proficiency level of individual users. In addition, by controlling how to change the display mode of the score according to the proficiency level of the user group, the proficiency level of the user group is taken into consideration, and the performance practice to deepen the proficiency level is effectively assisted. It will be possible.

Further, the present invention can be implemented not only as an invention of the device, but also as an invention of a program for causing a computer to execute a step of realizing the function of the score display control device, or as an invention of a method of executing the step by the computer. And may be configured.

According to the present invention, the proficiency level of the user can be appropriately determined based on the accumulated history, and the score is displayed in an appropriate display mode according to the determined proficiency level. It has the excellent effect of being able to assist performance practice efficiently and appropriately according to the level of proficiency.

The conceptual block diagram explaining the structural example of the musical score display control apparatus which concerns on one Embodiment of this invention. A block diagram showing an electrical hardware configuration of a score display controller. The flowchart which shows the musical score display control processing example. An example of a table configuration that associates the degree of proficiency with the degree of change in display mode. (A) An example of a musical score displayed on the display unit, (b) An example of changing the display mode of musical notes in the musical score. An example of auxiliary display in the score. An example of a table configuration in which the proficiency level to which the auxiliary display threshold value is added and the degree of change in the display mode are associated. A flowchart showing an example of music score display control processing according to another embodiment.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a conceptual block diagram illustrating a configuration example of a score display control device according to an embodiment. The score display control device 100 is based on a display unit 10 that displays a score that serves as a model for performance, an input unit 11 that inputs sound data according to the performance, and the score and the input sound data. The display mode of the score is changed according to the scoring unit 12 for scoring the performance, the storage unit 13 for accumulating the scoring results, and the proficiency level based on at least a part of the scoring results accumulated in the storage unit. It includes a display control unit 14 that controls the music, a music data storage unit 15 that stores one or more music data, and a performance unit 16 that automatically plays based on the music data.

The score display control device 100 may be composed of a computer device capable of executing a program for performing operations (various processes for analyzing audio signals described later) of each part 10 to 16, such as a general-purpose personal computer or an electronic musical instrument. Alternatively, it may consist of a dedicated hardware device (integrated circuit or the like) configured to be able to execute the operation of each part.

FIG. 2 is a block diagram illustrating an example of an electrical hardware configuration of the score display control device 100. The score display control device 100 includes a microprocessor unit (CPU) 101, a memory 102, an operation device 103, a display device 104, an audio interface (“audio I / F”) 105, and a communication interface (“communication I / F”). ”) 106, including an external storage device 107. The CPU 101 executes various programs stored in the memory 102 to control the overall operation of the score display control device 100. The memory 102 includes a ROM, a RAM, and the like. The external storage device 107 is made of an appropriate recording medium such as a magnetic disk, an optical disk, or a flash memory.

The operation device 103 includes an operation device such as a mouse and a keyboard and a mechanism for detecting the operation input, and inputs various instructions such as selection of a musical piece, start and stop of automatic performance, and the like according to the operation of the user. .. The display device 104 corresponds to the display unit 10 of FIG. 1, and includes a display capable of displaying various screens including a musical score and a mechanism for controlling the display of the display. The display comprises, for example, an LCD or an organic EL.

The audio I / F 105 includes an input terminal for an acoustic signal (audio signal), an output terminal for the acoustic signal, an analog-to-digital converter, and a digital-analog converter. The score display control device 100 can acquire sound data generated by the performance of a user's musical instrument, for example, via the audio I / F 105. The audio I / F 105 constitutes the input unit 11.

The communication I / F 106 includes an Ethernet (registered trademark) interface, a USB interface, a MIDI interface, and the like. The score display control device 100 can be connected to a communication network such as the Internet via the communication I / F 106. The score display control device 100 can acquire MIDI performance information from an external MIDI device (for example, a MIDI keyboard or the like) via the communication I / F 106.

The music data storage unit 15 of FIG. 1 may be configured by, for example, an external storage device 107, or may be configured by a music database server device on the Internet connected via a communication I / F 106. The individual music data stored in the music data storage unit 15 is associated with the music, and the automatic performance data for automatically playing the music and the score display data for displaying the score of the music, respectively. including. The automatic performance data consists of, for example, MIDI format data. In one example, the automatic performance data included in the music data stored in the music data storage unit 15 is composed of "minus one performance data" excluding the musical instrument part played by the user from the automatic performance data. When the user sings, the minus one performance data is the data excluding the singing part (“Karaoke accompaniment data”). It is assumed that the user sets in advance the part names (singing, musical instrument names, etc.) in which he / she practices, in the score display control device 100.

The user uses the operation device 103 to select a music (practice song) to be practiced from the music data stored in the music data storage unit 15 (memory 102 in FIG. 2). When a song is selected by the user, the CPU 101 displays a score based on the score display data included in the music data of the selected song on the display unit 10 (display device 104 in FIG. 2). The musical score displays information necessary for the user to play or sing a musical piece, such as the pitch, time position, and length of each sound constituting the musical piece. An example of a musical score is a staff notation.

The user can instruct the start of playback of the selected music data by using the operation device 103. The performance unit 16 of FIG. 1 reproduces a minus one performance (or karaoke accompaniment) of the selected music based on the automatic performance data included in the selected music data in response to the playback start instruction. The CPU 101 can execute the operation of the performance unit 16 by reading the music data stored in the memory 102 or the like and performing an automatic performance process based on the read music data.

By the automatic performance of the performance unit 16, the reproduced sound of the minus one performance (or karaoke accompaniment) of the selected music is output to a headphone or the like (not shown) via the audio I / F 105. The user practices playing or singing a musical piece in accordance with the reproduced sound. The performance and singing of musical instruments are also collectively referred to as "performance".

The input unit 11 of FIG. 1 inputs an acoustic signal corresponding to the performance or singing of the acoustic instrument captured via the audio I / F 105 to the score display control device 100 as sound data according to the performance. In another example, the input unit 11 inputs MIDI performance information according to the performance of the electronic musical instrument captured via the communication I / F 106 to the score display control device 100 as sound data according to the performance.

The scoring unit 12 scores the performance of the musical instrument by the user or the singing of the user based on the sound data input from the input unit 11 and the reference (model) data acquired from the performance unit 16, and all in one practice. Generate scoring analysis data based on the scoring results of.

The reference data is data based on music data (automatic performance data in MIDI format) used by the performance unit 16 for automatic performance, and includes information for specifying a sound to be played. Information that identifies a sound includes, for example, various sound elements such as pitch (pitch), length, and sounding timing. The information for identifying the sound may also include various information necessary for the performance. The necessary information may include elements represented by performance symbols in the staff notation, such as strength, speed, idea, articulation, repetition, decoration, and omission.

The scoring of the scoring unit 12 includes, for example, evaluation of each sound of the music. One note corresponds to one note in the score. The scoring includes expressing the score according to the evaluation numerically. The evaluation items include evaluations related to various sound elements such as pitch, volume, length, and pronunciation timing, and evaluations related to performance techniques such as strength, speed, idea, articulation, repetition, decoration, and omission. The evaluation includes, for example, evaluating the stability of each item and evaluating the fidelity to the reference data. The CPU 101 operates the scoring unit 12 by analyzing the input sound data, extracting the feature amounts of various sound elements for each sound, and comparing the feature amounts of individual sounds with the reference data. obtain. The scoring by the scoring unit 12 may be performed by substantially real-time processing according to the user's performance, or may be performed on sound data (for example, data for one song) recorded from the performance of one practice. Is also good.

The scoring analysis data includes, for example, scoring result information, music identification information, user identification information, and individual scoring analysis data identification information. The scoring result information may include scores related to various sound elements of individual sounds constituting the music and scores related to the playing technique. The information for identifying a song is a song title of the song, a song ID for each song, or the like. The information that identifies the user is a user name, a user ID for each user, or the like. Information for identifying the scoring analysis data, for example, the date and time when scoring (practice) was performed, the practice ID for each practice, and the like.

The storage unit 13 is a database device that stores and manages the scoring analysis data generated by the scoring unit 12. In one example, the storage unit 13 may be composed of a local database device (external storage device 107) provided in the score display control device 100. The CPU 100 stores the scoring analysis data generated by the scoring unit 12 in the storage unit 13 for each practice. As a result, the storage unit 13 holds a history of scoring analysis data showing the scoring results for the exercises performed by the user in the past. As an example, the storage unit 13 holds only the history of the scoring analysis data of the user.

The display control unit 14 estimates the user's proficiency level regarding the performance of the song based on at least a part of the past scoring analysis data for each song stored in the storage unit 13, and according to the estimated proficiency level. Then, control is performed to change the display mode of the score displayed on the display unit 10 (display device 104). The proficiency level indicates whether or not the user is correctly playing the performance points in the music, such as individual notes and phrases in the music, that is, whether or not a high score is obtained. Further, as the proficiency level, information such as the total average score at the relevant place when the same song is played several times or the average identification within the latest predetermined period may be used. The change of the display mode of the score according to the proficiency level includes, for example, changing the display mode of the part corresponding to the proficiency level in the score according to the proficiency level of the user regarding each performance part in the music. .. For example, it includes changing the display mode of a highly proficient part. The change in the display mode includes simplifying the display of the location, changing the display color of the musical note at the location, making the location invisible, and the like.

FIG. 3 is a flowchart showing an example of the display control process (operation of the display control unit 14) performed by the CPU 101. In one example, the CPU 101 executes the process of FIG. 3 when displaying the score of a certain music on the display unit 10 (display device 104) according to the selection of the music by the user. In step S1, the CPU 101 determines the scoring analysis data for the performance of the music by the user based on the information for identifying the user (for example, the user name) and the information for identifying the currently selected music (for example, the music name). The group is acquired from the storage unit 13. As an example, all the scoring analysis data groups for the performance of the music by the user may be acquired from the storage unit 13. The individual scoring analysis data includes scoring results (scores) for each evaluation item such as pitch, volume, strength, etc. of each sound in the corresponding music.

In step S2, the CPU 101 estimates the user's proficiency level regarding the performance of the music based on the acquired scoring analysis data group. The proficiency level is estimated based on the statistical value of the score (scoring result) of each sound in the music included in the individual scoring analysis data. For example, the CPU 101 can estimate the proficiency level of individual sounds by obtaining the average value of the pitch scores of the individual sounds in the music. It is estimated that the higher the average value, the higher the proficiency level (that is, the player is playing correctly), and the lower the average value, the lower the proficiency level (that is, the player is not playing correctly). By doing this, it is possible to analyze the playing tendency of each note in the music, and to understand, for example, which note or phrase is often played incorrectly, which note or phrase is played correctly, and so on. Can be done. The playing tendency depends on the proficiency level and individuality of the user. From the performance tendency grasped in this way, it is possible to determine which part of the music should be changed in display mode. The CPU 101 records the proficiency level of each sound in the estimated music in association with a time position (for example, a note) on the corresponding musical score. As a result, aggregated data of the scoring analysis data group corresponding to the selected music is created. The aggregated data is data in which the proficiency level of individual sounds in a musical composition based on a scoring analysis data group is recorded in association with a time position (for example, a note) on a corresponding musical score.

In step S3, the CPU 101 controls to change the display mode of the corresponding portion on the score according to the proficiency level of each sound in the music based on the created aggregated data. When the aggregated data is created based on all the scoring analysis data groups for the performance of the music by the user, the display control unit 14 (step S3) has all the scoring of the music accumulated in the storage unit 13. Based on the result of the above, control for changing the display mode will be performed.

As an example, the control for changing the display mode may be performed when displaying the score including the note on the display unit 10, that is, without interlocking with the progress of the minus one performance by the performance unit 16. When displaying the score on the display unit 10, the CPU 101 changes individual corresponding parts in the score (for example, musical notes and the like where the proficiency level is obtained) to a display mode according to each proficiency level. In this case, each corresponding portion in the score is fixedly displayed in a display mode according to the proficiency level when displayed on the display unit 10. As another example, the control for changing the display mode of the corresponding part on the score is performed in substantially real time according to the progress of the minus one performance by the performance unit 16, that is, according to the timing when the part is played by the performance unit 16. Do. In that case, time information indicating the progress of the automatic performance is supplied from the performance unit 16 to the display unit 10 and the display control unit 14 (see FIG. 1), and the CPU 101 displays a location corresponding to the current performance position based on the time information. The aspect is changed according to the proficiency level. As another example, the CPU 101 changes the display mode of the part after the current performance position based on the time information (for example, a few bars ahead, a few seconds ahead, a few notes ahead, etc.) according to the proficiency level. You may. As a result, the display mode of the part to be played can be changed at the timing according to the user's performance in accordance with the automatic performance (minus one performance) or prior to the performance.

Corresponding parts on the score are, for example, musical notes. The change in the display mode includes, for example, changing the display color and display size of the notes according to the average pitch score of each sound. As an example, the memory 102 of the score display control device 100 stores a table in which the proficiency level (average value of pitch scores) of individual sounds in the music and the degree of different display modes are associated with each other, and the CPU 101 stores the table. In step S3, based on the table, the display mode of the corresponding portion on the score is changed to a different degree depending on the proficiency level.

FIG. 4A shows an example of a table configuration in which the average value of pitch scores and the density of the display color of musical notes are associated with each other. In the table configuration example of (a), the vertical axis represents the average pitch score. What is the average pitch score? In the figure, the values from the lowest score (0 points) to 100 points (highest score) are taken from the bottom to the top. The horizontal axis indicates the density of the display color, and the density decreases from left to right in the figure. The state in which the density is highest is defined as a normal display color in which the display mode has not been changed. According to the table configuration example of (a), the notes having a low proficiency level from 0 to 50 points do not change with the darkest display color density. For notes with 50 points or more, the density of the display color gradually decreases as the score increases. Therefore, the display color of notes with low proficiency remains unchanged in the normal state. As the proficiency level increases, the display color of the notes becomes lighter and the visibility of the notes deteriorates. The lightest display color, for example, makes the notes invisible (the notes disappear). For example, FIG. 5A is an example of a musical score (staff) displayed on the display unit 10 (display device 104). When the display of the three notes 50, 51, and 52 in the third measure disappears by the process of step S3, the score becomes as shown in FIG. 5 (b).

In one example, in step S3, the overall display mode of the notes is changed. In another example, the display mode of only a part of the note, not the whole note, is changed. For example, only the display mode of the note head of a note may be changed. For example, in FIG. 5B, reference numeral 53 indicates an example in which the note head of a note is erased. In that case, since the stem and stem are displayed in the normal mode even after the display mode is changed, the user can know the pronunciation timing. As another example, only the display mode of the stem and stem of a note may be changed. In that case, since the stem is displayed in the normal mode even after the display mode is changed, the user can know the pitch. For example, in FIG. 5B, reference numeral 54 shows an example in which the stem of a note is erased.

In step S3, the method of changing the display mode is not limited to changing the density of the display color of the musical note, for example, changing the display color of the musical note according to the proficiency level (for example, the display color if the proficiency level increases). Change from red to black and leave it red if the proficiency level does not increase), change the transparency of the note display (for example, if the proficiency level increases, the note display will be displayed transparently, and the proficiency level will increase. If it does not increase, it remains opaque, etc.), or change the display size of the notes (for example, if the proficiency level increases, the note display size is reduced, and if the proficiency level does not increase, the size remains large, etc.). May include.

Further, another example of the method of changing the display mode in step S3 includes performing mosaic processing on the musical notes. In that case, for example, as the proficiency level increases, the CPU 101 coarsens the particle size of the mosaic processing so that the visibility of the notes deteriorates. Further, another example of the method of changing the display mode in step S3 includes performing blinking processing on the musical notes. In that case, for example, as the proficiency level increases, the CPU 101 lengthens the turn-off time during the blinking cycle so that the visibility of the notes deteriorates.

Further, another example of the method of changing the display mode in step S3 includes hiding at least a part of the musical notes by, for example, black-painting processing, shading processing, or superimposing an image of some pattern. In that case, for example, as the proficiency level increases, the CPU 101 increases the visibility of the notes by increasing the density of the black coating, making the grain size of the shading (mesh pitch) finer, expanding the range of hiding the notes, and the like. Make it worse. For example, in FIG. 5B, reference numeral 55 shows an example in which notes are hidden by blackening.

The place where the display mode is changed in step S3 is not limited to the musical note, but may be a display object other than the musical note, or a time position on the musical score (background portion of the musical score). For example, when displaying the scale name of each sound and the fingering for playing the sound (for example, the position of the keyboard and the fingering of a guitar, wind instrument, etc.) on the score, the scale name and fingering The display mode may be changed. FIG. 4B shows another example of the table configuration referred to in step S3, which is a table configuration in which the average pitch score is associated with the scale name and the display transparency of fingering. According to (b), as the average pitch score (proficiency level) increases, the scale name and fingering are transparently displayed and invisible.

Further, in another embodiment, the table in which the proficiency level referred to in step S3 and the change degree of the display mode are associated may be added with an offset value according to the difficulty level of each note in the music. .. The difficulty level for each note can be estimated based on the reference data. Thereby, for example, it is possible to control the change of the display mode reflecting the difficulty level, such as making it difficult to change the display mode even if the proficiency level is high in a part where the performance is difficult.

Further, the step S2 is not limited to estimating the proficiency level regarding the pitch of each sound in the music, and various evaluation items (for example, pitch, volume, pronunciation timing, strength, speed, idea) of each sound in the music. , Articulation, repetition, decoration, omission, etc.) is obtained as the proficiency level, and step S3 is performed on the score according to the proficiency level of the one evaluation item of each sound. It may be configured to change the display mode of the corresponding portion of.

In another embodiment, the step S2 may be configured to obtain the average value of a plurality of types of evaluation items of individual sounds in the music as the proficiency level. In that case, step S3 is configured to change the display mode of the corresponding portion on the score according to the total proficiency level (average score) for each of one or more types of evaluation items of each sound. You can do it. In another example, the step S3 may be configured to change the display mode of the corresponding portion on the score according to the total weighted to the individual evaluation items. In another embodiment, the method of changing the display mode according to the proficiency level may be different for each of the plurality of types of evaluation items. For example, the density of the note display color is changed according to the average pitch score, the note display color is changed according to the average volume score, and the blinking process is performed according to the average sounding timing score. is there.

Further, in another embodiment, the step S3 is a staff symbol, a clef symbol, a time signature, a rest, a key signature, an accidental symbol, a dynamic symbol, a speed symbol, an idea symbol, an articulation symbol, a repetition symbol, and a decoration. It may be configured to change the display mode of various symbols necessary for playing, such as symbols and various playing style instruction symbols. When changing the display mode of the symbols indicating these various playing techniques, refer to the evaluation by comparison with the reference data (for example, whether the performance was performed according to the model) and the score based on the evaluation of the length, volume, etc. of the sound. To do.

Further, the step S3 may be configured to change, for example, the display mode of the lyrics information and the chord (chord) information.

Further, in another embodiment, the step S2 estimates the proficiency level for the performance over a plurality of sounds in the music, and the step S3 changes the display mode according to the proficiency level for the performance over the plurality of sounds. It may be configured as follows. A performance over multiple sounds is, for example, a performance of a chord consisting of multiple sounds, a performance of a certain phrase, a performance of a specific pattern such as an arpeggio, or a performance of a specific section in a music, another specific from a specific pitch. The performance that shifts to the pitch of, the performance for each scale (for example, the performance of do, the performance of Les, etc.), the treble range played by the fradiolet playing method in a specific range (for example, sax, etc.), and the bridge range in the flute. Etc.), including the performance of multiple sounds belonging to.

Further, in another embodiment, in step S2, a scoring analysis data group for the performance of a plurality of musical pieces is acquired from the storage unit 13, and the accuracy of each pitch and the sound change are obtained from the acquired scoring analysis data. The proficiency level regarding the accuracy of each pattern may be estimated, and the step S3 may be configured to change the display mode of the corresponding note according to the proficiency level. The sound change pattern is, for example, an arpeggio pattern that changes in the order of do, mi, and so. In this case, it is possible to change the display mode according to the proficiency level of the performance for each pitch of the user or for each sound change pattern, instead of the proficiency level of the performance for each song.

Further, in another embodiment, the proficiency level estimated in step S2 is obtained by performing a low-pass filter process on the scoring analysis data group and extracting a certain score (for example, a low score) for each individual sound in the music. It may be estimated. Further, as another example, the proficiency level estimated in step S2 may be the median score for each sound in the music included in the individual scoring analysis data.

Further, in another embodiment, the scoring analysis data group acquired in step S1 is a scoring analysis data group within the latest predetermined period among the scoring analysis data for the performance of the music by the user. The scoring analysis data group within the latest predetermined period is a scoring analysis data group for a certain period in the past, for example, a scoring analysis data group within one month. Another example of the scoring analysis data group within the latest predetermined period is the scoring analysis data group for the last few times. In this case, the display control unit 14 (previous step S3) displays based on the scoring result within the latest predetermined period of the scoring analysis data group for the performance of the user of the practice song accumulated in the storage unit. Control to change the mode.

Further, in another embodiment, when the display mode of step S3 is changed to the position after the current performance position in accordance with the progress of the automatic performance of the minus one performance of the performance unit 16, the display mode is further changed. The timing of making the change may be determined according to the proficiency level. For example, as the proficiency level of the relevant part increases, the display mode may be changed at a time farther from the current playing position. As a result, for example, when the proficiency level of the location is low, the display mode is changed immediately before the performance timing of the location, and as the proficiency level of the location increases, the display mode significantly precedes the performance timing of the location. That is, the time during which the display mode of the relevant portion is changed can be lengthened.

In short, in step S3, if the control is such that the score is simplified, the visibility is deteriorated, or the score is invisible depending on the proficiency level, the display mode is changed by any method. It may be changed. By deteriorating the visibility of highly proficient notes and, by extension, erasing the notes, it can be expected to have the effect of encouraging the user to memorize the relevant part. Since the location for prompting the memorization is determined according to the proficiency level of the user, it is possible to provide the user with efficient and effective practice assistance (memorization assistance).

In one embodiment, the step S3 may be further configured to perform an auxiliary display suggesting that the display mode has been changed to the location where the display mode has been changed. FIG. 6 shows an example of the auxiliary display displayed on the score. In FIG. 6, as an example of changing the display mode, the notes 50, 51, 52 (see FIG. 5 (a)) in the third measure are erased, and the location is hidden by the question mark symbol 56. The balloon image 60 is displayed for the display change portion. In the balloon image 60, character information of "quarter note" is displayed as hint information suggesting erased notes (in other words, hint information suggesting a display mode before change). Although the notes at the display change portion are invisible, only the note value at the location can be known to the user by the balloon image 60 of the auxiliary display.

As an example, the CPU 101 performs the auxiliary display based on a comparison between the proficiency level of the portion where the display mode is changed and a predetermined threshold value. FIG. 7 shows a table in which the auxiliary display threshold value 70 is added to the association table of the average pitch score and the density of the display color of the musical note shown in FIG. 4 (a). In the table of FIG. 8, as the auxiliary display threshold value 70, a section for performing auxiliary display is set in a numerical range of a certain pitch score average value. As an example, when the proficiency level (pitch score average value) of the portion where the display mode is changed is included in the range of the auxiliary display threshold value 70, the CPU 101 performs auxiliary display on the portion where the display mode is changed.

In one embodiment, the auxiliary display threshold 70 does not have to be a fixed value. For example, the auxiliary display threshold value 70 is determined based on at least a part of the scoring analysis data accumulated in the storage unit 13. As an example, the auxiliary display threshold 70 may be determined based on all scoring analysis data for the performance of the music by the user. As another example, the auxiliary display threshold 70 may be determined based on all scoring analysis data for the performance of the song by the user. Further, in another example, the auxiliary display threshold value 70 may be determined based on the scoring analysis data group within the latest predetermined period among the scoring analysis data for the performance of the music by the user. The scoring analysis data group within the latest predetermined period is, for example, a scoring analysis data group for a certain period in the past, or a scoring analysis data group for the latest several times.

Other examples of hint information presented as ancillary displays are chord names, whether notes have accidentals, showing some of the invisible notes, and musical components (eg, intros, chorus melody, etc.). ) May be specified, the playing technique may be specified, and the like.

The auxiliary display is not limited to the balloon image 60 of FIG. 6, and may be in any format as long as the user can recognize it. For example, character information that serves as an auxiliary display may be displayed in a choice format in association with the display change location. The choice format is, for example, in the example of FIG. 6, as information suggesting a note value as an auxiliary display in association with the question mark symbol 56, for example, "(1) half note", "(2)). It includes presenting several options, such as "quarter note" and "(3) eighth note". The options displayed may include incorrect answers. Further, another example of the auxiliary display includes displaying information indicating an inaccurate performance content at a display change portion. Displaying information indicating inaccurate performance content is, for example, displaying the note of "re" on the score where the note indicated by the reference data is "do", or the balloon image 60 of FIG. Including displaying inaccurate content such as "Re?". The erroneous answer included in the options and the "inaccurate performance content" displayed as an auxiliary display may be determined based on, for example, a mistake in the performance performed by the user in the past. The user's past mistakes can be determined based on the scoring analysis data. Further, another example of the auxiliary display may be a quiz format in which the user is asked the correct performance content of the display change portion.

In one embodiment, the auxiliary display may be displayed at a corresponding portion on the score when the score is displayed on the display unit 10. When displaying the score on the display unit 10, the CPU 101 performs auxiliary display according to each proficiency level at each corresponding part (for example, a musical note or the like where the proficiency level is obtained) in the displayed score. .. In this case, the auxiliary display is fixedly displayed at the corresponding portion on the score displayed on the display unit 10. Further, in another embodiment, the auxiliary display may be performed according to the progress of the automatic performance of the performance unit 16. For example, the CPU 101 may perform an auxiliary display on a place where the auxiliary display should be performed when the user plays the place based on the time information indicating the progress of the automatic performance of the performance unit 16. As another example, the CPU 101 displays the auxiliary display for the part before the user plays the part to be the auxiliary display, that is, after the current performance position, based on the time information indicating the progress of the automatic performance of the performance unit 16. You may go. The timing of performing the auxiliary display according to the progress of the automatic performance may be determined, for example, according to the proficiency level of the relevant portion. As a result, the auxiliary display can be performed on the part to be played at the timing according to the user's performance in accordance with the automatic performance (minus one performance) or prior to the performance.

In one embodiment, the content of the auxiliary display may be changed according to the proficiency level. For example, in a place where the proficiency level is relatively low, more hint information can be provided as an auxiliary display, and the hint information can be reduced as the proficiency level increases.

The auxiliary display may also be transmitted by sound. For example, the CPU 101 illustrates the sound related to the auxiliary display via the audio I / F 105 before the user plays the portion to be the auxiliary display based on the time information indicating the progress of the automatic performance of the performance unit 16. Output to outside headphones, etc. The sound related to the auxiliary display is, for example, a warning that the display mode has been changed, a sound name whose display mode has been changed, or the like. Further, in another embodiment, the auxiliary display may transmit to the user that the display mode has been changed by light such as light emission or blinking.

By performing the above-mentioned auxiliary display at a place where the display mode is changed according to the proficiency level, it is possible to provide an effective practice assistance (for example, a secret score assistance) to the user.

In another embodiment, the storage unit 13 may be configured by a database server device on the Internet connected via communication I / F 106. In this case, the scoring unit 12 stores the scoring analysis data for the performance of the music by the user in the database server device (accumulation unit 13) via the Internet. The database server device (storage unit 13) can collect scoring analysis data from various terminal devices. Therefore, for example, when a user uses a plurality of different score display control devices 100, the scoring analysis data from any of the devices 100 is scored by the user in one database server device (accumulation unit 13). It is accumulated as analysis data (performance history).

In another embodiment, the storage unit 13 may be configured to store and manage scoring analysis data of a plurality of users other than the user. For example, the storage unit 13 is composed of a database server device on the Internet connected via the communication I / F 106, acquires scoring analysis data from a plurality of users via the Internet, and performs scoring analysis of the acquired plurality of users. Accumulate and manage data. In this case, the proficiency level is estimated by including the scoring analysis data of the user and the scoring analysis data of a plurality of other users, or the proficiency level is estimated based on the scoring analysis data of a plurality of other users. Depending on the situation, the process (display mode change and auxiliary display) of step S3 in FIG. 3 may be performed.

FIG. 8 is a flowchart showing a processing example in which the database server device constituting the storage unit 13 generates and transmits the aggregated data used for the processing of FIG. 3 based on the scoring analysis data of a large number of users. For example, the score display control device 100 sends aggregated data related to the selected music to the database server device via the Internet in response to a user's instruction to select a music (score display instruction or automatic performance start instruction). Request. In response to the request, the database server device performs the process shown in FIG.

In step S4, the database server device identifies the scoring analysis data group corresponding to the requested music from the storage unit 13. In step S5, the database server device is based on the specified scoring analysis data group, for example, a statistical value (for example, an average value, a median value, a deviation value, etc.) of scores for each evaluation item of individual sounds in a musical piece. By obtaining, the proficiency level of the performance is estimated, and the proficiency level of each sound in the estimated music is recorded in association with the time position (for example, a note) on the corresponding musical score. As a result, aggregated data of the scoring analysis data group corresponding to the requested music is created. The aggregated data is data in which the proficiency level of individual sounds in a musical composition based on a scoring analysis data group is recorded in association with a time position (for example, a note) on a corresponding musical score. In step S6, the database server device transmits the aggregated data to the score display control device 100. The score display control device 100 changes the display mode and performs auxiliary display in step S3 based on the aggregated data acquired from the database server device.

In one embodiment, the aggregated data creation process of step S5 may include determining the priority of the portion where the display mode is changed. For example, in a large number of users, notes with lower proficiency are given higher priority. In generating aggregated data based on the scoring analysis data of a large number of users, more effective performance assistance is provided by preferentially changing the display mode in places where the proficiency level is lower for a large number of users. It is expected.

As an example, the database server device generates aggregated data based on all the identified scoring analysis data (S5). In this case, the CPU 101 of the score display control device 100 changes the display mode and / or performs the auxiliary display in step S3 according to the degree of learning of performance by all users including the user. As another example, the database server device generates aggregated data based on the scoring analysis data of a user group including a plurality of users including the user among all the specified scoring analysis data (S5). In this case, the CPU 101 of the score display control device 100 changes the display mode and / or performs the auxiliary display in step S3 according to the degree of learning of the performance in the user group including the user. As another example, the database server device generates aggregated data based on the scoring analysis data of a user group including a plurality of users not including the user among all the specified scoring analysis data (S5). In this case, the CPU 101 of the score display control device 100 changes the display mode and / or performs the auxiliary display in step S3 according to the degree of learning of the performance in the user group consisting of a plurality of users including the user. As a result, the display mode can be changed and / or the auxiliary display (S3) can be performed by another person according to the proficiency level including the scoring result for the performance. As a result, for example, the display mode of the part with low proficiency level is preferentially changed by other users other than itself, and conversely, the display mode of the part with low proficiency level is not easily changed by other users. Auxiliary display can be displayed in places where the level of proficiency is low. The user group consisting of the plurality of users is a specific user group such as a member belonging to the same orchestra, a practice instructor and a student, or a designated performer (for example, a professional performer who serves as a model). You can do it.

Further, in another embodiment, the CPU 101 of the score display control device 100 is created by the database server device in step S5 with respect to the table in which the proficiency level referred to in step S3 and the degree of change in the display mode are associated with each other. An offset value according to the proficiency level indicated by the aggregated data may be added. As a result, it is possible to control the change of the display mode to reflect the playing tendency of a large number of users, such as making it difficult for the display mode to change even if the user's proficiency level is low, for example, in a place where the score is low among a large number of users. ..

Further, in another embodiment, the CPU 101 of the score display control device 100 has the proficiency level and the display mode referred to in the step S3 according to the proficiency level indicated by the aggregated data created by the database server device in the step S5. The auxiliary display threshold value 70 of the table (FIG. 7) associated with the degree of change may be changed. For example, by widening the auxiliary display threshold value 70 for a portion having a low proficiency level in a large number of users, the auxiliary display can be easily performed. This makes it easier for the user to pay attention to the relevant part. When the CPU 101 of the score display control device 100 performs the auxiliary display according to the proficiency level (S3), the CPU 101 refers to the auxiliary display threshold value included in the supplied aggregated data and the proficiency level (average score) to display the auxiliary display. Do.

Although one embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and various aspects are described within the scope of claims and the technical ideas described in the specification and drawings. It can be transformed. For example, the various components, various examples, various modifications, and different embodiments described above may be combined in any combination. In addition, the score is not limited to the staff notation, but also the tablature of the guitar, "piano roll" in which symbols indicating individual pronunciation events are arranged in association with the keyboard position and time position, drum notation, chord notation, lyrics display, fingering. Anything can be used as long as it displays information necessary for the user to play or sing a song, such as a tablature indicating. The lyrics display is, for example, one in which the lyrics subtitles are scrolled and displayed according to the automatic performance.

Further, the score display control device 100 of the present invention may be configured so that at least a part of each part 10 to 16 of FIG. 1 is distributed by one or more computer devices on the network. Further, according to the present invention, the performance is performed based on the step of displaying the score as a model of the performance on the computer, the step of inputting the sound data according to the performance, and the score and the input sound data. Based on the step of scoring, the step of accumulating the scoring results, and the scoring results of at least a part accumulated in the accumulating portion, the display mode of the highly proficient part in the score is displayed other than that. It may be configured and implemented as an invention of a program that performs a step of controlling to be modified to be different from a portion. Further, the present invention may be configured and implemented as an invention of a method including each step of the program.

100 Score display control device, 10 display unit, 11 input unit, 12 scoring unit, 13 storage unit, 14 display control unit, 15 music data storage unit, 16 performance unit

Claims (12)

A display unit that displays a score that serves as a model for the user to play a certain piece of music.
An input unit for inputting sound data according to the performance, and
A scoring unit for scoring the performance based on the score and the input sound data,
A storage unit that accumulates the scoring results as a history of performances performed by the user in the past for the same song, and stores the history for a plurality of users .
A means for statistically determining the proficiency level of at least a part of the music based on the history of the scoring results accumulated in the storage unit, and the proficiency level determined here is determined by an individual user. Includes proficiency and proficiency in user groups consisting of multiple users
A display control unit that controls to change the display mode of the score for at least a part of the music according to the proficiency level of the individual user, and is a display control unit of the display mode of the score according to the proficiency level of the user group. A musical score display control device that controls how to change. The display control unit includes a table in which the degree of proficiency is associated with the degree of different display modes, and the degree of proficiency varies depending on the degree of proficiency of the individual user based on the table. the is configured to change the display mode, and musical score display control apparatus according to claim 1, associating the contents of the table is characterized that you offset depending on the proficiency of the user group. The means for determining the proficiency level determines the proficiency level for each individual sound included in the musical piece.
The display control unit is characterized in that it controls to change the display mode of the portion corresponding to each sound in the score according to the proficiency level of the individual user for each individual sound. 2. The musical score display control device according to 2. The means for determining the proficiency level determines the proficiency level for at least one or more sections over a plurality of sounds included in the music.
Any one of claims 1 to 3, wherein the display control unit controls to change the display mode for each of at least one or more sections according to the proficiency level of the individual user with respect to the section. The score display control device described in. The score display control device according to any one of claims 1 to 4, wherein the control by the display control unit impairs the visibility of at least a part of the score according to the proficiency level of the individual user. The score display control device according to claim 5, wherein the control by the display control unit includes blindfolding at least a part of the score with an image of some symbol according to the proficiency level of the individual user. The score display control device according to any one of claims 1 to 6, wherein the control by the display control unit includes changing at least a part of the display colors in the score according to the proficiency level of the individual user. The display control unit is further configured to perform an auxiliary display suggesting that the display mode has been changed to a portion where the display mode has been changed. The score display control device according to any one of 7. The score display control device according to claim 8, wherein the display control unit is configured to display hint information suggesting a display mode before the change as the auxiliary display. The musical score display according to claim 8 or 9, wherein the display control unit changes the portion where the display mode is changed to a display indicating an erroneous performance different from the model as the auxiliary display. Control device. On the computer
For a song, the step of displaying a score that serves as a model for the user to play,
The step of inputting sound data according to the performance and
A step of scoring the performance based on the score and the input sound data,
A step of accumulating the scoring result as a history of performances performed by the user in the past for the same music, that is, accumulating the history for a plurality of users .
Based on the history of the accumulated scoring results, the step of statistically determining the proficiency level for at least a part of the music, and the proficiency level determined here are the proficiency level of individual users and the proficiency level. Includes proficiency in user groups consisting of multiple users
It is a step of controlling to change the display mode of the score for at least a part of the music according to the proficiency level of the individual user, and the display mode of the score is changed according to the proficiency level of the user group. A program that controls how it is done and <br /> executes it. For a song, the step of displaying a score that serves as a model for the user to play,
The step of inputting sound data according to the performance and
A step of scoring the performance based on the score and the input sound data,
It is a step of accumulating the result of the scoring as a history related to the performance performed by the user in the past for the same music, and accumulating the history for a plurality of users .
Based on the history of the accumulated scoring results, the step of statistically determining the proficiency level for at least a part of the music, and the proficiency level determined here are the proficiency level of individual users and the proficiency level. Includes proficiency in user groups consisting of multiple users
It is a step of controlling to change the display mode of the score for at least a part of the music according to the proficiency level of the individual user, and the display mode of the score is changed according to the proficiency level of the user group. How to control how and how to do <br /> with a computer.

Images