WO2023153033A1 - Procédé de traitement d'informations, programme et dispositif de traitement d'informations - Google Patents

Procédé de traitement d'informations, programme et dispositif de traitement d'informations Download PDF

Info

Publication number
WO2023153033A1
WO2023153033A1 PCT/JP2022/040701 JP2022040701W WO2023153033A1 WO 2023153033 A1 WO2023153033 A1 WO 2023153033A1 JP 2022040701 W JP2022040701 W JP 2022040701W WO 2023153033 A1 WO2023153033 A1 WO 2023153033A1
Authority
WO
WIPO (PCT)
Prior art keywords
performance
musical
musical score
sound
symbol
Prior art date
Application number
PCT/JP2022/040701
Other languages
English (en)
Japanese (ja)
Inventor
秀一 松本
Original Assignee
ヤマハ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ヤマハ株式会社 filed Critical ヤマハ株式会社
Publication of WO2023153033A1 publication Critical patent/WO2023153033A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10G—REPRESENTATION OF MUSIC; RECORDING MUSIC IN NOTATION FORM; ACCESSORIES FOR MUSIC OR MUSICAL INSTRUMENTS NOT OTHERWISE PROVIDED FOR, e.g. SUPPORTS
    • G10G1/00—Means for the representation of music
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L13/00—Speech synthesis; Text to speech systems

Definitions

  • an information processing method is implemented by a computer system, and based on musical score data representing a musical score including one or more performance symbols, sounds related to the performance symbols are generated. Generates an acoustic signal representing the
  • a program causes a computer system to function as a generation unit that generates an acoustic signal representing a sound related to one or more performance symbols based on musical score data representing a musical score including one or more performance symbols.
  • An information processing apparatus includes a generation unit that generates an acoustic signal representing a sound associated with one or more performance symbols based on musical score data representing a musical score including one or more performance symbols.
  • FIG. 4 is a diagram schematically showing processing of a text generation unit 32; FIG. FIG. 4 is a diagram illustrating a musical score; FIG. 10 is a diagram schematically showing read-out timings of read-out text; FIG. 10 is a diagram schematically showing read-out timings of read-out text; FIG. 11 is a diagram illustrating a display screen during reading of a musical score; 4 is a flowchart illustrating a specific procedure of processing for the control device 11 to execute a musical score reading application. 4 is a diagram exemplifying an instruction reception screen by an instruction reception unit 30.
  • FIG. FIG. 11 is a diagram illustrating a display screen during execution of a table-of-contents presentation function; FIG. 11 is a block diagram illustrating a functional configuration of a control device 11A in a third embodiment; FIG.
  • the control device 11 is composed of one or more processors that control each element of the information processing device 10 .
  • the control device 11 includes a CPU (Central Processing Unit), an SPU (Sound Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or an ASIC (Application Specific Integral). one or more types such as rated circuit) It consists of a processor.
  • the storage device 12 is a single or multiple memories that store the program PG (see FIG. 2) executed by the control device 11 and various data used by the control device 11 .
  • the storage device 12 is composed of a known recording medium such as a magnetic recording medium or a semiconductor recording medium, or a combination of a plurality of types of recording media.
  • a portable recording medium that can be attached to and detached from the information processing device 10, or a recording medium (for example, cloud storage) that can be written or read by the control device 11 via a communication network is used as the storage device 12.
  • the sound collection device 13 detects ambient sounds (air vibrations) and outputs them as acoustic signals.
  • the sound pickup device 13 is, for example, a microphone. Note that the sound collecting device 13, which is separate from the information processing device 10, may be connected to the information processing device 10 by wire or wirelessly.
  • the sound emitting device 14 reproduces the sound represented by the acoustic signal.
  • the sound emitting device 14 is, for example, a speaker or headphones.
  • a D/A converter that converts an acoustic signal from digital to analog and an amplifier that amplifies the acoustic signal are omitted from the drawing for the sake of convenience.
  • the sound emitting device 14, which is separate from the information processing device 10 may be connected to the information processing device 10 by wire or wirelessly.
  • the display device 16 displays images under the control of the control device 11 .
  • various display panels such as a liquid crystal display panel or an organic EL (Electroluminescence) panel are used as the display device 16 .
  • the display device 16, which is separate from the information processing device 10 may be connected to the information processing device 10 by wire or wirelessly.
  • the musical score data SD is stored in the storage device 12 after being distributed to the information processing device 10 via a communication network such as the Internet from, for example, a distribution device (typically a web server).
  • a plurality of score data SD may be stored in the storage device 12 .
  • one score data SD is created corresponding to one piece of music.
  • a musical score is a representation of a piece of music using musical symbols, including performance symbols.
  • musical symbols include musical note symbols, clefs, time signatures, key signatures, and performance symbols.
  • musical note symbols include notes, rests and accidentals that are attached to notes in a musical score.
  • the clef is marked on the left end of the staff and specifies the relationship between the position on the staff and the pitch of the sound.
  • the time signature specifies the number of beats in one measure and the type of note that constitutes one beat.
  • a key signature is a set of accidental symbols for designating the key of a piece of music.
  • Performance symbols are written on the score as a supplement to indicate to the performer the nuances that cannot be expressed with only notes and rests when performing a piece of music.
  • Performance symbols include speed symbols such as adagio and andante, expressions such as affettuoso and agitato, dynamics such as fortissimo and pianissimo, and articulation symbols such as tenuto and staccato (hereinafter referred to as articulation symbols). ”), repeat marks such as da capo and se ⁇ o, decorative marks such as trills and turns, abbreviations such as ottava alta and ottava bassa, and playing style marks indicating specific playing styles for instruments such as pedals and pizzicato.
  • the performance symbols also include finger numbers that designate fingers used when playing the notes written in the musical score.
  • the musical score data SD includes attribute information B for each musical symbol forming the target musical piece.
  • the attribute information B is information that defines musical attributes of each musical symbol, and includes a beat identifier B1 and a symbol identifier B2.
  • the beat identifier B1 is information specifying the temporal position of the musical symbol in the target music. If the music symbol is a musical note symbol or performance symbol, the number of beats from the beginning of the target song to the corresponding music symbol (for example, the beat number obtained by counting an eighth note as one beat) is preferably used as the beat identifier B1.
  • the symbol identifier B2 is information for identifying the type of musical symbol.
  • the symbol identifier B2 includes the note name (note number) and note value.
  • the note name represents the pitch of the note
  • the note value represents the duration of the note on the score.
  • a character string indicating the name of the musical symbol is preferably used as the symbolic identifier B2.
  • the musical score data SD includes musical score image data MD.
  • the musical score image data MD is data representing an image of the musical score of the target musical piece (hereinafter referred to as "score image").
  • an image file for example, a PDF file
  • a musical score image as a plane image in raster format or vector format is suitable as the musical score image data MD.
  • the symbol text data TD is data containing text corresponding to musical symbols written on the musical score.
  • the symbolic text data TD includes symbolic identifiers C1, name texts C2 and semantic texts C3.
  • the symbolic identifier C1 is information for identifying the type of musical symbol, and is information in the same format as the symbolic identifier B2 of the attribute information B.
  • FIG. The symbolic identifier C1 corresponding to the musical note symbol may be only the pitch name.
  • the name text C2 is text indicating the name of the symbol specified by the symbol identifier C1.
  • the symbolic identifier C1 and the name text C2 may be the same character string. If the musical symbol is a note symbol, the name text C2 is "do", "re”, and so on. If the musical symbol is a performance symbol, the name text C2 is "crescendo", "forte”, or the like.
  • the musical score reading application is capable of reading musical scores in multiple languages.
  • the languages that can be selected during reading are Japanese or English. Therefore, the name text C2 includes Japanese text indicating the name of the musical symbol and English text indicating the name of the musical symbol.
  • the meaning text C3 is text indicating the meaning of the symbol specified by the symbol identifier C1. For example, if the name of the musical symbol is "adagio", the semantic text C3 is "slowly”. If the musical symbols are note symbols, the semantic text C3 may not be provided.
  • the semantic text C3 also includes Japanese text indicating the meaning of the musical symbol and English text indicating the meaning of the musical symbol.
  • a word used as a phrase indicating the rest when taking a beat such as "Un” for a quarter rest, "U” for an eighth rest, etc. may be used.
  • FIG. 4 is a block diagram illustrating the functional configuration of the control device 11.
  • the control device 11 generates sound signals representing sounds related to musical symbols forming a musical score (hereinafter referred to as “symbolic sounds”) based on musical score data SD representing the musical score.
  • the musical score represented by the musical score data SD includes one or more performance symbols. Therefore, it can also be said that the control device 11 generates an acoustic signal representing a sound related to the performance symbol based on the musical score data SD representing a musical score including one or more performance symbols.
  • the sound related to the musical performance symbol is, for example, a sound indicating the name of the musical performance symbol or a sound indicating a phrase corresponding to the meaning of the musical performance symbol.
  • the names of the performance symbols correspond to the name texts C2 of the performance symbols
  • the words corresponding to the meanings of the performance symbols correspond to the meaning texts C3 of the performance symbols.
  • the musical score includes musical note symbols in addition to performance symbols. Accordingly, the control device 11 generates acoustic signals representing sounds related to performance symbols and sounds related to musical note symbols.
  • a sound related to a musical note symbol is, for example, a sound indicating the pitch name of the note indicated by the musical note symbol.
  • the acoustic signal is a signal for causing the sound emitting device 14 to reproduce the symbol sound.
  • the control device 11 has a plurality of functions for generating and reproducing sound signals (the instruction receiving unit 30, the text generating unit 32, the voice synthesizing unit 34, the performance The analysis unit 38 and the output control unit 40) are realized.
  • FIG. 5 to 9 are diagrams exemplifying instruction reception screens by the instruction reception unit 30.
  • the instruction receiving unit 30 causes the touch panel T to display a reception screen SC1 for selecting a song to be read aloud, as shown in FIG. 5, for example.
  • the user designates the musical score data SD to be read out by touching the display NA1 to NA5 corresponding to the desired musical score data SD.
  • musical score data "yyy.xml" corresponding to display NA2 is specified.
  • the instruction receiving unit 30 causes the touch panel T to display a reception screen SC2 for selecting a staff notation to be read out of the musical score data SD, as shown in FIG.
  • the reception screen SC2 displays options NB1 and NB2 for designating a staff notation to be read aloud out of the grand staff.
  • the option NB1 designates the reading of the right-hand staff positioned above the grand staff.
  • the option NB2 designates the reading of the staff notation for the left hand located on the lower stage of the grand staff.
  • the user designates the staff notation to be read out by checking the check box CK of at least one of the options NB1 and NB2.
  • the musical score represented by the musical score data SD includes a plurality of performance symbols, and each of the plurality of performance symbols belongs to one of a plurality of classifications.
  • a plurality of classifications correspond to the types of performance symbols shown in FIG.
  • the instruction receiving unit 30 receives selection of at least one of a plurality of classifications of performance symbols.
  • the control device 11 generates acoustic signals for performance symbols belonging to one or more selected categories among the plurality of performance symbols.
  • the instruction receiving unit 30 causes the touch panel T to display a reception screen SC4 for selecting a setting for reading out the musical score data SD, as shown in FIG. 8, for example.
  • Options ND1 and ND2 for designating information to be output when reading a musical score are displayed in the upper area E1 of the reception screen SC4.
  • the option ND1 designates only the reading of musical scores. That is, the option ND1 designates only audio output.
  • the option ND2 designates displaying a musical score image in addition to reading out the musical score. That is, the option ND2 designates output of audio and images.
  • Either option ND1 or ND2 can be selected using a radio button. By touching the radio button corresponding to the option ND1 or the radio button corresponding to the option ND2, the user specifies the information to be output when reading out the musical score.
  • options NE1 to NE4 for specifying the tempo for reading out the score are displayed in the lower region E2 of the reception screen SC4.
  • the option NE1 designates reading at the tempo designated by the musical score.
  • the option NE2 designates reading out all the symbols designated on the reception screen SC3 regardless of the tempo designated by the musical score.
  • Option NE3 designates reading aloud at a tempo synchronized with the performance of the musical score by the user.
  • the user designates an arbitrary tempo. In the illustrated example, the user specifies the reading tempo by specifying the number of beats per minute.
  • the speaking time for one syllable in other words, the number of reading syllables per unit time
  • the speaking time for one syllable in other words, the number of reading syllables per unit time
  • the speaking time for one syllable in other words, the number of reading syllables per unit time
  • the instruction reception unit 30 causes the touch panel T to display a selection instruction reception screen SC5 for further setting, as shown in FIG. 9, for example.
  • Options NF1 to NF2 for designating the language to be used when reading out the musical score are displayed in the upper area E3 of the reception screen SC5.
  • Option NF1 designates reading in Japanese. Reading in Japanese corresponds to, for example, using "do-re-mi-fa-so-la-si" as the note name, using Japanese-like pronunciation when uttering the names of performance symbols, and the like.
  • Choice NF2 designates reading in English.
  • Reading out in English corresponds to, for example, using "C, D, E, F, G, A, B" as note names, using English-like pronunciation when uttering names of performance symbols, and the like.
  • a language other than Japanese and English may be specified on the reception screen SC5.
  • the symbolic text data TD includes the name text C2 and the meaning text C3 of the language.
  • options NG1 to NG2 are displayed for designating the contents of reading out the performance symbols.
  • the option NG1 designates reading out the name of the performance symbol.
  • the name text C2 of the symbol text data TD is read out when the performance symbols are read out.
  • the option NG2 designates the reading of a phrase indicating the meaning of the performance symbol.
  • the meaning text C3 of the symbol text data TD is read out when reading out the performance symbol.
  • the type of voice for reading musical note symbols and the type of voice for reading performance symbols may be designated as different types. Also, if four or more voice types can be specified, for example, the right-hand staff note symbol, the left-hand staff note symbol, the right-hand staff performance symbol, and the left-hand staff symbol It may also be possible to designate the performance symbols to be read out in different voices.
  • the sound emitting device 14 may be set so that the right speaker reads out the staff notation for the right hand, and the left speaker reads out the staff notation for the left hand. Further, when the sound emitting device 14 is a stereo speaker, it may be possible to designate a speaker for outputting the reading sound of musical note symbols and a speaker for outputting the reading sound of performance symbols separately.
  • the user when reading out a chord, the user may be able to select whether to read out each note constituting the chord individually or read out the code name corresponding to the chord.
  • the name text C2 of the symbol text data TD may be stored with the text indicating the pitch name of each note forming the chord
  • the semantic text C3 may be stored with the text indicating the chord name of the chord.
  • the control device 11 starts the acoustic signal generation process. Further, the touch panel T displays, for example, a button for instructing the start of reading out a musical score (hereinafter referred to as "performance start button"). The user presses the performance start button at an appropriate timing to start reading out the musical score.
  • performance start button a button for instructing the start of reading out a musical score
  • the text generation unit 32 shown in FIG. 4 generates text indicating the content of the musical score.
  • FIG. 10 is a diagram schematically showing processing of the text generation unit 32.
  • the text generator 32 reads the score data SD specified on the reception screen SC1 shown in FIG. 5 (S1).
  • the text generation unit 32 classifies the musical score data SD into right-hand data representing a staff notation for the right hand and left-hand data representing a staff notation for the left hand (S2).
  • the data corresponding to the staff notation to be read aloud specified on the reception screen SC2 shown in FIG. 6 is to be processed thereafter.
  • the staff notation data to be read out includes attribute information B of all types of musical symbols (S3).
  • the text generation unit 32 extracts the attribute information B of the symbol to be read aloud specified on the reception screen SC3 shown in FIG. ).
  • the text generation unit 32 collates the symbolic identifier B2 of the extracted attribute information B with the symbolic identifier C1 of the symbolic text data TD, and reads out the name text C2 or the meaning text C3 corresponding to the symbolic identifier C1 (S5). Which of the name text C2 and the meaning text C3 is read depends on which of the options NG1 or NG2 on the reception screen SC5 shown in FIG. 9 is selected. Further, which of the Japanese text and the English text is read depends on which of the options NF1 and NF2 on the reception screen SC5 is selected. In the drawing, these selection contents are described as "designation of reading contents”. The read texts are arranged in the same order as the attribute information B (time series). Through the above processing, a text indicating the content of the musical score (hereinafter referred to as "read-aloud text”) is generated (S6).
  • FIG. 11 is a diagram illustrating musical scores. 12 and 13 are diagrams schematically showing read-out timings of read-out texts.
  • a musical score G shown in FIG. 11 indicates, for example, the first two bars of the musical score data (yyy.xml) specified to be read out on the reception screen SC1 shown in FIG.
  • the musical score G includes a musical score for the right hand and a musical score for the left hand. Based on the tempo information TP, the musical score G is specified as 120 beats per minute, with one quarter note as one beat.
  • FIG. 12 shows right-hand read-out sounds representing right-hand read-out sounds and left-hand read-out sounds representing left-hand read-out sounds.
  • a time axis is shown between the reading sound for the right hand and the reading sound of the musical score for the left hand.
  • One scale (t1) of the time axis is based on the eighth note, which is the shortest note in the musical score G. Based on the tempo of the musical score G described above, one scale (t1) on the time axis is 0.25 seconds.
  • the control device 11 controls when the target point in time at which the piece of music progresses at a speed corresponding to the tempo designated by the tempo information TP reaches the point in time corresponding to the performance symbol. , generates an acoustic signal so that the sound associated with the performance symbol is pronounced.
  • the text generation unit 32 may reduce the text to be read out. For example, if the speech time for one syllable is determined during reading, it is possible to determine whether or not reading is possible at the time the read-aloud text is generated. The text generation unit 32 determines whether or not the read-out text can be read out within the time based on the generated words, the tempo of the music, and the utterance time of one syllable.
  • the text generation unit 32 reads a part of the plurality of performance symbols. The remaining performance symbols may be excluded from reading. Taking the period P1 as an example, the text generation unit 32 does not need to read out one of "mesopiano” or “staccato” and not read out the other. That is, when “mesopiano” and “staccato” are pronounced together, the control device 11 selects either “mesopiano” or “staccato” and reads out the text corresponding to the selected performance symbol.
  • the control device 11 selects either “mesopiano” or “staccato” and reads out the text corresponding to the selected performance symbol.
  • include in “Mezzo piano” is an example of a sound associated with the first performance symbol
  • staccato is an example of a sound associated with the second performance symbol.
  • the instruction receiving unit 30 may allow the user to set the reading priority for each classification of performance symbols.
  • the text generator 32 deletes from the read-aloud text in order from the text of the performance symbols belonging to the lower priority category.
  • the non-verbal sound is included in the read-aloud text instead of the text corresponding to the performance symbol. good too.
  • the user can grasp the rhythm of the musical piece as well as the pitches of the notes indicated in the musical score.
  • FIG. 13 shows only the reading sounds for the right hand and omits the reading sounds for the left hand.
  • symbol Mti (i is an integer from 1 to 9) indicates a metronome sound. The user can grasp the break of the beat by the metronome sound Mti.
  • control device 11 may generate an acoustic signal indicating a sound produced by pronouncing a sound related to the performance symbol regardless of the tempo of the music.
  • the user can comprehend all the symbols of the specified type, and can comprehend the content of the musical score without omission.
  • the option NE4 is a mode in which the user designates an arbitrary tempo by designating the number of beats per minute.
  • the user may designate the progress of reading using the manipulator.
  • the operator may be an operation button displayed on the touch panel T, for example.
  • the operator may be a member of the musical instrument.
  • the musical instrument is a piano, pedals can be used as operators. In this case, for example, when the user steps on the damper pedal once, reading progresses in units of one note symbol or one bar, and when the user steps on the soft pedal once, the reading progresses by one note symbol. , or the reading may be reversed in units of one bar or the like.
  • the voice synthesizing unit 34 shown in FIG. 4 uses the read-out text generated by the text generating unit 32 and the voice data VD to generate an acoustic signal.
  • the speech synthesizer 34 is an example of a generator.
  • the speech synthesizing unit 34 sequentially selects speech segments corresponding to the text to be read out of a plurality of speech segments included in the speech data VD, adjusts the pitch of each speech segment, and then connects them to each other. to generate an acoustic signal.
  • the pitch of the sound related to the musical note symbol in the reading text may be matched with the pitch of the musical note symbol, or may be a predetermined pitch.
  • the performance analysis unit 38 operates only when the user designates reading out at a tempo synchronized with the performance of the musical score (option NE3) on the acceptance screen SC4 shown in FIG. good.
  • the output control unit 40 controls the output of sound based on the acoustic signal and the output of the musical score image based on the musical score image data MD.
  • the output control unit 40 causes the sound emitting device 14 to reproduce the sound represented by the acoustic signal generated by the voice synthesis unit 34.
  • the output control unit 40 outputs the sound represented by the acoustic signal generated by the voice synthesis unit 34 to the sound emitting device 14.
  • the musical score image data MD is displayed on the display device 16 while being reproduced.
  • FIG. 14 is a diagram illustrating a display screen during reading of musical scores.
  • the display of the touch panel T which is the display device 16
  • the display of the touch panel T is switched to the display screen SC6 shown in FIG.
  • a message 601 indicating that the reading sound of the musical score is being reproduced a musical score image 602, a pause button 604, a fast forward button 606, a rewind button 608, a repeat button 610,
  • An end button 612 is displayed.
  • the musical score image 602 is an image displaying the musical score image data MD included in the musical score data SD to be read aloud.
  • a bar 603 indicating the reading position is superimposed on the musical score image 602 and displayed.
  • the output control unit 40 scrolls the musical score image 602 based on the timing label attached to the reading text. At this time, the output control unit 40 adjusts the scrolling speed of the musical score image 602 so that the music symbol being read out and the bar 603 are superimposed. It should be noted that instead of displaying the read-out position with the bar 603, the musical symbols to be read-out may be highlighted.
  • FIG. 14 exemplifies a musical score using staff notation as the musical score image 602
  • the musical score image 602 may be displayed as a piano roll, for example. Further, in the reception screen SC4 shown in FIG. 8, when only the reading of the musical score is specified (option ND1), the musical score image 602 is not displayed.
  • a pause button 604, a fast-forward button 606, a rewind button 608, a review button 610, and an end button 612 accept operations related to reading out the musical score.
  • the output control unit 40 pauses reading out the score.
  • the fast-forward button 606 the output control unit 40 fast-forwards reading of the musical score. For example, when the fast-forward button 606 is touched once, the output control unit 40 changes the readout position to the beginning of the bar next to the bar containing the current readout position.
  • the rewind button 608 is operated, the output control unit 40 rewinds the reading of the musical score.
  • the output control unit 40 selects reading aloud at a tempo synchronized with the performance of the musical score by the user (option NE3). Based on this, adjust the output timing of the reading sound.
  • the output control unit 40 adjusts the output speed of the reading sound so that, for example, on the musical score, the position ahead of the performance position by a predetermined beat is read out.
  • the predetermined beat may be specified by the user.
  • the output control unit 40 may control the music included in the N+1-th measure immediately before the performance of the N-th measure to end.
  • the symbols may be read aloud.
  • Immediately before the performance of the N-th measure ends is, for example, after the last note of the N-th measure is played. This imitates, for example, a teaching method in which a chorus conductor prereads and shows lyrics to be sung next to the chorus members.
  • the generation of the read-out text (S102) and the generation of the acoustic signal (S104) may be performed after the user instructs to read out the score (S106: YES).
  • acoustic signals representing sounds related to performance symbols are generated based on musical score data SD including one or more performance symbols. Therefore, the performance symbols included in the musical score can be comprehended aurally, and even visually impaired people, beginners and small children who are not accustomed to reading musical scores can easily comprehend the musical score.
  • the sound related to the performance symbol is a sound indicating the name of the performance symbol or a sound indicating a phrase corresponding to the meaning of the performance symbol. If the sound associated with the performance symbol is the sound indicating the name of the performance symbol, the description on the musical score can be accurately grasped. In addition, if the sound related to the musical performance symbol is a sound indicating a phrase corresponding to the meaning of the musical performance symbol, even if the user lacks knowledge of the musical performance symbol and cannot understand the meaning of the performance symbol only by the name of the musical performance symbol, Able to grasp the contents indicated by musical scores.
  • sounds corresponding to performance symbols are pronounced at timings corresponding to the tempo of music. This makes it easier for the user to grasp the positions of the performance symbols in the music, thereby improving convenience.
  • an acoustic signal representing the sound related to the musical note symbol is generated. Therefore, the musical note symbols included in the musical score can be comprehended aurally, and the comprehension of the musical score can be further facilitated.
  • a non-verbal notification sound is pronounced as the sound related to the rest.
  • the user can immediately recognize that the sound associated with the rest corresponds to the rest when the sound related to the rest is pronounced.
  • the musical score data SD specified by the user is read out.
  • a list of data names of the musical score data SD such as the reception screen SC1 shown in FIG. 5 is displayed, the user may not be able to identify the musical score data SD corresponding to the desired song.
  • a part of a plurality of musical score data SD is continuously read out so that the user can specify the musical score data SD corresponding to the desired music piece.
  • the function of continuously reading a part of a plurality of musical score data SD is hereinafter referred to as a "table of contents presentation function".
  • FIG. 16 is a diagram illustrating an instruction reception screen by the instruction reception unit 30.
  • the instruction receiving unit 30 when the score reading application is activated, the instruction receiving unit 30 causes the touch panel T to display a menu selection instruction receiving screen SC7 as shown in FIG. 16, for example. Choices NI1 and NI2 are displayed on the acceptance screen SC7.
  • the option NI1 specifies reading out the musical score data SD selected by the user as shown in the first embodiment.
  • the instruction receiving unit 30 displays the receiving screen SC1 shown in FIG. 5, and receives the designation of the musical score data SD to be read out from the user.
  • a part of the musical score is, for example, a part or all of a specific structural section among multiple sections (hereinafter referred to as "structural sections") that divide a piece of music according to its musical meaning.
  • Structural sections are, for example, sections such as an intro, an A melody, a B melody, a chorus, and an outro.
  • the text generation unit 32 generates, for each of the plurality of musical score data SD, a text for reading out, for example, the structure section of the “chorus” of the music.
  • the text generation unit 32 generates, for each of the plurality of musical score data SD, a text for reading out the structural section of the "intro" (predetermined number of bars at the beginning of the musical score) of the music, for example.
  • the musical score data SD is the first musical score data
  • the storage device 12 also stores second musical score data different from the first musical score data.
  • the control device 11 generates a first acoustic signal representing a sound related to the performance symbol and a sound related to the musical note symbol included in a portion of the first musical score corresponding to the first musical score data, and a second acoustic signal corresponding to the second musical score data. Generating a second acoustic signal indicating sounds related to performance symbols and sounds related to musical note symbols included in a portion of the musical score.
  • the control device 11 causes the sound emitting device 14 to sequentially reproduce the first acoustic signal and the second acoustic signal.
  • the first musical score data is musical score data "xxx.xml”
  • the second musical score data is musical score data "yyy.xml”.
  • the control device 11 selects a portion from each of a plurality of musical score data, and sequentially reproduces sounds related to performance symbols and musical note symbols included in the selected portion.
  • the user can easily grasp which musical score of each piece of musical score data SD corresponds to, and quickly select the desired musical score data SD from among the plurality of musical score data SD. can do.
  • the information processing device 10 reads out the musical score data SD.
  • the information processing apparatus 10 assists the user in making the sound of the performance closer to the sound shown in the score.
  • FIG. 18 is a block diagram illustrating the functional configuration of the control device 11A in the third embodiment.
  • the control device 11A includes a performance evaluation section 42 in addition to the configuration of the control device 11 (see FIG. 4) according to the first embodiment.
  • the performance evaluation section 42 evaluates the performance of the musical instrument by the user based on the analysis result of the performance analysis section 38 .
  • the performance analysis unit 38 analyzed the performance position of the musical instrument by the user.
  • the performance analysis unit 38 analyzes the volume of the performance sound of the musical instrument in addition to the analysis of the performance position.
  • the performance evaluation unit 42 evaluates whether the user's performance conforms to the musical symbols of the score. More specifically, the performance evaluation unit 42 detects the difference between the performance sound, which is the sound of the musical composition played by the user, and the sound indicated by the musical symbols included in the musical score representing the musical composition, and determines that the difference is determined in advance. Determine whether or not it is out of the allowable range.
  • the user sets the allowable range of the difference, for example, based on his or her performance skill level. Generally, it is considered that the allowable difference becomes smaller as the skill level of the user increases. If there is a portion where the difference is out of the allowable range, the text generation unit 32 generates text pointing out the portion. Specifically, for example, "Right hand, 2nd measure, 'Fa, re, me' was changed to 'Fa, mi, le'.” In addition, it generates a text that reads out the pitch and duration of the performance performed by the user. Such text is referred to as "supporting text".
  • the performance evaluation unit 42 evaluates the performance by detecting the difference between the volume of the sound played by the user and the volume of the performance along the dynamic symbol. Further, the performance evaluation section 42 detects the difference between the duration of the sound played by the user when the performance symbol is an articulation symbol, and the duration when the performance is performed along the articulation symbol. by doing. For example, the smaller the difference, the performance evaluation unit 42 evaluates that the performance is performed along the performance symbols of the musical score, that is, the performance skill is high.
  • the user sets the allowable range of the difference, for example, based on his or her performance skill level. If there is a portion where the difference is out of the allowable range, the text generation unit 32 generates support text pointing out the portion. Specifically, for example, "Right hand, 1st measure, 'staccato, mi, staccato, fa', staccato momentum is weak," etc. Generate supporting text to indicate that the performance symbols were not reflected.
  • the speech synthesis unit 34 uses the supporting text and the speech data VD to generate an acoustic signal.
  • the text indicating the pitch of the musical note symbol may be read aloud with a voice of a pitch corresponding to the pitch.
  • the output control unit 40 causes the sound emitting device 14 to reproduce sound based on the acoustic signal.
  • the performance sound of the user may be recorded, and the part of the recorded performance sound corresponding to the point pointed out in the support text may be reproduced together with the support text read aloud.
  • the performance symbols may always be read aloud regardless of the presence or absence of differences. In this case, for example, if the difference between the performance symbols and the performance is large, the read-out voice is increased (the greater the difference, the louder the read-out voice is) so that the user can check whether the performance follows the performance symbols. can be grasped.
  • the control device 11 acquires the performance sound, which is the sound of the music played by the user, and detects the difference between the sound indicated by the musical symbols included in the musical score representing the music and the performance sound. do.
  • the control device 11 When the difference is out of a predetermined allowable range, the control device 11 generates an acoustic signal representing a sound related to the musical symbol included in the portion of the musical score corresponding to the location where the difference occurs.
  • the user can grasp the difference between his/her own performance and the content indicated by the musical score, and can efficiently master the performance of the musical piece indicated by the musical score.
  • the control device 11 indicates to the user the position on the musical score of the portion where the difference occurs by reading out the musical symbol of the portion where the difference occurs.
  • the user can intuitively grasp the location on the musical score where the difference occurs, compared to, for example, simply reading out the position (bar number, etc.) in the musical score mechanically.
  • the content of the user's performance is verbalized.
  • the control device 11 reads aloud the pitch and value of the performance performed by the user. This allows the user to objectively grasp the details of his/her own error.
  • the speech synthesizing unit 34 performs segment-connected speech synthesis, but the method of speech synthesis is not limited to the above examples.
  • statistical model type speech synthesis using statistical models such as deep neural networks or HMMs (Hidden Markov Models) may be used.
  • the information processing device 10 may be implemented by a server device that communicates with an information device such as a smart phone or a tablet terminal.
  • the information processing device 10 receives designation of musical score data SD from the information device, and generates an acoustic signal by speech synthesis processing using the designated musical score data SD.
  • the information processing device 10 transmits an acoustic signal generated by speech synthesis processing to the information device.
  • the information device reproduces the acoustic signal.
  • the functions of the information processing device 10 include the control device 11 as described above. It is realized by cooperation of one or more constituent processors and the program PG stored in the storage device 12 .
  • the manner in which the user looks at the items displayed on the touch panel T and touches the touch panel T when performing various settings and instructions in the musical score reading application has been described.
  • the presentation of information (such as selection items in settings) to the user may be performed by reading aloud.
  • the input from the user to the information processing device 10 may be performed by voice input.
  • voice input when a visually impaired person uses a musical score reading application, the use of voice is effective.
  • the above program can be provided in a form stored in a computer-readable recording medium and installed in the computer.
  • the recording medium is, for example, a non-transitory recording medium, and an optical recording medium (optical disc) such as a CD-ROM is a good example.
  • a recording medium for storing the program in the distribution device corresponds to the non-transitory recording medium described above.
  • An information processing apparatus is realized by a computer system, and generates an acoustic signal representing a sound related to the performance symbols based on musical score data representing a musical score including one or more performance symbols. do. Therefore, the performance symbols included in the musical score can be grasped by hearing, and even visually handicapped people, beginners who are not accustomed to reading musical scores, and small children can easily grasp the musical score.
  • the sound related to the performance symbol is a sound indicating the name of the performance symbol or a sound indicating a phrase corresponding to the meaning of the performance symbol.
  • the sound associated with the performance symbol is the sound indicating the name of the performance symbol
  • the sound associated with the performance symbol is a sound indicating a word or phrase corresponding to the meaning of the performance symbol, even if the user lacks knowledge of the performance symbol and cannot understand the meaning of the performance symbol only by the name of the performance symbol, the musical score can be used. It is possible to grasp the contents indicated by .
  • the musical score data includes tempo information specifying a tempo of a piece of music indicated by the musical score, and in generating the acoustic signal, the tempo specified by the tempo information.
  • the acoustic signal is generated such that when the target point in time at which the music progresses at the speed corresponding to reaches the point in time corresponding to the performance symbol, the sound associated with the performance symbol is pronounced.
  • sounds corresponding to performance symbols are produced at timings corresponding to the tempo of music. Therefore, it becomes easier for the user to grasp the positions of the performance symbols in the music, and convenience can be improved.
  • the one or more performance symbols include a first performance symbol and a second performance symbol, and in generating the acoustic signal, When the sound related to the first performance symbol and the sound related to the second performance symbol are superimposed, selecting either the first performance symbol or the second performance symbol, generating said acoustic information representing a sound;
  • the first performance symbol and the second performance symbol are superimposed, either the first performance symbol or the second performance symbol is selected.
  • Generates acoustic information Therefore, the sound related to the first performance symbol and the sound related to the second performance symbol do not overlap, and the audibility of the sound related to the performance symbol can be improved.
  • the musical score data includes tempo information specifying the tempo of the music indicated by the musical score, and in generating the acoustic signal, the musical performance symbols are used regardless of the tempo of the music. Generating an acoustic signal indicative of the sound produced.
  • sounds corresponding to performance symbols are produced regardless of the tempo of music. Therefore, the sounds related to the performance symbols are not over-pronounced, and the audibility of the sounds related to the performance symbols can be improved.
  • the one or more performance symbols are a plurality of performance symbols, and each of the plurality of performance symbols belongs to one of a plurality of classifications. , receiving a selection of at least one of the plurality of categories, and generating the acoustic signal for a musical performance symbol belonging to the one or more categories related to the selection among the plurality of musical performance symbols. Generate.
  • acoustic signals are generated for performance symbols belonging to the selected classification. Therefore, it is possible to selectively produce sounds related to the performance symbols required by the user, thereby improving convenience.
  • the musical score includes musical note symbols in addition to the performance symbols, and generating the acoustic signal includes sounds associated with the performance symbols and generating said acoustic signal representing a sound associated with a musical note symbol.
  • acoustic signals representing sounds related to musical note symbols are generated. Therefore, the musical note symbols included in the musical score can be comprehended aurally, and the comprehension of the musical score can be further facilitated.
  • an acoustic signal representing a non-verbal notification sound is generated as the sound related to the rest.
  • non-verbal notification sounds are used as the sounds related to rests. Therefore, when a sound related to a rest is pronounced, the user can immediately recognize that the sound corresponds to the rest.
  • the musical score data is first musical score data
  • the generation of the acoustic signal is included in a portion of the first musical score corresponding to the first musical score data.
  • generating a second acoustic signal indicative of the sound associated with the musical note symbol and further comprising causing a sound emitting device to sequentially reproduce the first acoustic signal and the second acoustic signal.
  • a portion is selected from each of a plurality of musical score data, and sounds associated with performance symbols and musical note symbols included in the selected portion are sequentially reproduced. Therefore, the user can easily grasp the musical score of which musical piece each of the plurality of musical score data corresponds to, and can quickly select the desired musical score data from the plurality of musical score data.
  • a program according to one aspect (a tenth aspect) of the present disclosure functions as a generation unit that generates an acoustic signal representing a sound related to one or more performance symbols based on musical score data representing a musical score including one or more performance symbols.
  • An information processing apparatus includes a generation unit that generates an acoustic signal representing a sound associated with one or more performance symbols based on musical score data representing a musical score including one or more performance symbols.
  • Braille sheet music requires about three times as much paper space to write the same content as regular sheet music, and it takes time to read. For this reason, for example, when a user forgets the title of a piece of music and wants to find the desired score from the contents of the score, the user has to spend time reading a plurality of scores, which is inconvenient. .
  • An information processing apparatus is implemented by a computer system, and includes a first musical symbol representing a sound related to the musical symbol included in a portion of a first musical score corresponding to first musical score data including one or more musical symbols. generating an acoustic signal, and generating a second acoustic signal indicating a sound associated with the musical symbol included in a portion of a second musical score that includes one or more of the musical symbols and corresponds to second musical score data that is different from the first musical score data; and causing a sound emitting device to sequentially reproduce the first acoustic signal and the second acoustic signal.
  • An information processing apparatus is realized by a computer system, acquires a performance sound that is a sound of a musical piece played by a user, and obtains a sound indicated by a musical symbol included in a musical score indicating the musical piece, Detecting a difference from the performance sound, and generating an acoustic signal indicating a sound related to a musical symbol included in a portion of the musical score corresponding to the portion where the difference occurs when the difference is out of a predetermined allowable range. do.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Auxiliary Devices For Music (AREA)

Abstract

La présente invention concerne un dispositif de traitement d'informations 10 qui génère, sur la base de données de score SD représentant un score comprenant au moins une marque de performance, un signal acoustique représentant un son relatif à la marque de performance.
PCT/JP2022/040701 2022-02-10 2022-10-31 Procédé de traitement d'informations, programme et dispositif de traitement d'informations WO2023153033A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022-019211 2022-02-10
JP2022019211A JP2023116866A (ja) 2022-02-10 2022-02-10 情報処理方法、プログラム、および情報処理装置

Publications (1)

Publication Number Publication Date
WO2023153033A1 true WO2023153033A1 (fr) 2023-08-17

Family

ID=87564095

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/040701 WO2023153033A1 (fr) 2022-02-10 2022-10-31 Procédé de traitement d'informations, programme et dispositif de traitement d'informations

Country Status (2)

Country Link
JP (1) JP2023116866A (fr)
WO (1) WO2023153033A1 (fr)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000214848A (ja) * 1999-01-21 2000-08-04 Yamaha Corp 演奏支援装置、演奏支援方法、および演奏支援プログラムを記録した記録媒体

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000214848A (ja) * 1999-01-21 2000-08-04 Yamaha Corp 演奏支援装置、演奏支援方法、および演奏支援プログラムを記録した記録媒体

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
SHINYA YAMADA, TOSHIYUKI GOTOH, NAOYOSHI TAMURA: "Automated Generation System of DAISY Content with Braill and Oral from Digital Music Scores", IEICE TECHNICAL REPORT, WIT, IEICE, JP, vol. 109, no. 358 (WIT2009-71), 1 January 2010 (2010-01-01), JP, pages 19 - 24, XP009548320 *

Also Published As

Publication number Publication date
JP2023116866A (ja) 2023-08-23

Similar Documents

Publication Publication Date Title
JP5821824B2 (ja) 音声合成装置
US7754955B2 (en) Virtual reality composer platform system
KR20180063163A (ko) 언어 및/또는 그래픽 아이콘 기반 음악적 경험 디스크립터를 채택한 자동화된 음악 작곡 및 생성 기계, 시스템 및 프로세스
JP5949607B2 (ja) 音声合成装置
JP2008015195A (ja) 楽曲練習支援装置
JP6728754B2 (ja) 発音装置、発音方法および発音プログラム
WO2015194423A1 (fr) Dispositif de commande et système de génération de voix sur la base de caractères
US20220076658A1 (en) Electronic musical instrument, method, and storage medium
US20220076651A1 (en) Electronic musical instrument, method, and storage medium
JP2008026622A (ja) 評価装置
JP7180587B2 (ja) 電子楽器、方法及びプログラム
KR20170120234A (ko) 영어 단어의 음절과 연동된 악보 제공 시스템
JP4929604B2 (ja) 歌データ入力プログラム
JP2006259471A (ja) 歌唱練習システムおよび歌唱練習システム用プログラム
JP6589356B2 (ja) 表示制御装置、電子楽器およびプログラム
JP4038836B2 (ja) カラオケ装置
JP2009169103A (ja) 練習支援装置
WO2023153033A1 (fr) Procédé de traitement d'informations, programme et dispositif de traitement d'informations
JP2008039833A (ja) 音声評価装置
JP6044284B2 (ja) 音声合成装置
US20220044662A1 (en) Audio Information Playback Method, Audio Information Playback Device, Audio Information Generation Method and Audio Information Generation Device
JP2022065554A (ja) 音声合成方法およびプログラム
JP4501874B2 (ja) 楽曲練習装置
WO2022190502A1 (fr) Dispositif de génération de son, son procédé de commande, programme et instrument de musique électronique
JP2002182675A (ja) 音声合成装置、ボーカルデータ生成装置および歌唱装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22926049

Country of ref document: EP

Kind code of ref document: A1