JP2011164162A - Support device for giving expression to performance - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a support device for giving an expression to performance easily by acquiring the relationship among tempo, dynamics, and articulation easily when correcting an expression curve. <P>SOLUTION: The support device 10 for giving an expression to performance includes a display device 18 and supports to give an expression to performance based on phrasing. When giving an expression to performance for the phrase specified by a user, the display device 18 multi-displays the expression curve 50 of three performance expressions including tempo, dynamics, and articulation on the same screen 42 by the same time axis. The expression curve 50 can change a shape by operating a mouse or the like. The user gives the expression to the performance for the specified phrase based on the correction of the expression curve 50. As a result, it becomes easy for the user to acquire the relationship among tempo, dynamics, and articulation so that giving the expression to the performance can be easily and efficiently performed. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a performance expression support apparatus, and more particularly to a performance expression support apparatus that supports expression of a performance based on phrasing performed by the user by providing music analysis information to a user.

  In recent years, the importance of facial expression of performance has been increasing in the design of music content. The expression of a performance is a work that changes the performance expression (performance parameters) including volume, tempo, and articulation for a specified sequence of sounds and materializes it as a lively piece of music. It is implemented as a manifestation of the intellect and sensitivity. The articulation is a representation of the shape of a sound that affects the sound and the connection of the sound, and indicates symbols such as slur, staccato, and accent, and their representation.

  Research on expression of musical performance has been addressed as one of the researches on music-related artificial intelligence, and many of them have been focused on the development of automation technology based on rule learning and case-based reasoning. Such an automation technology improves the productivity of music content, but on the other hand, there remains a problem that it is not always easy to reach the performance expression that the user desires.

Therefore, in the non-patent document 1 and the like, the present inventors perform facial expression in the performance design by the user, and substitute (support) the complicated work part with an automated technique, thereby providing the desired facial expression. A system that can be executed efficiently is proposed. The technique of Non-Patent Document 1 is a user-driven performance expression design system focused on phrasing, and includes a phrase structure analysis support function and a performance expression edit function. The phrase structure analysis support function is a function that can analyze the remaining phrase structure by applying GTTM (A Generative Theory of Tonal Music) if the user partially specifies the boundary (structure) of the phrase. The facial expression editing function allows the user to arbitrarily select a phrase that the user wants to add a facial expression to display each tempo and volume (dynamics) facial expression curve corresponding to that phrase on each edit screen, and to change each facial expression curve according to the user's operation. This is a function to form a performance expression desired by the user by individually correcting the sound. When facial expression is applied to the phrase selected by the user, the facial expression is reflected in the entire music piece, so that the performance expression of the performance is improved.
“Mixtract: Performance expression design support environment that responds to user's intention” (17th Osaka University Health Center Health Science Forum “Interdisciplinary Fusion of Music and Wellness”, 36-37, November 2009)

  The experienced musician understands the expression method of the phrase as tacit knowledge, but the combination of performance parameters (tempo, dynamics, etc.) for realizing “hearing” in performance expression is not necessarily unique. The performance expression of the phrase structure expresses its individuality by organically combining tempo expression and dynamic expression. However, in the technique of Non-Patent Document 1, the expression curve of tempo and dynamics for a specified phrase is displayed on a separate edit screen, and each expression curve is individually corrected on each edit screen. It was difficult to grasp the relationship between each other, and it was difficult to grasp what kind of combination of performance parameters realized “hearing” of the performance expression.

  In addition, the user performs desired expression by repeating correction (editing) of the performance expression (expression curve) for the selected phrase and audition, but the methodology related to performance generation is not specified, and there are no guidelines or clues. It is not always easy to express a performance with reference to only a state where information is scarce, for example, only musical score information (piano roll information).

  Therefore, a main object of the present invention is to provide a new performance expression support device.

  Another object of the present invention is to provide a performance expression support device that can easily execute a desired expression for a performance.

  The present invention employs the following configuration in order to solve the above problems. Note that reference numerals in parentheses and supplementary explanations indicate correspondence with embodiments described later in order to help understanding of the present invention, and do not limit the present invention.

  A first invention is a performance expression support device that supports expression of a performance based on phrasing, wherein a tempo is provided for each phrase constituting a hierarchical phrase structure designated or automatically analyzed by a user. Performance expression support that includes first display means for displaying three expression curves of dynamics and articulation on the same screen in the same time axis, and correction accepting means for receiving correction of the expression curve displayed on the screen Device.

  In the first invention, the performance expression support device (10) includes the first display means (12, 18, 42, S13) and supports the expression of the performance based on phrasing performed by the user. The first display means, for example, for a phrase arbitrarily selected from a plurality of phrases constituting the hierarchical phrase structure after the hierarchical phrase structure of the music is analyzed by designation or automatic analysis by the user, Expression curves (50) of three performance expressions of tempo, dynamics, and articulation are displayed in multiple on the same screen on the same time axis. The correction accepting means (12, 16, 42, S51) accepts the correction of the expression curve by the operation of the mouse or the like performed by the user. By correcting the expression curve, the user adds a performance expression to the phrase.

  According to the first invention, the three facial expression curves are displayed on the same screen in the same time axis, so that the relationship between the tempo, the dynamics, and the articulation can be easily grasped. That is, since the editing can be performed while comprehensively judging the relationship between these three performance expressions, the user can easily perform the expression of the performance.

  In addition, it is possible to externalize how phrases are expressed as tacit knowledge, that is, how the performance parameters such as tempo are combined to realize the expression, so that you can formalize your knowledge. , It will be possible to convey the technique of performance expression to a third party more specifically.

  The second invention is dependent on the first invention, and further includes calculation means for calculating the apex-likeness of each note in the phrase, and second display means for displaying the apex-likeness of each note in an identifiable manner.

  In the second invention, the calculation means (12, 14, S7) for calculating the apex-likeness of each note, that is, an index for quantitatively distinguishing the importance of each note in the performance expression, and each calculated note Is provided with second display means (12, 18, 40, S9) for identifiably displaying the vertices. The second display means displays, for example, the apex-likeness of each note as shade information of the drawing color of each note on the piano roll.

  According to the second invention, since it becomes possible to quantitatively determine which note in the phrase is important, the user can easily and efficiently express the performance by referring to this. It becomes like this.

  A third invention is dependent on the second invention, and the calculating means stores rule groups for storing a rule group for determining the likelihood of a note based on a predetermined music theory, and each note and rule in the phrase Each note of each group is checked, and a note having a corresponding rule includes an adding means for adding the evaluation point assigned to the rule as an energy value indicating the likelihood of the note.

  In the third invention, the calculation means (12, 14, S7) includes a rule storage means (14), and a rule group based on, for example, music interpretation theory proposed by Hoshina is stored in the rule storage means. In each rule, an evaluation point is set according to the importance. In other words, the conditions and evaluation point values for calculating the vertex likeness of a note are described as rules. The adding means (12, S47) compares and matches each note in the phrase (specifically, its pitch, note value, harmony, etc.) and the rule group stored in the rule storage means. For a note with a rule, the evaluation point assigned to the rule is added as an energy value indicating the likelihood of the vertex of the note. The calculation result of the energy value for each note is appropriately stored in the memory (14) or the like.

  According to the third aspect, it is possible to quantitatively represent the apex-likeness of each note, that is, the importance of each note, based on the magnitude of the calculated energy value. In addition, since the conditions and evaluation point values for calculating the likelihood of a note are described as rules, it is easy to show the basis of the calculation to the user. Therefore, it is possible to support the expression of the performance from the viewpoint of helping the user to estimate and grasp the vertex.

  According to the first invention, when the facial expression curve is corrected, the relationship between the tempo, the dynamics, and the articulation can be easily grasped, so that the performance can be easily expressed.

  According to the second invention, since it becomes possible to quantitatively determine which note in the phrase is important, the user can easily and efficiently express the performance by referring to this. It becomes like this.

  According to the third aspect, since the conditions and the evaluation point values for calculating the likelihood of the note apex are described as rules, it is easy to show the basis of the calculation to the user. Therefore, it is possible to support the expression of the performance from the viewpoint of helping the user to estimate and grasp the vertex.

  The above object, other objects, features, and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

It is a block diagram which shows the internal structure of the performance expression addition assistance apparatus of one Example of this invention. It is an illustration figure which shows an example of the phrase structure analysis screen displayed on the display apparatus of the performance expression addition assistance apparatus of FIG. It is an illustration figure which shows the classification | category of a vertex estimation rule. It is a table which shows a part of vertex estimation rule. It is a table which shows the other part of vertex estimation rules. It is an illustration figure which shows an example of the performance expression edit screen displayed on the display apparatus of the performance expression addition assistance apparatus of FIG. It is a flowchart which shows an example of the operation | movement of the whole process of the performance design which CPU of the performance expression addition support apparatus of FIG. 1 performs. It is a flowchart which shows an example of operation | movement of the phrase structure analysis process in FIG. It is a flowchart which shows an example of the operation | movement of the calculation process of the vertex in FIG. It is a flowchart which shows an example of the operation | movement of the edit process of the expression curve in FIG.

  Referring to FIG. 1, a performance expression support device (hereinafter simply referred to as “support device”) 10 according to an embodiment of the present invention is a user-driven performance based on a hierarchical phrase expression. It is a device for supporting facial expression. The support device 10 is configured by using an electronic computer such as a personal computer, for example, as will be described in detail later, in order to support the expression of a performance by a user, a phrase structure analysis support function, a vertex analysis function, and a performance expression It has editing functions.

  The support device 10 includes a CPU 12. The CPU 12 is also called a microcomputer or a processor, and executes overall control of the support apparatus 10. A memory 14, an input device 16, a display device 18, a sound source 20, and the like are connected to the CPU 12 via a bus or the like.

  The memory 14 includes a ROM, an HDD, a RAM, and the like, although not shown. The ROM and HDD store a program and data for the CPU 12 to perform this performance expression support process, and other necessary programs and data. The RAM is used as a work area or a buffer area for the CPU 12. That is, the CPU 12 loads a program and data stored in the ROM or HDD into the RAM, controls operations of each unit of the support apparatus 10 according to these programs, and executes various processes. Further, the CPU 12 stores temporary data such as performance data, which is appropriately generated according to the progress of the performance expression process, in the RAM.

  As a program stored in the memory 14 (ROM or HDD), a screen display program for displaying a predetermined screen on the display device 18, and the remaining phrase structure is analyzed based on a partial phrase structure input by the user. Phrase structure analysis support program, vertex estimation program for estimating the vertices of each note, facial expression curve generation program for generating facial expression curves for phrases specified by the user, facial expression curve correction by the user, etc. There are a performance data generation program for generating (updating) performance data and a performance sound output program for outputting sound (performance sound) corresponding to the generated performance data.

  The data stored in the memory 14 includes character data and video data displayed on the display device 18, data relating to rules for estimating the likelihood of the vertices of each note, and musical score data of desired music (note arrangement and performance). Symbol information). As the score data, for example, data in a standard MIDI file (SMF) format or MusicXML format can be used. The score data is appropriately input using a MIDI score editor or a MusicXML import function, and stored in the memory 14.

  The programs and data described above may be read out and used from various information recording media such as CD-ROM, DVD-ROM and memory card, or on a network such as a LAN or the Internet. It may be obtained from the server or the like.

  The input device 16 is operation means operated by a user, and includes a keyboard, a pointing device, and the like. As the pointing device, a mouse, a touch panel, a track ball, a track pad, a digitizer, a tablet, or the like is used. An operation signal (operation information) according to the operation of the input device 16 by the user is given to the CPU 12, and the CPU 12 executes processing based on the given operation signal.

  The display device 18 is an LCD, a CRT, or the like, and displays various screens such as a piano roll screen and an edit screen, which will be described later, based on a display signal given from the CPU 12.

  The sound source 20 generates an audio signal based on performance data (such as performance data with a facial expression) generated in the RAM under the control of the CPU 12 and outputs the audio signal to the audio output device 22. The sound output device 22 includes a D / A converter, an amplifier, a speaker, and the like, and outputs sound (performance sound) corresponding to the sound signal. As the sound source 20, an internal sound source such as a sound card provided with a sound source can be used, or an external sound source such as an electronic piano can be used.

  As described above, the support device 10 having such a configuration focuses on hierarchical phrase expression (phrasing) and supports the expression of performance performed by the user. Here, the phrase structure in music expression will be briefly described within the scope necessary for understanding the present invention. Note that, depending on the performer or composer, the phrase may be expressed as a group, but the phrase and the group are synonyms, so the phrase name is used here.

  In any music, a group having a certain relationship is formed between the sound at a certain time and the sound around it. This unit is called a phrase. A plurality of phrases are further associated to form a hierarchical phrase structure. In addition, phrasing is to give a breath to a song by dividing the flow of music at an appropriate position (that is, by setting a phrase). In other words, it is no exaggeration to say that how to convey the hierarchical phrase structure of music to the listener is the subject of musical expression.

  By the way, the phrase structure interpreted from the score prepared by the composer is not necessarily one, and there is ambiguity. Performers, conductors, etc. define one possible interpretation of the phrase structure and embody it as a performance expression so that the phrase structure is transmitted. Also, the performance expression of the phrase structure is not necessarily one, and the individuality of the performance is expressed by a combination of tempo, dynamics and articulation. That is, in order to perform the performance design desired by the user, it is important how to interpret (set) the phrase structure of the music and how to express the phrase structure. However, it takes a lot of labor for the user to determine all of the hierarchical phrase structure of the entire song and the method of expressing the frame structure without any clues, and it is difficult for beginners It will be a difficult task.

  Therefore, the support apparatus 10 provides the user with music analysis information (phrase structure analysis information, vertex analysis information, etc.) based on specific rules, case references, etc., on the premise that the expression of the performance is led by the user. Thus, the user-driven performance expression is supported.

  Although this embodiment will be described with respect to a percussion instrument such as a piano, the support apparatus 10 can be applied to instruments other than the percussion instrument, for example, string instruments, wind instruments, percussion instruments, and combinations thereof.

  Below, an example is given and demonstrated about a series of flows of the performance facial expression assistance process which the assistance apparatus 10 provides. The task of performing performance expression using the support device 10 is simply as follows: (1) determination of a group of phrases (primary phrase line) connected in a row; (2) apex with respect to phrases in the primary phrase line. (3) Performance design of the primary phrase line using vertex information, and (4) Giving a sense of unity to the upper structure of the primary phrase line. The support device 10 displays a GUI (Graphical User Interface) screen for allowing the user to perform input and selection operations on the display device 18 as appropriate, and performs interactive performance while providing music analysis information in accordance with an input instruction from the user. Perform facial expression support processing. In accordance with the progress of the performance design, performance data for outputting performance sounds from the audio output device 22 is appropriately generated. For example, a preview button (not shown) displayed on the screen of the display device 18 is clicked. By doing so, the user can audition the performance sound at an arbitrary stage. This will be specifically described below.

  A user who performs a performance expression work using the support apparatus 10 first inputs musical score information (score data) of a desired music piece to the support apparatus 10 or stores musical score information in the memory 14 or the like of the support apparatus 10 in advance. The desired music is selected from the selected music. When the music to be used for expression is determined, the user uses the phrase structure analysis support function of the support device 10 to determine the phrase structure of the music. FIG. 2 shows an example of a phrase structure analysis screen displayed on the screen of the display device 18.

  Specifically, when analyzing the phrase structure, the support apparatus 10 displays the piano roll screen 30 showing the musical score information of the music on the display device 18 as a GUI (see the upper part of FIG. 2). The user selects an arbitrary part of the piano roll, for example, the first few bars of the music or the middle bar, and sets a series of phrases (primary phrase lines) in a row, using the length of about one to several bars as a guide. Form. That is, the phrase boundary is partially designated by putting several dividing lines on the piano roll at an arbitrary part of the music piece.

  When the user partially specifies a phrase boundary, the support device 10 uses a rule (GPR: Rule for determining a boundary of GTTM (A Generative Theory of Tonal Music) for phrase boundaries and hierarchical phrase structures other than the part specified by the user). ExGTTM using Grouping Preference Rule (Hamanaka, M., Hirata, K. and Tojo, S .: Implementing “A Generative Theory of Tonal Music”, Journal of New Music Research, Vol.35, No.4, pp.249 -277 (2006)).

  When the phrase structure analysis process ends, the hierarchical phrase structure 32 is displayed on the display device 18 as a GUI (see the lower part of FIG. 2). In FIG. 2, a phrase structure (phrase U (0) -U (5)) 32a indicated by diagonal lines is a phrase structure specified by the user, and a phrase structure (C (0) -C (27) indicated by a dot pattern is shown. ) 32b are upper and lower phrase structures automatically analyzed by the support device 10. A plurality of vertical lines 34 shown in the hierarchical phrase structure 32 are phrase boundary candidates presented by automatic processing that does not depend on the phrase boundary designation of the user. The user can also correct the phrase structure by referring to the phrase boundary candidates. Also, for convenience of drawing, the user-specified phrase structure 32a is distinguished from the phrase structure 32b by automatic analysis by adding diagonal lines and dot patterns, but in actuality, they are distinguished by arranging arbitrary colors. In the phrase structure 32b by automatic analysis, different colors are further arranged for the upper structure and the lower structure so that they can be distinguished.

  Further, in the support device 10, when the phrase boundary is input or updated by the user, performance data for reproducing the automatic performance is sequentially generated. Although it is not always easy for a user to determine a phrase boundary using only musical score information as a clue, a user other than an expert can perform this operation almost intuitively by using a music preview function.

  When the intended phrase structure is not obtained in the above-described operation, the user appropriately edits the phrase structure. For example, the correction work is performed by selecting a certain phrase and moving the boundary position between the phrase and the phrase immediately before or after the phrase. The upper structure and the lower structure of the phrase structure corrected by the user are automatically reanalyzed. By repeating this work, the phrase structure intended by the user can be finally brought close to the phrase structure.

  Note that the phrase boundary specified by the user may conflict with the GTTM preference. In this case, the phrase boundary specified by the user is prioritized and automatically reduced by lowering the weight of the default preference supported by GTTM. User adaptation of the analysis process itself is achieved. The support device 10 may be provided with a function of updating the GPR user preference so as to support the phrase boundary specified by the user. In this case, regarding the automatic analysis of the phrase structure executed by the support apparatus 10, the user may be able to select which of the GPR basic preference and the user preference updated as needed is prioritized. Further, as a means for assisting the user in specifying a phrase, a function for detecting a phrase of the same type as the phrase specified by the user may be provided by calculating a melody outline.

  When the phrase structure desired by the user is obtained by the above-described operation, the vertex for the phrase in the primary phrase line is subsequently determined using the phrase vertex analysis function of the support apparatus 10.

  Specifically, when determining the apex for a phrase, first, the user selects one phrase from a phrase group included in the primary phrase line. When the user selects a phrase, the piano roll screen of the selected phrase portion is displayed on the display device 18 of the support device 10. In the support device 10, the likelihood (likelihood) of the vertices of each note included in the selected phrase. Is estimated.

  In this example, see Hoshina's music interpretation theory (Hoshishina Hiroshi: Approach to live music expression, performance interpretation based on energy thinking, Ongaku no Tomosha, (1999), hereinafter referred to as "Hoshina theory". ), Vertex analysis is performed.

  In Hoshina theory, in the phrase obtained from the analysis of the score, the note with the highest energy value is defined as the apex or the center of gravity. Then, it is described that the performance according to the composer's intention can be performed by replacing the energy value of the note with the control on the performance expression such as the volume and the tempo. Although the expression method of the performance of the vertex is not unique, the expression of the performance (performance design) that avoids musical failure is possible by drawing a facial expression curve centered on the vertex.

  Therefore, in this embodiment, a method for estimating the likelihood of vertices in accordance with Hoshina theory is formulated, and the energy value of the notes included in the phrase is estimated and displayed, so that the user can perform the facial expression of the performance. It provides information that is a guide or clue.

  Specifically, in Hoshina's theory, high-pitched sounds (that is, apex sounds in tone contours), long-tone sounds, and tense sounds in the tone-relaxation structure of each sound are energy values (that is, apex-likeness). Is said to be high. Also, the principle that the apex is changed by harmony and the final sound of the phrase does not reach the apex is also given. In this embodiment, these tone shape contours and chord progression principles are categorized so that they can be used as rule clauses, and rules regarding temporary symbols and the like are added. FIG. 3-5 shows an example of a group of rules used for calculating the likelihood of vertices. As shown in FIG. 3, the rule group used for calculating the likelihood of apex is classified into six values: tone value, pitch, melodic tone-relaxation, tone product (harmonic), group, and others. In addition, in order to support the expression of musically meaningful sounds such as dominant motion, each rule sets evaluation points for the preceding and subsequent sounds in addition to the corresponding notes. There is.

  As shown in FIGS. 4 and 5, as a rule belonging to the tone value, for example, there is a rule that “one point is added to the succeeding sound when two adjacent first sounds are shorter than the succeeding sound”. As a rule belonging to, for example, there is a rule that “one point is added to the succeeding sound when the two adjacent first sounds are lower than the succeeding sound.” A rule belonging to the melodic tension-relaxation is, for example, “ There is a rule that when the value of a stuttering sound is longer than the subsequent sound, 3 points are added to that stuttering. Further, as a rule belonging to the sound product, for example, there is a rule “1 point is added to the chord of I6”, and as a rule belonging to the group, for example, “1 point is added to the start sound of the group (that is, phrase)”. There is a rule “Yes.”, And another rule belonging to this is, for example, a rule “add one point to a sound with an accent mark”.

  The calculation of the likelihood of the vertex of each note is made by determining whether or not it matches each rule. When there is a corresponding rule, the evaluation point of the likelihood of vertex assigned to that rule is used as the energy value of that note. Is executed by the process of adding. The total point of the energy value represents the apex-likeness of a note, and a note with a high total point is preferably expressed as a vertex sound. The calculation result for each note is appropriately stored in the memory 14 or the like.

  Note that the rule group applied here is merely an example, and other rules may be applied to calculate the likelihood of the vertex of a note. For example, only a part of the rule group shown in FIGS. 4 and 5 can be used, or another rule can be added to the rule group shown in FIGS. A rule group based on a theory different from Hoshina theory can also be applied. Furthermore, it is not limited to estimating the likelihood of a vertex depending on whether or not a specific rule is met. For example, the likelihood of a vertex can be estimated by referring to a case.

  When the apex-likeness of each note in the phrase is estimated by the above-described method, the display device 18 of the support device 10 displays the note-likeness of each note on the piano roll so that the user can identify the apex-likeness of each note. It is displayed as shading information of the drawing color. The user designates the apex sound of the phrase by referring to the information on the likelihood of the apex of each note as appropriate. However, the apex sound of the phrase may be automatically determined by the support device 10. For example, a note having the highest apex point may be automatically determined as the apex sound.

  When the apex sound of the phrase is determined, subsequently, using the performance expression editing function of the support device 10, performance design (performance expression of the performance) for the selected phrase is performed.

  Specifically, when the apex sound of the phrase is determined, the support device 10 sets default values of three facial expression curves of tempo, dynamics, and articulation, and the facial expression curve is set as a GUI according to an instruction from the user. It is displayed on the display device 18. The default value of the facial expression curve is a simple mountain-shaped facial expression curve for the determined vertex sound, a facial expression curve according to the calculated vertices of each note, or a facial expression curve that does not depend on the vertex sound or a free curve Is done. These may be made selectable by the user or may be determined automatically. The user performs expression of the performance by correcting (editing) the shapes of the three expression curves displayed on the display screen 18. For example, the shape of the expression curve may be corrected by a drag operation with the mouse.

  FIG. 6 shows an example of a performance expression editing screen displayed on the display device 18. The performance expression editing screen includes a piano roll screen 40 at the top of the screen and an edit screen 42 at the bottom of the screen. In the piano roll screen 40 and the edit screen 42, the horizontal axis represents the time axis. On the left side of the edit screen 42, three display buttons, that is, a D (dynamics) button 44, a T (tempo) button 46, and an A (articulation) button 48 are provided. , 48 are turned on, the expression curve 50 of the corresponding performance expression is displayed as a dotted line on the edit screen 42. For example, when only the D button 44 is turned on, only the dynamics curve is displayed, and when all the three display buttons 44, 46, and 48 are turned on, the dynamics curve, the tempo curve, and the articulation curve are displayed in a multiplexed manner. That is, on the edit screen 42, three facial expression curves 50 of tempo, dynamics, and articulation can be displayed in multiple on the same time axis.

  The expression curve 50 selected by the edit radio button 52 provided beside the display buttons 44, 46, 48 is displayed by a solid line, and the user operates the mouse of the input device 16 to edit the edit screen 42. You can modify (edit) the shape directly above. By performing this editing operation on each expression curve 50, the user forms a desired performance expression.

  At this time, if the Show apeces check box 54 provided at the bottom of the edit screen 42 is designated, information on the likelihood of the vertices of each note is displayed on the piano roll screen 40. In FIG. 6, for the convenience of the drawing, the pattern of filling the sound 56 is changed so that the density is different, and the difference in the vertices of each sound 56 is expressed by the difference in density expressed by the pattern. However, in practice, an arbitrary color is arranged and the apex-likeness is displayed in an identifiable manner based on the shading information. In this embodiment, the color is darker when the apex likelihood is high, and the color is lightened when the apex likelihood is low. However, the difference in the likelihood of the vertices of each note may be expressed by other methods, for example, by assigning numbers in descending order of the likelihood of vertices.

  If the Show articulation check box 58 provided at the bottom of the edit screen 42 is designated, for example, the default expression curve displayed on the edit screen 42 becomes an expression curve corresponding to the vertices of each note. If it is not specified, it becomes a simple mountain-shaped expression curve for the vertex sound.

  Then, when the Reset curves button 60 is pressed, the expression curve for the selected phrase is updated, and this expression is reflected to the remaining phrases. In addition, the setting device 10 automatically sets the default value of the expression curve based on the vertex information for the upper structure of the primary phrase line. For this upper structure, a modest expression curve is given to the extent that the structure is heard together. Note that the user can edit the expression curve of the upper phrase by performing the same operation as described above.

  When the expression curves of each phrase having a hierarchical phrase structure are set, these expression curves are combined to generate an expression curve for the entire music. The calculation method of each expression curve in the whole music (the whole hierarchical phrase structure) is shown below.

  The tempo curve itself is given as an exponential expression, that is, 0 when there is no change, 1 when the tempo is doubled, and -1 when the tempo is halved. The tempo curve Tempo (t) for the entire music is given by equation (1).

Formula (1)

Here, t is a musical score time, Group Tempo _k (t) is a tempo curve of a phrase having a time t on the musical score within the range, and w _k is a weight of the phrase. Further, the sound generation time and the mute time of each sound are determined by the integral value of Tempo (t). That is, the elapsed time Time (t ₀ , t ₁ ) between the events arranged at t ₀ and t ₁ as the time on the score is that the tempo curve is represented by an exponent and that the tempo is the reciprocal of time progression. Therefore, it is expressed by the equation (2).

Formula (2)

  Here, DeltaDuration is a value defined by the bpm (Beats Per Minute) of the entire song and the resolution of the integration interval in Equation (2). By carrying out such a calculation, when configuring a real-time scheduler such as a command system, it is possible to simply implement processes such as increasing or decreasing the speed in units of beats or less.

  Similar to the tempo curve described above, the dynamics curve Dyn (t) for the entire music is given by equation (3). The sound volume of each sound is also calculated by adding together the dynamic curves of phrases having the time t on the score in the same range as in the above-described pronunciation / mute.

Formula (3)

Here, t is a musical score time, GroupDyn _k (t) is a dynamic curve of a phrase having a time t on the musical score within the range, and w _k is a weight of the phrase. Since the dynamics curve is set on the basis of velocity (sound intensity) in MIDI, the velocity of the note at time t on the score is calculated by the equation (4).

Formula (4)

  Here, StdVel is a default velocity setting value.

Further, in this embodiment, since a keyed musical instrument is targeted, the main control target in articulation expression is the note-off timing of each note. The end time N _Offset (i) on the score of the i-th note is set by Equation (5).

Formula (5)

Here, each of N _Onset (i) and N _TV (i) indicates a start time and a note value on the score of the i-th note. GroupArtcltn (t) indicates the ratio of the articulation curve of the phrase containing the note at time t. The time when N _Offset (i) is actually issued is calculated according to equation (2).

  In the support device 10, when performance symbols such as crescendo, ritardant, and staccato are added as musical score information (score data), they can be developed into facial expression curves by rule processing. When a performance symbol is applied, a new phrase corresponding to the performance symbol is formed, and a default value of the corresponding expression curve is set.

  In this way, the expression curve for the entire music is generated. When editing the expression curve of the selected phrase, the user can individually set the start time, end time, and volume of each sound as necessary. Also, if you want to emphasize the performance design of the upper structure rather than the primary phrase line, lower the preset primary phrase line weighting factor and re-set the upper structure as the primary phrase line for editing operation. May be continued. Further, the series of processes described above can return to the previous operation and repeat the process as necessary.

  In the support device 10, performance data for reproducing the automatic performance is generated or updated in accordance with the correction of the expression curve by the user as described above, and the preview of the piano roll screen 40 is also updated. Then, the user repeats the audition and the editing operation described above with reference to the visual information displayed on the piano roll screen 40 and the edit screen 42, thereby forming a desired performance expression.

  During this editing operation, three facial expression curves 50 of tempo, dynamics and articulation are displayed in multiple on the same time axis on the edit screen 42, so the user grasps the relationship between these performance expressions. It becomes easy to do. That is, the editing operation can be executed while comprehensively determining the relationship between these three performance expressions. In addition, it is possible to recognize how a performance expression is completed by combining performance expressions such as tempo and volume, so that the user can easily perform the expression of the performance.

  In addition, since the information about the apex is displayed on the piano roll screen 40, it is possible to quantitatively determine which note is important, and this also allows the user to easily perform the expression of the performance. become.

  In addition, when performing expression on a piano song for intermediate players of about 1 minute, the target notes are approximately 500 sounds, and it is difficult to add a performance expression to each. However, when the support device 10 is used, there are a dozen or so phrases (boundaries) specified by the user, and the expression of the performance can be executed simply by correcting the expression curve of the phrase specified by the user. It can be seen that the support device 10 is effective in terms of efficiency.

  Then, an example of operation | movement of the above assistance apparatuses 10 is demonstrated according to a flowchart. Specifically, the CPU 12 of the support apparatus 10 executes the entire process shown in FIG. Referring to FIG. 7, when starting the entire process, CPU 12 acquires score information (score data) in step S1, and presents the score information in a piano roll format. That is, the musical score information of the user-desired music stored in the memory 14 or the like is read out, and a piano roll screen is generated based on the information and displayed on the display screen of the display device 18. In the next step S3, a phrase structure analysis process, which will be described later in detail, is executed. When the phrase structure of the music is determined in step S3, the process proceeds to step S5.

  In step S5, it is determined whether or not there is an input for phrase selection. That is, it is determined whether or not the user has selected one phrase from the series of phrase groups in the phrase structure determined in step 3 using the input device 16 or the like. If “YES” in the step S5, that is, if there is an input of phrase selection, the process proceeds to a step S7. On the other hand, if “NO” in the step S5, that is, if there is no input of phrase selection, the process proceeds to a step S17.

  In step S7, an apex likelihood estimation process, which will be described later in detail, is performed, and the process proceeds to step S9. In step S9, the likelihood of a vertex, that is, a candidate for a vertex sound is presented. For example, an arbitrary color is arranged on the sound on the piano roll screen, and the apex-likeness is displayed in an identifiable manner based on the shading information.

  In the next step S11, input of apex designation is accepted. However, if there is no apex designation by the user, or if automatic determination is set, the apex sound of the phrase may be automatically determined based on the vertices of each note calculated in step S7. Good. In addition, when there is an input of apex designation by the user, the information is appropriately stored as a user preference, and the apex estimation rule parameters and the like may be updated to suit the user. it can.

  In the next step S13, an initial value of the expression curve based on the vertex information is set and presented. For example, a default value of a dynamics curve or a tempo curve is set so as to have a peak shape with the vertex sound determined in step S11 as a vertex, or a dynamics so as to have a shape according to the vertex-likeness estimated in step S7. Set default values for curves and tempo curves. The user can also select what default value to take. Each facial expression curve is displayed as a GUI in response to an input instruction from the user. For example, when the user inputs to display three expression curves of tempo, dynamics, and articulation, these three expression curves are displayed in multiple on the same screen on the same time axis.

  In the next step S15, an expression curve editing process described later is executed, and the process proceeds to step S17. In step S17, it is determined whether or not to end the entire process. For example, when there is an end instruction from the user, or when an input operation by the user has not been performed for a predetermined time or more, it is determined that the entire process is finished. If “YES” in the step S17, the entire process is terminated as it is. On the other hand, if “NO” in the step S17, the process returns to an appropriate step according to the designation of the step S3 or the user.

  FIG. 8 is a flowchart showing the phrase structure analysis processing in step S3 shown in FIG. Referring to FIG. 8, when starting the phrase structure analysis process, CPU 12 accepts designation of a GPR preference in step S21. In other words, the selection by the user regarding whether to give priority to the GPR basic preference or the user preference is accepted, and the GPR preference is set according to the user selection.

  In the next step S23, an input of a phrase boundary is accepted. That is, it is detected in which part on the piano roll presented in step S1 of FIG. 7 that the phrase boundary is input by the user, and is temporarily stored in the memory 14 or the like. In step S25, the user GPR preference is updated. That is, according to the phrase boundary designation by the user in step S23, the user GPR preference is updated to suit the user.

  In the next step S27, a similar phrase is detected. That is, a phrase similar to the phrase specified by the user in step S23 is extracted from the entire music, and the process proceeds to step S29. In step S29, the remaining phrase structure is analyzed based on eXGTTM. That is, the phrase structure before and after the phrase specified by the user and the upper and lower phrase structures are analyzed. In this case, the GPR preference (user or basic) set in step S21 is prioritized. Further, when the phrase boundary specified by the user conflicts with the GTTM preference, the weight of the basic GPR preference is reduced and the user GPR preference is updated to match the user. In step S31, a phrase structure is presented. That is, the hierarchical phrase structure analyzed in step S29 is displayed on the display device 18 as a GUI.

  In the next step 33, it is determined whether or not the phrase structure is determined. That is, it is determined whether or not there is a phrase structure determination instruction from the user. For example, the user determines whether the intended phrase structure is achieved by listening to the performance at this stage and visually confirming the phrase structure, and inputs the determination result if satisfied. If “YES” in the step S33, that is, if the phrase structure is determined, the process returns to the entire process of FIG. On the other hand, if “NO” in the step S33, that is, if the phrase structure is analyzed again, the process returns to the step S23 or an appropriate step according to the user's designation.

  FIG. 9 is a flowchart showing the vertex-likeness calculation process in step S7 shown in FIG. In FIG. 9, “i” is a variable assigned in the order of the notes in order to specify the notes. Note (i) describes information such as the pitch, note value and harmony of the i-th note, and ApexVal (i) is a value (energy value) representing the apex-likeness of the i-th note. ). Referring to FIG. 9, when starting the vertex-likeness calculation process, CPU 12 sets ApexVal (i) = 0 to initialize vertex-likeness in step S41, and sets i = 0 in step S43. To do.

  In the next step S45, it is determined whether i = max. That is, it is determined whether or not the vertex-likeness has been calculated for all target notes. If “YES” in the step S45, that is, if the vertex-likeness is calculated for all the target notes, the process returns to the entire process of FIG. On the other hand, if “NO” in the step S45, that is, if there is a note for which the vertex-likeness calculation is not performed, the process proceeds to a step S47.

  In step S47, the information of Note (i) is compared with each of the vertex estimation rules (see FIGS. 4 and 5), and if there is a rule corresponding to Note (i), it is assigned to the corresponding rule. The evaluation point is added to ApexVal (i). The calculated ApexVal (i) value, that is, the total points indicating the likelihood of vertices, is appropriately stored in the memory 14 or the like in association with each note. In the next step S49, i is incremented (i = i + 1), and the process returns to step S45.

  FIG. 10 is a flowchart showing the expression curve editing process in step S15 shown in FIG. Referring to FIG. 10, when starting the expression curve editing process, CPU 12 determines in step S51 whether or not there is an expression curve correction. That is, it is determined whether or not the tempo curve or dynamics curve has been modified by the user. If “YES” in the step S51, that is, if there is correction of the expression curve, the process proceeds to a step S53. On the other hand, if “NO” in the step S51, that is, if the facial expression curve is not corrected, the process proceeds to a step S61.

  In step S53, the expression curve of the entire music is calculated. That is, the default value of the expression curve is also set based on the vertex information for the superstructure of the phrase modified in step S51, and the expression curve of each phrase in the hierarchical phrase structure is synthesized to express the expression as the entire song. Generate a curve. The user can input a performance symbol such as crescendo when correcting the expression curve. When a performance symbol is input, the user develops the performance symbol into the expression curve by rule processing.

  In the next step S55, it is determined whether or not there is an individual correction of each note. That is, it is determined whether or not the sound generation time, the mute time, and the volume of each note have been corrected by the user. If “YES” in the step S55, that is, if there is an individual correction of each note, the process proceeds to a step S57. On the other hand, if “NO” in the step S55, that is, if there is no individual correction of each note, the process proceeds to a step S59.

  In step S57, the performance expression is corrected in note units in accordance with the correction instruction received in step S55. In step S59, the editing result is displayed on the piano roll screen and the edit screen, and the performance data for outputting a performance corresponding to the expression curve editing is updated.

  In step 61, it is determined whether or not a facial expression curve is determined. That is, it is determined whether or not there is an instruction to determine the expression curve from the user. For example, the user determines whether or not the intended expression of the performance has been performed by auditioning the performance with the expression and visual confirmation of the piano roll screen or the edit screen. Enter the judgment result. If “YES” in the step S61, that is, if the facial expression curve is determined, the process returns to the entire process of FIG. On the other hand, if “NO” in the step S61, that is, if the facial expression curve is corrected again, the process returns to the step S51 or an appropriate step according to the user's designation.

  According to this embodiment, when editing a facial expression curve, three facial expression curves of tempo, dynamics, and articulation are displayed on the edit screen in the same time axis. Easy to grasp the relationship. In other words, since it is possible to recognize how the performance expression is completed by combining performance expressions such as tempo and volume, the user can efficiently and easily perform expression of the performance. In addition, by performing expression of performance with such recognition, the user's thinking about performance expression shifts to an objective one, and the technique of music analysis and performance expression can be understood more specifically. . That is, since the expression method of the phrase that has existed as tacit knowledge, that is, how the performance parameters such as tempo are combined to realize the expression can be externalized, its own knowledge can be formalized. Therefore, since the technique of performance expression can be conveyed more specifically to a third party, the support device 10 can be suitably used as an educational tool.

  In addition, since the peak sound of a phrase cannot be determined explicitly by trial listening like a phrase boundary, it is difficult for those who have not experienced music to determine the peak sound. However, according to this embodiment, the information (guidance) about the apex is displayed on the piano roll screen, so that the user can quantitatively determine which note is important and refer to this. This makes it easier and more efficient to express performance. In addition, since it is possible to visually determine what kind of sound is likely to be a vertex sound, the user can easily understand how to find the vertex sound. Also in this respect, it can be said that the support device 10 can be suitably used as an educational tool.

  Furthermore, in this embodiment, since the conditions for calculating the likelihood of the vertex of a note and the evaluation point value are described as rules, it is easy to show the basis of the calculation to the user. Therefore, it is possible to support the expression of the performance from the viewpoint of helping the user to estimate and grasp the vertex.

  In the above-described embodiment, after the user partially specifies the phrase boundary (designates the primary phrase line), the phrase structure of the remaining part is automatically analyzed, but the present invention is not limited to this. For example, the user can specify all of the hierarchical phrase structure of the music, or can automatically analyze all of the hierarchical phrase structure. Of course, the phrase boundary can be corrected by the user after the entire hierarchical phrase structure is automatically analyzed.

  Further, in the above-described embodiment, for the phrase included in the primary phrase line, the apex-likeness is calculated and the correction of the expression curve is accepted. However, the present invention is not limited to this, and the hierarchical phrase structure of the music is configured. It is possible to calculate the likelihood of vertices for each of all the phrases to be performed and accept correction of the expression curve.

10 ... Performance expression support device 12 ... CPU
DESCRIPTION OF SYMBOLS 14 ... Memory 16 ... Input device 18 ... Display device 20 ... Sound source 40 ... Piano roll screen 42 ... Edit screen 50 ... Expression curve

Claims (3)

A performance expression support device that supports expression of performance based on phrasing,
First, multiple expression curves of tempo, dynamics, and articulation are displayed on the same screen in the same time axis for each phrase constituting a hierarchical phrase structure designated or automatically analyzed by the user. A performance expression support apparatus, comprising: display means; and correction accepting means for accepting correction of the expression curve displayed on the screen. The performance expression support apparatus according to claim 1, further comprising: a calculation unit that calculates the apex-likeness of each note in the phrase; and a second display unit that displays the apex-likeness of each note in an identifiable manner. The calculating means includes
Rule storage means for storing a group of rules for determining the apex-likeness of a note based on a predetermined music theory, and by comparing each note in the phrase with each of the rule group, The performance expression support device according to claim 2, further comprising addition means for adding the evaluation point assigned to the rule as an energy value indicating the likelihood of the note apex.

Images