JP2015146035A - Display control device, sound processing device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To intuitively grasp a characteristic which is attached to each specified sound.SOLUTION: An information creation unit 22 creates note information UN indicating a pitch and a sounding period of each specified sound, variably, according to indication from a user. A sound creation unit 24 creates a sound signal SOUT according to the note information UN. A display control unit 26 displays a sound indicator 50 in which a position in a pitch axis direction is determined according to the pitch indicated by the tone information UN, and a position and length of a time axis direction is determined according to the sounding period indicated by the tone information UN, for each specified sound on a display unit 16. The display control unit 26 changes a mode of the sound indicator 50 corresponding to a reproduction point p of the sound signal SOUT, according to a characteristic of the specified sound together with reproduction of the sound signal SOUT.

Description

  The present invention relates to a technique for displaying information related to each sound of a music piece (hereinafter referred to as “designated sound”).

  2. Description of the Related Art Conventionally, a technique for synthesizing sound such as singing sound using musical score information (score data) that specifies a pitch and a pronunciation period for each designated sound has been proposed. For example, Patent Document 1 discloses an image (piano roll image) in which sound indicators indicating respective designated sounds are arranged along a time axis, and a time of a control variable for designating characteristics (for example, musical expression) of each designated sound. A technique for displaying a graph showing a typical change on a common time axis is disclosed. When the designated sound is played, an indicator (pointer) indicating the playback point is displayed, and moves along the time axis as the playback progresses.

JP 2008-165130 A

  However, under the technique of Patent Document 1, since the indicator indicating the reproduction point is only displayed at the time of reproduction of each designated sound, the user cannot intuitively grasp the characteristics given to each designated sound. There is a problem. In view of the above circumstances, an object of the present invention is to make it possible to intuitively understand the characteristics imparted to each designated sound.

  In order to solve the above problems, the display control device of the present invention is a device that displays note information indicating the pitch and the sound generation period of each designated sound, and the first control is performed according to the pitch indicated by the note information. The direction indicator is determined, and the sound indicator whose position and length in the second direction intersecting the first direction is determined according to the sound generation period of the note information is displayed on the display device for each designated sound, and the time axis The sound indicator corresponding to the upper reproduction point is changed over time with the movement of the reproduction point so that the sound indicator indicates the characteristic of the designated sound at the reproduction point in the sound generation period indicated by the sound indicator. Display control means for changing is provided. In the above mode, the mode of the sound indicator corresponding to the playback point is variably controlled according to the characteristics of the designated sound. Therefore, there is an advantage that the user can intuitively grasp the characteristics of each designated sound.

  Note that the sound indicator means a state of an image that can be visually identified by an observer. For example, the shape and size of the sound indicator (width and length), position, gradation of gradation, display color (hue, brightness, saturation), pattern, or a combination thereof is the concept of the sound indicator. Is included.

  In a preferred aspect of the present invention, the display control means displays the sound indicator corresponding to the reproduction point according to the characteristic indicated by the control information indicating the temporal change in the characteristic of each designated sound for the reproduction point. To control. In the above aspect, the sound indicator of each designated sound is controlled according to the characteristic indicated by the control information regarding the playback point, so that the user intuitively grasps the characteristic of each designated sound indicated by the control information. There is an advantage that you can.

  The acoustic processing apparatus according to the present invention generates information generating means that variably generates musical note information indicating the pitch and sound generation period of each designated sound in accordance with an instruction from a user, and generates an acoustic signal corresponding to the musical note information. A sound in which the position in the first direction is determined according to the sound generation means and the pitch indicated by the note information, and the position and length in the second direction intersecting the first direction are determined according to the sound generation period of the note information The indicator is displayed on the display device for each designated sound, and the mode of the sound indicator corresponding to the playback point of the acoustic signal generated by the sound generation means is set at the playback point in the sound generation period indicated by the sound indicator. Display control means for changing the sound signal with time so as to be in a mode according to the characteristics of the designated sound. In the above mode, the mode of the sound indicator corresponding to the playback point is variably controlled according to the characteristics of the designated sound. Therefore, there is an advantage that the user can intuitively grasp the characteristics of each designated sound.

  The information generation means variably generates control information (for example, control information DC in FIG. 1 or control information UC in FIG. 5) indicating temporal changes in characteristics of each designated sound in accordance with an instruction from the user, The generation unit generates an acoustic signal according to the control information, and the display control unit displays a sound indicator corresponding to the reproduction point of the acoustic signal generated by the acoustic generation unit according to the characteristic that the control information indicates for the reproduction point. Control to different aspects. In the above aspect, the sound indicator of each designated sound is controlled in accordance with the characteristics indicated by the control information regarding the reproduction point of the acoustic signal, so that the user can intuitively understand the characteristics of each designated sound indicated by the control information. There is an advantage that can be grasped.

  For example, when the control information (for example, control information DC in FIG. 1) indicates a change in the volume of the designated sound, the display control means controls at least one of the length of the sound indicator and the tone of the sound indicator in the first direction. A configuration in which control is performed according to information is preferable. According to the above aspect, there exists an advantage that the user can grasp | ascertain the volume of each designated sound intuitively from the length and gradation of a sound indicator. A specific example of the above aspect will be described later as the first embodiment, for example.

  Further, when the control information (for example, control information UC in FIG. 5) indicates a minute change in pitch (for example, vibrato, preparation, overshoot), a part of the sound indicator in the first direction (for example, the end of FIG. 6). 52) or a configuration in which the display control means controls all positions in accordance with the control information. According to the above aspect, there exists an advantage that a user can grasp | ascertain the minute change of the pitch of each designated sound intuitively from the position of the sound indicator in the 1st direction which shows a pitch. In addition, the specific example of the above aspect is later mentioned, for example as 2nd Embodiment.

  In a preferred aspect of the present invention, the display control means is configured such that the playback point approaches the start point of the sound generation period when the start point of the sound generation period of the designated sound is delayed or preceded by beat points set at a predetermined interval. Then, the end point (for example, the end point E in FIGS. 7 and 8) of the sound indicator corresponding to the designated sound is moved to one side in the second direction and then returned to the original position. In the above aspect, when the sound of the designated sound precedes or is delayed with respect to the beat point, the sound indicator moves in the second direction when the playback point approaches the sound production period. There is an advantage that the user can intuitively grasp that the point is delayed (accumulated) or preceding (running).

  The information display device and the sound processing device according to each of the aspects described above are realized by hardware (electronic circuit) such as a DSP (Digital Signal Processor) dedicated to the processing of note information, as well as a CPU (Central Processing Unit) and the like This is also realized by the cooperation of the general-purpose arithmetic processing device and the program. A program according to one aspect of the present invention is a program for displaying note information indicating a pitch and a pronunciation period of each designated sound, and the position in the first direction is in accordance with the pitch indicated by the note information. A sound indicator whose position and length in the second direction intersecting with the first direction is determined according to the sound generation period of the note information is displayed on the display device for each designated sound, and the playback point on the time axis Display control that changes the mode of the sound indicator corresponding to the time with the movement of the playback point so that the mode according to the characteristic of the designated sound at the playback point in the sound generation period indicated by the sound indicator Make a computer function as a means. In addition, the program according to another aspect of the present invention includes an information generating unit that variably generates note information indicating a pitch and a sound generation period of each designated sound according to an instruction from a user, and an acoustic according to the note information. The position of the first direction is determined in accordance with the sound generating means for generating the signal and the pitch indicated by the note information, and the position and length in the second direction intersecting the first direction depends on the sound generation period of the note information. The sound indicator determined in this manner is displayed on the display device for each designated sound, and the sound indicator corresponding to the reproduction point of the acoustic signal generated by the acoustic generation means is included in the sound generation period indicated by the sound indicator. The computer is caused to function as display control means that changes with time along with the reproduction of the acoustic signal so as to be in a mode according to the characteristics of the designated sound at the reproduction point.

  The program according to each of the above aspects is provided to the user in a form stored in a computer-readable recording medium and installed in the computer, and is also provided from the server device in the form of distribution via a communication network. Installed on the computer.

1 is a block diagram of a sound processing apparatus according to a first embodiment of the present invention. It is a schematic diagram of musical score information. It is a schematic diagram of an edit screen. It is a schematic diagram for demonstrating the change of the aspect of a sound indicator. It is a schematic diagram of the score information in 2nd Embodiment. It is a schematic diagram for demonstrating the change of the aspect of a sound indicator. It is a schematic diagram for demonstrating the change of the aspect of the sound indicator in 3rd Embodiment. It is a schematic diagram of the change of the aspect of the sound indicator in the modification of 3rd Embodiment.

<A: First Embodiment>
FIG. 1 is a block diagram of a sound processing apparatus 100 according to the first embodiment of the present invention. The sound processing device 100 is a device that generates sound such as a singing sound (hereinafter referred to as “synthetic sound”) in which a plurality of designated sounds are arranged in time series. As shown in FIG. 1, the control device 10 and the storage device 12 are used. And a computer system including the input device 14, the display device 16, and the sound emitting device 18.

  The control device (CPU) 10 implements a plurality of functions (information generation unit 22, sound generation unit 24, display control unit 26) necessary for generating the acoustic signal SOUT by executing the program PG stored in the storage device 12. To do. The acoustic signal SOUT is a signal indicating a waveform of the synthesized sound on the time axis. A configuration in which each function of the control device 10 is realized by a dedicated electronic circuit (DSP) or a configuration in which each function of the control device 10 is distributed over a plurality of integrated circuits may be employed.

  The input device 14 is a device (for example, a mouse or a keyboard) that receives an instruction from a user. The display device 16 (for example, a liquid crystal display device) displays an image instructed from the control device 10. The sound emitting device 18 (for example, a speaker or headphones) reproduces sound according to the sound signal SOUT generated by the control device 10.

  The storage device 12 stores a program PG executed by the control device 10 and various data used by the control device 10 (segment information DV, score information DS, control information DC). A known recording medium such as a semiconductor recording medium or a magnetic recording medium or a combination of a plurality of types of recording media can be arbitrarily employed as the storage device 12. A configuration in which the program PG and each data (DV, DS, DC) are distributed and stored in a plurality of recording media may be employed.

  The unit information DV is a data group used as a material for synthesized speech, and includes a large number of unit data corresponding to different speech units (for example, data indicating the waveform and feature amount of a speech unit). Is done. The phoneme segment is a phoneme which is the smallest unit that can be distinguished auditorily or a phoneme chain in which a plurality of phonemes are connected.

  The musical score information DS is information (score data) instructing each designated sound, and is composed of a plurality of unit information U corresponding to different designated sounds as shown in FIG. Each unit information U includes note information UN and character information UL. The note information UN designates the pitch of the designated sound and the pronunciation period (starting point and duration of the designated sound), and the character information UL designates the pronunciation character (lyric syllable) of the designated sound.

  The control information DC in FIG. 1 is information indicating a temporal change in the control variable X indicating the characteristic (musical expression) of each designated sound. In the present embodiment, volume (dynamics) is exemplified as the control variable X. However, the type of characteristics of the designated sound indicated by the control variable X and the total number of control variables X can be changed as appropriate. The information generation unit 22 in FIG. 1 variably sets the musical score information DS and the control information DC in accordance with a user operation on the input device 14.

  The display control unit 26 shown in FIG. 1 generates the editing screen 40 shown in FIG. 3 that is visually recognized by the user for generating and editing the musical score information DS and the control information DC, and displays the editing screen 40 on the display device 16. As shown in FIG. 3, the editing screen 40 includes a score area 42 that displays a time series of each designated sound, and a variable area 44 that displays changes over time of the control variable X.

  The musical score area 42 is a piano roll type image area in which a vertical axis indicating pitch (hereinafter referred to as “pitch axis”) and a horizontal axis indicating time (hereinafter referred to as “time axis”) are set. In the score area 42, a plurality of straight lines 48 parallel to the pitch axis are arranged at equal intervals in the direction of the time axis. The position indicated by each straight line 48 on the time axis corresponds to the time point of the beat in the music.

  The display control unit 26 arranges the sound indicator 50 at an arbitrary position designated by the user by operating the input device 14 in the score area 42. The sound indicator 50 is a rectangular image set for each designated sound. The position of the sound indicator 50 in the direction of the pitch axis indicates the pitch of the designated sound, and the position of the sound indicator 50 in the direction of the time axis indicates the point in time when the designated sound is generated. Further, the display control unit 26 variably sets the length of the sound indicator 50 in the direction of the time axis in accordance with an operation from the user on the input device 14. The length of the sound indicator 50 in the direction of the time axis indicates the duration of the designated sound (the time length from the start point to the end point of the sound generation). Further, the display control unit 26 adds a pronunciation character (lyric) that the user designates for each designated sound by operating the input device 14 to the sound indicator 50 of the designated sound.

  The information generation unit 22 in FIG. 1 generates unit information U of the score information DS for each sound indicator 50 arranged in the score area 42 by the user and stores it in the storage device 12. That is, the note information UN of each unit information U includes a pitch according to the position of the sound indicator 50 in the direction of the pitch axis, and a sound generation period according to the position and length of the sound indicator 50 in the direction of the time axis. And the character information UL of each unit information U designates a pronunciation character added to the sound indicator 50.

  A variable area 44 in FIG. 3 is an image area in which the variable transition image 442 is arranged. The variable transition image 442 is a graph (line graph) showing the change over time of the control variable X, and is displayed on the same time axis as in the score area 42. The display control unit 26 updates the variable transition image 442 at any time according to an operation from the user with respect to the input device 14. The information generation unit 22 generates control information DC indicating the temporal change of the control variable X indicated by the variable transition image 442 and stores it in the storage device 12.

  When the user instructs the reproduction of the synthesized sound by an operation on the input device 14, the sound generation unit 24 in FIG. 1 generates the sound signal SOUT of each designated sound indicated by the score information DS stored in the storage device 12. . Specifically, the sound generation unit 24 converts the unit data U corresponding to the phonetic character specified by the character information UL of the unit information U from the unit information DV stored in the storage device 12. The sound signal SOUT is generated (speech synthesized) by adjusting the pitches and the sound generation periods specified by the note information UN, and connecting the adjusted segment data to each other. Control information DC is applied to the generation of the acoustic signal SOUT by the acoustic generator 24. That is, the acoustic generator 24 generates the acoustic signal SOUT so that the characteristic (volume) of the acoustic signal SOUT varies with time in accordance with each numerical value of the control variable X specified by the control information DC in time series. The sound signal SOUT generated by the sound generation unit 24 is reproduced as sound by being sequentially supplied to the sound emitting device 18.

  When the reproduction of the acoustic signal SOUT is started, the display control unit 26, as shown in FIG. 3, displays a reproduction instruction that designates a time point (hereinafter referred to as “reproduction point”) p of the acoustic signal SOUT that is actually reproduced. A child (pointer) 46 is displayed on the editing screen 40. The reproduction indicator 46 moves along the time axis with the progress of reproduction of the acoustic signal SOUT. The display control unit 26 sequentially changes the mode of the sound indicator 50 corresponding to the reproduction point p among the plurality of sound indicators 50 arranged in the score area 42. Control of the mode of the sound indicator 50 will be described below.

  FIG. 4 is a schematic diagram showing a change in the mode of the sound indicator 50 to which the phonetic character “sa [s_a]” is added. In FIG. 4, a variable transition image 442 within the sound generation period of the sound indicator 50 (the period from the start point ts to the end point te) is also shown for convenience. When the playback point p of the acoustic signal SOUT indicated by the playback indicator 46 is located before the start of the sound generation period of the sound indicator 50, the display control unit 26, as shown in part (A) of FIG. The indicator 50 is displayed in a standard mode (hereinafter referred to as “standard mode”). In the standard mode, the length (width) W of the sound indicator 50 in the direction of the pitch axis is set to the reference value W0, and the tone (display color shading) G of the sound indicator 50 is set to the reference value G0. The

  When the reproduction point p reaches the sound generation period of the sound indicator 50 (part (B) or part (C) in FIG. 4), the display control unit 26 responds to the control variable X indicated by the control information DC in time series. The mode (width W and gradation G) of the sound indicator 50 is variably controlled. For example, at the time t1 when the numerical value x1 of the control variable X exceeds the predetermined reference value xref, the width W of the sound indicator 50 is set to a numerical value W1 exceeding the reference value W0, as shown in part (B) of FIG. The tone G of the sound indicator 50 is set to a numerical value G1 on the higher density side (darker tone) than the reference value G0. On the other hand, at the time t2 when the numerical value x2 of the control variable X falls below the predetermined reference value xref, the width W of the sound indicator 50 is set to a numerical value W2 below the reference value W0, as shown in part (C) of FIG. The tone G of the sound indicator 50 is set to a numerical value G2 on the lower density side (lighter tone) than the reference value G0. In the above description, the time t1 and the time t2 are focused. However, the mode of the sound indicator 50 changes continuously with the progress of reproduction of the acoustic signal SOUT (movement of the reproduction indicator 46).

  As illustrated above, the display control unit 26 responds to the control variable X specified by the control information DC so that the width W and the gradation G of the sound indicator 50 increase as the numerical value of the control variable X increases. The mode of the sound indicator 50 is changed with time. That is, a change in the characteristic (volume) of the acoustic signal SOUT is represented in a simulated manner in the form of the sound indicator 50 (width W and gradation G).

  On the other hand, when the reproduction point p of the sound signal SOUT has passed the end point te of the sound generation period of the sound indicator 50, the display control unit 26 sets the sound indicator 50 to the standard mode as shown in part (D) of FIG. To do. The processing described above (control of the mode of the sound indicator 50) is sequentially executed for each sound indicator 50 in the score area 42 as the acoustic signal SOUT is reproduced.

  According to the above embodiment, the mode of the sound indicator 50 changes over time according to the control variable X as the acoustic signal SOUT is reproduced, so that the user can intuitively change the characteristics (volume) of each designated sound. There is an advantage that can be grasped. Further, as compared with the configuration in which the mode of each sound indicator 50 is fixed, it is possible to improve the fun of editing each designated sound using the editing screen 40.

<B: Second Embodiment>
Next, a second embodiment of the present invention will be described. In addition, about the element which an effect | action and function are equivalent to 1st Embodiment in each following illustration, the code | symbol referred by the above description is diverted and each detailed description is abbreviate | omitted suitably.

  FIG. 5 is a schematic diagram of the score information DS in the second embodiment. As shown in FIG. 5, each unit information U constituting the score information DS of the second embodiment may include control information UC in addition to the note information UN and the character information UL. The control information UC is information for specifying a minute change in the pitch of the specified sound, and includes head variation information VA and vibrato information VB. The head variation information VA and the vibrato information VB are variably generated by the information generation unit 22 in accordance with an operation from the user on the input device 14. Note that the unit information U of the designated sound to which a minute change in pitch is not added does not include the control information UC.

  FIG. 6 is a schematic diagram showing a minute change in the pitch of the designated sound. When the pitch changes between successive designated sounds, as shown in FIG. 6, if preparation (period TA1) and overshoot (period TA2) are added to the beginning of the sound generation period of the rear designated sound, The change in pitch between the specified sounds gives an acoustically natural impression. Preparation is a singing method that instantaneously changes the pitch in the opposite direction to the change in pitch between the specified sounds, and overshoot sounds so as to instantaneously exceed the changed pitch (N0). It is a singing method that changes the height.

  The head variation information VA in FIG. 5 indicates the characteristics of the preparation and overshoot added to the designated sound. Specifically, the head fluctuation information VA includes a pitch fluctuation amount (difference from the pitch of the immediately preceding designated sound) δA1 and a preparation period (start point and duration) TA1, and a sound due to overshoot. A high fluctuation amount (difference from the target pitch N0) δA2 and an overshoot period (start point and duration) TA2 are designated.

  The vibrato information VB in FIG. 5 indicates the characteristics of the vibrato added to the vibrato period TB in the sound generation period of the designated sound. For example, the vibrato information VB includes a vibrato period TB (start point and continuation length), a vibrato depth δB1, and a vibrato speed δB2. As shown in FIG. 6, the vibrato depth δB1 means a fluctuation range of the pitch due to vibrato, and the vibrato speed δB2 means a speed (frequency) of a change in pitch due to vibrato.

  The sound generation unit 24 generates the sound signal SOUT so that the pitch of each designated sound specified by the note information UN changes minutely according to the control information UC (lead variation information VA, vibrato information VB). On the other hand, when the reproduction of the acoustic signal SOUT by the acoustic generation unit 24 is started, the display control unit 26 responds to a minute change in the pitch of each designated sound designated by the control information UC (lead variation information VA, vibrato information VB). The mode of the sound indicator 50 is controlled. In the following description, the case where the reproduction point p of the acoustic signal SOUT is located within the preparation period TA1 or the overshoot period TA2 of the sound generation period of the designated sound (pronunciation character “sa [s_a]”), and the vibrato period The change in the mode of the sound indicator 50 will be described in detail by dividing the case into the case where it is located in TB.

(1) Preparation period TA1, Overshoot period TA2
When the reproduction point p of the acoustic signal SOUT is located within the preparation period TA1 or the overshoot period TA2 specified by the head fluctuation information VA, the display control unit 26 displays the end of the sound indicator 50 (in the direction of the time axis). The position of the left end portion 52 in the direction of the pitch axis is variably controlled so as to be linked to minute pitch fluctuations (preparation and overshoot) indicated by the head fluctuation information VA.

  As shown in part (A1) of FIG. 6, when the playback point p of the acoustic signal SOUT indicated by the playback indicator 46 is located before the start of the preparation period TA1 specified by the head fluctuation information VA, The shape of the indicator 50 is set to be linear along the time axis as a whole.

  When the playback of the acoustic signal SOUT progresses and the playback point p reaches the preparation period TA1, the display control unit 26, as shown in part (A2) of FIG. The sound indicator 50 is deformed (curved) so that the end 52 moves to the low pitch side in the direction of the pitch axis. Specifically, the movement amount ΔA1 of the end portion 52 in the direction of the pitch axis increases from the start point tA1s of the preparation period TA1 to the center point, and reaches a maximum value corresponding to the variation amount δA1 indicated by the head variation information VA. The sound indicator 50 is continuously deformed with the movement of the reproduction point p from the start point tA1s to the end point tA1e of the preparation period TA1 so as to reach the end point tA1e of the preparation period TA1.

  On the other hand, when the reproduction point p of the acoustic signal SOUT reaches within the overshoot period TA2, the display control unit 26, as shown in the part (A3) of FIG. The sound indicator 50 is deformed (curved) so that the end 52 moves to the high sound side in the direction of. Specifically, the movement amount ΔA2 of the end portion 52 in the direction of the pitch axis increases from the start point tA2s of the overshoot period TA2 and reaches the maximum value corresponding to the variation amount δA2 in the head variation information VA. The sound indicator 50 is continuously deformed so as to decrease from the end point tA2e of the overshoot period TA2.

  When the playback point p of the acoustic signal SOUT has passed the end point tA2e of the overshoot period TA2, the display control unit 26 changes the shape of the sound indicator 50 to the shape before deformation (on the time axis as shown in part (A4) of FIG. 6). Along the straight line). As understood from the above description, the minute fluctuation of the pitch due to the preparation indicated by the head fluctuation information VA and the overshoot is simulated by the movement of the end 52 of the sound indicator 50.

(2) Vibrato period TB
As shown in part (B1) of FIG. 6, when the reproduction point p of the acoustic signal SOUT is located before the start of the vibrato period TB (start point tBs), the display control unit 26 performs the pitch specified by the note information UN. The sound indicator 50 is arranged so that the center line coincides with a position (hereinafter referred to as “reference position”) Pref corresponding to N0 (hereinafter referred to as “designated pitch”).

  When the playback point p of the acoustic signal SOUT arrives within the vibrato period TB specified by the vibrato information VB in the sound generation period of the specified sound, the display control unit 26 is linked to the pitch fluctuation (vibrato) indicated by the vibrato information VB. Thus, the position of the sound indicator 50 (entire) in the direction of the pitch axis is variably controlled. Specifically, the display control unit 26 continuously vibrates the sound indicator 50 in the direction of the pitch axis around the reference position Pref so as to be interlocked with the minute fluctuation of the pitch of the acoustic signal SOUT by vibrato. .

  For example, at the time point tB1 when the pitch of the acoustic signal SOUT exceeds the designated pitch N0 due to vibrato, the display control unit 26 makes the center line of the sound indicator 50 be located on the higher pitch side (upward) than the reference position Pref. The position of the sound indicator 50 is controlled. On the other hand, at the time point tB2 when the pitch of the acoustic signal SOUT falls below the designated pitch N0 due to vibrato, the display control unit 26 ensures that the center line of the sound indicator 50 is located on the bass side (downward) from the reference position Pref. The position of the sound indicator 50 is controlled. In the above description, only the time tB1 and the time tB2 are focused. However, the sound indicator 50 continuously moves (vibrates) along the pitch axis as the acoustic signal SOUT is reproduced.

  The amplitude of the vibration of the sound indicator 50 is variably set according to the vibrato depth δB1 specified by the vibrato information VB. For example, the amplitude of vibration of the sound indicator 50 increases as the vibrato depth δB1 increases. The frequency of vibration of the sound indicator 50 is variably set according to the vibrato speed δB2 specified by the vibrato information VB. For example, the vibration frequency of the sound indicator 50 increases as the vibrato speed δB2 increases. As can be understood from the above description, the fluctuation of the pitch of the designated sound due to the vibrato indicated by the vibrato information VB is simulated by the movement (vibration) of the sound indicator 50.

  According to the above embodiment, since the mode (shape and position) of the sound indicator 50 changes in conjunction with the minute change in the pitch specified by the control information UC, the minute change in the pitch of the acoustic signal SOUT is detected by the user. There is an advantage that can be grasped intuitively. Further, as compared with the configuration in which the mode of each sound indicator 50 is fixed, it is possible to improve the fun of editing each designated sound using the editing screen 40.

<C: Third Embodiment>
In FIG. 3, the starting point of the pronunciation period of each designated sound of the pronunciation character “a [a]” and the pronunciation character “sa [s_a]” coincides with the beat point indicated by the straight line 48. On the other hand, a singing method (so-called “reservation”) that delays the starting point of the pronunciation period from the beat point is designated for the designated sound whose pronunciation character is “[g_a]”. The display control unit 26 of the third embodiment variably controls the mode of the sound indicator 50 of the designated sound whose starting point of the sound generation period does not match the beat point.

  FIG. 7 is a schematic diagram for explaining a change in the mode of the sound indicator 50 in the third embodiment. In the following, it is assumed that the start point ts of the sound generation period specified by the note information UN is delayed from the beat point b corresponding to the straight line 48. As shown in part (A) of FIG. 7, at the stage where the playback point p of the acoustic signal SOUT indicated by the playback indicator 46 has not reached the beat point b, the same as in the first and second embodiments. In addition, the end point E of the sound indicator 50 is located at the time ts (starting point of the sound generation period).

  When the reproduction of the acoustic signal SOUT proceeds and the reproduction point p reaches the beat point b, the display control unit 26 determines that the end point E of the sound indicator 50 is a distance from the time ts as shown in part (B) of FIG. The sound indicator 50 is moved and maintained in the direction of the time axis so as to be positioned at the time point tx on the right side (time passage side) by d. The distance d of movement of the sound indicator 50 is variably set according to, for example, the time difference between the beat point b and the time point ts (the amount of sound generation delay with respect to the beat point b). For example, the distance d increases as the time difference between the beat point b and the time point tx increases.

  When the reproduction point p of the sound signal SOUT reaches the start point ts of the sound generation period specified by the note information UN, the display control unit 26 keeps the sound indicator 50 along the time axis until the end point E matches the time point ts. Move. Specifically, the display control unit 26 gradually attenuates the vibration while vibrating the sound indicator 50 in the direction of the time axis, and finally the vibration of the sound indicator 50 so that the end point E matches the time point ts. To converge. That is, the position of the sound indicator 50 on the time axis is controlled so that the behavior of the object when the load acting on the elastically supported object is released is simulated at the end point E.

  As described above, the sound indicator 50 of the designated sound that is sounded delayed from the beat point b (that is, the singing sound that is accumulated and sung) is moved in the direction of the time axis, and the reproduction point p is the actual sounding period. The sound indicator 50 returns to the initial position when approaching the starting point ts. That is, the manner in which the pronunciation is accumulated is simulated by the behavior of the sound indicator 50. Therefore, there is an advantage that the user can intuitively grasp the accumulation (characteristic) of pronunciation of each designated sound. Further, as compared with the configuration in which the mode of each sound indicator 50 is fixed, it is possible to improve the fun of editing each designated sound using the editing screen 40.

  In the above description, the case where the start point ts of the sound generation period is delayed from the beat point b (reservation) is illustrated, but for example, as shown in part (A) of FIG. The designated sound instructed by the singing method preceded by b (so-called “run”) can also be controlled in the same manner as in the case where the sound indicator 50 is a reservoir as described below.

  For example, when the reproduction point p of the acoustic signal SOUT arrives at a time t0 before the starting point ts of the sound generation period specified by the note information UN, the display control unit 26, as shown in part (B) of FIG. The sound indicator 50 is moved so that the end point E is located from the time point ts to the right time point tx by a distance d corresponding to the time difference between the time point ts and the beat point b. When the playback point p of the sound signal SOUT reaches the start point ts of the sound generation period, the display control unit 26, as shown in the part (C) of FIG. 8, displays the sound indicator 50 until the end point E matches the time point ts. Is moved over time while vibrating. According to the above configuration, the state of singing can be simulated by the behavior of the sound indicator 50. Therefore, there is an advantage that the user can intuitively grasp the running of each designated sound.

<D: Modification>
Each of the above forms can be variously modified. Specific modifications are exemplified below. Two or more aspects arbitrarily selected from the following examples can be appropriately combined.

(1) Modification 1
The characteristics of the designated sound applied to the change in the mode of the sound indicator 50 include the volume of the designated sound (control variable X) in the first embodiment, the minute variation in the pitch of the designated sound in the second embodiment, It is not limited to the time difference between the starting point ts of the sound generation period of the designated sound and the beat point b in the third embodiment. For example, for example, the breath component added to the specified sound (breathiness), clarity (brightness, clearness), opening degree during pronunciation (opening), gender factor (sender), and pitch are continuously changed. An arbitrary variable (control variable X) that can be applied to the generation of the acoustic signal SOUT, such as a portamento-timing point, is applied to control the mode of the sound indicator 50 as a numerical value indicating the characteristics of the designated sound. obtain.

(2) Modification 2
The types of modes of the sound indicator 50 controlled by the display control unit 26 according to the characteristics of the designated sound are the width W and the gradation G of the sound indicator 50 in the first embodiment, and the sound indicator in the second embodiment. It is not limited to the position on the pitch axis of 50 (all or the end portion 52) or the position on the time axis of the sound indicator in the third embodiment. For example, the display color (hue, brightness, saturation), shape, and pattern of the sound indicator 50 can be variably controlled. In the above embodiment, the sound indicator 50 is a two-dimensional figure, but a configuration in which the sound indicator 50 is displayed three-dimensionally can also be adopted. For example, a configuration in which the depth of the three-dimensionally displayed sound indicator 50 is variably controlled according to the characteristics of the designated sound is suitable. As understood from the above examples, the mode of the sound indicator 50 refers to the state of an image that can be visually recognized by an observer (typically, the shape, size, position, gradation, and display of the sound indicator 50). Color).

(3) Modification 3
A combination of the type of characteristic of the sound indicator 50 and the type of aspect of the sound indicator 50 that is variably controlled according to the characteristic is arbitrary. For example, the configuration in which the width W and the gradation G of the sound indicator 50 are changed in conjunction with a minute change (preparation, overshoot, vibrato) indicated by the control information UC in FIG. Instead of the configuration for changing the position of the sound indicator 50 (or together with this configuration), a configuration for changing other modes (for example, shape and gradation) of the sound indicator 50 may be employed.

(4) Modification 4
Each form illustrated above may be merged suitably. For example, the width W and gradation G of the sound indicator 50 are variably controlled according to the control information DC (first embodiment), and the pitch of the sound indicator 50 is determined according to the control information UC of each designated sound. A configuration in which the position on the shaft is variably controlled (second embodiment) is preferably employed.

(5) Modification 5
In each of the above embodiments, the playback indicator 46 that indicates the playback point p of the acoustic signal SOUT is arranged on the editing screen 40, but the playback indicator 46 may be omitted. Since the mode of each sound indicator 50 changes according to the playback point p of the acoustic signal SOUT, the user can check the current playback point p even with the configuration in which the playback indicator 46 is omitted.

DESCRIPTION OF SYMBOLS 100 ... Sound processing apparatus, 10 ... Control apparatus, 12 ... Memory | storage device, 14 ... Input device, 16 ... Display apparatus, 18 ... Sound emission device, 22 ... Information generation part, 24 ... Sound generation , 26... Display control section, 40... Editing screen, 42... Score area, 44 .. variable area, 46 .. reproduction indicator, 50.

Claims (7)

A device for displaying note information indicating the pitch and the pronunciation period of each designated sound,
A sound indicator in which the position in the first direction is determined according to the pitch indicated by the note information, and the position and length in the second direction intersecting the first direction are determined according to the sound generation period of the note information. Is displayed on the display device for each designated sound, and the mode of the sound indicator corresponding to the reproduction point on the time axis is set to the characteristic of the designated sound at the reproduction point in the sound generation period indicated by the sound indicator. A display control device comprising display control means for changing over time with the movement of the reproduction point so as to achieve a corresponding mode. The display control unit controls the mode of the sound indicator corresponding to the reproduction point to a mode according to the characteristic that the control information indicating the temporal change in the characteristic of each designated sound indicates the reproduction point. Item 1. The display control device according to Item 1. The control information indicates a change in volume of the designated sound,
The display control apparatus according to claim 2, wherein the display control unit controls at least one of a length of the sound indicator and a gradation of the sound indicator in the first direction according to the control information. The control information indicates a minute change in the pitch of the designated sound,
The display control device according to claim 2, wherein the display control unit controls a position of a part or all of the sound indicator in the first direction according to the control information. When the playback point approaches the start point of the sounding period when the start point of the sounding period of the specified sound is delayed or preceded by beat points set at predetermined intervals, the display control means The display control device according to claim 1, wherein the end point of the corresponding sound indicator is moved to one side in the second direction and then returned to the original position. Information generating means for variably generating note information indicating a pitch and a pronunciation period of each designated sound according to an instruction from a user;
Sound generating means for generating an acoustic signal corresponding to the note information;
A sound indicator in which the position in the first direction is determined according to the pitch indicated by the note information, and the position and length in the second direction intersecting the first direction are determined according to the sound generation period of the note information. Is displayed on the display device for each designated sound, and the mode of the sound indicator corresponding to the playback point of the sound signal generated by the sound generation means is displayed at the playback point in the sound generation period indicated by the sound indicator. A sound processing apparatus comprising: display control means that changes over time with reproduction of the sound signal so as to be in a mode according to the characteristics of the designated sound. A program for displaying note information indicating the pitch and pronunciation period of each designated sound,
A sound indicator in which the position in the first direction is determined according to the pitch indicated by the note information, and the position and length in the second direction intersecting the first direction are determined according to the sound generation period of the note information. Is displayed on the display device for each designated sound, and the mode of the sound indicator corresponding to the reproduction point on the time axis is set to the characteristic of the designated sound at the reproduction point in the sound generation period indicated by the sound indicator. A program that causes a computer to function as display control means that changes over time with the movement of the reproduction point so as to achieve a corresponding mode.

Images